HttpClient 4.x how to use cookies? - java

I'm trying to use the cookies I get in a response to my post method using HttpClient 4.0.3;
Here is my code:
public void generateRequest()
{
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost("http://mysite.com/login");
httpclient.getParams().setParameter("http.useragent", "Custom Browser");
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
httpclient.getParams().setParameter(ClientPNames.COOKIE_POLICY,
CookiePolicy.BROWSER_COMPATIBILITY);
CookieStore cookieStore = new BasicCookieStore();
HttpContext localContext = new BasicHttpContext();
localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
try
{
LOG.info("Status Code: sending");
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("email", "john%40gmail.com"));
nameValuePairs.add(new BasicNameValuePair("password", "mypassword"));
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
httppost.setHeader("ContentType", "application/x-www-form-urlencoded");
HttpResponse response = httpclient.execute(httppost, localContext);
HttpEntity entity = response.getEntity();
if (entity != null)
{
entity.consumeContent();
}
iterateCookies(httpclient);
}
catch (ClientProtocolException e)
{
LOG.error("ClientProtocolException", e);
}
catch (IOException e)
{
LOG.error("IOException", e);
}
}
private void iterateCookies(DefaultHttpClient httpclient)
{
List<Cookie> cookies = httpclient.getCookieStore().getCookies();
if (cookies.isEmpty())
{
System.out.println("No cookies");
}
else
{
for (Cookie c : cookies)
{
System.out.println("-" + c.toString());
}
}
}
But I keep getting the No cookies logged out even though when I use web-sniffer.net, I get this response:
Status: HTTP/1.1 302 Found
Cache-Control: private, no-store
Content-Type: text/html; charset=utf-8
Location: http://www.mysite.com/loginok.html
Server: Microsoft-IIS/7.0
X-AspNet-Version: 2.0.50727
Set-Cookie: USER=DDA5FF4E1C30661EC61CFA; domain=.mysite.com; expires=Tue, 08-Jan-2013 18:39:53 GMT; path=/
Set-Cookie: LOGIN=D6CC13A23DCF56AF81CFAF; domain=.mysite.com; path=/ Date: Mon, 09 Jan 2012 18:39:53 GMT
Connection: close
Content-Length: 165
All the examples I've found online that make any sort of sense refer to HttpClient 3.x where you can set the CookiePolicy to IGNORE and handle the Set-Cookie header manually. I can't understand why this is so difficult in 4.x. I need access to the USER hash for a number of reasons. Can anyone please tell me how in the hell I can get access to it?
UPDATE
I have found the following C# code which does the same thing and works correctly.
private static string TryGetCookie(string user, string pass, string baseurl)
{
string body = string.Format("email={0}&password={1}", user, pass);
byte[] bodyData = StringUtils.StringToASCIIBytes(body);
HttpWebRequest req = WebRequest.Create(baseurl) as HttpWebRequest;
if (null != req.Proxy)
{
req.Proxy.Credentials = CredentialCache.DefaultCredentials;
}
req.AllowAutoRedirect = false;
req.Method = "Post";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = bodyData.Length;
using (Stream reqBody = req.GetRequestStream())
{
reqBody.Write(bodyData, 0, bodyData.Length);
reqBody.Close();
}
HttpWebResponse resp1 = req.GetResponse() as HttpWebResponse;
string cookie = resp1.Headers["Set-Cookie"];
if( string.IsNullOrEmpty(cookie))
{
if (0 < resp1.ContentLength)
{
// it's probably not an event day, and the server is returning a singlecharacter
StreamReader stringReader = new StreamReader(resp1.GetResponseStream());
return stringReader.ReadToEnd();
}
return null;
}
return ParseCookie(cookie);
}
I believe my java code is not forming the post request correctly because when I use a URLConnection and print the request header from web-sniffer.net below:
POST /reg/login HTTP/1.1[CRLF]
Host: live-timing.formula1.com[CRLF]
Connection: close[CRLF]
User-Agent: Web-sniffer/1.0.37 (+http://web-sniffer.net/)[CRLF]
Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7[CRLF]
Cache-Control: no-cache[CRLF]
Accept-Language: de,en;q=0.7,en-us;q=0.3[CRLF]
Referer: http://web-sniffer.net/[CRLF]
Content-type: application/x-www-form-urlencoded[CRLF]
Content-length: 53[CRLF]
[CRLF]
email=john%40gmail.com&password=mypassword
I get a response from the server that contains the set-cookies header. Is my java code not generating the request the same as web-sniffer.net?
I have seen a post method generated using this code:
PostMethod authPost = new PostMethod("http://localhost:8000/webTest/j_security_check");
// authPost.setFollowRedirects(false);
NameValuePair[] data = {
new NameValuePair("email", "john%40gmail.com"),
new NameValuePair("password", "mypassword")
};
authPost.setRequestBody(data);
status = client.executeMethod(authPost);
The main difference here being that the NameValuePair data is set in the request body rather than set as the entity. Does this make a difference? Would this produce the correct request header?

Both cookies look suspicious. Both use outdated Netscape cookie draft format. Both have invalid domain attribute value. The LOGIN appears malformed (semicolon is missing after the path attribute) on top of that. So, most likely both cookies got rejected by HttpClient.
You can find out whether this is the case by running HttpClient with the context logging turned on as described here:
http://hc.apache.org/httpcomponents-client-ga/logging.html
One last remark. Generally one should not meddle with cookie policies when using HttpClient 4.x. The default BEST_MATCH policy will automatically delegate processing of cookies to a particular cookie spec implementation based on the composition of the Set-Cookie header value. In order to disable cookie processing entirely one should remove cookie processing protocol interceptors from the protocol processing chain.
Hope this helps.

I believe the problem is that you are mixing two "styles" here: on one hand you create your own BasicCookieStore and put that in your HttpContext; on the other hand, when print the cookies, you loop over the cookie store in the DefaultHttpClient.
So either change the iterateCookies to use your local cookie store, or just use the one provided by the DefaultHttpClient. As you can see in the javadoc of DefaultHttpClient, it should automatically add response cookies to its internal cookie store.

It's always a simple answer! After debugging the C# and another C program I found. I was jumping the gun and doing my own encoding on the email address to remove the # character. This was the problem!!! Nothing else seemed to make a difference whether it was there or not! The code now looks like:
public void postData(final String email, final String password)
{
DefaultHttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(LOGIN_URL);
client.getParams().setParameter("http.useragent", "Custom Browser");
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
try
{
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("email", email));
nameValuePairs.add(new BasicNameValuePair("password", password));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(nameValuePairs, HTTP.UTF_8);
entity.setContentType("application/x-www-form-urlencoded");
post.setEntity(entity);
printHeaders(client.execute(post));
printCookies(client.getCookieStore());
}
catch (ClientProtocolException e)
{
LOG.error("ClientProtocolException", e);
}
catch (IOException e)
{
LOG.error("IOException", e);
}
}
and the output now looks like:
Response Headers:
-Cache-Control: private, no-store
-Content-Type: text/html; charset=utf-8
-Server: Microsoft-IIS/7.0
-X-AspNet-Version: 2.0.50727
-Set-Cookie: USER=3D907C0EB817FD7...92F79616E6E96026E24; domain=.mysite.com; expires=Thu, 10-Jan-2013 20:22:16 GMT; path=/
-Set-Cookie: LOGIN=7B7028DC2DA82...5CA6FB6CD6C2B1; domain=.mysite.com; path=/
-Date: Wed, 11 Jan 2012 20:22:16 GMT
-Content-Length: 165
-Cookie:: [version: 0][name: USER][value: 3D907C0E...E26E24][domain: .mysite.com][path: /][expiry: Thu Jan 10 20:22:16 GMT 2013]
-Cookie:: [version: 0][name: LOGIN][value: 7B7028D...D6C2B1][domain: .mysite.com][path: /][expiry: null]

Related

HTTP/1.1 400 Bad Request using base64 encoding in java

I am using http request. I have two image to call api. Image are base64encoding..but in postman it is working fine. It is not working in my java code..it shows bellow error
response = HTTP/1.1 400 Bad Request [Server: nginx/1.14.0 (Ubuntu), Date: Fri, 13 Nov 2020 09:24:20 GMT, Content-Type: application/json; charset=utf-8, Content-Length: 3866, Connection: keep-alive, X-Powered-By: Express, Access-Control-Allow-Origin: *, Access-Control-Expose-Headers: Origin, X-Requested-With, Content-Type, Accept, x-auth-token, x-login-token, x-verification-token, ETag: W/"f1a-xqn4nWVQUqRNj8g2mv5av6fbTrg"]
statusCode = 400
Postman header image
Postman body
I have used bellow code..
byte[] photo1=null;
byte[] photo2=null;
String base64encodedString = Base64.getEncoder().encodeToString(photo1);
String base64encodedString2 = Base64.getEncoder().encodeToString(photo2);
try {
DefaultHttpClient httpclient = new DefaultHttpClient(new BasicHttpParams());
HttpPost httppost = new HttpPost("url");
JSONObject jsonObject = new JSONObject();
try {
jsonObject.put("photo", base64encodedString);
jsonObject.put("nidFront", base64encodedString2);
} catch (Exception e) {
e.printStackTrace();
}
System.out.println("jsonObject = " + jsonObject);
StringEntity params = new StringEntity(String.valueOf(jsonObject));
httppost.addHeader("content-type", "application/json");
httppost.setHeader("x-auth-token", token);
httppost.setEntity(params);
HttpResponse response = httpclient.execute(httppost);
int statusCode = response.getStatusLine().getStatusCode();
System.out.println("statusCode = " + statusCode);
Please help me what is wrong in code

Not able to get proper response while calling POST method with parameter

Trying to calling post method using main method but getting 400 Bad request. I am using below code
private static String POST_URL = "http://localhost:8080/RestApi/api/v1/getDetails";
public static void main(String[] args) {
try {
sendPost();
} catch (Exception e) {
System.out.println("Exception occurred while calling service and exception is " + e);
}
}
private static void sendPost() throws Exception {
HttpPost post = new HttpPost(POST_URL);
post.addHeader(HttpHeaders.CONTENT_TYPE, "application/json");
// add request parameter, form parameters
List<NameValuePair> urlParameters = new ArrayList<>();
urlParameters.add(new BasicNameValuePair("id", "1004906799"));
urlParameters.add(new BasicNameValuePair("identifer", "Customer"));
post.setEntity(new UrlEncodedFormEntity(urlParameters));
try (CloseableHttpClient httpClient = HttpClients.createDefault(); CloseableHttpResponse response = httpClient.execute(post)) {
System.out.println(EntityUtils.toString(response.getEntity()));
}
}
> Response : HTTP/1.1 400 Bad Request [Connection: keep-alive,
> X-Powered-By: Undertow/1, Server: WildFly/8, Content-Length: 0, Date:
> Mon, 14 Oct 2019 09:44:34 GMT]
Try by setting the encoding. Replace post.setEntity(new UrlEncodedFormEntity(urlParameters)); line with
post.setEntity(new UrlEncodedFormEntity(urlParameters, "UTF-8"));
Haven't run it but should work.
The problem is that you post a form content rather than a JSON payload:
Replace those lines:
List<NameValuePair> urlParameters = new ArrayList<>();
urlParameters.add(new BasicNameValuePair("id", "1004906799"));
urlParameters.add(new BasicNameValuePair("identifer", "Customer"));
post.setEntity(new UrlEncodedFormEntity(urlParameters));
with something like this:
String payload = "{\n" +
" \"id\": \"1004906799\",\n" +
" \"identifier\": \"Customer\"\n" +
"}";
post.setEntity(payload);
See the section 4. POST with JSON in this tutorial.

Apache HttpClient not receiving entire response

Update: If I use System.out.println(EntityUtils.toString(response.getEntity())); the output is what appears to be the missing lines of HTML (including the closing body and html tags). However, printing to a file still only gives me the first 2000 odd lines missing the last 1000.
I am using the following code to perform a http post request:
public static String Post(CloseableHttpClient httpClient, String url, Header[] headers,
List<NameValuePair> data, HttpClientContext context) throws IOException
{
HttpPost httpPost = new HttpPost(url);
httpPost.setEntity(new UrlEncodedFormEntity(data));
httpPost.setHeaders(headers);
CloseableHttpResponse response = httpClient.execute(httpPost, context);
if (response.getEntity() == null)
throw new NullPointerException("Unable to get html for: " + url);
// Get the data then close the response object
String responseData = EntityUtils.toString(response.getEntity());
EntityUtils.consume(response.getEntity());
response.close();
return responseData;
}
However I am not receiving the full response entity. I am missing about 1000 lines of html (including the closing body and html tags. I think this is because the data is being sent in chunks although I am not entirely sure.
Here are the response headers:
Cache-Control:max-age=0, no-cache, no-store
Connection:Transfer-Encoding
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/html; charset=utf-8
Date:Sat, 04 Jul 2015 15:14:58 GMT
Expires:Sat, 04 Jul 2015 15:14:58 GMT
Pragma:no-cache
Server:Microsoft-IIS/7.5
Transfer-Encoding:chunked
Vary:User-Agent
Vary:Accept-Encoding
X-Content-Type-Options:nosniff
X-Frame-Options:SAMEORIGIN
How can I ensure that I receive the full response entity?
To gather all the essence of the comments. There is nothing wrong with your code here - using EntityUtils is the recomended way to deal with all kinds of responses. You have error in code that stores your response to the file.
I've got a similar problem, and solved it by ensuring lcosing the connection like :
} finally {
try {
EntityUtils.consume(entity);
try {
response.getOutputStream().flush();
} catch (IOException e) {
logger.warn("Error while flushing the response output connection. It will ensure to close the connection.", e);
}
if (null != httpResponse) {
httpResponse.close();
}
} catch (IOException ignore) {
}
}
Or event better with try-resources :
try(CloseableHttpResponse response = httpClient.execute(httpPost, context)){
if (response.getEntity() == null){
throw new NullPointerException("Unable to get html for: " + url);
}
String responseData = EntityUtils.toString(response.getEntity());
EntityUtils.consume(response.getEntity());
}

Apache HTTP Client Post appears to work but variable not received

I have the following code that executes a httpclient post request
public void upload() throws Exception{
//HTTP POST Service
try{
HttpClient httpclient = HttpClientBuilder.create().build();
URI uri = new URIBuilder()
.setScheme("http")
.setHost("www.mysite.com")
.setPath("/mypage.php")
.setParameter("Username", userID)
.setParameter("Password", password)
.build();
HttpPost httppost = new HttpPost(uri);
httpclient.execute(httppost);
BasicHttpContext localContext = new BasicHttpContext();
HttpResponse response = httpclient.execute(httppost, localContext);
HttpUriRequest currentReq = (HttpUriRequest) localContext.getAttribute(ExecutionContext.HTTP_REQUEST);
HttpHost currentHost = (HttpHost)localContext.getAttribute(ExecutionContext.HTTP_TARGET_HOST);
String currentUrl = currentHost.toURI() + currentReq.getURI();
System.out.println(currentUrl);
System.out.println(response);
HttpEntity httpEntity = response.getEntity();
String str = "";
if (httpEntity != null) {
str = EntityUtils.toString(httpEntity);
System.out.println(str);
}
}catch (Exception e) {
e.printStackTrace();
}
}
which returns
HTTP/1.1 200 OK [Date: Sat, 11 Jan 2014 16:17:22 GMT, Server: Apache, Expires: Thu, 19 Nov 1981 08:52:00 GMT, Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0, Pragma: no-cache, Vary: Accept-Encoding, Keep-Alive: timeout=10, max=30, Content-Type: text/html, Via: 1.1 NDC1-B2-CE01, Connection: keep-alive]
As if everything had worked fine but my php script on the other end doesn't seem to pick up the variable.
I've tried something as simple as:
<?php
error_log($_POST["Username"]);
?>
But get an index undefined error printed
You are setting the query parameters of the URI which builds a URI your URI like http://www.mysite.com/mypage.php?Username=userId&Password=pass
You need to set the parameters of the HttpPost with NameValuePair.
HttpPost post = new HttpPost("http://www.mysite.com/mypage.php");
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("Username", userId));
params.add(new BasicNameValuePair("Password", pass));
post.setEntity(new UrlEncodedFormEntity(params));
HttpResponse response = post.execute(post);
Additionally, I would recommend handling authentication with an Authorization header, such as Basic authentication as well as sending credentials over HTTPS.

how to set user name/password in httpget

I use Apache HttpComponents to access a web service, and don' know how to set user/password in the request, here is my code:
URI url = new URI(query);
HttpGet httpget = new HttpGet(url);
DefaultHttpClient httpclient = new DefaultHttpClient();
Credentials defaultcreds = new UsernamePasswordCredentials("test", "test");
httpclient.getCredentialsProvider().setCredentials(new AuthScope(HOST, AuthScope.ANY_PORT), defaultcreds);
HttpResponse response = httpclient.execute(httpget);
..
but still it got the 401 unauthorized error.
HTTP/1.1 401 Unauthorized [Server: Apache-Coyote/1.1, Pragma: No-cache, Cache-Control: no-cache, Expires: Wed, 31 Dec 1969 16:00:00 PST, WWW-Authenticate: Basic realm="MemoryRealm", Content-Type: text/html;charset=utf-8, Content-Length: 954, Date: Wed, 04 Apr 2012 02:28:49 GMT]
I m not sure if its the right way to set user/password? anyone can help? thanks.
I think you are on the right track. Perhaps you should check your user credential as the http error response could probably means incorrect username/password or the user does not have the privilege to access the resources. I have the below code which I do basic http authentication and it is working fine.
import org.apache.http.HttpResponse;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
public class Authentication
{
public static void main(String[] args)
{
DefaultHttpClient dhttpclient = new DefaultHttpClient();
String username = "abc";
String password = "def";
String host = "abc.example.com";
String uri = "http://abc.example.com/protected";
try
{
dhttpclient.getCredentialsProvider().setCredentials(new AuthScope(host, AuthScope.ANY_PORT), new UsernamePasswordCredentials(username, password));
HttpGet dhttpget = new HttpGet(uri);
System.out.println("executing request " + dhttpget.getRequestLine());
HttpResponse dresponse = dhttpclient.execute(dhttpget);
System.out.println(dresponse.getStatusLine() );
}
catch (Exception e)
{
e.printStackTrace();
}
finally
{
dhttpclient.getConnectionManager().shutdown();
}
}
}

Categories