Should I regenerate cookies CookieStore using apache HttpClient? - java

I need to login to external web site, and use HttpClient in combination with CookieStore. As a guide I used http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientCustomContext.java and http://hc.apache.org/httpcomponents-client-4.4.x/tutorial/html/statemgmt.html
Here is my code:
RequestConfig globalConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.BEST_MATCH).build();
CookieStore cookieStore = new BasicCookieStore();
HttpClientContext context = HttpClientContext.create();
context.setCookieStore(cookieStore);
CloseableHttpClient httpClient = HttpClients.custom()
.setDefaultRequestConfig(globalConfig)
.setDefaultCookieStore(cookieStore)
.build();
HttpGet httpGet = new HttpGet(urlAddress);
httpGet.addHeader("Accept", "text/html, application/xhtml+xml, */*");
httpGet.addHeader("Accept-Language", "en-US,en;q=0.7,ru;q=0.3");
httpGet.addHeader("User-Agent", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)");
httpGet.addHeader("Accept-Encoding","gzip, deflate");
//httpGet.addHeader("Host", host); // should I copy host here?
httpGet.addHeader("Connection", "Keep-Alive");
CloseableHttpResponse response1 = httpClient.execute(httpGet);
List<Cookie> cookies1 = context.getCookieStore().getCookies();
try {
System.out.println(response1.getStatusLine());
HttpEntity entity1 = response1.getEntity();
// do something useful with the response body
streamToFile(entity1.getContent(), "./login_result_1.html");
// and ensure it is fully consumed
EntityUtils.consume(entity1);
} finally {
response1.close();
}
That works fine.
When I do further request to the site, should I do any special code to pass cookies obtained in the request above?
Should I regenerate cookies or that will be done by Apache?
I was not able to find clear answer. Also my further request is not handled by web site properly and it doesn't allow me to login.
HttpPost httpPost = new HttpPost(urlAddress);
httpPost.addHeader("Accept", "text/html, application/xhtml+xml, */*");
httpPost.addHeader("Accept-Language", "en-US,en;q=0.7,ru;q=0.3");
httpPost.addHeader("Referer", "http://virtonomica.ru/");
httpPost.addHeader("User-Agent", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)");
httpPost.addHeader("Accept-Encoding","gzip, deflate");
httpPost.addHeader("Content-Type", "application/x-www-form-urlencoded");
httpPost.addHeader("Accept-Encoding", "gzip, deflate");
//httpPost.addHeader("Host", "virtonomica.ru");
//httpPost.addHeader("Content-Length", "51");//LENGTH
httpPost.addHeader("Connection", "Keep-Alive");
httpPost.addHeader("Cache-Control", "no-cache");
httpPost.setEntity(new UrlEncodedFormEntity(nvps)); // nvps is List<NameValuePair> which contains POST data
CloseableHttpResponse response = httpClient.execute(httpPost);
Please advise.

Related

Upload File in form data using Java

I am trying to perform a HTTP Post Request in Java using the Apache API.
With curl the request looks like this
curl https://host/upload
-X POST
-H "Authorization: Bearer xxx"
-H "Content-Type: multipart/form-data"
-H "Accept: application/json"
-F "file=#{PathToImage}" -F "type=file"
While this work fine when running it with CURL the server returns a 500er result when running it with the following Java code
final HttpPost httppost = new HttpPost("https://host/upload");
httppost.addHeader("Authorization", "Bearer xxx");
httppost.addHeader("Accept", "application/json");
httppost.addHeader("Content-Type", "multipart/form-data");
final MultipartEntityBuilder builder = MultipartEntityBuilder.create();
final File file = new File("c:\\tmp\\myfile.pdf");
builder.addBinaryBody("file", file);
builder.addTextBody("type", "file");
final HttpEntity entity = builder.build();
httppost.setEntity(entity);
final HttpResponse response = httpclient.execute(httppost);
httpclient.close();
Any idea what I am missing here?
This question is similar. But I believe the answer is changing your addBinary to addPart.
final HttpPost httppost = new HttpPost("https://host/upload");
httppost.addHeader("Authorization", "Bearer xxx");
httppost.addHeader("Accept", "application/json");
httppost.addHeader("Content-Type", "multipart/form-data");
final MultipartEntityBuilder builder = MultipartEntityBuilder.create();
final File file = new File("c:\\tmp\\myfile.pdf");
builder.addPart("file", new FileBody(file));
builder.addTextBody("type", "file");
final HttpEntity entity = builder.build();
httppost.setEntity(entity);
final HttpResponse response = httpclient.execute(httppost);
httpclient.close();
Try syntax as baeldung multipart upload article suggest:
String textFileName = "c:\\tmp\\myfile.pdf";
final File file = new File(textFileName);
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
builder.addBinaryBody("file", file, ContentType.DEFAULT_BINARY, textFileName);
builder.addTextBody("type", "file", ContentType.DEFAULT_BINARY);
create a multipart entity is to use the addBinaryBody and AddTextBody methods. These methods work for uploading text, files, character arrays, and InputStream objects.
Try something like this instead
//import javax.net.ssl.HttpsURLConnection;
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
String encoding = Base64.getEncoder().encodeToString((user + ":" + psw).getBytes("UTF-8"));
con.setRequestProperty("Authorization", String.format("Basic %s", encoding));
con.setDoOutput(true);
con.setDoInput(true);
con.setRequestProperty("Content-Type", "application/xml; charset=UTF-8");
con.setRequestMethod("POST");
con.setRequestProperty("User-Agent", "Java client");
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.write(val);

How to handle the session in HttpUnit

try{
HttpEntity entity=null;
HttpGet httpget=null;
HttpResponse response=null;
httpclient = new DefaultHttpClient();
httpget = new HttpGet(credentialsURL);
httpget.setHeader(HttpHeaders.USER_AGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2");
response = httpclient.execute(httpget);
entity = response.getEntity();
if (entity != null) {
EntityUtils.consume(entity);
}
}
catch(Exception e){
}
return httpclient;
I am using HttpClient to handle session.Here is my code to handle session using httpclient. But httpclient won't handle ajax calls.
I am planning to move to httpunit to overcome my problem.
Basically I have a referral URL(or authentication server) from which I need to get cookies and session and store it in client.
And use this client across website to get logged user information.
What is the appropriate solution for this using httpunit.
credentialsURL="https://www.,,,,.com./sapLogin.aspx?HOOK_URL=https://authentication.server.address&username=&password=
";

Exception org.apache.http.client.RedirectException: Maximum redirects (50) exceeded

When am trying to hit the following URL :
http://www.wsj.com/video/khorasan-meet-the-new-us-terrorist-target/157FA4EF-D44F-49BA-938B-002A91A090A3.html
with HttpClient, am getting the following exception =
org.apache.http.client.RedirectException: Maximum redirects (50) exceeded
Code Snippet :
HttpClient httpclient = HttpClientBuilder.create().build();
RequestConfig config = RequestConfig.custom().setRedirectsEnabled(true).setCircularRedirectsAllowed(true).build();
HttpGet httpGet = new HttpGet(("http://www.wsj.com/video/khorasan-meet-the-new-us-terrorist-target/157FA4EF-D44F-49BA-938B-002A91A090A3.html"));
httpGet.setConfig(config);
httpGet.addHeader("Host", "online.wsj.com");
httpGet.addHeader("User-Agent", USER_AGENT);
httpGet.addHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
httpGet.addHeader("Accept-Encoding", "gzip, deflate");
httpGet.addHeader("Accept-Language", "en-US,en;q=0.5");
httpGet.addHeader("Connection", "keep-alive");
httpGet.addHeader("Content-Type", "application/x-www-form-urlencoded");
HttpResponse httpResponse = httpclient.execute(httpGet);
StringWriter writer = new StringWriter();
IOUtils.copy(httpResponse.getEntity().getContent(), writer, "UTF-8");
String theString = writer.toString();
System.out.println(theString);
Any way out to resolve it ? Would be of great help !

Execute GET/POST on Google App Engine using UrlFetchTransport

I'm porting an Android app on GAE.
The app compile and post a login form to enter in a
password protected site.
I'm using UrlFetchTransport instead of DefaultHttpClient.
I can get web page without problem using the following piece of code:
HttpTransport HTTP_TRANSPORT = new UrlFetchTransport();
HttpRequestFactory requestFactory = HTTP_TRANSPORT.createRequestFactory();
GenericUrl genericUrl = new GenericUrl(url);
HttpRequest request = requestFactory.buildGetRequest(genericUrl);
com.google.api.client.http.HttpHeaders httpHeaders = request.getHeaders();
httpHeaders.setUserAgent("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36");
httpHeaders.setAccept("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
String paginaRicevuta = request.execute().parseAsString();
I would try a POST request using the following:
UrlFetchTransport HTTP_TRANSPORT = new UrlFetchTransport();
HttpRequestFactory requestFactory = HTTP_TRANSPORT.createRequestFactory();
GenericUrl genericUrl = new GenericUrl("https://www.......");
List<BasicNameValuePair> nvps = new ArrayList<BasicNameValuePair>();
nvps.add(new BasicNameValuePair("ReturnUrl", ""));
nvps.add(new BasicNameValuePair("userName", "username"));
nvps.add(new BasicNameValuePair("password","password"));
//HERE HOW DEFINE httpContent????
HttpContent httpContent=null;
final HttpRequest postRequest = requestFactory.buildPostRequest(genericUrl,httpContent);
postRequest.setContent(httpContent);
postRequest.setFollowRedirects(true);
HttpHeaders httpHeaders = postRequest.getHeaders();
httpHeaders.setUserAgent("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36");
httpHeaders.setAccept("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
httpHeaders.setContentType("application/x-www-form-urlencoded");
postRequest.setHeaders(httpHeaders);
String pagePost = postRequest.execute().parseAsString();
but I don't know how format a right POST request?
Thanks for your help.

How to send XML POST request using Apache HttpClient?

I want to do a HTTP POST of the format as follows,
<?xml version="1.0" encoding="UTF-8" ?>
<authRequest>
<username>someusernamehere</username>
<password>somepasswordhere</password>
</authRequest>
I usually work with the following mechanism for any login based POST,
HttpParams params = new BasicHttpParams();
params.setParameter(
"http.useragent",
"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2) Gecko/20100115 Firefox/3.6");
DefaultHttpClient httpclient = new DefaultHttpClient(params);
HttpPost httppost = new HttpPost("http://mysite.com/login");
List<NameValuePair> formparams = new ArrayList<NameValuePair>();
formparams.add(new BasicNameValuePair("username", "stackoverflow"));
formparams.add(new BasicNameValuePair("password", "12345"));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formparams, "UTF-8");
httppost.setEntity(entity);
HttpResponse httpresponse = httpclient.execute(httppost);
But with this way, the POST data will be look like,
username=stackoverflow&password=12345
How can I format this request as per the specified XML format that I mentioned above?
Thanks in advance.
Use a different kind of HttpEntity. There are a number of implementations listed at the top of the documentation.

Categories