We Keep Coding

HttpClient redirecting

I am trying to write a video downloader from kissanime.to . I am using HttpClient library. This site is using cloudflare. It redirects after 5 secs. How can I set so my application will go to the redirected link? My code below isnt working. Where am I going wrong and how can I fix it.
HttpGet request = new HttpGet(url);
HttpClient httpClient = HttpClientBuilder.create()
.setRedirectStrategy(new LaxRedirectStrategy()).build();
HttpResponse response = httpClient.execute(request);
HttpEntity entity = response.getEntity();
String entityContents = EntityUtils.toString(entity);`

It means that given site is under DDOS protection mode (maybe you try to open it too often?) to workaround you would need to stop hitting it that much (e.g. wait some time between tries). Or if you insist use some javascript executing library (Rhino?) that would execute javascript that cloudflare is using.

HttpClientBuilder not working

I have a code in my java application that connects to a url with a post request and get response.
The code worked for a long time, and it still does for most people. I have one person that the code works for him most of the times, but sometimes I am getting timeout exception.
The code:
HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost("http://example.com/link-to-api");
post.addHeader( "Data1", "my_data" );
HttpResponse response = null;
BufferedReader rd = null;
try {
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("post_key1","post_data1"));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
response = client.execute(post);
} catch (IOException e) {
System.out.println("error: "+ e);
}
The exception that I get is:
org.apache.http.conn.HttpHostConnectException: Connect to example.com:80 [example.com/__IP__] failed: Connection timed out: connect
I sniffed the network, and I can see the packets when the connection is ok, but every few minutes, when I get the above exception, I don't even see the attempt connection in the sniffer.
I did ping and traceroute to the server while the timeout is occuring, and both are ok (I get response from the server)
So it seems to me that the problem is in the java application, not even attempting the connection.
I am using httpclient 4.4.1, if it is of any importance.
Why I am getting these timeouts?

I solved it . At least in my case, all I did was reset the router on my computer, and the problem disappeared.
I have read on the internet many suggestions to change server's tcp/ip parameters. So if this happens to you, before you do anything, I suggest that you check your hardware as well.

Handling HTTP request redirect in java

I'm writing a network android application that uses http requests to get data. The data is HTML format. I use Apache HttpClient and JSoup.
When I'm out of traffic with my mobile internet provider, I am always redirected to the providers' page saying that I should pay some money. Of course, it is a bad idea to parse this page.
How to detect occured page substitution?

This code will help you to know with is the final target of your request, if isn't the page that you asked for, is the provider page.
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("http://www.google.com/");
HttpResponse response = httpclient.execute(httpget, localContext);
HttpHost target = (HttpHost) localContext.getAttribute(
ExecutionContext.HTTP_TARGET_HOST);// this is the final page of the request
System.out.println("Final target: " + target);
HttpEntity entity = response.getEntity();
EntityUtils.consume(entity);
Thanks

If your provider is lying to you by immediately returning a 200 OK but not giving you the resource you've requested, your best option is probably to set a custom HTTP response header that your client can check before continuing.

Android error : MultipartEntity , request sent by the client was syntactically incorrect

I am suppose to send song (mp3/wav) file and some data through secure restful web service. I am using MultipartEntity to make HttpPost request. but When I execute it through HttpClient, the server replies this error
HTTP Status 400 - Bad Request
type: Status report
message : Bad Request
The request sent by the client was syntactically incorrect (Bad Request).
But the service is doing very well if we call it from its Web interface. please help
its the code
HttpClient httpclient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost();
try {
MultipartEntity reqEntity = new MultipartEntity();
reqEntity.addPart("email", new StringBody("test#testmail.com"));
reqEntity.addPart("password", new StringBody("123"));
reqEntity.addPart("title", new StringBody("My new song"));
reqEntity.addPart("musicData", new FileBody(new File(FilePath)));
// FIlePath is path to file and contains correct file location
postRequest.setEntity(reqEntity);
postRequest.setURI(new URI(ServiceURL));
HttpResponse response = httpclient.execute(postRequest);
} catch (URISyntaxException e) {
Log.e("URISyntaxException", e.toString());
}
I also included apache-mime4j, httpclient, httpcore and httpmime jars for MultipartEntity.
This is HTML page snap for the Service.

Try removing the setURI method and passing the URL in when you create your HttpPost object, as follows. This worked for me (more here).
HttpClient httpclient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost(ServiceURL);
try {
MultipartEntity reqEntity = new MultipartEntity();
reqEntity.addPart("email", new StringBody("test#testmail.com"));
reqEntity.addPart("password", new StringBody("123"));
reqEntity.addPart("title", new StringBody("My new song"));
reqEntity.addPart("musicData", new FileBody(new File(FilePath)));
postRequest.setEntity(reqEntity);
HttpResponse response = httpclient.execute(postRequest);
} catch (URISyntaxException e) {
Log.e("URISyntaxException", e.toString());
}

It seems header of the request is incorrect, this problem can occur if you use a different Auth protocol or upper/lower case or simply wrong things in header that server side can't handle.

Dont waste your time by trying different different combinations.There are some HTTP Request tools available for HTTP with which you can track request and response you are getting.Ex. HTTP Analyzer download trial version
Call URL from your working webinterface , copy request and response
then do same with from program the tool is enogh capable to capture your request and response data.
Now compare working and non working request you will surely able to dignose the issue whether it can be header issue or some authentication related issue.

How to emulate a browser HTTPS POST request with Java (Apache HTTP Client)?

There is a website with an AJAX API. I have opened Firebug to look into the details of the login HTTPS POST request.
Then I have tried to do the same POST request from my Java program using Apache HTTP Client. But somehow the server identified my request as a non browser request. It sends a security exception message, which tells me that.
When all request headers are the same, what else could identify my client as not a browser?

My guess is that it's a cookie issue (e.g. JSESSIONID the browser has stored). Include the session information with your POST. Have a look at the cookies of this site. Try disabling cookies for this site a have a look a the request again.

user-agent header? "httpclient.useragent" property
Use debug mode to see full wire logging and compare the request with firebug's one.

Dont know about the POST request but there is this for a multipart request
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
see if its of any help
EDIT: Code sample for a multipart request
String createOrderUrl = Constants.CREATE_ORDER_URL;
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(createOrderUrl);
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
// add the information to the multipart request
entity.addPart("msisdn", new StringBody("something"));
entity.addPart("recipientname", new StringBody("something"));
entity.addPart("recipientnumber", new StringBody("something"));
entity.addPart("recipientaddress", new StringBody("something"));
// add the images
for (String imagePath : selectedImages)
{
FileBody bin = new FileBody(new File(imagePath));
entity.addPart("image", bin);
}
httpPost.setEntity(entity);
return httpClient.execute(httpPost);