I am trying to write a video downloader from kissanime.to . I am using HttpClient library. This site is using cloudflare. It redirects after 5 secs. How can I set so my application will go to the redirected link? My code below isnt working. Where am I going wrong and how can I fix it.
HttpGet request = new HttpGet(url);
HttpClient httpClient = HttpClientBuilder.create()
.setRedirectStrategy(new LaxRedirectStrategy()).build();
HttpResponse response = httpClient.execute(request);
HttpEntity entity = response.getEntity();
String entityContents = EntityUtils.toString(entity);`
It means that given site is under DDOS protection mode (maybe you try to open it too often?) to workaround you would need to stop hitting it that much (e.g. wait some time between tries). Or if you insist use some javascript executing library (Rhino?) that would execute javascript that cloudflare is using.
Related
I'm doing a GET request with version 4.3.3 of Apache HttpClient, like this:
HttpGet httpGet = new HttpGet("http://www.revenue.ie/en/tax/it/forms/med1.pdf");
CloseableHttpClient client = HttpClients.createDefault();
CloseableHttpResponse response = client.execute(httpGet);
client.close();
The response status code tells me 200, and the content length as returned by response.getEntity().getContentLength() is 1213954, but the InputStream as returned from a call to:
response.getEntity().getContent()
...is reporting 0 bytes available.
I have been successfully making GET calls like this to retrieve and parse the HTML of other URLs, but is there something different I need to do here since it's file contents that I'm interested in?
The problem was that I was closing the http client too early i.e. client.close() before I tried to retrieve the response InputStream with a call to response.getEntity().getContent().
I am trying to get some data from a webpage that needs login in a page. I could do with htmlunit but gave me problems import these libraries on Android. So I am trying to do it with apache http client.
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("https://inside.cineca.it/cgi-bin/uinside/marcature.pl");
BasicNameValuePair usernameBasicNameValuePair = new BasicNameValuePair("j_username", "user");
BasicNameValuePair passwordBasicNameValuePAir = new BasicNameValuePair("j_password", "pass");
List<NameValuePair> nameValuePairList = new ArrayList<NameValuePair>();
nameValuePairList.add(usernameBasicNameValuePair);
nameValuePairList.add(passwordBasicNameValuePAir);
UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(nameValuePairList);
HttpResponse httpResponse = httpClient.execute(httpPost);
System.out.println(EntityUtils.toString(httpResponse.getEntity()));
But is giving me error. I am affraid i should manage the redirect but i don't know how. I would be happy if someone could give me some advice.
Best regards
You're performing an http POST to a login page. Maybe the server that serves you back the login page doesn't know to handle the post. Anyway, I am sure your intention is to perform the login actually. You need to figure out what is the server API where you should post the credentials and process the result. Performing a quick investigation with Chrome Network profiler, the login API for above link actually resides at: https://idp-is.cineca.it/idp/Authn/Multilogin. That is the URL that you should be using and there is where you need to POST.
Also, make sure the request you're making is done on a non-UI thread as this is a common mistake done by beginners.
I'm writing a network android application that uses http requests to get data. The data is HTML format. I use Apache HttpClient and JSoup.
When I'm out of traffic with my mobile internet provider, I am always redirected to the providers' page saying that I should pay some money. Of course, it is a bad idea to parse this page.
How to detect occured page substitution?
This code will help you to know with is the final target of your request, if isn't the page that you asked for, is the provider page.
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("http://www.google.com/");
HttpResponse response = httpclient.execute(httpget, localContext);
HttpHost target = (HttpHost) localContext.getAttribute(
ExecutionContext.HTTP_TARGET_HOST);// this is the final page of the request
System.out.println("Final target: " + target);
HttpEntity entity = response.getEntity();
EntityUtils.consume(entity);
Thanks
If your provider is lying to you by immediately returning a 200 OK but not giving you the resource you've requested, your best option is probably to set a custom HTTP response header that your client can check before continuing.
I am developing one application which is connecting to server to get some data.
In this I want to check first if application is connected to server or not. And then, if server is on or off? Based on the result I want to do my further manipulations.
So how do I get the result of server status?
Here is the code which I am using:
Code:
try {
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(
"http://192.168.1.23/sip_chat_api/getcountry.php");
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
is = entity.getContent();
} catch (Exception e) {
}
Maintaining session cookies is best choice here, please see how to use session cookie: How do I make an http request using cookies on Android?
here, before sending request to server, check for session cookie. If it exists, proceed for the communication.
Update:
The Java equivalent -- which I believe works on Android -- should be:
InetAddress.getByName(host).isReachable(timeOut)
Check getStatusLine() method of HttpResponse
any status code other than 200 means there is a problem , and each status codes points to different problems happened.
http://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/HttpResponse.html?is-external=true
http://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/StatusLine.html#getStatusCode()
I'm posting data to a website form using Apache's HttpClient class. The form is retrieved using the following lines of code:
HttpGet get = new HttpGet(url);
HttpResponse response = client.execute(get);
The website that I'm retrieving the form from requires authentication to access the form. If the request isn't authenticated, the website redirects the request to a login form page that will subsequently redirect back to the original page on successful authentication.
I want to cleanly detect whether or not the GET request returns the login page or the desired form page so that I can either POST login data or form data. The only way I can think of to do this is by reading from the content InputStream of the entity of the response and parsing each line. But that seems somewhat convoluted. I haven't worked with the Apache HttpComponents api before so I'm not sure if this would be the only and best way to accomplish what I want to accomplish.
EDIT: To clarify question, I'm asking if there is a set way to handle forms with Apache's HttpClient. I somewhat know how to achieve what I'm looking to do, but it looks very ugly and I'm hoping there is an easier and faster way to achieve it. For example, if there was some way to do the following:
HttpGet get = new HttpGet(url);
HttpResponse response = client.execute(get);
if(parseElements(response.getEntity()).hasFormWithId("login")) {
// post authentication data
} else {
// post actual form data
}
Because of my inexperience with Apache's HttpClient api, I'm not sure if what I'm looking for in the API is too abstract for the intent of the API.
You can modify the behavior of the HttpClient by setting the HttpClient Parameters
DefaultHttpClient client = new DefaultHttpClient();
client.setDefaultHttpParams(client.getParams().setBoolean(ClientPNames.HANDLE_REDIRECTS, false));
Which disables handling redirects automatically.
See also:
Automatic redirect handling
HTTP Authentication
DefaultHttpClient API