I'm writing a network android application that uses http requests to get data. The data is HTML format. I use Apache HttpClient and JSoup.
When I'm out of traffic with my mobile internet provider, I am always redirected to the providers' page saying that I should pay some money. Of course, it is a bad idea to parse this page.
How to detect occured page substitution?
This code will help you to know with is the final target of your request, if isn't the page that you asked for, is the provider page.
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("http://www.google.com/");
HttpResponse response = httpclient.execute(httpget, localContext);
HttpHost target = (HttpHost) localContext.getAttribute(
ExecutionContext.HTTP_TARGET_HOST);// this is the final page of the request
System.out.println("Final target: " + target);
HttpEntity entity = response.getEntity();
EntityUtils.consume(entity);
Thanks
If your provider is lying to you by immediately returning a 200 OK but not giving you the resource you've requested, your best option is probably to set a custom HTTP response header that your client can check before continuing.
Related
I'm doing a GET request with version 4.3.3 of Apache HttpClient, like this:
HttpGet httpGet = new HttpGet("http://www.revenue.ie/en/tax/it/forms/med1.pdf");
CloseableHttpClient client = HttpClients.createDefault();
CloseableHttpResponse response = client.execute(httpGet);
client.close();
The response status code tells me 200, and the content length as returned by response.getEntity().getContentLength() is 1213954, but the InputStream as returned from a call to:
response.getEntity().getContent()
...is reporting 0 bytes available.
I have been successfully making GET calls like this to retrieve and parse the HTML of other URLs, but is there something different I need to do here since it's file contents that I'm interested in?
The problem was that I was closing the http client too early i.e. client.close() before I tried to retrieve the response InputStream with a call to response.getEntity().getContent().
There is a website with an AJAX API. I have opened Firebug to look into the details of the login HTTPS POST request.
Then I have tried to do the same POST request from my Java program using Apache HTTP Client. But somehow the server identified my request as a non browser request. It sends a security exception message, which tells me that.
When all request headers are the same, what else could identify my client as not a browser?
My guess is that it's a cookie issue (e.g. JSESSIONID the browser has stored). Include the session information with your POST. Have a look at the cookies of this site. Try disabling cookies for this site a have a look a the request again.
user-agent header? "httpclient.useragent" property
Use debug mode to see full wire logging and compare the request with firebug's one.
Dont know about the POST request but there is this for a multipart request
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
see if its of any help
EDIT: Code sample for a multipart request
String createOrderUrl = Constants.CREATE_ORDER_URL;
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(createOrderUrl);
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
// add the information to the multipart request
entity.addPart("msisdn", new StringBody("something"));
entity.addPart("recipientname", new StringBody("something"));
entity.addPart("recipientnumber", new StringBody("something"));
entity.addPart("recipientaddress", new StringBody("something"));
// add the images
for (String imagePath : selectedImages)
{
FileBody bin = new FileBody(new File(imagePath));
entity.addPart("image", bin);
}
httpPost.setEntity(entity);
return httpClient.execute(httpPost);
I need to coding an Http Client using java which interact with a stateful http server. The client needs to
navigate to login page and accept cookies
submit login page with http form field filled
select goods and add to shopping cart
submit shopping cart
I am trying to use HttpClient to implement this client. However I found even I submitted the login form, it still return the login form just like my submit is invalid. Here is my code:
HttpClient agent = new DefaultHttpClient();
agent.getParams().setParameter(ClientPNames.COOKIE_POLICY,
CookiePolicy.RFC_2965);
CookieStore cookieStore = new BasicCookieStore();
HttpContext localContext = new BasicHttpContext();
localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
HttpGet httpget = new HttpGet(site);
HttpResponse response = agent.execute(httpget, localContext);
HttpEntity entity = response.getEntity();
entity.getContent().close();
HttpPost post = new HttpPost(site + "/login.aspx");
post.getParams().setParameter("LoginControl1$ctlLoginName", "myusername");
post.getParams().setParameter("LoginControl1$ctlPassword", "mypassword");
response = agent.execute(post, localContext);
entity = response.getEntity();
String s = IO.readContentAsString(entity.getContent());
System.out.println(s);
Any idea where I am wrong? Or do you have better way to implement this?
Thanks a lot
Green
Not really sure what might be wrong, but have you considered using tcpdump/Wireshark to grab the raw HTTP conversation with the server and compare what you're sending/receiving in HTTPClient with what you send/receive in your web browser when you submit the form?
I using HttpClient api to authenticate to a web site:
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, 443),
new UsernamePasswordCredentials(args[0], args[1]));
HttpGet httpget = new HttpGet("http://..........");
HttpResponse response = httpclient.execute(httpget);
System.out.println(response.getStatusLine());
if (entity != null) {
System.out.println("Response content length: "
+ entity.getContentLength());
}
I have this answer:
HTTP/1.1 403 Forbidden
Response content length: -1
But with a browser i have access to this page with the same login and password !!!!
How can i fix this problem ?
You construct the AuthScope object with the port parameter set to 443 (default port for HTTPS). However, you create the HttpGet object with the URL pointing to HTTP (with default port 80).
Either try to construct the AuthScope using:
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT)
or make sure that ports will match.
You need to look carefully at how the browser is actually authenticating.
What you are trying to do is (I think) send the credentials using HTTP Basic Authentication. If the site is set up to only allow form-based authentication and a session cookie, then it will ignore the header containing the credentials.
Check if the Version of the HttpClient you are using is whats causing the 403.
Try
HttpClient httpClient = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.build();
I'm using DefaultHttpClient to make an http connection. I [think] we can set the preferred locale in the http headers [http accept-language] when making a connection, which the server can check and send back content in a matching language (if it wants).
Anyone know if this is possible, or how to go about doing this with DefaultHttpClient?
Thanks
You have to add your header to HttpRequest object
HttpClient client = new DefaultHttpClient();
HttpPost request = new HttpPost(URL);
request.addHeader("Accept-Language", "en");
HttpResponse response = client.execute(request);