I'm adding a cookie to Closeable HttpClient object and attempt to retrieve the same at the Server as below.
BasicClientCookie mCookie = new BasicClientCookie("myCookie", "dummyValue");
mCookie.setDomain(".myCookie.net");
mCookie.setPath("/");
mCookie.setSecure(true);
mCookie.setAttribute(ClientCookie.PATH_ATTR, "/");
mCookie.setAttribute(ClientCookie.DOMAIN_ATTR, ".myCookie.net");
mfCookies[0] = mCookie;
BasicCookieStore basicCookieStore = new BasicCookieStore();
basicCookieStore.addCookies(mfCookies);
CloseableHttpClient httpClient = HttpClients.custom()
.setDefaultRequestConfig(
RequestConfig.custom().setConnectTimeout(connectionTimeOut).build())
.setDefaultCookieStore(basicCookieStore)
.setConnectionManager(poolingConnectionManager)
.build();
I connect to local url using HttpGet object as below
httpGet = new HttpGet(myLocalUrl);
httpResponse = httpClient.execute(httpGet);
HttpClient hits the local url correctly connecting to Rest end point on local host but when I try to retrieve cookies using code below
(HttpServletRequest) request.getCookies() ;//returns null
I get null.
Can someone please help - I'm using spring boot to make a Rest end point (server) and attempt to retrive the cookie.
I figured it out myself, actually I was setting the domain as .myCookie.net while I was connecting to http://localhost:8080/..., if running on local, the localhost should resolve to the domain name by dns so in this case .myCookie.net should be mapped to localhost in file /etc/hosts file.
eg set 127.0.0.1 www.myCookie.net in etc/hosts file and hit the url http://www.myCookie.net/...
If the url the httpclient connects to and the domain set in the cookie dont match, httpclient will just drop the cookies.
I'm doing a GET request with version 4.3.3 of Apache HttpClient, like this:
HttpGet httpGet = new HttpGet("http://www.revenue.ie/en/tax/it/forms/med1.pdf");
CloseableHttpClient client = HttpClients.createDefault();
CloseableHttpResponse response = client.execute(httpGet);
client.close();
The response status code tells me 200, and the content length as returned by response.getEntity().getContentLength() is 1213954, but the InputStream as returned from a call to:
response.getEntity().getContent()
...is reporting 0 bytes available.
I have been successfully making GET calls like this to retrieve and parse the HTML of other URLs, but is there something different I need to do here since it's file contents that I'm interested in?
The problem was that I was closing the http client too early i.e. client.close() before I tried to retrieve the response InputStream with a call to response.getEntity().getContent().
I'm writing a network android application that uses http requests to get data. The data is HTML format. I use Apache HttpClient and JSoup.
When I'm out of traffic with my mobile internet provider, I am always redirected to the providers' page saying that I should pay some money. Of course, it is a bad idea to parse this page.
How to detect occured page substitution?
This code will help you to know with is the final target of your request, if isn't the page that you asked for, is the provider page.
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("http://www.google.com/");
HttpResponse response = httpclient.execute(httpget, localContext);
HttpHost target = (HttpHost) localContext.getAttribute(
ExecutionContext.HTTP_TARGET_HOST);// this is the final page of the request
System.out.println("Final target: " + target);
HttpEntity entity = response.getEntity();
EntityUtils.consume(entity);
Thanks
If your provider is lying to you by immediately returning a 200 OK but not giving you the resource you've requested, your best option is probably to set a custom HTTP response header that your client can check before continuing.
I'm trying to use the Apache/Jakarta HttpClient 4.1.1 to connect to an arbitrary web page using the given credentials. To test this, I have a minimal install of IIS 7.5 on my dev machine running where only one authentication mode is active at a time. Basic authentication works fine, but Digest and NTLM return 401 error messages whenever I try to log in. Here is my code:
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("http://localhost/");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY,
new NTCredentials("user", "password", "", "localhost"));
if (!new File(System.getenv("windir") + "\\krb5.ini").exists()) {
List<String> authtypes = new ArrayList<String>();
authtypes.add(AuthPolicy.NTLM);
authtypes.add(AuthPolicy.DIGEST);
authtypes.add(AuthPolicy.BASIC);
httpclient.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF,
authtypes);
httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF,
authtypes);
}
localContext.setAttribute(ClientContext.CREDS_PROVIDER, credsProvider);
HttpResponse response = httpclient.execute(httpget, localContext);
System.out.println("Response code: " + response.getStatusLine());
The one thing I've noticed in Fiddler is that the hashes sent by Firefox versus by HttpClient are different, making me think that maybe IIS 7.5 is expecting stronger hashing than HttpClient provides? Any ideas? It'd be great if I could verify that this would work with NTLM. Digest would be nice too, but I can live without that if necessary.
I am not an expert on the subject but during the NTLM authentication using http components I have seen that the client needs 3 attempts in order to connect to an NTML endpoint in my case. It is kinda described here for Spnego but it is a bit different for the NTLM authentication.
For NTLM in the first attempt client will make a request with Target auth state: UNCHALLENGED and Web server returns HTTP 401 status and a header: WWW-Authenticate: NTLM
Client will check for the configured Authentication schemes, NTLM should be configured in client code.
Second attempt, client will make a request with Target auth state: CHALLENGED, and will send an authorization header with a token encoded in base64 format: Authorization: NTLM TlRMTVNTUAABAAAAAYIIogAAAAAoAAAAAAAAACgAAAAFASgKAAAADw==
Server again returns HTTP 401 status but the header: WWW-Authenticate: NTLM now is populated with encoded information.
3rd Attempt Client will use the information from WWW-Authenticate: NTLM header and will make the final request with Target auth state: HANDSHAKE and an authorisation header Authorization: NTLM which contains more information for the server.
In my case I receive an HTTP/1.1 200 OK after that.
In order to avoid all this in every request documentation at chapter 4.7.1 states that the same execution token must be used for logically related requests. For me it did not worked.
My code:
I initialize the client once in a #PostConstruct method of an EJB
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
cm.setMaxTotal(18);
cm.setDefaultMaxPerRoute(6);
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(30000)
.setConnectTimeout(30000)
.setTargetPreferredAuthSchemes(Arrays.asList(AuthSchemes.NTLM))
.setProxyPreferredAuthSchemes(Arrays.asList(AuthSchemes.BASIC))
.build();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new NTCredentials(userName, password, hostName, domainName));
// Finally we instantiate the client. Client is a thread safe object and can be used by several threads at the same time.
// Client can be used for several request. The life span of the client must be equal to the life span of this EJB.
this.httpclient = HttpClients.custom()
.setConnectionManager(cm)
.setDefaultCredentialsProvider(credentialsProvider)
.setDefaultRequestConfig(requestConfig)
.build();
Use the same client instance in every request:
HttpPost httppost = new HttpPost(endPoint.trim());
// HttpClientContext is not thread safe, one per request must be created.
HttpClientContext context = HttpClientContext.create();
response = this.httpclient.execute(httppost, context);
Deallocate the resources and return the connection back to connection manager, at the #PreDestroy method of my EJB:
this.httpclient.close();
I had the same problem with HttpClient4.1.X After upgrading it to
HttpClient 4.2.6 it woked like charm. Below is my code
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("url");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY,
new NTCredentials("username", "pwd", "", "domain"));
List<String> authtypes = new ArrayList<String>();
authtypes.add(AuthPolicy.NTLM);
httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF,authtypes);
localContext.setAttribute(ClientContext.CREDS_PROVIDER, credsProvider);
HttpResponse response = httpclient.execute(httpget, localContext);
HttpEntity entity=response.getEntity();
The easiest way troubleshoot such situations I found is Wireshark. It is a very big hammer, but it really will show you everything. Install it, make sure your server is on another machine (does not work with Localhost) and start logging.
Run your request that fails, run one that works. Then, filter by http (just put http in the filter field), find the first GET request, find the other GET request and compare. Identify meaningful difference, you now have specific keywords or issues to search code/net for. If not enough, narrow down to first TCP conversation and look at full request/response. Same with the other one.
I solved an unbelievable number of problems with that approach. And Wireshark is very useful tool to know. Lots of super-advanced functions to make your network debugging easier.
You can also run it on either client or server end. Whatever will show you both requests to allow you to compare.
I had a similar problem with HttpClient 4.1.2. For me, it was resolved by reverting to HttpClient 4.0.3. I could never get NTLM working with 4.1.2 using either the built-in implementation or using JCIFS.
Updating our application to use the jars in the httpcomponents-client-4.5.1 resolved this issue for me.
I finally figured it out. Digest authentication requires that if you use a full URL in the request, the proxy also needs to use the full URL. I did not leave the proxy code in the sample, but it was directed to "localhost", which caused it to fail. Changing this to 127.0.0.1 made it work.
I'm using DefaultHttpClient to make an http connection. I [think] we can set the preferred locale in the http headers [http accept-language] when making a connection, which the server can check and send back content in a matching language (if it wants).
Anyone know if this is possible, or how to go about doing this with DefaultHttpClient?
Thanks
You have to add your header to HttpRequest object
HttpClient client = new DefaultHttpClient();
HttpPost request = new HttpPost(URL);
request.addHeader("Accept-Language", "en");
HttpResponse response = client.execute(request);