We Keep Coding

DefaultHttpClient execute bring different location from browser

its seems i have a bug but i cant find any clue in google.
i have the following code:
DefaultHttpClient httpClient = new DefaultHttpClient();
setupHTTPS(httpClient);
HttpContext context = new BasicHttpContext();
// set proxy data, if available
if (proxyHost != null) {
httpClient.getCredentialsProvider().setCredentials(new AuthScope(proxyHost, proxyPort), new UsernamePasswordCredentials(proxyUsername, proxyPassword));
HttpHost proxy = new HttpHost(proxyHost, proxyPort);
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
}
RedirectStrategy redirectStrategy = new DefaultRedirectStrategy();
httpClient.setRedirectStrategy(redirectStrategy);
HttpGet request = new HttpGet(url);
// allow circle redirect with limiting the numbers of redirects
httpClient.getParams().setParameter(ClientPNames.ALLOW_CIRCULAR_REDIRECTS, true);
httpClient.getParams().setParameter(ClientPNames.MAX_REDIRECTS, 50);
// set userAgent , if available
if(userAgent!=null){
setUserAgentForHttpClient(userAgent,httpClient);
}
HttpResponse response = httpClient.execute(request, context);
RedirectLocations locations = (RedirectLocations) context.getAttribute(DefaultRedirectStrategy.REDIRECT_LOCATIONS);
and i tried to run it on url: http://annemergmed.com/action/consumeSharedSessionAction?JSESSIONID=aaaiJSrshFbnYctW3Q4Rv&MAID=I1JsdfY3Lc%2FzCjqdBXuiSQ%3D%3D&SERVER=WZ6myaEXBLHj3ZzqSv9HPw%3D%3D&ORIGIN=378011271&RD=RD&acw=&utt=
but i got different location from what i see in firebug. totaly different! (i am runing firebug on privat browsing)
how can it be? what am i missing?

Is the ProxyFactory replacement in RESTEasy thread safe?

I developed a service in RESTEasy using ProxyFactory and ClientExecutor like this:
PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager();
DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, 5000);
HttpConnectionParams.setSoTimeout(params, 5000);
ClientExecutor clientExecutor = new ApacheHttpClient4Executor(httpClient);
MyClass client = ProxyFactory.create(MyClass.class, "http://www.example.com", clientExecutor);
It always worked perfectly. After RESTEasy deprecated both ClientExecutor and ProxyFactory, they provided a new ResteasyClient for external connections, but I don't know if this new ResteasyClient is threadsafe. This is the new sample code from the documentation:
ResteasyClient client = new ResteasyClientBuilder().build();
ResteasyWebTarget target = client.target("http://example.com/base/uri");
SimpleClient simple = target.proxy(SimpleClient.class);
UPDATE: I used the code with the ResteasyClient and I got many of these errors:
javax.ws.rs.ProcessingException: Unable to invoke request
Caused by
java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one.

We use this:
final ResteasyClient client = new ResteasyClientBuilder()
.connectionPoolSize(10)
.maxPooledPerRoute(5)
.build();
And after debugging I found out that (at least in our situation) the RESTEasy client uses the ThreadSafeClientConnManager by default so I think there is no need to specify a different one although according to the JavaDoc it is deprecated in favour of PoolingHttpClientConnectionManager (note the extra Http). But this has been fixed in RESTEasy client 3.0.5.Final: https://issues.jboss.org/browse/RESTEASY-948
It's a jungle of HTTP connection managers out there..

This worked for me. Just needed to find the hook into setting up the Apache HTTP engine. Mostly based on RestEasy 3.0.5.Final API
public static Object setupServiceProxy(#NotNull Class responseClass) {
ResteasyProviderFactory factory = ResteasyProviderFactory.getInstance();
ResteasyClientBuilder builder = new ResteasyClientBuilder().providerFactory(factory);
ResteasyClient client = builder.httpEngine(setupHttpDefaults()).build();
ResteasyWebTarget target = client.target(url);
return target.proxy(responseClass);
}
public static ClientHttpEngine setupHttpDefaults() {
PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager();
DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, 30000);
HttpConnectionParams.setSoTimeout(params, 30000);
BasicHttpContext localContext = new BasicHttpContext();
return new ApacheHttpClient4Engine(httpClient, localContext);
}

Trying to add a http parameter but it isn't working

HttpClient httpclient = new DefaultHttpClient();
try {
HttpPost httpMethod = new HttpPost(this.transformURL(request));
BasicHttpParams params = new BasicHttpParams();
params.setParameter("name", name);
httpMethod.setParams(params);
ResponseHandler<String> responseHandler = new BasicResponseHandler();
httpclient.execute(httpMethod, responseHandler);
}catch{
LOG.error("Error");
} finally {
httpclient.getConnectionManager().shutdown();
}
I have the above code, and I'm trying to pass in a name variable as a paramter to get picked up in another method by request.getParameter("name").
It doesn't seem to be working, when I debug I can see the parameters get set but when I follow it through to the next method that gets executed, it doesn't pick up the parameters.
Any suggestions?
EDIT:
I added this and it worked great
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("name", request.getParameter("name")));
httpMethod.setEntity(new UrlEncodedFormEntity(nameValuePairs));

Did you check this example? it uses the class BasicNameValuePair instead of BasicHttpParams as you do.
Also, the documentation for the version 3.x of HttpClient does it:
PostMethod post = new PostMethod("http://jakarata.apache.org/");
NameValuePair[] data = {
new NameValuePair("user", "joe"),
new NameValuePair("password", "bloggs")
};
post.setRequestBody(data);
// execute method and handle any error responses.
...
InputStream in = post.getResponseBodyAsStream();
// handle response.
Update: The BasicHttpParams class is an implementation of the HttpParams interface, which as #Perception notes below, is a set of properties "that customize the behavior of the HTTP client". From the HttpParams javadoc: "HttpParams is expected to be used in 'write once - read many' mode. Once initialized, HTTP parameters are not expected to mutate in the course of HTTP message processing."

Apache HTTPClient does not make more than 2 connections

I've been trying to implement connection pooling for my application using Apache HTTPClient (v4.1). The problem is that the client always makes only two connections when run, though there are enough threads running parallel. I have been trying to modify the code for a while now, but nothing has helped yet.
I'm using ThreadSafeClientConnManager for connection pooling and set the MaxTotal and DefaulMaxPerRoute to values I want.
Is there anything that comes to your mind first that I might want to check?
Here's that code segment that I use to create the client.
DefaultHttpClient createClient() {
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, new Integer(60000));
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, new Integer(60000));
params.setBooleanParameter(CoreConnectionPNames.TCP_NODELAY, true);
SchemeRegistry registry = new SchemeRegistry();
registry.register(new Scheme("https", sf, 6443));
registry.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
ThreadSafeClientConnManager cm = new ThreadSafeClientConnManager(params, registry);
cm.setMaxTotal(2 * maxConnections);
cm.setDefaultMaxPerRoute(maxConnections);
HttpHost localhost = new HttpHost("localhost");
cm.setMaxForRoute(new HttpRoute(localhost), maxConnections);
HttpHost sdpTargetHost = new HttpHost("webserviceIP", webservicePort, "https");
cm.setMaxForRoute(new HttpRoute(sdpTargetHost, null, true), maxConnections);
return new DefaultHttpClient(cm, params);
}
The client returned by this function is used in Runnables managed by a ThreadPoolExecutor. The Runnables use the client, and has these lines:
HttpResponse response = httpClient.execute(httpPost, context);
HttpEntity entity = response.getEntity();
....
EntityUtils.consume(entity);
From what I know, the EntityUtils.consume(entity) will notify the connection manager that the connection is no longer used, and thus will release the connection to be used by other threads. So I'm guessing the connection management is alright.
I guess I've provided enough info, please tell me if I'm to add anything more.
Thanks

OK. I've found the solution, thanks to oleg for pointing out the logging, and to google and all the forums. All I had to do was define the class with only the connection manager, and then set HttpParams using HttpClient.setParams(). So the code will look something like this:
DefaultHttpClient createClient() {
SchemeRegistry registry = new SchemeRegistry();
registry.register(new Scheme("https", sf, 6443));
ThreadSafeClientConnManager cm = new ThreadSafeClientConnManager(registry);
cm.setMaxTotal(maxConnections);
cm.setDefaultMaxPerRoute(maxConnections);
HttpHost targetHost = new HttpHost("webserviceIP", webservicePort, "https");
cm.setMaxForRoute(new HttpRoute(targetHost, null, true), maxConnections);
return new DefaultHttpClient(cm);
}
And right before using the client,
DefaultHttpClient httpClient = createClient();
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, new Integer(60000));
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, new Integer(60000));
params.setBooleanParameter(CoreConnectionPNames.TCP_NODELAY, true);
httpClient.setParams(params);
There apparently is no difference in the code logically, but this fixed my problem. I presume this probably is some sorta bug in the HttpClient 4.1 API.

I was not able to set this option in cm
cm.setDefaultMaxPerRoute(maxConnections);
It was necessary to do so:
ConnPerRoute perRoute = new ConnPerRouteBean(100);
ConnManagerParams.setMaxConnectionsPerRoute(params, perRoute);
ConnManagerParams.setMaxTotalConnections(params, 100);
ConnManagerParams.setTimeout(params, 15000);

Apache HttpComponents HttpClient timeout

How do I set the connection timeout in httpcomponents httpclient? I have found the documentation at: http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html but it is not clear how these parameters are actually set.
Also, an explanation of the difference between SO_TIMEOUT and CONNECTION_TIMEOUT would be helpful.

In version 4.3 of Apache Http Client the configuration was refactored (again). The new way looks like this:
RequestConfig.Builder requestBuilder = RequestConfig.custom();
requestBuilder.setConnectTimeout(timeout);
requestBuilder.setConnectionRequestTimeout(timeout);
HttpClientBuilder builder = HttpClientBuilder.create();
builder.setDefaultRequestConfig(requestBuilder.build());
HttpClient client = builder.build();

In HttpClient 4.3 version you can use below example.. let say for 5 seconds
int timeout = 5;
RequestConfig config = RequestConfig.custom()
.setConnectTimeout(timeout * 1000)
.setConnectionRequestTimeout(timeout * 1000)
.setSocketTimeout(timeout * 1000).build();
CloseableHttpClient client =
HttpClientBuilder.create().setDefaultRequestConfig(config).build();
HttpGet request = new HttpGet("http://localhost:8080/service"); // GET Request
response = client.execute(request);

The answer from #jontro is correct, but it's always nice to have a code snippet on how to do this. There are two ways to do this:
Version 1: Set a 10 second timeout for each of these parameters:
HttpClient httpclient = new DefaultHttpClient();
// this one causes a timeout if a connection is established but there is
// no response within 10 seconds
httpclient.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, 10 * 1000);
// this one causes a timeout if no connection is established within 10 seconds
httpclient.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 10 * 1000);
// now do the execute:
HttpGet httpget = new HttpGet(URL);
HttpResponse response = httpclient.execute(httpget);
Version 2: Also set a 10 second timeout for each of these parameters:
HttpParams params = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(params, 10 * 1000);
HttpConnectionParams.setSoTimeout(params, 10 * 1000);
HttpClient httpclient = new DefaultHttpClient(params);
HttpGet httpget = new HttpGet(URL);
HttpResponse response = httpclient.execute(httpget);

In section 2.5 you see an example of how to set the CONNECTION_TIMEOUT parameter.
CONNECTION_TIMEOUT is the time waiting for the initial connection and SO_TIMEOUT is the timeout that you wait for when reading a packet after the connection is established.

I set a hard timeout for the entire request to workaround the java.net.SocketInputStream.socketRead0 problem.
private static final ScheduledExecutorService SCHEDULED_EXECUTOR = Executors.newSingleThreadScheduledExecutor()
HttpGet request = new HttpGet("http://www.example.com")
final Runnable delayedTask = new Runnable() {
#Override
public void run() {
request.abort()
}
}
SCHEDULED_EXECUTOR.schedule(delayedTask, 100000, TimeUnit.MILLISECONDS)