I developed a service in RESTEasy using ProxyFactory and ClientExecutor like this:
PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager();
DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, 5000);
HttpConnectionParams.setSoTimeout(params, 5000);
ClientExecutor clientExecutor = new ApacheHttpClient4Executor(httpClient);
MyClass client = ProxyFactory.create(MyClass.class, "http://www.example.com", clientExecutor);
It always worked perfectly. After RESTEasy deprecated both ClientExecutor and ProxyFactory, they provided a new ResteasyClient for external connections, but I don't know if this new ResteasyClient is threadsafe. This is the new sample code from the documentation:
ResteasyClient client = new ResteasyClientBuilder().build();
ResteasyWebTarget target = client.target("http://example.com/base/uri");
SimpleClient simple = target.proxy(SimpleClient.class);
UPDATE: I used the code with the ResteasyClient and I got many of these errors:
javax.ws.rs.ProcessingException: Unable to invoke request
Caused by
java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one.
We use this:
final ResteasyClient client = new ResteasyClientBuilder()
.connectionPoolSize(10)
.maxPooledPerRoute(5)
.build();
And after debugging I found out that (at least in our situation) the RESTEasy client uses the ThreadSafeClientConnManager by default so I think there is no need to specify a different one although according to the JavaDoc it is deprecated in favour of PoolingHttpClientConnectionManager (note the extra Http). But this has been fixed in RESTEasy client 3.0.5.Final: https://issues.jboss.org/browse/RESTEASY-948
It's a jungle of HTTP connection managers out there..
This worked for me. Just needed to find the hook into setting up the Apache HTTP engine. Mostly based on RestEasy 3.0.5.Final API
public static Object setupServiceProxy(#NotNull Class responseClass) {
ResteasyProviderFactory factory = ResteasyProviderFactory.getInstance();
ResteasyClientBuilder builder = new ResteasyClientBuilder().providerFactory(factory);
ResteasyClient client = builder.httpEngine(setupHttpDefaults()).build();
ResteasyWebTarget target = client.target(url);
return target.proxy(responseClass);
}
public static ClientHttpEngine setupHttpDefaults() {
PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager();
DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, 30000);
HttpConnectionParams.setSoTimeout(params, 30000);
BasicHttpContext localContext = new BasicHttpContext();
return new ApacheHttpClient4Engine(httpClient, localContext);
}
Related
I am trying to set proxy for a request I am making using HttpClientBuilder as follows:
CredentialsProvider credsProvider = new BasicCredentialsProvider();
UsernamePasswordCredentials usernamePasswordCredentials = new UsernamePasswordCredentials(proxyUser, proxyPassword);
credsProvider.setCredentials(new AuthScope(proxyHost, proxyPort), usernamePasswordCredentials);
builder.useSystemProperties();
builder.setProxy(new HttpHost(proxyHost, proxyPort));
builder.setDefaultCredentialsProvider(credsProvider);
builder.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy());
where builder is:
HttpClientBuilder builder = HttpClientBuilder.create();
However, I get this exception when I execute this request:
java.lang.RuntimeException: org.apache.http.conn.UnsupportedSchemeException: http protocol is not supported
Caused by: org.apache.http.conn.UnsupportedSchemeException: http protocol is not supported
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:108) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:338) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:388) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) ~[httpclient-4.5.1.jar:4.5.1]
(exception shortened for brevity)
Since this is an HTTP proxy, I don't want to change the scheme to HTTPS, which anyways won't work. How do I get this working?
java.lang.RuntimeException:
org.apache.http.conn.UnsupportedSchemeException: http protocol is not
supported
Why this problem occurs?
Ans: This actually happens because you forget to register a connection socket factory for the 'http' scheme.
Plain 'http' scheme must be used to establish an intermediate connection
to the proxy itself before 'https' tunneling could be employed.
For operational purpose, you can try this code:
CloseableHttpClient client = HttpClients.custom()
.setRoutePlanner(new
SystemDefaultRoutePlanner(ProxySelector.getDefault()))
.build();
I would also suggest simple code for your research. Hope it can save you.
ClientExecuteProxy.java
package org.apache.http.examples.client;
import org.apache.http.HttpHost;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
/**
* How to send a request via proxy.
*
* #since 4.0
*/
public class ClientExecuteProxy {
public static void main(String[] args)throws Exception {
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpHost target = new HttpHost("httpbin.org", 443, "https");
HttpHost proxy = new HttpHost("127.0.0.1", 8080, "http");
RequestConfig config = RequestConfig.custom()
.setProxy(proxy)
.build();
HttpGet request = new HttpGet("/");
request.setConfig(config);
System.out.println("Executing request " + request.getRequestLine() + " to " + target + " via " + proxy);
CloseableHttpResponse response = httpclient.execute(target, request);
try {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
System.out.println(EntityUtils.toString(response.getEntity()));
} finally {
response.close();
}
} finally {
httpclient.close();
}
}
}
Are you using using CloudantClient java API for Cloudant DB?
Ans:
If YES, then It turned out the issue with HTTP when setting a proxy was a bug at our end (sorry about that). We released 1.2.1 with the fix for this problem. You can download jar file from here. (Collected from mike-rhodes's answer)
UPDATE
How do I specify the credentials for the proxy here?
From HTTP authentication,
By default, httpclient will not provide credentials preemptively, it will first create a HTTP request without authentication parameters. This is by design, as a security precaution, and as part of the spec. But, this causes issues if you don't retry the connection, or wherever you're connecting to expects you to send authentication details on the first connection. It also causes extra latency to a request, as you need to make multiple calls, and causes 401s to appear in the logs.
The workaround is to use an authentication cache to pretend that you've already connected to the server once. This means you'll only make one HTTP call.
CloseableHttpClient httpclient = HttpClientBuilder.create().build();
HttpHost targetHost = new HttpHost("localhost", 80, "http");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(targetHost.getHostName(), targetHost.getPort()),
new UsernamePasswordCredentials("username", "password"));
// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();
// Generate BASIC scheme object and add it to the local auth cache
BasicScheme basicAuth = new BasicScheme();
authCache.put(targetHost, basicAuth);
// Add AuthCache to the execution context
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credsProvider);
context.setAuthCache(authCache);
HttpGet httpget = new HttpGet("/");
for (int i = 0; i < 3; i++) {
CloseableHttpResponse response = httpclient.execute(
targetHost, httpget, context);
try {
HttpEntity entity = response.getEntity();
} finally {
response.close();
}
}
N.B: You need to trust the host you're connecting to, and if you're
using HTTP, your username and password will be sent in cleartext
(well, base64, but that doesn't count).
You should also be using a much more specific Authscope rather than
relying on AuthScope.ANY_HOST and AuthScope.ANY_PORT like in your
example.
Credit goes to Cetra
Related Links:
HttpClientBuilder basic auth
Apache HttpClient 4.1 - Proxy Authentication
What you have should be very close to working. I would make the following simple changes:
builder.useSystemProperties();
Delete the call to useSystemProperties. It isn't documented well, but when you set the Proxy (as you do in the next line), it overrides this, so just remove that line.
builder.setProxy(new HttpHost(proxyHost, proxyPort));
Call the HttpHost constructor with the explicit 'scheme' parameter. This is where you are getting the error, so make it explicit:
String proxyScheme = "http";
builder.setProxy(new HttpHost(proxyHost, proxyPort, proxyScheme));
Note: you did not say, but based on the usage of "BasicCredentialsProvider", this is only giving you "Basic" authentication. Basic is only encoded and is not really secure. For Digest or NTLM or Kerberos you will need different code.
I think the problem is with your HttpClient, not the proxy. Did you try to create your HttpClient by using HttpClientBuilder.build()
HttpClient client = builder.build();
ChallengeState.PROXY would provide proxy-authorization header.
However since v4.3, the code is deprecated. It still works in v4.5.
HttpHost proxyHost = this.getProxyHttpHost(config);
authCache.put(proxyHost, new BasicScheme(ChallengeState.PROXY));
Another way to have proxy-authorization header
credsProvider.setCredentials(new AuthScope("127.0.0.1","8080"),
new UsernamePasswordCredentials("username", "password"));
builder.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy());
builder.setDefaultCredentialsProvider(credsProvider);
I am resolving RDF URIs in a Servlet using Jena:
final Model rdfModel = ModelFactory.createDefaultModel();
rdfModel.read(resource);
Is there a possibility to set Http Connect and Socket timeouts in Jena?
Or is it the only option to do handle the http connection 'manually' using Apache HttpClient?
final HttpClient httpclient = new DefaultHttpClient();
final HttpParams params = httpclient.getParams();
params.setParameter(HttpConnectionParams.CONNECTION_TIMEOUT, 1000);
params.setParameter(HttpConnectionParams.SO_TIMEOUT, 5000);
...
Here's the code based on #rob-hall's hint:
final Model rdfModel = ModelFactory.createDefaultModel();
final HttpClient httpclient = new DefaultHttpClient();
final HttpParams params = httpclient.getParams();
params.setParameter(HttpConnectionParams.CONNECTION_TIMEOUT, 1000);
params.setParameter(HttpConnectionParams.SO_TIMEOUT, 5000);
HttpOp.setDefaultHttpClient(httpclient);
rdfModel.read(resource);
From the design of the API, it appears that you should be able to use HttpOp#setDefaultHttpClient() to modify the default HttpClient utilized within Jena.
HttpOp#execHttpGet(String,String) is delegated to by LocatorURL#open(String), which is called by the StreamManager used by RDFDataMgr. Elsewhere, HttpOp#ensureClient(...) substitutes in the default client, if it is present.
I guys, I'm using retrofit and I wonder how to transparently handle the session cookie.
For that I extend the given ApacheClient and use a CookieStore in the custom call to ApacheClient.execute(HttpClient, HttpUriRequest) :
Client client = new ApacheClient() {
final CookieStore cookieStore = new BasicCookieStore();
#Override
protected HttpResponse execute(HttpClient client, HttpUriRequest request) throws IOException {
// BasicHttpContext is not thread safe
// CookieStore is thread safe
BasicHttpContext httpContext = new BasicHttpContext();
httpContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
return client.execute(request, httpContext);
}
};
RestAdapter restAdapter = new RestAdapter.Builder()
.setServer(API_URL)
.setClient(client)
.build();
Is there a better way to do this with the build-in retrofit API (with no HttpClient extension) ?
Starting from API 9 you have java.net.CookieManager and can set system-wide cookie handler like this:
CookieManager cookieManager = new CookieManager();
cookieManager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
CookieHandler.setDefault(cookieManager);
Yes, Apache Http client uses its own cookie-handling mechanism. But it should not be the problem because starting from API 9 HttpURLConnection is recommended HTTP client.
If you use Retrofit from Square you may also like their OkHttp lib - custom URLConnection implementation with lots of useful capabilities.
I've been trying to implement connection pooling for my application using Apache HTTPClient (v4.1). The problem is that the client always makes only two connections when run, though there are enough threads running parallel. I have been trying to modify the code for a while now, but nothing has helped yet.
I'm using ThreadSafeClientConnManager for connection pooling and set the MaxTotal and DefaulMaxPerRoute to values I want.
Is there anything that comes to your mind first that I might want to check?
Here's that code segment that I use to create the client.
DefaultHttpClient createClient() {
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, new Integer(60000));
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, new Integer(60000));
params.setBooleanParameter(CoreConnectionPNames.TCP_NODELAY, true);
SchemeRegistry registry = new SchemeRegistry();
registry.register(new Scheme("https", sf, 6443));
registry.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
ThreadSafeClientConnManager cm = new ThreadSafeClientConnManager(params, registry);
cm.setMaxTotal(2 * maxConnections);
cm.setDefaultMaxPerRoute(maxConnections);
HttpHost localhost = new HttpHost("localhost");
cm.setMaxForRoute(new HttpRoute(localhost), maxConnections);
HttpHost sdpTargetHost = new HttpHost("webserviceIP", webservicePort, "https");
cm.setMaxForRoute(new HttpRoute(sdpTargetHost, null, true), maxConnections);
return new DefaultHttpClient(cm, params);
}
The client returned by this function is used in Runnables managed by a ThreadPoolExecutor. The Runnables use the client, and has these lines:
HttpResponse response = httpClient.execute(httpPost, context);
HttpEntity entity = response.getEntity();
....
EntityUtils.consume(entity);
From what I know, the EntityUtils.consume(entity) will notify the connection manager that the connection is no longer used, and thus will release the connection to be used by other threads. So I'm guessing the connection management is alright.
I guess I've provided enough info, please tell me if I'm to add anything more.
Thanks
OK. I've found the solution, thanks to oleg for pointing out the logging, and to google and all the forums. All I had to do was define the class with only the connection manager, and then set HttpParams using HttpClient.setParams(). So the code will look something like this:
DefaultHttpClient createClient() {
SchemeRegistry registry = new SchemeRegistry();
registry.register(new Scheme("https", sf, 6443));
ThreadSafeClientConnManager cm = new ThreadSafeClientConnManager(registry);
cm.setMaxTotal(maxConnections);
cm.setDefaultMaxPerRoute(maxConnections);
HttpHost targetHost = new HttpHost("webserviceIP", webservicePort, "https");
cm.setMaxForRoute(new HttpRoute(targetHost, null, true), maxConnections);
return new DefaultHttpClient(cm);
}
And right before using the client,
DefaultHttpClient httpClient = createClient();
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, new Integer(60000));
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, new Integer(60000));
params.setBooleanParameter(CoreConnectionPNames.TCP_NODELAY, true);
httpClient.setParams(params);
There apparently is no difference in the code logically, but this fixed my problem. I presume this probably is some sorta bug in the HttpClient 4.1 API.
I was not able to set this option in cm
cm.setDefaultMaxPerRoute(maxConnections);
It was necessary to do so:
ConnPerRoute perRoute = new ConnPerRouteBean(100);
ConnManagerParams.setMaxConnectionsPerRoute(params, perRoute);
ConnManagerParams.setMaxTotalConnections(params, 100);
ConnManagerParams.setTimeout(params, 15000);
I am using jersey client with ApacheConnection Provider.
Builder builder = RequestConfig.custom().setConnectTimeout(timeout);
List<Proxy> proxies = ProxyManager.getInstance().select(baseUrl.toURI());
if (useProxy) {
...
builder.setProxy(new HttpHost(proxyUri.getHost(), proxyUri.getPort()));
}
RequestConfig requestConfig = builder.build();
final ClientConfig clientConfig = new ClientConfig();
clientConfig.property(ApacheClientProperties.REQUEST_CONFIG, requestConfig);
clientConfig.connectorProvider(new ApacheConnectorProvider());
client = ClientBuilder.newBuilder().withConfig(clientConfig).sslContext(getSSLContext()).build();
client.property(ClientProperties.CONNECT_TIMEOUT, 5000);
But how to add username and password for Proxy authentication?
Seems like apache connection provider does not use the standard java proxy selector mechanisms.
I finally found the solution by myself. Unfortunately this is documented nowhere:
HttpHost proxyhost = new HttpHost(host,pw);
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope(proxyhost), new UsernamePasswordCredentials(user, pw));
clientConfig.property(ApacheClientProperties.CREDENTIALS_PROVIDER, credsProvider);
builder.setProxy(proxyhost);
I think you should add few more lines of code
builder.setProxy(proxyhost).setDefaultCredentialsProvider(credsProvider)
.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy());
otherwise it wont really authenticate the proxy host I feel. In your case it might be bypassing the proxy. ?