I have inherited the code
import org.apache.http.client.HttpClient;
...
HttpClient httpclient = createHttpClientOrProxy();
...
private HttpClient createHttpClientOrProxy() {
HttpClient httpclient = new DefaultHttpClient();
/*
* Set an HTTP proxy if it is specified in system properties.
*
* http://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html
* http://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientExecuteProxy.java
*/
if( isSet(System.getProperty("http.proxyHost")) ) {
int port = 80;
if( isSet(System.getProperty("http.proxyPort")) ) {
port = Integer.parseInt(System.getProperty("http.proxyPort"));
}
HttpHost proxy = new HttpHost(System.getProperty("http.proxyHost"), port, "http");
// #Deprecated methods here... getParams() and ConnRoutePNames
httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
}
return httpclient;
}
httpClient.getParams() is #Deprecated and reads "
HttpParams getParams()
Deprecated.
(4.3) use RequestConfig.
There are no class docs for RequestConfig and I do not know what method should be used to replace getParams() and ConnRoutePNames.DEFAULT_PROXY.
This is more of a follow-up to the answer given by #Stephane Lallemagne
There is a much conciser way of making HttpClient pick up system proxy settings
CloseableHttpClient client = HttpClients.custom()
.setRoutePlanner(
new SystemDefaultRoutePlanner(ProxySelector.getDefault()))
.build();
or this if you want an instance of HttpClient fully configured with system defaults
CloseableHttpClient client = HttpClients.createSystem();
You are using apache HttpClient 4.3 library with apache HttpClient 4.2 code.
Please notice that getParams() and ConnRoutePNames are not the only deprecated methods in your case. The DefaultHttpClient class itself rely on 4.2 implementation and is also deprecated in 4.3.
In regard to the 4.3 documentation here (http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/connmgmt.html#d5e473), you can rewrite it this way:
private HttpClient createHttpClientOrProxy() {
HttpClientBuilder hcBuilder = HttpClients.custom();
// Set HTTP proxy, if specified in system properties
if( isSet(System.getProperty("http.proxyHost")) ) {
int port = 80;
if( isSet(System.getProperty("http.proxyPort")) ) {
port = Integer.parseInt(System.getProperty("http.proxyPort"));
}
HttpHost proxy = new HttpHost(System.getProperty("http.proxyHost"), port, "http");
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
hcBuilder.setRoutePlanner(routePlanner);
}
CloseableHttpClient httpClient = hcBuilder.build();
return httpClient;
}
Related
How can I use a custom configured Apache HttpClient with Jersey Client 2?
HttpClient with Caching (from apache docs)
CacheConfig cacheConfig = CacheConfig.custom()
.setMaxCacheEntries(1000)
.setMaxObjectSize(8192)
.build();
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(30000)
.setSocketTimeout(30000)
.build();
CloseableHttpClient cachingClient = CachingHttpClients.custom()
.setCacheConfig(cacheConfig)
.setDefaultRequestConfig(requestConfig)
.build();
Using Apache Http in general works by setting
protected ClientConfig getClientConfig() {
ClientConfig config = new ClientConfig();
config.property(HttpUrlConnectorProvider.SET_METHOD_WORKAROUND, true);
final PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
config.property(ApacheClientProperties.CONNECTION_MANAGER, connectionManager);
final ApacheConnectorProvider connector = new ApacheConnectorProvider();
config.connectorProvider(connector);
return config;
}
But how to use custom http client config like above?
You can use ApacheHttpClientBuilderConfigurator. So you can do:
config.register(new ApacheHttpClientBuilderConfigurator() {
public HttpClientBuilder configure(HttpClientBuilder httpClientBuilder) {
return httpClientBuilder.setDefaultRequestConfig(requestConfig);
}
});
See the example.
The Apache caching client is not properly supported at the moment. You may return CachingHttpClientBuilder from the method, but the HttpClientBuilder does not have getters to see what is already set, so you need to set everything from scratch.
I am trying to set proxy for a request I am making using HttpClientBuilder as follows:
CredentialsProvider credsProvider = new BasicCredentialsProvider();
UsernamePasswordCredentials usernamePasswordCredentials = new UsernamePasswordCredentials(proxyUser, proxyPassword);
credsProvider.setCredentials(new AuthScope(proxyHost, proxyPort), usernamePasswordCredentials);
builder.useSystemProperties();
builder.setProxy(new HttpHost(proxyHost, proxyPort));
builder.setDefaultCredentialsProvider(credsProvider);
builder.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy());
where builder is:
HttpClientBuilder builder = HttpClientBuilder.create();
However, I get this exception when I execute this request:
java.lang.RuntimeException: org.apache.http.conn.UnsupportedSchemeException: http protocol is not supported
Caused by: org.apache.http.conn.UnsupportedSchemeException: http protocol is not supported
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:108) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:338) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:388) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) ~[httpclient-4.5.1.jar:4.5.1]
(exception shortened for brevity)
Since this is an HTTP proxy, I don't want to change the scheme to HTTPS, which anyways won't work. How do I get this working?
java.lang.RuntimeException:
org.apache.http.conn.UnsupportedSchemeException: http protocol is not
supported
Why this problem occurs?
Ans: This actually happens because you forget to register a connection socket factory for the 'http' scheme.
Plain 'http' scheme must be used to establish an intermediate connection
to the proxy itself before 'https' tunneling could be employed.
For operational purpose, you can try this code:
CloseableHttpClient client = HttpClients.custom()
.setRoutePlanner(new
SystemDefaultRoutePlanner(ProxySelector.getDefault()))
.build();
I would also suggest simple code for your research. Hope it can save you.
ClientExecuteProxy.java
package org.apache.http.examples.client;
import org.apache.http.HttpHost;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
/**
* How to send a request via proxy.
*
* #since 4.0
*/
public class ClientExecuteProxy {
public static void main(String[] args)throws Exception {
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpHost target = new HttpHost("httpbin.org", 443, "https");
HttpHost proxy = new HttpHost("127.0.0.1", 8080, "http");
RequestConfig config = RequestConfig.custom()
.setProxy(proxy)
.build();
HttpGet request = new HttpGet("/");
request.setConfig(config);
System.out.println("Executing request " + request.getRequestLine() + " to " + target + " via " + proxy);
CloseableHttpResponse response = httpclient.execute(target, request);
try {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
System.out.println(EntityUtils.toString(response.getEntity()));
} finally {
response.close();
}
} finally {
httpclient.close();
}
}
}
Are you using using CloudantClient java API for Cloudant DB?
Ans:
If YES, then It turned out the issue with HTTP when setting a proxy was a bug at our end (sorry about that). We released 1.2.1 with the fix for this problem. You can download jar file from here. (Collected from mike-rhodes's answer)
UPDATE
How do I specify the credentials for the proxy here?
From HTTP authentication,
By default, httpclient will not provide credentials preemptively, it will first create a HTTP request without authentication parameters. This is by design, as a security precaution, and as part of the spec. But, this causes issues if you don't retry the connection, or wherever you're connecting to expects you to send authentication details on the first connection. It also causes extra latency to a request, as you need to make multiple calls, and causes 401s to appear in the logs.
The workaround is to use an authentication cache to pretend that you've already connected to the server once. This means you'll only make one HTTP call.
CloseableHttpClient httpclient = HttpClientBuilder.create().build();
HttpHost targetHost = new HttpHost("localhost", 80, "http");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(targetHost.getHostName(), targetHost.getPort()),
new UsernamePasswordCredentials("username", "password"));
// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();
// Generate BASIC scheme object and add it to the local auth cache
BasicScheme basicAuth = new BasicScheme();
authCache.put(targetHost, basicAuth);
// Add AuthCache to the execution context
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credsProvider);
context.setAuthCache(authCache);
HttpGet httpget = new HttpGet("/");
for (int i = 0; i < 3; i++) {
CloseableHttpResponse response = httpclient.execute(
targetHost, httpget, context);
try {
HttpEntity entity = response.getEntity();
} finally {
response.close();
}
}
N.B: You need to trust the host you're connecting to, and if you're
using HTTP, your username and password will be sent in cleartext
(well, base64, but that doesn't count).
You should also be using a much more specific Authscope rather than
relying on AuthScope.ANY_HOST and AuthScope.ANY_PORT like in your
example.
Credit goes to Cetra
Related Links:
HttpClientBuilder basic auth
Apache HttpClient 4.1 - Proxy Authentication
What you have should be very close to working. I would make the following simple changes:
builder.useSystemProperties();
Delete the call to useSystemProperties. It isn't documented well, but when you set the Proxy (as you do in the next line), it overrides this, so just remove that line.
builder.setProxy(new HttpHost(proxyHost, proxyPort));
Call the HttpHost constructor with the explicit 'scheme' parameter. This is where you are getting the error, so make it explicit:
String proxyScheme = "http";
builder.setProxy(new HttpHost(proxyHost, proxyPort, proxyScheme));
Note: you did not say, but based on the usage of "BasicCredentialsProvider", this is only giving you "Basic" authentication. Basic is only encoded and is not really secure. For Digest or NTLM or Kerberos you will need different code.
I think the problem is with your HttpClient, not the proxy. Did you try to create your HttpClient by using HttpClientBuilder.build()
HttpClient client = builder.build();
ChallengeState.PROXY would provide proxy-authorization header.
However since v4.3, the code is deprecated. It still works in v4.5.
HttpHost proxyHost = this.getProxyHttpHost(config);
authCache.put(proxyHost, new BasicScheme(ChallengeState.PROXY));
Another way to have proxy-authorization header
credsProvider.setCredentials(new AuthScope("127.0.0.1","8080"),
new UsernamePasswordCredentials("username", "password"));
builder.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy());
builder.setDefaultCredentialsProvider(credsProvider);
My application goes to Performance review stage. So Performs team told me there are some deprecated methods are available, write latest versons.
My part of the code is :
private void workAroundReverseDnsBugInHoneycombAndEarlier(HttpClient client) {
// Android had a bug where HTTPS made reverse DNS lookups (fixed in Ice Cream Sandwich)
// http://code.google.com/p/android/issues/detail?id=13117
SocketFactory socketFactory = new LayeredSocketFactory() {
SSLSocketFactory delegate = SSLSocketFactory.getSocketFactory();
#Override
public Socket createSocket() throws IOException {
return delegate.createSocket();
}
#Override
public Socket connectSocket(Socket sock, String host, int port, InetAddress localAddress, int localPort, HttpParams params) throws IOException {
return delegate.connectSocket(sock, host, port, localAddress, localPort, params);
}
#Override
public boolean isSecure(Socket sock) throws IllegalArgumentException {
return delegate.isSecure(sock);
}
#Override
public Socket createSocket(Socket socket, String host, int port, boolean autoClose) throws IOException {
injectHostname(socket, host);
return delegate.createSocket(socket, host, port, autoClose);
}
private void injectHostname(Socket socket, String host) {
try {
Field field = InetAddress.class.getDeclaredField("hostName");
field.setAccessible(true);
field.set(socket.getInetAddress(), host);
} catch (Exception ignored) {
}
}
};
client.getConnectionManager().getSchemeRegistry().register(new Scheme("https", socketFactory, 443));
}
Here SocketFactory ,LayeredSocketFactory,SSLSocketFactory,HttpProtocolParams ... says
Multiple markers at this line
- The method getParams() from the type HttpClient is deprecated
- The method setUserAgent(HttpParams, String) from the type HttpProtocolParams is
deprecated
- The type HttpProtocolParams is deprecated
- The method getParams() from the type HttpClient is deprecated
- The method setContentCharset(HttpParams, String) from the type HttpProtocolParams is
deprecated
- The type HttpProtocolParams is deprecated
another sample deprecated
Multiple markers at this line
- The type SocketFactory is deprecated
- The type LayeredSocketFactory is deprecated
- The type SSLSocketFactory is deprecated
- The type SSLSocketFactory is deprecated
- The method getSocketFactory() from the type SSLSocketFactory is
deprecated
- The type SSLSocketFactory is deprecated
Please guide me . I am using HttpClient 4.3.jar file.
Please tell me any ideas...
The method getParams() from the type HttpClient is deprecated
Before 4.3.2 you could set the parameters to the client using the getParams() method (deprecated now), after 4.3.2 you can set the request params via the RequestConfig class using a Builder
Builder requestConfigBuilder = RequestConfig.custom();
requestConfigBuilder.setConnectionRequestTimeout(1000).setMaxRedirects(1);
and then set to the HttpMethod only (not to client like before)
HttpGet request = new HttpGet(builder.build());
request.setConfig(requestConfigBuilder.build());
The method setUserAgent(HttpParams, String) from the type HttpProtocolParams is deprecated
CloseableHttpClient httpclient = HttpClients.custom().setUserAgent("Agent").build();
The method setContentCharset(HttpParams, String) from the type HttpProtocolParams is deprecated
Use AbstractHttpEntity subclasses (BasicHttpEntity, ByteArrayEntity, EntityTemplate, FileEntity, InputStreamEntity, SerializableEntity, StringEntity)
HttpPost post = new HttpPost("http://");
post.setEntity(new StringEntity("content", "UTF-8"));
The type HttpProtocolParams is deprecated
(4.3) use configuration classes provided 'org.apache.http.config' and 'org.apache.http.client.config'
Builder requestConfigBuilder = RequestConfig.custom();
The type SocketFactory is deprecated & The type LayeredSocketFactory is deprecated
SSLContext sslcontext = SSLContexts.createSystemDefault();
// Allow TLSv1 protocol only
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
sslcontext,
new String[] { "TLSv1" },
null,
SSLConnectionSocketFactory.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
or
SSLContext sslContext = SSLContexts.custom()
.useTLS()
.loadTrustMaterial(myTrustStore)
.build();
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext);
then
CloseableHttpClient httpclient = HttpClients.custom().setSSLSocketFactory(sslsf).build();
You can take a look at HttpClient Tutorial (4.3.x) for more examples and explanations
I hope this helps you.
I developed a service in RESTEasy using ProxyFactory and ClientExecutor like this:
PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager();
DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, 5000);
HttpConnectionParams.setSoTimeout(params, 5000);
ClientExecutor clientExecutor = new ApacheHttpClient4Executor(httpClient);
MyClass client = ProxyFactory.create(MyClass.class, "http://www.example.com", clientExecutor);
It always worked perfectly. After RESTEasy deprecated both ClientExecutor and ProxyFactory, they provided a new ResteasyClient for external connections, but I don't know if this new ResteasyClient is threadsafe. This is the new sample code from the documentation:
ResteasyClient client = new ResteasyClientBuilder().build();
ResteasyWebTarget target = client.target("http://example.com/base/uri");
SimpleClient simple = target.proxy(SimpleClient.class);
UPDATE: I used the code with the ResteasyClient and I got many of these errors:
javax.ws.rs.ProcessingException: Unable to invoke request
Caused by
java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one.
We use this:
final ResteasyClient client = new ResteasyClientBuilder()
.connectionPoolSize(10)
.maxPooledPerRoute(5)
.build();
And after debugging I found out that (at least in our situation) the RESTEasy client uses the ThreadSafeClientConnManager by default so I think there is no need to specify a different one although according to the JavaDoc it is deprecated in favour of PoolingHttpClientConnectionManager (note the extra Http). But this has been fixed in RESTEasy client 3.0.5.Final: https://issues.jboss.org/browse/RESTEASY-948
It's a jungle of HTTP connection managers out there..
This worked for me. Just needed to find the hook into setting up the Apache HTTP engine. Mostly based on RestEasy 3.0.5.Final API
public static Object setupServiceProxy(#NotNull Class responseClass) {
ResteasyProviderFactory factory = ResteasyProviderFactory.getInstance();
ResteasyClientBuilder builder = new ResteasyClientBuilder().providerFactory(factory);
ResteasyClient client = builder.httpEngine(setupHttpDefaults()).build();
ResteasyWebTarget target = client.target(url);
return target.proxy(responseClass);
}
public static ClientHttpEngine setupHttpDefaults() {
PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager();
DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, 30000);
HttpConnectionParams.setSoTimeout(params, 30000);
BasicHttpContext localContext = new BasicHttpContext();
return new ApacheHttpClient4Engine(httpClient, localContext);
}
I am using jersey client with ApacheConnection Provider.
Builder builder = RequestConfig.custom().setConnectTimeout(timeout);
List<Proxy> proxies = ProxyManager.getInstance().select(baseUrl.toURI());
if (useProxy) {
...
builder.setProxy(new HttpHost(proxyUri.getHost(), proxyUri.getPort()));
}
RequestConfig requestConfig = builder.build();
final ClientConfig clientConfig = new ClientConfig();
clientConfig.property(ApacheClientProperties.REQUEST_CONFIG, requestConfig);
clientConfig.connectorProvider(new ApacheConnectorProvider());
client = ClientBuilder.newBuilder().withConfig(clientConfig).sslContext(getSSLContext()).build();
client.property(ClientProperties.CONNECT_TIMEOUT, 5000);
But how to add username and password for Proxy authentication?
Seems like apache connection provider does not use the standard java proxy selector mechanisms.
I finally found the solution by myself. Unfortunately this is documented nowhere:
HttpHost proxyhost = new HttpHost(host,pw);
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope(proxyhost), new UsernamePasswordCredentials(user, pw));
clientConfig.property(ApacheClientProperties.CREDENTIALS_PROVIDER, credsProvider);
builder.setProxy(proxyhost);
I think you should add few more lines of code
builder.setProxy(proxyhost).setDefaultCredentialsProvider(credsProvider)
.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy());
otherwise it wont really authenticate the proxy host I feel. In your case it might be bypassing the proxy. ?