I have been trying to use a custom SocketFactory in the httpclient library from the Apache HTTPComponents project. So far without luck. I was expecting that I could just set a socket factory for a HttpClient instance, but it is obviously not so easy.
The documentation for HttpComponents at http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html does mention socket factories, but does not say how to use them.
Does anybody know how this is done?
oleg's answer is of course correct, I just wanted to put the information directly here, in case the link goes bad. In the code that creates a HttpClient, I use this code to let it use my socket factory:
CustomSocketFactory socketFactory = new CustomSocketFactory();
Scheme scheme = new Scheme("http", 80, socketFactory);
httpclient.getConnectionManager().getSchemeRegistry().register(scheme);
CustomSocketFactory is my own socket factory, and I want to use it for normal HTTP traffic, that's why I use "http" and 80 as parameters.
My CustomSchemeSocketFactory looks similar to this:
public class CustomSchemeSocketFactory implements SchemeSocketFactory {
#Override
public Socket connectSocket( Socket socket, InetSocketAddress remoteAddress, InetSocketAddress localAddress, HttpParams params ) throws IOException, UnknownHostException, ConnectTimeoutException {
if (localAddress != null) {
socket.setReuseAddress(HttpConnectionParams.getSoReuseaddr(params));
socket.bind(localAddress);
}
int connTimeout = HttpConnectionParams.getConnectionTimeout(params);
int soTimeout = HttpConnectionParams.getSoTimeout(params);
try {
socket.setSoTimeout(soTimeout);
socket.connect(remoteAddress, connTimeout );
} catch (SocketTimeoutException ex) {
throw new ConnectTimeoutException("Connect to " + remoteAddress + " timed out");
}
return socket;
}
#Override
public Socket createSocket( HttpParams params ) throws IOException {
// create my own socket and return it
}
#Override
public boolean isSecure( Socket socket ) throws IllegalArgumentException {
return false;
}
}
We use a custom socket factory to allow HttpClient connections to connect to HTTPS URLs with untrusted certificates.
Here is how we did it:
We adapted implementations of both the 'EasySSLProtocolSocketFactory' and 'EasyX509TrustManager' classes from the examples source directory referenced by Oleg.
In our HttpClient startup code, we do the following to enable the new socket factory:
HttpClient httpClient = new HttpClient();
Protocol easyhttps = new Protocol("https", new EasySSLProtocolSocketFactory(), 443);
Protocol.registerProtocol("https", easyhttps);
So that any time we request an https:// URL, this socket factory is used.
Related
This question is worded the same way as another question on SO (HttpClient: how do I obtain the underlying socket from an existing connection?), but that question is actually about tunneling other protocols through HTTP(S) and thus the answer is quite a bit different.
What I'm trying to do is make an HTTPS connection, then find out the details of that connection. Java's SSLSocket class will give me what I need, but I need to be able to get a hold of the Socket itself in order to interrogate it.
Is there a way to get to the underlying Socket? httpclient/httpcore has become a maze of factories and private/protected implementations of things, so it's really difficult to poke-around the API to figure out how to actually get things once they have been configured.
HttpClient intentionally makes it difficult to get hold of the underlying connection object and the socket it is bound to, primarily to ensure the connection state is consistent and persistent connections in the connection pool are safe to be re-used by another transaction.
However, one can get hold of the underlying connection from a response interceptor.
CloseableHttpClient httpclient = HttpClients.custom()
.addInterceptorLast(new HttpResponseInterceptor() {
#Override
public void process(
final HttpResponse response,
final HttpContext context) throws HttpException, IOException {
HttpClientContext clientContext = HttpClientContext.adapt(context);
ManagedHttpClientConnection connection = clientContext.getConnection(ManagedHttpClientConnection.class);
// Can be null if the response encloses no content
if (connection != null) {
Socket socket = connection.getSocket();
System.out.println(socket);
}
}
})
.build();
try (CloseableHttpResponse response = httpclient.execute(new HttpGet("http://www.google.com/"))) {
System.out.println(response.getStatusLine());
EntityUtils.consume(response.getEntity());
}
I ended up using a somewhat different technique, but #oleg got me on the right track. Here's my one-time code:
HttpClientContext ctx = HttpClientContext.create();
HttpResponse response = getHttpClient().execute(method, ctx);
if(log.isDebugEnabled())
{
ManagedHttpClientConnection connection = ctx.getConnection(ManagedHttpClientConnection.class);
// Can be null if the response encloses no content
if(null != connection)
{
Socket socket = connection.getSocket();
if(socket instanceof SSLSocket)
{
SSLSocket sslSock = (SSLSocket)socket;
log.debug("Connected to " + getEndpointURL()
+ " using " + sslSock.getSession().getProtocol()
+ " and suite " + sslSock.getSession().getCipherSuite());
}
}
}
I wish to force Apache Commons HTTP-Client (version 3.1) to use TLS 1.2 as the only protocol for HTTPS.
This is due to the server supposedly being upgraded to TLS 1.2 and not accepting any older protocol anymore (causing 'Connection Reset' to be returned).
For further context, probably irrelevant, the HTTP-Client is used along with Axis2 to make a SOAP; some of the code used for setting up the HttpClient is below:
MultiThreadedHttpConnectionManager connMgr = new MultiThreadedHttpConnectionManager();
this.httpClient = new HttpClient(connMgr);
// initialize HttpClient parameters
HttpClientParams hcParams = this.httpClient.getParams();
// Maximum time to wait to receive connection from pool
hcParams.setConnectionManagerTimeout(this.maxWait);
hcParams.setSoTimeout(this.timeout);
hcParams.setParameter(HttpMethodParams.RETRY_HANDLER, new DefaultHttpMethodRetryHandler(this.retryCount, false));
// Initialize global Connection manager parameters
HttpConnectionManagerParams cmParams = connMgr.getParams();
cmParams.setDefaultMaxConnectionsPerHost(this.maxActive);
cmParams.setStaleCheckingEnabled(this.checkStaleConnections);
cmParams.setConnectionTimeout(this.timeout);
Thanks a lot for the help!
Too bad nobody answered; I was able to do it, first you write a CustomHttpSocketFactory, then you do:
String scheme = "https";
Protocol baseHttps = Protocol.getProtocol(scheme);
int defaultPort = baseHttps.getDefaultPort();
ProtocolSocketFactory baseFactory = baseHttps.getSocketFactory();
ProtocolSocketFactory customFactory = new CustomHttpsSocketFactory(baseFactory);
Protocol customHttps = new Protocol(scheme, customFactory, defaultPort);
Protocol.registerProtocol(scheme, customHttps);
A sample custom socket factory code is found here, but instead I did:
public class CustomHttpsSocketFactory implements SecureProtocolSocketFactory
{
private final SecureProtocolSocketFactory base;
public CustomHttpsSocketFactory(ProtocolSocketFactory base)
{
if(base == null || !(base instanceof SecureProtocolSocketFactory)) throw new IllegalArgumentException();
this.base = (SecureProtocolSocketFactory) base;
}
private Socket acceptOnlyTLS12(Socket socket)
{
if(!(socket instanceof SSLSocket)) return socket;
SSLSocket sslSocket = (SSLSocket) socket;
sslSocket.setEnabledProtocols(new String[]{"TLSv1.2" });
return sslSocket;
}
#Override
public Socket createSocket(String host, int port) throws IOException
{
return acceptOnlyTLS12(base.createSocket(host, port));
}
#Override
public Socket createSocket(String host, int port, InetAddress localAddress, int localPort) throws IOException
{
return acceptOnlyTLS12(base.createSocket(host, port, localAddress, localPort));
}
#Override
public Socket createSocket(String host, int port, InetAddress localAddress, int localPort, HttpConnectionParams params) throws IOException
{
return acceptOnlyTLS12(base.createSocket(host, port, localAddress, localPort, params));
}
#Override
public Socket createSocket(Socket socket, String host, int port, boolean autoClose) throws IOException
{
return acceptOnlyTLS12(base.createSocket(socket, host, port, autoClose));
}
}
You need a Socket reference in your code. Then you can set enabled protocols on it like this:
if (socket != null && (socket instanceof SSLSocket)) {
((SSLSocket)socket).setEnabledProtocols(new String[] {"TLSv1.2"});
}
It depends on how you are writing your clients and what JRE versions you are using:
If you are using JRE8 (unless you have replaced the default SunJSSE that comes with JRE8), there is a system property "jdk.tls.client.protocols". By default, whatever you mention here will be used for all client communication.
If you are using HttpsURLConnection object for client connection, u can use the system property "https.protocols". This will work for all JRE versions, not just JRE8.
If you don't specify anything, for TLS clients, in JRE8, TLSv1, v1.1 and v1.2 are enabled, so it will work with a server what supports any one of this versions. However in JRE7 by default TLSv1 alone is enabled.
In your code u can always override the default or what u pass through the system properties. What u set in the code will take higher precedence. To override in the code...
1) If you are using raw socket and SSLEngine, u can set the protocol and ciphers in the SSLEngine (sslEngine.setEnabledProtocols(..)
2) If you are using SSLSocket, u can set the protocol and ciphers in the SSLSocket (sslSocket.setEnabledProtocols(..)
You can also get an SSLContext with the required protocol enabled and use that for whatever SSL components you use. SSLContext.getInstance("TLSvx.x"). Note that by default it will return a context with all the protocols lesser that TLSvx.x enabled. If u have configured "jdk.tls.client.protocols", this will return a context with those protocols enabled.
It would not be a good idea to hard coded the protocols in the code. Quite often, we will encounter certain customers want specific version either because they use old servers or some serious vulnerabilities are encountered in some TLS versions. Either set it through the system properties or even if you are explicitly setting in sslsocket or sslengine, read that from some conf file.
Also refer:
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html
http://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html
I need some help with implementing a manual ssl handshake between an android smartphone and a server.
The background why is in my case the encryption/decryption of the communication. Android would be just a proxy which should have no information about what data is exchanged between the server and my javacard applet which only can communicate with the android device.
My first thought was to establich a connection to a server like a normal http just on the ssl port of the server: (in android)
Socket socket = new Socket("google.de", 443);
DataOutputStream os = new DataOutputStream(socket.getOutputStream());
DataInputStream is = new DataInputStream(socket.getInputStream());
// send data for the handshake
os.writeChars("Params for the SSL handshake");
// read the server response
in.read(...)
// answer with the next ssl handshake step
....
and then send the information for the handshake and wait for the response.
Problem here is, I don't know in which format I would have to send the parameters (like the client hello: protocolVersion, random, sessionID, cipherSuites, compressMethod). Or if it would work?
The other way I inspected was the
javax.net.ssl.SSLSocket;
which makes the handshake by itself.
For example:
public static void main(final String[] args) throws Exception {
final SSLContext sslContext = SSLContext.getInstance("TLSv1");
sslContext.init(null, null, null);
// getDefault();
final SSLSocketFactory fac = sslContext.getSocketFactory();
final SSLSocket socket = (SSLSocket) fac.createSocket("google.de", 443);
socket.addHandshakeCompletedListener(new HandshakeCompletedListener() {
#Override
public void handshakeCompleted(final HandshakeCompletedEvent event) {
System.out.println("Cipher:" + event.getCipherSuite());
}
});
final String[] ciphers = fac.getSupportedCipherSuites();
final String[] protocols = { "TLSv1" };
final SSLParameters params = new SSLParameters(ciphers, protocols);
params.setNeedClientAuth(false);
socket.setSSLParameters(params);
socket.startHandshake();
}
In this case, all the relevant security information (like the secret exchange) and so on will happen on Android... (in detail: in the SSLSocket itself I think) That's exactly the way I don't want!
I want to have the possibility to send the handshake parameters by my own to the server and can catch the response from the server and will forward it to my javacard applet. I know I have to take care of the encryption later on by myself.
I hope my problem is understandable and the background why I need this too. Else please give a small hint and I will do my best to complete the information.
Thanks in advance!
I'm using Java's HttpUrlConnection to hit foo.com
foo.com has multiple A-Records that point to different IP addresses (1.1.1.1 and 1.1.1.2)
If my first connect call resolves to 1.1.1.1, but then that machine goes down, will a subsequent connect call recognize this and try to connect on 1.1.1.2 instead?
Or do I need to implement this sort of logic myself using the INetAddress api?
I was able to resolve this by using Apache Commons HttpClient, see the code snippet below.
Like I feared, the URLConnection provided by java.net is a very simplistic implementation and will only try the first IP address from the resolved list. If you really are not allowed to use another library, you will have to write your own error handling. It's kinda messy, since you will need to resolve all IPs before hand using InetAddress, and connect to each IP passing the "Host: domain.name" header to the HTTP stack yourself until one of the IPs responds.
The Apache library is greatly more robust and allows for a great deal of customization. You can control how many times it will retry and, most importantly, it will automatically try all IP addresses resolved to the same name until one of them responds successfully.
HttpRequestRetryHandler myRetryHandler = new HttpRequestRetryHandler() {
#Override
public boolean retryRequest(IOException exception, int count, HttpContext context) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
return count < 30;
}
};
ConnectionKeepAliveStrategy keepAlive = new ConnectionKeepAliveStrategy() {
#Override
public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
return 500;
}
};
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter("http.socket.timeout", new Integer(2000));
httpclient.getParams().setParameter("http.connection.timeout", new Integer(2000));
httpclient.setHttpRequestRetryHandler(myRetryHandler);
httpclient.setKeepAliveStrategy(keepAlive);
HttpGet httpget = new HttpGet("http://remotehost.com");
HttpResponse httpres = httpclient.execute(httpget);
InputStream is = httpres.getEntity().getContent();
I hope this helps!
I have been using the following code to connect to one of google's service. This code worked fine on my local machine :
HttpClient client=new DefaultHttpClient();
HttpPost post = new HttpPost("https://www.google.com/accounts/ClientLogin");
post.setEntity(new UrlEncodedFormEntity(myData));
HttpResponse response = client.execute(post);
I put this code in a production environment, which had blocked Google.com. On request, they allowed communication with Google server by allowing me to accessing an IP : 74.125.236.52 - which is one of Google's IPs. I edited my hosts file to add this entry too.
Still I could not access the URL, which I wonder why. So I replaced the above code with :
HttpPost post = new HttpPost("https://74.125.236.52/accounts/ClientLogin");
Now I get an error like this :
javax.net.ssl.SSLException: hostname in certificate didn't match:
<74.125.236.52> != <www.google.com>
I guess this is because Google has multiple IPs. I cant ask the network admin to allow me access to all those IPs - I may not even get this entire list.
What should I do now ? Is there a workaround at Java level ? Or is it totally in hands of the network guy ?
You can also try to set a HostnameVerifier as described here. This worked for me to avoid this error.
// Do not do this in production!!!
HostnameVerifier hostnameVerifier = org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER;
DefaultHttpClient client = new DefaultHttpClient();
SchemeRegistry registry = new SchemeRegistry();
SSLSocketFactory socketFactory = SSLSocketFactory.getSocketFactory();
socketFactory.setHostnameVerifier((X509HostnameVerifier) hostnameVerifier);
registry.register(new Scheme("https", socketFactory, 443));
SingleClientConnManager mgr = new SingleClientConnManager(client.getParams(), registry);
DefaultHttpClient httpClient = new DefaultHttpClient(mgr, client.getParams());
// Set verifier
HttpsURLConnection.setDefaultHostnameVerifier(hostnameVerifier);
// Example send http request
final String url = "https://encrypted.google.com/";
HttpPost httpPost = new HttpPost(url);
HttpResponse response = httpClient.execute(httpPost);
The certificate verification process will always verify the DNS name of the certificate presented by the server, with the hostname of the server in the URL used by the client.
The following code
HttpPost post = new HttpPost("https://74.125.236.52/accounts/ClientLogin");
will result in the certificate verification process verifying whether the common name of the certificate issued by the server, i.e. www.google.com matches the hostname i.e. 74.125.236.52. Obviously, this is bound to result in failure (you could have verified this by browsing to the URL https://74.125.236.52/accounts/ClientLogin with a browser, and seen the resulting error yourself).
Supposedly, for the sake of security, you are hesitant to write your own TrustManager (and you musn't unless you understand how to write a secure one), you ought to look at establishing DNS records in your datacenter to ensure that all lookups to www.google.com will resolve to 74.125.236.52; this ought to be done either in your local DNS servers or in the hosts file of your OS; you might need to add entries to other domains as well. Needless to say, you will need to ensure that this is consistent with the records returned by your ISP.
I had similar problem. I was using Android's DefaultHttpClient. I have read that HttpsURLConnection can handle this kind of exception. So I created custom HostnameVerifier which uses the verifier from HttpsURLConnection. I also wrapped the implementation to custom HttpClient.
public class CustomHttpClient extends DefaultHttpClient {
public CustomHttpClient() {
super();
SSLSocketFactory socketFactory = SSLSocketFactory.getSocketFactory();
socketFactory.setHostnameVerifier(new CustomHostnameVerifier());
Scheme scheme = (new Scheme("https", socketFactory, 443));
getConnectionManager().getSchemeRegistry().register(scheme);
}
Here is the CustomHostnameVerifier class:
public class CustomHostnameVerifier implements org.apache.http.conn.ssl.X509HostnameVerifier {
#Override
public boolean verify(String host, SSLSession session) {
HostnameVerifier hv = HttpsURLConnection.getDefaultHostnameVerifier();
return hv.verify(host, session);
}
#Override
public void verify(String host, SSLSocket ssl) throws IOException {
}
#Override
public void verify(String host, X509Certificate cert) throws SSLException {
}
#Override
public void verify(String host, String[] cns, String[] subjectAlts) throws SSLException {
}
}
A cleaner approach ( only for test environment) in httpcliet4.3.3 is as follows.
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext,SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
CloseableHttpClient httpclient = HttpClients.custom().setSSLSocketFactory(sslsf).build();
In httpclient-4.3.3.jar, there is another HttpClient to use:
public static void main (String[] args) throws Exception {
// org.apache.http.client.HttpClient client = new DefaultHttpClient();
org.apache.http.client.HttpClient client = HttpClientBuilder.create().build();
System.out.println("HttpClient = " + client.getClass().toString());
org.apache.http.client.methods.HttpPost post = new HttpPost("https://www.rideforrainbows.org/");
org.apache.http.HttpResponse response = client.execute(post);
java.io.InputStream is = response.getEntity().getContent();
java.io.BufferedReader rd = new java.io.BufferedReader(new java.io.InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
This HttpClientBuilder.create().build() will return org.apache.http.impl.client.InternalHttpClient. It can handle the this hostname in certificate didn't match issue.
Thanks Vineet Reynolds. The link you provided held a lot of user comments - one of which I tried in desperation and it helped. I added this method :
// Do not do this in production!!!
HttpsURLConnection.setDefaultHostnameVerifier( new HostnameVerifier(){
public boolean verify(String string,SSLSession ssls) {
return true;
}
});
This seems fine for me now, though I know this solution is temporary. I am working with the network people to identify why my hosts file is being ignored.
The concern is we should not use ALLOW_ALL_HOSTNAME_VERIFIER.
How about I implement my own hostname verifier?
class MyHostnameVerifier implements org.apache.http.conn.ssl.X509HostnameVerifier
{
#Override
public boolean verify(String host, SSLSession session) {
String sslHost = session.getPeerHost();
System.out.println("Host=" + host);
System.out.println("SSL Host=" + sslHost);
if (host.equals(sslHost)) {
return true;
} else {
return false;
}
}
#Override
public void verify(String host, SSLSocket ssl) throws IOException {
String sslHost = ssl.getInetAddress().getHostName();
System.out.println("Host=" + host);
System.out.println("SSL Host=" + sslHost);
if (host.equals(sslHost)) {
return;
} else {
throw new IOException("hostname in certificate didn't match: " + host + " != " + sslHost);
}
}
#Override
public void verify(String host, X509Certificate cert) throws SSLException {
throw new SSLException("Hostname verification 1 not implemented");
}
#Override
public void verify(String host, String[] cns, String[] subjectAlts) throws SSLException {
throw new SSLException("Hostname verification 2 not implemented");
}
}
Let's test against https://www.rideforrainbows.org/ which is hosted on a shared server.
public static void main (String[] args) throws Exception {
//org.apache.http.conn.ssl.SSLSocketFactory sf = org.apache.http.conn.ssl.SSLSocketFactory.getSocketFactory();
//sf.setHostnameVerifier(new MyHostnameVerifier());
//org.apache.http.conn.scheme.Scheme sch = new Scheme("https", 443, sf);
org.apache.http.client.HttpClient client = new DefaultHttpClient();
//client.getConnectionManager().getSchemeRegistry().register(sch);
org.apache.http.client.methods.HttpPost post = new HttpPost("https://www.rideforrainbows.org/");
org.apache.http.HttpResponse response = client.execute(post);
java.io.InputStream is = response.getEntity().getContent();
java.io.BufferedReader rd = new java.io.BufferedReader(new java.io.InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
SSLException:
Exception in thread "main" javax.net.ssl.SSLException: hostname in certificate didn't match: www.rideforrainbows.org != stac.rt.sg OR stac.rt.sg OR www.stac.rt.sg
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:231)
...
Do with MyHostnameVerifier:
public static void main (String[] args) throws Exception {
org.apache.http.conn.ssl.SSLSocketFactory sf = org.apache.http.conn.ssl.SSLSocketFactory.getSocketFactory();
sf.setHostnameVerifier(new MyHostnameVerifier());
org.apache.http.conn.scheme.Scheme sch = new Scheme("https", 443, sf);
org.apache.http.client.HttpClient client = new DefaultHttpClient();
client.getConnectionManager().getSchemeRegistry().register(sch);
org.apache.http.client.methods.HttpPost post = new HttpPost("https://www.rideforrainbows.org/");
org.apache.http.HttpResponse response = client.execute(post);
java.io.InputStream is = response.getEntity().getContent();
java.io.BufferedReader rd = new java.io.BufferedReader(new java.io.InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
Shows:
Host=www.rideforrainbows.org
SSL Host=www.rideforrainbows.org
At least I have the logic to compare (Host == SSL Host) and return true.
The above source code is working for httpclient-4.2.3.jar and httpclient-4.3.3.jar.
Updating the java version from 1.8.0_40 to 1.8.0_181 resolved the issue.
SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(
SSLContexts.custom().loadTrustMaterial(null, new TrustSelfSignedStrategy()).build(),
SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
CloseableHttpClient httpClient = HttpClients.custom().setSSLSocketFactory(sslConnectionSocketFactory).build();