I was a bit envious of our Ruby on Rails team who needs to consume the same web services requiring HTTPS request be signed with a certificate. They grab a gem, sign the request with a single line of code, dishes are done...
Java, however, requires that we import the cert to our keystore (trying to avoid this), or create one in memory...that's fine. But, once I do that, it seems like the only option to actually signing requests is using the HttpUrlConnection class. HttpUrlConnection example
I'd like to be able to sign requests using my existing code, which uses apache's DefaultHttpClient client = new DefaultHttpClient(); - but I don't see a way to have the Http client sign requests using an in memory keystore.
Anyone faced this?
Have a look at AuthSSLProtocolSocketFactory.java. You should be able to perform SSL client authentication like this (untested code);
Protocol.registerProtocol("https",
new Protocol("https", new AuthSSLProtocolSocketFactory(keystoreUrl, keystorePassword, truststoreUrl, truststorePassword), 443));
HttpClient httpclient = new HttpClient();
GetMethod httpget = new GetMethod("https://www.whatever.com/");
try {
httpclient.executeMethod(httpget);
System.out.println(httpget.getStatusLine());
} finally {
httpget.releaseConnection();
}
Your client certificate goes into keystore pointed by keystoreUrl. Read more about this on HttpClient SSL Guide.
Related
Here there is an example for custom SSL:
https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientCustomSSL.java
/**
* This example demonstrates how to create secure connections with a custom SSL
* context.
*/
public class ClientCustomSSL {
public final static void main(String[] args) throws Exception {
// Trust own CA and all self-signed certs
SSLContext sslcontext = SSLContexts.custom()
.loadTrustMaterial(new File("my.keystore"), "nopassword".toCharArray(),
new TrustSelfSignedStrategy())
.build();
// Allow TLSv1 protocol only
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
sslcontext,
new String[] { "TLSv1" },
null,
SSLConnectionSocketFactory.getDefaultHostnameVerifier());
CloseableHttpClient httpclient = HttpClients.custom()
.setSSLSocketFactory(sslsf)
.build();
try {
HttpGet httpget = new HttpGet("https://httpbin.org/");
System.out.println("Executing request " + httpget.getRequestLine());
CloseableHttpResponse response = httpclient.execute(httpget);
try {
HttpEntity entity = response.getEntity();
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
EntityUtils.consume(entity);
} finally {
response.close();
}
} finally {
httpclient.close();
}
}
}
Why we need that? I've tested an HttpClient request without any SSL thing on it and I'm getting the correct response from HTTPS urls without errors.
What is the problem if I don't add any SSLContext?
And if it's important to make it more secure, what is this line?:
.loadTrustMaterial(new File("my.keystore"), "nopassword".toCharArray(),
it seems we need some file and also some password?
If you don't specify (a factory using) a context, Java (JSSE) uses a default context containing the default truststore, which defaults to the file JRE/lib/security/cacerts (or jssecacerts if present) unless overridden with system properties; see https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#CustomizingStores . Depending on whether you are using an Oracle-was-Sun Java package, an IBM or Apple package or Android system, or OpenJDK, this default truststore usually contains more or less the same set of public CAs as most OSes and browsers, like Verisign Symantec Digicert and GoDaddy and LetsEncrypt/Identrust. Whether you consider the default cacerts 'secure' is a choice for you to make; if not you can either change the contents of the default file, or have your code use a different file and to do the latter yes you must specify the filename of the keystore file and its password.
That example uses a custom store because it is an example of custom SSL. If it used the defaults, it would be an example of default SSL not an example of custom SSL. For many actual applications using the defaults is fine.
Aside: specifying only TLSv1 (meaning 1.0) for protocol is way out of date, and is likely to be considered insecure or at least borderline. It hasn't actually been broken outright like SSLv3 (and long ago SSLv2), because BEAST proved tamer than feared, but TLSv1.1 and 1.2 are now widely implemented and used, and 1.3 hopefully not too far away, so using 1.0 is widely considered substandard and for one example applicable to many people TLSv1.0 for payment-card transactions is prohibited outright as of last weekend.
So I wanted to use this XML parser class to parse an XML file from one of my sites, which acts as an API.
So I came across this XMLParser class over here: XMLParser.java
Since I use https over at my site, I quickly found out this isn't going to work with it, unless SSL is implemented within the code, where it fetches XML file from the URL using the following method:
getXmlFromUrl(String url)
So for my test environment I was quick to create a new httpClient method, which would accept any SSL certificate from.
public static HttpClient createHttpClient()
{
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, HTTP.DEFAULT_CONTENT_CHARSET);
HttpProtocolParams.setUseExpectContinue(params, true);
SchemeRegistry schReg = new SchemeRegistry();
schReg.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
schReg.register(new Scheme("https", SSLSocketFactory.getSocketFactory(), 443));
ClientConnectionManager conMgr = new ThreadSafeClientConnManager(params, schReg);
return new DefaultHttpClient(conMgr, params);
}
So this is my final XMLParser.java after I did it.
XMLParser_New.java
But this all went in vain, as I am repeatedly getting the following exception.
javax.net.ssl.SSLPeerUnverifiedException: No peer certificate
Where have I gone wrong or what is it that I don't understand.
Thank you :)
How are you implementing SSL. Have you followed these steps to implement it:
Need to create X.509 certificate on Server.
Store trusted CAs on terminal side.
Encrypt and decrypt messages on both ends.
Are you using self signed certificate. Then you need to store that certificate on device and load your own trust store to check the SSL certificates, rather than android default trust store.
Here is very good article to implement SSL on android:
http://www.codeproject.com/Articles/826045/Android-security-Implementation-of-Self-signed-SSL
I think in your case trust certificate on terminal side is missing. That's why it is throwing this exception.
I'm trying to call heroku's developer api from java, but I get the following exception:
Exception in thread "main" javax.net.ssl.SSLException: hostname in certificate didn't match: <50.19.233.255> != <*.heroku.com>
My code looks like this:
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("https://api.heroku.com/apps");
String token = "d6d7ea6e-6e71-4f13-b0ff-ed9ee9d56c37";
request.addHeader("Authorization", "Bearer "+token);
HttpResponse response = client.execute(request);
If I try it with curl it works fine:
curl "https://api.heroku.com/apps" -H"Authorization: Bearer d6d7ea6e-6e71-4f13-b0ff-ed9ee9d56c37"
Why does the java code act differently to curl?
P.S. I'm aware that others have asked this questions, but all the answers, e.g:
https://stackoverflow.com/a/7266768
https://stackoverflow.com/a/3904473
https://stackoverflow.com/a/25356821
suggest that I should override the certificate hostname check, which surely defeats the point (and certainly isn't production-ready)?
This problem is described in Apache HttpClient resolving domain to IP address and not matching certificate.
It appears to be a bug in the version of HTTPClient you are using, where it compares the target IP instead of the target hostname with the subject certificate. Please use a fixed version of HTTPClient instead.
I'm developing a mobile app that involves parsing data from JSON object via Kimono Labs. I am having problem when it comes to SSL and certificates when accessing the APIs at Kimono Labs.
According to Kimono Labs, calling the API requires setting the header "Authorization: Bearer " header at the call.
GET Request
Calling an auth API using a GET request is just like calling any kimono api, but you will additionally need to pass your secure token in the Authentication request header, like so: Authorization: Bearer {YOUR_SECURE_TOKEN}
I'm using the following code to get InputStream
HttpUriRequest request = new HttpGet(apiURL);
request.addHeader("authorization", "Bearer " + securityToken);
HttpClient httpclient = new DefaultHttpClient();
InputStream is = httpclient.execute(request).getEntity().getContent();
Executing this code gives me an exception: javax.net.ssl.SSLPeerUnverifiedException: No peer certificate
Alternatively I tried implementing with the following code:
URL url = new URL(apiURL);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestProperty("Authorization", "Bearer " + securityToken);
urlConnection.setRequestMethod("GET"); urlConnection.setConnectTimeout(activity.getResources().getInteger(R.integer.internet_timeout)); urlConnection.setReadTimeout(activity.getResources().getInteger(R.integer.internet_timeout));
InputStream inputStream = urlConnection.getInputStream();
With this code I am having the exception javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
Any ideas?
I am able to "hack" solve it by allowing all certificates via CustomTrustManager and CustomAllVerifier returning true for all attempts but heard this is totally unsafe and is not recommended for production. Is this true in my case?
Thank you in advance.
I ran into exactly the same problem with a node.js build using kimono as api. The problem is that the intermediate signed cert is not trusted by Mozilla, ergo nodejs throws a leaf error.
I solved it by doing the following, find which cert is being used, download the pem for it and add it to your root CA's before doing the request to kimono, its a hell of a lot safer than disabling TLS/SSL security checks.
Just a heads up! Turned out that this is a temporary problem with kimonolabs' server.
I'm trying to use the Apache/Jakarta HttpClient 4.1.1 to connect to an arbitrary web page using the given credentials. To test this, I have a minimal install of IIS 7.5 on my dev machine running where only one authentication mode is active at a time. Basic authentication works fine, but Digest and NTLM return 401 error messages whenever I try to log in. Here is my code:
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("http://localhost/");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY,
new NTCredentials("user", "password", "", "localhost"));
if (!new File(System.getenv("windir") + "\\krb5.ini").exists()) {
List<String> authtypes = new ArrayList<String>();
authtypes.add(AuthPolicy.NTLM);
authtypes.add(AuthPolicy.DIGEST);
authtypes.add(AuthPolicy.BASIC);
httpclient.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF,
authtypes);
httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF,
authtypes);
}
localContext.setAttribute(ClientContext.CREDS_PROVIDER, credsProvider);
HttpResponse response = httpclient.execute(httpget, localContext);
System.out.println("Response code: " + response.getStatusLine());
The one thing I've noticed in Fiddler is that the hashes sent by Firefox versus by HttpClient are different, making me think that maybe IIS 7.5 is expecting stronger hashing than HttpClient provides? Any ideas? It'd be great if I could verify that this would work with NTLM. Digest would be nice too, but I can live without that if necessary.
I am not an expert on the subject but during the NTLM authentication using http components I have seen that the client needs 3 attempts in order to connect to an NTML endpoint in my case. It is kinda described here for Spnego but it is a bit different for the NTLM authentication.
For NTLM in the first attempt client will make a request with Target auth state: UNCHALLENGED and Web server returns HTTP 401 status and a header: WWW-Authenticate: NTLM
Client will check for the configured Authentication schemes, NTLM should be configured in client code.
Second attempt, client will make a request with Target auth state: CHALLENGED, and will send an authorization header with a token encoded in base64 format: Authorization: NTLM TlRMTVNTUAABAAAAAYIIogAAAAAoAAAAAAAAACgAAAAFASgKAAAADw==
Server again returns HTTP 401 status but the header: WWW-Authenticate: NTLM now is populated with encoded information.
3rd Attempt Client will use the information from WWW-Authenticate: NTLM header and will make the final request with Target auth state: HANDSHAKE and an authorisation header Authorization: NTLM which contains more information for the server.
In my case I receive an HTTP/1.1 200 OK after that.
In order to avoid all this in every request documentation at chapter 4.7.1 states that the same execution token must be used for logically related requests. For me it did not worked.
My code:
I initialize the client once in a #PostConstruct method of an EJB
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
cm.setMaxTotal(18);
cm.setDefaultMaxPerRoute(6);
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(30000)
.setConnectTimeout(30000)
.setTargetPreferredAuthSchemes(Arrays.asList(AuthSchemes.NTLM))
.setProxyPreferredAuthSchemes(Arrays.asList(AuthSchemes.BASIC))
.build();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new NTCredentials(userName, password, hostName, domainName));
// Finally we instantiate the client. Client is a thread safe object and can be used by several threads at the same time.
// Client can be used for several request. The life span of the client must be equal to the life span of this EJB.
this.httpclient = HttpClients.custom()
.setConnectionManager(cm)
.setDefaultCredentialsProvider(credentialsProvider)
.setDefaultRequestConfig(requestConfig)
.build();
Use the same client instance in every request:
HttpPost httppost = new HttpPost(endPoint.trim());
// HttpClientContext is not thread safe, one per request must be created.
HttpClientContext context = HttpClientContext.create();
response = this.httpclient.execute(httppost, context);
Deallocate the resources and return the connection back to connection manager, at the #PreDestroy method of my EJB:
this.httpclient.close();
I had the same problem with HttpClient4.1.X After upgrading it to
HttpClient 4.2.6 it woked like charm. Below is my code
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("url");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY,
new NTCredentials("username", "pwd", "", "domain"));
List<String> authtypes = new ArrayList<String>();
authtypes.add(AuthPolicy.NTLM);
httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF,authtypes);
localContext.setAttribute(ClientContext.CREDS_PROVIDER, credsProvider);
HttpResponse response = httpclient.execute(httpget, localContext);
HttpEntity entity=response.getEntity();
The easiest way troubleshoot such situations I found is Wireshark. It is a very big hammer, but it really will show you everything. Install it, make sure your server is on another machine (does not work with Localhost) and start logging.
Run your request that fails, run one that works. Then, filter by http (just put http in the filter field), find the first GET request, find the other GET request and compare. Identify meaningful difference, you now have specific keywords or issues to search code/net for. If not enough, narrow down to first TCP conversation and look at full request/response. Same with the other one.
I solved an unbelievable number of problems with that approach. And Wireshark is very useful tool to know. Lots of super-advanced functions to make your network debugging easier.
You can also run it on either client or server end. Whatever will show you both requests to allow you to compare.
I had a similar problem with HttpClient 4.1.2. For me, it was resolved by reverting to HttpClient 4.0.3. I could never get NTLM working with 4.1.2 using either the built-in implementation or using JCIFS.
Updating our application to use the jars in the httpcomponents-client-4.5.1 resolved this issue for me.
I finally figured it out. Digest authentication requires that if you use a full URL in the request, the proxy also needs to use the full URL. I did not leave the proxy code in the sample, but it was directed to "localhost", which caused it to fail. Changing this to 127.0.0.1 made it work.