We Keep Coding

Java : HTTP(S)/WebServices connections through NTLM proxy

We have a java client side application deployed in our customer (a java application, not an applet).
This application checks connectivity with an url.openConnection() and calls web services (with CXF/JAX-WS) through internet.
Some of our customer network use proxies to access to the external world.
The client side application sets the proxy parameter in java system properties :
System.setProperty("proxySet", "true"); //Obsolete ?
System.setProperty("http.keepAlive", "false");
System.setProperty("java.net.useSystemProxies", "false");
System.setProperty("https.proxyHost", httpsProxyHost);
System.setProperty("https.proxyPort", httpsProxyPort);
System.setProperty("https.proxyUser", httpsProxyUser);
System.setProperty("https.proxyPassword", httpsProxyPassword);
System.setProperty("http.proxyHost", httpProxyHost);
System.setProperty("http.proxyPort", httpProxyPort);
System.setProperty("http.proxyUser", httpProxyUser);
System.setProperty("http.proxyPassword", httpProxyPassword);
Authenticator.setDefault(new NtlmAuthenticator(httpsProxyUser, httpsProxyPassword));
The NtlmAuthenticator class :
public class NtlmAuthenticator extends Authenticator {
private final String username;
private final char[] password;
public NtlmAuthenticator(final String username, final String password) {
super();
this.username = username;
this.password = password.toCharArray();
}
public PasswordAuthentication getPasswordAuthentication() {
return (new PasswordAuthentication (username, password));
}
}
We're using Java 6 (client side application embbed a JRE 1.6.0_39), and application is deployed on Windows (XP / Seven). I read that NTLM protocol is supported since 1.4.2 on Windows platform.
So we made tests with a Trend proxy and succeed to perform NTLM proxy authentication (we see the 3 packets with Wireshark NTLMSSP_NEGOCIATE (from app) / NTLMSSP_CHALLENGE (from proxy) / NTLMSSP_AUTH (from app))
But with one of our customers, who use a Bluecoat proxy, NTLM authentication failed after NTLMSSP_CHALLENGE. With Wireshark, we only see the 2 first packets NTLMSSP_NEGOCIATE (from app) and NTLMSSP_CHALLENGE (from proxy), NTLMSSP_AUTH is never sent by our application.
In the application we catch a SocketException : socket is closed
We also try to use jCIFS HttpUrlNltmHandler, but authentication failed too (same diagnostic).
I found this thread with similar issue but it doesn't provide any clues.
I found also this thread about NTLM session security
Any ideas ?
Thanks.
Find the solution just by setting http.keepalive to true :
System.setProperty("http.keepAlive", "true");
But i don't know why, with false value, it works with our Trend proxy and doesn't work with our customer's bluecoat proxy

It's due to a bug on the underlying implementation. It's described on Java 6 NTLM proxy authentication and HTTPS - has anyone got it to work?

Is there any way with apache httpclient to set credentials based of a URI

Currently I have some uris that I call using resteasy that That looks something like this:
http://host.com/api/project1/getsomestuff
and
http://host.com/api/project2/getsomestuff
Both use digest authentication but require a different username and password for authentication. Currently in code I have to handle this by creating a different client instance for each project like this:
DefaultHttpClient project1Client = new DefaultHttpClient();
Credentials project1Credentials = new UsernamePasswordCredentials("user1", "password1");
project1Client.getCredentialsProvider().setCredentials(AuthScope.ANY, project1Credentials);
ClientExecutor executor1 = new ApacheHttpClient4Executor(project1Client);
MyService project1Proxy = ProxyFactory.create(MyService.class, executor1);
project1Proxy.getSomeStuff("project1");
DefaultHttpClient project2Client = new DefaultHttpClient();
Credentials project2Credentials = new UsernamePasswordCredentials("user2", "password2");
project2Client.getCredentialsProvider().setCredentials(AuthScope.ANY, project1Credentials);
ClientExecutor executor2 = new ApacheHttpClient4Executor(project1Client);
MyService project2Proxy = ProxyFactory.create(MyService.class, executor2);
project2Proxy.getSomeStuff("project2");
I have looked through AuthScope and CredentialsProvider and I just can't see any way how this would be possible. Does anybody know of any way to use both sets of credentials using the same client or am I stuck using multiple clients?
Also, please note that I do not control the server side of this rest call and both calls use the same realm so I am unable to set the different credentials per realm.

If those applications have a different security contexts they are also likely to have different authentication realms (or at least they are expected to). One can explicitly set the realm attribute on the AuthScope object to explicitly match it to a specific authentication context with the given realm name.

I think you are out of luck. Looking through the docs you cannot create an Authscope which sends a different password to different paths on the same host and port.

Java Proxy Authentication

I have a Java webapp, running in Tomcat 6, that loads RSS feeds from remote URLs.
I use Rome to handle the RSS feeds and different formats for me. The connection part looks like like that :
try{
feedSource = new URL(rssObject.getAsset());
}catch(MalformedURLException mue){
logger.error(...);
throw mue;
}
try{
URLConnection connection = feedSource.openConnection();
feed = new SyndFeedInput().build(new XmlReader(connection));
}catch(Exception){handle...}
The code works fine, except at this new client, where they use a proxy.
In order to use the proxy, I set the http.proxyHost and proxyPort system properties :
System.setProperty("http.proxyHost", proxyHost);
System.setProperty("http.proxyPort", proxyPort);
System.setProperty("https.proxyHost", proxyHost);
System.setProperty("https.proxyPort", proxyPort);
HTTP GET is made to the proxy alright, but now I get a HTTP 502 error (bad gateway or something similar).
Analysing the HTTP exchange with Wireshark, I noticed that the proxy is requiring authentication. It sends a HTTP 507. Java is somehow trying to authenticate but it uses the wrong username and passwords. It seems to use the host name as the username, as for the password I don't know.
So I tried to implement the Authenticator method of specifying a username+password :
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
logger.info(MessageFormat.format("Generating PasswordAuthentitcation for proxy authentication, using username={0} and password={1}.", username, password));
return new PasswordAuthentication(username, password.toCharArray());
}
});
Now my problem is that it is ignored. The getPasswordAuthentication method is never called. I don't see the logging statement in the log file and using Wireshark I can see that it still uses the host name as the user name.
Why ? It seems that java somehow tries to authenticate by itself without consulting the Authenticator.
The proxy seems to be a MS device that uses NTLM for authentication. Is there some built-in mechanism in java to handle this ? The machine on which the app runs is Win Server 2008 R2.

We did the same here for authenticating on a NTLM based proxy.
The authentication on the proxy is actually a normal HTTP Basic Authentication.
We used the following method:
protected URLConnection newURLConnection(URL pURL) throws IOException {
URLConnection urlConnection = super.newURLConnection(pURL);
String auth = new String(Base64.base64Encode(new String("username:password").getBytes()));
auth = "Basic " + auth;
urlConnection.setRequestProperty("Proxy-Connection","Keep-Alive");
urlConnection.setRequestProperty("Proxy-Authorization",auth);
return urlConnection;
}
That, together with the proxy jvm settings, did the trick.
See http://en.wikipedia.org/wiki/Basic_access_authentication.

Simple Kerberos client in Java?

Applications such a Google's Chrome and IE can transparently handle Kerberos authentication; however I can not find a "simple" Java solution to match this transparency. All of the solutions I have found require the presence of a krb5.conf file and a login.conf file which nether of the above apps seem to require.
What is the best way to build a Java application with Kerberos SSO capabilities that just work?
[update]: to be clear I need a CLIENT side solution for creating tickets not validating them. Also, it seems that SPNEGO is the default "wrapper" protocol that will eventually delegate to Kerberos but I need to be able to handle the SPNEGO protocol as well.

There is now a simple solution for this using the Apache HTTP Components Client 4.5 or greater. This is still marked as experimental in 4.5 so your milage may vary, but this is working fine for me in an enterprise context.
In addition to the HC 4.5 client jars you will need to have the httpclient-win, jna and jna-platform jars on your classpath, as provided with http-component-client. You then construct a Kerberos enabled HC-client as follows:
CloseableHttpClient httpclient = WinHttpClients.createDefault();
Or using the builder:
HttpClientBuilder clientBuilder = WinHttpClients.custom();
Which can then be customised as required before building the client:
CloseableHttpClient client = clientBuilder.build();
This solution works without any external configuration, and most importantly solves the issue where the in-built JRE mechanism breaks for users with local Admin rights on Windows 7+. This is possible because the Kerberos ticket is being retrieved directly from the SSPI API via JNA, rather than going through the GSSAPI provided by the JRE.
Example code from the http-components team
This was all made possible by the good work of Daniel Doubrovkine Timothy Wall
and Ryan McKinley

Adding to David Roussels answer on url specific http based kerberos authentication:-
The reason why your code works is because your target SPN(server side principal) is configured to with HTTP/serverhostname.realm.com#DOMAIN.COM. In that case it will work because you are not explicitly setting the token. URLConnection internally sets a token with that SPN
1 Perform steps(from my previous answer) to get a subject
2 Use gss api init sec context to generate a context token. There are numerous tutorials out there for this step
3 Base 64 encode the token
4 Attach the token to urlconnection:-
URL url = new URL("http://myhost/myapp")
HttpURLConnection urlConn = (HttpURLConnection)url.openConnection(); =
urlConn.setRequestProperty("Authorization", "Negotiate " + encodedToken);
5 Implement a priviledged action:-
//this internally calls the getInputStream
public class PrivilegedGetInputStream implements PrivilegedExceptionAction<InputStream>
6 Wrap the whole thing in Subject.doAs
//use prev answer instructions to get subject
Subject.doAs(subject, new PrivilegedGetInputStream(urlConnection)

Oracle has an example using Java's SaslClient. I'm not a Java programmer, but when I pointed this out once to someone who is, they were able to make it work pretty quickly. It may still require a "conf" file somewhere (n.b. Kerberos uses environment variables, often starting with KRB5_, to know where to look for such files). Also note that Kerberos itself does not include a transport of any kind--your app needs to know how to send and receive the Kerberos payloads the way the server expects (and this is different depending on the server you are trying to authenticate with).
Edit: you edited your question, so here's a link related to SPNEGO in Java which might be of some use:
http://download.oracle.com/javase/6/docs/technotes/guides/security/jgss/lab/part5.html

You don't actually need to do anything. In Java 6, on a Windows client machine you can do this:
new URL("http://myhost/myapp").openStream();
And negotiate authentication just works. At least it does for me. And the server I tested on only supports Negotiate, not NTLM auth.

Ok if you want to avoid using a login.conf file you need to code differently:-
//define your own configuration
import javax.security.auth.login.Configuration;
public class CustomLoginConfiguration extends Configuration
//pass certain parameters to its constructor
//define an config entry
import javax.security.auth.login.AppConfigurationEntry;
private AppConfigurationEntry configEntry;
//define a map of params you wish to pass and fill them up
//the map contains entries similar to one you have in login.conf
Map<String, String> params = new HashMap<String, String>();
//define the configuration
configEntry = new AppConfigurationEntry(
"com.sun.security.auth.module.Krb5LoginModule",
AppConfigurationEntry.LoginModuleControlFlag.REQUIRED, params);
//implement getappconfig method
public AppConfigurationEntry[] getAppConfigurationEntry() {
return new AppConfigurationEntry[] { configEntry };
}
Now once you are done with this definition you can use this in you use this to fetch tickets from kdc
//get ticket in login context
LoginContext lc = null;
lc = new LoginContext("lc", null, callback, new CustomLoginConfiguration(argumentlist));
lc.login();
Now from here on you can fetch jaas subject and can basically do a ton of authentication stuff.
In case you need further pointers just leave a comment.

You can use system properties instead of config files to specify the KDC hostname and service name, but those things (at least) are mandatory....
Waffle will actually give you the information you need to set most of the properties, even if it won't get you a ticket. Look at the WindowsAuthProviderImpl class (the Waffle.chm help file shows the API).
I use JAAS do obtain a service ticket from Active Directory in two steps:
Use Krb5LoginModule to retrieve the cached TGT and add it to the Subject.
Use the Subject and GSS-API to retrieve a service ticket from the KDC.
There's a lot of good information and example code at The Java Way of Active Directory.

I created a small tool to simplify connecting with httpclient to kerberos, you might want to give it a try.
https://github.com/DovAmir/httpclientAuthHelper
DefaultHttpClient httpclient = new DefaultHttpClient();
AuthUtils.securityLogging(SecurityLogType.KERBEROS,true);
CredentialsUtils.setKerberosCredentials(client, new UsernamePasswordCredentials("xxx", "xxx"), "domain", "kdc");
client.executeMethod(httpget);

Use WAFFLE

Here's a good blog post on having a java client to use with Kerberos
http://sachithdhanushka.blogspot.com/2014/02/kerberos-java-client-configuration.html

Java Authenticator on a per connection basis?

I'm building an Eclipse plugin that talks to a REST interface which uses Basic Authentication. When the authentication fails I would like to popup my plugin's settings dialog and retry. Normally I could use the static Authenticator.setDefault() to setup an authenticator for all HttpURLConnection's for this, but since I am writing a plugin I don't want to overwrite Eclipse's default Authenticator (org.eclipse.ui.internal.net.auth);
I thought of setting my custom Authenticator before loading and putting Eclipse's default back afterwards, but I imagine this will cause all sorts of race issues with multithreading so I quickly lost that notion.
Google searches yield all sorts of results basically telling me it's not possible:
The Java URLConnection API should have a setAuthenticator(Authenticator) method for making it easier to use this class in multi-threaded context where authentication is required.
Source
If applications contains few third party plugins and each plugin use its own Authenticator what we should do? Each invocation of "Authenticator.setDefault()" method rewrite previously defined Authenticator...
Source
Are there any different approaches that might help me overcome this issue?

If it is not possible with HttpURLConnection I would suggest using the httpclient library from Apache.
A quick example:
HttpClient client = new HttpClient();
client.getState().setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("test","test"));
GetMethod getMethod = new GetMethod("http://www.example.com/mylogin");
client.executeMethod(getMethod);
System.out.println(getMethod.getResponseBodyAsString());

Another approach would be to perform the basic authentication yourself on the connection.
final byte[] encodedBytes = Base64.encodeData((username + ':' + new String(password)).getBytes("iso-8859-1"));
final String encoded = new String(encodedBytes, "iso-8859-1");
connection.setRequestProperty("Authorization", "Basic " + encoded);
This would also have the advantage of not requiring an unauthenticated request to receive a 401 before providing the credential on a subsequent request. Similar behavior can be leveraged in the apache http-client by requesting preemptive authentication.