I need to do a HTTP GET to a URL that needs NTLM authentication. I can access the URL using Firefox or Chrome on a MacBook Pro. The browser asks for the username/password combo and it works. I am now trying to do the same from Groovy using HttpClient. I followed the NTLM support guide, but I always get a 401 Unauthorized back. There is also this sentence in the response:
You do not have permission to view this directory or page using the
credentials that you supplied because your Web browser is sending a
WWW-Authenticate header field that the Web server is not configured to
accept.
I tried all kinds of combinations for the servername and domain (the remote windows pc is not on a domain) in this piece of code, but I always get the same response.
httpclient.getCredentialsProvider().setCredentials(
new AuthScope("myserver", -1),
new NTCredentials("username", "password", "MYSERVER", "MYDOMAIN"));
Anybody had the same problem and managed to solve it? Note that this is an external program that uses IIS under the hood, so I don't think I can change any settings there.
EDIT:
Unlike what I have said, I managed to change the security settings in IIS to accept BASIC authentation, so I don't have the problem anymore.
EDIT:
In my experience with setting up Kerberos or NTLM (both are single sign on), you don't have to enter username/password at all when you are already logged in to your system.
I am pretty sure that when the browser asked for username/password combo, that's not an NTLM authentication at all. Most likely the server side application has a fallback scheme to HTTP Basic Digest (that why it displayed the username/password combo). With NTLM you'll never have to enter your username/password (principal/credentials) at all, as the server will recognize who you are through the negotiation mechanism between your browser, your operating system, server and Active Directory server.
If your MacBook Pro is running on OS/X, you also need to add your OS/X to the domain.
Your server also needs to be in the same domain where the client OS/X being added.
This may not be a trivial case. Some external tools/driver may be needed. This one may be a good candidate (but I haven't tried that).
NTLM needs both the client to be a member of the same domain as the server, hence both needs to be registered in the Active Directory domain. If your server is not in the domain, than that will be another set of problem.
In order to get your browser works with NTLM, you need to install plugin (ntlmauth-plugin?). But I have never try that on MacOS/X yet. Even in Windows you still need a plugin in order to run Firefox successfully with NTLM.
HttpClient did not work for me but finally the code below worked.
Reference - http://docs.oracle.com/javase/7/docs/technotes/guides/net/http-auth.html
For quick reference -
public static String getResponse(String url, String userName, String password) throws IOException {
Authenticator.setDefault(new Authenticator() {
#Override
public PasswordAuthentication getPasswordAuthentication() {
System.out.println(getRequestingScheme() + " authentication");
return new PasswordAuthentication(userName, password.toCharArray());
}
});
URL urlRequest = new URL(url);
HttpURLConnection conn = (HttpURLConnection) urlRequest.openConnection();
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("GET");
StringBuilder response = new StringBuilder();
InputStream stream = conn.getInputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(stream));
String str = "";
while ((str = in.readLine()) != null) {
response.append(str);
}
in.close();
return response.toString();
}
Related
I have a problem with HTTP digest authentication on a Java client. I'm running a Web Service (SOAP) client on Wildfly, but the same problem occurs if I use URL.openConnection(), ie. the failing part is the underlying sun.net.www.protocol.http.HttpURLConnection and its authentication handling. I'm setting the credentials through java.net.Authenticator.
java.net.Authenticator.setDefault(new java.net.Authenticator() {
#Override
protected java.net.PasswordAuthentication getPasswordAuthentication() {
return new java.net.PasswordAuthentication("username", "password");
}
});
URL url = new URL(requestUri);
HttpURLConnection conn = (HttpURLConnection) url.openConnection(Proxy.NO_PROXY);
conn.setRequestMethod("GET");
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while (rd.readLine() != null) { }
rd.close();
All is running well initially, but after a couple of weeks, all the HTTP calls gets response '400 Bad request' from the server. If I restart the JVM all the calls work again. The only thing I can think of is that the server thinks that there is something wrong with the digest auth (such as an incorrect nonce counter) and responds with HTTP 400. HttpURLConnection (or actually Authenticator?) does not understand HTTP 400 as failed authentication and continues to use the same auth tokens. After restarting JVM the calls naturally request new server nonce and starts nonce counter from 0, and everything works again.
Is there something I can do? Is there something I'm doing wrong? Can I somehow force the underlying JRE implementation to either understand HTTP 400 as failed authentication or to force the client to request new server nonce for all requests?
The server I'm making the request to is MS Dynamics NAV. Another section of my app is making similar HTTP requests to the same server but using the Apache HTTP Client, and there are no problems whatsoever.
Update:
I'm running Oracle's Server JRE 8u152
I'm using HttpURLConnection class to manage HTTP connections in my Android app. Problem is, on some servers, I have a code 401 using an URL which working on a Internet navigator. I initialize connection with basic authentication, but I think these servers need an other mode. I know it works with libCURL and the following instruction:
curl_easy_setopt(pCurlHandle, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
But is there an way to doing something like that on Android? Currently, I do that:
...
connection.setRequestProperty("Authorization", "Basic " + Base64.encode("username:password"));
...
I tried with Authenticator too:
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication()
{
return new PasswordAuthentication("username", "password".toCharArray());
}
});
But I still have a 401 in my app. Any idea?
Well, this post really helped me (but I don't use Authenticator class at all) and now my code do the job like that (if authentication is needed):
Initialize my HttpURLConnection object with Basic authentication (see the first Java sample in my question's code)
Test if I receive a code 401 and if it is, check if server was excepted for a Digest authentication analyzing response headers.
Retry with a manually-built Digest request property (see the post I mentioned)
And it works!
I hope this answer'll help someone!
How can I get the redirected URL of http://at.atwola.com/?adlink/5113/1649059/0/2018/AdId=4041444;BnId=872;itime=15692006;impref=13880156912668385284; in Java?
My code (given below) is constructed according to answers to similar questions on stack-overflow (https://stackoverflow.com/a/5270162/1382251 in particular).
But it just yields the original URL. I suspect that there are other similar cases, so I would like to resolve this one in specific and use the solution in general.
String ref = "http://at.atwola.com/?adlink/5113/1649059/0/2018/AdId=4041444;BnId=872;itime=15692006;impref=13880156912668385284;";
try
{
URLConnection con1 = new URL(ref).openConnection();
con1.connect();
InputStream is = con1.getInputStream();
URL url = con1.getURL();
is.close();
String finalPage = url.toString();
if (finalPage.equals(ref))
{
HttpURLConnection con2 = (HttpURLConnection)con1;
con2.setInstanceFollowRedirects(false);
con2.connect();
if (con2.getResponseCode()/100 == 3)
finalPage = con2.getHeaderField("Location");
}
System.out.println(finalPage);
}
catch (Exception error)
{
System.out.println("error");
}
I played a bit with your URL with telnet, wget, and curl and I noticed that in some cases the server returns response 200 OK, and sometimes 302 Moved Temporarily. The main difference seems to be the request User-agent header. Your code works if you add the following before con1.connect():
con1.setRequestProperty("User-Agent","");
That is, with empty User-Agent (or if the header is not present at all), the server issues a redirect. With the Java User-Agent (in my case User-Agent: Java/1.7.0_45) and with the default curl User-Agent (User-Agent: curl/7.32.0) the server responds with 200 OK.
In some cases you might need to also set:
System.setProperty("http.agent", "");
See Setting user agent of a java URLConnection
The server running the site is the Adtech Adserver and apparently it is doing user agent sniffing. There is a long history of user agent sniffing. So it seems that the safest thing to do would be to set the user agent to Mozilla:
con1.setRequestProperty("User-Agent","Mozilla"); //works with your code for your URL
Maybe the safest option would be to use a user agent used by some of the popular web browsers.
I am getting Unauthorized error when accessing restful web services. My sample program looks like this.
public static void main(String[] args){
// Use apache commons-httpclient to create the request/response
HttpClient client = new HttpClient();
Credentials defaultcreds = new UsernamePasswordCredentials("aaa", "cdefg");
client.getState().setCredentials(AuthScope.ANY, defaultcreds);
GetMethod method = new GetMethod(
"http://localhost:8080/userService/usersByID/1234");
try {
client.executeMethod(method);
InputStream in = method.getResponseBodyAsStream();
// Use dom4j to parse the response and print nicely to the output stream
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder out = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
out.append(line);
}
System.out.println(out.toString());
} catch (IOException e) {
e.printStackTrace();
}
}
My credentials are correct. My web services will consume Basic Http Authentication.
I have doubt at scope of authentication.
client.getState().setCredentials(AuthScope.ANY, defaultcreds);
My credentials are correct.
Can any one help to resolve this issue.
Thanks.
First check your url via browser and verify ?? as mentioned here
Fixing 401 errors - general
Each Web Server manages user authentication in its own way. A security officer (e.g. a Web Master) at the site typically decides which users are allowed to access the URL. This person then uses Web server software to set up those users and their passwords. So if you need to access the URL (or you forgot your user ID or password), only the security officer at that site can help you. Refer any security issues direct to them.
If you think that the URL Web page *should* be accessible to all and sundry on the Internet, then a 401 message indicates a deeper problem. The first thing you can do is check your URL via a Web browser. This browser should be running on a computer to which you have never previously identified yourself in any way, and you should avoid authentication (passwords etc.) that you have used previously. Ideally all this should be done over a completely different Internet connection to any you have used before (e.g. a different ISP dial-up connection). In short, you are trying to get the same behaviour a total stranger would get if they surfed the Internet to the Web page.
If this type of browser check indicates no authority problems, then it is possible that the Web server (or surrounding systems) have been configured to disallow certain patterns of HTTP traffic. In other words, HTTP communication from a well-known Web browser is allowed, but automated communication from other systems is rejected with an 401 error code. This is unusual, but may indicate a very defensive security policy around the Web server.
Manual Fix
Hit the url from the browser and record the HTTP traffic (Headers,body)
Run the Java client code and record the HTTP traffic (Headers,body)
Analyze and fix the differences
I have a Java webapp, running in Tomcat 6, that loads RSS feeds from remote URLs.
I use Rome to handle the RSS feeds and different formats for me. The connection part looks like like that :
try{
feedSource = new URL(rssObject.getAsset());
}catch(MalformedURLException mue){
logger.error(...);
throw mue;
}
try{
URLConnection connection = feedSource.openConnection();
feed = new SyndFeedInput().build(new XmlReader(connection));
}catch(Exception){handle...}
The code works fine, except at this new client, where they use a proxy.
In order to use the proxy, I set the http.proxyHost and proxyPort system properties :
System.setProperty("http.proxyHost", proxyHost);
System.setProperty("http.proxyPort", proxyPort);
System.setProperty("https.proxyHost", proxyHost);
System.setProperty("https.proxyPort", proxyPort);
HTTP GET is made to the proxy alright, but now I get a HTTP 502 error (bad gateway or something similar).
Analysing the HTTP exchange with Wireshark, I noticed that the proxy is requiring authentication. It sends a HTTP 507. Java is somehow trying to authenticate but it uses the wrong username and passwords. It seems to use the host name as the username, as for the password I don't know.
So I tried to implement the Authenticator method of specifying a username+password :
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
logger.info(MessageFormat.format("Generating PasswordAuthentitcation for proxy authentication, using username={0} and password={1}.", username, password));
return new PasswordAuthentication(username, password.toCharArray());
}
});
Now my problem is that it is ignored. The getPasswordAuthentication method is never called. I don't see the logging statement in the log file and using Wireshark I can see that it still uses the host name as the user name.
Why ? It seems that java somehow tries to authenticate by itself without consulting the Authenticator.
The proxy seems to be a MS device that uses NTLM for authentication. Is there some built-in mechanism in java to handle this ? The machine on which the app runs is Win Server 2008 R2.
We did the same here for authenticating on a NTLM based proxy.
The authentication on the proxy is actually a normal HTTP Basic Authentication.
We used the following method:
protected URLConnection newURLConnection(URL pURL) throws IOException {
URLConnection urlConnection = super.newURLConnection(pURL);
String auth = new String(Base64.base64Encode(new String("username:password").getBytes()));
auth = "Basic " + auth;
urlConnection.setRequestProperty("Proxy-Connection","Keep-Alive");
urlConnection.setRequestProperty("Proxy-Authorization",auth);
return urlConnection;
}
That, together with the proxy jvm settings, did the trick.
See http://en.wikipedia.org/wiki/Basic_access_authentication.