We Keep Coding

HttpURLConnection authentication mode

I'm using HttpURLConnection class to manage HTTP connections in my Android app. Problem is, on some servers, I have a code 401 using an URL which working on a Internet navigator. I initialize connection with basic authentication, but I think these servers need an other mode. I know it works with libCURL and the following instruction:
curl_easy_setopt(pCurlHandle, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
But is there an way to doing something like that on Android? Currently, I do that:
...
connection.setRequestProperty("Authorization", "Basic " + Base64.encode("username:password"));
...
I tried with Authenticator too:
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication()
{
return new PasswordAuthentication("username", "password".toCharArray());
}
});
But I still have a 401 in my app. Any idea?

Well, this post really helped me (but I don't use Authenticator class at all) and now my code do the job like that (if authentication is needed):
Initialize my HttpURLConnection object with Basic authentication (see the first Java sample in my question's code)
Test if I receive a code 401 and if it is, check if server was excepted for a Digest authentication analyzing response headers.
Retry with a manually-built Digest request property (see the post I mentioned)
And it works!
I hope this answer'll help someone!

NTLM authentication with httpclient 4.2.1

I need to do a HTTP GET to a URL that needs NTLM authentication. I can access the URL using Firefox or Chrome on a MacBook Pro. The browser asks for the username/password combo and it works. I am now trying to do the same from Groovy using HttpClient. I followed the NTLM support guide, but I always get a 401 Unauthorized back. There is also this sentence in the response:
You do not have permission to view this directory or page using the
credentials that you supplied because your Web browser is sending a
WWW-Authenticate header field that the Web server is not configured to
accept.
I tried all kinds of combinations for the servername and domain (the remote windows pc is not on a domain) in this piece of code, but I always get the same response.
httpclient.getCredentialsProvider().setCredentials(
new AuthScope("myserver", -1),
new NTCredentials("username", "password", "MYSERVER", "MYDOMAIN"));
Anybody had the same problem and managed to solve it? Note that this is an external program that uses IIS under the hood, so I don't think I can change any settings there.
EDIT:
Unlike what I have said, I managed to change the security settings in IIS to accept BASIC authentation, so I don't have the problem anymore.

EDIT:
In my experience with setting up Kerberos or NTLM (both are single sign on), you don't have to enter username/password at all when you are already logged in to your system.
I am pretty sure that when the browser asked for username/password combo, that's not an NTLM authentication at all. Most likely the server side application has a fallback scheme to HTTP Basic Digest (that why it displayed the username/password combo). With NTLM you'll never have to enter your username/password (principal/credentials) at all, as the server will recognize who you are through the negotiation mechanism between your browser, your operating system, server and Active Directory server.
If your MacBook Pro is running on OS/X, you also need to add your OS/X to the domain.
Your server also needs to be in the same domain where the client OS/X being added.
This may not be a trivial case. Some external tools/driver may be needed. This one may be a good candidate (but I haven't tried that).
NTLM needs both the client to be a member of the same domain as the server, hence both needs to be registered in the Active Directory domain. If your server is not in the domain, than that will be another set of problem.
In order to get your browser works with NTLM, you need to install plugin (ntlmauth-plugin?). But I have never try that on MacOS/X yet. Even in Windows you still need a plugin in order to run Firefox successfully with NTLM.

HttpClient did not work for me but finally the code below worked.
Reference - http://docs.oracle.com/javase/7/docs/technotes/guides/net/http-auth.html
For quick reference -
public static String getResponse(String url, String userName, String password) throws IOException {
Authenticator.setDefault(new Authenticator() {
#Override
public PasswordAuthentication getPasswordAuthentication() {
System.out.println(getRequestingScheme() + " authentication");
return new PasswordAuthentication(userName, password.toCharArray());
}
});
URL urlRequest = new URL(url);
HttpURLConnection conn = (HttpURLConnection) urlRequest.openConnection();
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("GET");
StringBuilder response = new StringBuilder();
InputStream stream = conn.getInputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(stream));
String str = "";
while ((str = in.readLine()) != null) {
response.append(str);
}
in.close();
return response.toString();
}

301/302 Redirect not working in Android (work differently in different versions)

When using a URLConnection, the 301 redirect doesn't work, doesn't even show a Location header, using getHeaderFields(). It is a blank list, except in newer Android (I tested 4.1 and it worked). It looks like something this has been reported in the default browser here as well, though in my test it worked in the Android browser. Is there some workaround for this bug in older Android?
I tried:
URLConnection conn = u.openConnection();
HttpURLConnection conn = (HttpURLConnection) u.openConnection();
(conn).setInstanceFollowRedirects(true);
but it still returns an empty list, except in newer Android.
Update: It may be a related issue, it seems sometimes the URLConnection isn't even sending a request in some cases. (I checked with Wireshark on a pc with emulator). Is there a way to work-around this bug?
Update: I tried testing for 3xx redirect, redirects worked fine, but normal links didn't work with Ian's Cookie Manager. After making sure the setCookies was called directly after openConnection, it works great:
URL u = new URL(_url);
...
int tries = 4;
int code = 301;
URLConnection conn = null;
while (tries > 0 && code/100 == 3) {
conn = null;
conn = u.openConnection();
_CM.setCookies(conn);
((HttpURLConnection)conn).setInstanceFollowRedirects(false);//Required
code =((HttpURLConnection)conn).getResponseCode();
if (code/100 == 3) {
String loc = conn.getHeaderField("Location");
u = new URL(loc);
}
}
//conn.addRequestProperty("Accept-Encoding", "gzip");
conn.connect();
_CM.storeCookies(conn);
The really strange thing is, for newer Android (4.1 emulator) the FollowRedirect line (commented "Required") is not necessary. On older Android (2.2), it gives Connection Reset by Peer error. This was probably the reason my redirect experimental code was failing on 2.2, not 4.1. Any reason for the differences in functionality? According to comments here, redirection https apparently has different behavior depending on the JVM version, could it be that Android's URLConnection/HTTPUrlConnection has changed in different versions as well?

Not sure about URLConnection, but I know that HttpClient honors redirects and we use it all the way back to Android 2.1
http://developer.android.com/reference/org/apache/http/client/HttpClient.html
(Based on apache commons HttpClient)

Can I override the Host header where using java's HttpUrlConnection class?

I'm using the following code to open a http connection in java:
URL url = new URL("http://stackoverflow.com");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("GET");
conn.setRequestProperty("Host", "Test:8080");
conn.getOutputStream();
However calling conn.setRequestProperty("Host", "Test:8080") appears to have no effect regardless of what order I call the methods and the Host is reset to the destination server. Is there any way to override the Host header without using a different library?
TIA Matt

This used to work in the past, but it has been disabled as part of a security-fix. Apparently without a note in the changelog. There are even bugs like #7022056 for this at bugs.sun.com.
There is a similar question for another header, where the answer goes more into the details, so I just link it instead of writing it myself. :-)
The only workarounds seem to be setting sun.net.http.allowRestrictedHeaders to true or use another http-library like the already mentioned http components.

The Host header is filled by the HttpURLConnection based on the URL. You can't open foo.com with Host=bar.com. From the RFC
The Host request-header field specifies the Internet host and port number of the resource being requested, as obtained from the original URI given by the user or referring resource (generally an HTTP URL)
Btw, you can also try apache http components.

This is an issue with how volley handles HTTPUrlConnection and retry policy.
A Quick fix for it is to extend "HurlStack" class and override the "createConnection" function to return a HTTPUrlConnection with ChunkStreamMode of 0
public class CustomHurlStack extends HurlStack {
public CustomHurlStack(){
super();
}
#Override
protected HttpURLConnection createConnection(URL url) throws IOException {
HttpURLConnection connection = super.createConnection(url);
connection.setChunkedStreamingMode(0);
return connection;
}
}

Using java class HttpsURLConnection

I have a small piece of code which basically impements a HTTP-Client,
i.e. it POSTS request and works with re RESPONSE. As long as HTTP is
concenerned everthing work well. For some reason I now have to support
HTTPS too. So here is briefly what I do in order to get a connection opened:
URL url = new URL(serverAddress);
HttpsURLConnection httpsConn = (HttpsURLConnection) url.openConnection();
This fails, stating:
sun.net.www.protocol.https.HttpsURLConnectionImpl cannot be cast to com.sun.net.ssl.HttpsURLConnection
I guess this is kinda trivial, but I just don't get what I'm doing wrong in this one...
Googled it, and the code just looks right - not?
any ideas are appreciated!

Just keep it java.net.URLConnection or cast it to java.net.HttpURLConnection instead. Both offers methods to do the desired task as good.
A side remark unrelated to the technical problem: you should never explicitly import/use Sun Java SE implementation specific classes in your code. Those are undocumented classes and are subject to changes which may cause your code break when you upgrade the JVM. On the other hand, your code may also break when you run it at a different brand JVM.
Update: since you seem to accidentally have imported it, go to Window > Preferences > Java > Appearance > Type Filters and Add com.sun.* and sun.* to the list. This way you won't ever import them accidentally:

Your url's protocol should also be https and not http. Check your url.

The above problem is only caused by two issues
Using wrong import
Using http in the string you create url from use instead https

Instead of creating a URL object using standard constructor like
URL wsURL = new URL(url);
Use
java.net.URL wsURL = new URL(null, url,new sun.net.www.protocol.https.Handler());
which would solve this problem

Check your imports, you should be using
java.net.HttpURLConnection
or
javax.net.ssl.HttpsURLConnection

Check value of your "serverAddress" variable. It should https and not http

Change:
import javax.net.ssl.HttpsURLConnection;
and
URL url = new URL(serverAddress);
HttpsURLConnection httpsConn = (HttpsURLConnection) url.openConnection();
Change To:
import java.net.HttpURLConnection;
and
URL url = new URL(serverAddress);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();

Hard to tell without seeing the whole file, but it looks like you're importing com.sun.net.ssl.HttpsURLConnection when you really want javax.net.ssl.HttpsURLConnection.

In my case, the protocol and port were not correct while invoking the httpsUrlConnection.
Port and protocol were defined as static class variables. And the step prior to the failed step, was invoking an httpUrlConnection. That method changed the port/protocol to 80/http, but didn't set it back to /https at the end. So eventhough httpsUrlConnection was invoked, it was still using http/80. Once I reset those at the end of the httpUrlConnection method, the error disappeared.