We Keep Coding

Unable to load data from URL using HTTP Proxy Android

I'm using java.net.URL and java.net.Proxy to load my app's data from the desired URL. When I am using Proxy.Type.SOCKS proxy, everything is OK, but I need to use Proxy.Type.HTTP. My code is the following:
URL url = new URL("https://google.com/");
final Proxy proxy = getProxy();
HttpURLConnection connection = (HttpURLConnection) url.openConnection(proxy);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
Where getProxy (simplified) looks as follows:
private static Proxy getProxy(){
if(!StringUtils.isNullOrBlank(username)){
Authenticator.setDefault(new Authenticator() {
#Override
public PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password.toCharArray());
}
});
} else {
Authenticator.setDefault(null);
}
return new Proxy(Proxy.Type.SOCKS HTTP, new InetSocketAddress(address, port));
}
I am getting java.io.IOException: unexpected end of stream on null, Caused by: java.io.EOFException: \n not found: size=0 content=....
System.setProperty option doesn't seem to work, as well.
Is there any way to use HTTP proxy correctly on Android?

Server returned HTTP response code: 401 for URL: https

I'm using Java to access a HTTPS site which returns the display in an XML format. I pass the login credentials in the URL itself. Here is the code snippet:
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
requestURL = "https://Administrator:Password#localhost:8443/abcd";
try {
InputStream is = null;
URL url = new URL(requestURL);
InputStream xmlInputStream =new URL(requestURL).openConnection().getInputStream();
byte[] testByteArr = new byte[xmlInputStream.available()];
xmlInputStream.read(testByteArr);
System.out.println(new String(testByteArr));
Document doc = db.parse(xmlInputStream);
System.out.println("DOC="+doc);
} catch (MalformedURLException e) {
}
I'm creating a trust manager in the program which does not validate signed/unsigned certificates. But, on running the above program, I get the error
Server returned HTTP response code: 401 for URL: https://Administrator:Password#localhost:8443/abcd
I can use the same url on my browser and it displays the xml correctly. Kindly let me know how to make this work within the Java program.

401 means "Unauthorized", so there must be something with your credentials.
I think that java URL does not support the syntax you are showing. You could use an Authenticator instead.
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(login, password.toCharArray());
}
});
and then simply invoking the regular url, without the credentials.
The other option is to provide the credentials in a Header:
String loginPassword = login+ ":" + password;
String encoded = new sun.misc.BASE64Encoder().encode (loginPassword.getBytes());
URLConnection conn = url.openConnection();
conn.setRequestProperty ("Authorization", "Basic " + encoded);
PS: It is not recommended to use that Base64Encoder but this is only to show a quick solution. If you want to keep that solution, look for a library that does. There are plenty.

Try This. You need pass the authentication to let the server know its a valid user. You need to import these two packages and has to include a jersy jar. If you dont want to include jersy jar then import this package
import sun.misc.BASE64Encoder;
import com.sun.jersey.core.util.Base64;
import sun.net.www.protocol.http.HttpURLConnection;
and then,
String encodedAuthorizedUser = getAuthantication("username", "password");
URL url = new URL("Your Valid Jira URL");
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
httpCon.setRequestProperty ("Authorization", "Basic " + encodedAuthorizedUser );
public String getAuthantication(String username, String password) {
String auth = new String(Base64.encode(username + ":" + password));
return auth;
}

Java HTTP PUT with Digest authentication in Java

I am trying to upload a file with Java using PUT, server does Digest authentication. I want to keep it lean, so I try to use HttpURLConnection.
public void putData(String path, byte [] data) throws IOException, MalformedURLException {
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(user,password.toCharArray());
}});
debug("Default authenticator set");
//Safeguard against double slashes
if (path.startsWith("/")) {
path = path.replaceFirst("/","");
}
debug(hostname + path);
URL url = new URL(hostname + path);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
debug("HttpURLConnection acquired");
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("PUT");
conn.setRequestProperty("Content-Length",String.valueOf(data.length));
conn.setRequestProperty("Content-type","application/binary");
conn.setFixedLengthStreamingMode(data.length);
conn.connect();
debug("Properties set");
OutputStream out = conn.getOutputStream();
debug("Outputstrem acquired");
out.write(data,0,data.length);
out.flush();
out.close();
debug("Data written, stream closed.");
}
For some reason, this fails hopelessly: I see a 401 coming back, and then it's done. If I disable authorization, same code works. Downloading a file with similar code using digest authentication "just works". Any ideas? I'd really rather not start using the next so many libraries like htclient from Apache or so (...it's 2010... you'd expect http requests with digest authN to work in any standard library).

You should at least try conn.getInputStream() at the end of your method, to force evaluation of the server response. Otherwise, potential error messages from the server will not be detected properly.

Using HTTP Basic-Auth with Google App Engine URLFetch service

How can I specify the username and password for making Basic-Auth requests with App Engine's URLFetch service (in Java)?
It seems I can set HTTP headers:
URL url = new URL("http://www.example.com/comment");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("X-MyApp-Version", "2.7.3");
What are the appropriate headers for Basic-Auth?

This is a basic auth header over http:
Authorization: Basic base64 encoded(username:password)
eg:
GET /private/index.html HTTP/1.0
Host: myhost.com
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
You will need to do this:
URL url = new URL("http://www.example.com/comment");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("Authorization",
"Basic "+codec.encodeBase64String(("username:password").getBytes());
And to do that you will want to get a base64 codec api, like the Apache Commons Codec

For those interested in doing this in Python (as I was), the code looks like this:
result = urlfetch.fetch("http://www.example.com/comment",
headers={"Authorization":
"Basic %s" % base64.b64encode("username:pass")})

You set up an Authenticator before you call openConnection() like this,
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password.toCharArray());
}
});
Since there is only one global default authenticator, this doesn't really work well when you have multiple users doing the URLFetch in multiple threads. I would use Apache HttpClient if that's the case.
EDIT: I was wrong. App Engine doesn't allow Authenticator. Even if it's allowed, we would have the multi-thread issue with a global authenticator instance. Even though you can't create threads, your requests may still get served in different threads. So we just add the header manually using this function,
import com.google.appengine.repackaged.com.google.common.util.Base64;
/**
* Preemptively set the Authorization header to use Basic Auth.
* #param connection The HTTP connection
* #param username Username
* #param password Password
*/
public static void setBasicAuth(HttpURLConnection connection,
String username, String password) {
StringBuilder buf = new StringBuilder(username);
buf.append(':');
buf.append(password);
byte[] bytes = null;
try {
bytes = buf.toString().getBytes("ISO-8859-1");
} catch (java.io.UnsupportedEncodingException uee) {
assert false;
}
String header = "Basic " + Base64.encode(bytes);
connection.setRequestProperty("Authorization", header);
}

Using HttpURLConnection gave me some problems (for some reason the server I was trying to connect to didn't accept auth credentials), and finally I realized that it's actually much easier to do using GAE's low-level URLFetch API (com.google.appengine.api.urlfetch) like so:
URL fetchurl = new URL(url);
String nameAndPassword = credentials.get("name")+":"+credentials.get("password");
String authorizationString = "Basic " + Base64.encode(nameAndPassword.getBytes());
HTTPRequest request = new HTTPRequest(fetchurl);
request.addHeader(new HTTPHeader("Authorization", authorizationString));
HTTPResponse response = URLFetchServiceFactory.getURLFetchService().fetch(request);
System.out.println(new String(response.getContent()));
This worked.

There is a wrapper on Apache HttpClient for App Engine
please go through the post http://esxx.blogspot.com/2009/06/using-apaches-httpclient-on-google-app.html
http://peterkenji.blogspot.com/2009/08/using-apache-httpclient-4-with-google.html

Note on the first answer: setRequestProperty should get the property name without the colon ("Authorization" rather than "Authorization:").

Logging off with java.net.Authenticator

I was wondering if any of you know how to 'log-off' basic authentication (BA) using the java.net.Authenticator class. I know that BA doesn't have a log-off method, and that you have to close and reopen the browser to end the session. Question is, how do you 'close and reopen the browser' within java code? That is, I'm connecting via java, not a borwser, so how to I get the JVM to 'deauthenticate' itself?
Context: I'm writing an app to post tweets to multiple twitter accounts using BA. I can use the java.net.* to post the tweet for one account (and can see that it calls my authenticator class) but when I try and post the tweet for the second, I can't see any second call to the authenticator, and the tweet get's fired off to the first account.
Is it possible to get the Authenticator to re-authenticate, or is this a dead end?If so, I may just end up using OAuth instead.
Many thanks in advance for any insight you can offer!
Shane
static class MyAuthenticator extends Authenticator {
private String username, password;
public MyAuthenticator(String user, String pass) {
username = user;
password = pass;
}
protected PasswordAuthentication getPasswordAuthentication() {
System.out.println("Requesting Host : " + getRequestingHost());
System.out.println("Requesting Scheme : " + getRequestingScheme());
System.out.println("Requesting Site : " + getRequestingSite());
return new PasswordAuthentication(username, password.toCharArray());
}
}
public void tweet(AutoTwitterAccount acc, String tweet) {
Authenticator.setDefault(null);
Authenticator.setDefault(new MyAuthenticator(acc.getUserName(), acc.getPassword()));
try {
/* First login the session and fire off tweet*/
URL url = new URL(AutoTweeter.TWEET_URL);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("User-Agent", "Mozilla/4.0");
conn.setRequestProperty("User-Agent", AGENT);
conn.setRequestProperty("Content-Type", TYPE);
conn.setRequestProperty("Content-Length", "" + tweet.length());
...fire off tweet....
conn.disconnect();
Thanks once again!

If the Authenticator indeed does not get called again (and even resetting it to another one does not work, which apparently is the case due to a bug), then you can forgo Authenticator and send the HTTP Basic Auth header manually:
URL url = new URL("http://www.example.com/comment");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("Authorization:",
"Basic "+codec.encodeBase64String(("username:password").getBytes());

I tried that but it still wouldn't authenticate, but I've gotten around this by using a co-workers utility class to construct HTTP request myself and writing it to the socket directly, bypassing Sun's Http classes. But thanks for your answer anyway!

If you are using Authenticator for a Proxy, try the following:
import org.apache.commons.codec.binary.Base64;
...
StringBuilder auth = new StringBuilder("Basic ");
auth.append(Base64.encodeBase64String((username + ':' + password).getBytes()));
connection.setRequestProperty("Proxy-Authorization", auth.toString());