We Keep Coding

HTTP connection made in Java fails with too many redirects?

I've got a simple Tomcat-based Java app that functions as a sort of firewall - I take requests from the "outside", reroute them to resources on the "inside", and return the result to the "outside."
This works fine for GETs, but I'm trying to add a POST function for a different request and I cannot get it working. The "inside" remote server is password protected and I cannot get the remote server to accept the authentication credentials (they work for the GET so the credentials are fine.) Instead, the Tomcat server calls the Authenticator over and over, and finally fails. Here's the error I'm getting:
java.net.ProtocolException: Server redirected too many times (20)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1848)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at com.mystuff.house.server.MyServlet.doPost(MyServlet.java:191)
I'm sure I'm doing something stupid, but I can't see where it is. Here's the guts of the servlet doPost() routine:
URL url = new URL("HTTP", "10.10.1.101", -1, "/myresource");
URLConnection con = url.openConnection();
HttpURLConnection http = (HttpURLConnection) con;
http.setRequestMethod("POST");
http.setDoOutput(true);
String encoded = String.valueOf(Base64.getEncoder().encode((a.getUsername().concat(":").concat(a.getPassword())).getBytes()));
http.setRequestProperty("Authorization", "Basic "+encoded);
http.setRequestProperty("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");
// Read the POST payload from the front end post, write to back end post
InputStream r = request.getInputStream();
OutputStream os = http.getOutputStream();
int j = 0;
while ((j = r.read()) != -1) {
os.write((byte) j);
}
http.connect();
// Try reading the result from the back end, push it back to the front end
try {
InputStream i = http.getInputStream();
OutputStream o = response.getOutputStream();
// read/write bytes until EOF
j = 0;
while ((j = i.read()) != -1) {
o.write((byte) j);
}
} catch (Exception ex) {
System.out.println("AIEEEE! Error receiving page from HTTP call");
ex.printStackTrace();
}

The problem with this, after some investigation, turned out to be that the authentication was not valid for the specific URL that I was trying to hit on the remote server.
I had expected to get a 403, 401 or 407 back from the remote server but that never happened, instead this "redirect" happened. So that's something to be aware of if you are trying to hit password-protected URLs from Java code.

Playing radiostreams using javax.media.Player vs. javazoom.jl.player.Player

I am new to playing audio with Java. I have written code to play some radio streams.
I find that there are some streaming urls like http://fm939.wnyc.org/wnycfm. This has no port number and contains slashes. I am able to play this type of url only with javax.media.Player.
There are other streaming urls that come with a port number and no slashes. For example, I have a url for National Public Radio 140.254.23.68:8000. I can play these types of url with javazoom.jl.player.Player since this player takes a url string and a port number.
Can someone tell me a little more about the types of streams and how to use the above players correctly. For example, is it possible to play the stream http://fm939.wnyc.org/wnycfm with javazoom.jl.player.Player ? If so, how?
Any help will be much appreciated.
Addendum:
Sorry I didn't mean to say javazoom Player accepts a url. I am using the following code snippet to create a javazoom player. As you can see from the code, I am using SocketFactory createSocket method to create a connection. The createSocket method takes a url and port. I'd like to know how to play a url like http://fm939.wnyc.org/wnycfm, in other words, a url without an explicit port number.
see code snippet below-
response = null;
try {
SocketFactory sf = SocketFactory.getDefault();
connection = sf.createSocket(url,port);
request = "GET / HTTP/1.1\n\n";
outputStream = connection.getOutputStream();
if(outputStream!=null) {
outputStream.flush();
byte[] b = null;
try {b = request.getBytes(StandardCharsets.US_ASCII);}
catch(NullPointerException npe) {..}
if(b != null) {
outputStream.write(b);
outputStream.flush();
response = connection.getInputStream();
}
} catch (IOException e) {e.printStackTrace();}
javazoom.jl.player.Player zoomPlayer = null;
if(response!=null) {
try {zoomPlayer = new javazoom.jl.player.Player(response);
} catch (JavaLayerException e) {e.printStackTrace();}
}
return zoomPlayer;

"A URL can optionally specify a "port"".
so you can set the port.
I dont see where javazoom.jl.player.Player uses a url - maybe a newer version.
But in any case see the document for URL. One of the constructors says
URL(String protocol, String host, int port, String file, URLStreamHandler handler)
Creates a URL object from the specified protocol, host, port number, file, and handler.
The jmf player can be considered more stable. and documented.
javazoom.jl.player.Player is one line.
--
In fact I can play both streams with regular javax.sound procedures.
--
The way to get the url stream is this:
String u="http://140.254.23.68:8000";
URL url=new URL(u);
URLConnection uc = new URL(u).openConnection();
InputStream is=(InputStream)uc.getInputStream();

How to Pass a File through an HttpURLConnection

I'm trying to get an image hosting on our server available to be displayed on a client. As per the specs of the project:
"When a Client receives such a URL, it must download the
contents (i.e., bytes) of the file referenced by the URL.
Before the Client can display the image to the user, it must first retrieve (i.e., download) the bytes of the
image file from the Server. Similarly, if the Client receives the URL of a known data file or a field help file
from the Server, it must download the content of those files before it can use them."
I'm pretty sure we have the server side stuff down, because if I put the url into a browser it retrieves and displays just fine. So it must be something with the ClientCommunicator class; can you take a look at my code and tell me what the problem is? I've spent hours on this.
Here is the code:
Where I actually call the function to get and display the file: (This part is working properly insofar as it is passing the right information to the server)
JFrame f = new JFrame();
JButton b = (JButton)e.getSource();
ImageIcon image = new ImageIcon(ClientCommunicator.DownloadFile(HOST, PORT, b.getLabel()));
JLabel l = new JLabel(image);
f.add(l);
f.pack();
f.setVisible(true);
From the ClientCommunicator class:
public static byte[] DownloadFile(String hostname, String port, String url){
String image = HttpClientHelper.doGetRequest("http://"+hostname+":"+port+"/"+url, null);
return image.getBytes();
}
The pertinent httpHelper:
public static String doGetRequest(String urlString,Map<String,String> headers){
URL url;
HttpURLConnection connection = null;
try {
//Create connection
url = new URL(urlString);
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Content-Language", "en-US");
connection.setUseCaches (false);
connection.setDoInput(true);
connection.setDoOutput(true);
if(connection.getResponseCode() == 500){
return "failed";
}
//Get Response
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
StringBuffer response = new StringBuffer();
while((line = rd.readLine()) != null) {
response.append(line);
}
rd.close();
return response.toString();
} catch (Exception e) {
e.printStackTrace();
return null;
} finally {
if(connection != null) {
connection.disconnect();
}
}
}
After that, it jumps into the server stuff, which as I stated I believe is working correctly because clients such as Chrome can retrieve the file and display it properly. The problem has to be somewhere in here.
I believe that it has to do with the way the bytes are converted into a string and then back, but I do not know how to solve this problem. I've looked at similar problems on StackOverflow and have been unable to apply them to my situation. Any pointers in the right direction would be greatly appreciated.

If your server is sending binary data, you do not want to use an InputStreamReader, or in fact a Reader of any sort. As the Java API indicates, Readers are for reading streams of characters (not bytes) http://docs.oracle.com/javase/7/docs/api/java/io/Reader.html, which means you will run into all sorts of encoding issues.
See this other stack overflow answer for how to read bytes from a stream:
Convert InputStream to byte array in Java

Do your homework.
Isolate the issue. Modify the server side to send only 256 all possible bytes. Do a binary search and reduce it to small set of bytes.
Use http proxy tools to monitor the bytes as they are transmitted. Fiddler in windows world. Find other ones for the *nix environments.
Then see where the problem is happening and google/bing the suspicions or share the result.

HttpsURLConnection Connection Problems

I'm a problem with a HttpsURLConnection that I can't seem to solve. Basically, I'm sending up some info to a server and if some of that data is wrong, the server sends me a 500 response code. However, it also sends a message in the response telling me which bit of data was wrong. The problem is that the message is always empty when I read it in. I think this is because a filenotfound exception always gets thrown before the stream can be read. Am I right? I tried reading the errorstream as well but this is always empty. Here's a snippet:
conn = (HttpsURLConnection) connectURL.openConnection();
conn.setDoOutput(true);
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Length",
Integer.toString(outString.getBytes().length));
DataOutputStream wr = new DataOutputStream(conn
.getOutputStream());
wr.write(outString.getBytes());
wr.flush();
wr.close();
if(conn.getResponseCode>400{
String response = getErrorResponse(conn);
public String getErrorResponse(HttpsURLConnection conn) {
Log.i(TAG, "in getResponse");
InputStream is = null;
try {
//is = conn.getInputStream();
is = conn.getErrorStream();
// scoop up the reply from the server
int ch;
StringBuffer sb = new StringBuffer();
while ((ch = is.read()) != -1) {
sb.append((char) ch);
}
//System.out.println(sb.toString());
return sb.toString();
// return conferenceId;
}
catch (Exception e){
e.printStackTrace();
}
}

So just to follow up on this, here is how I solved it:
public static String getResponse(HttpsURLConnection conn) {
Log.i(TAG, "in getResponse");
InputStream is = null;
try {
if(conn.getResponseCode()>=400){
is = conn.getErrorStream();
}
else{
is=conn.getInputStream();
}
...read stream...
}
It seems that calling them like this produced an error stream with a message. Thanks for the suggestions!

Try setting content-type request property to "application/x-www-form-urlencoded"
The same is mentioned on this link:
http://developers.sun.com/mobility/midp/ttips/HTTPPost/
The Content-Length and Content-Type headers are critical because they tell the web server how many bytes of data to expect, and what kind, identified by a MIME type.
In MIDP clients the two most popular MIME types are application/octet-stream, to send raw binary data, and application/x-www-form-urlencoded, to send name-value pairs

Are you in control of the server? In other words, did you write the process that runs on the server and listens to the port you're trying to access?
If you did, then you should also be able to debug it and see why your process returns 404.
If you didn't, then describe your architecture (HTTP server, the component it invokes to respond to your HTTP(S) request, etc) and we'll take it from there.
In the very simplest case, of an HTTP server being an Apache server yielding control to some PHP script, it means that Apache couldn't assign your request to anything. Most likely a Web server misconfiguration. Provide some more details and we'll help you out.

Java URLConnection : how can I find out the size of a web file?

I'm working on a project for school, and I'm implementing a tool which can be used to download files from the web ( with a throttling option ). The thing is, I'm gonna have a GUI for it, and I will be using a JProgressBar widget, which I would like to show the current progress of the download. For that I would need to know the size of the file. How do you get the size of the file prior to downloading the file.

Any HTTP response is supposed to contain a Content-Length header, so you could query the URLConnection object for this value.
//once the connection has been opened
List values = urlConnection.getHeaderFields().get("content-Length")
if (values != null && !values.isEmpty()) {
// getHeaderFields() returns a Map with key=(String) header
// name, value = List of String values for that header field.
// just use the first value here.
String sLength = (String) values.get(0);
if (sLength != null) {
//parse the length into an integer...
...
}
It might not always be possible for a server to return an accurate Content-Length, so the value could be inaccurate, but at least you would get some usable value most of the time.
update: Or, now that I look at the URLConnection javadoc more completely, you could just use the getContentLength() method.

As mentioned, URLConnection's getContentLengthLong() is your best bet, but it won't always give a definite length. That's because the HTTP protocol (and others that could be represented by a URLConnection) doesn't always convey the length.
In the case of HTTP, the length of dynamic content typically isn't known in advance—when the content-length header would normally be sent. Instead, another header, transfer-encoding, specifies that a "chunked" encoding is used. With chunked encoding, the length of the entire response is unspecified, and the response is sent back in pieces, where the size of each piece is specified. In practice, the server buffers output from the servlet. Whenever the buffer fills up, another chunk is sent. Using this mechanism, HTTP could actually start streaming a response of infinite length.
If a file is larger than 2 Gb, its size can't be represented as an int, so the older method, getContentLength() will return -1 in that case.

Using a HEAD request, i got my webserver to reply with the correct content-length field which otherwise was empty. I don't know if this works in general but in my case it does:
private int tryGetFileSize(URL url) {
HttpURLConnection conn = null;
try {
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("HEAD");
conn.getInputStream();
return conn.getContentLength();
} catch (IOException e) {
return -1;
} finally {
conn.disconnect();
}
}

You'll want to use the content length (URLConnection.getContentLength()). Unfortunately, this won't always be accurate, or may not always be provided, so it's not always safe to rely on it.

//URLConnection connection
private int FileSize(String url) {
// this is the method and it get the url as a parameter.
// this java class will allow us to get the size of the file.
URLConnection con;
// its in a try and catch incase the url given is wrong or invalid
try{
// we open the stream
con = new URL(url).openConnection()
return con.getContentLength();
}catch (Exception e){
e.printStackTrace();
// this is returned if the connection went invalid or failed.
return 0;
}
}

As #erickson said, sometimes there is header "Transfer-Encoding: chunked", instead of "Content-Length: " and of course you have null value for length.
About the available() method - nobody can guarantee to you that it will return proper value, so I recommend you to not use it.