I am using HttpURLConnection and I have a problem understanding on how client and server sync. Assume simple download file example. This example is copied somewhere from web. I am only using the code to state the standard process.
Servlet code is like:
response.setContentType(mimeType);
response.setContentLength((int) downloadFile.length());
String headerKey = "Content-Disposition";
String headerValue = String.format("attachment; filename=\"%s\"", downloadFile.getName());
response.setHeader(headerKey, headerValue);
// obtains response's output stream
OutputStream outStream = response.getOutputStream();
//write to stream
//close the stream
And Client code is like :
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
String disposition = httpConn.getHeaderField("Content-Disposition");
String contentType = httpConn.getContentType();
int contentLength = httpConn.getContentLength();
//parse content-disposition
....
InputStream inputStream = httpConn.getInputStream();
String saveFilePath = saveDir + File.separator + fileName;
// opens an output stream to save into file
FileOutputStream outputStream = new FileOutputStream(saveFilePath);
//write to stream
//close stream
} else {
//if non Ok status
}
My first question is : Is httConn.getResponseCode() a blocking call that waits for the servlet to finish processing? Otherwise, if error comes or servlet call response.sendError(), when you are inside if (responseCode == HttpURLConnection.HTTP_OK) {, what will happen.
Second Question : is an extension of first question. If responseCode is not blocking, then when i am accessing disposition, contentType, cotentLength, how am i sure that they are already set.
Third Question. If httConn.getResponseCode() is blocking. So if I want to send send some message to the client, how correct is to send it to the client in response headers like : resposnse.setHeader("my-message", "some message I want to send"); rather than using response.getWriter() to write to the stream. So that I am sure client will definitely read that.
Fourth Question : If I am writing two objects to streams on servlet, how will client distinguish or can it distinguish? Suppose I am writing a class object using response.getObjectOutputStream() and then I may be writing some string using writer or may be write a file after that. Can client distinguish these different items coming in stream or do I have to use multiple request. One request per object or file or String to be read from stream.
Yes, as the javadoc indicates
Gets the status code from an HTTP response message. For example, in the case of the following status lines:
HTTP/1.0 200 OK
HTTP/1.0 401 Unauthorized
It will return 200 and 401 respectively. Returns -1 if no code can be discerned from the response (i.e., the response is not valid HTTP).
NA
You can use headers if you want, but headers are limited to text only and limited in length (AFAIR). The response body is usually used to contain... the response body. Whereas headers are typically used for metadata.
server and client have to agree on a protocol. If the protocol is that a response contains two objects, then the client should read two objects. I would not do that though. You'd better send a unique container object rather than 2 in sequence. HTTP can be used to transport anything, but a JSON or XML document is usually used to transport structured data.
Related
Most importantly I want to get the exceptionMessage that I am able to view as part of the data returned when I use SoapUI to make the request. I don't see anything in HttpURLConnection that includes this detailed information. Only responseCode and responseMessage, which are nice, but are lacking the description I'm looking for.
Also, does SoapUI parse this raw data into JSON and XML itself, or is there a simple way I can get it as JSON through java as well?
Thanks
The server returns HTTP headers and in most cases a body. To get the body in case of a error, you have to do something like that:
InputStream is;
if (conn.getResponseCode() / 100 == 2) { // HTTP status code 2xx, e.g. 200
is = conn.getInputStream();
// read input stream -> this is the content you wanted
} else {
is = conn.getErrorStream();
// read input stream -> contains a description of the error
// depending on header "Content-Type" you can also parse the stream
// as JSON or XML or HTML ...
}
I have a home grown protocol which uses HttpURLConnection (from Java 1.6) & Jetty (6.1.26) to POST a block of xml as a request and receive a block of xml as a response. The amounts of xml are approx. 5KB.
When running both sender and receiver on Linux EC2 instances in different parts of the world I'm finding that in about 0.04% of my requests the Jetty handler sees the xml request (the post body) as an empty string. I've checked and the client outputs that it's consistently trying to send the correct (> 0 length) xml request string.
I have also reproduced this by looping my JUnit tests on my local (Win 8) box.
I assume the error must be something like:
Misuse of buffers
An HttpURLConnection bug
A network error
A Jetty bug
A random head slapping stupid thing I've done in the code
The relevant code is below:
CLIENT
connection = (HttpURLConnection) (new URL (url)).openConnection();
connection.setReadTimeout(readTimeoutMS);
connection.setConnectTimeout(connectTimeoutMS);
connection.setRequestMethod("POST");
connection.setAllowUserInteraction(false);
connection.setDoOutput(true);
// Send request
byte[] postBytes = requestXML.getBytes("UTF-8");
connection.setRequestProperty("Content-length", "" + postBytes.length);
OutputStream os = connection.getOutputStream();
os.write(postBytes);
os.flush();
os.close();
// Read response
InputStream is = connection.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
is.close();
connection.disconnect();
return writer.toString();
SERVER (Jetty handler)
public void handle(java.lang.String target, javax.servlet.http.HttpServletRequest request, javax.servlet.http.HttpServletResponse response, int dispatch) {
InputStream is = request.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
is.close();
String requestXML = writer.toString();
// requestXML is 0 length string about 0.04% of time
Can anyone think of why I'd randomly get the request as an empty string?
Thanks!
EDIT
I introduced some more trace and getContentLength() returns -1 when the error occurs, but the client output still shows it's sending the right amount of bytes.
I can't think of why you are getting a empty string. Code looks correct. If you update you code to check for empty string and if found report the content-length and transfer-encoding of the request, that would be helpful to identify the culprit. A wireshark trace of the network data would also be good.
But the bad new is that jetty-6 is really end of life, and we are unlikely to be updating it. If you are writing the code today, then you really should be using jetty-7 or 8. Perhaps even jetty-9 milestone release if you are brave. If you find such and error in jetty-9, I'd be all over it like a rash trying to fix it for you!
Make sure you set connection.setRequestProperty("Content-Type", "application/xml"); It's possible POST data may be discarded without some Content-type. This was the case when I replicated your problem locally (against a Grails embedded Tomcat instance), and supplying this fixed it.
I try to use HTTP POST to send some Data to a Server.
The Server expects the binary Data in $_POST['file'].
URL url = new URL("http://example.com");
URLConnection connection = url.openConnection();
connection.setDoOutput(true);
OutputStream outputStream = connection.getOutputStream();
outputStream.write("file=".getBytes());
//byte[] buffer contains the data
outputStream.write(buffer);
outputStream.close();
Is OutputStream.write the right method to write into the stream? Do I have to handle the string ("file=") other then the buffer?
I recommend converting your data to Base64 String (Compatibility with all systems).
string result = Convert.ToBase64String(Encoding.UTF8.GetBytes(utf8Text));
Yes, to write text with POST, you will need to write to `OutputStream.
For parameters, you will need to create a String of key-value pair (separated with &) and write the byte array of that data in OutputStream as follows:
String parameterString = "file=" + parameters.get("file") + "&" + "other=" + parameter.get("other");
outputStream.write(parameterString.getBytes("UTF-8"); //Don't forget, HTTP protocol supports UTF-8 encoding.
outputStream.flush();
To do file upload with URLConnection, see BalusC's article How to use java.net.URLConnection to fire and handle HTTP requests?
Everything works fine, but only if file is small, about 1MB, when I tried it with bigger files, like 20MB my browser display it, instead of force to download, I tried many headers so far, now my code looks:
PrintWriter out = response.getWriter();
String fileName = request.getParameter("filename");
File f= new File(fileName);
InputStream in = new FileInputStream(f);
BufferedInputStream bin = new BufferedInputStream(in);
DataInputStream din = new DataInputStream(bin);
while(din.available() > 0){
out.print(din.readLine());
out.print("\n");
}
response.setContentType("application/force-download");
response.setContentLength((int)f.length());
response.setHeader("Content-Transfer-Encoding", "binary");
response.setHeader("Content-Disposition","attachment; filename=\"" + "xxx\"");//fileName);
in.close();
bin.close();
din.close();
You are setting the response headers after writing the contents of the file to the output stream. This is quite late in the response lifecycle to be setting headers. The correct sequence of operations should be to set the headers first, and then write the contents of the file to the servlet's outputstream.
Therefore, your method should be written as follows (this won't compile as it is a mere representation):
response.setContentType("application/force-download");
response.setContentLength((int)f.length());
//response.setContentLength(-1);
response.setHeader("Content-Transfer-Encoding", "binary");
response.setHeader("Content-Disposition","attachment; filename=\"" + "xxx\"");//fileName);
...
...
File f= new File(fileName);
InputStream in = new FileInputStream(f);
BufferedInputStream bin = new BufferedInputStream(in);
DataInputStream din = new DataInputStream(bin);
while(din.available() > 0){
out.print(din.readLine());
out.print("\n");
}
The reason for the failure is that it is possible for the actual headers sent by the servlet would be different from what you are intending to send. After all, if the servlet container does not know what headers (which appear before the body in the HTTP response), then it may set appropriate headers to ensure that the response is valid; setting the headers after the file has been written is therefore futile and redundant as the container might have already set the headers. You could confirm this by looking at the network traffic using Wireshark or a HTTP debugging proxy like Fiddler or WebScarab.
You may also refer to the Java EE API documentation for ServletResponse.setContentType to understand this behavior:
Sets the content type of the response being sent to the client, if the response has not been committed yet. The given content type may include a character encoding specification, for example, text/html;charset=UTF-8. The response's character encoding is only set from the given content type if this method is called before getWriter is called.
This method may be called repeatedly to change content type and character encoding. This method has no effect if called after the response has been committed.
...
Set content-type and other headers before you write the file out. For small files the content is buffered, and the browser gets the headers first. For big ones the data come first.
This is from a php script which solves the problem perfectly with every browser I've tested (FF since 3.5, IE8+, Chrome)
header("Content-Disposition: attachment; filename=\"".$fname_local."\"");
header("Content-Type: application/force-download");
header("Content-Transfer-Encoding: binary");
header("Content-Length: ".filesize($fname));
So as far as I can see, you're doing everything correctly. Have you checked your browser settings?
I'am writing a HTTP proxy that is part of a test/verification
system. The proxy filters all requests coming from the client device
and directs them towards various systems under test.
The proxy is implemented as a servlet where each request is forwarded
to the target system, it handles both GET and POST. Somtimes the
response from the target system is altered to fit various test
conditions, but that is not the part of the problem.
When forwarding a request, all headers are copied except for those
that is part of the actual HTTP transfer such as Content-Length and
Connection headers.
If the request is a HTTP POST, then the entity body of the request is
forwarded as well and here is where it doesnt work sometimes.
The code reading the entity body from the servlet request is the following:
URL url = new URL(targetURL);
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
String method = request.getMethod();
java.util.Enumeration headers = request.getHeaderNames();
while(headers.hasMoreElements()) {
String headerName = (String)headers.nextElement();
String headerValue = request.getHeader(headerName);
if (...) { // do various adaptive stuff based on header
}
conn.setRequestProperty(headerName, headerValue);
}
// here is the part that fails
char postBody[] = new char[1024];
int len;
if(method.equals("POST")) {
logger.debug("guiProxy, handle post, read request body");
conn.setDoOutput(true);
BufferedReader br = request.getReader();
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(conn.getOutputStream()));
do {
logger.debug("Read request into buffer of size: " + postBody.length);
len = br.read(postBody, 0, postBody.length);
logger.debug("guiProxy, send request body, got " + len + " bytes from request");
if(len != -1) {
bw.write(postBody, 0, len);
}
} while(len != -1);
bw.close();
}
So what happends is that the first time a POST is received, -1
characters are read from the request reader, a wireshark trace shows
that the entity body containing URL encoded post parameters are there
and it is in one TCP segment so there is no network related
differences.
The second time, br.read successfully returns the 232 bytes in the
POST request entity body and every forthcoming request works as well.
The only difference between the first and forthcoming POST requests is
that in the first one, no cookies are present, but in the second one,
a cookie is present that maps to the JSESSION.
Can it be a side effect of entity body not being available since the
request processing in the servlet container allready has read the POST
parameters, but why does it work on forthcoming requests.
I believe that the solution is of course to ignore the entity body on
POST requests containing URL encoded data and fetch all parameters
from the servlet request instead using getParameter and reinsert them
int the outgoing request.
Allthough that is tricky since the POST request could contain GET
parameters, not in our application right now, but implementing it
correctly is some work.
So my question is basically: why do the reader from
request.getReader() return -1 when reading and an entity body is
present in the request, if the entity body is not available for
reading, then getReader should throw an illegal state exception. I
have also tried with InputStream using getInputStream() with the same
results.
All of this is tested on apache-tomcat-6.0.18.
So my question is basically: why do the reader from request.getReader() return -1 when reading.
It will return -1 when there is no body or when it has already been read. You cannot read it twice. Make sure that nothing before in the request/response chain has read it.
and an entity body is present in the request, if the entity body is not available for reading, then getReader should throw an illegal state exception.
It will only throw that when you have already called getInputStream() on the request before, not when it is not available.
I have also tried with InputStream using getInputStream() with the same results.
After all, I'd prefer streaming bytes than characters because you then don't need to take character encoding into account (which you aren't doing as far now, this may lead to future problems when you will get this all to work).
Seems, that moving
BufferedReader br = request.getReader()
before all operations, that read request (like request.getHeader() ), works for me well .