Is there a way to determine the size of the HTTPServletResponse content? I read this get-size-of-http-response-in-java question but sadly where I work I do not have access to CommonsIO :(
The response content consists of a single complex object so I have considered writing it out to a temp file and then checking that file. This is not something I want to be doing as a diagnostic while the application is running in production though so want to avoid it if at all possible.
PS I read erickson's answer but it mentioned input streams I want to know the size of the object being written out... Would be really nice if the writeObject() method returned a number representing bytes written instead of void...
If you have access to the response header, you can read the Content-Length.
Here is a example of a response header:
(Status-Line):HTTP/1.1 200 OK
Connection:Keep-Alive
Date:Fri, 25 Mar 2011 16:26:56 GMT
Content-Length:728
Check this out: Header Field Definitions
This seems to be what you're looking for:
DataOutputStream dos = new DataOutputStream(response.getOutputStream());
...
int len = dos.size();
I eventually found a way to get what I wanted:
URLConnection con = servletURL.openConnection();
BufferedInputStream bif = new BufferedInputStream(con.getInputStream());
ObjectInputStream input = new ObjectInputStream(bif);
int avail = bif.available();
System.out.println("Response content size = " + avail);
This allowed me to see the response size on the client. I still would like to know what it is on the server side before it is sent but this was the next best thing.
Assuming the use of ObjectOutputStream, build it around a java.io.ByteArrayOutputStream:
ByteArrayOutputStream contentBytes = new ByteArrayOutputStream();
ObjectOutputStream objectOut = new ObjectOutputStream(contentBytes);
objectOut.writeObject(content);
int contentLength = contentBytes.size();
And then you can send the content with
contentBytes.writeTo(connection.getOutputStream());
where connection is whatever you're getting your OutputStream from.
Better late than never, right?
Related
I wrote a Java program to make a simple HTTP get call and read content from a REST webservice. I include 2 custom HTTP headers and provide values to them.
In SOAPUI, when I make a REST call with updating those 2 headers, I get proper response, however when I make the same call in my Java program, I get truncated output. Below is my code:
try {
URL url = new URL(lnk);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("HeaderParameter1", param1);
connection.setRequestProperty("HeaderParamter2", param2);
connection.setConnectTimeout(60000);
InputStream is = connection.getInputStream();
byte [] b = new byte [is.available()];
is.read(b);
String input = new String(b);
System.out.println("The payload is \n"+input);
connection.disconnect();} catch(Exception e){}
Calling is.available() returns the number of bytes that are available to be read without blocking, but that may be less than the full content of the response. In the documentation for this method, it says: "It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream."
A simple way to read the entire response into a String could be:
InputStream is = connection.getInputStream();
byte [] b = is.readAllBytes();
String input = new String(b);
There are two caveats with this approach:
The new String(byte[]) constructor uses the platform's default charset to convert the bytes to characters. This is generally not a good idea for network protocols, since the client and server need to agree on the character encoding. You can hardcode the charset encoding if it will always be the same, for example: new String(b, "UTF-8").
If the response is very large, this will allocate a large byte array. If your handling of the response can be written in a streaming manner, it would be better to read iteratively with a fixed buffer size, as Anders mentioned in his comment.
I am doing a project to get the picture from a website(anyone will be OK),and I know that I could use the URL to get it . But I want to know better about the TCP ,so I use the socket to get it . That's all be OK, but the problem is that the data stream I received contain the respond of the HTTP ,and I don't know how to filter it.
Here is my code (just a part of it)
Socket socket = new Socket(netAdress, 80);
bw = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
bw.write("GET HTTP://" + sources + " HTTP/1.0\r\n");
bw.write("\r\n");
bw.flush()//connect
BufferedOutputStream writeImg = new BufferedOutputStream(new FileOutputStream(adj));
byte[] data = new byte[512];
int len=0;
boolean OK=false;
while ((len=in.read(data))>0) {
writeImg.write(data,0,len);
writeImg.flush();
}//receive the data stream
and this is what I received,and the picture couldn't open.
the detail of the data stream
If you know how to solved the problem or you have a better idea of get the picture By socket ,please contact me.Thanks.
... this is what I received,and the picture couldn't open
Yup. The response starts with an HTTP response header.
If you know how to solved the problem ...
Well, this is a hack, and NOT recommended (and it won't work in general!) but the HTTP response header ends with the first <CR> <NL> <CR> <NL> sequence (ASCII control codes). So if you strip off everything up to and including that sequence you should have an image. (Unless it is compressed, or encoded, or a multi-part, or .....)
... or you have a better idea of get the picture.
A better idea is to use the URL. Seriously.
We are in the process of migrating functionality from a Weblogic Server 8,1 sp5 with java 1.4 to 10.3.6 with java 1.7.
The case that is described below is working properly in the old server, however we are facing an issue when transferring the handling to the new one.
The problem lies while retrieving and parsing an XML response that was retrieved from an external system through SOAP calls.
The following libraries and procedure are used in the method:
java.net.HttpURLConnection to make the connection
java.io.OutputStream to send the request
java.io.InputStream to get the response
byte[] to store the result before transforming in to String
javax.xml.parsers.DocumentBuilder, java.io.StringReader and org.xml.sax.InputSource to transform the String into org.w3c.dom.Document
The following exception is thrown:
"org.xml.sax.SAXParseException - Content is not allowed in trailing section."
When opening the logs of the application with notepad++ many null characters appear after the end of the file which seem to cause the issue. I repeat that no such cases appear when executing the request from the old server.
The respective code is the following:
//Creating the connection
URL u = new URL(default_server);
URLConnection uc = u.openConnection();
HttpURLConnection connection = (HttpURLConnection) uc;
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod(requestMethod);
connection.setRequestProperty("SOAPAction", soap_action);
OutputStream out = connection.getOutputStream();
Writer wout = new OutputStreamWriter(out);
wout.write(xmlString);
wout.flush();
wout.close();
InputStream in = connection.getInputStream();
int c = in.available();
byte r[] = new byte[c];
in.read(r);
String response = new String(r);
connection.disconnect();
//Transorming the String to XML Doc
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
StringReader theReader = new StringReader(response);
InputSource theInputSource = new InputSource();
theInputSource.setCharacterStream(theReader);
Document doc = builder.parse(theInputSource);
//Here occurs org.xml.sax.SAXParseException-Content is not allowed in trailing section
return doc;
I know that i can solve the issue by getting the stripping the response from junk characters but this not a safe resolution.
Do you have any information to share on the matter? Do you think it is a java version issue or maybe a server configuration issue?
Thank you in advance your time.
Best Regards,
George
I see two issues
in.available() as per the javadoc: Returns an estimate of the number of bytes... Do not rely on this. Loop over buffers of 8K to read the stream until you reach the end or even better, do not re-invent the wheel, use commons-io from Apache and use a single call to ÌOUtils.read
String response = new String(r); doing this you are assuming that the bytes received are encoded using the same charset as your platform encoding/charset. It is unlikely to be the case if you are on Windows or OSX. You must pass the charset and use the constructor String(byte[] bytes, Charset charset).
I have a home grown protocol which uses HttpURLConnection (from Java 1.6) & Jetty (6.1.26) to POST a block of xml as a request and receive a block of xml as a response. The amounts of xml are approx. 5KB.
When running both sender and receiver on Linux EC2 instances in different parts of the world I'm finding that in about 0.04% of my requests the Jetty handler sees the xml request (the post body) as an empty string. I've checked and the client outputs that it's consistently trying to send the correct (> 0 length) xml request string.
I have also reproduced this by looping my JUnit tests on my local (Win 8) box.
I assume the error must be something like:
Misuse of buffers
An HttpURLConnection bug
A network error
A Jetty bug
A random head slapping stupid thing I've done in the code
The relevant code is below:
CLIENT
connection = (HttpURLConnection) (new URL (url)).openConnection();
connection.setReadTimeout(readTimeoutMS);
connection.setConnectTimeout(connectTimeoutMS);
connection.setRequestMethod("POST");
connection.setAllowUserInteraction(false);
connection.setDoOutput(true);
// Send request
byte[] postBytes = requestXML.getBytes("UTF-8");
connection.setRequestProperty("Content-length", "" + postBytes.length);
OutputStream os = connection.getOutputStream();
os.write(postBytes);
os.flush();
os.close();
// Read response
InputStream is = connection.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
is.close();
connection.disconnect();
return writer.toString();
SERVER (Jetty handler)
public void handle(java.lang.String target, javax.servlet.http.HttpServletRequest request, javax.servlet.http.HttpServletResponse response, int dispatch) {
InputStream is = request.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
is.close();
String requestXML = writer.toString();
// requestXML is 0 length string about 0.04% of time
Can anyone think of why I'd randomly get the request as an empty string?
Thanks!
EDIT
I introduced some more trace and getContentLength() returns -1 when the error occurs, but the client output still shows it's sending the right amount of bytes.
I can't think of why you are getting a empty string. Code looks correct. If you update you code to check for empty string and if found report the content-length and transfer-encoding of the request, that would be helpful to identify the culprit. A wireshark trace of the network data would also be good.
But the bad new is that jetty-6 is really end of life, and we are unlikely to be updating it. If you are writing the code today, then you really should be using jetty-7 or 8. Perhaps even jetty-9 milestone release if you are brave. If you find such and error in jetty-9, I'd be all over it like a rash trying to fix it for you!
Make sure you set connection.setRequestProperty("Content-Type", "application/xml"); It's possible POST data may be discarded without some Content-type. This was the case when I replicated your problem locally (against a Grails embedded Tomcat instance), and supplying this fixed it.
Everything works fine, but only if file is small, about 1MB, when I tried it with bigger files, like 20MB my browser display it, instead of force to download, I tried many headers so far, now my code looks:
PrintWriter out = response.getWriter();
String fileName = request.getParameter("filename");
File f= new File(fileName);
InputStream in = new FileInputStream(f);
BufferedInputStream bin = new BufferedInputStream(in);
DataInputStream din = new DataInputStream(bin);
while(din.available() > 0){
out.print(din.readLine());
out.print("\n");
}
response.setContentType("application/force-download");
response.setContentLength((int)f.length());
response.setHeader("Content-Transfer-Encoding", "binary");
response.setHeader("Content-Disposition","attachment; filename=\"" + "xxx\"");//fileName);
in.close();
bin.close();
din.close();
You are setting the response headers after writing the contents of the file to the output stream. This is quite late in the response lifecycle to be setting headers. The correct sequence of operations should be to set the headers first, and then write the contents of the file to the servlet's outputstream.
Therefore, your method should be written as follows (this won't compile as it is a mere representation):
response.setContentType("application/force-download");
response.setContentLength((int)f.length());
//response.setContentLength(-1);
response.setHeader("Content-Transfer-Encoding", "binary");
response.setHeader("Content-Disposition","attachment; filename=\"" + "xxx\"");//fileName);
...
...
File f= new File(fileName);
InputStream in = new FileInputStream(f);
BufferedInputStream bin = new BufferedInputStream(in);
DataInputStream din = new DataInputStream(bin);
while(din.available() > 0){
out.print(din.readLine());
out.print("\n");
}
The reason for the failure is that it is possible for the actual headers sent by the servlet would be different from what you are intending to send. After all, if the servlet container does not know what headers (which appear before the body in the HTTP response), then it may set appropriate headers to ensure that the response is valid; setting the headers after the file has been written is therefore futile and redundant as the container might have already set the headers. You could confirm this by looking at the network traffic using Wireshark or a HTTP debugging proxy like Fiddler or WebScarab.
You may also refer to the Java EE API documentation for ServletResponse.setContentType to understand this behavior:
Sets the content type of the response being sent to the client, if the response has not been committed yet. The given content type may include a character encoding specification, for example, text/html;charset=UTF-8. The response's character encoding is only set from the given content type if this method is called before getWriter is called.
This method may be called repeatedly to change content type and character encoding. This method has no effect if called after the response has been committed.
...
Set content-type and other headers before you write the file out. For small files the content is buffered, and the browser gets the headers first. For big ones the data come first.
This is from a php script which solves the problem perfectly with every browser I've tested (FF since 3.5, IE8+, Chrome)
header("Content-Disposition: attachment; filename=\"".$fname_local."\"");
header("Content-Type: application/force-download");
header("Content-Transfer-Encoding: binary");
header("Content-Length: ".filesize($fname));
So as far as I can see, you're doing everything correctly. Have you checked your browser settings?