I am decoding http packets.
And I faced a problem that chunk problem.
When I get a http packet it has a header and body.
When transefer-encoding is chunked I don't know what to do ?
Is there a useful API or class for dechunk the data in JAVA ?
And if someone , experienced about http decoding , please show me a way how to do this ?
Use a fullworthy HTTP client like Apache HttpComponents Client or just the Java SE provided java.net.URLConnection (mini tutorial here). Both handles it fully transparently and gives you a "normal" InputStream back. HttpClient in turn also comes with a ChunkedInputStream which you just have to decorate your InputStream with.
If you really insist in homegrowing a library for this, then I'd suggest to create a class like ChunkedInputStream extends InputStream and write logic accordingly. You can find more detail how to parse it in this Wikipedia article.
Apache HttpComponents
Oh, and if we are talking about the client side, HttpUrlConnection does this as well.
If you are looking for a simple API try Jodd Http library (http://jodd.org/doc/http.html).
It handles Chunked transfer encoding for you and you get the whole body as a string back.
From the docs:
HttpRequest httpRequest = HttpRequest.get("http://jodd.org");
HttpResponse response = httpRequest.send();
System.out.println(response);
Here is quick-and-dirty alternative that requires no dependency except Oracle JRE:
private static byte[] unchunk(byte[] content) throws IOException {
ByteArrayInputStream bais = new ByteArrayInputStream(content);
ChunkedInputStream cis = new ChunkedInputStream(bais, new HttpClient() {}, null);
return readFully(cis);
}
It uses the same sun.net.www.http.ChunkedInputStream as java.net.HttpURLConnection does behind the scene.
This implementation doesn't provide detailed exceptions (line numbers) on wrong content format.
It works with Java 8 but could fail in with next release. You've been warned.
Could be useful for prototyping though.
You can choose any readFully implementation from Convert InputStream to byte array in Java.
Related
I am loading model in apache jena using function FileManager.get().loadModel(url).And I also know that there may be some URLs in HTTP Response Link Header .I want to load model also from the links(URLs) in link header.How to do that ? Is there any inbuilt fuctionality to get access to header and process link header in Response header?
FileManager.get().loadModel(url) packages up reading a URL and parsing the results into a model. It is packing up a common thing to do; it is not claiming to be comprehensive. It is quite an old interface.
If you wanted detailed control over the HTTP handling, see if HttpOp (a lower level) mechanism helps, otherwise do the handling in the application and hand the input stream for the response directly to the parser.
You may also find it useful to look at the code in RDFDataMgr.process for help with content negotiation.
I don't think that this is supported by Jena. I don't see any reason in doing so. The HTTP request is done to get the data and maybe also to get the response type. If you want to get the URLs in some header fields, why not simply use plain old Java:
URL url = new URL("http://your_ontology.owl");
URLConnection conn = url.openConnection();
Map<String, List<String>> map = conn.getHeaderFields();
I have a custom java console app I am writing to upload non-binary files to a java app server I own. It is performing an HTTPS multipart/form-data POST with the file to a REST api. While it works great for small files, I would like to apply GZIP content-encoding to the post request, so it more efficiently handles large files.
Is there a JAVA library I can use to gzip the post, including the file content and then un-zip it on the other side? I would like to avoid having to zip the file first and would rather rely on HTTP encoding to handle it.
To be pedantic, you wouldn't gzip the entire POST. You would just Gzip the content data, and then in your POST set the Content-Encoding as gzip.
You haven't posted your code (get it???), so some assumptions need to be made to give an example:
import java.util.zip.GZIPOutputStream;
import java.io.ByteArrayOutputStream;
...
final String yourData = "butts";
final ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
final GZipOutputStream gzipOutputStream;
try {
gzipOutputStream = new GZipOutputStream(byteArrayOutputStream);
gzipOutputStream.write(yourData.getBytes("utf-8"));
} finally {
gzipOutputStream.close();
}
final byte[] gzippedButts = byteArrayOutputStream.toByteArray();
/*
* Now use the gzipped data as the data in your POST, and also
* make sure to set the Content-Encoding of your HTTP POST to "gzip".
*/
Edit: Reading the question again, it sounds like OP wants a library that will abstract away all of the handling and just Gzip a request body under the hood. Unfortunately, I am not aware of any such library.
I am calling a restful service that returns JSON using the Apache HttpClient.
The problem is I am getting different results in the encoding of the response when I run the code on different platforms.
Here is my code:
GetMethod get = new GetMethod("http://urltomyrestservice");
get.addRequestHeader("Content-Type", "text/html; charset=UTF-8");
...
HttpResponse response = httpexecutor.execute(request, conn, context);
response.setParams(params);
httpexecutor.postProcess(response, httpproc, context);
StringWriter writer = new StringWriter();
IOUtils.copy(response.getEntity().getContent(), writer);
When I run this on OSX, asian characters etc return fine e.g. 張惠妹 in the response. But when I run this on a linux server the same code displays the characters as ???
The linux server is an Amazon EC2 instance running Java 1.6.0_26-b03
My local OSX is running 1.6.0_29-b11
Any ideas really appreciated!!!!!
If you look at the javadoc of org.apache.commons.io.IOUtils.copy(InputStream, Writer):
Copy bytes from an InputStream to chars on a Writer using the default
character encoding of the platform.
So that will give different answers depending on the client (which is what you're seeing)
Also, Content-Type is usually a response header (unless you're using POST or PUT). The server is likely to ignore it (though you might have more luck with the Accept-Charset request header).
You need to parse the content type's charset-encoding parameter of the response header, and use that to convert the response into a String (if it's a String you're actually after). I expect Commons HTTP has code that will do that automatically for you. If it doesn't, Spring's RESTTemplate definitely does.
I believe that the problem is not in the HTTP encoding but elsewhere (e.g. while reading or forming the answer). Where do you get the content from and how? Is this stored in a DB or file?
I am attempting to have my android phone connect to my servlet and send it a certain image. The way I figured I would do this, is to use the copyPixelsToBuffer() function and then attempt to send this to the servlet through some output stream(similar to how I would do it in a normal stand alone java application). Will this way work? If so, what kind of stream do I use exactly? Should I just use DataOutputStream and just do something like the following:
ByteBuffer imgbuff;
Bitmap bm = BitmapFactory.decodeResource(getResources(), R.drawable.icon);
bm.copyPixelsToBuffer(bm);
...code...
URLConnection sc = server.openConnection();
sc.setDoOutput(true);
DataOutputStream out = new DataOutputStream( sc.getOutputStream() );
out.write(imgbuff.array());
out.flush();
out.close();
Note: I understand that this may not be the proper way of connecting to a server using the Android OS but at the moment I'm working on just how to send the image, not the connection (unless this is relevant on how the image is sent).
If this is not a way you'd recommend sending the image to the servlet (I figured a byte buffer would be best but I could be wrong), how would you recommend this to be done?
Since a HttpServlet normally listens on HTTP requests, you'd like to use multipart/form-data encoding to send binary data over HTTP, instead of raw (unformatted) like that.
From the client side on, you can use URLConnection for this as outlined in this mini tutorial, but it's going to be pretty verbose. You can also use Apache HttpComponents Client for this. This adds however extra dependencies, I am not sure if you'd like to have that on Android.
Then, on the server side, you can use Apache Commons FileUpload to parse the items out of a multipart/form-data encoded request body. You can find a code example in this answer how the doPost() of the servlet should look like.
As to your code example: wrapping in the DataOutputStream is unnecessary. You aren't taking benefit of the DataOutputStream's facilities. You are just using write(byte[]) method which is already provided by the basic OutputStream as returned by URLConnection#getOutputStream(). Further, the Bitmap has a compress() method which you can use to compress it using a more standard and understandable format (PNG, JPG, etc) into an arbitrary OutputStream. E.g.
output = connection.getOutputStream();
// ...
bitmap.compress(CompressFormat.JPEG, 100, output);
Do this instead of output.write(bytes) as in your code.
How can I post a file to a php using java (as a html form would do) ?
If you simply need to generate an HTTP Post, check out HttpClient, and in particular the PostMethod. Since you're talking HTTP the implementing technology on the server (in your case, PHP) is immaterial.
There's an example here.
It is in fact possible to do a POST using only the classes that are in the JDK, but as others have pointed out it will probably be easier to use a library like HttpClient.
In addition to HttpClient you may also want to look at the client-side java libraries supplied by RESTful frameworks such as Restlet and Jersey. While primarily designed for interacting with web services they offer a very high-level abstraction for GETing and POSTing to just about anything.
Code NOT tested or even compiled (probably doesn't compile), but this is kinda sorta would you'd do if you wanted to roll your own:
URL url = new URL("http://hostname/foo.php");
URLConnection connection = url.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
// ignoring possible encoding issues
byte[] body = "param=value¶m2=value2".getBytes();
connection.setRequestProperty("Content-type", "application/x-www-form-urlencoded" );
connection.setRequestProperty("Content-length", String.valueOf(body.length) );
// ignoring possible IOExceptions
OutputStream out = connection.getOutputStream();
out.write(body);
out.flush();
// use this to read back from server
InputStream in = connection.getInputStream();
As you can see, it's pretty low-level stuff. Which is why you want to use a library.
The httpclient library contains all the tools you need to talk to a web server.