New Servlet 3.0 API provide us with convenient way to parse multi-part form data. But it stores content of uploaded files in file system or in memory
Is there streaming API for Servlet 3.0 ?
Something like Commons FileUpload. I have to write content directly from InputStream and write to another OutputStream adn I don't want to store temporary file content in disc or memory
I used this once for something similar, though not with servlets. It doesn't fill up your memory with data. Hope it helps:
http://code.google.com/p/io-tools/wiki/Tutorial_EasyStream
Looking at the Servlet 3.0 spec it may not be possible to have a streaming implementation
For parts with form-data as the Content-Disposition, but without a
filename, the string value of the part will also be available via the
getParameter /getParameterValues methods on HttpServletRequest, using
the name of the part.
So the request must be parsed up front so that all the non-file parts can be exposed as HttpServletRequest parameters.
You have to use third party libraries if you need streaming.
Related
I want to download a very large file using camel, but I don't want to hold the entire file in memory and THEN save it to file.
I want to stream the file in and save or write to a file in chunks.
Is this possible with Camel, and if so, how do I do this?
Note: Is it possible that the endpoint I am downloading the file does not support streaming/chunking? If yes, how can I verify this?
Camel's HTTP component uses Netty to make the request. Netty reads the entire response into memory, so there is no way to do what you are asking for.
You would need to implement your own endpoint for Camel that utilizes another HTTP library which has support for HTTP response streaming.
More documentation is available here :
https://cwiki.apache.org/confluence/display/CAMEL/Netty4+HTTP
You can 3 option to download the file i.e. using:
ftp://[username#]hostname[:port]/directoryname[?options]
sftp://[username#]hostname[:port]/directoryname[?options]
ftps://[username#]hostname[:port]/directoryname[?options]
There is a option of streamDownload in it.
For more check out http://camel.apache.org/ftp.html
Is is possible to exchange PDF file as a input for RESTful webservices and also I would like to send PNG image as a response to it.
If we can do it through REST services please provide me the references link to implement it using REST services.
I was able to achieve a similar output by converting the file to string/byte[] data and sending it via REST.
My implementation was in Java and the steps used is outline below
Convert the file on disk to byte[] array (apache common-io can convert the file to byte[] in easy step. Try the IOUtils class)
Encoded the byte[] as String (apache common-codec was used for the encoding)
Wrapped the string data in a model class
Converted the model class to json format (GSON was used for the conversion)
Sent the json data over to the server
The server application reversed the process, and the file was available on the server
A rest service isn't the right way for what you want. The input for this kind of services are HTTP request attributes or some kind of pushed data. Maybe it's possible to implement a file upload but it's not typical.
For restful services is also common to tell your service how to handle the requested resource via the used request method (GET, POST, PUT, DELETE)
The response of rest services is normally some kind of structured text output - for instance json.
All in all rest services seems to me not the way to implement your desired scenario.
What about a normal cgi or servlet solution?
I'm writing a Java desktop client which will send multiple files over the wire to a servlet using a post request. In the servlet I'm getting the input stream from the request to receive the files. The servlet will write the files to disk, one by one as they're read from the stream.
The implementation has a couple of requirements:
Only one HTTP request must be used to the server (so only a single stream)
The servlet must use a reasonable fixed amount of memory, no matter what the size of the files.
I had considered inserting markers into the stream so I know when one file ends and the next one begins. I'd then write some code to parse the stream in the servlet, and start writing the next file as appropriate.
Here's the thing... surely there's a library to do that. I've looked through apache commons and found nothing. Commons File Upload is interesting but since the upload comes from a Java app, not a browser it only solves the receiving end, not the sending.
Any ideas for a library which easily allows multiple file transfers across a single stream with fixed memory expectations even for very large files?
Thanks.
Just use HTTP multipart/form-data encoding on the POST request body. It's described in RFC-2388 and a standard way of uploading (multiple) files by HTTP.
You can do it with just java.net.URLConnection as described in this mini-tutorial, although it would generate lot of boilerplate code. A more convenienced approach would be using Apache Commons HttpClient.
In the servlet side you can then just use Apache Commons Fileupload to process the uploaded files the usual HTTP way (or when you're already on Servlet 3.0, the HttpServletRequest#getParts(), see also this answer for examples).
I have two web applications say App1 and App2. I want to call a servlet which is in App2 from a servlet in App1. I'm using URLConnection for this. I'm able to pass parameters to the servlet in App2 also and I'm also able to receive response from the servlet as string. But I want to send java objects from the servlet in App2 and receive them in servlet of App1. How to achieve this?
Depends.
If those webapplications runs at physically the same webserver in the same servletcontainer, then just set it as a request attribute and forward the request to the other context:
request.setAttribute("name", object);
ServletContext app2 = getServletContext().getContext("app2");
app2.getRequestDispacher("servletUrl").forward(request, response);
The other context will be able to obtain the object as follows:
Object object = request.getAttribute("name");
This only requires a server setting that the contexts are accessible by each other. How to do this depends on the servletcontainer. In Tomcat for example, you just need to set crossContext attribute of the webapp's <Context> element to true.
<Context crossContext="true">
Then it will be available to other contexts. For other servers, consult its documentation.
If those webapplications runs at physically different webserver, then there are several options:
Convert to String and send as parameter. On retrieval, convert back from String. JSON is a nice format for this. Google Gson offers possibilities to convert between fullworthy Java objects and JSON and vice versa. If you're using GET and the request URI gets pretty long, over 2KB, then consider using POST instead of GET, else the URI may be truncated by the server. Pros: better reuseable service. Cons: hard to send binary data.
See also: Converting JSON to Java.
Send a multipart/form-data HTTP POST request using URLConnection or Apache HttpComponents Client as per RFC2388 and process it on the other side using Apache Commons FileUpload. Pros: standard specification, possible to send binary data. Cons: more code.
See also: How to use URLConnection.
Serialize the Java object, write it raw to the URLConnection#getOutputStream() using ObjectOutputStream and retrieve it raw from the HttpServletRequest#getInputStream() and unserialize it using ObjectInputStream. Pros: easy. Cons: not reuseable, tight coupled.
See also: Object Streams and Lesson: Serialization.
Use Serialization
Instead of sending HTML you'll send objects streams.
Just, be aware, to add extra security, you don't want to some external source inject poisoned objects into your calls.
I'm looking for a way to get the form parameters of a HTTP multi-part request in a Servlet-filter without uploading files (yet).
request.getParameterMap() returns empty. I understand this is because of the request being multi-part.
I've looked at commons.HttpFileUpload but this seems to be overkill for my situation. In this filter I'm only interested in the normal parameters, and don't want to handle the file-upload yet.
Edit: the main problem is that I need to have an intact HttpRequestObject further down the filter stack. The HttpFileUpload seems to consume part of the request data (probably by using the data stream object and closing it again.)
It's certainly not overkill, it's the right way and always better than writing the parser yourself. The Apache Commons FileUpload is developed and maintained for years and has proven its robustness in handling multipart/form-data requests. You don't want to reinvent the wheel. If you really want to do it (I don't recommend it), then read on the multipart/form-data specification and start with reading the HttpServletRequest#getInputStream() (warning: this is a mix of binary and character data!).
You can if necessary also write a Filter which makes use of Apache Commons FileUpload under the hood and checks every request if it is multipart/form-data and if so, then put the parameters back in the request parameter map with help of Commons FileUpload and put the uploaded files (or exceptions) as request attributes, so that it's finally a bit more transparently in your servlet code. You can find here a basic example to get the idea.
Hope this helps.
Just to add to the answers already provided - I had a very similar problem in that I was trying to add some CSRF validation to our existing web app. We decided to include a special token in each form using some JS and add a servlet filter to check that the token existed (therefore a generic, isolated solution).
The servlet would check if the token was present but broke for every form that provided a file upload option. Hence I landed at this page frequently while doing some googling.
The work around we used (while attempting to avoid any dealings with the uploaded files) was to get some JavaScript to add the token as a GET parameter, i.e. We modified the form's action URL to include the token and therefore could use the HttpServletRequest.getParameter() method for the token (and only the token).
I have tested this in IE, FF and Chrome and all seem to be happy.
Hope this helps anyone who also finds themselves in a similar situation.
The Oreilly Servlets website has some sample code which you can download customise and use. This includes MultipartRequest which sounds like it does what you require, it un-boxes a multipart request and allows access to the parameters and the files separately.
Commons FileUpload provides a mechanism to read request params from a multipart form upload.
There's a really great example of how to grab the request parameters here:
How to upload files to server using JSP/Servlet?