I need to send a tar.gzip file from one java app (via a Servlet) to another - I'm using HTTP client with a MultipartEntity to achieve this.
During the file transfer, the file seems to double in size - as if it's being decompressed - and it's no longer recognizable as either a tar.gz or tar file.
Here's the send method:
HttpClient http = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
MultipartEntity multipart = new MultipartEntity();
ContentBody fileContent = new FileBody(file, "application/octet-stream");
ContentBody pathContent = new StringBody(file.getAbsolutePath());
multipart.addPart("package", fileContent);
multipart.addPart("path", pathContent);
post.setEntity(multipart);
HttpResponse response = null;
try {
response = http.execute(post);
StringWriter sw = new StringWriter();
IOUtils.copy(response.getEntity().getContent(), sw);
} catch (Exception ex){
log.error("Unable to POST to ["+url+"].",ex);
}
return result;
Here is the servlet method the above code is POSTing to:
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
log.info("File transfer request received, collecting file information and saving to server.");
Part filePart = req.getPart("package");
Part filePathPart = req.getPart("path");
StringWriter sw = new StringWriter();
IOUtils.copy(filePathPart.getInputStream(), sw);
String path = sw.getBuffer().toString();
File outputFile = new File(path);
FileWriter out = new FileWriter(outputFile);
IOUtils.copy(filePart.getInputStream(), out);
log.info("File ["+path+"] has been saved to the server.");
out.close();
sw.close();
}
I'm no expert on this stuff - and there doesn't appear to be much help via Google... Any help would be great.
Thanks,
Pete
Your concrete problem is caused because you're converting the incoming bytes to characters by using FileWriter instead of FileOutputStream here:
FileWriter out = new FileWriter(outputFile);
ZIP files are binary files, represented by a specific sequence of bytes, not character files like text, HTML, XML, etc. With converting bytes to characters this way, you're only malforming the original binary content which causes that the file is not recognizeable as a ZIP file anymore. You end up with a corrupted file.
If you use FileOutputStream instead, then your problem will be solved. There's absolutely no need to replace this all with Commons FileUpload.
See also:
How to upload files to server using JSP/Servlet?
Unrelated to the concrete problem, reusing a client side specific absolute path in the server side is not a good idea for security reasons, but you'll find out that sooner or later. Rather reuse at highest the filename, preferably in combination with File#createTempFile() to autogenerate an unique filename suffix.
I made this work by using Apache commons File Upload:
Send code:
HttpClient http = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
post.addHeader("path", file.getAbsolutePath());
MultipartEntity multipart = new MultipartEntity();
ContentBody fileContent = new FileBody(file); //For tar.gz: "application/x-gzip"
multipart.addPart("package", fileContent);
post.setEntity(multipart);
Receive code:
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
log.info("File transfer request received, collecting file information and saving to server.");
FileItemFactory factory = new DiskFileItemFactory();
ServletFileUpload upload = new ServletFileUpload(factory);
try {
List fileItems = upload.parseRequest(req);
Iterator iterator = fileItems.iterator();
if (iterator.hasNext()){
FileItem fileItem = (FileItem) iterator.next();
File file = new File(req.getHeader("path"));
fileItem.write(file);
log.info("File ["+fileItem.getName()+"] has been saved to the server.");
}
} catch (Exception ex) {
log.error("Unable to retrieve or write file set...",ex);
}
}
Related
I am uploading a File from GWT to a different domain
File Uploads well , But the response i sent from the server always reaches as "null" at the client side
response.setContentType("text/html");
response.setHeader("Access-Control-Allow-Origin", "*");
response.getWriter().print("TEST");
response is NULL only when i upload the file on a different domain ... (on same domain all is OK)
I also see this in GWT documentation
Tip:
The result html can be null as a result of submitting a form to a different domain.
http://www.gwtproject.org/javadoc/latest/com/google/gwt/user/client/ui/FormPanel.SubmitCompleteEvent.html
Is there any way I can receive back a response at my client side when i am uploading file to a different domain
There are 2 possible answer:
Use JSONP Builder
JsonpRequestBuilder requestBuilder = new JsonpRequestBuilder();
requestBuilder.requestObject(url, new AsyncCallback<FbUser>() {
#Override
public void onFailure(Throwable ex) {
throw SOMETHING_EXCEPTION(ex);
}
#Override
public void onSuccess(ResponseModel resp) {
if (resp.isError()) {
// on response error on something
log.error(resp.getError().getMessage())
log.error(resp.getError().getCode())
}
log.info(resp.getAnyData())
}
Not to use GWT to upload, rather use other client like apache HttpClient
public uploadFile() {
HttpClient httpClient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpPost httpPost = new HttpPost(url);
FileBody bin = new FileBody(new File(UPLOADED_FILE));
long size = bin.getContentLength();
MultipartEntity reqEntity = new MultipartEntity();
reqEntity.addPart("PART", bin);
String content = "-";
try {
httpPost.setEntity(reqEntity);
HttpResponse response = httpClient.execute(httpPost, localContext);
HttpEntity ent = response.getEntity();
InputStream st = ent.getContent();
StringWriter writer = new StringWriter();
IOUtils.copy(st, writer);
content = writer.toString();
} catch (IOException e) {
return "false";
}
return content;
}
Hope it helps
Here is the code I have so far.
What does works is reading all form name/values from the original request.
What does not work is the new server does not receive any of the newly assigned form name/values. Basically they dont seem to get transmitted to the secondary server.
There might be an easier way to do so?? All I need is to trigger on a specific form field from the new server and redirect to a sub-server that will handle the request and pass back the results thru the main server to the client (proxying).
String value = String.format("https://%s.myotherserver.com%s", "sub1", request.getRequestURI());
HttpPost uploadFile = new HttpPost(value);
uploadFile.addHeader("Content-Type", request.getContentType());
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
ContentBody cbFile = new InputStreamBody(request.getPart("audio").getInputStream(), ContentType.create("audio/webm"),"audio.ogg");
builder.addPart("audio", cbFile);
builder.addPart("text", new StringBody(request.getParameter("text"),ContentType.DEFAULT_TEXT));
builder.addPart("email", new StringBody(request.getParameter("email"),ContentType.DEFAULT_TEXT));
// now add the other original form name/values to new request
do
{
String parameterName = reqParameterNames.nextElement().toString();
Object parameterValue = request.getParameter(parameterName);
if (!privateParameters.contains("p_"+parameterName)) {
builder.addPart(new FormBodyPart(parameterName, new StringBody((String) parameterValue,ContentType.DEFAULT_TEXT)));
}
} while (reqParameterNames.hasMoreElements());
HttpEntity multipart = builder.build();
uploadFile.setEntity(multipart);
CloseableHttpClient httpClient2 = HttpClients.createDefault();
CloseableHttpResponse statusCode = httpClient2.execute(uploadFile);
HttpEntity responseEntity = statusCode.getEntity();
StringBuffer responseBuffer = new StringBuffer();
OutputStream output = response.getOutputStream();
ByteStreams.copy(responseEntity.getContent(), output);
output.flush();
I finally managed to get it to work with the following code. I hope this can help someone else;
MultipartEntityBuilder mb = null;
org.apache.http.HttpEntity entity =null;
String value = String.format("https://%s.myotherserver.com%s", "sub1", request.getRequestURI());
mb = MultipartEntityBuilder.create();
mb.addTextBody("noproxy", "true");
mb.addTextBody("text", request.getParameter("text"));
mb.addTextBody("email", request.getParameter("email"));
mb.addBinaryBody("audio", new File(inputAudioFilename));
entity = mb.build();
URLConnection conn = new URL(urlStr[i]).openConnection();
conn.setDoOutput(true);
conn.addRequestProperty(entity.getContentType().getName(), entity.getContentType().getValue());
conn.addRequestProperty("Content-Length", String.valueOf(entity.getContentLength()));
OutputStream fout = conn.getOutputStream();
entity.writeTo(fout);//write multi part data...
fout.flush();
fout.close();
OutputStream output = response.getOutputStream();
output.flush();
ByteStreams.copy(conn.getInputStream(),response.getOutputStream());
conn.getInputStream().close();
I am trying to POST a file using HttpClient. However, I don't have access to the actual File but I have access to its InputStream. Is there a way I can still POST the file?
This is what I have done so far:
public void sendFile (InputStream instream) {
HttpClient client = new HttpClient();
PostMethod method = new PostMethod("http://localhost:8080/myservice/testupload");
Part[] parts = new Part[] {
//new FilePart("myFile", file.getName(), file)
};
method.setRequestEntity(
new MultipartRequestEntity(parts, method.getParams()));
client.executeMethod(method);
}
as you can see, the FilePart needs file but I have InputStream. How can I POST the input stream as a file?
Looking at the javadoc for FilePart, there is a constructor which accepts PartSource instead of File, and there is a subclass of PartSource called ByteArrayPartSource, which you can construct from byte[]; you can obtain this from the InputStream as described here.
How do I get an OutputStream using org.apache.http.impl.client.DefaultHttpClient?
I'm looking to write a long string to an output stream.
Using HttpURLConnection you would implement it like so:
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
OutputStream out = connection.getOutputStream();
Writer wout = new OutputStreamWriter(out);
writeXml(wout);
Is there a method using DefaultHttpClient similar to what I have above? How would I write to an OutputStream using DefaultHttpClient instead of HttpURLConnection?
e.g
DefaultHttpClient client = new DefaultHttpClient();
OutputStream outstream = (get OutputStream somehow)
Writer wout = new OutputStreamWriter(out);
I know that another answer has already been accepted, just for the record this is how one can write content out with HttpClient without intermediate buffering in memory.
AbstractHttpEntity entity = new AbstractHttpEntity() {
public boolean isRepeatable() {
return false;
}
public long getContentLength() {
return -1;
}
public boolean isStreaming() {
return false;
}
public InputStream getContent() throws IOException {
// Should be implemented as well but is irrelevant for this case
throw new UnsupportedOperationException();
}
public void writeTo(final OutputStream outstream) throws IOException {
Writer writer = new OutputStreamWriter(outstream, "UTF-8");
writeXml(writer);
writer.flush();
}
};
HttpPost request = new HttpPost(uri);
request.setEntity(entity);
You can't get an OutputStream from BasicHttpClient directly. You have to create an HttpUriRequest object and give it an HttpEntity that encapsulates the content you want to sent. For instance, if your output is small enough to fit in memory, you might do the following:
// Produce the output
ByteArrayOutputStream out = new ByteArrayOutputStream();
Writer writer = new OutputStreamWriter(out, "UTF-8");
writeXml(writer);
// Create the request
HttpPost request = new HttpPost(uri);
request.setEntity(new ByteArrayEntity(out.toByteArray()));
// Send the request
DefaultHttpClient client = new DefaultHttpClient();
HttpResponse response = client.execute(request);
If the data is large enough that you need to stream it, it becomes more difficult because there's no HttpEntity implementation that accepts an OutputStream. You'd need to write to a temp file and use FileEntity or possibly set up a pipe and use InputStreamEntity
EDIT See oleg's answer for sample code that demonstrates how to stream the content - you don't need a temp file or pipe after all.
This worked well on android. It should also work for large files, as no buffering is needed.
PipedOutputStream out = new PipedOutputStream();
PipedInputStream in = new PipedInputStream();
out.connect(in);
new Thread() {
#Override
public void run() {
//create your http request
InputStreamEntity entity = new InputStreamEntity(in, -1);
request.setEntity(entity);
client.execute(request,...);
//When this line is reached your data is actually written
}
}.start();
//do whatever you like with your outputstream.
out.write("Hallo".getBytes());
out.flush();
//close your streams
I wrote an inversion of Apache's HTTP Client API [PipedApacheClientOutputStream] which provides an OutputStream interface for HTTP POST using Apache Commons HTTP Client 4.3.4.
Calling-code looks like this:
// Calling-code manages thread-pool
ExecutorService es = Executors.newCachedThreadPool(
new ThreadFactoryBuilder()
.setNameFormat("apache-client-executor-thread-%d")
.build());
// Build configuration
PipedApacheClientOutputStreamConfig config = new
PipedApacheClientOutputStreamConfig();
config.setUrl("http://localhost:3000");
config.setPipeBufferSizeBytes(1024);
config.setThreadPool(es);
config.setHttpClient(HttpClientBuilder.create().build());
// Instantiate OutputStream
PipedApacheClientOutputStream os = new
PipedApacheClientOutputStream(config);
// Write to OutputStream
os.write(...);
try {
os.close();
} catch (IOException e) {
logger.error(e.getLocalizedMessage(), e);
}
// Do stuff with HTTP response
...
// Close the HTTP response
os.getResponse().close();
// Finally, shut down thread pool
// This must occur after retrieving response (after is) if interested
// in POST result
es.shutdown();
Note - In practice the same client, executor service, and config will likely be reused throughout the life of the application, so the outer prep and close code in the above example will likely live in bootstrap/init and finalization code rather than directly inline with the OutputStream instantiation.
I am sending a image from android phone to server which is a servlet I am using the HttpClient and HttpPost for this and ByteArrayBody for storing the image before sending.
how do i extract the image from the post request in Servlet.
Here is my code for sending the post request
String postURL = //server url;
HttpClient httpClient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost(postURL);
ByteArrayBody bab = new ByteArrayBody(imageBytes,"file_name_ignored");
MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
reqEntity.addPart("source", bab);
postRequest.setEntity(reqEntity);
HttpResponse response = httpClient.execute(postRequest);
Use something like commons fileupload.
There are examples in the Apache docs, and all over the web.
Servlet 3.0 has support for reading multipart data. MutlipartConfig support in Servlet 3.0
If a servelt is annotated using #MutlipartConfig annotation, the container is responsible for making the Multipart parts available through
HttpServletRequest.getParts()
HttpServletRequest.getPart("name");
use http://commons.apache.org/fileupload/using.html
private DiskFileItemFactory fif = new DiskFileItemFactory();
protected void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
boolean isMultipart = ServletFileUpload.isMultipartContent(req);
if(!isMultipart)
throw new ServletException("upload using multipart");
ServletFileUpload upload = new ServletFileUpload(fif);
upload.setSizeMax(1024 * 1024 * 10 /* 10 mb */);
List<FileItem> items;
try {
items = upload.parseRequest(req);
} catch (FileUploadException e) {
throw new ServletException(e);
}
if(items == null || items.size() == 0)
throw new ServletException("No items uploaded");
FileItem item = items.get(0);
// do something with file item...
}