How do I get an OutputStream using org.apache.http.impl.client.DefaultHttpClient?
I'm looking to write a long string to an output stream.
Using HttpURLConnection you would implement it like so:
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
OutputStream out = connection.getOutputStream();
Writer wout = new OutputStreamWriter(out);
writeXml(wout);
Is there a method using DefaultHttpClient similar to what I have above? How would I write to an OutputStream using DefaultHttpClient instead of HttpURLConnection?
e.g
DefaultHttpClient client = new DefaultHttpClient();
OutputStream outstream = (get OutputStream somehow)
Writer wout = new OutputStreamWriter(out);
I know that another answer has already been accepted, just for the record this is how one can write content out with HttpClient without intermediate buffering in memory.
AbstractHttpEntity entity = new AbstractHttpEntity() {
public boolean isRepeatable() {
return false;
}
public long getContentLength() {
return -1;
}
public boolean isStreaming() {
return false;
}
public InputStream getContent() throws IOException {
// Should be implemented as well but is irrelevant for this case
throw new UnsupportedOperationException();
}
public void writeTo(final OutputStream outstream) throws IOException {
Writer writer = new OutputStreamWriter(outstream, "UTF-8");
writeXml(writer);
writer.flush();
}
};
HttpPost request = new HttpPost(uri);
request.setEntity(entity);
You can't get an OutputStream from BasicHttpClient directly. You have to create an HttpUriRequest object and give it an HttpEntity that encapsulates the content you want to sent. For instance, if your output is small enough to fit in memory, you might do the following:
// Produce the output
ByteArrayOutputStream out = new ByteArrayOutputStream();
Writer writer = new OutputStreamWriter(out, "UTF-8");
writeXml(writer);
// Create the request
HttpPost request = new HttpPost(uri);
request.setEntity(new ByteArrayEntity(out.toByteArray()));
// Send the request
DefaultHttpClient client = new DefaultHttpClient();
HttpResponse response = client.execute(request);
If the data is large enough that you need to stream it, it becomes more difficult because there's no HttpEntity implementation that accepts an OutputStream. You'd need to write to a temp file and use FileEntity or possibly set up a pipe and use InputStreamEntity
EDIT See oleg's answer for sample code that demonstrates how to stream the content - you don't need a temp file or pipe after all.
This worked well on android. It should also work for large files, as no buffering is needed.
PipedOutputStream out = new PipedOutputStream();
PipedInputStream in = new PipedInputStream();
out.connect(in);
new Thread() {
#Override
public void run() {
//create your http request
InputStreamEntity entity = new InputStreamEntity(in, -1);
request.setEntity(entity);
client.execute(request,...);
//When this line is reached your data is actually written
}
}.start();
//do whatever you like with your outputstream.
out.write("Hallo".getBytes());
out.flush();
//close your streams
I wrote an inversion of Apache's HTTP Client API [PipedApacheClientOutputStream] which provides an OutputStream interface for HTTP POST using Apache Commons HTTP Client 4.3.4.
Calling-code looks like this:
// Calling-code manages thread-pool
ExecutorService es = Executors.newCachedThreadPool(
new ThreadFactoryBuilder()
.setNameFormat("apache-client-executor-thread-%d")
.build());
// Build configuration
PipedApacheClientOutputStreamConfig config = new
PipedApacheClientOutputStreamConfig();
config.setUrl("http://localhost:3000");
config.setPipeBufferSizeBytes(1024);
config.setThreadPool(es);
config.setHttpClient(HttpClientBuilder.create().build());
// Instantiate OutputStream
PipedApacheClientOutputStream os = new
PipedApacheClientOutputStream(config);
// Write to OutputStream
os.write(...);
try {
os.close();
} catch (IOException e) {
logger.error(e.getLocalizedMessage(), e);
}
// Do stuff with HTTP response
...
// Close the HTTP response
os.getResponse().close();
// Finally, shut down thread pool
// This must occur after retrieving response (after is) if interested
// in POST result
es.shutdown();
Note - In practice the same client, executor service, and config will likely be reused throughout the life of the application, so the outer prep and close code in the above example will likely live in bootstrap/init and finalization code rather than directly inline with the OutputStream instantiation.
Related
Here is the code I have so far.
What does works is reading all form name/values from the original request.
What does not work is the new server does not receive any of the newly assigned form name/values. Basically they dont seem to get transmitted to the secondary server.
There might be an easier way to do so?? All I need is to trigger on a specific form field from the new server and redirect to a sub-server that will handle the request and pass back the results thru the main server to the client (proxying).
String value = String.format("https://%s.myotherserver.com%s", "sub1", request.getRequestURI());
HttpPost uploadFile = new HttpPost(value);
uploadFile.addHeader("Content-Type", request.getContentType());
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
ContentBody cbFile = new InputStreamBody(request.getPart("audio").getInputStream(), ContentType.create("audio/webm"),"audio.ogg");
builder.addPart("audio", cbFile);
builder.addPart("text", new StringBody(request.getParameter("text"),ContentType.DEFAULT_TEXT));
builder.addPart("email", new StringBody(request.getParameter("email"),ContentType.DEFAULT_TEXT));
// now add the other original form name/values to new request
do
{
String parameterName = reqParameterNames.nextElement().toString();
Object parameterValue = request.getParameter(parameterName);
if (!privateParameters.contains("p_"+parameterName)) {
builder.addPart(new FormBodyPart(parameterName, new StringBody((String) parameterValue,ContentType.DEFAULT_TEXT)));
}
} while (reqParameterNames.hasMoreElements());
HttpEntity multipart = builder.build();
uploadFile.setEntity(multipart);
CloseableHttpClient httpClient2 = HttpClients.createDefault();
CloseableHttpResponse statusCode = httpClient2.execute(uploadFile);
HttpEntity responseEntity = statusCode.getEntity();
StringBuffer responseBuffer = new StringBuffer();
OutputStream output = response.getOutputStream();
ByteStreams.copy(responseEntity.getContent(), output);
output.flush();
I finally managed to get it to work with the following code. I hope this can help someone else;
MultipartEntityBuilder mb = null;
org.apache.http.HttpEntity entity =null;
String value = String.format("https://%s.myotherserver.com%s", "sub1", request.getRequestURI());
mb = MultipartEntityBuilder.create();
mb.addTextBody("noproxy", "true");
mb.addTextBody("text", request.getParameter("text"));
mb.addTextBody("email", request.getParameter("email"));
mb.addBinaryBody("audio", new File(inputAudioFilename));
entity = mb.build();
URLConnection conn = new URL(urlStr[i]).openConnection();
conn.setDoOutput(true);
conn.addRequestProperty(entity.getContentType().getName(), entity.getContentType().getValue());
conn.addRequestProperty("Content-Length", String.valueOf(entity.getContentLength()));
OutputStream fout = conn.getOutputStream();
entity.writeTo(fout);//write multi part data...
fout.flush();
fout.close();
OutputStream output = response.getOutputStream();
output.flush();
ByteStreams.copy(conn.getInputStream(),response.getOutputStream());
conn.getInputStream().close();
Currently, I'm working on tuneup client/server application communicated via http using HttpURLConnection.
Here, the client sends a java-serializable object as the request to a server (HttpServlet receives this) and it receives a JSON string as the response.
Then that JSON response is forwarded to a JSON parser to process.
Previously, first we downloaded and constructed the complete JSON string comming through the stream and then we forwarded it to the parser. Then I tuned it up to forward the stream it-self to the parser (Jackson Streaming parser) and let it to process on the part that it receives without waiting to download the entire JSON string.
This gives good results when the download speed is good. But here I'm getting not that much good results in one environment, since it is getting so much time to download and it process very faster. (sometimes 17(s) to download 4-5MB JSON string).
How can tune this up to download and parse the JSON faster than this?
Client
URL url = new URL(http://myhost.com/Connectors/url2Service);
HttpURLConnection servletConnection = (HttpURLConnection) url.openConnection();
servletConnection.setDoInput(true);
servletConnection.setDoOutput(true);
servletConnection.setUseCaches (false);
servletConnection.setDefaultUseCaches (false);
servletConnection.setRequestMethod("POST");
servletConnection.setRequestProperty("content-type","application/x-java-serialized-object");
servletConnection.setRequestProperty("Accept","application/json");
servletConnection.setRequestProperty("Accept-Encoding","gzip,deflate");
servletConnection.setRequestProperty("user-agent","Mozilla(MSIE)");
servletConnection.setConnectTimeout(30000);
servletConnection.setReadTimeout(60000);
servletConnection.connect();
String encodingHeader = servletConnection.getHeaderField("Content-Encoding");
String contentType = servletConnection.getHeaderField("content-type");
if(contentType.equals("application/json")){
InputStream inputStream = servletConnection.getInputStream();
if(encodingHeader != null && encodingHeader.toLowerCase().indexOf("gzip") != -1) {
gzipis = new GZIPInputStream(inputStream);
} else {
iSR = new InputStreamReader(inputStream);
br = new BufferedReader(iSR);
}
ResposeObject resposeObject = new JsonString2ResposeObject().parse(gzipis); // previously we passed complete String here
}
Server
public class url2Service extends HttpServlet{
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
ObjectInputStream ois = new ObjectInputStream(request.getInputStream());
RequestObject requestObject = (RequestObject) ois.readObject();
response.setContentType("application/json");
String jsonResponseString = new ServerApplication().doMoreWithRequest(requestObject);
OutputStreamWriter wr = null;
if (null != aEncoding && aEncoding.toLowerCase().indexOf("gzip") != -1) {
gzipos = new GZIPOutputStream(response.getOutputStream());
wr = new OutputStreamWriter(gzipos);
response.addHeader("Content-Encoding", "gzip,deflate");
} else {
wr = new OutputStreamWriter(response.getOutputStream());
}
wr.write(jsonString);
wr.flush();
}
}
I am trying to POST a file using HttpClient. However, I don't have access to the actual File but I have access to its InputStream. Is there a way I can still POST the file?
This is what I have done so far:
public void sendFile (InputStream instream) {
HttpClient client = new HttpClient();
PostMethod method = new PostMethod("http://localhost:8080/myservice/testupload");
Part[] parts = new Part[] {
//new FilePart("myFile", file.getName(), file)
};
method.setRequestEntity(
new MultipartRequestEntity(parts, method.getParams()));
client.executeMethod(method);
}
as you can see, the FilePart needs file but I have InputStream. How can I POST the input stream as a file?
Looking at the javadoc for FilePart, there is a constructor which accepts PartSource instead of File, and there is a subclass of PartSource called ByteArrayPartSource, which you can construct from byte[]; you can obtain this from the InputStream as described here.
I need to send a tar.gzip file from one java app (via a Servlet) to another - I'm using HTTP client with a MultipartEntity to achieve this.
During the file transfer, the file seems to double in size - as if it's being decompressed - and it's no longer recognizable as either a tar.gz or tar file.
Here's the send method:
HttpClient http = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
MultipartEntity multipart = new MultipartEntity();
ContentBody fileContent = new FileBody(file, "application/octet-stream");
ContentBody pathContent = new StringBody(file.getAbsolutePath());
multipart.addPart("package", fileContent);
multipart.addPart("path", pathContent);
post.setEntity(multipart);
HttpResponse response = null;
try {
response = http.execute(post);
StringWriter sw = new StringWriter();
IOUtils.copy(response.getEntity().getContent(), sw);
} catch (Exception ex){
log.error("Unable to POST to ["+url+"].",ex);
}
return result;
Here is the servlet method the above code is POSTing to:
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
log.info("File transfer request received, collecting file information and saving to server.");
Part filePart = req.getPart("package");
Part filePathPart = req.getPart("path");
StringWriter sw = new StringWriter();
IOUtils.copy(filePathPart.getInputStream(), sw);
String path = sw.getBuffer().toString();
File outputFile = new File(path);
FileWriter out = new FileWriter(outputFile);
IOUtils.copy(filePart.getInputStream(), out);
log.info("File ["+path+"] has been saved to the server.");
out.close();
sw.close();
}
I'm no expert on this stuff - and there doesn't appear to be much help via Google... Any help would be great.
Thanks,
Pete
Your concrete problem is caused because you're converting the incoming bytes to characters by using FileWriter instead of FileOutputStream here:
FileWriter out = new FileWriter(outputFile);
ZIP files are binary files, represented by a specific sequence of bytes, not character files like text, HTML, XML, etc. With converting bytes to characters this way, you're only malforming the original binary content which causes that the file is not recognizeable as a ZIP file anymore. You end up with a corrupted file.
If you use FileOutputStream instead, then your problem will be solved. There's absolutely no need to replace this all with Commons FileUpload.
See also:
How to upload files to server using JSP/Servlet?
Unrelated to the concrete problem, reusing a client side specific absolute path in the server side is not a good idea for security reasons, but you'll find out that sooner or later. Rather reuse at highest the filename, preferably in combination with File#createTempFile() to autogenerate an unique filename suffix.
I made this work by using Apache commons File Upload:
Send code:
HttpClient http = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
post.addHeader("path", file.getAbsolutePath());
MultipartEntity multipart = new MultipartEntity();
ContentBody fileContent = new FileBody(file); //For tar.gz: "application/x-gzip"
multipart.addPart("package", fileContent);
post.setEntity(multipart);
Receive code:
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
log.info("File transfer request received, collecting file information and saving to server.");
FileItemFactory factory = new DiskFileItemFactory();
ServletFileUpload upload = new ServletFileUpload(factory);
try {
List fileItems = upload.parseRequest(req);
Iterator iterator = fileItems.iterator();
if (iterator.hasNext()){
FileItem fileItem = (FileItem) iterator.next();
File file = new File(req.getHeader("path"));
fileItem.write(file);
log.info("File ["+fileItem.getName()+"] has been saved to the server.");
}
} catch (Exception ex) {
log.error("Unable to retrieve or write file set...",ex);
}
}
I need to send an Object from client to server by serializing it.
This is my code:
HttpURLConnection con = null;
ObjectOutputStream out = null;
ObjectInputStream inputStream = null;
URL servlet = new URL("MY_URL");
con = (HttpURLConnection) servlet.openConnection();
con.setDoInput(true);
con.setDoOutput(true);
con.setUseCaches(false);
con.setDefaultUseCaches(false);
con.setRequestProperty("Content-type", "application/octet-stream");
con.setRequestMethod("POST");
out = new ObjectOutputStream(con.getOutputStream());
out.writeObject(myobject);
out.flush();
out.close();
inputStream = new ObjectInputStream(con.getInputStream());
inputStream.close();
}
catch (Exception e)
{
e.printStackTrace();
}
finally
{
// inputStream.close();
con.disconnect();
}
return true;
Now, I am able to reach the Servlet, and I can retrieve the object through there.
The only problem is that as soon as I reach to this line:
inputStream = new ObjectInputStream(con.getInputStream());
I get an exception StreamCorruptedException, at the client side. (at the server side everything working great!)
And if I take this line off, the servlet not being triggered (I mean the doGet() or doPost() not being called in the servlet)
What am I doing wrong?
This is the exact error:
06-02 12:41:53.549: WARN/System.err(4260): java.io.StreamCorruptedException
06-02 12:41:53.549: WARN/System.err(4260): java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:2399)
06-02 12:41:53.549: WARN/System.err(4260): at java.io.ObjectInputStream.<init>(ObjectInputStream.java:447)
Thanks,
Ray
The client is expecting that the servlet writes an object back to the response something like:
ObjectOutputStream oos = new ObjectOutputStream(response.getOutputStream());
oos.writeObject(someObject);
But the servlet apparently actually doesn't write any object back. So the client should not decorate it with an ObjectInputStream. Just do so:
InputStream inputStream;
// ...
inputStream = connection.getInputStream();
or simply
connection.connect();
if you're not interested in the response anyway. The connection is executed on demand only. The getInputStream() will do that implicitly. That's why the request is not been fired until you call getInputStream(). Also see this answer for more hints.
Don't do this stuff yourself, look at HttpClient and spring's HttpInvoker.