After almost 2 workdays of Googling and trying several different possibilities I found throughout the web, I'm asking this question here, hoping that I might finally get an answer.
First of all, here's what I want to do:
I'm developing a client and a server application with the purpose of exchanging a lot of large files between multiple clients on a single server. The client is developed in pure Java (JDK 1.6), while the web application is done in Grails (2.0.0).
As the purpose of the client is to allow users to exchange a lot of large files (usually about 2GB each), I have to implement it in a way, so that the uploads are resumable, i.e. the users are able to stop and resume uploads at any time.
Here's what I did so far:
I actually managed to do what I wanted to do and stream large files to the server while still being able to pause and resume uploads using raw sockets. I would send a regular request to the server (using Apache's HttpClient library) to get the server to send me a port that was free for me to use, then open a ServerSocket on the server and connect to that particular socket from the client.
Here's the problem with that:
Actually, there are at least two problems with that:
I open those ports myself, so I have to manage open and used ports myself. This is quite error-prone.
I actually circumvent Grails' ability to manage a huge amount of (concurrent) connections.
Finally, here's what I'm supposed to do now and the problem:
As the problems I mentioned above are unacceptable, I am now supposed to use Java's URLConnection/HttpURLConnection classes, while still sticking to Grails.
Connecting to the server and sending simple requests is no problem at all, everything worked fine. The problems started when I tried to use the streams (the connection's OutputStream in the client and the request's InputStream in the server). Opening the client's OutputStream and writing data to it is as easy as it gets. But reading from the request's InputStream seems impossible to me, as that stream is always empty, as it seems.
Example Code
Here's an example of the server side (Groovy controller):
def test() {
InputStream inStream = request.inputStream
if(inStream != null) {
int read = 0;
byte[] buffer = new byte[4096];
long total = 0;
println "Start reading"
while((read = inStream.read(buffer)) != -1) {
println "Read " + read + " bytes from input stream buffer" //<-- this is NEVER called
}
println "Reading finished"
println "Read a total of " + total + " bytes" // <-- 'total' will always be 0 (zero)
} else {
println "Input Stream is null" // <-- This is NEVER called
}
}
This is what I did on the client side (Java class):
public void connect() {
final URL url = new URL("myserveraddress");
final byte[] message = "someMessage".getBytes(); // Any byte[] - will be a file one day
HttpURLConnection connection = url.openConnection();
connection.setRequestMethod("GET"); // other methods - same result
// Write message
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
out.writeBytes(message);
out.flush();
out.close();
// Actually connect
connection.connect(); // is this placed correctly?
// Get response
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line = null;
while((line = in.readLine()) != null) {
System.out.println(line); // Prints the whole server response as expected
}
in.close();
}
As I mentioned, the problem is that request.inputStream always yields an empty InputStream, so I am never able to read anything from it (of course). But as that is exactly what I'm trying to do (so I can stream the file to be uploaded to the server, read from the InputStream and save it to a file), this is rather disappointing.
I tried different HTTP methods, different data payloads, and also rearranged the code over and over again, but did not seem to be able to solve the problem.
What I hope to find
I hope to find a solution to my problem, of course. Anything is highly appreciated: hints, code snippets, library suggestions and so on. Maybe I'm even having it all wrong and need to go in a totally different direction.
So, how can I implement resumable file uploads for rather large (binary) files from a Java client to a Grails web application without manually opening ports on the server side?
HTTP GET method have special headers for range retrieval: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35 It's used by most downloaders to do resumable download from server.
As I understand, there are no standard practice for using this headers for POST/PUT request, but it's up to you, right? You can make pretty standard Grails controller, that will accept standard http upload, with header like Range: bytes=500-999. And controller should put this 500 uploaded bytes from client into file, starting at position 500
At this case you don't need to open any socket, and make own protocols, etc.
P.S. 500 bytes is just a example, probably you're using much bigger parts.
Client Side Java Programming:
public class NonFormFileUploader {
static final String UPLOAD_URL= "http://localhost:8080/v2/mobileApp/fileUploadForEOL";
static final int BUFFER_SIZE = 4096;
public static void main(String[] args) throws IOException {
// takes file path from first program's argument
String filePath = "G:/study/GettingStartedwithGrailsFinalInfoQ.pdf";
File uploadFile = new File(filePath);
System.out.println("File to upload: " + filePath);
// creates a HTTP connection
URL url = new URL(UPLOAD_URL);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
httpConn.setDoOutput(true);
httpConn.setRequestMethod("POST");
// sets file name as a HTTP header
httpConn.setRequestProperty("fileName", uploadFile.getName());
// opens output stream of the HTTP connection for writing data
OutputStream outputStream = httpConn.getOutputStream();
// Opens input stream of the file for reading data
FileInputStream inputStream = new FileInputStream(uploadFile);
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer)) != -1) {
System.out.println("bytesRead:"+bytesRead);
outputStream.write(buffer, 0, bytesRead);
outputStream.flush();
}
System.out.println("Data was written.");
outputStream.flush();
outputStream.close();
inputStream.close();
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
// reads server's response
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String response = reader.readLine();
System.out.println("Server's response: " + response);
} else {
System.out.println("Server returned non-OK code: " + responseCode);
}
}
}
Server Side Grails Programme:
Inside the controller:
def fileUploadForEOL(){
def result
try{
result = mobileAppService.fileUploadForEOL(request);
}catch(Exception e){
log.error "Exception in fileUploadForEOL service",e
}
render result as JSON
}
Inside the Service Class:
def fileUploadForEOL(request){
def status = false;
int code = 500
def map = [:]
try{
String fileName = request.getHeader("fileName");
File saveFile = new File(SAVE_DIR + fileName);
System.out.println("===== Begin headers =====");
Enumeration<String> names = request.getHeaderNames();
while (names.hasMoreElements()) {
String headerName = names.nextElement();
System.out.println(headerName + " = " + request.getHeader(headerName));
}
System.out.println("===== End headers =====\n");
// opens input stream of the request for reading data
InputStream inputStream = request.getInputStream();
// opens an output stream for writing file
FileOutputStream outputStream = new FileOutputStream(saveFile);
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = inputStream.read(buffer);
long count = bytesRead
while(bytesRead != -1) {
outputStream.write(buffer, 0, bytesRead);
bytesRead = inputStream.read(buffer);
count += bytesRead
}
println "count:"+count
System.out.println("Data received.");
outputStream.close();
inputStream.close();
System.out.println("File written to: " + saveFile.getAbsolutePath());
code = 200
}catch(Exception e){
mLogger.log(java.util.logging.Level.SEVERE,"Exception in fileUploadForEOL",e);
}finally{
map <<["code":code]
}
return map
}
I have tried with above code it is worked for me(only for file size 3 to 4MB, but for small size files some bytes of code missing or not even coming but in request header content-length is coming, not sure why it is happening.)
Related
I am reading this file: https://www.reddit.com/r/tech/top.json?limit=100 into a BufferedReader from a HttpUrlConnection. I've got it to read some of the file, but it only reads about a 1/10th of what it should. It doesn't change anything if I change the size of the input buffer - it prints the same thing just in smaller chunks:
try{
URL url = new URL(urlString);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
StringBuilder sb = new StringBuilder();
int charsRead;
char[] inputBuffer = new char[500];
while(true) {
charsRead = reader.read(inputBuffer);
if(charsRead < 0) {
break;
}
if(charsRead > 0) {
sb.append(String.copyValueOf(inputBuffer, 0, charsRead));
Log.d(TAG, "Value read " + String.copyValueOf(inputBuffer, 0, charsRead));
}
}
reader.close();
return sb.toString();
} catch(Exception e){
e.printStackTrace();
}
I believe the issue is that the text is all on one line since it's not formatted in json correctly, and BufferedReader can only take a line so long. Is there any way around this?
read() should continue to read as long as charsRead > 0. Every time it makes a call to read, the reader marks where it last read from and the next call starts at that place and continues on until there is no more to read. There is no limit to the size it can read. The only limit is the size of the array but the overall size of the file there is none.
You could try the following:
try(InputStream is = connection.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
int read = 0;
byte[] buffer = new byte[4096];
while((read = is.read(buffer)) > 0) {
baos.write(buffer, 0, read);
}
return new String(baos.toByteArray(), StandardCharsets.UTF_8);
} catch (Exception ex){}
The above method is using purely the bytes from the stream and reading it into the output stream, then creating the string from that.
I suggest using 3d party Http client. It could reduce your code literally to just a few lines and you don't have to worry about all those little details. Bottom line is - someone already wrote the code that you are trying to write. And it works and already well tested. Few suggestions:
Apache Http Client - A well known and popular Http client, but might be a bit bulky and complicated for a simple case like yours.
Ok Http Client - Another well-known Http client
And finally, my favorite (because it is written by me) MgntUtils Open Source library that has Http Client. Maven artifacts can be found here, GitHub that includes the library itself as a jar file, source code, and Javadoc can be found here and JavaDoc is here
Just to demonstrate the simplicity of what you want to do here is the code using MgntUtils library. (I tested the code and it works like a charm)
private static void testHttpClient() {
HttpClient client = new HttpClient();
client.setContentType("application/json; charset=utf-8");
client.setConnectionUrl("https://www.reddit.com/r/tech/top.json?limit=100");
String content = null;
try {
content = client.sendHttpRequest(HttpMethod.GET);
} catch (IOException e) {
content = client.getLastResponseMessage() + TextUtils.getStacktrace(e, false);
}
System.out.println(content);
}
My wild guess is that your default platform charset was UTF-8 and encoding problems were raised. For remote content the encoding should be specified, and not assumed to be equal to the default encoding on your machine.
The charset of the response data must be correct. For that the headers must be inspected. The default should be Latin-1, ISO-8859-1, but browsers interprete that
as Windows Latin-1, Cp-1252.
String charset = connection.getContentType().replace("^.*(charset=|$)", "");
if (charset.isEmpty()) {
charset = "Windows-1252"; // Windows Latin-1
}
Then you can better read bytes, as there is no exact correspondence to the number of bytes read and the number of chars read. If at the end of a buffer is the first char of a surrogate pair, two UTF-16 chars that form a Unicode glyph, symbol, code point above U+FFFF, I do not know the efficiency of the underlying "repair."
BufferedInputStream in = new BufferedInputStream(connection.getInputStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buffer = new byte[512];
while (true) {
int bytesRead = in.read(buffer);
if (bytesRead < 0) {
break;
}
if (bytesRead > 0) {
out.write(buffer, 0, bytesRead);
}
}
return out.toString(charset);
And indeed it is safe to do:
sb.append(inputBuffer, 0, charsRead);
(Taking a copy was probably a repair attempt.)
By the way char[500] takes almost twice the memory of byte[512].
I saw that the site uses gzip compression in my browser. That makes sense for text such as json. I mimicked it by setting a request header Accept-Encoding: gzip.
URL url = new URL("https://www.reddit.com/r/tech/top.json?limit=100");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("Accept-Encoding", "gzip");
try (InputStream rawIn = connection.getInputStream()) {
String charset = connection.getContentType().replaceFirst("^.*?(charset=|$)", "");
if (charset.isEmpty()) {
charset = "Windows-1252"; // Windows Latin-1
}
boolean gzipped = "gzip".equals(connection.getContentEncoding());
System.out.println("gzip=" + gzipped);
try (InputStream in = gzipped ? new GZIPInputStream(rawIn)
: new BufferedInputStream(rawIn)) {
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buffer = new byte[512];
while (true) {
int bytesRead = in.read(buffer);
if (bytesRead < 0) {
break;
}
if (bytesRead > 0) {
out.write(buffer, 0, bytesRead);
}
}
return out.toString(charset);
}
}
It might be for not gzip conform "browsers" the content length of the compressed content was erroneously set in the response. Which is a bug.
I believe the issue is that the text is all on one line since it's not formatted in json correctly, and BufferedReader can only take a line so long.
This explanation is not correct:
You are not reading a line at a time, and BufferedReader is not treating the text as line based.
Even when you do read from a BufferedReader a line at a time (i.e. using readLine()) the only limits on the length of a line are the inherent limits of a Java String length (2^31 - 1 characters), and the size of your heap.
Also, note that "correct" JSON formatting is subjective. The JSON specification says nothing about formatting. It is common for JSON emitters to not waste CPU cycles and network bandwidth on formatting for JSON that a human will only rarely read. Application code that consumes JSON needs to be able cope with this.
So what is actually going on?
Unclear, but here are some possibilities:
A StringBuilder also has an inherent limit of 2^31 - 1 characters. However, with (at least) some implementations, if you attempt to grow a StringBuilder beyond that limit, it will throw an OutOfMemoryError. (This behavior doesn't appear to be documented, but it is clear from reading the source code in Java 8.)
Maybe you are reading the data too slowly (e.g. because your network connection is too slow) and the server is timing out the connection.
Maybe the server has a limit on the amount of data that it is willing to send in a response.
Since you haven't mentioned any exceptions and you always seem to get the same amount of data, I suspect the 3rd explanation is the correct one.
I'm downloading video files that are larger than the memory space that Android apps are given. When they're *on the device, the MediaPlayer handles them quite nicely, so their overall size isn't the issue.
The problem is that if they exceed the relatively small number of megabytes that a byte[] can be then I get the dreaded OutOfMemory exception as I download them.
My intended solution is to just write the incoming byte stream straight to the SD card, however, I'm using the Apache Commons library and the way I'm doing it tries to get the entire video read in before it hands it back to me.
My code looks like this:
HttpClient client = new HttpClient();
PostMethod filePost = new PostMethod(URL_PATH);
client.setConnectionTimeout(timeout);
byte [] ret ;
try{
if(nvpArray != null)
filePost.setRequestBody(nvpArray);
}catch(Exception e){
Log.d(TAG, "download failed: " + e.toString());
}
try{
responseCode = client.executeMethod(filePost);
Log.d(TAG,"statusCode>>>" + responseCode);
ret = filePost.getResponseBody();
....
I'm curious what another approach would be to get the byte stream one byte at a time and just write it out to disk as it comes.
You should be able to use the GetResponseBodyAsStream method of your PostMethod object and stream it to a file. Here's an untested example....
InputStream inputStream = filePost.getResponseBodyAsStream();
FileInputStream outputStream = new FileInputStream(destination);
// Per your question the buffer is set to 1 byte, but you should be able to use
// a larger buffer.
byte[] buffer = new byte[1];
int bytesRead;
while ((bytesRead = input.read(buffer)) != -1)
{
outputStream.write(buffer, 0, bytesRead);
}
outputStream.close();
inputStream.close();
I have custom socket client server data (file or text) transmission code. Now when I transfer binary files, some bytes convert onto out of range characters. So I send them in hex string. That works. But for another problem this is not the solution. This has a performance problems as well.
I took help from Java code To convert byte to Hexadecimal.
When I download images from the net, same thing happens. Some bytes change into something else. I have compared bytes by bytes.
Converting into String show ? instead of the symbol. I have tried readers and byte array input stream. I have tried all the examples on the net. What is the mistake I could be doing?
My Code to save bytes to file:
void saveFile(String strFileName){
try{
URL url = new URL(strImageRoot + strFileName);
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
BufferedWriter bw = new BufferedWriter(new FileWriter(strImageDownloadPath + strFileName));
String line = null;
while ( (line = reader.readLine()) != null) {
bw.write(line);
}
}catch(FileNotFoundException fnfe){
System.out.println("FileNotFoundException occured!!!");
}catch(IOException ioe){
}catch(Exception e){
System.out.println("Exception occured : " + e);
}finally{
System.out.println("Image downloaded!!!");
}
}
i had a similar issue when i was building a Socket client server application. The bytes would be some weird characters and i tried all sorts of things to try and compare them. Then i came across a discussion where some1 pointed out to me that i should use a datainputstream, dataoutstream and let that do the conversion to and from bytes. that worked for me totally. i never touched the bytes at all.
use this code
File root = android.os.Environment.getExternalStorageDirectory();
File dir = new File (root.getAbsolutePath() + "/image");
if(dir.exists()==false) {
dir.mkdirs();
}
URL url = new URL("http://4.bp.blogspot.com/-zqJs1fVcfeY/TiZM7e-pFqI/AAAAAAAABjo/aKTtTDTCgKU/s1600/Final-Fantasy-X-Night-Sky-881.jpg");
//URL url = new URL(DownloadUrl);
//you can write here any link
File file = new File(dir,"Final-Fantasy-X-Night-Sky-881.jpg");
long startTime = System.currentTimeMillis();
//Open a connection to that URL.
URLConnection ucon = url.openConnection();
//* Define InputStreams to read from the URLConnection.
InputStream is = ucon.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
//* Read bytes to the Buffer until there is nothing more to read(-1).
ByteArrayBuffer baf = new ByteArrayBuffer(6000);
int current = 0;
while ((current = bis.read()) != -1) {
baf.append((byte) current);
}
//Convert the Bytes read to a String.
FileOutputStream fos = new FileOutputStream(file);
fos.write(baf.toByteArray());
fos.flush();
fos.close();
You should take the help of this link: How to encode decode in base64 in Android.
You can send byte array obtained from a file as string by encoding into Base64. This reduces the amount of data transmitted as well.
At the receiving end just decode the string using Base64 and obtain byte array.
Then you can use #Deepak Swami's solution to save bytes in file.
I recently found out that PHP service APIs do not know about what is byte array. Any String can be byte stream at the same time, so the APIs expect Base64 string in the request parameter. Please see the posts:
String to byte array in php
Passing base64 encoded strings in URL
Hence Base64 has quite importance as also it allows you to also save byte arrays in preferences, and increases performance if you have to send file data across network using Serialization.
Happy Coding :-)
I am using Apache HTTPClient 4 to connect to twitter's streaming api with default level access. It works perfectly well in the beginning but after a few minutes of retrieving data it bails out with this error:
2012-03-28 16:17:00,040 DEBUG org.apache.http.impl.conn.SingleClientConnManager: Get connection for route HttpRoute[{tls}->http://myproxy:80->https://stream.twitter.com:443]
2012-03-28 16:17:00,040 WARN com.cloudera.flume.core.connector.DirectDriver: Exception in source: TestTwitterSource
java.lang.IllegalStateException: Invalid use of SingleClientConnManager: connection still allocated.
at org.apache.http.impl.conn.SingleClientConnManager.getConnection(SingleClientConnManager.java:216)
Make sure to release the connection before allocating another one.
at org.apache.http.impl.conn.SingleClientConnManager$1.getConnection(SingleClientConnManager.java:190)
I understand why I am facing this issue. I am trying to use this HttpClient in a flume cluster as a flume source. The code looks like this:
public Event next() throws IOException, InterruptedException {
try {
HttpHost target = new HttpHost("stream.twitter.com", 443, "https");
new BasicHttpContext();
HttpPost httpPost = new HttpPost("/1/statuses/filter.json");
StringEntity postEntity = new StringEntity("track=birthday",
"UTF-8");
postEntity.setContentType("application/x-www-form-urlencoded");
httpPost.setEntity(postEntity);
HttpResponse response = httpClient.execute(target, httpPost,
new BasicHttpContext());
BufferedReader reader = new BufferedReader(new InputStreamReader(
response.getEntity().getContent()));
String line = null;
StringBuffer buffer = new StringBuffer();
while ((line = reader.readLine()) != null) {
buffer.append(line);
if(buffer.length()>30000) break;
}
return new EventImpl(buffer.toString().getBytes());
} catch (IOException ie) {
throw ie;
}
}
I am trying to buffer 30,000 characters in the response stream to a StringBuffer and then return this as the data received. I am obviously not closing the connection - but I do not want to close it just yet I guess. Twitter's dev guide talks about this here It reads:
Some HTTP client libraries only return the response body after the
connection has been closed by the server. These clients will not work
for accessing the Streaming API. You must use an HTTP client that will
return response data incrementally. Most robust HTTP client libraries
will provide this functionality. The Apache HttpClient will handle
this use case, for example.
It clearly tells you that HttpClient will return response data incrementally. I've gone through the examples and tutorials, but I haven't found anything that comes close to doing this. If you guys have used a httpclient (if not apache) and read the streaming api of twitter incrementally, please let me know how you achieved this feat. Those who haven't, please feel free to contribute to answers. TIA.
UPDATE
I tried doing this: 1) I moved obtaining stream handle to the open method of the flume source. 2) Using a simple inpustream and reading data into a bytebuffer. So here is what the method body looks like now:
byte[] buffer = new byte[30000];
while (true) {
int count = instream.read(buffer);
if (count == -1)
continue;
else
break;
}
return new EventImpl(buffer);
This works to an extent - I get tweets, they are nicely being written to a destination. The problem is with the instream.read(buffer) return value. Even when there is no data on the stream, and the buffer has default \u0000 bytes and 30,000 of them, so this value is getting written to the destination. So the destination file looks like this.. " tweets..tweets..tweeets.. \u0000\u0000\u0000\u0000\u0000\u0000\u0000...tweets..tweets... ". I understand the count won't return a -1 coz this is a never ending stream, so how do I figure out if the buffer has new content from the read command?
The problem is that your code is leaking connections. Please make sure that no matter what you either close the content stream or abort the request.
InputStream instream = response.getEntity().getContent();
try {
BufferedReader reader = new BufferedReader(
new InputStreamReader(instream));
String line = null;
StringBuffer buffer = new StringBuffer();
while ((line = reader.readLine()) != null) {
buffer.append(line);
if (buffer.length()>30000) {
httpPost.abort();
// connection will not be re-used
break;
}
}
return new EventImpl(buffer.toString().getBytes());
} finally {
// if request is not aborted the connection can be re-used
try {
instream.close();
} catch (IOException ex) {
// log or ignore
}
}
It turns out that it was a flume issue. Flume is optimized to transfer events of size 32kb. Anything beyond 32kb, Flume bails out. (The workaround is to tune event size to be greater than 32KB). So, I've changed my code to buffer 20,000 characters at least. It kind of works, but it is not fool proof. This can still fail if the buffer length exceeds 32kb, however, it hasn't failed so far in an hour of testing - I believe it has to do with the fact that Twitter doesn't send a lot of data on its public stream.
while ((line = reader.readLine()) != null) {
buffer.append(line);
if(buffer.length()>20000) break;
}
Hey, I've tried researching how to POST data from java, and nothing seems to do what I want to do. Basically, theres a form for uploading an image to a server, and what I want to do is post an image to the same server - but from java. It also needs to have the right parameter name (whatever the form input's name is). I would also want to return the response from this method.
It baffles me as to why this is so difficult to find, since this seems like something so basic.
EDIT ---- Added code
Based on some of the stuff BalusC showed me, I created the following method. It still doesn't work, but its the most successful thing I've gotten yet (seems to post something to the other server, and returns some kind of response - I'm not sure I got the response correctly though):
EDIT2 ---- added to code based on BalusC's feedback
EDIT3 ---- posting code that pretty much works, but seems to have an issue:
....
FileItemFactory factory = new DiskFileItemFactory();
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload(factory);
// Parse the request
List<FileItem> items = upload.parseRequest(req);
// Process the uploaded items
for(FileItem item : items) {
if( ! item.isFormField()) {
String fieldName = item.getFieldName();
String fileName = item.getName();
String itemContentType = item.getContentType();
boolean isInMemory = item.isInMemory();
long sizeInBytes = item.getSize();
// POST the file to the cdn uploader
postDataRequestToUrl("<the host im uploading too>", "uploadedfile", fileName, item.get());
} else {
throw new RuntimeException("Not expecting any form fields");
}
}
....
// Post a request to specified URL. Get response as a string.
public static void postDataRequestToUrl(String url, String paramName, String fileName, byte[] requestFileData) throws IOException {
URLConnection connection=null;
try{
String boundary = Long.toHexString(System.currentTimeMillis()); // Just generate some unique random value.
String charset = "utf-8";
connection = new URL(url).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
PrintWriter writer = null;
OutputStream output = null;
try {
output = connection.getOutputStream();
writer = new PrintWriter(new OutputStreamWriter(output, charset), true); // true = autoFlush, important!
// Send binary file.
writer.println("--" + boundary);
writer.println("Content-Disposition: form-data; name=\""+paramName+"\"; filename=\"" + fileName + "\"");
writer.println("Content-Type: " + URLConnection.guessContentTypeFromName(fileName));
writer.println("Content-Transfer-Encoding: binary");
writer.println();
output.write(requestFileData, 0, requestFileData.length);
output.flush(); // Important! Output cannot be closed. Close of writer will close output as well.
writer.println(); // Important! Indicates end of binary boundary.
// End of multipart/form-data.
writer.println("--" + boundary + "--");
} finally {
if (writer != null) writer.close();
if (output != null) output.close();
}
//* screw the response
int status = ((HttpURLConnection) connection).getResponseCode();
logger.info("Status: "+status);
for (Map.Entry<String, List<String>> header : connection.getHeaderFields().entrySet()) {
logger.info(header.getKey() + "=" + header.getValue());
}
} catch(Throwable e) {
logger.info("Problem",e);
}
}
I can see this code uploading the file, but only after I shutdown the tomcat. This leads me to believe that I'm leaving some sort of connection open.
This worked!
The core API you'd like to use is java.net.URLConnection. This is however pretty low level and verbose. You'd like to learn about the HTTP specifics in detail and take them into account (headers, etcetera). You can find here a related question with lot of examples.
A more convenient HTTP client API is the Apache Commons HttpComponents Client. You can find an example here.
Update: as per your update: you should read the response as a character stream, not as a binary stream and attempt to cast a byte to a char. This ain't going to work. Head to the Gathering HTTP response information part in the linked question with examples. Here's how it should look like:
BufferedReader reader = null;
StringBuilder builder = new StringBuilder();
try {
reader = new BufferedReader(new InputStreamReader(response, charset));
for (String line; (line = reader.readLine()) != null;) {
builder.append(line);
}
} finally {
if (reader != null) try { reader.close(); } catch (IOException logOrIgnore) {}
}
return builder.toString();
Update 2: as per your second update. Seeing the way how you continue to attampt reading/writing streams, I think it's high time to learn the basic Java IO :) Well, this part is also answered in the linked question. You would like to use Apache Commons FileUpload to parse a multipart/form-data request in a servlet. How to use it is also described/linked in the linked question. Look at the bottom of the Uploading files chapter. By the way, the content length header would return zero since you are not explicitly setting it (and also cannot do without buffering the entire request in memory).
Update 3:
I can see this code uploading the file, but only after I shutdown the tomcat. This leads me to believe that I'm leaving some sort of connection open.
You need to close the OutputStream with which you wrote the file to the disk. Once again, read the above linked basic Java IO tutorial.
What have you tried? If you google for Http Post Java, dozens of pages appear - what's wrong with them? This one, http://www.devx.com/Java/Article/17679/1954 for example, appears pretty decent.