We Keep Coding

File Upload using HTTP protocol through java socket

I am trying to understand how HTTP protocol works, So I tried to add headers manually to java Socket to send a request to httpbin.org as shown below:
BufferedWriter wr = new BufferedWriter(/*socket Outputstream*/)
wr.write("POST post HTTP/1.1\r\n");
wr.write("Host: httpbin.org\r\n");
wr.write("Accept: */*\r\n");
wr.write("Content-Length: "+data.length()+"\r\n");
wr.write("Content-Type: multipart/form-data; boundary=---WebKitFormBoundary67\r\n");
wr.write("\r\n");
wr.write(data);
wr.flush();
In above code data is the payload of HTTP request that looks exactly as below:
---WebKitFormBoundary67
Content-Disposition: form-data; name="field1"
value1
---WebKitFormBoundary67
Content-Disposition: form-data; name="field2"; filename="example.txt"
Java is better when it run long
---WebKitFormBoundary67--
But the server httpbin.org is not identifying any files attached, am I missing anything?

multipart/form-data is a multipart MIME message as defined in RFC 2046. The basic structure of a multipart MIME message in an example of a multipart/form-data message looks like this:
Content-type: multipart/form-data; boundary=foo
--foo
Content-Disposition: form-data; name=key1
abcde
--foo
Content-Disposition: form-data; name=key2; filename=foo.txt
01234
--foo--
As you can see, the boundary foo is defined in the boundary attribute, is used as delimiter between the parts with --foo and is used as the final boundary as --foo--.
Your code instead defines the boundary not as foo but as --foo and then tries to still use only --foo as a separator between the parts. To correct your code you would either need to set the boundary to only -WebKitFormBoundary67 instead of ---WebKitFormBoundary67, or use -----WebKitFormBoundary67 as separator instead of only ---WebKitFormBoundary67.

How to parse http header to get uploaded file and save it to disk

I am developing a http web server in java using socket which gets post header InputStream and then I processed the header with some String split by the header 'boundary' and '\r\n' and got all Headers, Cookies in HashMap(s) and got the contents of the file in a String and saved that String to a file on the server. It works fine when I upload text file or java source file to the server but in case of doc, pdf and image it shows corrupted file and corrupted image.
PrintWriter out;
try {
out = new PrintWriter(new OutputStreamWriter(
new FileOutputStream(UploadPath + "\\" + FileName)));
out.print(FileData);
out.close();
} catch (Exception e) {
}
Above code will save contents of 'FileData' at 'UploadPath' with 'FileName'.
In case of jpg or doc file String FileData is having binary contents of the uploaded file which saved by the above code and also I checked both files for their size in bytes and both were having equal size in byte and I also matched contents of the actual file and content FileData String by debugging the application.
I also checked actual uploaded image file and the FileData String and both matches byte by byte but the image uploaded is totally corrupted.
After searching on internet for this complete day I am not able to find the solution for this. Please help.
I do not want to use apache commons which was suggested on most of the pages.
If you want to see more codes then I will post them.

As you are dealing with binary data, you should use byte and OutputStream instead of String and Writer: If you put some bytes in a string, they are decoded
So if you have found the boundaries of the binary data in your request (represented by a byte array), copy the content byte-wise directly to an output stream.
This only works, if your request is already completely in memory. Regarding file upload, this is not always possible, because you can run out of memory, if you have large files.
So the best way to implement a file upload is to read only the next byte from the stream: This is the difference between splitting and parsing. Actually you need a real parser for multipart form data. Now things get complex, and this is the reason why everybody uses commons-fileupload: It's not that easy to detect the boundaries, if your "look ahead" is just some bytes.
I had to implement a clean-room implementation for legal reasons. If that is not your situation, look in the the source of commons-fileupload. And have a look at the RFC

Since you use Java 7, this is quite easy: use Files.copy().
Also, DO NOT store file contents as Strings, those will only ever be valid for text files. Use classical InputStream/OutputStreams to read/write.

You could read it using an array of bytes like the following
InputStream is = ...
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int nRead;
byte[] data = new byte[16384];
while ((nRead = is.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
}
buffer.flush();
return buffer.toByteArray();

I solved my problem like this,
while (inputRequest.available()>0) {
try {
int t = inputRequest.read();
ch = (char) t;
//here i checked each byte data
} catch (IOException e) {
}
}
Problem was that the input stream was having http header fields along with the file content located anywhere in the stream, so I firstly stored the bytes in a temp String until i get '\r' and '\n' in the stream. In this way I got the boundary for multipart/form-data HTTP header and then I compared the temp String until I found the boundary and other known header contents and then I sent the input-stream to file output-stream. But in some cases header may contain other contents after file content so and definitely it will have a ending boundary so I was continuously keeping track of each byte that I have read and then I sent each byte individually to the file output-stream. Here is the sample http header-
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://localhost/index.html
Connection: keep-alive
Content-Type: multipart/form-data; boundary=---------------------------274761981030199
Content-Length: 1405
-----------------------------274761981030199
Content-Disposition: form-data; name="name1"
pppppp
-----------------------------274761981030199
Content-Disposition: form-data; name="name2"
rrrrrrrrr
-----------------------------274761981030199
Content-Disposition: form-data; name="name3"
eeeeeeee
-----------------------------274761981030199
Content-Disposition: form-data; name="name4"
2
-----------------------------274761981030199
Content-Disposition: form-data; name="name5"; filename="CgiPost.java"
Content-Type: text/x-java-source
import java.io.*;
// This appears in Core Web Programming from
// Prentice Hall Publishers, and may be freely used
// or adapted. 1997 Marty Hall, hall#apl.jhu.edu.
public class CgiPost extends CgiGet
{
public static void main(String[] args)
{
try
{
DataInputStream in
= new DataInputStream(System.in);
String[] data = { in.readLine() };
CgiPost app = new CgiPost("CgiPost", data, "POST");
app.printFile();
} catch(IOException ioe) {
System.out.println
("IOException reading POST data: " + ioe);
}
}
public CgiPost(String name, String[] args,
String type) {
super(name, args, type);
}
}
-----------------------------274761981030199
Content-Disposition: form-data; name="name6"
pppppppppp
-----------------------------274761981030199--
NOTE: In some cases there are chances that your application code reaches to inputRequest.available() but the browser haven't sent the request yet, in this case inputRequest.available() will always return 0 and your while loop will exit immediately. To avoid this first read one byte using inputRequest.read() and then execute code because you can guess the first byte from others in case of http header.
If you are using some count int then use long instead of int, because stream stops in some cases where int variable reaches its limit.
Try to transfer the int value returned from int t = inputRequest.read() to fileoutputstream.write(t).
inputRequest.available() keeps decreasing as you are reading byte form inputstream, it returns number of bytes available in the stream.
In this way you can upload files of large size without any corruption in it.
Leave your comment if anyone needs more details about this.

file size increased during upload

I am uploading file using httpclient. After uploading file size get changed. During file upload some extra things get added in to file.
Before uploading file it contains:
hi this is vipin check
After uploading the file contains:
--j9q7PmvnWSP9wKHHp2w_KCI4Q2jCniJvPbrE0
Content-Disposition: form-data; name="vipin.txt"; filename="vipin.txt"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
hi this is vipin check
--j9q7PmvnWSP9wKHHp2w_KCI4Q2jCniJvPbrE0--
Why file size is changing?
Why does this extra contents get added?
My httpclient code is:
HttpPut httppost = new HttpPut(URIUtil.encodeQuery(newUrl));
httppost.setHeader("X-Auth-Token", cred.getAuthToken());
httppost.addHeader("User-Agent", "NetMagic-file-upload");
System.out.println("Dest : " + dest.getAbsolutePath());
MultipartEntity mpEntity = new MultipartEntity();
ContentBody cbFile = (ContentBody) new FileBody(src);
mpEntity.addPart(dest.getName(), cbFile);
httppost.setEntity(mpEntity);
System.out.println("executing request " + httppost.getRequestLine());
HttpResponse response = httpclient.execute(httppost);

You're doing a PUT request, yet your client uses multipart encoding as commonly uses in HTML form posts.

What appears to be happening is that the client is sending the file to be uploaded as multipart entity, but the server is treating it as an plain file. It is not entirely clear where the fault lies.
It is possible that the server is ignoring the content type in the request header. That would most likely be a bug in the servlet (or whatever) that is responsible for handing the upload request.
It is possible that the client is not setting a content type in the request header. I'd have expected that the client library would take care of that for you. But it is possible that you need to do it explicitly.
I'd advise looking at the request headers as they are sent by the client or received by the server to see if there is a proper multi-part content-type. That will help you determine where the problem is.
But there is an obvious solution. If the server cannot cope with multiparts, change the client side to not send them.

Servlet handling file-upload, Why bigger than the original?

Servlet doPost handing file-uploads,
InputStream in = req.getInputStream();
File file = new File("c:/8.dat");
OutputStream out = new FileOutputStream(file);
byte[] buffer = new byte[1024];
int len =0;
while((len=in.read(buffer))!=-1){
out.write(buffer, 0, len);
}
bao.close();
out.close();
in.close();
Dose Request's getInputStream Method take the http header information?
Why is the uploaded file bigger than the original?

Sending files in a HTTP request is usually done using multipart/form-data encoding. This enables the server to distinguish multiple form data parts in a single request (it would otherwise not be possible to send multiple files and/or input fields along in a single request). Each part is separated by a boundary and preceeded by form data headers. The entire request body roughly look like this (taking an example form with 3 plain <input type="text"> fields with names name1, name2 and name3 which have the values value1, value2 and value3 filled):
--SOME_BOUNDARY
content-disposition: form-data;name="name1"
content-type: text/plain;charset=UTF-8
value1
--SOME_BOUNDARY
content-disposition: form-data;name="name2"
content-type: text/plain;charset=UTF-8
value2
--SOME_BOUNDARY
content-disposition: form-data;name="name3"
content-type: text/plain;charset=UTF-8
value3
--SOME_BOUNDARY--
With a single <input type="file"> field with the name file1 the entire request body look like this:
--SOME_BOUNDARY
content-disposition: form-data;name="file1";filename="some.ext"
content-type: application/octet-stream
binary file content here
--SOME_BOUNDARY--
That's thus basically what you're reading by request.getInputStream(). You should be parsing the binary file content out of the request body. It's exactly that boundary and the form data header which makes your uploaded file to seem bigger (and actually also corrupted). If you're on servlet 3.0, you should have used request.getPart() instead to get the sole file content.
InputStream content = request.getPart("file1").getInputStream();
// ...
If you're still on servlet 2.5 or older, then you can use among others Apache Commons FileUpload to parse it.
See also:
How to upload files to server using JSP/Servlet?

REST - HTTP Post Multipart with JSON

I need to receive an HTTP Post Multipart which contains only 2 parameters:
A JSON string
A binary file
Which is the correct way to set the body?
I'm going to test the HTTP call using Chrome REST console, so I'm wondering if the correct solution is to set a "label" key for the JSON parameter and the binary file.
On the server side I'm using Resteasy 2.x, and I'm going to read the Multipart body like this:
#POST
#Consumes("multipart/form-data")
public String postWithPhoto(MultipartFormDataInput multiPart) {
Map <String, List<InputPart>> params = multiPart.getFormDataMap();
String myJson = params.get("myJsonName").get(0).getBodyAsString();
InputPart imagePart = params.get("photo").get(0);
//do whatever I need to do with my json and my photo
}
Is this the way to go?
Is it correct to retrieve my JSON string using the key "myJsonName" that identify that particular content-disposition?
Are there any other way to receive these 2 content in one HTTP multipart request?

If I understand you correctly, you want to compose a multipart request manually from an HTTP/REST console. The multipart format is simple; a brief introduction can be found in the HTML 4.01 spec. You need to come up with a boundary, which is a string not found in the content, let’s say HereGoes. You set request header Content-Type: multipart/form-data; boundary=HereGoes. Then this should be a valid request body:
--HereGoes
Content-Disposition: form-data; name="myJsonString"
Content-Type: application/json
{"foo": "bar"}
--HereGoes
Content-Disposition: form-data; name="photo"
Content-Type: image/jpeg
Content-Transfer-Encoding: base64
<...JPEG content in base64...>
--HereGoes--