We Keep Coding

File Upload using HTTP protocol through java socket

I am trying to understand how HTTP protocol works, So I tried to add headers manually to java Socket to send a request to httpbin.org as shown below:
BufferedWriter wr = new BufferedWriter(/*socket Outputstream*/)
wr.write("POST post HTTP/1.1\r\n");
wr.write("Host: httpbin.org\r\n");
wr.write("Accept: */*\r\n");
wr.write("Content-Length: "+data.length()+"\r\n");
wr.write("Content-Type: multipart/form-data; boundary=---WebKitFormBoundary67\r\n");
wr.write("\r\n");
wr.write(data);
wr.flush();
In above code data is the payload of HTTP request that looks exactly as below:
---WebKitFormBoundary67
Content-Disposition: form-data; name="field1"
value1
---WebKitFormBoundary67
Content-Disposition: form-data; name="field2"; filename="example.txt"
Java is better when it run long
---WebKitFormBoundary67--
But the server httpbin.org is not identifying any files attached, am I missing anything?

multipart/form-data is a multipart MIME message as defined in RFC 2046. The basic structure of a multipart MIME message in an example of a multipart/form-data message looks like this:
Content-type: multipart/form-data; boundary=foo
--foo
Content-Disposition: form-data; name=key1
abcde
--foo
Content-Disposition: form-data; name=key2; filename=foo.txt
01234
--foo--
As you can see, the boundary foo is defined in the boundary attribute, is used as delimiter between the parts with --foo and is used as the final boundary as --foo--.
Your code instead defines the boundary not as foo but as --foo and then tries to still use only --foo as a separator between the parts. To correct your code you would either need to set the boundary to only -WebKitFormBoundary67 instead of ---WebKitFormBoundary67, or use -----WebKitFormBoundary67 as separator instead of only ---WebKitFormBoundary67.

Setting the Content-Transfer-Encoding of upload file in multi-part HTTP request part with Apache HttpClient

I am using apache http library to build and send a multi-part http request that has a file part in the body. Here is a little sample of my request
Request POST HTTPS://hostname:9443/di/resources/upload?logonId=user1 HTTP/1.1:
Headers: Content-Type: multipart/form-data Set-Cookie: Path=/; HttpOnly TrustToken: -1000%2CCaKOjiTFmje3%2Fw0GGcw5%2BDwgxXHjHdQShQgW1QGiHYk%3D
Body: --ncFZGuKp50zCWWImlBFZjxbanSSoJt
Content-Disposition: form-data; name="File 1"; filename="SampleData_en.csv"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
Identifier,title,,,,,,,,,,,,,,
Plan: Entries (1) or ent2 (2) ,2,"Set to 1 to specify plan is for Catalog Entries, set to 2 etc .....
--ncFZGuKp50zCWWImlBFZjxbanSSoJt--
The project is that the service is based on apache wink that has a problem decoding the headers in the body which gives this kind of error
Caused by: java.lang.StringIndexOutOfBoundsException
at java.lang.String.substring(String.java:1240)
at org.apache.wink.common.internal.providers.multipart.MultiPartParser.parseHeaders(MultiPartParser.java:264)
at org.apache.wink.common.internal.providers.multipart.MultiPartParser.nextPart(MultiPartParser.java:109)
at org.apache.wink.common.model.multipart.InMultiPart.hasNext(InMultiPart.java:83)
I believe the fix would be to remove the Content-Transfer-Encoding from the body? or change it to maybe a different encoding, maybe base64.
The only problem is that I dont know how to do this using the apache library and haven't been able to find anything examples. Here is the code I am using apache to create the entity portion of my HttpPost request:
MultipartEntityBuilder reqEntity = MultipartEntityBuilder.create();
File file1 = RequestUtils.getFileBody(filePath);
FileBody fileBodyFile1 = new FileBody(file1, org.apache.http.entity.ContentType.create("application/octet-stream"),
file1 .getName());
reqEntity.addPart("File 1", fileBodyFile1);

Adding content-id and encoding to multipart using apache http client/mime

We are using httpclient & httpmime of version 4.3.6.
We are not able to find a way to add Content-Id, encoding fields to mimepart!
Below is the code snippet in is:
HttpEntity mimeEntity = MultipartEntityBuilder.create()
.setBoundary("_boundary_")
.addTextBody("xml_body", xmlString, ContentType.TEXT_XML)
.addBinaryBody("pdf_file_name", pdf, ContentType.create(" application/pdf"), pdf.getName())
.build();
The objective is to have the below result in the http post envelope:
Content-ID: <attachment_1>
content-transfer-encoding: base64
Content-Disposition: attachment; filename="attachment.pdf"
content-type: application/pdf
What is the default content-transfer-encoding value to be used for images/pdf/.doc files, which the server should accept by default? We are posting to "https" url of Ariba Network. Is encoding of base64 mandatory for posting to Ariba Network server?
In some places i have seen the encoding was 7bit/8bit, which is not supported in this api (dont know the reason & background).
The tutorial did not provide an example on this. Great, if you can provide a code snippet that can be of help or a link!!!
Thanks

Tumblr API Photo Post Returns 401 (Not Authorized)

I'm attempting to use the Tumblr API in an Android app to authorize users and make text and photo posts. I'm using the Scribe library. So for, I can successfully obtain an access token and use it to get user info. I can also make text posts without any issues. This tells me that I'm signing requests correctly.
However, I've spent the last week and a half attempting to make photo posts without success. I continuously receive 401 errors (Not Authorized) I've read through many posts on the Tumblr support forum as well as here on Stack Overflow, but was unable to find a solution.
I'm reluctant to include the Jumblr library because I'm trying to keep my app as lean as possible. That said, I reviewed the Jumblr code and decided to mimic how photo posts are sent (https://github.com/tumblr/jumblr/blob/master/src/main/java/com/tumblr/jumblr/request/MultipartConverter.java). I'm still receiving the exact same error.
Below is an example my multipart POST request and the response I receive. I've replace the blog name, and OAuth signature, consumer key, and token variables, and have removed the binary image data for brevity sake. Everything else is untouched. I have a few questions...
Are there any other variables that should be included in the
multipart section? A Stack Overflow user stated that placing the
"oauth_" signature variables in there fixed his problem. I didn't
have success with this, but maybe there was something I was missing.
The Jumblr app doesn't appear to do any encoding of the image data,
although the Tumblr documentation states that it should be URL
encoded. Right now I'm sending it as the Jumblr app appears to (raw
binary). Is this correct?
Does anything else in my request look
incorrect?
REQUEST:
NOTE: I learned that the OAuth signature should be generated WITHOUT the multipart form. My code takes that into account when building this request!
POST http://api.tumblr.com/v2/blog/**REMOVED**.tumblr.com/post HTTP/1.1
Content-Type: multipart/form-data, boundary=cbe6b79db1b3cbe6b79e104e
Authorization: OAuth oauth_signature="**REMOVED**", oauth_version="1.0", oauth_nonce="3181201716", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="**REMOVED**", oauth_timestamp="1388791537", oauth_token="**REMOVED**"
Content-Length: 1001
User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.3; SM-N900T Build/JSS15J)
Host: api.tumblr.com
Connection: Keep-Alive
Accept-Encoding: gzip
--cbe6b79db1b3cbe6b79e104e
Content-Disposition: form-data; name="type"
photo
--cbe6b79db1b3cbe6b79e104e
Content-Disposition: form-data; name="caption"
Another pic test...
--cbe6b79db1b3cbe6b79e104e
Content-Disposition: form-data; name="data[0]"; filename="postr_media_file_1388791537-1709648435.jpg"
Content-Type: image/jpeg
---- BINARY DATA REMOVED FOR BREVITY ----
RESPONSE:
HTTP/1.1 401 Not Authorized
Server: nginx
Date: Fri, 03 Jan 2014 23:25:39 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Set-Cookie: tmgioct=52c746f34266840643527780; expires=Mon, 01-Jan-2024 23:25:39 GMT; path=/; httponly
P3P: CP="ALL ADM DEV PSAi COM OUR OTRo STP IND ONL"
3c
{"meta":{"status":401,"msg":"Not Authorized"},"response":[]}

I posted the answer in the "Tumblr API Discussion" Google Group. This is what I did:
The key to doing it correctly is NOT just signing without the multipart form!!! Here are the steps...
Add all fields EXCEPT the data field as regular url encoded POST
body variables
Sign the request
Remove ALL off the post variables you just added from the request
Add the multipart form, including the data field this time
Some things to consider...
The Content-Type in the header should be "multipart/form-data"
The Content-Disposition of all form parts should be "form-data" and, of course, include a valid "name" attribute (ie. type, caption, etc...)
The Content-Disposition of the data part should also include a "filename" attribute
The only form part that should contain a Content-Type is data, and it should be set to the mime type of the file you are uploading (ie. "image/jpeg")
I used "data[0]" as the name of the data field. I haven't tested this with just "data", but according to everything I've read it should work that way as well. If you are creating a photo set, I believe you simple add additional parts (ie. data1. data[2], etc...). Again, I haven't tested anything except "data[0]", so do your due diligence!!!
I did NOT encode the binary image data!!! I saw people spending considerable amount of time on this in other posts when adding the image as a POST body variable. If doing this as a multipart form, you can skip the encoding and send raw binary data! ;-)
I hope this helps someone! I've spent two weeks banging my head on random solid objects trying to figure this out. The implementation is very easy to do, but there is zero documentation available on how exactly to build POST requests for photos properly. The official docs really should include that. If I had know what I just posted above I could have completed this in minutes instead of weeks!!!
The last request I posted earlier is still valid, but here it is again. Just remember what I mentioned about the signature!!!
REQUEST:
POST http://api.tumblr.com/v2/blog/REMOVED.tumblr.com/post HTTP/1.1
Content-Type: multipart/form-data, boundary=c60f7c041c02c60f7c046e9b
Authorization: OAuth oauth_signature="***REMOVED***", oauth_version="1.0", oauth_nonce="315351812", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="***REMOVED***", oauth_timestamp="1388785116", oauth_token="***REMOVED***"
Content-Length: 1001
User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.3; SM-N900T Build/JSS15J)
Host: api.tumblr.com
Connection: Keep-Alive
Accept-Encoding: gzip
--c60f7c041c02c60f7c046e9b
Content-Disposition: form-data; name="type"
photo
--c60f7c041c02c60f7c046e9b
Content-Disposition: form-data; name="caption"
Another pic test...
--c60f7c041c02c60f7c046e9b
Content-Disposition: form-data; name="data[0]"; filename="postr_media_file_1388785116-1709648435.jpg"
Content-Type: image/jpeg
***** BINARY DATA REMOVED FOR BREVITY *****
--c60f7c041c02c60f7c046e9b--

file size increased during upload

I am uploading file using httpclient. After uploading file size get changed. During file upload some extra things get added in to file.
Before uploading file it contains:
hi this is vipin check
After uploading the file contains:
--j9q7PmvnWSP9wKHHp2w_KCI4Q2jCniJvPbrE0
Content-Disposition: form-data; name="vipin.txt"; filename="vipin.txt"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
hi this is vipin check
--j9q7PmvnWSP9wKHHp2w_KCI4Q2jCniJvPbrE0--
Why file size is changing?
Why does this extra contents get added?
My httpclient code is:
HttpPut httppost = new HttpPut(URIUtil.encodeQuery(newUrl));
httppost.setHeader("X-Auth-Token", cred.getAuthToken());
httppost.addHeader("User-Agent", "NetMagic-file-upload");
System.out.println("Dest : " + dest.getAbsolutePath());
MultipartEntity mpEntity = new MultipartEntity();
ContentBody cbFile = (ContentBody) new FileBody(src);
mpEntity.addPart(dest.getName(), cbFile);
httppost.setEntity(mpEntity);
System.out.println("executing request " + httppost.getRequestLine());
HttpResponse response = httpclient.execute(httppost);

You're doing a PUT request, yet your client uses multipart encoding as commonly uses in HTML form posts.

What appears to be happening is that the client is sending the file to be uploaded as multipart entity, but the server is treating it as an plain file. It is not entirely clear where the fault lies.
It is possible that the server is ignoring the content type in the request header. That would most likely be a bug in the servlet (or whatever) that is responsible for handing the upload request.
It is possible that the client is not setting a content type in the request header. I'd have expected that the client library would take care of that for you. But it is possible that you need to do it explicitly.
I'd advise looking at the request headers as they are sent by the client or received by the server to see if there is a proper multi-part content-type. That will help you determine where the problem is.
But there is an obvious solution. If the server cannot cope with multiparts, change the client side to not send them.