We Keep Coding

POST file (as binary stream) using java.net.HttpURLConnection as a param file=<file binary stream>

I am trying to upload (POST) a file to an endpoint using java.net.HttpURLConnection but I keep getting http code 400 (bad request).
I refered to Send File And Parameters To Server With HttpURLConnection in android API 23
but problem is that I need to send this file as request body param (file=).
Note: The files will be of small size only (4-5mb) so I am reading it entirely in memory.
Corresponding curl request is:
curl -X POST "API" -H "Content-Type: multipart/form-data" -F "file="
Excerpts of Code that I am using:
Proxy webproxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("
<proxy host>", <proxy_port>));
HttpURLConnection http_conn = (HttpURLConnection)
url.openConnection(webproxy);
String authorization = getAuthorization(access_token);
http_conn.setRequestMethod("POST");
http_conn.setRequestProperty("Accept-Charset", "UTF-8");
http_conn.setRequestProperty("Authorization", authorization);
http_conn.setRequestProperty("Connection", "Keep-Alive");
http_conn.setRequestProperty("Content-Type", "multipart/form-data);
http_conn.setDoOutput(true);
http_conn.setDoInput(true);
DataOutputStream outputStream;
outputStream = new DataOutputStream(http_conn.getOutputStream());
File file_obj = new File(this.file);
byte[] allBytes = new byte[(int) file_obj.length()];
FileInputStream fileInputStream = new FileInputStream(file_obj);
outputStream.write("file=".getBytes("UTF-8")); <---Trying to add file param here
fileInputStream.read(allBytes);
outputStream.write(allBytes);
Post that I just read response using below piece of code (works fine for different GET requests):
InputStream inputStream = http_conn.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new
InputStreamReader(inputStream));
String line = "";
while ((line = bufferedReader.readLine()) != null) {
data = data + line;
}
Note: I use java rarely an am not very familiar with it so please be descriptive in your response.

When looking at your curl command line, it shows that the file needs to be send as a multipart/form-data request. This is actually a complex way of formatting your data when it is requires.
An example of the format you need to send is:
Headers:
Content-Type: multipart/form-data; boundary=AaB03x
Body:
--AaB03x
Content-Disposition: form-data; name="files"; filename="file1.txt"
Content-Type: text/plain
... contents of file1.txt ...
--AaB03x--
At the moment, your code is sending the file as a POST/GET formatted request, and this doesn't work as the backend isn't expecting that.
To solve this problem, we need to format the source files into the format required by the backend, and once you know that the "boundary" header option is just a randomly generated value, it becomes more easy to send the request.
String boundary = "MY_AWESOME_BOUNDARY"
http_conn.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
try(DataOutputStream outputStream = new DataOutputStream(http_conn.getOutputStream())) {
File file_obj = new File(this.file);
// Write form-data header
outputStream.write(("--" + boundary + "\r\n").getBytes("UTF-8"));
outputStream.write(("Content-Disposition: form-data; name=\"file\"; filename=\"file1.txt\"\r\n").getBytes("UTF-8"));
outputStream.write(("Content-Type: text/plain\r\n").getBytes("UTF-8"));
outputStream.write(("\r\n").getBytes("UTF-8"));
// Write form-data body
Files.copy(file_obj.toPath(), outputStream)
// Write form-data "end"
outputStream.write(("--" + boundary + "--\r\n").getBytes("UTF-8"));
}
// Read backend response here
try(InputStream inputStream = http_conn.getInputStream()) {
BufferedReader bufferedReader = new BufferedReader(new
InputStreamReader(inputStream));
StringBuilder lines = new StringBuilder(); // StringBuilder is faster for concatination than appending strings
while ((line = bufferedReader.readLine()) != null) {
lines.append(line);
}
System.out.println(lines);
}
Note that I used "try-with-resource" blocks, these blocks make sure that any external resources are closed and disposed when you are done using them, generally the open resource limit of the OS is very low, compared to the amount of memory your program has, so what happens is that your program could give weird errors that only happens after some time of running or when the user executes certain actions inside your application

The above didnt worked for me so I switched to different package (okhttp3), here is what worked for me:
File file_obj = new File(this.file);
String authorization = "my authorization string";
Proxy webproxy = new Proxy(Proxy.Type.HTTP, new
InetSocketAddress("proxy", <port>));
OkHttpClient client = new OkHttpClient.Builder().proxy(webproxy).build();
RequestBody requestBody = new MultipartBody.Builder().setType(MultipartBody.FORM).addFormDataPart("file", "filename",
RequestBody.create(MediaType.parse("application/octet-stream"), file_obj)).build();
Request request = new Request.Builder().header("Authorization", authorization).url(this.url).post(requestBody).build();
try (Response response = client.newCall(request).execute()){
if(!response.isSuccessful()) return "NA";
return (response.body().string());
}

replicate wget command in java

i have this command:
wget -O prova.csv --header="prova-user: guest" --header="prova-passwd: guest"
"http://www.....................80&albedo=0.2&horizon=1"
i want to do a batch scheduled in Java but I can not connect. When I try to take the imputstream return me this error:
ERROR message -8: Unregistered IP address
This is my piece of code:
URL myURL = new URL(url);
HttpURLConnection myURLConnection = (HttpURLConnection)myURL.openConnection();
String userCredentials = "guest:guest";
String basicAuth = "Basic " + new String(new Base64().encode(userCredentials.getBytes()));
myURLConnection.setRequestProperty ("Authorization", basicAuth);
myURLConnection.setRequestMethod("POST");
myURLConnection.setRequestProperty("Content-Language", "en-US");
myURLConnection.setUseCaches(false);
myURLConnection.setDoInput(true);
myURLConnection.setDoOutput(true);
// Show page.
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(myURLConnection.getInputStream(), "UTF-8"));
for (String line; ((line = reader.readLine()) != null);) {
System.out.println(line);
}
} finally {
if (reader != null) try { reader.close(); } catch (IOException ignore) {}
}
is it possible? and how can I do it?
Thanks in advance

You had provided 2 completely different commands.
The first is a wget that send in HTTP headers a sort of authentication infos, and GET a result.
The second is a java program that perform an HTTP request in POST with basic authentication.
If the first command is working, than you should forget about the basic authentication and set the proper HTTP headers as you did in the wget command.
I don't know why you try a POST, if the wget looks as a normal GET request.
Just use a GET request in java too.
And it should work.
About the error, I suppose is the server that sent you such error message.
So it could be as you haven't correctly authenticated.
But it is a strange error, I'm expecting such kind of error if the server have a white list of IP addresses allowed to connect.
Are you running the wget and the java code on the same server?

How Do You Upload a File to a Server With PHP using Java [duplicate]

This question already has answers here:
How to send PUT, DELETE HTTP request in HttpURLConnection?
(8 answers)
Closed 9 years ago.
I have a Java applet that needs to upload a file from the user's computer to the server.
I am unable to add other libraries (such as com.apache). Is there a low level way of doing so.
Currently, I have a php file on the server which contains:
//Sets the target path for the upload.
$target_path = "spelling/";
$_FILES = $_POST;
var_dump($_FILES);
move_uploaded_file($_FILES["tmp_name"], $target_path . $_FILES["name"]);
?>
Currently my Java program is sending parameters via POST to this php file.
It sends these parameters by POST using the following code:
try {
//Creates a new URL containing the php file that writes files on the ec2.
url = new URL(WEB_ADDRESS + phpFile);
//Opens this connection and allows for the connection to output.
connection = url.openConnection();
connection.setDoOutput(true);
//Creates a new streamwriter based on the output stream of the connection.
OutputStreamWriter wr = new OutputStreamWriter(connection.getOutputStream());
//Writes the parameters to the php file on the ec2 server.
wr.write(data);
wr.flush();
//Gets the response from the server.
//Creates a buffered input reader from the input stream of the connection.
BufferedReader rd = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line;
//Loops through and reads the response. Loops until reaches null line.
while ((line = rd.readLine()) != null) {
//Prints to console.
System.out.println(line);
}
//Closes reader and writer.
wr.close();
rd.close();
} catch (Exception e) {
e.printStackTrace();
}
This works for POST'ing data but when I try to send a file using this method, nothing happens (no response from the server nor is the file uploaded).
If anyone has any hints I would be grateful :)

Are you using java.net.URLConnection?
You may want to get some help on this page:
http://www.codejava.net/java-se/networking/upload-files-by-sending-multipart-request-programmatically
Here is the main part:
boundary = "===" + System.currentTimeMillis() + "===";
URL url = new URL(requestURL);
httpConn = (HttpURLConnection) url.openConnection();
httpConn.setDoOutput(true); // indicates POST method
httpConn.setDoInput(true);
httpConn.setRequestProperty("Content-Type",
"multipart/form-data; boundary=" + boundary);
outputStream = httpConn.getOutputStream();
writer = new PrintWriter(new OutputStreamWriter(outputStream, charset),
true);
But, you will need to have the php script be on the same server where your applet came from.

Does Java GAE UrlFetch support HTTP PUT?

How do I do an HTTP PUT request using the UrlFetch service for Google App Engine's Java runtime?
The following code in my application will send a POST request:
URL url = ...;
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(MAX_TIMEOUT); // Maximum allowable timeout on GAE of 60 seconds
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setRequestProperty("Content-Type", "application/json; charset=utf-8");
OutputStreamWriter writer = null;
try {
writer = new OutputStreamWriter(conn.getOutputStream(), Charset.forName("utf-8"));
writer.write(jsonContent);
} finally {
if (writer != null)
writer.close();
}
}
InputStream inStream = null;
E response = null; // Unmarshalled using Jackson
try {
inStream = conn.getInputStream();
if (status != 204) { // NO CONTENT => No need to use Jackson to deserialize!
response = mapper.readValue(inStream, typeRef);
}
} finally {
if (inStream != null) {
inStream.close();
}
}
I try doing the same as POST for PUT, but I keep on getting a 405 HTTP error for Method Not Allowed. Thanks for the help.
Reference: https://developers.google.com/appengine/docs/java/urlfetch/overview
Unfortunately, GAE Java doesn't have as good of documentation as Python's version of GAE URL fetch(https://developers.google.com/appengine/docs/python/urlfetch/overview). Anyone from Google wanna say why?

I think your code works. The HTTP 405 you are seeing is coming from the server you're contacting (not all services support PUT). You can try using curl to hit the URL if you'd like confirmation that it's not your code at fault:
curl -X PUT -d "foo=bar" http://your_site.com/path/to/your/resource

Curl Object to URL Generation

I have webservice that I am trying to call:
The following Curl command works for that
curl -F fa=c.apiupload -F sessiontoken=EA3237F922644115A0F7DB75D0AE388F -F destfolderid=52482BD488DB4AD6887C5C7BF47BD6FC -F filedata=#/Users/cpinera/tmp/panda2.jpg -F zip_extract=1 -F metadata=1 -F meta_img_description="This is a very nice panda" -F meta_img_keywords="panda,happy panda" http://domain.com/razuna/raz1/dam/index.cfm
but the equivalent URL that I generates as my HttpURLConnection for accessing RESTFul API does not :
http://domain.com/razuna/raz1/dam/index.cfm?fa=c.apiupload&sessiontoken=F46D2226463C4ADE866819AACD7D2F5E&filedata=C:\JobInterview\BatchUpload\auth.xml&destfolderid=52482BD488DB4AD6887C5C7BF47BD6FC&zip_extract=1
I get this response for the REST Request:
The content was :: <?xml version="1.0" encoding="UTF-8"?><Response><responsecode>1</responsecode><message>Upload failed This was not an uploaded form type</message></Response>
Here is the Java Code
Method that uses queries the URL:
public static String doQuery(String loginUrl) throws IOException{
URL url = new URL(loginUrl);
HttpURLConnection conn =
(HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
if (conn.getResponseCode() != 200) {
throw new IOException(conn.getResponseMessage());
}
InputStream is = conn.getInputStream();
//Buffer the result into a string
BufferedReader rd = new BufferedReader(
new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line;
while ((line = rd.readLine()) != null) {
line = StringEscapeUtils.unescapeHtml4(line);
sb.append(line);
}
rd.close();
conn.disconnect();
System.out.println("The content was :: " + sb.toString());
return sb.toString();
}
Method that should upload:
public static void testUpload(String seesionToken, String file ) throws IOException{
String upload = "http://domain.com:8080/razuna/raz1/dam/index.cfm?fa=c.apiupload&sessiontoken="+seesionToken+"&filedata="+file+"&destfolderid=52482BD488DB4AD6887C5C7BF47BD6FC&zip_extract=1" ;
System.out.println(upload);
Authenticate.doQuery(upload);
}
After adding in doQuery:
conn.setRequestProperty("Content-Type", "multipart/form-data");
And changing RESTFul URL to:
http://domain.com:8080/razuna/raz1/dam/index.cfm?fa=c.apiupload&sessiontoken="+seesionToken+"&filedata="+files.toURI()+"&destfolderid=52482BD488DB4AD6887C5C7BF47BD6FC&zip_extract=1"
It seems to detect that URL is trying to send a File but still cannot upload file:
The content was :: <?xml version="1.0" encoding="UTF-8"?><Response><responsecode>1</responsecode><message>Upload failed There was no appropriate FILE found in the upload</message></Response>

Your curl command is uploading the file #/Users/cpinera/tmp/panda2.jpg. Using the URL by itself will not, it will simply access the page, hence your error that 'this was not an uploaded form'.

When you run curl with -F parameters, it sends an HTTP POST request with the specified name-value pairs passed in the body of the HTTP request. When you access a URL using query string parameters, an HTTP GET is performed. It seems likely that the server handles those two request methods differently.
Update, now that you've posted some Java code:
The values in the query string portion of the URL are used to convey information to the web server that will be handling the request. Of course, this remote server is not going to know what C:/path/to/your/file is, because this is a path to a file on your local machine. The proper way to send this file data is to open the file in Java, and then read it and write it to the HttpUrlConnection's output stream.
For example, see the code snippet under "Posting Content" on the Android documentation's discussion of HttpUrlConnection: http://developer.android.com/reference/java/net/HttpURLConnection.html