We Keep Coding

Why isn't my node.js restful api recieving this output stream from Java Android?

I have a class audio sender which makes a connection to the nodejs server and uploads an audio file in POST method mode.
public class AudioSender implements Callable<JSONObject> {
String outputFile;
AudioSender(String fileLocation){
outputFile=fileLocation;
}
public JSONObject call(){
URL url = null;
HttpURLConnection urlConnection = null;
InputStream audioInputStream=null;
JSONObject response=null;
byte buffer[]=new byte[16];
try {
url = new URL("http://192.168.0.106:3000/upload");
} catch (MalformedURLException e) {
e.printStackTrace();
} finally {
try {
urlConnection = (HttpURLConnection) url.openConnection();
} catch (IOException e) {
e.printStackTrace();
} finally {
urlConnection.setDoOutput(true);
urlConnection.setChunkedStreamingMode(16);
urlConnection.setRequestProperty("Content-Type","multipart/form-data");
urlConnection.setRequestProperty("Connection", "Keep-Alive");
urlConnection.setRequestProperty("Cache-Control", "no-cache");
urlConnection.setRequestProperty(
"Content-Type", "multipart/form-data;boundary=*****");
try {
OutputStream out = new BufferedOutputStream(urlConnection.getOutputStream());
try {
audioInputStream = new BufferedInputStream(new FileInputStream(outputFile));
Log.d("hello","audioinputstream");
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
while(audioInputStream.read(buffer)!=-1) {
out.write(buffer);
Log.d("buffer",buffer.toString());
}
try {
audioInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
StringBuilder total = new StringBuilder();
while(in.read(buffer)!=-1)
total.append(buffer);
Log.d("response",total.toString());
try {
response = new JSONObject(total.toString());
}catch(JSONException e){
Log.e("Response Parse Error", "Could not parse malformed JSON");
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
return response;
}
This is the upload that executes the AudioSender callable.
public void upload() {
JSONObject response=null;
AudioSender sender=new AudioSender(outputFile);
FutureTask<JSONObject> uploadTask=new FutureTask<JSONObject>(sender);
ExecutorService executorService= Executors.newFixedThreadPool(1);
executorService.execute(uploadTask);
Log.d("s","was here");
while(true){
if(uploadTask.isDone()){
try{
response=uploadTask.get();
}catch(InterruptedException|ExecutionException e){
e.printStackTrace();
Log.e("ee","ee",e.getCause());
}
}
}
}
I pretty much know this isn't node js's fault but here's the server code:
app.post('/upload',function(req,res){
console.log("someone called!");
req.on('data',function(chunk){
console.log('res');
console.log(chunk);
});
req.on('end',function(){
console.log("done!");
res.send({'name':'sg'});
});});
When I call upload(), the server console prints
someone called!
done!
I was debugging and found that indeed I am receiving the responded json object from the server. And I don't know if out.write(buffer) is doing the job, but debugging it shows that the buffer value is changing and is in par with my audio file's size.
Please do not suggest using ION or anything else.

I solved the problems by setting up the URLConnection as follows:
boundary = "===" + System.currentTimeMillis() + "===";
urlConnection.setUseCaches(false);
urlConnection.setDoOutput(true); // indicates POST method
urlConnection.setDoInput(true);
urlConnection.setRequestProperty("Content-Type","multipart/form-data; boundary=" + boundary);

How to send a file using PUT method to a web service

I am trying to upload a file using PUT method. I have been advised before to use POST and do a multipart upload but my web service only allows GET, PUT, PATCH, DELETE, HEAD and OPTIONS. And the main reason for that is that I am updating the JSON array that contains id, target, source, file. Now I just have to update the file part in the web service i.e. I have to upload the file there.
Here is my code:
URLConnection urlconnection = null;
try{
File file = new File("/Users/pathtofile/file.pdb");
URL url = new URL("http://example.website/api/jobs/5/");
urlconnection = url.openConnection();
urlconnection.setDoOutput(true);
urlconnection.setDoInput(true);
if (urlconnection instanceof HttpURLConnection) {
try {
((HttpURLConnection)urlconnection).setRequestMethod("PUT");
((HttpURLConnection)urlconnection).setRequestProperty("Content-Type",
"application/json");
((HttpURLConnection)urlconnection).connect();
} catch (ProtocolException e) {
// TODO Auto-generated catch block
System.out.println("ProtocolException");
e.printStackTrace();
}
}
BufferedOutputStream bos = new BufferedOutputStream(urlconnection
.getOutputStream());
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(
file));
int i;
// read byte by byte until end of stream
while ((i = bis.read()) >= 0) {
bos.write(i);
}
bos.close();
// Getting error here (Bad request).
System.out.println(((HttpURLConnection)urlconnection).getResponseMessage());
} catch(Exception e) {
System.out.println("Exception");
e.printStackTrace();
}
try {
InputStream inputStream;
int responseCode = ((HttpURLConnection)urlconnection).getResponseCode();
if ((responseCode >= 200) && (responseCode <= 202)) {
inputStream = ((HttpURLConnection)urlconnection).getInputStream();
int j;
while ((j = inputStream.read()) >= 0) {
System.out.println(j);
}
} else {
System.out.println(responseCode);
inputStream = ((HttpURLConnection)urlconnection).getErrorStream();
System.out.println(inputStream);
}
((HttpURLConnection)urlconnection).disconnect();
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println("IOException");
e.printStackTrace();
}
And I get the following error:
Bad Request
400
Is there a way to send a file using PUT method? The file is going to be in a place lıke this:
{"target":["This field is required."],
"project":["This field is required."],
"option":["This field is required."],
"result_file":["No file was submitted."],
"com_status":["This field is required."]}
The above is output from curl PUT method that I tried without giving any parameters. And it also shows that the parameters are required in a JSON format.

Java, FileNotfound Exception, While reading conn.getInputStream()

Please tell me some one, How to resolve this problem,
Sometime I am getting Filenotfound Exception and Some time this code working fine.
Below is my code,
public String sendSMS(String data, String url1) {
URL url;
String status = "Somthing wrong ";
try {
url = new URL(url1);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
conn.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
conn.setRequestProperty("Accept","*/*");
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
// Get the response
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String s;
while ((s = rd.readLine()) != null) {
status = s;
}
rd.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
wr.close();
} catch (MalformedURLException e) {
status = "MalformedURLException Exception in sendSMS";
e.printStackTrace();
} catch (IOException e) {
status = "IO Exception in sendSMS";
e.printStackTrace();
}
return status;
}

Rewrite like this and let me know how you go... (note closing of reading and writing streams, also the cleanup of streams if an exception is thrown).
public String sendSMS(String data, String url1) {
URL url;
OutputStreamWriter wr = null;
BufferedReader rd = null;
String status = "Somthing wrong ";
try {
url = new URL(url1);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
conn.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
conn.setRequestProperty("Accept","*/*");
wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
wr.close();
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String s;
while ((s = rd.readLine()) != null) {
status = s;
}
rd.close();
} catch (Exception e) {
if (wr != null) try { wr.close(); } catch (Exception x) {/*cleanup*/}
if (rd != null) try { rd.close(); } catch (Exception x) {/*cleanup*/}
e.printStackTrace();
}
return status;
}

This issue seems to be known, but for different reasons so its not clear why this happend.
Some threads would recommend closing the OutputStreamWriter as flushing it is not enough, therefor i would try to clos it directly after fushing as you are not using it in the code between the flush and close.
Other threads show that using a different connections like HttpURLConnection are avoiding this problem from occuring (Take a look here)
Another article suggests to use the URLEncoder class’ static method encode. This method takes a string and encodes it to a string that is ok to put in a URL.
Some similar questions:
URL is accessable with browser but still FileNotFoundException with URLConnection
URLConnection FileNotFoundException for non-standard HTTP port sources
URLConnection throwing FileNotFoundException
Wish you good luck.

It returns FileNotFoundException when the server response to HTTP request is code 404.
Check your URL.

Android Java UTF-8 HttpClient Problem

I am having weird character encoding issues with a JSON array that is grabbed from a web page. The server is sending back this header:
Content-Type text/javascript; charset=UTF-8
Also I can look at the JSON output in Firefox or any browser and Unicode characters display properly. The response will sometimes contain words from another language with accent symbols and such. However I am getting those weird question marks when I pull it down and put it to a string in Java. Here is my code:
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, "utf-8");
params.setBooleanParameter("http.protocol.expect-continue", false);
HttpClient httpclient = new DefaultHttpClient(params);
HttpGet httpget = new HttpGet("http://www.example.com/json_array.php");
HttpResponse response;
try {
response = httpclient.execute(httpget);
if(response.getStatusLine().getStatusCode() == 200){
// Connection was established. Get the content.
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String jsonText = convertStreamToString(instream);
Toast.makeText(getApplicationContext(), "Response: "+jsonText, Toast.LENGTH_LONG).show();
}
}
} catch (MalformedURLException e) {
Toast.makeText(getApplicationContext(), "ERROR: Malformed URL - "+e.getMessage(), Toast.LENGTH_LONG).show();
e.printStackTrace();
} catch (IOException e) {
Toast.makeText(getApplicationContext(), "ERROR: IO Exception - "+e.getMessage(), Toast.LENGTH_LONG).show();
e.printStackTrace();
} catch (JSONException e) {
Toast.makeText(getApplicationContext(), "ERROR: JSON - "+e.getMessage(), Toast.LENGTH_LONG).show();
e.printStackTrace();
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader;
try {
reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
} catch (UnsupportedEncodingException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
StringBuilder sb = new StringBuilder();
String line;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
As you can see, I am specifying UTF-8 on the InputStreamReader but every time I view the returned JSON text via Toast it has strange question marks. I am thinking that I need to send the InputStream to a byte[] instead?
Thanks in advance for any help.

Try this:
if (entity != null) {
// A Simple JSON Response Read
// InputStream instream = entity.getContent();
// String jsonText = convertStreamToString(instream);
String jsonText = EntityUtils.toString(entity, HTTP.UTF_8);
// ... toast code here
}

#Arhimed's answer is the solution. But I cannot see anything obviously wrong with your convertStreamToString code.
My guesses are:
The server is putting a UTF Byte Order Mark (BOM) at the start of the stream. The standard Java UTF-8 character decoder does not remove the BOM, so the chances are that it would end up in the resulting String. (However, the code for EntityUtils doesn't seem to do anything with BOMs either.)
Your convertStreamToString is reading the character stream a line at a time, and reassembling it using a hard-wired '\n' as the end-of-line marker. If you are going to write that to an external file or application, you should probably should be using a platform specific end-of-line marker.

It is just that your convertStreamToString is not honoring encoding set in the HttpRespnose. If you look inside EntityUtils.toString(entity, HTTP.UTF_8), you will see that EntityUtils find out if there is encoding set in the HttpResponse first, then if there is, EntityUtils use that encoding. It will only fall back to the encoding passed in the parameter(in this case HTTP.UTF_8) if there isn't encoding set in the entity.
So you can say that your HTTP.UTF_8 is passed in the parameter but it never get used because it is the wrong encoding. So here is update to your code with the helper method from EntityUtils.
HttpEntity entity = response.getEntity();
String charset = getContentCharSet(entity);
InputStream instream = entity.getContent();
String jsonText = convertStreamToString(instream,charset);
private static String getContentCharSet(final HttpEntity entity) throws ParseException {
if (entity == null) {
throw new IllegalArgumentException("HTTP entity may not be null");
}
String charset = null;
if (entity.getContentType() != null) {
HeaderElement values[] = entity.getContentType().getElements();
if (values.length > 0) {
NameValuePair param = values[0].getParameterByName("charset");
if (param != null) {
charset = param.getValue();
}
}
}
return TextUtils.isEmpty(charset) ? HTTP.UTF_8 : charset;
}
private static String convertStreamToString(InputStream is, String encoding) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*/
BufferedReader reader;
try {
reader = new BufferedReader(new InputStreamReader(is, encoding));
} catch (UnsupportedEncodingException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
StringBuilder sb = new StringBuilder();
String line;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

Archimed's answer is correct. However, that can be done simply by providing an additional header in the HTTP request:
Accept-charset: utf-8
No need to remove anything or use any other library.
For example,
GET / HTTP/1.1
Host: www.website.com
Connection: close
Accept: text/html
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.10 Safari/537.36
DNT: 1
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: utf-8
Most probably your request doesn't have any Accept-Charset header.

Extract the charset from the response content type field. You can use the following method to do this:
private static String extractCharsetFromContentType(String contentType) {
if (TextUtils.isEmpty(contentType)) return null;
Pattern p = Pattern.compile(".*charset=([^\\s^;^,]+)");
Matcher m = p.matcher(contentType);
if (m.find()) {
try {
return m.group(1);
} catch (Exception e) {
return null;
}
}
return null;
}
Then use the extracted charset to create the InputStreamReader:
String charsetName = extractCharsetFromContentType(connection.getContentType());
InputStreamReader inReader = (TextUtils.isEmpty(charsetName) ? new InputStreamReader(inputStream) :
new InputStreamReader(inputStream, charsetName));
BufferedReader reader = new BufferedReader(inReader);

java.io.FileNotFoundException when retrieving a url with umlauts in the filename

I am trying to retrieve a url with an umlaut in the filename, something like "http://somesimpledomain.com/some/path/überfile.txt", but it gives me a java.io.FileNotFoundException. I suspect that the filename on the remote server is encoded in latin1, though my url is in utf8. But my tries to change the encoding of the url weren't successful and I don't know how to debug it further. Please help!
Code is as follows:
HttpURLConnection conn = null;
try {
conn = (HttpURLConnection) new URL(uri).openConnection();
conn.setRequestMethod("GET");
} catch (MalformedURLException ex) {}
} catch (IOException ex){}
// Filter headers
int i=1;
String hKey;
while ((hKey = conn.getHeaderFieldKey(i)) != null) {
conn.getHeaderField(i);
i++;
}
// Open the file and output streams
InputStream in = null;
OutputStream out = null;
try {
in = conn.getInputStream();
} catch (IOException ex) {
ex.printStackTrace();
}
try {
out = response.getOutputStream();
} catch (IOException ex) {
}
Regards,
Hendrik

URL needs to be properly encoded. You have to know what charset/encoding your server is expecting. You can try this first,
String uri = "http://somesimpledomain.com/some/path/" +
URLEncoder.encode(filename, "ISO-8859-1");
If that doesn't work, replace "ISO-8859-1" with "UTF-8" and try again.
If that doesn't work either, file doesn't exist :)

Have you tried urlencoding it? E.g.
%FCberfile