We Keep Coding

Persisting the same HttpURLConnection to send multiple JSON lists separately: what should ContentLength be?

Example problem:
I have an initial list of size 1000.
I want to POST the data of this list to a URL in 10 HTTP requests, where each request writes a ("sublist") of size 100 (since 1000 / 10 = 100).
Now I want to do the above using Java's HttpURLConnection, I have this method:
private static HttpURLConnection open(String endpoint) throws IOException {
URL url = new URL(endpoint);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setConnectTimeout(5000);
connection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
connection.setDoOutput(true);
connection.setDoInput(true);
// Should I also set Content-Length? Why or why not?
connection.setRequestMethod("POST");
return connection;
}
My issue is that I don't know what the content length should be. Given the example above, should it be 1000 (the total size of the list fetched)? Or should it be 100 (the size of an individual list that will be sent throughout the session)? Note that I wish for the connection to be persistent; no need to open a new connection for every list to be sent.
For reference, my function that does the POST looks like the below:
private static int post(String endpoint, String payload) throws IOException {
int response;
try {
HttpURLConnection connection = open(endpoint, payload.length());
try (BufferedWriter writer = new BufferedWriter(
new OutputStreamWriter(connection.getOutputStream()))) {
writer.write(payload);
}
try (InputStream is = connection.getInputStream()) {
IOUtils.consume(is);
} catch (IOException e) {
InputStream es = connection.getErrorStream();
IOUtils.consume(es);
es.close();
}
response = connection.getResponseCode();
System.out.println(String.format("Sent batch with response code %d.", response));
} catch (IOException e) {
throw new IOException(e);
}
return response;
}

Copying Image from URL to HTTP URL

Someone is providing a S3 Presigned URL so that I can upload my images to that link. All my images are on the website. Is there a way in JAVA to copy the image URL to the new URL provided ?
I am trying to do this. Seems like an overkill
try {
// Get Image from URL
URL urlGet = new URL("http://something.com/something.png");
BufferedImage image = ImageIO.read(urlGet);
//for png
ImageIO.write(image, "png",new File("/something.png"));
// for jpg
//ImageIO.write(image, "jpg",new File("/something.jpg"));
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ImageIO.write(image, "png", outputStream);
outputStream.flush();
byte[] imageInBytes = outputStream.toByteArray();
outputStream.close();
URL url = new URL(putUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod(HttpMethod.PUT);
connection.setRequestProperty(HttpHeaders.CONTENT_TYPE, PNG_MIME_TYPE);
OutputStream stream = connection.getOutputStream();
try {
stream.write(imageInBytes);
} finally {
stream.close();
connection.disconnect();
}
switch (connection.getResponseCode()) {
case HttpURLConnection.HTTP_OK:
return "";
default:
break;
}
} catch (Exception e) {
log.error("Exception occured", e);
throw e;
}

There would be no point converting to BufferedImage and back for the copy when you can preserve the byte stream of the original files. The first part can be replaced with simple call to extract the bytes off your website:
byte[] imageInBytes = read(urlGet);
Where read() is:
private static byte[] read(URL url) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream(16*1024);
try (var in = url.openStream()) {
in.transferTo(out);
}
return out.toByteArray();
}
If you use JDK11 onwards you could try the HttpClient class for the GET and POSTs, for example this does same as above if passing it urlGet.toURI():
private static byte[] read(URI uri) throws IOException, InterruptedException
{
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder().uri(uri).build();
var resp = client.send(request, BodyHandlers.ofByteArray());
return resp.body();
}

Java HttpURLConnection is not sending a body

I'm trying to send a PUT request with a body to my API in my Android app with Java. I'm using the java.net HttpURLConnection and the request gets through. But without the body.
I refactored my code over and over again but it still does not work, at the moment I have following Task:
#Override
protected String doInBackground(String... urls) {
String result = "";
URL url;
HttpURLConnection urlConnection = null;
JSONObject body = new JSONObject();
try {
body.put("title", title);
body.put("content", content);
try {
url = new URL(urls[0]);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("PUT");
urlConnection.addRequestProperty("username", username);
urlConnection.addRequestProperty("password", password);
urlConnection.addRequestProperty("Content-Type", "application/json; utf-8");
urlConnection.addRequestProperty("Accept", "application/json");
urlConnection.setDoOutput(true);
try(OutputStream os = urlConnection.getOutputStream()){
byte[] input = body.toString().getBytes("utf-8");
os.write(input, 0, input.length);
}
InputStream in = urlConnection.getInputStream();
InputStreamReader reader = new InputStreamReader(in);
int data = reader.read();
while (data != -1){
char current = (char) data;
result += current;
data += current;
data = reader.read();
}
return result;
}catch (Exception e){
e.printStackTrace();
return null;
}
} catch (JSONException e) {
e.printStackTrace();
return null;
}
}
But this code leads always to an empty body.
EDIT
When changing content type to application/x-www-form-urlencoded the server gets the body. But this way I don't get a JSON file for further processing it.

The java code is working correctly. The API wasn't accepting JSON. Other languages worked without problems.
Server Info
Node.js
Express
Body Parser
I didn't add app.use(bodyParser.json());

Call httpconnection with encoding not working

i need to call a service get using http connection, the response contains arabic characters, but when i call it using the code below
try {
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
InputStream in = new BufferedInputStream(conn.getInputStream());
response = IOUtils.toString(in, "UTF-8");
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
the reponse is
1|U|����� ������|$2|L|���� �������|$3|S|����
I tried another solution not using Commons-io but also not working
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setConnectTimeout(5000);
connection.setRequestMethod("GET");
connection.connect();
int statusCode = connection.getResponseCode();
//Log.e("statusCode", "" + statusCode);
if (statusCode == 200) {
sb = new StringBuilder();
reader = new BufferedReader(new InputStreamReader(connection.getInputStream(),"UTF-8"));
char[] tmp = new char[1024];
int l;
while((l = reader.read(tmp)) != -1) {
sb.append(tmp, 0, l);
}
//sb = buffer.toString();
}
connection.disconnect();
if (sb != null)
serverResponse = sb.toString();
Do i need to change anything from web service??? but when i call it from browser all characters show clearly with no problem
any suggestion?

Maybe the server is not using UTF-8, your code is trying to use UTF-8 to decode the data but that will only work if the server is using the same encoding.
The browser works because maybe it is using the HTTP header "Content-Encoding" which should indicate the encoding used for the data.

Please decode your string response
String dateStr = URLDecoder.decode(yourStringResponse, "utf-8");

Java InputStream NullPointerException with InputStream

I've got two methods in a class:
private static InputStream getSongStream(String ip, String id){
try {
URL url = new URL("http://"+ ip + "/" + Client.streamphp);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data); //Data is a simple Http Post that is know to work
wr.flush();
wr.close();
return conn.getInputStream();
} catch (MalformedURLException badurl) {
System.out.println(badurl);
return null;
} catch (IOException noconnection) {
System.out.println(noconnection);
return null;
}
}
public static void downloadSong(String ip, String id, String path){
InputStream rd = Client.getSongStream(ip, id);
try {
OutputStream stream = new FileOutputStream(new File(path));
byte[] buffer = new byte[4096];
int len;
while ((len = rd.read(buffer)) > 0) { //Here I get NullPointerException
stream.write(buffer, 0, len);
}
stream.close();
rd.close();
} catch (IOException noconnection) {
System.out.println(noconnection);
}
}
The line commented in the second method is the problem, if I put all in the same method I can download the song without problems, but not if I separate them.
Any ideas? I would like to have them separated to reuse getSongStream.

The problem is that you're swallowing exceptions in getSongStream and returning null. Don't do that - let the exception propagate up, possibly having wrapped it in another form... so declare that your method can throw (say) IOException. Your downloadSong method should probably declare that it can throw IOException too. Note that you should have finally blocks to make sure you close your streams appropriately even if an exception is thrown.
It's almost always a bad idea to catch an exception, write it out to standard output and then proceed as if everything is okay.