We Keep Coding

REST API Client response in java encoding trouble

I have a problem with an API call response in java.
See below the API response of my request, server replies with content that is unreadable as text:
Here is my code:
String urlt = "xxxxxx";
URL url = new URL(urlt);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Connection", "Keep-Alive");
conn.addRequestProperty("Accept-Encoding", "gzip");
conn.addRequestProperty("User-Agent", "okhttp/3.4.1");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
String output;
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
and an example of response headers:

With the following line, you tell the server that you are ready to accept a compressed response:
conn.addRequestProperty("Accept-Encoding", "gzip");
As shown in picture of response headers, the server obliges and gives you gzip-encoded (compressed) content.
But then you proceed to read the response, assuming it's just text... so yeah it prints as garbage in your console.
Either you remove that header above, or be ready to uncompress gzipped content.

HttpsUrlConnection Android vs Native Java

I wrote some code that tries to communicate with a website "as a browser" (in terms of cookies & headers). I currently have four requests (GET, POST, POST, GET).
The code is pretty straightforward: opening a connection, adding headers and cookies, parsing response.
GET code:
conn = (HttpsURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setUseCaches(false);
conn.setRequestProperty("User-Agent", userAgent);
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
if (cookies != null) {
for (String cookie : this.cookies) {
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
}
int responseCode = conn.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
setCookies(conn.getHeaderFields().get("Set-Cookie"));
return response.toString();
POST code:
conn = (HttpsURLConnection) url.openConnection();
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.setRequestProperty("Host", "...");
conn.setRequestProperty("User-Agent", userAgent);
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
conn.setRequestProperty("Accept-Encoding","identity");
for (String cookie : this.cookies) {
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
conn.setRequestProperty("Connection", "keep-alive");
conn.setRequestProperty("Referer", "https://...");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));
conn.setDoOutput(true);
conn.setDoInput(true);
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(postParams);
wr.flush();
wr.close();
int responseCode = conn.getResponseCode();
InputStream is = responseCode != 400 ? conn.getInputStream() : conn.getErrorStream();
BufferedReader in = new BufferedReader(new InputStreamReader(is));
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
return response.toString();
The program works well on my PC. However, when running it on an Android device, I face multiple problems on the second POST; on one device there's Too many redirections exception, and in another one (which is the one I focus on) I simply receive 400 Bad Request although the exact same request returns 200 on my PC.
I noticed that there are actually two different implementations: On PC I'm using sun.net.www.protocol.https.DelegateHttpsURLConnection, and on Android it's com.android.okhttp.internal.http.HttpURLConnectionImpl. Object conn looks a bit different during runtime. However, I failed to find a meaningful difference (if needed, I can post the whole content of these objects); I found one difference in the cookies set in the first GET, but a manual manipulation resulted in the same 400.
I tried to capture the Android output request using Wireshark, but the result is encrypted and I didn't manage to decrypt it.
I basically thought of two possible scenarios:
Find the difference(s) between those implementations and act accordingly.
Find a way to use the sun.net.www.protocol.https.DelegateHttpsURLConnection on Android.
So far I didn't manage to figure out any of these. Is there any known difference/issue with these implementations? Is there a way to run the native Java library on Android? Any help will be appreciated, thanks.

Yes, there are some difference between those SDKs, the desktop is the Oracle sdk and on Android is the one published by Google.
Ony way, when doing http requests on Android it's recommended using Retrofit or Volley

Java Post Data HttpsUrlConnection or HttpClient 4.5

I am trying to do what I thought was a simple task. I need to POST data to a PHP server. I have tried this solution but in Apache HttpClient 4.5 I can't find BasicNameValuePair in the package. Upon further research I thought I'd try StringEntity...nope not in 4.5 either (that I can find at least). So I tried to do it with HttpsURLConnection. The problem with that is I can't figure out how to add a name to my parameter and with a name, I don't know how to access in PHP with $_POST['name'].
My Current Code:
String json = gson.toJson(data);
URL url = new URL("https://www.domain.com/test.php");
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Content-Length", String.valueOf(json.length()));
OutputStream os = conn.getOutputStream();
os.write(json.getBytes());
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String decodedString;
while ((decodedString = in.readLine()) != null) {
System.out.println(decodedString);
}
in.close();

Try to use DataOutputStream and flush it afterward.
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeChars(json);
wr.flush();
wr.close();

How to add header to HttpRequest of GET method in Java

I have to pass a token as part of validation for each GET request to access RESTful web service. Below is the code I'm using it to access REST api:
public static String httpGet(String urlStr, String[] paramName, String[] paramVal) throws Exception {
URL url = new URL(urlStr);
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setDoOutput(true);
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
OutputStream out = conn.getOutputStream();
Writer writer = new OutputStreamWriter(out, "UTF-8");
for (int i = 0; i < paramName.length; i++) {
writer.write(paramName[i]);
writer.write("=");
writer.write(URLEncoder.encode(paramVal[i], "UTF-8"));
writer.write("&");
}
writer.close();
out.close();
if (conn.getResponseCode() != 200) {
System.out.println("Response code: "+conn.getResponseCode());
throw new IOException(conn.getResponseMessage());
}
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = rd.readLine()) != null) {
sb.append(line);
}
rd.close();
conn.disconnect();
return sb.toString();
}
I can't see any such method to set Header conn.setHeader() provided for HttpsURLConnection. It should be something like X-Cookie: token={token}; please help me to find a way to set header.

You can use:
conn.addRequestProperty("X-Cookie", "token={token}");
or setRequestProperty() also works

You are already setting headers on your request in your code when you do the following:
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
I.e. if the service you are communicating with requires that you send your token in the "X-Cookie" header you can simply do the same for that header:
conn.setRequestProperty("X-Cookie", "token={token}");

HttpURLConnection always return error 500

i'm trying to run a soap request in a basic http request...naturally i tried with external tools the message and is correct, like the endpoint i'm using as targetUrl, the wsdl is in something like
http://00.00.00.00/a-ws/services/basic?wsdl
and my actual end point is
http://00.00.00.00/a-ws/services/basic.targetservice
and i'm using this last as target url
URL url = new URL(targetUrl);
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "text/xml;charset=UTF-8");
connection.setRequestProperty("SOAPAction", action);
connection.setRequestProperty("User-Agent", "myagent");
connection.setRequestProperty("Host", "localhost");
//connection.setRequestProperty("Content-Length", "" + Integer.toString(message.getBytes().length));
connection.setUseCaches (false);
connection.setDoInput(true);
connection.setDoOutput(true);
//Send request
OutputStream wr = connection.getOutputStream ();
wr.write (message.getBytes());
wr.flush ();
wr.close ();
//Get Response
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line=null;
StringBuffer response = new StringBuffer();
while( (line = rd.readLine()) != null) {
if (line!=null)
response.append(line);
}
rd.close();
return response.toString();
the raw message is tested with chrome plugin, the only thing i can't test is headers but the result is always an exception on getInputStream
java.io.IOException: Server returned HTTP response code: 500 for URL:
why?

It was a very stupid issue of encoding (like I was supposing)...i didn't escape double quote inside the message.
The evidence of problem was visible using a fake http server that just echo contents.
UPDATE:
Another thing nobody already pointed out is that is useful in case of exception to retrieve
connection.getErrorStream()
that contains the response in case of error!