We Keep Coding

How to decompress invocation API response In Java?

I want to decompress an API response (header : Accept-encoding=gzip).
Here are the lines I tried but always failed due to:
java.util.zip.ZipException: Not in GZIP format
String entity = response.readEntity(String.class);
// When I print "entity" I will get Compress message as below:
��{o�ʒ��
���c^MC��������ut4"�ms���fWGگ�_o?�V�G�ę�=�PF�3��i�������ɚ���G�[�eEA����$��M�(�rW�nk��97�m�r��6�$�$T�a^ZaIj�"�5U�4�����4:oW�C{�-��A�c0�hސ�*l���JP�ƚ.������t�
}ˏ�r�kIxjk��!���m�0��Z�9 ���r[�����6!٦^fQ�X_d)hބe���m��\RP�Y��Xg�:�F�IEE�5]U��f�\jϋ�?N�ߖ�<�κ;�+��j�xQ�{����40�]4N�NOib�=o(r�mL�rLϱ�>Rۖ�l4
{2jʁk��f�*�љw��a�l���������^�V�a�ӱ���w[2�V>n��'��n���;�ȧ�#�p-ch}<>�9>�IB�~X��X���=��lz9)H��2#O?�R����*�����q�c�V�t����c�ܩ<��A��[���4
��
byte[] input = entity.getBytes();
// helper method
public static String decompress(byte[] compressed) throws IOException {
ByteArrayInputStream bis = new ByteArrayInputStream(compressed);
// Always failed at "GZIPInputStream" line due to java.util.zip.ZipException: Not in GZIP format
GZIPInputStream gis = new GZIPInputStream(bis);
BufferedReader br = new BufferedReader(new InputStreamReader(gis, "UTF-8"));
StringBuilder sb = new StringBuilder();
String line;
while((line = br.readLine()) != null) {
sb.append(line);
}
br.close();
gis.close();
bis.close();
return sb.toString();
}

How to read error details from REST API

I am trying to register a user to my Web Server. If i send valid details to the server then i get 201 code in response which stands for "created". But when i send wrong credentials to the server i.e same username as before then i get 400 FileNotFoundException. I need to interpret error details not only 400 code. Because if i send wrong details using curl from command line then i get error details also e.g this username already exists.
Here is my code to read response from the server. Actually i've tried two different methods but they all end up in same error which is 400(Bad Request) nothing else.
public static String readResponse(HttpURLConnection connection)
throws IOException, JSONException {
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
StringBuilder response = new StringBuilder();
while((line = rd.readLine()) != null) {
response.append(line);
response.append('\r');
}
return response.toString();
}
public static String readResponseFromServer(HttpURLConnection connection) throws IOException {
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader
(connection.getInputStream()));
String line = "";
StringBuilder stringBuilder = new StringBuilder();
while ((line = bufferedReader.readLine()) != null) {
stringBuilder.append(line).append("\n");
}
return stringBuilder.toString();
}
In the above code there are two methods for reading response from the server.
And here is how i am using these methods to read response from the server
System.out.println("Server Response" + WebServiceHelpers.readResponseFromServer(urlConnection));
And i am also using the 2nd method which is readResponse() from the above code.
And here is the screenshot of curl command in which i am sending wrong details to the server and getting error details.
I've also tried this with HTTPIE and i am also getting the same response as using curl command i.e A user with that username already exists.
I need these error details in my Java code also. I have searched the internet but didn't found a solution.
Any suggestions?

Try this
public static String readResponse(HttpURLConnection connection)
throws IOException, JSONException {
int respCode = connection.getResponseCode();
InputStream is = null;
if (isErrorCode(respCode)) {
is = connection.getErrorStream();
} else if (connection.getErrorStream() != null) {
is = connection.getInputStream();
}
//FIXME: InputStreamReader must be constructed with right charset
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
StringBuilder response = new StringBuilder();
while((line = rd.readLine()) != null) {
response.append(line);
}
return response.toString();
}
Write isErrorCode method. It should interpret response code 400 as error and other codes as you need. Also notice fixme comment. When you are constructing InputStreamReader without a charset it uses default charset (UTF-8 if you don't provide file.encoding property) but correct way is to get charset from Content-Type response header and process response body with that encoding. Method for extracting charset from response may looks like this
private String getCharset(HttpURLConnection con) {
String charset = "";
String contentType = con.getContentType();
if (contentType != null) {
String[] values = contentType.split(";");
for (String value : values) {
String trimValue = value.trim();
if (trimValue.toLowerCase().startsWith("charset=")) {
charset = trimValue.substring("charset=".length());
}
}
}
if ("".equals(charset)) {
charset = "UTF-8";
}
return charset;
}

HTTP GET request in java returns meaningless data from App Engine

I'm requesting a json file from an App Engine URL
http://1-1-26a.wordbuzzweb.appspot.com/json/level-images.json
The file encoding is UTF-8 without a BOM. If I look at this file on my local disk it's size is 12414 bytes. If I get the file in Chrome is reads it perfectly well. If I then save it it's 12414 bytes. However, if I try and download the file with a GET request in java I only get 780 bytes returned and the returned data would appear to be meaningless.
I've tried several different types of get request, both of the methods below I have used elsewhere perfectly effectively. The response code on the GET requests is 200. Interestingly, if I do a POST with no content instead of a GET, then I get the valid response.
If I download the file from this URL on Google Drive instead, then the GET methods below work perfectly.
edit This code is now working, however, this is a recurring issue that comes and goes. If anyone has any ideas what might be causing it then please say so!
This doesn't work
public static String doGetSync(String urlToRead) throws IOException {
URL url = new URL(urlToRead);
InputStream is = url.openStream();
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int nRead;
byte[] data = new byte[BUFFER_SIZE];
while ((nRead = is.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
}
buffer.flush();
byte[] bytes = buffer.toByteArray();
return new String(bytes, "UTF-8");
}
Neither does this
public static String doGetSync2(String urlToRead) throws IOException {
final String charset = "UTF-8";
// Create the connection
HttpURLConnection connection = (HttpURLConnection) new URL(urlToRead).openConnection();
// Check the error stream first, if this is null then there have been no issues with the request
InputStream inputStream = connection.getErrorStream();
if (inputStream == null)
inputStream = connection.getInputStream();
// Read everything from our stream
BufferedReader responseReader = new BufferedReader(new InputStreamReader(inputStream, charset));
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = responseReader.readLine()) != null) {
response.append(inputLine);
}
responseReader.close();
return response.toString();
}
This code works
public static String doPostSync(final String url, final String content) throws IOException {
final String charset = "UTF-8";
// Create the connection
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
// setDoOutput(true) implicitly set's the request type to POST
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", charset);
connection.setRequestProperty("Content-type", "application/json");
// Write to the connection
OutputStream output = connection.getOutputStream();
output.write(content.getBytes(charset));
output.close();
// Check the error stream first, if this is null then there have been no issues with the request
InputStream inputStream = connection.getErrorStream();
if (inputStream == null)
inputStream = connection.getInputStream();
// Read everything from our stream
BufferedReader responseReader = new BufferedReader(new InputStreamReader(inputStream, charset));
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = responseReader.readLine()) != null) {
response.append(inputLine);
}
responseReader.close();
return response.toString();
}

How to read the blob data from servlet request object

There is client and server components, the client is sending the data in more secure way by converting the data in blob using POST method to the server.
Can any suggest me how to convert that blob data to string object in server side(Java).i have tried some code below
Way 1):
==============================
String streamLength = request.getHeader("Content-Length");
int streamIntLength = Integer.parseInt(streamLength);
byte[] bytes = new byte[streamIntLength];
request.getInputStream().read(bytes, 0, bytes.length);
String content = DatatypeConverter.printBase64Binary(bytes);
System.out.println(content);
Output for above code is : some junk data is displaying.
dABlAG0AcABsAGEAdABlAD0AMgAzADUAUgBfAFAAcgBvAHYAaQBkAGUAcgBfA
Way 2) :
======
BufferedReader reader = new BufferedReader(new InputStreamReader(
request.getInputStream()));
StringBuilder sb = new StringBuilder();
for (String line; (line = reader.readLine()) != null;) {
String str = new String(line.getBytes());
System.out.println(str);
}
Please suggest me any one, above both ways are not worked out.

Below code works for me.
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
String streamLength = request.getHeader("Content-Length");
int streamIntLength = Integer.parseInt(streamLength);
InputStream inputStream = request.getInputStream();
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(
inputStream));
char[] charBuffer = new char[streamIntLength];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
} else {
stringBuilder.append("");
}
} catch (IOException ex) {
throw ex;
}
String body = stringBuilder.toString();
//System.out.println(body);
byte[] bytes = body.getBytes();
System.out.println(StringUtils.newStringUtf16Le(bytes));

From the first approach, it looks like the data is encoded (possibly in Base64 format). After decoding it, what is the problem you are facing ? If the data is String and then encoded to Base64, you should get the actual string after decoding it. (Assuming platform locales on client and server side are same).
If its a binary data, better you keep it inside a byte stream only. If you anyhow want it to convert to a string, then the first approach looks okay.
If this binary data represents some kind of file, you can get the related information using the HTTP headers and write it to temp location for further use.

Is it possible to check progress of URLconnection.getInputStream()?

I want to check progress of downloading file by URLconnection. Is it possible or should I use another library? This is my urlconnection function:
public static String sendPostRequest(String httpURL, String data) throws UnsupportedEncodingException, MalformedURLException, IOException {
URL url = new URL(httpURL);
URLConnection conn = url.openConnection();
//conn.addRequestProperty("Content-Type", "text/html; charset=iso-8859-2");
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream(), "ISO-8859-2"));
String line, all = "";
while ((line = rd.readLine()) != null) {
all = all + line;
}
wr.close();
rd.close();
return all;
}
I understand that whole file is downloaded in this line (or worng)?:
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream(), "ISO-8859-2"));
So is it possible to do this in this code?

Just check if the HTTP Content-Length header is present in the response.
int contentLength = connection.getContentLength();
if (contentLength != -1) {
// Just do (readBytes / contentLength) * 100 to calculate the percentage.
} else {
// You're lost. Show "Progress: unknown"
}
Update as per your update, you're wrapping the InputStream inside a BufferedReader and reading inside a while loop. You can count the bytes as follows:
int readBytes = 0;
while ((line = rd.readLine()) != null) {
readBytes += line.getBytes("ISO-8859-2").length + 2; // CRLF bytes!!
// Do something with line.
}
The + 2 is to cover the CRLF (carriage return and linefeed) bytes which are eaten by BufferedReader#readLine(). More clean approach would be to just read it by InputStream#read(buffer) so that you don't need to massage the bytes forth and back from characters to calculate the read bytes.
See also:
How to use java.net.URLConnection to fire and handle HTTP requests?

Wrap it in a javax.swing.ProgressMonitorInputStream. But note that Java may buffer the entire response before it starts delivering it to the stream ...

BufferedReader rd = new BufferedReader(new InputStreamReader(new FilterInputStream(conn.getInputStream())
{
public int read(byte[] buffer, int byteOffset, int byteCount) throws IOException
{
int count = super.read(buffer, byteOffset, byteCount);
// do whatever with count, i.e. mDownloaded += count;
return count;
}
}, "ISO-8859-2"));