Java - StackExchange get JSON - java

Here is my Java code to get some information from stackexchange.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.InputStreamReader;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
public class StackExchangeConnect {
private static final String URL = "http://api.stackexchange.com/2.2/answers?key=U4DMV*8nvpm3EOpvf69Rxw((&site=stackoverflow&fromdate=1396310400&todate=1398729600&order=desc&sort=activity&filter=default";
public static void main(String[] args) throws Exception {
StackExchangeConnect http = new StackExchangeConnect();
System.out.println("Testing 1 - Send Http GET request");
http.sendGet();
}
// HTTP GET request
private void sendGet() throws Exception {
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(URL);
// add request header
request.addHeader("Accept", "application/json");
HttpResponse response = client.execute(request);
System.out.println("Response Code : "
+ response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(new InputStreamReader(response
.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
System.out.println(result.toString());
}
}
I am getting following in my console.
Testing 1 - Send Http GET request
Response Code : 200
�
I tried writing the contents to file, and it's like
�\00\00\00\00\00\00�Z�n�H�Cy�;_�`df1��&���.F�ٔK���|I���eƒ�M�,GO�Ev��:U�_gE����_g���U����ʭ7�i�r5���t��]uUd�ʤZW����]���n\���l]�y�pW���[�uӬ�$���;�W��4�:��2��&��:�6�L�N8�2���)�,18?ԯ1Q���E�VMa�ի����_�1�-+���<\�̲}��sS��E��^�co�[W��{y;�:f�n�NgE}e�u��r��E�Ng�-+x8����{S���Uf���V�c!�`wY�xl�r݂n�����әY�w��J�.��quw�h��v\00�0!�`"�k𘀩��U����q�i�:��L�$C4�y���Se�6��Y�hȆ�%u�5��F�0b�̓�P�0��՘�b�*�x������b�����+λ�;/��M���?�����z>#2�1g^���S���ӏ9y��.�ͪ]�v!��F��h/��&�/�I^D3ċ!��6dI�0\00l�a� �S���B��8�q�P���h� �H<�ݼ��g�^|.����y���G/���c�\00u2a���LQ�����_ņ=ů7w�A?�Ĵ>�$%X� �͇f��Ib�%AA�_8���T����(C�����S�,�Y��A�nYu<K��$��1y��(\00Rg�E�6��Lx�B#���d&x#�->����(?^�pu�^M�粹6����-0�X#�q'���l9���'��~`�&J�}`\'"E,�ڦFJJ�^�>�H*r�q��ܦVZ�X�[ vtzKAn��g�nG ���!�z��:HJ:�oPH�oiFH�m11rt���L#i\00$όu� 'J��+�,KU4fs[5e������vh�� h��K+��4�T>�!*,���JH2.��>$�rC�)�9�!�3k��eee�O��2�`�N�ݯ]���~^ɦ}#��g��>rM��LT�Ѡ\00�m�8�Z�&�z����L�]�j.Ք�8Z�g�3(!�U�a��D\�#��Y�G�m�^lV��&�o��G�cGu���e|���8��(:\9��D&,�T������O���8y�����OQ��|�L�{\00PR�l>\000�d�"����!J�Pa%A�vs����!��j�2�p�3ąS�aĴ��M���&�B�A���9,Dx���b�,�)�&��mI�2]��g9M2F K&s�5��+����5���++xc4\�Ԓy��|ĘV�l�2b�3���D%��m!`㇇7�f}}���K��[(�4���d]�fS���^�~��ֿ}����=��ɶ��kY(�"��|]p��m�2�4DA�����3$��tѩ�j*�T�Y��� Џ.齦W�lzٹ%���Y �(d�#���P���m���{�-"���T)�(IS�n��)}��!I����bq#���O��1a*��0ؼ�("��C}9�<LNۖ�B0�ڮС�qŹ�0�e�N%#.�X ���F�+�kS�h�3K�i��$bd"��#H�H5�)F=�DL�r�r���E�������<ٶ^e�{8�x���(oب�F��� g���]�B�l�6/��Q�gR`O�g��p%1s�I[�G���Ooq�J8�*(m榮��ӏ9�TP�������ᬳ�Z�w[1{��P���`�s�Kn���HG�3�p��Kᣦ����$2�9v1f5U����Ɲ�x�!~K��[����c�Li�˽ͧҰ�|�G(x�J��)��vk��N{s��?~{�F�N>�|�/�G�q������\uF�OƵ���<}��؇q�٭�p�K�8�RN��I�\00�b�" W,�q$�z��[v��ɶ�1hu�\00Z���"�G�c�����C"���> ��#�(ÿ��4��$���e$%qb)C9�^������r��h��i%�����`�!��Y�b�;<B���f;�P�����쮎7��r�{A�Ru��ܬ6l&t�M��ۤ^�ӊ�0���$B���C�)Ti{��)9�� 刋�U�ζ��,�Tk����'������o�u�L6��Y=x?_�I.C���\ ����S�"0�����Fi�d�f{V�R��V���j#e�s-�Ny�s�3���.���4��X�7:�&u����H%oy��km�(U�O����\B�?N<x�)���j_L'���[%�5ÔZ��J�4��9�hR�\����0�d����M��櫐8�T�P��}�aE[%N<�"v>a��������[�T�ɫ�nV��L|nw��=�,��f�����X��:�]�!�,�� hƩ<t�`�CT��r =O�CiNS��pim�����-��T7����#'�֩��({=>�K���C1<���Ngצ�Zv#꿡?7ec����#�o�Tni�U���.�C���Ý�=1\00\00
When I try the same URL in browser it's getting the JSON. If I try other URL in program, it's getting the JSON properly. Why is that?
And also, is there any Java API so that the JSON String I can convert to some Object?
NOTE:
Apache HTTP Client - 4.3.3, Apache HTTP Core - 4.3.2, Commons logging - 1.1.3
UPDATE 1
I am printing response. System.out.println(response.toString()); And I am getting the following output.
HTTP/1.1 200 OK [Cache-Control: private, Content-Type: application/json; charset=utf-8, Content-Encoding: gzip, Access-Control-Allow-Origin: *, Access-Control-Allow-Methods: GET, POST, Access-Control-Allow-Credentials: false, Date: Tue, 29 Apr 2014 06:31:45 GMT, Content-Length: 2551]

It looks like it's compressed, w hich should be handled by HTTPClient; try using the builder and disabling compression.

The content being returned is gzip compressed. you can see this in your output of the response:
... Content-Encoding: gzip ...
Given that you're using the 4.3.3 version of the Apache HTTP client library, you simply need to do the following:
HttpClient client = HttpClientBuilder.create().build();
This will construct a HttpClient that has no problem decompressing the content, and you will get the JSON you're expecting ( The DefaultHttpClient does not support compression.)

The content is gzipped. You can verify that using
System.out.println(connection.getContentEncoding());
To get the content, use something like this.(This is java.net.URl version. Please change accordingly for httpclient)
GZIPInputStream gs = new GZIPInputStream(connection.getInputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(gs));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();

You should render the response as UTF-8.
And your second question is the Jackson library. It can conver json objects into Java objects.

Related

Cannot get XML from webpage to String in Java

I have a problem with getting XML from this webpage. In the browser it shows correctly and there is no issue, but when it comes to Java, it is different.
I've tried two methods which both of them resulted in exception.
// Method 1 - Using Java's URL
URL url = new URL(/* mentioned link */);
String rawXML = new String(url.openStream().readAllBytes(), StandardCharsets.UTF_8); // java.io.IOException: Invalid Http response
// Method 2 - Using Apache's HTTP client
HttpGet httpGet = new HttpGet(/* mentioned link */);
String rawXML = EntityUtils.toString(HttpClients.createDefault().execute(httpGet).getEntity()); // org.apache.http.ProtocolException: The server failed to respond with a valid HTTP response
Downloading this webpage with wget and using argument --content-on-error works but it is unreliable since wget is not always available on all systems like Windows.
The response does not contains headers so java rejects it
wget "https://www.strava.cz/foxisapi/foxisapi.dll/istravne.istravne.process?xmljidelnickyA&zarizeni=3148" -O so-69226464.html
--2021-09-17 13:44:29-- https://www.strava.cz/foxisapi/foxisapi.dll/istravne.istravne.process?xmljidelnickyA&zarizeni=3148
Resolving www.strava.cz (www.strava.cz)... 82.99.180.77
Connecting to www.strava.cz (www.strava.cz)|82.99.180.77|:443... connected.
HTTP request sent, awaiting response... 200 No headers, assuming HTTP/0.9
Length: unspecified
This java class making a raw HTTP GET request is able to get the contents. Based on this page.
The request sent is
GET /foxisapi/foxisapi.dll/istravne.istravne.process?xmljidelnickyA&zarizeni=3148 HTTP/1.1\r\n
User-Agent: RawHttpGet\r\n
Host: www.strava.cz\r\n
Accept: */*\r\n
Java code:
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.net.Socket;
import java.nio.charset.StandardCharsets;
import javax.net.ssl.SSLSocketFactory;
public class RawHttpGet {
private static String hostname = "www.strava.cz";
public static void main(String[] args) throws IOException {
Socket socket = SSLSocketFactory.getDefault().createSocket(hostname, 443);
// UTF-8 encdoding
//BufferedWriter out = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream(), StandardCharsets.UTF_8));
// Encoding for this request
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream(), "Cp1250"));
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
StringBuffer buff = new StringBuffer("GET /foxisapi/foxisapi.dll/istravne.istravne.process?xmljidelnickyA&zarizeni=3148 HTTP/1.1\r\n");
buff.append("User-Agent: RawHttpGet\r\n");
buff.append("Accept: */*\r\n");
buff.append("Host: " + hostname + "\r\n");
buff.append("\r\n");
System.out.println(" * Request");
System.out.println(buff.toString());
// send message
out.write(buff.toString());
out.flush();
// read response
System.out.println(" * Response");
// Default system encoding
//System.out.println(new String(socket.getInputStream().readAllBytes()));
// Encoding for this request
System.out.println(new String(socket.getInputStream().readAllBytes(), "Cp1250"));
out.close();
in.close();
}
}

HTTP post request fails

I wrote the below code to send post request to an url. When I ran the code I am getting 500 error code. But, when I tried the same url in SOAP UI with the below headers I got the response back. May I know what is wrong in my code. Thanks in advance. I doubt I didn't add the headers properly.
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:arm="http://siebel.com/Webservice">
<soapenv:Header>
<UsernameToken xmlns="http://siebel.com/webservices">username</UsernameToken>
<PasswordText xmlns="http://siebel.com/webservices">password</PasswordText>
<SessionType xmlns="http://siebel.com/webservices">Stateless</SessionType>
</soapenv:Header>
<soapenv:Body>
<arm:QueryList_Input>
<arm:SRNum></arm:SRNum>
</arm:QueryList_Input>
</soapenv:Body>
</soapenv:Envelope>
Below is my code.
package com.siebel.Webservice;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import javax.net.ssl.HttpsURLConnection;
public class HttpQueryList {
private final String USER_AGENT = "Mozilla/5.0";
public static void main(String[] args) throws Exception {
HttpQueryList http = new HttpQueryList();
System.out.println("\nTesting 2 - Send Http POST request");
http.sendPost();
}
// HTTP POST request
private void sendPost() throws Exception {
String url = "https://mywebsite.org/start.swe";
URL obj = new URL(url);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
//add reuqest header
con.setRequestMethod("POST");
con.setRequestProperty("UsernameToken", "username");
con.setRequestProperty("PasswordText", "password");
String urlParameters = "SWEExtSource=WebService&SWEExtCmd=Execute&WSSOAP=1";
// Send post request
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();
int responseCode = con.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + urlParameters);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//print result
System.out.println(response.toString());
}
}
In your XML, you are specifying a token. When I have done this using SOAP UI, I have a certificate file that I use. In my case, I put it in my C:\Program Files (x86)\SmartBear\SoapUI-5.2.1\bin folder. Then I configured SOAP UI to use this. Do you have a certificate? If yes, are you referencing it?

Java Apache http library file upload issue

I Have a webservice that uploads a file. (details in the first link)
To communicate with this webservice I use org.apache.http library (Apache HttpComponents). I have found most of this code if not all of it here.
Sadly the solution only works with images and despite the efforts when trying to upload a video is shows the error of Content Length too long. To try and fix this I decided to replace it with what's in current use. At first using HttpClientBuilder
HttpClient httpclient = HttpClientBuilder.create().disableContentCompression().build();
My try failed as it still gives the same error and after research I found out that I need to use the CloseableHttpClient as follow to disable the automatic headers
CloseableHttpClient httpclient = HttpClients.custom()
.setHttpProcessor(HttpProcessorBuilder.create().build())
.build();
That worked but when setting the headers something doesn't pass I can't figure it out. I did it as follow
httppost.addHeader("Keep-Alive", "timeout=5, max=100");
httppost.addHeader("Connection", "Keep-Alive");
httppost.addHeader("Content-Type", "text/html; charset=UTF-8");
But when tracing the response this is what I have
Http Client Builder Trace
Date,Tue, 28 Mar 2017 18:30:50 GMT
Server,Apache/2.4.23 (Win64) PHP/7.0.10
X-Powered-By,PHP/7.0.10
Content-Length,43
Keep-Alive,timeout=5, max=100
Connection,Keep-Alive
Content-Type,text/html; charset=UTF-8
Closable Response Trace
Date,Tue, 28 Mar 2017 18:31:27 GMT
Server,Apache/2.4.23 (Win64) PHP/7.0.10
Content-Length,311
Connection,close
Content-Type,text/html; charset=iso-8859-1
I tried to set the headers to the response but I am haven't figured how to instantiate it properly to set the headers. And also I am trying to understand why the httppost is not being taken in consideration. Did I miss something?
One last thing when I use Closable The way I did it doesn't upload anymore due to missing headers I guess.
EDIT
Error when uploading video
Exception in thread "main" org.apache.http.ContentTooLongException: Content length is too long: 16363535
at org.apache.http.entity.mime.MultipartFormEntity.getContent(MultipartFormEntity.java:103)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:199)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:306)
at testfileupload.PostFile.main(PostFile.java:100)
C:\Users\User\AppData\Local\NetBeans\Cache\8.2\executor-snippets\run.xml:53: Java returned: 1
BUILD FAILED (total time: 0 seconds)
EDIT
I changed the max file size for php following this but it still shows the same error
Update
This is how I handle the multipart (I use multipartform)
HttpEntity resEntity = MultipartEntityBuilder.create()
.addBinaryBody("userfile", file, ContentType.create("video/mp4"), file.getName())
.build();
I rememeber reading something about it (not sure) And as for the php.ini params that need to be of higher value upload_max_filesize = 40M and post_max_size = 40M and memory_limit = 128M which I am kinda sure its enough (using wamp64).
I kept trying to udnerstand the content length issue since ..
I found out that the HttpEntity sets calculates the length and I checked the value its 2 bits off I believe but no matter the conenction type it just doesn't work and sticks to content length error.
I also tried
HttpClient httpclient = HttpClients.custom().setDefaultHeaders(Arrays.asList(CustomHeader)).build();
But still nothing. At this point I highly doubt its about setting the header and more related to HttpEntity with the Multipart or the HttpPost but It could be else.
Upload
I tried with a 9s Video of a size ~350Kb but it still gives the content length error which means there might be a default content set or none at all
So I dont know what I did / changed but now its working with the error.
I would really want to know why this cause an error it doesn't make sense so far unless its calling something that can't take too high of a value (method of param type given value more than it can)
To be clear the error is generated when I use this (verifcation area)
System.out.println(response.getStatusLine());
if (resEntity != null) {
// error from below, kinda forgot about it
System.out.println(EntityUtils.toString(resEntity));
}
if (resEntity != null) {
// resEntity.consumeContent();
}
The webservice is the same as in the first link.
Below is my code (dont mind the comments I didn't clear the decrepated / changed / replaced / test / different approaches)
import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpResponse;
import org.apache.http.HttpVersion;
import org.apache.http.ProtocolVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.ContentBody;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.params.CoreProtocolPNames;
import org.apache.http.protocol.HTTP;
import org.apache.http.protocol.HttpProcessorBuilder;
import org.apache.http.util.EntityUtils;
public class PostFile {
public static void main(String[] args) throws Exception {
// HttpClient httpclient = new DefaultHttpClient();
// CloseableHttpClient httpclient = HttpClients.custom()
// .setHttpProcessor(HttpProcessorBuilder.create().build())
// .build();
// Header headerCustom = new BasicHeader(HttpHeaders.CONTENT_LENGTH, "15");
// HttpClient httpclient = HttpClients.custom().build();
// HttpClient httpclient = HttpClientBuilder.create().disableContentCompression().build();
HttpClient httpclient = HttpClientBuilder.create().build();
// httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
HttpPost httppost = new HttpPost("http://localhost/userVids/upload.php");
httppost.setProtocolVersion(new ProtocolVersion("HTTP", 1, 1));
File file = new File("C:/Users/User/Desktop/test_video.mp4");
// httppost.addHeader("Content-Length", String.valueOf(file.length()));
// MultipartEntity mpEntity = new MultipartEntity();
// ContentBody cbFile = new FileBody(file, ContentType.MULTIPART_FORM_DATA);
// mpEntity.addPart("userfile", cbFile);
// System.out.println(mpEntity.getContentLength());
//
// httppost.setEntity(mpEntity);
/*
Date,Tue, 28 Mar 2017 18:30:50 GMT
Server,Apache/2.4.23 (Win64) PHP/7.0.10
X-Powered-By,PHP/7.0.10
Content-Length,43
Keep-Alive,timeout=5, max=100
Connection,Keep-Alive
Content-Type,text/html; charset=UTF-8
*/
/*
Date,Tue, 28 Mar 2017 18:31:27 GMT
Server,Apache/2.4.23 (Win64) PHP/7.0.10
Content-Length,311
Connection,close
Content-Type,text/html; charset=iso-8859-1
*/
HttpEntity resEntity = MultipartEntityBuilder.create()
.addBinaryBody("userfile", file, ContentType.create("video/mp4"), file.getName())
.build();
System.out.println("Entity Length " + resEntity.getContentLength());
// httppost.addHeader("Keep-Alive", "timeout=5, max=100");
// httppost.addHeader("Connection", "Keep-Alive");
// httppost.addHeader("Content-Type", "text/html; charset=UTF-8");
// httppost.addHeader("Content-Disposition", "form-data; name=\"userfile\"; filename=\"star_empty.png\"");
// httppost.addHeader("Content-Type", "image/png");
// httppost.addHeader("Content-Transfer-Encoding", "binary");
// httppost.addHeader("Content-Length", "99999");
httppost.setEntity(resEntity);
System.out.println("executing request " + httppost.getRequestLine());
HttpResponse response = httpclient.execute(httppost);
// HttpEntity resEntity = response.getEntity();
System.out.println("Headers (name,value)");
List<Header> httpHeaders = Arrays.asList(response.getAllHeaders());
httpHeaders.forEach((header) -> {
System.out.println(header.getName() + "," + header.getValue());
});
System.out.println(response.getStatusLine());
if (resEntity != null) {
System.out.println(EntityUtils.toString(resEntity));
}
if (resEntity != null) {
// resEntity.consumeContent();
}
// httpclient.getConnectionManager().shutdown();
}
}
BUT A HUGE NOTICE I had my php.ini (changed all of them in wamp didn't know if it would consider only the running version or reads all (didn't test) ) and changed the values as Lucas Holt (he fixed it i.e, I am just providing for who needs or maybe explanation) said and he deserves credit for it. (I am sure it didn't work before that I just tested a lot and dont remember really.)

HTTP 403 Service Error when trying to post XML to HTTPS URL

I am trying to write a small class using the Apache HttpClient library that would do an HTTPS post to a specified URL sending some XML. When I run my code, the HTTP status line I receive back is "403 Service Error". Here's the complete error HTML returned:
$errorDump java.net.SocketTimeoutException:Read timed out
$errorInfo
$errorDump java.net.SocketTimeoutException:Read timed out
$error Read timed out
$localizedError Read timed out
$errorType java.net.SocketTimeoutException
$user
$time 2011-10-25 09:39:29 EDT
$error Read timed out
$errorType java.net.SocketTimeoutException
This is the code I am using:
import java.io.ByteArrayInputStream;
import java.io.InputStreamReader;
import java.io.BufferedReader;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.InputStreamEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
public class HttpXmlPost {
public static void main(String[] args)
{
String url = "https://someurlhere.com";
String xmlStr = "<?xml version=\"1.0\" encoding=\"UTF-8\" ?><xmlTag></xmlTag>";
String content = request(xmlStr, url);
System.out.println(content);
}
private static String request(String xmlStr, String url) {
boolean success = false;
String content = "";
HttpClient httpclient = new DefaultHttpClient();
try {
HttpPost httpPost = new HttpPost(url.trim());
InputStreamEntity reqEntity = new InputStreamEntity(new ByteArrayInputStream(xmlStr.getBytes() ), -1);
reqEntity.setContentType("application/xml");
reqEntity.setChunked(true);
httpPost.setEntity(reqEntity);
System.out.println("Executing request " + httpPost.getRequestLine());
HttpResponse response = httpclient.execute(httpPost);
HttpEntity resEntity = response.getEntity();
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
if(response.getStatusLine().getStatusCode() == 200){
success = true;
}
if (resEntity != null) {
System.out.println("Response content length: " + resEntity.getContentLength());
System.out.println("Chunked?: " + resEntity.isChunked());
}
BufferedReader reader = new BufferedReader(new InputStreamReader(resEntity.getContent()));
StringBuilder buf = new StringBuilder();
char[] cbuf = new char[ 2048 ];
int num;
while ( -1 != (num=reader.read( cbuf ))) {
buf.append( cbuf, 0, num );
}
content = buf.toString();
EntityUtils.consume(resEntity);
}
catch (Exception e) {
System.out.println(e);
}
finally {
httpclient.getConnectionManager().shutdown();
}
return content;
}
}
Whatever XML I pass in doesn't seem to matter, it gives the same error no matter what. Note that this actually works with some URLs. For example, if I put https://www.facebook.com, it goes through. However, it doesn't work for my specified URL. I thought it might be a certificate issue, tried to add some code to trust any certificate, didn't seem to work either, though I may have done it wrong. Any help is appreciated.
Based on the SocketTimeoutException in the first line of the response HTML, I'm guessing that the component which implements the handler for the URL to which you are posting is having some connection problems to a source system it needs to generate the response data.
Basically, it looks like the problem is on the server, not your client.

How can I get an HTTP response body as a string?

I know there used to be a way to get it with Apache Commons as documented here:
http://hc.apache.org/httpclient-legacy/apidocs/org/apache/commons/httpclient/HttpMethod.html
...and an example here:
http://www.kodejava.org/examples/416.html
...but I believe this is deprecated.
Is there any other way to make an http get request in Java and get the response body as a string and not a stream?
Here are two examples from my working project.
Using EntityUtils and HttpEntity
HttpResponse response = httpClient.execute(new HttpGet(URL));
HttpEntity entity = response.getEntity();
String responseString = EntityUtils.toString(entity, "UTF-8");
System.out.println(responseString);
Using BasicResponseHandler
HttpResponse response = httpClient.execute(new HttpGet(URL));
String responseString = new BasicResponseHandler().handleResponse(response);
System.out.println(responseString);
Every library I can think of returns a stream. You could use IOUtils.toString() from Apache Commons IO to read an InputStream into a String in one method call. E.g.:
URL url = new URL("http://www.example.com/");
URLConnection con = url.openConnection();
InputStream in = con.getInputStream();
String encoding = con.getContentEncoding();
encoding = encoding == null ? "UTF-8" : encoding;
String body = IOUtils.toString(in, encoding);
System.out.println(body);
Update: I changed the example above to use the content encoding from the response if available. Otherwise it'll default to UTF-8 as a best guess, instead of using the local system default.
Here's an example from another simple project I was working on using the httpclient library from Apache:
String response = new String();
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("j", request));
HttpEntity requestEntity = new UrlEncodedFormEntity(nameValuePairs);
HttpPost httpPost = new HttpPost(mURI);
httpPost.setEntity(requestEntity);
HttpResponse httpResponse = mHttpClient.execute(httpPost);
HttpEntity responseEntity = httpResponse.getEntity();
if(responseEntity!=null) {
response = EntityUtils.toString(responseEntity);
}
just use EntityUtils to grab the response body as a String. very simple.
This is relatively simple in the specific case, but quite tricky in the general case.
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("http://stackoverflow.com/");
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
System.out.println(EntityUtils.getContentMimeType(entity));
System.out.println(EntityUtils.getContentCharSet(entity));
The answer depends on the Content-Type HTTP response header.
This header contains information about the payload and might define the encoding of textual data. Even if you assume text types, you may need to inspect the content itself in order to determine the correct character encoding. E.g. see the HTML 4 spec for details on how to do that for that particular format.
Once the encoding is known, an InputStreamReader can be used to decode the data.
This answer depends on the server doing the right thing - if you want to handle cases where the response headers don't match the document, or the document declarations don't match the encoding used, that's another kettle of fish.
Below is a simple way of accessing the response as a String using Apache HTTP Client library.
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
//...
HttpGet get;
HttpClient httpClient;
// initialize variables above
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpClient.execute(get, responseHandler);
The Answer by McDowell is correct one. However if you try other suggestion in few of the posts above.
HttpEntity responseEntity = httpResponse.getEntity();
if(responseEntity!=null) {
response = EntityUtils.toString(responseEntity);
S.O.P (response);
}
Then it will give you illegalStateException stating that content is already consumed.
How about just this?
org.apache.commons.io.IOUtils.toString(new URL("http://www.someurl.com/"));
We can use the below code also to get the HTML Response in java
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.HttpResponse;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import org.apache.log4j.Logger;
public static void main(String[] args) throws Exception {
HttpClient client = new DefaultHttpClient();
// args[0] :- http://hostname:8080/abc/xyz/CheckResponse
HttpGet request1 = new HttpGet(args[0]);
HttpResponse response1 = client.execute(request1);
int code = response1.getStatusLine().getStatusCode();
try (BufferedReader br = new BufferedReader(new InputStreamReader((response1.getEntity().getContent())));) {
// Read in all of the post results into a String.
String output = "";
Boolean keepGoing = true;
while (keepGoing) {
String currentLine = br.readLine();
if (currentLine == null) {
keepGoing = false;
} else {
output += currentLine;
}
}
System.out.println("Response-->" + output);
} catch (Exception e) {
System.out.println("Exception" + e);
}
}
Here's a lightweight way to do so:
String responseString = "";
for (int i = 0; i < response.getEntity().getContentLength(); i++) {
responseString +=
Character.toString((char)response.getEntity().getContent().read());
}
With of course responseString containing website's response and response being type of HttpResponse, returned by HttpClient.execute(request)
Following is the code snippet which shows better way to handle the response body as a String whether it's a valid response or error response for the HTTP POST request:
BufferedReader reader = null;
OutputStream os = null;
String payload = "";
try {
URL url1 = new URL("YOUR_URL");
HttpURLConnection postConnection = (HttpURLConnection) url1.openConnection();
postConnection.setRequestMethod("POST");
postConnection.setRequestProperty("Content-Type", "application/json");
postConnection.setDoOutput(true);
os = postConnection.getOutputStream();
os.write(eventContext.getMessage().getPayloadAsString().getBytes());
os.flush();
String line;
try{
reader = new BufferedReader(new InputStreamReader(postConnection.getInputStream()));
}
catch(IOException e){
if(reader == null)
reader = new BufferedReader(new InputStreamReader(postConnection.getErrorStream()));
}
while ((line = reader.readLine()) != null)
payload += line.toString();
}
catch (Exception ex) {
log.error("Post request Failed with message: " + ex.getMessage(), ex);
} finally {
try {
reader.close();
os.close();
} catch (IOException e) {
log.error(e.getMessage(), e);
return null;
}
}
Here is a vanilla Java answer:
import java.net.http.HttpClient;
import java.net.http.HttpResponse;
import java.net.http.HttpRequest;
import java.net.http.HttpRequest.BodyPublishers;
...
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(targetUrl)
.header("Content-Type", "application/json")
.POST(BodyPublishers.ofString(requestBody))
.build();
HttpResponse response = client.send(request, HttpResponse.BodyHandlers.ofString());
String responseString = (String) response.body();
If you are using Jackson to deserialize the response body, one very simple solution is to use request.getResponseBodyAsStream() instead of request.getResponseBodyAsString()
Using Apache commons Fluent API, it can be done as mentioned below,
String response = Request.Post("http://www.example.com/")
.body(new StringEntity(strbody))
.addHeader("Accept","application/json")
.addHeader("Content-Type","application/json")
.execute().returnContent().asString();

Categories