I am having weird character encoding issues with a JSON array that is grabbed from a web page. The server is sending back this header:
Content-Type text/javascript; charset=UTF-8
Also I can look at the JSON output in Firefox or any browser and Unicode characters display properly. The response will sometimes contain words from another language with accent symbols and such. However I am getting those weird question marks when I pull it down and put it to a string in Java. Here is my code:
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, "utf-8");
params.setBooleanParameter("http.protocol.expect-continue", false);
HttpClient httpclient = new DefaultHttpClient(params);
HttpGet httpget = new HttpGet("http://www.example.com/json_array.php");
HttpResponse response;
try {
response = httpclient.execute(httpget);
if(response.getStatusLine().getStatusCode() == 200){
// Connection was established. Get the content.
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String jsonText = convertStreamToString(instream);
Toast.makeText(getApplicationContext(), "Response: "+jsonText, Toast.LENGTH_LONG).show();
}
}
} catch (MalformedURLException e) {
Toast.makeText(getApplicationContext(), "ERROR: Malformed URL - "+e.getMessage(), Toast.LENGTH_LONG).show();
e.printStackTrace();
} catch (IOException e) {
Toast.makeText(getApplicationContext(), "ERROR: IO Exception - "+e.getMessage(), Toast.LENGTH_LONG).show();
e.printStackTrace();
} catch (JSONException e) {
Toast.makeText(getApplicationContext(), "ERROR: JSON - "+e.getMessage(), Toast.LENGTH_LONG).show();
e.printStackTrace();
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader;
try {
reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
} catch (UnsupportedEncodingException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
StringBuilder sb = new StringBuilder();
String line;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
As you can see, I am specifying UTF-8 on the InputStreamReader but every time I view the returned JSON text via Toast it has strange question marks. I am thinking that I need to send the InputStream to a byte[] instead?
Thanks in advance for any help.
Try this:
if (entity != null) {
// A Simple JSON Response Read
// InputStream instream = entity.getContent();
// String jsonText = convertStreamToString(instream);
String jsonText = EntityUtils.toString(entity, HTTP.UTF_8);
// ... toast code here
}
#Arhimed's answer is the solution. But I cannot see anything obviously wrong with your convertStreamToString code.
My guesses are:
The server is putting a UTF Byte Order Mark (BOM) at the start of the stream. The standard Java UTF-8 character decoder does not remove the BOM, so the chances are that it would end up in the resulting String. (However, the code for EntityUtils doesn't seem to do anything with BOMs either.)
Your convertStreamToString is reading the character stream a line at a time, and reassembling it using a hard-wired '\n' as the end-of-line marker. If you are going to write that to an external file or application, you should probably should be using a platform specific end-of-line marker.
It is just that your convertStreamToString is not honoring encoding set in the HttpRespnose. If you look inside EntityUtils.toString(entity, HTTP.UTF_8), you will see that EntityUtils find out if there is encoding set in the HttpResponse first, then if there is, EntityUtils use that encoding. It will only fall back to the encoding passed in the parameter(in this case HTTP.UTF_8) if there isn't encoding set in the entity.
So you can say that your HTTP.UTF_8 is passed in the parameter but it never get used because it is the wrong encoding. So here is update to your code with the helper method from EntityUtils.
HttpEntity entity = response.getEntity();
String charset = getContentCharSet(entity);
InputStream instream = entity.getContent();
String jsonText = convertStreamToString(instream,charset);
private static String getContentCharSet(final HttpEntity entity) throws ParseException {
if (entity == null) {
throw new IllegalArgumentException("HTTP entity may not be null");
}
String charset = null;
if (entity.getContentType() != null) {
HeaderElement values[] = entity.getContentType().getElements();
if (values.length > 0) {
NameValuePair param = values[0].getParameterByName("charset");
if (param != null) {
charset = param.getValue();
}
}
}
return TextUtils.isEmpty(charset) ? HTTP.UTF_8 : charset;
}
private static String convertStreamToString(InputStream is, String encoding) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*/
BufferedReader reader;
try {
reader = new BufferedReader(new InputStreamReader(is, encoding));
} catch (UnsupportedEncodingException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
StringBuilder sb = new StringBuilder();
String line;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
Archimed's answer is correct. However, that can be done simply by providing an additional header in the HTTP request:
Accept-charset: utf-8
No need to remove anything or use any other library.
For example,
GET / HTTP/1.1
Host: www.website.com
Connection: close
Accept: text/html
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.10 Safari/537.36
DNT: 1
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: utf-8
Most probably your request doesn't have any Accept-Charset header.
Extract the charset from the response content type field. You can use the following method to do this:
private static String extractCharsetFromContentType(String contentType) {
if (TextUtils.isEmpty(contentType)) return null;
Pattern p = Pattern.compile(".*charset=([^\\s^;^,]+)");
Matcher m = p.matcher(contentType);
if (m.find()) {
try {
return m.group(1);
} catch (Exception e) {
return null;
}
}
return null;
}
Then use the extracted charset to create the InputStreamReader:
String charsetName = extractCharsetFromContentType(connection.getContentType());
InputStreamReader inReader = (TextUtils.isEmpty(charsetName) ? new InputStreamReader(inputStream) :
new InputStreamReader(inputStream, charsetName));
BufferedReader reader = new BufferedReader(inReader);
Related
I've been trying to download the csv from the UK government coronavirus API. you can download the file manually by using the URL: https://api.coronavirus.data.gov.uk/v1/data?filters=areaType=overview&structure=%7B%22areaName%22:%22areaName%22,%22date%22:%22date%22,%22newCasesBySpecimenDate%22:%22newCasesBySpecimenDate%22,%22cumCasesBySpecimenDate%22:%22cumCasesBySpecimenDate%22%7D&format=csv
I've tried multiple methods of downloading files, such as JavaIO:
InputStream in = null;
try {
in = new URL("https://api.coronavirus.data.gov.uk/v1/data?filters=areaType=overview&structure=%7B%22areaName%22:%22areaName%22,%22date%22:%22date%22,%22newCasesBySpecimenDate%22:%22newCasesBySpecimenDate%22,%22cumCasesBySpecimenDate%22:%22cumCasesBySpecimenDate%22%7D&format=csv").openStream();
} catch (IOException e) {
e.printStackTrace();
}
try {
Files.copy(in, Paths.get("test.csv"), StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
e.printStackTrace();
}
and Java NIO:
ReadableByteChannel readableByteChannel = Channels.newChannel(url.openStream());
FileOutputStream fileOutputStream = null;
try {
fileOutputStream = new FileOutputStream("test.txt");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
FileChannel fileChannel = fileOutputStream.getChannel();
try {
fileOutputStream.getChannel()
.transferFrom(readableByteChannel, 0, Long.MAX_VALUE);
} catch (IOException e) {
e.printStackTrace();
}
however, it only shows this in the file:
‹ 0w®_ÿ}ZKŽX5Üs
àHm÷ÏÞ;$6ˆŒÈe‘€HâöTy²L=)Š”y»ÝŸru{^þ~}ùååãëxÿòåu|zý÷ǗϯŸøï׿^ÿðñõÓOüñïÿ|üÆ¿ûíÓ‡/¯ï¿ÿùç?Þÿùq,[önNü3,Ç\Çæ ³1ÛN·wï£pvÆjÛ\Ï:Kâö˜§66_ez=ì¸}½Vœµ%®Æò•°rY•™Ä%p½`–õÔ¸ ®xŽ™§VKœ• 8ÏÒ¶†Ï¨nZ˜ö <’1¦ÁË^ßÆÙ;®bÃ;V>•Ÿ³±V]\T[*ÜBÜÜž¶Âæ–¸M¿,œ×Ìjk\8ë8݈„!lE÷\'µuI¯¸³*¦KÜV+±¼²M³Ýc#`Ka1¥¼ÆnØ6%l""|ì8=õˆô™c¯X-ã5óiïÑgŸl CA2x4¼RGÂ÷èêÑyæ–Q˜HK„ °ˆ˜z5Dar¤WõÔG–ö‰X[ÃP[{ƒQE©i"Õ¦UXTF4†Êߣ ¬OJ‚H¬Y¾gËMÁbÈ7d²©tö’ÄÎ;!mûÊas u¤{Q°ìÁL°
Q#Éì‘YXTÂœ«9JP–Œ]†Ë¹qú¤ˆ‚WŸ‘`—1%{ùìBç·7µCRBÓ`Žo'‘æ#šËÕ#I¼0À®-ˆ($Úe‘µED‰ªQ#ÊoJT¢à3Týt½* PZáHomjØò Ñz° 9¨æXÈò–(¤X¢¢EL¢ 9ÁµìÌ#½
"Š]ð½ùnix(Nùðõ…Íʉ‚Òž¸®B;hYQ9 TZʃ»5ãáŒ÷" ª·½#îjÜß ÑÔgœØ•î
µ¢w4X³ö*èçÞÅ^å-íû ë k6Ûµî¶!§|ŠèëZm=ÅËÖÓCç=M2çZöPC ‡†½3$
¾?Œ#Œ{#á
¦ös”H‰¼ßW2¥1L¸Í5
Ü4™Ñ¾MÈR HMk1LµÅmCÔ¦!*k¶DázÈ»£q…Û~
NJ…<ãß
‘ATÐzäÄ:ûa-§Vh—¹-€
îÖÞ‚ç'³~ÕI½<ôÃÊNéÓIÏb
7¹õ»è7Ó;ÎÛÎ Ÿ¬“^ʃZ£¼Î-Õ=¡Ù–‘¦2šè Øýèýx%³Ê -VHÔ¢à!
ÂMzž²È!) <wëµèù¾Jê<+ñ:Õg!ý R‚Nˆá‘ 'Xÿ³]‡r膙¬E5ÄëÓÑ›š6ýÔ$j£i’(g—7X׃ñkP/¹UêHƒfò®TR¸4 b Å“M>:,U‹=K®Rrž—§Dmô(p¨!çS‚Ð1aðO!ñ BIð6jG ‚ \Üñi ¢HâJf
‡F:‡7*Aë+Â#o7G#§»Ò-A66/}ôKžŽú(¸RÍ)#|å:á“¥ø¥ÉBÙ4µ§W*ÈÄ (R$ÔMÑz»#…AsJ€ƒØn¯³TƒLÒ²x:t5ÚO‘ ùÃJ¤qqPŸ2vàŸžÜÎÜdf‚š34[Úþ)j†½¥0oòOS²eu‚~íÂÚ;K‚‚ú˜ ü#‚œ}#UKoÇV“Û•…t&NVµ(r$è·AEZ©É•Ð)4>;Ð÷KrŠ¢7Ô]·û‚#HŠªzk¾¾{A¬J{/úi–¢ßº¬ÒŒÒ†CßlºÀ¦’]uõ
ô'šc7éÌEãt½ãátðø†MJ%A`•ƒž}‘žävWÜ0.»GºœâÆ&MO‰âD’µÐhͤ«8ù1æ'aÚ®€¤æÑn˜ô:mÇå®Aœ/2_X1S¢&OˆµêÌÔ(
úƒŒ©=]ÚΉ4P…&U¢èù»V¶bŽ"½4çx¥jÙ$ôr)=몛É9dù,iª¶?«–…Œ!Õ“‰ zJ£àù8Ø×̃·øhàì[ͽ,oÃe›sOÔ~Iãs‡¨PLE…“œg¸˜b…ø w jÉv$oõ9G¸\¢PÅë-ô˜$oÃU´ýéƒõqÇg#åy°ÞyF¢"}K‚cÌÔ ü’(:´‘_`¤ÔÖßjåSÇví¯+œyË¡-i= 1É]èòT{—•¶ó±Æ|VZFMšW,Ÿd%zócšôÇ&|Áüíò·ñÍZ¦fýD-N°ÖìR•–w””X3Vh]ã&: ‰â{ ÿÞèé%jßVÿF‡&ãx§SÌBCO¥QèÞØÒO¨íX…¾¥6gó™Sæ=•óí èÒ_v9gé;´ïù‚†•Ñ2sŒÓâ‹B¼EN9'NÝWT×zs ¡#b½Ò[¢6wäs\ËÛ8È9A9‰Xºª¡ ç8ñ™]J9Ç;85
YAÎAïaœy…ºÑ‚œ“¸<AçðªF¡ç
ˆ—Hm}_œ8ÄvíUø~ƒ¿&#¨ârΦ]¨ðEßóê^ W¹#ÇÏ'ö½¤^ =-Ä>hÛd'ß6’ívÅ”ùÎáÓ å‰ë8pßð¥Ð"ícÇ8ýú„dðå>’ƃ„s‚óMFÚ8öO¹:µ#79‘Ñ]¥ã ¶¹Ï¢=S»Óê><*Ó§#×#Ž õé-ê ܇h¢æѧƒÇûmŸÚO äzá/Ü4ßV‚HREï#¨!“Cmç—‹p óbQÛù}žgÑ,ùJ[5^ü%ËÏID(a
9ßñËCF•n{éíòês’èÃvÍç{s½’ó*ä0¶R:“¿s€ìæ kIg¾Mš7
Aüj1÷õÙÛ)À)3ú!´uj²á$ŸsÞ® apPz'4½ÕÒ‡“ï‘wâ(ÏÄ÷.ª„šÒdŠñ¥(ÐÉ5±o#5„/‚÷·…ô‰Ø‰€Aøm¶ç^òÈdÞ˜z‰ÅçPéöWdE½˜œ}¯<…„. Žlx£Ë !“ð7jdºÞ‡«¡Ûº
^ŸùkQCåÖº³\¤ü ŽÃþò34Ô·ne5ú\·™ª?Y¬fzF~žüßú³áK¯¡xŸ¾îǯýøµ¿æã×xüê_×ãW:K¥¯dYŠ_¡¯†L?£¯dX=~Ej=|¥¯dbš?~]_§þ:ya?}Å7|ý:7é…)
I then moved on to trying to make an http request, which I used this code:
URL url = new URL("https://api.coronavirus.data.gov.uk/v1/data?filters=areaType=overview&structure=%7B%22areaName%22:%22areaName%22,%22date%22:%22date%22,%22newCasesBySpecimenDate%22:%22newCasesBySpecimenDate%22,%22cumCasesBySpecimenDate%22:%22cumCasesBySpecimenDate%22%7D&format=csv");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
int status = con.getResponseCode();
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer content = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
content.append(inputLine);
}
in.close();
con.disconnect();
which only returns
0w�_�}ZK�X5�s�Hm���;$6��e��H��Ty�L=)��y�ݟru{^�~}�����x���u|z��Ǘϯ���^�����O����|�Ə���Ӈ/����ç?���q,[�nN�3,�\�� �1�N�w�pv�j�\�:K����66_ez=�}�V��%���rY���%p�`��Ը �x���VK��� 8����ϨnZ�� <�1���^���;�b�;V>�����V]\T[*�B�܁���斸M�,���jk\�8�8��!lE��\'�uI����*�K�V+���M���c#`Ka1���n�6%l""|�8=����c�X-�5�i��g�l CA2x4�RG�����y�Q�HK� ���z5Dar�W�����X[�P[{�Q�E�i"��UXTF�4��ߣ �OJ�H�Y�g�M�b�7d��t����;!m��asu�{�Q���L�Q#��YXT���9JP��]����q����W��`�1%{��B�7�CRB�`�o'���#����#I�0��-��($ځe��ED��Q�#�oJT��3T�t�*�PZ�Homj�� �z� 9��X��(�X��EL� 9����#�"�]��nix(N������ʉ�Ґ���B;hY�Q9 TZ���5���" ���#�j�� ��g������w4X����*����^�-�� � k6۵�!��|���Zm=����C�=M2�Z�PC ���3$�?�#�{#���s�H���W2�1L��5�4�ѾM�R�HMk1L��mCԦ!*k�D�zȻ�q��~NJ�<��AT�z��:�a-�Vh��-����ނ�'�~�I�<���N��I�b7����7�;��� ���^ʃZ����-�=�ٖ��2�� ����x%�ʠ-VHԢ�!�Mz���!) <w����J�<+�:Õg!� R�N�ᑠ'X��]�r����E5���ћ�6���$j�i�(g�7X׃�kP/�U�H�f�TR�4 b� œM>:,U�=K��Rr����Dm�(p�!�S��1a�O!� BI�6jG � \��i �H�Jf��F:�7*A��#o7G#���-A66/}�K���(�R�)#|�:ᓥ���B�4��W*�� (R$�MÑz�#�AsJ���n��T�LҲx:t5�O���� ��J��qqP�2v������df��34[��)j���0o�OS�eu�~��ڍ;K���� �#��}#UKo�V�ە�t&NV�(r$�AEZ�ɕ�)�4>;��Kr��7ԭ]���#H��zk���{A�J{/�i��ߺ�Ҍ��҆C�l����]u��'�c7��E��t���t���MJ%A`���}���vW�0.�G����&M��D���hͤ�8�1�'aڮ����n��:m��A�/2_X1S�&O�����(����=]�Ή4P�&U����V�b�"�4�x�j�$�r)=몛�9d�,i��?����!Ս�� zJ���8��̃��h��[ͽ,o�e�sO�~I�s��PLE���g��b���w j�v$oõ9G�\�P��-��$o�U����q�g#�y��yF�"}K�c�Ԡ��(:��_`����j�S�v�+�yˡ�-i= 1�]��T{������|VZFM�W,�d%z�c����&|�������Z�f�D-N���R��w��X3Vh]�&: ��{ ����%j�V�F�&�x�S�BCO�Q����O��X���6g�S�=���� ��_v9g�;�������2s���B�EN9'N�W�T�z�s �#b��[�6w�s\��8�9A9�X��� �8�]J9�;85YA�A�a�y��т���<A��F����Hm}_�8�v�U�~��&#���rΦ]��E���^�W�#��'����^ =-�>h�d'�6��vŔ���� ��8p����"�c�8���d��>�ƃ�s���MF�8��:�#79��]�� ��Ϣ=S���><�*ӧ#�#����-�� ܇h��ѧ���m��O �z�/�4�V�HRE�#�!�Cm痋p �bQ��}�g�,�J[5^�%��ID(a9���CF�n{����s���v��{s���*�0�R:��s���kIg�M�7A�j1����)�)3�!�uj��$�s a�pPz'4��҇��w�(���.����d��(��5�o#5�/�����؉�A�m��^��dޘz���P��WdE���}�<���. �lx�� !��7jd�އ��ۺ^���kQC�ֺ�\��� ����34��ne5�\���?Y�fzF~������K��x���ǯ�������x��_��W:K��dY�_���L?��dX=~Ej=|��db�?~]�_��:ya?}�7|�:7��)
when outputting content.
Can anyone help me?
In violation of the HTTP protocol, the server returns the csv data with Content-Encoding: gzip, even if you explicitly send an Accept-Encoding: identity header. You will need to decompress the response body.
This constraint is actually documented in the Developers guide for this API:
All API responses are compressed using GZip. The request client must therefore accept GZip encoded content.
Something like this should work, and continue to work if the server starts sending uncompressed responses against our (now explicit) request:
URLConnection connection;
try {
connection = new URL("...").openConnection();
connection.setRequestProperty("Accept-Encoding", "gzip");
connection.connect();
} catch (IOException e) {
e.printStackTrace();
}
InputStream in = connection.getInputStream();
String encoding = connection.getContentEncoding();
if ("identity".equals(encoding)) {
// Nothing to do.
} else if ("gzip".equals(encoding)) {
in = new GZIPInputStream(in);
} else {
throw IOException("Unsupported encoding " + encoding);
}
try {
Files.copy(in, Paths.get("test.csv"), StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
e.printStackTrace();
}
Note that neither this code, nor your own, closes the connections and streams properly. I recommend to use try-with-resources to close things gracefully even in case of an exception.
You could save yourself some headaches by using a third-party HTTP library, such as OkHttp or Apache HttpClient.
I'm trying to make a post to a node.js server but for some reason the body is always empty for me no matter what I try.
I'm testing now towards requestb.in and its always empty there too.
This is the code I use for posting:
public static String post(String url, String json) {
StringBuilder stringBuffer = new StringBuilder();
BufferedReader bufferedReader = null;
try {
HttpClient httpClient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpPost httpPost = new HttpPost("http://requestb.in/14a9s7m1");
StringEntity se = new StringEntity("{'string':'string'}", HTTP.UTF_8);
se.setContentType("application/json; charset=UTF-8");
httpPost.setEntity(se);
httpPost.setHeader("Accept", "application/json");
httpPost.setHeader("Content-type", "application/json");
httpPost.setHeader("hmac", Methods.getMethods().getHmac(json));
HttpResponse httpResponse = httpClient.execute(httpPost, localContext);
InputStream inputStream = httpResponse.getEntity().getContent();
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
String readLine = bufferedReader.readLine();
while (readLine != null) {
stringBuffer.append(readLine);
stringBuffer.append("\n");
readLine = bufferedReader.readLine();
}
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return stringBuffer.toString();
}
This is the requestb.in http://requestb.in/14a9s7m1?inspect
raw body should contain the json string, right?
Any suggestions?
You can make many mistakes when using HttpUrlConnection. I admit that I don't see any error, but this doesn't mean anything.
Since Google doesn't recommend using HttpClient and AndroidHttpClient (except for FROYO or older), but we should use HttpUrlConnection instead, you're on the right way (from a Android perspective).
When using a very lightweight wrapper for HttpUrlConnection called DavidWebb, the code looks like this (I left out hmac-generation):
public class TestWebbRequestBin {
#Test public void stackOverflow20543115() throws Exception {
Webb webb = Webb.create();
webb.setBaseUri("http://requestb.in");
JSONObject jsonObject = new JSONObject();
jsonObject.put("string", "string");
String json = jsonObject.toString(); // {"string":"string"}
Response<String> response = webb
.post("/1g7afwn1")
.header("Accept", "application/json")
.header("Content-type", "application/json")
.header("hmac", "some-hmac-just-a-test")
.body(json)
.asString();
assertEquals(200, response.getStatusCode());
assertTrue(response.isSuccess());
String body = response.getBody();
assertEquals("ok\n", body);
}
}
When the JSON I post looks like in your example, requestb.in does accept it:
json = "{'string':'string'}";
But this is not valid JSON (here tested in node.js):
> JSON.parse("{'string':'string'}")
SyntaxError: Unexpected token '
at Object.parse (native)
at repl:1:7
at REPLServer.self.eval (repl.js:110:21)
at Interface.<anonymous> (repl.js:239:12)
tl;dr
Take care to send valid JSON
Master HttpUrlConnection or use a simple abstraction library
For nasty bugs you could either debug your node.js code (or console.log(req)) or use a tool like Wireshark.
Try this code to send the string.... In HttpPost you should use key value pairs to send the data.
HttpPost httppost = new HttpPost(SERVER_URL);
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("REQUEST", req));
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs, "UTF-8"));
HttpResponse response = httpclient.execute(httppost);
I am not sure is this the problems. Can you give a tried?
You are sending invalid JSON format string. This make server unable to accept your invalid json string so your body is empty. To solve this, change following code.
StringEntity se = new StringEntity("{\"string\":\"string\"}", HTTP.UTF_8);
I couldn't get HttpPost to work, but HttpUrlConnection works instead. It solves my problem, but doesn't solve the mysterious no body thing of httpPost.
Here is my solution:
public static String post(String ur2l, String json) {
StringBuilder stringBuffer = new StringBuilder();
BufferedReader bufferedReader = null;
try {
URL url = new URL(ur2l);
HttpURLConnection conn;
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Accept", "application/json");
conn.setRequestProperty("Content-type", "application/json");
conn.setRequestProperty("hmac", Methods.getMethods().getHmac(json));
conn.setDoOutput(true);
OutputStream os = null;
try {
os = conn.getOutputStream();
os.write(json.getBytes());
} catch (Exception e) {
e.printStackTrace();
}
os.close();
conn.connect();
int respCode = conn.getResponseCode();
if (respCode == 200) {
InputStream inputStream = conn.getInputStream();
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
String readLine = bufferedReader.readLine();
while (readLine != null) {
stringBuffer.append(readLine);
stringBuffer.append("\n");
readLine = bufferedReader.readLine();
}
}
} catch (MalformedURLException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return stringBuffer.toString();
}
I am having problems calling a simple JSON web service from an Android app. The .execute() completes successfully with an 200-OK Status however I am unable to read any JSON output or text.
For the record, if I HttpPost a regular webpage, like Google.com, I can read and parse all the markup. Also, I am able to call the complete urlWithParams string from the device's browser and I see JSON output in the browser. This works in device's browser:
http://maps.googleapis.com/maps/api/distancematrix/json?origins=Seattle&destinations=San+Francisco&mode=bicycling&language=fr-FR&sensor=false
When the code runs, the reader is always blank and reader.readLine() never runs. Returns an empty string. If I change the URL to Google.com, it works and returns 17,000 characters. Thanks!
protected String doInBackground(String... uri) {
String responseString = null;
try {
//String urlGoogle = "http://google.com";
//String urlWithParams = "http://maps.googleapis.com/maps/api/distancematrix/json?origins=Seattle&destinations=San+Francisco&mode=bicycling&language=fr-FR&sensor=false";
String urlOnly = "http://maps.googleapis.com/maps/api/distancematrix/json";
HttpClient httpclient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(urlOnly);
httpPost.setHeader("Accept", "application/json");
httpPost.setHeader("Content-type", "application/json");
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("origins", "Seattle"));
nameValuePairs.add(new BasicNameValuePair("destinations", "Cleveland"));
nameValuePairs.add(new BasicNameValuePair("sensor", "false"));
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = httpclient.execute(httpPost);
int status = response.getStatusLine().getStatusCode();
HttpEntity entity = response.getEntity();
InputStream is = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append((line + "\n"));
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
responseString = sb.toString();
}}
catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
catch (ClientProtocolException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
return responseString;
}
Maybe you should test other mime types instead of application/json.
1 - Check in your manifest file having INTENET Permission or not.
2 - Use this code its returning data
BufferedReader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
try {
String inputLine;
while ((inputLine = reader.readLine()) != null) {
responseString += inputLine;
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Solved! The blank return when calling the JSON page was due to not having the proxy settings defined. Proxy settings were setup on the device however per this post, HttpClient does NOT inherit them.
Adding the following line resolved my issue. The code is now returning JSON.
HttpHost proxy = new HttpHost("172.21.31.239", 8080);
httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
Good day. Have just switched from objective-c to java and trying to read url contents normally to string. Read tons of posts and still it gives garbage.
public class TableMain {
/**
* #param args
*/
#SuppressWarnings("deprecation")
public static void main(String[] args) throws Exception {
URL url = null;
URLConnection urlConn = null;
try {
url = new URL("http://svo.aero/timetable/today/");
} catch (MalformedURLException err) {
err.printStackTrace();
}
try {
urlConn = url.openConnection();
} catch (IOException e) {
e.printStackTrace();
}
try {
BufferedReader input = new BufferedReader(new InputStreamReader(
urlConn.getInputStream(), "UTF-8"));
StringBuilder strB = new StringBuilder();
String str;
while (null != (str = input.readLine())) {
strB.append(str).append("\r\n");
System.out.println(str);
}
input.close();
} catch (IOException err) {
err.printStackTrace();
}
}
}
What's wrong? I get something like this
??y??'??)j1???-?q?E?|V??,??< 9??d?Bw(?э?n?v?)i?x?????Z????q?MM3~??????G??љ??l?U3"Y?]????zxxDx????t^???5???j??k??u?q?j6?^t???????W??????????~?????????o6/?|?8??{???O????0?M>Z{srs??K???XV??4Z??'??n/??^??4????w+?????e???????[?{/??,??WO???????????.?.?x???????^?rax??]?xb??& ??8;?????}???h????H5????v?e?0?????-?????g?vN
Here is a method using HttpClient:
public HttpResponse getResponse(String url) throws IOException {
httpClient.getParams().setParameter("http.protocol.content-charset", "UTF-8");
return httpClient.execute(new HttpGet(url));
}
public String getSource(String url) throws IOException {
StringBuilder sb = new StringBuilder();
HttpResponse response = getResponse(url);
if (response.getEntity() == null) {
throw new IOException("Response entity not set");
}
BufferedReader contentReader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line = contentReader.readLine();
while ( line != null ){
sb.append(line)
.append(NEW_LINE);
line = contentReader.readLine();
}
return sb.toString();
}
Edit: I edited the response to ensure it uses utf-8.
This is a result of:
You are fetching data that is UTF-8 encoded
You are didn't specify, but I surmise you are printing it to the console on a Windows system
The data is being received and stored correctly, but when you print it the destination is incapable of rendering the Russian text. You will not be able to just "print" the text to stdout unless the ultimate display handler is capable of rendering the characters involved.
I am making a little program thath will read data from website. String in the html file is already managed every info is divided with ; . Now i should read complete line here is example of this line:
14:47;24.02.12;18.7°C;18.7°C;285;0.5m/s; 6:48;17:37; Warm ;36;1.8;0.0;
So first how should i read them with HTTP Get or is there anything other? And then i would like to save each info, they are seperated with ; into a variable. And how should i cut each info from this line.
You definitely need to do some homework, but this methods will help you:
public static String getContentFromUrl(String url) throws ClientProtocolException, IOException {
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
HttpResponse response;
response = httpClient.execute(httpGet);
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream inStream = entity.getContent();
String result = HttpService.convertStreamToString(inStream);
inStream.close();
return result;
}
return null;
}
private static String convertStreamToString(InputStream is) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
This allows you to get data from a URL. Then lookup String.split to chop your string into usable entities.
Hope this helps!
Use GET request to fetch the data from the website
Separate the string-data from the HTML-markup
Parse the string into multiple strings or a list of strings, using ';' as the delimiter.
Android Java SDK String reference