Displaying NON-ASCII Characters using HttpClient - java

So, i am using this code to get the whole HTML of a website. But i dont seem to get non-ascii characters with me. all i get is diamonds with question mark.
characters like this: å, appears like this: �
I doubt its because of the charset, what could it then be?
Log.e("HTML", "henter htmlen..");
String url = "http://beep.tv2.dk";
HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
client.getParams().setParameter(CoreProtocolPNames.HTTP_ELEMENT_CHARSET, "UTF-8");
HttpGet request = new HttpGet(url);
HttpResponse response = client.execute(request);
Header h = HeaderValueFormatter
response.addHeader(header)
String html = "";
InputStream in = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder str = new StringBuilder();
String line = null;
while((line = reader.readLine()) != null)
{
str.append(line);
}
in.close();
//b = false;
html = str.toString();

Thank you. This worked (in case others have the issue):
HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
client.getParams().setParameter(CoreProtocolPNames.HTTP_ELEMENT_CHARSET, "iso-8859-1");
HttpGet request = new HttpGet(url);
request.setHeader("Accept-Charset", "iso-8859-1, unicode-1-1;q=0.8");
HttpResponse response = client.execute(request);
String html = "";
InputStream in = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(in,"iso-8859-1"));

use the new InputStreamReader(in, "UTF-8") constructor
Set the Accept-Charset request header to, say, Accept-Charset: iso-8859-5, unicode-1-1;q=0.8
Make sure the page opens properly in a browser. If it does not, then it might be a server-side issue.
If none of the above works, check other headers using firebug (or similar tool)

This really helped me get started, but I was having the same problem while reading a text file. It was fixed using the following command:
BufferedReader br = new BufferedReader(new InputStreamReader(new
FileInputStream(fileName), "iso-8859-1"));
...and of course, the HTTP Response needs to have the encoding set as well:
response.setCharacterEncoding("UTF-8");
Thanks for the help!

Related

Code works in Xamarin Android but doestn't work in Java (HttpPost JSON)

I started developing in Xamarin, and then decided that license may be a bit expensive for playing around, so I'm transferring my code to java.
I have a small chunk that performs a POST with a JSON object, and it works in Xamarin and doest work in Java.
Xamarin:
var client = new HttpClient ();
var content = new FormUrlEncodedContent(new Dictionary<string, string>() {
{"action", "getEpisodeJSON"},
{"episode", "11813"}
});
client.DefaultRequestHeaders.Referrer = new Uri(link);
var resp = client.PostAsync("http://www.ts.kg/ajax", content).Result;
var repsStr = resp.Content.ReadAsStringAsync().Result;
dynamic res = JsonConvert.DeserializeObject (repsStr);
Android:
HttpClient httpclient = new DefaultHttpClient();
// 2. make POST request to the given URL
HttpPost httpPost = new HttpPost("http://www.ts.kg/ajax");
String json = "";
// 3. build jsonObject
JSONObject jsonObject = new JSONObject();
jsonObject.accumulate("action", "getEpisodeJSON");
jsonObject.accumulate("episode", "11813");
// 4. convert JSONObject to JSON to String
json = jsonObject.toString();
// 5. set json to StringEntity
StringEntity se = new StringEntity(json);
// 6. set httpPost Entity
httpPost.setEntity(se);
// 7. Set some headers to inform server about the type of the content
httpPost.setHeader("Accept", "application/json");
httpPost.setHeader("Content-type", "application/json");
httpPost.addHeader("Referer", "http://www.ts.kg");
// 8. Execute POST request to the given URL
HttpResponse httpResponse = httpclient.execute(httpPost);
// 9. receive response as inputStream
InputStream inputStream = httpResponse.getEntity().getContent();
// 10. convert inputstream to string
String result;
if(inputStream != null)
result = convertInputStreamToString(inputStream);
What is a correct way to make such a POST in Android?
UPD
Current problem is that i'm getting an empty result string;
private static String convertInputStreamToString(InputStream inputStream) throws IOException{
BufferedReader bufferedReader = new BufferedReader( new InputStreamReader(inputStream));
String line = "";
String result = "";
while((line = bufferedReader.readLine()) != null)
result += line;
inputStream.close();
return result;
}
I ended up catching all requests of my device via Fiddle (good tutorial is here: http://tech.vg.no/2014/06/04/how-to-monitor-http-traffic-from-your-android-phone-through-fiddler/)
The difference was in cookie, so I used and HttpContex variable as described here:
Android HttpClient Cookie
And I also had a different Content-Type, so I set this header manually as this:
httpPost.setHeader("Content-Type", "application/x-www-form-urlencoded");

Java Code for quering REST API is very slow

I am using following code for querying REST API but it is very slow. Where as response time for browser is in milliseconds. I think there is no issue in API.
Could anyone please suggest what changes I should do to optimize it? Thanks for your time
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(URI);
HttpResponse response = client.execute(request);
BufferedReader rd = new BufferedReader (new InputStreamReader(response.getEntity().getContent()));
String line = "";
StringBuilder outputLine=new StringBuilder();
while ((line = rd.readLine()) != null) {
outputLine.append("\n"+line);
}
return (JSONObject)new JSONParser().parse(outputLine.toString());
I would expect the biggest delays are in the time;
it takes to get a reply
the time it takes to build a JSONObject.
You can speed up how the text is copied as follows but it might not make much difference.
HttpResponse response = client.execute(request);
char[] chars = new char[4096];
Reader rd = new InputStreamReader(response.getEntity().getContent());
StringWriter sw = new StringWriter();
for (int len; ((len = rd.read(chars)) > 0;)
sw.write(chars, 0, len);
return (JSONObject)new JSONParser().parse(sw.toString());
This is faster as it copies up to 4 KB at a time without breaking the text into lines and having to assemble it back into one block of text.
You should look in toont retrofit with okhttp. With those libraries you can easly communicate with a rest API and it also parses the body to json automatically.

How to parse a large amount of data from a JSON webservice

I have a URL which returns a large amount of data in response, but I can't get a full response from the webservice URL. The webservice responds as follows:
\"},
{\"minute_usage_end_time\":\"11:59\",\"minute_usage_start_time\":\"11:00\",\"kwh_usage\":\"0\",\"meter_reading_date\":\"08-02-2011\"},{...
What should I do?
StringEntity entity = new StringEntity(jsonObject.toString(),HTTP.UTF_8);
httpPost.setEntity(entity);
HttpResponse response = httpClient.execute(httpPost);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
Log.v("mas\n", reader.readLine());
String jsonResultStr = reader.readLine();

Reading JSON data from the net (Twitter)

I found this great tutorial on how to use JSON to retrieve Twitter updates, and post it in a TextView:
http://www.ibm.com/developerworks/xml/library/x-andbene1/
I've followed this tutorial step by step, so my code is the same.
In the method examineJSONFile(), we have this line:
InputStream is = this.getResources().openRawResource(R.raw.jsontwitter);
This file is downloaded directly from the Twitter website, as mentioned in the second paragraph of http://www.ibm.com/developerworks/xml/library/x-andbene1/#aotf.
All this is great, except for one thing: it's absolutely no use that one has to download the Twitter updates (tweets) and then build the app using this as a raw file. It should be possible to download this JSON file at runtime, and then show the tweets in the TextView afterwards.
I have tried to create the InputStream in another way, like this:
String url = "http://twitter.com/statuses/user_timeline/bbcnews.json";
HttpClient httpclient = new DefaultHttpClient();
HttpResponse response = httpclient.execute(new HttpGet(url));
InputStream is = response.getEntity().getContent();
BufferedReader br = new BufferedReader(new InputStreamReader(is,"UTF-8"));
StringBuffer sb = new StringBuffer();
try
{
String line = null;
while ((line = br.readLine())!=null)
{
sb.append(line);
sb.append('\n');
}
}
catch (IOException e)
{
e.printStackTrace();
}
String jsontext = new String(sb.toString());
But it seems this line: HttpResponse response = httpclient.execute(new HttpGet(url)); throws an exception.
Any help please?
You seem to be missing the INTERNET permission. Look at the logs and it would be clear what exactly is the problem.

Java client has trouble to read big responses from wcf rest service

I am trying to consume a rest WCF service using java client on android 2.1.
It works perfectly on small responses. When i tried to push a little further by getting 1000+ char response reader.read(buffer) failed to read all the characters. This caused the last line of the script to return: JsonException unterminated string at character 8193 "{plate,...
My android device starts to get this error before the emulator android stars to get it (character 1194 instead of 8193). Anyone knows how get the full message?
Client Code:
HttpGet request = new HttpGet(SERVICE_URI + "/GetPlates");
request.setHeader("Accept", "application/json");
request.setHeader("Content-type", "application/json");
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpResponse response = httpClient.execute(request);
HttpEntity responseEntity = response.getEntity();
char[] buffer = new char[(int)responseEntity.getContentLength()];
InputStream stream = responseEntity.getContent();
InputStreamReader reader = new InputStreamReader(stream);
reader.read(buffer);
stream.close();
JSONArray plates = new JSONArray(new String(buffer));
Server Config:
<webHttpBinding>
<binding name="big_webHttpBinding" maxReceivedMessageSize="4097152" maxBufferSize="4097152" maxBufferPoolSize="4097152">
<readerQuotas maxStringContentLength="4097152" maxArrayLength="4097152" maxBytesPerRead="4097152" maxNameTableCharCount="4097152" maxDepth="4097152"/>
</binding>
</webHttpBinding>
Instead of
char[] buffer = new char[(int)responseEntity.getContentLength()];
InputStream stream = responseEntity.getContent();
InputStreamReader reader = new InputStreamReader(stream);
reader.read(buffer);
stream.close();
do this:
String jsonText = EntityUtils.toString(responseEntity, HTTP.UTF_8);

Categories