Apache html response returns gibberish - java

I'm trying to get an HTML response from a remote website, and I get something like this :
ס×?×? ×?×? ×?×? ×?×?
instead of Hebrew letters or symbols.
Here is my code:
CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultCookieStore(cookieStore)
.build();
HttpGet httpget = new HttpGet(URL);
CloseableHttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
String s=null;
if (entity != null) {
s= EntityUtils.toString(entity);
}
Does anyone know what the problem is?

As per the docs,
The content is converted using the character set from the entity (if any), failing that, "ISO-8859-1" is used.
The default charset is being used because you don't provide one, which doesn't map those characters correctly - you should probably use UTF-8 instead. Try this.
s= EntityUtils.toString(entity, "UTF-8");

Related

how to get json object from HttpEntity?

I have an HttpEntity object created as below:
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpGet get = new HttpGet(new URI(currentUrl));
HttpResponse resp = httpclient.execute(get);
HttpEntity entity = resp.getEntity();
String respbody = EntityUtils.toString(entity);
JSONObject jsonobj = new JSONObject(respbody);
Result:
org.json.JSONException: A JSONObject text must begin with '{' at character 2
Observation:
respbody string when printed is not the same text returned from currentUrl, it includes some non ascii characters.
I tried adding charset to tostring method but no luck!
It will be very helpful if anyone can suggest why this string is not normal text?

Posting to REST api using Java, specifically Android

I'm posting to the Wufoo api inside of an Android app and I am hitting a bit of a snag. My data does not seem to be formatting in a way that the server likes (or there is some other issue). Here is my code (note authkey and authpass are placeholders in the exmaple):
HttpClient httpclient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
String json = "";
JSONObject jsonObject = new JSONObject();
jsonObject.accumulate("Field17", "Some Value");
json = jsonObject.toString();
StringEntity postData = new StringEntity(json, "UTF8");
httpPost.setEntity(postData);
String authorizationString = "Basic " + Base64.encodeToString(
("authkey" + ":" + "authpass").getBytes(),
Base64.NO_WRAP);
httpPost.setHeader("Content-type", "application/json");
httpPost.setHeader("Authorization", authorizationString);
HttpResponse httpResponse = httpclient.execute(httpPost);
inputStream = httpResponse.getEntity().getContent();
The response I get back from the server looks like this:
{"Success":0,"ErrorText":"Errors have been <b>highlighted<\/b> below.","FieldErrors":
[{"ID":"Field17","ErrorText":"This field is required. Please enter a value."}]}
This is the response for a failure (obviously) which leads me to believe I'm doing the authentication correctly, and that it just doesn't like my JSON string, I've looked through the API docs which are located here:
http://www.wufoo.com/docs/api/v3/entries/post/
and by all accounts this should work? Any suggestions?
I would start by looking at this line:
StringEntity postData = new StringEntity(json, "UTF8");
It's "UTF-8", not "UTF8".
Note: I would suggest you using the HTTP.UTF_8 constant in order to avoid this kind of problem again.
StringEntity postData = new StringEntity(json, HTTP.UTF_8);
The Field17 may be of specific field type other than string.
After reading the document, I think you missed the point. The server accepted fields parameter from http post, not from a json string.
Your problem looks like this one.
So your request should like this:
ArrayList<NameValuePair> postParameters = new ArrayList<NameValuePair>();
postParameters.add(new BasicNameValuePair("Field17", "Some Value"));
httpPost .setEntity(new UrlEncodedFormEntity(postParameters));
Hope this can help.
I actually figured this out, this isn't a problem with the code anyone here gave me, it's the fact that I was sending the wrong header info. This must be a quirk of the Wufoo API.
If I use the BasicNameValuePair objects like what was suggestion by R4j and I remove the line
httpPost.setHeader("Content-type", "application/json");
everything works perfectly!
Thanks for all the help and I hope this helps anyone who is having trouble with the Wufoo API and Java.

Space between http url in java

This code working fine
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("http://localhost:8090/Servlet/ServletFirst?to=1234&from=567&text=testtest");
If i use space between parameter value. It throws exception
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("http://localhost:8090/Servlet/ServletFirst?to=1234&from=567&textParam=test test");
space between test test throws error. How to resolve?
You must URL encode the parameter in your URL; use %20 instead of the space.
HttpGet request = new HttpGet("http://localhost:8090/Servlet/ServletFirst?to=1234&from=567&textParam=test%20test");
Java has a class to do do URL encoding for you, URLEncoder:
String param = "test test";
String enc = URLEncoder.encode(param, "UTF-8");
String url = "http://...&textParam=" + enc;
Just use a %20 to represent a space.
This is all part of the URL encoding: http://www.w3schools.com/tags/ref_urlencode.asp
So you would want:
HttpGet request = new HttpGet("http://localhost:8090/Servlet/ServletFirst?to=1234&from=567&text=test%20test");
Use
URLEncoder.encode("test test","UTF-8")
So change your code to
HttpGet request = new HttpGet("http://localhost:8090/Servlet/ServletFirst?to=1234&from=567&textParam="+URLEncoder.encode("test test","UTF-8"));
Note Don't Encode Whole url
URLEncoder.encode("http://...test"); // its Wrong because it will also encode the // in http://
Use %20 to indicate space in the URL, as space is not an allowed character. See the Wikipedia entry for character data in a URL.
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(
"http://localhost:8090/Servlet/ServletFirst?to=1234&from=567&textParam=test%20test");

URL Encoding with httpclient

I have a list of URLs which I need to get the content of.
The URL is with special characters and thus needs to be encoded.
I use Commons HtpClient to get the content.
when I use:
GetMethod get = new GetMethod(url);
I get a " Invalid "illegal escape character" exception.
when I use
GetMethod get = new GetMethod();
get.setURI(new URI(url.toString(), false, "UTF-8"));
I get 404 when trying to get the page, because a space is turned to %2520 instead of just %20.
I've seen many posts about this problem, and most of them advice to build the URI part by part. The problem is that it's a given list of URLs, not a one that I can handle manually.
Any other solution for this problem?
thanks.
What if you create a new URL object from it's string like URL urlObject = new URL(url), then do urlObject.getQuery() and urlObject.getPath() to split it right, parse the Query Params into a List or a Map or something and do something like:
EDIT: I just found out that HttpClient Library has a URLEncodedUtils.parse() method which you can use easily with the code provided below. I'll edit it to fit, however is untested.
With Apache HttpClient it would be something like:
URI urlObject = new URI(url,"UTF-8");
HttpClient httpclient = new DefaultHttpClient();
List<NameValuePair> formparams = URLEncodedUtils.parse(urlObject,"UTF-8");
UrlEncodedFormEntity entity;
entity = new UrlEncodedFormEntity(formparams);
HttpPost httppost = new HttpPost(urlObject.getPath());
httppost.setEntity(entity);
httppost.addHeader("Content-Type","application/x-www-form-urlencoded");
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity2 = response.getEntity();
With Java URLConnection it would be something like:
// Iterate over query params from urlObject.getQuery() like
while(en.hasMoreElements()){
String paramName = (String)en.nextElement(); // Iterator over yourListOfKeys
String paramValue = yourMapOfValues.get(paramName); // replace yourMapOfNameValues
str = str + "&" + paramName + "=" + URLEncoder.encode(paramValue);
}
try{
URL u = new URL(urlObject.getPath()); //here's the url path from your urlObject
URLConnection uc = u.openConnection();
uc.setDoOutput(true);
uc.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
PrintWriter pw = new PrintWriter(uc.getOutputStream());
pw.println(str);
pw.close();
BufferedReader in = new BufferedReader(new
InputStreamReader(uc.getInputStream()));
String res = in.readLine();
in.close();
// ...
}
If you need to manipulate with request URIs it is strongly advisable to use URIBuilder shipped with Apache HttpClient.
try it out
GetMethod get = new GetMethod(url.replace(" ","%20")).toASCIIString());
Please use the URLEncoder class.
I used it in an exact scenario and it worked just fine for me.
What I did is to use the URL class, to get the part that comes after the host
(for example - at www.bla.com/mystuff/bla.jpg this would be "mystuff/bla.jpg" - you should URLEncode only this part, and then consturct the URL again.
For example, if the orignal string is "http://www.bla.com/mystuff/bla foo.jpg" then:
Encode - "mystuff/bla foo.jpg" and get "mystuff/bla%20foo.jpg" and then attach this to the host and protocol parts:
"http://www.bla.com/mystuff/bla%20foo.jpg"
I hope this helps

IllegalStateException: Content has been consumed - How to resolve?

I have the following code:
HttpGet httpGet = new HttpGet(serverAdress + "/rootservices");
httpGet.setHeader("Accept", "text/xml");
HttpResponse response = client.execute(httpGet, localContext);
String projectURL = XMLDocumentParser.parseDocument(response.getEntity().getContent(), "oslc_scm:scmServiceProviders", "rdf:resource");
String workItemURL = XMLDocumentParser.parseDocument(response.getEntity().getContent(), "oslc_cm:cmServiceProviders", "rdf:resource");
The problem here is that I read two times the HttpResponse object. So the second time I get the exception. But although I know the problem, I can´t find an easy solution. So what is a good way to solve that problem?
Read the input stream returned by response.getEntity().getContent() into a byte[], stored in a local variable. See Convert InputStream to byte array in Java.
byte[] content = IOUtils.toByteArray(response.getEntity().getContent());
String projectURL = XMLDocumentParser.parseDocument(new ByteArrayInputStream(content), "oslc_scm:scmServiceProviders", "rdf:resource");
String workItemURL = XMLDocumentParser.parseDocument(new ByteArrayInputStream(content), "oslc_cm:cmServiceProviders", "rdf:resource");
Read the response content only once, and copy it to some kind of buffer compatible with XMLDocumentParser.parseDocument. Then parse the data directly from your buffer, as many times as you want.
Why don't you try it this way?
HttpGet httpGet = new HttpGet(serverAdress + "/rootservices");
httpGet.setHeader("Accept", "text/xml");
InputStream in = null;
HttpResponse response = client.execute(httpGet, localContext);
in = response.getEntity().getContent();
String projectURL = XMLDocumentParser.parseDocument(in, "oslc_scm:scmServiceProviders", "rdf:resource");
String workItemURL = XMLDocumentParser.parseDocument(in, "oslc_cm:cmServiceProviders", "rdf:resource");

Categories