We Keep Coding

Preserve a HTTPEntity without consuming the stream

I'm using org.apache.http.client.HttpClient and I'm trying to access the payload of the request HTTPEntity without consuming the underlying stream.
I tried using
EntityUtils.toString(someEntity);
but this consumes the stream.
I just want to preserve the payload which was sent in a HTTP request to a String object for e.g.
Sample Code:
String uri = "someURI";
HttpPut updateRequest = new HttpPut(uri);
updateRequest.setEntity(myHttpEntity);
Any hint appreciated.

A HttpEntity must be repeatable for it to be repeatedly consumable. The method isRepeatable() shows whether or not this is the case.
Two entities are repeatable:
StringEntity
ByteArrayEntity
This means you have to add one of these to the original request so you can keep using using its content.
public void doExample() {
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpPut httpPut = new HttpPut("some_url");
httpPut.setHeader(CONTENT_TYPE, ContentType.APPLICATION_JSON.toString());
StringEntity jsonEntityOrso = new StringEntity("{ \"hello\": \"some message\" }");
httpPut.setEntity(jsonEntityOrso);
StringEntity reusableEntity = (StringEntity) httpPut.getEntity();
String hello = readInputStream(reusableEntity.getContent());
String hello2 = readInputStream(reusableEntity.getContent());
boolean verify = hello.equals(hello2); // returns true
}
private String readInputStream(InputStream stream) {
return new BufferedReader(
new InputStreamReader(stream, StandardCharsets.UTF_8))
.lines()
.collect(Collectors.joining("\n"));
}

Apache html response returns gibberish

I'm trying to get an HTML response from a remote website, and I get something like this :
ס×?×? ×?×? ×?×? ×?×?
instead of Hebrew letters or symbols.
Here is my code:
CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultCookieStore(cookieStore)
.build();
HttpGet httpget = new HttpGet(URL);
CloseableHttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
String s=null;
if (entity != null) {
s= EntityUtils.toString(entity);
}
Does anyone know what the problem is?

As per the docs,
The content is converted using the character set from the entity (if any), failing that, "ISO-8859-1" is used.
The default charset is being used because you don't provide one, which doesn't map those characters correctly - you should probably use UTF-8 instead. Try this.
s= EntityUtils.toString(entity, "UTF-8");

HTTP request not returning desired json

I am trying to send a http request to a website which is supposed to return a json response. The problem is that i am not getting the json data. But when i paste the url in a browser it displays the json output. Am a newbie. Kindly help.
Here is my code
HttpClient client = new DefaultHttpClient();
String url="http://directclientvendors.com/news24/api/get.php?type=news";
HttpGet request = new HttpGet(url);
HttpResponse response;
response = client.execute(request);
BufferedReader br =
new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line = "";
while(br.ready())
{
line+=br.readLine();
}
System.out.println("line "+line);

You should be executing a GET request and not a POST. Please change the request type to HttpGet. The browser executes a GET on the URL when you paste it on the address bar and hit enter.
Additionally use a Reader + StringBuilder / JsonReader / GSON to read from the URL's response content. String concatenation leads to the creation of additional objects unnecessarily.
[EDIT]
To my astonishment the API call works even when a POST call is made to get the resource. The problem must be in your parsing logic. Using a JsonReader works fine for me. This is just template code, but you can fill in the rest to get the other JSON elements. Regardless of whether POST works or not, you should still use GET for this call.
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("http://directclientvendors.com/news24/api/get.php?type=news");
HttpResponse response = client.execute(request);
InputStream content = response.getEntity().getContent();
JsonReader jsonReader = new JsonReader(new InputStreamReader(content, "UTF-8"));
jsonReader.beginObject();
if(jsonReader.hasNext())
{
System.out.println(jsonReader.nextName()); // prints 'news'
// BEGIN_ARRAY etc to parse the rest
}
// END_OBJECT and cleanup

Removing White spaces from a string for http connection

I am trying to send a query url
String url = String.format(
"http://xxxxx/xxx/xxx&message=%s",myEditBox.getText.toString());
// Create a new HttpClient and Post Header
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
httpclient.getCookieStore().addCookie(cooki);
try {
ResponseHandler<String> responseHandler = new BasicResponseHandler();
httpclient.getParams().setParameter("http.connection-manager.timeout", 15000);
String response = httpclient.execute(httppost, responseHandler);
gives me error, illegal character at query. That's white space probably. How to deal with this issue?
Best Regards

You need to encode your url.
String query = URLEncoder.encode(myEditBox.getText.toString(), "utf-8");
String url = "http://stackoverflow.com/search?q=" + query;

Can you try
httpclient.setRequestProperty("Accept-Charset","UTF-8");
url="http://xxxxx/xxx/xxx&message="+URLEncoder.encode(myEditBox.getText.toString(), "UTF-8");

Try .trim() while get value from edittext.
May be whitespace come from edittext and also use "utf-8".
see below code.
String value = URLEncoder.encode(myEditBox.getText.toString().trim(), "utf-8");
String url = "http://xxxxx/xxx/xxx&message=%s" + value;

URL Encoding with httpclient

I have a list of URLs which I need to get the content of.
The URL is with special characters and thus needs to be encoded.
I use Commons HtpClient to get the content.
when I use:
GetMethod get = new GetMethod(url);
I get a " Invalid "illegal escape character" exception.
when I use
GetMethod get = new GetMethod();
get.setURI(new URI(url.toString(), false, "UTF-8"));
I get 404 when trying to get the page, because a space is turned to %2520 instead of just %20.
I've seen many posts about this problem, and most of them advice to build the URI part by part. The problem is that it's a given list of URLs, not a one that I can handle manually.
Any other solution for this problem?
thanks.

What if you create a new URL object from it's string like URL urlObject = new URL(url), then do urlObject.getQuery() and urlObject.getPath() to split it right, parse the Query Params into a List or a Map or something and do something like:
EDIT: I just found out that HttpClient Library has a URLEncodedUtils.parse() method which you can use easily with the code provided below. I'll edit it to fit, however is untested.
With Apache HttpClient it would be something like:
URI urlObject = new URI(url,"UTF-8");
HttpClient httpclient = new DefaultHttpClient();
List<NameValuePair> formparams = URLEncodedUtils.parse(urlObject,"UTF-8");
UrlEncodedFormEntity entity;
entity = new UrlEncodedFormEntity(formparams);
HttpPost httppost = new HttpPost(urlObject.getPath());
httppost.setEntity(entity);
httppost.addHeader("Content-Type","application/x-www-form-urlencoded");
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity2 = response.getEntity();
With Java URLConnection it would be something like:
// Iterate over query params from urlObject.getQuery() like
while(en.hasMoreElements()){
String paramName = (String)en.nextElement(); // Iterator over yourListOfKeys
String paramValue = yourMapOfValues.get(paramName); // replace yourMapOfNameValues
str = str + "&" + paramName + "=" + URLEncoder.encode(paramValue);
}
try{
URL u = new URL(urlObject.getPath()); //here's the url path from your urlObject
URLConnection uc = u.openConnection();
uc.setDoOutput(true);
uc.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
PrintWriter pw = new PrintWriter(uc.getOutputStream());
pw.println(str);
pw.close();
BufferedReader in = new BufferedReader(new
InputStreamReader(uc.getInputStream()));
String res = in.readLine();
in.close();
// ...
}

If you need to manipulate with request URIs it is strongly advisable to use URIBuilder shipped with Apache HttpClient.

try it out
GetMethod get = new GetMethod(url.replace(" ","%20")).toASCIIString());

Please use the URLEncoder class.
I used it in an exact scenario and it worked just fine for me.
What I did is to use the URL class, to get the part that comes after the host
(for example - at www.bla.com/mystuff/bla.jpg this would be "mystuff/bla.jpg" - you should URLEncode only this part, and then consturct the URL again.
For example, if the orignal string is "http://www.bla.com/mystuff/bla foo.jpg" then:
Encode - "mystuff/bla foo.jpg" and get "mystuff/bla%20foo.jpg" and then attach this to the host and protocol parts:
"http://www.bla.com/mystuff/bla%20foo.jpg"
I hope this helps