We Keep Coding

How to extract document id from response - ElasticSearch Java Low Level Rest Client

I'm indexing document without id into ElasticSearch using the following code:
Response response = restClient.performRequest(
HttpPost.METHOD_NAME,
"/posts/doc/",
Collections.emptyMap(),
entity);
I want to extract the document id that was generated by ElasticSearch from the response. Is there any way to do that?

You need to read the response object for that response.getEntity().getContent().
If you are using Jackson, you can then deserialize the stream as a Map mapper.readValue(response.getEntity().getContent(), new TypeReference<Map<String, Object>>(){});
Then from the map, read the _id field.
Hope this helps.

response.getEntity().getContent() is InputStream object.
So, you could you BufferedReader also.
Try this:
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuilder out = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
out.append(line);
}
System.out.println(out.toString());

Java: Reading from a URL produces gibberish

So I've been trying to read the html code from kickass.to(it works fine on other sites) but all I get is some weird gibberish.
My code:
BufferedReader in = new BufferedReader(
new InputStreamReader(new URL("http://kickass.to/").openStream()));
String s = "";
while ((s=in.readLine())!=null) System.out.println(s);
in.close();
For example:
Does anyone knows why it does that?
thanks!

The problem here is a server that is probably not configured correctly, as it returns its response gzip compressed, even if the client does not send an Accept-Encoding: gzip header.
So what you're seeing is the compressed version of the page. To decompress it, pass it through a GZIPInputStream:
BufferedReader in = new BufferedReader(
new InputStreamReader(
new GZIPInputStream(new URL("http://kickass.to/").openStream())));

HttpURLConnection response is incorrect

When using this code below to make a get request:
private String get(String inurl, Map headers, boolean followredirects) throws MalformedURLException, IOException {
URL url = new URL(inurl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setInstanceFollowRedirects(followredirects);
// Add headers to request.
Iterator entries = headers.entrySet().iterator();
while (entries.hasNext()) {
Entry thisEntry = (Entry) entries.next();
Object key = thisEntry.getKey();
Object value = thisEntry.getValue();
connection.addRequestProperty((String)key, (String)value);
}
// Attempt to parse
InputStream stream = connection.getInputStream();
InputStreamReader isReader = new InputStreamReader(stream );
BufferedReader br = new BufferedReader(isReader );
System.out.println(br.readLine());
// Disconnect
connection.disconnect();
return connection.getHeaderField("Location");
}
The resulting response is completely nonsensical (e.g ���:ks�6��﯐9�rђ� e��u�n�qש�v���"uI*�W��s)
However I can see in Wireshark that the response is HTML/XML and nothing like the string above. I've tried a myriad of different methods for parsing the InputStream but I get the same result each time.
Please note: this only happens when it's HTML/XML, plain HTML works.
Why is the response coming back in this format?
Thanks in advance!
=== SOLVED ===
Gah, got it!
The server is compressing the response when it contains XML, so I needed to use GZIPInputStream instead of InputSream.
GZIPInputStream stream = new GZIPInputStream(connection.getInputStream());
Thanks anyway!

use an UTF-8 encoding in input stream like below
InputStreamReader isReader = new InputStreamReader(stream, "UTF-8");

Using Perl data on server to display on Android

I am developing an Android application. I am calling a Perl file on a server. This Perl file has different print statements.
I want to make the collective text available to a variable in android Java file of mine.
I have tried this :
URL url= new URL("http://myserver.com/cgi-bin/myfile.pl?var=97320");
here goes my request to the server file. But how can i get the data from the Perl file available there?

In your perl service:
use CGI qw(param header);
use JSON;
my $var = param('var');
my $json = &fetch_return_data($var);
print header('application/json');
print to_json($json); # or encode_json($json) for utf-8
to return data as a JSON object. Then use one of many JSON libraries for Java to read the data. For instance http://json.org/java/:
Integer var = 97320;
InputStream inputStream = new URL("http://myserver.com/cgi-bin/myfile.pl?var=" + var).openStream();
try {
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
// Or this if you returned utf-8 from your service
//BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream, Charset.forName("UTF-8")));
JSONObject json = new JSONObject(readAll(bufferedReader));
} catch (Exception e) {
}

Reading JSON data from the net (Twitter)

I found this great tutorial on how to use JSON to retrieve Twitter updates, and post it in a TextView:
http://www.ibm.com/developerworks/xml/library/x-andbene1/
I've followed this tutorial step by step, so my code is the same.
In the method examineJSONFile(), we have this line:
InputStream is = this.getResources().openRawResource(R.raw.jsontwitter);
This file is downloaded directly from the Twitter website, as mentioned in the second paragraph of http://www.ibm.com/developerworks/xml/library/x-andbene1/#aotf.
All this is great, except for one thing: it's absolutely no use that one has to download the Twitter updates (tweets) and then build the app using this as a raw file. It should be possible to download this JSON file at runtime, and then show the tweets in the TextView afterwards.
I have tried to create the InputStream in another way, like this:
String url = "http://twitter.com/statuses/user_timeline/bbcnews.json";
HttpClient httpclient = new DefaultHttpClient();
HttpResponse response = httpclient.execute(new HttpGet(url));
InputStream is = response.getEntity().getContent();
BufferedReader br = new BufferedReader(new InputStreamReader(is,"UTF-8"));
StringBuffer sb = new StringBuffer();
try
{
String line = null;
while ((line = br.readLine())!=null)
{
sb.append(line);
sb.append('\n');
}
}
catch (IOException e)
{
e.printStackTrace();
}
String jsontext = new String(sb.toString());
But it seems this line: HttpResponse response = httpclient.execute(new HttpGet(url)); throws an exception.
Any help please?

You seem to be missing the INTERNET permission. Look at the logs and it would be clear what exactly is the problem.