I'm using org.elasticsearch.client.RestHighLevelClient to get data from elasticsearch.
I want to know is it possible to get all documents using RestHighLevelClient for given index?
like http://localhost:9200/test/_search?
It is definitely possible.
First of all you need to initialize the client
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("localhost", 9200, "http"),
new HttpHost("localhost", 9201, "http")));
then you need to execute a search query.
If you would like to fetch all docs you will have to use the scrolling API.
You can find a complete example here.
If you do not need all, you can simply use the search API.
And don't forget to close the connection when the work is done
client.close();
Related
I'm writing a unit test where I need to write to an Elasticsearch 6 index using a RestHighLevelClient in the Java Elasticsearch 6 library, then read from the index. How can I know when the results of an IndexRequest are ready to be read from the index via RestHighLevelClient.search? For example:
RestHighLevelClient client;
//client initialization
BulkRequest request = new BulkRequest();
request.add(new IndexRequest(...));
BulkResponse response = client.bulk(request);
//process response
SearchRequest request = new SearchRequest(...);
SearchResponse scrollResponse = client.search(request);
//scrollResponse is empty!
Basically, if I put a Thread.sleep between the write and the read, the response has the content I wrote, so I think the requests are being made properly. Is there a way I can be sure to wait until the client.bulk(request) part has completely finished writing before I do the read operation?
This will force a refresh as part of this request.
request.setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE);
Figured it out. If anyone in the future happens to have this very specific problem, you need to include:
client.refreshIndex(indexName)
In between the write and the read. Elasticsearch refreshes by default every 1 second, but you can do this explicitly as well if you need to read <1 second after writing.
For example: I can get 'max_result_window' value using an HTTP request, like http://esIp:9200/index/_settings.
But I want to get this value in Java runtime environment using [elasticsearch client, Java Api]. I look through the official document but result in finding no relevant infomation about it.
So, How can I get 'max_result_window' value using [Elasticsearch Clients, Java API]?
If you are talking about native transport client you can use admin API. Here is the example:
TransportClient client = new PreBuiltTransportClient(Settings.EMPTY)
.addTransportAddress(new TransportAddress(InetAddress.getLoopbackAddress(), 9300));
GetSettingsResponse response = client.admin().indices().prepareGetSettings("test").get();
String maxResultWindow = response.getSetting("test", "index.max_result_window");
I tried to use the example in here:
https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-search-scrolling.html
on how to use scroll with java in elasticsearch.
this is the code:
QueryBuilder qb = termQuery("multi", "test");
SearchResponse scrollResp = client.prepareSearch("test")
.addSort(FieldSortBuilder.DOC_FIELD_NAME, SortOrder.ASC)
.setScroll(new TimeValue(60000))
.setQuery(qb)
.setSize(100).get(); //max of 100 hits will be returned for each scroll
//Scroll until no hits are returned
do {
for (SearchHit hit : scrollResp.getHits().getHits()) {
//Handle the hit...
}
scrollResp = client.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new TimeValue(60000)).execute().actionGet();
} while(scrollResp.getHits().getHits().length != 0); // Zero hits mark the end of the scroll and the while loop.
though for some reasons I have an error which says The method prepareSearch(String) is undefined for the type RestHighLevelClient.
my client variable is indeed RestHighLevelClient but in the tutorial it is what it should be.
ant ideas what is the problem?
RestHighLevelClient works differently than a TransportClient.
Following are the steps you must follow if you wish to use scroll with RestHighLevelClient:
Create a SearchRequest:
SearchRequest request = new SearchRequest("test").scroll(new TimeValue(60000));
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(qb);
searchSourceBuilder.sort(FieldSortBuilder.DOC_FIELD_NAME, SortOrder.ASC);
request.source(searchSourceBuilder);
Perform the first search:
SearchResponse scrollResp = client.search(sreq);
here client is the RestHighLevelClient.
For subsequent scroll searches create a SearchScrollRequest and then use it for scroll:
scrollResp = client.searchScroll(new SearchScrollRequest(scrollResponse.getScrollId()).scroll(new TimeValue(60000)));
For more information refer :Search Scroll API
From elasticsearch 6 there are two apis
One is Rest Api
One is transport api.
Error says that you have used client of REST Api and Code of TRANSPORT Api.
You need to use this Client Api : https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html
But it would be beneficial if you use REST api as elasticsearch will remove TRANSPORT Api in future.
Here is scroll request for REST Api : https://www.elastic.co/guide/en/elasticsearch/client/java-rest/6.3/java-rest-high-search-scroll.html
My Guidelines
If followed this Google documentation about verifying Google-Account-Tokens on the server side, but I am kinda confused.
My Problem
GoogleIdTokenVerifier googleIdTokenVerifier = new GoogleIdTokenVerifier.Builder(new NetHttpTransport(), new JacksonFactory())
.setAudience(Collections.singletonList(CLIENT_ID))
.build();
In this piece of code I figured out that the transport and jsonFactory arguments can be filled as new NetHttpTransport() and new JacksonFactory() here. It also describes how to get AudienceString, but I couldn't figure out what it is for. I couldn't test it, but my question is if I can use it without .setAudience() or if I need it and what it is for.
In .setAudience() you have to pass all client ID's. You can get the ID for your client from the Credentials Page. It's explained here.
Thanks to #StevenSoneff.
If you didn't get the basic concept
For every client you want your server to accept, you need to create a project in the `Developer Console`. Clients are differentiated by their `SHA-1` fingerprint. You can for example have a debug project (will take your debug fingerprint) and a release one. To make both work, you have to add both `ID`'s to your server's `GoogleIdTokenVerifier`'s `.setAudience()`.
In my case, If you're using Firebase to get the id token on Android or iOS. You should follow these instructions to verify it on your backend server.
Verify ID tokens using a third-party JWT library
For me, I'm using Google OAuth Client as the third-party library so it's easy to use.
But it's a little bit different from this document.
Verify the Google ID token on your server side
The CLIENT_ID is your firebase project ID.
The Issuer has to be set as https://securetoken.google.com/<projectId>.
You need to use GooglePublicKeysManager and call setPublicCertsEncodedUrl to set it as https://www.googleapis.com/robot/v1/metadata/x509/securetoken#system.gserviceaccount.com
GooglePublicKeysManager manager = new GooglePublicKeysManager.Builder(HTTP_TRANSPORT, JSON_FACTORY)
.setPublicCertsEncodedUrl(PUBLIC_KEY_URL)
.build();
GoogleIdTokenVerifier verifier = new GoogleIdTokenVerifier.Builder(manager)
.setAudience(Collections.singletonList(FIREBASE_PROJECT_ID))
.setIssuer(ISSUER)
.build();
If you have multiple issuers, then you have to create GoogleIdTokenVerifier for each one.
I'm trying to connect from Java to ElasticSearch but I can only connect over HTTP. I can't use the TransportClient. Is there a Java client wrapper around the ElasticSearch REST APIs? If so, how do I use it?
Hi There is a brand new project just matching your needs. It Java based Rest API for Elasticsearch
Check it out! its name JEST
A new "official" REST-based java client will be available starting with v5.0.0-alpha4.
We just open sourced Flummi, a Java HTTP/REST client for Elastic Search. It imitates the transport client's API as closely as possible, making it easy to port existing code. It also provides a better abstraction level than Jest, because it reports all the errors with Exceptions. Give it a try!
Simple usage example:
Flummi flummi = new Flummi("http://elasticsearch.base.url:9200");
SearchResponse searchResponse = flummi
.prepareSearch("products")
.setQuery(
QueryBuilders.termQuery("color", "yellow").build()
)
.execute();
System.out.println("Found "
+ searchResponse.getHits().getTotalHits()
+ " products");
searchResponse.getHits()
.stream().map(hit -> hit.getSource().get("name").getAsString())
.forEach(name -> System.out.println("Name: " + name));
Since version 5.6 of the Elasticsearch Java SDK they provide a Java REST Client.
RestClient restClient = RestClient.builder(
new HttpHost("localhost", 9200, "http"),
new HttpHost("localhost", 9201, "http")).build();
// for the RestHighLevelClient
RestHighLevelClient client =
new RestHighLevelClient(restClient);