We Keep Coding

How to translate Elastic Search query in Java by using the query builder?

I am trying to translate the following elastic search query in Java, using the query builder? Can someone give any ideas of it?
GET <index-name>/_search
{
“size”: 1,
“sort”: [
{
“Date”: {
“order”: “desc”
}
}
]
}

You need to use SortBuilder to sort on fields. Below request will sort Date field in desc order
RestHighLevelClient client = new RestHighLevelClient(restClientBuilder);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.size(1).sort(new FieldSortBuilder("Date").order(SortOrder.DESC));
SearchRequest searchRequest = new SearchRequest("my-index");
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

Elasticsearch - Terms Aggregation nested field

I have following problem. I have a nested field ("list") with 2 properties (fieldB & fieldC).
This is how a document looks like:
"fieldA: "1",
"list": [
{"fieldB": "ABC",
"fieldC": "DEF"},
{"fieldB": "ABC",
"fieldC": "GHI"},
{"fieldB": "UVW",
"fieldC": "XYZ"},...]
},
I want to get a distinct list of all possible fieldC values for "ABC" (fieldB) over all documents. So far I've tried this in Java (Java REST Client):
SearchRequest searchRequest = new SearchRequest("abc*");
QueryBuilder matchQueryBuilder = QueryBuilders.boolQuery()
.must(QueryBuilders.nestedQuery("aList",
QueryBuilders.matchQuery("list.fieldB.keyword", "ABC"), ScoreMode.None));
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(matchQueryBuilder)
.aggregation(AggregationBuilders.nested("listAgg","list")
.subAggregation(AggregationBuilders.terms("fieldBAgg")
.field("list.fieldB.keyword")));
searchRequest.source(sourceBuilder);
SearchResponse searchResponse = null;
try {
searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
e.printStackTrace();
}
Nested list = searchResponse.getAggregations().get("listAgg");
Terms fieldBs = list.getAggregations().get("fieldBAgg");
With that query I get all documents which include "ABC" in fieldB and I get all fieldC values. But I just want the fieldC values where fieldB is "ABC".
So in that example I get DEF, GHI and XYZ. But i just want DEF and GHI. Does anybody have an idea how to solve this?

The nested constraint in the query part will only select all documents that do have a nested field satisfying the constraint. You also need to add that same constraint in the aggregation part, otherwise you're going to aggregate all nested fields of all the selected documents, which is what you're seeing. Proceed like this instead:
// 1. terms aggregation on the desired nested field
nestedField = AggregationBuilders.terms("fieldBAgg").field("list.fieldC.keyword");
// 2. filter aggregation on the desired nested field value
onlyBQuery = QueryBuilders.termQuery("list.fieldB.keyword", "ABC");
onlyBFilter = AggregationBuilders.filter("onlyFieldB", onlyBQuery).subAggregation(nestedField);
// 3. parent nested aggregation
nested = AggregationBuilders.nested("listAgg", "list").subAggregation(onlyBFilter);
// 4. main query/aggregation
sourceBuilder.query(matchQueryBuilder).aggregation(nested);

How to remove duplicate search result in elasticsearch when using scroll search?

I am developing a project that integrates Java and Elasticsearch.
And I am using scroll api because of searching a large amount of data.
I want to see unique result( like distint in oracle).
How to remove duplicate search result in elasticsearch?
I searched, but couldn't find the Java version.
My code is like this (this is a just sample code):
final Scroll scroll = new Scroll(TimeValue.timeValueMinutes(1L));
SearchRequest searchRequest = new SearchRequest("posts");
searchRequest.scroll(scroll);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(matchQuery("title", "Elasticsearch"));
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
String scrollId = searchResponse.getScrollId();
SearchHit[] searchHits = searchResponse.getHits().getHits();
while (searchHits != null && searchHits.length > 0) {
SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
scrollRequest.scroll(scroll);
searchResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT);
scrollId = searchResponse.getScrollId();
searchHits = searchResponse.getHits().getHits();
}
Is there any way to search the data on the elastic without duplication?

Scroll will return all the documents in elasticsearch. You cannot perform distinct operation in elasticsearch
There are two ways to resolve your issue
Field collapsing
It does a group by on different fields and return top 1 document
Terms Aggregation
{
"aggs": {
"t": {
"terms": {
"script": "doc['title.keyword'] + ' '+doc['description.keyword']"
}
}
}
}
Scroll is intended to fetch large number of documents and above options are not for bulk data. So you need to perform distinct operation client side(outside of elastic search)

How can I use avg aggregation for keyword typed field in elasticsearch

Hi tried some solutions but due to lack of elasticsearch experience i could concluded.
First of all I have index like this;
field1 | field2 | **status: 200, response_time: 2** (type of this row keyword)
//other rows omitted for brevity
My business requirement is averaging response_time seconds. I want to parse status: 200, response_time: 2 row and get time from with regexp then do averaging.
I tried it like:
private RestHighLevelClient client;
public void averaging() {
SearchRequest searchRequest = new SearchRequest("index_name");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
AvgAggregationBuilder aggregation = AggregationBuilders.avg("response_time_average")
.field("doc['Message.keyword']").script(
new Script(
"def value = doc['Message.keyword']; return Integer.parseInt(value.split('regexp'))"));
searchSourceBuilder.aggregation(aggregation);
searchRequest.source(searchSourceBuilder);
try {
SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
Avg agg = search.getAggregations().get("response_time_average");
double value = agg.getValue();
//value returns as _infinity_
} catch (IOException e) {
log.error(e.getMessage(), e);
}
}
I don't have enough information about elasticsearch jargon. I tried a lot of solutions but I think this one is more close to solution.
My elasticservice client version is 7.3.1
thank you :)

Elasticsearch query through Java API

I am using the following elasticsearch query to fetch the details,
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"bool": {
"should": [
{"match": {
"val": "GET"
}}]
}
}
}
}
}
It is working fine and given the result as required.
I want to execute the same query through java and get the same results and tried the following,
getClient().prepareSearch(esIndex)
.setQuery(QueryBuilders.queryStringQuery(QUERY)).execute().actionGet();
It is not returning the data and throw some query format wrong exception as well.
Is there any Java API available using which the same query can be executed?
NOTE: There is a possibility available to create the boolquery and aggregation builders in java api and execute the same. I am just curious to find a way to execute this query directly through elasticsearch java api

If you really want to use the Query String Query, your query has to follow Query String Syntax:
getClient().prepareSearch(esIndex)
.setQuery(QueryBuilders.queryStringQuery("val: \"GET\""))
.execute()
.actionGet();
As already stated, you should construct your query by using the provided QueryBuilders instead of strings. This will keep your code clean and readable even for complex queries.
getClient().prepareSearch(esIndex)
.setQuery(QueryBuilders.boolQuery()
.should(QueryBuilders.matchQuery("val", "GET"))
.execute()
.actionGet();

BoolQueryBuilder bool = boolQuery();
bool.must(QueryBuilders.matchAllQuery());
bool.filter(QueryBuilders.boolQuery().should(QueryBuilders.matchQuery("Val", "GET")));
AggregationBuilder agg = AggregationBuilders.terms("").field("");
SearchResponse reponse = getClient().prepareSearch().setIndices("indexName").setTypes("indexType")
.setQuery(bool).addAggregation(agg).execute().actionGet();

you should use boolQuery() when you construct your QueryBuilder:
QueryBuilder qb = boolQuery()
.must(termQuery("content", "test1"))
.must(termQuery("content", "test4"))
.mustNot(termQuery("content", "test2"))
.should(termQuery("content", "test3"))
.filter(termQuery("content", "test5"));
Official docs:
https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-compound-queries.html