this is my code, where I want to gain pagination ability, for not getting the whole bucket list and not to overload RAM memory. But anyway, it's unclear to me:
what is this actually 'afterKey' for, and what is the use case. Yeah I understand that here, as a key of that 'aggregateAfter' map should be set the field, on which it's going to be aggregated the results, but what about the value of that map, what to set there, this is what I don't understand at all. Please have a look on this code, and introduce the changes that should be done here, in order the pagination works for me.
Am I thinking correct way, that here the response(searchResponse) should contain only the paginated results, or I should do more from here to gain that miracle.
public BucketList getListOfBucket(final BucketListInfo bucketListInfo, int from, int size) {
final SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
for (final String aggrField : bucketListInfo.getAggrFieldList()) {
CompositeAggregationBuilder aggregationBuilder = AggregationBuilders
.composite(aggrField, List.of(new TermsValuesSourceBuilder(aggrField).field(aggrField)))
.aggregateAfter(Map.of(aggrField, aggrField))
.size(bucketListInfo.getTopResultsCount());
searchSourceBuilder
.from(from)
.size(size)
.aggregation(aggregationBuilder);
}
final SearchRequest searchRequest = new SearchRequest(bucketListInfo.getIndexName())
.source(searchSourceBuilder);
try {
final SearchResponse response = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
// here with the response all buckets are coming, instead of only the specified amount(as pagination 'size', and 'from') of buckets to come
return extractBucketsFromResponse(bucketListInfo, response);
} catch (Exception e) {
log.error(e.getMessage(), e);
return null;
}
}
Related
I have a Query for getting lastSeenTime only for one user
but what I need is to get a map of ids by their last seen for a list of users in elastic search
can somebody help me with converting this query to find last seen of a list of users ssoIds?
static Map<String, Object> getLastSeen(String ssoId) {
SearchResponse response = transportClient.prepareSearch(ChatSettings.ELASTIC_LAST_SEEN_INDEX_NAME)
.setTypes(ChatSettings.ELASTIC_DB_NAME)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(QueryBuilders.idsQuery().addIds(ssoId))
.setFrom(0).setSize(1).setExplain(true)
.get();
checkResponse(response);
Map<String, Object> result = null;
if (response.getHits().getTotalHits() > 0) {
result = response.getHits().getAt(0).getSource();
}
return result;
}
actually I want something like this
static Map<String, Object> getLastSeens(List<String> ssoIdList)
{
//elsticQuery
}
You can use fetch in your elastic query to return only selected fields:
.setFetchSource(new String[]{"field1","field2}, null)
And for passing multiple IDs, you can pass the Array of ids to the idsQuery()
So, in your case it will become:
SearchResponse response = transportClient
//.prepareSearch(ChatSettings.ELASTIC_LAST_SEEN_INDEX_NAME) // you might need to pass the columns here
.setTypes(ChatSettings.ELASTIC_DB_NAME)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setFetchSource(new String[]{"id","lastSeenTime"}, null) // or you can pass the columns here
.setQuery(QueryBuilders.idsQuery().addIds(ssoIds)) // where ssoIds is a array of Ids
.setFrom(0).setSize(1).setExplain(true)
.get();
post this, rest of the code will work as it is.
Hi tried some solutions but due to lack of elasticsearch experience i could concluded.
First of all I have index like this;
field1 | field2 | **status: 200, response_time: 2** (type of this row keyword)
//other rows omitted for brevity
My business requirement is averaging response_time seconds. I want to parse status: 200, response_time: 2 row and get time from with regexp then do averaging.
I tried it like:
private RestHighLevelClient client;
public void averaging() {
SearchRequest searchRequest = new SearchRequest("index_name");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
AvgAggregationBuilder aggregation = AggregationBuilders.avg("response_time_average")
.field("doc['Message.keyword']").script(
new Script(
"def value = doc['Message.keyword']; return Integer.parseInt(value.split('regexp'))"));
searchSourceBuilder.aggregation(aggregation);
searchRequest.source(searchSourceBuilder);
try {
SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
Avg agg = search.getAggregations().get("response_time_average");
double value = agg.getValue();
//value returns as _infinity_
} catch (IOException e) {
log.error(e.getMessage(), e);
}
}
I don't have enough information about elasticsearch jargon. I tried a lot of solutions but I think this one is more close to solution.
My elasticservice client version is 7.3.1
thank you :)
I use Java and Java High Level REST Client.
I need to build query that will find documents. This query should contain many fields for searching.
For example, I have such json:
{
"timeStamp": "Fri, 29 Dec 2017 15:32:22 +0000",
"value": 314,
"operation": "http://software-testing.ru/library/testing/testing-tools/2638-postman",
"type": "duration",
"system": "front-admin"
}
and I need find all document due to the all fields.
I understand how constract query due to the one field (parametr) but don't know how I can use a lot of fields for query:
#PostMapping("/findMetricsByValues") #Transactional public ResponseEntity findMetricsByValues(#RequestBody ElasticSearchMetrics metrics){
SearchRequest searchRequest = new SearchRequest();
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.termQuery("value", metrics.getValue()));
searchRequest.source(sourceBuilder);
SearchResponse searchResponse = null;
try {
searchResponse = client.search(searchRequest);
} catch (IOException e) {
e.printStackTrace();
}
return new ResponseEntity<>(new GenericResponse(searchResponse, CODE_200), HttpStatus.OK); }
Could somebody help with it or give a hint?
You can use QueryBuilders.queryStringQuery(String query) method instead
In this you can write query according to may fields
Below is a link that can help you with this ->
https://www.elastic.co/guide/en/elasticsearch/reference/2.4/query-dsl-query-string-query.html
There are many clauses like AND , OR for joining multiple queries.
Here is syntax for writing query in a string -> https://www.elastic.co/guide/en/elasticsearch/reference/2.4/query-dsl-query-string-query.html#query-string-syntax
You can use bool query -> https://www.elastic.co/guide/en/elasticsearch/reference/2.4/query-dsl-bool-query.html
I'm using the Java API for ElasticSearch. I'm attempting to highlight my fields but it's not working. The correct results that match the search term are being returned, so there is content to highlight, but it simply won't do it. I set my SearchResponse and HighlightBuilder like this:
QueryBuilder matchQuery = simpleQueryStringQuery(searchTerm);
...
HighlightBuilder highlightBuilder = new HighlightBuilder()
.postTags("<highlight>")
.preTags("</highlight>")
.field("description");
SearchResponse response = client.prepareSearch("mediaitems")
.setTypes("mediaitem")
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(matchQuery) // Query
.setFrom(from)
.setSize(pageSize)
.setExplain(true)
.highlighter(highlightBuilder)
.get();
and in my JSON->POJO code, I check to see which fields have been highlighted, but the returned Map is empty.
Arrays.stream(hits).forEach((SearchHit hit) -> {
String source = hit.getSourceAsString();
Map<String, HighlightField> highlightFields = hit.getHighlightFields();
try {
MediaItem mediaItem = objectMapper.readValue(source, MediaItem.class);
mediaItemList.add(mediaItem);
} catch (IOException e) {
e.printStackTrace();
}
});
Why on earth is my highlighting request being ignored?
Any help is greatly appreciated.
You have to set the highlighted field in HighlightBuilder.
For example:
HighlightBuilder.Field field = new HighlightBuilder.Field(fieldName);
highlightBuilder.field(field);
I saw you are using simple query string query, so you can do the following:
Your query string: fieldname: searched text
So for example your query string is the following:
price: >2000 && city: Manchaster
With this query string you specified the fields in the query too.
Now highlighter should work.
I'm trying to use Lucene 4.8.1's SearchAfter methods to implement paging of search results in a web application.
A similar question has been asked before, but the accepted answer given there does not work for me:
Stack Overflow Question: Lucene web paging
When I create a Lucene ScoreDoc from scratch in this way to use as an argument for SearchAfter:
ScoreDoc sd = new ScoreDoc(14526, 0.0f);
TopDocs td = indexSearcher.searchAfter(sd, query, null, PAGEHITS);
I get this exception:
java.lang.IllegalArgumentException: after must be a FieldDoc
This appears contrary to the documentation. But in any case, when I create a Field Doc instead, I get:
java.lang.IllegalArgumentException: after.fields wasn't set
after.fields is an Object array, so I can hardly set that with information I can pass in a URI!
I cannot find any working code examples using SearchAfter. My original plan was obviously to create a new ScoreDoc as the previous question suggests. Can anybody suggest what I might be doing wrong, or link to any working code examples of SearchAfter?
Thanks!
I don't believe you can create a scoredoc and then pass it to searchAfter. You need to use the ScoreDocs returned from a previous search.
can you have a try.
#Test
public void searchAfter() {
Object[] objects = new Object[]{"1"};
List<Map<String, Object>> data = new ArrayList<Map<String, Object>>();
boolean type = true;
while (type) {
SearchHits searchHits = searchAfter(objects);
SearchHit[] hits = searchHits.getHits();
if (hits != null && hits.length > 0){
objects = hits[hits.length-1].getSortValues();
if (hits.length < size) type = false;
for (SearchHit hit : hits) {
data.add(hit.getSourceAsMap());
System.out.println(JsonUtil.objectToJson(hit.getSourceAsMap()));
}
}
}
Iterator<Map<String, Object>> iterator = data.iterator();
while (iterator.hasNext()) {
System.out.println(iterator.next().toString());
}
System.out.println(data.size() + "-----------------");
}
public SearchHits searchAfter(Object[] objects) {
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.termQuery("age", "33"));
sourceBuilder.size(size);
sourceBuilder.sort("account_number", SortOrder.ASC);
sourceBuilder.searchAfter(objects);
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("bank");
searchRequest.source(sourceBuilder);
ActionFuture<SearchResponse> response = elasticsearchTemplate.getClient().search(searchRequest);
SearchHits searchHits = response.actionGet().getHits();
return searchHits;
}