I have an Elastic search query and I would like to retrieve a certain column, not all.
I make my request in java with BoolQUEryBuilder which gives:
BoolQueryBuilder query = boolQuery();
query.must(wildcardQuery('value', value + "*"));
return findAll(query);
The method findAll :
protected List<T> findAll(final BoolQueryBuilder query) {
Query searchQuery = (new NativeSearchQueryBuilder()).withQuery(query).build();
SearchHits<T> searchHits = this.elasticsearchRestTemplate.search(searchQuery, this.getClazz(), this.elasticsearchRestTemplate.getIndexCoordinatesFor(this.getClazz()));
return (List)SearchHitSupport.unwrapSearchHits(searchHits);
}
I would like to add a filter on the columns. To illustrate in SQL this gives:
Select column_one, column_two from table;
Refer source filtering to fetch only few fields from Elasticsearch query results.
As explained in the same document example below code shows which fields to include and which to exclude.
String[] includeFields = new String[] {"title", "innerObject.*"};
String[] excludeFields = new String[] {"user"};
sourceBuilder.fetchSource(includeFields, excludeFields);
With Spring Data Elasticsearch, you should try this instead:
...
//include only specific fields
final SourceFilter sourceFilter = new FetchSourceFilter(new String[]{"column_one", "column_two"}, null);
// assemble the query
Query searchQuery = new NativeSearchQueryBuilder().withQuery(query).build();
searchQuery.addSourceFilter(sourceFilter);
...
Related
I have a list of studies. And I want to search over them with a typeahead function in a front page.
For that I use Hibernate Search with Spring Boot (2.1.5).
I indexed my STUDY table and some fields are marked with #Field attribute to be indexed.
The search works well.
But now I want to add a filter to use the same typeahead function but searching of a subset of my studies.
For that I created a filter like the Hibernate Search documentation but I didn't found a way to filter on two field with a OR between them.
My actual filter but filtering only on one field (avisDefinitifCet):
/**
* etudeFilter
*/
public class EtudeFilterFactory {
private String clasCet1ErPassage;
private String avisDefinitifCet;
public void setClasCet1ErPassage(String clasCet1ErPassage) {
this.clasCet1ErPassage = clasCet1ErPassage;
}
public void setAvisDefinitifCet(String avisDefinitifCet) {
this.avisDefinitifCet = avisDefinitifCet;
}
#Factory
public Query getFilter() {
System.out.println("Filter avisDefinitifCet : " + this.avisDefinitifCet.toLowerCase());
return new TermQuery(new Term("avisDefinitifCet", this.avisDefinitifCet.toLowerCase()));
}
}
How can I filter with a second field in my case clasCet1ErPassage?
At this end make a search on the standard search query and applying the filter like this
SELECT *
FROM STUDY
WHERE
A=t OR B=t OR C=t -- Normal search
AND (avisDefinitifCet='acceptation' OR clasCet1ErPassage='acceptation') -- Filter on two fields
My search function:
public List<Etude> search(String text, Map<String, String> allParams) {
text = stripAccents(text);
// get the full text entity manager
FullTextEntityManager fullTextEntityManager = getFullTextEntityManager(entityManager);
// create the query using Hibernate Search query DSL
QueryBuilder queryBuilder = fullTextEntityManager
.getSearchFactory()
.buildQueryBuilder()
.forEntity(Etude.class)
.get();
// Simple Query String queries
Query query = queryBuilder
.simpleQueryString()
.onFields("n0Cet")
.andField("anneeCet")
.andField("noDansAnneeCet")
.andField("sigleEtude")
.andField("titreEtude")
.andField("traitement1")
.andField("traitement2")
.andField("traitement3")
.andField("traitement4")
.andField("traitement5")
.andField("demandeurIgr.nomInvestigateurIgr")
.andField("investigateurHorsIgr.nomInvestigateur")
.andField("investigateurIgr.nomInvestigateurIgr")
.andField("promoteur.nomPromoteur")
.matching(text)
.createQuery();
// wrap Lucene query in an Hibernate Query object
FullTextQuery fullTextQuery = fullTextEntityManager
.createFullTextQuery(query, Etude.class)
.setMaxResults(101);
// Here allParams contains
// avisDefinitifCet => 'acceptation',
// clasCet1ErPassage => 'acceptation'
allParams.forEach((key, value) -> {
fullTextQuery.enableFullTextFilter("etudeFilter").setParameter(key, value);
});
return (List<Etude>) fullTextQuery.getResultList();
}
Am I thinking in the right way to implement it or I'm going wrong?
EDIT: Appparently you're using full-text filters, where you indeed have to use Lucene APIs directly.
In this case, just use a boolean junction with "should" clauses. When there's only "should" clauses in a boolean junction, at least one of them has to match.
#Factory
public Query getFilter() {
String valueToMatch = this.avisDefinitifCet.toLowerCase();
return new BooleanQuery.Builder()
.add(new TermQuery(new Term("avisDefinitifCet", valueToMatch)), Occur.SHOULD)
.add(new TermQuery(new Term("clasCet1ErPassage", valueToMatch)), Occur.SHOULD)
.build();
}
Previous answer:
If you're new to Lucene, you really should give the Hibernate Search DSL a try.
In your case, you'll want a keyword query that targets two fields:
EntityManager em = ...;
FullTextEntityManager fullTextEntityManager =
org.hibernate.search.jpa.Search.getFullTextEntityManager(em);
QueryBuilder queryBuilder = fullTextEntityManager.getSearchFactory()
.buildQueryBuilder().forEntity( Etude.class ).get();
Query luceneQuery = queryBuilder.keyword()
.onField("avisDefinitifCet").andField("clasCet1ErPassage")
.matching("acceptation")
.createQuery();
Note that, despite the method being called andField, it's actually an "OR": the query will match if any field matches.
For more advanced combination of queries, have a look at boolean junctions.
Here what I used to solve my problem thanks to #yrodiere
#Factory
public Query getFilter() {
return new BooleanQuery.Builder()
.add(new TermQuery(new Term("avisDefinitifCet", this.avisDefinitifCet.toLowerCase())), BooleanClause.Occur.SHOULD)
.add(new TermQuery(new Term("clasCet1ErPassage", this.clasCet1ErPassage.toLowerCase())), BooleanClause.Occur.SHOULD)
.build();
}
I am using java QueryBuilder to search values in elasticQuery.Below is my code snippet.
Sample 2 json store in elastic search
{
"dataLayer":"sourcefeed"
}
{
"dataLayer":"sandbox"
}
BoolQueryBuilder qb = QueryBuilders.boolQuery();
qb.must(QueryBuilders.matchQuery("dataLayer",inputValue);
The input value can be either sourcefeed or sandbox or both,Depending on my input Value it should return the output.For single value it is working fine,But if I give input as InputValue="sourceFeed,sandbox" it is not working.Tried termsQuery also but it is not fetching correctly.Kindly help me to solve the problem.
Thanks Prakash
SearchSourceBuilder query = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
List<QueryBuilder> queryBuilders1 = new ArrayList<QueryBuilder>();
TermsQueryBuilder termsQuery = QueryBuilders.termsQuery("dataLayer.keyword", listOfValues);
queryBuilders1.add(termsQuery);
boolQueryBuilder.must(queryBuilders1.get(0));
query.query(boolQueryBuilder);
add sourceFeed and sandbox etc in one list and pass it in QueryBuilders.termsQuery.
Try this.Hope this will work.
I am trying to query elastic search in Java using Jest. I am using a query builder to construct the query.
QueryBuilder matchQuery = QueryBuilders.boolQuery()
.must(QueryBuilders.matchQuery("id", "3434"))
.must(QueryBuilders.matchQuery("name", "name"))
.must(QueryBuilders.matchQuery("action", "login"))
.must(rangeQuery);
//i have this map now
Map<String , String> parameters = new HashMap<>();
parameters.put("id", "3433");
parameters.put("name", "name");
parameters.put("action", "login");
It would be great if some one can tell me if this can be constructed dynamically , like the matchQueries. For instance i would have three match queries , but i would have more. I can put my match query attributes in a Map. But if i iterate that how would we can define this boolQuery ? Anyone has any idea on this ?
Looking for something like this
QueryBuilder matchQuery = QueryBuilders.boolQuery()
.must(QueryBuilders.matchQuery(map.key, map.value))
.must(rangeQuery);
but how would i contruct this dynamically ?
You can simply iterate over your map and add each condition to the query, like this:
BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
for(String key : parameters.keySet()){
boolQuery.must(QueryBuilders.matchQuery(key, parameters.get(key)));
}
boolQuery.must(rangeQuery);
....
StringHandle rawHandle = new StringHandle(jsonString);
RawQueryByExampleDefinition querydef = queryManager.newRawQueryByExampleDefinition(rawHandle);
querydef.setCollections(collection);
StringHandle report = queryManager.validate(querydef, new StringHandle());
LOGGER.info("Query Def valididity: {}",report.toString());
StringHandle resultsHandle = new StringHandle().withFormat(Format.JSON);
queryManager.setPageLength(size);
queryManager.search(querydef, resultsHandle, startIndex);
....
I'm using the code above for MarkLogic search Query By Example, my question is how could you pass in a "sort by" criteria into the RawQueryByExampleDefinition to sort or order the resultset. For example I want the result to be sorted by emailAddress similar to the below query:
{
"$query":
{
"identifier":"user",
"$sort-by":"emailAddress"
}
}
How do I achieve the "sortby" as well as specifying desc or asc?
I think that would be done using a "combined query" from https://docs.marklogic.com/guide/java/searches#id_76144 with the sort-order option from https://docs.marklogic.com/search:search
As long as I limit my query to:
SolrQuery solrQuery = new SolrQuery();
solrQuery.set("q", query); //where query is solr query string (e.g. *:*)
solrQuery.set("start", 0);
solrQuery.set("rows", 10);
everything works fine - results are returned and so on.
Things are getting worse when I try to group results by my field "Token_group" to avoid duplicates:
SolrQuery solrQuery = new SolrQuery();
solrQuery.set("q", query); //where query is solr query string (e.g. *:*)
solrQuery.set("start", 0);
solrQuery.set("rows", 10);
solrQuery.set("group", true);
solrQuery.set("group.field", "token_group");
solrQuery.set("group.ngroups", true);
solrQuery.set("group.limit", 20);
Using this results in HttpSolrServer no exceptions are being thrown, but trying to access results ends up in NPE.
My querying Solr method:
public SolrDocumentList query(SolrQuery query) throws SolrServerException {
QueryResponse response = this.solr.query(query); //(this.solr is handle to HttpSolrSelver)
SolrDocumentList list = response.getResults();
return list;
}
note that similar grouping (using the very same field) is made in our other apps (PHP) and works fine, so this is not a schema issue.
I solved my issue. In case someone needs this in future:
When you perform a group query, you should use different methods to get and parse results.
While in ungrouped queries
QueryResponse response = this.solr.query(query); //(this.solr is handle to HttpSolrSelver)
SolrDocumentList list = response.getResults();
will work, when you want to query for groups, it won't.
So, how do I make and parse query?
Below code for building query is perfectly fine:
SolrQuery solrQuery = new SolrQuery();
solrQuery.set("q", query); //where query is solr query string (e.g. *:*)
solrQuery.set("start", 0);
solrQuery.set("rows", 10);
solrQuery.set("group", true);
solrQuery.set("group.field", "token_group");
solrQuery.set("group.ngroups", true);
solrQuery.set("group.limit", 20);
where last four lines define that Solr should group results and parameters of grouping. In this case group.limit will define how many maximum results within a group you want, and rows will tell how many max results should be there.
Making grouped query looks like this:
List<GroupCommand> groupCommands = this.solr.query(query).getGroupResponse().getValues();
referring to documentation, GroupCommand contains info about grouping as well as list of results, divided by groups.
Okay, I want to get to the results. How to do it?
Well, in my example there's only one position in List<GroupCommand> groupCommands, so to get list of found groups within it:
GroupCommand groupCommand = groupCommands.get(0);
List<Group> groups = groupCommand.getValues();
This will result in list of groups. Each group contains its own SolrDocumentList. To get it:
for(Group g : groups){
SolrDocumentList groupList = g.getResult();
(...)
}
Having this, well just proceed with SolrDocumentList for each group.
I used grouping query to get list of distinct results. How to do it?
This was exacly my case. It seems easy but there's a tricky part that can catch you if you're refactoring already running code that uses getNumFound() from SolrDocumentList.
Just analyze my code:
/**
* Gets distinct resultlist from grouped query
*
* #param query
* #return results list
* #throws SolrServerException
*/
public SolrDocumentList queryGrouped(SolrQuery query) throws SolrServerException {
List<GroupCommand> groupCommands = this.solr.query(query).getGroupResponse().getValues();
GroupCommand groupCommand = groupCommands.get(0);
List<Group> groups = groupCommand.getValues();
SolrDocumentList list = new SolrDocumentList();
if(groups.size() > 0){
long totalNumFound = groupCommand.getNGroups();
int iteratorLimit = 1;
for(Group g : groups){
SolrDocumentList groupList = g.getResult();
list.add(groupList.get(0));
//I wanted to limit list to 10 records
if(iteratorLimit++ > 10){
break;
}
}
list.setNumFound(totalNumFound);
}
return list;
}