I have a bunch of documents stored in elasticsearch with fields title and abstract. I have to search documents for queries like 'word1 word2 ..'. Currently I am using spring data
MultiMatchQueryBuilder multiMatchQueryBuilder = new MultiMatchQueryBuilder(query, "abstract", "title");
Iterable<Document> result = documentRepository.search(multiMatchQueryBuilder);
This gives me all the documents that contain word1 or word2. How can I match all the keywords? It should give me documents that have all the words in the query word1 and word2. Basically I want an and and not or of all the keywords in the search query.
You can specify the AND operator like this:
MultiMatchQueryBuilder multiMatchQueryBuilder = new MultiMatchQueryBuilder(query, "abstract", "title")
.operator(Operator.AND); // <---- add this
Iterable<Document> result = documentRepository.search(multiMatchQueryBuilder);
Related
I have an Elastic search query and I would like to retrieve a certain column, not all.
I make my request in java with BoolQUEryBuilder which gives:
BoolQueryBuilder query = boolQuery();
query.must(wildcardQuery('value', value + "*"));
return findAll(query);
The method findAll :
protected List<T> findAll(final BoolQueryBuilder query) {
Query searchQuery = (new NativeSearchQueryBuilder()).withQuery(query).build();
SearchHits<T> searchHits = this.elasticsearchRestTemplate.search(searchQuery, this.getClazz(), this.elasticsearchRestTemplate.getIndexCoordinatesFor(this.getClazz()));
return (List)SearchHitSupport.unwrapSearchHits(searchHits);
}
I would like to add a filter on the columns. To illustrate in SQL this gives:
Select column_one, column_two from table;
Refer source filtering to fetch only few fields from Elasticsearch query results.
As explained in the same document example below code shows which fields to include and which to exclude.
String[] includeFields = new String[] {"title", "innerObject.*"};
String[] excludeFields = new String[] {"user"};
sourceBuilder.fetchSource(includeFields, excludeFields);
With Spring Data Elasticsearch, you should try this instead:
...
//include only specific fields
final SourceFilter sourceFilter = new FetchSourceFilter(new String[]{"column_one", "column_two"}, null);
// assemble the query
Query searchQuery = new NativeSearchQueryBuilder().withQuery(query).build();
searchQuery.addSourceFilter(sourceFilter);
...
I have a string list with the dates of the days of a given week.
String daysweek[] = ["10/05/2020", "11/05/2020", "12/05/2020", "13/05/2020", "14/05/2020", "15/05/2020", "16/05/2020" ]
My goal is to be able to find several documents that belong to a certain week. The comparison field is "firstday".
Follows the image of the document structure in the database:
Document insert = new Document().append("$elemMatch", daysweek[]);
Document filterstar = new Document().append("id_motorista", idmotorista).append("pagamento", false).append("firstday", insert);
coll.find(filterstar).projection(new Document().append("_id", 1).append("origem",1).append("destino", 1).append("formadepagamento", 1).append("valordaviagem",1)
.append("notamotorista",1).append("pagamento",1).append("iniciodaviagem", 1).append("fimdaviagem",1).append("viagemcancelada", 1).append("horadaaceitacao",1)
.append("horacancelamentomotorista", 1).append("horacancelamentousuario", 1).append("taxadecancelamento", 1).append("valordaviagemmotorista", 1).append("valordaviagemusuario", 1).append("id_acompanhamento",1)
.append("taxaaplicativo", 1).append("taxacartao", 1).append("taxamotorista", 1)).sort(new Document().append("firstday", 1)).limit(100)
.into(docs).addOnSuccessListener(new OnSuccessListener<List<Document>>() {
#Override
public void onSuccess(List<Document> documents) {}
But the search finds no documents. The number of queries expected would be 35.
I would like to know if there is any way to find documents through a given document field, match any of the items within an arraylist.
$elemMatch is used when you're querying against an array field, but in your scenario you're querying against a string field and input is an array, then you can just use $in operator.
Mongo Shell Syntax :
db.collection.find({firstday : {$in : ["10/05/2020", "11/05/2020", "12/05/2020", "13/05/2020", "14/05/2020", "15/05/2020", "16/05/2020"]}})
Test : mongoplayground
The advice of #whoami works for me :D
So i change part of the code.
I changed that:
Document insert = new Document().append("$elemMatch", daysweek[]);
to this:
Document insert = new Document().append("$in", daysweek[]);
FINAL CODE:
Document insert = new Document().append("$in", daysweek[]);
Document filterstar = new Document().append("id_motorista", idmotorista).append("pagamento", false).append("firstday", insert);
coll.find(filterstar).projection(new Document().append("_id", 1).append("origem",1).append("destino", 1).append("formadepagamento", 1).append("valordaviagem",1)
.append("notamotorista",1).append("pagamento",1).append("iniciodaviagem", 1).append("fimdaviagem",1).append("viagemcancelada", 1).append("horadaaceitacao",1)
.append("horacancelamentomotorista", 1).append("horacancelamentousuario", 1).append("taxadecancelamento", 1).append("valordaviagemmotorista", 1).append("valordaviagemusuario", 1).append("id_acompanhamento",1)
.append("taxaaplicativo", 1).append("taxacartao", 1).append("taxamotorista", 1)).sort(new Document().append("firstday", 1)).limit(100)
.into(docs).addOnSuccessListener(new OnSuccessListener<List<Document>>() {
#Override
public void onSuccess(List<Document> documents) {}
I am using mysql grammar from here: https://github.com/antlr/grammars-v4/tree/master/mysql and have generated java files using Maven. Now, I was trying to parse a query but I am not getting how to do so.
I basically want to 'get' all the different components of a query, like the list columns selected, where conditions, sub queries, table names, etc. But I have no idea how to proceed. I have written below code as of now. Can someone please suggest with a simple example so that I can understand the usage and take up more complex tasks? Here is my code:
public static void main( String[] args )
{
String sql="select cust_name from database..table where cust_name like 'Kash%'";
ANTLRInputStream input = new ANTLRInputStream(sql);
MySqlLexer mySqlLexer = new MySqlLexer(input);
CommonTokenStream tokens = new CommonTokenStream(mySqlLexer);
MySqlParser mySqlParser = new MySqlParser(tokens);
ParseTree tree = mySqlParser.dmlStatement();
ParseTreeWalker walker = new ParseTreeWalker();
MySqlParserBaseListener listener=new MySqlParserBaseListener();
ParseTreeWalker.DEFAULT.walk(listener, tree);
System.out.println(?);
}
Using the above code, I am getting the following output:
line 1:11 no viable alternative at input '_'
(dmlStatement _ . . _ 'Kash%')
Thanks For Help :)
I basically want to 'get' all the different components of a query, like the list columns selected, where conditions, sub queries, table names, etc.
Your tree variable holds all that data: ParseTree tree = mySqlParser.dmlStatement();
line 1:11 no viable alternative at input '_'
If you look at the lexer rules:
SELECT: 'SELECT';
ID: ID_LITERAL;
fragment ID_LITERAL: [A-Z_$0-9]*?[A-Z_$]+?[A-Z_$0-9]*;
it appears that keywords and identifiers cannot contain lowercase letters.
If you run it like this:
String sql = "SELECT CUST_NAME FROM CUSTOMERS WHERE CUST_NAME LIKE 'Kash%'";
MySqlLexer lexer = new MySqlLexer(CharStreams.fromString(sql));
MySqlParser parser = new MySqlParser(new CommonTokenStream(lexer));
ParseTree root = parser.dmlStatement();
System.out.println(root.toStringTree(parser));
you will see the following output (indented for easier reading):
(dmlStatement
(selectStatement
(querySpecification SELECT
(selectElements
(selectElement
(fullColumnName
(uid
(simpleId CUST_NAME)))))
(fromClause FROM
(tableSources
(tableSource
(tableSourceItem
(tableName
(fullId
(uid
(simpleId CUSTOMERS))))))) WHERE
(expression
(predicate
(predicate
(expressionAtom
(fullColumnName
(uid
(simpleId CUST_NAME))))) LIKE
(predicate
(expressionAtom
(constant
(stringLiteral 'Kash%'))))))))))
I have an implementation of hibernate-search-orm (5.9.0.Final) with hibernate-search-elasticsearch (5.9.0.Final).
I defined a custom analyzer on an entity (see beelow) and I indexed two entities :
id: "1"
title: "Médiatiques : récit et société"
abstract:...
id: "2"
title: "Mediatique Com'7"
abstract:...
The search works fine when I search on title field :
"title:médiatique" => 2 results.
"title:mediatique" => 2 results.
My problem is when I do a global search with accents (or not) :
search on "médiatique => 1 result (id:1)
search on "mediatique => 1 result (id:2)
Is there a way to resolve this?
Thanks.
Entity definition:
#Entity
#Table(name="bibliographic")
#DynamicUpdate
#DynamicInsert
#Indexed(index = "bibliographic")
#FullTextFilterDefs({
#FullTextFilterDef(name = "fieldsElasticsearchFilter",
impl = FieldsElasticsearchFilter.class)
})
#AnalyzerDef(name = "customAnalyzer",
tokenizer = #TokenizerDef(factory = StandardTokenizerFactory.class),
filters = {
#TokenFilterDef(factory = LowerCaseFilterFactory.class),
#TokenFilterDef(factory = ASCIIFoldingFilterFactory.class),
})
#Analyzer(definition = "customAnalyzer")
public class BibliographicHibernate implements Bibliographic {
...
#Column(name="title", updatable = false)
#Fields( {
#Field,
#Field(name = "titleSort", analyze = Analyze.NO, store = Store.YES)
})
#SortableField(forField = "titleSort")
private String title;
...
}
Search method :
FullTextEntityManager ftem = Search.getFullTextEntityManager(entityManager);
QueryBuilder qb = ftem.getSearchFactory().buildQueryBuilder().forEntity(Bibliographic.class).get();
QueryDescriptor q = ElasticsearchQueries.fromQueryString(queryString);
FullTextQuery query = ftem.createFullTextQuery(q, Bibliographic.class).setFirstResult(start).setMaxResults(rows);
if (filters!=null){
filters.stream().map((filter) -> filter.split(":")).forEach((f) -> {
query.enableFullTextFilter("fieldsElasticsearchFilter")
.setParameter("field", f[0])
.setParameter("value", f[1]);
}
);
}
if (facetFields!=null){
facetFields.stream().map((facet) -> facet.split(":")).forEach((f) ->{
query.getFacetManager()
.enableFaceting(qb.facet()
.name(f[0])
.onField(f[0])
.discrete()
.orderedBy(FacetSortOrder.COUNT_DESC)
.includeZeroCounts(false)
.maxFacetCount(10)
.createFacetingRequest() );
}
);
}
List<Bibliographic> bibs = query.getResultList();
To be honest I'm more surprised document 1 would match at all, since there's a trailing "s" on "Médiatiques" and you don't use any stemmer.
You are in a special case here: you are using a query string and passing it directly to Elasticsearch (that's what ElasticsearchQueries.fromQueryString(queryString) does). Hibernate Search has very little impact on the query being run, it only impacts the indexed content and the Elasticsearch mapping here.
When you run a QueryString query on Elasticsearch and you don't specify any field, it uses all fields in the document. I wouldn't bet that the analyzer used when analyzing your query is the same analyzer that you defined on your "title" field. In particular, it may not be removing accents.
An alternative solution would be to build a simple query string query using the QueryBuilder. The syntax of queries is a bit more limited, but is generally enough for end users. The code would look like this:
FullTextEntityManager ftem = Search.getFullTextEntityManager(entityManager);
QueryBuilder qb = ftem.getSearchFactory().buildQueryBuilder().forEntity(Bibliographic.class).get();
Query q = qb.simpleQueryString()
.onFields("title", "abstract")
.matching(queryString)
.createQuery();
FullTextQuery query = ftem.createFullTextQuery(q, Bibliographic.class).setFirstResult(start).setMaxResults(rows);
Users would still be able to target specific fields, but only in the list you provided (which, by the way, is probably safer, otherwise they could target sort fields and so on, which you probably don't want to allow). By default, all the fields in that list would be targeted.
This may lead to the exact same result as the query string, but the advantage is, you can override the analyzer being used for the query. For instance:
FullTextEntityManager ftem = Search.getFullTextEntityManager(entityManager);
QueryBuilder qb = ftem.getSearchFactory().buildQueryBuilder().forEntity(Bibliographic.class)
.overridesForField("title", "customAnalyzer")
.overridesForField("abstract", "customAnalyzer")
.get();
Query q = qb.simpleQueryString()
.onFields("title", "abstract")
.matching(queryString)
.createQuery();
FullTextQuery query = ftem.createFullTextQuery(q, Bibliographic.class).setFirstResult(start).setMaxResults(rows);
... and this will use your analyzer when querying.
As an alternative, you can also use a more advanced JSON query by replacing ElasticsearchQueries.fromQueryString(queryString) with ElasticsearchQueries.fromJsonQuery(json). You will have to craft the JSON yourself, though, taking some precautions to avoid any injection from the user (use Gson to build the Json), and taking care to follow the Elasticsearch query syntax.
You can find more information about simple query string queries in the official documentation.
Note: you may want to add FrenchMinimalStemFilterFactory to your list of token filters in your custom analyzer. It's not the cause of your problem, but once you manage to use your analyzer in search queries, you will very soon find it useful.
I am using Lucene 1.3. I am trying to index one key and values were a list. When ever I search for the key, I should get list of results, but I am getting 0 results.i am declaring pricelistkey as Long
Here is my code which I tried.
String fieldName = pliBean.getProductID();
Field skuField = new Field(String.valueOf(priceListKey),fieldName, true, true, false);
doc.add(skuField);
writer.addDocument(doc);
This is indexed in key:listofvalues
The query I am passing is term query.
TermQuery qry = new TermQuery(new Term(key,Key));
Search(qry);