We Keep Coding

Match few fields of pojo with mongo db documents

Background
I am trying to fetch data from mongo by matching custom class with few fields.
Mongo Collection:
"_id":{
"sNo":"1001",
"name": "Sameer",
"city": "Pune",
"state": "Maharashtra"
}
Pojo to match above details
#Document
class Id {
private String sNo;
private String name;
private String city;
private String state;
// getter & setters
}
Java code to fetch data from mongo where I want to fetch data based on sNo, name & city and want to skip state for matching:
ArrayList<Id> ids = new ArrayList<Id>();
...// code to populate Ids
MatchOperation match = Aggregation.match(Criteria.where("_id").in(ids));
Aggregation agg = Aggregation.newAggregation(match);
AggregationResults<Document> result = mongoTemplate.aggregate(agg, detailsCollection,
Document.class);
Problem:
I didn't find any way to skip few of the fields while match.
for now I am getting all data from collection.
Please help to suggest, how this can be achieved?

How to define custom analyzer to do global search with hibernate-search and elasticsearch

I have an implementation of hibernate-search-orm (5.9.0.Final) with hibernate-search-elasticsearch (5.9.0.Final).
I defined a custom analyzer on an entity (see beelow) and I indexed two entities :
id: "1"
title: "Médiatiques : récit et société"
abstract:...
id: "2"
title: "Mediatique Com'7"
abstract:...
The search works fine when I search on title field :
"title:médiatique" => 2 results.
"title:mediatique" => 2 results.
My problem is when I do a global search with accents (or not) :
search on "médiatique => 1 result (id:1)
search on "mediatique => 1 result (id:2)
Is there a way to resolve this?
Thanks.
Entity definition:
#Entity
#Table(name="bibliographic")
#DynamicUpdate
#DynamicInsert
#Indexed(index = "bibliographic")
#FullTextFilterDefs({
#FullTextFilterDef(name = "fieldsElasticsearchFilter",
impl = FieldsElasticsearchFilter.class)
})
#AnalyzerDef(name = "customAnalyzer",
tokenizer = #TokenizerDef(factory = StandardTokenizerFactory.class),
filters = {
#TokenFilterDef(factory = LowerCaseFilterFactory.class),
#TokenFilterDef(factory = ASCIIFoldingFilterFactory.class),
})
#Analyzer(definition = "customAnalyzer")
public class BibliographicHibernate implements Bibliographic {
...
#Column(name="title", updatable = false)
#Fields( {
#Field,
#Field(name = "titleSort", analyze = Analyze.NO, store = Store.YES)
})
#SortableField(forField = "titleSort")
private String title;
...
}
Search method :
FullTextEntityManager ftem = Search.getFullTextEntityManager(entityManager);
QueryBuilder qb = ftem.getSearchFactory().buildQueryBuilder().forEntity(Bibliographic.class).get();
QueryDescriptor q = ElasticsearchQueries.fromQueryString(queryString);
FullTextQuery query = ftem.createFullTextQuery(q, Bibliographic.class).setFirstResult(start).setMaxResults(rows);
if (filters!=null){
filters.stream().map((filter) -> filter.split(":")).forEach((f) -> {
query.enableFullTextFilter("fieldsElasticsearchFilter")
.setParameter("field", f[0])
.setParameter("value", f[1]);
}
);
}
if (facetFields!=null){
facetFields.stream().map((facet) -> facet.split(":")).forEach((f) ->{
query.getFacetManager()
.enableFaceting(qb.facet()
.name(f[0])
.onField(f[0])
.discrete()
.orderedBy(FacetSortOrder.COUNT_DESC)
.includeZeroCounts(false)
.maxFacetCount(10)
.createFacetingRequest() );
}
);
}
List<Bibliographic> bibs = query.getResultList();

To be honest I'm more surprised document 1 would match at all, since there's a trailing "s" on "Médiatiques" and you don't use any stemmer.
You are in a special case here: you are using a query string and passing it directly to Elasticsearch (that's what ElasticsearchQueries.fromQueryString(queryString) does). Hibernate Search has very little impact on the query being run, it only impacts the indexed content and the Elasticsearch mapping here.
When you run a QueryString query on Elasticsearch and you don't specify any field, it uses all fields in the document. I wouldn't bet that the analyzer used when analyzing your query is the same analyzer that you defined on your "title" field. In particular, it may not be removing accents.
An alternative solution would be to build a simple query string query using the QueryBuilder. The syntax of queries is a bit more limited, but is generally enough for end users. The code would look like this:
FullTextEntityManager ftem = Search.getFullTextEntityManager(entityManager);
QueryBuilder qb = ftem.getSearchFactory().buildQueryBuilder().forEntity(Bibliographic.class).get();
Query q = qb.simpleQueryString()
.onFields("title", "abstract")
.matching(queryString)
.createQuery();
FullTextQuery query = ftem.createFullTextQuery(q, Bibliographic.class).setFirstResult(start).setMaxResults(rows);
Users would still be able to target specific fields, but only in the list you provided (which, by the way, is probably safer, otherwise they could target sort fields and so on, which you probably don't want to allow). By default, all the fields in that list would be targeted.
This may lead to the exact same result as the query string, but the advantage is, you can override the analyzer being used for the query. For instance:
FullTextEntityManager ftem = Search.getFullTextEntityManager(entityManager);
QueryBuilder qb = ftem.getSearchFactory().buildQueryBuilder().forEntity(Bibliographic.class)
.overridesForField("title", "customAnalyzer")
.overridesForField("abstract", "customAnalyzer")
.get();
Query q = qb.simpleQueryString()
.onFields("title", "abstract")
.matching(queryString)
.createQuery();
FullTextQuery query = ftem.createFullTextQuery(q, Bibliographic.class).setFirstResult(start).setMaxResults(rows);
... and this will use your analyzer when querying.
As an alternative, you can also use a more advanced JSON query by replacing ElasticsearchQueries.fromQueryString(queryString) with ElasticsearchQueries.fromJsonQuery(json). You will have to craft the JSON yourself, though, taking some precautions to avoid any injection from the user (use Gson to build the Json), and taking care to follow the Elasticsearch query syntax.
You can find more information about simple query string queries in the official documentation.
Note: you may want to add FrenchMinimalStemFilterFactory to your list of token filters in your custom analyzer. It's not the cause of your problem, but once you manage to use your analyzer in search queries, you will very soon find it useful.

spring data mongodb - aggregation with dates

Following code doesn't give the expected results, i've tried with multiple documents, those documents structured as follows.
_id: 5a7714d44c75220958e6aa01
imei:355227045347655
point: [3.143453333333333,80.10954]
heading: 0
speed:0
timestamp: 2018-02-04 19:42:36.000
point_distance: 525.25
Now i need to calculate sum of point_distance of every record match with given imei and time period. I've tried to achieve this with following code but returns nothing even if required data exists.
public Object findDistance(long imei, Date from, Date to) {
List aggregationOps = new ArrayList<>();
//operations
aggregationOps.add(match(Criteria.where("imei").is(imei)));
aggregationOps.add(match(Criteria.where("timestamp").gte(from)));
aggregationOps.add(match(Criteria.where("timestamp").lte(to)));
aggregationOps.add(group("imei").sum("point_distance").as("distance"));
aggregationOps.add(project("imei").and("distance").previousOperation());
AggregationOptions agOps = new AggregationOptions.Builder().allowDiskUse(true).cursor(new BasicDBObject()).build();
return (DistanceInfo) getMongoTemplate()
.aggregate(newAggregation(aggregationOps).withOptions(agOps), Location.class, DistanceInfo.class)
.getUniqueMappedResult();
}
Date > from and to are formatted as yyyy-MM-dd hh:mm:ss
DistanceInfo class
public class DistanceInfo {
long imei;
double distance;
}
I'm new to this mongodb stuff and no idea what did i do wrong, how can i correct this ? any help is much appreciated.

Try this. this should work
MatchOperation matchOperation = match(Criteria.where("imei").is(imei)
.and("timestamp").gte(from.getTime()).lte(to.getTime()));
GroupOperation groupOperation = group("imei").sum("point_distance").as("distance");
ProjectionOperation projectionOperation = project().andExpression("imei").as("imei")
.andExpression("distance").as("distance");
Aggregation aggregation = newAggregation(matchOperation, groupOperation, projectionOperation);
AggregationResults<DistanceInfo> results = mongoTemplate.aggregate(aggregation, "location", DistanceInfo.class);
return results.getMappedResults();

MongoDB - group by - aggregation - java

I have a doc in my mongodb that looks like this -
public class AppCheckInRequest {
private String _id;
private String uuid;
private Date checkInDate;
private Double lat;
private Double lon;
private Double altitude;
}
The database will contain multiple documents with the same uuid but different checkInDates
Problem
I would like to run a mongo query using java that gives me one AppCheckInRequest doc(all fields) per uuid who's checkInDate is closest to the current time.
I believe I have to the aggregation framework, but I can't figure out how to get the results I need. Thanks.

In the mongo shell :-
This will give you the whole groupings:
db.items.aggregate({$group : {_id : "$uuid" , value : { $push : "$somevalue"}}} )
And using $first instead of $push will only put one from each (which is what you want i think?):
db.items.aggregate({$group : {_id : "$uuid" , value : { $first : "$somevalue"}}} )
Can you translate this to the Java api? or i'll try to add that too.
... ok, here's some Java:
Assuming the docs in my collection are {_id : "someid", name: "somename", value: "some value"}
then this code shows them grouped by name:
Mongo client = new Mongo("127.0.0.1");
DBCollection col = client.getDB("ajs").getCollection("items");
AggregationOutput agout = col.aggregate(
new BasicDBObject("$group",
new BasicDBObject("_id", "$name").append("value", new BasicDBObject("$push", "$value"))));
Iterator<DBObject> results = agout.results().iterator();
while(results.hasNext()) {
DBObject obj = results.next();
System.out.println(obj.get("_id")+" "+obj.get("value"));
}
and if you change $push to $first, you'll only get 1 per group. You can then add the rest of the fields once you get this query working.

Aggregate via Spring MongoTemplate to get max value of a collection

I want to find users by most recent date (Assume the User object has a date field). The data is stored in MongoDB and accessed via a Spring MongoTemplate.
Example of the raw data:
{userId:1, date:10}
{userId:1, date:20}
{userId:2, date:50}
{userId:2, date:10}
{userId:3, date:10}
{userId:3, date:30}
The query should return
{{userId:1, date:20}, {userId:2, date:50}, {userId:3, date:30}}
The aggregation method Ï am using is
db.table1.aggregate({$group:{'_id':'$userId', 'max':{$max:'$date'}}},
{$sort:{'max':1}}).result

You could Sort it first by date DESC and select the first while grouping by userID
final Aggregation aggregation = newAggregation(
Aggregation.sort(Sort.Direction.DESC, "date"),
Aggregation.group("userId").first("date").as("Date")
);
final AggregationResults<User> results = mongoTemplate.aggregate(aggregation, "user", User.class);