We Keep Coding

count the number of record by $match before $group java mongoDB

I have used aggregation for fetching records from MongoDB. I need to count the number of records return by $match before I do the groupBy (MongoDB) in java. Any help will be appreciated.
List countries =col2.distinct("country");
for(int i=0;i<countries.size();i++){
String country1= (String) countries.get(i);
List<DBObject> pipeline = new ArrayList<DBObject>(Arrays.asList(
new BasicDBObject("$match",new BasicDBObject("country", country1)),
new BasicDBObject("$group",
new BasicDBObject("_id","$job_id" )
.append("count", new BasicDBObject("$sum",1)
))));
AggregationOutput output = col2.aggregate(pipeline);}
This is a sample data:

You need to autowire
#Autowired
MongoTemplate mongoTemplate;
Then the converted mongo script in java
public List<Object> test(){
Aggregation aggregation = Aggregation.newAggregation(
match(Criteria.where("country").is("us")),
group(null)
.push("$$ROOT").as("data")
.count().as("count")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_COLLECTION.class), Object.class).getMappedResults();
}
I haven't tested it, but hope, this should work.
Working Mongo playground

MongoDB - Implementing nor query in java

I have the following query which I am trying to implement in Java (I am using 3.4.2 mongo-java driver) :
Json:
{
"_id" : "Team:2334918",
"fieldName" : [
"Delivery",
"Support"
],
"isDeleted" : false
}
Here the query:
db.myCollection.find({$nor: [{“fieldName”: {$exists: false}},{“fieldName”:{$size:0}}]})
I have written the following code to implement the above query:
MongoClient mc = ctm.getMongoConn();
db = ctm.getDatabase(mc);
col = db.getCollection(collectionName);
BasicDBObject searchObject = new BasicDBObject();
List<BasicDBObject> searchArguments = new ArrayList<BasicDBObject>();
searchArguments.add(new BasicDBObject("fieldName",new BasicDBObject("$exists",false)));
searchArguments.add(new BasicDBObject("fieldName",new BasicDBObject("$size",0)));
searchObject.put("$nor", searchArguments);
MongoCursor<Document> curs = (MongoCursor<Document>) col.find(searchObject);
while (curs.hasNext()){
Document doc = curs.next();
Object name1 = doc.get("fieldName");
System.out.println(name1.toString());
j++;
}
I am getting the following error while running:
java.lang.ClassCastException: com.mongodb.FindIterableImpl cannot be cast to com.mongodb.client.MongoCursor

Replace this:
MongoCursor<Document> curs = (MongoCursor<Document>) col.find(searchObject);
while (curs.hasNext()) {
}
With this:
FindIterable<Document> docs = col.find(searchObject);
for (Document d : docs) {
}
The suggestion from #Veeram (in the comments above) is correct too.

Try
MongoCursor<Document> curs = col.find(searchObject).iterator(); to get cursor.
You don't have to get hold of cursor. You can use the methods provided in the FindIterable which iterates the cursor for you. example at the end.
Also you are mixing Document (3.x) and BasicDBObject (2.x). You can replace your search builder with
Filters searchObject = Filters.nor(Filters.exists("fieldName"), Filters.size("fieldName", 0);
FindIterable<Document> curs = col.find(searchObject);
curs.forEach();
More info here by the author.

Top Hits Aggregation support in Jest SearchResult

I am using below function to make aggregation query:
private TermsBuilder getAggregations(String[] outputFields) {
TermsBuilder topLevelAggr = AggregationBuilders.terms("level1").field("field1").size(0);
TermsBuilder aggr2 = AggregationBuilders.terms("level2").field("field2").size(0);
TermsBuilder aggr3 = AggregationBuilders.terms("level3").field("field3").size(0);
TermsBuilder aggr4 = AggregationBuilders.terms("level4").field("field4").size(0);
TopHitsBuilder topHitsBuilder = AggregationBuilders.topHits("doc").setSize(1).addSort("fieldValue", SortOrder.DESC);
aggr4.subAggregation(topHitsBuilder);
aggr3.subAggregation(aggr4);
aggr2.subAggregation(aggr3);
topLevelAggr.subAggregation(aggr2);
topHitsBuilder.setFetchSource(outputFields, new String[]{});
return topLevelAggr;
}
I am getting the correct aggregation query from this code, but after executing the query I am not able to extract the top_hits aggregation result. I am using
io.searchbox.core.SearchResult class to get the query result. In this class I couldn't find any support for Top_Hits aggregation.
Please help. Thanks.

You can use the top hits aggregation java API.
Here's an example from elasticsearch docs:
// sr is here your SearchResponse object
Terms agg = sr.getAggregations().get("agg");
// For each entry
for (Terms.Bucket entry : agg.getBuckets()) {
String key = entry.getKey(); // bucket key
long docCount = entry.getDocCount(); // Doc count
logger.info("key [{}], doc_count [{}]", key, docCount);
// We ask for top_hits for each bucket
TopHits topHits = entry.getAggregations().get("top");
for (SearchHit hit : topHits.getHits().getHits()) {
logger.info(" -> id [{}], _source [{}]", hit.getId(), hit.getSourceAsString());
}
}

Streaming the result of an aggregate operation using spring-data-mongodb

I am using spring-data-mongodb and I want to use a cursor for an aggregate operation.
MongoTemplate.stream() gets a Query, so I tried creating the Aggregation instance, convert it to a DbObject using Aggregation.toDbObject(), created a BasicQuery using the DbObject and then invoke the stream() method.
This returns an empty cursor.
Debugging the spring-data-mongodb code shows that MongoTemplate.stream() uses the FindOperation, which makes me thinkspring-data-mongodb does not support streaming an aggregation operation.
Has anyone been able to stream the results of an aggregate query using spring-data-mongodb?
For the record, I can do it using the Java mongodb driver, but I prefer using spring-data.
EDIT Nov 10th - adding sample code:
MatchOperation match = Aggregation.match(Criteria.where("type").ne("AType"));
GroupOperation group = Aggregation.group("name", "type");
group = group.push("color").as("colors");
group = group.push("size").as("sizes");
TypedAggregation<MyClass> agg = Aggregation.newAggregation(MyClass.class, Arrays.asList(match, group));
MongoConverter converter = mongoTemplate.getConverter();
MappingContext<? extends MongoPersistentEntity<?>, MongoPersistentProperty> mappingContext = converter.getMappingContext();
QueryMapper queryMapper = new QueryMapper(converter);
AggregationOperationContext context = new TypeBasedAggregationOperationContext(MyClass.class, mappingContext, queryMapper);
// create a BasicQuery to be used in the stream() method by converting the Aggregation to a DbObject
BasicQuery query = new BasicQuery(agg.toDbObject("myClass", context));
// spring-mongo attributes the stream() method to find() operationsm not to aggregate() operations so the stream returns an empty cursor
CloseableIterator<MyClass> iter = mongoTemplate.stream(query, MyClass.class);
// this is an empty cursor
while(iter.hasNext()) {
System.out.println(iter.next().getName());
}
The following code, not using the stream() method, returns the expected non-empty result of the aggregation:
AggregationResults<HashMap> result = mongoTemplate.aggregate(agg, "myClass", HashMap.class);

For those who are still trying to find the answer to this:
From spring-data-mongo version 2.0.0.M4 onwards (AFAIK) MongoTemplate got an aggregateStream method.
So you can do the following:
AggregationOptions aggregationOptions = Aggregation.newAggregationOptions()
// this is very important: if you do not set the batch size, you'll get all the objects at once and you might run out of memory if the returning data set is too large
.cursorBatchSize(mongoCursorBatchSize)
.build();
data = mongoTemplate.aggregateStream(Aggregation.newAggregation(
Aggregation.group("person_id").count().as("count")).withOptions(aggregationOptions), collectionName, YourClazz.class);

using multiple criteria for grouping with the mongo 3 aggregation pipeline

In the example found here (http://mongodb.github.io/mongo-java-driver/3.2/builders/aggregation/), the aggregation grouping is based on "$customerId":
collection.aggregate(Arrays.asList(match(eq("author", "Dave")),
group("$customerId", sum("totalQuantity", "$quantity"),
avg("averageQuantity", "$quantity"))
out("authors")));
How would one add another grouping field into the aggregation pipepline, like say, "$systemId"?
Within the mongo-shell, you would do something like this:
db.coll.aggregate(
[
{
$group : {
_id : {"customerId":"$customerId", "systemId":"$systemId"},
//etc
}
}
]
)
In Java, you could build a query:
DBObject groupFields = new BasicDBObject();
DBObject groupIdFields = new BasicDBObject();
groupIdFields.put("customerId", "$customerID");
groupIdFields.put("systemId", "$systemId");
groupFields.put( "_id", groupIdFields );
//...
//... here some more aggregate criteria
//...
final AggregationOutput output = coll.aggregate(group);
Now, this seems like an awful lot of code just to get more than one grouping criterion. How is this possible with the aggregation pipeline in the mongo 3 java driver?