We Keep Coding

Custom field into Aggregation with MONGO

Is it possible to add 2 custom fields (that are not exist at my collections), into aggregate group query, using java with mongoDB? My approach is below but it doesn't work
Double custom_calculated_field1 = 0d;
Double custom_calculated_field2 = 0d;
Document customGroup_fields = new Document();
customGroup_fields.append("customer_id", "$customer_id");
//custom_calculated fields
customGroup_fields.append("custom_calculated_field1", custom_calculated_field1);
customGroup_fields.append("custom_calculated_field2", custom_calculated_field2);
AggregateIterable<Document> customers_results = customerID.aggregate(
Arrays.asList(
Aggregates.match(values),
Aggregates.group(customGroup_fields)
));

This is a problem with the way you structure your aggregation. You can read more about it on the Mongo aggregation docs here: https://docs.mongodb.com/manual/aggregation/
Effectively you assume that your pipeline knows what operations you are trying to execute. A pipeline however works very similar to what a unix pipe | does. It executes any command and passes the output to the next. So you need to tell it that your match is a match and the group is a group. I have an example for both Document and DBObject, where DBObject is the more up to date approach I believe. Consider this example:
#Test
public void testAggProjection() {
DBCollection collection = template.getCollection("test-collection");
DBObject ob = new BasicDBObject(ImmutableMap.of("test", "value1", "something", "here"));
DBObject ob2 = new BasicDBObject(ImmutableMap.of("test", "value1", "hello", "world"));
DBObject ob3 = new BasicDBObject(ImmutableMap.of("test", "value2", "other", "fields"));
WriteResult insert = collection.insert(ob, ob2, ob3);
Assert.assertEquals(3, insert.getN());
DBObject match = new BasicDBObject("$match", new BasicDBObject("test", "value1")); // match the test value
Map<String, Object> grouping = new HashMap<>();
grouping.put("_id", "$test");
grouping.put("my_count", new BasicDBObject("$sum", 1));
DBObject group = new BasicDBObject("$group", grouping); // group by
AggregationOutput aggregate = collection.aggregate(match, group);
System.out.println(aggregate.results());
DBObject next = aggregate.results().iterator().next();
Assert.assertTrue(next.containsField("_id"));
Assert.assertTrue(next.containsField("my_count"));
// with documents raw
MongoClient mc = (MongoClient) client;
MongoCollection<Document> collection2 = mc.getDatabase(template.getDb().getName()).getCollection("test-collection");
Document dob = new Document(ImmutableMap.of("test", "value1", "something", "here"));
Document dob2 = new Document(ImmutableMap.of("test", "value1", "hello", "world"));
Document dob3 = new Document(ImmutableMap.of("test", "value2", "other", "fields"));
collection2.insertMany(Arrays.asList(dob, dob2, dob3));
long count = collection2.count();
Assert.assertEquals(3, count);
Document match2 = new Document("$match", new Document("test", "value1"));
Document group2 = new Document("$group", ImmutableMap.of("_id", "$test", "my_count", new Document("$sum", 1)));
AggregateIterable<Document> aggregate2 = collection2.aggregate(Arrays.asList(match2, group2));
Document next2 = aggregate2.iterator().next();
Assert.assertTrue(next2.get("_id") != null);
Assert.assertTrue(next2.get("my_count") != null);
}
This is a working unit test. What this does is (in both cases):
Create a new collection and insert 3 documents.
Aggregate them with a match, matching 2 documents, and a group where we count the objects we found.
The important bits for you are here:
Document match2 = new Document("$match", new Document("test", "value1"));
Document group2 = new Document("$group", ImmutableMap.of("_id", "$test", "my_count", new Document("$sum", 1)));
AggregateIterable<Document> aggregate2 = collection2.aggregate(Arrays.asList(match2, group2));
In my match2 and group2 documents, I define what stage the aggregation is executing as $match and $group.
The resulting Json is then:
Document{{_id=value1, my_count=2}}
I hope that helps!
Please note that the _id field inside the group is mandatory.
Artur

How can i groupby a column in Mongodb using java? [duplicate]

I'd like to learn how to implement group by query using Mongo DB Java 3.x Driver. I want to group my collection through the usernames, and sort the results by the count of results DESC.
Here is the shell query which I want to implement the Java equivalent:
db.stream.aggregate({ $group: {_id: '$username', tweetCount: {$sum: 1} } }, { $sort: {tweetCount: -1} } );
Here is the Java code that I have implemented:
BasicDBObject groupFields = new BasicDBObject("_id", "username");
// count the results and store into countOfResults
groupFields.put("countOfResults", new BasicDBObject("$sum", 1));
BasicDBObject group = new BasicDBObject("$group", groupFields);
// sort the results by countOfResults DESC
BasicDBObject sortFields = new BasicDBObject("countOfResults", -1);
BasicDBObject sort = new BasicDBObject("$sort", sortFields);
List < BasicDBObject > pipeline = new ArrayList < BasicDBObject > ();
pipeline.add(group);
pipeline.add(sort);
AggregateIterable < Document > output = collection.aggregate(pipeline);
The result I need is the count of documents grouped by username. countOfResults returns the total number of the documents the collection has.

You should try not to use old object (BasicDBObject) types with Mongo 3.x. You can try something like this.
import static com.mongodb.client.model.Accumulators.*;
import static com.mongodb.client.model.Aggregates.*;
import static java.util.Arrays.asList;
Bson group = group("$username", sum("tweetCount", 1));
Bson sort = sort(new Document("tweetCount", -1));
AggregateIterable <Document> output = collection.aggregate(asList(group, sort));

How to use "$or" operator using documents in mongodb using java

Document query = new Document();
query.put("firstName", "abc");
query.put("firstName", "xyz");
Here,"abc" is overrided by the "xyz".So,in the query I'm getting only one value i.e,xyz.I don't know,why the query getting overrided.can u please help me out...
My code:
MongoClient mongo = new MongoClient("localhost",27017);
MongoDatabase db = mongo.getDatabase("sample");
MongoCollection<Document> coll = db.getCollection("users");
Document query = new Document();
query.put("firstName", "abc");
query.put("firstName", "xyz");
Document docStatus =new Document("$or",query );
try (MongoCursor<Document> cursor = coll.find(query).iterator()) {
while (cursor.hasNext()) {
System.out.println(cursor.next());
}
}

The arguments to the $or operator are a List, therefore you are just contructing a List of Document:
Document query = new Document(
"$or", Arrays.asList(
new Document("firstname", "abc"),
new Document("firstname", "xyz")
)
)
But really, when you want an $or condition on the same property, use $in instead. Which is of course another List but just of values:
Document query = new Document(
"firstname",
new Document(
"$in", Arrays.asList("abc","xyz")
)
)
Which is much shorter syntax for the selection of "either" value on the same document property.

You need to construct your query as below :
DBObject clause1 = new BasicDBObject("firstName", "abc");
DBObject clause2 = new BasicDBObject("firstName", "xyz");
BasicDBList or = new BasicDBList();
or.add(clause1);
or.add(clause2);
DBObject query = new BasicDBObject("$or", or);

BasicDBList orList = new BasicDBList();
orList.add(new BasicDBObject("FirstName","xxx"));
orList.add(new BasicDBObject("LastName","yyy"));
BasicDBObject dbObj = new BasicDBObject("$or",orList);

What's the update equivalent of insertMany in Mongo / Java

I am trying to do a bulk upsert, but am having problems to figure out how to do it. This does not work:
BasicDBObject filter = new BasicDBObject();
BasicDBObject update = new BasicDBObject("$set", list);
UpdateOptions options = new UpdateOptions().upsert(true);
collection.updateMany(filter, update, options);
Exception in thread "main" com.mongodb.MongoWriteException: Modifiers operate on fields but we found a Array instead.
Any examples would just be great!
UpsertMany items in one roundtrip
This is what I meant:
public static BulkWriteResult upsertAll(MongoCollection<Document> coll, List<Document> docs, String keyTag) {
List<UpdateOneModel<Document>> requests = new ArrayList<UpdateOneModel<Document>>();
UpdateOptions opt = new UpdateOptions().upsert(true);
for (Document doc : docs ) {
BasicDBObject filter = new BasicDBObject(keyTag, doc.get(keyTag));
BasicDBObject action = new BasicDBObject("$set", doc);
requests.add(new UpdateOneModel<Document>(filter, action, opt));
}
return coll.bulkWrite(requests);
}

You can perform bulk upsert using below code :
MongoClient mongo = new MongoClient("localhost", 27017);
DB db = (DB) mongo.getDB("testDB");
DBCollection collection = db.getCollection("collection");
BasicDBObject obj = new BasicDBObject();
obj.append("$set", new BasicDBObject("my_field", dbList));
BasicDBList dbList = new BasicDBList();
dbList.add(1);
dbList.add(2);
dbList.add(3);
BulkWriteOperation bwo = collection.initializeUnorderedBulkOperation();
bwo.find(new BasicDBObject()).upsert().update(obj);
bwo.execute();
For more details you can check : http://www.journaldev.com/6324/mongodb-upsert-example-using-mongo-shell-and-java-driver

This might work
db.collection.update({},{$set : {"my_field":[1, 2, 3]}},false,true)

Getting data from mongo db like we get in mysql?

I am inserting some value into mongodb in this way.
MongoClient mongo = new MongoClient( "localhost" , 27017 );
DB db = mongo.getDB("test");
DBCollection table = db.getCollection("paramDescMapper");
String key = UUID.randomUUID().toString();
String value = "{\"param0\":\"Car Make\",\"param1\":\"Car Model\",\"param2\":\"Car Variant\",\"param3\":\"Car Year\",\"param4\":\"Car Number\"}";
JSONObject jsonObject = new JSONObject(value);
// create a document to store key and value
BasicDBObject document = new BasicDBObject();
document.put("apiKey", key);
document.put("apiParamDesc", jsonObject.toString());
table.insert(document);
It is inserting data in this way.
{ "_id" : { "$oid" : "534251125f1ab7ec747298cd"} , "apiKey" : "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6" , "apiParamDesc" : "{\"param0\":\"Car Make\",\"param1\":\"Car Model\",\"param2\":\"Car Variant\",\"param3\":\"Car Year\",\"param4\":\"Car Number\"}"}
Now i want to get apiParamDesc value using apiKey. Like how we get data in mysql.
Select apiParamDesc where apiKey =
'1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6';
I googled a lot but could not found anything. This is how i am trying to get this apiParamDesc
BasicDBObject whereQuery = new BasicDBObject();
whereQuery.put("apiKey", "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6");
DBCursor cursor = table.find(whereQuery);
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
But this is giving me entire row. I want only apiParamDesc in a String.
Please help me.
Thanks

You can easily do it with aggregation framework. Below is the example which can resolve your issue:
// create our pipeline operations, first with the $match
DBObject match = new BasicDBObject("$match", new BasicDBObject("apiKey", "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6"));
// build the $projection operation
DBObject fields = new BasicDBObject("apiParamDesc", 1);
fields.put("_id", 0);
DBObject project = new BasicDBObject("$project", fields );
// run aggregation
AggregationOutput output = collection.aggregate( match, project);
You can also make use of only db.coll.find(< criteria >, < projection >);
BasicDBObject query = new BasicDBObject(new BasicDBObject("apiKey", "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6"), new BasicDBObject("apiParamDesc", 1).append("_id", 0));
//Which is equivalent to a follwoing query
//'db.coll.find({"apiKey": "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6"}, {"apiParamDesc": 1,"_id": 0});'
cursor = coll.find(query);
One more thing to update you on "I want only apiParamDesc in a String.", is if you are storing string like
"apiParamDesc" : "{\"param0\":\"Car Make\",\"param1\":\"Car Model\",\"param2\":\"Car Variant\",\"param3\":\"Car Year\",\"param4\":\"Car Number\"}
You cannot query on those sub level fields like param0, param1 ...
You data should look like :
{
"_id":{
"$oid":"534251125f1ab7ec747298cd"
},
"apiKey":"1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6",
"apiParamDesc":{
"param0":"Car Make",
"param1":"Car Model",
"param2":"Car Variant",
"param3":"Car Year",
"param4":"Car Number"
}
}

I want only apiParamDesc in a String.
You cannot however, you can get a document (object) returned with only the apiParamDesc as its single field (my Java is rusty):
BasicDBObject whereQuery = new BasicDBObject();
whereQuery.put("apiKey", "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6");
BasicDBObject fields = new BasicDBObject();
fields.put("apiParamDesc", 1);
fields.put("_id", 0);
DBCursor cursor = table.find(whereQuery, fields);
while(cursor.hasNext()) {
System.out.println(cursor.next());
}