I'd like to learn how to implement group by query using Mongo DB Java 3.x Driver. I want to group my collection through the usernames, and sort the results by the count of results DESC.
Here is the shell query which I want to implement the Java equivalent:
db.stream.aggregate({ $group: {_id: '$username', tweetCount: {$sum: 1} } }, { $sort: {tweetCount: -1} } );
Here is the Java code that I have implemented:
BasicDBObject groupFields = new BasicDBObject("_id", "username");
// count the results and store into countOfResults
groupFields.put("countOfResults", new BasicDBObject("$sum", 1));
BasicDBObject group = new BasicDBObject("$group", groupFields);
// sort the results by countOfResults DESC
BasicDBObject sortFields = new BasicDBObject("countOfResults", -1);
BasicDBObject sort = new BasicDBObject("$sort", sortFields);
List < BasicDBObject > pipeline = new ArrayList < BasicDBObject > ();
pipeline.add(group);
pipeline.add(sort);
AggregateIterable < Document > output = collection.aggregate(pipeline);
The result I need is the count of documents grouped by username. countOfResults returns the total number of the documents the collection has.
You should try not to use old object (BasicDBObject) types with Mongo 3.x. You can try something like this.
import static com.mongodb.client.model.Accumulators.*;
import static com.mongodb.client.model.Aggregates.*;
import static java.util.Arrays.asList;
Bson group = group("$username", sum("tweetCount", 1));
Bson sort = sort(new Document("tweetCount", -1));
AggregateIterable <Document> output = collection.aggregate(asList(group, sort));
Related
This is my current code
Document bitcoin = new Document("Currency", "Bitcoin").append("Values", Arrays.asList());
//bitCoinCollection.insertOne(bitcoin);
for(int i = 0; i<100;i++){
BasicDBObject setNewFieldQuery = new BasicDBObject()
.append("$set", new BasicDBObject().append("value", 100));
mongoClient.getDatabase("Binance").getCollection("Binance")
.updateOne(new BasicDBObject().append("_id", "tjena"), setNewFieldQuery);
}
It doesnt add any values to the excisting arraylist... is there a way to add it to an excisting arraylist in a document? Thanks in advance
For this sample collection of one document with an array field using MongoDB Java driver:
{ _id: 1, fruits: [ "orange", "banana" ] }
The following Java code adds one new element to the fruits array:
Bson update = push("fruits", "guava");
Bson filter = eq("_id", new Integer(1));
UpdateResult result = collection.updateOne(filter, update);
To add multiple elements all at once from a List collection:
Bson update = pushEach("fruits", Arrays.asList("grape", "apple", "peach"));
Bson filter = eq("_id", new Integer(1));
UpdateResult result = collection.updateOne(filter, update);
I have a collection processedClickLog in MongoDB.
{
"_id" : ObjectId("58ffb4cefbe21fa7896e2d73"),
"ID" : "81a5d7f48e5df09c9bc006e7cc89d6e6",
"USERID" : "206337611536",
"DATETIME" : "Fri Mar 31 17:29:34 -0400 2017",
"QUERYTEXT" : "Tom",
"DOCID" : "www.demo.com",
"TITLE" : "Harry Potter",
"TAB" : "People-Tab",
"TOTALRESULTS" : "1",
"DOCRANK" : 1
}
{ "id":
....
}
I am trying to execute a complex query in java. My query is to get processedClickLog collection where
TAB is not equal to People-Tab
DOCRANK is not equal to 0
only return "USERID", "DOCID", "DOCRANK", "QUERYTEXT" fields
Group by USERID
Below is my Java code. I am able to satisfy the first three condition. But I am stuck on 4th condition which is group by USERID.
String jsonResult = "";
MongoClient mongoClient = new MongoClient("localhost", 27017);
MongoDatabase database = mongoClient.getDatabase("test1");
MongoCollection<Document> collection = database.getCollection("processedClickLog");
//add condition where TAB is not equal to "People-Tab" and DOCRANK is not equal to 0
List<DBObject> criteria = new ArrayList<DBObject>();
criteria.add(new BasicDBObject("DOCRANK", new BasicDBObject("$ne", 0)));
criteria.add(new BasicDBObject("TAB", new BasicDBObject("$ne", "People-Tab")));
//combine the above two conditions
BasicDBObject query = new BasicDBObject("$and", criteria);
//to retrieve all the documents with specific fields
MongoCursor<Document> cursor = collection.find(query)
.projection(Projections.include("USERID", "DOCID", "DOCRANK", "QUERYTEXT")).iterator();
try {
while (cursor.hasNext()) {
System.out.println(cursor.next().toJson());
}
} finally {
cursor.close();
}
System.out.println(hashMap);
mongoClient.close();
}
How should I define my whole query to add the condition "group by USERID" in java? Any help is appreciated
You've to use aggregation framework. Statically import all the methods of helper classes and use the below code.
Use of BasicDBObject is not required in newer 3.x driver api. You should use the new class Document for similar needs.
import static com.mongodb.client.model.Accumulators.*;
import static com.mongodb.client.model.Aggregates.*;
import static java.util.Arrays.asList;
import static com.mongodb.client.model.Filters.*;
import static com.mongodb.client.model.Projections.*;
Bson match = match(and(ne("DOCRANK", 0), ne("TAB", "People-Tab")));
Bson group = group("$USERID", first("USERID", "$USERID"), first("DOCID", "$DOCID"), first("DOCRANK", "$DOCRANK"), first("QUERYTEXT", "$QUERYTEXT"));
Bson projection = project(fields(include("USERID", "DOCID", "DOCRANK", "QUERYTEXT"), excludeId()));
MongoCursor<Document> cursor = collection.aggregate(asList(match, group, projection)).iterator();
Projection stage is optional, only added to give a complete example.
More about aggregation here https://docs.mongodb.com/manual/reference/operator/aggregation/
I am inserting some value into mongodb in this way.
MongoClient mongo = new MongoClient( "localhost" , 27017 );
DB db = mongo.getDB("test");
DBCollection table = db.getCollection("paramDescMapper");
String key = UUID.randomUUID().toString();
String value = "{\"param0\":\"Car Make\",\"param1\":\"Car Model\",\"param2\":\"Car Variant\",\"param3\":\"Car Year\",\"param4\":\"Car Number\"}";
JSONObject jsonObject = new JSONObject(value);
// create a document to store key and value
BasicDBObject document = new BasicDBObject();
document.put("apiKey", key);
document.put("apiParamDesc", jsonObject.toString());
table.insert(document);
It is inserting data in this way.
{ "_id" : { "$oid" : "534251125f1ab7ec747298cd"} , "apiKey" : "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6" , "apiParamDesc" : "{\"param0\":\"Car Make\",\"param1\":\"Car Model\",\"param2\":\"Car Variant\",\"param3\":\"Car Year\",\"param4\":\"Car Number\"}"}
Now i want to get apiParamDesc value using apiKey. Like how we get data in mysql.
Select apiParamDesc where apiKey =
'1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6';
I googled a lot but could not found anything. This is how i am trying to get this apiParamDesc
BasicDBObject whereQuery = new BasicDBObject();
whereQuery.put("apiKey", "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6");
DBCursor cursor = table.find(whereQuery);
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
But this is giving me entire row. I want only apiParamDesc in a String.
Please help me.
Thanks
You can easily do it with aggregation framework. Below is the example which can resolve your issue:
// create our pipeline operations, first with the $match
DBObject match = new BasicDBObject("$match", new BasicDBObject("apiKey", "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6"));
// build the $projection operation
DBObject fields = new BasicDBObject("apiParamDesc", 1);
fields.put("_id", 0);
DBObject project = new BasicDBObject("$project", fields );
// run aggregation
AggregationOutput output = collection.aggregate( match, project);
You can also make use of only db.coll.find(< criteria >, < projection >);
BasicDBObject query = new BasicDBObject(new BasicDBObject("apiKey", "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6"), new BasicDBObject("apiParamDesc", 1).append("_id", 0));
//Which is equivalent to a follwoing query
//'db.coll.find({"apiKey": "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6"}, {"apiParamDesc": 1,"_id": 0});'
cursor = coll.find(query);
One more thing to update you on "I want only apiParamDesc in a String.", is if you are storing string like
"apiParamDesc" : "{\"param0\":\"Car Make\",\"param1\":\"Car Model\",\"param2\":\"Car Variant\",\"param3\":\"Car Year\",\"param4\":\"Car Number\"}
You cannot query on those sub level fields like param0, param1 ...
You data should look like :
{
"_id":{
"$oid":"534251125f1ab7ec747298cd"
},
"apiKey":"1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6",
"apiParamDesc":{
"param0":"Car Make",
"param1":"Car Model",
"param2":"Car Variant",
"param3":"Car Year",
"param4":"Car Number"
}
}
I want only apiParamDesc in a String.
You cannot however, you can get a document (object) returned with only the apiParamDesc as its single field (my Java is rusty):
BasicDBObject whereQuery = new BasicDBObject();
whereQuery.put("apiKey", "1eb9b9e3-3af1-4b25-b7ea-1f2fcb1d9af6");
BasicDBObject fields = new BasicDBObject();
fields.put("apiParamDesc", 1);
fields.put("_id", 0);
DBCursor cursor = table.find(whereQuery, fields);
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
I have the document structured as shown
I would like to get a collection of moduleDataItems which has version say more than 0.
This is my attempt :
Query qu = Query.query(Criteria.where("appKey")
.is("MOCK_APP").and("modules._id")
.is("APP_1_MOD_1")
.and("modules.moduleDataItems.version")
.gt(0));
List<DataItem> dList = mongoTemplate.find(qu,
DataItem.class,
ApplicationConstants.MONGO_APPLICATION_COLLECTION_NAME);
I m pretty sure I'm not doing the right thing. I do not get any DataItem in the result.
The classes represent the json structure.
Any help is appreciated.
Thanks
Finally solved it using this -
DBObject unwindParam = new BasicDBObject("$unwind","$dataItems");
DBObject matchParam = new BasicDBObject("$match",
new BasicDBObject("dataItems.version",
new BasicDBObject("$gt",requestedModule.getVersion())));
DBObject fields = new BasicDBObject("dataItems", 1);
DBObject projectParam = new BasicDBObject("$project", fields);
AggregationOutput output = mongoTemplate.getCollection(
"appModules").aggregate(
unwindParam, matchParam,projectParam);
CommandResult updatedData = output.getCommandResult();
BasicDBList resList = (BasicDBList) updatedData.get("result");
I need to implement "Popular Keyword" in my web app so that most searched keywords will show there.. For that i have created a db in mongo. I need to retrieve data from mongo so that mostly occuring keywords should sort and should return to my page. In sql it is just like as,
select names,count(names) from keyword_db group by names order by names;
need both java code and mongo shell query..
You might want to try for the mongo shell query (I'm sorting in descending order:
db.keyword_db.aggregate ( [
{ $group: { _id: "$names", count: { $sum: 1 } } },
{ $sort: { count: -1 } } ] )
For the Java version:
DBCollection myColl = db.getCollection("keyword_db");
// for the $group operator
DBObject groupFields = new BasicDBObject( "_id", "$names");
groupFields.put("count", new BasicDBObject( "$sum", 1));
DBObject group = new BasicDBObject("$group", groupFields );
// for the $sort operator
DBObject sortFields = new BasicDBObject("count", -1);
DBObject sort = new BasicDBObject("$sort", sortFields );
// run the aggregation
AggregationOutput output = myColl.aggregate(group, sort);
System.out.println( output.getCommandResult() );
The following page may help you also:
http://docs.mongodb.org/manual/reference/sql-aggregation-comparison/
For sorting you can use following..1 for ascending and -1 for desc.
db.keyword_db.find( { names: "A" } ).sort( { user_id: 1 } )
and for groupBy You can use Aggregation framework..
http://docs.mongodb.org/manual/reference/method/db.collection.group/