I have a collection processedClickLog in MongoDB.
{
"_id" : ObjectId("58ffb4cefbe21fa7896e2d73"),
"ID" : "81a5d7f48e5df09c9bc006e7cc89d6e6",
"USERID" : "206337611536",
"DATETIME" : "Fri Mar 31 17:29:34 -0400 2017",
"QUERYTEXT" : "Tom",
"DOCID" : "www.demo.com",
"TITLE" : "Harry Potter",
"TAB" : "People-Tab",
"TOTALRESULTS" : "1",
"DOCRANK" : 1
}
{ "id":
....
}
I am trying to execute a complex query in java. My query is to get processedClickLog collection where
TAB is not equal to People-Tab
DOCRANK is not equal to 0
only return "USERID", "DOCID", "DOCRANK", "QUERYTEXT" fields
Group by USERID
Below is my Java code. I am able to satisfy the first three condition. But I am stuck on 4th condition which is group by USERID.
String jsonResult = "";
MongoClient mongoClient = new MongoClient("localhost", 27017);
MongoDatabase database = mongoClient.getDatabase("test1");
MongoCollection<Document> collection = database.getCollection("processedClickLog");
//add condition where TAB is not equal to "People-Tab" and DOCRANK is not equal to 0
List<DBObject> criteria = new ArrayList<DBObject>();
criteria.add(new BasicDBObject("DOCRANK", new BasicDBObject("$ne", 0)));
criteria.add(new BasicDBObject("TAB", new BasicDBObject("$ne", "People-Tab")));
//combine the above two conditions
BasicDBObject query = new BasicDBObject("$and", criteria);
//to retrieve all the documents with specific fields
MongoCursor<Document> cursor = collection.find(query)
.projection(Projections.include("USERID", "DOCID", "DOCRANK", "QUERYTEXT")).iterator();
try {
while (cursor.hasNext()) {
System.out.println(cursor.next().toJson());
}
} finally {
cursor.close();
}
System.out.println(hashMap);
mongoClient.close();
}
How should I define my whole query to add the condition "group by USERID" in java? Any help is appreciated
You've to use aggregation framework. Statically import all the methods of helper classes and use the below code.
Use of BasicDBObject is not required in newer 3.x driver api. You should use the new class Document for similar needs.
import static com.mongodb.client.model.Accumulators.*;
import static com.mongodb.client.model.Aggregates.*;
import static java.util.Arrays.asList;
import static com.mongodb.client.model.Filters.*;
import static com.mongodb.client.model.Projections.*;
Bson match = match(and(ne("DOCRANK", 0), ne("TAB", "People-Tab")));
Bson group = group("$USERID", first("USERID", "$USERID"), first("DOCID", "$DOCID"), first("DOCRANK", "$DOCRANK"), first("QUERYTEXT", "$QUERYTEXT"));
Bson projection = project(fields(include("USERID", "DOCID", "DOCRANK", "QUERYTEXT"), excludeId()));
MongoCursor<Document> cursor = collection.aggregate(asList(match, group, projection)).iterator();
Projection stage is optional, only added to give a complete example.
More about aggregation here https://docs.mongodb.com/manual/reference/operator/aggregation/
Related
In SQL you can do the following;
SELECT * FROM CUSTOMERS WHERE ID=43 AND 1=1
How can I feed boolean expressions like '1=1' or '4<6' as criteria to MongoDB logical operators in Java?
For instance, I can do;
collection.find("$or",Arrays.asList(new Document("field1",value1),new Document("field2",value2)))
however, the above criteria are always based on already existing fields, while what I would like to do instead is more something like this (won't compile);
collection.find("$and",Arrays.asList(new Document(1,1),new Document("field2",value2)))
The reason I need this, is because I have a list of '$or' criteria but this list might be empty - in this case I want to have no criteria at all.
Update 1
While #gil.fernandes's solution is great for find queries, it will not work in aggregation queries (which is also what I need);
AggregateIterable aggregationQuery = collection.aggregate(Arrays.asList(
[...]
new Document("$match", new Document("$or", Arrays.asList(new Document("$where","1==1"))))
));
MongoCommandException: Command failed with error 16395: '$where is
not allowed inside of a $match aggregation expression'
Any ideas on how we can use the '1=1' logic in $match operators of aggregation?
Update 2
I applied #Veeram's second solution using mongo server version 3.4.7
However, if I include the addFields and match objects into my aggregation query I get 0 results.
If I remove them, I get all the results.
collection = mongoClient.getDatabase("testDatabase").getCollection("testColletion");
collection.insertOne(new Document("testField","testValue"));
Bson addFields = Aggregates.addFields(new Field<>("cmp", new Document("$or", Arrays.asList(new Document("$eq", Arrays.asList(1, 1))))));
Bson match = Aggregates.match(Filters.eq("cmp", 1));
AggregateIterable aggregationQuery = collection.aggregate(Arrays.asList(
new Document("$match", new Document("testField", "testValue")),
addFields,
match
));
boolean hasDocuments = aggregationQuery.iterator().hasNext()
You can $expr on 3.6 mongo server version.
import static com.mongodb.client.model.Aggregates.addFields;
import static com.mongodb.client.model.Aggregates.match;
import static com.mongodb.client.model.Filters.*;
import static java.util.Arrays.asList;
Something like
Bson match = match(expr(new Document("$or", asList(new Document("$eq", asList(1, 1))))));
AggregateIterable aggregationQuery = collection.aggregate(asList(
[...]
match
));
This should output query something like
{ "$match" : { "$expr" : { "$or" : [{ "$eq" : [1, 1] }] } } }
For lower version 3.4 you can use combination of $addFields and $match to achieve the similar query.
Bson addFields = addFields(new Field<>("cmp", new Document("$or", asList(new Document("$eq", asList(1, 1))))));
Bson match = match(eq("cmp", true));
AggregateIterable aggregationQuery = collection.aggregate(asList(
[...]
addFields,
match
));
This should output query something like
{ "$addFields" : { "cmp" : { "$or" : [{ "$eq" : [1, 1] }] } } }
{ "$match" : { "cmp" : true } }
In normal MongoDB Javascript queries you can express filters with 1=1.
Here is an example:
db.customer.find(
{"customer.id" : "1081965259", "customer.status": "Live", "$where": "1 == 1"},
{"customer.id": 1, "customer.status": 1})
.pretty();
The relevant query is "$where": "1 == 1". If you set this to "$where": "1 == 2" never any results will be retrieved.
Update
You can also use this clause in an or statement:
db.customer.find({"$or": [{ "customer.id" : "1081959342" }, { "$where": "1 == 1" }]});
Beware: this returns literally all records in the collection.
Update 2
Actually the combination of the $or clause with the $where can also be done in Java as well as well as a simple and combination. Here is an example:
// Or Logic
private static FindIterable<Document> findDocumentByKeysOrClause(String id, MongoCollection<Document> collection) {
BasicDBObject query = new BasicDBObject();
List<BasicDBObject> orQueryList =
Arrays.asList(new BasicDBObject("customer.customerId.sourceKeys.keies.id", id),
new BasicDBObject("$where", "1 == 1"));
query.put("$or", orQueryList);
return collection.find(query);
}
Here is my full test class for reference:
import com.mongodb.BasicDBObject;
import com.mongodb.MongoClient;
import com.mongodb.client.FindIterable;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;
import org.bson.types.ObjectId;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
/**
* Tests Mongo custom queries.
*/
public class MongoClauseTester {
public static void main(String[] args) {
MongoClient mongoClient = new MongoClient("localhost", 27017);
MongoDatabase db = mongoClient.getDatabase("customers");
MongoCollection<Document> customerCollection = db.getCollection("customer_UK");
final FindIterable<Document> documentByKeysOrClause = findDocumentByKeysOrClause("C#0248870263#DHUB", customerCollection);
MongoCursor<Document> iter = documentByKeysOrClause.iterator();
for(int i = 0; i < 3 && iter.hasNext(); i++) {
System.out.println(iter.next().toJson());
}
}
// And logic
private static FindIterable<Document> findDocumentByKeys(String id, MongoCollection<Document> collection) {
BasicDBObject query = new BasicDBObject();
query.put("customer.customerId.sourceKeys.keies.id", id);
query.put("$where", "1 == 1");
return collection.find(query);
}
// Or Logic
private static FindIterable<Document> findDocumentByKeysOrClause(String id, MongoCollection<Document> collection) {
BasicDBObject query = new BasicDBObject();
List<BasicDBObject> orQueryList =
Arrays.asList(new BasicDBObject("customer.customerId.sourceKeys.keies.id", id),
new BasicDBObject("$where", "1 == 1"));
query.put("$or", orQueryList);
return collection.find(query);
}
}
Here is the version of the MongoDB driver which was used:
<!-- https://mvnrepository.com/artifact/org.mongodb/mongo-java-driver -->
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
<version>3.4.1</version>
</dependency>
I'd like to learn how to implement group by query using Mongo DB Java 3.x Driver. I want to group my collection through the usernames, and sort the results by the count of results DESC.
Here is the shell query which I want to implement the Java equivalent:
db.stream.aggregate({ $group: {_id: '$username', tweetCount: {$sum: 1} } }, { $sort: {tweetCount: -1} } );
Here is the Java code that I have implemented:
BasicDBObject groupFields = new BasicDBObject("_id", "username");
// count the results and store into countOfResults
groupFields.put("countOfResults", new BasicDBObject("$sum", 1));
BasicDBObject group = new BasicDBObject("$group", groupFields);
// sort the results by countOfResults DESC
BasicDBObject sortFields = new BasicDBObject("countOfResults", -1);
BasicDBObject sort = new BasicDBObject("$sort", sortFields);
List < BasicDBObject > pipeline = new ArrayList < BasicDBObject > ();
pipeline.add(group);
pipeline.add(sort);
AggregateIterable < Document > output = collection.aggregate(pipeline);
The result I need is the count of documents grouped by username. countOfResults returns the total number of the documents the collection has.
You should try not to use old object (BasicDBObject) types with Mongo 3.x. You can try something like this.
import static com.mongodb.client.model.Accumulators.*;
import static com.mongodb.client.model.Aggregates.*;
import static java.util.Arrays.asList;
Bson group = group("$username", sum("tweetCount", 1));
Bson sort = sort(new Document("tweetCount", -1));
AggregateIterable <Document> output = collection.aggregate(asList(group, sort));
I am using mongodb with Java 3.0 driver. I have a scenario where I have to perform logical and i.e, $and on my queries. For example, I have two documents already created and I am trying to do something like this:
iterable = mongoDatabase.getCollection("restaurants").find(
new Document("$and", asList(abc,
updatedDocumentTypeOne)));
where abc is one document and updatedDocumentTypeOne is another document. I found this in mongodb manual but I am getting error as first create asList Method.
Or how to replicate the following in Java:
db.inventory.find( {
$and : [
{ $or : [ { price : 0.99 }, { price : 1.99 } ] },
{ $or : [ { sale : true }, { qty : { $lt : 20 } } ] }
]
} )`
You can also try the code below that adds a filter for query replication in Java:
// Where db is the object reference of "inventory" collection's database
MongoCollection<Document> inventory = db.getCollection("inventory");
//List for result return
List<Document> result = new ArrayList<Document>();
//Query replication in Java and return result into the list
inventory.find(Filters.and(
Filters.or(Filters.eq("price", 0.99),Filters.eq("price", "1.99")),
Filters.or(Filters.eq("sale", true),Filters.lt("qty", 20))
)).into(result);
Change from asList() to Arrays.asList()
Instead of writing Arrays.asList(), you have specified as asList(). So compiler is searching for the method asList(), which is NOT available.
Check the below code :
iterable = mongoDatabase.getCollection("restaurants").find(
new Document("$and", Arrays.asList(abc,
updatedDocumentTypeOne)));
For your above query, You can code as below :
/* First OR condition */
Document price1 = new BasicDBObject("price",0.99);
Document price2 = new BasicDBObject("price",1.99);
BasicDBList or_first = new BasicDBList();
or_first.add(price1);
or_first.add(price2);
DBObject query = new BasicDBObject("$or", or_first);
/* Second OR condition */
boolean val = true;
Document sale = new BasicDBObject("sale",val);
Document qty = new BasicDBObject("qty", new BasicDocument("$lt",20));
BasicDBList or_second = new BasicDBList();
or_second.add(sale);
or_second.add(qty);
DBObject query = new BasicDBObject("$or", or_second);
/* AND condition logic */
BasicDBList and_op = new BasicDBList();
and_op.add(or_first);
and_op.add(or_second);
iterable = mongoDatabase.getCollection("restaurants").find( new Document("$and", and_op ));
Hi I want to auto increment _id in mongodb using java. I am completely new to this. In the document I found the solution like this:
db.counters.insert(
{
_id: "userid",
seq: 0
}
)
function getNextSequence(name) {
var ret = db.counters.findAndModify(
{
query: { _id: name },
update: { $inc: { seq: 1 } },
new: true
}
);
return ret.seq;
}
db.users.insert(
{
_id: getNextSequence("userid"),
name: "Sarah C."
}
)
Can any one suggest how do I do this using java ? I am completely new to this.
Using Create an Auto-Incrementing Sequence Field first you should create collection using mongoDB shell and collection should be as :
db.counters.insert(
{
_id: "userid",
seq: 0
})
So you get counters collections which contains field like _id,seq, now create getNextSequence function in java and this function having parameter userid as string so getNextSequence function like this :
public static Object getNextSequence(String name) throws Exception{
MongoClient mongoClient = new MongoClient( "localhost" , 27017 );
// Now connect to your databases
DB db = mongoClient.getDB("demo");
DBCollection collection = db.getCollection("counters");
BasicDBObject find = new BasicDBObject();
find.put("_id", name);
BasicDBObject update = new BasicDBObject();
update.put("$inc", new BasicDBObject("seq", 1));
DBObject obj = collection.findAndModify(find, update);
return obj.get("seq");
}
The above function return seq count and used this function in main method as like :
public static void main(String[] args) throws Exception {
MongoClient mongoClient = new MongoClient( "localhost" , 27017 );
// Now connect to your databases
DB db = mongoClient.getDB("demo");
DBCollection collection = db.getCollection("counters");
BasicDBObject document = new BasicDBObject();
document.put("_id", getNextSequence("userid"));
document.put("name","Sarah C.");
collection.insert(document); // insert first doc
document.put("_id", getNextSequence("userid"));
document.put("name", "Bob D.");
collection.insert(document); // insert second doc
}
Now in counters collection contains three documents which contains name as Sarah C. and Bob D. respectively and one default documents which we inserted manually at first time and it increment seq like this { "_id" : "userid", "seq" : 2 }
DBCollection collection = database.getCollection("Table Name");
DBObject modifier = new BasicDBObject("counter", 1);
DBObject incQuery = new BasicDBObject("$id", modifier);
From a collection named "persons", I want to retreive all records where date is between
"14-11-2014" and "20-11-2014" <-- These are both in string format (dd-mm-yyyy)
AND
user: "Erik"
My mongoDB
{
"_id" : "546c9f26dbeaa7186ab042c4", <------this one should NOT be retreived
"Task: "Sometask" because of the user
"date" : "20-11-2014",
"user" : "Dean"
},
{
"_id" : "546caef6dbeaa7186ab042c5", <--------- This one should be retreived
"task": "A task",
"date" : "20-11-2014",
"user" : "Erik"
}
{
"_id" : "546caef6dbeaa7186ab042c5", <----- This one should NOT be retreived
"task": "A task", because of the date
"date" : "13-11-2014",
"user" : "Erik"
}
I am using java mongo java driver 2.11.3
Maybe there is some solution using BasicDBObject?
I'm very curious.. thanks
EDIT
I'm using:
public static String findTimelines(String begin, String end, String username) throws UnknownHostException, JSONException{
DBCollection dbCollection = checkConnection("timelines");
BasicDBObject query = new BasicDBObject();
query.put("date", BasicDBObjectBuilder.start("$gte", begin).add("$lte",end).get());
query.put("user", username);
dbCollection.find(query).sort(new BasicDBObject("date", -1));
DBCursor cursor = dbCollection.find(query);
return JSON.serialize(cursor);
}
Does work until you query something like "28-11-2014" to "01-12-2014", It doesn't return anything even though there is a object with date: "30-11-2014". I think this is because of the month change.
Also that object is retreived when you do: "28-11-2014" to "30-11-2014" because of the month staying the same
Please help!
Try something like this
BasicDBObject query = new BasicDBObject();
query.put("date", BasicDBObjectBuilder.start("$gte", fromDate).add("$lte", toDate).get());
collection.find(query).sort(new BasicDBObject("date", -1));
This is the query you would use:
db.posts.find({date: {$gte: start, $lt: end}, user: 'Erik'});
You should first parse your date using SimpleDateFormat or alike to get a Date object.
Then put together your query using BasicDBObject:
BasicDBObject q = new BasicDBObject();
q.append("date", new BasicDBObject().append("$gte", start).append("$lt", end));
q.append("user", "Erik");
collection.find(q);