Can anyone explain me why in Java when i do an aggregation pipeline with "$out" don't write the result in the new collection when i write only this:
Document match = new Document("$match", new Document("top_speed",new Document("$gte",350)));
Document out=new Document("$out", "new_collection");
coll.aggregate(Arrays.asList(
match,out
)
);
When I save the aggregation result and I iterate on it, the new collection is created and the result of the match is inside (Java has an error obviously in this case):
AggregateIterable<Document> resultAgg=
coll.aggregate(Arrays.asList(
match,out
)
);
for (Document doc : resultAgg){
System.out.println("The result of aggregation match:-"+ doc.toJson());
}
I can't understand why.
You can call toCollection() method instead of iterating.
Document match = new Document("$match", new Document("top_speed", new Document("$gte", 350)));
Document out = new Document("$out", "new_collection");
coll.aggregate(Arrays.asList(match, out)).toCollection();
Related
I'm trying to find all the documents with first name starting with Ram or Shyam.
I tried the following which actually worked in mongoDb
db.collection.find({
$or: [
{
name: {
$regex: "^Ram"
}
},
{
"name": {
"$regex": "^Shyam"
}
}
]
})
I got around count of 4k documents with this find.
Now when I tried to convert this mongo shell query into java.
Things do work but only name starting with Shyam get filtered. As a result the count also decreases to 2k.
Can someone please look at the below code and tell me why is it happening?
Why things are working in mongodb and not in java.
Java Equivalent Code --
MongoDatabase database = mongoClient.getDatabase("Friends");
MongoCollection<Document> collection = database.getCollection("Friend");
BasicDBObject filter = new BasicDBObject("$or", Arrays.asList
(new BasicDBObject("name", new BasicDBObject("$regex", "^Ram")).append("name", new
BasicDBObject("$regex", "^Shyam"))));
collection.find(filter).forEach((Consumer<Document>) doc -> {
// some java function
}
I figured out. There is just one some thing as $or operator always take array
In my java code I have converted it into array but just forgot that there is no need to append .
The solution will be --
BasicDBObject filter = new BasicDBObject("$or", Arrays.asList
(new BasicDBObject("name", new BasicDBObject("$regex", "^Ram")),new BasicDbObject("name", new
BasicDBObject("$regex", "^Shyam"))));
collection.find(filter).forEach((Consumer<Document>) doc -> {
// some java function
}
As per your mongo query, you need to match against name field but not on id field. That's the mistake
BasicDBObject filter = new BasicDBObject("$or", Arrays.asList
(new BasicDBObject("name", new BasicDBObject("$regex", "^Ram"))
.append("id", new //Here is the mistake
BasicDBObject("$regex", "^Shyam"))));
It should be
BasicDBObject filter = new BasicDBObject("$or", Arrays.asList
(new BasicDBObject("name", new BasicDBObject("$regex", "^Ram"))
.append("name", new BasicDBObject("$regex", "^Shyam"))));
I'm trying to load the matched documents into a temporary collection using aggregation query. Actually, I'm able to load all the matched documents into the temporary collection of MongoDB but my java program is throwing Null pointer exception at the for loop.
I'm totally stuck over here. May I know the reason for Null Pointer exception in this scenario . And can anyone please suggest me regarding the same ...
Document query = {"$or":[{"roll":1,"joiningDate":{"$gte":ISODate("2017-04-11T00:00:00Z")}},{"roll":2,"joiningDate":{"$gte": ISODate("2017-03-17T00:00:00Z")}}]};
Document match = new Document("$match",new Document("$or",query));
Document out =new Document("$out","TempCol");
System.out.println("Before Aggregation");
AggregateIterable<Document> resultAgg = collection.aggregate(Arrays.asList(match,out));
System.out.println("After aggregation");
for (Document doc : resultAgg){
System.out.println("The result of aggregation match:-");
}
System.out.println("Completed");
I generally prefer to keep the pipeline structured in one variable.
But the general idea here is use Document where you see {} and Arrays.asList where you see []:
List<Document> pipeline = Arrays.<Document>asList(
new Document("$match",
new Document("$or", Arrays.<Document>asList(
new Document("roll", 1)
.append("joiningDate", new Document(
"$gte", new DateTime(2017,04,11,0,0,0, DateTimeZone.UTC).toDate()
)),
new Document("controlId", 2)
.append("joiningDate", new Document(
"$gte", new DateTime(2017,03,17,0,0,0, DateTimeZone.UTC).toDate()
))
))
),
new Document("$out","TempCol")
);
AggregateIterable<Document> resultAgg = controlIssueCollection.aggregate(pipeline);
Also make sure when constructing a Date object with whatever your favorite construction method here is ( for me org.joda.time.DateTime ) that you are working with time in UTC, unless you really mean otherwise. And if you are comparing with values stored in MongoDB as shown in the shell, then you mean UTC.
I am trying to upsert millions of data using BulkWriteOperation, but my code is giving exception when my query condition is not satisfying but a document is available with that id.
Here is my code :-
if(provisionSubscriberList.size()>0){
Map<String, Object> map = new HashMap<String, Object>();
map.put("id", campaignTO.getId());
map.put("testSample", false);
map.put("status", "Active");
map.put("controlGroup", false);
try{
WriteConcern wc = WriteConcern.ACKNOWLEDGED;
BulkWriteOperation bulk = mongoTemplate.getCollection("provisionSubscriber").initializeOrderedBulkOperation();
for (ProvisionSubscriberEntity provisionalSubscriber : provisionSubscriberList) {
Query queryForAddSubscriber = new Query();
Update updateFieldsForAddSubscriber = new Update();
updateFieldsForAddSubscriber.set("msisdn", provisionalSubscriber.getMsisdn());
updateFieldsForAddSubscriber.set("deviceType", provisionalSubscriber.getDeviceType());
updateFieldsForAddSubscriber.addToSet("campaignIdList", map);
List<DBObject> criteria = new ArrayList<DBObject>();
criteria.add(new BasicDBObject("_id",new ObjectId(provisionalSubscriber.getId())));
criteria.add(new BasicDBObject("campaignIdList.id", new BasicDBObject("$ne", campaignTO.getId())));
criteria.add(new BasicDBObject("campaignIdList.controlGroup", new BasicDBObject("$ne", true)));
criteria.add(new BasicDBObject("campaignIdList.status", new BasicDBObject("$ne", "Active")));
BasicDBObject queryCriteria = new BasicDBObject("$and", criteria);
bulk.find(queryCriteria).upsert().updateOne(updateFieldsForAddSubscriber.getUpdateObject());
}
BulkWriteResult results =bulk.execute(wc);
System.out.println(results);
for (BulkWriteUpsert up : results.getUpserts()) {
System.out.println(up.getId());
}
And Here is the Exception I am getting:-
com.mongodb.BulkWriteException: Bulk write operation error on server 192.168.1.113:27017. Write errors: [BulkWriteError{index=0, code=11000, message='E11000 duplicate key error index: jmailer_digiengage.provisionSubscriber.$_id_ dup key: { : ObjectId('58c8f33301de9614143f5812') }', details={ }}].
at com.mongodb.BulkWriteHelper.translateBulkWriteException(BulkWriteHelper.java:56)
at com.mongodb.DBCollection.executeBulkWriteOperation(DBCollection.java:2310)
at com.mongodb.BulkWriteOperation.execute(BulkWriteOperation.java:136)
at com.lumatadigital.digiengage.daoImpl.ProvisioningDaoImpl.provisionOnCampaign(ProvisioningDaoImpl.java:120)
at com.lumatadigital.digiengage.schedular.service.SchedularJobConfig.provisioningJob(SchedularJobConfig.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:269)
at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:257)
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:75)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
EDIT: Basically, I want to insert data if document is not available or update data if the document is available and my query is satisfying to that document, otherwise skip that document. Also, I want to track the upserted documents.
It is because in your query criteria:
List<DBObject> criteria = new ArrayList<DBObject>();
criteria.add(new BasicDBObject("_id",new ObjectId(provisionalSubscriber.getId())));
criteria.add(new BasicDBObject("campaignIdList.id", new BasicDBObject("$ne", campaignTO.getId())));
criteria.add(new BasicDBObject("campaignIdList.controlGroup", new BasicDBObject("$ne", true)));
criteria.add(new BasicDBObject("campaignIdList.status", new BasicDBObject("$ne", "Active")));
BasicDBObject queryCriteria = new BasicDBObject("$and", criteria);
If the _id field is inserted in the database already with the insert statement, and when the update statement runs for the next time, the criteria "$ne" (not equal to) in the campaign list object fails that will create new row with the same _id tries to insert instead of update since the previous data do not match with the current data.
Hence you are getting the below error:
E11000 duplicate key error index: jmailer_digiengage.provisionSubscriber.$_id_ dup key: { : ObjectId('58c8f33301de9614143f5812') }
To do bulk update, you can use the below code
MongoCollection<Document> collection = database.getCollection("collection");
List<WriteModel<Document>> updates = new ArrayList<WriteModel<Document>>();
UpdateOptions options = new UpdateOptions();
options.upsert(true);
// Doc1 update
Document doc1 = new Document("$set", new Document("key1", "value1"));
updates.add(new UpdateOneModel<Document>(new Document("_id",new ObjectId("562a44971bca3c0001953f42")), doc1, options));
//Doc2 update
Document doc2 = new Document("$set", new Document("key1", "value2"));
updates.add(new UpdateOneModel<Document>(new Document("_id",new ObjectId("562a44971bca3c0001954071")), doc2, options));
BulkWriteResult result = collection.bulkWrite(updates);
System.out.println("Updated count : " + result.getModifiedCount());
In the below snippet
updates.add(new UpdateOneModel<Document>(new Document("_id",new ObjectId("562a44971bca3c0001954071")), doc2, options));
The 1st condition is the filter condition where you can use any key present in the doc to filter out the document you want to update, the 2nd parameter is the fields that needs to be updated for the doc, the 3rd parameter is additional options that can be passed to the model
I want to execute query in MongoDB 3.2 with Java Driver 3.2, which contains both $and and $or clauses at the same time.
With the reference, I tried the following approach:
List<Document> criteria1 = new ArrayList<>();
List<Document> criteria2 = new ArrayList<>();
criteria1.add(new Document("fetchStatus", new Document("$gte", FetchStatus.PROCESSED_NLP.getID())));
criteria1.add(new Document("fetchStatus", new Document("$lte", fetchStatusParam)));
criteria1.add(new Document("episodeID", new Document("$in", episodeIDs)));
criteria2.add(new Document("fetchStatus", new Document("$eq", PROCESSED_FETCH.getID())));
criteria2.add(new Document("isFullTextRet", new Document("$eq", false)));
BasicDBList or = new BasicDBList();
or.add(criteria1);
or.add(criteria2);
DBObject query = new BasicDBObject("$or", or);
ArrayList<Document> results = dbC_Coll.find(query).into(new ArrayList<>());
Where the criteria1 and criteria2 should be connected with $or while within criteria1 clause the $and should be applied.
The problem is that in MongoDB Java Driver 3.2 there is such no method and I get the Cannot resolve method find(com.mongodb.DBObject) error.
How can I compose a query such as (A && B) || (X && Y) in MongoDB Java Driver 3.2?
Personally, I find it far less confusing to construct the object sequences just like U would with a JSON structure to enhance readability. But it's still just Document() wherever you see {} and List wherever you see []:
Document query = new Document(
"$or", Arrays.asList(
// First document in $or
new Document(
"fetchStatus",
new Document( "$gte", FetchStatus.PROCESSED_NLP.getID() )
.append("$lte", fetchStatusParam)
)
.append("episodeID", new Document( "$in", episodeIDs)),
// Second document in $or
new Document("fetchStatus", PROCESSED_FETCH.getID())
.append("isFullTextRet", false)
)
);
Which is basically the same as:
{
"$or": [
{
"fetchStatus": {
"$gte": FetchStatus.PROCESS_NLP.getID(),
"$lte": fetchStatusParam
},
"episodeID": { "$in": episodeIDs }
},
{
"fetchStatus": PROCESSED_FETCH.getID(),
"isFullTextRet": false
}
]
}
Also there is no need for "explicit" $eq operators, since "equals" is actually the default meaning of a value assignment in a query property anyway.
I want to perform a query on a field that is greater than or equal to, AND less than or equal to(I'm using java btw). In other words. >= and <=. As I understand, mongoDB has $gte and $lte operators, but I can't find the proper syntax to use it. The field i'm accessing is a top-level field.
I have managed to get this to work:
FindIterable<Document> iterable = db.getCollection("1dag").find(new Document("timestamp", new Document("$gt", 1412204098)));
as well ass...
FindIterable<Document> iterable = db.getCollection("1dag").find(new Document("timestamp", new Document("$lt", 1412204098)));
But how do you combine these with each other?
Currently I'm playing around with a statement like this, but it does not work:
FindIterable<Document> iterable5 = db.getCollection("1dag").find(new Document( "timestamp", new Document("$gte", 1412204098).append("timestamp", new Document("$lte",1412204099))));
Any help?
Basically you require a range query like this:
db.getCollection("1dag").find({
"timestamp": {
"$gte": 1412204098,
"$lte": 1412204099
}
})
Since you need multiple query conditions for this range query, you can can specify a logical conjunction (AND) by appending conditions to the query document using the append() method:
FindIterable<Document> iterable = db.getCollection("1dag").find(
new Document("timestamp", new Document("$gte", 1412204098).append("$lte", 1412204099)));
The constructor new Document(key, value) only gets you a document with one key-value pair. But in this case you need to create a document with more than one. To do this, create an empty document, and then add pairs to it with .append(key, value).
Document timespan = new Document();
timespan.append("$gt", 1412204098);
timespan.append("$lt", 1412204998);
// timespan in JSON:
// { $gt: 1412204098, $lt: 1412204998}
Document condition = new Document("timestamp", timespan);
// condition in JSON:
// { timestamp: { $gt: 1412204098, $lt: 1412204998} }
FindIterable<Document> iterable = db.getCollection("1dag").find(condition);
Or if you really want to do it with a one-liner without temporary variables:
FindIterable<Document> iterable = db.getCollection("1dag").find(
new Document()
.append("timestamp", new Document()
.append("$gt",1412204098)
.append("$lt",1412204998)
)
);