We Keep Coding

How can I find Documents with Duplicate Array Elements?

Here is my Document:
{
"_id":"5b1ff7c53e3ac841302cfbc2",
"idProf":"5b1ff7c53e3ac841302cfbbf",
"pacientes":["5b20d2c83e3ac841302cfbdb","5b20d25f3e3ac841302cfbd0"]
}
I want to know how to find a duplicate entry in the array using MongoCollection in Java.
This is what I'm trying:
BasicDBObject query = new BasicDBObject("idProf", idProf);
query.append("$in", new BasicDBObject().append("pacientes", idJugador.toString()));
collection.find(query)

We can try to solve this in your Java-application code.
private final MongoCollection collection;
public boolean hasDuplicatePacientes(String idProf) {
Document d = collection.find(eq("idProf", idProf)).first();
List<String> pacientes = (List<String>) d.get("pacientes");
int original = pacientes.size();
if (original == 0) {
return false;
}
Set<String> unique = new HashSet(pacientes);
return original != unique.size();
}
Or if you're searching for a way to do this fully on db-side, I believe it's also possible with something like Neil Lunn provided.

The best approach really is to compare the length of the array to the length of an array which would have all duplicates removed. A "Set" does not have duplicate entries, so what you need to do is convert an array into a "Set" and test against the original.
Modern MongoDB $expr
Modern MongoDB releases have $expr which can be used with aggregation expressions in a regular query. Here the expressions we would use are $setDifference and $size along with $ne for the boolean comparison:
Document query = new Document(
"$expr", new Document(
"$ne", Arrays.asList(
new Document("$size", "$pacientes"),
new Document("$size",
new Document("$setDifference", Arrays.asList("$pacientes", Collections.emptyList()))
)
)
)
);
MongoCursor<Document> cursor = collection.find(query).iterator();
Which serializes as:
{
"$expr": {
"$ne": [
{ "$size": "$pacientes" },
{ "$size": { "$setDifference": [ "$pacientes", [] ] } }
]
}
}
Here it is actually the $setDifference which is doing the comparison and returning only unique elements. The $size is returning the length, both of the original document array content and the newly reduced "set". And of course where these are "not equal" ( the $ne ) the condition would be true meaning that a duplicate was found in the document.
The $expr operates on receiving a boolean true/false value in order whether to consider the document a match for the condition or not.
Earlier Version $where clause
Basically $where is a JavaScript expression that evaluates on the server
String whereClause = "this.pacientes.length != Object.keys(this.pacientes.reduce((o,e) => Object.assign(o, { [e.valueOf()]: null}), {})).length";
Document query = new Document("$where": whereClause);
MongoCursor<Document> cursor = collection.find(query).iterator();
You do need to have not explicitly disabled JavaScript evaluation on the server ( which is the default ) and it's not as efficient as using $expr and the native aggregation operators. But JavaScript expressions can be evaluated in the same way using $where, and the argument in Java code is basically sent as a string.
In the expression the .length is a property of all JavaScript arrays, so you have the original document content and the comparison to the "set". The Array.reduce() uses each array element as a "key" in a resulting object, from which the Object.keys() will then return those "keys" as a new array.
Since JavaScript objects work like a Map, only unique keys are allowed and this is a way to get that "set" result. And of course the same != comparison will return true when the removal of duplicate entries resulted in a change of length.
In either case of $expr or $where these are computed conditions which cannot use an index where present on the collection. As such it is generally recommended that additional criteria which use regular equality or range based query expressions which can indeed utilize an index be used alongside these expressions. Such additional criteria in the predicate would improve query performance greatly where an index is in place.

Mongo Template find element in a list inside of a document [duplicate]

This question already has answers here:
Retrieve only the queried element in an object array in MongoDB collection
(18 answers)
Spring data Match and Filter Nested Array
(1 answer)
Closed 3 years ago.
This is a document I want to search. Collection is CustmObjects. This is one CustomObject
{
"id" : "id_1",
"name" : "name_1";
"additionalData" : [
{
"additionalDataId" : "id_1_1",
"additionalDataName" : "name_1_1",
"longText" : "A long story about..."
},
{
"additionalDataId" : "id_1_2",
"additionalDataName" : "name_1_2",
"longText" : "A longer story about danger..."
},
{
"additionalDataId" : "id_1_3",
"additionalDataName" : "name_1_3",
"longText" : "A longer story about danger and courage"
},
]
}
To retrieve a document with name name_1 is easy.
final nameToSearchBy = "name_1";
Query query = new Query();
Criteria criteria = Criteria.where("name").is(nameToSearchBy);
query.addCriteria(criteria);
mongoTemplate.findOne(query, CustomObject.class, "CUSTOM_OBJECTS_COLLECTION");
Now my question. How to retrieve additionalData with name "id_1_2"? I do not need to retrieve the whole document, only an entry in the array.
Of course, I can retrive a whole document and iterate through its additionalData values in java program. But I want to delegate this job to database server.
I tried
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(Criteria.where("name").is("name_1").and("additionalData.additionalDataName").is("name_1_2")),
Aggregation.project("additionalData"),
);
mongoTemplate.aggregate(aggregation, "CustmObjects", Object.class);
It returns all elements of the array. I need only one matching element.
Update
This is what I want to do
Retrieve only the queried element in an object array in MongoDB collection
db.find({shapes: {"$elemMatch": {name: "name_1_2"}}}, ... )
As I understand, what the second part is projection to select fields. I need all the fields from array element.
{"shapes.color": 1}
The second link
Spring data Match and Filter Nested Array
The result I want to get is this
{
"additionalDataId" : "id_1_2",
"additionalDataName" : "name_1_2",
"longText" : "A long story about..."
}
is more comlex, it has array inside of element which is inside of array.
Please, I need something more simple and working.

How to get just the desired field from an array of sub documents in Mongodb using Java

I have just started using Mongo Db . Below is my data structure .
It has an array of skillID's , each of which have an array of activeCampaigns and each activeCampaign has an array of callsByTimeZone.
What I am looking for in SQL terms is :
Select activeCampaigns.callsByTimeZone.label,
activeCampaigns.callsByTimeZone.loaded
from X
where skillID=50296 and activeCampaigns.campaign_id= 11371940
and activeCampaigns.callsByTimeZone='PT'
The output what I am expecting is to get
{"label":"PT", "loaded":1 }
The Command I used is
db.cd.find({ "skillID" : 50296 , "activeCampaigns.campaignId" : 11371940,
"activeCampaigns.callsByTimeZone.label" :"PT" },
{ "activeCampaigns.callsByTimeZone.label" : 1 ,
"activeCampaigns.callsByTimeZone.loaded" : 1 ,"_id" : 0})
The output what I am getting is everything under activeCampaigns.callsByTimeZone while I am expecting just for PT
DataStructure :
{
"skillID":50296,
"clientID":7419,
"voiceID":1,
"otherResults":7,
"activeCampaigns":
[{
"campaignId":11371940,
"campaignFileName":"Aaron.name.121.csv",
"loaded":259,
"callsByTimeZone":
[{
"label":"CT",
"loaded":6
},
{
"label":"ET",
"loaded":241
},
{
"label":"PT",
"loaded":1
}]
}]
}
I tried the same in Java.
QueryBuilder query = QueryBuilder.start().and("skillID").is(50296)
.and("activeCampaigns.campaignId").is(11371940)
.and("activeCampaigns.callsByTimeZone.label").is("PT");
BasicDBObject fields = new BasicDBObject("activeCampaigns.callsByTimeZone.label",1)
.append("activeCampaigns.callsByTimeZone.loaded",1).append("_id", 0);
DBCursor cursor = coll.find(query.get(), fields);
String campaignJson = null;
while(cursor.hasNext()) {
DBObject campaignDBO = cursor.next();
campaignJson = campaignDBO.toString();
System.out.println(campaignJson);
}
the value obtained is everything under callsByTimeZone array. I am currently parsing the JSON obtained and getting only PT values . Is there a way to just query the PT fields inside activeCampaigns.callsByTimeZone .
Thanks in advance .Sorry if this question has already been raised in the forum, I have searched a lot and failed to find a proper solution.
Thanks in advance.

There are several ways of doing it, but you should not be using String manipulation (i.e. indexOf), the performance could be horrible.
The results in the cursor are nested Maps, representing the document in the database - a Map is a good Java-representation of key-value pairs. So you can navigate to the place you need in the document, instead of having to parse it as a String. I've tested the following and it works on your test data, but you might need to tweak it if your data is not all exactly like the example:
while (cursor.hasNext()) {
DBObject campaignDBO = cursor.next();
List callsByTimezone = (List) ((DBObject) ((List) campaignDBO.get("activeCampaigns")).get(0)).get("callsByTimeZone");
DBObject valuesThatIWant;
for (Object o : callsByTimezone) {
DBObject call = (DBObject) o;
if (call.get("label").equals("PT")) {
valuesThatIWant = call;
}
}
}
Depending upon your data, you might want to add protection against null values as well.
The thing you were looking for ({"label":"PT", "loaded":1 }) is in the variable valueThatIWant. Note that this, too, is a DBObject, i.e. a Map, so if you want to see what's inside it you need to use get:
valuesThatIWant.get("label"); // will return "PT"
valuesThatIWant.get("loaded"); // will return 1
Because DBObject is effectively a Map of String to Object (i.e. Map<String, Object>) you need to cast the values that come out of it (hence the ugliness in the first bit of code in my answer) - with numbers, it will depend on how the data was loaded into the database, it might come out as an int or as a double:
String theValueOfLabel = (String) valuesThatIWant.get("label"); // will return "PT"
double theValueOfLoaded = (Double) valuesThatIWant.get("loaded"); // will return 1.0
I'd also like to point out the following from my answer:
((List) campaignDBO.get("activeCampaigns")).get(0)
This assumes that "activeCampaigns" is a) a list and in this case b) only has one entry (I'm doing get(0)).
You will also have noticed that the fields values you've set are almost entirely being ignored, and the result is most of the document, not just the fields you asked for. I'm pretty sure you can only define the top-level fields you want the query to return, so your code:
BasicDBObject fields = new BasicDBObject("activeCampaigns.callsByTimeZone.label",1)
.append("activeCampaigns.callsByTimeZone.loaded",1)
.append("_id", 0);
is actually exactly the same as:
BasicDBObject fields = new BasicDBObject("activeCampaigns", 1).append("_id", 0);
I think some of the points that will help you to work with Java & MongoDB are:
When you query the database, it will return you the whole document of
the thing that matches your query, i.e. everything from "skillID"
downwards. If you want to select the fields to return, I think those will only be top-level fields. See the documentation for more detail.
To navigate the results, you need to know that a DBObjects are returned, and that these are effectively a Map<String,
Object> in Java - you can use get to navigate to the correct node,
but you will need to cast the values into the correct shape.

Replacing while loop from your Java code with below seems to give "PT" as output.
`while(cursor.hasNext()) {
DBObject campaignDBO = cursor.next();
campaignJson = campaignDBO.get("activeCampaigns").toString();
int labelInt = campaignJson.indexOf("PT", -1);
String label = campaignJson.substring(labelInt, labelInt+2);
System.out.println(label);
}`

Parsing strings to mongodb query documents with operators in java

In a project I'm working on, at one point I read a query to mongodb from a string. I've been using com.mongodb.util.JSON.parse(querystring) to read the query, which worked fine until I started reading queries that contained operators like $max and $min. At that point, rather than using mongodb's $max operator, the parser instead creates a "$max" field. For instance,
the input string:
{ $query : { state : "AL" } , $max : { pop : 9058 } }
is parsed to the DBObject:
{ "$query" : { "state" : "AL"} , "$max" : { "pop" : 9058}}
When I then look for a DBCursor with that query document, I get a cursor of size 0 (no matching document found in the databse), presumably because there are no documents with "$query" or "$max" fields.
Is there something I can use besides JSON.parse()? I'm not averse to writing my own function for it, but how can I get a DBObject that recognizes the $ operators as operators and not fields?
Any advice would be appreciated!

The following code snippet using query modification operator $max seems to work fine.
/* {$query:{state:"AL"}, "$max":{pop:10000}}*/
String s = "{$query:{state:\"AL\"}, \"$max\":{pop:10000}}";
DBObject dbObject = (DBObject) JSON.parse(s);
System.out.println("\nFind all: ");
DBCursor cursor = collection.find(dbObject);
try {
while (cursor.hasNext()) {
DBObject cur = cursor.next();
System.out.println(cur);
}
} finally {
cursor.close();
}
Make sure you have specified index on pop.
db.zips.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "test.zips",
"name" : "id"
},
{
"v" : 1,
"key" : {
"pop" : 1
},
"ns" : "test.zips",
"name" : "pop_1"
}
]
See the following link for detail.
http://docs.mongodb.org/manual/reference/operator/max/
Just in case you are interested in using aggregation operators $max or $min, the following link provide details and sample code.
http://docs.mongodb.org/ecosystem/tutorial/use-aggregation-framework-with-java-driver/

So it turns out the DBObject as given up there worked out fine. It returns a cursor with a size of 0, true, but the DBCursor's length is actually the thing I was looking for. (Previously, I had been checking whether the cursor's size was 0, and if it was, returning null.)
I'm not quite sure what the difference between size and length is in a DBCursor (the difference between size and count is apparent, but I'm not sure what length is supposed to be), but it works now. In the case above, size and count were both 0 but length was the desired number.

How to use AND & OR condition in JAVA mongo db array collection

I have the Java Mongo Collection in the following format.
{
"field1": ["f1","f2"],
"field2": ["g1","g2"],
"Ans": a1
}
{
"field1": ["f4","f5"],
"field2": ["g5","g6"],
"Ans": a2
}
{
"field1": ["f1","f6"],
"field2": ["g2","g3"],
"Ans": a3
}
I need to do the "OR" condition in field1 and field2 separately
From the result i need to do and condition to select the row.
For example: f1,f5,g2,g3 is the input i get.
i need to do or condtion in field 1 with f1,f5 so i will get all the three rows
i need to do or condtion in field 2 with g,g3 so i will get 1st and 3rd rows
while doing and condition of both i need to get only 1st and 3rd row.
Follwing is the code im using
DBObject query1 = new BasicDBObject("field1","f1");
DBObject query2 = new BasicDBObject("field1","f5");
BasicDBList condtionalOperator = new BasicDBList();
condtionalOperator.add(query1);
condtionalOperator.add(query2);
DBObject query = new BasicDBObject("$or", condtionalOperator);
Similar OR query for field 2 and And query for their result
I know its pretty long. Any one knows simple code than this?

(I'm responding using the Mongo shell JavaScript syntax, since it's a bit less verbose than Java. The equivalent Java should be straightforward to produce.)
db.collection.find({$and:[
{field1:{$in:["f1","f5"]}},
{field2:{$in:["g1","g3"]}}
]});
The $in operator basically does an OR operation, so the first operand of the $and is essentially equivalent to {$or:[{field1:"f1"},{field1:"f5"}]}.