In MongoDB, is it possible to update the value of a field using the value from another field? The equivalent SQL would be something like:
UPDATE Person SET Name = FirstName + ' ' + LastName
And the MongoDB pseudo-code would be:
db.person.update( {}, { $set : { name : firstName + ' ' + lastName } );
The best way to do this is in version 4.2+ which allows using the aggregation pipeline in the update document and the updateOne, updateMany, or update(deprecated in most if not all languages drivers) collection methods.
MongoDB 4.2+
Version 4.2 also introduced the $set pipeline stage operator, which is an alias for $addFields. I will use $set here as it maps with what we are trying to achieve.
db.collection.<update method>(
{},
[
{"$set": {"name": { "$concat": ["$firstName", " ", "$lastName"]}}}
]
)
Note that square brackets in the second argument to the method specify an aggregation pipeline instead of a plain update document because using a simple document will not work correctly.
MongoDB 3.4+
In 3.4+, you can use $addFields and the $out aggregation pipeline operators.
db.collection.aggregate(
[
{ "$addFields": {
"name": { "$concat": [ "$firstName", " ", "$lastName" ] }
}},
{ "$out": <output collection name> }
]
)
Note that this does not update your collection but instead replaces the existing collection or creates a new one. Also, for update operations that require "typecasting", you will need client-side processing, and depending on the operation, you may need to use the find() method instead of the .aggreate() method.
MongoDB 3.2 and 3.0
The way we do this is by $projecting our documents and using the $concat string aggregation operator to return the concatenated string.
You then iterate the cursor and use the $set update operator to add the new field to your documents using bulk operations for maximum efficiency.
Aggregation query:
var cursor = db.collection.aggregate([
{ "$project": {
"name": { "$concat": [ "$firstName", " ", "$lastName" ] }
}}
])
MongoDB 3.2 or newer
You need to use the bulkWrite method.
var requests = [];
cursor.forEach(document => {
requests.push( {
'updateOne': {
'filter': { '_id': document._id },
'update': { '$set': { 'name': document.name } }
}
});
if (requests.length === 500) {
//Execute per 500 operations and re-init
db.collection.bulkWrite(requests);
requests = [];
}
});
if(requests.length > 0) {
db.collection.bulkWrite(requests);
}
MongoDB 2.6 and 3.0
From this version, you need to use the now deprecated Bulk API and its associated methods.
var bulk = db.collection.initializeUnorderedBulkOp();
var count = 0;
cursor.snapshot().forEach(function(document) {
bulk.find({ '_id': document._id }).updateOne( {
'$set': { 'name': document.name }
});
count++;
if(count%500 === 0) {
// Excecute per 500 operations and re-init
bulk.execute();
bulk = db.collection.initializeUnorderedBulkOp();
}
})
// clean up queues
if(count > 0) {
bulk.execute();
}
MongoDB 2.4
cursor["result"].forEach(function(document) {
db.collection.update(
{ "_id": document._id },
{ "$set": { "name": document.name } }
);
})
You should iterate through. For your specific case:
db.person.find().snapshot().forEach(
function (elem) {
db.person.update(
{
_id: elem._id
},
{
$set: {
name: elem.firstname + ' ' + elem.lastname
}
}
);
}
);
Apparently there is a way to do this efficiently since MongoDB 3.4, see styvane's answer.
Obsolete answer below
You cannot refer to the document itself in an update (yet). You'll need to iterate through the documents and update each document using a function. See this answer for an example, or this one for server-side eval().
For a database with high activity, you may run into issues where your updates affect actively changing records and for this reason I recommend using snapshot()
db.person.find().snapshot().forEach( function (hombre) {
hombre.name = hombre.firstName + ' ' + hombre.lastName;
db.person.save(hombre);
});
http://docs.mongodb.org/manual/reference/method/cursor.snapshot/
Starting Mongo 4.2, db.collection.update() can accept an aggregation pipeline, finally allowing the update/creation of a field based on another field:
// { firstName: "Hello", lastName: "World" }
db.collection.updateMany(
{},
[{ $set: { name: { $concat: [ "$firstName", " ", "$lastName" ] } } }]
)
// { "firstName" : "Hello", "lastName" : "World", "name" : "Hello World" }
The first part {} is the match query, filtering which documents to update (in our case all documents).
The second part [{ $set: { name: { ... } }] is the update aggregation pipeline (note the squared brackets signifying the use of an aggregation pipeline). $set is a new aggregation operator and an alias of $addFields.
Regarding this answer, the snapshot function is deprecated in version 3.6, according to this update. So, on version 3.6 and above, it is possible to perform the operation this way:
db.person.find().forEach(
function (elem) {
db.person.update(
{
_id: elem._id
},
{
$set: {
name: elem.firstname + ' ' + elem.lastname
}
}
);
}
);
I tried the above solution but I found it unsuitable for large amounts of data. I then discovered the stream feature:
MongoClient.connect("...", function(err, db){
var c = db.collection('yourCollection');
var s = c.find({/* your query */}).stream();
s.on('data', function(doc){
c.update({_id: doc._id}, {$set: {name : doc.firstName + ' ' + doc.lastName}}, function(err, result) { /* result == true? */} }
});
s.on('end', function(){
// stream can end before all your updates do if you have a lot
})
})
update() method takes aggregation pipeline as parameter like
db.collection_name.update(
{
// Query
},
[
// Aggregation pipeline
{ "$set": { "id": "$_id" } }
],
{
// Options
"multi": true // false when a single doc has to be updated
}
)
The field can be set or unset with existing values using the aggregation pipeline.
Note: use $ with field name to specify the field which has to be read.
Here's what we came up with for copying one field to another for ~150_000 records. It took about 6 minutes, but is still significantly less resource intensive than it would have been to instantiate and iterate over the same number of ruby objects.
js_query = %({
$or : [
{
'settings.mobile_notifications' : { $exists : false },
'settings.mobile_admin_notifications' : { $exists : false }
}
]
})
js_for_each = %(function(user) {
if (!user.settings.hasOwnProperty('mobile_notifications')) {
user.settings.mobile_notifications = user.settings.email_notifications;
}
if (!user.settings.hasOwnProperty('mobile_admin_notifications')) {
user.settings.mobile_admin_notifications = user.settings.email_admin_notifications;
}
db.users.save(user);
})
js = "db.users.find(#{js_query}).forEach(#{js_for_each});"
Mongoid::Sessions.default.command('$eval' => js)
With MongoDB version 4.2+, updates are more flexible as it allows the use of aggregation pipeline in its update, updateOne and updateMany. You can now transform your documents using the aggregation operators then update without the need to explicity state the $set command (instead we use $replaceRoot: {newRoot: "$$ROOT"})
Here we use the aggregate query to extract the timestamp from MongoDB's ObjectID "_id" field and update the documents (I am not an expert in SQL but I think SQL does not provide any auto generated ObjectID that has timestamp to it, you would have to automatically create that date)
var collection = "person"
agg_query = [
{
"$addFields" : {
"_last_updated" : {
"$toDate" : "$_id"
}
}
},
{
$replaceRoot: {
newRoot: "$$ROOT"
}
}
]
db.getCollection(collection).updateMany({}, agg_query, {upsert: true})
(I would have posted this as a comment, but couldn't)
For anyone who lands here trying to update one field using another in the document with the c# driver...
I could not figure out how to use any of the UpdateXXX methods and their associated overloads since they take an UpdateDefinition as an argument.
// we want to set Prop1 to Prop2
class Foo { public string Prop1 { get; set; } public string Prop2 { get; set;} }
void Test()
{
var update = new UpdateDefinitionBuilder<Foo>();
update.Set(x => x.Prop1, <new value; no way to get a hold of the object that I can find>)
}
As a workaround, I found that you can use the RunCommand method on an IMongoDatabase (https://docs.mongodb.com/manual/reference/command/update/#dbcmd.update).
var command = new BsonDocument
{
{ "update", "CollectionToUpdate" },
{ "updates", new BsonArray
{
new BsonDocument
{
// Any filter; here the check is if Prop1 does not exist
{ "q", new BsonDocument{ ["Prop1"] = new BsonDocument("$exists", false) }},
// set it to the value of Prop2
{ "u", new BsonArray { new BsonDocument { ["$set"] = new BsonDocument("Prop1", "$Prop2") }}},
{ "multi", true }
}
}
}
};
database.RunCommand<BsonDocument>(command);
MongoDB 4.2+ Golang
result, err := collection.UpdateMany(ctx, bson.M{},
mongo.Pipeline{
bson.D{{"$set",
bson.M{"name": bson.M{"$concat": []string{"$lastName", " ", "$firstName"}}}
}},
)
I'm using mongodb version 3.4, with java driver.
I have a collection of jsons of this format:
{
"foo": {
"d4231ds": {
"type":"A",
"color":"red"
},
"dmncxa3s": {
"type":"B",
"color":"yellow"
},
"JsdjS8": {
"type":"A",
"color":"red"
},
"SKJDcxar3": {
"type":"C",
"color":"green"
}
},
"bar": "100011"
}
I need to get all the documents which have "type":"A", using java. The keys under foo (d4231ds,dmncxa3s etc) aren't known in advance, and differ between the documents. Hence, using the dot operator is not working in this case.
I've tried looking on several methods, but non seemed to achieve this.
You can try the below aggregation in latest 3.4 server version.
Use $objectToArray to convert the named keys into key value pair ( k, v ) and use $in to query the value document for field type = A.
$redact which acts on the results from query filter, $$KEEP when match else $$PRUNE
MongoClient mongoClient = new MongoClient();
MongoDatabase database = mongoClient.getDatabase(db_name);
MongoCollection<Document> mongoCollection = database.getCollection(collection_name);
Document query = Document.parse("{\"$redact\":{\"$cond\":[{\"$let\":{\"vars\":{\"subelems\":{\"$objectToArray\":\"$foo\"}},\"in\":{\"$in\":[\"A\",\"$$subelems.v.type\"]}}},\"$$KEEP\",\"$$PRUNE\"]}}");
List<Document> result = mongoCollection.aggregate(Arrays.asList(query)).into(new ArrayList<>());
Mongo Shell Query:
db.collection_name.aggregate([
{
"$redact": {
"$cond": [
{
"$let": {
"vars": {
"subelems": {
"$objectToArray": "$foo"
}
},
"in": {
"$in": [
"A",
"$$subelems.v.type"
]
}
}
},
"$$KEEP",
"$$PRUNE"
]
}
}
])
I'm having trouble creating aggregation in Morphia, the documentation is really not clear. This is the original query:
db.collection('events').aggregate([
{
$match: {
"identifier": {
$in: [
userId1, userId2
]
},
$or: [
{
"info.name": "messageType",
"info.value": "Push",
"timestamp": {
$gte: newDate("2015-04-27T19:53:13.912Z"),
$lte: newDate("2015-08-27T19:53:13.912Z")
}
}
]
}{
$unwind: "$info"
},
{
$match: {
$or: [
{
"info.name": "messageType",
"info.value": "Push"
}
]
}
]);
The only example in their docs was using out and there's some example here but I couldn't make it to work.
I didn't even made it past the first match, here's what I have:
ArrayList<String> ids = new ArrayList<>();
ids.add("199941");
ids.add("199951");
Query<Event> q = ads.getQueryFactory().createQuery(ads);
q.and(q.criteria("identifier").in(ids));
AggregationPipeline pipeline = ads.createAggregation(Event.class).match(q);
Iterator<Event> iterator = pipeline.aggregate(Event.class);
Some help or guidance and how to start with the query or how it works will be great.
You need to create the query for the match() pipeline by breaking your code down into manageable pieces that will be easy to follow. So let's start
with the query to match the identifier field, you have done the great so far. We need to then combine with the $or part of the query.
Carrying on from where you left, create the full query as:
Query<Event> q = ads.getQueryFactory().createQuery(ads);
Criteria[] arrayA = {
q.criteria("info.name").equal("messageType"),
q.criteria("info.value").equal("Push"),
q.field("timestamp").greaterThan(start);
q.field("timestamp").lessThan(end);
};
Criteria[] arrayB = {
q.criteria("info.name").equal("messageType"),
q.criteria("info.value").equal("Push")
};
q.and(
q.criteria("identifier").in(ids),
q.or(arrayA)
);
Query<Event> query = ads.getQueryFactory().createQuery(ads);
query.or(arrayB);
AggregationPipeline pipeline = ads.createAggregation(Event.class)
.match(q)
.unwind("info")
.match(query);
Iterator<Event> iterator = pipeline.aggregate(Event.class);
The above is untested but will guide you somewhere closer home, so make some necessary adjustments where appropriate. For some references, the following SO questions may give you some pointers:
Complex AND-OR query in Morphia
Morphia query with or operator
and of course the AggregationTest.java Github page
I'm making a MongoDB statistic system using the Java driver, and I am wondering if it is possible (and how) to change the value of a key nested inside many objects. Here is how my database is formatted:
{
location : “chicago”,
stats : [
{
"employee" : "rob",
"stat1" : 1,
"stat2" : 3,
"stat3" : 2
},
{
"employee" : "krista",
"stat1" : 1,
"stat2" : 3,
"stat3" : 2
}
]
}
So, for example, how could I change Rob's "stat2" to another value? I am new to JSON and the MongoDB Java driver. Any help is appreciated!
You need to use the positional $ operator and $set in order to update what you want.
db.collection.update(
{ _id: <docId>, "stats.employee": "rob" },
{ "$set": { "stats.$.stat2": <value> } }
)
So you match your document and the required element of the array. The update side uses that array index to know in which element to update. The $set operator only updates the specified field.
In Java, Build with BasicDBObject.
BasicDBObject query = new BasicDBObject("_id", id);
query.append( new BasicDBObject("stats.employee", "rob") );
BasicDBObject update = new BasicDBObject("$set",
new BasicDBObject("stats.$.stat2", value));
collection.update(query,update);
I am very new to MongoDB and its Java... SDK? Api? I have a very simple question, but I haven't been able to find a satisfactory answer.
Let's say I have a collection of instances that are like:
{
"_id": {
"$oid": "5156171e5d451c136236e738"
},
"_types": [
"Sample"
],
"last_z": {
"$date": "2012-12-30T09:12:12.250Z"
},
"last": {
"$date": "2012-12-30T04:12:12.250Z"
},
"section": "5156171e5d451c136236e70f",
"s_mac": "AA:AA:AA:AA:AA:AA",
"_cls": "Sample",
}
And I have a hard-coded Java list:
static List<String> MAC_LIST = Arrays.asList("90:27:E4:0E:3D:D2", "A8:26:D9:E6:1D:8B");
What I would like to know is how to query the MongoDB so it will give me all the objects whose s_mac field has a value that appears in the MAC_LIST List.
I'm guessing I should use the $in operator, but I don't know how to translate it to Java code.
Any hint or link to pages with explanations of the use of the $in operator through the Java SDK would be appreciated!
Here is a contrived example that works for me (driver version 2.10.1) - you can adjust the IP address and run it as is to check if you get the same outcome:
public void gss() throws Exception{
MongoClient mongo = new MongoClient("192.168.1.1");
DB db = mongo.getDB("test");
DBCollection collection = db.getCollection("stackoverflow");
DBObject o1 = new BasicDBObject();
o1.put("s_mac", "AA:AA:AA:AA:AA:AA");
o1.put("_cls", "Sample1");
DBObject o2 = new BasicDBObject();
o2.put("s_mac", "90:27:E4:0E:3D:D2");
o2.put("_cls", "Sample2");
DBObject o3 = new BasicDBObject();
o3.put("s_mac", "A8:26:D9:E6:1D:8B");
o3.put("_cls", "Sample3");
collection.insert(o1, o2, o3);
System.out.println(collection.find().count());
List<String> MAC_LIST = Arrays.asList("90:27:E4:0E:3D:D2", "A8:26:D9:E6:1D:8B");
System.out.println(collection.find(new BasicDBObject("s_mac", new BasicDBObject("$in", MAC_LIST))).count());
}
It inserts the following documents:
{ "_id" : ObjectId("5159ff98567e143bff0668e9"),
"s_mac" : "AA:AA:AA:AA:AA:AA",
"_cls" : "Sample1"
}
{ "_id" : ObjectId("5159ff98567e143bff0668ea"),
"s_mac" : "90:27:E4:0E:3D:D2",
"_cls" : "Sample2"
}
{ "_id" : ObjectId("5159ff98567e143bff0668eb"),
"s_mac" : "A8:26:D9:E6:1D:8B",
"_cls" : "Sample3"
}
A call to collection.find().count() returns 3 and a call to collection.find(new BasicDBObject("s_mac", new BasicDBObject("$in", MAC_LIST))).count() returns 2 which I think is what you expected.