I am using Spring data mongo to insert a record to Mongo,
here is my code
mongoTemplate.save(person,"personCollection");
Here is my person object
public class Person implements Serializable {
int age;
String address;
String name;
//Getters and setters including hashcode and equals
}
my address is null here , after inserting the record in the collection, the data is populated with only age and name
i know that mongodb treats null value and noKey as the same thing, but my requirement is to even populate the address:null to have the schema consistent how do i do this with Spring Data mongo
current o/p: {"age":21,"name":"john Doe"}
expected o/p: {"age":21,"name":"john Doe","address":null}
NoSQL DB works in a different way compared to RDBMS.
the document {"age":21,"name":"john Doe"} is same as {"age":21,"name":"john Doe";"address":null}
instead of storing the key's as null better to not store the key at all this improves the performance of your reads/updates against the DB.
However, if your usecase still demands to sore null due to whatever the reasons it might be convert your POJO to BSONObject and then persist the BSONObject in the MongoDB.
Here is the example ( but however it will be only a work around to get the things going)
BSONObject personBsonObj = BasicDBObjectBuilder.start()
.add("name","John Doe")
.add("age",21)
.add("address",null).get();
if you are using spring data mongo use
mongoTemplate.insert(personBsonObj,"personCollection");
document in the db:
db.personCollection.findOne().pretty();
{"age":21,"name":"John Doe";"address":null}*
I've solved this problem using the below code
final Document document = new Document();
document.put("test1", "test1");
document.put("test2", null);
document.put("test3", "test3");
mongoTemplate.getCollection("your-collection-name").insert(document);
Here instead of using BSONObject, I used Document object and it worked fine.
Document inserted in DB
{
"_id" : ObjectId("some-id"),
"test1" : "test1",
"test2" : null,
"test3" : "test3"
}
Related
I am using mongodb 3.4 and I want to get the last inserted document id. I have searched all and I found out below code can be used if I used a BasicDBObject.
BasicDBObject docs = new BasicDBObject(doc);
collection.insertOne(docs);
ID = (ObjectId)doc.get( "_id" );
But the problem is am using Document type not BasicDBObject so I tried to get it as like this, doc.getObjectId();. But it asks a parameter which I actually I want, So does anyone know how to get it?
EDIT
This is the I am inserting it to mongo db.
Document doc = new Document("jarFileName", jarDataObj.getJarFileName())
.append("directory", jarDataObj.getPathData())
.append("version", jarDataObj.getVersion())
.append("artifactID", jarDataObj.getArtifactId())
.append("groupID", jarDataObj.getGroupId());
If I use doc.toJson() it shows me whole document. is there a way to extract only _id?
This gives me only the value i want it like the objectkey, So I can use it as reference key.
collection.insertOne(doc);
jarID = doc.get( "_id" );
System.out.println(jarID); //59a4db1a6812d7430c3ef2a5
Based on ObjectId Javadoc, you can simply instantiate an ObjectId from a 24 byte Hex string, which is what 59a4db1a6812d7430c3ef2a5 is if you use UTF-8 encoding. Why don't you just do new ObjectId("59a4db1a6812d7430c3ef2a5"), or new ObjectId("59a4db1a6812d7430c3ef2a5".getBytes(StandardCharsets.UTF_8))? Although, I'd say that exposing ObjectId outside the layer that integrates with Mongo is a design flaw.
Using Mongo Java Driver 2.13 and Mongo 3.0.
I am trying to move from Spring Data save() to MongoDB API's Bulk Writing since I am saving/updating about 100K objects. I am trying to write the Service/Repository layer code where I can pass in a Collection of my specific Objects and be able to either create new records or update existing records, or in other words upsert. When I do an insert the performance is very acceptable.
If I update the code to do upserts the performance is just way too slow. Am I doing something wrong in the following code sample (note it is scaled down to just the necessary logic, i.e. no error handling):
public void save(Collection<MyDomainObject> objects) {
BulkWriteOperation bulkWriter = dbCollection.initializeUnorderedBulkOperation();
for(MyDomainObject mdo : objects) {
DBObject dbObject = convert(mdo);
bulkWriter.find(new BasicDBObject("id",mdo.getId()))
.upsert().updateOne(new BasicDBObject("$set",dbObject));
}
bulkWriter.execute(writeConcern);
}
Note that I also tried replaceOne() instead of updateOne() with the same results.
I also noticed in the Mongo log that "nscannedObjects" keeps increasing while "nMatched", "nModified" and "upsert" are never larger than 1. Does this mean that it is table scanning for each record?
Am I using upsert the correct way? Any other suggestions?
Thanks to ry_donahue I figured out the issue.
It was not using the correct ID field, which is the index. In the conversion of the domain object to a DBObject there ended up being an "id" and an "_id" field.
I also changed updateOne() to replaceOne(). So now the code looks like this:
public void save(Collection<MyDomainObject> objects) {
BulkWriteOperation bulkWriter = dbCollection.initializeUnorderedBulkOperation();
for(MyDomainObject mdo : objects) {
DBObject dbObject = convert(mdo);
bulkWriter.find(new BasicDBObject("_id",new ObjectId(mdo.getId()))).upsert().replaceOne(dbObject);
}
bulkWriter.execute(writeConcern);
}
This now gives very good performance.
I am using JDO and DataNucleus to persist runtime-generated objects into MongoDB. The database objects have their own unique identifier, a string, which I put into Mongo's _id field. This works fine and I end up with, for instance, this object:
(in the mongo shell)
> db.CollectionName.find({"_id":"01e293bc-970d-e0b3-aac1-14109fdb7235_ZMUkU234ufY3opYPeov38T4EilNLURIb8ki"}).pretty()
{
"_id" : "01e293bc-970d-e0b3-aac1-14109fdb7235_ZMUkU234ufY3opYPeov38T4EilNLURIb8ki",
...
When I want to get an object back out of Mongo I make the call which I think should work:
PersistenceManager pm = pmf.getPersistenceManager();
String keyString = "01e293bc-970d-e0b3-aac1-14109fdb7235_ZMUkU234ufY3opYPeov38T4EilNLURIb8ki";
Object dbObject = pm.getObjectById(keyString);
But I don't get a dbObject, instead JDO throws a JDONotFoundException. I thought perhaps I needed to specify the class of the DB object which is tough because it is runtime generated, but I added a hack which saves a pointer to the class when I persist, so that I can use it later:
this.savedDBclass = obj.getClass();
pm.makePersistent(obj);
...
PersistenceManager pm = pmf.getPersistenceManager();
String keyString = "01e293bc-970d-e0b3-aac1-14109fdb7235_ZMUkU234ufY3opYPeov38T4EilNLURIb8ki";
Object dbObject = pm.getObjectById(this.savedDBclass, keyString);
and in that case I get a JDOFatalUserException "No metadata has been registered for class".
When I look at the documentation it seems like this procedure should be straightforward: "You can then go back to your data layer and retrieve the object as follows: Object obj = pm.getObjectById(id);"
I figure my problem is that I'm using a String instead of an ObjectId but I can't figure out the voodoo to make String IDs work. I read in the documentation "A DataNucleus extension is to pass in a String form of the identity to the above method" but we aren't using that extension, to the best of my knowledge.
Suggest you read the JDO spec which is very clear what is an "identity" (a String is not it), and what is a PK value. You don't post the class itself, so people are left to guesswork. To get an "identity" you can easily enough do
Object identity = pm.newObjectIdInstance(MyObject.class, "my_pk_value_when_string");
and that is what goes into pm.getObjectById(id). You then look at the log if having problems.
I've started to fiddle with mongo db and came up with a question.
Say, I have an object (POJO) with an id field (say, named 'ID') that I would like to represent in JSON and store/load in/from Mongo DB.
As far as I understood any object always has _id field (with underscore, lowercased).
What I would like to do is: during the query I would like the mongo db to return me my JSON with field ID instead of _id.
In SQL I would use something like
SELECT _id as ID ...
My question is whether its possible to do this in mongo db, and if it is, the Java based Example will be really appreciated :)
I understand that its possible to iterate over the records and substitute the _id with ID manually but I don't want this O(n) loop.
I also don't really want to duplicate the lines and store both "id" and "_id"
So I'm looking for solution at the level of query or maybe Java Driver.
Thanks in advance and have a nice day
Mongodb doesnt use SQL , its more like Object Query Language and Collections.
what you can try is , some thing similar to below code using Mongo Java Driver
Pojo obj = new PojoInstance();
obj.setId(id);
db.yourdb.find(obj);
I've end up using the following approach in the Java Driver:
DBCursor cursor = runSomeQuery();
try {
while(cursor.hasNext()) {
DBObject dbObject = cursor.next();
ObjectId id = (ObjectId) dbObject.get("_id");
dbObject.removeField("_id");
dbObject.put("ID", id.toString());
System.out.println(dbObject);
}
} finally {
cursor.close();
}
I was wondering whether this is the best solution or I have other better options
Mark
Here's an example of what I am doing in Javascript. It may be helpful to you. In my case I am removing the _id field and aliasing the two very nested fields to display simpler names.
db.players.aggregate([
{ $match: { accountId: '12345'}},
{ $project: {
"_id": 0,
"id": "$id",
"masterVersion": "$branches.master.configuration.player.template.version",
"previewVersion": "$branches.preview.configuration.player.template.version"
}
}
])
I hope you find this helpful.
I am newbie to mongodb. May I know how to avoid duplicate entries. In relational tables, we use primary key to avoid it. May I know how to specify it in Mongodb using java?
Use an index with the {unique:true} option.
// everyone's username must be unique:
db.users.createIndex({email:1},{unique:true});
You can also do this across multiple fields. See this section in the docs for more details and examples.
A unique index ensures that the indexed fields do not store duplicate values; i.e. enforces uniqueness for the indexed fields. By default, MongoDB creates a unique index on the _id field during the creation of a collection.
If you wish for null values to be ignored from the unique key, then you have to also make the index sparse (see here), by also adding the sparse option:
// everyone's username must be unique,
//but there can be multiple users with no email field or a null email:
db.users.createIndex({email:1},{unique:true, sparse:true});
If you want to create the index using the MongoDB Java Driver. Try:
Document keys = new Document("email", 1);
collection.createIndex(keys, new IndexOptions().unique(true));
This can be done using "_id" field although this use is discouraged.
suppose you want the names to be unique, then you can put the names in "_id" column and as you might know "_id" column is unique for each entry.
BasicDBObject bdbo = new BasicDBObject("_id","amit");
Now , no other entry can have name as "amit" in the collection.This can be one of the way you are asking for.
As of Mongo's v3.0 Java driver, the code to create the index looks like:
public void createUniqueIndex() {
Document index = new Document("fieldName", 1);
MongoCollection<Document> collection = client.getDatabase("dbName").getCollection("CollectionName");
collection.createIndex(index, new IndexOptions().unique(true));
}
// And test to verify it works as expected
#Test
public void testIndex() {
MongoCollection<Document> collection = client.getDatabase("dbName").getCollection("CollectionName");
Document newDoc = new Document("fieldName", "duplicateValue");
collection.insertOne(newDoc);
// this will throw a MongoWriteException
try {
collection.insertOne(newDoc);
fail("Should have thrown a mongo write exception due to duplicate key");
} catch (MongoWriteException e) {
assertTrue(e.getMessage().contains("duplicate key"));
}
}
Theon solution didn't work for me, but this one did:
BasicDBObject query = new BasicDBObject(<fieldname>, 1);
collection.ensureIndex(query, <index_name>, true);
I am not a Java programmer however you can probably convert this over.
MongoDB by default does have a primary key known as the _id you can use upsert() or save() on this key to prevent the document from being written twice like so:
var doc = {'name': 'sam'};
db.users.insert(doc); // doc will get an _id assigned to it
db.users.insert(doc); // Will fail since it already exists
This will stop immediately duplicates. As to multithread safe inserts under certain conditions: well, we would need to know more about your condition in that case.
I should add however that the _id index is unqiue by default.
using pymongo it looks like:
mycol.create_index("id", unique=True)
where myCol is the collection in the DB
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import pymongo
myclient = pymongo.MongoClient("mongodb://localhost:27017/")
mydb = myclient["mydatabase"]
mycol = mydb["customers"]
mycol.create_index("id", unique=True)
mydict = {"name": "xoce", "address": "Highway to hell 666", "id": 1}
x = mycol.insert_one(mydict)
Prevent mongoDB to save duplicate email
UserSchema.path('email').validate(async(email)=>{
const emailcount = await mongoose.models.User.countDocuments({email})
return !emailcount
}, 'Email already exits')
May this help ur question...
worked for me..
use in user model.
refer for explaination
THANKS...