In my Mongo database, I have a "Books" collection with a "category" field. In the past, "category" was an enum mapped as String in Mongo but now I need "category" to be an object within my "Books" collection. What can I do? Is there a query that can be executed to convert all documents at once?
Example, today category is like that:
"category" : "Sci-Fi and Fantasy"
But I need to convert to this:
"category" : {
"_id" : ObjectId("3f07bc56po324021df23a8f1"),
"code" : NumberLong(1),
"name" : "Sci-Fi and Fantasy"
}
MongoDB is no SQL you can have both kind of records present in your mongo collection, although while reading the records you might have to frame some logic for creating identical entities. This might cause issues if you search based on that particular property.
You can create a new property with different name keeping the category as well and then add the new property to all the records.
Related
I have so many documents in a collection and would like to change one of the filed name in all the documents. Also, want to change the value with some prefixed constant in all the docs.
Example,
{
"_id" : ObjectId("56e9e6e9083378ba4e5e8832"),
"name" : "Mike"
}
Should be changed to,
{
"_id" : ObjectId("56e9e6e9083378ba4e5e8832"),
"firstName" : "First-Mike"
}
I used the following Java code to rename the field,
final MongoDatabase mongoDb = mongo.getDatabase(database);
final MongoCollection<Document> collection = mongoDb.getCollection("<CollectionName>");
Bson rename = Updates.rename("name", "firstName");
collection.updateMany(new Document(), rename);
But not sure, how to change the value with some prefixed constants for all the documents in the collection.
I can iterate all the documents in the collection and do the change, but trying to understand, if there is any way we can do this without iterating all the documents, like single update.
Thanks
Why not use regular expressions for prefix matching? A single update command can do your job if you use mongodb's regex:
https://docs.mongodb.org/manual/reference/operator/query/regex/
At work, we supply files for other services. Size of them are between 5mo and 500mo.
We want to use Json instead of XML, but i m wondering how our customers could
upload those files in an easy way in their database, Oracle or Mysql.
I mean, i can t find on the web APi or functions or tools, in Mysql or oracle, to do that.
I know that it s easy to work data by data to load a small Json file, decode each
object or array and put them at the right place in database.
But is there an other way to do this, like sqlloader in Oracle ?
And if so, size of our file aren t they too big to produce JSON file, in JAVA for example ?
I guess it might be difficult to do this load job automatically, especially because of arrays like this :
{"employees":[
{"firstName":"John", "lastName":"Doe", "salaryHistory":[1000,2000,3000]},
{"firstName":"Anna", "lastName":"Smith", "salaryHistory":[500,800]},
{"firstName":"Peter", "lastName":"Jones", "salaryHistory":[400]}
]}
where salaryHistory must produce problems because their sizes are different, and data are not madatoryly
in the same table.
Any ideas or help would be welcomed !
Edit
i m looking for a solution to put each data in the good column of a table, i don t need to store a Json structure in a single column of simple table.
like this :
table employees : column are id, FirstName, lastName and
table salaryHistory : column are id, order, salary
and each data must go in the good column like "John" in firstname, "Doe" in lastname, then "1000" in a new row of table salaryHistory , "2000" in another new row of salaryHistory and so on.
Starting with MySQL 5.7 there is a new data type: JSON.
Take a look here for more details.
Example for Oracle 12c:
create table transactions (
id number not null primary key,
trans_msg clob,
constraint
check_json check (trans_msg is json)
);
regular insert:
insert into transactions
values
(
sys_guid(),
systimestamp,
'{
"TransId" : 3,
"TransDate" : "01-JAN-2015",
"TransTime" : "10:05:00",
"TransType" : "Deposit",
"AccountNumber" : 125,
"AccountName" : "Smith, Jane",
"TransAmount" : 300.00,
"Location" : "website",
"CashierId" : null,
"ATMDetails" : null,
"WebDetails" : {
"URL" : "www.proligence.com/acme/dep.htm"
},
"Source" : "Transfer",
"TransferDetails" :
{
"FromBankRouting" : "012345678",
"FromAccountNo" : "1234567890",
"FromAccountType" : "Checking"
}
}'
)
/
SQL*Loader control file and data file:
load data into table transactions
fields terminated by ','
(
trans_id sequence(max,1),
fname filler char(80),
trans_body lobfile(fname) terminated by EOF
)
There I am stuck, how to remove embedded document in mongodb. I am using spring data mongodb criteria, I am doing it like the following:
// database
"_id" : ObjectId("55683d51e4b0b6050c5b0db7"),
"_class" : "com.samepinch.domain.metadata.Metadata",
"preferenceType" : "SHOPPING",
"subtypes" : [
{
"_id" : ObjectId("55683d51e4b0b6050c5b0db6"),
"leftValue" : "VEG",
"rightValue" : "NON_VEG",
"preferencePoint" : 0
}
],
"createdDate" : ISODate("2015-05-29T10:20:01.610Z"),
"updatedDate" : ISODate("2015-05-29T10:20:01.610Z")
// query
mongoTemplate.updateMulti(new Query(),
new Update().pull("subtypes", Query.query(Criteria.where("subtypes._id").is(new objectId("55683d51e4b0b6050c5b0db6"))),Metadata.class);
What i am doing wrong?
Thanks in advance!
subtypes is in nested objects so you should first pass this in $elemMatch is matched first matching array elements of given conditions. Update query as :
db.updateMulti.update({"subtypes":{"$elemMatch":{"_id":ObjectId("55683d51e4b0b6050c5b0db6")}}},
{"$pull":{"subtypes":{"_id":ObjectId("55683d51e4b0b6050c5b0db6")}}})
this query pull exact matching array elements from subtypes array .
And with the help of this spring elemMatch ( not that much expertise in spring mongo ) I converted this query in spring format as below :
mongoTemplate.updateMulti(new Query(
where("subtypes").elemMatch(where("_id").is(ew objectId("55683d51e4b0b6050c5b0db6"))).pull(
pull("subtypes", Query.query(Criteria.where("_id").is(new objectId("55683d51e4b0b6050c5b0db6"))),Metadata.class
));
this above spring query not tested I hope you will convert mongo update query in spring mongo query format.
We have a user_audit entity(table) in mongodb which stores all the login/logout information.
For example,
"_id" : ObjectId("5228cf0156961de6693b74c0"),
"active" : true,
"fname" : "Tom",
"lastlogin" : ISODate("2013-09-05T18:35:45.608Z"),
"lastloginip" : "0:0:0:0:0:0:0:1",
"lname" : "Bailey",
"lastlogout" : ISODate("2013-09-05T18:36:45.568Z"),
There are thousands of records in this table in production.
Now, the admin wants to look for all the logins on a particular date. i am not able to look for exact match of date because of the "time" information attached to ISODate in the "lastlogin" field. In Java, new Date() had been used to insert this field value.
The issue is the time information keeps changing for logins on a particular day.
Tried,
query.put("lastlogin", new BasicDBObject("$lte", givenDate).append("$gte", givenDate));
Obviously it does not return results because the time part of the date does not match.
The query passed from Java to mongo is:
query={ "lastlogin" : { "$lte" : { "$date" : "2013-09-05T04:00:00.000Z"} , "$gte" : {
"$date" : "2013-09-05T04:00:00.000Z"}}}
[Note: It defaults to 04:00:00.000Z always if we format with MM_dd_yyyy in java, not sure why..]
The issue is we have a lot of records in production database. For fresh records, i can blank out the time portion before inserting from Java. But for existing records, not sure how to handle it. How can i get the records on a given date?
According to mongodb cookbook, you are in the right direction. You may just query for something like
query.put("lastlogin", new BasicDBObject("$lt", givenDatePlusOneDay).append("$gte", givenDateAt00h00min));
which is an interval and not the same date.
I'm just getting started with MongoDb and I've noticed that I get a lot of duplicate records for entries that I meant to be unique. I would like to know how to use a composite key for my data and I'm looking for information on how to create them. Lastly, I am using Java to access mongo and morphia as my ORM layer so including those in your answers would be awesome.
Morphia: http://code.google.com/p/morphia/
You can use objects for the _id field as well. The _id field is always unique. That way you kind of get a composite primary key:
{ _id : { a : 1, b: 1} }
Just be careful when creating these ids that the order of keys (a and b in the example) matters, if you swap them around, it is considered a different object.
The other possibility is to leave _id alone and create a unique compound index.
db.things.ensureIndex({firstname: 1, lastname: 1}, {unique: true});
//Deprecated since version 3.0.0, is now an alias for db.things.createIndex()
https://docs.mongodb.org/v3.0/reference/method/db.collection.ensureIndex/
You can create Unique Indexes on the fields of the document that you'd want to test uniqueness on. They can be composite as well (called compound key indexes in MongoDB land), as you can see from the documentation. Morphia does have a #Indexed annotation to support indexing at the field level. In addition with morphia you can define compound keys at the class level with the #Indexed annotation.
I just noticed that the question is marked as "java", so you'd want to do something like:
final BasicDBObject id = new BasicDBObject("a", aVal)
.append("b", bVal)
.append("c", cVal);
results = coll.find(new BasicDBObject("_id", id));
I use Morphia too, but have found (that while it works) it generates lots of errors as it tries to marshall the composite key. I use the above when querying to avoid these errors.
My original code (which also works):
final ProbId key = new ProbId(srcText, srcLang, destLang);
final QueryImpl<Probabilities> query = ds.createQuery(Probabilities.class)
.field("id").equal(key);
Probabilities probs = (Probabilities) query.get();
My ProbId class is annotated as #Entity(noClassnameStored = true) and inside the Probabilities class, the id field is #Id ProbId id;
I will try to explain with an example:
Create a table Music
Add Artist as a primary key
Now since artist may have many songs we have to figure out a sort key.
The combination of both will be a composite key.
Meaning, the Artist + SongTitle will be unique.
something like this:
{
"Artist" : {"s" : "David Bowie"},
"SongTitle" : {"s" : "changes"},
"AlbumTitle" : {"s" : "Hunky"},
"Genre" : {"s" : "Rock"},
}
Artist key above is: Partition Key
SongTitle key above is: sort key
The combination of both is always unique or should be unique. Rest are attributes which may vary per record.
Once you have this data structure in place you can easily append and scan as per your custom queries.
Sample Mongo queries for reference:
db.products.insert(json file path)
db.collection.drop(json file path)
db.users.find(json file path)