We are using mongo db to store certain records in production database.
We see our records having "_id" : { "$oid" : "50585fbcb046b2709a534502"} in production database , while we see same record as "_id" : ObjectId(" 50585fbcb046b2709a534502 ") in the qa database.
For dates we see "ld" : { "$date" : "2011-12-03T17:00:00Z"} in prod database, while "ld" :ISODate("2011-12-03T17:00:00Z") in qa database.
We have tested our queries successfully in qa environment, but worried it might fail in production
1) Will my java queries work seamlessly on prod & qa both? (I am using morphia apis to query)
2) Are they internally being stored in the same identical way?
To answer the two questions:
Yes they will
Yes they are the same, it is merely the representation within the item you are looking in (console or app) as to how they display. Console (later versions anyway, about 1.4+) will display ObjectId and ISODate (normally) whereas picking it out directly from the server language (Java in your case) will tend to show the full objects properties ($oid and $date in this case).
Related
I have have a mongo collection that keeps state records for devices. Thus, there could be multiple records per device. What I would like to do is create a query through the mongoTemplate that gets the latest record for each device.
Here's the constraints:
Pass in a Set<'String'> name_ids, regular field within mongo collection not the _id or found within the _id
get only the latest record for each device with matching name_id
return List<'DeviceStateData'> (No duplicates should be found with the same name_id)
example of collection object:
{
_id: "241324123412",
name_id: "flyingMan",
powerState:"ON",
timeStamp: ISODate('')
}
Thanks
You should look on Distinct function.
Here you can find details with Spring.
I've setup full text search and MongoDB and it's working quite well (Mongo 2.6.5).
However it does an OR instead of and AND.
1) Is it possible to make the query an AND query, while still getting all the benefits of full text search (stemming etc.)
2) And if so, is it possible to add this option via the Morphia wrapper library
EDIT
I see that the full text search includes a 'score' for each document returned. Is it possible to only return docs with a certain score or above. Is there some score that would represent a 'fuzzy' and query. That is usually all tokens are in the document but not absolutely always. If so this would solve the problem as well.
Naturally if possible to do this via Morphia that would be super helpful. But I can use the native java driver as well.
Any pointers in the correct direction, much appreciated.
EDIT
Code looks like this, I'm using Morphia 1.0.1:
Datastore ds = Dao.instance().getDatabase();
Query<Product> q = ds.createQuery(Product.class).search("grey vests");
List<Product> prods = q.asList();
Printing the query gives:
{ "$text" : { "$search" : "grey vests"}}
Note: I am able to do take an intersection of multiple result sets to create an AND query. However this is very slow since something like "grey" will return a massive result set and be slow at feeding the results back.
EDIT
I've tried to chain the search() calls and add a single 'token' to each call. But I am getting a run time error. Code becomes:
q.search("grey").search("vests");
The query I get is (which seems like it's doing the right thing) ...
{ "$and" : [ { "$text" : { "$search" : "grey"}} , { "$text" : { "$search" : "vests"}}]}
The error is:
com.mongodb.MongoQueryException: Query failed with error code 17287 and error message 'Can't canonicalize query: BadValue Too many text expressions' on server ...
at com.mongodb.connection.ProtocolHelper.getQueryFailureException(ProtocolHelper.java:93)
I know it is a non-relational database but this does not mean that relational data does not exist.
For example, I have a table that holds urls like this ( simplified ):
url | domain
and I have a table that holds domains like this ( simplified ):
domain | favicon_path
Because many different urls may have the same domain, I did not want to repeat the favicon_path for each domain when pulling the data for sending to the view.
Hence I used a simple ( simplified for example ) join command when I need the data.
"SELECT bookmarks.*, domains.favicon FROM bookmarks JOIN
domains ON bookmarks.domain=domains.domain"
How would I handle this scenario using no-sql?
I plan on implementing no-sql using indexedDB on the client ( javascript ) and MongoDB on the server ( java ).
If you want to use document-oriented DB, you can use this structure of documents:
URL_ID : {
"domain":"id_of_domain",
"another_staff": "..."
}
DOMAIN_ID : {
"favicon_path" : "path or id of another document",
"another_staff": "..."
}
So you can get document with URL_ID by id from database and then get document of type Domain.
ADDITION:
You can use the following approach for generating id. Create special document (like sequence) which will have only one field - current_value_of_sequence. Every insert to DB you have to get this sequence and increment it. Some DB like Couchbase have low-level support of this mechanism, which very efficient and thread-safety.
From years of work expierence in IT area, I would say most of the business models could be normalized as simple as these two types of data structure:
Entity info.
Entity list.
For example, in a book store business, we will have the Book entity, and many list that containing all of the books or a subset of the whole books.
With a NoSQL database, such as Redis or SSDB, the Book entity is stored with Key-Value, where key is the book sn, and value is the stringified book info(title, publish date, description, etc). While book list(list by publish date, list by price, etc) are stored in zset data type.
We have a user_audit entity(table) in mongodb which stores all the login/logout information.
For example,
"_id" : ObjectId("5228cf0156961de6693b74c0"),
"active" : true,
"fname" : "Tom",
"lastlogin" : ISODate("2013-09-05T18:35:45.608Z"),
"lastloginip" : "0:0:0:0:0:0:0:1",
"lname" : "Bailey",
"lastlogout" : ISODate("2013-09-05T18:36:45.568Z"),
There are thousands of records in this table in production.
Now, the admin wants to look for all the logins on a particular date. i am not able to look for exact match of date because of the "time" information attached to ISODate in the "lastlogin" field. In Java, new Date() had been used to insert this field value.
The issue is the time information keeps changing for logins on a particular day.
Tried,
query.put("lastlogin", new BasicDBObject("$lte", givenDate).append("$gte", givenDate));
Obviously it does not return results because the time part of the date does not match.
The query passed from Java to mongo is:
query={ "lastlogin" : { "$lte" : { "$date" : "2013-09-05T04:00:00.000Z"} , "$gte" : {
"$date" : "2013-09-05T04:00:00.000Z"}}}
[Note: It defaults to 04:00:00.000Z always if we format with MM_dd_yyyy in java, not sure why..]
The issue is we have a lot of records in production database. For fresh records, i can blank out the time portion before inserting from Java. But for existing records, not sure how to handle it. How can i get the records on a given date?
According to mongodb cookbook, you are in the right direction. You may just query for something like
query.put("lastlogin", new BasicDBObject("$lt", givenDatePlusOneDay).append("$gte", givenDateAt00h00min));
which is an interval and not the same date.
I have let's say two pc's.PC-a and PC-b which both have the same application installed with java db support.I want from time to time to copy the data from the database on PC-a to database to PC-b and vice-versa so the two PC's to have the same data all the time.
Is there an already implemented API in the database layer for this(i.e 1.export-backup database from PC-a 2.import-merge databases to PC-b) or i have to do this in the sql layer(manually)?
As you mention in the comments that you want to "merge" the databases, this sounds like you need to write custom code to do this, as presumably there could be conficts - the same key in both, but with different details against it, for example.
In short: You can't do this without some work on your side. SalesLogix fixed this problem by giving everything a site code, so here's how your table looked:
Customer:
SiteCode varchar,
CustomerID varchar,
....
primary key(siteCode, CustomerID)
So now you would take your databases, and match up each record by primary key. Where there are conflicts you would have to provide a report to the end-user, on what data was different.
Say machine1:
SiteCode|CustomerID|CustName |phone |email
1 XXX |0001 |Customer1 |555.555.1212 |darth#example.com
and on machine2:
SiteCode|CustomerID|CustName |phone |email
2 XXY |0001 |customer2 |555.555.1213 |darth#nowhere.com
3 XXX |0001 |customer1 |555.555.1212 |darth#nowhere.com
When performing a resolution:
Record 1 and 3 are in conflict, because the PK matches, but the data doesnt (email is different).
Record 2 is unique, and can freely exist in both databases.
There is NO way to do this automatically without error or data corruption or referential integrity issues.
I guess you are using Java DB (aka Derby) - in which case, assuming you just can't use a single instance, you can do a backup/restore.
Why dont you have the database on one pc. and have all other pc's request data from the host pc