Concurrent Read/Write in MongoDB - java

I have a collection from which I am getting max id and while inserting using max id + 1. The id column is unique in this collection.
When multiple instances of this service is invoked the concurrent application reads the same collection and gets the max id. But since the same collection is accessed the same max id is returned to multiple instances, can I get an explicit lock on the collection while reading the data from this collection and release the lock after writing in Mongo DB?

Using mongoDB method collections.findAndModify() you can create your own "get-and-increment" query.
For example:
db.collection_name.findAndModify({
query: { document_identifier: "doc_id_1" },
update: { $inc: { max_id: 1 } },
new: true //return the document AFTER it's updated
})
https://docs.mongodb.com/manual/reference/method/db.collection.findAndModify/
Take a look at this page for more help:
https://www.tutorialspoint.com/mongodb/mongodb_autoincrement_sequence.htm

Try this approach
Instead of getting the max id in read of the collection and increment it as max id + 1.
While read for multiple instances just give the document/collection, and while updating follow the below logic
Let us have the below part in a synchronized block, so that no two threads gets the same max id
synchronize() {
getMaxId from collection
increase it by 1
insert the new document
}
Please refer:
https://docs.mongodb.com/v3.0/tutorial/create-an-auto-incrementing-field/
https://www.tutorialspoint.com/mongodb/mongodb_autoincrement_sequence.htm
Hope it Helps!

Related

synchronized method for accessing database with spring and hibernate

I have a table that maintains a sequence number that is used as an identifier for multiple tables (multiple invoice tables all the tables are using single sequence).
Whenever i want to insert a new record in invoice table I read the current sequence number from the table and update it with +1.
The problem is when there are multiple requests for new invoice numbers the sequence number returns duplicate numbers.I tried synchronized block but still it returning duplicate values when multiple requests are hitting at same time.
Here is the method to retrieve the sequence number
synchronized public int getSequence(){
Sequence sequence = getCurrentSession().get(Sequence.class,1); //here 1 is the id of the row
int number = sequence.getSequenceNumber();
sequence.setSequenceNumber(number+1);
getCurrentSession().saveOrUpdate(sequence);
return number;
}
Is there something I am missing?
First of all I won't recommend you to use table implementation of the sequence. Explanation why
But if you have to - hibernate knows how to manage it. Take a look
And one more thing. I strongly recommend you to implement synchronization on the data base side. Imagine you have 2 instances of your application connected to the same database instance and working simultaneously.
Using transactions also not worked for me. I tried all the isolations in mysql but nothing helps me. I solved it with below solution.
synchronized public int getSequence(){
Sequence sequence = getCurrentSession().get(Sequence.class,1); //here 1 is the id of the row
int prevNumber = sequence.getSequenceNumber();
Query<Sequence> query = getCurrentSession().createQuery("UPDATE Sequence SET sequenceNumber = :number WHERE sequenceNumber = :prevNumber",Sequence.class);
query.setParameter("number",prevNumber+1);
query.setParameter("prevNumber",prevNumber);
int affectedRows = query.executeUpdate();
if(accectedRows > 0)
return number;
else
throw new Exception();
}
So whenever a conflict happens it will throw an exception.

Firestore order by and limit not working in kotlin

I have the following firestore setup:
-Root
- Queue
- item1
- time : 20
- item2
- time : 1
- 2000 more items, with a random time value
What i want is to show 40 items, with smallest time first so i do the following in kotlin:
val ref = firestore.collection("Queue")
orderBy?.let{
ref.orderBy(it)
}
limit?.let{
ref.limit(it)
}
return ref.get().get().toObjects(Queue::class.java)
It actually completly ignore my order by and limit statements. and is returning all items in the Queue collection, what am i doing wrong.
The documentation here:
https://firebase.google.com/docs/reference/android/com/google/firebase/firestore/Query
says that the orderBy and limit methods return a new query object, so maybe you should try
val ref = firestore.collection("Queue").orderBy("time").limit(40)
As per the update to your question, you could create a function that returns the query you want based on whether or not the orderBy and limit query modifiers are present. You would have to make that query object a var in order to make it mutable.

Dynamodb AWS Java scan withLimit is not working

I am trying to use the DynamoDBScanExpression withLimit of 1 using Java aws-sdk version 1.11.140
Even if I use .withLimit(1) i.e.
List<DomainObject> result = mapper.scan(new DynamoDBScanExpression().withLimit(1));
returns me list of all entries i.e. 7. Am I doing something wrong?
P.S. I tried using cli for this query and
aws dynamodb scan --table-name auditlog --limit 1 --endpoint-url http://localhost:8000
returns me just 1 result.
DynamoDBMapper.scan will return a PaginatedScanList - Paginated results are loaded on demand when the user executes an operation that requires them. Some operations, such as size(), must fetch the entire list, but results are lazily fetched page by page when possible.
Hence, The limit parameter set on DynamoDBScanExpression is the maximum number of items to be fetched per page.
So in your case, a PaginatedList is returned and when you do PaginatedList.size it attempts to load all items from Dynamodb, under the hood the items were loaded 1 per page (each page is a fetch request to DynamoDb) till it get's to the end of the PaginatedList.
Since you're only interested in the first result, a good way to get that without fetching all the 7 items from Dynamo would be :
Iterator it = mapper.scan(DomainObject.class, new DynamoDBScanExpression().withLimit(1)).iterator();
if ( it.hasNext() ) {
DomainObject dob = (DomainObject) it.next();
}
With the above code, only the first item will fetched from dynamodb.
The take away is that : The limit parameter in DynamoDBQueryExpression is used in the pagination purpose only. It's a limit on the amount of items per page not a limit on the number of pages that can be requested.

Mongo DB / No duplicates

I have have a mongo collection that keeps state records for devices. Thus, there could be multiple records per device. What I would like to do is create a query through the mongoTemplate that gets the latest record for each device.
Here's the constraints:
Pass in a Set<'String'> name_ids, regular field within mongo collection not the _id or found within the _id
get only the latest record for each device with matching name_id
return List<'DeviceStateData'> (No duplicates should be found with the same name_id)
example of collection object:
{
_id: "241324123412",
name_id: "flyingMan",
powerState:"ON",
timeStamp: ISODate('')
}
Thanks
You should look on Distinct function.
Here you can find details with Spring.

Concurrency - Getting the MongoDB generated ID of an object inserted via Java in a thread safe way

What is the best method to get the Mongo generated ID of a document inserted via Java.
The Java process inserting the documents is multi-thread, meaning that we need some atomic way to insert and return the ID of the object.
Also, if we setup a unique index, in the event that the object is a duplicate, will an ID be returned?
Thanks!
Generate the ObjectId early, use it in the insert, and there will no need to have the database return it to you.
ObjectId doesn't use a shared sequence number to be unique, so it doesn't matter if you generate one before inserting or retrieve it after.
public ObjectId createThing() {
ObjectId result = new ObjectId();
BasicDBObject thingToInsert = new BasicDbObject();
thingToInsert.put('_id', result);
//set other fields here
collection.insert(thingToInsert);
return result;
}
native ObjectId's which are generated by Mongo are globally unique and can be safely used from the multi-threaded application.
generated ObjectId can be obtained from the DbObject under _id key.
If inserted document violates a unique index constraint - java driver may throw an exception, depending on a value of WriteConcern:
http://api.mongodb.org/java/current/com/mongodb/WriteConcern.html
If it's value is higher then NORMAL- exception will be thrown.
WriteConcern can be specified for every individual insert (or update) method, or globally by using DBCollection.setWriteConcern
I retrieve the document with _id but when I get the data into my java class eg mobile, _id attribute which is of type ObjectID me I change it set the value of the document in mongodb.

Categories