I have an application which gets data from a database (Mongo) when a user connects, and saves it when a user disconnects and at fixed intervals to reduce the likelihood of data loss if a server goes down. I am using data access objects to save users to the database which updates every field regardless of if it has been changed. This can lead to problems such as when a user joins multiple servers and makes changes on one of them but the changes are overwritten when the user disconnects from another.
Are there any established ways of persisting only modified fields or any frameworks that do this? I would rather not use a boolean for every field as I have many fields inside the User object and adding a dirty flag to each of them would increase the class size dramatically.
The steps your application takes:
User gets data from MongoDB
This data get's partially modified
The modifications should get saved
This means: The part of your application that modifies the data should take care of that.
The Spring team introduces some Diff tool, a few months ago: https://spring.io/blog/2014/10/22/introducing-spring-sync
Using that, you'll get a Patch object, which only contains the changes.
Patch patch = Diff.diff(original, modified);
Here's an approach that might work:
Object data = mongoClient.getData();
Object modifiedData = modify(data);
Patch patch = Diff.diff(data, modifiedData);
The patch now contains everything that has changed. Now you must somehow use the internals of the Patch object and map that to MongoDB's $set commands.
Related
I have a scenario where any update/change in the data by a cms user through application/CMS needs the approval of the admin/authorizer user. There may be multiple changes in one update in a single document/record. This approval will not be done in real-time and may take few hours or may be days. Authorizer may also reject the change. So in this case what would be the best way to keep this data alive without comitting it to the database till approval or rejection. Should I create temporary or duplicate tables to keep this data temporarily in the db? But this will result in large number of temporary tables(one for each table). Or is there any other option at developer/application/java end? I am using here Oracle with Java.
You need to better understand the problem.
You do not require one datastore,
you require two datastores.
Datastore one (possible table one) will contain unapproved changes.
This is the "proposed" state.
You will write and commit all data into this datastore as soon as the user requests the change.
Datastore two (possible table two) will contain the approved changes;
this is the "real" state.
Once a change that is in datastore one has been reviewed and approved,
you must apply the change here.
A possible other solution is to use a Kafka topic:
Use a Kafka topic to store the unapproved changes.
Feed the topic to reviewers.
When approved, note the decision (in the same topic) and write the change to the database.
Note:
datastore 1 and datastore 2 can be the same table,
just have a column to indicate "approved change",
"declined change",
and "pending change".
You can always have draft and final copy of the data. Draft copy will save your work in draft mode, committed and operation like save / confirm from app can copy this into final version.
This requires one more record to identify draft / final version and you should be using draft data to show on UI.
Let's assume there is a form frontend, which has several dropdowns with data(objects, not just strings) that likely not changing in the future, but it has reasonably size, so it looks a little bit weird putting it into frontend.
Do you create tables for these data backend and fetch it from there even though the backend likely not using or changing it ever?
Could you give me some resources where I can find about these conventions?
If you are the owner of this data it is more efficient to have this on frontend stored in some constants file, no problem whether they are objects or strings. For example,create class DropdownOption and store array of these objects.
If you decide to keep it in database and provide data via REST API count on the performance - every request will reach your endpoint first, create transaction, get data from db, close transaction, map objects to dtos and only after that return to your frontend. More data more time.
Further from Ilia Ilin's answer, an additional thing to consider is if this data set is referenced anywhere, how you'd like the data to behave once a value is updated or removed.
If you load the data on the front end, then any modification will not apply to previously stored data.
If you store the data in a relational DB, fetch it in front end, any modification will cascade to all previous data references.
I have implement a grid which displays document metadata and the user is able to edit the document on right click. I wanted to implement a locking mechanism for this. What would be the best way to put a lock on the document when one user has opened the editor ? These documents do reside in the database.
Just add a column that specifies who currently has the file checked out. When a person tries to check out a file, if that column is set, they will not be able to check it out, and will be notified of who has it checked out. Unless you have thousands of requests per second for a single document, this method will work fine.
In addition to adding a column to say who has the file checked out and preventing access using that. You can add a timestamp for when the lock was requested.
This way, if someone requests it and the lock is, for example, 30 mins old with no changes made, they can take the lock. (If the original user didn't quit gracefully or something).
If the documents are in a database, the database itself should have support for preventing inconsistent access.
http://docs.oracle.com/javase/6/docs/api/java/sql/Connection.html#setTransactionIsolation%28int%29
If the editor does not keep database transactions/connections open for the duration of file editing, however, and the java application runs client-side rather than server-side (as you could simply create a lock in the editor for concurrency on the server side), then things get a bit trickier and I haven't yet had enough database experience to say how you would resolve that, as using a field in the database to indicate editing status would have concurrency problems with that type of setup (unless the database itself supports locking on records, but that would depend on the DB engine in use).
Oh, one possibility would be to use file modification times (have a timestamp field in the database and update it each time a file is modified) and keep a no-dirty-reads-allowed transaction in use while checking the timestamp and determining if the file was modified by another user after the user attempting to save last accessed it; if so, it won't save the file to the database and will instead alert the user that the server-side file was changed and ask if they want to view the changes (similar to how version control systems work). By disallowing dirty reads for all such transactions, that should prevent other users from changing the file's record while the first transaction is open (to mark a record as "dirty", you could perhaps use a dummy field that would be updated at the start of each transaction with some random value). (Note: aglassman's answer would work similarly to this.)
I wrote an application that uses JPA (and hibernate as persistence provider).
It works on a database with several tables.
I need to create an "offline mode", where a copy of the programa, which acts as a client, allows the same functionality while keeping their data synchronized with the server when it is reachable.
The aim is to get a client that you can "detach" from the server, make changes on the data and then merge changes back. A bit like a revision control system.
It is not important to manage conflicts, in case the user will decide which version to keep.
My idea, but it can't work, was to assign to each row in the database the last edit timestamp. The client initially download a copy of the entire database and also records a second timestamp when it modify a row while non connected to the server. In this way, it knows what data has changed and the last timestamp where it is synchronized with the server. When you reconnect to the server, he will have to ask what are the data that have been changed since the last synchronization from the server and sends the data it has changed. (a bit simplified, but the management of conflicts should not be a big problem)
This, of course, does not work in case of deleting a row. If both the server or the client deletes a row they will not notice it and the other will never know.
The solution would be to maintain a table with the list of deleted rows, but it seems too expensive.
Does anyone know a method that works? there is already something similar?
Enver:
If you like to have a simple solution, you can create Version-Fields that acts like your "Timestamp".
Audit:
If you like to have a complex, powerfull solution, you should use the Hibernateplugin
I'm hoping for some advice or suggestions on how best to handle multi threaded access to a value store.
My local value storage is designed to hold onto objects which are currently in use. If the object is not in use then it is removed from the store.
A value is pumped into my store via thread1, its entry into the store is announced to listeners, and the value is stored. Values coming in on thread1 will either be totally new values or updates for existing values.
A timer is used to periodically remove any value from the store which is not currently in use and so all that remains of this value is its ID held locally by an intermediary.
Now, an active element on thread2 may wake up and try to access a set of values by passing a set of value IDs which it knows about. Some values will be stored already (great) and some may not (sadface). Those values which are not already stored will be retrieved from an external source.
My main issue is that items which have not already been stored and are currently being queried for may arrive in on thread1 before the query is complete.
I'd like to try and avoid locking access to the store whilst a query is being made as it may take some time.
It seems that you are looking for some sort of cache. Did you try to investigate existing cache implementation, maybe some of them will do?
For example Guava cache implementations seems to cover a lot of your requirements - http://code.google.com/p/guava-libraries/wiki/CachesExplained.