Is an attached jOOQ Record (UpdatableRecord) thread-safe, i.e. can I attach (fetch) a Record in one thread, and store it later in another thread without negative effects? Should I detach it in the original thread and attach it back in the new thread?
I know about the jOOQ manual page about thread-safety of the DSLContext. I'm using the Spring Boot Autoconfiguration of jOOQ, so that should all be thread-safe (with Spring's DataSourceTransactionManager and Hikari pooling).
But the following questions remain:
How does an attached Record behave when a transaction in the original thread is opened, and store() is called in another thread either before or after the original transaction has been committed? Does jOOQ open a new connection every time for each operation?
Would the attached Record be keeping a connection open across threads, which might then lead to resource leaks?
A jOOQ record is not thread safe. It is a simple mutable container backed by an ordinary Object[]. As such, all the usual issues may arise when sharing mutable state across threads.
But your question isn't really about the thread safety of the record.
How does an attached Record behave when a transaction in the original thread is opened, and store() is called in another thread either before or after the original transaction has been committed? Does jOOQ open a new connection every time for each operation?
This has nothing to do with Record, but how you configure jOOQ's ConnectionProvider. jOOQ doesn't hold a connection or even open one. You do that, explicitly, or implicitly, by passing jOOQ a connection via a ConnectionProvider (probably via some Spring configured DataSource). jOOQ will, for each database interaction, acquire() a connection, and release() it again after the interaction. The Record doesn't know how this connection is obtained. It just runs jOOQ queries that acquire and release connections.
In fact, jOOQ doesn't even really care about your transactions (unless you're using jOOQ's transaction API, but you aren't).
Would the attached Record be keeping a connection open across threads, which might then lead to resource leaks?
No, a Record is "attached" to a Configuration, not a connection. That Configuration contains a ConnectionProvider, which does whatever you configured it to do.
Related
I am using ORMLite (Java) with HikariCP and I have a lot of code that's called asynchronously i.e. CompletableFuture.supplyAsync(Supplier). Inside a Supplier I have various calls to different ORMLite Dao objects inside these tasks. I ran into a deadlock scenario and found that increasing the size of the connection pool seemed to have fixed the issue.
After some research, I found that this is a well-documented problem. Having looked at the information provided here:
"Each thread is waiting for a second database connection, and none would become available since all threads are blocked."
This caught my eye. When looking at the ORMLite source code, I found that for each Dao method called, a connection is acquired. In my case, I am calling multiple Dao methods in each asynchronous task I create via CompletableFuture which brings me to my question.
My understanding is that I should only acquire a single connection in each thread so is there a way to do this with ORMLite? Bearing in mind I have calls to different Dao objects and having done some further research, Dao#callBatchTasks(Callable) caught my eye but I am unsure whether I can have calls to different Dao objects inside the Callable. Or is there a flaw in my logic or something fundamental I am missing in my write-up?
I found that for each Dao method called, a connection is acquired
If ORMLite needs to go to the database, it needs to get a connection, right? Is there something that it is not doing right?
My understanding is that I should only acquire a single connection in each thread so is there a way to do this with ORMLite?
I don't think that there is anything that says that a thread can't open multiple database connections. As long as they are all closed when you are done with te operation, then I think this is fine. If you are querying different DAOs that all live on the same database then you can actually reuse the connection each time as long as you are done with it. I.e. if you are paging across the results using an open connection then in the middle query another table, then this might screw up the cursor. However, if you do a dao1.queryAll() and then a dao2.queryAll(), this can use the same connection since the first query will have finished.
But if you are using a connection pool, the pool should do the connection management for you with help from ORMLite's ConnectionSource. The pool is supposed to create a new one for you if needed and otherwise reuse old connections. It's a bit strange to have a limit on your database connections.
Dao#callBatchTasks(Callable) caught my eye but I am unsure whether I can have calls to different Dao objects inside the Callable.
You can call different dao objects as long as they are all on the same database. You actually could call a dao with connections to a different database, but you'd have no guarantee around transactions between the databases.
For example if you did something like the following.
You call dao1.callBatchTasks(callable)
In your Callable, you call dao2.delete(entity1) which works.
Still in the Callable, you call dao2.delete(entity2) which throws.
The dao1 transaction will be rolled back, but the entity1 would still be deleted.
The scenario you quoted in the statement
Each thread is waiting for a second database connection, and none
would become available since all threads are blocked.
would result in a deadlock only when a thread is holding a lock to connection A AND trying to acquire Connection B, which might be held by another thread which might be trying to get hold of Connection A. As long as you are not "holding" one resource while trying to "grab" the other one, you should be good.
Context
I have a RESTful API for a versus fighting game, using JAX-RS, tomcat8 and Neo4j embedded.
Today I figured that a lot of queries will be done in a limited time, I'm using embedded for faster queries but I still want to go as fast as possible.
Problem
In fact, the problem is a bit different but not that much.
Actually, I'm using a Singleton with a getDabatase() method returning the current GraphDatabaseServiceinstance to begin a transaction, once it's done, the transaction is closed... and that's all.
I don't know if the best solution for optimal perfs is a Singleton pattern or a pool one (like creating XX instances of database connection, and reuse them when the database operation is finished).
I can't test it myself actually, because I don't have enough connections to even know which one is the fastest (and the best overall).
Also, I wonder if I create a pool of GraphDatabaseService instances, will they all be able to access the same datas without getting blocked by the lock?
Crate only one on GraphDatabaseService instance and use it everywhere. There are no need to create instance pool for them. GraphDatabaseService is completely thread-safe, so you can not worry about concurrency (note: transaction are thread-bound, so you can't run multiple transactions in same thread).
All operations in Neo4j should be executed in Transaction. On commit transaction is written in transaction log, and then persisted into database. General rules are:
Always close transaction as early as possible (use try-with-resource)
Close all resources as early as possible (ResourceIterator returned by findNodes() and execute())
Here you can find information about locking strategy.
To be sure that you have best performance, you should:
Check database settings (memory mapping)
Check OS settings (file system)
Check JVM settings (GC, heap size)
Data model
Here you can find some articles about Neo4j configuration & optimizations. All of them have useful information.
Use a pool - definitely.
Creating a database connection is generally very expensive. Using a pool will ensure that connections are kept for a reasonable mount of time and re-used whenever possible.
I want to use Spring-Hibernate and JDBC together in my application.
Hibernate should do all the updating and writing from one thread and other threads should just be able to read from the database without too much synchronization effort.
Will those JDBC-using threads deliver correct results (if they read from the database a short time after calling persist() or merge()) or could it happen, that Hibernate
has not flushed any updates and therefore other threads return wrong database entries?
"Wrong" depends on the isolation level you set for your connection pool.
I think it can work if Hibernate and Spring share the same connection pool and you set the isolation level to SERIALIZABLE for all connections.
Long-running transactions will be the problem. If all your write operations are fast you won't block. If you don't commit and flush updates quickly the read operations will either have to block and wait OR allow "dirty reads".
That depends. You're basically describing a race condition - if you want to make sure that your read-thread only reads after the write-thread has persisted, you will have to look into thread synchronization methodology.
Cheers,
On a Java EE server using CMT, I am using ehcache to implement a caching layer between the business object layer (EJBs) and the Data Access layer (POJOs using JDBC). I seem to be experiencing a race condition between two threads accessing the same record while using a self-populating Ehcache. The cache is keyed on the primary key of the record.
The scenario is:
The first thread updates the record in the database and removes the record from cache (but the database commit doesn't necessarily happen immediately - there may be other queries to follow.)
The second thread reads the record, causing the cache to be re-populated.
The first thread commits transaction.
This is all happening in a fraction of a second. It results in the cache being out of sync with the database, and subsequent reads of the record returning the stale cached data until another update is performed, or the entry expires from the cache. I can handle stale data for short periods (the typical length of a transaction), but not minutes, which is how long I would like to cache objects.
Any suggestions for avoiding this race condition?
UPDATE:
Clearing the cache after the transaction has committed would certainly be ideal. The question is, in a J2EE environment using CMT, when the caching layer is sandwiched between the business layer (stateless session EJBs) and the data access layer, how to do this?
To be clear about the constraints this imposes, the method call in question may or may not be in the same transaction as additional method calls that happen before or after. I can't force a commit (or do this work in a separate transaction) since that would change the transaction boundaries from what the client code expects. Any subsequent exceptions would not roll back the entire transaction (unneseccarily clearing the cache in this case is an acceptable side-effect). I can't control the entry points into the transaction, as it is essentially an API that clients can use. It is not reasonable to push the resonsiblity of clearing the cache to the client application.
I would like to be able to defer any cache clearing operations until the entire transaction is committed by the EJB container, but I have found no way to hook into that logic and run my own code with a stateless session bean.
UPDATE #2:
The most promising solution so far, short of a major design change, is to use ehcache 2.0's JTA support: http://ehcache.org/documentation/apis/jta
This means upgrading to ehcache 2.x and enabling XA transactions for the database as well, which could potentially have negative side-effects. But it seems like the "right" way.
You are using transactions - it makes more sense to remove the cache after the commit, that is when the change really happens.
That way you see the old data only during the length of the transaction, and all reads afterwards have the latest view.
Update: Since this is CMT specific, you should look at the SessionSynchronization interface, and it's afterCompletion() method. This is showed in this tutorial.
Just wondering if beginning a new transaction in Hibernate actually allocates a connection to the DB?
I'm concerned b/c our server begins a new transaction for each request received, even if that request doesn't interact with the DB. We're seeing DB connections as a major bottleneck, so I'm wondering if I should take the time narrow the scope of my transactions.
Searched everywhere and haven't been able to find a good answer. The very simple code is here:
SessionFactory sessionFactory = (SessionFactory) Context.getContext().getBean("sessionFactory");
sessionFactory.getCurrentSession().beginTransaction();
sessionFactory.getCurrentSession().setFlushMode(FlushMode.AUTO);
thanks very much!
a
According to the section 11.1. Session and transaction scopes of the Hibernate documentation:
A SessionFactory is an
expensive-to-create, threadsafe
object, intended to be shared by all
application threads. It is created
once, usually on application startup,
from a Configuration instance.
A Session is an inexpensive,
non-threadsafe object that should be
used once and then discarded for: a
single request, a conversation or a
single unit of work. A Session will
not obtain a JDBC Connection, or a
Datasource, unless it is needed. It
will not consume any resources until
used.
In order to reduce lock contention in
the database, a database transaction
has to be as short as possible. Long
database transactions will prevent
your application from scaling to a
highly concurrent load. It is not
recommended that you hold a database
transaction open during user think
time until the unit of work is
complete.
Now, to answer your question:
getting a Session does not immediately acquire a connection (the connection is lazy loaded)
but calling beginTransaction() will cause the load of the connection for the given Session
subsequent calls will reuse the same connection
Look at org.hibernate.impl.SessionImpl#beginTransaction() and go through the code for more details.
(Updated per Pascal Thivent's comment)
Each Session creates a database connection if there is a need for that - e.g. if a transaction is started. The connection is not opened with the mere creation of the session.
To overcome this, you can use a connecetion pool so that connections are reused. Or you can make sure (as it appears you did) that no transaction is started automatically.
(This discusses read-only transactions. Take a look.)