I just have a problem relative to concurrency whose logic flow is when a client (called Oracle Forms) will submit a request （called concurrent program ) and call a plsql procedure, this procedure eventually will call a java static method.
What I find is that when I submit two request in the same time or in a very short interval(like 1 second), some concurrency problem will be noticed.
The java method is the start point of doing something that search from the database suggest that which records should be inserted into database.
The problem is that, they will result in duplicated result since when I query, both request find it fine to insert new records.
I tried add synchronized in the static java method, but this does not solve this problem, why?
What I do is:
public static synchronized void execute
Please note that the insert will be called in plsql, which means I do a not sufficient synchronize if only synchronize the java method. But when I look into the log, it shows the two request run in the same second, which I do not think it is normal! since query database and doing the suggestion is time consuming.
To make the java method really time consuming, I add a code call Thread.sleep(5000), and log for the time after this code and log the thread id.
Surprise to see that the Thread id is 1! And also, the time where they pass the sleep is in the same time. Why is that?
What can I do to solve the problem? Any lock on the java method or pl sql?
PS: I am now trying to use DMBS_LOCK, and which seems to be working but I still hope to know the reason why the java method is not synchronized.
I have no idea how the JVM inside the Oracle DB is implemented, but since (at least in some common configurations) every database connection gets its own server process, then if a separate JVM is embedded into each of those, a synchronized block won't do you much good. You'd need to use database locks.
Assuming that the calls to the Java static method are done within the same classloader, then synchronized is all you need.
Your logging may be faulty. How exactly are you logging?
Your statement about the database lookup being "time consuming" is not convincing. Databases tend to cache data, for example.
In a nutshell: if, by your definition, an "atomic operation" is a combination of lookup + insert, then you should "synchronize" over both. Acquiring a database lock seems like a reasonable way to go about it.
Related
I know synchronized keyword makes method run only on single class at a time. But here is the problem.
I have a database class with methods e.g. insertAccount, updateSetting, etc. If I make insertAccount, updateSetting synchronized, each of them will be able to run only on one thread at a time.
If there was one method for whole database, it would be great, but there are not one. If one thread calls insertAccount and another thread calls updateSetting at the same time, it will go bad, right?
Because only one of these methods can be run at any time. So what do I do?
Is there a way to apply something like synchronized to the whole class? So that if 1st thread calls insertAccount and 2nd thread calls updateSetting at the same time, 2nd thread has to wait until 1st thread finishes accessing database.
The real answer here: step back and do some studying. You should not be using synchronized here, but rather look into a lock object that a reader/writer needs to acquire prior turning to that "DB class". See here for more information.
On the other hand, you should understand what transactions are, and how your database supports those. Meaning: there are different kinds of problems; and the different layers (application code, database) have different responsibilities.
You see, using "trial and error" isn't an approach that will work out here. You should spend some serious time studying the underlying concepts. Otherwise you are risking to damage your data set; and worse: you risk writing code that works fine most of the time; but fails in obscure ways "randomly". Because that is what happens when multiple threads manipulate shared data in an uncontrolled manner.
You misunderstood how synchronized work.
If you mark two method of class by synchronized only one of them could be executed at any moment of time (except if you invoke wait).
Also note that if you have several instances of this class you can execute methods of different instances simultaneously.
#Test(singleThreaded = true) Use above annotation above class and its tests will be run using a single thread even though you have used parallel="methods" in your testng.xml file
After I create a new object 'Order', I would like to get its generated ID and put it on an AMQP queue, so that a worker can do other stuff with it. The worker takes the generated ID (message) and looks up the order but complains that no record exists, even though I just created one. I am trying to figure out either how long to wait for after I call my .persist() before I put the message (generated ID) on the queue (which I dont think is a good idea); have the worker loop over and over until mysql DOES return a record (which I dont like either); or find a point where I can put the message on the queue after I know the data is safe in mysql (this sounds best). Im thinking that it needs to be done outside of any #Transactional method.
The worker that is going to read the data back out of mysql is part of a different system on a different server. So when can I tell the worker that the data is in mysql so that it can get started with its task?
Is it true that after the #Transactional method finishes the data is done being written to mysql, I am having trouble understanding this.
Thanks a million in advanced.
Is it true that after the #Transactional method finishes the data is
done being written to mysql, I am having trouble understanding this.
Thanks a million in advanced.
So first, as Kayamann and Ralf wrote in comments, it is guaranteed that data is stored and available for other processes when the transaction commits (ends)
#Transactional methods are easy to understand. When you have #Transactional method, it means that the container (application that is going to actually invoke that method) will begin the transaction before the method is invoked, and auto commit or rollback the transaction in case of success or error.
So if we have
#Transactional
public void modify(){
doSomething();
}
And when you call somewhere in the code (or invokation via contaier eg due to some bindings) the actuall frol will be as follows
tx=entityManager.beginTransaction();
object.modify();
tx.commit();
There is quite simple. Such approach will mean that transactions are Container Controlled
As four your situation, well to let your external system know that transaction has been complete, you have to either use message queue (that you are using already) with the message that transaction is complete for some id and it can start processing stuff, or use different technology, REST for example.
Remote systems can signal eachoter for various of events via queues and REST services, so there is no difference.
I'm making a series of connections asynchronously via MySQL, and I have a class which contains a bunch of easy-accesible static methods to update/remove/clear/get/etc data.
The issue I'm confronted with is that the getter methods won't return the proper value (practically ever) because they are returned before the async connection gets a chance to update the value to be returned.
Example:
public static int getSomething(final UUID user)
{
Connection c = StatsMain.getInstance().getSQL().getConnection();
PreparedStatement ps;
try
{
ps = c.prepareStatement("select something from stats where uuid=?");
ps.setString(1, user.toString());
ResultSet result = ps.executeQuery();
return result.getInt("something");
}
catch (SQLException e)
{
return false;
}
}
(Not copy & pasted, but pretty close)
I realize I can use a 'callback' effect by passing an interface to the method and doing such, but that becomes very tedious when the database stores 10 values for a key.
Sounds like you're looking for Futures since Java 6 or CompletableFuture, which is new in Java 8
Solution 1:
The best method I've come up with is have a thread with a loop in it that waits for MySQL to return values and responds to each value. This is rather like the callback in the get routine, but you only have the one loop. Of course, the loop has to know what to do with each possible returned piece of data.
This means rethinking a bit how your program works. Instead of: ask a question, get an answer, use the answer, you have two completely independent operations. The first is: ask a question, then forget about it. The second is: get an answer, then, knowing nothing about the question, use the answer. It's a completely different approach, and you need to get your head around it before using it.
(One possible further advantage of this approach is that MySQL end can now send data without being prompted. You have the option of feeding changes made by another user to your user in real time.)
Solution 2:
My other solution is simpler in some ways, but it can have you firing off lots of threads. Just have your getSomething method block until it has the answer and returns. To keep your program from hanging, just put the whole block of code that calls the method in its own thread.
Hybrid:
You can use both solutions together. The first one makes for cleaner code, but the second lets you answer a specific question when you get the reply. (If you get a "Customer Name" from the DB, and you have a dozen fields it could go in, it might help to know that you did ask for this field specifically, and that you asked because the user pushed a button to put the value in a specific text box on the screen.)
Lastly:
You can avoid a lot of multithreading headaches by using InvokeLater to put all changes to your data structures on your EventQueue. This can nicely limit the synchronization problems. (On the other hand, having 20 or 30 threads going at once can make good use of all your computer's cores, if you like to live dangerously.)
You may want to stick with synchronized calls, but if you do want to go asynchronous, this is how I'd do it. It's not too bad once you get some basic tools written and get your brain to stop thinking synchronously.
I'm maintaining a very old application, and recently I came across a 'multi thread' bug.
Here, in a method, to insert a value into a db, the record is first checked that it exists or not, then if it does not exist, it is inserted into the db.
createSomething(params)
{
....
....
if( !isPresentInDb(params) )
{
.....
.....
.....
insertIntoDb(params)
}
...
}
Here when multiple threads invoke this method, two or more threads with same params may cross the isPresentInDb check, one thread inserts successfully, the other threads fail.
To solve this problem I enclosed both the db interactions into a single synchronized(this) block. But is there a better way of doing this?
Edit: it is more like selective synchronization, only threads with same params need to by synchronized. Is selective synchronization possible?
I'd say the better way to do this would be to let the database do it for you if at all possible. Assuming the row on the database that you are wanting to either update or insert has a unique constraint on it, then my usual approach would be
unconditionally insert the row
if an SQLException occurs, check to see if it is due to a duplicate key on insert error, if it is, do the update, otherwise rethrow the SQLException.
If you can wrap those statements in a database transaction, then you don't have to worry about two threads trampling on each other.
If the logic is really "create this if it doesn't already exist", it could be better still to push the logic down into the database. For example, MySQL has "INSERT IGNORE" syntax that will cause it to ignore the insert if it would violate a primary key constraint. It may not be possible for your code, but worth considering.
This way of doing would only work if this object instance is the only one which inserts something in the table. If it's not, then two threads will synchronize on two different objects, and the synchronization won't work. To make it short : the object should be a singleton, and no other object should insert into this table.
Even if there is a unique object instance inserting, if you have any other application, or any other JVM, inserting in this table, then the synchronization won't bring you any guarantee.
Doing this is better than nothing, but doesn't guarantee that the insert will always succeed. If it doesn't, then the transaction will rollback due (hopefully) to a constraint violation. If you don't have any unique constraint to guarantee the uniqueness in the database, and you have several applications inserting in parallel, then you can't do anything to avoid duplicates.
Since you only want to forbid this method from running with the same params, you can use a ConcurrentMap instead and then call putIfAbsent and check its return value before proceeding. This will allow you to run the method concurrently for different arguments.
Looks fine to me. You can use some of the java.util.concurrent aids, like a ReentrantLock.
It will be better to utilize some sort of optimistic transactions: try to insert, and catch an exception. If the records has just been inserted, simply do nothing.
In one word NO. there is not better way than this. Since to make check-then-update kind of operations atomic you must have to put the logic inside a synchronized block.
You could make the whole method synchronized. I tend to find that a good marker for "this method only gets run by one thread at a time". That's my personal preference though.
The downside of too coarse-grained locking is performance degradation. If the method is called often, it will become a performance bottleneck. Here are two other approaches:
Move your concurrent code into your database statement, if possible.
Use a non-blocking data structure such as ConcurrentMap and maintain a list of known entries (must be warmed up on startup). This allows you two run the method with minimal locking, and without synchronizing the code. An atomic putIfAbsent() can be used to check if it must be added or not.
As others have stated your current approach is fine. Although depending on your requirements there are other things to consider
Is this the only place in your application where you insert these records into the db? If no then the insert could still fail even with synchronisation
How often does theoperation fail? If the number of times the operations fail compared to the number of times you run the method it may be beneficial to detect the failure by catching an appropriate exception. This may be beneficial due to the overhead involved in synchronising threads.
What does your application need to do when it detects this kind of failure?
On first sight your solution seems ok, but if you want to change it here are two options:
use db transactions
use locks from java.util.concurrent.locks
Lock lock = new ReentrantLock();
.....
createSomething(params)
{
....
....
try {
lock.lock();
if( !isPresentInDb(params) )
{
.....
.....
.....
insertIntoDb(params)
}
finally {
lock.unlock;
}
}
Suppose that I have a method called doSomething() and I want to use this method in a multithreaded application (each servlet inherits from HttpServlet).I'm wondering if it is possible that a race condition will occur in the following cases:
doSomething() is not staic method and it writes values to a database.
doSomething() is static method but it does not write values to a database.
what I have noticed that many methods in my application may lead to a race condition or dirty read/write. for example , I have a Poll System , and for each voting operation, a certain method will change a single cell value for that poll as the following:
[poll_id | poll_data ]
[1 | {choice_1 : 10, choice_2 : 20}]
will the JSP/Servlets app solve these issues by itself, or I have to solve all that by myself?
Thanks..
It depends on how doSomething() is implemented and what it actually does. I assume writing to the database uses JDBC connections, which are not threadsafe. The preferred way of doing that would be to create ThreadLocal JDBC connections.
As for the second case, it depends on what is going on in the method. If it doesn't access any shared, mutable state then there isn't a problem. If it does, you probably will need to lock appropriately, which may involve adding locks to every other access to those variables.
(Be aware that just marking these methods as synchronized does not fix any concurrency bugs. If doSomething() incremented a value on a shared object, then all accesses to that variable need to be synchronized since i++ is not an atomic operation. If it is something as simple as incrementing a counter, you could use AtomicInteger.incrementAndGet().)
The Servlet API certainly does not magically make concurrency a non-issue for you.
When writing to a database, it depends on the concurrency strategy in your persistence layer. Pessimistic locking, optimistic locking, last-in-wins? There's way more going on when you 'write to a database' that you need to decide how you're going to handle. What is it you want to have happen when two people click the button at the same time?
Making doSomething static doesn't seem to have too much bearing on the issue. What's happening in there is the relevant part. Is it modifying static variables? Then yes, there could be race conditions.
The servlet api will not do anything for you to make your concurrency problems disappear. Things like using the synchronized keyword on your servlets are a bad idea because you are basically forcing your threads to be processed one at a time and it ruins your ability to respond quickly to multiple users.
If you use Spring or EJB3, either one will provide threadlocal database connections and the ability to specify transactions. You should definitely check out one of those.
Case 1, your servlet uses some code that accesses a database. Databases have locking mechanisms that you should exploit. Two important reasons for this: the database itself might be used from other applications that read and write that data, it's not enough for your app to deal with contending with itself. And: your own application may be deployed to a scaled, clustered web container, where multiple copies of your code are executing on separate machines.
So, there are many standard patterns for dealing with locks in databases, you may need to read up on Pessimistic and Optimistic Locking.
The servlet API and JBC connection pooling gives you some helpful guarantees so that you can write your servlet code without using Java synchronisation provided your variables are in method scope, in concept you have
Start transaction (perhaps implicit, perhaps on entry to an ejb)
Get connection to DB ( Gets you a connection from pool, associated with your tran)
read/write/update code
Close connection (actually keeps it for your thread until your transaction commits)
Commit (again maybe implictly)
So your only real issue is dealing with any contentions in the DB. All of the above tends to be done rather more nicely using things such as JPA these days, but under the covers thats more or less what's happening.
Case 2: static method, this presumably implies that you now keep everything in a memory structure. This (barring remote invocation of some sort) impies a single JVM and you managing your own locking. Should your JVM or machine crash I guess you lose your data. If you care about your data then using a DB is probably better.
OR, how about a completely other approach: servlet simply records the "vote" by writing a message to a persistent JMS queue. Have some other processes pick up the votes from the queue and adds them up. You won't give immediate feedback to the voter this way, but you decouple the user's experience from the actual (in similar scenarios) quite complex processing .
I thing that the best solution for your problem is to use something like "synchronized" keyword and wait/notify!