As a number of people have noted and encountered HashMap.put can go into an infinite execution loop when used concurrently (see GRIZZLY-1207, JGRP-525, possibly HHH-6414, and this SO answer).
HashMap is clearly documented as not thread safe. Obviously, the correct fix is to use a thread-safe implementation of Map, ConncurrentHashMap in particular. I'm more curious about the concurrent timing that causes the infinite loop. I encountered this loop recently with a Java 7 JRE and would like to understand the exact causes. For example, is this caused by multiple puts at the same time?
A look inside HashMap.put shows that HashMap.Entry contains a link to the next node (in the bucket?). I assume these links are getting corrupting to contain circular references, which is causing the infinite loop. However, I still don't understand exactly how that corruption is occurring.
To the contrary of what many people think, the main issue with multi-threading and HashMaps is not just a duplicate entry or a vanishing one... As you said, an infinite loop might occur when two or multiple Threads concurrently decide to resize the HashMap.
If the size of the HashMap passes a given threshold, several threads might end up trying to resize it at the same time, and if we are lucky enough (you deployed the code in production already) they will keep going forever...
The issue is caused by the way the void resize(int newCapacity); and void transfer(Entry[] newTable); are implemented, you can take a look at the openjdk source code yourself. A mix of bad luck, good timing, entries that get reversed (ordering is not required in this data structure) and that end up to mistakenly referring to each other while a thread keep going while(e != null)...
While I could try to give you an explanation myself, I want to give credit to Paul Tyma's post (I cannot do better than him anyway) where I learned how this worked the first time I decided to figure out why I wasn't hired for a job several months ago...
http://mailinator.blogspot.com/2009/06/beautiful-race-condition.html
As Paul says, the best word to describe this race is condition is: beautiful
Related
Can anyone please explain to me the consequences of mutating a collection in java that is not thread-safe and is being used by multiple threads?
The results are undefined and somewhat random.
With JDK collections that are designed to fail fast, you might receive a ConcurrentModificationException. This is really the only consequence that is specific to thread safety with collections, as opposed to any other class.
Problems that occur generally with thread-unsafe classes may occur:
The internal state of the collection might be corrupted.
The mutation may appear to be successful, but the changes may not, in fact, be visible to other threads at any given time. They might be invisible at first and become visible later.
The changes might actually be successful under light load, but fail randomly under heavy load with lots of threads in contention.
Race conditions might occur, as was mentioned in a comment above.
There are lots of other possibilities, none of them pleasant. Worst of all, these things tend to most commonly reveal themselves in production, when the system is stressed.
In short, you probably don't want to do that.
The most common outcome is it looks like it works, but doesn't work all the time.
This can mean you have a problem which
works on one machine but doesn't on another.
works for a while but something apparently unrelated changes and your program breaks.
whenever you have a bug you don't know if it's a multi-threading issue or not if you are not using thread safe data structures.
What can happen is;
you rarely/randomly get an error and strange behaviour
your code goes into an infinite loop and stops working (HashMap used to do this)
The only option is to;
limit the amount of state which is shared between threads, ideally none at all.
be very careful about how data is updated.
don't rely on unit tests, you have to understand what the code doing and be confident it will be behave correctly in all possible situations.
The invariants of the data structure will not be guaranteed.
For example:
If thread 2 does a read whilst thread 1 is adding to the DS thread 1 may consider this element added while thread 2 doesn't see that the element has been added yet.
There are plenty of data structures that aren't thread-safe that will still appear to function(i.e. not throw) in a multi threaded environment and they might even perform correctly under certain circumstances(like if you aren't doing any writes to the data structure).
To fully understand this topic exploring the different classes of bugs that occur in concurrent systems is recommended: this short document seems like a good start.
http://pages.cs.wisc.edu/~remzi/OSTEP/threads-bugs.pdf
I'm having a little problem at the moment. My latest assignment is to create a rock paper scissors program which will run concurrently and output various values. Sounds simple and trivial, I know, but a little fun and I have been allowed to use scala which I had no knowledge of so thought it would be interesting.
Now the error is confusing me. I am receiving a NullPointerException on Thread 10, and have spent an awful long time trying to find it to no success. I have 6 classes so it would be impractical of me to put all of the code in, but I will put a few snippets.
result = Shapes.Winner(player1.getChoice(), player2.getChoice())
This is the line giving the error. Player1 and Player2 are simply objects from a class I have made, and they have been dequeued in.
Now for the confusing part. If I add a simple line in another class:
println(wait.getChoice())
it all works. This line has no relation to the first line above, and was simply for testing. But now when I take it out, I get the error. I would just leave it, but as you can see, it prints a load of rubbish onto the terminal.
I really have no idea what I'm doing wrong, and would love some help.
Feel free to ask for more snippets of the code, and thank you in advance for any help.
The problem is that you use synchronized incorrectly. You should wrap all calls to Referee.queue into Referee.synchronized {} blocks in order to synchronization to work. Otherwise the calls are not synchronized, and it's possible for one thread to modify Referee.queue without other thread notice.
Otherwise you should use thread safe collection. For example when I substitute scala.collection.mutable.Queue with scala.collection.mutable.SynchronizedQueue in the Referee class everything works ok, and you don't need to synchronize access to queue.
Even better use java.util.concurrent.ConcurrentLinkedQueue instead, because
SynchronizedQueue is deprecated in scala_2.11
I'm currently reading about game development, every time I see a game loop implemented it is always the same way.
A while(true) block, with a sleep inside for the FPS.
And I'm asking myself, why shouldn't I use a scheduledExcecutor, it seems like the obvious choice?
I mean, I'm sure I'm wrong here, it's unlikely that I'm the first one to think of this, but WHY not ?
First of all, look at some of the evolution of the traditional game loop. Your while(true) + sleep example is the simplest one.
Then people noticed that games were affected by the speed of the computer they were running on. Look up "variable time step game loop" on Google, for examples of how this is dealt with.
Now read over the docs on the ScheduledExecutorService. Neither scheduleAtFixedRate nor scheduleWithFixedDelay are quite what you'd want for a game.
Also, scrolling down in the docs, you'll notice "If any execution of the task encounters an exception, subsequent executions are suppressed." This can cause your game loop to abruptly terminate if you don't handle every possible exception.
There's nothing to stop you from extending the ScheduledExecutorService and modifying it to suit your needs, but I wouldn't recommend it as-is.
The main reason is probably simply that most of the code you are reading was coded before (or following patterns that were coded before) the concurrent package came out/was in heavy use.
A secondary might be trust. When you are coding something that critical to your app you want to be sure exactly how it behaves. The while(true) loop makes it very clear exactly what's going on... You'd have to spend some time dealing with the various executors before you became truly comfortable with one.
There should be a solution that solves your problem, I'm sure you can find something that fires regularly, and has an adjustment in case one iteration takes a little longer than expected.
You may even want to build your own executor. The pattern itself is fine, it's just finding a way to make it work for you.
Similar to the question I posted yesterday, I have this problem that I just can't understand. The code is pretty simple and should (I think) generate a deadlock. I even reduced the number of accounts to 2, to increase the probability of deadlocks.
The code is really easy to understand but to put some context. I have a bank with accounts and I'm doing lots of transfers between accounts. The transfer method should generate a deadlock. Why isn't that happening?
I can only think that the code is running way too fast, but that seems improbable to happen all the time.
Here's the whole code:
http://pastebin.com/HWJpuT38
Problem is on this line:
mAccounts = new ArrayList<Account>(Collections.nCopies(slots, new Account()));
Basically, there is only one Account object, but lots of references to it. Thus you're only ever locking on a single object.
If you create lots of different Account objects, you should be able to see the deadlock quite quickly.
The only place where you have a 'contested' resource is where you synchronize on fromaccount and then on toaccount - everything else depends on one lock only.
If you had another method which synchronized on toaccount and then on fromaccount you might have a chance of causing deadlock, but as the code currently is it should be perfectly well-behaved.
I think you need to add some sort of sleep to your loop in AccountTransferRunnable otherwise the Scheduler will run the thread until end before starting the other one.
With a Sleep you will give the Scheduler the chance to switch to the other thread will the first one is still running, which will give your code the chance to run into a deadlock.
mAccounts = new ArrayList<Account>(Collections.nCopies(slots, new Account()));
http://download.oracle.com/javase/1.4.2/docs/api/java/util/Collections.html#nCopies%28int,%20java.lang.Object%29
You end up with a list of 2 references to the same object.
That object can only be locked by one thread at a time. You can never have a deadlock.
I assume you wanted to initialize mAccounts with 2 different instances of Account class.
I have a piece of code that looks like this:
Algorithm a = null;
while(a == null)
{
a = grid.getAlgorithm();
}
getAlgorithm() in my Grid class returns some subtype of Algorithm depending on what the user chooses from some options.
My problem is that even after an algorithm is selected, the loop never terminates. However, that's not the tricky bit, if I simply place a System.out.println("Got here"); after my call to getAlgorithm(), the program runs perfectly fine and the loop terminates as intended.
My question is: why does adding that magic print statement suddenly make the loop terminate?
Moreover, this issue first came up when I started using my new laptop, I doubt that's related, but I figured it would be worth mentioning.
Edit: The program in question is NOT multithreaded. The code for getAlgorithm() is:
public Algorithm getAlgorithm ()
{
return algorithm;
}
Where algorithm is initially null, but will change value upon some user input.
I believe the issue has to deal with how grid.getAlgorithm is executed. If there is very little cost associated with executing the method, then your while loop will cycle very quickly as long the method continues to return null. That is often referred to as a busy wait.
Now it sounds like your new laptop is encountering a starvation issue which didn't manifest on your old computer. It is hard to say why but if you look at the link I included above, the Wikipedia article does indicate that busy waits do have unpredictable behavior. Maybe your old computer handles user IO better than your new laptop. Regardless, on your new laptop, that loop is taking resources away from whatever is handling your user IO hence it is starving the process that is responsible for breaking the loop.
You are doing active polling. This is a bad practice. You should at least let the polling thread sleep (with Thread.sleep). Since println does some io, it probably does just that. If your app is not multithreaded it is unlikely to work at all.
If this loop is to wait for user input in a GUI then ouch. Bad, bad idea and even with Thread.sleep() added I'd never recommend it. Instead, you most likely want to register an event listener on the component in question, and only have the validation code fire off when the contents change.
It's more than likely you're program is locking up because you've reached some form of deadlock more than anything else, especially if your application is multithreaded. Rather than try to solve this issue and hack your way round it, I'd seriously consider redesigning how this part of the application works.
You should check getAlgorithm(), there must be something wrong in the method.
There are two scenarios:
Your code is really not meant to be multi-threaded. In this case you need to insert some sort of user input in the loop. Otherwise you might as well leave it as Algorithm a = grid.getAlgorithm(); and prevent the infinite loop.
Your code is multi-threaded in which case you have some sort of 'visibility' problem. Go to Atomicity, Visibility and Ordering or read Java Concurrency in Practice to learn more about visibility. Essentially it means that without some sort of synchronization between threads, the thread you are looping in may never find out that the value has changed due to optimizations the JVM may perform.
You did not mention any context around how this code is run. If it is a console based application and you started from a 'main' function, you would know if there was multi-threading. I am assuming this is not the case since you say there is no multithreading. Another option would be that this is a swing application in which case you should read Multithreaded Swing Applications. It might be a web application in which case a similar case to swing might apply.
In any case you could always debug the application to see which thread is writing to the 'algorithm' variable, then see which thread is reading from it.
I hope this is helpful. In any case, you may find more help if you give a little more context in your question. Especially for a question with such an intriguing title as 'Weird Java problem, while loop termination'.