We Keep Coding

Run functions concurrently in java with wait [duplicate]

During the course of my program execution, a number of threads are started. The amount of threads varies depending on user defined settings, but they are all executing the same method with different variables.
In some situations, a clean up is required mid execution, part of this is stopping all the threads, I don't want them to stop immediately though, I just set a variable that they check for that terminates them. The problem is that it can be up to 1/2 second before the thread stops. However, I need to be sure that all threads have stopped before the clean up can continues. The cleanup is executed from another thread so technically I need this thread to wait for the other threads to finish.
I have thought of several ways of doing this, but they all seem to be overly complex. I was hoping there would be some method that can wait for a group of threads to complete. Does anything like this exist?

Just join them one by one:
for (Thread thread : threads) {
thread.join();
}
(You'll need to do something with InterruptedException, and you may well want to provide a time-out in case things go wrong, but that's the basic idea...)

If you are using java 1.5 or higher, you can try CyclicBarrier. You can pass the cleanup operation as its constructor parameter, and just call barrier.await() on all threads when there is a need for cleanup.

Have you seen the Executor classes in java.util.concurrent? You could run your threads through an ExecutorService. It gives you a single object you can use to cancel the threads or wait for them to complete.

Define a utility method (or methods) yourself:
public static waitFor(Collection<? extends Thread) c) throws InterruptedException {
for(Thread t : c) t.join();
}
Or you may have an array
public static waitFor(Thread[] ts) throws InterruptedException {
waitFor(Arrays.asList(ts));
}
Alternatively you could look at using a CyclicBarrier in the java.util.concurrent library to implement an arbitrary rendezvous point between multiple threads.

If you control the creation of the Threads (submission to an ExecutorService) then it appears you can use an ExecutorCompletionService
see ExecutorCompletionService? Why do need one if we have invokeAll? for various answers there.
If you don't control thread creation, here is an approach that allows you to join the threads "one by one as they finish" (and know which one finishes first, etc.), inspired by the ruby ThreadWait class.
Basically by newing up "watching threads" which alert when the other threads terminate, you can know when the "next" thread out of many terminates.
You'd use it something like this:
JoinThreads join = new JoinThreads(threads);
for(int i = 0; i < threads.size(); i++) {
Thread justJoined = join.joinNextThread();
System.out.println("Done with a thread, just joined=" + justJoined);
}
And the source:
public static class JoinThreads {
java.util.concurrent.LinkedBlockingQueue<Thread> doneThreads =
new LinkedBlockingQueue<Thread>();
public JoinThreads(List<Thread> threads) {
for(Thread t : threads) {
final Thread joinThis = t;
new Thread(new Runnable() {
#Override
public void run() {
try {
joinThis.join();
doneThreads.add(joinThis);
}
catch (InterruptedException e) {
// "should" never get here, since we control this thread and don't call interrupt on it
}
}
}).start();
}
}
Thread joinNextThread() throws InterruptedException {
return doneThreads.take();
}
}
The nice part of this is that it works with generic Java threads, without modification, any thread can be joined. The caveat is it requires some extra thread creation. Also this particular implementation "leaves threads behind" if you don't call joinNextThread() the full number of times, and doesn't have an "close" method, etc. Comment here if you'd like a more polished version created. You could also use this same type of pattern with "Futures" instead of Thread objects, etc.

Why does value not change inside thread?

When i assign a new value to variable it doesn't change after start(), however after i use join() it does. Why does this happen and in this case should int a be volatile or not?
class SampleThread extends Thread {
private static int a = 0;
#Override
public void run() {
a = 3;
}
public static void main(String[] args) throws InterruptedException {
Thread t2 = new Thread(new SampleThread());
t2.start();
System.out.println(a);
}
}

for seeing what is going on, try this:
...
#Override
public void run() {
System.out.println("start of Thread");
a = 3;
System.out.println("end of Thread");
}
...
only run method changed, rest of code unchanged

Yes, it needs volatile.
Every thread has an evil coin. The thread flips the coin anytime it reads or writes a field: Heads, and the thread uses its own local (to the thread) copy of it; if writing, that update simply does not reflect to all the other threads, who will still see the 'old' value, and if reading, same deal: Reads whatever it had, even though other threads updated it already. Even if they did so an hour ago. Tails, and it does refresh other threads' views of this thing, and won't use the local copy.
The coin is evil: it is not a fair coin. It will work every time today, and every time tomorrow, and every time during the test suite, and all throughout that week you have it live for the early adopting customers. And then juuust as that big client comes in and you're giving the demo? It flips to break your app every time. That kind of evil.
So, you must eliminate all coin flips, or at least ensure that the result of a coin flip does not affect your app whatsoever.
The way to do this, is to establish comes-before relationships. Between any 2 lines of java code as executed by a VM, there is a set of rules to determine if these 2 lines have such a relationship: That one is guaranteed to be run after the other. It's not about whether they did (timestamps of when they ran is completely irrelevant), it's whether the Java Memory Model decrees that such a relationship exists.
If yes, no coin is flipped: Anything the 'line that came before as per the JMM' did, is definitely visible to the line that came after. But if the JMM does not explicitly spell out that this relationship exists, the coin is flipped, and you lose.
One trivial 'comes before' relation ship is within a single thread: x = 5; System.out.println(x); trivially has such a relationship; they ran in the same thread, one came after the other. That's the freebie.
But between threads, oh dear. You need synchronized, volatile, or call code that does these things internally or has other mechanisms to ensure it (tip: There is lots of great stuff in the java.util.concurrent package, and as the name suggests, it's generally thread safe in very efficient ways. For example, an AtomicInteger is almost always far better than a volatile int, and can do far more, such as CAS operations, which volatile ints cannot do.

If you expect things in different threads to happen in a particular order -- in this case, that a = 2 is executed before `System.out.println(a)' -- then you have to write code to make that order happen.
In this trivial case, where no real work is being done, almost anything you can do makes the use of threads pointless. The main thread could 'join' the thread that's setting a to 2, but then all you've achieved is an expensive way to execute code that could be executed in a single thread.

main thread should wait that the other was executed
One solution is to use join()
class SampleThread extends Thread {
private static int a = 0;
#Override
public void run() {
a = 3;
}
public static void main(String[] args) throws InterruptedException {
Thread t2 = new Thread(new SampleThread());
t2.start();
// wait for completion of t2
t2.join()
System.out.println(a);
}
}

Why did the definers of the Java Language Specification decide not to allow a Java Thread to restart? [duplicate]

I know that it is not possible to restart a used Java Thread object, but I don't find an explanation why this is not allowed; even if it is guaranteed that the thread has finished (see example code below).
I don't see why start() (or at least a restart()) method should not be able to somehow reset the internal states - whatever they are - of a Thread object to the same values they have when the Thread object is freshly created.
Example code:
class ThreadExample {
public static void main(String[] args){
Thread myThread = new Thread(){
public void run() {
for(int i=0; i<3; i++) {
try{ sleep(100); }catch(InterruptedException ie){}
System.out.print(i+", ");
}
System.out.println("done.");
}
};
myThread.start();
try{ Thread.sleep(500); }catch(InterruptedException ie){}
System.out.println("Now myThread.run() should be done.");
myThread.start(); // <-- causes java.lang.IllegalThreadStateException
} // main
} // class

I know that it is not possible to
restart a used Java Thread object, but
I don't find an explanation why this
is not allowed; even if it is
guaranteed that the thread has
finished (see example code below).
My guestimation is that Threads might be directly tied (for efficiency or other constrains) to actual native resources that might be re-startable in some operating systems, but not in others. If the designers of the Java language had allowed Threads to be re-started, they might limit the number of operating systems on which the JVM can run.
Come to think of it, I cannot think of a OS that allows a thread or process to be restarted once it is finished or terminated. When a process completes, it dies. You want another one, you restart it. You never resurrect it.
Beyond the issues of efficiency and limitations imposed by the underlying OS, there is the issue of analysis and reasoning. You can reason about concurrency when things are either immutable or have a discrete, finite life-time. Just like state machines, they have to have a terminal state. Is it started, waiting, finished? Things like that cannot be easily reasoned about if you allow Threads to resurrect.
You also have to consider the implications of resurrecting a thread. Recreate its stack, its state, is is safe to resurrect? Can you resurrect a thread that ended abnormally? Etc.
Too hairy, too complex. All that for insignificant gains. Better to keep Threads as non-resurrectable resources.

I'd pose the question the other way round - why should a Thread object be restartable?
It's arguably much easier to reason about (and probably implement) a Thread that simply executes its given task exactly once and is then permanently finished. To restart threads would require a more complex view on what state a program was in at a given time.
So unless you can come up with a specific reason why restarting a given Thread is a better option than just creating a new one with the same Runnable, I'd posit that the design decision is for the better.
(This is broadly similar to an argument about mutable vs final variables - I find the final "variables" much easier to reason about and would much rather create multiple new constant variables rather than reuse existing ones.)

Because they didn't design it that way. From a clarity standpoint, that makes sense to me. A Thread represents a thread of execution, not a task. When that thread of execution has completed, it has done its work and it just muddies things were it to start at the top again.
A Runnable on the other hand represents a task, and can be submitted to many Threads as many times as you like.

Why don't you want to create a new Thread? If you're concerned about the overhead of creating your MyThread object, make it a Runnable and run it with a new Thread(myThread).start();

Java Threads follow a lifecycle based on the State Diagram below. Once the thread is in a final state, it is over. That is simply the design.

You can kind of get around this, either by using a java.util.concurrent.ThreadPoolExecutor, or manually by having a thread that calls Runnable.run() on each Runnable that it is given, not actually exiting when it is finished.
It's not exactly what you were asking about, but if you are worried about thread construction time then it can help solve that problem. Here's some example code for the manual method:
public class ReusableThread extends Thread {
private Queue<Runnable> runnables = new LinkedList<Runnable>();
private boolean running;
public void run() {
running = true;
while (running) {
Runnable r;
try {
synchronized (runnables) {
while (runnables.isEmpty()) runnables.wait();
r = runnables.poll();
}
}
catch (InterruptedException ie) {
// Ignore it
}
if (r != null) {
r.run();
}
}
}
public void stopProcessing() {
running = false;
synchronized (runnables) {
runnables.notify();
}
}
public void addTask(Runnable r) {
synchronized (runnables) {
runnables.add(r);
runnables.notify();
}
}
}
Obviously, this is just an example. It would need to have better error-handling code, and perhaps more tuning available.

If you are concerned with the overhead of creating a new Thread object then you can use executors.
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
public class Testes {
public static void main(String[] args) {
Executor executor = Executors.newSingleThreadExecutor();
executor.execute(new Testes.A());
executor.execute(new Testes.A());
executor.execute(new Testes.A());
}
public static class A implements Runnable{
public void run(){
System.out.println(Thread.currentThread().getId());
}
}
}
Running this you will see that the same thread is used for all Runnable objects.

A Thread is not a thread. A thread is an execution of your code. A Thread is an object that your program uses to create and, manage the life-cycle of, a thread.
Suppose you like playing tennis. Suppose you and your friend play a really awesome set. How would your friend react if you said, "That was incredible, let's play it again." Your friend might think you were nuts. It doesn't make sense even to talk about playing the same set again. If you play again you're playing a different set.
A thread is an execution of your code. It doesn't make sense to even talk about "re-using" a thread of execution for same reason that it makes no sense to talk about re-playing the same set in tennis. Even if another execution of your code executes all the same statements in the same order, it's still a different execution.
Andrzej Doyle's asked, "Why would you want to re-use a Thread?" Why indeed? If a Thread object represents a thread of execution---an ephemeral thing that you can't even talk about re-using---then why would you want or expect the Thread object to be re-useable?

i've been searching the same solution which you seem to be looking for, and i resolved it in this way. if you occur mousePressed Event you can terminate it also reuse it, but it need to be initialized, as you can see below.
class MouseHandler extends MouseAdapter{
public void mousePressed(MouseEvent e) {
if(th.isAlive()){
th.interrupt();
th = new Thread();
}
else{
th.start();
}
}
}

join() doesn't block other threads (except main)?

Our teacher gave us the following code:
public static void main(String[]args) {
Thread a = new Thread(new T(2));
Thread b = new Thread(new T(5));
a.start();
b.start();
try {
a.join(); //Thread a now runs completely to the end, before the main-method gets back to a "runnable" state
b.join(); //Thread b runs to death before the main methods u
} catch (InterruptedException ie) {}
System.out.println("done"); //Result: Random Thread a and b outputs
//and in the end "done" from main
}
public class T extends Thread {
private int nr;
public T(int nr) {
this.nr = nr;
}
public void run() {
for (int i=0; i<10; i++) {
System.out.println("Hello " + nr + " " + i);
}
}
Thread a and b are the same and both write (in a for-loop) 10 prints to the console.
Thread a and b were finished, before the main method stopped and all results were random except the main method.
My question was, if it shouldn't also block the other threads(not just main), if you call join() on one thread. He said, that join() just freezes the main method. But for what reason should this be good? He also said, that this is totally random and managed by the scheduler, which doesn't make sense for this part in my opinion (the scheduler commands the thread-states, this is clear, but not after calling join(), at least not for the java application. Or am I false?). My point would be, that Thread a and b ran completely to the end, before the main-thread even called the join method. Javadoc tells me the same, if I understand it correct.
I hope someone of you can give me an answer. :)

The call to join() on an instance of Thread will not complete until the thread corresponding to that instance dies.
Corrollary 1: if that thread is already dead, join() returns immediately.
Corrollary 2: no threads except the current are affected by this call.
He also said, that this is totally random and managed by the scheduler
You probably didn't catch exactly what the teacher said here. Thread scheduling, which means making decisions when a thread will be given some CPU time to run, and how much of it, is done by the thread scheduler. It is definitely not "totally random" and for most practical considerations, all threads run all the time. Again, this has little to do with the behavior of the join method.

The point of join is not to give a single thread priority over all others. Rather, it's to express that one thread needs to wait for another thread to complete before that (first) thread can go on. It's not always the main thread calling join. It's a single constraint being placed on the scheduler: "Don't do A until you've done B". Of course, by using multiple joins, you can accomplish more complex dependencies.

I suspect that the point your teacher was trying to make is that you cannot assume anything other than the contract of join. I.e. the main thread will not continue until a has run to completion.
It is quite possible for a.join() to allow b to continue but it is also possible for it to completely block b until a is complete.
If you tested this code on a single-core machine it is actually quite likely that a.join() will exclude b but on a multi-core machine it may not.

Simple thread problem in java

Works except for when I free the crawler:
public void setCrawlerFree(WebCrawler w)
{
synchronized(myFreeCrawlers)
{
synchronized(numToGo)
{
myFreeCrawlers.add(w);
myFreeCrawlers.notifyAll();
numToGo--;
numToGo.notify();
}
}
}
When the crawler is done, I can add it back on the list. I also want to subtract 1 from the number of things I still need to do. I have one main thread waiting until numToGo is at 0. I'm getting an IllegalMonitorStateException on numToGo.notify() but since its inside of the synchronization block, doesn't that mean I own it?

Consider rewriting it to ExecutorService.
ThreadPoolExecutor executor = new ThreadPoolExecutor(corePoolSize,
maximumPoolSize, keepAliveTime, timeUnit,
new LinkedBlockingQueue<Runnable>());
executor.submit(new Callable<...>() { ... });
It would greatly simplify your code and eliminate thread synchronization issues.

So I thought I needed to call wait and
notify on the object that all the
threads have in common, but that's not
correct either.
Yes, it is. But:
public class IllegalMonitorStateException
extends RuntimeException
Thrown to indicate that a thread has
attempted to wait on an object's
monitor or to notify other threads
waiting on an object's monitor without
owning the specified monitor.
You need to synchronize on an object before calling wait() or notify() on it.

Is your numToGo field is a primitive type which is being wrapped ? (int to Integer, long to Long, etc). Remember these wrappers are immutable and will cause you to have different object every time the boxing/unboxing happens. It's always recommended to use final objects when synchronization is needed.
Instead of using and integer create your own object to maintain the value and synchronization.
class Counter {
private int value ;
private final Object lock = new Object() ;
public ExecutionStatus() { }
public void increment() {
synchronized(lock) {
value ++ ;
}
}
public void decrease() {
synchronized(lock) {
value-- ;
}
}
// Read dirty
public int count() {
return value ;
}
public int safeCount() {
synchronize(lock) {
return count() ;
}
}
}
Never the less, you can add the line private final Object lock = new Object() to your current code and use that to control the synchronization of the numToGo variable.
Hope this helps.

you are synchronising on a non-final member variable. Your sync(numToGo) syncs on some value of numToGo and then you change the reference: numToGo--. You now have a different instance on which you call notify, hence the exception.

Some good posts there, there are plenty of alternatives but I imagine this is some kind of academic exercise? As people have pointed out, you'd probably wouldn't use wait/notify/notifyAll when there are more modern alternatives that makes things easier. The wait/notify thing though is interesting and is well worth understanding as a basis for concurrency work.
I'm assuming this is some kind of consumer/producer thing? One thread is trapping a crawler, the other setting it free? If that's the case, you might want to wait for the trap to have occupants before setting free? it might look something like this...
private final List<Object> trap = new ArrayList<Object>();
public class BugCatcher {
public void trapCrawler(Object crawler) {
synchronized (trap) {
trap.add(crawler);
System.out.println("caught bug number " + trap.size() + "!");
trap.notifyAll();
}
}
}
public class Hippy {
public void setCrawlerFree(Object crawler) throws InterruptedException {
synchronized (trap) {
trap.wait();
trap.clear();
System.out.println("set bugs free! time to hug a tree");
}
}
}
If the BugCatcher can catch bugs quicker than the hippy releases them, the hippy waits for the trap to have something in it before attempting to release the bugs (hence the wait call).
If you leave out the wait/notify part, things will rely just on the synchronized keyword, only one thread will access the trap at a time and its a race as to which gets there first (the hippy might try an empty an already empty trap).
In order to co-ordinate the wait and notify, the VM will use an object monitor. A thread acquires the object's monitor when it enters a synchronized block. An object has just a single monitor which acts as a mutually exclusivity lock (mutex). If you try and wait or notify without first getting the object's monitor (without executing wait or notify within a synchronized block), the VM can't set things up and so throws the IllegalMonitorException. It's saying "I can't allow this because if, for example, I wait, when will I know that I can progress when somebody calls notify? what/who are they notifying?". It uses the monitor to coordinate and so forces you to acquire the monitor.
So, the error you get is because numToGo isn't syncrhonised in the other thread (as Michael said previously).
I can't quite see why you need the numToGo, if it is producer/consumer, do you want to stop after a certain number? After the bug catcher catches 10 bugs and the hippy releases 10? Doesn't sound like that's what you're trying to do (as they could both have unrelated internal counters), so I'm not sure what you trying to do there. It'd be good to outline what you're trying to do in case I've gone off on completely the wrong tangent!