We Keep Coding

multithread: java condition await timeout but can't return

Lock sharedLock = new ReentrantLock();
Condition condition = lock.newCondition();
main thread:
sharedLock.lock();
childThread.start();
condition.await(5, TimeUnit.SECONDS);
sharedLock.unlock();
child thread:
sharedLock.lock();
//do something, may take a long time
Thread.sleep(10);// sleep to simulate a long execution
condition.signal();
sharedLock.unlock();
Suppose child thread send a network request and wait for response, I want main thread wait at most 5 seconds, if timeout, retry the request. but when the await() timeout, it cannot acquire lock because child thread still hold it, so it still wait the lock until child thread release it, which takes 10 seconds.
How can I achieve my requirement that main thread wait child thread's signal, but have a bounded timeout?

This is not how your are supposed to do it, you are supposed to:
Create an ExecutorService (thread pool) for that you should check the methods of the class Executors to choose the best one in your case but Executors.newFixedThreadPool is a good start
Submit your task as a FutureTask to the thread pool
Then call get with a timeout
Manage properly the TimeoutException
Here is how it could be done:
// Total tries
int tries = 3;
// Current total of tries
int tryCount = 1;
do {
// My fake task to execute asynchronously
FutureTask<Void> task = new FutureTask<>(
() -> {
Thread.sleep(2000);
return null;
}
);
// Submit the task to the thread pool
executor.submit(task);
try {
// Wait for a result during at most 1 second
task.get(1, TimeUnit.SECONDS);
// I could get the result so I break the loop
break;
} catch (TimeoutException e) {
// The timeout has been reached
if (tryCount++ == tries) {
// Already tried the max allowed so we throw an exception
throw new RuntimeException(
String.format("Could execute the task after %d tries", tries),
e
);
}
}
} while (true);
How can I achieve my requirement that main thread wait child thread's
signal, but have a bounded timeout?
Here is how you can achieve your requirements:
Main Thread:
lock.lock();
try {
childThread.start();
condition.await(5, TimeUnit.SECONDS);
} finally {
sharedLock.lock();
}
The child thread:
try {
//do something, may take a long time
Thread.sleep(10);// sleep to simulate a long execution
} finally {
// Here we notify the main thread that the task is complete whatever
// the task failed or not
lock.lock();
try {
condition.signal();
} finally {
lock.unlock();
}
}
As you can see to work, the task must not be performed within the critical section, we only acquire the lock to notify the main thread nothing more. Otherwise if you execute the task within the critical section after the timeout the main thread will still need to acquire the lock once again and since the lock is actually owned by the child thread, it will need to wait anyway until the end of the task which makes the timeout totally useless.
NB: I renamed sharedLock to lock as a ReentrantLock is an exclusive lock not as shared lock, if you need a shared lock check the class Semaphore to define the total amount of permits.

Your code can be simplified with intrinsic lock.
Object sharedObj = new Object();
main thread:
synchronized (sharedObj) {
int retryCount = 0;
while (retryCount < maxRetry) {
sharedObj.wait(5000);
retryCount++;
}
}
child thread:
synchronized (sharedObj) {
//do something, may take a long time
Thread.sleep(10);// sleep to simulate a long execution
sharedObj.notify();
}
java condition await timeout but can't return
That's because the lock must be released so wait/await can return. So your child thread should be like:
//do something, may take a long time
Thread.sleep(10);// sleep to simulate a long execution
synchronized (sharedObj) {
sharedObj.notify();
}
Java's wait/notify is usually used to solve producer-consumer problem. And usually sharedObj shouldn't be holded for too long. Then your main thread can hold the lock again when the wait timeout.
Take a look at an in-production example: hadoop/hdfs/DFSOutputStream.java
The logic is simple, the producer creates packet and put it in dataQueue
// takes a long time to create packet
synchronized (dataQueue) {
dataQueue.addLast(packet);
dataQueue.notifyAll();
}
The consumer wait while dataQueue is empty:
synchronized (dataQueue) {
while ((!shouldStop() && dataQueue.size() == 0 &&... ) {
try {
dataQueue.wait(timeout);
} catch (InterruptedException e) {
LOG.warn("Caught exception", e);
}
doSleep = false;
now = Time.monotonicNow();
}
As you can see, the dataQueue are unlocked for most of the time!
How can I achieve my requirement that main thread wait child thread's signal, but have a bounded timeout?
If your child thread are mostly in a loop, your Main thread can set a isRunning flag to make child thread stop by itself. If your child thread are mostly blocking by an I/O operation, your Main thread can interrupt the child thread.
The sharedObj is used for coordination and protects sharedObj. If there's other resources should be protected, you have 2 choices:
1. If the operation on the resource is quick, like ackQueue in DFSOutputStream.java, protect it together inside the sharedObj.
2. If the operation on the resource is time-consuming, do it and protect it outside the sharedObj.

The valid confusion in the question is because the "Thread.sleep(10)" is done inside the lock block. When await(long time, TimeUnit unit) has to return because of timeout, it still needs the lock. So, as suggested in the other answer the long running task should not be inside the lock for it to work properly.
But it would be nice to have proper documentation stressing this fact. For example, if we await(5, TimeUnit.SECONDS) i.e wait for 5 seconds and the lock is available 10 seconds after the call, it will still return false even though the lock is available now at the moment of return.

synchronized block locking object and wait/notify

According to what I understood, when I use a synchronized block it acquires the lock on an object and releases it when the code block is done executing. In the following code
public class WaitAndNotify extends Thread{
long sum;
public static void main(String[] args) {
WaitAndNotify wan = new WaitAndNotify();
//wan.start();
synchronized(wan){
try {
wan.wait();
} catch (InterruptedException ex) {
Logger.getLogger(WaitAndNotify.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println("Sum is : " + wan.sum);
}
}
#Override
public void run(){
synchronized(this){
for(int i=0; i<1000000; i++){
sum = sum + i;
}
notify();
}
}
}
what happens if the synchronized block inside the run method acquires the lock first? Then the synchronized block inside the main method has to wait (not because of the wait(), because the other thread acquired the lock). After the run method is done executing, won't the main method enter its synchronized block and wait for a notify which it will never get? What did I misunderstand here?

wait() implicitly exits the respective monitor temporarily and re-enters it upon returning:
See wait()
The current thread must own this object's monitor. The thread releases
ownership of this monitor and waits until another thread notifies
threads waiting on this object's monitor to wake up either through a
call to the notify method or the notifyAll method. The thread then
waits until it can re-obtain ownership of the monitor and resumes
execution.
That's why and how this sort of synchronization does work at all.

Yes, it's possible to perform a notify() before a wait() causing a hung thread, so you need to be careful that it can't happen.
For that reason (and others) it's generally better to use the higher level constructs of java.util.concurrent, since they generally give you less possibilities to shoot yourself in the foot.

You won't see the 'waiting forever' issue here, because you are calling the version of wait() with a timeout; so, after 5 seconds it returns even if it doesn't receive a notify. The 'wait forever' version of the wait() call could indeed exhibit the problem you describe.

You've got two threads here: your WaitAndNotify (WAN) thread, and Java's main execution thread. Both are vying for the same lock.
If the WAN thread gets the lock first, the main thread will be blocked. Being in a blocked state is NOT the same as being in a wait state. A thread in the wait state will wait for notification before moving forward. A thread in the blocked state will actively try to get the lock when it becomes available (and keep trying until it does).
Assuming the run method executes normally, it will call notify(), which will have no effect because no other threads are currently in a wait state. Even if there were, WAN still holds the lock until it exits the synchronized block of code. Once WAN exits the block, THEN Java would notify a waiting thread (if there was one, which there is not).
At this point, the main execution thread now obtains the lock (it is no longer blocked) and enters the wait state. Now you've used the version of wait that will wait up to 5000 milliseconds before continuing. If you used the vanilla version (wait()) it would wait forever because no other process would notify it.

Here is a version of the example program changed to introduce a loop that tests a condition variable. This way you avoid bad assumptions about the state of things after a thread re-acquires a lock upon waking from a wait, and there's no order dependence between the two threads:
public class W extends Thread {
long sum;
boolean done;
public static void main(String[] args) throws InterruptedException {
W w = new W();
w.start();
synchronized(w) {
while (!w.done) {
w.wait();
}
// move to within synchronized block so sum
// updated value is required to be visible
System.out.println(w.sum);
}
}
#Override public synchronized void run() {
for (int i = 0; i < 1000000; i++) {
sum += i;
}
done = true;
// no notify required here, see nitpick at end
}
}
It's not sufficient to wait on a notification, for the reason you point out (order dependence, where you're relying on a race condition hoping one thread acquires the monitor before another) as well as for other reasons. For one thing, a thread can wake up from waiting without ever having received a notification, you can't assume that there was a notify call at all.
When a thread waits, it needs to do so in a loop, where in the test on the loop it checks some condition. The other thread should set that condition variable so the first thread can check it. The recommendation that the Oracle tutorial makes is:
Note: Always invoke wait inside a loop that tests for the condition being waited for. Don't assume that the interrupt was for the particular condition you were waiting for, or that the condition is still true.
Other nitpicks:
As your example is written, the JVM is not required to make the changes to your sum variable visible to the main thread. If you add a synchronized instance method to access the sum variable, or access the sum within a synchronized block, then the main thread will be guaranteed to see the updated value of sum.
Looking at your logging, there is nothing SEVERE about an InterruptedException, it doesn't mean anything went wrong. An InterruptedException is caused when you call interrupt on a thread, setting its interrupt flag, and that thread is either currently waiting or sleeping, or enters a wait or sleep method with the flag still set. In my toy example at the top of this answer I put the exception in the throws clause because I know it's not going to happen.
When the thread terminates it issues a notifyAll that anything waiting on that object will receive (again, that's how join is implemented). It's better style to use Runnable instead of Thread, partly because of this.
In this particular example it would make more sense to call Thread#join on the summing thread, rather than calling wait.
Here's the example re-written to use join instead:
public class J extends Thread {
private long sum;
synchronized long getSum() {return sum;}
public static void main(String[] args) throws InterruptedException {
J j = new J();
j.start();
j.join();
System.out.println(j.getSum());
}
#Override public synchronized void run() {
for (int i = 0; i < 1000000; i++) {
sum += i;
}
}
}
Thread#join calls wait, locking on the thread object. When the summing thread terminates it sends a notification and sets its isAlive flag to false. Meanwhile in the join method, the main thread is waiting on the summing thread object, it receives the notification, checks the isAlive flag, and realizes it doesn't have to wait anymore, so it can leave the join method and print the result.

how to synchronize a set of multiple threads with respect to a single thread in Java

Suppose that I have an arraylist called myList of threads all of which are created with an instance of the class myRunnable implementing the Runnable interface, that is, all the threads share the same code to execute in the run() method of myRunnable. Now suppose that I have another single thread called singleThread that is created with an instance of the class otherRunnable implementing the Runnable interface.
The synchornization challenge I have to resolve for these threads is the following: I need all of the threads in myList to execute their code until certain point. Once reached this point, they shoud sleep. Once all and only all of the threads in myList are sleeping, then singleThread should be awakened (singleThread was already asleep). Then singleThread execute its own stuff, and when it is done, it should sleep and all the threads in myList should be awakened. Imagine that the codes are wrapped in while(true)'s, so this process must happen again and again.
Here is an example of the situation I've just described including an attempt of solving the synchronization problem:
class myRunnable extends Runnable
{
public static final Object lock = new Object();
static int count = 0;
#override
run()
{
while(true)
{
//do stuff
barrier();
//do stuff
}
}
void barrier()
{
try {
synchronized(lock) {
count++;
if (count == Program.myList.size()) {
count = 0;
synchronized(otherRunnable.lock) {
otherRunnable.lock.notify();
}
}
lock.wait();
}
} catch (InterruptedException ex) {}
}
}
class otherRunnable extend Runnable
{
public static final Object lock = new Object();
#override
run()
{
while(true)
{
try {
synchronized(lock) {
lock.wait();
} catch (InterruptedException ex) {}
// do stuff
try {
synchronized(myRunnable.lock) {
myRunnable.notifyAll();
}
}
}
}
class Program
{
public static ArrayList<Thread> myList;
public static void main (string[] args)
{
myList = new ArrayList<Thread>();
for(int i = 0; i < 10; i++)
{
myList.add(new Thread(new myRunnable()));
myList.get(i).start();
}
new Thread(new OtherRunnable()).start();
}
}
Basically my idea is to use a counter to make sure that threads in myList just wait except the last thread incrementing the counter, which resets the counter to 0, wakes up singleThread by notifying to its lock, and then this last thread goes to sleep as well by waiting to myRunnable.lock. In a more abstract level, my approach is to use some sort of barrier for threads in myList to stop their execution in a critical point, then the last thread hitting the barrier wakes up singleThread and goes to sleep as well, then singleThread makes its stuff and when finished, it wakes up all the threads in the barrier so they can continue again.
My problem is that there is a flaw in my logic (probably there are more). When the last thread hitting the barrier notifies otherRunnable.lock, there is a chance that an immediate context switch could occur, giving the cpu to singleThread, before the last thread could execute its wait on myRunnable.lock (and going to sleep). Then singleThread would execute all its stuff, would execute notifyAll on myRunnable.lock, and all the threads in myList would be awakened except the last thread hitting the barrier because it has not yet executed its wait command. Then, all those threads would do their stuff again and would hit the barrier again, but the count would never be equal to myList.size() because the last thread mentioned earlier would be eventually scheduled again and would execute wait. singleThread in turn would also execute wait in its first line, and as a result we have a deadlock, with everybody sleeping.
So my question is: what would be a good way to synchronize these threads in order to achieve the desired behaviour described before but at the same time in a way safe of deadlocks??

Based on your comment, sounds like a CyclicBarrier would fit your need exactly. From the docs (emphasis mine):
A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point. CyclicBarriers are useful in programs involving a fixed sized party of threads that must occasionally wait for each other. The barrier is called cyclic because it can be re-used after the waiting threads are released.
Unfortunately, I haven't used them myself, so I can't give you specific pointers on them. I think the basic idea is you construct your barrier using the two-argument constructor with the barrierAction. Have your n threads await() on this barrier after this task is done, after which barrierAction is executed, after which the n threads will continue.
From the javadoc for CyclicBarrier#await():
If the current thread is the last thread to arrive, and a non-null barrier action was supplied in the constructor, then the current thread runs the action before allowing the other threads to continue. If an exception occurs during the barrier action then that exception will be propagated in the current thread and the barrier is placed in the broken state.

How does wait and notify work?

I need to know how wait() and notify() works exactly? I couldn't achieve its working by using wait() and notify() as such. Instead if I use a while() loop for wait, it works properly. How is it so? Why can't I use just wait() and notify() simply?

have you read the documentation of the wait-notify functions ?
anyway, for the best way to achieve a wait-notify mechanism, use something like this (based on this website) :
public class WaitNotifier {
private final Object monitoredObject = new Object();
private boolean wasSignalled = false;
/**
* waits till another thread has called doNotify (or if this thread was interrupted), or don't if was already
* notified before
*/
public void doWait() {
synchronized (monitoredObject) {
while (!wasSignalled) {
try {
monitoredObject.wait();
} catch (final InterruptedException e) {
break;
}
}
wasSignalled = false;
}
}
/**
* notifies the waiting thread . will notify it even if it's not waiting yet
*/
public void doNotify() {
synchronized (monitoredObject) {
wasSignalled = true;
monitoredObject.notify();
}
}
}
do note, that each instance of this class should be used only once, so you might want to change it if you need to use it multiple times.

wait() and notify() are used in synchronized block while using threads to suspend and resume where left off.
Wait immediately looses the lock, whereas Nofity will leave the lock only when the ending bracket is encountered.
You can also refer this sample example:
public class MyThread implements Runnable {
public synchronized void waitTest() {
System.out.println("Before Wait");
wait();
System.out.println("After Wait");
}
public synchronized void notifyTest() {
System.out.println("Before Notify");
notify();
System.out.println("After Notify");
}
}
public class Test {
public static void main(String[] args) {
Thread t = new Thread(new MyThread());
t.start();
}
}

I think you are asking why does it work with while loop and does not without.
The answer is when your program calls wait() the operation system suspends your thread and activates (starts) another, and there will happen so called context switch.When OS suspend a thread it needs to save some "meta data" about your thread in order to be able to resume that thread later, PC register is what will answer your question.Basically PC (Program Counter) is a pointer to next instruction which the thread should do or is going to do, after being resumed a thread uses it to understand which instruction it was going to do when OS suspended him, and continues by that instruction (in this case, if you want to look at it by the means of Java program, the next instruction will be the next line after call to wait()).As written in "Java Concurrency in Practice"
Every call to wait is implicitly associated with a specific condition predicate. When calling wait regarding a particular
condition predicate, the caller must already hold the lock associated with the condition queue, and that lock must also
guard the state variables from which the condition predicate is composed.
Because your thread waits because some condition was not met (it should be) after returning to the method that it was suspended in, it needs to recheck that condition to see is it met yet.If condition is met it will not wait anymore, if it's not met it will call wait() again ( as it is in while loop).The important thing to know here is
PC (Program Counter) concept
and
The fact that a Thread that calls wait() on your method will not exit the method -> wait -> get resumed again -> call the method again, instead it will wait -> get resumed again -> continue from the point (instruction/line) where it was suspended (called wait())

infinite loop in a function called within a thread causes the thread to stay alive

public class Ex4 extends Thread {
boolean ans;
boolean change=false;
public boolean isPrime(long n, double maxTime) throws RuntimeException {
final Thread a;
Thread b;
final RuntimeException e;
final long num = n ;
final double mtime = maxTime;
a = new Thread(){
public void run(){
try{
ans = Ex4_tester.isPrime(num);
change=true;
}
catch ( RuntimeException exp){
throw exp;
};
}
};
a.start();
try {
sleep((long)mtime);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
if(a.isAlive()){
a.interrupt();
}
if(a.isInterrupted()){
throw new RuntimeException("time for check exceeded");
}
return ans;
}
}
all works fine, except sometimes Ex4_tester.isPrime goes into infinite loop (its ok, it should do that).
the problem that even though the thread gets interrupted the process keeps running and the thread doesnt actually stop

A thread getting interrupted doesn't stop it. It just causes the thread to get out of certain wait and sleep states. To have the thread exit an infinite loop based on input from another thread, put an explicit check against an Atomic<> in the loop and have the control thread modify the state of the atomic variable to where thread 'a' will exit the loop.

Only certain methods like Object.wait() will throw InterruptedException. If your thread doesn't call any of those methods, it won't get interrupted.
In other words, you can't asynchronously interrupt a Java thread. Very few threading systems support asynchronous interruption.

The most easy way is to set your thread to be daemon, that is call a.setDaemon(true) before start. That will make your process will be ended if this is only thread that stayed started.
If you want to interrupt your thread read this documentation for interrupt method

interrupt method causes InterruptedException only when the thread is in certain points of execution (such as sleep). Your code has invalid cancellation policy. You should consider another way of stopping your thread.

An interrupt requires an explicit check at a point where it is safe by design to interrupt a thread. If you don't perform this check (or call a method which does) the thread will keep running.