We Keep Coding

Why calling notify in Java requires holding the lock?

Thread thread = new Thread(() -> {
synchronized (this){
try {
this.wait();
System.out.println("Woke");
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
});
thread.start();
TimeUnit.SECONDS.sleep(1);
this.notify();
When calling notify it says
java.lang.IllegalMonitorStateException: current thread is not owner
The typical usage of notify is that you call it and then you release the lock implicitly (by leaving the synchronized block) so that the waiting threads may re-acquire the lock.
But code above calls notify even before it has the lock, so other threads can just try to acquire the lock, why not? I think the holding the lock is not necessary.

I think the holding the lock is not necessary.
It is necessary because the javadoc for Object.notify() says it is necessary. It states:
"This method should only be called by a thread that is the owner of
this object's monitor. A thread becomes the owner of the object's
monitor in one of three ways:
By executing a synchronized instance method of that object.
By executing the body of a synchronized statement that synchronizes on the object.
For objects of type Class, by executing a synchronized static method of that class."
But your real question is why is it necessary? Why did they design it this way?
To answer that, we need to understand that Java's wait / notify mechanism is primarily designed for implementing condition variables. The purpose of a condition variable is to allow one thread to wait for a condition to become true and for another thread to notify it that this has occurred. The basic pattern for implementing condition variables using wait() / notify() is as follows:
// Shared lock that provides mutual exclusion for 'theCondition'.
final Object lock = new Object();
// Thread #1
synchronized (lock) {
// ...
while (! theCondition) { // One reason for this loop will
// become later ...
lock.wait();
}
// HERE
}
// Thread # 2
synchronized (lock) {
// ...
if (theCondition) {
lock.notify();
}
}
This when thread #1 reaches // HERE, it knows that theCondition is now true. Furthermore it is guaranteed the current values variables that make up the condition, and any others controlled by the lock monitor will now be visible to thread #1.
But one of the prerequisites for this actually working is that both thread #1 and thread #2 are synchronized on the same monitor. That will guarantee the visibility of the values according to a happens before analysis based on the Java Memory Model (see JLS 17.4).
A second reason that the above needs synchronization is because thread #1 needs exclusive access to the variables to check the condition and then use them. Without mutual exclusion for the shared state between threads #1 and #2, race conditions are possible that can lead to a missed notification.
Since the above only works reliably when threads #1 and #2 hold the monitor when calling wait and notify, the Java designers decided to enforce this in implementations of the wait and notify methods themselves. Hence the javadoc that I quoted above.
Now ... your use-case for wait() / notify() is simpler. No information is shared between the two threads ... apart from the fact that the notify occurred. But it is still necessary to follow the pattern above.
Consider the consequences of this caveat in the javadoc for the wait() methods:
"A thread can wake up without being notified, interrupted, or timing out, a so-called "spurious wakeup". While this will rarely occur in practice, applications must guard against it ..."
So one issue is that a spurious wakeup could cause the child thread to be woken before the main thread's sleep(...) completes.
A second issue is that is the child thread is delayed, the main thread may notify the child before the child has reached the wait. The notification then be lost. (This might happen due to system load.)
What these issues mean is that your example is incorrect ... in theory, if not in reality. And in fact, it is not possible to solve your problem using wait / notify without following the pattern above/
A corrected version of your example (i.e. one that is not vulnerable to spurious wakeups, and race conditions) looks like this:
final Object lock = new Object;
boolean wakeUp = false;
Thread thread = new Thread(() -> {
synchronized (lock){
try {
while (!wakeUp) {
this.wait();
}
System.out.println("Woke");
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
});
thread.start();
TimeUnit.SECONDS.sleep(1);
synchronized (lock) {
wakeUp = true;
this.notify();
}
Note that there are simpler and more obviously correct ways to do this using various java.concurrent.* classes.

The case where using synchronized makes sense is where the thing using the lock has state that needs to be protected. In that case the lock has to be held while notifying because there are going to be state changes that go along with the notification, so that requiring notify to be called with the lock makes sense.
Using wait/notify without state that indicates when the thread should wait is not safe, it allows race conditions that can result in hanging threads, or threads can stop waiting without having been notified. It really isn't safe to use wait and notify without keeping state.
If you have code that doesn't otherwise need that state, then synchronized is an overcomplicated/tricky/buggy solution. In the case of the posted code example you could use a CountdownLatch instead, and have something that is simple and safe.

Using this.wait() inside run() in a synchronized block

I have this code:
public class Nit extends Thread {
public void run() {
try {
synchronized(this) {
this.wait();
}
System.out.println("AAA");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
Nit n = new Nit();
n.start();
synchronized(n) {
n.notify();
}
}
}
When I run it from cmd it never exits like it is an infinite loop. I don't understand why. Only thing i can think of is that Nit n is still waiting but I don't get why?

You are observing a race condition. You notify before the wait happens. Therefore the wait sits there and waits forever.
If you would invoke this code often enough, you might see it passing sometimes - when the new thread advanced faster then the main thread. One way to make the example work: try adding a call to Thread.sleep(1000) or so before calling notify(). Alternatively, even a println() call on the main thread (before the notify() might change timing enough).
Beyond that: such subtleties are the main reason why you actually avoid using the "low level" primitives such as as wait/notify. Instead, you use the powerful abstractions (like queues) that standard APIs have to offer.

The notify method tells the scheduler to pick a thread to notify, choosing from only those threads that are currently waiting on the same lock that notify was called on.
In this case the n thread doesn't start waiting until after the notification has already happened, so nothing ever wakes the thread up from waiting. You may have assumed that waiting threads will see notifications made before they started waiting, or that the JVM would have to give the n thread CPU time before the main thread proceeds past the call to start, but those assumptions aren't valid.
Introduce a condition flag as an instance member of Nit:
public class Nit extends Thread {
boolean notified = false;
and change Nit's run method to check it:
synchronized (this) {
while (!notified) {
wait();
}
}
Then add a line to the main method so that the main thread can set the flag:
synchronized (n) {
n.notified = true;
n.notify();
}
This way the notify can still happen before n starts waiting, but in that case n will check the flag, see it's true already, and skip waiting.
See Oracle's guarded blocks tutorial:
Note: Always invoke wait inside a loop that tests for the condition being waited for.
Also the API documentation (see Thread.join) discourages the practice of locking on thread objects.

An unsynchronized method can still get access despite the object being locked, why?

Consider the following:
class A {
public static void main(String[] args) throws InterruptedException
{
final A a = new A();
new Thread()
{
public void run()
{
a.intrudeLock();
};
}.start();
Thread.sleep(1000);
new Thread()
{
public void run()
{
a.doSomethingAfterLocking();
};
}.start();
}
synchronized void doSomethingAfterLocking() throws InterruptedException
{
System.out.println("aquired lock");
Thread.sleep(10000);
System.out.println("finished stuff");
}
void intrudeLock()
{
System.out.println("don't need object's lock");
}
}
Going by the locking mechanism - the expected output is (at least in most of the cases):
aquired lock
don't need object's lock
finished stuff
I am not asking why this output, and understand the reason that the second thread doesn't require a lock for its method call and can thus intrude.
Now here is my doubt - When a thread acquires lock, its intention is gaining exclusivity over the object and intuitively the execution environment should prevent any state change by other threads. But this is not how Java implements it. Is there reason why this mechanism has been designed so?

When a thread acquires lock, its intention is gaining exclusivity over the object and intuitively the execution environment should
prevent any state change by other threads. .
Small correction :
When a thread acquires lock, its intention is gaining exclusivity over
the monitor of the object and intuitively the execution environment
should prevent any state change by other threads which are waiting
(which need) to acquire the same lock.*
Its completely left to the programmer to specify whether he wants some field / resource to be used only after acquiring a lock. If you have a field that can only be accessed by one thread, then it doesn't need synchronization (getting lock).
The important point that must be noted is that, it is completely left to the programmer to synchronize access to fields based on the code paths in the program. For example a field could be accessed by multiple threads in one code path (which calls for synchronization) and may be accessed by only one thread in another path. But since there is good probability that both the code paths can be accessed at the same time by different threads, you should acquire lock before entering any of the above mentioned code paths.
Now, the JIT may decide to ignore your lock requests (lock-elision) if it thinks that they are unnecessary (like trying to lock method local fields which never escape).

multithreading beginner question

As you can tell I'm new to multithreading and a bit stuck here. For my program I need a thread (PchangeThread in the below example) that can be toggled on and off from another thread at any point during execution of the program.
The thread should be suspended on start and resume when pixelDetectorOn() is called.
The two threads will most likely not need to share any data except for a start/stop flag. I included a reference to the main thread anyway, just in case.
However, in the below code the only message that is ever output is "before entering loop", which indicates that the thread never wakes up from wait() for some reason. I'm guessing this is some kind of locking problem but I haven't been able to figure out what exactly is going wrong. Locking on this.detector from the main thread gives me the same result. Also I'm wondering if the wait()/notify() paradigm is really the way to go for suspending and waking the thread.
public class PchangeThread extends Thread {
Automation _automation;
private volatile boolean threadInterrupted;
PchangeThread(Automation automation)
{
this._automation = automation;
this.threadInterrupted = true;
}
#Override
public void run()
{
while (true) {
synchronized (this) {
System.out.println("before entering loop");
while (threadInterrupted == true) {
try {
wait();
System.out.println("after wait");
} catch (InterruptedException ex) {
System.out.println("thread2: caught interrupt!");
}
}
}
process();
}
}
private void process()
{
System.out.println("thread is running!");
}
public boolean isThreadInterrupted()
{
return threadInterrupted;
}
public synchronized void resumeThread()
{
this.threadInterrupted = false;
notify();
}
}
resumeThread() is called from the main thread the following way:
public synchronized void pixelDetectorOn(Context stateInformation) {
this.detector.resumeThread();
}
detector is a reference to an instance of PchangeThread.
The "detector"-thread is instantiated in the program's main module the following way:
detector=new PchangeThread(this);

As you said, you need to protect access to the shared flag. You declared threadInterrupted volatile, but than are still using syncronized. You only need one. I prefer to just use syncronized as it makes things simpler. Multi-threading is complicated enough, keep it simple unless you know you need more complicated controls. This means that any time threadInterrupted is read or written to, the access should be synchronized. Currently, you are not doing that in setThreadInterrupt() and isThreadInterrupted().
Secondly, you want to synchronize on as small of a code block as possible. Inside of run(), you are synchronizing over the inner loop. In actuality, you only need to to synchronize on the read of threadInterrupted. When the implementation of isThreadInterrupted() is fixed as mentioned above, you can use that directly and remove the synchronized block from the inner loop.
The fact that you are synchronizing on the inner loop, is the error that is causing your code to never print "thread is running!". PchangeThread acquires the lock on itself and calls wait() to suspend the thread. However, the thread is still holding the lock at this point. At some point later, the main thread calls resumeThread() in order to restart the thread. However, that method can not begin its execution because it must first wait to acquire the lock. However, it will never get the lock until the PchangeThread is notified.
You are providing two ways to set threadInterrupted, but only one of them notifies the thread when the value is set to false. Do you really need setThreadInterrupt()? I expect you don't. If you keep it, it should act the same as resumeThread() when the argument is false.
Lastly, it is better to lock on a private object instead of the instance itself. You have complete control over the private lock object. However, anyone with a reference to your thread instance could also use it as the lock for a synchronized block, which could potentially lead to a hard to find deadlock.
Your code altered to use my edits:
public class PchangeThread extends Thread {
private final Object _lock = new Object();
Automation _automation;
private final boolean _threadInterrupted;
PchangeThread(Automation automation)
{
_automation = automation;
_threadInterrupted = true;
}
#Override
public void run()
{
while (true) {
System.out.println("before entering loop");
while (isThreadInterrupted()) {
try {
wait();
System.out.println("after wait");
} catch (InterruptedException ex) {
System.out.println("thread2: caught interrupt!");
}
}
process();
}
}
private void process()
{
System.out.println("thread is running!");
}
public boolean isThreadInterrupted()
{
synchronized (_lock) {
return _threadInterrupted;
}
}
public void resumeThread()
{
synchronized (_lock) {
_threadInterrupted = false;
notify();
}
}
}

I personally would ask myself the following question in this case: Is the
isInterrupted
flag set only by the main thread e.g. the worker thread just reads it and decides whether to wait or not based on the flag BUT doesn't update it. Or can it be set by both the main thread and the worker thread.
If it is the former - go for a volatile boolean. That way the worker thread will not cache the volatile's value and will always read it from memory. This won't create a race condition because only 1 thread will be updating it - the main one. Think of it as a publish/subscribe scenario.
If you scenario falls in the latter category - use an AtomicBoolean variable. Both cases are going to be more efficient than the synchronized keyword, since you won't acquire any locks but in the case of Atomic* variables you will be utilizing CAS operations which are more lightweight than lock acquisition.

Your code is not wrong (though is not ideal).
I ran it and it prints all the expected messages. Likely, you just do not invoke resumeThread().
A couple of advises:
do not sync on Thread, make a Runnable and sync on it.
you want to start some computation, but what are the data to compute? Looks like they go in a separate way. This is a ground for errors. Use single channel for both data and control. The preferred way is to use a Queue for such a channel. For example, LinkedBlockingQueue is already synchronized in a proper way.

I doubt that anyone will read this, but just in case someone's interested in knowing:
When I checked the debugger log I noticed something strange - it read "debugging stopped on uncompilable source code: )Void;". Since I couldn't think of anything in my source that could have caused this error , I guessed that Netbeans had a problem with some part of the external code I was using (it was not caused by a breakpoint and the project compiled fine!). So, I just updated the third party library I'm using to it's latest version. And behold: after that I suddenly got a null pointer exception when I called resumeThread()!. I checked the rest of my code and quickly found the bug (indeed the reference to the thread was null).
So, to sum it up: The strange behaviour was caused by a minor bug in my program, but something in the external jar led to the suppression of the exception that should have been thrown. Just out of curiosity I double checked by downgrading the jar and "unfixing" the bug and again, the exception was swallowed and the debugger exited with the above mentioned strange message.
Netbeans version 7.1.1

Simple thread problem in java

Works except for when I free the crawler:
public void setCrawlerFree(WebCrawler w)
{
synchronized(myFreeCrawlers)
{
synchronized(numToGo)
{
myFreeCrawlers.add(w);
myFreeCrawlers.notifyAll();
numToGo--;
numToGo.notify();
}
}
}
When the crawler is done, I can add it back on the list. I also want to subtract 1 from the number of things I still need to do. I have one main thread waiting until numToGo is at 0. I'm getting an IllegalMonitorStateException on numToGo.notify() but since its inside of the synchronization block, doesn't that mean I own it?

Consider rewriting it to ExecutorService.
ThreadPoolExecutor executor = new ThreadPoolExecutor(corePoolSize,
maximumPoolSize, keepAliveTime, timeUnit,
new LinkedBlockingQueue<Runnable>());
executor.submit(new Callable<...>() { ... });
It would greatly simplify your code and eliminate thread synchronization issues.

So I thought I needed to call wait and
notify on the object that all the
threads have in common, but that's not
correct either.
Yes, it is. But:
public class IllegalMonitorStateException
extends RuntimeException
Thrown to indicate that a thread has
attempted to wait on an object's
monitor or to notify other threads
waiting on an object's monitor without
owning the specified monitor.
You need to synchronize on an object before calling wait() or notify() on it.

Is your numToGo field is a primitive type which is being wrapped ? (int to Integer, long to Long, etc). Remember these wrappers are immutable and will cause you to have different object every time the boxing/unboxing happens. It's always recommended to use final objects when synchronization is needed.
Instead of using and integer create your own object to maintain the value and synchronization.
class Counter {
private int value ;
private final Object lock = new Object() ;
public ExecutionStatus() { }
public void increment() {
synchronized(lock) {
value ++ ;
}
}
public void decrease() {
synchronized(lock) {
value-- ;
}
}
// Read dirty
public int count() {
return value ;
}
public int safeCount() {
synchronize(lock) {
return count() ;
}
}
}
Never the less, you can add the line private final Object lock = new Object() to your current code and use that to control the synchronization of the numToGo variable.
Hope this helps.

you are synchronising on a non-final member variable. Your sync(numToGo) syncs on some value of numToGo and then you change the reference: numToGo--. You now have a different instance on which you call notify, hence the exception.

Some good posts there, there are plenty of alternatives but I imagine this is some kind of academic exercise? As people have pointed out, you'd probably wouldn't use wait/notify/notifyAll when there are more modern alternatives that makes things easier. The wait/notify thing though is interesting and is well worth understanding as a basis for concurrency work.
I'm assuming this is some kind of consumer/producer thing? One thread is trapping a crawler, the other setting it free? If that's the case, you might want to wait for the trap to have occupants before setting free? it might look something like this...
private final List<Object> trap = new ArrayList<Object>();
public class BugCatcher {
public void trapCrawler(Object crawler) {
synchronized (trap) {
trap.add(crawler);
System.out.println("caught bug number " + trap.size() + "!");
trap.notifyAll();
}
}
}
public class Hippy {
public void setCrawlerFree(Object crawler) throws InterruptedException {
synchronized (trap) {
trap.wait();
trap.clear();
System.out.println("set bugs free! time to hug a tree");
}
}
}
If the BugCatcher can catch bugs quicker than the hippy releases them, the hippy waits for the trap to have something in it before attempting to release the bugs (hence the wait call).
If you leave out the wait/notify part, things will rely just on the synchronized keyword, only one thread will access the trap at a time and its a race as to which gets there first (the hippy might try an empty an already empty trap).
In order to co-ordinate the wait and notify, the VM will use an object monitor. A thread acquires the object's monitor when it enters a synchronized block. An object has just a single monitor which acts as a mutually exclusivity lock (mutex). If you try and wait or notify without first getting the object's monitor (without executing wait or notify within a synchronized block), the VM can't set things up and so throws the IllegalMonitorException. It's saying "I can't allow this because if, for example, I wait, when will I know that I can progress when somebody calls notify? what/who are they notifying?". It uses the monitor to coordinate and so forces you to acquire the monitor.
So, the error you get is because numToGo isn't syncrhonised in the other thread (as Michael said previously).
I can't quite see why you need the numToGo, if it is producer/consumer, do you want to stop after a certain number? After the bug catcher catches 10 bugs and the hippy releases 10? Doesn't sound like that's what you're trying to do (as they could both have unrelated internal counters), so I'm not sure what you trying to do there. It'd be good to outline what you're trying to do in case I've gone off on completely the wrong tangent!