I just want to know why Non Blocking concurrency is better than Blocking concurrency. In Blocking Concurrency Your thread must wait till other thread completes its execution. So thread would not consuming CPU in that case.
But if I talk about Non Blocking Concurrency, Threads do not wait to get a lock they immediately returns if certain threads is containing the lock.
For Example in ConcurrentHashMap class , inside put() method there is tryLock() in a loop. Other thread will be active and continuously trying to check if lock has been released or not because tryLock() is Non Blocking. I assume in this case, CPU is unnecessary used.
So Is it not good to suspend the thread till other thread completes its execution and wake the thread up when work is finished?
Whether or not blocking or non-blocking concurrency is better depends on how long you expect to have to wait to acquire the resource you're waiting on.
With a blocking wait (i.e. a mutex lock, in C parlance), the operating system kernel puts the waiting thread to sleep. The CPU scheduler will not allocate any time to it until after the required resource has become available. The advantage here is that, as you said, this thread won't consume any CPU resources while it is sleeping.
There is a disadvantage, however: the process of putting the thread to sleep, determining when it should be woken, and waking it up again is complex and expensive, and may negate the savings achieved by not having the thread consume CPU while waiting. In addition (and possibly because of this), the OS may choose not to wake the thread immediately once the resource becomes available, so the lock may be waited on longer than is necessary.
A non-blocking wait (also known as a spinlock) does consume CPU resource while waiting, but saves the expense of putting the thread to sleep, determining when it should be woken, and waking it. It also may be able to respond faster once the lock becomes free, as it is less at the whim of the OS in terms of when it can proceed with execution.
So, as a very general rule, you should prefer a spinlock if you expect to only wait a short time (e.g. the few CPU cycles it might take for another thread to finish with an entry in ConcurrentHashMap). For longer waits (e.g. on synchronized I/O, or a number of threads waiting on a single complex computation), a mutex (blocking wait) may be preferable.
If you consider ConcurrentHashMap as an example , considering the overhead due to multiple threads performing update operations (like put) , and block waiting for the locks to release (as you mention other thread will be active and continuously trying to check if lock has been released), is not going to be the case,always.
Compared to HashTable , Concurrency control in ConcurrentHashMap is split up. So multiple threads can acquire lock(on segments of the table).
Originally, the ConcurrentHashMap class supports a hard-wired preset concurrency level of 32. This allows a maximum of 32 put and/or remove operations to proceed concurrently(factors other than synchronization tend to be bottlenecks when more than 32 threads concurrently attempt updates.)
Also, successful retrievals (when the key is present) using get(key) and containsKey(key) usually run without locking.
So for instance, one thread might be in the process of adding an element, what cannot be done with such a locking strategy is operations like add an element only if it is not already present (ConcurrentReaderHashMap provides such facilities).
Also, the size() and isEmpty() methods require accumulations across 32 control segments, and so might be slightly slower.
Related
I would like to ask you a question related to multithreading in Java.
I have a monitor and multiple threads are eager to own it.
Also inside the critical section this.wait() is invoked based on some conditions.
AFAIK, the monitor has 2 sets of threads:
entry set - where just arrived threads congregate and wait for their turn to own the monitor
wait set - where this.wait() threads that are waiting to be awakened
But how do they compete when notify/notifyAll is called?
Do threads from wait set have a priority in acquiring the monitor over threads in entry set or do they move to entry set?
Can I be sure that in case of notify the next executed thread will be one from the wait set?
No. The scheduler is in charge of which thread gets the lock next. It might be one from the wait set that got notified. It might be a thread that is just arrived and hasn't entered the wait set. Assuming the thread that just got notified will get the monitor next is not safe.
The standard advice is to call wait in a loop where we check the condition being waited on:
synchronized (lock) {
while (!condition) {
lock.wait();
}
...
That way when a thread comes out of a wait, it makes the same check as any thread that hasn't waited yet to know whether to progress or not.
If you need fairness, where you want the longest-waiting thread to acquire the lock next, then you might try one of the explicit Locks from java.util.concurrent.locks, such as ReentrantLock, but read the fine print:
The constructor for this class accepts an optional fairness parameter. When set true, under contention, locks favor granting access to the longest-waiting thread. Otherwise this lock does not guarantee any particular access order. Programs using fair locks accessed by many threads may display lower overall throughput (i.e., are slower; often much slower) than those using the default setting, but have smaller variances in times to obtain locks and guarantee lack of starvation. Note however, that fairness of locks does not guarantee fairness of thread scheduling. Thus, one of many threads using a fair lock may obtain it multiple times in succession while other active threads are not progressing and not currently holding the lock. Also note that the untimed tryLock method does not honor the fairness setting. It will succeed if the lock is available even if other threads are waiting.
Let's say you have two threads, thread1 and thread2. If you call thread1.start() and thread2.start() at the same time and they both print out numbers between 1 and 5, they will both run at the same time and they will randomly print out the numbers in any order, if I am not mistaken. To prevent this, you use the .join() method to make sure that a certain thread gets executed first. If this is what the .join() method does, what is the Lock object used for?
Thread.join is used to wait for another thread to finish. The join method uses the implicit lock on the Thread object and calls wait on it. When the thread being waited for finishes it notifies the waiting thread so it can stop waiting.
Java has different ways to use locks to protect access to data. There is implicit locking that uses a lock built into every Java object (this is where the synchronized keyword comes in), and then there are explicit Lock objects. Both of them protect data from concurrent access, the difference is the explicit Locks are more flexible and powerful, while implicit locking is designed to be easier to use.
With implicit locks, for instance, I can't not release the lock at the end of a synchronized method or block, the JVM makes sure that the lock gets released as the thread leaves. But programming with implicit locks can be limiting. For instance, there aren't separate condition objects so if there are different threads accessing a shared object for different things, notifying only a subset of them is not possible.
With explicit Locks you get separate condition objects and can notify only those threads waiting on a particular condition (producers might wait on one condition while consumers wait on another, see the ArrayBlockingQueue class for an example), and you can implement more involved kinds of patterns, like hand-over-hand locking. But you need to be much more careful, because the extra features introduce complications, and releasing the lock is up to you.
Locking typically prevents more than one thread from running a block of code at the same time. This is because only ONE thread at a time can acquire the lock and run the code within. If a thread wants the lock but it is already taken, then that thread goes into a wait state until the lock is released. If you have many threads waiting for the lock to be released, which one gets the lock next is INDETERMINATE (can't be predicted). This can lead to "thread starvation" where a thread is waiting for the lock, but it just never gets it because other threads always seem to get it instead. This is a very generic answer because you didn't specify a language. Some languages may differ slightly in that they might have a determinate method of deciding who gets the lock next.
Suppose we have a ConcurrentHashMap of size 16 .i.e concurrency level 16. If number of threads working on this map is greater than 16 i.e. somewhere 20 to 25 then what will happen to the extra threads whether they will be in waiting state?
If yes then how long they will be in waiting state and internally how they will communicate with other threads?
There's nothing magic about the concurrency level. Even if you have fewer than 16 threads in your example, they still can collide when attempting to access the map.
If the concurrency level is 16, then when any two threads try to put two different keys into the map, there will be one chance in 16 that the two keys are assigned to the same segment. If that's the case, then their access to the segment will have to be serialized. (i.e., one of the threads will have to be blocked until the other thread has finished.)
Of course, if two threads try to access the same key at the same time then it's guaranteed that they will access the same segment and one will be blocked, no matter what the concurrency level.
what will happen to the extra threads whether they will be in waiting state?
Nothing. That's what a synchronized statement does. It does nothing (a.k.a., it "blocks") until the lock becomes available. Then, it acquires the lock, performs the body of the statement, and releases the lock.
how long they will be in waiting state?
A thread that is blocked on a mutex lock will remain blocked until the lock is released by some other thread. In well designed code, that should only be a very short time---just long enough for the other thread to update a few fields.
internally how they will communicate with other threads?
Not sure what you mean by "internally". A thread that is blocked can't communicate with other threads. It can't do anything until it is unblocked.
Maybe you are asking how the operating system knows what to unblock, and when.
When a thread attemts to lock a mutex that already is in use by some other thread, the operating system will suspend the thread, save its state, and put some token that represents the thread into a queue that is associated with the mutex. When the owner of the thread releases the mutex, the OS picks a thread from the queue, transfers ownership of the mutex to the selected thread, restores the thread's saved state, and lets it run.
The algorithms that different operating systems use to select which thread to unblock, and the means of saving and restoring thread contexts (a.k.a., "context switch") are deeper subjects than I have room or time to discuss here.
What is "Busy Spin" in multi-threaded environment?
How it is useful and how can it be implemented in java in a multi-threaded environment?
In what way can it be useful in improving the performance of an application?
Some of the other answers miss the real problem with busy waiting.
Unless you're talking about an application where you are concerned with conserving electrical power, then burning CPU time is not, in and of itself, a Bad Thing. It's only bad when there is some other thread or process that is ready-to-run. It's really bad when one of the ready-to-run threads is the thread that your busy-wait loop is waiting for.
That's the real issue. A normal, user-mode program running on a normal operating system has no control over which threads run on which processors, a normal operating system has no way to tell the difference between a thread that is busy waiting and a thread that is doing work, and even if the OS knew that the thread was busy-waiting, it would have no way to know what the thread was waiting for.
So, it's entirely possible for the busy waiter to wait for many milliseconds (practically an eternity), waiting for an event, while the the only thread that could make the event happen sits on the sideline (i.e., in the run queue) waiting for its turn to use a CPU.
Busy waiting is often used in systems where there is tight control over which threads run on which processors. Busy waiting can be the most efficient way to wait for an event when you know that the thread that will cause it is actually running on a different processor. That often is the case when you're writing code for the operating system itself, or when you're writing an embedded, real-time application that runs under a real-time operating system.
Kevin Walters wrote about the case where the time to wait is very short. A CPU-bound, ordinary program running on an ordinary OS may be allowed to execute millions of instructions in each time slice. So, if the program uses a spin-lock to protect a critical section consisting of just a few instructions, then it is highly unlikely that any thread will lose its time slice while it is in the critical section. That means, if thread A finds the spin-lock locked, then it is highly likely that thread B, which holds the lock, actually is running on a different CPU. That's why it can be OK to use spin-locks in an ordinary program when you know it's going to run on a multi-processor host.
Busy-waiting or spinning is a technique in which a process repeatedly checks to see if a condition is true instead of calling wait or sleep method and releasing CPU.
1.It is mainly useful in multicore processor where condition is going to be true quite quickly i.e. in millisecond or micro second
2.Advantage of not releasing CPU is that, all cached data and instruction are remained unaffected, which may be lost, had this thread is suspended on one core and brought back to another thread
Busy spin is one of the techniques to wait for events without releasing CPU. It's often done to avoid losing data in CPU cached which is lost if the thread is paused and resumed in some other core.
So, if you are working on a low latency system where your order processing thread currently doesn't have any order, instead of sleeping or calling wait(), you can just loop and then again check the queue for new messages. It's only beneficial if you need to wait for a very small amount of time e.g. in microseconds or nanoseconds.
LMAX Disrupter framework, a high-performance inter-thread messaging library has a BusySpinWaitStrategy which is based on this concept and uses a busy spin loop for EventProcessors waiting on the barrier.
A "busy spin" is constantly looping in one thread to see if the other thread has completed some work. It is a "Bad Idea" as it consumes resources as it is just waiting. The busiest of spins don't even have a sleep in them, but spin as fast as possible waiting for the work to get finished. It is less wasteful to have the waiting thread notified by the completion of the work directly and just let it sleep until then.
Note, I call this a "Bad Idea", but it is used in some cases on low-level code to minimize latency, but this is rarely (if ever) needed in Java code.
Busy spinning/waiting is normally a bad idea from a performance standpoint. In most cases, it is preferable to sleep and wait for a signal when you are ready to run, than to do spinning. Take the scenario where there are two threads, and thread 1 is waiting for thread 2 to set a variable (say, it waits until var == true. Then, it would busy spin by just doing
while (var == false)
;
In this case, you will take up a lot of time that thread 2 can potentially be running, because when you wake up you are just executing the loop mindlessly. So, in a scenario where you are waiting for something like this to happen, it is better to let thread 2 have all control by putting yourself to sleep and having it wake you up when it is done.
BUT, in rare cases where the time you need to wait is very short, it is actually faster to spinlock. This is because of the time it takes to perform the signalng functions; spinning is preferable if the time used spinning is less than the time it would take to perform the signaling. So, in that way it may be beneficial and could actually improve performance, but this is definitely not the most frequent case.
Spin Waiting is that you constantly wait for a condition comes true. The opposite is waiting for a signal (like thread interruption by notify() and wait()).
There are two ways of waiting, first semi-active (sleep / yield) and active (busy waiting).
On busy waiting a program idles actively using special op codes like HLT or NOP or other time consuming operations. Other use just a while loop checking for a condition comming true.
The JavaFramework provides Thread.sleep, Thread.yield and LockSupport.parkXXX() methods for a thread to hand over the cpu. Sleep waits for a specific amount of time but alwasy takes over a millisecond even if a nano second was specified. The same is true for LockSupport.parkNanos(1). Thread.yield allows for a resolution of 100ns for my example system (win7 + i5 mobile).
The problem with yield is the way it works. If the system is utilized fully yield can take up to 800ms in my test scenario (100 worker threads all counting up a number (a+=a;) indefinitively). Since yield frees the cpu and adds the thread to the end of all threads within its priority group, yield is therefore unstable unless the cpu is not utilized to a certain extend.
Busy waiting will block a CPU (core) for multiple milliseconds.
The Java Framework (check Condition class implementations) uses active (busy) wait for periodes less then 1000ns (1 microsecond). At my system an average invocation of System.nanoTime takes 160ns so busy waiting is like checking the condition spend 160ns on nanoTime and repeat.
So basically the concurrency framework of Java (queues etc) has something like wait under a microsecond spin and hit the waiting periode within a N granulairty where N is the number of nanoseconds for checking time constraints and wait for one ms or longer (for my current system).
So active busy waiting increases utilization but aid in the overall reactiveness of the system.
While burning CPU time one should use special instructions reducing the power consumption of the core executing the time consuming operations.
Busy spin is nothing but looping over until thread(s) completes. E.g. You have say 10 threads, and you want to wait all the thread to finish and then want to continue,
while(ALL_THREADS_ARE_NOT_COMPLETE);
//Continue with rest of the logic
For example in java you can manage multiple thread with ExecutorService
ExecutorService executor = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
Runnable worker = new WorkerThread('' + i);
executor.execute(worker);
}
executor.shutdown();
//With this loop, you are looping over till threads doesn't finish.
while (!executor.isTerminated());
It is a to busy spins as it consumes resources as CPU is not sitting ideal, but keeping running over the loop. We should have mechanism to notify the main thread
(parent thread) to indicate that all thread are done and it can continue with the rest of the task.
With the preceding example, instead of having busy spin, you can use different mechanism to improve performance.
ExecutorService executor = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
Runnable worker = new WorkerThread('' + i);
executor.execute(worker);
}
executor.shutdown();
try {
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
log.fatal("Exception ",e);
}
I understand that Thread.currentThread().yield() is a notification to thread scheduler that it may assign cpu cycle to some other thread of same priority if any such is present.
My question is: If current thread has got lock on some object and calls yield(), will it loses that lock right away? And when thread scheduler finds out there is no such thread to assign cpu cycle, then the thread which has called yield() will again be in fight to get lock on the object which it has lost earlier??
I couldn't find it in javadoc and forums [http://www.coderanch.com/t/226223/java-programmer-SCJP/certification/does-sleep-yield-release-lock] have 50-50 answers.
I think yield() (lets say thread1) should release lock because if some thread (lets say thread2) of same priority wants to operate on same object, then it can have chance when thread scheduler eventually assign cup to thread2.
No. Thread.yield() is not like Object.wait(). It just gives up control to allow a thread switch. It will have no effect on the concurrency of your program.
There is no guarantee which thread the scheduler will run after a yield.
In Java Language specification
17.3 Sleep and Yield
It is important to note that neither Thread.sleep nor Thread.yield have any synchronization semantics. In particular, the compiler does not have to flush writes cached in registers out to shared memory before a call to Thread.sleep or Thread.yield, nor does the compiler have to reload values cached in registers after a call to Thread.sleep or Thread.yield.
My comment:
In java's early days, when it did not really supported parallel executions, but only concurrent (green threads), yield() was suspending the current thread, and the jvm was picking up another thread to resume. Now-days, yield does not have much meaning as usually the tread scheduling is on OS level.
So, yield is just a hint to the JVM that current thread wants to take a rest and nothing else, it is up to the thread scheduler to decide what to do. yield does not have any synchronization semantic. If thread holds lock, it will continue to hold it.
Only wait methods of the Object class release the intrinsic lock of the current instance (the thread may have other locks acquired, they don't get released). Yield, sleep, join do not bother about locks. However, join is a little more special, you are guaranteed to see all the changes made by the thread you're waiting for to finish.