I have a dilemma regarding the use of multithreading in the application I am working on. I have a workflow in which the state of the object changes, which presents no issues for single-threaded operation. However, In order to improve the performance, I am planning to use multiple threads.
It is my understanding that since the state is going to be shared among the threads, every thread must acquire a lock on the state before execution, so doesn't this defeat the purpose of multithreading? It seems like multiple threads won't produce any actual concurrency, so it wouldn't be any better than single threaded.
Is my analysis correct? If I am misunderstanding then would someone please clarify the concept?
The short answer: concurrency is hard. Real concurrency, with multiple concurrent writers, is really hard.
What you need to determine is what your actual consistency guarantees need to be. Does every reader need to be able to see every write, guaranteed? Then you'll be forced into linearizing all the threads somehow (e.g. using locks) -- your next effort should be to ensure you do as much work as possible outside of the lock, to keep the lock held for the shortest possible time.
One way to keep the lock held for the shortest possible time is to use a lock-free algorithm. Most lock-free algorithms are based on an atomic compare-and-set primitive, such as those provided by the java.util.concurrent.atomic package. These can be very high-performance, but designing a successful lock-free algorithm can be subtle. One simple kind of lock-free algorithm is to just build a new (immutable) state object and then atomically make it the "live" state, retrying in a loop if a different state was made live by another writer in the interim. (This approach is good enough for many applications, but it's vulnerable to livelock if you have too many writers.)
If you can get by with a looser consistency guarantee, then many other optimizations are possible. For example, you can use thread-local caches so that each thread sees its own view of the data and can be writing in parallel. Then you need to deal with the consequences of data being stale or inconsistent. Most techniques in this vein strive for eventual consistency: writes may not be visible to all readers immediately, but they are guaranteed to be visible to all readers eventually.
This is an active area of research, and a complete answer could fill a book (really, several books!). If you're just getting started in this area, I'd recommend you read Java Concurrency in Practice by Goetz et al, as it provides a good introduction to the subject and lots of practical advice about how to successfully build concurrent systems.
Your interpretation of the limits of multithreading and concurrency are correct. Since the state must be acquired and controlled by threads in order for them to perform work (and waiting when not working), you are essentially splitting the work of a single thread among multiple threads.
The best way to fix this is to adjust your program design to limit the size of the critical section. As we learned in my operating systems course with process synchronization,
only one critical section must be executing at any given time
The specific term critical section may not directly apply to Java concurrency, but it still illustrates the concept.
What does it mean to limit this critical section? For example, let's say you have a program managing a single bank account (unrealistic, but illustrates my point). If a lock on the account must be acquired by a thread for the balance to be updated, the basic option would be to have a single thread working on updating the balance at all times (without concurrency). The critical section would be the entire program. However, let's say there was also other logic to be executed, such as alerting other banks of the balance update. You could require the lock on the bank account state only while updating the balance, and not when alerting other banks, decreasing the size of critical section and allowing other threads to perform work by alerting other banks while one thread is updating the balance.
Please comment if this was unclear. Your seem to already understand the constraints of concurrency, but hopefully this will reveal possible steps towards implementing concurrency.
Your need is not totally clear but you guess well the limitations that multi threading may have.
Running parallel threads have a sense if some "relatively autonomous" tasks can be concurrently performed by distinct threads or group of threads.
If your scenario looks like : you start 5 threads and finally only a single thread is active while the others are waiting for a locking resource, using multithreading makes no sense and could even introduce an overhead because of cpu context switches.
I think that in your use case, the multithreading could be used for :
tasks that don't change the state
performing a task that changes the state if the task may be divided in multiple processing with a minimal set of instructions that may do profitable the multithreading use.
It is my understanding that since the state is going to be shared among the threads, every thread must acquire a lock on the state before execution, so doesn't this defeat the purpose of multithreading?
The short answer is "it depends". It is rare that you have a multithreaded application that has no shared data. So sharing data, even if it needs a full lock, doesn't necessarily defeat the performance improvements when making a single threaded application be multi-threaded.
The big question is what the frequency that the state needs to be updated by each thread. If the threads read in the state, do their concurrent processing which takes time, and then alter the state at the end then you may see performance gains. On the other hand, if every step in the processing needs to somehow be coordinated between threads then they may all spend them time contending for the state object. Reducing this dependence on shared state will then improve your multi-threaded performance.
There are also more efficient ways to update a state variable which can avoid locks. Something like the following pattern is used a lot:
private AtomicReference<State> sharedState;
...
// inside a thread processing loop
// do the processing job
while (true) {
State existingState = sharedState.get();
// create a new state object from the existing and our processing
State newState = updateState(state);
// if the application state hasn't changed, then update it
if (sharedState.compareAndSet(existingState, newState)) {
break;
}
// otherwise we need to get the new existing state and try again
}
One way to handle state changes is to have a coordinating thread. It is the only thread which reads from the state and generates jobs. As jobs finish they put updates to the state on a BlockingQueue which is then read by the coordinating thread which updates the state in turn. Then the processing threads don't have to all be contending for access to the shared state.
Imagine it this way :
Synchronization is blocking
Concurrency is parallelization
You don't have to use synchronization. You can use an Atomic reference object as a wrapper for your shared mutable state.
You can also use stamped locks which improves concurrency by allowing for optimistic reads. You may also use Accumulators to write concurrent code. These features are part of Java 8.
Another way to prevent synchronization is to use immutable objects which can be shared and published freely and need no synchronization. I should add that you should use immutable objects anyway regardless of concurrency for that makes your state space of the object easier to reason about
Related
I have a shared collection an ArrayList and also i use a ReentrantReadWriteLock lock to secure the entering on a critical area from different threads. My threads are three a writer,read,delete thread. i acquire the correct lock on each case. The logic is that i insert data to ArrayList, read them when necessary, and also when the timer reaches limits delete some entries. The process runs smoothly and everything is perfect.
My question now is can i transfer the above logic somehow and implemented it with an LMAX disruptor in order to avoid lock overheads and improve performance. If yes can you describe me an ideal case and if you are able to also post code i would really appreciate it.
i assume that instead of ArrayList data will be entered in ringbuffer and i will have 2 producers write, delete, and a consumer for read. Also i must make sure that i use producer barriers. Will the performance will be increased from lock case. i am not sure if i understand everything correctly please help me and give me directions?
If your shared state is the ArrayList and you have one thread that is reading and processing the elements in the ArrayList and you want the updates to that shared state synchronised then usually the ArrayList would be owned by one EventHandler that processes events such as Process, Write, Delete and updates and processes the shared state.
This would all run on one thread but that pretty much is what is happening now as you cannot Read at the same time as Writing/Deleting.
As you only have one reading thread there is not much to be gained from using a ReadWriteLock as you will never have concurrent reads.
I have a Results object which is written to by several threads concurrently. However, each thread has a specific purpose and owns certain fields, so that no data is actually modified by more than one thread. The consumer of this data will not try to read it until all of the writer threads are done writing it. Because I know this to be true, there is no synchronization on the data writes and reads.
There is a RunningState object associated with this Results object which serves to coordinate this work. All of its methods are synchronized. When a thread is done with its work on this Results object, it calls done() on the RunningState object, which does the following: decrements a counter, checks if the counter has gone to 0 (indicating that all writers are done), and if so, puts this object on a concurrent queue. That queue is consumed by a ResultsStore which reads all of the fields and stores data in the database. Before reading any data, the ResultsStore calls RunningState.finalizeResult(), which is an empty method whose sole purpose is to synchronize on the RunningState object, to ensure that writes from all of the threads are visible to the reader.
Here are my concerns:
1) I believe that this will work correctly, but I feel like I'm violating good design principles to not synchronize on the data modifications to an object that is shared by multiple threads. However, if I were to add synchronization and/or split things up so each thread only saw the data it was responsible for, it would complicate the code. Anyone who modifies this area had better understand what's going on in any case or they're likely to break something, so from a maintenance standpoint I think the simpler code with good comments explaining how it works is a better way to go.
2) The fact that I need to call this do-nothing method seems like an indication of wrong design. Is it?
Opinions appreciated.
This seems mostly right, if a bit fragile (if you change the thread-local nature of one field, for instance, you may forget to synchronize it and end up with hard-to-trace data races).
The big area of concern is in memory visibility; I don't think you've established it. The empty finalizeResult() method may be synchronized, but if the writer threads didn't also synchronize on whatever it synchronizes on (presumably this?), there's no happens-before relationship. Remember, synchronization isn't absolute -- you synchronize relative to other threads that are also synchronized on the same object. Your do-nothing method will indeed do nothing, not even ensure any memory barrier.
You somehow need to establish a happens-before relationship between each thread doing its writes, and the thread that eventually reads. One way to do this without synchronization is via a volatile variable, or an AtomicInteger (or other atomic classes).
For instance, each writer thread can invoke counter.incrementAndGet(1) on the object, and the reading thread can then check that counter.get() == THE_CORRECT_VALUE. There's a happens-before relationship between a volatile/atomic field being written and it being read, which gives you the needed visibility.
Your design is sound, but it can be improved if you are using a true concurrent queue since a concurrent queue from the java.util.concurrent package already guarantees a happens before relationship between the thread putting an item into the queue, and the thread taking an item out, so this precludes needing to call finalizeResult() in the taking thread (so no need for that "do nothing" method call).
From java.util.concurrent package description:
The methods of all classes in java.util.concurrent and its subpackages
extend these guarantees to higher-level synchronization. In
particular:
Actions in a thread prior to placing an object into any
concurrent collection happen-before actions subsequent to the access
or removal of that element from the collection in another thread.
The comments in another answer concerning using an AtomicInteger instead of synchronization are also wise (as using an AtomicInteger to do your thread counting will likely perform better than synchronization), just make sure to get the value of the count after the atomic decrement (e.g. decrementAndGet()) when comparing to 0 in order to avoid adding to the queue twice.
What you've described is indeed safe, but it also sounds, frankly, brittle and (as you note) maintenance could become an issue. Without sample code, it's really hard to tell what's really easiest to understand, so an already subjective question becomes frankly unanswerable. Could you ask a coworker for a code review? (Particularly one that's likely to have to deal with this pattern.) I'm going to trust you that this is indeed the simplest approach, but doing something like wrapping synchronized blocks around writes would increase safety now and in the future. That said, you obviously know your code better than I do.
This were the only two questions I couldn't answer in the interview I got rejected from last night.
Q: When should you use multithreading?
A: "Your question is very broad. There are few non-trivial systems where the functionality can be met simply, quickly and reliably with only one thread. For example: [pick out a typical system that the target company sells and pick out a couple aspects of its function that would be better threaded off - heavy CPU, comms, multi-user - just pick out something likely & explain].
Q: Would multithreading be beneficial if the different threads execute mutually independent tasks?
A: "Depends on what you mean by 'executing tasks'. Multithreading would surely be beneficial if the threads process mutually independent data in a concurrent fashion - it reduces requirements for locks and probabilty of deadlocks increases in a super-linear fashion with the number of locks. OTOH, there is no issue with threads executing the same code, this is safe and very common."
You should use multithreading when you want to perform heavy operations without "blocking" the flow.
Example in UIs where you do a heavy processing in a background thread but the UI is still active.
If the threads execute mutually exclusive tasks it is the best since there is no overhead for synchronization among threads needed
Multithreading is a way to introduce parallelness in your program. In any case if there can be parallel paths (parts which do not depend on result from a other part) in your program, use can make use of it.
Specially with all these multiple core machines now days, this is a feature which one should exploit.
Some examples would be processing of large data where you can divide it in chunks and get it done in multiple threads, file processing, long running I/O works like network data transfers etc.
To your second question, it would be best if the tasks are mutually independent - reasons
no shared data means no contentions
no need for any ordered processing (dependency), so each thread can work when have resources
more easy to implement
You should definitely use multithreading in GUI applications when you invoke time consuming tasks from the main event loop. Same applies for server application that might block while doing the I/O.
For the second question, it is usually yes when you have machine with multiple CPU cores. In this case these independent tasks can be executed in parallel.
You can use multithreading if the tasks can be broken down which can be executed in parallel. Like produce and consume , Validate and save , Read and Validate.
For the second question , Yes, it is beneficial for make a program into Multi threading if they are executing independent tasks.
This article gives very good reasons:
https://marcja.wordpress.com/2007/04/06/four-reasons-to-use-multithreading/
To summarize, the reasons are:
Keep your program responsive.
Make better use of your CPU. CPU may be blocked by IO or other stuff. While waiting, why not letting other threads use it
Multiple threads can be scheduled to multiple CPU cores
Some problems are naturally to be solved by multi-threading. Such solution can simplify your code.
In general, multithreading is used in cases where execution time is throttled/bottlenecked by the CPU as opposed to other areas such as IO. The second question is really quite subjective to the circumstance. For example if they are mutually independent but both do heavy IO, you might not necessarily get a large gain.
Multithreading is used when we can divide our job into several independent parts. For example, suppose you have to execute a complex database query for fetching data and if you can divide that query into sereval independent queries, then it will be better if you assign a thread to each query and run all in parallel.
In that way, your final result output will be faster.
Again, this is an example when you have the leverage to run mutliple database queries.
And to answer your second question, it is better to have threads for independent tasks. Otherwise, you will have to take care of synchronization, global variables, etc.
When should you use multithreading?
Multithreading is a process of executing multiple threads simultaneously. You should use multithreading when you can perform multiple operations together so that it can save time.
Would multithreading be beneficial if the different threads execute mutually independent tasks?
it is usually yes. Multithreading would usually be beneficial if the different threads execute mutually independent tasks so that it doesn't affect other threads if exception occur in a single thread.
I have old code in Java which deadlocks... I never used netbeans as a development tool... however, I need to fix the code.
I ran the application in debug mode, clicked on check for deadlock and netBeans brought a screen. Two out of four threads were in red... see the screen dump below.
I'm new to multithreading, and on the top of that code is not mine...
What's most likely causing the problem?
As far as I can tell the problem is very likely related to the way in which (or more specifically the order in which) the multiple threads acquire and release locks.
In the above example the two threads need access to two locks (or monitors):
nano.toolbox.strategies.ESMarketMaker
nano.toolbox.strategies.ExecutionManager
From the stack trace on the two threads currently in a deadlock, we can see that thread 'ExecutionManager' has aquired the ExecutionManager monitor but is awaiting acquisition (while still holding the 'ExecutionManager' monitor) of the 'ESMarketMaker' monitor.
The 'StrategyManager' thread on the other hand, has acquired the 'ESMarketMaker' monitor but is awaiting acqusition (while still holding the 'ESMarketMaker' monitor) of the 'ExecutionManager' monitor.
This is a class example of deadlocks and the many ways in which order of acquisition of locks can cause deadlocks.
There are many ways to address these kind of problems:
If possible, all threads needing some set of locks to operate, must acquire the shared locks in the same order (the inversed order is the problem in the above deadlock). But this is not always possible, as multiple threads may have only semi-overlapping lock usage in different conditions, why it may be hard or impossible to design a protocol of acquisition that will ensure uniform ordering.
You may also use tryLock() instead, which is a non-blocking acquisition, it returns a flag to indicate success or failure and gives you the option to do something else before re-trying. One thing I would recommend in this case, is that if acquisition fails, it is to drop all currently owned locks and try from scratch again (thus giving way for any who is blocked on any or all locks the current thread holds, to complete their work, maybe freeing the locks this thread needs when it retries).
One thing to note though, is that sometimes when deciding on the protocol to use, you need more explicit control over your locks, rather than normal synchronization in Java. In these cases, the usage of explicit ReentrantLock instances can be a benefit, as these allows you to do stuff like inspecting whether a lock is unlocked or currently locked, and do non-blocking try-locks as described above.
I hope this helps, I'm sorry I can't be more specific, but I would need to see the source code for that. :-)
(Oh an p.s., a third thing one might opt for, if deadlock is something that must be avoided by all cost, is to look into modeling tools, to model a state machine over the states of the program and locks, which can be used together with analysis tools which can check for possible deadlocks in such a model and give you examples if any such is found).
Guys, can anyone give a simple practical example of LockSupport & AbstractQueuedSynchronizer use? Example given in javadocs is quite strained.
Usage of Semaphore permits is understood by me.
Thanks for any response.
If youre talking about using a locking mechanism (or even sync barriers) just use a java.util.concurrent.Lock. The obvious suggestion is to user a ReentrantLock which delegates to a Synch. The synch is an AQS which in turn uses LockSupport.
Its all done under the covers for you.
Edit:
No let's go over the practical uses of AbstractQueuedSynchronizer (AQS).
Concurrency constructs though can be very different in their usage all can have the same underlying functions.
I.e. Under some condition park this thread. Under some other condition wake a thread up.
This is a very broad set of instructions but makes it obvious that most concurrency structures would need some common functionality that would be able to handle those operations for them. Enter AQS. There are five major synchronization barriers.
ReentrantLock
ReadLock
WriteLock
Semaphore
CountDownLatch
Now, all these five structures have very different set of rules when using them. A CountdownLatch can allow many threads to run at the same time but forces one (or more) threads to wait until at least n number of threads count down on said latch.
ReentrantLock forces only one thread at a time to enter a critical section and queues up all other threads to wait for it to completed.
ReadLock allows any number of reading threads into the critical section until a write lock is acquiered.
The examples can go on, but the big picture here is they all use AQS. This is because they are able to use the primitive functions that AQS offers and implements more complex functionality on top of it. AQS allows you to park unpark and wake up threads ( interruptibly if need be) but in such a way that you can support many complex functions.
they are not meant for direct use in client code; more for helping building new concurrent classes.
AQS is a wonderful class for building concurrency primitives – but it is complex and requires a bit of study to use it properly. I have used it for a few things like lazy initialisation and a simple fast reusable latch.
As complex as it is, I don't think AQS is particularly vague, it has excellent javadocs describing how to use it properly.
2.7 release of Disruptor uses LockSupport.parkNanos instead of Thread.sleep to reduce latency:
http://code.google.com/p/disruptor/
AFAIK, AbstractQueuedSynchronizer is used to manage state transitions. The JDK uses it to extend Sync, an internal class for java.util.concurrent.FutureTask. The Sync class manages the states (READY, RUNNING, RAN, and CANCELLED) of FutureTask and the transitions between them.
This allows, as you may know, FutureTask to block on FutureTask.get() until the RAN state is reached, for example.