I have the classic problem of a thread pushing events to the incoming queue of a second thread. Only this time, I am very interested about performance. What I want to achieve is:
I want concurrent access to the queue, the producer pushing, the receiver poping.
When the queue is empty, I want the consumer to block to the queue, waiting for the producer.
My first idea was to use a LinkedBlockingQueue, but I soon realized that it is not concurrent and the performance suffered. On the other hand, I now use a ConcurrentLinkedQueue, but still I am paying the cost of wait() / notify() on each publication. Since the consumer, upon finding an empty queue, does not block, I have to synchronize and wait() on a lock. On the other part, the producer has to get that lock and notify() upon every single publication. The overall result is that I am paying the cost of
sycnhronized (lock) {lock.notify()} in every single publication, even when not needed.
What I guess is needed here, is a queue that is both blocking and concurrent. I imagine a push() operation to work as in ConcurrentLinkedQueue, with an extra notify() to the object when the pushed element is the first in the list. Such a check I consider to already exist in the ConcurrentLinkedQueue, as pushing requires connecting with the next element. Thus, this would be much faster than synchronizing every time on the external lock.
Is something like this available/reasonable?
I think you can stick to java.util.concurrent.LinkedBlockingQueue regardless of your doubts. It is concurrent. Though, I have no idea about its performance. Probably, other implementation of BlockingQueue will suit you better. There's not too many of them, so make performance tests and measure.
Similar to this answer https://stackoverflow.com/a/1212515/1102730 but a bit different.. I ended up using an ExecutorService. You can instantiate one by using Executors.newSingleThreadExecutor(). I needed a concurrent queue for reading/writing BufferedImages to files, as well as atomicity with reads and writes. I only need a single thread because the file IO is orders of magnitude faster than the source, net IO. Also, I was more concerned about atomicity of actions and correctness than performance, but this approach can also be done with multiple threads in the pool to speed things up.
To get an image (Try-Catch-Finally omitted):
Future<BufferedImage> futureImage = executorService.submit(new Callable<BufferedImage>() {
#Override
public BufferedImage call() throws Exception {
ImageInputStream is = new FileImageInputStream(file);
return ImageIO.read(is);
}
})
image = futureImage.get();
To save an image (Try-Catch-Finally omitted):
Future<Boolean> futureWrite = executorService.submit(new Callable<Boolean>() {
#Override
public Boolean call() {
FileOutputStream os = new FileOutputStream(file);
return ImageIO.write(image, getFileFormat(), os);
}
});
boolean wasWritten = futureWrite.get();
It's important to note that you should flush and close your streams in a finally block. I don't know about how it performs compared to other solutions, but it is pretty versatile.
I would suggest you look at ThreadPoolExecutor newSingleThreadExecutor. It will handle keeping your tasks ordered for you, and if you submit Callables to your executor, you will be able to get the blocking behavior you are looking for as well.
You can try LinkedTransferQueue from jsr166: http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/jsr166y/
It fulfills your requirements and have less overhead for offer/poll operations.
As I can see from the code, when the queue is not empty, it uses atomic operations for polling elements. And when the queue is empty, it spins for some time and park the thread if unsuccessful.
I think it can help in your case.
I use the ArrayBlockingQueue whenever I need to pass data from one thread to another. Using the put and take methods (which will block if full/empty).
Here is a list of classes implementing BlockingQueue.
I would recommend checking out SynchronousQueue.
Like #Rorick mentioned in his comment, I believe all of those implementations are concurrent. I think your concerns with LinkedBlockingQueue may be out of place.
Related
I need help with my multithreading code.
I have a callable class which returns a value. I have a cachedThreadPool to submit ~60,000 tasks. I collect all the Futures in a List. After the ExecutiveService has shutdown, I loop through the list of Futures, and write the returned values using a bufferedWriter. Is this correct way of implementation?
ExecutorService execService = Executors.newCachedThreadPool();
List<Future<ValidationDataObject<String, Boolean>>> futureList = new ArrayList<>();
for (int i = 0; i < emailArrayList.size(); i++) {
String emailAddress = emailArrayList.get(i);
ValidateEmail validateEmail = new ValidateEmail(emailAddress);
Future<ValidationDataObject<String, Boolean>> future =
execService.submit(validateEmail);
futureList.add(future);
}
execService.shutdown();
for (Future<ValidationDataObject<String, Boolean>> future: futureList) {
ValidationDataObject<String, Boolean> validationObject = future.get();
bufferedWriter.write(validationObject.getEmailAddress() + "|"
+ validationObject.getIsValid());
bufferedWriter.newLine();
bufferedWriter.flush();
}
if (execService.isTerminated()) bufferedWriter.close();
Should I using synchronized block for the bufferedWriter? I am thinking, It doesn't need to be synchronized because, I am using the bufferedWriter from the main Thread, right?
I have a cachedThreadPool to submit ~60,000 tasks.
Off the bat, a cached thread-pool and 60k tasks is a red flag. That is going to start 60k threads which I doubt you really want. You should use a fixed thread pool and vary the number of threads until you achieve a good balance of throughput versus overwhelming your server. Maybe start with 2x the number of CPUs and then vary it depending on the server load.
You might also might consider using a fixed size queue which will limit the number of tasks outstanding although 60k tasks is fine unless those objects are heavy.
I collect all the Futures in a List. After the ExecutiveService has shutdown, I loop through the list of Futures, and write the returned values using a bufferedWriter. Is this correct way of implementation?
Yes, that's a good pattern. You don't show the writer being created but it is certainly fine for the main thread to own that.
Should I using synchronized block for the bufferedWriter? I am thinking, It doesn't need to be synchronized because, I am using the bufferedWriter from the main Thread, right?
Right. No other threads are using it so that's fine. It is a very typical pattern to have a writer thread managing the output of a multi-thread application.
One final comment is that you might want to look at the ExecutionCompletionService which allows you to process the tasks as they finish instead of having to wait for them in order. You might require the output to be in order in which case this isn't helpful but it's good technology to know about anyway.
Apart from the fact, that executor.shutdown() will most likely not do, what you believe it to do (it simply stops the Executor from accepting new Tasks, it will not wait for all tasks to terminate), your code looks fine.
You are right, there is no need for synchronization with respect to the writer, as you access it only single threaded.
There are things, that can be improved, though. Firstly, you are not doing a lot of Exception handling. Future.get() will throw an ExecutionException, if the Callable hits an Exception.
I'm not certain, how large the deviations in execution-time of your Callables are. Assume, there are notable deviations look at the following case: Say we submit Callables A, B and C, you receive FutA, FutB and FutC. Calling the get methods will block until the calculation behind the Future is finished. In your setting, you might be waiting for FutA to complete, while FutB/FutC might already be finished and ready for writing. Worst case here is, that processing of A will delay writing for all 60000 tasks.
I think, I would go for another approach, where every Callable gets the reference to the same ConcurrentLinkedQueue and instead of returning the result via Future writes the result into that queue. In this scenario, the ordering of the result is not dependent on the ordering of the Callables but on the time, the Callables finish execution. Whether or not this results in a speedup depends on your setting (especially time to write result and deviation in execution times of the Callables).
Suppose that I have a List with role of cache. Most time list is read-only buy every few seconds I want to do atomic replacement of all List contents.
In atomic I mean that I don't want to allow cache clients to hit read between for example clear() and addAll().
What list implementation to use and how to perform replacement for best performance?
It is better to replace list contents or to replace reference value itself?
Java has a ReadWriteLock which supports reads concurrently and writes exclusively. As mentioned in the JavaDoc, it is a good choice if updates occur not very frequently and reads occur often. The faster your writer updates the List the better the performance you get.
The methods readLock() has to be called by readers and writeLock() by writers. Then you have to call lock() on the Lock obtained. If it is available, the tread will continue working, otherwise it will block, until the lock is available.
Use fairness when constructing the ReadWriteLock to enable reader and writer threads to obtain their locks in the order they requested it. Otherwise some thread could wait forever (in a worst case scenario).
The benefit of a ReadWriteLock is that many reader may share the same lock without obtaining it, which is an expansive operation. This benefit is only observable if the ratio between reading and writing is heavily in favor of reading.
If you really want to use the List<> type for a cache, you can use a lock like:
private static final Objet cacheLock = new Object();
public void updateListCache()
{
synchronized(lock)
{
//update your list cache
}
}
Your read operation would also need to implement this lock, or it wouldn't be much use.
But for caching purposes I would personally go with ConcurrentHashMap since it's thread safe.
To begin with, I have used search and found n topics related to this question. Unfortunately, they didin't help me, so it'll be n++ topics :)
Situation: I will have few working threads (the same class, just many dublicates) (let's call them WT) and one result writing thread (RT).
WT will add objects to the Queue, and RT will take them. Since there will be many WT won't there be any memory problems(independant from the max queue size)? Will those operations wait for each other to be completed?
Moreover, as I understand, BlockingQueue is quite slow, so maybe I should leave it and use normal Queue while in synchronized blocks? Or should I consider my self by using SynchronizedQueue?
LinkedBlockingQueue is designed to handle multiple threads writing to the same queue. From the documentation:
BlockingQueue implementations are thread-safe. All queuing methods achieve their effects atomically using internal locks or other forms of concurrency control. However, the bulk Collection operations addAll, containsAll, retainAll and removeAll are not necessarily performed atomically unless specified otherwise in an implementation.
Therefore, you are quite safe (unless you expect the bulk operations to be atomic).
Of course, if thread A and thread B are writing to the same queue, the order of A's items relative to B's items will be indeterminate unless you synchronize A and B.
As to the choice of queue implementation, go with the simplest that does the job, and then profile. This will give you accurate data on where the bottlenecks are so you won't have to guess.
I want to use a ConcurrentLinkedQueue in an atomic lock-free manner:
Several concurrent threads push events into the queue and some other thread will process them. The queue is not bound and I don't want any thread to wait or get locked. The reading part however may notice that the queue got empty. In a lock free implementation the reading thread must not block but will just end its task and proceeds executing other tasks (i.e. as an ExecutorService). Thus the writer pushing the first new event into an empty queue must become aware about it and should restart the reader (i.e. by submitting a new Runnable to the ExecutorService) to process the queue. Any further threads submitting a second or third event won't care about, as they may assume some reader was already prepared/submitted.
Unfortunately the add() method of ConcurrentLinkedQueue always returns true. Asking the queue if isEmpty() before or after adding the event won't help, as it is not atomic.
Should I use some additional AtomicInteger for monitoring the queue size() or is there some smarter solution for that?
Dieter.
I don't quite understand why you wouldn't just use an ExecutorService directly for this. It uses a BlockingQueue internally and takes care of all of the signaling itself.
// open ended thread pool
ExecutorService threadPool = Executors.newFixedThreadPool(1);
for (Job job : jobsToDo) {
threadPool.submit(new MyJobProcessor(job));
}
Unless you have good reasons, I would not rewrite the same logic yourself.
If you are trying to make use of dormant threads somehow, I would strongly recommend not bothering. Threads are relatively cheap so assigning a thread to process your queued tasks is fine. Re-using threads is unnecessary and seems like premature optimization to me.
Using of AtomicInteger for resolving submit contention is more efficient than locks or synchronized block.
Here is an example how it can be implemented with Java.
Also there is more efficient structure for multi-producer / single-writer queue than ConcurrentLinkedQueue.
Example of using it for actor implementations.
Another example.
We have a system in which each thread (there can be dozens of them) works as an individual agent.
It has its own inner variables and objects, and it monitors other threads' objects as well as its own) in order to make decisions.
Unfortunately the system is deadlocking quite often.
Going through java tutorial (http://download.oracle.com/javase/tutorial/essential/concurrency/index.html) and through other topics here at stackoverflow, I managed to avoid some of these deadlocks by synchronizing the methods and using a monitor, as in:
Producer->monitor->Consumer.
However, not all communication between threads can be modeled like this. As I've mentioned before, at a given time one thread must have access to the objects (variables, lists, etc) of the other threads.
The way it's being done now is that each thread has a list with pointers to every other thread, forming a network. By looping through this list, one thread can read all the information it needs from all the others. Even though there is no writing involved (there shouldn't be any problems with data corruption), it still deadlocks.
My question is: is there an already known way for dealing with this sort of problem? A standard pattern such as the monitor solution?
Please let me know if the question needs more explanation and I'll edit the post.
Thank you in advance!
-Edit----
After getting these answers I studied more about java.concurrency and also the actor model. At the moment the problem seems to be fixed by using a reentrant lock:
http://download.oracle.com/javase/tutorial/essential/concurrency/newlocks.html
Since it can back out from an attempt to acquire the locks, it doesn't seem to have the problem of waiting forever for the them.
I also started implementing an alternate version following the actor model since it seems to be an interesting solution to this case.
My main mistakes were:
-Blindly trusting synchronize
-When in the tutorial they say "the lock is on the object" what they actually mean is the whole object running the thread (in my case), not the object I would like to access.
Thank you all for the help!
Look at higher-level concurrency constructs such as the java.util.concurrent package and the Akka framework/library. Synchronizing and locking manually is a guaranteed way to fail with threads in Java.
I would recommend to apply Actor model here (kind of share nothing parallelism model).
Using this model means that all your thread don't interrupt each other explicitely and you don't need to do any synchronization at all.
Instead of making synchronization you'll use messages. When one Actor (thread) needs to get info about another Actor, it just asynchronously send a correspondent message to that Actor.
Each Actor can also respond to messages of certain types. So, when a new message comes, Actor analyses it and sends a response (or does any other activity). The key point here is that processing of incoming messages is being done synchronously (i.e. it's the only point where you need the simplest way of synchronization - just mark the method which processes messages with synchronized modifier).
When one thread needs to synchronize with many other threads in a manner that a deadlock may occur, greedily acquire all your resources, and in the case that you can't acquire a single resource out of the set, release all resources and try again.
It's an algorithm based on the dining philosophers problem.
One important thing to remember is, that you have to aquire all locks in a consistent order across all your threads, in order to avoid the following situation:
Thread 1 Thread 2
acquire A acquire B
acquire B acquire A
One way to do it would be to have only objects used as locks, which can be ordered.
class Lock {
static final AtomicLong counter = new AtomicLong()
final long id = counter.incrementAndGet();
}
which must be used like
if (lock1.id < lock2.id) {
synchronized (lock1) {
synchronized (lock2) {
...
}
}
} else {
synchronized (lock2) {
synchronized (lock1) {
...
}
}
}
Obviously, this becomes tedious soon, in particular, the more locks are involved. Using explicit ReentrantLocks might help, as it more easily allows all that stuff to be factored out into a generic “grab multiple locks method“.
Another strategy, which might be applicable for your problem, would be "hand-over-hand" locking. Consider
class Node {
final ReentrantLock lock = new ReentrantLock();
Node previous;
Node next;
}
with a traversal operation like
Node start = ...;
Node successor;
start.lock.lock();
try {
successor = start.next;
successor.lock.lock();
} finally {
start.lock.unlock();
}
// Here, we own the lock on start's next sibling. We could continue
// with this scheme, traversing the entire graph, at any time holding
// at most two locks: the node we come from and the node we want to
// go to.
The above scheme still requires, that the locks are acquired in a consistent order across all threads. This means, that you can only every traverse the graph either in "forward" direction (i.e., following the thread of next pointers) or "backward" direction (going via previous). As soon as you start using both at random, things become prone to deadlocks again. This is potentially true also, if you make arbitrary changes to the graph structure, changing the positions of nodes.
How about actor model? Shortly speaking, in actor-based programming all threads work as independent actors (or, as you said, agents). Communication is done via messages. Each actor has its own message queue and processes these messages one by one. This model is implemented in a Scala programming language, and one of its frameworks - Akka - may be used from Java.
What I do is use ExecutorServices for each Thread Pool. When you want another thread to do work, you pass it copies (or immutable data) of all the information it will need. This way you have state which is local to a thread or thread pool and you have information which is passed to another thread. i.e. you never pass mutable state to another thread. This avoid the need to ever lock another threads data.