When to prefer LinkedBlockingQueue over ArrayBlockingQueue?
Which data structure to use among LinkedBlockingQueue and ArrayBlockingQueue when:
You want an efficient read and write
should have lesser memory footprints
Although there is a similar question but it does not highlight the fact that which should be preferred?
Links:
Java: ArrayBlockingQueue vs. LinkedBlockingQueue
What is the Difference between ArrayBlockingQueue and LinkedBlockingQueue
Boris the Spider has already outlined the most visible difference between ArrayBlockingQueue and LinkedBlockingQueue - the former is always bounded, while the latter can be unbounded.
So in case you need an unbounded blocking queue, LinkedBlockingQueue or a LinkedTransferQueue used as a BlockingQueue are your best bets from the java.util.concurrent toolbox.
But let's say you need a bounded blocking queue.
In the end, you should choose an implementation based on extensive experimenting with a simulation of your real-world workload.
Nevertheless, here are some notes that can help you with your choice or with interpreting the results from the experiment:
ArrayBlockingQueue can be created with a configurable (on/off) scheduling fairness policy. This is great if you need fairness or want to avoid producer/consumer starvation, but it will cost you in throughput.
ArrayBlockingQueue pre-allocates its backing array, so it doesn't allocate nodes during its usage, but it immediately takes what can be a considerable chunk of memory, which can be a problem if your memory is fragmented.
ArrayBlockingQueue should have less variability in performance, because it has less moving parts overall, it uses a simpler and less-sophisticated single-lock algorithm, it does not create nodes during usage, and its cache behavior should be fairly consistent.
LinkedBlockingQueue should have better throughput, because it uses separate locks for the head and the tail.
LinkedBlockingQueue does not pre-allocate nodes, which means that its memory footprint will roughly match its size, but it also means that it will incur some work for allocation and freeing of nodes.
LinkedBlockingQueue will probably have worse cache behavior, which may affect its own performance, but also the performance of other components due to false sharing.
Depending on your use-case and how much do you care about performance, you may also want to look outside of java.util.concurrent and consider Disruptor (an exceptionally fast, but somewhat specialized bounded non-blocking ring buffer) or JCTools (a variety of bounded or unbounded queues with different guarantees depending on the number of producers and consumers).
From the JavaDoc for ArrayBlockingQueue
A bounded blocking queue backed by an array.
Emphasis mine
From the JavaDoc for LinkedBlockingQueue:
An optionally-bounded blocking queue based on linked nodes.
Emphasis mine
So if you need a bounded queue you can use either, if you need an unbounded queue you must use LinkedBlockingQueue.
For a bounded queue, then you would need to benchmark to work out which is better.
Related
I understand that java parallelStream(or ForkJoinPool) is designed to maximize CPU Utilization.
Because of the great combination of java stream functional interfaces (such as map, reduce, filter) and ForkJoinPool I use java parallelstream and ForkJoinPool.
The problem is that ForkJoin does not limit the number of active thread.
If some threads blocked while it hold large memory, ForkJoinPool tries to create more threads to meet the parallelism(running thread).
In this case cpu utilization will be maximized but heap memory will be Exhausted or even go OOM.
How can I limit the number of active threads of ForkJoinPool?
Or is there any java stream interface alternatives?
I understand that java parallelStream(or ForkJoinPool) is designed to maximize CPU Utilization.
That's not exactly the goal. It may have the effect of maximizing CPU utilization, but the goal is to speed up the computation. That is NOT the same thing.
How can I limit the number of active threads of ForkJoinPool?
According to this mailing list thread, one way prevent the forkjoin thread pool from exploding in pathological situations is to supply a custom ForkJoinThreadFactory that keeps track of the number of extant threads (somehow) and returns null when too many threads already exist.
Be aware that if you do hit the limit (imposed by your factory), you will get RejectedExecutionExceptions on task submission.
Or is there any java stream interface alternatives?
I'm not aware of one.
But I'm not convinced that you would encounter this problem at all when using Stream.parallelStream() in the normal way.
If you do encounter it and rejected executions are problematic, you probably need to look for another way to express the computation; e.g. using coroutines rather than threads, or with work queues and a Executor, or something else that I haven't thought of :-)
I am looking for a collection which has the most efficient "add item" concurrent operation. Scala or Java is good.
I typically:
insert 100.000s of entries, one at a time
don't care about order
only read and clear the collection when there is no append. So not critical
Also, it should work with multiple thread (hence the concurrent constraint). But I need it to be most effective when there is no concurrency: the design for concurrent-safety should not have too much impact when there is no concurrent access.
I use this collection to record performance measurements. This is why it should be the most efficient possible to not bias too much actual performance. However, because the collection size is possibly big and not known in advance, it should cope efficiently with size increase.
So which collection would be best to use?
I currently used a mutable.ListBuffer with buffer.synchronized{ ... } around append (and clear) operation. I tried to use a var buf: List (scala) with similar synchronized{ ... } block but it seriously impacted the measurements.
I'd say the ConcurrentLinkedQueue. It's O(1) insertion using CAS. So under moderate load you probably won't have faster inserts. If you have very very high load, you may want to consider a LinkedBlockingQueue.
Since you say it's likely going to be single-threaded on adds, using CLQ and CAS will be your best option.
I have a program that calculates Pi from the Chudnovsky formula. It's written in Java and it uses a shared Vector that is used to save intermediate calculations like factorials and powers that include the index of the element.
However, I believe that since it's a synchronized Vector (thread safe by default) only one thread can read or write to it. So when we have lots of threads, instead of having increasing speedup, we see the computation time becomes constant.
Is there anything that I can do to circumvent that? What to do when there are too many threads reading/writing to the same shared memory?
When the access pattern is lots of reads and occasional writes, you can protect an unsyncronized data structure with a ReentrantReadWriteLock. It allows multiple readers, but only a single writer.
Depending on your implementation, you might also benefit from using a ConcurrentHashMap.
You might be able to cheat a bit and use either an AtomicIntegerArray or an AtomicReferenceArray of Futures/CompletionStages.
Store the results of each thread in a stack. One thread collects results from every thread and adds them together. Of course the stack should not be empty.
If you want multiple threads to work on factorials why not create a thread or two that produce a list of factorial results. Other threads can just look up results if needed.
Instead of having the same shared memory, you can have multiple threads with individual memories in a stack. Eventually, add all these up together (or occasionally) with one thread!
If you need high throughput, you can consider using Disruptor and RingBuffer.
At a crude level you can think of a Disruptor as a multicast graph of queues where producers put objects on it that are sent to all the consumers for parallel consumption through separate downstream queues. When you look inside you see that this network of queues is really a single data structure - a ring buffer.
Each producer and consumer has a sequence counter to indicate which slot in the buffer it's currently working on. Each producer/consumer writes its own sequence counter but can read the others' sequence counters
Few useful links:
https://lmax-exchange.github.io/disruptor
http://martinfowler.com/articles/lmax.html
https://softwareengineering.stackexchange.com/questions/244826/can-someone-explain-in-simple-terms-what-is-the-disruptor-pattern
For the logging feature I am working on, I need to have a processing thread which will sit waiting for jobs and execute them in batches when the count reaches or exceeds certain number. Since it is a standard case of producer consumer problem, I intend to use BlockingQueues. I have a number of producers adding entries to the queue using add() method, whereas there is only one consumer thread that uses take() to wait on the queue.
LinkedBlockingQueue seems to be a good option since it does not have any size restriction on it, however I am confused reading this from the documentation.
Linked queues typically have higher throughput than array-based queues but less predictable performance in most concurrent applications.
It was not clearly explained what they mean by this statement. Can some one please throw light on it? Does it mean LinkedBlockingQueue is not thread safe? Did any of you encounter any issues using LinkedBlockingQueue.
Since the number of producers are lot more, there is always a scenario I can run into where the queue is overwhelmed with large number of entries to be added. If I were to use ArrayBlockingQueue instead, which takes size of the queue as parameter in the constructor, I could always run into capacity full related exceptions. In order to avoid this, I am not sure how to determine what size I should instantiate my ArrayBlockingQueue with. Did you have to solve a similar problem using ArrayBlockingQueue?
Does it mean LinkedBlockingQueue is not thread safe?
It certainly does not mean that. The phrase "less predictable performance" is talking about just that -- performance -- and not some violation of the thread-safety or Java collections contract.
I suspect this is more around the fact that it is a linked-list so iterating and other operations on the collection will be slower so the class will hold locks longer. It also has to deal with more memory structures since each element has it's one linked-list node as opposed to just an entry in an array. This means that it has to flush more dirty memory pages between processors when synchronizing. Again, this impacts performance.
What they are trying to say is that if you can, you should use the ArrayBlockingQueue but otherwise I wouldn't worry about it.
Did any of you encounter any issues using LinkedBlockingQueue.
I've used it a lot and not seen any problems. It also is used a lot in the ExecutorService classes which are used everywhere.
As far as I know both the linked list and array can grow without bounds or am I wrong ? But when I have gone through the documentation in the Executor Service I see this :
Unbounded queues. Using an unbounded queue (for example a
LinkedBlockingQueue without a predefined capacity) will cause new
tasks to wait in the queue when all corePoolSize threads are busy.
Thus, no more than corePoolSize threads will ever be created. (And the
value of the maximumPoolSize therefore doesn't have any effect.)
So does the Unbounded Queue property changes when the LinkedBlockingQueue has a defined capacity ?
And this written for ArrayBlockingQueue:
Bounded queues. A bounded queue (for example, an ArrayBlockingQueue)
helps prevent resource exhaustion when used with finite
maximumPoolSizes, but can be more difficult to tune and control. Queue
sizes and maximum pool sizes may be traded off for each other: Using
large queues and small pools minimizes CPU usage, OS resources, and
context-switching overhead, but can lead to artificially low
throughput. If tasks frequently block (for example if they are I/O
bound), a system may be able to schedule time for more threads than
you otherwise allow. Use of small queues generally requires larger
pool sizes, which keeps CPUs busier but may encounter unacceptable
scheduling overhead, which also decreases throughput.
Why do you think that an ArrayBlockingQueue can grow without bounds? From its own documentation:
This is a classic "bounded buffer", in which a fixed-sized array holds elements inserted by producers and extracted by consumers. Once created, the capacity cannot be increased. Attempts to put an element into a full queue will result in the operation blocking; attempts to take an element from an empty queue will similarly block.
In other words, once it gets full, it's full - it doesn't grow.
Are you getting confused with an ArrayList by any chance - which is also backed by an array, but which expands this as required?
So does the Unbounded Queue property changes when the LinkedBlockingQueue has a defined capacity ?
Yes, hence why it's described as "optionally-bounded" in its Javadocs. Furthermore, the docs state that (emphasis mine):
The optional capacity bound constructor argument serves as a way to prevent excessive queue expansion. The capacity, if unspecified, is equal to Integer.MAX_VALUE. Linked nodes are dynamically created upon each insertion unless this would bring the queue above capacity.
The javadoc for LinkedBlockingQueue says:
An optionally-bounded blocking queue based on linked nodes.[...]
The optional capacity bound constructor argument serves as a way to
prevent excessive queue expansion. The capacity, if unspecified, is
equal to Integer.MAX_VALUE.
The javadoc of ArrayBlockingQueue says:
A bounded blocking queue backed by an array.[...]
This is a classic "bounded buffer", in which a fixed-sized array holds
elements inserted by producers and extracted by consumers. Once
created, the capacity cannot be increased
So, a LinkedBlockingQueue can be bounded or unbounded, whereas an ArrayBlockingQueue is always bounded.
As far as I know both the linked list and array can grow without bounds or am I wrong
A linked list as an unlimited size. An array has fixed size. An ArrayList wraps an array and replaces it when it needs a bigger one.
So does the Unbounded Queue property changes when the LinkedBlockingQueue has a defined capacity
When LinkedBlockingQueue has a maximum capacity, it is bounded but it not used this way by default.
From the documentataion for ArrayBlockingQueue
A bounded blocking queue backed by an array. This queue orders elements FIFO (first-in-first-out). The head of the queue is that element that has been on the queue the longest time. The tail of the queue is that element that has been on the queue the shortest time. New elements are inserted at the tail of the queue, and the queue retrieval operations obtain elements at the head of the queue.
If you notice all the constructors of ArrayBlockingQueue take a capacity because this class was designed to be bounded. This choice was made because if you want a concurrent queue, you probably don't want the overhead that comes with resizing an ArrayList. Hence, if you want an unbounded queue LinkedBlockingQueue is a better option since it does not involve this overhead.
other answer is very right! I provide another way to explain.
well, I also get confused by the term "unbound and bound" passed. you can look at the source code blew.
/** The queued items */
final Object[] items;
/** items index for next take, poll, peek or remove */
int takeIndex;
/** items index for next put, offer, or add */
int putIndex;
/** Number of elements in the queue */
int count;
from the source code, we can see the array is final, so we can not resize the array. if use LinkedBlockingQueue, we can always add more elements...and in the source code, the next reference is not final. NOTE, in theory, LinkedBlockingQueue is not unbounded. because it can only store MAX_INTEGER minus 8 elements. from javadoc, the unbounded queue is PriorityBlockingQueue. but the PriorityBlockingQueue also can only store MAX_INTEGER -8 elements. so i think there is no perfect unbounded queue...