We Keep Coding

Using Lock to lock a position in an array instead of using synchronized for concurrency (Fine Grained Locking)

I have been asked to implement fine grained locking on a hashlist. I have done this using synchronized but the questions tells me to use Lock instead.
I have created a hashlist of objects in the constructor
private LinkedList<E> data[];;
private Lock lock[];
private Lock lockR = new ReentrantLock();
// The constructors ensure that both the data and the dataLock are the same size
#SuppressWarnings("unchecked")
public ConcurrentHashList(int n){
if(n > 1000) {
data = (LinkedList<E>[])(new LinkedList[n/10]);
lock = new Lock [n/10];
}
else {
data = (LinkedList<E>[])(new LinkedList[100]);
lock = new Lock [100]; ;
}
for(int j = 0; j < data.length;j++) {
data[j] = new LinkedList<E>();
lock[j] = new ReentrantLock();// Adding a lock to each bucket index
}
}
The original method
public void add(E x){
if(x != null){
lock.lock();
try{
int index = hashC(x);
if(!data[index].contains(x))
data[index].add(x);
}finally{lock.unlock();}
}
}
Using synchronization to grab a handle on the object hashlist to allow mutable Threads to work on mutable indexes concurrently.
public void add(E x){
if(x != null){
int index = hashC(x);
synchronized (dataLock[index]) { // Getting handle before adding
if(!data[index].contains(x))
data[index].add(x);
}
}
}
I do not know how to implement it using Lock though I can not lock a single element in a array only the whole method which means it is not coarse grained.
Using an array of ReentrantLock
public void add(E x){
if(x != null){
int index = hashC(x);
dataLock[index].lock();
try {
// Getting handle before adding
if(!data[index].contains(x))
data[index].add(x);
}finally {dataLock[index].unlock();}
}
}
The hash function
private int hashC(E x){
int k = x.hashCode();
int h = Math.abs(k % data.length);
return(h);
}

Presumably, hashC() is a function that is highly likely to produce unique numbers. As in, you have no guarantee that the hashes are unique, but the incidence of non-unique hashes is extremely low. For a data structure with a few million entries, you have a literal handful of collisions, and any given collision always consists of only a pair or maybe 3 conflicts (2 to 3 objects in your data structure have the same hash, but not 'thousands').
Also, assumption: the hash for a given object is constant. hashC(x) will produce the same value no matter how many times you call it, assuming you provide the same x.
Then, you get some fun conclusions:
The 'bucket' (The LinkedList instance found at array slot hashC(x) in data) that your object should go into, is always the same - you know which one it should be based solely on the result of hashC.
Calculating hashC does not require a lock of any sort. It has no side effects whatsoever.
Thus, knowing which bucket you need for a given operation on a single value (Be it add, remove, or check-if-in-collection) can be done without locking anything.
Now, once you know which bucket you need to look at / mutate, okay, now locking is involved.
So, just have 1 lock for each bucket. Not a List<Object> locks[];, that's a whole list worth of locks per bucket. Just Object[] locks is all you need, or ReentrantLock[] locks if you prefer to use lock/unlock instead of synchronized (lock[bucketIdx]) { ... }.
This is effectively fine-grained: After all, the odds that one operation needs to twiddle its thumbs because another thread is doing something, even though that other thread is operating on a different object, is very low; it would require the two different objects to have a colliding hash, which is possible, but extremely rare - as per assumption #1.
NB: Note that therefore lock can go away entirely, you don't need it, unless you want to build into your code that the code may completely re-design its bucket structure. For example, 1000 buckets feels a bit meh if you end up with a billion objects. I don't think 'rebucket everything' is part of the task here, though.

CopyOnWriteArrayList is too slow

I have following case,
public class Test {
private static final int MAX_NUMBER = 10_00_00;
public static void main(String[] args) {
List<Integer> list = new CopyOnWriteArrayList<>();
long start = System.nanoTime();
for(int i = 0; i < MAX_NUMBER; i++) {
list.add(i * 2);
}
long end = System.nanoTime();
System.out.println(((end - start) / Math.pow(10, 9)));
}
}
OUTPUT
6.861539857
It adds element quite slowly compared to ArrayList which took approximately 0.004690843. I came to know the reason in the documentation,
A thread-safe variant of ArrayList in which all mutative operations
(add, set, and so on) are implemented by making a fresh copy of the
underlying array.
So, my understanding is, whenever I add new element in this list it will create new fresh array and add element at last index of this array. I find a lock in add method and apart from that that method is actually creating new array every time.
When I increased MAX_NUMBER to 10_00_000 my programs keeps on running and never end (it would but I can't wait for so long).
I think Collections.synchronizedList is the better choice when you want thread safety with speed. I used it and it took about 0.007673728.
My questions :
Why internally it create new array does thread safety is related to this ?
Why it took so much time in case of MAX_NUMBER = 10_00_000 ? (as it took about 6 seconds with MAX_NUMBER = 10_00_00) Is this happening because mutative operation creates new array every time ?
Does this mean CopyOnWriteArrayList has performance drawback when you have huge number of elements and better to choose something else (i.e. Collections.synchronizedList)?
Is this the reason we usually don't see CopyOnWriteArrayList in public APIs ? Is there any drawbacks other than this ?

CopyOnWriteArrayList is the preferred option ONLY WHEN there are very less number of writes and huge number of reads (if multiple threads are accessing this list)

Problems with adding string to a list by many threads

this is a follow-up post to Do I need a concurrent collection for adding elements to a list by many threads?
everybody there has focused on expaning of the list. I understand how that can be a problem, but .. what about adding elements ? is that thread-safe?
example code
static final Collection<String> FILES = new ArrayList<String>(1000000);
and I execute in many threads (I add less than 1000000 elements)
FILES.add(string)
is that thread safe ? what are possible problems with doing it that way ?

ArrayList<String> is not synchronized by itself. Use Collections.synchronizedList(new ArrayList<String>()) instead.

use Collections.syncronizedList(new ArrayList<String>(1000000))

Even if the list doesn't expand, it maintains its internal state, such as the current size. In a wider sense, ArrayList is specified a mutable object which is not thread-safe, therefore it is illegal to concurrently call any mutator methods on it.

public boolean add(E e) {
ensureCapacityInternal(size + 1); // Increments modCount!!
elementData[size++] = e;
return true;
}
This is the source cod of the ArrayList in 7u40-b43.
The method ensureCapacityInternal() is not thread safe,and size++ is not either.
The size++ is done by three steps,1)get the size,2) size +1,3) write the new value back.

Java concurrency - improving a copy-on-read collection

I have a multithreaded application, where a shared list has write-often, read-occasionally behaviour.
Specifically, many threads will dump data into the list, and then - later - another worker will grab a snapshot to persist to a datastore.
This is similar to the discussion over on this question.
There, the following solution is provided:
class CopyOnReadList<T> {
private final List<T> items = new ArrayList<T>();
public void add(T item) {
synchronized (items) {
// Add item while holding the lock.
items.add(item);
}
}
public List<T> makeSnapshot() {
List<T> copy = new ArrayList<T>();
synchronized (items) {
// Make a copy while holding the lock.
for (T t : items) copy.add(t);
}
return copy;
}
}
However, in this scenario, (and, as I've learned from my question here), only one thread can write to the backing list at any given time.
Is there a way to allow high-concurrency writes to the backing list, which are locked only during the makeSnapshot() call?

synchronized (~20 ns) is pretty fast and even though other operations can allow concurrency, they can be slower.
private final Lock lock = new ReentrantLock();
private List<T> items = new ArrayList<T>();
public void add(T item) {
lock.lock();
// trivial lock time.
try {
// Add item while holding the lock.
items.add(item);
} finally {
lock.unlock();
}
}
public List<T> makeSnapshot() {
List<T> copy = new ArrayList<T>(), ret;
lock.lock();
// trivial lock time.
try {
ret = items;
items = copy;
} finally {
lock.unlock();
}
return ret;
}
public static void main(String... args) {
long start = System.nanoTime();
Main<Integer> ints = new Main<>();
for (int j = 0; j < 100 * 1000; j++) {
for (int i = 0; i < 1000; i++)
ints.add(i);
ints.makeSnapshot();
}
long time = System.nanoTime() - start;
System.out.printf("The average time to add was %,d ns%n", time / 100 / 1000 / 1000);
}
prints
The average time to add was 28 ns
This means if you are creating 30 million entries per second, you will have one thread accessing the list on average. If you are creating 60 million per second, you will have concurrency issues, however you are likely to be having many more resourcing issue at this point.
Using Lock.lock() and Lock.unlock() can be faster when there is a high contention ratio. However, I suspect your threads will be spending most of the time building the objects to be created rather than waiting to add the objects.

You could use a ConcurrentDoublyLinkedList. There is an excellent implementation here ConcurrentDoublyLinkedList.
So long as you iterate forward through the list when you make your snapshot all should be well. This implementation preserves the forward chain at all times. The backward chain is sometimes inaccurate.

First of all, you should investigate if this really is too slow. Adds to ArrayLists are O(1) in the happy case, so if the list has an appropriate initial size, CopyOnReadList.add is basically just a bounds check and an assignment to an array slot, which is pretty fast. (And please, do remember that CopyOnReadList was written to be understandable, not performant.)
If you need a non-locking operation, you can have something like this:
class ConcurrentStack<T> {
private final AtomicReference<Node<T>> stack = new AtomicReference<>();
public void add(T value){
Node<T> tail, head;
do {
tail = stack.get();
head = new Node<>(value, tail);
} while (!stack.compareAndSet(tail, head));
}
public Node<T> drain(){
// Get all elements from the stack and reset it
return stack.getAndSet(null);
}
}
class Node<T> {
// getters, setters, constructors omitted
private final T value;
private final Node<T> tail;
}
Note that while adds to this structure should deal pretty well with high contention, it comes with several drawbacks. The output from drain is quite slow to iterate over, it uses quite a lot of memory (like all linked lists), and you also get things in the opposite insertion order. (Also, it's not really tested or verified, and may actually suck in your application. But that's always the risk with using code from some random dude on the intertubes.)

Yes, there is a way. It is similar to the way ConcurrentHashMap made, if you know.
You should make your own data structure not from one list for all writing threads, but use several independent lists. Each of such lists should be guarded by it's own lock. .add() method should choose list for append current item based on Thread.currentThread.id (for example, just id % listsCount). This will gives you good concurrency properties for .add() -- at best, listsCount threads will be able to write without contention.
On makeSnapshot() you should just iterate over all lists, and for each list you grab it's lock and copy content.
This is just an idea -- there are many places to improve it.

You can use a ReadWriteLock to allow multiple threads to perform add operations on the backing list in parallel, but only one thread to make the snapshot. While the snapshot is being prepared all other add and snapshot request are put on hold.
A ReadWriteLock maintains a pair of associated locks, one for
read-only operations and one for writing. The read lock may be held
simultaneously by multiple reader threads, so long as there are no
writers. The write lock is exclusive.
class CopyOnReadList<T> {
// free to use any concurrent data structure, ConcurrentLinkedQueue used as an example
private final ConcurrentLinkedQueue<T> items = new ConcurrentLinkedQueue<T>();
private final ReadWriteLock rwLock = new ReentrantReadWriteLock();
private final Lock shared = rwLock.readLock();
private final Lock exclusive = rwLock.writeLock();
public void add(T item) {
shared.lock(); // multiple threads can attain the read lock
// try-finally is overkill if items.add() never throws exceptions
try {
// Add item while holding the lock.
items.add(item);
} finally {
shared.unlock();
}
}
public List<T> makeSnapshot() {
List<T> copy = new ArrayList<T>(); // probably better idea to use a LinkedList or the ArrayList constructor with initial size
exclusive.lock(); // only one thread can attain write lock, all read locks are also blocked
// try-finally is overkill if for loop never throws exceptions
try {
// Make a copy while holding the lock.
for (T t : items) {
copy.add(t);
}
} finally {
exclusive.unlock();
}
return copy;
}
}
Edit:
The read-write lock is so named because it is based on the readers-writers problem not on how it is used. Using the read-write lock we can have multiple threads achieve read locks but only one thread achieve the write lock exclusively. In this case the problem is reversed - we want multiple threads to write (add) and only thread to read (make the snapshot). So, we want multiple threads to use the read lock even though they are actually mutating. Only thread is exclusively making the snapshot using the write lock even though snapshot only reads. Exclusive means that during making the snapshot no other add or snapshot requests can be serviced by other threads at the same time.
As #PeterLawrey pointed out, the Concurrent queue will serialize the writes aqlthough the locks will be used for as minimal a duration as possible. We are free to use any other concurrent data structure, e.g. ConcurrentDoublyLinkedList. The queue is used only as an example. The main idea is the use of read-write locks.

What is the use of AtomicReferenceArray?

When is it a good idea to use AtomicReferenceArray? Please explain with an example.

looks like it's functionally equivalent to AtomicReference[], occupying a little less memory though.
So it's useful when you need more than a million atomic references - can't think of any use case.

If you had a shared array of object references, then you would use an AtomicReferenceArray to ensure that the array couldn't be updated simultaneously by different threads i.e. only one element can be updated at a time.
However, in an AtomicReference[] (array of AtomicReference) multiple threads can still update different elements simulateously, because the atomicity is on the elements, not on the array as a whole.
More info here.

It could be useful if you have a large number of objects that are updated concurrently, for example in a large multiplayer game.
An update of reference i would follow the pattern
boolean success = false;
while (!success)
{
E previous = atomicReferenceArray.get(i);
E next = ... // compute updated object
success = atomicReferenceArray.compareAndSet(i, previous, next);
}
Depending on the circumstances this may be faster and/or easier to use than locking (synchronized).

One possible use case would have been ConcurrentHashMap which extensively uses array internally. Array can be volatile but at per element level sematics can't be volatile. it's one of the reason automic array came into existence.

some notes from a C++ programmer below, please don't condemn my Java much :)
AtomicReferenceArray allows to avoid false sharing, when multiple CPU logical cores access the same cache line that is changed by one of the thread. Invalidating and re-fetching the cache is very expensive. Unfortunately there is no sizeof in Java, so we don't know how many bytes each AtomicReference takes, but assuming it's at least 8 bytes (the size of a pointer on 64-bit architectures), you can allocate as follows:
// a lower bound is enough
private final int sizeofAtomicReference = 8;
// good for x86/x64
private final int sizeofCacheLine = 64;
// the number of CPU cores
private final int nLogicalCores = Runtime.getRuntime().availableProcessors();
private final int refsPerCacheLine = (sizeofCacheLine + sizeofAtomicReference - 1) / sizeofAtomicReference;
private AtomicReferenceArray<Task> tasks = new AtomicReferenceArray<Task>(nLogicalCores * refsPerCacheLine);
Now if you assign a task to i-th thread via
tasks.compareAndSet(i*refsPerCacheLine, null, new Task(/*problem definition here*/));
you guarantee that the task references are assigned to different CPU cache lines. Thus there is no expensive false sharing. So the latency of passing tasks from the producer thread to the consumer threads is minimal (for Java, but not for C++/Assembly).
Bonus:
You then poll the tasks array in the worker threads like this:
// consider iWorker is the 0-based index of the logical core this thread is assigned to
final int myIndex = iWorker*refsPerCacheLine;
while(true) {
Task curTask = tasks.get(myIndex);
if(curTask == null) continue;
if(curTask.isTerminator()) {
return; // exit the thread
}
// ... Process the task here ...
// Signal the producer thread that the current worker is free
tasks.set(myIndex, null);
}

import java.util.concurrent.atomic.AtomicReferenceArray;
public class AtomicReferenceArrayExample {
AtomicReferenceArray<String> arr = new AtomicReferenceArray<String>(10);
public static void main(String... args) {
new Thread(new AtomicReferenceArrayExample().new AddThread()).start();
new Thread(new AtomicReferenceArrayExample().new AddThread()).start();
}
class AddThread implements Runnable {
#Override
public void run() {
// Sets value at the index 1
arr.set(0, "A");
// At index 0, if current reference is "A" then it changes as "B".
arr.compareAndSet(0, "A", "B");
// At index 0, if current value is "B", then it is sets as "C".
arr.weakCompareAndSet(0, "B", "C");
System.out.println(arr.get(0));
}
}
}
// Result:
// C
// C