I'm supernew to multi-threading, but I think I got the overall idea more or less. I'm trying to fill a matrix multi-threaded, but my code is clearly not thread-safe, I have duplicate columns in my matrix, which is not the case when the matrix is filled regularly. Below is an example block of code. Note that the reader is a Scanner object and someOperationOnText(someText) returns an int[100] object.
int[][] mat = new int[100][100];
ExecutorService threadPool = Executors.newFixedThreadPool(8);
for (int i = 0; i < 100; i++) {
Set<Integer>someText = new HashSet<>(reader.next());
int lineIndex = i;
threadPool.submit(() -> mat[lineIndex] = someOperationOnText(someText);
}
Do you see any reason why this is not thread-safe? I can't seem to get my head around it, since the reading is done outside the thread-pool, I didn't think it would be at risk.
Thanks a lot for any debugging tips!
Grts
There is a happens-before between the submit calls and the execution of the lambda by a thread in the executor's thread pool. (See javadoc: "Memory consistency effects"). So that means the lambda will see the correct values for someText, mat and lineIndex.
The only thing that is not thread-safe about this is the (implied) code that uses the values in mat in the main thread. Calling shutdown() on the executor should be sufficient ... though the javadocs don't talk about the memory consistency effects of shutdown() and awaitTermination().
(By my reading of the code for ThreadPoolExecutor, the awaitTermination() method provides a happens-before between the pool threads (after completion of all tasks) and the method's return. The happens-before is due the use of the executor's main lock to synchronize the pool shutdown. It is hard to see how they could implement shutdown correctly without this (or equivalent), so it is more than an "implementation artefact" ... IMO.)
Well, accessing elements in the matrix is a very fast operation, a computing is not.
I think making access to a matrix synchronized while computing can be concurrent is a right approach for you.
int[][] mat = new int[100][100];
Object lock = new Object();
ExecutorService threadPool = Executors.newFixedThreadPool(8);
for (int i = 0; i < 100; i++) {
Set<Integer> someText = new HashSet<>(reader.next());
int lineIndex = i;
threadPool.submit(() -> {
int result = someOperationOnText(someText);
synchronized (lock) {
mat[lineIndex] = result;
}
});
}
Related
I'm wondering whether there is any advantage to keeping the same threads over the course of the execution of an object, rather than re-using the same Thread objects. I have an object for which a single (frequently used) method is parallelized using local Thread variables, such that every time the method is called, new Threads (and Runnables) are instantiated. Because the method is called so frequently, a single execution may instantiate upwards of a hundred thousand Thread objects, even though there are never more than a few (~4-6) active at any given time.
Following is a cut down example of how this method is currently implemented, to give a sense of what I mean. For reference, n is of course the pre-determined number of threads to use, whereas this.dataStructure is a (thread-safe) Map which serves as the input to the computation, as well as being modified by the computation. There are other inputs involved, but as they are not relevant to this question, I've omitted their usage. I've also omitted exception handling for the same reason.
Runnable[] tasks = new Runnable[n];
Thread[] threads = new Thread[n];
ArrayBlockingQueue<MyObject> inputs = new ArrayBlockingQueue<>(this.dataStructure.size());
inputs.addAll(this.dataStructure.values());
for (int i = 0; i < n; i++) {
tasks[i] = () -> {
while (true) {
MyObject input = inputs.poll(1L, TimeUnit.MICROSECONDS);
if (input == null) return;
// run computations over this.dataStructure
}
};
threads[i] = new Thread(tasks[i]);
threads[i].start();
}
for (int i = 0; i < n; i++)
threads[i].join();
Because these Threads (and their runnables) always execute the same way using a single ArrayBlockingQueue as input, an alternative to this would be to just "refill the queue" every time the method is called and just re-start the same Threads. This is easily implemented, but I'm unsure as to whether it would make any difference one way or the other. I'm not too familiar with concurrency, so any help is appreciated.
PS.: If there is a more elegant way to handle the polling, that would also be helpful.
It is not possible to start a Thread more than once, but conceptually, the answer to your question is yes.
This is normally accomplished with a thread pool. A thread pool is a set of Threads which rarely actually terminate. Instead, an application is passes its task to the thread pool, which picks a Thread in which to run it. The thread pool then decides whether the Thread should be terminated or reused after the task completes.
Java has some classes which make use of thread pools quite easy: ExecutorService and CompletableFuture.
ExecutorService usage typically looks like this:
ExecutorService executor = Executors.newCachedThreadPool();
for (int i = 0; i < n; i++) {
tasks[i] = () -> {
while (true) {
MyObject input = inputs.poll(1L, TimeUnit.MICROSECONDS);
if (input == null) return;
// run computations over this.dataStructure
}
};
executor.submit(tasks[i]);
}
// Doesn't interrupt or halt any tasks. Will wait for them all to finish
// before terminating its threads.
executor.shutdown();
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
Executors has other methods which can create thread pools, like newFixedThreadPool() and newWorkStealingPool(). You can decide for yourself which one best suits your needs.
CompletableFuture use might look like this:
Runnable[] tasks = new Runnable[n];
CompletableFuture<?>[] futures = new CompletableFuture<?>[n];
for (int i = 0; i < n; i++) {
tasks[i] = () -> {
while (true) {
MyObject input = inputs.poll(1L, TimeUnit.MICROSECONDS);
if (input == null) return;
// run computations over this.dataStructure
}
};
futures[i] = CompletableFuture.runAsync(tasks[i]);
}
CompletableFuture.allOf(futures).get();
The disadvantage of CompletableFuture is that the tasks cannot be canceled or interrupted. (Calling cancel will mark the task as completing with an exception instead of completing successfully, but the task will not be interrupted.)
Per definition, you cannot restart a thread. According to the documentation:
It is never legal to start a thread more than once. In particular, a thread may not be restarted once it has completed execution.
Nevertheless a thread is a valuable resource, and there are implementations to reuse threads. Have a look at the Java Tutorial about Executors.
My question is extremely basic: once I have written some array values by one or more threads (phase 1), how can I 'publish' my array to make all the changes visible to other threads (phase 2)?
I have code that does all the array writing, then all the array reading, then again all the writing, then again all the reading etc. I'd like to do it in multiple threads, so multiple threads first would do the array writing phase, then multiple threads would do the array reading phase etc.
My concern is how to safely publish the array writes after each writing phase.
Consider the following simplified thread-unsafe code, that does just one writing phase with just one thread and then just one reading phase with multiple threads:
ExecutorService executor = Executors.newFixedThreadPool(5);
double[] arr = new double[5];
for (int i=0; i<5; ++i) {
arr[i] = 1 + Math.random();
}
for (int i=0; i<5; ++i) {
final int j=i;
executor.submit(() -> System.out.println(String.format("arr[%s]=%s", j, arr[j])));
}
The code normally prints non-zero values, but I understand that it might occasionally print zeros as well, as the array is not properly published by the writing thread, so some writes might not be visible to other threads.
I'd like to fix this problem and write the above code properly, in a thread-safe manner, i.e. to make sure that all my writes will be visible to the reading threads.
1. Could you advise on the best way to do so?
The concurrent collections and AtomicXxxArray are not an option for me because of performance (and also code clarity), as I have 2D arrays etc.
2. I can think of the following possible solutions, but I am not 100% sure they would work. Could you also advise on the solutions below?
Solution 1: assignment to a final array
Justification: I expect a final field to be always properly initialized with the latest writes, including all its recursive dependencies.
for (int i=0; i<5; ++i) {
arr[i] = 1 + Math.random();
}
final double[] arr2 = arr; //<---- safe publication?
for (int i=0; i<5; ++i) {
final int j=i;
executor.submit(() -> System.out.println(String.format("arr[%s]=%s", j, arr2[j])));
}
Solution 2: a latch
Justification: I expect the latch to establish a perfect happens-before relationship between the writing thread(s) and the reading threads.
CountDownLatch latch = new CountDownLatch(1); //1 = the number of writing threads
for (int i=0; i<5; ++i) {
arr[i] = Math.random();
}
latch.countDown(); //<- writing is done
for (int i=0; i<5; ++i) {
final int j=i;
executor.submit(() -> {
try {latch.await();} catch (InterruptedException e) {...} //happens-before(writings, reading) guarantee?
System.out.println(String.format("arr[%s]=%s", j, arr[j]));
});
}
Update: this answer https://stackoverflow.com/a/5173805/1847482 suggests the following solution:
volatile int guard = 0;
...
//after the writing is done:
guard = guard + 1; //write some new value
//just before the reading: read the volatile variable, e.g.
guard = guard + 1; //includes reading
... //do the reading
This solution uses the following rule: "if thread A writes some non-volatile stuff and a volatile variable after that, thread B is guaranteed to see the changes of the volatile stuff as well if it reads the volatile variable first".
Your first example is perfectly safe, because the tasks originate from the writer thread. As the docs say:
Actions in a thread prior to the submission of a Runnable to an Executor happen-before its execution begins.
I need to calculate an array via a thread like this:
public class Calculator implements Callable<int[]> {
#Override
public int[] call() {
int[] result = new int[100];
for (int i = 0; i < result.length; i++) {
result[i]++;
}
return result;
}
}
And a main thread may look like this:
ExecutorService executor = Executors.newFixedThreadPool(1);
Future<int[]> future = executor.submit(new Calculator());
int[] result = future.get();
I cannot figure out how to ensure the visibility of all elements in result. According to the Memory Consistency Properties section, there seems to be no such happens-before relation.
One way to walk around is to wrap the array into an AtomicIntegerArray. But I wonder whether there is some other solution which can be generalized to any kind of primitive arrays.
Thanks
From the javadoc of Future https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Future.html
Memory consistency effects: Actions taken by the asynchronous computation happen-before actions following the corresponding Future.get() in another thread.
I'm trying to implement a fast version of LZ77 and I have a question to ask you about concurrent programming.
For now I have a final byte[] buffer and a final int[] resultHolder, both of the same length. The program does the following:
The main thread writes all the buffer, then notifies the Threads and wait for them to complete.
The single working Thread processes a portion of the buffer saving the results in the same portion of the result holder. Worker's portion is exclusive. After that the main thread is notified and the worker pauses.
When all the workers have paused, the main thread reads the data in resultHolder and updates the buffer, then (if needed) the process begins again from point 1.
Important things in manager (main Thread) are declared as follow:
final byte[] buffer = new byte[SIZE];
final MemoryHelper memoryHelper = new MemoryHelper();
final ArrayBlockingQueue<Object> waitBuffer = new ArrayBlockingQueue<Object>(TOT_WORKERS);
final ArrayBlockingQueue<Object> waitResult = new ArrayBlockingQueue<Object>(TOT_WORKERS);
final int[] resultHolder = new int[SIZE];
MemoryHelper simply wraps a volatile field and provides two methods: one for reading it and one for writing to it.
Worker's run() code:
public void run() {
try {
// Wait main thread
while(manager.waitBuffer.take() != SHUTDOWN){
// Load new buffer values
manager.memoryHelper.readVolatile();
// Do something
for (int i = a; i <= b; i++){
manager.resultHolder[i] = manager.buffer[i] + 10;
}
// Flush new values of resultHolder
manager.memoryHelper.writeVolatile();
// Signal job done
manager.waitResult.add(Object.class);
}
} catch (InterruptedException e) { }
}
Finally, the important part of main Thread:
for(int i=0; i < 100_000; i++){
// Start workers
for (int j = 0; j < TOT_WORKERS; j++)
waitBuffer.add(Object.class);
// Wait workers
for (int j = 0; j < TOT_WORKERS; j++)
waitResult.take();
// Load results
memoryHelper.readVolatile();
// Do something
processResult();
setBuffer();
// Store buffer
memoryHelper.writeVolatile();
}
Synchronization on ArrayBlockingQueue works well. My doubt is in using readVolatile() and writeVolatile(). I've been told that writing to a volatile field flushes to memory all the previously changed data, then reading it from another thread makes them visible.
So is it enough in this case to ensure a correct visibility? There is never a real concurrent access to the same memory areas, so a volatile field should be a lot cheaper than a ReadWriteLock.
You don't even need volatile here, because BlockingQueues already provide necessary memory visibility guarantees:
Memory consistency effects: As with other concurrent collections, actions in a thread prior to placing an object into a BlockingQueue happen-before actions subsequent to the access or removal of that element from the BlockingQueue in another thread.
In general, if you already have some kind of synchronization, you probably don't need to do anything special to ensure memory visibility, because it's already guaranteed by synchronization primitives you use.
However, volatile reads and writes can be used to ensure memory visibility when you don't have explicit synchronization (e.g. in lock-free algorithms).
P. S.
Also it looks like you can use CyclicBarrier instead of your solution with queues, it's especially designed for similar scenarios.
Suppose I have the following code which I wan't to optimize by spreading the workload over the multiple CPU cores of my PC:
double[] largeArray = getMyLargeArray();
double result = 0;
for (double d : largeArray)
result += d;
System.out.println(result);
In this example I could distribute the work done within the for-loop over multiple threads and verify that the threads have all terminated before proceeding to printing the result. I therefore came up with something that looks like this:
final double[] largeArray = getMyLargeArray();
int nThreads = 5;
final double[] intermediateResults = new double[nThreads];
Thread[] threads = new Thread[nThreads];
final int nItemsPerThread = largeArray.length/nThreads;
for (int t = 0; t<nThreads; t++) {
final int t2 = t;
threads[t] = new Thread(){
#Override public void run() {
for (int d = t2*nItemsPerThread; d<(t2+1)*nItemsPerThread; d++)
intermediateResults[t2] += largeArray[d];
}
};
}
for (Thread t : threads)
t.start();
for (Thread t : threads)
try {
t.join();
} catch (InterruptedException e) { }
double result = 0;
for (double d : intermediateResults)
result += d;
System.out.println(result);
Assume that the length of the largeArray is dividable by nThreads. This solution works correctly.
However, I am encountering the problem that the above threading of for-loops occurs a lot in my program, which causes a lot of overhead due to the creation and garbage collection of threads. I am therefore looking at modifying my code by using a ThreadPoolExecutor. The threads giving the intermediate results would then be reused in the next execution (summation, in this example).
Since I store my intermediate results in an array of a size which has to be known beforehand, I was thinking of using a thread pool of fixed size.
I am having trouble, however, with letting a thread know at which place in the array it should store its result.
Should I define my own ThreadFactory?
Or am I better of using an array of ExecutorServices created by the method Executors.newSingleThreadExecutor(ThreadFactory myNumberedThreadFactory)?
Note that in my actual program it is very hard to replace the double[] intermediateResults with something of another type. I would prefer a solution which is confined to creating the right kind of thread pool.
I am having trouble, however, with letting a thread know at which place in the array it should store its result. Should I define my own ThreadFactory?
No need for that. The interfaces used by executors (Runnable and Callable) are run by threads, and you can pass whatever arguments to implementations you want to pass (for instance, an array, a begin index and an end index).
A ThreadPoolExecutor is indeed a good solution. Also look at FutureTask if you have runnables bearing results.
ExecutorService provides you with API to get the result from thread pool via Future interface:
Future<Double> futureResult = executorService.submit(new Callable<Double>() {
Double call() {
double totalForChunk = 0.0;
// do calculation here
return totalForChunk;
}
});
Now all you need to do is to submit tasks (Callable instances) and wait for result to be available:
List<Future<Double>> results = new ArrayList<Double>();
for (int i = 0; i < nChunks; i++) {
results.add(executorService.submit(callableTask));
}
Or even simpler:
List<Future<Double>> results = executorService.invokeAll(callableTaskList);
The rest is easy, iterate over results and collect total:
double total = 0.0;
for (Future<Double> result : results) {
total += result.get(); // this will block until your task is completed by executor service
}
Having that said, you do not care how much threads you have in executor service. You just submit tasks and collect results when they are available.
You would be better off creating "worker" threads that take information about work to be performed from a queue. Your process would then be to create an initially empty WorkQueue and then create and start the worker threads. Each worker thread would pick up its work from the queue, do the work, and put the work on a "finished" queue for the master to pick up and handle.