This is a simplification of my code:
for(int i = 0; i < 500; i++) {
someMethod(i);
}
someMethod() takes a long time to execute, so I want to use multithreading to break up the for-loop into 5 intervals of 100:
for(int i = 0; i < 100; i++) {
someMethod(i);
}
for(int i = 100; i < 200; i++) {
someMethod(i);
}
...
for(int i = 400; i < 500; i++) {
someMethod(i);
}
so I can execute someMethod() concurrently for different i.
How do I use multithreading to accomplish this?
Please help! Thanks!
It is recommended to use the great ExecutorService code for situations like this. What you do is submit all of your tasks (0 to 499) into the thread pool and they will be run concurrently by the 5 threads in the pool.
Something like the following:
// create a thread pool with 5 workers
ExecutorService threadPool = Executors.newFixedThreadPool(5);
// submit all of your jobs here
for (int i = 0; i < 500; i++) {
threadPool.submit(new MyJob(i));
}
// once we have submitted all jobs to the thread pool, it should be shutdown
threadPool.shutdown();
// if you need to wait for the pool you can do
threadPool.awaitTerminatation(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
private static class MyJob implements Runnable {
private int i;
public MyJob(int i) {
this.i = i;
}
public void run() {
// do the thread stuff here
}
}
Related
im doing a few concurrency experiments in java.
I have this prime calculation method, which is just for mimicking a semi-expensive operation:
static boolean isprime(int n){
if (n == 1)
return false;
boolean flag = false;
for (int i = 2; i <= n / 2; ++i) {
if (n % i == 0) {
flag = true;
break;
}
}
return ! flag;
}
And then I have this main method, which simply calculates all prime number from 0 to N, and stores results in a array of booleans:
public class Main {
public static void main(String[] args) {
final int N = 100_000;
int T = 1;
boolean[] bool = new boolean[N];
ExecutorService es = Executors.newFixedThreadPool(T);
final int partition = N / T;
long start = System.nanoTime();
for (int j = 0; j < N; j++ ){
boolean res = isprime(j);
bool[j] = res;
}
System.out.println(System.nanoTime()-start);
}
This gives me results like: 893888901 n/s 848995600 n/s
And i also have this drivercode, where I use a executorservice where I use one thread to do the same:
public class Main {
public static void main(String[] args) {
final int N = 100_000;
int T = 1;
boolean[] bool = new boolean[N];
ExecutorService es = Executors.newFixedThreadPool(T);
final int partition = N / T;
long start = System.nanoTime();
for (int i = 0; i < T; i++ ){
final int current = i;
es.execute(new Runnable() {
#Override
public void run() {
for (int j = current*partition; j < current*partition+partition; j++ ){
boolean res = isprime(j);
bool[j] = res;
}
}
});
}
es.shutdown();
try {
es.awaitTermination(1, TimeUnit.MILLISECONDS);
} catch (Exception e){
System.out.println("what?");
}
System.out.println(System.nanoTime()-start);
}
this gives results like: 9523201 n/s , 15485300 n/s.
Now the second example is, as you can read, much faster than the first. I can't really understand why that is? should'nt the exercutorservice thread (with 1 thread) be slower, since it's basically doing the work sequentially + overhead from "awaking" the thread, compared to the main thread?
I was expecting the executorservice to be faster when I started adding multiple threads, but this is a little counterintuitive.
It's the timeout at the bottom of your code. If you set that higher you arrive at pretty similar execution times.
es.awaitTermination(1000, TimeUnit.MILLISECONDS);
The execution times you mention for the first main are much higher than the millisecond you allow the second main to wait for the threads to finish.
public class CounterThread extends Thread {
private static int counter = 0;
#Override
public void run() {
for (int i = 0; i < 1000; i++) {
counter++;
}
}
public static void main(String[] args) throws InterruptedException {
int num = 1000;
Thread[] threads = new Thread[num];
for (int i = 0; i < num; i++) {
threads[i] = new CounterThread();
threads[i].start();
// The comment section guarantees counter synchronization and why. .
--------------------------------------------------------------------
//threads[i].join();
}
for (int i = 0; i < num; i++) {
threads[i].join();
}
System.out.println(counter);
}
}
start() and immediate join() means you execute threads sequentially, not in parallel. Sequential modification of static variable does not introduce concurrent modification issues so code is safe. But it is not "multi-threaded" in common meaning of this term :)
I have to write program that finds the sum of a 2D array of int,
I coded every thing as I know and there is no syntax error but when I use someways to check my code the thread is not working at all but sometimes work some of thread not all of them
I put the number 1 to check the summation
and I put lock to make sure not two of thread in same method of summation only for make sure
and the n for see how much time it's join the add method
public class extend extends Thread {
int a, b;
private static int sum = 0;
static int n;
boolean lock;
int[][] arr;
public extend() {
arr = new int[45][45];
for (int i = 0; i < 45; i++) {
for (int j = 0; j < 45; j++)
arr[i][j] = 1;
}
n = 0;
lock = false;
}
public extend(int a, int b) {
arr = new int[45][45];
for (int i = 0; i < 45; i++) {
for (int j = 0; j < 45; j++)
arr[i][j] = 1;
}
n = 0;
lock = false;
this.a = a;
this.b = b;
}
public void run() {
add(a, b);
}
public void add(int st, int e) {
n++;
while (lock) ;
lock = true;
int sums = 0;
synchronized (this) {
for (int i = st; i < e; i++) {
for (int j = 0; j < 45; j++) {
sums += arr[i][j];
}
}
}
sum = sums;
lock = false;
}
public int getSum() {
return sum;
}
public static void main(String[] args) {
long ss = System.currentTimeMillis();
Thread t1 = new Thread(new extend(0, 9));
Thread t2 = new Thread(new extend(9, 18));
Thread t3 = new Thread(new extend(18, 27));
Thread t4 = new Thread(new extend(27, 36));
Thread t5 = new Thread(new extend(36, 45));
t1.start();
t2.start();
t3.start();
t4.start();
t5.start();
long se = System.currentTimeMillis();
System.out.println("The sum for 45*45 array is: " + sum);
System.out.println("time start;" + (se - ss));
System.out.print(n);
}
}
I'm sorry to say, but there's so much wrong with this code, it's hard to point at one problem:
You start your threads, but you don't wait for them to finish using .join()
Extending Thread when you actually meant implementing Runnable
Using busy waiting in your thread with while (true)
Using of static intfor counting
But, if there's only one thing you must fix, wait for your threads:
t1.join();
...
t5.join();
Your lockout of the sum variable may not even result in a speedup while taking into account the overhead of creating Threads, but your main problem is you are not adding sums to sum.
Change:
sum = sums;
to:
sum += sums;
This will make your code work for some of the time. It is not guaranteed to work and will sometimes output weird results like 1620 instead of 2025. You should learn more about how to properly handle multithreading, race conditions, and atomic locks.
I separate the array and sum its parts separately, at the end, adding everything to a single variable using join.
class code Main
int partArray = array.length / THREAD;
int first = 0;
AtomicInteger result = new AtomicInteger(0);
Thread[] thr = new Thread[THREAD];
for(i = 0; i < THREAD; ++i) {
thr[i] = new Thread(new ThreadSum(first, first + partArray, array, result));
thr[i].start();
first += partArray;
}
for(i = 0; i < THREAD; ++i) {
thr[i].join();
}
class code Thread
int first;
int end;
private int[] array;
private AtomicInteger result;
public ThreadSum(int first, int end, int[] array, AtomicInteger result) {
this.first = first;
this.end = end;
this.array = array;
this.result = result;
}
public synchronized void run() {
int sum = 0;
for(int i = first; i < end; ++i) {
sum += array[i];
}
result.getAndAdd(sum);
}
How do I implement this without using join?
Any help guys.
In the end, all the answers and comments from #Tudor and #JBNizet helped me solve the problem. I used CountDownLatch.
CountDownLatch countDownLatch = new CountDownLatch(THREAD);
for(i = 0; i < THREAD; ++i) {
thr[i] = new Thread(new ThreadSum(first, first + partArray, array, result,countDownLatch));
thr[i].start();
first += partArray;
}
countDownLatch.await();
class code Thread
CountDownLatch countDownLatch;
public ThreadSum(int first, int end, int[] array, AtomicInteger result, CountDownLatch countDownLatch) {
this.first = first;
this.end = end;
this.array = array;
this.result = result;
this.countDownLatch = countDownLatch;
}
#Override
public void run() {
int sum = 0;
System.out.println(currentThread().getName());
for(int i = first; i < end; ++i) {
sum += array[i];
}
countDownLatch.countDown();
result.getAndAdd(sum);
}
Here is version of the code using a thread pool that technically fulfills the requirement of not using join():
int partArray = array.length / THREAD;
int first = 0;
AtomicInteger result = new AtomicInteger(0);
ExecutorService threadPool = Executors.newCachedThreadPool();
for(i = 0; i < THREAD; ++i) {
threadPool.execute(new ThreadSum(first, first + partArray, array, result));
first += partArray;
}
threadPool.shutdown();
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
There are a couple of things not ok here:
Using join() is required because otherwise there is no precise point in the program execution where you can safely retrieve the computed sum and know that the parallel summation is finished.
synchronized on the run method is not required because individual array chunks can be summed up in parallel and you are already using AtomicInteger for synchronization.
I am experimenting with techniques for ensuring the visibility of side effects accomplished by concurrent tasks executed using the Java Executor framework.
As a simple scenario, considere an hypothetic problem of matrix multiplication.
Let's say that the matrices to multiply could be considerably large (e.g., few thousands rows and columns) and that to speed up the multiplication of such matrices I implement a concurrent algorithm where the calculation of each cell in the result matrix is considered as an independent (i.e., parallelizable) task.
To simplify a bit, let's ignore that for small input matrices this parallelization may be not such a good idea.
So considere below the first version of my program:
public class MatrixMultiplier {
private final int[][] m;
private final int[][] n;
private volatile int[][] result; //the (lazily computed) result of the matrix multiplication
private final int numberOfMRows; //number of rows in M
private final int numberOfNColumns; //number of columns in N
private final int commonMatrixDimension; //number of columns in M and rows in N
public MatrixMultiplier(int[][] m, int[][] n) {
if(m[0].length != n.length)
throw new IllegalArgumentException("Uncompatible arguments: " + Arrays.toString(m) + " and " + Arrays.toString(n));
this.m = m;
this.n = n;
this.numberOfMRows = m.length;
this.numberOfNColumns = n[0].length;
this.commonMatrixDimension = n.length;
}
public synchronized int[][] multiply() {
if (result == null) {
result = new int[numberOfMRows][numberOfNColumns];
ExecutorService executor = createExecutor();
Collection<Callable<Void>> tasks = new ArrayList<>();
for (int i = 0; i < numberOfMRows; i++) {
final int finalI = i;
for (int j = 0; j < numberOfNColumns; j++) {
final int finalJ = j;
tasks.add(new Callable<Void>() {
#Override
public Void call() throws Exception {
calculateCell(finalI, finalJ);
return null;
}
});
}
}
try {
executor.invokeAll(tasks);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
executor.shutdownNow();
}
}
return result;
}
private ExecutorService createExecutor() {
final int availableProcessors = Runtime.getRuntime().availableProcessors();
final int processorsBound = availableProcessors + 1;
final int maxConcurrency = numberOfMRows * numberOfNColumns;
final int threadPoolSize = maxConcurrency < processorsBound ? maxConcurrency : processorsBound;
return Executors.newFixedThreadPool(threadPoolSize);
}
private void calculateCell(int mRow, int nColumn) {
int sum = 0;
for (int k = 0; k < commonMatrixDimension; k++) {
sum += m[mRow][k] * n[k][nColumn];
}
result[mRow][nColumn] = sum;
}
}
As far as I understand there is a problem with this implementation: some modifications to the result matrix by the executed tasks may not be necessarily visible to the thread invoking multiply().
Assuming the previous is correct, consider the alternative implementation of multiply() relying on explicit locks (the new lock related code is commented with //<LRC>):
public synchronized int[][] multiply() {
if (result == null) {
result = new int[numberOfMRows][numberOfNColumns];
final Lock[][] locks = new Lock[numberOfMRows][numberOfNColumns]; //<LRC>
for (int i = 0; i < numberOfMRows; i++) { //<LRC>
for (int j = 0; j < numberOfNColumns; j++) { //<LRC>
locks[i][j] = new ReentrantLock(); //<LRC>
} //<LRC>
} //<LRC>
ExecutorService executor = createExecutor();
Collection<Callable<Void>> tasks = new ArrayList<>();
for (int i = 0; i < numberOfMRows; i++) {
final int finalI = i;
for (int j = 0; j < numberOfNColumns; j++) {
final int finalJ = j;
tasks.add(new Callable<Void>() {
#Override
public Void call() throws Exception {
try { //<LRC>
locks[finalI][finalJ].lock(); //<LRC>
calculateCell(finalI, finalJ);
} finally { //<LRC>
locks[finalI][finalJ].unlock(); //<LRC>
} //<LRC>
return null;
}
});
}
}
try {
executor.invokeAll(tasks);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
executor.shutdownNow();
}
for (int i = 0; i < numberOfMRows; i++) { //<LRC>
for (int j = 0; j < numberOfNColumns; j++) { //<LRC>
locks[i][j].lock(); //<LRC>
locks[i][j].unlock(); //<LRC>
} //<LRC>
} //<LRC>
}
return result;
}
The usage of explicit locks above has as unique goal to ensure the publication of the changes to the invoking thread, since there is no any possibility of contention.
My main question is if this is a valid solution to the problem of publishing side effects in my scenario.
As a secondary question: is there a more efficient/elegant way to solve this problem ? Please note that I am not looking for alternative algorithm implementations (e.g., the Strassen's algorithm) for parallelizing matrix multiplication, since mine is just a simple case study. I am rather interested on alternatives for ensuring the visibility of changes in an algorithm like the one presented here.
UPDATE
I think the alternative implementation below improves on the previous implementation. It makes use of one single internal lock without affecting much the concurrency:
public class MatrixMultiplier {
...
private final Object internalLock = new Object();
public synchronized int[][] multiply() {
if (result == null) {
result = new int[numberOfMRows][numberOfNColumns];
ExecutorService executor = createExecutor();
Collection<Callable<Void>> tasks = new ArrayList<>();
for (int i = 0; i < numberOfMRows; i++) {
final int finalI = i;
for (int j = 0; j < numberOfNColumns; j++) {
final int finalJ = j;
tasks.add(new Callable<Void>() {
#Override
public Void call() throws Exception {
calculateCell(finalI, finalJ);
synchronized (internalLock){}
return null;
}
});
}
}
try {
executor.invokeAll(tasks);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
executor.shutdownNow();
}
}
synchronized (internalLock){}
return result;
}
...
}
This alternative is just more efficient but both it and the previous implementation which makes use of many locks look correct to me. Are all my observations correct ? Is there a more efficient/elegant way to deal with the synchronization problem in my scenario?
Declaring result as volatile only ensures that changing the reference of result (i.e. result = ...; operations) is visible to everyone.
The most obvious way to resolve this is to eliminate the side effect. In this case this is easy: just make calculateCell() and the Callable invoking it return the value and let the main thread write the values into the array.
You could of course do explicit locking, like you did in your second example but it seems an overkill to use nxm locks when you could use just one lock. Of course one lock would kill the parallelism in your example, so once again the solution is to make calculateCell() return the value and only lock for the duration of writing the result in the result array.
Or indeed you can use ForkJoin and forget about the whole thing because it will do it for you.