We Keep Coding

Why the speedup of parallel programming using Executor larger than cores?

I am writing a program dealing with matrix parallel programming using Executorservice framework. And I set the fixedpoolsize to 4, however what surprises me is that when the matrix dimension is set to 5000, the speedup of using multithreading against serial execution is greater than 4 (which is also my CPU cores). And I have checked that my CPU does not support for hyperthreading.
Actually I use the Callable and Future container since my multithreading task requires the result to be returned.
// Part of code for parallel programming
double[][] x = new double[N][N];
List<Future<double[]>> futureList = new ArrayList<>();
for (int k=0;k<N;k++)
{
Future<double[]>temp=service.submit(new Thread.Task(N,k,matrix,vector));
futureList.add(temp);
}
for (int j = 0; j < N; j++) {
x[j]=futureList.get(j).get();
}
public double[] call() throws Exception {
for (int i = N - 1; i >= 0; i--)
{
double sum = 0;
for (int j = i + 1; j < N; j++)
{
sum += matrix[i][j] * x[j];
}
x[i] = (vector[i][k] - sum) / matrix[i][i];
}
return x;
}
// Part of code for Serial programming
double[][] x = new double[N][N];
for (int k=0;k<N;k++)
{
for (int i = N - 1; i >= 0; i--)
{
double sum = 0;
for (int j = i + 1; j < N; j++)
{
sum += matrix[i][j] * x[j][k];
}
x[i][k] = (vector[i][k] - sum) / matrix[i][i];
}
}
In short, I just take the inside loop away to let it be run by the thread and leave the outside loop unchanged.
But how can the speedup be like this?
Since from my previous concept it is that the maximum speedup can only be 4. And I have checked that the task is just done by 4 threads actually.

Threads can be utilized on the same cpu. You do not need a multi core processor to execute multi threaded applications.
Think of a thread as small process, that gets created by the parent program and destroyed once its done. Even single cpu computers can run multiple threads at once.
ExecutorService schedules threads to be executed and will run as many parallel threads as there are resources available including the cores.
Here is the docs on fixedThreadPool
public static ExecutorService newFixedThreadPool(int nThreads)
Creates a thread pool that reuses a fixed number of threads operating
off a shared unbounded queue. At any point, at most nThreads threads
will be active processing tasks. If additional tasks are submitted
when all threads are active, they will wait in the queue until a
thread is available. If any thread terminates due to a failure during
execution prior to shutdown, a new one will take its place if needed
to execute subsequent tasks. The threads in the pool will exist until
it is explicitly shutdown
You can also try workStealingPool
public static ExecutorService newWorkStealingPool()
Creates a
work-stealing thread pool using all available processors as its target
parallelism level.

This could be an effect of CPU cache affinity. If each core works on a different part of a problem it may achieve greater efficiency in cache use. Because RAM is up to 10's or more times slower than cache this can make a HUGE difference.

Threads not finishing when incremented by 1 more Java

The code below works fine, however I want value to reach the end of the array, as at the moment it start from 1 then goes to the end of the array.
Booth[] boot = new Booth[numberOfBooths];
for (int j = 1; j < boot.length; j++) {
boot[j] = new Booth(j, buff);
boot[j].start();
}
for (int j =1 ; j < boot.length; j++) {
try {
boot[j].join();
} catch (InterruptedException ex) {
System.out.println(ex);
}
}
I altered the code so the loops start from 0.. such:
for (int j = 0; j < boot.length; j++) {
boot[j] = new Booth(j, buff);
boot[j].start();
}
for (int j =0 ; j < boot.length; j++) {
try {
boot[j].join();
} catch (InterruptedException ex) {
System.out.println(ex);
}
But after debugging as soon as it gets to the join the program stops. I read about Deadlock and maybe that was the reason for this, could there be a way around it, is there a general solution to this problem?
Edit: Sorry I haven't been so clear. The code works either way, however when I run it a second time ( I have my program in a while loop ) it doesn't do the join

It sounds as if the thread pointed to by boot[1] completes, but the thread pointed to by boot[0] does not.
Therefore something about new Booth(0,buff) creates an object for which run() doesn't terminate.
Try unit testing Booth without using threads, and just run:
Booth b = new Booth(0,buff); // initialise buff first, of course
b.run();
... and see whether this returns. If not, work out why, perhaps by stepping through it with a debugger.
The other thing to look out for is deadlock around the shared buff object. Unless there is some locking around buff, you can certainly expect issues. However that seems less likely if starting your index at 1 solves the problem.
If the whole thing works the first time around, but not on a second attempt, then you should consider the state of buff. Perhaps new Booth(0,buff).run() is non-terminating when supplied a certain state of buff.
Aside: It's cleaner to not do class Booth extends Thread but instead class Booth implements Runnable.
Then instead of Booth.start() use
Thread t = new Thread(booth);
t.start();
That way you're not polluting the Booth code with the knowledge that it's going to be a thread -- it's better encapsulation.

All java threads are running on a single core, ultimately taking too much time to execute

This is my homework problem:
I have to do matrix multiplication. My code should create a thread to each element in the resultant matrix. i.e., if resultant matrix is mXn size then there should be m*n threads.
(http://iit.qau.edu.pk/books/OS_8th_Edition.pdf page 179)
Yes I finished it by my own. Which is executing fine with all my test cases. Unfortunately, I end up getting 70% credit :(
This is my code and test cases.
Matrix Multiplication.zip
When I met my professor regarding my marks. He told me that my code taking too long to execute larger size matrix.
I argued with him that it is a expected behavior as it is obvious that large size data takes more time. However, he disagree with me.
I attached my code and test cases . My code is taking 3 hours. As per my professor it should only take 5 min to execute.
I tried to figured out for last couple of days but I couldn't find exact solution :(
outline of my code
ExecutorService executor = Executors
.newFixedThreadPool(threadCount); // creating thread pool
// with fixed threads
int mRRowLen = matrix1.length; // m result rows length
int mRColLen = matrix2[0].length; // m result columns length
mResult = new long[mRRowLen][mRColLen];
for (int i = 0; i < mRRowLen; i++) { // rows from m1
for (int j = 0; j < mRColLen; j++) { // columns from m2
Runnable r = new MatrixMultipliactionThread(matrix1ColLen, i, j, matrix1,
matrix2);
executor.execute(r);
}
}
executor.shutdown();
while (!executor.isTerminated()) {
// wait untill it get shutdown
}
Run method :
public void run() {
for (int k = 0; k < m1ColLength; k++) { // columns from m1
Matrix.mResult[i][j] += matrix1[i][k] * matrix2[k][j];
}
Thanks in Advance

Ok, I downloaded your zip and ran the program. Your problem isn't in the matrix multiplication at all. The advice in the comments is still valid about reducing the number of threads, however as it stands the multiplication happens very quickly.
The actual problem is in your writeToAFile method - all the single-threaded CPU utilization you are seeing is actually happening in there, after the multiplication is already complete.
The way you're appending your strings:
fileOutputString = fileOutputString + resultMatrix[i][j]
creates thousands of new String objects which then immediately become garbage; this is very inefficient. You should be using a StringBuilder instead, something like this:
StringBuilder sb=new StringBuilder();
for (int i = 0; i < resultMatrix.length; i++) {
for (int j = 0; j < resultMatrix[i].length; j++) {
sb.append(resultMatrix[i][j]);
if (j != resultMatrix[i].length - 1) sb.append(",");
}
sb.append("\n");
}
String fileOutputString=sb.toString();
That code executes in a fraction of a second.

How to correctly shutdown all threads?

I have 8 threads which each execute loops similar to this in their run method:
for (int i = 0; i < someVar; i++) {
for (int j = 0; j < someOtherVar; j++) {
if (Thread.currentThread().isInterrupted()) {
return;
}
// another loop and some calculations here
}
}
I start the Threads like this:
executor = Executors.newFixedThreadPool(threads);
// generate the threads and run them
for (int i = 0; i < threads; i++) {
MyClass cls = new MyClass(i);
executor.execute(cls);
}
Now I need to be able to pretty much instantly kill my threads, that's why I added the if statement above and use at another point executor.shutdownNow(). But it just isn't working fast enough. Some calculations might take a few seconds before they finish and the next iteration starts.
Is there any other reliable way to pretty much immediately shutdown threads?

You need an interrupt operation that calls into the thread and stops the current execution. Here's some documentation: http://docs.oracle.com/javase/tutorial/essential/concurrency/interrupt.html
An interrupt is an indication to a thread that it should stop what it is doing and do something else. It's up to the programmer to decide exactly how a thread responds to an interrupt, but it is very common for the thread to terminate.

Code inside thread slower than outside thread..?

I'm trying to alter some code so it can work with multithreading. I stumbled upon a performance loss when putting a Runnable around some code.
For clarification: The original code, let's call it
//doSomething
got a Runnable around it like this:
Runnable r = new Runnable()
{
public void run()
{
//doSomething
}
}
Then I submit the runnable to a ChachedThreadPool ExecutorService. This is my first step towards multithreading this code, to see if the code runs as fast with one thread as the original code.
However, this is not the case. Where //doSomething executes in about 2 seconds, the Runnable executes in about 2.5 seconds. I need to mention that some other code, say, //doSomethingElse, inside a Runnable had no performance loss compared to the original //doSomethingElse.
My guess is that //doSomething has some operations that are not as fast when working in a Thread, but I don't know what it could be or what, in that aspect is the difference with //doSomethingElse.
Could it be the use of final int[]/float[] arrays that makes a Runnable so much slower? The //doSomethingElse code also used some finals, but //doSomething uses more. This is the only thing I could think of.
Unfortunately, the //doSomething code is quite long and out-of-context, but I will post it here anyway. For those who know the Mean Shift segmentation algorithm, this a part of the code where the mean shift vector is being calculated for each pixel. The for-loop
for(int i=0; i<L; i++)
runs through each pixel.
timer.start(); // this is where I start the timer
// Initialize mode table used for basin of attraction
char[] modeTable = new char [L]; // (L is a class property and is about 100,000)
Arrays.fill(modeTable, (char)0);
int[] pointList = new int [L];
// Allcocate memory for yk (current vector)
double[] yk = new double [lN]; // (lN is a final int, defined earlier)
// Allocate memory for Mh (mean shift vector)
double[] Mh = new double [lN];
int idxs2 = 0; int idxd2 = 0;
for (int i = 0; i < L; i++) {
// if a mode was already assigned to this data point
// then skip this point, otherwise proceed to
// find its mode by applying mean shift...
if (modeTable[i] == 1) {
continue;
}
// initialize point list...
int pointCount = 0;
// Assign window center (window centers are
// initialized by createLattice to be the point
// data[i])
idxs2 = i*lN;
for (int j=0; j<lN; j++)
yk[j] = sdata[idxs2+j]; // (sdata is an earlier defined final float[] of about 100,000 items)
// Calculate the mean shift vector using the lattice
/*****************************************************/
// Initialize mean shift vector
for (int j = 0; j < lN; j++) {
Mh[j] = 0;
}
double wsuml = 0;
double weight;
// find bucket of yk
int cBucket1 = (int) yk[0] + 1;
int cBucket2 = (int) yk[1] + 1;
int cBucket3 = (int) (yk[2] - sMinsFinal) + 1;
int cBucket = cBucket1 + nBuck1*(cBucket2 + nBuck2*cBucket3);
for (int j=0; j<27; j++) {
idxd2 = buckets[cBucket+bucNeigh[j]]; // (buckets is a final int[] of about 75,000 items)
// list parse, crt point is cHeadList
while (idxd2>=0) {
idxs2 = lN*idxd2;
// determine if inside search window
double el = sdata[idxs2+0]-yk[0];
double diff = el*el;
el = sdata[idxs2+1]-yk[1];
diff += el*el;
//...
idxd2 = slist[idxd2]; // (slist is a final int[] of about 100,000 items)
}
}
//...
}
timer.end(); // this is where I stop the timer.
There is more code, but the last while loop was where I first noticed the difference in performance.
Could anyone think of a reason why this code runs slower inside a Runnable than original?
Thanks.
Edit: The measured time is inside the code, so excluding startup of the thread.

All code always runs "inside a thread".
The slowdown you see is most likely caused by the overhead that multithreading adds. Try parallelizing different parts of your code - the tasks should neither be too large, nor too small. For example, you'd probably be better off running each of the outer loops as a separate task, rather than the innermost loops.
There is no single correct way to split up tasks, though, it all depends on how the data looks and what the target machine looks like (2 cores, 8 cores, 512 cores?).
Edit: What happens if you run the test repeatedly? E.g., if you do it like this:
Executor executor = ...;
for (int i = 0; i < 10; i++) {
final int lap = i;
Runnable r = new Runnable() {
public void run() {
long start = System.currentTimeMillis();
//doSomething
long duration = System.currentTimeMillis() - start;
System.out.printf("Lap %d: %d ms%n", lap, duration);
}
};
executor.execute(r);
}
Do you notice any difference in the results?

I personally do not see any reason for this. Any program has at least one thread. All threads are equal. All threads are created by default with medium priority (5). So, the code should show the same performance in both the main application thread and other thread that you open.
Are you sure you are measuring the time of "do something" and not the overall time that your program runs? I believe that you are measuring the time of operation together with the time that is required to create and start the thread.

When you create a new thread you always have an overhead. If you have a small piece of code, you may experience performance loss.
Once you have more code (bigger tasks) you make get a performance improvement by your parallelization (the code on the thread will not necessarily run faster, but you are doing two thing at once).
Just a detail: this decision of how big small can a task be so parallelizing it is still worth is a known topic in parallel computation :)

You haven't explained exactly how you are measuring the time taken. Clearly there are thread start-up costs but I infer that you are using some mechanism that ensures that these costs don't distort your picture.
Generally speaking when measuring performance it's easy to get mislead when measuring small pieces of work. I would be looking to get a run of at least 1,000 times longer, putting the whole thing in a loop or whatever.
Here the one different between the "No Thread" and "Threaded" cases is actually that you have gone from having one Thread (as has been pointed out you always have a thread) and two threads so now the JVM has to mediate between two threads. For this kind of work I can't see why that should make a difference, but it is a difference.
I would want to be using a good profiling tool to really dig into this.