We Keep Coding

How to make parallel programming with input values in java

I implement a recommendation algorithm in Java program.
However, I have serious problems. The dataset is too large and it's computation is too slow. So, I need to do parallel programming in Java.
For example,
for (int i=0; i < 10000000 ; i++) { ~~~ }
I want to split this sentences such as
process 1: for (int i=0; i < 10000 ; i++)
process 2: for (int i=10001; i < 20000 ; i++)
process 3: for (int i=20001; i < 30000 ; i++)
...
I know similar methods in Python. How to do parallel programming in Java?

Hope this will help you.
public class MyRunnable implements Runnable {
private final long countUntil;
MyRunnable(long countUntil) {
this.countUntil = countUntil;
}
#Override
public void run() {
long sum = 0;
for (long i = 1; i < countUntil; i++) {
sum += i;
}
System.out.println(sum);
}
}
public class Main {
public static void main(String[] args) {
// We will store the threads so that we can check if they are done
List<Thread> threads = new ArrayList<Thread>();
// We will create 500 threads
for (int i = 0; i < 500; i++) {
Runnable task = new MyRunnable(10000000L + i);
Thread worker = new Thread(task);
// We can set the name of the thread
worker.setName(String.valueOf(i));
// Start the thread, never call method run() direct
worker.start();
// Remember the thread for later usage
threads.add(worker);
}
int running = 0;
do {
running = 0;
for (Thread thread : threads) {
if (thread.isAlive()) {
running++;
}
}
System.out.println("We have " + running + " running threads. ");
} while (running > 0);
}
}
i got it from here

Round Robin Scheduler

I need to implement a "round-robin" scheduler with a job class that I cannot modify. Round-robin scheduler should process the job that has been waiting the longest first, then reset timer to zero. If two jobs have same wait time, lower id is processed first. The job class only gives three values (job id, remaining duration, and priority(which is not needed for this). each job has a start time, so only a couple of jobs may be available during first cycle, few more next cycle, etc. Since the "job array" I am calling is different every time I call it, I'm not sure how to store the wait times.
This is the job class:
public class Jobs{
private int[] stas = new int[0];
private int[] durs = new int[0];
private int[] lefs = new int[0];
private int[] pris = new int[0];
private int[] fins = new int[0];
private int clock;
public Jobs()
{
this("joblist.csv");
}
public Jobs(String filename)
{
BufferedReader fp = null;
String line = "";
String[] b = null;
int[] tmp;
try
{
fp = new BufferedReader(new FileReader(filename));
while((line = fp.readLine()) != null)
{
b = line.split(",");
if(b.length == 3)
{
try
{
int sta = Integer.parseInt(b[0]);
//System.out.println("sta: " + b[0]);
int dur = Integer.parseInt(b[1]);
//System.out.println("dur: " + b[1]);
int pri = Integer.parseInt(b[2]);
//System.out.println("pri: " + b[2]);
stas = app(stas, sta);
//System.out.println("stas: " + Arrays.toString(stas));
durs = app(durs, dur);
//System.out.println("durs: " + Arrays.toString(durs));
lefs = app(lefs, dur);
//System.out.println("lefs: " + Arrays.toString(lefs));
pris = app(pris, pri);
//System.out.println("pris: " + Arrays.toString(pris));
fins = app(fins, -1);
//System.out.println("fins: " + Arrays.toString(fins));
}
catch(NumberFormatException e) {}
}
}
fp.close();
}
catch(FileNotFoundException e) { e.printStackTrace(); }
catch(IOException e) { e.printStackTrace(); }
clock = 0;
}
public boolean done()
{
boolean done = true;
for(int i=0; done && i<lefs.length; i++)
if(lefs[i]>0) done=false;
return done;
}
public int getClock() { return clock; }
public int[][] getJobs()
{
int count = 0;
for(int i=0; i<stas.length; i++)
if(stas[i]<=clock && lefs[i]>0)
count++;
int[][] jobs = new int[count][3];
count = 0;
for(int i=0; i<stas.length; i++)
if(stas[i]<=clock && lefs[i]>0)
{
jobs[count] = new int[]{i, lefs[i], pris[i]};
count++;
}
return jobs;
}
public int cycle() { return cycle(-1); }
public int cycle(int j)
{
if(j>=0 && j<lefs.length && clock>=stas[j] && lefs[j]>0)
{
lefs[j]--;
if(lefs[j] == 0) fins[j] = clock+1;
}
clock++;
return clock;
}
private int[] app(int[] a, int b)
{
int[] tmp = new int[a.length+1];
for(int i=0; i<a.length; i++) tmp[i] = a[i];
tmp[a.length] = b;
return tmp;
}
public String report()
{
String r = "JOB,PRIORITY,START,DURATION,FINISH,DELAY,PRI*DELAY\n";
float dn=0;
float pdn=0;
for(int i=0; i<stas.length; i++)
{
if(fins[i]>=0)
{
int delay = ((fins[i]-stas[i])-durs[i]);
r+= ""+i+","+pris[i]+","+stas[i]+","+durs[i]+","+fins[i]+","+delay+","+(pris[i]*delay)+"\n";
dn+= delay;
pdn+= pris[i]*delay;
}
else
{
int delay = ((clock*10-stas[i])-durs[i]);
r+= ""+i+","+pris[i]+","+stas[i]+","+durs[i]+","+fins[i]+","+delay+","+(pris[i]*delay)+"\n";
dn+= delay;
pdn+= pris[i]*delay;
}
}
if(stas.length>0)
{
r+= "Avg,,,,,"+(dn/stas.length)+","+pdn/stas.length+"\n";
}
return r;
}
public String toString()
{
String r = "There are "+stas.length+" jobs:\n";
for(int i=0; i<stas.length; i++)
{
r+= " JOB "+i+": START="+stas[i]+" DURATION="+durs[i]+" DURATION_LEFT="+lefs[i]+" PRIORITY="+pris[i]+"\n";
}
return r;
}
I don't need full code, just an idea of how to store wait times and cycle the correct job.

While a array based solution 'may' work, I would advocate a more object oriented approach. Create 'Job' class with the desire attributes (id, start_time, wait etc). Using the csv file, create Job objects and hold them in a list. Write a comparator to sort this jobs-list (in this case based on job wait/age would be the factor).
The job executor then has to do the following:
while(jobs exist) {
iterate on the list {
if job is executable // start_time > current sys_time
consume cycles/job for executable jobs
mark completed jobs (optional)
}
remove the completed jobs
}

//\ This loop will add +1 to each job
for(int i = 0; i < jobs.length; i++)
{
waitTime[jobs[i][0]] += 1;
}
int longestWait = 0;//\ This holds value for greatest wait time
int nextJob = 0; //\ This holds value for index of job with greatest wait time
//\ this loop will check for the greatest wait time and and set variables accordingly
for(int i = 0; i < waitTime.length; i++)
{
if(waitTime[i] > longestWait)
{
longestWait = waitTime[i];
nextJob = i;
}
}
//\ this cycles the job with the highest wait time
jobsource.cycle(nextJob);
//\ this resets the wait time for processed job
waitTime[nextJob] = 0;

Java instantiating a class in a loop

Currently my code instantiates a class for a number of cores one after the other but I would like to create a loop that instantiates the class detectors() for an arbitrary number of cores, running concurrently.
int processors = Runtime.getRuntime().availableProcessors(); // finds the number of available threads
detectors.getStartingConditions();
long startTime = System.currentTimeMillis();
detectors core1= new detectors();
detectors core2= new detectors();
detectors core3= new detectors();
//etc
core1.start();
core2.start();
core3.start();
//etc
try
{ // wait for completion of all thread and then sum
core1.join();
core2.join();
core3.join();
//etc
}
catch(InterruptedException IntExp) {}
long endTime = System.currentTimeMillis();
System.out.println("That took " + (endTime - startTime) + " milliseconds");
My attempt at a solution:
I created a array of objects as follows but the processor cores run one after the other as opposed to concurrently.
edit: the cores now run concurrently.
int processors = Runtime.getRuntime().availableProcessors(); // finds the number of available threads
detectors[] theCores = new detectors[processors];
detectors.getStartingConditions();
long startTime = System.currentTimeMillis();
for(int i = 0; i <= processors-1; i++){
theCores[i] = new detectors();
theCores[i].start();
}
for(int i = 0; i <= processors-1; i++){
try{
theCores[i].join();}
catch(InterruptedException IntExp) {}
}
long endTime = System.currentTimeMillis();
System.out.println("That took " + (endTime - startTime) + " milliseconds");

Your code creates threads and joins it before creating the next thread. That causes the sequential execution. You have to use two loops instead. The first loop creates all threads, whereas the second loop joins all threads.
for (int i = 0; i < processors; ++i) {
theCores[i] = new detectors();
theCores[i].start();
}
for (int i = 0; i < processors; ++i) {
try {
theCores[i].join();
} catch (InterruptedException ie) {
RuntimeException re = new RuntimeException("unsupported interruption", ie);
for (++i; i < processors; ++i) {
try {
theCores[i].join();
} catch (InterruptedException e) {
re.addSuppressed(e);
}
}
throw re;
}
}

How to remove manually created pauses in Main-thread?

Problem description:
We have a given matrix randomly filled with digits and have to create separate threads for each row of the matrix that count how many times the digits encounter in that row.
Without these sleeps in the main thread, it's not working correctly..
Here's my solution.
Also it's following here:
public class TestingMatrixThreads {
public static void main(String[] arr) throws InterruptedException {
int[][] a = new int[67][6];
// class.Count works with class.Matrix, that's why I've made it this way
Matrix m = new Matrix(a);
m.start();
Thread.sleep(1000); // Here comes the BIG question -> how to avoid these
// manually created pauses
Count c;
Thread t;
// Creating new threads for each row of the matrix
for (int i = 0; i < Matrix.matr.length; i++) {
c = new Count(i);
t = new Thread(c);
t.start();
}
//Again - the same question
System.out.println("Main - Sleep!");
Thread.sleep(50);
System.out.println("\t\t\t\t\tMain - Alive!");
int sum = 0;
for (int i = 0; i < Count.encounters.length; i++) {
System.out.println(i + "->" + Count.encounters[i]);
sum += Count.encounters[i];
}
System.out.println("Total numbers of digits: " + sum);
}
}
class Count implements Runnable {
int row;
public static int[] encounters = new int[10]; // here I store the number of each digit's(array's index) encounters
public Count(int row) {
this.row = row;
}
public synchronized static void increment(int number) {
encounters[number]++;
}
#Override
public void run() {
System.out.println(Thread.currentThread().getName() + ", searching in row " + row + " STARTED");
for (int col = 0; col < Matrix.matr[0].length; col++) {
increment(Matrix.matr[row][col]);
}
try {
Thread.sleep(1); // If it's missing threads are starting and stopping consequently
} catch (InterruptedException e) {
}
System.out.println(Thread.currentThread().getName() + " stopped!");
}
}
class Matrix extends Thread {
static int[][] matr;
public Matrix(int[][] matr) {
Matrix.matr = matr;
}
#Override
public void run() {
//print();
fill();
System.out.println("matrix filled");
print();
}
public static void fill() {
for (int i = 0; i < matr.length; i++) {
for (int j = 0; j < matr[0].length; j++) {
matr[i][j] = (int) (Math.random() * 10);
}
}
}
public static void print() {
for (int i = 0; i < matr.length; i++) {
for (int j = 0; j < matr[0].length; j++) {
System.out.print(matr[i][j] + " ");
}
System.out.println();
}
}
}
P.S. I'm sorry if this question is too stupid for you to answer, but I'm a newbie in Java programming, as well as it's my very first post in stackoverflow, so please excuse me for the bad formatting, too :)
Thank you in advance!

Change the Thread.sleep by m.join()
Doing this will make the main thread wait for the other to complete its work and then it will continu its execution.
Cheers

To answer your main question:
Thread.join();
For example:
public static void main(String[] args) throws Exception {
final Thread t = new Thread(new Runnable() {
#Override
public void run() {
System.out.println("Do stuff");
}
});
t.start();
t.join();
}
The start call, as you know, kicks off the other Thread and runs the Runnable. The join call then waits for that started thread to finish.
A more advanced way to deal with multiple threads is with an ExecutorService. This detaches the threads themselves from the tasks they do. You can have a pool of n threads and m > n tasks.
Example:
public static void main(String[] args) throws Exception {
final class PrintMe implements Callable<Void> {
final String toPrint;
public PrintMe(final String toPrint) {
this.toPrint = toPrint;
}
#Override
public Void call() throws Exception {
System.out.println(toPrint);
return null;
}
}
final List<Callable<Void>> callables = new LinkedList<>();
for (int i = 0; i < 10; ++i) {
callables.add(new PrintMe("I am " + i));
}
final ExecutorService es = Executors.newFixedThreadPool(4);
es.invokeAll(callables);
es.shutdown();
es.awaitTermination(1, TimeUnit.DAYS);
}
Here we have 4 threads and 10 tasks.
If you go down this route you probably need to look into the Future API to so that you can check whether the tasks completed successfully. You can also return a value from the task; in your case a Callable<Integer> would seem to be appropriate so that you can return the result of your calculation from the call method and gather up the results from the Future.

As other Answers have stated, you can do this simply using join; e.g.
Matrix m = new Matrix(a);
m.start();
m.join();
However, I just want to note that if you do that, you are not going to get any parallelism from the Matrix thread. You would be better of doing this:
Matrix m = new Matrix(a);
m.run();
i.e. executing the run() method on the main thread. You might get some parallelism by passing m to each "counter" thread, and having them all join the Matrix thread ... but I doubt that it will be worthwhile.
Frankly, I'd be surprised if you get a worthwhile speedup for any of the multi-threading you are trying here:
If the matrix is small, the overheads of creating the threads will dominate.
If the matrix is large, you are liable to run into memory contention issues.
The initialization phase takes O(N^2) computations compared with the parallelized 2nd phase that has N threads doing O(N) computations. Even if you can get a decent speedup in the 2nd phase, the 1st phase is likely to dominate.

Need Help intrepretting the results of this program

So, I made a small program to test Multithreading in java and compare the time it takes to scale an array using a while loop and then creating multiple threads and running those threads. I'm unsure about then numbers I'm getting when the program finishes, so I was wondering if I made a boneheaded error at some point and messed something up to get very disparate numbers.
Code below:
import java.util.Scanner;
public class arrayScaling {
public static void main(String[] args) throws InterruptedException {
Scanner input = new Scanner(System.in);
System.out.println("Enter the amount of number you want the program to generate:");
int numOfNumbs = input.nextInt();
int [] arrayForNumbers = new int [numOfNumbs];
int [] newArrayForNumbers = new int [numOfNumbs];
for (int i = 0; i < arrayForNumbers.length; i++) {
arrayForNumbers[i] = (int) ((Math.random() * 25) + 1);
}
long startTime = System.nanoTime();
for (int i = 0; i < arrayForNumbers.length; i++) {
newArrayForNumbers[i] = newArrayForNumbers[i] * 3;
}
long endTime = System.nanoTime();
System.out.println();
long totalExecutionTime = endTime-startTime;
System.out.println("Time it takes execute scaling is " +
totalExecutionTime + " nanoseconds");
System.out.println();
int numOfNumLeftOver = numOfNumbs % 5;
int numOfNumDivided = numOfNumbs / 5;
int [] temp = null;
int [] temp2 = null;
int [] temp3 = null;
int [] temp4 = null;
int [] temp5 = null;
MyThread thread1 = new MyThread (numOfNumbs/5);
MyThread thread2 = new MyThread (numOfNumbs/5);
MyThread thread3 = new MyThread (numOfNumbs/5);
MyThread thread4 = new MyThread (numOfNumbs/5);
MyThread thread5;
if (numOfNumLeftOver != 0) {
numOfNumDivided = numOfNumDivided + numOfNumLeftOver;
thread5 = new MyThread (numOfNumDivided);
}
else {
thread5 = new MyThread (numOfNumbs/5);
}
int tempNum = 0;
for ( int i = 0; i < thread1.getArray().length; i ++) {
temp = thread1.getArray();
temp[tempNum] = arrayForNumbers[tempNum];
tempNum++;
}
for ( int i = 0; i < thread2.getArray().length; i ++) {
temp2 = thread2.getArray();
temp2[i] = arrayForNumbers[tempNum];
tempNum++;
}
for ( int i = 0; i < thread3.getArray().length; i ++) {
temp3 = thread3.getArray();
temp3[i] = arrayForNumbers[tempNum];
tempNum++;
}
for ( int i = 0; i < thread4.getArray().length; i ++) {
temp4 = thread4.getArray();
temp4[i] = arrayForNumbers[tempNum];
tempNum++;
}
for ( int i = 0; i < thread5.getArray().length; i ++) {
temp5 = thread5.getArray();
temp5[i] = arrayForNumbers[tempNum];
tempNum++;
}
thread1.setArray(temp);
thread2.setArray(temp2);
thread3.setArray(temp3);
thread4.setArray(temp4);
thread5.setArray(temp5);
long startTime2 = System.nanoTime();
thread1.start();
thread2.start();
thread3.start();
thread4.start();
thread5.start();
thread1.join();
thread2.join();
thread3.join();
thread4.join();
thread5.join();
long endTime2 = System.nanoTime();
long newTotalExecutionTime = endTime2 - startTime2;
System.out.println("Time it takes execute scaling w/ multiple threads is " +
newTotalExecutionTime + " nanoseconds");
if (newTotalExecutionTime < totalExecutionTime) {
System.out.println("Multithreading was more effective");
}
else if (totalExecutionTime < newTotalExecutionTime) {
System.out.println("The original algorithm was more effective");
}
else if (totalExecutionTime == newTotalExecutionTime) {
System.out.println("Both method worked at the same speed");
}
input.close();
}
}
public class MyThread extends Thread {
private int [] array;
private int [] scaleArray;
public MyThread(int size) {
array = new int [size];
scaleArray = new int [size];
}
public int[] getArray() {
return array;
}
public void setArray(int[] array) {
this.array = array;
}
public int[] getScaleArray() {
return scaleArray;
}
public void setScaleArray(int[] scaleArray) {
this.scaleArray = scaleArray;
}
public void run () {
for (int z = 0; z < array.length; z++){
scaleArray[z] = 3 * array[z];
}
}
}
And the output of this program is:
Enter the amount of number you want the program to generate:
16
Time it takes execute scaling is 893 nanoseconds
Time it takes execute scaling w/ multiple threads is 590345 nanoseconds
The original algorithm was more effective

Your results don't surprise me in the slightest. There's a lot of overhead to creating threads, starting them, waiting for them to finish and so on. Don't forget, 590345ns is still less than a millisecond; but most of that is to do with shuffling threads, not with multiplying the numbers.
If you want to see the threaded part of the program outperform the other part, try generating a whole lot more than 16 numbers.