We Keep Coding

Getting results back to the caller from worker threads

There are some answers on SO but I'm looking for a solution using my code for clear understanding.
I've created worker threads using Executor, and after performing work, the workers are to return results back to the caller (main)
Worker
public class Worker implements Runnable {
private final int num;
public Worker(int num) {
this.num = num;
}
#Override
public void run() {
System.out.println("Starting job: " + num);
try {
Thread.sleep(2000);
System.out.println("end job:" + num);
String result = "result " + num; // how to pass all the results back to the caller
} catch (Exception e) {
e.printStackTrace();
}
}
}
Worker Test
public class WorkerTest {
private static List<String> result = new ArrayList<>();
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
Runnable worker = new Worker(i);
executorService.execute(worker);
}
executorService.shutdown();
while(!executorService.isTerminated());
}
}

Here is your complete code
public class WorkerTest {
private static List result = new ArrayList<>();
public static void main(String[] args) throws Exception {
Future[] futures = new Future[10];
ExecutorService ex = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
Callable worker = new Worker(i);
futures[i] = ex.submit(worker);
}
for (int i = 0; i < 10; i++) {
String resultString = (String) futures[i].get();
System.out.println(resultString);
}
ex.shutdown();
while (!ex.isTerminated());
}
}
class Worker implements Callable<String> {
private final int num;
public Worker(int num) {
super();
this.num = num;
}
#Override
public String call() throws Exception {
String result = null;
System.out.println("starting job " + num);
try {
Thread.sleep(2000);
System.out.println("end job " + num);
result = "result" + num;
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
}
Or you can do this way if you want to store results in list.
public class ExecutorTest {
private static List<String> result = new ArrayList<>();
public static void main(String[] args) throws Exception {
ExecutorService ex = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
Callable worker = new Worker(i);
Future<String> stringResult= ex.submit(worker);
String output = stringResult.get();
result.add(output);
System.out.println(output);
}
ex.shutdown();
while (!ex.isTerminated());
System.out.println("All results received frmo executor service ");
System.out.println(result);
}
}

Instead of implementing Runnable you should implement Callable and then use a method returning a Future
public class Worker implements Callable<String> {
#Override
public String call() {
return "123";
}
}
ExecutorService es = Executors.newFixedThreadPool(5);
Future<?> f = es.submit(new Worker());
String result = f.get(); // 123

> I can't wrap my head around [Futures].
Maybe a brief glimpse behind the curtain would help
Future is an interface, but there is some standard library class, whose name we do not know, that implements Future. Let's pretend it's called FutureImpl.
A FutureImpl instance has a get() method that is specified by the Future API, and it has another method that we can't see, but let's call it put(x).
When Karol Dowbecki's example code submits a task to the executor service, the executor service creates a new FutureImpl instance and does two things with it:
It gives a reference to the worker thread that is going to execute your task, and,
It returns a reference from the es.submit(...) call.
Future<?> f = es.submit(...);
The worker thread will call the task function that you give it, and then it will take the returned value, and stash it in the FutureImpl object:
futureImpl.put(task.Call());
Note! f and futureImpl are two different names for the same object. The only difference is, the worker thread knows that it is an instance of the FutureImpl class, but the main thread only knows that it is an instance of some class that implements Future.
Anyway, back to the main thread... It can do other stuff while the worker is doing its thing, and then eventually, the main thread can try to retrieve the value:
String result = f.get();
If the task has already completed, and the worker has already called futureImpl.put(...) before the main thread calls f.get() then f.get() will immediately return the value. But, if the main thread calls f.get() first, then the get() call will wait until the worker thread completes the task and stashes the value.

How to wait for one of the running threads in ExecutorService to finish to assign another task

I have loop that assign task to ExecutorService with fixed size thread, I want the main program wait for threadPool to free one of its' threads to assign another task to it.
Here is my sample code: in this sample code I want finished! be printed at end and want to use ExecutorService.
public static void main(String[] args) {
ExecutorService ex = Executors.newFixedThreadPool(3);
for(int i=0; i< 100; i++) {
ex.execute(new TestThread(i)); // I want the program wait here for at least one thread to free
}
System.out.println("finished!");
}
private static class TestThread implements Runnable {
private int i;
public TestThread(int i) {
this.i = i;
}
#Override
public void run() {
System.out.println("hi: " + i);
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

I understand you want for the thread that is submitting a job, to block in the case when there is not a free, readily available worker thread in the executor service. This can be useful to apply back-pressure.
At the core the executor service is "simply" composed of a queue of runnables, and of a pool of worker threads.
You can obtain this behaviour by building an executor service with a work-queue of fixed size (in your case, size one).
In code: (note that, your caller thread will still continue after submitting the last job; it will not wait for that job to be completed)
package stackOv;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public class BackPressure {
public static void main(String[] args) {
// this is the backing work queue; in this case, it is of bounded size
ArrayBlockingQueue<Runnable> q = new ArrayBlockingQueue<>(1);
ExecutorService ex = new ThreadPoolExecutor(3, 3, 30, TimeUnit.SECONDS, q,
new ThreadPoolExecutor.CallerRunsPolicy());
for(int i=0; i< 100; i++) {
ex.execute(new TestWork(i));
}
System.out.println("finished!");
}
private static class TestWork implements Runnable {
private int i;
public TestWork(int i) {
this.i = i;
}
#Override
public void run() {
System.out.println("hi: " + i);
try {
Thread.sleep(100);
} catch (InterruptedException e) { e.printStackTrace(); }
}
}
}

All you need is:
ex.awaitTermination();

How to pass a parameter to a thread and get a return value?

public class CalculationThread implements Runnable {
int input;
int output;
public CalculationThread(int input)
{
this.input = input;
}
public void run() {
output = input + 1;
}
public int getResult() {
return output;
}
}
Somewhere else:
Thread thread = new Thread(new CalculationThread(1));
thread.start();
int result = thread.getResult();
Of course, thread.getResult() doesn't work (it tries to invoke this method from the Thread class).
You get what I want. How can I achieve this in Java?

This a job for thread pools. You need to create a Callable<R> which is Runnable returning a value and send it to a thread pool.
The result of this operation is a Future<R> which is a pointer to this job which will contain a value of the computation, or will not if the job fails.
public static class CalculationJob implements Callable<Integer> {
int input;
public CalculationJob(int input) {
this.input = input;
}
#Override
public Integer call() throws Exception {
return input + 1;
}
}
public static void main(String[] args) throws InterruptedException {
ExecutorService executorService = Executors.newFixedThreadPool(4);
Future<Integer> result = executorService.submit(new CalculationJob(3));
try {
Integer integer = result.get(10, TimeUnit.MILLISECONDS);
System.out.println("result: " + integer);
} catch (Exception e) {
// interrupts if there is any possible error
result.cancel(true);
}
executorService.shutdown();
executorService.awaitTermination(1, TimeUnit.SECONDS);
}
Prints:
result: 4

The accepted answer is great. But it is not the simplest approach. There's no real need to use an ExecutorService if you just want to wait for the result of a thread. You can simply use java.util.concurrent.FutureTask, which is basically a Runnable wrapping a Callable which also implements the Future interface.
So step 1 is still make the calculation a Callable :
public class Calculation implements Callable<Integer> {
private final int input;
public Calculation(int input) {
this.input = input;
}
#Override
public Integer call() throws Exception {
return input + 1;
}
}
So where you need the asynchronous calculation you can do :
FutureTask<Integer> task = new FutureTask<>(new Calculation(1561));
new Thread(task).start();
// ... do other stuff
// when I really need the result :
try {
int result = task.get(); // this will wait for the task to finish, if it hasn't yet.
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (ExecutionException e) {
e.getCause().printStackTrace(); // e.getCause() holds the exception that happened on the calculation thread
}
What the ExecutorService adds is managing a pool of threads to run the task on, but under the hood of an ExecutorService, basically the same thing happens.

You are looking for Callable. It's a Runnable on steroids, it can return a result. Check the javadoc:http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Callable.html
Here is a tutorial: http://www.journaldev.com/1090/java-callable-future-example

Old school style
public class CalculationThread extends Thread {
int input;
int output;
public CalculationThread(int input){
this.input = input;
}
public void run() {
output = input + 1;
}
public int getResult() {
return output;
}
}
CalculationThread thread = new CalculationThread(1);
thread.start();
thread.join();
int result = thread.getResult();

Use constructor and pass value by reference.
public CalculationThread(int[] input)
Do you want to pass the value of int to the thread called CalculationThread you should create a thread but if you pass a parameter by value the thread can use that value but you can't get the data back. Passing it by reference you can modify the value but not reference in the thread and the reference if it's available to both threads point to the value which is modifiable in both threads. Of cause you code should be synchronized to share the same reference.

Ordering threads to run in the order they were created/started

How can i order threads in the order they were instantiated.e.g. how can i make the below program print the numbers 1...10 in order.
public class ThreadOrdering {
public static void main(String[] args) {
class MyRunnable implements Runnable{
private final int threadnumber;
MyRunnable(int threadnumber){
this.threadnumber = threadnumber;
}
public void run() {
System.out.println(threadnumber);
}
}
for(int i=1; i<=10; i++){
new Thread(new MyRunnable(i)).start();
}
}
}

Sounds like you want ExecutorService.invokeAll, which will return results from worker threads in a fixed order, even though they may be scheduled in arbitrary order:
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class ThreadOrdering {
static int NUM_THREADS = 10;
public static void main(String[] args) {
ExecutorService exec = Executors.newFixedThreadPool(NUM_THREADS);
class MyCallable implements Callable<Integer> {
private final int threadnumber;
MyCallable(int threadnumber){
this.threadnumber = threadnumber;
}
public Integer call() {
System.out.println("Running thread #" + threadnumber);
return threadnumber;
}
}
List<Callable<Integer>> callables =
new ArrayList<Callable<Integer>>();
for(int i=1; i<=NUM_THREADS; i++) {
callables.add(new MyCallable(i));
}
try {
List<Future<Integer>> results =
exec.invokeAll(callables);
for(Future<Integer> result: results) {
System.out.println("Got result of thread #" + result.get());
}
} catch (InterruptedException ex) {
ex.printStackTrace();
} catch (ExecutionException ex) {
ex.printStackTrace();
} finally {
exec.shutdownNow();
}
}
}

"I actually have some parts that i want to execute in parallel, and then once the results are generated, I want to merge the results in certain order." Thanks, this clarifies what you're asking.
You can run them all at once, but the important thing is to get their results in order when the threads finish their computation. Either Thread#join() them in the order in which you want to get their results, or just Thread#join() them all and then iterate through them to get their results.
// Joins the threads back to the main thread in the order we want their results.
public class ThreadOrdering {
private class MyWorker extends Thread {
final int input;
int result;
MyWorker(final int input) {
this.input = input;
}
#Override
public void run() {
this.result = input; // Or some other computation.
}
int getResult() { return result; }
}
public static void main(String[] args) throws InterruptedException {
MyWorker[] workers = new MyWorker[10];
for(int i=1; i<=10; i++) {
workers[i] = new MyWorker(i);
workers[i].start();
}
// Assume it may take a while to do the real computation in the threads.
for (MyWorker worker : workers) {
// This can throw InterruptedException, but we're just passing that.
worker.join();
System.out.println(worker.getResult());
}
}
}

Simply put, the scheduling of threads is indeterminate.
http://www.janeg.ca/scjp/threads/scheduling.html Dead domain - do not click
WaybackMachine version of the above page
The longer answer is that if you want to do this, you'll need to manually wait for each thread to complete its work before you allow another to run. Note that in this fashion, all the threads will still interleave but they won't accomplish any work until you give the go-ahead. Have a look at the synchronize reserved word.

You can chain them – that is, have the first one start the second, the second start the third, etc. They won't really be running at the same time except for a bit of overlap when each one is started.
public class ThreadOrdering
{
public static void main(String[] args)
{
MyRunnable[] threads = new MyRunnable[10];//index 0 represents thread 1;
for(int i=1; i<=10; i++)
threads[i] = new MyRunnable(i, threads);
new Thread(threads[0].start);
}
}
public class MyRunnable extends Runnable
{
int threadNumber;
MyRunnable[] threads;
public MyRunnable(int threadNumber, MyRunnable[] threads)
{
this.threadnumber = threadnumber;
this.threads = threads;
}
public void run()
{
System.out.println(threadnumber);
if(threadnumber!=10)
new Thread(threadnumber).start();
}
}

Here's a way to do it without having a master thread that waits for each result. Instead, have the worker threads share an object which orders the results.
import java.util.*;
public class OrderThreads {
public static void main(String... args) {
Results results = new Results();
new Thread(new Task(0, "red", results)).start();
new Thread(new Task(1, "orange", results)).start();
new Thread(new Task(2, "yellow", results)).start();
new Thread(new Task(3, "green", results)).start();
new Thread(new Task(4, "blue", results)).start();
new Thread(new Task(5, "indigo", results)).start();
new Thread(new Task(6, "violet", results)).start();
}
}
class Results {
private List<String> results = new ArrayList<String>();
private int i = 0;
public synchronized void submit(int order, String result) {
while (results.size() <= order) results.add(null);
results.set(order, result);
while ((i < results.size()) && (results.get(i) != null)) {
System.out.println("result delivered: " + i + " " + results.get(i));
++i;
}
}
}
class Task implements Runnable {
private final int order;
private final String result;
private final Results results;
public Task(int order, String result, Results results) {
this.order = order;
this.result = result;
this.results = results;
}
public void run() {
try {
Thread.sleep(Math.abs(result.hashCode() % 1000)); // simulate a long-running computation
}
catch (InterruptedException e) {} // you'd want to think about what to do if interrupted
System.out.println("task finished: " + order + " " + result);
results.submit(order, result);
}
}

If you need that fine-grained control, you should not use threads but instead look into using a suitable Executor with Callables or Runnables. See the Executors class for a wide selection.

A simple solution would be to use an array A of locks (one lock per thread). When thread i begins its executions, it acquires its associated lock A[i]. When it's ready to merge its results, it releases its lock A[i] and waits for locks A[0], A[1], ..., A[i - 1] to be released; then it merges the results.
(In this context, thread i means the i-th launched thread)
I don't know what classes to use in Java, but it must be easy to implement. You can begin reading this.
If you have more questions, feel free to ask.

public static void main(String[] args) throws InterruptedException{
MyRunnable r = new MyRunnable();
Thread t1 = new Thread(r,"A");
Thread t2 = new Thread(r,"B");
Thread t3 = new Thread(r,"C");
t1.start();
Thread.sleep(1000);
t2.start();
Thread.sleep(1000);
t3.start();
}

Control of thread execution order may be implemented quite easily with the semaphores. The code attached is based on the ideas presented in Schildt's book on Java (The complete reference....).
// Based on the ideas presented in:
// Schildt H.: Java.The.Complete.Reference.9th.Edition.
import java.util.concurrent.Semaphore;
class Manager {
int n;
// Initially red on semaphores 2&3; green semaphore 1.
static Semaphore SemFirst = new Semaphore(1);
static Semaphore SemSecond = new Semaphore(0);
static Semaphore SemThird = new Semaphore(0);
void firstAction () {
try {
SemFirst.acquire();
} catch(InterruptedException e) {
System.out.println("Exception InterruptedException catched");
}
System.out.println("First: " );
System.out.println("-----> 111");
SemSecond.release();
}
void secondAction() {
try{
SemSecond.acquire();
} catch(InterruptedException e) {
System.out.println("Exception InterruptedException catched");
}
System.out.println("Second: ");
System.out.println("-----> 222");
SemThird.release();
}
void thirdAction() {
try{
SemThird.acquire();
} catch(InterruptedException e) {
System.out.println("Exception InterruptedException catched");
}
System.out.println("Third: ");
System.out.println("-----> 333");
SemFirst.release();
}
}
class Thread1 implements Runnable {
Manager q;
Thread1(Manager q) {
this.q = q;
new Thread(this, "Thread1").start();
}
public void run() {
q.firstAction();
}
}
class Thread2 implements Runnable {
Manager q;
Thread2(Manager q) {
this.q = q;
new Thread(this, "Thread2").start();
}
public void run() {
q.secondAction();
}
}
class Thread3 implements Runnable {
Manager q;
Thread3(Manager q) {
this.q = q;
new Thread(this, "Thread3").start();
}
public void run() {
q.thirdAction();
}
}
class ThreadOrder {
public static void main(String args[]) {
Manager q = new Manager();
new Thread3(q);
new Thread2(q);
new Thread1(q);
}
}

This can be done without using synchronized keyword and with the help of volatile keyword. Following is the code.
package threadOrderingVolatile;
public class Solution {
static volatile int counter = 0;
static int print = 1;
static char c = 'A';
public static void main(String[] args) {
// TODO Auto-generated method stub
Thread[] ths = new Thread[4];
for (int i = 0; i < ths.length; i++) {
ths[i] = new Thread(new MyRunnable(i, ths.length));
ths[i].start();
}
}
static class MyRunnable implements Runnable {
final int thID;
final int total;
public MyRunnable(int id, int total) {
thID = id;
this.total = total;
}
#Override
public void run() {
while(true) {
if (thID == (counter%total)) {
System.out.println("thread " + thID + " prints " + c);
if(c=='Z'){
c='A';
}else{
c=(char)((int)c+1);
}
System.out.println("thread " + thID + " prints " + print++);
counter++;
} else {
try {
Thread.sleep(30);
} catch (InterruptedException e) {
// log it
}
}
}
}
}
}
Following is the github link which has a readme, that gives detailed explanation about how it happens.
https://github.com/sankar4git/volatile_thread_ordering

How can a Thread return a value after finishing its job?

Let's say we have this simple example:
public Example extends Thread{
String temp;
public Example(){
}
#Override
public void run(){
.
.
.
.
temp = "a_value";
}
public static void main(String[] args) {
Example th = new Example();
th.start();
}
}
How can the Thread after finishing its job return me the String temp?

Make use of the (relatively) new Callable<T> instead of Runnable (available in 1.5 and newer versions):
Here is a (simple) example:
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class Main {
public static void main(final String[] argv) {
final ExecutorService service;
final Future<String> task;
service = Executors.newFixedThreadPool(1);
task = service.submit(new Foo());
try {
final String str;
// waits the 10 seconds for the Callable.call to finish.
str = task.get(); // this raises ExecutionException if thread dies
System.out.println(str);
} catch(final InterruptedException ex) {
ex.printStackTrace();
} catch(final ExecutionException ex) {
ex.printStackTrace();
}
service.shutdownNow();
}
}
class Foo implements Callable<String> {
public String call() {
try {
// sleep for 10 seconds
Thread.sleep(10 * 1000);
} catch(final InterruptedException ex) {
ex.printStackTrace();
}
return ("Hello, World!");
}
}

Look at Future interface javadoc. It has sample usage showing you how to do this.

You can achieve this by the Observer pattern.
on finishing the thread notifies all listeners that it's finished and they can retrieve the value (through a getter). Or it can even already send the computed value.
Or you can use a task, see FutureTask, a runnable ( indeed as stated below a Callable ) that returns a result and can throw exceptions.

If you don't want to swap the solution to use Callable objects then you can use also queues and return the result from the threads that way.
I re-wrote your example like this:
import java.util.PriorityQueue;
import java.util.Queue;
public class GetResultFromThread {
public static void main(String[] args) throws Exception {
Queue<String> queue = new PriorityQueue<String>();
int expectedResults = 2;
for (int i = 0; i < expectedResults; i++) {
new Example(queue).start();
}
int receivedResults = 0;
while (receivedResults < expectedResults) {
if (!queue.isEmpty()) {
System.out.println(queue.poll());
receivedResults++;
}
Thread.sleep(1000);
}
}
}
class Example extends Thread {
private final Queue<String> results;
public Example(Queue<String> results) {
this.results = results;
}
#Override
public void run() {
results.add("result from thread");
}
}
Note that you shall think of synchronization and concurrency!