We Keep Coding

Calling a method with multiple argument

I've been using the Java 8 Streams for a while. I came across a situation where I need to stream through a List and pass each element to a method of a class that is not static.
List<String> emps = new ArrayList<>();
emps.add("ABC");
emps.add("DEF");
emps.add("GHI");
I want to call the "start" method of EmpDataGenerator.
EmpDataGenerator generator = new EmpDataGenerator(
Executors.newFixedThreadPool(emps.size()));
I have tried this, but it's not working
emps.stream().map(e-> generator.start(e));
public class EmpDataGenerator {
// Used to signal a graceful shutdown
private volatile boolean stop = false;
private final ExecutorService executor;
public EmpDataGenerator(ExecutorService executor) {
this.executor = executor;
}
public void start(String name ) {
Runnable generator = () -> {
try {
while (!stop) {
//do some processing
}
System.out.println("Broke while loop, stop " + stop);
} catch (Exception e) {
System.out.println("EmpDataGenerator thread caught an exception and halted!");
throw e;
}
};
executor.execute(generator);
}
public void stop() {
stop = true;
// The shutdown the executor (after waiting a bit to be nice)
try {
executor.awaitTermination(1000, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
// Purposely ignore any InterruptedException
Thread.currentThread().interrupt();
}
executor.shutdownNow();
}
}

A map must take an input and transform it to something. The start method is void.
There is no need for streams here. A simple forEach should do.
emps.forEach(e-> generator.start(e));
or
emps.forEach(generator::start);

Unit testing asynchronous computation that has to be interrupted manually

I have got a class that records eyetracking data asynchronously. There are methods to start and stop the recording process. The data is collected in a collection and the collection can only be accessed if the recording thread has finished its work. It basically encapsulates all the threading and synchronizing so the user of my library doesn't have to do it.
The heavily shortened code (generics and error handling omitted):
public class Recorder {
private Collection accumulatorCollection;
private Thread recordingThread;
private class RecordingRunnable implements Runnable {
...
public void run() {
while(!Thread.currentThread().isInterrupted()) {
// fetch data and collect it in the accumulator
synchronized(acc) { acc.add(Eyetracker.getData()) }
}
}
}
public void start() {
accumulatorCollection = new Collection();
recordingThread = new Thread(new RecordingRunnable(accumulatorCollection));
recordingThread.start();
}
public void stop() {
recordingThread.interrupt();
}
public void getData() {
try {
recordingThread.join(2000);
if(recordingThread.isAlive()) { throw Exception(); }
}
catch(InterruptedException e) { ... }
synchronized(accumulatorCollection) { return accumulatorCollection; }
}
}
The usage is quite simple:
recorder.start();
...
recorder.stop();
Collection data = recorder.getData();
My problem with the whole thing is how to test it. Currently i am doing it like this:
recorder.start();
Thread.sleep(50);
recorder.stop();
Collection data = recorder.getData();
assert(stuff);
This works, but it is non-deterministic and slows down the test suite quite a bit (i marked these tests as integration tests, so they have to be run separately to circumvent this problem).
Is there a better way?

There is a better way using a CountDownLatch.
The non-deterministic part of the test stems from two variables in time you do not account for:
creating and starting a thread takes time and the thread may not have started executing the runnable when Thread.start() returns (the runnable will get executed, but it may be a bit later).
the stop/interrupt will break the while-loop in the Runnable but not immediately, it may be a bit later.
This is where a CountDownLatch comes in: it gives you precise information about where another thread is in execution. E.g. let the first thread wait on the latch, while the second "counts down" the latch as last statement within a runnable and now the first thread knows that the runnable finished. The CountDownLatch also acts as a synchronizer: whatever the second thread was writing to memory, can now be read by the first thread.
Instead of using an interrupt, you can also use a volatile boolean. Any thread reading the volatile variable is guaranteed to see the last value set by any other thread.
A CountDownLatch can also be given a timeout which is useful for tests that can hang: if you have to wait to long you can abort the whole test (e.g. shutdown executors, interrupt threads) and throw an AssertionError. In the code below I re-used the timeout to wait for a certain amount of data to collect instead of 'sleeping'.
As an optimization, use an Executor (ThreadPool) instead of creating and starting threads. The latter is relative expensive, using an Executor can really make a difference.
Below the updated code, I made it runnable as an application (main method). (edit 28/02/17: check maxCollect > 0 in while-loop)
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class Recorder {
private final ExecutorService executor;
private Thread recordingThread;
private volatile boolean stopRecording;
private CountDownLatch finishedRecording;
private Collection<Object> eyeData;
private int maxCollect;
private final AtomicBoolean started = new AtomicBoolean();
private final AtomicBoolean stopped = new AtomicBoolean();
public Recorder() {
this(null);
}
public Recorder(ExecutorService executor) {
this.executor = executor;
}
public Recorder maxCollect(int max) { maxCollect = max; return this; }
private class RecordingRunnable implements Runnable {
#Override public void run() {
try {
int collected = 0;
while (!stopRecording) {
eyeData.add(EyeTracker.getData());
if (maxCollect > 0 && ++collected >= maxCollect) {
stopRecording = true;
}
}
} finally {
finishedRecording.countDown();
}
}
}
public Recorder start() {
if (!started.compareAndSet(false, true)) {
throw new IllegalStateException("already started");
}
stopRecording = false;
finishedRecording = new CountDownLatch(1);
eyeData = new ArrayList<Object>();
// the RecordingRunnable created below will see the values assigned above ('happens before relationship')
if (executor == null) {
recordingThread = new Thread(new RecordingRunnable());
recordingThread.start();
} else {
executor.execute(new RecordingRunnable());
}
return this;
}
public Collection<Object> getData(long timeout, TimeUnit tunit) {
if (started.get() == false) {
throw new IllegalStateException("start first");
}
if (!stopped.compareAndSet(false, true)) {
throw new IllegalStateException("data already fetched");
}
if (maxCollect <= 0) {
stopRecording = true;
}
boolean recordingStopped = false;
try {
// this establishes a 'happens before relationship'
// all updates to eyeData are now visible in this thread.
recordingStopped = finishedRecording.await(timeout, tunit);
} catch(InterruptedException e) {
throw new RuntimeException("interrupted", e);
} finally {
stopRecording = true;
}
// if recording did not stop, do not return the eyeData (could stil be modified by recording-runnable).
if (!recordingStopped) {
throw new RuntimeException("recording");
}
// only when everything is OK this recorder instance can be re-used
started.set(false);
stopped.set(false);
return eyeData;
}
public static class EyeTracker {
public static Object getData() {
try { Thread.sleep(1); } catch (Exception ignored) {}
return new Object();
}
}
public static void main(String[] args) {
System.out.println("Starting.");
ExecutorService exe = Executors.newSingleThreadExecutor();
try {
Recorder r = new Recorder(exe).maxCollect(50).start();
int dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(100).start();
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(0).start();
Thread.sleep(100);
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
} catch (Exception e) {
e.printStackTrace();
} finally {
exe.shutdownNow();
System.out.println("Done.");
}
}
}
Happy coding :)

handle multiple infinite tasks in a single thread? P.S run one task at a time and control its task behavior(i.e starting/stoping task) from outside

I want to make a single thread which would contain 3 infinite tasks.
I want one task to run at a time and start/stop running task when required.
For example first I want task 1 to run, then I want task 2 to run but after stopping task 1 and again I want task 1 to run but after stopping of task 2 and so on.
Infinite task needs to check some condition and if that condition is satisfied perform some operations and if not satisfied sleep for few seconds and after wake up perform the above same operations again.
Infinite Runnable task looks some thing like this:
new Runnable(){
while(1){
if(TaskQueue.getInstance().size()<= 100){
TaskQueue.getInstance().push("add command to the end of queue");
}else{
try {
Thread.sleep(10000);
}catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
Any help would be appreciated?
Edit : I modified my question. I want a continuous single running thread(some thing like looper ) to monitor 3 infinite tasks and control this single continuous running thread tasks from outside.

Use this for start/stop thread in real-time:
class MyThread extends Thread {
private volatile boolean running = true; // Run unless told to pause
...
#Override
public void run() {
// Only keep painting while "running" is true
// This is a crude implementation of pausing the thread
while (true) {
if (Thread.currentThread().isInterrupted()) {
return;
}
if (running) {
//Your code
} else yield;
}
}
public void pauseThread() throws InterruptedException {
running = false;
}
public void resumeThread() {
running = true;
}
}
For pause thread use this:
myThread.pauseThread();
For resume thread use this:
myThread.resumeThread();
For stop thread use this (Not recommended):
myThread.stop();
For currently stop thread use this:
myThread.interrupt();

You must use a class like Thread that already implements Runnable.
new Thread(){....};
And the way it works it's:
Thread t = new Thread(){.....};
t.start();
t.stop();

You could also initialize a new thread, like:
Thread exampleThread = new thread();
After this you can start it at any point in your code by:
exampleThread.start();

you can use Semaphore,
to Manage the amount of signal.
private final static Semaphore semaphore = new Semaphore(0);
public static void main(String[] args) throws Exception {
//入口
threadTest();
}
public static void thread1() {
try{
//…… some code
}
finally{
semaphore.release();
}
}
public static void thread2() {
semaphore.acquire(1);
}
The question is my first answer,thanks.

I finally made my task scheduler. The API of which looks something like this:
TaskScheduler taskScheduler = TaskScheduler.getInstance();
taskScheduler.startTaskOne();
taskScheduler.stopTaskOne();
taskScheduler.startTaskTwo();
taskScheduler.stopTaskTwo();
Runs one task at a time (because I used Executors.newSingleThreadExecutor()).
We can control the execution of the task from outside:
public class TaskScheduler {
private static ExecutorService mTaskRunningService;
private static TaskScheduler mInstance;
private Future mFirstTaskFuture = null;
private Future mSecondTaskFuture = null;
static {
configure();
}
private static void configure() {
mTaskRunningService = Executors.newSingleThreadExecutor();
}
public static TaskScheduler getInstance() {
if (mInstance == null) {
mInstance = new TaskScheduler();
}
return mInstance;
}
private Runnable mTaskOneRunnable = new Runnable() {
#Override
public void run() {
try {
while (true) {
/** stop this single thread (i.e executing one task at time) service if this thread is interrupted
* from outside because documentation of {#link java.util.concurrent.ThreadPoolExecutor#shutdownNow()}
* says we need to do this*/
if (Thread.currentThread().isInterrupted()) {
return;
}
// task one work.......
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
private Runnable mTaskTwoRunnable = new Runnable() {
#Override
public void run() {
try {
while (true) {
/** stop this single thread (i.e executing one task at time) service if this thread is interrupted
* from outside because documentation of {#link java.util.concurrent.ThreadPoolExecutor#shutdownNow()}
* says we need to do this*/
if (Thread.currentThread().isInterrupted()) {
return;
}
// task two work......
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
public synchronized void startTaskOne() {
if (mFirstTaskFuture == null) {
// start executing runnable
mFirstTaskFuture = mTaskRunningService.submit(mTaskOneRunnable);
}
}
public synchronized boolean stopTaskOne() {
if (mFirstTaskFuture != null) {
// stop general reading thread
mFirstTaskFuture.cancel(true);
// cancel status
boolean status = mFirstTaskFuture.isDone();
// assign null because startTaskOne() again be called
mGeneralFuture = null;
return status;
}
return true;
}
public synchronized void startTaskTwo() {
if (mSecondTaskFuture == null) {
// start executing runnable
mSecondTaskFuture = mTaskRunningService.submit(mTaskTwoRunnable);
}
}
public synchronized boolean stopTaskTwo() {
if (mSecondTaskFuture != null) {
// clear task queue
mTaskQueue.clearTaskQueue();
// stop 22 probes reading thread
mSecondTaskFuture.cancel(true);
// cancel status
boolean status = mSecondTaskFuture.isDone();
// assign null because startTaskTwo() again be called
mSecondTaskFuture = null;
return status;
}
return true;
}
}

Waiting on a list of Future

I have a method which returns a List of futures
List<Future<O>> futures = getFutures();
Now I want to wait until either all futures are done processing successfully or any of the tasks whose output is returned by a future throws an exception. Even if one task throws an exception, there is no point in waiting for the other futures.
Simple approach would be to
wait() {
For(Future f : futures) {
try {
f.get();
} catch(Exception e) {
//TODO catch specific exception
// this future threw exception , means somone could not do its task
return;
}
}
}
But the problem here is if, for example, the 4th future throws an exception, then I will wait unnecessarily for the first 3 futures to be available.
How to solve this? Will count down latch help in any way? I'm unable to use Future isDone because the java doc says
boolean isDone()
Returns true if this task completed. Completion may be due to normal termination, an exception, or cancellation -- in all of these cases, this method will return true.

You can use a CompletionService to receive the futures as soon as they are ready and if one of them throws an exception cancel the processing. Something like this:
Executor executor = Executors.newFixedThreadPool(4);
CompletionService<SomeResult> completionService =
new ExecutorCompletionService<SomeResult>(executor);
//4 tasks
for(int i = 0; i < 4; i++) {
completionService.submit(new Callable<SomeResult>() {
public SomeResult call() {
...
return result;
}
});
}
int received = 0;
boolean errors = false;
while(received < 4 && !errors) {
Future<SomeResult> resultFuture = completionService.take(); //blocks if none available
try {
SomeResult result = resultFuture.get();
received ++;
... // do something with the result
}
catch(Exception e) {
//log
errors = true;
}
}
I think you can further improve to cancel any still executing tasks if one of them throws an error.

If you are using Java 8 then you can do this easier with CompletableFuture and CompletableFuture.allOf, which applies the callback only after all supplied CompletableFutures are done.
// Waits for *all* futures to complete and returns a list of results.
// If *any* future completes exceptionally then the resulting future will also complete exceptionally.
public static <T> CompletableFuture<List<T>> all(List<CompletableFuture<T>> futures) {
CompletableFuture[] cfs = futures.toArray(new CompletableFuture[futures.size()]);
return CompletableFuture.allOf(cfs)
.thenApply(ignored -> futures.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList())
);
}

Use a CompletableFuture in Java 8
// Kick of multiple, asynchronous lookups
CompletableFuture<User> page1 = gitHubLookupService.findUser("Test1");
CompletableFuture<User> page2 = gitHubLookupService.findUser("Test2");
CompletableFuture<User> page3 = gitHubLookupService.findUser("Test3");
// Wait until they are all done
CompletableFuture.allOf(page1,page2,page3).join();
logger.info("--> " + page1.get());

You can use an ExecutorCompletionService. The documentation even has an example for your exact use-case:
Suppose instead that you would like to use the first non-null result of the set of tasks, ignoring any that encounter exceptions, and cancelling all other tasks when the first one is ready:
void solve(Executor e, Collection<Callable<Result>> solvers) throws InterruptedException {
CompletionService<Result> ecs = new ExecutorCompletionService<Result>(e);
int n = solvers.size();
List<Future<Result>> futures = new ArrayList<Future<Result>>(n);
Result result = null;
try {
for (Callable<Result> s : solvers)
futures.add(ecs.submit(s));
for (int i = 0; i < n; ++i) {
try {
Result r = ecs.take().get();
if (r != null) {
result = r;
break;
}
} catch (ExecutionException ignore) {
}
}
} finally {
for (Future<Result> f : futures)
f.cancel(true);
}
if (result != null)
use(result);
}
The important thing to notice here is that ecs.take() will get the first completed task, not just the first submitted one. Thus you should get them in the order of finishing the execution (or throwing an exception).

If you are using Java 8 and don't want to manipulate CompletableFutures, I have written a tool to retrieve results for a List<Future<T>> using streaming. The key is that you are forbidden to map(Future::get) as it throws.
public final class Futures
{
private Futures()
{}
public static <E> Collector<Future<E>, Collection<E>, List<E>> present()
{
return new FutureCollector<>();
}
private static class FutureCollector<T> implements Collector<Future<T>, Collection<T>, List<T>>
{
private final List<Throwable> exceptions = new LinkedList<>();
#Override
public Supplier<Collection<T>> supplier()
{
return LinkedList::new;
}
#Override
public BiConsumer<Collection<T>, Future<T>> accumulator()
{
return (r, f) -> {
try
{
r.add(f.get());
}
catch (InterruptedException e)
{}
catch (ExecutionException e)
{
exceptions.add(e.getCause());
}
};
}
#Override
public BinaryOperator<Collection<T>> combiner()
{
return (l1, l2) -> {
l1.addAll(l2);
return l1;
};
}
#Override
public Function<Collection<T>, List<T>> finisher()
{
return l -> {
List<T> ret = new ArrayList<>(l);
if (!exceptions.isEmpty())
throw new AggregateException(exceptions, ret);
return ret;
};
}
#Override
public Set<java.util.stream.Collector.Characteristics> characteristics()
{
return java.util.Collections.emptySet();
}
}
This needs an AggregateException that works like C#'s
public class AggregateException extends RuntimeException
{
/**
*
*/
private static final long serialVersionUID = -4477649337710077094L;
private final List<Throwable> causes;
private List<?> successfulElements;
public AggregateException(List<Throwable> causes, List<?> l)
{
this.causes = causes;
successfulElements = l;
}
public AggregateException(List<Throwable> causes)
{
this.causes = causes;
}
#Override
public synchronized Throwable getCause()
{
return this;
}
public List<Throwable> getCauses()
{
return causes;
}
public List<?> getSuccessfulElements()
{
return successfulElements;
}
public void setSuccessfulElements(List<?> successfulElements)
{
this.successfulElements = successfulElements;
}
}
This component acts exactly as C#'s Task.WaitAll. I am working on a variant that does the same as CompletableFuture.allOf (equivalento to Task.WhenAll)
The reason why I did this is that I am using Spring's ListenableFuture and don't want to port to CompletableFuture despite it is a more standard way

In case that you want combine a List of CompletableFutures, you can do this :
List<CompletableFuture<Void>> futures = new ArrayList<>();
// ... Add futures to this ArrayList of CompletableFutures
// CompletableFuture.allOf() method demand a variadic arguments
// You can use this syntax to pass a List instead
CompletableFuture<Void> allFutures = CompletableFuture.allOf(
futures.toArray(new CompletableFuture[futures.size()]));
// Wait for all individual CompletableFuture to complete
// All individual CompletableFutures are executed in parallel
allFutures.get();
For more details on Future & CompletableFuture, useful links:
1. Future: https://www.baeldung.com/java-future
2. CompletableFuture: https://www.baeldung.com/java-completablefuture
3. CompletableFuture: https://www.callicoder.com/java-8-completablefuture-tutorial/

I've got a utility class that contains these:
#FunctionalInterface
public interface CheckedSupplier<X> {
X get() throws Throwable;
}
public static <X> Supplier<X> uncheckedSupplier(final CheckedSupplier<X> supplier) {
return () -> {
try {
return supplier.get();
} catch (final Throwable checkedException) {
throw new IllegalStateException(checkedException);
}
};
}
Once you have that, using a static import, you can simple wait for all futures like this:
futures.stream().forEach(future -> uncheckedSupplier(future::get).get());
you can also collect all their results like this:
List<MyResultType> results = futures.stream()
.map(future -> uncheckedSupplier(future::get).get())
.collect(Collectors.toList());
Just revisiting my old post and noticing that you had another grief:
But the problem here is if, for example, the 4th future throws an exception, then I will wait unnecessarily for the first 3 futures to be available.
In this case, the simple solution is to do this in parallel:
futures.stream().parallel()
.forEach(future -> uncheckedSupplier(future::get).get());
This way the first exception, although it will not stop the future, will break the forEach-statement, like in the serial example, but since all wait in parallel, you won't have to wait for the first 3 to complete.

maybe this would help (nothing would replaced with raw thread, yeah!)
I suggest run each Future guy with a separated thread (they goes parallel), then when ever one of the got error, it just signal the manager(Handler class).
class Handler{
//...
private Thread thisThread;
private boolean failed=false;
private Thread[] trds;
public void waitFor(){
thisThread=Thread.currentThread();
List<Future<Object>> futures = getFutures();
trds=new Thread[futures.size()];
for (int i = 0; i < trds.length; i++) {
RunTask rt=new RunTask(futures.get(i), this);
trds[i]=new Thread(rt);
}
synchronized (this) {
for(Thread tx:trds){
tx.start();
}
}
for(Thread tx:trds){
try {tx.join();
} catch (InterruptedException e) {
System.out.println("Job failed!");break;
}
}if(!failed){System.out.println("Job Done");}
}
private List<Future<Object>> getFutures() {
return null;
}
public synchronized void cancelOther(){if(failed){return;}
failed=true;
for(Thread tx:trds){
tx.stop();//Deprecated but works here like a boss
}thisThread.interrupt();
}
//...
}
class RunTask implements Runnable{
private Future f;private Handler h;
public RunTask(Future f,Handler h){this.f=f;this.h=h;}
public void run(){
try{
f.get();//beware about state of working, the stop() method throws ThreadDeath Error at any thread state (unless it blocked by some operation)
}catch(Exception e){System.out.println("Error, stopping other guys...");h.cancelOther();}
catch(Throwable t){System.out.println("Oops, some other guy has stopped working...");}
}
}
I have to say the above code would error(didn't check), but I hope I could explain the solution. please have a try.

/**
* execute suppliers as future tasks then wait / join for getting results
* #param functors a supplier(s) to execute
* #return a list of results
*/
private List getResultsInFuture(Supplier<?>... functors) {
CompletableFuture[] futures = stream(functors)
.map(CompletableFuture::supplyAsync)
.collect(Collectors.toList())
.toArray(new CompletableFuture[functors.length]);
CompletableFuture.allOf(futures).join();
return stream(futures).map(a-> {
try {
return a.get();
} catch (InterruptedException | ExecutionException e) {
//logger.error("an error occurred during runtime execution a function",e);
return null;
}
}).collect(Collectors.toList());
};

The CompletionService will take your Callables with the .submit() method and you can retrieve the computed futures with the .take() method.
One thing you must not forget is to terminate the ExecutorService by calling the .shutdown() method. Also you can only call this method when you have saved a reference to the executor service so make sure to keep one.
Example code - For a fixed number of work items to be worked on in parallel:
ExecutorService service = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
CompletionService<YourCallableImplementor> completionService =
new ExecutorCompletionService<YourCallableImplementor>(service);
ArrayList<Future<YourCallableImplementor>> futures = new ArrayList<Future<YourCallableImplementor>>();
for (String computeMe : elementsToCompute) {
futures.add(completionService.submit(new YourCallableImplementor(computeMe)));
}
//now retrieve the futures after computation (auto wait for it)
int received = 0;
while(received < elementsToCompute.size()) {
Future<YourCallableImplementor> resultFuture = completionService.take();
YourCallableImplementor result = resultFuture.get();
received ++;
}
//important: shutdown your ExecutorService
service.shutdown();
Example code - For a dynamic number of work items to be worked on in parallel:
public void runIt(){
ExecutorService service = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
CompletionService<CallableImplementor> completionService = new ExecutorCompletionService<CallableImplementor>(service);
ArrayList<Future<CallableImplementor>> futures = new ArrayList<Future<CallableImplementor>>();
//Initial workload is 8 threads
for (int i = 0; i < 9; i++) {
futures.add(completionService.submit(write.new CallableImplementor()));
}
boolean finished = false;
while (!finished) {
try {
Future<CallableImplementor> resultFuture;
resultFuture = completionService.take();
CallableImplementor result = resultFuture.get();
finished = doSomethingWith(result.getResult());
result.setResult(null);
result = null;
resultFuture = null;
//After work package has been finished create new work package and add it to futures
futures.add(completionService.submit(write.new CallableImplementor()));
} catch (InterruptedException | ExecutionException e) {
//handle interrupted and assert correct thread / work packet count
}
}
//important: shutdown your ExecutorService
service.shutdown();
}
public class CallableImplementor implements Callable{
boolean result;
#Override
public CallableImplementor call() throws Exception {
//business logic goes here
return this;
}
public boolean getResult() {
return result;
}
public void setResult(boolean result) {
this.result = result;
}
}

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public class Stack2 {
public static void waitFor(List<Future<?>> futures) {
List<Future<?>> futureCopies = new ArrayList<Future<?>>(futures);//contains features for which status has not been completed
while (!futureCopies.isEmpty()) {//worst case :all task worked without exception, then this method should wait for all tasks
Iterator<Future<?>> futureCopiesIterator = futureCopies.iterator();
while (futureCopiesIterator.hasNext()) {
Future<?> future = futureCopiesIterator.next();
if (future.isDone()) {//already done
futureCopiesIterator.remove();
try {
future.get();// no longer waiting
} catch (InterruptedException e) {
//ignore
//only happen when current Thread interrupted
} catch (ExecutionException e) {
Throwable throwable = e.getCause();// real cause of exception
futureCopies.forEach(f -> f.cancel(true));//cancel other tasks that not completed
return;
}
}
}
}
}
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(3);
Runnable runnable1 = new Runnable (){
public void run(){
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
}
}
};
Runnable runnable2 = new Runnable (){
public void run(){
try {
Thread.sleep(4000);
} catch (InterruptedException e) {
}
}
};
Runnable fail = new Runnable (){
public void run(){
try {
Thread.sleep(1000);
throw new RuntimeException("bla bla bla");
} catch (InterruptedException e) {
}
}
};
List<Future<?>> futures = Stream.of(runnable1,fail,runnable2)
.map(executorService::submit)
.collect(Collectors.toList());
double start = System.nanoTime();
waitFor(futures);
double end = (System.nanoTime()-start)/1e9;
System.out.println(end +" seconds");
}
}

This is what i use to wait for a certain time on a list of futures. I think its cleaner.
CountDownLatch countDownLatch = new CountDownLatch(partitions.size());
// Some parallel work
for (Something tp : somethings) {
completionService.submit(() -> {
try {
work(something)
} catch (ConnectException e) {
} finally {
countDownLatch.countDown();
}
});
}
try {
if (!countDownLatch.await(secondsToWait, TimeUnit.SECONDS)){
}
} catch (InterruptedException e) {
}

A Guava-based solution can be implemented using Futures.FutureCombiner.
Here is the code example given in the javadoc:
final ListenableFuture<Instant> loginDateFuture =
loginService.findLastLoginDate(username);
final ListenableFuture<List<String>> recentCommandsFuture =
recentCommandsService.findRecentCommands(username);
ListenableFuture<UsageHistory> usageFuture =
Futures.whenAllSucceed(loginDateFuture, recentCommandsFuture)
.call(
() ->
new UsageHistory(
username,
Futures.getDone(loginDateFuture),
Futures.getDone(recentCommandsFuture)),
executor);
For more info, see the ListenableFutureExplained section of the user's guide.
If you're curious about how it works under the hood, I suggest looking at this part of the source code: AggregateFuture.java#L127-L186

Java: notify main class when all threads in threadpool are finished / same instance of object in different threads

How do I notify my main class which instantiates a ThreadPoolExecutor when all threads within the ThreadPoolExecutor are completed?
ThreadPoolExecutor threadPool = null;
ThreadClass threadclass1;
ThreadClass threadclass2;
final ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(maxPoolSize);
puclic MyClass(){
threadPool = new ThreadPoolExecutor(poolSize, maxPoolSize, keepAliveTime, TimeUnit.SECONDS, queue);
threadClass1 = new ThreadClass;
threadClass2 = new ThreadClass;
threadPool.execute(threadClass1);
threadPool.execute(threadClass2);
//Now I would like to do something until the threadPool is done working
//The threads fill a ConcurrentLinkedQueueand I would like to poll
//the queue as it gets filled by the threads and output
//it to XML via JAX-RS
}
EDIT 1
Wile my threads fetch data from somewhere and fill this information into a ConcurrentLinkedQueue I basically would like to perform some action in MyClass to update the XML output with the results. When all threads are terminated I would like to return true to the JAX-RS webservice which instantiated MyClass so the webservice knows all data has been fetched and it can now display the final XML file
EDIT 2
I am passing a Queue to threads so they can add items to the queue. When one driver is done adding items to the articleQueue I want to perform an action within my main class, polling the entity from the Queue and handing it over to the response object to display it in some way.
When I pass the queue to the threads, are they working with the same object or with a "copy" of the object so that changes within the thread do not effect the main object? That is not the behavior I want. When I check the size of the articleQueue within the Driver it is 18, the size of the articleQueue in the DriverController is 0.
Is there a nicer way to react when a thread has added something to the queue other than my while loop? How do I have to modify my code to acces the same object within different classes?
DriverController
public class DriverController {
Queue<Article> articleQueue;
ThreadPoolExecutor threadPool = null;
final ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(
maxPoolSize);
public DriverController(Response response) {
articleQueue = new ConcurrentLinkedQueue<Article>();
threadPool = new ThreadPoolExecutor();
Driver driver = new Driver(this.articleQueue);
threadPool.execute(driver);
// More drivers would be executed here which add to the queue
while (threadPool.getActiveCount() > 0) {
// this.articleQueue.size() gives back 0 here ... why?
if(articleQueue.size()>0){
response.addArticle(articleQueue.poll());
}
}
}
}
Driver
public class Driver implements Runnable{
private Queue<Article> articleQueue;
public DriverAlliedElectronics(Queue articleQueue) {
this.articleQueue = articleQueue;
}
public boolean getData() {
// Here would be the code where the article is created ...
this.articleQueue.offer(article);
return true;
}
public void run() {
this.getData();
// this.articleQueue.size() gives back 18 here ...
}
}

You should try to use following snippet
//Now I would like to wait until the threadPool is done working
threadPool.shutdown();
while (!threadPool.isTerminated()) {
try {
threadPool.awaitTermination(10, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
}

Maybe a ExecutorCompletionService might be the right thing for you:
http://download.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ExecutorCompletionService.html
Example from the link above:
void solve(Executor e, Collection<Callable<Result>> solvers)
throws InterruptedException, ExecutionException {
CompletionService<Result> ecs = new ExecutorCompletionService<Result>(e);
for (Callable<Result> s : solvers)
ecs.submit(s);
int n = solvers.size();
for (int i = 0; i < n; ++i) {
Result r = ecs.take().get();
if (r != null)
use(r);
}
}

Instead of using execute you should use submit. This will return a Future instance on which you can wait for the task(s) to complete. That way you don't need polling or shutting down the pool.

I don't think there's a way to do this explicitly. You could poll the getCompletedTaskCount() to wait for that to become zero.
Why not collect the Future objects returned upon submission and check for all of those being completed ? Simply call get() on each one in turn. Since that call blocks you'll simply wait for each in turn and gradually fall through the set until you've waited on each on.
Alternatively you could submit the threads, and call shutdown() on the executor. That way, the submitted tasks will be executed, and then the terminated() method is called. If you override this then you'll get a callback once all tasks are completed (you couldn't use that executor again, obviously).

Judging from the reference documentation you have a few options:
ThreadPoolExecutor threadPool = null;
ThreadClass threadclass1;
ThreadClass threadclass2;
final ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(maxPoolSize);
puclic MyClass(){
threadPool = new ThreadPoolExecutor(poolSize, maxPoolSize, keepAliveTime, TimeUnit.SECONDS, queue);
threadClass1 = new ThreadClass;
threadClass2 = new ThreadClass;
threadPool.execute(threadClass1);
threadPool.execute(threadClass2);
//Now I would like to wait until the threadPool is done working
//Option 1: shutdown() and awaitTermination()
threadPool.shutDown();
try {
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS)
}
catch (InterruptedException e) {
e.printStackTrace();
}
//Option 2: getActiveCount()
while (threadPool.getActiveCount() > 0) {
try {
Thread.sleep(1000);
}
catch (InterruptedException ignored) {}
}
//Option 3: getCompletedTaskCount()
while (threadPool.getCompletedTaskCount() < totalNumTasks) {
try {
Thread.sleep(1000);
}
catch (InterruptedException ignored) {}
}
}
All things considered, I think shutdown() and awaitTermination() is the best option of the three.

I think you're overengineering things a bit. You don't really care about the threads or the thread pool, and rightly so. Java provides nice abstractions so that you don't have to. You just need to know when your tasks are complete, and methods exist for that. Just submit your jobs, and wait for the futures to say they're done. If you really want to know as soon as a single task completes, you can watch all the futures and take action as soon as any one is finished. If not and you only care that everything is finished, you can remove some complexity from the code I'm about to post. Try this on for size (note MultithreadedJaxrsResource is executable):
import javax.ws.rs.*;
import javax.ws.rs.core.MediaType;
import java.util.*;
import java.util.concurrent.*;
#Path("foo")
public class MultithreadedJaxrsResource {
private ExecutorService executorService;
public MultithreadedJaxrsResource(ExecutorService executorService) {
this.executorService = executorService;
}
#GET
#Produces(MediaType.APPLICATION_XML)
public AllMyArticles getStuff() {
List<Future<Article>> futures = new ArrayList<Future<Article>>();
// Submit all the tasks to run
for (int i = 0; i < 10; i++) {
futures.add(executorService.submit(new Driver(i + 1)));
}
AllMyArticles articles = new AllMyArticles();
// Wait for all tasks to finish
// If you only care that everything is done and not about seeing
// when each one finishes, this outer do/while can go away, and
// you only need a single for loop to wait on each future.
boolean allDone;
do {
allDone = true;
Iterator<Future<Article>> futureIterator = futures.iterator();
while (futureIterator.hasNext()) {
Future<Article> future = futureIterator.next();
if (future.isDone()) {
try {
articles.articles.add(future.get());
futureIterator.remove();
} catch (InterruptedException e) {
// thread was interrupted. don't do that.
throw new IllegalStateException("broken", e);
} catch (ExecutionException e) {
// execution of the Callable failed with an
// exception. check it out.
throw new IllegalStateException("broken", e);
}
} else {
allDone = false;
}
}
} while (!allDone);
return articles;
}
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(10);
AllMyArticles stuff =
new MultithreadedJaxrsResource(executorService).getStuff();
System.out.println(stuff.articles);
executorService.shutdown();
}
}
class Driver implements Callable<Article> {
private int i; // Just to differentiate the instances
public Driver(int i) {
this.i = i;
}
public Article call() {
// Simulate taking some time for each call
try {
Thread.sleep(1000 / i);
} catch (InterruptedException e) {
System.err.println("oops");
}
return new Article(i);
}
}
class AllMyArticles {
public final List<Article> articles = new ArrayList<Article>();
}
class Article {
public final int i;
public Article(int i) {
this.i = i;
}
#Override
public String toString() {
return "Article{" +
"i=" + i +
'}';
}
}
Done that way, you can plainly see that the tasks are returned in the order they complete, as the last task finishes first thanks to sleeping the shortest time. If you don't care about completion order and just want to wait for all to finish, the loop becomes much simpler:
for (Future<Article> future : futures) {
try {
articles.articles.add(future.get());
} catch (InterruptedException e) {
// thread was interrupted. don't do that.
throw new IllegalStateException("broken", e);
} catch (ExecutionException e) {
// execution of the Callable failed with an exception. check it out.
throw new IllegalStateException("broken", e);
}
}