We Keep Coding

How do javafx multithreading execute sequentially?

I don't know if there are any other good ways to achieve the results I want, thank you.
I have a requirement, according to the URL, create multiple webview threads, and execute them in order, such as thread execution, then trigger thread two execution, and so on, I use the synchronized (lobject) method, but in JAVAfx encountered a problem, the code is as follows:
public class LockObject {
public int orderNum = 1;
public final static int MaxValue=9;
public LockObject(int orderNum){
this.orderNum = orderNum;
}
}
public class DownloadThread extends Thread{
private LockObject lobject;
private int printNum =0;
private String url;
public DownloadThread(LockObject lobject,int printNum,String url){
this.lobject=lobject;
this.printNum = printNum;
this.url = url;
}
#Override
public void run() {
synchronized(lobject){
while(lobject.orderNum <= lobject.MaxValue){
if(lobject.orderNum == printNum){
System.out.print(printNum);
Platform.runLater(new Runnable() {
#Override
public void run() {
webView.getEngine().load(url);
webView.getEngine().getLoadWorker().stateProperty().addListener(new ChangeListener<Worker.State>() {
#Override
public void changed(ObservableValue<? extends Worker.State> observable, Worker.State oldValue, Worker.State newValue) {
if (newValue == Worker.State.SUCCEEDED) {
try {
//xxxxx
// java.lang.IllegalMonitorStateException
lobject.notifyAll();
} catch (Exception e) {
e.printStackTrace();
}
}
}
});
}
});
lobject.orderNum++;
if(lobject.orderNum==downloadThreads.length){
saveCsvFile(goodCSVS);
}
//lobject.notifyAll(); is ok
}else{
try {
lobject.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
}
Place of call
private DownloadThread[] downloadThreads;
LockObject lobject = new LockObject(1);
downloadThreads = new DownloadThread[tableView.getItems().size()];
for (int i = 0; i < tableView.getItems().size(); i++) {
UrlModel item = tableView.getItems().get(i);
downloadThreads[i] = new DownloadThread(lobject,tableView.getItems().size()-i,item.getLink());
downloadThreads[i].start();
}
Calling lobject.notifyAll() in the run method in Platform.runLater will report an IllegalMonitorStateException. After the address is processed, I want to wake up the next thread to execute.

If you need to execute multiple tasks in order, there's no need to create multiple threads. Just using a single thread will guarantee the next task only executes after the previous one has completed. You should also consider using a CountDownLatch instead of synchronizing on an object.
ExecutorService executor = Executors.newSingleThreadExecutor();
try {
for (UrlModel model : tableView.getItems()) {
executor.submit(() -> {
CountDownLatch latch = new CountDownLatch(1);
Platform.runLater(() -> {
engine.load(model.getLink())
engine.getLoadWorker().runningProperty().addListener((obs, ov, nv) -> {
if (!nv) {
latch.countDown();
}
});
});
latch.await();
// do whatever needs to happen after the WebEngine finishes loading
return null; // using #submit(Callable) and Callable needs to return something
});
}
} finally {
executor.shutdown();
}
Some notes:
You may want to avoid creating the ExecutorService if the table has no items to process. That is, assuming you don't reuse the same ExecutorService every time.
If you reuse the ExecutorService, don't call shutdown().
This ExecutorService uses non-daemon threads. You can customize this by supplying a ThreadFactory that creates daemon threads.
I added a listener to the Worker#running property instead of the status property to make it easier to ensure countDown() is invoked no matter the terminal status of the load (i.e. whether it's SUCCEEDED, CANCELLED or FAILED).
You may want to remove the the listener added to the Worker's property when it's finished. You can do this by using an anonymous class (rather than the lambda expression I used) and calling obs.removeListener(this) inside the changed method, where obs is the ObservableValue argument.

Unit testing asynchronous computation that has to be interrupted manually

I have got a class that records eyetracking data asynchronously. There are methods to start and stop the recording process. The data is collected in a collection and the collection can only be accessed if the recording thread has finished its work. It basically encapsulates all the threading and synchronizing so the user of my library doesn't have to do it.
The heavily shortened code (generics and error handling omitted):
public class Recorder {
private Collection accumulatorCollection;
private Thread recordingThread;
private class RecordingRunnable implements Runnable {
...
public void run() {
while(!Thread.currentThread().isInterrupted()) {
// fetch data and collect it in the accumulator
synchronized(acc) { acc.add(Eyetracker.getData()) }
}
}
}
public void start() {
accumulatorCollection = new Collection();
recordingThread = new Thread(new RecordingRunnable(accumulatorCollection));
recordingThread.start();
}
public void stop() {
recordingThread.interrupt();
}
public void getData() {
try {
recordingThread.join(2000);
if(recordingThread.isAlive()) { throw Exception(); }
}
catch(InterruptedException e) { ... }
synchronized(accumulatorCollection) { return accumulatorCollection; }
}
}
The usage is quite simple:
recorder.start();
...
recorder.stop();
Collection data = recorder.getData();
My problem with the whole thing is how to test it. Currently i am doing it like this:
recorder.start();
Thread.sleep(50);
recorder.stop();
Collection data = recorder.getData();
assert(stuff);
This works, but it is non-deterministic and slows down the test suite quite a bit (i marked these tests as integration tests, so they have to be run separately to circumvent this problem).
Is there a better way?

There is a better way using a CountDownLatch.
The non-deterministic part of the test stems from two variables in time you do not account for:
creating and starting a thread takes time and the thread may not have started executing the runnable when Thread.start() returns (the runnable will get executed, but it may be a bit later).
the stop/interrupt will break the while-loop in the Runnable but not immediately, it may be a bit later.
This is where a CountDownLatch comes in: it gives you precise information about where another thread is in execution. E.g. let the first thread wait on the latch, while the second "counts down" the latch as last statement within a runnable and now the first thread knows that the runnable finished. The CountDownLatch also acts as a synchronizer: whatever the second thread was writing to memory, can now be read by the first thread.
Instead of using an interrupt, you can also use a volatile boolean. Any thread reading the volatile variable is guaranteed to see the last value set by any other thread.
A CountDownLatch can also be given a timeout which is useful for tests that can hang: if you have to wait to long you can abort the whole test (e.g. shutdown executors, interrupt threads) and throw an AssertionError. In the code below I re-used the timeout to wait for a certain amount of data to collect instead of 'sleeping'.
As an optimization, use an Executor (ThreadPool) instead of creating and starting threads. The latter is relative expensive, using an Executor can really make a difference.
Below the updated code, I made it runnable as an application (main method). (edit 28/02/17: check maxCollect > 0 in while-loop)
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class Recorder {
private final ExecutorService executor;
private Thread recordingThread;
private volatile boolean stopRecording;
private CountDownLatch finishedRecording;
private Collection<Object> eyeData;
private int maxCollect;
private final AtomicBoolean started = new AtomicBoolean();
private final AtomicBoolean stopped = new AtomicBoolean();
public Recorder() {
this(null);
}
public Recorder(ExecutorService executor) {
this.executor = executor;
}
public Recorder maxCollect(int max) { maxCollect = max; return this; }
private class RecordingRunnable implements Runnable {
#Override public void run() {
try {
int collected = 0;
while (!stopRecording) {
eyeData.add(EyeTracker.getData());
if (maxCollect > 0 && ++collected >= maxCollect) {
stopRecording = true;
}
}
} finally {
finishedRecording.countDown();
}
}
}
public Recorder start() {
if (!started.compareAndSet(false, true)) {
throw new IllegalStateException("already started");
}
stopRecording = false;
finishedRecording = new CountDownLatch(1);
eyeData = new ArrayList<Object>();
// the RecordingRunnable created below will see the values assigned above ('happens before relationship')
if (executor == null) {
recordingThread = new Thread(new RecordingRunnable());
recordingThread.start();
} else {
executor.execute(new RecordingRunnable());
}
return this;
}
public Collection<Object> getData(long timeout, TimeUnit tunit) {
if (started.get() == false) {
throw new IllegalStateException("start first");
}
if (!stopped.compareAndSet(false, true)) {
throw new IllegalStateException("data already fetched");
}
if (maxCollect <= 0) {
stopRecording = true;
}
boolean recordingStopped = false;
try {
// this establishes a 'happens before relationship'
// all updates to eyeData are now visible in this thread.
recordingStopped = finishedRecording.await(timeout, tunit);
} catch(InterruptedException e) {
throw new RuntimeException("interrupted", e);
} finally {
stopRecording = true;
}
// if recording did not stop, do not return the eyeData (could stil be modified by recording-runnable).
if (!recordingStopped) {
throw new RuntimeException("recording");
}
// only when everything is OK this recorder instance can be re-used
started.set(false);
stopped.set(false);
return eyeData;
}
public static class EyeTracker {
public static Object getData() {
try { Thread.sleep(1); } catch (Exception ignored) {}
return new Object();
}
}
public static void main(String[] args) {
System.out.println("Starting.");
ExecutorService exe = Executors.newSingleThreadExecutor();
try {
Recorder r = new Recorder(exe).maxCollect(50).start();
int dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(100).start();
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(0).start();
Thread.sleep(100);
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
} catch (Exception e) {
e.printStackTrace();
} finally {
exe.shutdownNow();
System.out.println("Done.");
}
}
}
Happy coding :)

Using polling with isDone and Cancel on Java Future instead of blocking get

I am working on some legacy code which I can not refactor immediately.
The code uses blocking Java Future. It uses future.get(withTimeOut,...). This, means that we need to have a decent size thread pool to be responsive enough. As the calls will get blocked till they finish or times out.
Question:
I was thinking to grab the future and put it in a data structure which will be aware of start of execution of a task. Then have a dedicated thread or pool which will loop over the data structure and check if the future.isDone or has exceeded timeout limit. If yes, it can either get the result or cancel the execution. This way not many threads would be required. Will it be a correct implementation or it is not recommended at all?
Thanks in advance.
Edit:
Just to provide more context. These threads are used for logging to a downstream service. We really do not care about the response but we do not want the connection to hung. Thus, we need to grab the future and ensure that it is cancelled or timed out.
Here is a basic simulation which I wrote after asking the question.
#Component
public class PollingService {
private ExecutorService executorService = Executors.newFixedThreadPool(1);
PoorMultiplexer poorMultiplexer = new PoorMultiplexer();
private ConcurrentMap<Integer, Map<Future, Long>> futures = new ConcurrentHashMap<>();
public void startHandler(){
Thread handler = new Thread(new Runnable() {
#Override
public void run() {
while(true){
try {
//This should be handled better. If there is not anything stop and re-start it later.
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
for(Iterator<ConcurrentMap.Entry<Integer, Map<Future, Long>>> it = futures.entrySet().iterator(); it.hasNext();){
ConcurrentMap.Entry<Integer, Map<Future, Long>> entry = it.next();
Map<Future, Long> futureMap = entry.getValue();
boolean isProcessed = false;
if(futureMap.keySet().iterator().next().isDone()){
//mark completed
isProcessed = true;
}
if(futureMap.values().iterator().next() < (300 + System.currentTimeMillis()) && !isProcessed){
//cancel
futureMap.keySet().iterator().next().cancel(true);
isProcessed = true;
}
if(isProcessed){
futures.remove(entry.getKey());
System.out.println("Completed : " + entry.getKey());
}
}
System.out.println("Run completed");
}
}
});
handler.start();
}
public void run(int i) throws InterruptedException, ExecutionException{
System.out.println("Starting : " + i);
poorMultiplexer.send(new Runnable() {
#Override
public void run() {
long startTime = System.currentTimeMillis();
Future future = poorMultiplexer.send(execute());
Map<Future, Long> entry = new HashMap<>();
entry.put(future, startTime);
futures.put(i, entry);
System.out.println("Added : " + i);
}
});
}
public void stop(){
executorService.shutdown();
}
public Runnable execute(){
Worker worker = new Worker();
return worker;
}
}
//This is a placeholder for a framework
class PoorMultiplexer {
private ExecutorService executorService = Executors.newFixedThreadPool(20);
public Future send(Runnable task){
return executorService.submit(task);
}
}
class Worker implements Runnable{
#Override
public void run() {
//service call here
}
}

Asynchronously polling a set of Futures using a separate thread does sound like a reasonable implementation to me. That said, if you're able to add a library dependency, you might find it easier to switch to Guava's ListenableFuture, as Guava provides a wealth of utilities for doing asynchronous work.

Run a new thread and don't wait this thread finish

I am new to the Threading, so if please give me an advice for my case.
I would like create a new thread to do something and I don't care this thread can do complete or not.
I intend to use ExecutorCompletionService to do my job but this class is not suitable for me. It must call take or poll to drain a queue to avoid memory leak. So, this means I must wait until the thread complete. I read this from this question
This is the current code
ExecutorService executor = Executors.newCachedThreadPool();
CompletionService<Entity> completion = new ExecutorCompletionService<>(executor);
DoSomeThingClass doSomething = getInstance();
completion.submit(doSomething);
executor.shutdown();
// Continue to do other job and I don't care whenever doSomeThing is complete.
// However when doSomeThing finish, I don't need to do anything to avoid memory leak
For that reason, please give me an approach for my case and some skeleton code for example.
Thank you so much

You can mark this thread as "Daemon". And when your main thread completed, your app will exit.
public static void main(String[] args)
{
Thread t = new Thread(new Runnable() {
#Override
public void run() {
try {
TimeUnit.SECONDS.sleep(2);
} catch(InterruptedException e) {}
System.out.println("Thread 2 is finished");
}
});
t.setDaemon(true);
t.start();
System.out.println("Thread 1 is finished");
}

You can use Spring TaskExecutor, it is very useful to raise a thread to run a task.
import org.springframework.core.task.TaskExecutor;
public class TaskExecutorExample {
private class MessagePrinterTask implements Runnable {
private String message;
public MessagePrinterTask(String message) {
this.message = message;
}
public void run() {
System.out.println(message);
}
}
private TaskExecutor taskExecutor;
public TaskExecutorExample(TaskExecutor taskExecutor) {
this.taskExecutor = taskExecutor;
}
public void printMessages() {
for(int i = 0; i < 25; i++) {
taskExecutor.execute(new MessagePrinterTask("Message" + i));
}
}
}
You can check Spring Task Execution documentation here:
http://docs.spring.io/spring/docs/3.0.x/spring-framework-reference/html/scheduling.html

Along with you code your Future concept
Future ft=completion.submit(doSomething);
ft.get(timeOut, TimeUnit.MILLISECONDS);
here you can specify Time to execute Thread if it fail to get execute thread get kill(not 100% sure)means it try to interrupt the thread and try to kill

I can resolve my problem as the following code
public static void main(
String[] args) {
ExecutorService executor = Executors.newCachedThreadPool();
executor.execute(new Runnable() {
#Override
public void run() {
try {
TimeUnit.SECONDS.sleep(2);
} catch (InterruptedException e) {
} finally {
System.out.println("Thread 2 is finished");
}
}
});
executor.shutdown();
System.out.println("Thread 1 is finished");
}

Java: notify main class when all threads in threadpool are finished / same instance of object in different threads

How do I notify my main class which instantiates a ThreadPoolExecutor when all threads within the ThreadPoolExecutor are completed?
ThreadPoolExecutor threadPool = null;
ThreadClass threadclass1;
ThreadClass threadclass2;
final ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(maxPoolSize);
puclic MyClass(){
threadPool = new ThreadPoolExecutor(poolSize, maxPoolSize, keepAliveTime, TimeUnit.SECONDS, queue);
threadClass1 = new ThreadClass;
threadClass2 = new ThreadClass;
threadPool.execute(threadClass1);
threadPool.execute(threadClass2);
//Now I would like to do something until the threadPool is done working
//The threads fill a ConcurrentLinkedQueueand I would like to poll
//the queue as it gets filled by the threads and output
//it to XML via JAX-RS
}
EDIT 1
Wile my threads fetch data from somewhere and fill this information into a ConcurrentLinkedQueue I basically would like to perform some action in MyClass to update the XML output with the results. When all threads are terminated I would like to return true to the JAX-RS webservice which instantiated MyClass so the webservice knows all data has been fetched and it can now display the final XML file
EDIT 2
I am passing a Queue to threads so they can add items to the queue. When one driver is done adding items to the articleQueue I want to perform an action within my main class, polling the entity from the Queue and handing it over to the response object to display it in some way.
When I pass the queue to the threads, are they working with the same object or with a "copy" of the object so that changes within the thread do not effect the main object? That is not the behavior I want. When I check the size of the articleQueue within the Driver it is 18, the size of the articleQueue in the DriverController is 0.
Is there a nicer way to react when a thread has added something to the queue other than my while loop? How do I have to modify my code to acces the same object within different classes?
DriverController
public class DriverController {
Queue<Article> articleQueue;
ThreadPoolExecutor threadPool = null;
final ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(
maxPoolSize);
public DriverController(Response response) {
articleQueue = new ConcurrentLinkedQueue<Article>();
threadPool = new ThreadPoolExecutor();
Driver driver = new Driver(this.articleQueue);
threadPool.execute(driver);
// More drivers would be executed here which add to the queue
while (threadPool.getActiveCount() > 0) {
// this.articleQueue.size() gives back 0 here ... why?
if(articleQueue.size()>0){
response.addArticle(articleQueue.poll());
}
}
}
}
Driver
public class Driver implements Runnable{
private Queue<Article> articleQueue;
public DriverAlliedElectronics(Queue articleQueue) {
this.articleQueue = articleQueue;
}
public boolean getData() {
// Here would be the code where the article is created ...
this.articleQueue.offer(article);
return true;
}
public void run() {
this.getData();
// this.articleQueue.size() gives back 18 here ...
}
}

You should try to use following snippet
//Now I would like to wait until the threadPool is done working
threadPool.shutdown();
while (!threadPool.isTerminated()) {
try {
threadPool.awaitTermination(10, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
}

Maybe a ExecutorCompletionService might be the right thing for you:
http://download.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ExecutorCompletionService.html
Example from the link above:
void solve(Executor e, Collection<Callable<Result>> solvers)
throws InterruptedException, ExecutionException {
CompletionService<Result> ecs = new ExecutorCompletionService<Result>(e);
for (Callable<Result> s : solvers)
ecs.submit(s);
int n = solvers.size();
for (int i = 0; i < n; ++i) {
Result r = ecs.take().get();
if (r != null)
use(r);
}
}

Instead of using execute you should use submit. This will return a Future instance on which you can wait for the task(s) to complete. That way you don't need polling or shutting down the pool.

I don't think there's a way to do this explicitly. You could poll the getCompletedTaskCount() to wait for that to become zero.
Why not collect the Future objects returned upon submission and check for all of those being completed ? Simply call get() on each one in turn. Since that call blocks you'll simply wait for each in turn and gradually fall through the set until you've waited on each on.
Alternatively you could submit the threads, and call shutdown() on the executor. That way, the submitted tasks will be executed, and then the terminated() method is called. If you override this then you'll get a callback once all tasks are completed (you couldn't use that executor again, obviously).

Judging from the reference documentation you have a few options:
ThreadPoolExecutor threadPool = null;
ThreadClass threadclass1;
ThreadClass threadclass2;
final ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(maxPoolSize);
puclic MyClass(){
threadPool = new ThreadPoolExecutor(poolSize, maxPoolSize, keepAliveTime, TimeUnit.SECONDS, queue);
threadClass1 = new ThreadClass;
threadClass2 = new ThreadClass;
threadPool.execute(threadClass1);
threadPool.execute(threadClass2);
//Now I would like to wait until the threadPool is done working
//Option 1: shutdown() and awaitTermination()
threadPool.shutDown();
try {
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS)
}
catch (InterruptedException e) {
e.printStackTrace();
}
//Option 2: getActiveCount()
while (threadPool.getActiveCount() > 0) {
try {
Thread.sleep(1000);
}
catch (InterruptedException ignored) {}
}
//Option 3: getCompletedTaskCount()
while (threadPool.getCompletedTaskCount() < totalNumTasks) {
try {
Thread.sleep(1000);
}
catch (InterruptedException ignored) {}
}
}
All things considered, I think shutdown() and awaitTermination() is the best option of the three.

I think you're overengineering things a bit. You don't really care about the threads or the thread pool, and rightly so. Java provides nice abstractions so that you don't have to. You just need to know when your tasks are complete, and methods exist for that. Just submit your jobs, and wait for the futures to say they're done. If you really want to know as soon as a single task completes, you can watch all the futures and take action as soon as any one is finished. If not and you only care that everything is finished, you can remove some complexity from the code I'm about to post. Try this on for size (note MultithreadedJaxrsResource is executable):
import javax.ws.rs.*;
import javax.ws.rs.core.MediaType;
import java.util.*;
import java.util.concurrent.*;
#Path("foo")
public class MultithreadedJaxrsResource {
private ExecutorService executorService;
public MultithreadedJaxrsResource(ExecutorService executorService) {
this.executorService = executorService;
}
#GET
#Produces(MediaType.APPLICATION_XML)
public AllMyArticles getStuff() {
List<Future<Article>> futures = new ArrayList<Future<Article>>();
// Submit all the tasks to run
for (int i = 0; i < 10; i++) {
futures.add(executorService.submit(new Driver(i + 1)));
}
AllMyArticles articles = new AllMyArticles();
// Wait for all tasks to finish
// If you only care that everything is done and not about seeing
// when each one finishes, this outer do/while can go away, and
// you only need a single for loop to wait on each future.
boolean allDone;
do {
allDone = true;
Iterator<Future<Article>> futureIterator = futures.iterator();
while (futureIterator.hasNext()) {
Future<Article> future = futureIterator.next();
if (future.isDone()) {
try {
articles.articles.add(future.get());
futureIterator.remove();
} catch (InterruptedException e) {
// thread was interrupted. don't do that.
throw new IllegalStateException("broken", e);
} catch (ExecutionException e) {
// execution of the Callable failed with an
// exception. check it out.
throw new IllegalStateException("broken", e);
}
} else {
allDone = false;
}
}
} while (!allDone);
return articles;
}
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(10);
AllMyArticles stuff =
new MultithreadedJaxrsResource(executorService).getStuff();
System.out.println(stuff.articles);
executorService.shutdown();
}
}
class Driver implements Callable<Article> {
private int i; // Just to differentiate the instances
public Driver(int i) {
this.i = i;
}
public Article call() {
// Simulate taking some time for each call
try {
Thread.sleep(1000 / i);
} catch (InterruptedException e) {
System.err.println("oops");
}
return new Article(i);
}
}
class AllMyArticles {
public final List<Article> articles = new ArrayList<Article>();
}
class Article {
public final int i;
public Article(int i) {
this.i = i;
}
#Override
public String toString() {
return "Article{" +
"i=" + i +
'}';
}
}
Done that way, you can plainly see that the tasks are returned in the order they complete, as the last task finishes first thanks to sleeping the shortest time. If you don't care about completion order and just want to wait for all to finish, the loop becomes much simpler:
for (Future<Article> future : futures) {
try {
articles.articles.add(future.get());
} catch (InterruptedException e) {
// thread was interrupted. don't do that.
throw new IllegalStateException("broken", e);
} catch (ExecutionException e) {
// execution of the Callable failed with an exception. check it out.
throw new IllegalStateException("broken", e);
}
}