We Keep Coding

Unit testing asynchronous computation that has to be interrupted manually

I have got a class that records eyetracking data asynchronously. There are methods to start and stop the recording process. The data is collected in a collection and the collection can only be accessed if the recording thread has finished its work. It basically encapsulates all the threading and synchronizing so the user of my library doesn't have to do it.
The heavily shortened code (generics and error handling omitted):
public class Recorder {
private Collection accumulatorCollection;
private Thread recordingThread;
private class RecordingRunnable implements Runnable {
...
public void run() {
while(!Thread.currentThread().isInterrupted()) {
// fetch data and collect it in the accumulator
synchronized(acc) { acc.add(Eyetracker.getData()) }
}
}
}
public void start() {
accumulatorCollection = new Collection();
recordingThread = new Thread(new RecordingRunnable(accumulatorCollection));
recordingThread.start();
}
public void stop() {
recordingThread.interrupt();
}
public void getData() {
try {
recordingThread.join(2000);
if(recordingThread.isAlive()) { throw Exception(); }
}
catch(InterruptedException e) { ... }
synchronized(accumulatorCollection) { return accumulatorCollection; }
}
}
The usage is quite simple:
recorder.start();
...
recorder.stop();
Collection data = recorder.getData();
My problem with the whole thing is how to test it. Currently i am doing it like this:
recorder.start();
Thread.sleep(50);
recorder.stop();
Collection data = recorder.getData();
assert(stuff);
This works, but it is non-deterministic and slows down the test suite quite a bit (i marked these tests as integration tests, so they have to be run separately to circumvent this problem).
Is there a better way?

There is a better way using a CountDownLatch.
The non-deterministic part of the test stems from two variables in time you do not account for:
creating and starting a thread takes time and the thread may not have started executing the runnable when Thread.start() returns (the runnable will get executed, but it may be a bit later).
the stop/interrupt will break the while-loop in the Runnable but not immediately, it may be a bit later.
This is where a CountDownLatch comes in: it gives you precise information about where another thread is in execution. E.g. let the first thread wait on the latch, while the second "counts down" the latch as last statement within a runnable and now the first thread knows that the runnable finished. The CountDownLatch also acts as a synchronizer: whatever the second thread was writing to memory, can now be read by the first thread.
Instead of using an interrupt, you can also use a volatile boolean. Any thread reading the volatile variable is guaranteed to see the last value set by any other thread.
A CountDownLatch can also be given a timeout which is useful for tests that can hang: if you have to wait to long you can abort the whole test (e.g. shutdown executors, interrupt threads) and throw an AssertionError. In the code below I re-used the timeout to wait for a certain amount of data to collect instead of 'sleeping'.
As an optimization, use an Executor (ThreadPool) instead of creating and starting threads. The latter is relative expensive, using an Executor can really make a difference.
Below the updated code, I made it runnable as an application (main method). (edit 28/02/17: check maxCollect > 0 in while-loop)
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class Recorder {
private final ExecutorService executor;
private Thread recordingThread;
private volatile boolean stopRecording;
private CountDownLatch finishedRecording;
private Collection<Object> eyeData;
private int maxCollect;
private final AtomicBoolean started = new AtomicBoolean();
private final AtomicBoolean stopped = new AtomicBoolean();
public Recorder() {
this(null);
}
public Recorder(ExecutorService executor) {
this.executor = executor;
}
public Recorder maxCollect(int max) { maxCollect = max; return this; }
private class RecordingRunnable implements Runnable {
#Override public void run() {
try {
int collected = 0;
while (!stopRecording) {
eyeData.add(EyeTracker.getData());
if (maxCollect > 0 && ++collected >= maxCollect) {
stopRecording = true;
}
}
} finally {
finishedRecording.countDown();
}
}
}
public Recorder start() {
if (!started.compareAndSet(false, true)) {
throw new IllegalStateException("already started");
}
stopRecording = false;
finishedRecording = new CountDownLatch(1);
eyeData = new ArrayList<Object>();
// the RecordingRunnable created below will see the values assigned above ('happens before relationship')
if (executor == null) {
recordingThread = new Thread(new RecordingRunnable());
recordingThread.start();
} else {
executor.execute(new RecordingRunnable());
}
return this;
}
public Collection<Object> getData(long timeout, TimeUnit tunit) {
if (started.get() == false) {
throw new IllegalStateException("start first");
}
if (!stopped.compareAndSet(false, true)) {
throw new IllegalStateException("data already fetched");
}
if (maxCollect <= 0) {
stopRecording = true;
}
boolean recordingStopped = false;
try {
// this establishes a 'happens before relationship'
// all updates to eyeData are now visible in this thread.
recordingStopped = finishedRecording.await(timeout, tunit);
} catch(InterruptedException e) {
throw new RuntimeException("interrupted", e);
} finally {
stopRecording = true;
}
// if recording did not stop, do not return the eyeData (could stil be modified by recording-runnable).
if (!recordingStopped) {
throw new RuntimeException("recording");
}
// only when everything is OK this recorder instance can be re-used
started.set(false);
stopped.set(false);
return eyeData;
}
public static class EyeTracker {
public static Object getData() {
try { Thread.sleep(1); } catch (Exception ignored) {}
return new Object();
}
}
public static void main(String[] args) {
System.out.println("Starting.");
ExecutorService exe = Executors.newSingleThreadExecutor();
try {
Recorder r = new Recorder(exe).maxCollect(50).start();
int dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(100).start();
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(0).start();
Thread.sleep(100);
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
} catch (Exception e) {
e.printStackTrace();
} finally {
exe.shutdownNow();
System.out.println("Done.");
}
}
}
Happy coding :)

Java Multithreading doesn't seem to be correctly working

I have a class which processes something. I'm trying to run a number of instances of this class in parallel.
However, I'm not sure if in TaskManager.startAll(), when I call r.go(), whether this would cause r to start running in its own thread, or within the main thread?
The total execution time that I'm getting seems to be very high, and despite my attempts at optimizing, nothing seems to be having any effect. Also, if I run a profiler on my project in Netbeans, it shows all the threads as sleeping. So I'd like to know if I'm doing something wrong?
This is the structure of the class:
public class TaskRunner implements Runnable {
private boolean isRunning = false;
public void run() {
while(true) {
while (! running) {
try {
Thread.sleep(1);
} catch (Exception e) {
e.printStackTrace();
}
}
process();
}
}
public void go() {
isRunning = true;
}
public void stop() {
isRunning = false;
}
private void process() {
//Do some number crunching and processing here
}
}
Here's how these are being run / managed:
public class TaskManager {
private ArrayList<TaskRunner> runners = new ArrayList<>();
public TaskManager() {
for (int i = 0; i < 10; i++) {
TaskRunner r = new TaskRunner();
new Thread(r).start();
runners.add(r);
}
}
public void startAll() {
for (TaskRunner r : runners) {
r.go();
}
}
}

Indeed, you are not "doing it right." If you want to create a multi-threaded Java application, the place to start is with the java.util.concurrent package.
It appears from your code that you want to run ten tasks in parallel. I assume that after "number crunching and processing," you'll want to aggregate the results and do something with them in the main thread. For this, the invokeAll() method of ExecutorService works well.
First, implement Callable to do the work you show in your process() method.
final class YourTask implements Callable<YourResults> {
private final YourInput input;
YourTask(YourInput input) {
this.input = input;
}
#Override
public YourResults call()
throws Exception
{
/* Do some number crunching and processing here. */
return new YourResults(...);
}
}
Then create your tasks and run them. This would take the place of your main() method:
Collection<Callable<YourResults>> tasks = new List<>(inputs.size());
for (YourInput i : inputs)
tasks.add(new YourTask(i));
ExecutorService workers = Executors.newFixedThreadPool(10);
/* The next call blocks while the worker threads complete all tasks. */
List<Future<YourResult>> results = workers.invokeAll(tasks);
workers.shutdown();
for (Future<YourResult> f : results) {
YourResult r = f.get();
/* Do whatever it is you do with the results. */
...
}

However, I'm not sure if in TaskManager.startAll(), when I call r.go(), whether this would cause r to start running in its own thread, or within the main thread?
So my first comment is that you should make isRunning be volatile since it is being shared between threads. If the threads are not starting when it goes to true (or seem to be delayed in starting) then I suspect that's your problem. volatile provides memory synchronization between the threads so the thread that calls go() and makes a change to isRunning will be seen immediately by the thread waiting for the change.
Instead of spinning like this, I would use wait/notify:
// this synchronizes on the instance of `TaskRunner`
synchronized (this) {
// always do your wait in a while loop to protect against spurious wakeups
while (!isRunning && !Thread.currentThread().isInterrupted()) {
try {
// wait until the notify is called on this object
this.wait();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
e.printStackTrace();
}
}
Then in the go() method you should do the following. stop() would be similar.
public void go() {
synchronized (this) {
isRunning = true;
this.notifyAll();
}
}
Notice that you should handle thread interrupts carefully. Test for isInterrupted() in the while running loop and re-interrupt a thread when InterruptedException is thrown is always a good pattern.
The total execution time that I'm getting seems to be very high, and despite my attempts at optimizing, nothing seems to be having any effect. Also, if I run a profiler on my project in Netbeans, it shows all the threads as sleeping.
So although the threads are mostly sleeping, they are still each looping 1000 times a second because of your Thread.sleep(1). If you increased the time sleeping (after making isRunning be volatile) they would loop less but the right mechanism is to use the wait/notify to signal the thread.

Awful solution, terrible. first I highly recommend you start reading some tutorial like [this]
Second, if threads should wait for a signal to go for some job, so why just don't you wait them!!!!!, something like this
import java.util.ArrayList;
public class TaskManager
{
//////////////////////
public volatile static Signal wait=new Signal();
//////////////////////
private ArrayList<TaskRunner> runners = new ArrayList<>();
public TaskManager()
{
for (int i = 0; i < 10; i++)
{
TaskRunner r = new TaskRunner();
new Thread(r).start();
runners.add(r);
}
try {
Thread.sleep(1000);
startAll();
Thread.sleep(1000);
pauseAll();
Thread.sleep(1000);
startAll();
Thread.sleep(1000);
haltAll();System.out.println("DONE!");
}catch(Exception ex){}
}
public void startAll()
{
synchronized(wait){
wait.setRun(true);;
wait.notifyAll();
}
}
public void pauseAll(){
wait.setRun(false);
}
public void haltAll(){
for(TaskRunner tx:runners){tx.halt();}
}
public static void main(String[] args) {
new TaskManager();
}
}
class TaskRunner implements Runnable
{
private Thread thisThread;
private volatile boolean run=true;
public void run()
{
thisThread=Thread.currentThread();
while(run){
if(!TaskManager.wait.isRun()){
synchronized(TaskManager.wait)
{
if(!TaskManager.wait.isRun()){
System.out.println("Wait!...");
try
{
TaskManager.wait.wait();
}
catch (Exception e)
{
e.printStackTrace();
break;
}
}
}}
process();
}
}
private double r=Math.random();
private void process(){System.out.println(r);try {
Thread.sleep(10);
} catch (Exception e) {
// TODO: handle exception
}}
public void halt(){run=false;thisThread.interrupt();}
}
class Signal{
private boolean run=false;
public boolean isRun() {
return run;
}
public void setRun(boolean run) {
this.run = run;
}
}
in above sample, all runners works till the Signal run boolean is true, and simple TaskManager class set tit as false for every time it needs to pause the threads. and about the halt, it just set the shutdown(run) flag to false, and also interrupt the thread because of if thread is in wait state.
I hope I could prove your solution is like dream-on story, and also could explained enough about my solution.
have a good parallel application :)

Java code speed improvement

Hi all may such a code cause lag of process?
while(true)
if(!connected) break;
As we see it checks for condition all the time. Would it be faster if I set small sleep in each iteration.
The code runs on Android GingerBread.
EDIT:
It waits for other Thread to finish and set variable Connected to false. Variable connected is used like lock. thread usually finds it true and waits to be false. then sets to true and only changes it at the end.

I think it would be better to add small sleep in your loop, to free processor resources for other processes, especially on one-core processor.

I belive it could be like this:
while(connected) {
// do stuff...

Try something like this:
private Integer connected;
private ConnectedListener connectedListener;
public interface ConnectedListener {
public void onDisconnected();
}
private void startThreads() {
for (int i = 0; i < 10; i++) {
new Thread(new Runnable() {
#Override
public void run() {
synchronized (connected) {
connected++;
}
// do some random long work
synchronized (connected) {
connected--;
if (connected == 0 && connectedListener != null) {
//let the listener know that we are completely disconnected
connectedListener.onDisconnected();
}
}
}
}).start();
}
}

Runnable/Thread possible misconception

Not sure I understand threads correctly, could someone tell me whether I´m right or wrong in the following example:
class Task {
String taskName;
private Thread thread;
boolean isFinished;
public Task(String name){
taskName = name;
}
public void createTask(final Runnable r) {
thread = new Thread(r){
public void run(){
if(r != null) {
r.run();
isFinished = true;
}
}
};
thread.start();
}
}
What I'm actually doing in my app is that I set the isFinished to true, and have an observer that whenever the isFinished is true does some stuff. I'm afraid that isFinished is set to true, before all code in the Runnable I pass as parameter is actually terminated.
Isn't the run method suppose to put the code I pass, in a separate thread and run that code asynchronously?

Close, but your new thread is already given the runnable object to execute. You really want to give it a wrapper which runs the r.run() method and then sets isFinished.
Change:
public void createTask(final Runnable r) {
thread = new Thread(r){
public void run(){
if(r != null) {
r.run();
isFinished = true;
}
}
};
thread.start();
}
to
public void createTask(final Runnable r) {
thread = new Thread( new Runnable {
public void run(){
if(r != null) {
r.run();
isFinished = true;
}
}
});
thread.start();
}
I would be remiss if I didn't point out the thread-unsafetiness of isFinished. You will not be guaranteed to notice when the thread finishes, without adding synchronization. I recommend you add:
public synchronized boolean getIsFinished()
{
return isFinished;
}
public synchronized void setIsFinished(boolean finished)
{
isFinished = finished;
}
And use these methods to get or set the isFinished flag.
Given your lack of synchronization here, you may be seeing other thread-safety oddities, depending on whether or not your r.run() method and your other "observer" are sharing data without synchronization as well.

You should almost never pass a Runnable into the constructor of a Thread and override the Thread's run() method.
The following two pieces of code are essentially identical:
Runnable r = new Runnable( )
{
public void run( )
{
// do stuff...
}
};
new Thread( r ).start( );
An here's another way to accomplish the same thing by overriding run():
(new Thread( )
{
public void run( )
{
// do stuff...
}
}).start( );

The way you wrote your code, isFinished will not be set to true until r.run() is complete. It may be appearing otherwise because you may have some data visibility issues due to missing synchronization or missing volatile declarations.
It's a little bit odd since you're both passing in the Runnable to the constructor, but calling it using the reference from your method declaration, not the one inside the thread. But it "works", there's just a redundancy there.
As an aside, don't forget #Override in your anonymous class :)

No, the run method simply is a normal function, that you can override when extending the Thread class in order to implement your own behaviour.
It's the start method of the Thread class that starts a new thread and runs that code async.

Well your code is partially right, and partially wrong.
You are correct that isFinished will only be set to true once everything inside the runnable you are passing in the parameter has finished executing.
However, due to the particular semantics of the java memory model (I'll get into more details about that below), it is possible that when you set isFinished to true, that change is only visible to the thread that has set that variable to true. If you want your code to work as expected, you need to declare isFinished as volatile. This will make any changes you make to that variable immediately visible by other threads.
Another way to do it is to declare isFinished as AtomicBoolean rather than boolean. This class has many methods that allow you to check and set the boolean in an atomic way, helping you to avoid many common multithreading pitfalls.

I suggest you use the synchronization primitive specifically designed for your problem.
This primitive is called CountDownLatch.
Here is the updated code:
class Task {
String taskName;
private Thread thread;
CountDownLatch finishedSignal = new CountDownLatch( 1 );
public Task(String name){
taskName = name;
}
public void createTask(final Runnable r) {
thread = new Thread(r){
public void run(){
if(r != null) {
r.run();
finishedSignal.countDown( );
}
}
};
thread.start();
finishedSignal.await( );
}
}

You should use a FutureTask instead of your own Task class. It has an isDone() method, and it is integrated nicely with the Executor framework.
Most importantly the happens-before relationships are maintained as you expect it (actually in your code the problem is not that isFinished is set to true, before all code in the Runnable is terminated, but the other way: possibly it will not be set to true in the original thread even if the Runnable is terminated)
Example:
Runnable runnable = new Runnable() {
#Override
public void run() {
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("work done");
}
};
FutureTask<Void> task = new FutureTask<Void>(runnable, null);
ExecutorService es = Executors.newSingleThreadExecutor();
es.submit (task);
while (!task.isDone()) {
System.out.println("waiting...");
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}

Java: NotifyObserver and DeadLock in thread

I have a problem with Observer-Pattern and deadlock using threads.
package observerDeadLock;
import java.util.Observable;
public class MyModel extends Observable {
Integer foo;
public MyModel() {
foo = 0;
}
public void changeStatus(Integer newStatus) {
foo = newStatus;
notifyObservers(newStatus);
}
}
package observerDeadLock;
public class Job extends Thread {
public MyModel model;
public Job(MyModel model) {
super();
this.model = model;
}
public void run() {
prepareJob();
runMyJob();
}
private void runMyJob() {
// Some stuff
Integer choice = 1;
if (choice == 3) {
return;
}
else if (choice == 2) {
return;
}
else if (choice == 1) {
model.changeStatus(123); // Set a particalar status that MyController receive as wrong!
// PROBLEM: The Controller listen the changeStatus(123) of notifyObserver of MyModel and call a join() because I want the thread join and quit()
return; // <<- NEVER EXECUTED! join(timeout) isn't the solution IHMO...s
}
return;
}
private void prepareJob() {
// Do some stuff
}
}
package observerDeadLock;
import java.util.Observable;
import java.util.Observer;
public class MyController implements Observer {
private Job myJob;
private MyModel model;
public MyController() {
}
public void startJob() {
model = new MyModel();
model.addObserver(this);
myJob = new Job(model);
myJob.start();
}
#Override
public void update(Observable o, Object arg) {
if (o instanceof MyModel) {
if (((Integer) arg) == 1) {
// do other stuff
}
else if (((Integer) arg) == 2) {
// do other stuff
}
else if (((Integer) arg) == 123) { // 123 => Wrong state for myController, so must stop Job
// Stop myJob!!!
try {
//myJob.join(timeout); // isn' the solution IHMO
myJob.join(); // PROBLEM HERE!!! In job, the "return" statment is locked in changeStatus() -> myModel.notifyobserver() that lock here in JOIN();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
What I can do to resolve that problem? I suppose that use Thread.join() than Thread.interrupt() is a bad practice...
Suggestion?
Thanks!

You're apparently trying to join() the current thread (you can see this for yourself if you replace myJob.join() with System.out.println( Thread.currentThread() )), which isn't a very good idea, as the thread will be stuck forever, or until someone else interrupts it from the outside.
Instead of join(), just call interrupt(), there's nothing shameful about that. (Although whether it will have any effect depends on what else is in runMyJob())
Update: I can only assume that you edited out parts of your code for brevity, but if MyController really does nothing else if it encounters the exit value 123 than to stop the updating thread, then the whole construction is unnecessary. You should simply return from the runMyJob() method without setting anything and the thread will stop gracefully.

It looks like you have some logic errors here:
In runMyJob() you are calling model.changeStatus(123)
That calls notifyObservers() in Observable
That notifies the observers by calling MyController.update()
That then tries to join() on the thread.
Since the thread is the one that is calling the update() then it is trying to join with itself which obviously won't work. I'd have the main thread just do a join after the start:
myJob.start();
myJob.join();
You could also set some sort of value during the update (maybe a volatile int or AtomicInteger) that then can be read by the main thread.
If you edit your question to let us know what you are trying to accomplish, I'll adjust my answer to provide some better recommendations.