Trying to test that one of my classes can handle being accessed on multiple threads. I have a JUnit test in which I have created a class to implement Runnable and run my class.
When I run execute, it reaches a line in the run() method and just exits on that thread without reporting back any issue, I've wrapped it with Try-Catch(Throwable), but still no sign of what's gone wrong.
Here's my code:
class ConcurrencyTests {
class ConcurrentComponentTracker implements Runnable {
private String component;
ConcurrentComponentTracker(String component) {
this.component = component;
}
#Override
public void run() {
try {
System.out.printf("Component to track: [%s]\n", component);
ParserHandler parserHandler = new ParserHandler();
System.out.println(parserHandler.componentTracker(component));
}
catch (Throwable t) {
t.printStackTrace();
}
}
}
#Test
void runRunner() {
ExecutorService executor = Executors.newFixedThreadPool(4);
String[] componentsToTrack = {"x", "y", "z"};
executor.execute(new ConcurrentComponentTracker(componentsToTrack[0]));
executor.execute(new ConcurrentComponentTracker(componentsToTrack[1]));
executor.execute(new ConcurrentComponentTracker(componentsToTrack[2]));
}
}
And the output:
Component to track: [x]
Component to track: [y]
Component to track: [z]
It just seems to exit on the ParserHandler instantiation line without reporting anything. When attempting to debug and step into that line, it just skips to the end without letting my inspect the ParserHandler class instantiation.
Whomever answered in a comment and then deleted it solved the problem for me. My main JUnit thread wasn't waiting for the other threads to finish. So I tacked on these lines to the end and it worked as expected:
executor.shutdown();
try {
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
}
catch (InterruptedException e) {
e.printStackTrace();
}
JUnit test should actually test and verify result. In this particular case your test produces output which should be verified by the developer/tester, thus this is not automation.
I suggest you to use Future<?> to track status of your tasks execution. For Runnable tasks get() method of Future<?> will return null if task executed successfully, otherwise it will forward you exception wrapped with ExecutionException. Code will look like this:
#Test
void runRunner() {
ExecutorService executor = Executors.newFixedThreadPool(3);
String[] componentsToTrack = {"x", "y", "z"};
assertNull(executor.submit(new ConcurrentComponentTracker(componentsToTrack[0]).get());
assertNull(executor.submit(new ConcurrentComponentTracker(componentsToTrack[1]).get());
assertNull(executor.submit(new ConcurrentComponentTracker(componentsToTrack[2]).get());
// cleanup and shutdown pool
executor.shutdown();
try {
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
// ignore this
}
}
Before doing so, you should remove try/catch block inside your run() method.
Another thing you probably can improve is to make executor a field of your test class and manage it with traditional JUnit lifecycle using #BeforeClass, #Before, ... to initialize, clean between tests, shutdown pool gracefully. Bear in mind that thread pools are not so cheap from resources point of view.
Hope it helps!
P.S. if your goal is parallel processing of your ConcurrentComponentTracker's you can store Future<?> in some collection and run assertions after all tasks will be submitted.
Related
final ExecutorService executor = Executors.newFixedThreadPool(1);
final Future<?> future = executor.submit(myRunnable);
executor.shutdown();
if(executor.awaitTermination(10, TimeUnit.SECONDS)) {
System.out.println("task completed");
}else{
System.out.println("Executor is shutdown now");
}
//MyRunnable method is defined as task which I want to execute in a different thread.
Here is run method of executor class:
public void run() {
try {
Thread.sleep(20 * 1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}}
Here it is waiting for 20 second but when i run the code it throws an exception:
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
I am not able to close the concurrent thread ruining in Java Executor class. Here is my Code flow:
Created a new Thread with Java executor class to run some task i.e written in MyRunnable
executor wait for 10 second to complete the tasks.
If the task has completed then runnable thread also got terminated.
If the task is not completed within 10 second then executor class should terminate the thread.
Everything works fine except the termination of tasks in the last scenario. How should I do it?
The shutDown() method simply prevents additional tasks from being scheduled. Instead, you could call shutDownNow() and check for thread interruption in your Runnable.
// in your Runnable...
if (Thread.interrupted()) {
// Executor has probably asked us to stop
}
An example, based on your code, might be:
final ExecutorService executor = Executors.newFixedThreadPool(1);
executor.submit(new Runnable() {
public void run() {
try {
Thread.sleep(20 * 1000);
} catch (InterruptedException e) {
System.out.println("Interrupted, so exiting.");
}
}
});
if (executor.awaitTermination(10, TimeUnit.SECONDS)) {
System.out.println("task completed");
} else {
System.out.println("Forcing shutdown...");
executor.shutdownNow();
}
It is generally a bad idea to terminate a running thread from the outside, because you don't know the state the thread is currently in. It's possible that it needs to do some cleanups, and it won't be able to do that when you forcefully shut it down. That's why all methods of Thread which do that are marked as deprecated.
It's much better to use one of the many techniques which are available for interprocess communication to signal the procedure running in the thread itself that it has to abort its work and exit normally. One way to do this is to add an abort() method to your runnable, which raises a flag declared as volatile. The inner loop of your Runnable checks that flag and exits (in a controlled fashion) when that flag is raised.
I'm querying an API to create a report and want to load that report 5 minutes later. I want to use a ScheduledExecutorService for that. As I don't want the Executor to block my thread, I'm creating a new thread for that, but am unsure if this is the correct way of doing this. Here's my code:
Thread thread = new Thread() {
public void run() {
log.info("Starting...");
new RequestReport().runScheduledTask(requestId);
}
};
thread.start();
private void runScheduledTask(String requestId) {
log.info("Starting five-minute countdown now...");
ScheduledFuture<?> countdown = scheduler.schedule(() -> {
try {
new GetReportList().run(requestId);
} catch (Exception e) {
e.printStackTrace();
}
}, 5, TimeUnit.MINUTES);
try {
countdown.get();
} catch (InterruptedException | ExecutionException e) {
log.info("catched Exception");
e.printStackTrace();
}
scheduler.shutdown();
}
Is there a better way to run a function 5 minutes after another? Is the way I'm doing this good? What should I change?
BTW, I'm using spring - is there something that could make that better?
ScheduledExecutorService is a good choice, but you use it incorrectly:
First of all, you don't need to create a Thread just to schedule a task from it. It adds nothing to the functionality, only wastes resources.
Second, after calling shutdown() your scheduler will no longer accept tasks, which is bad if you need to generate more than one report.
Third, since your code does nothing after the task completion, you don't need to call get() at all.
So, the only code you need is:
scheduler.schedule(() -> {
try {
new GetReportList().run(requestId);
} catch (Exception e) {
e.printStackTrace();
}
}, 5, TimeUnit.MINUTES);
It will schedule a task and release your thread immediately. The task will be executed five minutes later in a separate thread that is managed by scheduler.
If you need some control on the scheduled tasks (check their states, cancel them etc.), you can obtain Future from schedule() and save it somewhere, but based on the code from your question, you don't need it.
If you'd like to be more 'correct' about the code, you could separate the RequestReport into it's own class implementing the Runnable interface, and pass along an instance of said class to the Thread's constructor
I am using Java executor in the following way, but not sure if every line is necessary and if this is the correct way to use it :
ExecutorService executor=Executors.newFixedThreadPool(30);
...
int N=200;
CountDownLatch doneSignal=new CountDownLatch(N);
for (int i=0;i<N;i++) executor.execute(new Test_Runner(doneSignal,...));
doneSignal.await();
executor.shutdown();
while (!executor.isTerminated()) { Thread.sleep(1000); }
// Blocks until all tasks have completed execution after a shutdown request
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
...
class Test_Runner implements Runnable
{
private CountDownLatch doneSignal;
Thread Test_Runner_Thread;
public Tes_Runner(CountDownLatch doneSignal,...)
{
this.doneSignal=doneSignal;
}
// Define some methods
public void run()
{
try
{
// do some work
}
catch (Exception e)
{
e.printStackTrace();
}
doneSignal.countDown();
}
public void start()
{
if (Test_Runner_Thread==null)
{
Test_Runner_Thread=new Thread(this);
Test_Runner_Thread.setPriority(Thread.NORM_PRIORITY);
Test_Runner_Thread.start();
}
}
public void stop() { if (Test_Runner_Thread!=null) Test_Runner_Thread=null; }
}
Looks correct to me. In the past I have followed the suggested implementation from the Java 7 JavaDoc for ExecutorService for stopping it. You can get it fromt he Java 7 Javadoc but I provide it below for convenience. Edit it to fit your needs, for example you might want to pass the number of seconds to wait. The good thing about using a CountDownLatch is that by the time it is done waiting you know the ExecutorService will terminate right away. Also, you might want to add a timeout to your latch's await if needed in future real world cases. Also, put your latch.countDOwn() in a try's finally block when using in real world application.
void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}
You can further simplify the code.
You can remove CountDownLatch.
Change Test_Runner to Callable task.
Create a ArrayList of Callable Tasks.
List<Test_Runner> callables = new ArrayList<Test_Runner>();
for (int i=0;i<N;i++) {
callables.add(new Test_Runner());
}
Use invokeAll() on executorService.
List<Future<String>> futures = executorService.invokeAll(callables);
From javadocs,
<T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks)
throws InterruptedException
Executes the given tasks, returning a list of Futures holding their status and results when all complete. Future.isDone() is true for each element of the returned list. Note that a completed task could have terminated either normally or by throwing an exception. The results of this method are undefined if the given collection is modified while this operation is in progress.
And you can shutdown executorService as proposed by Jose Martinez
Relate SE question : How to shutdown an ExecutorService?
According to book Java Concurrency in Practice at Listing 12.3 we could test a concurrent code using the following sample code:
void testTakeBlocksWhenEmpty() {
final BoundedBuffer<Integer> bb = new BoundedBuffer<Integer>(10);
Thread taker = new Thread() {
public void run() {
try {
int unused = bb.take();
fail(); // if we get here, it’s an error
} catch (InterruptedException success) { }
}
};
try {
taker.start();
Thread.sleep(LOCKUP_DETECT_TIMEOUT);
taker.interrupt();
taker.join(LOCKUP_DETECT_TIMEOUT);
assertFalse(taker.isAlive());
} catch (Exception unexpected) {
fail();
}
}
Let's say that the following steps are executed:
taker thread started.
bb.take() returned successfully and we are just a little bit before the fail() method run.
It is called the interrupt() method.
We are at the catch block of the taker thread.
So, we are at the catch block at the moment but actually the test method failed. It is failed and we are never informed.
Is this right? If yes how could we fix this?
take is supposed to block on an empty queue. So the expected sequence of events is:
taker.start(); => start the thread
Thread.sleep(LOCKUP_DETECT_TIMEOUT); wait to make sure the thread is started and take has been called. The actual value of the constant is hard to estimate, but anything above a few hundreds of millis should be enough - alternatively you could use a CountDownLatch to know when the taker thread is started
in taker thread: bb.take(); => is supposed to block - if it doesn't fail() is called and the test fails
in main thread: taker.interrupt(); => the take() method is supposed to exit with InterruptedException
in main thread: taker.join(); => wait for some time to allow the taker thread to finish
in main thread: assertFalse(taker.isAlive()); => confirm that the taker thread has exited and is not blocked in the take method any more
Version with a latch (it assumes that if the thread is interrupted before take is called, take will exit with an InterruptedException - if not then you have no other way but to add some random sleep before calling started.await()):
void testTakeBlocksWhenEmpty() {
final CountDownLatch started = new CountDownLatch(1);
final CountDownLatch ended = new CountDownLatch(1);
final BoundedBuffer<Integer> bb = new BoundedBuffer<Integer>(10);
Thread taker = new Thread() {
public void run() {
try {
started.countDown();
int unused = bb.take();
fail(); // if we get here, it’s an error
} catch (InterruptedException success) { }
ended.countDown();
}
};
try {
taker.start();
started.await();
taker.interrupt();
assertTrue(ended.await());
} catch (Exception unexpected) {
fail();
}
}
You should add a timeout to your test method or to the latch (long enough to not interfere if the test passes, for example 5 seconds). That will avoid blocking your whole test suite.
I have the following piece of code:
public class Test {
List<Future> future = new ArrayList<Future>();
public static void main(String args[]) throws Exception {
Adapter b1 = new Adapter();
final ExecutorService threadPool = Executors.newCachedThreadPool();
for(//iterate for number of files) {
while(data exists in file) {
//Call a function to process and update values in db
future.add(threadPool.submit(new Xyz(b1)));
//read next set of data in file;
}
}
try {
for(Future f: future) {
f.get();
}
}
catch(Exception e) {
throw e;
}
}
}
class Xyz implements Runnable {
private Adapter a1;
public Xyz(Adapter al) {
this.a1=a1;
}
#Override
public void run() {
try {
a1.abc();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
When the number of files is 1 (for loop runs for 1 time), the code runs fine.
But, when the number of files increases, the code never returns back from future.get() method.
just out of curiosity.. do i need to shutdown the executor somewhere ??
Yes, and this is likely the problem. Each Future.get() will block until the corresponding task is complete, then once all the tasks are complete your main thread will exit. But your java process will not exit because the thread pool threads are still active in the background. You should shut down the executor once you have finished with it, most likely as the last thing in your main method.
I also note that you're submitting many tasks that wrap the same Adapter instance and all call its abc() method - check that there's nothing in there that will deadlock when called simultaneously in more than one thread.
Your Callable::call / Runable::run does not return. Otherwise the corresponding future would not block.
Additional executor.shutdown or future.cancel will thow an InterruptedException to stop the thread processing the object you submitted but it is up to you if to catch it or not. Your are responsible for making the jobs you submitted stop.
When you submit thousands Callables/Runnables to a CachedExecutor that it might spawn so many threads that your machine gets so slow that you think it takes forever. But you would have noticed that.
When dealing with an undefined number of parallelizable tasks i suggest to use a FixedThreadPool with not much more threads that there are cpu cores.
Edit: Therefore when you set a breakpoints at a1.abc(); and step forward you will probably find out that it never returns.