We Keep Coding

ReentrantLock as replacement for CountdownLatch

I have myCountDownLatch (which works as expected):
public static void myCountDownLatch() {
CountDownLatch countDownLatch = new CountDownLatch(1);
Thread t = new Thread(() ->
{
try {
log.info("CountDownLatch: in thread..");
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
countDownLatch.countDown();
});
t.start();
try {
countDownLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
log.info("CountDownLatch: out thread..");
}
I am trying to understand the difference of CountdownLatch and ReentrantLock and tried to rewrite myCountDownLatch by using ReentrantLock instead of CountdownLatch:
public static void myRentrantLock() {
ReentrantLock lock = new ReentrantLock();
Condition condition = lock.newCondition();
Thread t = new Thread(() ->
{
try {
log.info("ReentrantLock: in thread..");
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
});
lock.lock();
t.start();
lock.unlock();
log.info("ReentrantLock: out thread..");
}
I only want to stop the main thread as long as Thread t is not finished by using ReentrantLock instead of CountDownLatch.
However, myRentrantLock does not behave equal to my myCountDownLatch. Why?

You can not replace a countdown latch with a ReentrantLock, which is a tool for mutual exclusion and notification, but you could use a ReentrantLock to implement a similar functionality.
It may look like
public class MyLatch {
final ReentrantLock lock = new ReentrantLock();
final Condition zeroReached = lock.newCondition();
int remaining;
MyLatch(int count) {
if(count < 0) throw new IllegalArgumentException();
remaining = count;
}
public void await() throws InterruptedException {
lock.lock();
try {
while(remaining != 0) zeroReached.await();
}
finally {
lock.unlock();
}
}
public boolean await(long timeout, TimeUnit unit) throws InterruptedException {
lock.lock();
try {
if(remaining == 0) return true;
long deadLine = System.nanoTime() + unit.toNanos(timeout);
while(remaining != 0) {
final long remainingTime = deadLine - System.nanoTime();
if(remainingTime <= 0) return false;
zeroReached.await(remainingTime, TimeUnit.NANOSECONDS);
}
return true;
}
finally {
lock.unlock();
}
}
public void countDown() {
lock.lock();
try {
if(remaining > 0 && --remaining == 0) zeroReached.signalAll();
}
finally {
lock.unlock();
}
}
public long getCount() {
lock.lock();
try {
return remaining;
}
finally {
lock.unlock();
}
}
}
The ReentrantLock guards the internal state, which is the remaining field. The associated Condition zeroReached is used to allow threads waiting for the remaining field to become zero.
This can be used the same way as the builtin CountDownLatch:
public class MyLatchTest {
public static void main(String[] args) {
int num = 10;
MyLatch countDownLatch = new MyLatch(num);
for(int i = 0; i < num; i++) {
Thread t = new Thread(() ->
{
try {
System.out.println("CountDownLatch: in thread..");
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("CountDownLatch: one thread finished..");
countDownLatch.countDown();
});
t.start();
}
try {
countDownLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("CountDownLatch: out thread..");
}
}
Note that you don’t need an explicit Lock here, Java’s intrinsic locking feature would work as well:
public class MyLatch {
int remaining;
MyLatch(int count) {
if(count < 0) throw new IllegalArgumentException();
remaining = count;
}
public synchronized void await() throws InterruptedException {
while(remaining != 0) wait();
}
public synchronized boolean await(long timeout, TimeUnit unit) throws InterruptedException {
if(remaining == 0) return true;
long deadLine = System.nanoTime() + unit.toNanos(timeout);
while(remaining != 0) {
long remainingTime = deadLine - System.nanoTime();
if(remainingTime <= 0) return false;
wait(remainingTime / 1_000_000, (int)(remainingTime % 1_000_000));
}
return true;
}
public synchronized void countDown() {
if(remaining > 0 && --remaining == 0) notifyAll();
}
public synchronized long getCount() {
return remaining;
}
}
But in either case, the builtin CountDownLatch is more efficient…

Monitor in my java program gets into a deadlock

I'm trying to solve single consumer/producer problem using monitor in Java, and the code is as follows. When I run this code, it will finally get stucked. The most typical case is that the consumer calls wait(), and then the producer keeps producing but cannot notify the consumer (although it will call notify()). I don't know why it's happening. Java code:
import java.util.*;
class Monitor {
int length;
int size;
int begin, end;
int queue[];
private static Random randGenerator;
public Monitor() {}
public Monitor(int length) {
this.length = length;
this.size = 0;
begin = end = 0;
queue = new int[length];
randGenerator = new Random(10);
}
public synchronized void produce() throws InterruptedException {
while(size == length) {
System.out.println("Producer waiting");
wait();
}
int produced = randGenerator.nextInt();
size++;
queue[end] = produced;
end = (end + 1) % length;
System.out.println("Produce element " + produced + " size "+size);
// When size is not 1, no thread is blocked and therefore don't need to notify
if(size == 1) {
System.out.println("Notify consumer");
notify();
}
}
public synchronized void consume() throws InterruptedException {
while(size == 0) {
System.out.println("Consumer waiting, size " + size);
wait();
}
size--;
System.out.println("Consume element " + queue[begin] + " size " + size);
begin = (begin + 1) % length;
if(size == length - 1) {
System.out.println("Notify producer");
notify();
}
}
}
class Producer implements Runnable {
Monitor producer;
public Producer(Monitor m) {
producer = m;
}
#Override
public void run() {
producer = new Monitor();
System.out.println("Producer created");
try {
while(true) {
producer.produce();
}
}
catch (InterruptedException e) {
e.printStackTrace();
}
}
}
class Consumer implements Runnable {
Monitor consumer;
public Consumer(Monitor m) {
consumer = m;
}
#Override
public void run() {
System.out.println("Consumer created");
consumer = new Monitor();
try {
while(true) {
consumer.consume();
}
}
catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public class monitorTest {
public static void main(String args[]) {
Monitor monitor = new Monitor(10);
Thread t1 = new Thread(new Producer(monitor));
Thread t2 = new Thread(new Consumer(monitor));
t1.start();
t2.start();
}
}

When the control of each thread enters the produce() or consume() methods, the size and length are both zero and hence both threads are waiting for the other to notify. Break this and your code will come out of the deadlock.
public synchronized void produce() throws InterruptedException {
while(size == length) { // size is 0 and length is 0; so wait
System.out.println("Producer waiting");
wait();
}
public synchronized void consume() throws InterruptedException {
while(size == 0) { // size is 0 so wait
System.out.println("Consumer waiting, size " + size);
wait();
}
This is happening because you have a default constructor which you are calling inside the run() method of your Producer and Consumer objects.
class Producer implements Runnable {
Monitor producer;
public Producer(Monitor m) {
producer = m;
}
#Override
public void run() {
producer = new Monitor(); // REMOVE THIS
class Consumer implements Runnable {
Monitor consumer;
public Consumer(Monitor m) {
consumer = m;
}
#Override
public void run() {
System.out.println("Consumer created");
consumer = new Monitor(); // AND REMOVE THIS
Hope this helps!

Safe thread utilization

I am using single thread executor for long-running threads like this:
executor = Executors.newSingleThreadExecutor(THREAD_FACTORY);
executor.submit(new LongRunnable());
which checks a flag to be stopped:
private class LongRunnable implements Runnable {
#Override
public void run() {
while (isRunning.get()) {
try {
doSomething();
} catch (InterruptedException e) {
...
}
}
}
}
and whole execution is interrupted that way:
#Override
public void close() throws Exception {
isRunning.set(false);
executor.shutdownNow();
}
Still I can see some threads not gc-ed in profiler (while by logs, runnable they were executing has quit outermost while loop).
Question: does provided working with threads strategy memory-leak-free and thread-leak-free?

I am not able to see any issue with executor or shutDownNow. Probably you are looking at different threads in your profiler.
Try this program which is similar to the one in your question and you can see the thread is no longer there after successful shutdown.
public class ExecutorShutdownTest {
private static ExecutorService executor;
private static AtomicLong executorThreadId = new AtomicLong(0);
public static void main(String[] args) {
// get thread MX bean
ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();
// create an executor and start the task
executor = Executors.newSingleThreadExecutor(new TestThreadFactory());
LongRunnable runnable = new LongRunnable();
executor.submit(runnable);
// main thread: keep running for sometime
int count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace(
"\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// main thread: stop the task
try {
runnable.close();
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (Exception e) {
e.printStackTrace();
}
// main thread: run some more time to verify the executor thread no longer exists
count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private static class LongRunnable implements Runnable {
private volatile boolean isRunning = true;
#Override
public void run() {
while (isRunning) {
System.out.println("Running");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
//ignore
}
}
System.out.println("Stopped");
}
public void close() throws Exception {
System.out.println("Stopping");
isRunning = false;
executor.shutdownNow();
}
}
private static class TestThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String namePrefix;
TestThreadFactory() {
SecurityManager s = System.getSecurityManager();
group = (s != null) ? s.getThreadGroup() : Thread.currentThread().getThreadGroup();
namePrefix = "pool-" + poolNumber.getAndIncrement() + "-thread-";
}
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0) {
#Override protected void finalize() throws Throwable {
super.finalize();
// probably bad idea but lets see if it gets here
System.out.println("Executor thread removed from JVM");
}
};
if (t.isDaemon())
t.setDaemon(false);
if (t.getPriority() != Thread.NORM_PRIORITY)
t.setPriority(Thread.NORM_PRIORITY);
executorThreadId.set(t.getId());
System.out.println("Executor thread created");
return t;
}
}
}

Here's a sample program using the single-thread Executor that manages to strand a thread so that the JVM can't shut down, but it only manages to do it by not calling shutdownNow:
import java.util.concurrent.*;
public class Exec {
public static void main(String[] args) throws Exception {
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(new MyTask());
Thread.sleep(20000L);
// executor.shutdownNow();
int retryCount = 4;
while (!executor.isTerminated() && retryCount > 0) {
System.out.println("waiting for tasks to terminate");
Thread.sleep(500L);
retryCount -= 1;
}
}
}
class MyTask implements Runnable {
public void run() {
int count = 0;
try {
while (!Thread.currentThread().isInterrupted() && count < 10) {
Thread.sleep(1000L);
count += 1;
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("all done");
}
}
The thread used by the executor has a separate life cycle from the task, this example shows how the task finishes but the thread goes on. Uncommenting the shutdownNow results in the executor's thread terminating. Otherwise the main thread sleeps for a while and exits, leaving the executor's thread hanging out, preventing the JVM from exiting.
My guess is that your close method isn't getting called and your executor never gets shut down. To get more useful answers please add a MVCE so that we can reproduce the problem.
Consider that with interruption there's no need to keep a reference to the Runnable to set the flag. As I read the question the task not finishing is not an issue here, but it would still be better to make the Runnable respond to interruption and lose the flag, just because having less things to keep track of is always an improvement.

Tracking Executing Threads

I am trying to figure out how I can track all the threads that my application is spawning. Initially, I thought I had it figured out using a CyclicBarrier, however I am seeing threads executing after my await call.
Below is the working pseudo code:
public class ThreadTesterRunner {
public static void main(String[] args) throws InterruptedException {
final CyclicBarrier cb = new CyclicBarrier(1);
ThreadRunner tr = new ThreadRunner(cb);
Thread t = new Thread(tr, "Thread Runner");
t.start();
boolean process = true;
// wait until all threads process, then print reports
while (process){
if(tr.getIsFinished()){
System.out.println("Print metrics");
process = false;
}
Thread.sleep(1000);
}
}
}
class ThreadRunner implements Runnable {
static int timeOutTime = 2;
private ExecutorService executorService = Executors.newFixedThreadPool(10);
private final CyclicBarrier barrier;
private boolean isFinished=false;
public ThreadRunner(CyclicBarrier cb) {
this.barrier = cb;
}
public void run(){
try {
boolean stillLoop = true; int i = 0;
while (stillLoop){
int size;
Future<Integer> future = null;
try {
future = executorService.submit(new Reader()); // sleeps
size = future.get();
} catch (InterruptedException | ExecutionException ex) {
// handle Errs
}
if(i == 3){
stillLoop = false;
this.barrier.await();
this.isFinished=true;
}
//System.out.println("i = "+i+" Size is: "+size+"\r");
i++;
}
} catch (InterruptedException | BrokenBarrierException e1) {
e1.printStackTrace();
}
}
public boolean getIsFinished(){
return this.isFinished;
}
}
class Reader implements Callable {
private ExecutorService executorService = Executors.newFixedThreadPool(1);
#Override
public Object call() throws Exception {
System.out.println("Reading...");
Thread.sleep(2000);
executorService.submit(new Writer());
return 1000;
}
}
class Writer implements Callable {
#Override
public Void call() throws Exception {
Thread.sleep(4000);
System.out.println("Wrote");
return null;
}
}
Can anyone suggest a way to ONLY print "print metrics" after all threads have run?

It doesn't seem like you're doing anything to coordinate with your Reader and Writer threads, which are the ones you want to wait for. If you pass your synchronization barrier through to those threads so that they can register and signal when they are done, it works just fine.
Here's a version rewritten to do so, using a Phaser instead of a CyclicBarrier. Note that each Reader and Writer registers itself upon construction, and notifies the synchronization barrier when it is done executing:
public class ThreadTesterRunner {
public static void main(String[] args) throws InterruptedException {
final Phaser cb = new Phaser();
ThreadRunner tr = new ThreadRunner(cb);
Thread t = new Thread(tr, "Thread Runner");
t.start();
boolean process = true;
// wait until all threads process, then print reports
while (process){
if(tr.getIsFinished()){
System.out.println("Print metrics");
process = false;
}
//else {
// System.out.println("Waiting: registered=" + cb.getRegisteredParties() + ", arrived=" + cb.getArrivedParties() + ", unarrived=" + cb.getUnarrivedParties());
//}
Thread.sleep(1000);
}
}
}
class ThreadRunner implements Runnable {
static int timeOutTime = 2;
private ExecutorService executorService = Executors.newFixedThreadPool(10);
private final Phaser barrier;
private boolean isFinished=false;
public ThreadRunner(Phaser phaser) {
this.barrier = phaser;
}
public void run(){
try {
boolean stillLoop = true; int i = 0;
while (stillLoop){
int size;
Future<Integer> future = null;
try {
future = executorService.submit(new Reader(this.barrier)); // sleeps
size = future.get();
} catch (InterruptedException | ExecutionException ex) {
// handle Errs
}
if(i == 3){
stillLoop = false;
this.barrier.awaitAdvance(0);
this.isFinished=true;
}
//System.out.println("i = "+i+" Size is: "+size+"\r");
i++;
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
public boolean getIsFinished(){
return this.isFinished;
}
}
class Reader implements Callable {
private Phaser barrier;
private ExecutorService executorService = Executors.newFixedThreadPool(1);
public Reader(Phaser phase) {
phase.register();
this.barrier = phase;
}
#Override
public Object call() throws Exception {
System.out.println("Reading...");
Thread.sleep(2000);
executorService.submit(new Writer(this.barrier));
this.barrier.arrive();
return 1000;
}
}
class Writer implements Callable {
private Phaser barrier;
public Writer(Phaser phase) {
phase.register();
this.barrier = phase;
}
#Override
public Void call() throws Exception {
Thread.sleep(4000);
System.out.println("Wrote");
this.barrier.arrive();
return null;
}
}

From what I can see you aren't waiting for the Writer to finish in the Reader. Is that the problem you are seeing?
You are also accessing isFinished from more than one thread without synchronization (which however, merely may delay the termination of the loop in this situation).
I don't see CyclicBarrier doing anything.
Not sure what you are trying to do, but I'd think about how simpler I can make it. For example, can Reader and Writer be combined into one task? Then, waiting for them to finish would merely be:
executorService.invokeAll(tasks);
System.out.println("Print metrics");
where tasks is a collection of tasks (see also this javadoc)

Java native process timeout

At the moment I execute a native process using the following:
java.lang.Process process = Runtime.getRuntime().exec(command);
int returnCode = process.waitFor();
Suppose instead of waiting for the program to return I wish to terminate if a certain amount of time has elapsed. How do I do this?

All other responses are correct but it can be made more robust and efficient using FutureTask.
For example,
private static final ExecutorService THREAD_POOL
= Executors.newCachedThreadPool();
private static <T> T timedCall(Callable<T> c, long timeout, TimeUnit timeUnit)
throws InterruptedException, ExecutionException, TimeoutException
{
FutureTask<T> task = new FutureTask<T>(c);
THREAD_POOL.execute(task);
return task.get(timeout, timeUnit);
}
final java.lang.Process[] process = new Process[1];
try {
int returnCode = timedCall(new Callable<Integer>() {
public Integer call() throws Exception {
process[0] = Runtime.getRuntime().exec(command);
return process[0].waitFor();
}
}, timeout, TimeUnit.SECONDS);
} catch (TimeoutException e) {
process[0].destroy();
// Handle timeout here
}
If you do this repeatedly, the thread pool is more efficient because it caches the threads.

If you're using Java 8 or later (API 26 or later for Android) you could simply use the waitFor with timeout:
Process p = ...
if(!p.waitFor(1, TimeUnit.MINUTE)) {
//timeout - kill the process.
p.destroy(); // consider using destroyForcibly instead
}

This is how the Plexus CommandlineUtils does it:
Process p;
p = cl.execute();
...
if ( timeoutInSeconds <= 0 )
{
returnValue = p.waitFor();
}
else
{
long now = System.currentTimeMillis();
long timeoutInMillis = 1000L * timeoutInSeconds;
long finish = now + timeoutInMillis;
while ( isAlive( p ) && ( System.currentTimeMillis() < finish ) )
{
Thread.sleep( 10 );
}
if ( isAlive( p ) )
{
throw new InterruptedException( "Process timeout out after " + timeoutInSeconds + " seconds" );
}
returnValue = p.exitValue();
}
public static boolean isAlive( Process p ) {
try
{
p.exitValue();
return false;
} catch (IllegalThreadStateException e) {
return true;
}
}

What about the Groovy way
public void yourMethod() {
...
Process process = new ProcessBuilder(...).start();
//wait 5 secs or kill the process
waitForOrKill(process, TimeUnit.SECONDS.toMillis(5));
...
}
public static void waitForOrKill(Process self, long numberOfMillis) {
ProcessRunner runnable = new ProcessRunner(self);
Thread thread = new Thread(runnable);
thread.start();
runnable.waitForOrKill(numberOfMillis);
}
protected static class ProcessRunner implements Runnable {
Process process;
private boolean finished;
public ProcessRunner(Process process) {
this.process = process;
}
public void run() {
try {
process.waitFor();
} catch (InterruptedException e) {
// Ignore
}
synchronized (this) {
notifyAll();
finished = true;
}
}
public synchronized void waitForOrKill(long millis) {
if (!finished) {
try {
wait(millis);
} catch (InterruptedException e) {
// Ignore
}
if (!finished) {
process.destroy();
}
}
}
}

just modified a bit according to my requirement. time out is 10 seconds here. process is getting destroyed after 10 seconds if it is not exiting.
public static void main(String arg[]) {
try {
Process p = Runtime.getRuntime().exec("\"C:/Program Files/VanDyke Software/SecureCRT/SecureCRT.exe\"");
long now = System.currentTimeMillis();
long timeoutInMillis = 1000L * 10;
long finish = now + timeoutInMillis;
while ( isAlive( p ) ) {
Thread.sleep( 10 );
if ( System.currentTimeMillis() > finish ) {
p.destroy();
}
}
} catch (Exception err) {
err.printStackTrace();
}
}
public static boolean isAlive( Process p ) {
try {
p.exitValue();
return false;
} catch (IllegalThreadStateException e) {
return true;
}
}

You'd need a 2. thread that interrupts the thread that calls .waitFor();
Some non trivial synchronization will be needed to make it robust, but the basics are:
TimeoutThread:
Thread.sleep(timeout);
processThread.interrupt();
ProcessThread:
try {
proc.waitFor();
} catch (InterruptedException e) {
proc.destroy();
}