We Keep Coding

Unit testing asynchronous computation that has to be interrupted manually

I have got a class that records eyetracking data asynchronously. There are methods to start and stop the recording process. The data is collected in a collection and the collection can only be accessed if the recording thread has finished its work. It basically encapsulates all the threading and synchronizing so the user of my library doesn't have to do it.
The heavily shortened code (generics and error handling omitted):
public class Recorder {
private Collection accumulatorCollection;
private Thread recordingThread;
private class RecordingRunnable implements Runnable {
...
public void run() {
while(!Thread.currentThread().isInterrupted()) {
// fetch data and collect it in the accumulator
synchronized(acc) { acc.add(Eyetracker.getData()) }
}
}
}
public void start() {
accumulatorCollection = new Collection();
recordingThread = new Thread(new RecordingRunnable(accumulatorCollection));
recordingThread.start();
}
public void stop() {
recordingThread.interrupt();
}
public void getData() {
try {
recordingThread.join(2000);
if(recordingThread.isAlive()) { throw Exception(); }
}
catch(InterruptedException e) { ... }
synchronized(accumulatorCollection) { return accumulatorCollection; }
}
}
The usage is quite simple:
recorder.start();
...
recorder.stop();
Collection data = recorder.getData();
My problem with the whole thing is how to test it. Currently i am doing it like this:
recorder.start();
Thread.sleep(50);
recorder.stop();
Collection data = recorder.getData();
assert(stuff);
This works, but it is non-deterministic and slows down the test suite quite a bit (i marked these tests as integration tests, so they have to be run separately to circumvent this problem).
Is there a better way?

There is a better way using a CountDownLatch.
The non-deterministic part of the test stems from two variables in time you do not account for:
creating and starting a thread takes time and the thread may not have started executing the runnable when Thread.start() returns (the runnable will get executed, but it may be a bit later).
the stop/interrupt will break the while-loop in the Runnable but not immediately, it may be a bit later.
This is where a CountDownLatch comes in: it gives you precise information about where another thread is in execution. E.g. let the first thread wait on the latch, while the second "counts down" the latch as last statement within a runnable and now the first thread knows that the runnable finished. The CountDownLatch also acts as a synchronizer: whatever the second thread was writing to memory, can now be read by the first thread.
Instead of using an interrupt, you can also use a volatile boolean. Any thread reading the volatile variable is guaranteed to see the last value set by any other thread.
A CountDownLatch can also be given a timeout which is useful for tests that can hang: if you have to wait to long you can abort the whole test (e.g. shutdown executors, interrupt threads) and throw an AssertionError. In the code below I re-used the timeout to wait for a certain amount of data to collect instead of 'sleeping'.
As an optimization, use an Executor (ThreadPool) instead of creating and starting threads. The latter is relative expensive, using an Executor can really make a difference.
Below the updated code, I made it runnable as an application (main method). (edit 28/02/17: check maxCollect > 0 in while-loop)
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class Recorder {
private final ExecutorService executor;
private Thread recordingThread;
private volatile boolean stopRecording;
private CountDownLatch finishedRecording;
private Collection<Object> eyeData;
private int maxCollect;
private final AtomicBoolean started = new AtomicBoolean();
private final AtomicBoolean stopped = new AtomicBoolean();
public Recorder() {
this(null);
}
public Recorder(ExecutorService executor) {
this.executor = executor;
}
public Recorder maxCollect(int max) { maxCollect = max; return this; }
private class RecordingRunnable implements Runnable {
#Override public void run() {
try {
int collected = 0;
while (!stopRecording) {
eyeData.add(EyeTracker.getData());
if (maxCollect > 0 && ++collected >= maxCollect) {
stopRecording = true;
}
}
} finally {
finishedRecording.countDown();
}
}
}
public Recorder start() {
if (!started.compareAndSet(false, true)) {
throw new IllegalStateException("already started");
}
stopRecording = false;
finishedRecording = new CountDownLatch(1);
eyeData = new ArrayList<Object>();
// the RecordingRunnable created below will see the values assigned above ('happens before relationship')
if (executor == null) {
recordingThread = new Thread(new RecordingRunnable());
recordingThread.start();
} else {
executor.execute(new RecordingRunnable());
}
return this;
}
public Collection<Object> getData(long timeout, TimeUnit tunit) {
if (started.get() == false) {
throw new IllegalStateException("start first");
}
if (!stopped.compareAndSet(false, true)) {
throw new IllegalStateException("data already fetched");
}
if (maxCollect <= 0) {
stopRecording = true;
}
boolean recordingStopped = false;
try {
// this establishes a 'happens before relationship'
// all updates to eyeData are now visible in this thread.
recordingStopped = finishedRecording.await(timeout, tunit);
} catch(InterruptedException e) {
throw new RuntimeException("interrupted", e);
} finally {
stopRecording = true;
}
// if recording did not stop, do not return the eyeData (could stil be modified by recording-runnable).
if (!recordingStopped) {
throw new RuntimeException("recording");
}
// only when everything is OK this recorder instance can be re-used
started.set(false);
stopped.set(false);
return eyeData;
}
public static class EyeTracker {
public static Object getData() {
try { Thread.sleep(1); } catch (Exception ignored) {}
return new Object();
}
}
public static void main(String[] args) {
System.out.println("Starting.");
ExecutorService exe = Executors.newSingleThreadExecutor();
try {
Recorder r = new Recorder(exe).maxCollect(50).start();
int dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(100).start();
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(0).start();
Thread.sleep(100);
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
} catch (Exception e) {
e.printStackTrace();
} finally {
exe.shutdownNow();
System.out.println("Done.");
}
}
}
Happy coding :)

Java Concurrency: Paired locks with shared access

I am looking for a Java implementation of the following concurrency semantics. I want something similar to ReadWriteLock except symmetrical, i.e. both the read and write sides can be shared amongst many threads, but read excludes write and vice versa.
There are two locks, let's call them A and B.
Lock A is shared, i.e. there may be multiple threads holding it concurrently. Lock B is also shared, there may be multiple threads holding it concurrently.
If any thread holds lock A then no thread may take B – threads attempting to take B shall block until all threads holding A have released A.
If any thread holds lock B then no thread may take A – threads attempting to take A shall block until all threads holding B have released B.
Is there an existing library class that achieves this? At the moment I have approximated the desired functionality with a ReadWriteLock because fortunately the tasks done in the context of lock B are somewhat rarer. It feels like a hack though, and it could affect the performance of my program under heavy load.

Short answer:
In the standard library, there is nothing like what you need.
Long answer:
To easily implement a custom Lock you should subclass or delegate to an AbstractQueuedSynchronizer.
The following code is an example of a non-fair lock that implements what you need, including some (non exhausting) test. I called it LeftRightLock because of the binary nature of your requirements.
The concept is pretty straightforward:
AbstractQueuedSynchronizer exposes a method to atomically set the state of an int using the Compare and swap idiom ( compareAndSetState(int expect, int update) ), we can use the exposed state keep the count of the threads holding the lock, setting it to a positive value in case the Right lock is being held or a negative value in case the Left lock is being held.
Than we just make sure of the following conditions:
- you can lock Left only if the state of the internal AbstractQueuedSynchronizer is zero or negative
- you can lock Right only if the state of the internal AbstractQueuedSynchronizer is zero or positive
LeftRightLock.java
import java.util.concurrent.locks.AbstractQueuedSynchronizer;
import java.util.concurrent.locks.Lock;
/**
* A binary mutex with the following properties:
*
* Exposes two different {#link Lock}s: LEFT, RIGHT.
*
* When LEFT is held other threads can acquire LEFT but thread trying to acquire RIGHT will be
* blocked. When RIGHT is held other threads can acquire RIGHT but thread trying to acquire LEFT
* will be blocked.
*/
public class LeftRightLock {
public static final int ACQUISITION_FAILED = -1;
public static final int ACQUISITION_SUCCEEDED = 1;
private final LeftRightSync sync = new LeftRightSync();
public void lockLeft() {
sync.acquireShared(LockSide.LEFT.getV());
}
public void lockRight() {
sync.acquireShared(LockSide.RIGHT.getV());
}
public void releaseLeft() {
sync.releaseShared(LockSide.LEFT.getV());
}
public void releaseRight() {
sync.releaseShared(LockSide.RIGHT.getV());
}
public boolean tryLockLeft() {
return sync.tryAcquireShared(LockSide.LEFT) == ACQUISITION_SUCCEEDED;
}
public boolean tryLockRight() {
return sync.tryAcquireShared(LockSide.RIGHT) == ACQUISITION_SUCCEEDED;
}
private enum LockSide {
LEFT(-1), NONE(0), RIGHT(1);
private final int v;
LockSide(int v) {
this.v = v;
}
public int getV() {
return v;
}
}
/**
* <p>
* Keep count the count of threads holding either the LEFT or the RIGHT lock.
* </p>
*
* <li>A state ({#link AbstractQueuedSynchronizer#getState()}) greater than 0 means one or more threads are holding RIGHT lock. </li>
* <li>A state ({#link AbstractQueuedSynchronizer#getState()}) lower than 0 means one or more threads are holding LEFT lock.</li>
* <li>A state ({#link AbstractQueuedSynchronizer#getState()}) equal to zero means no thread is holding any lock.</li>
*/
private static final class LeftRightSync extends AbstractQueuedSynchronizer {
#Override
protected int tryAcquireShared(int requiredSide) {
return (tryChangeThreadCountHoldingCurrentLock(requiredSide, ChangeType.ADD) ? ACQUISITION_SUCCEEDED : ACQUISITION_FAILED);
}
#Override
protected boolean tryReleaseShared(int requiredSide) {
return tryChangeThreadCountHoldingCurrentLock(requiredSide, ChangeType.REMOVE);
}
public boolean tryChangeThreadCountHoldingCurrentLock(int requiredSide, ChangeType changeType) {
if (requiredSide != 1 && requiredSide != -1)
throw new AssertionError("You can either lock LEFT or RIGHT (-1 or +1)");
int curState;
int newState;
do {
curState = this.getState();
if (!sameSide(curState, requiredSide)) {
return false;
}
if (changeType == ChangeType.ADD) {
newState = curState + requiredSide;
} else {
newState = curState - requiredSide;
}
//TODO: protect against int overflow (hopefully you won't have so many threads)
} while (!this.compareAndSetState(curState, newState));
return true;
}
final int tryAcquireShared(LockSide lockSide) {
return this.tryAcquireShared(lockSide.getV());
}
final boolean tryReleaseShared(LockSide lockSide) {
return this.tryReleaseShared(lockSide.getV());
}
private boolean sameSide(int curState, int requiredSide) {
return curState == 0 || sameSign(curState, requiredSide);
}
private boolean sameSign(int a, int b) {
return (a >= 0) ^ (b < 0);
}
public enum ChangeType {
ADD, REMOVE
}
}
}
LeftRightLockTest.java
import org.junit.Test;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
public class LeftRightLockTest {
int logLineSequenceNumber = 0;
private LeftRightLock sut = new LeftRightLock();
#Test(timeout = 2000)
public void acquiringLeftLockExcludeAcquiringRightLock() throws Exception {
sut.lockLeft();
Future<Boolean> task = Executors.newSingleThreadExecutor().submit(() -> sut.tryLockRight());
assertFalse("I shouldn't be able to acquire the RIGHT lock!", task.get());
}
#Test(timeout = 2000)
public void acquiringRightLockExcludeAcquiringLeftLock() throws Exception {
sut.lockRight();
Future<Boolean> task = Executors.newSingleThreadExecutor().submit(() -> sut.tryLockLeft());
assertFalse("I shouldn't be able to acquire the LEFT lock!", task.get());
}
#Test(timeout = 2000)
public void theLockShouldBeReentrant() throws Exception {
sut.lockLeft();
assertTrue(sut.tryLockLeft());
}
#Test(timeout = 2000)
public void multipleThreadShouldBeAbleToAcquireTheSameLock_Right() throws Exception {
sut.lockRight();
Future<Boolean> task = Executors.newSingleThreadExecutor().submit(() -> sut.tryLockRight());
assertTrue(task.get());
}
#Test(timeout = 2000)
public void multipleThreadShouldBeAbleToAcquireTheSameLock_left() throws Exception {
sut.lockLeft();
Future<Boolean> task = Executors.newSingleThreadExecutor().submit(() -> sut.tryLockLeft());
assertTrue(task.get());
}
#Test(timeout = 2000)
public void shouldKeepCountOfAllTheThreadsHoldingTheSide() throws Exception {
CountDownLatch latchA = new CountDownLatch(1);
CountDownLatch latchB = new CountDownLatch(1);
Thread threadA = spawnThreadToAcquireLeftLock(latchA, sut);
Thread threadB = spawnThreadToAcquireLeftLock(latchB, sut);
System.out.println("Both threads have acquired the left lock.");
try {
latchA.countDown();
threadA.join();
boolean acqStatus = sut.tryLockRight();
System.out.println("The right lock was " + (acqStatus ? "" : "not") + " acquired");
assertFalse("There is still a thread holding the left lock. This shouldn't succeed.", acqStatus);
} finally {
latchB.countDown();
threadB.join();
}
}
#Test(timeout = 5000)
public void shouldBlockThreadsTryingToAcquireLeftIfRightIsHeld() throws Exception {
sut.lockLeft();
CountDownLatch taskStartedLatch = new CountDownLatch(1);
final Future<Boolean> task = Executors.newSingleThreadExecutor().submit(() -> {
taskStartedLatch.countDown();
sut.lockRight();
return false;
});
taskStartedLatch.await();
Thread.sleep(100);
assertFalse(task.isDone());
}
#Test
public void shouldBeFreeAfterRelease() throws Exception {
sut.lockLeft();
sut.releaseLeft();
assertTrue(sut.tryLockRight());
}
#Test
public void shouldBeFreeAfterMultipleThreadsReleaseIt() throws Exception {
CountDownLatch latch = new CountDownLatch(1);
final Thread thread1 = spawnThreadToAcquireLeftLock(latch, sut);
final Thread thread2 = spawnThreadToAcquireLeftLock(latch, sut);
latch.countDown();
thread1.join();
thread2.join();
assertTrue(sut.tryLockRight());
}
#Test(timeout = 2000)
public void lockShouldBeReleasedIfNoThreadIsHoldingIt() throws Exception {
CountDownLatch releaseLeftLatch = new CountDownLatch(1);
CountDownLatch rightLockTaskIsRunning = new CountDownLatch(1);
Thread leftLockThread1 = spawnThreadToAcquireLeftLock(releaseLeftLatch, sut);
Thread leftLockThread2 = spawnThreadToAcquireLeftLock(releaseLeftLatch, sut);
Future<Boolean> acquireRightLockTask = Executors.newSingleThreadExecutor().submit(() -> {
if (sut.tryLockRight())
throw new AssertionError("The left lock should be still held, I shouldn't be able to acquire right a this point.");
printSynchronously("Going to be blocked on right lock");
rightLockTaskIsRunning.countDown();
sut.lockRight();
printSynchronously("Lock acquired!");
return true;
});
rightLockTaskIsRunning.await();
releaseLeftLatch.countDown();
leftLockThread1.join();
leftLockThread2.join();
assertTrue(acquireRightLockTask.get());
}
private synchronized void printSynchronously(String str) {
System.out.println(logLineSequenceNumber++ + ")" + str);
System.out.flush();
}
private Thread spawnThreadToAcquireLeftLock(CountDownLatch releaseLockLatch, LeftRightLock lock) throws InterruptedException {
CountDownLatch lockAcquiredLatch = new CountDownLatch(1);
final Thread thread = spawnThreadToAcquireLeftLock(releaseLockLatch, lockAcquiredLatch, lock);
lockAcquiredLatch.await();
return thread;
}
private Thread spawnThreadToAcquireLeftLock(CountDownLatch releaseLockLatch, CountDownLatch lockAcquiredLatch, LeftRightLock lock) {
final Thread thread = new Thread(() -> {
lock.lockLeft();
printSynchronously("Thread " + Thread.currentThread() + " Acquired left lock");
try {
lockAcquiredLatch.countDown();
releaseLockLatch.await();
} catch (InterruptedException ignore) {
} finally {
lock.releaseLeft();
}
printSynchronously("Thread " + Thread.currentThread() + " RELEASED left lock");
});
thread.start();
return thread;
}
}

I don't know any library that does that you want. Even if there is such a library it possess little value because every time your request changes the library stops doing the magic.
The actual question here is "How to I implement my own lock with custom specification?"
Java provides tool for that named AbstractQueuedSynchronizer. It has extensive documentation. Apart from docs one would possibly like to look at CountDownLatch and ReentrantLock sources and use them as examples.
For your particular request see code below, but beware that it is 1) not fair 2) not tested
public class MultiReadWriteLock implements ReadWriteLock {
private final Sync sync;
private final Lock readLock;
private final Lock writeLock;
public MultiReadWriteLock() {
this.sync = new Sync();
this.readLock = new MultiLock(Sync.READ, sync);
this.writeLock = new MultiLock(Sync.WRITE, sync);
}
#Override
public Lock readLock() {
return readLock;
}
#Override
public Lock writeLock() {
return writeLock;
}
private static final class Sync extends AbstractQueuedSynchronizer {
private static final int READ = 1;
private static final int WRITE = -1;
#Override
public int tryAcquireShared(int arg) {
int state, result;
do {
state = getState();
if (state >= 0 && arg == READ) {
// new read
result = state + 1;
} else if (state <= 0 && arg == WRITE) {
// new write
result = state - 1;
} else {
// blocked
return -1;
}
} while (!compareAndSetState(state, result));
return 1;
}
#Override
protected boolean tryReleaseShared(int arg) {
int state, result;
do {
state = getState();
if (state == 0) {
return false;
}
if (state > 0 && arg == READ) {
result = state - 1;
} else if (state < 0 && arg == WRITE) {
result = state + 1;
} else {
throw new IllegalMonitorStateException();
}
} while (!compareAndSetState(state, result));
return result == 0;
}
}
private static class MultiLock implements Lock {
private final int parameter;
private final Sync sync;
public MultiLock(int parameter, Sync sync) {
this.parameter = parameter;
this.sync = sync;
}
#Override
public void lock() {
sync.acquireShared(parameter);
}
#Override
public void lockInterruptibly() throws InterruptedException {
sync.acquireSharedInterruptibly(parameter);
}
#Override
public boolean tryLock() {
return sync.tryAcquireShared(parameter) > 0;
}
#Override
public boolean tryLock(long time, TimeUnit unit) throws InterruptedException {
return sync.tryAcquireSharedNanos(parameter, unit.toNanos(time));
}
#Override
public void unlock() {
sync.releaseShared(parameter);
}
#Override
public Condition newCondition() {
throw new UnsupportedOperationException(
"Conditions are unsupported as there are no exclusive access"
);
}
}
}

After my nth attempt to make a simple fair implementation, I think I understand why I could not find another library/example of the "mutual exclusive lock-pair": it requires a pretty specific user-case. As OP mentioned, you can get a long way with the ReadWriteLock and a fair lock-pair is only useful when there are many requests for a lock in quick succession (else you might as well use one normal lock).
The implementation below is more of a "permit dispenser": it is not re-entrant. It can be made re-entrant though (if not, I fear I failed to make the code simple and readable) but it requires some additional administration for various cases (e.g. one thread locking A twice, still needs to unlock A twice and the unlock-method needs to know when there are no more locks outstanding). An option to throw a deadlock error when one thread locks A and wants to lock B is probably a good idea.
The main idea is that there is an "active lock" that can only be changed by the lock-method when there are no (requests for) locks at all and can be changed by the unlock-method when the active locks outstanding reaches zero. The rest is basically keeping count of lock-requests and making threads wait until the active lock can be changed. Making threads wait involves working with InterruptedExceptions and I made a compromise there: I could not find a good solution that works well in all cases (e.g. application shutdown, one thread that gets interrupted, etc.).
I only did some basic testing (test class at the end), more validation is needed.
import java.util.concurrent.Semaphore;
import java.util.concurrent.locks.ReentrantLock;
/**
* A pair of mutual exclusive read-locks: many threads can hold a lock for A or B, but never A and B.
* <br>Usage:<pre>
* PairedLock plock = new PairedLock();
* plock.lockA();
* try {
* // do stuff
* } finally {
* plock.unlockA();
* }</pre>
* This lock is not reentrant: a lock is not associated with a thread and a thread asking for the same lock
* might be blocked the second time (potentially causing a deadlock).
* <p>
* When a lock for A is active, a lock for B will wait for all locks on A to be unlocked and vice versa.
* <br>When a lock for A is active, and a lock for B is waiting, subsequent locks for A will wait
* until all (waiting) locks for B are unlocked.
* I.e. locking is fair (in FIFO order).
* <p>
* See also
* stackoverflow-java-concurrency-paired-locks-with-shared-access
*
* #author vanOekel
*
*/
public class PairedLock {
static final int MAX_LOCKS = 2;
static final int CLOSE_PERMITS = 10_000;
/** Use a fair lock to keep internal state instead of the {#code synchronized} keyword. */
final ReentrantLock state = new ReentrantLock(true);
/** Amount of threads that have locks. */
final int[] activeLocks = new int[MAX_LOCKS];
/** Amount of threads waiting to receive a lock. */
final int[] waitingLocks = new int[MAX_LOCKS];
/** Threads block on a semaphore until locks are available. */
final Semaphore[] waiters = new Semaphore[MAX_LOCKS];
int activeLock;
volatile boolean closed;
public PairedLock() {
super();
for (int i = 0; i < MAX_LOCKS; i++) {
// no need for fair semaphore: unlocks are done for all in one go.
waiters[i] = new Semaphore(0);
}
}
public void lockA() throws InterruptedException { lock(0); }
public void lockB() throws InterruptedException { lock(1); }
public void lock(int lockNumber) throws InterruptedException {
if (lockNumber < 0 || lockNumber >= MAX_LOCKS) {
throw new IllegalArgumentException("Lock number must be 0 or less than " + MAX_LOCKS);
} else if (isClosed()) {
throw new IllegalStateException("Lock closed.");
}
boolean wait = false;
state.lock();
try {
if (nextLockIsWaiting()) {
wait = true;
} else if (activeLock == lockNumber) {
activeLocks[activeLock]++;
} else if (activeLock != lockNumber && activeLocks[activeLock] == 0) {
// nothing active and nobody waiting - safe to switch to another active lock
activeLock = lockNumber;
activeLocks[activeLock]++;
} else {
// with only two locks this means this is the first lock that needs an active-lock switch.
// in other words:
// activeLock != lockNumber && activeLocks[activeLock] > 0 && waitingLocks[lockNumber] == 0
wait = true;
}
if (wait) {
waitingLocks[lockNumber]++;
}
} finally {
state.unlock();
}
if (wait) {
waiters[lockNumber].acquireUninterruptibly();
// there is no easy way to bring this lock back into a valid state when waiters do no get a lock.
// so for now, use the closed state to make this lock unusable any further.
if (closed) {
throw new InterruptedException("Lock closed.");
}
}
}
protected boolean nextLockIsWaiting() {
return (waitingLocks[nextLock(activeLock)] > 0);
}
protected int nextLock(int lockNumber) {
return (lockNumber == 0 ? 1 : 0);
}
public void unlockA() { unlock(0); }
public void unlockB() { unlock(1); }
public void unlock(int lockNumber) {
// unlock is called in a finally-block and should never throw an exception.
if (lockNumber < 0 || lockNumber >= MAX_LOCKS) {
System.out.println("Cannot unlock lock number " + lockNumber);
return;
}
state.lock();
try {
if (activeLock != lockNumber) {
System.out.println("ERROR: invalid lock state: no unlocks for inactive lock expected (active: " + activeLock + ", unlock: " + lockNumber + ").");
return;
}
activeLocks[lockNumber]--;
if (activeLocks[activeLock] == 0 && nextLockIsWaiting()) {
activeLock = nextLock(lockNumber);
waiters[activeLock].release(waitingLocks[activeLock]);
activeLocks[activeLock] += waitingLocks[activeLock];
waitingLocks[activeLock] = 0;
} else if (activeLocks[lockNumber] < 0) {
System.out.println("ERROR: to many unlocks for lock number " + lockNumber);
activeLocks[lockNumber] = 0;
}
} finally {
state.unlock();
}
}
public boolean isClosed() { return closed; }
/**
* All threads waiting for a lock will be unblocked and an {#link InterruptedException} will be thrown.
* Subsequent calls to the lock-method will throw an {#link IllegalStateException}.
*/
public synchronized void close() {
if (!closed) {
closed = true;
for (int i = 0; i < MAX_LOCKS; i++) {
waiters[i].release(CLOSE_PERMITS);
}
}
}
#Override
public String toString() {
StringBuilder sb = new StringBuilder(this.getClass().getSimpleName());
sb.append("=").append(this.hashCode());
state.lock();
try {
sb.append(", active=").append(activeLock).append(", switching=").append(nextLockIsWaiting());
sb.append(", lockA=").append(activeLocks[0]).append("/").append(waitingLocks[0]);
sb.append(", lockB=").append(activeLocks[1]).append("/").append(waitingLocks[1]);
} finally {
state.unlock();
}
return sb.toString();
}
}
The test class (YMMV - works fine on my system, but may deadlock on yours due to faster or slower starting and running of threads):
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class PairedLockTest {
private static final Logger log = LoggerFactory.getLogger(PairedLockTest.class);
public static final ThreadPoolExecutor tp = (ThreadPoolExecutor) Executors.newCachedThreadPool();
public static void main(String[] args) {
try {
new PairedLockTest().test();
} catch (Exception e) {
e.printStackTrace();
} finally {
tp.shutdownNow();
}
}
PairedLock mlock = new PairedLock();
public void test() throws InterruptedException {
CountDownLatch start = new CountDownLatch(1);
CountDownLatch done = new CountDownLatch(2);
mlock.lockA();
try {
logLock("la1 ");
mlock.lockA();
try {
lockAsync(start, null, done, 1);
await(start);
logLock("la2 ");
} finally {
mlock.unlockA();
}
lockAsync(null, null, done, 0);
} finally {
mlock.unlockA();
}
await(done);
logLock();
}
void lockAsync(CountDownLatch start, CountDownLatch locked, CountDownLatch unlocked, int lockNumber) {
tp.execute(() -> {
countDown(start);
await(start);
//log.info("Locking async " + lockNumber);
try {
mlock.lock(lockNumber);
try {
countDown(locked);
logLock("async " + lockNumber + " ");
} finally {
mlock.unlock(lockNumber);
//log.info("Unlocked async " + lockNumber);
//logLock("async " + lockNumber + " ");
}
countDown(unlocked);
} catch (InterruptedException ie) {
log.warn(ie.toString());
}
});
}
void logLock() {
logLock("");
}
void logLock(String msg) {
log.info(msg + mlock.toString());
}
static void countDown(CountDownLatch l) {
if (l != null) {
l.countDown();
}
}
static void await(CountDownLatch l) {
if (l == null) {
return;
}
try {
l.await();
} catch (InterruptedException e) {
log.error(e.toString(), e.getCause());
}
}
}

How about
class ABSync {
private int aHolders;
private int bHolders;
public synchronized void lockA() throws InterruptedException {
while (bHolders > 0) {
wait();
}
aHolders++;
}
public synchronized void lockB() throws InterruptedException {
while (aHolders > 0) {
wait();
}
bHolders++;
}
public synchronized void unlockA() {
aHolders = Math.max(0, aHolders - 1);
if (aHolders == 0) {
notifyAll();
}
}
public synchronized void unlockB() {
bHolders = Math.max(0, bHolders - 1);
if (bHolders == 0) {
notifyAll();
}
}
}
Update: As for "fairness" (or, rather, non-starvation), OPs requirements don't mention it. In order to implement OPs requirements + some form of fairness/non-starvation, it should be specified explicitly (what do you consider fair, how should it behave when flows of requests for currently dominant and non-dominant locks come in etc). One of the ways to implement it would be:
class ABMoreFairSync {
private Lock lock = new ReentrantLock(true);
public final Part A, B;
public ABMoreFairSync() {
A = new Part();
B = new Part();
A.other = B;
B.other = A;
}
private class Part {
private Condition canGo = lock.newCondition();
private int currentGeneration, lastGeneration;
private int holders;
private Part other;
public void lock() throws InterruptedException {
lock.lockInterruptibly();
try {
int myGeneration = lastGeneration;
if (other.holders > 0 || currentGeneration < myGeneration) {
if (other.currentGeneration == other.lastGeneration) {
other.lastGeneration++;
}
while (other.holders > 0 || currentGeneration < myGeneration) {
canGo.await();
}
}
holders++;
} finally {
lock.unlock();
}
}
public void unlock() throws InterruptedException {
lock.lockInterruptibly();
try {
holders = Math.max(0, holders - 1);
if (holders == 0) {
currentGeneration++;
other.canGo.signalAll();
}
} finally {
lock.unlock();
}
}
}
}
To be used as in:
sync.A.lock();
try {
...
} finally {
sync.A.unlock();
}
The idea of generations here is taken from "Java Concurrency in Practice", Listing 14.9.

All threads get locked in wait() state [duplicate]

This question already has answers here:
Notify not getting the thread out of wait state
(3 answers)
Closed 7 years ago.
Basically I have to create 3 classes (2 threaded).
First one holds some cargo (has a minimum capacity (0) and a maximum (200))
Second one supplies the cargo every 500ms.
Third one takes away from cargo every 500ms.
Main program has one cargo class(1), 2 supplier classes(2) and 2 substraction classes(3). Problem I'm having is that one by one, they're falling into a wait(); state and never get out. Eventually all of them get stucked in the wait() state, with the program running, but without them actually doing anything.
First class:
public class Storage {
private int maxCapacity;
private int currentCapacity;
public Storage( int currentCapacity, int maxCapacity ) {
this.currentCapacity = currentCapacity;
this.maxCapacity = maxCapacity;
}
public int getCapacity(){ return this.currentCapacity; }
public void increase( int q ) {
this.currentCapacity += q;
System.out.println("increase" + q + ". Total: " + currentCapacity);
}
public int getMax() { return this.maxCapacity; }
public void decrease( int q ) {
this.currentCapacity -= q;
System.out.println("decrease - " + q + ". Total: " + currentCapacity);
}
}
2nd class (supplier):
public class Supplier implements Runnable {
private int capacity;
private Storage storage;
private volatile boolean run;
public Supplier( int capacity, Storage storage ) {
this.capacity = capacity;
this.storage = storage;
this.run = true;
}
public void kiss_kill() { run = !run; }
public synchronized void add() {
while(storage.getCapacity() + capacity > storage.getMax()) {
try {
System.out.println("wait - supplier");
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
storage.increase(capacity);
notifyAll();
}
public void run() {
synchronized (this) {
while(run) {
add();
Thread.yield(); //would be wait(500), but this just speeds it up
}
}
}
}
3rd class (taker/demander):
public class Taker implements Runnable {
private int capacity;
private Storage storage;
private volatile boolean run;
public Taker( int capacity, Storage storage ) {
this.capacity = capacity;
this.storage = storage;
this.run = true;
}
public void kiss_kill() { run = !run; }
public synchronized void take() {
while(storage.getCapacity() - capacity < 0) {
try {
System.out.println("wait - taker");
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
storage.decrease(capacity);
notifyAll();
}
public void run() {
synchronized (this) {
while(run) {
take();
Thread.yield(); //again, wait(500) should be instead
}
}
}
}
Main is something like this:
public class Main{
public static void main(String... args) {
Storage sk = new Storage(100, 200);
Supplier[] s = { new Supplier(10, sk), new Supplier(15, sk) };
Taker[] p = { new Taker(15, sk), new Taker(20, sk) };
Thread t[] = {
new Thread(s[0]),
new Thread(s[1]),
new Thread(p[0]),
new Thread(p[1]) };
for(Thread th : t) th.start();
try {
Thread.sleep(60000); //program should last for 60s.
} catch (InterruptedException e) {
e.printStackTrace();
}
s[0].kiss_kill(); s[1].kiss_kill(); p[0].kiss_kill(); p[1].kiss_kill();
}
}
Why doesn't notifyAll() release the wait() state of other object? What could I do to fix this?
Sorry, I know it's a long example, I hate posting too many classes like this. Thanks for reading!
I translated the code, so if you spot anything that you're unsure about that I've missed, please tell me and I'll fix it right away!

Doing concurrency is easy:
Anyone can slap synchronized on methods and synchronized () {} around blocks of code. It does not mean it is correct. And then they can continue to slap synchronized on everything until it works until it doesn't.
Doing concurrency correctly is Hard:
You should lock on the data that needs to be consistent not the methods making the changes. And you have to use the same lock instance for everything.
In this case that is the currentCapacity in Storage. That is the only thing that is shared and the only thing that needs to be consistent.
What you are doing now is having the classes lock on instances of themselves which means nothing shared is being protected because there is no shared lock.
Think about it, if you are not locking on the same exact instance which must be final of an object then what are you protecting?
Also what about code that has access to the object that needs to be consistent and does not request a lock on it. Well it just does what it wants. synchronized() {} in calling classes is not how you protect shared data from external manipulation.
Thread safe objects are NOT about the synchronized keyword:
Read up on the java.util.concurrent package it has all the things you need already. Use the correct data structure for your use case.
In this particular case if you use AtomicInteger for your counter, you do not need any error prone manual locking, no need for synchronized anywhere, it is already thread safe.
Immutable Data:
If you work with immutable data exclusively you do not need any of this silly locking semantics that are extremely error prone for even those that understand it and even more so for those that think they understand it.
Here is a working idiomatic example:
This is a good chance to learn what non-deterministic means and how to use the step debugger in your IDE to debug concurrent programs.
Q33700412.java
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
import com.vertigrated.FormattedRuntimeException;
public class Q33700412
{
public static void main(final String[] args)
{
final Storage s = new Storage(100);
final int ap = Runtime.getRuntime().availableProcessors();
final ExecutorService es = Executors.newFixedThreadPool(ap);
for (int i = 0; i < ap; i++)
{
es.execute(new Runnable()
{
final Random r = new Random();
#Override
public void run()
{
while (true)
{
/* this if/else block is NOT thread safe, I did this on purpose
the state can change between s.remainingCapacity() and
the call to s.increase/s.decrease.
This is ok, because the Storage object is internally consistent.
This thread might fail if this happens, this is the educational part.
*/
if (s.remainingCapacity() > 0)
{
if (r.nextBoolean()) { s.increase(r.nextInt(10)); }
else { s.decrease(10); }
System.out.format("Current Capacity is %d", s.getCurrentCapacity());
System.out.println();
}
else
{
System.out.format("Max Capacity %d Reached", s.getMaxCapacity());
System.out.println();
}
try { Thread.sleep(r.nextInt(5000)); }
catch (InterruptedException e) { throw new RuntimeException(e); }
}
}
});
}
es.shutdown();
try
{
Thread.sleep(TimeUnit.MINUTES.toMillis(1));
es.shutdown();
}
catch (InterruptedException e) { System.out.println("Done!"); }
}
public static final class Storage
{
/* AtomicInteger is used so that it can be mutable and final at the same time */
private final AtomicInteger currentCapacity;
private final int maxCapacity;
public Storage(final int maxCapacity) { this(0, maxCapacity); }
public Storage(final int currentCapacity, final int maxCapacity)
{
this.currentCapacity = new AtomicInteger(currentCapacity);
this.maxCapacity = maxCapacity;
}
public int remainingCapacity() { return this.maxCapacity - this.currentCapacity.get(); }
public int getCurrentCapacity() { return this.currentCapacity.get(); }
public void increase(final int q)
{
synchronized (this.currentCapacity)
{
if (this.currentCapacity.get() < this.maxCapacity)
{
this.currentCapacity.addAndGet(q);
}
else
{
throw new FormattedRuntimeException("Max Capacity %d Exceeded!", this.maxCapacity);
}
}
}
public int getMaxCapacity() { return this.maxCapacity; }
public void decrease(final int q)
{
synchronized (this.currentCapacity)
{
if (this.currentCapacity.get() - q >= 0)
{
this.currentCapacity.addAndGet(q * -1);
}
else
{
this.currentCapacity.set(0);
}
}
}
}
}
Notes:
Limit the scope of synchronized blocks to the minimum they need to protect and lock on the object that needs to stay consistent.
The lock object must be marked final or the reference can change and you will be locking on different instances.
The more final the more correct your programs are likely to be the first time.

Jarrod Roberson gave you the "how" half of the answer. Here's the other half--the "why".
Your Supplier object's add() method waits on itself (i.e., on the supplier object), and it notifies itself.
Your Taker object's take() method waits on its self (i.e., on the taker object), and it notifies its self.
The supplier never notifies the taker, and taker never notifies the supplier.
You should do all of your synchronization on the shared object (i.e., on the Storage object.
So I should convert storage into a thread?
No, you don't want Storage to be a thread, you want it to be the lock. Instead of having your Supplier objects and your Taker objects synchronize on themselves, they should all synchronize on the shared Storage object.
E.g., do this:
public void take() {
synchronized(storage) {
while(...) {
try {
storage.wait();
} catch ...
}
...
storage.notifyAll();
}
}
Instead of this:
public synchronized void take() {
while(...) {
try {
wait();
} catch ...
}
...
notifyAll();
}
And do the same for all of your other synchronized methods.

Java Thread stop notifier

My task here is to perform unzip operation using multiple threads. I did it with following structure Way.
// A class for Unzipping files
public class UnzipClass extends Thread(){
private String zipfile;
private Thread t;
public UnzipClass(String zipFile){
this.zipFile = zipFile;
}
public String getZipFile() {
return zipFile;
}
public void setZipFile(String zipFile) {
this.zipFile = zipFile;
}
public void run() {
try {
unzipFolder(this.getZipFile());
} catch (IOException ex) {
Logger.getLogger(Unzipper.class.getName()).log(Level.SEVERE, null, ex);
}
}
public void start(String filename){
if (t == null){
t = new Thread(this,filename);
t.start();
}
}
public unzipFolder(String zipFile) throws ZipException, IOException
// Here is the Unzip Method
}
}
// Now I am calling this class from another class
public static void main(){
Thread t1 = new UnzipClass("filename1");
t1.start();
if(!(t1.isAlive())){
logEvent("Unzip Complete");
}
// Similarly I have Thread t2 with another file name
}
The above code works perfect and unzips the files but I have following problems.
I wanted to use implements Runnable , but I cannot use it because I did not find a way to pass variable(Filename) to another class which implements Runnable and do it. Literally: How to implement Runnable instead of extends Thread`
Using above method, How can I detect if the unzip process has been completed. To be specific how to stop the thread when the file unzip process is completed`.
Any sort of hint or solution would be really great.
Thanks in advance.

1.change
public class UnzipClass extends Thread
into
public class UnzipClass implements Runnable
and use
Runnable t1 = new UnzipClass("filename1");
to create the thread.
2.
use a while loop here
while((t1.isAlive())){
logEvent("Unziping...");
}
logEvent("Unzip Complete");
but using a flag like boolean isComplete in the UnzipClass will me more effective. like
in class UnzipClass add
private boolean complete=false;
then,
public void run() {
try {
unzipFolder(this.getZipFile());
complete=true;
} catch (IOException ex) {
Logger.getLogger(Unzipper.class.getName()).log(Level.SEVERE, null, ex);
}
}
//simple getter.
public boolean isComplete()
{
return this.complete;
}
in main...
while(!t1.isComplete()){
logEvent("Unziping...");
}
logEvent("Unzip Complete");

Just change extends Thread to implements Runnable.
While creating new thread, you will do
Thread t1 = new Thread(new UnzipClass("filename1"));
Instead of
Thread t1 = new UnzipClass("filename1");
As you want to unzip couple files simultaniously, try using ExecutorService for that. You can submit Runnable taks to be executed by thread pool - this way, you will reuse already existing threads.
Check Executors and ExecutorService

Please, check this solution.
It uses Java 8 features, but it can be easily upgraded to be used with Java 5 / 6 / 7 (and external library, like Apache Commons or Guava).
/**
* File unzipping service
*/
public class Unzipper {
/**
* Default number of threads
*/
private static byte DEFAULT_THREADS_COUNT = 5;
/**
* Completion handler
*/
private Consumer onComplete;
/**
* Unzipping tasks
*/
private Collection<File> unzippingTasks = new LinkedList<>();
/**
* Add task for unzipping file
*
* #param file Path to file to be unzipped
* #return upgraded <code>this</code> instance
*/
public Unzipper unzip(String file) {
//check validity of 'file' string: non-null and non-empty
//check that file pointed by 'file' exists
unzippingTasks.add(new File(file));
return this;
}
/**
* Add unzipping completion handler
*
* #param onComplete Unzipping completion handler
* #return upgraded <code>this</code> instance
*/
public Unzipper onComplete(Consumer onComplete) {
//check validity of 'onComplete' object: non-null
this.onComplete = onComplete;
return this;
}
/**
* Run files unzipping (with default number of threads)
*/
public void run() {
run(DEFAULT_THREADS_COUNT);
}
/**
* Run files unzipping
*
* #param threads Number of parallel threads
*/
public void run(byte threads) {
//check validity of 'threads' argument: non-negative and non-zero
//check that we have at least one 'unzipping' task
if(unzippingTasks.isEmpty()) {
//output to log that unzipping tasks list is empty
return;
}
CountDownLatch latch = new CountDownLatch(threads);
Executor executor = Executors.newFixedThreadPool(threads + 1); //we are not blocking in 'run' method, so we should create extra thread to wait for all tasks completion
for(File file: unzippingTasks) {
executor.execute(() -> {
//unzip file
latch.release();
});
}
executor.execute(() -> {
latch.await(); //wait for all unzipping tasks completion
if(onComplete) //you can use 'optional' here instead
onComplete.accept(null);
});
executor.shutdown();
}
}
....
//Usage example
new Unzipper()
.unzip("<path-to-some-file>")
.unzip("<path-to-another-file>")
.unzip("<path-to-some-another-file>")
.onComplete(() -> {
//make you log output
})
.run();

import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.zip.ZipException;
public class UnzipClass {
// Now I am calling this class from another class
public static void main() {
Thread t1 = new Thread(new UnzipClassRunner("filename1"));
t1.start();
try {
t1.join();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
class UnzipClassRunner implements Runnable {
private String zipfile;
public UnzipClassRunner(String zipFile) {
this.zipfile = zipFile;
}
public void run() {
try {
unzipFolder(zipfile);
} catch (IOException ex) {
Logger.getLogger(Unzipper.class.getName()).log(Level.SEVERE, null,
ex);
}
}
public void unzipFolder(String zipFile) throws ZipException, IOException {
//
}
}

Java, stopping (interrupting) the thread

Very recently I've asked this question, but wasn unable to fix this. So I have a thread hunter (2 of them actually), who "goes off to catch wild boars". He stores these boars in a container Fridge. He will continue to do so until his working hours expire. However, in case the Fridge is full he has to wait. The aim is to wait until a wild boar is removed from the fridge, but if it takes more then 5 seconds of waiting test must be terminated. So everything works except one thing. After running test and interrupting these threads, the program still continues to run. So how do I completely terminate/stop these threads?
TEST CLASS (main)
class Test {
public static void main(String[] args) {
test1();
}
public static void test1() {
Fridge fridge = new Fridge(4);
Hunter hunter1 = new Hunter("hunter1", 4, fridge);
Hunter hunter2 = new Hunter("hunter2", 7, fridge);
Thread hunterThread1 = new Thread(hunter1);
Thread hunterThread2 = new Thread(hunter2);
hunterThread1.start();
hunterThread2.start();
try { Thread.sleep(1000); } catch (InterruptedException e) {}
hunterThread1.interrupt();
hunterThread2.interrupt();
System.out.println(fridge.getSize());
System.out.println(hunter1.getWorkTime());
System.out.println(hunter2.getWorkTime());
}
}
HUNTER CLASS
class Hunter extends Worker {
private int workTime;
private Fridge fridge;
public Hunter(String name, int workTime, Fridge fridge) {
super(name);
this.workTime = workTime;
this.fridge = fridge;
}
public int getWorkTime() {
return workTime;
}
public void run() {
while (workTime > 0) {
/** Each hunt takes a random amount of time (1-50 ms) **/
try { Thread.sleep(workGen()); } catch (InterruptedException e) {}
/** Add new wild boars **/
try { fridge.add(new WildBoar()); } catch (InterruptedException e) {}
workTime--;
/** If thread is interupted break the loop **/
if( Thread.currentThread().isInterrupted()){
break;
}
}
}
}
FRIDGE CLASS
import java.util.Stack;
class Fridge extends Storage {
private Stack<WildBoar> boars;
public Fridge(int cap) {
super(cap);
boars = new Stack<WildBoar>();
}
public int getCap() {
return cap;
}
public int getSize() {
return boars.size();
}
public boolean hasFreeSpace() {
if ( boars.size() < cap )
return true;
else
return false;
}
public synchronized void add(WildBoar boar) throws InterruptedException {
/** If there's no free space available wait **/
while ( !hasFreeSpace() ) {
wait();
}
/** Once there's free space available add new item **/
boars.add(boar);
}
public synchronized WildBoar remove() {
return boars.pop();
}
}
ADDITIONAL CLASSES FOR COMPILING:
abstract class Worker implements Runnable {
private String name;
public Worker(String name) {
this.name = name;
}
public String getName() {
return name;
}
public int workGen() {
return 1 + (int)(Math.random() * (50 - 1));
}
}
class WildBoar {
public WildBoar() {}
}
abstract class Storage {
protected int cap;
public Storage(int cap) {
this.cap = cap;
}
public int getCap() {
return cap;
}
}

After you interrupt() the thread which is currently waiting, the native wait method will actually reset the interruption flag. So when you evaluate the isInterrupted() here, it is actually reset and will appear as not interrupted.
if( Thread.currentThread().isInterrupted()){
break;
}
You will have to re-interrupt the thread after an interruption occurs during the waiting
public synchronized void add(Object boar) {
/** If there's no free space available wait **/
while (!hasFreeSpace()) {
try{
wait();
}catch(InterruptedException e){
Thread.currentThread().interrupt();
return; //or rethrow
}
}
/** Once there's free space available add new item **/
boars.add(boar);
}

Currently, the run method in your Hunter thread is discarding interruptions:
try { fridge.add(new WildBoar()); }
catch (InterruptedException e) {}
Thus, nothing happens when you later check for interruptions
if( Thread.currentThread().isInterrupted()){
break;
}
To correct this, you need to set the thread's interrupt status:
try { fridge.add(new WildBoar()); }
catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
Summary - ignoring the InterruptedException resets the interrupt status. If you don't or re-throw it or break, then you will need to set the interrupt status manually.