We Keep Coding

Callable with while loop

I need to provide code that using ExecutorService, Callable and Future will be doing some calculations and printing the partial results until defined condition is reached. First thing that comes to my mind is using while loop. Unfortunately as I understand ExecutorService.get() waits until the task is done, so I cannot do something like (pseudo code):
public Object call() throws Exception {
try {
while(!condition) {
//perform calc
return partialCalculationResult;
}
}
catch(InterruptedException e){
}
}
Could anyone guide me what's the proper direction I should go for?

This here:
while(!condition) {
//perform calc
return partialCalculationResult;
}
indicates a "hole" in your logic. This should probably go like this instead:
while(!condition) {
// perform computation
push intermediate results somewhere
}
return finalResult;
In other words: you are talking about two different elements here. For those "progress" updates you will need some kind of shared data structure; for example a Queue.
You see, in difference to other languages, there is no built-in "generator" concept that would allow you to yield values from a loop; like you can do in python or scala for example.

The dirty option is putting a System.out.println within the while loop.
The cleaner option would be a publish/subscriber pattern, like:
interface Subscriber {
void onPartialResult(double partialResult);
}
class SystemOutSubscriber implements Subscriber{
#Override
void onPartialResult(double partialResult) {
System.out.println(partialResult);
}
}
class YourCalculatorClass {
List<Subscriber> subscribers = ...
public Object call() throws Exception {
while(!condition) {
//perform calc
for(Subscriber s : subscribers) {
s.onPartialResult(partialCalculationResult);
}
}
}
}

You can use Thread.interrupt to stop the thread inside while loop and add remaining result in list
while(!condition){ list.add(addResultHere)
Thread.interrupt(); }

Below is a small example of using an ExecutorService
to push callable tasks. I push them inside a while loop now for the ease of the example, but they can come from anywhere. The callable itself uses the most silly easy example of course where it takes in a number. If the number is below 5, all is good, and we return a text. If not, we return nothing. When the future is evaluated and the result is empty, we shut down the ExecutorService and call it a day. So, this is an example of using an ExecutorService, Callable, and Future to do something at least similar to what I could discern from your explanation.
public ExecutorServiceTest() {
ExecutorService service = Executors.newCachedThreadPool();
int num = 0;
while (true) {
Future<Optional<String>> future = service.submit(new MyCallable(num++));
try {
Optional<String> result = future.get();
if (!result.isPresent()) {
service.shutdown();
break;
}
System.out.println(result.get());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
service.shutdown();
}
}
}
private static class MyCallable implements Callable<Optional<String>> {
private final int num;
MyCallable(int num) {
this.num = num;
}
#Override
public Optional<String> call() throws Exception {
if (num < 5)
return Optional.of("My number is " + num);
return Optional.empty();
}
}
public static void main(String[] args) {
new ExecutorServiceTest();
}

Java signal/event mechanism to indicate some value is available

I have a generator class that owns a Thread in which a number of "records" to be generated is determined, then generates that many records (which get placed in a BlockingQueue for retrieval by another thread).
I'd like the other thread to know how many records are going to be generated (for sensible progress reporting among other things).
It seems Future gives me exactly the interface I'm after, but I'm new to Java, and not sure of the idiomatic way of implementing it.
My background is in C++/Win32, so I'd normally use a win32 "Event" (as created by CreateEvent(0, true, false, 0), with SetEvent and WaitForSingleObject for my signal and wait implementations). I've noticed Java has a CountDownLatch, but this somehow feels heavier than what I'm after (somewhat akin to using an int when I really want a boolean), and it seems unintuitive for this purpose (to me, anyway).
So here's my code using CountDownLatch and a Future. I've distilled my real code down a bit here (removed irrelevant implementation details and ignoring all error handling).
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Future;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
public abstract class Generator {
private CountDownLatch numRecordsSignal = new CountDownLatch(1);
private int numRecords;
private BlockingQueue<Record> queue = new LinkedBlockingQueue<Record>();
public Generator() {
new Thread(new Runnable() {
#Override
public void run() {
numRecords = calculateNumRecords();
numRecordsSignal.countDown();
for (Record r : generateRecords()) {
try {
queue.put(r);
} catch (InterruptedException e) {
// [ ... snip ... ]
}
}
}
}).start();
}
public Future<Integer> numRecords() {
return new Future<Integer>() {
// Ignore cancel for now (It wouldn't make sense to cancel
// just this part of the Generator's work, anyway).
public boolean cancel(boolean mayInterruptIfRunning) {
return false;
}
public Integer get() throws InterruptedException {
numRecordsSignal.await();
return numRecords;
}
public Integer get(long timeout, TimeUnit unit)
throws InterruptedException {
numRecordsSignal.await(timeout, unit);
return numRecords;
}
public boolean isCancelled() {
return false;
}
public boolean isDone() {
// Since we can't cancel, just check the state of the
// signal
return numRecordsSignal.getCount() == 0;
}
};
}
public Record nextRecord() throws InterruptedException {
return queue.take();
}
/** --- Boring stuff below this line --- */
public interface Record { }
protected abstract int calculateNumRecords();
protected abstract Iterable<Record> generateRecords();
}
Now for my actual questions:
Is there a better mechanism than CountDownLatch for single-shot signalling?
I want callers to be able to either wait or poll on the result, but don't need them to be able to cancel the operation. Is Future the right way to expose this stuff?
Does any of this stuff look particularly "un-Java"? Am I on the wrong track completely?
Edit:
Just to clarify, I expect the caller to be able to do the following:
Generator gen = new Generator();
Integer numRecords = gen.numRecords().get(); // This call might block waiting for the result
numRecords = gen.numRecords().get(); // This call will never block, as the result is already available.
It's just a slow-to-initialise value I'm trying to implement. Once the "initialised" condition is met, it should latch. The value doesn't get re-evaluated once it's known.

Side comment
You should not start a thread in a constructor - it is very conceivable that the Generator object is not fully created when the thread starts, and the countdown latch could well be null for example. You can create the thread in the constructor, but should start it in a seperate method. Your calling code would become:
Generator g = new Generator();
g.start();
Your question
You are reimplementing a Future yourself, which is not necessary nor desirable in my opinion. I would redesign the class and make Generator implement Callable<Integer> and run it through an executor. That provides you with several things:
remove the threading logic from the Generator, which enables you a more efficient management of your threads at a higher level in your call stack
the integer is returned via the future in your calling code and you rely on the JDK to handle the implementation
I have assumed that it's ok to first populate the queue then return the integer
you can call future.get() as many times as you want - it will only block the first time it is called.
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(1);
Future<Integer> future = executor.submit(new GeneratorImpl()); //a concrete implementation of Generator
int numRecords = 0;
try {
numRecords = future.get(); //you can use a get with timeout here
} catch (ExecutionException e) {
//an exception happened in Generator#call()
} catch (InterruptedException e) {
//handle it
}
//don't forget to call executor.shutdown() when you don't need it any longer
}
public abstract class Generator implements Callable<Integer> {
private BlockingQueue<Record> queue = new LinkedBlockingQueue<Record>();
#Override
public Integer call() {
int numRecords = calculateNumRecords();
for (Record r : generateRecords()) {
try {
queue.put(r);
} catch (InterruptedException e) {
// [ ... snip ... ]
}
}
return numRecords;
}
public Record nextRecord() throws InterruptedException {
return queue.take();
}
/**
* --- Boring stuff below this line ---
*/
public interface Record {
}
protected abstract int calculateNumRecords();
protected abstract Iterable<Record> generateRecords();
}
EDIT
If you need to return numRecods asap, you can populate your queue in a separate thread:
public Integer call() {
int numRecords = calculateNumRecords();
new Thread(new Runnable() {
#Override
public void run() {
for (Record r : generateRecords()) {
try {
queue.put(r);
} catch (InterruptedException e) {
// [ ... snip ... ]
}
}
}
}).start(); //returns immediately
return numRecords;
}

The standard Java equivalents of "WaitOnSingleEvent()" and "SetEvent()" for Java threads are "wait()", "notify()" and "notifyAll()".

After looking at implementing my own signal mechanism and following the bread-crumb trail left by others doing the same thing, I came across the javadoc for AbstractQueuedSynchronizer, which includes a code snippet for a "BooleanLatch", which perfectly meets my needs:
class BooleanLatch {
private static class Sync extends AbstractQueuedSynchronizer {
boolean isSignalled() { return getState() != 0; }
protected int tryAcquireShared(int ignore) {
return isSignalled()? 1 : -1;
}
protected boolean tryReleaseShared(int ignore) {
setState(1);
return true;
}
}
private final Sync sync = new Sync();
public boolean isSignalled() { return sync.isSignalled(); }
public void signal() { sync.releaseShared(1); }
public void await() throws InterruptedException {
sync.acquireSharedInterruptibly(1);
}
}
Doing a bit more searching, I found that a number of frameworks include a BooleanLatch (Apache Qpid being one). Some implementations (such as Atlassian's), are auto-resetting, which would make them inappropriate for my needs.

Standard observer notification pattern can be helpful here, if i understood your problem correctly.

For one shot signalling in this scenario Semaphore is better as it remembers the "signal".
Condition object [wait() is on a condition] won't remember the signal.
Semaphore numRecordsUpdated = new Semaphore(0);
In Generator
numRecordsUpdated.release();
In consumer
numRecordsUpdated.acquire();

can i know the Thread runnable class attributes in java?

probability this question have been asked before but i cant find anything in my searching mechanism. I am trying to create a multiple threads, in an array list but i want to retrieve them from an arraylist and filter them by the attribute of w1 i used in my code. any ideas ?
w1 = new FirstWorker(ProductsList, OrdersList, s);
FirstWorkerThread = new Thread(w1);
ThreadArrayList.add(FirstWorkerThread);
//I know i cant do the code below but i want to do that how ?
for(Thread x : ThreadArrayList){
x.ProductsList
}
this is FirstWorker class
import java.lang.String;
import java.util.HashMap;
/*
* To change this template, choose Tools | Templates and open the template in
* the editor.
*/
/**
*
* #author Dimitris
*/
public class FirstWorker extends Thread implements Runnable {
private OrderList orderlist;
private ProductList productlist;
private String Worker;
boolean Stop;
private int speed = 1000;
public FirstWorker(ProductList productlist, OrderList orderlist, String Worker) {
this.productlist = productlist;
this.orderlist = orderlist;
this.Worker = Worker;
this.Stop = true;
}
public void run() {
if (Stop == true) {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
}
while (orderlist.returnLengthofOrder() != 0) {
if (Thread.interrupted()) {
System.out.println("I am in the thread inturrupt");
// We've been interrupted: no more crunching.
return;
}
if (orderlist.getDone() == true) {
} else if (orderlist.getDone() == false) {
orderlist.setDoneTrue();
orderlist.Purchased(Worker);
orderlist.setDoneFalse();
try {
Thread.sleep(this.speed);
} catch (InterruptedException e) {
return;
}
}
}
}
}
public void setWork() {
Stop = false;
}
public void setSpeed(int speed) {
this.speed = speed;
}
}

If you want to access a member variable of your Runnable, you should extend Thread instead of implementing Runnable. Also, don't extend Thread AND implement Runnable. Pick one.
public class MyThread extends Thread
{
public int myarg;
public void run()
{
}
}
public void useThread(int inputArgs[])
{
ArrayList<MyThread> threadArray = new ArrayList<MyThread>();
for (int arg : inputArgs)
{
MyThread temp = new MyThread(arg);
temp.start();
threadArray.add(temp);
}
for (MyThread t : threadArray)
System.out.println(t.myarg);
}

The simple answer with constructing a Thread with a Runnable is no.
The constructor for Thread that you are using accepts a Runnable ... I assume that FirstWorker implements the Runnable interface.
But looking at the API docs for Thread http://docs.oracle.com/javase/6/docs/api/java/lang/Thread.html there is no method to return the Runnable.
Without knowing more context about what your trying to do, the simplest approach might be to change FirstWorker to extend Thread then the loop you have would work.
That would probably work, but would need to know more about what your doing to reccomend anything else.

If you want to retrieve properties from the Runnable instance within a Thread object, I do not believe that is generally possible, since the Thread class does not have a method to return its target Runnable object.
That said, you can always extend the Thread class itself, which would allow you to use instanceof with the Thread instances themselves before casting and getting to whatever property you need.
Keep in mind, though, that extending Thread is not a recommended practice and if you are intending to get the result of some computation straight from the Runnable object, you could run into some severe trouble if you are not careful.
In any case, recent Java versions (i.e. 1.5+) offer substantial capabilities for concurrency and I suspect that your application would benefit from a re-design that uses them.
We might be able to help more if you explained what exactly you are trying to do in broader terms...

You should consider using the new java.util.concurrent package. What you are trying to do can be implemented a lot easier and intuitively with an ExecutorService and a collection of Callables.
Check out this sample and the Executors API (specifically the fixedThreadPool).

How to wrap a callable in a type-friendly way?

I'm trying to implement a work queue in Java that limits the amount of work that can be taken at a time. In particular, it is trying to protect access to an external resource. My current approach is to use a Semaphore and a BlockingQueue so that I have something like this:
interface LimitingQueue<V> {
void put(Callable<V> work);
Callable<V> tryPoll();
}
It should behave like this:
#Test
public void workLimit() throws Exception {
final int workQueue = 2;
final LimitingQueue<Void> queue = new LimitingQueue<Void>(workQueue);
queue.put(new Work()); // Work is a Callable<Void> that just returns null.
queue.put(new Work());
// Verify that if we take out one piece of work, we don't get additional work.
Callable<Void> work = queue.tryPoll();
assertNotNull(work, "Queue should return work if none outstanding");
assertNull(queue.tryPoll(), "Queue should not return work if some outstanding");
// But we do after we complete the work.
work.call();
assertNotNull(queue.tryPoll(), "Queue should return work after outstanding work completed");
}
The implementation of tryPoll() uses Semaphore#tryAcquire and, if successful, creates an anonymous Callable that wraps the Semaphore#release call in a try/finally block around the call to work.call().
This works, but is somewhat unsatisfying in that if the user of this class puts work that is of some specific class that implements Callable, the user does not get access to that class back when looking at the result of tryPoll. Notably, tryPoll() returns a Callable<Void>, not a Work.
Is there a way to achieve what the work limitation effect while giving the caller back a usable reference to the work object that was submitted? (It's fine to strengthen the type signature of LimitingQueue to be more like LimitingQueue<R, T extends Callable<R>>.) I can't think of a way to ensure that the semaphore is released after calling the work item without doing this kind of wrapping.

EDIT2 I have replaced what was here with a suggestion on how to implement what you're looking for. Let me know if you want some of the old info back and I can restore it.
public class MyQueue<T> {
private Semaphore semaphore;
public void put(Work<T> w) {
w.setQueue(this);
}
public Work<T> tryPoll() {
return null;
}
public abstract static class Work<T> implements Callable<T> {
private MyQueue<T> queue;
private void setQueue(MyQueue<T> queue) {
if(queue != null) {
throw new IllegalStateException("Cannot add a Work object to multiple Queues!");
}
this.queue = queue;
}
#Override
public final T call() throws Exception {
try {
return callImpl();
} finally {
queue.semaphore.release();
}
}
protected abstract T callImpl() throws Exception;
}
}
Then use it like thus:
public class Test {
public static void main(String[] args) {
MyQueue<Integer> queue = new MyQueue<Integer>();
MyQueue.Work<Integer> work = new MyQueue.Work<Integer>() {
#Override
protected Integer callImpl() {
return 5;
}
};
queue.put(work);
MyQueue.Work<Integer> sameWork = queue.tryPoll();
}
}

Sounds to me like you should just use the builtin ExecutorService. Use Executors#newCachedThreadPool to get a pool, then submit Callable jobs which return back a Future.

How to demonstrate race conditions around values that aren't published properly?

I am reading "Java Concurrency in practice" and looking at the example code on page 51.
According to the book this piece of code is at risk of of failure if it has not been published properly. Because I like to code examples and break them to prove how they work. I have tried to make it throw an AssertionError but have failed. (Leading me to my previous question)
Can anyone post sample code so that an AssertionError is thrown? Rule: Do not modify the Holder class.
public class Holder{
private int n;
public Holder(int n){
this.n = n;
}
public void assertSanity(){
if (n != n) {
throw new AssertionError("This statement is false");
}
}
}
I have modified the class to make it more fragile but I still can not get an AssertionError thrown.
class Holder2 {
private int n;
private int n2;
public Holder2(int n) throws InterruptedException{
this.n = n;
Thread.sleep(200);
this.n2 = n;
}
public void assertSanity(){
if (n != n2) {
throw new AssertionError("This statement is false");
}
}
}
Is it possible to make either of the above classes throw an AssertionError? Or do we have to accept that they may occasionally do so and we can't write code to prove it?

I'd run this on a multiprocessor machine for a few hours and see what happens(remove the sleep if you use your Holder2). Such race conditions might be rare, or not existant on your particular machine - but atleast try to provoke these one on a million cases , by trying millions of times.
class Checker {
private Holder h;
public Checker() {
h = new Holder(42);
}
public void check() {
h.assertSanity();
}
public void create(int n) {
h = new Holder(n);
}
}
public class MyThread extends thread{
private bool check;
private final Checker c;
public MyThread(bool check,Checker c) {
this.check = check;
this.c = c;
}
public static void main(String[] args) {
Checker c = new Checker();
MyThread t1 = new MyThread(false,c);
MyThread t2 = new MyThread(true,c);
t1.start();
t2.start();
t1.join();
t2.join();
}
public void run() {
int n = 0;
while(true) {
if(check)
c.check();
else
c.create(n++);
}
}
}
}

As BobbyShaftoe said in the other thread, you can't rely on just running the code enough times to show that the error can or cannot happen. If you think about this from an Assembly level, it will be very hard for n != n as it is so few calls and relies on the process being switched out at a really precise time.
If you want to be able to show whether a concurrent system is provably valid it would be better to model it using something like Labelled Transition Systems. Try the LTSA tool if you're interested in proving concurrency or finding errors.
http://www.doc.ic.ac.uk/ltsa/

In the example the that book is giving the Holder class is not directly the cause of the problem, in fact it states that:
The problem here is not the Holder class itself, but that the Holder is not properly published. However, Holder can be made immune to improper publication by declaring the n field to be final, which would make Holder immutable; see Section 3.5.2.
Just prior to this it mentions the following code, which it the subject of the problem:
// Unsafe publication
public Holder holder;
public void initialize() {
holder = new Holder(42);
}
So to re-create it you will need to create a publisher class and two threads, one that calls initialize and one that calls the assert.
Having said that, I tried to re-create it myself and still failed to do so :(
Below is my first attempt, however there is a better explanation of the problem at http://forums.oracle.com/forums/thread.jspa?threadID=1140814&tstart=195
public class HolderTest {
#Test
public void testHolder() throws Exception {
for (int i = 0; i < 1000000000; i++) {
final CountDownLatch finished = new CountDownLatch(2);
final HolderPublisher publisher = new HolderPublisher();
final Thread publisherThread = new Thread(new Publisher(publisher,
finished));
final Thread checkerThread = new Thread(new Checker(publisher,
finished));
publisher.holder = null;
publisherThread.start();
checkerThread.start();
finished.await();
}
}
static class Publisher implements Runnable {
private final CountDownLatch finished;
private final HolderPublisher publisher;
public Publisher(final HolderPublisher publisher,
final CountDownLatch finished) {
this.publisher = publisher;
this.finished = finished;
}
#Override
public void run() {
try {
publisher.initialize();
} finally {
finished.countDown();
}
}
}
static class Checker implements Runnable {
private final CountDownLatch finished;
private final HolderPublisher publisher;
public Checker(final HolderPublisher publisher,
final CountDownLatch finished) {
this.publisher = publisher;
this.finished = finished;
}
#Override
public void run() {
try {
publisher.holder.assertSanity();
} catch (final NullPointerException e) {
// This isnt the error we are interested in so swallow it
} finally {
finished.countDown();
}
}
}
static class HolderPublisher {
// Unsafe publication
public Holder holder;
public void initialize() {
holder = new Holder(42);
}
}
}

I don't think the assertion error can occur without modifying the Holder class. I think the book is wrong.
The only reason to cause the assertion error is when assertSanity() is called on a partially constructed object. How can a thread, other than the constructor thread, reference a partially constructed object? AFAIK, it's only possible in the following two cases:
Publish this in the constructor. E.g. assign this to a shared variable. This can't happen in our sample code because Holder's constructor doesn't do that.
A class's non-static inner class can refer to its parent even when its parent is partially constructed. This can't happen either because Holder doesn't have any inner class.
Note that the following code in the book doesn't publish any partially constructed object:
public class GoodCode {
public Holder holder;
public void initialize () {
holder = new Holder(42);
}
}
If you disassemble initialize(), you get the following:
public void initialize();
Code:
0: aload_0
1: new #2 // class Holder
4: dup
5: bipush 42
7: invokespecial #3 // Method Holder."<init>":(I)V
10: putfield #4 // Field holder:LHolder;
13: return
Note that putfield holder executes after invokespecial <init>. This means the assignment of holder happens after the constructor is completed. The partially constructed object is only stored in the thread's stack. It's not published.
If you can trigger the assertion error in a reasonable way (e.g. reflection is not reasonable), put it here. I will up vote you.

You cant change value of n at any time by using:
Holder h = new Holder(5);
Field f = h.getClass().getDeclaredField("n");
f.setAccessible(true);
f.setInt(h, 10);
h.assertSanity();