We Keep Coding

Waiting for something to finsih, should I use thread.sleep or ReentrantLock?

I have a Java program. The logic is as follow:
place order out (relying on Interactive Broker / Binance API)
Once the order is filled (there will be a callback from the API), immediately execute a method called "calculateSomething"
The order is placed using Interactive Broker / Binance API. Once the order is filled, the API callback method will return a message.
The problem is that I do not know how to write out the code to identify that the order has been filled, so i can immediately execute the "calculateSomething" method with minimal waiting time.
I can think of two ways:
while loop and thread.sleep
ReentrantLock.
Method 1 works, but it's not instantaneous. Hence, I am exploring ReentrantLock and I am not sure the code is correct. Nonetheless, which method is the most efficient and can immediately execute the "calculateSomething" once the order is completed If there is a more efficient approach, please give me some help, as I have been stuck in this problem for many days.
pseudocode below.
Method 1 - thread.sleep
placeOrder(); // place order to binance <- API method
while(order is not completed){
Thread.sleep(1000)
if(order is completed){
return
}
}
calculateSomething();
Method 2 - ReentrantLock
ReentrantLock lock = new ReentrantLock();
lock.lock();
System.out.println("1. Locked");
try {
while(lock.isLocked()) {
if(isOrderCompleted() == true){
lock.unlock();
}
}
} catch(Exception e){
e.printStackTrace();
}finally {
if(lock.isLocked()) {
lock.unlock();
}
}
calculateSomething();

You can have a blocking queue.
BlockingQueue<?> finishedOrders = new ArrayBlockingQueue<>(512);
Then you have a loop that processes finished orders.
public void processFinishedOrders() throws InterruptedException{
while(!Thread.interrupted()){
finishedOrders.take();
doSomethingRelevant();
}
}
I would also suggest populating finishedOrders with a meaningful class.
BlockingQueue<Order> finishedOrders;
Order fin = finishedOrders.take();
doSomethingRelevant( fin );
That way the thread waiting on the api call can create a an order and add it to the finished orders queue, and the processing thread will have the relevant information.

java blockingqueue consumer block on full queue

I'm writing an small program to put tweets from the Twitter public stream into an HBase database. The program uses two threads, one to collect the tweets and one to process them.
The first thread uses twitter4j StatusListener to get the tweets and puts them in an ArrayBlockingQueue with an capacity of 100.
The second thread takes an status from the queue, filters the needed data and moves them to the database.
The processing takes more time, than the collecting of the status.
The producer looks like this:
public void onStatus(Status status) {
try {
this.queue.put(status);
} catch(Exception ex) {
ex.printStackTrace();
}
}
The consumer uses take and calls an function to process the new status:
public void run() {
try {
while(true) {
// Get new status to process
this.status = this.queue.take();
this.analyse();
}
} catch(Exception ex) {
ex.printStackTrace();
}
}
In the main function the two threads were created and started:
ArrayBlockingQueue<Status> queue_public = new ArrayBlockingQueue<Status>(100);
Thread ta_public = new Thread(new TweetAnalyser(cl.getOptionValue("config"), queue_public));
Thread st_public = new Thread(new RunPublicStream(cl.getOptionValue("config"), queue_public));
ta_public.start();
st_public.start();
The program runs for awhile without any problem, but then stops suddenly. At this time the queue is full and it seems, that the consumer is not able to take a new status from it. I tried several variations of the producer/consumer pattern without success. No exception is thrown.
I don't know were to look for the failure. I hope someone could give me an hint or an solution.

If working with blocking queues double check for blocking commands (put and take for ArrayBlockingQueue) in the code and typos if working with multiple lists.

Controlling thread using wait() and notify()

(Problem solved, solution below)
I have 2 classes: Equip and Command. The equip is an equipment that run commands, but I need it to be able to run only 1 command at the same time.
A command is a thread, that executes on the run() function, while Equip is a normal class that don't extend anything.
Currently I have the following setup to run the commands:
Command class:
#Override
public void run() {
boolean execute = equip.queueCommand(this);
if (!execute) {
// if this command is the only one on the queue, execute it, or wait.
esperar();
}
// executes the command.....
equip.executeNextCommand();
}
synchronized public void esperar() {
try {
this.wait();
} catch (Exception ex) {
Log.logErro(ex);
}
}
synchronized public void continue() {
this.notifyAll();
}
Equip class:
public boolean queueCommand(Command cmd) {
// commandQueue is a LinkedList
commandQueue.addLast(cmd);
return (commandQueue.size() == 1);
}
public void executeNextCommand() {
if (commandQueue.size() >= 1) {
Command cmd = commandQueue.pollFirst();
cmd.continue();
}
}
However, this is not working. Basically, the notify() isn't waking the command thread, so it'll never execute.
I searched about the wait and notify protocol, but I couldn't find anything wrong with the code. I also tried calling the wait() directly from the queueCommand() method, but then the execution of the queueCommand stopped, and it also didn't do what it was supposed to do.
Is this approach correct and I'm missing something or this is completely wrong and I should implement a Monitor class to manipulate the concurrent threads?
EDIT: I solved the problem using another completely different approach, using Executors, thanks to #Gray.
Here's the final code, it might help someone someday:
Equip class:
private ExecutorCompletionService commandQueue = new ExecutorCompletionService(Executors.newFixedThreadPool(1));
public void executeCommand(Command cmd, boolean waitCompletion) {
commandQueue.submit(cmd, null);
if (waitCompletion) {
try {
commandQueue.take();
} catch (Exception ex) {
}
}
}
In the Command class I just have a method to encapsulate the equip's execute method.
The boolean waitCompletion is used when I need the result of the command at the same time, and instead of calling a new thread to execute it, I just execute and wait, pretending that it's executing on the same thread. This question contains a good discussion on this matter: When would you call java's thread.run() instead of thread.start()?. And yes, this is a case where it's useful to call .run() instead of .start().

There are a large number of race conditions that exist in your code if Command.run() is called from multiple threads. Unless this is some sort of homework question where you have to implement the code yourself, I would highly recommend using one of the Java Executors which were added in 1.6. In this case the Executors.newSingleThreadExecutor() is what you need to limit the number of running background tasks to 1. This will allow an unlimited number of tasks to be submitted to the ExecutorService, but only one of those tasks will be executing at any one time.
If you need the thread that is submitting the tasks to block when another task is already running then you would use something like the following. This sets up a pool of a maximum of 1 thread and uses a SynchronousQueue which blocks until the worker thread consumes the job:
final ExecutorService executorServer =
new ThreadPoolExecutor(0, 1, 60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
But if that was the case then you would just call the task directly inside of a synchronized block and you wouldn't need the ExecutorService.
Lastly, for any new concurrency programmer (of any language) I would recommend that you take the time to read some documentation on the subject. Until you start recognizing the concurrent pitfalls inherent in threading even the simplest set of classes, it will be a frustrating process to get your code to work. Doug Lea's book is one of the bible's on the subject. My apologies if I have underestimated your experience in this area.

I think you should not have "synchronized" on the esperar method. That will block using the object instances as the locking object. Any other thread that attempts to wait will block AT ENTRY TO THE METHOD, not on the wait. So, the notifyAll will release the one thread that got into the method first. Of the remaining callers, only one will proceed with a call to esperar, which will then block on the wait(). Rinse and repeat.

ExectutorService is the way to go. But if you want to do-it-yourself, or need to do something fancier, I offer the following.
I gather than this whole thing is driven by Equip's queueCommand, which might be callled from any thread anywhere at any time. For starters, the two methods in Equip should by synchronized so commandQueue does not get trashed. (You might use ConcurrentLinkedQueue, but be careful with your counts.) Better still, put the code in each method in a block synchronized by queueCommand.
But further, I think your two classes work better combined. Switching Command to a simple Runnable, I'd try something like this:
class Equip {
private Object queueLock = new Object(); // Better than "this".
private LinkedList<Runnable> commandQueue = new LinkedList<Runnable>();
private void run() {
for (;;) {
Runnable cmd = equip.getNextCommand();
if (cmd == null) {
// Nothing to do.
synchronized (queueLock) { queueLock.wait(); }
}
else
cmd.run();
}
}
// Adds commands to run.
public boolean queueCommand( Runnable cmd ) {
synchronized (queueCommand) { commandQueue.addLast( cmd ); }
synchronized (queueLock) {
// Lets "run" know queue has something in it if it
// is in a wait state.
queueLock.notifyAll();
}
}
private Runnable getNextCommand() {
synchronized (queueCommand) { return commandQueue.pollFirst(); }
}
}
You'll need to catch some exceptions, and figure out how to start things up and shut them down, but this should give an idea of how the wait and notify work. (I'd look for some way to know when "run" was not waiting so I could skip synching on queueLock in queueCommand, but walk before you run.)

Have threads run indefinitely in a java application

I am trying to program a game in which I have a Table class and each person sitting at the table is a separate thread. The game involves the people passing tokens around and then stopping when the party chime sounds.
how do i program the run() method so that once I start the person threads, they do not die and are alive until the end of the game
One solution that I tried was having a while (true) {} loop in the run() method but that increases my CPU utilization to around 60-70 percent. Is there a better method?

While yes, you need a loop (while is only one way, but it is simplest) you also need to put something inside the loop that waits for things to happen and responds to them. You're aiming to have something like this pseudocode:
loop {
event = WaitForEvent();
RespondToEvent(event);
} until done;
OK, that's the view from 40,000 feet (where everything looks like ants!) but it's still the core of what you want. Oh, and you also need something to fire off the first event that starts the game, obviously.
So, the key then becomes the definition of WaitForEvent(). The classic there is to use a queue to hold the events, and to make blocking reads from the queue so that things wait until something else puts an event in the queue. This is really a Concurrency-101 data-structure, but an ArrayBlockingQueue is already defined correctly and so is what I'd use in my first implementation. You'll probably want to hide its use inside a subclass of Thread, perhaps like this:
public abstract class EventHandlingThread<Event> extends Thread {
private ArrayBlockingQueue<Event> queue = new ArrayBlockingQueue<Event>();
private boolean done;
protected abstract void respondToEvent(Event event);
public final void postEvent(Event event) throws InterruptedException {
queue.put(event);
}
protected final void done() {
done = true;
}
public final void run() {
try {
while (!done) {
respondToEvent(queue.take());
}
} catch (InterruptedException e) {
// Maybe log this, maybe not...
} catch (RuntimeException e) {
// Probably should log this!
}
}
}
Subclass that for each of your tasks and you should be able to get going nicely. The postEvent() method is called by other threads to send messages in, and you call done() on yourself when you've decided enough is enough. You should also make sure that you've always got some event that can be sent in which terminates things so that you can quit the game…

I would look into Locks and Conditions. This way you can write code that waits for a certain condition to happen. This won't take a lot of CPU power and is even much more efficient and better performing than sleeping .

To make a thread run for an infinite time:
final Object obj = new Object();
try {
Thread th = new Thread(new Runnable() {
public void run() {
synchronized(obj) {
try {
System.out.println("Waiting");
obj.wait();
System.out.println("Done waiting");
}catch(Exception ex) {
ex.printStackTrace();
}
}
}
});
th.start();
System.out.println("Waiting to join.");
// Dont notify; but wait for joining. This will ensure that main thread is running always.
th.join();
System.out.println("End of the Program");
} catch(Exception ex) {
ex.printStackTrace();
}

You may add Thread.sleep() with appropriate time to minimize useless loop iterations.
Another solution is using synchronization. While threads are not required to do anything, they enter into a sleeping state on a monitor using the wait() method, and then when the turn comes, required thread is woken up by the notify() method.

Actor model seems suitable for this scenario. Each person sitting on the table and the table itself can be modelled as actors and the event of passing the tokens and starting and stopping of the game can be modelled as messages to be passed between the actors.
As a bonus, by modelling the scenario as actors you get rid of explicit manipulation of threads, synchronization and locking.
On JVM I will prefer using Scala for modelling actors. For Java you can use libraries like Kilim. See this post for a comparison of Actor model related libraries in Java.

One Way is to use while loop but keep a check i.e
while(true){
if(condition!=true){
Thread.sleep(time);
}else{
break;
}
}
This way if your condition says game is not over it will keep person thread at sleep and memory consumption will be very low.

You should test for a condition in the while loop:
while (!gameOver)
{
do_intersting_stuff();
}
Heavy CPU load is typical for busy wait. Is your loop actually just checking a flag over and over, like
while (!gameOver)
{
if (actionNeeded)
{
do_something();
}
}
you might change to another notification system to sleep and wake up, as this just burns CPU time for nothing.

How can I wrap a method so that I can kill its execution if it exceeds a specified timeout?

I have a method that I would like to call. However, I'm looking for a clean, simple way to kill it or force it to return if it is taking too long to execute.
I'm using Java.
to illustrate:
logger.info("sequentially executing all batches...");
for (TestExecutor executor : builder.getExecutors()) {
logger.info("executing batch...");
executor.execute();
}
I figure the TestExecutor class should implement Callable and continue in that direction.
But all i want to be able to do is stop executor.execute() if it's taking too long.
Suggestions...?
EDIT
Many of the suggestions received assume that the method being executed that takes a long time contains some kind of loop and that a variable could periodically be checked.
However, this is not the case. So something that won't necessarily be clean and that will just stop the execution whereever it is is acceptable.

You should take a look at these classes :
FutureTask, Callable, Executors
Here is an example :
public class TimeoutExample {
public static Object myMethod() {
// does your thing and taking a long time to execute
return someResult;
}
public static void main(final String[] args) {
Callable<Object> callable = new Callable<Object>() {
public Object call() throws Exception {
return myMethod();
}
};
ExecutorService executorService = Executors.newCachedThreadPool();
Future<Object> task = executorService.submit(callable);
try {
// ok, wait for 30 seconds max
Object result = task.get(30, TimeUnit.SECONDS);
System.out.println("Finished with result: " + result);
} catch (ExecutionException e) {
throw new RuntimeException(e);
} catch (TimeoutException e) {
System.out.println("timeout...");
} catch (InterruptedException e) {
System.out.println("interrupted");
}
}
}

Java's interruption mechanism is intended for this kind of scenario. If the method that you wish to abort is executing a loop, just have it check the thread's interrupted status on every iteration. If it's interrupted, throw an InterruptedException.
Then, when you want to abort, you just have to invoke interrupt on the appropriate thread.
Alternatively, you can use the approach Sun suggest as an alternative to the deprecated stop method. This doesn't involve throwing any exceptions, the method would just return normally.

I'm assuming the use of multiple threads in the following statements.
I've done some reading in this area and most authors say that it's a bad idea to kill another thread.
If the function that you want to kill can be designed to periodically check a variable or synchronization primitive, and then terminate cleanly if that variable or synchronization primitive is set, that would be pretty clean. Then some sort of monitor thread can sleep for a number of milliseconds and then set the variable or synchronization primitive.

Really, you can't... The only way to do it is to either use thread.stop, agree on a 'cooperative' method (e.g. occassionally check for Thread.isInterrupted or call a method which throws an InterruptedException, e.g. Thread.sleep()), or somehow invoke the method in another JVM entirely.
For certain kinds of tests, calling stop() is okay, but it will probably damage the state of your test suite, so you'll have to relaunch the JVM after each call to stop() if you want to avoid interaction effects.
For a good description of how to implement the cooperative approach, check out Sun's FAQ on the deprecated Thread methods.
For an example of this approach in real life, Eclipse RCP's Job API's 'IProgressMonitor' object allows some management service to signal sub-processes (via the 'cancel' method) that they should stop. Of course, that relies on the methods to actually check the isCancelled method regularly, which they often fail to do.
A hybrid approach might be to ask the thread nicely with interrupt, then insist a couple of seconds later with stop. Again, you shouldn't use stop in production code, but it might be fine in this case, esp. if you exit the JVM soon after.
To test this approach, I wrote a simple harness, which takes a runnable and tries to execute it. Feel free to comment/edit.
public void testStop(Runnable r) {
Thread t = new Thread(r);
t.start();
try {
t.join(2000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
if (!t.isAlive()) {
System.err.println("Finished on time.");
return;
}
try {
t.interrupt();
t.join(2000);
if (!t.isAlive()) {
System.err.println("cooperative stop");
return;
}
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
System.err.println("non-cooperative stop");
StackTraceElement[] trace = Thread.getAllStackTraces().get(t);
if (null != trace) {
Throwable temp = new Throwable();
temp.setStackTrace(trace);
temp.printStackTrace();
}
t.stop();
System.err.println("stopped non-cooperative thread");
}
To test it, I wrote two competing infinite loops, one cooperative, and one that never checks its thread's interrupted bit.
public void cooperative() {
try {
for (;;) {
Thread.sleep(500);
}
} catch (InterruptedException e) {
System.err.println("cooperative() interrupted");
} finally {
System.err.println("cooperative() finally");
}
}
public void noncooperative() {
try {
for (;;) {
Thread.yield();
}
} finally {
System.err.println("noncooperative() finally");
}
}
Finally, I wrote the tests (JUnit 4) to exercise them:
#Test
public void testStopCooperative() {
testStop(new Runnable() {
#Override
public void run() {
cooperative();
}
});
}
#Test
public void testStopNoncooperative() {
testStop(new Runnable() {
#Override
public void run() {
noncooperative();
}
});
}
I had never used Thread.stop() before, so I was unaware of its operation. It works by throwing a ThreadDeath object from whereever the target thread is currently running. This extends Error. So, while it doesn't always work cleanly, it will usually leave simple programs with a fairly reasonable program state. For example, any finally blocks are called. If you wanted to be a real jerk, you could catch ThreadDeath (or Error), and keep running, anyway!
If nothing else, this really makes me wish more code followed the IProgressMonitor approach - adding another parameter to methods that might take a while, and encouraging the implementor of the method to occasionally poll the Monitor object to see if the user wants the system to give up. I'll try to follow this pattern in the future, especially methods that might be interactive. Of course, you don't necessarily know in advance which methods will be used this way, but that is what Profilers are for, I guess.
As for the 'start another JVM entirely' method, that will take more work. I don't know if anyone has written a delegating class loader, or if one is included in the JVM, but that would be required for this approach.

Nobody answered it directly, so here's the closest thing i can give you in a short amount of psuedo code:
wrap the method in a runnable/callable. The method itself is going to have to check for interrupted status if you want it to stop (for example, if this method is a loop, inside the loop check for Thread.currentThread().isInterrupted and if so, stop the loop (don't check on every iteration though, or you'll just slow stuff down.
in the wrapping method, use thread.join(timeout) to wait the time you want to let the method run. or, inside a loop there, call join repeatedly with a smaller timeout if you need to do other things while waiting. if the method doesn't finish, after joining, use the above recommendations for aborting fast/clean.
so code wise, old code:
void myMethod()
{
methodTakingAllTheTime();
}
new code:
void myMethod()
{
Thread t = new Thread(new Runnable()
{
public void run()
{
methodTakingAllTheTime(); // modify the internals of this method to check for interruption
}
});
t.join(5000); // 5 seconds
t.interrupt();
}
but again, for this to work well, you'll still have to modify methodTakingAllTheTime or that thread will just continue to run after you've called interrupt.

The correct answer is, I believe, to create a Runnable to execute the sub-program, and run this in a separate Thread. THe Runnable may be a FutureTask, which you can run with a timeout ("get" method). If it times out, you'll get a TimeoutException, in which I suggest you
call thread.interrupt() to attempt to end it in a semi-cooperative manner (many library calls seem to be sensitive to this, so it will probably work)
wait a little (Thread.sleep(300))
and then, if the thread is still active (thread.isActive()), call thread.stop(). This is a deprecated method, but apparently the only game in town short of running a separate process with all that this entails.
In my application, where I run untrusted, uncooperative code written by my beginner students, I do the above, ensuring that the killed thread never has (write) access to any objects that survive its death. This includes the object that houses the called method, which is discarded if a timeout occurs. (I tell my students to avoid timeouts, because their agent will be disqualified.) I am unsure about memory leaks...
I distinguish between long runtimes (method terminates) and hard timeouts - the hard timeouts are longer and meant to catch the case when code does not terminate at all, as opposed to being slow.
From my research, Java does not seem to have a non-deprecated provision for running non-cooperative code, which, in a way, is a gaping hole in the security model. Either I can run foreign code and control the permissions it has (SecurityManager), or I cannot run foreign code, because it might end up taking up a whole CPU with no non-deprecated means to stop it.
double x = 2.0;
while(true) {x = x*x}; // do not terminate
System.out.print(x); // prevent optimization

I can think of a not so great way to do this. If you can detect when it is taking too much time, you can have the method check for a boolean in every step. Have the program change the value of the boolean tooMuchTime to true if it is taking too much time (I can't help with this). Then use something like this:
Method(){
//task1
if (tooMuchTime == true) return;
//task2
if (tooMuchTime == true) return;
//task3
if (tooMuchTime == true) return;
//task4
if (tooMuchTime == true) return;
//task5
if (tooMuchTime == true) return;
//final task
}