I have a thread downloading data and I want to wait until the download is finished before I load the data. Is there a standard way of doing this?
More Info:
I have a Download class that gets data from a URL (Serialized POJOs). Download is Runnable and Observable. It keeps track of the bytes downloaded and download size. I have a progress bar that displays the progress to the User. The GUI observes Download to update the progress bar.
When the POJO is downloaded I want to get it and move to the next step. Each step has to wait for the previous to finish. The problem is I cant think of a way to pause my application to wait for the download thread. Once the download is finished I want to call download.getObject() which will return the data as an object. I can then cast it and get on with the next download.
I have a helper class that manages the URLs for download and makes all of the calls to Download. This call will call getObject and do the casting. The Gui calls helper.getUser(). helper starts the thread running and I want it to 'know' when it is finished so it can return the casted object.
Any suggestions/examples? I am in the beginning stages of this design so I am willing to change it.
Update:
I followed http://download.oracle.com/javase/6/docs/api/javax/swing/SwingWorker.html#get and used modal to block until the thread finished. The code was very messy and I don't like this approach. I will keep trying to find a 'clean' way to handle the workflow of the download processes.
Thread has a method that does that for you join which will block until the thread has finished executing.
You could use a CountDownLatch from the java.util.concurrent package. It is very useful when waiting for one or more threads to complete before continuing execution in the awaiting thread.
For example, waiting for three tasks to complete:
CountDownLatch latch = new CountDownLatch(3);
...
latch.await(); // Wait for countdown
The other thread(s) then each call latch.countDown() when complete with the their tasks. Once the countdown is complete, three in this example, the execution will continue.
Better alternatives to join() method have been evolved over a period of time.
ExecutorService.html#invokeAll is one alternative.
Executes the given tasks, returning a list of Futures holding their status and results when all complete. Future.isDone() is true for each element of the returned list.
Note that a completed task could have terminated either normally or by throwing an exception. The results of this method are undefined if the given collection is modified while this operation is in progress.
ForkJoinPool or Executors.html#newWorkStealingPool provides other alternatives to achieve the same purpose.
Example code snippet:
import java.util.concurrent.*;
import java.util.*;
public class InvokeAllDemo{
public InvokeAllDemo(){
System.out.println("creating service");
ExecutorService service = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
List<MyCallable> futureList = new ArrayList<MyCallable>();
for ( int i=0; i<10; i++){
MyCallable myCallable = new MyCallable((long)i);
futureList.add(myCallable);
}
System.out.println("Start");
try{
List<Future<Long>> futures = service.invokeAll(futureList);
}catch(Exception err){
err.printStackTrace();
}
System.out.println("Completed");
service.shutdown();
}
public static void main(String args[]){
InvokeAllDemo demo = new InvokeAllDemo();
}
class MyCallable implements Callable<Long>{
Long id = 0L;
public MyCallable(Long val){
this.id = val;
}
public Long call(){
// Add your business logic
return id;
}
}
}
You can use join() to wait for all threads to finish. Like below:
for (int i = 0; i < 10; i++)
{
Thread T1 = new Thread(new ThreadTest(i));
T1.start();
try {
T1.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
SwingWorker has doInBackground() which you can use to perform a task. You have the option to invoke get() and wait for the download to complete or you can override the done() method which will be invoked on the event dispatch thread once the SwingWorker completes.
The Swingworker has advantages to your current approach in that it has many of the features you are looking for so there is no need to reinvent the wheel. You are able to use the getProgress() and setProgress() methods as an alternative to an observer on the runnable for download progress. The done() method as I stated above is called after the worker finishes executing and is performed on the EDT, this allows you load the data after the download has completed.
I imagine that you're calling your download in a background thread such as provided by a SwingWorker. If so, then simply call your next code sequentially in the same SwingWorker's doInBackground method.
Generally, when you want to wait for a thread to finish, you should call join() on it.
Any suggestions/examples? I followed SwingWorker... The code was very messy and I don't like this approach.
Instead of get(), which waits for completion, use process() and setProgress() to show intermediate results, as suggested in this simple example or this related example.
The join() method allows one thread to wait for the completion of another.However, as with sleep, join is dependent on the OS for timing, so you should not assume that join will wait exactly as long as you specify.
Related
I am using an ExecutorService with a fixedThreadPool. I create threads by implementing the Runnable interface. In the run() method, I am calling a time consuming function (let's say Thread.sleep() for now) and finally add an element to a thread safe CopyOnWriteArrayList. I am submitting threads in a for-loop. At the end of the for-loop I shutdown the ExecutorService and await termination.
However, the number of elements in the list does not turn out to be the same as the number of loop-iterations. Where is my mistake in concurrency thinking?
Here is a code example for this problem:
public class TestProgram {
public static void main(String[] args) throws Exception {
ExecutorService executor = Executors.newFixedThreadPool(4);
CopyOnWriteArrayList<String> stringList = new CopyOnWriteArrayList<String>();
for (int i = 0; i < 1000; i++) {
executor.submit(new myThread(stringList));
}
executor.shutdown();
executor.awaitTermination(10, TimeUnit.SECONDS);
System.out.println(Integer.toString(stringList.size()));
}
}
class myThread implements Runnable {
CopyOnWriteArrayList<String> stringList;
public myThread(CopyOnWriteArrayList<String> stringList) {
this.stringList = stringList;
}
public void run() {
String string = new String("test");
try {
Thread.sleep(100);
} catch (InterruptedException ex) {
Logger.getLogger(myThread.class.getName()).log(Level.SEVERE, null, ex);
}
stringList.add(string);
}
}
You're just not giving it enough time to complete. If you want to execute this serially you would need 1000 * 100 milliseconds, which is 100 seconds. You are running it with four threads, so divide that by 4 and you get 25 seconds. You only wait 10 seconds to complete. Increase that to 26 seconds just to be safe and you should see your result.
The ExecutorService#awaitTermination will wait the N number of seconds to complete, if it doesn't complete it will simply return out and continue the execution. To prove my point check the return value
System.out.println(executor.awaitTermination(10, TimeUnit.SECONDS));
The better solution here, is to use a new CountDownLatch(1000), have each Runnable task countDown after it adds to the list and finally have the main thread await completion.
The documentation of ExecutorService.shutdown says the following:
Initiates an orderly shutdown in which previously submitted tasks are
executed, but no new tasks will be accepted. Invocation has no
additional effect if already shut down.
This method does not wait for previously submitted tasks to complete
execution. Use awaitTermination to do that.
This means that you indicate your ExecutorService that it must stop it's services. Thus, this method sets a flag which indicates that the ExecutorService will end soon, which allows the main thread to returns from this method without problems.
ExecutorService.awaitTermination, moreover, reads as follows:
Blocks until all tasks have completed execution after a shutdown
request, or the timeout occurs, or the current thread is interrupted,
whichever happens first.
This means that the main thread will be blocked inside the method, where it will only be returned when the tasks sent to ExecutorService finish. In both cases, the developer intends to end the ExecutorService function. But with awaitTermination, the developer is saying ExecutorService should be terminated regardless of their tasks have been carried out or not.
As each task takes at least 100 milliseconds to complete, no one will possibly be completed because ExecutorService has a tolerance of only 10 milliseconds for completion of all it's tasks.
There is no point in calling shutdown and awaitTermination at the same time (or preceded by another). According to your code, shutdown has no effect. If you want your tasks to become completed, and that ExecutorService is terminated without caring if your main thread will continue, just use shutdown. If you do not want your main thread to be blocked, and simply want to end with ExecutorService, use shutdownNow.
If, on the other hand, you want your main thread to wait for the execution of your tasks without a certain time to be specified, you may have to use Java Synchronizers. Some of Java Synchronizers are as follow:
Latches
FutureTasks
Semaphores
Barriers
For your case, I believe you can use CountDownLatch. Make a new CountDownLatch object have the number of tasks running on your ExecutorService. As they are completed, the count is decreased, and so, your main thread can expect completion and continue after all.
I hope you have understood and you can do what you want. If you have further questions, the documentation of CountDownLatch demonstrates perfectly how you can synchronize tasks. Good luck.
In that code snippet one issue will be there.
executor.awaitTermination(10, TimeUnit.SECONDS);
Here it will wait for 10 seconds after that it will break.
You will try following snippet.
while (true) {
if(executor.awaitTermination(10, TimeUnit.SECONDS))
break;
}
The main thing is that the callback should be on the PARENT thread and not called from the task, which is finishing the job (e.g. NOT like ThreadPoolExecutor's afterExecute() hook).
I'd like to have something like:
ExecutorService exec = Executors.newSingleThreadExecutor();
>>>>> exec.addAllJobsFinishedListener(someListener) <<<<<
exec.submit(task);
and someListener to have an Overrideable method like allJobsFinished()
Non-blocking method to start several threads and run a callback on the parent thread when all children have finished
If you want the callback on PARENT thread, then I'm afraid you need to have the parent thread calling exec.awaitTermination(...). After you have submitted all of the jobs and the exec.shutdown() has been called, this waits for the all of the jobs in the thread-pool to complete. If you want this to be non-blocking then you are going to have to do this in another thread (running in a different pool of course).
I don't see how it can be a "listener" running on the parent thread that is also non-blocking. You could have a background thread checking which will update some sort of shared state (like an AtomicBoolean) to be true when the exec.awaitTermination(...) completes.
ThreadPoolExecutor does have terminate() method that you can override. Is that not what you want? Here are the javadocs for the method. Your code would look something like:
ThreadPoolExecutor threadPool =
new ThreadPoolExecutor(10, 10, 0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue()) {
public void terminated() {
super.terminated();
// do something magic
}
};
threadPool.execute(task);
// ...
// need to shutdown the pool after you've submitted all of the tasks
threadPool.shutdown();
If the "something magic" code set a shared volatile boolean or an AtomicBoolean that the main thread could check, then it should work fine.
Looks like ThreadPoolExecutor class is really good option for you and you can call getActiveCount() method to check for running threads without being blocked. Also look on this article for example based on shutdown() and isTerminated() which is pretty simple and nice as for me.
What this solution does not cover is waiting cycle. Some kind of sleeping should handle this. Probably standard combination of Condition and Lock (look for POSIX condition variable example) is also acceptable solution for this but it lacks beauty of previous approach.
Look for this, I think it is right for you:
package com.journaldev.threadpool;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class SimpleThreadPool {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
Runnable worker = new WorkerThread('' + i);
executor.execute(worker);
}
executor.shutdown();
while (!executor.isTerminated()) {
// Put other work or calls here...
}
System.out.println('Finished all threads');
}
}
Please pay attention to the following facts:
executor.shutdown() just blocks new threads spawning.
executor.isTerminated() doesn't block you. You are free to do other job.
Using Java's ProcessBuilder I am creating a group of child processes. I can use the waitFor() method from the resulting Process object to wait for that particular child to exit.
Is it possible to block until any child exits, in the manner of the UNIX wait() system call?
First step is to represent the work done by each subprocess as a Future, like so:
final ProcessBuilder builder = ...;
// for each process you're going to launch
FutureTask task = new FutureTask(new Callable<Integer>() {
#Override public Integer call() {
return builder.start().waitFor();
}
};
Now submit all the tasks to an executor:
ExecutorService executor = Executors.newCachedThreadPool();
for (FutureTask task : tasks) {
executor.submit(task);
}
// no more tasks are going to be submitted, this will let the executor clean up its threads
executor.shutdown();
Now use the excellent ExecutorCompletionService class:
ExecutorCompletionService service = new ExecutorCompletionService(executor);
while (!executor.isTerminated()) {
Future<Integer> finishedFuture = service.take();
System.out.println("Finishing process returned " + finishedFuture.get());
}
This loop will iterate once for every completing task as it finishes. The returnValue will be the exit code of the child process.
Now, you don't know exactly which process has finished. You could change the Callable to instead of returning an Integer to just return the Process or even better a class of your own representing the output of the process.
Oh and of course if you don't care about waiting for all the tasks, you can just call take() once.
Read about the CountDownLatch
A CountDownLatch is initialized with a
given count. The await methods block
until the current count reaches zero
due to invocations of the countDown()
method, after which all waiting
threads are released and any
subsequent invocations of await return
immediately. This is a one-shot
phenomenon -- the count cannot be
reset. If you need a version that
resets the count, consider using a
CyclicBarrier.
you have to use some form of IPC to achieve this. If you are allowed to use native libraries and if you work on UNIX/Linux platform try using the same wait() system call by writing a simple JNI wrapper & calling the native method from java code.
If you cannot use native IPC mechanisms use TCP/IP server/client mechanism in which you control the child process exit from the server while the client connects/disconnects to/from the server. When there are no child connections you can exit the server program!
I have question about the Java threads. Here is my scenario:
I have a thread calling a method that could take while. The thread keeps itself on that method until I get the result. If I send another request to that method in the same way, now there are two threads running (provided the first did not return the result yet). But I want to give the priority to the last thread and don't want to get the results from the previously started threads. So how could I get rid of earlier threads when I do not have a stop method?
The standard design pattern is to use a local variable in the thread that can be set to stop it:
public class MyThread extends Thread {
private volatile boolean running = true;
public void stop() {
running = false;
}
public void run() {
while (running) {
// do your things
}
}
}
This way you can greacefully terminate the thread, i.e. without throwing an InterruptedException.
The best way really depends on what that method does. If it waits on something, chances are an interrupt will result in an InterruptedException which you handle and cleanly exit. If it's doing something busy, it won't:
class Scratchpad {
public static void main(String[] a) {
Thread t = new Thread(new Runnable() {
public void run() {doWork();}
});
t.start();
try {
Thread.sleep(50);
} catch (InterruptedException ie) {}
t.interrupt();
}
private static void doWork() {
for ( long i = 1; i != 0; i *=5 );
}
}
In the case above, the only viable solution really is a flag variable to break out of the loop early on a cancel, ala #inflagranti.
Another option for event-driven architectures is the poison-pill: if your method is waiting on a blocking queue for a new item, then you can have a global constant item called the "poison-pill" that when consumed (dequeued) you kill the thread:
try {
while(true) {
SomeType next = queue.take();
if ( next == POISON_PILL ) {
return;
}
consume(next);
}
} catch //...
EDIT:
It looks like what you really want is an executor service. When you submit a job to an executor service, you get back a Future which you can use to track results and cancel the job.
You can interrupt a Thread, its execution chain will throw an InterruptedException most of the time (see special cases in the documentation).
If you just want to slow down the other thread and not have it exit, you can take some other approach...
For one thing, just like exiting you can have a de-prioritize variable that, when set, puts your thread to sleep for 100ms on each iteration. This would effectively stop it while your other thread searched, then when you re-prioritize it it would go back to full speed.
However, this is a little sloppy. Since you only ever want one thing running but you want to have it remember to process others when the priority one is done, you may want to place your processing into a class with a .process() method that is called repeatedly. When you wish to suspend processing of that request you simply stop calling .process on that object for a while.
In this way you can implement a stack of such objects and your thread would just execute stack.peek().process(); every iteration, so pushing a new, more important task onto the stack would automatically stop any previous task from operating.
This leads to much more flexible scheduling--for instance you could have process() return false if there is nothing for it to do at which point your scheduler might go to the next item on the stack and try its' process() method, giving you some serious multi-tasking ability in a single thread without overtaxing your resources (network, I'm guessing)
There is a setPriority(int) method for Thread. You can set the first thread its priority like this:
Thread t = new Thread(yourRunnable);
t.start();
t.setPriority(Thread.MIN_PRIORITY); // The range goes from 1 to 10, I think
But this won't kill your thread. If you have only two threads using your runnable, then this is a good solution. But if you create threads in a loop and you always sets the priority of the last thread to minimum, you will get a lot of threads.
If this is what is application is going to do, take a look at a ThreadPool. This isn't an existing class in the Java API. You will have create one by yourself.
A ThreadPool is another Thread that manages all your other Threads the way you want. You can set a maximum number of running Threads. And in that ThreadPool, you can implement a system that manages the Thread priority automatically. Eg: You can make that older threads gain more priority, so you can properly end them.
So, if you know how to work with a ThreadPool, it can be very interesting.
According to java.lang.Thread API, you should use interrupt() method and check for isInterrupted() flag while you're doing some time-consuming cancelable operation. This approach allows to deal with different kind of "waiting situations":
1. wait(), join() and sleep() methods will throw InterruptedExcetion after you invoke interrupt() method
2. If thread blocked by java.nio.channels.Selector it will finish selector operation
3. If you're waiting for I/O thread will receive ClosedByInterruptException, but in this case your I/O facility must implement InterruptibleChannel interface.
If it's not possible to interrupt this action in a generic way, you could simply abandon previous thread and get results from a new one. You could do it by means of java.util.concurrent.Future and java.util.concurrent.ExecutorService.
Cosider following code snippet:
public class RequestService<Result> {
private ExecutorService executor = Executors.newFixedThreadPool(3);
private Future<Result> result;
public Future<Result> doRequest(){
if(result !=null){
result.cancel(true);
}
result = executor.submit(new Callable<Result>() {
public Result call() throws Exception {
// do your long-running service call here
}
});
return result;
}
}
Future object here represents a results of service call. If you invoke doRequest method one more time, it attempts to cancel previous task and then try to submit new request. As far as thread pool contain more than one thread, you won't have to wait until previous request is cancelled. New request is submitted immediately and method returns you a new result of request.
I'm trying to execute a SwingWorker (SubWorker) from another SwingWorker (MainWorker), and then I want the MainWorker to wait for the SubWorker to complete. In the mean time, the MainWorker should update itself according to property changes of the SubWorker.
public class MainWorker extends SwingWorker<Void,Void>
{
public Void doInBackground()
{
SubWorker sw = new SubWorker();
sw.execute();
try {
network.get(); // wait for completion
} catch (Exception ex) {
}
return null;
}
}
The problem is that the SubWorker's doInBackground method is not called until the MainWorker has finished, while the MainWorker is waiting for the SubWorker to finish.
How can I let the SubWorker run parallel to the MainWorker's activities?
If you have only one sub-worker, I'm not sure what is the point - why can't you just execute the task in the SW body?
If you have multiple parallel sub-tasks, and you want to group them and report on their progress, you don't really need separate SwingWorkers - just start threads or use an executor to schedule the tasks. If you want to publish interim results, pass a blocking queue to the tasks, where they would push the updates.
The main (or rather the only) SwingWorker would take items from that queue and update the GUI using the publish()/process() methods. When a subtask finishes work, it can push special token in the queue (or null). That's how you can keep track of the outstanding subtasks and decide when to terminate the loop.
Alternatively, if you have a bunch of self-contained tasks you can use CompletionService and update the status in similar way from the SW.