This question already has answers here:
wait until all threads finish their work in java
(17 answers)
Closed 6 years ago.
Say I'm using a HTTP requests library for downloading files. This library uses threads inside. Now, I want to wait on the main thread until other threads complete their execution.
All the other solutions that I found by googling only work if I have access to the Thread variables that were used in the library. But these are not accessible to me.
Here's what i'm using currently:
package smartzero.eightnoteight.testfirebase;
import com.firebase.client.AuthData;
import com.firebase.client.Firebase;
import com.firebase.client.FirebaseError;
import java.util.Scanner;
public class Main {
public static void main(String[] args) {
Scanner in = new Scanner(System.in);
System.out.print("email: ");
String email = in.nextLine();
System.out.print("password: ");
String password = in.nextLine();
Firebase fb = new Firebase("https://nullform.firebaseio.com");
fb.authWithPassword(email, password, new AuthResultHandler());
try {
Thread.sleep(1000000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public static class AuthResultHandler implements Firebase.AuthResultHandler {
#Override
public void onAuthenticated(AuthData authData) {
System.out.println("authentication successful");
String uid = authData.getUid();
new RunTests(uid);
}
#Override
public void onAuthenticationError(FirebaseError firebaseError) {
System.out.println("authentication failed.");
}
}
}
PS: i'm testing firebase using firebase-client-jvm on my pc.
You should use the events provided by Firebase:
fb.authWithPassword(email, password, new AuthResultHandler(){
#Override
public void onAuthenticated(AuthData authData) {
//do something if authentication successful
}
#Override
public void onAuthenticationError(FirebaseError error) {
//handle error
}
});
You could, if you really want to wait in the main do this:
void main(String[] args) {
boolean finished = false;
fb.authWithPassword(email, password, new AuthResultHandler(){
#Override
public void onAuthenticated(AuthData authData) {
finished = true;
}
});
while (!finished){
Thread.sleep(1);
}
}
Thats more of a pseudocode. It doesnt catch the interrupted exception and blocks forever if there is an error (onAuthenticationError). Also i would not recommend this. Busy waiting is almost never a good idea.
On non-Android runtimes the Firebase Java client uses daemon threads, which will not prevent a process from exiting. You must handle this using a CountdownLatch or a Semaphore.
CountdownLatch
A synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes.
Your code on it:
package smartzero.eightnoteight.testfirebase;
import com.firebase.client.AuthData;
import com.firebase.client.Firebase;
import com.firebase.client.FirebaseError;
import java.util.Scanner;
import java.util.concurrent.CountDownLatch;
public class Main {
public static void main(String[] args) throws InterruptedException {
Scanner in = new Scanner(System.in);
System.out.print("email: ");
String email = in.nextLine();
System.out.print("password: ");
String password = in.nextLine();
in.close();
Firebase fb = new Firebase("https://nullform.firebaseio.com");
CountDownLatch done = new CountDownLatch(1);
fb.authWithPassword(email, password, new Firebase.AuthResultHandler(){
#Override
public void onAuthenticated(AuthData authData) {
System.out.println("authentication successful");
done.countDown();
}
#Override
public void onAuthenticationError(FirebaseError error) {
System.out.println("authentication failed.");
done.countDown();
}
});
done.await();
}
}
Semaphore
It is used to control the number of concurrent threads that are using a resource. You could think of it as tickets to use a resource. You set the number of tickets available when you create it, and when acquire() is called with no tickets left, your process will wait for one to become available (on a release() call). On your code it is being created with zero "tickets" available:
package smartzero.eightnoteight.testfirebase;
import com.firebase.client.AuthData;
import com.firebase.client.Firebase;
import com.firebase.client.FirebaseError;
import java.util.Scanner;
import java.util.concurrent.Semaphore;
public class Main {
public static void main(String[] args) throws InterruptedException {
Scanner in = new Scanner(System.in);
System.out.print("email: ");
String email = in.nextLine();
System.out.print("password: ");
String password = in.nextLine();
in.close();
Firebase fb = new Firebase("https://nullform.firebaseio.com");
Semaphore semaphore = new Semaphore(0);
fb.authWithPassword(email, password, new Firebase.AuthResultHandler(){
#Override
public void onAuthenticated(AuthData authData) {
System.out.println("authentication successful");
semaphore.release();
}
#Override
public void onAuthenticationError(FirebaseError error) {
System.out.println("authentication failed.");
semaphore.release();
}
});
semaphore.acquire();
}
}
A CountdownLatch is really good for having one thread wait for one or more threads to complete one or more tasks before proceeding.
First, create the countdown latch with count of n, where n is the number events you want to wait on. Next, give the latch to the thread or threads doing the work. After that, the thread that should wait calls await() on the latch, and simultaneously the other threads begin working. When each of the worker threads is done, they call countdown() on the latch. When the latch counter hits zero, the waiting thread (or possibly threads) will unblock.
Related
I am generating a child thread when I receive data from user.
What are the steps if I want to dispose the previous user child thread if the same user sends data again and wants to generate a new user child thread again?
Right, so java can't dispose of the thread, a thread simply runs until it terminates.
So:
To get rid of the thread you need to allow the threads run method to end and then get rid of all references to the Thread and any Runnable it's constructed with.
You want to toggle the thread finishing so, a simple example:
class SimpleRunnable implements Runnable {
public volatile boolean run = true; //Volatile for thread safety.
public void run() {
while(run) {
System.out.println("WHOOOO!"); //Boy, will this be annoying
}
}
}
Creating a thread from this runnable:
SimpleRunnable run = new SimpleRunnable();
Thread thread = new Thread(run);
Thread.start(); //run thread
//Stop thread
run.run=false;
//Thread will be removed when out of scope
Youu need to create a Runnable per user in your case, and then call set the stop variable when a new thread is created.
For example, you could store each runnable in a ConcurrentHashMap by userId.
ConcurrentHashMap<String,SimpleRunnable> runnablesByUser = new ConcurrentHashMap<>();
public void startNewThreadForUser(String userId){
//Time passes, retrieve and kill old thread:
SimpleRunnable oldRunnable = runnableByUser.get(userId);
if(oldRunnable!=null){
oldRunnable.run=false;
}
SimpleRunnable newRunnableUserOne = new SimpleRunnable();
runnablesByUser.put(userId,newRunnableUserOne);
Thread thread = new Thread(newRunnableUserOne);
thread.start();
}
Calls to the method would then kill an old thread if found, release the old one from scope by replacing it with a new one in the ConcurrentHashMap and finally start the new thread.
Like so:
public void startThreeThreads(){
startNewThreadForUser("User1");//starts Thread for User1
startNewThreadForUser("User2");//starts Thread for User2
startNewThreadForUser("User1");//Replaces Thread for User1
}
Managing running threads is typically done in a thread pool and this is rough in all sorts of ways, but hopefully it's useful.
I can elaborate that mechanism if you want.
Starting a new thread every time that you receive data from a user will lead to running out of resources, besides causing an unnecessary overhead of managing too many threads. Your computer has a limited number of threads that can run at any single time and is limited by your CPU. to find out that number you can use command
Runtime.getRuntime().availableProcessors()
on the other hand, if the jobs that you want to process require a lot of I/O processing, you should launch a few more threads than "Runtime.getRuntime().availableProcessors()", or you will be under-using your CPU.
what I would do is to use a "ExecutorService" which will handle the threads for you (no need to manually start, stop threads). Just start an "ExecutorService" with the total number of threads that you want to execute simultaneously, and then every time that you get more work from a User, submit the new task (as a Callable) to the ExecutorService. The executorService will handle the execution of that task for you, and once it is done it will become available for garbage collection.
for example, see code below:
import java.util.Random;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class MultipleClientsExample {
public static final int TOTAL_THREADS_TO_PROCESS_CUSTOMER_WORK = 4;
public static final Random random = new Random();
public static int customerCounter = 0;
public static void main(String[] args) throws InterruptedException {
MultipleClientsExample multipleClientsExample = new MultipleClientsExample();
multipleClientsExample.doTheWork();
}
private void doTheWork() throws InterruptedException {
ExecutorService executorService = Executors.newFixedThreadPool(TOTAL_THREADS_TO_PROCESS_CUSTOMER_WORK);
while (customerCounter < 10) {
try {
CustomerInput customerInput = getWorkFromCustomer();
System.out.println("main program. received work from customer: " + customerInput.getCustomerId());
executorService.submit(new WorkToBeDone(customerInput.getCustomerId(), customerInput.getWorkInfo()));
} catch (InterruptedException e) {
break;
}
customerCounter++;
}
executorService.shutdown();
executorService.awaitTermination(5, TimeUnit.SECONDS);
}
private CustomerInput getWorkFromCustomer() throws InterruptedException {
while (true) {
String customerId = String.valueOf(random.nextInt(10));
CustomerInput customerInput = new CustomerInput(customerId, "work from customer: " + customerId);
return customerInput;
}
}
}
class WorkToBeDone implements Callable<Void> {
private String clientId;
private String workInfo;
public WorkToBeDone(String clientId, String workInfo) {
this.clientId = clientId;
this.workInfo = workInfo;
}
#Override
public Void call() throws Exception {
System.out.println("inside a working thread: it is going to do the work of customer: " + clientId);
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
System.out.println("worker processing job from customer: " + clientId + " was interrupted. ending now");
return null;
}
System.out.println("work completed for customer: " + clientId);
return null;
}
}
class CustomerInput {
private String customerId;
private String workInfo;
public CustomerInput(String customerId, String workInfo) {
this.customerId = customerId;
this.workInfo = workInfo;
}
public String getCustomerId() {
return customerId;
}
public String getWorkInfo() {
return workInfo;
}
}
In case you want the ability to cancel a task that has already been submitted to the thread pool, you will have to keep reference of the Future values of each task, and make sure to remove the reference of the tasks that completed and that you cancelled, so they are ready to be garbage collected (otherwise you will have a memory leak).
for example
import java.util.Iterator;
import java.util.Map;
import java.util.concurrent.Callable;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
public class MultipleClientsExample {
public static final int TOTAL_THREADS_TO_PROCESS_CUSTOMER_WORK = 4;
public static int customerCounter = 0;
public static void main(String[] args) throws InterruptedException {
MultipleClientsExample multipleClientsExample = new MultipleClientsExample();
multipleClientsExample.doTheWork();
}
private void doTheWork() throws InterruptedException {
final ExecutorService executorService = Executors.newFixedThreadPool(TOTAL_THREADS_TO_PROCESS_CUSTOMER_WORK);
Map<String, Future<String>> map = new ConcurrentHashMap<>();
while (customerCounter < 11) {
try {
WorkToBeDone workToBeDone = getWorkFromCustomer();
System.out.println("main program. received work from customer: " + workToBeDone.getClientId());
Future<String> resultFuture = executorService.submit(workToBeDone);
map.put(workToBeDone.getClientId(), resultFuture);
} catch (InterruptedException e) {
break;
}
customerCounter++;
}
// cancel job of customer with id: 10
Future<String> resultFuture = map.get("10");
System.out.println("cancelling job of customerId: 10");
resultFuture.cancel(true);
// remove references of all completed jobs
Thread.sleep(2000);
System.out.println("looking for jobs that completed or were cancelled.");
Iterator<Map.Entry<String, Future<String>>> iterator = map.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, Future<String>> entry = iterator.next();
if (entry.getValue().isCancelled() || entry.getValue().isDone()) {
System.out.println("removing reference of job for customer: " + entry.getKey());
iterator.remove();
}
}
// simpler way to remove entries from map (but doesn't print output of jobs removed from map)
// map.entrySet().removeIf(entry -> entry.getValue().isCancelled() || entry.getValue().isDone());
executorService.shutdown();
executorService.awaitTermination(5, TimeUnit.SECONDS);
}
private WorkToBeDone getWorkFromCustomer() throws InterruptedException {
String customerId = String.valueOf(customerCounter);
WorkToBeDone workToBeDone = new WorkToBeDone(customerId, "work from customer: " + customerId);
return workToBeDone;
}
}
class WorkToBeDone implements Callable<String> {
private String clientId;
private String workInfo;
public String getClientId() {
return clientId;
}
public WorkToBeDone(String clientId, String workInfo) {
this.clientId = clientId;
this.workInfo = workInfo;
}
#Override
public String call() throws Exception {
System.out.println("inside a working thread: it is going to do the work of customer: " + clientId);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
System.out.println("worker processing job from customer: " + clientId + " was interrupted. ending now");
return clientId;
}
System.out.println("work completed for customer: " + clientId);
return clientId;
}
}
I have a thread in Java that is connecting to a socket and sending information to another thread, which is processing that information.
Now, if the "producer" thread fails for any reason, I want the whole program to stop, so some sort of notification must happen.
Here's my program (very simplified):
public class Main {
private Queue<String> q = new ConcurrentLinkedQueue();
public static void main(String[] args) throws IOException {
new Thread(new Producer(q)).start();
new Thread(new Consumer(q)).start();
// Catch any error in the producer side, then stop both consumer and producer, and do some extra work to notify me that there's an error...
}
}
Main code just creates a shared queue, and starts both producer and consumer. So far, I guess it's ok? Now the Producer code is like this:
public class Producer implements Runnable {
private Queue<String> q;
public Producer(Queue<String> q) {
this.q = q;
}
public void run() {
try {
connectToSocket();
while(true) {
String data = readFromSocket()
q.offer(data);
}
} catch (Exception e) {
// Something really bad happened, notify the parent thread so he stops the program...
}
}
}
Producer connects to socket, reads and sends to queue the string data... The consumer:
public class Consumer implements Runnable {
private Queue<String> q;
public Consumer(Queue<String> q) {
this.q = q;
}
public void run() {
while(true) {
String dataFromSocket = q.poll();
saveData(dataFromSocket);
}
}
}
My code does a lot more than that, but I think it's now self-explanatory what I'm trying to do. I've read about wait() and notify() but I think that wouldn't work, because I don't want to wait my thread for an exception, I want to deal with it in a better way. What are the alternatives?
In general, does my code look reasonable? Would using ExecutorService help here at all?
Thanks a lot!
you can use Thread's UncaughtExceptionHandler
Thread.setDefaultExceptionHandler(
new UncaughtExceptionHandler() {
public void unchaughtException(Thread th, Throwable exception) {
System.out.println("Exception from Thread" + th + ". Exception:" + exception);
}
});
Java docs
http://docs.oracle.com/javase/7/docs/api/java/lang/Thread.UncaughtExceptionHandler.html
The simplest solution given your current code would be to wait for the producer thread to finish and then interrupt the consumer:
Thread producerThread = new Thread(new Producer(q));
producerThread.start();
Thread consumerThread = new Thread(new Consumer(q));
consumerThread.start();
try {
producerThread.join();
} finally {
consumerThread.interrupt();
}
As you mention, an executor would give you a more general purpose way to shut down everything when you need to exit (for example, when a interrupted in the terminal with ctrl-c).
ScheduledExecutorService executor = Executors.newScheduledThreadPool(10);
Producer producer = new Producer(q);
Consumer consumer = new Consumer(q);
executor.submit(producer::run);
executor.submit(consumer::run);
Runtime.getRuntime().addShutdownHook(new Thread(executor::shutdownNow));
Note that your cleanup would have to be more comprehensive than just shutting down the executor. You would have to close the socket beforehand to allow the threads to be interrupted.
Here is a more complete example that handles shutdown from both sides. You can test it by starting a test server with nc -l 1234. Killing either process (nc or the java client) will result in a clean exit of the other.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.Socket;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class Main {
private ExecutorService executor;
private Socket socket;
private AtomicBoolean running = new AtomicBoolean(true);
public static void main(String[] args) throws IOException {
Main main = new Main();
main.run();
}
private Main() throws IOException {
executor = Executors.newCachedThreadPool();
socket = new Socket("localhost", 1234);
}
private void run() throws IOException {
BlockingQueue<String> q = new SynchronousQueue<>();
Producer producer = new Producer(socket, q);
Consumer consumer = new Consumer(q);
// Start the producer. When it ends call stop
CompletableFuture.runAsync(producer, executor).whenComplete((status, ex) -> stop());
// Start the consumer.
CompletableFuture.runAsync(consumer, executor);
// Add a shutdown hook to stop everything on break
Runtime.getRuntime().addShutdownHook(new Thread(this::stop));
}
private void stop() {
if (running.compareAndSet(true, false)) { // only once
// Close the socket to unblock the producer
try {
socket.close();
} catch (IOException e) {
// ignore
}
// Interrupt tasks
executor.shutdownNow();
try {
// Give tasks some time to clean up
executor.awaitTermination(1, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// ignore
}
}
}
static class Producer implements Runnable {
private BufferedReader in;
private BlockingQueue<String> q;
public Producer(Socket socket, BlockingQueue<String> q) throws IOException {
this.in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
this.q = q;
}
#Override
public void run() {
try {
while (true) {
String data = in.readLine();
if (data == null) {
break;
}
q.put(data);
}
} catch (InterruptedException | IOException e) {
// Fall through
}
System.err.println("Producer done");
}
}
static class Consumer implements Runnable {
private BlockingQueue<String> q;
public Consumer(BlockingQueue<String> q) {
this.q = q;
}
#Override
public void run() {
try {
while (true) {
System.out.println(q.take());
}
} catch (InterruptedException e) {
// done
}
System.err.println("Client done");
}
}
}
Start consumer thread as 'daemon' thread
Mark the consumer thread as 'daemon' and let the main thread end too:
From the Java API doc for Thread.setDaemon(boolean):
Marks this thread as either a daemon thread or a user thread. The Java Virtual Machine exits when the only threads running are all daemon threads.
public class Main {
private Queue<String> q = new ConcurrentLinkedQueue();
public static void main(String[] args) throws IOException {
Thread producerThread = new Thread(new Producer(q));
// producerThread.setDefaultUncaughtExceptionHandler(...);
producerThread.start();
Thread consumerThread = new Thread(new Consumer(q));
consumerThread.setDeamon(true);
consumerThread.start();
}
}
This way, your application automatically stops, when the main thread and the producer-thread have terminated (sucessfully or by exception).
You could combine this with the UncaughtExceptionHandler as #Manish suggested, if the main thread needs to know about the producerThread failing...
How about volatile?
public class Main {
volatile boolean isStopMain = false;
private Queue<String> q = new ConcurrentLinkedQueue();
public static void main(String[] args) throws IOException {
new Thread(new Producer(q)).start();
new Thread(new Consumer(q)).start();
// Catch any error in the producer side, then stop both consumer and producer, and do some extra work to notify me that there's an error...
while (true) {
if(isStopMain){
System.exit(0); //or other operation to stop the main thread.
}
}
}
}
And In Producer:
public class Producer implements Runnable {
private Queue<String> q;
public Producer(Queue<String> q) {
this.q = q;
}
public void run() {
try {
connectToSocket();
while(true) {
String data = readFromSocket()
q.offer(data);
}
} catch (Exception e) {
// Something really bad happened, notify the parent thread so he stops the program...
Main.isStopMain = true;
}
}
}
And I wonder if you are trying to kill the parent thread in child thread? If yes, here is something you may need to know:How do I terminate parent thread from child?
i have to finish an exercise where i have to go find ".java" files in my folder path using the producer/consumer pattern with at least one producer thread and x consumer threads.
ProducerConsumer-class:
First i tried to stop the consumer when the producer is finished finding files with setting a while loop from true to false which doesn't work. It doesn't work because the threads are still running obviously just not doing anything useful. Now i use a closePool() function (as well).
So the function does work if i dont put up with my locks called locka. And thats basically something i don't understand.
So if i have
loka.lock();
ende = false;
loka.unlock();
and
while(ende){
loka.lock();
System.out.println(xy.getQueue());
loka.unlock();
}
the closePool() function will never get called. And this is something i don't understand. If i put away the locks in the while loop it does work and the threads do stop.
questions:
1) The ende parameter will be set false anyway so the lock will be finally released.
2) Secondly i did only lock a part of the method and not the object?! As far as i understand it other code in other methods in the same object will still work at the same time. Or is the lock like synchronized and i synchronize the whole object while it is in the lock state?
In my understanding the while loop in the consumer-thread is locked but the producer-thread will still call closePool();
on a extra note: maybe i didn't even design my Producer/Consumer pattern the right way.
import java.io.IOException;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
public class FindJavaVisitorp extends SimpleFileVisitor<Path> {
private BlockingQueue<String> xxx = new ArrayBlockingQueue<String>(10);
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
if (file.toString().endsWith(".java")) {
try {
xxx.put(file.toString());
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
return FileVisitResult.CONTINUE;
}
public String getQueue() throws InterruptedException {
return xxx.take();
}
}
public class ProducerConsumer {
private volatile boolean ende = true;
private Path path;
private FindJavaVisitorp xy;
private Lock loka = new ReentrantLock();
private ExecutorService pepe;
public ProducerConsumer(Path path, FindJavaVisitorp xy, ExecutorService xyz) {
this.path = path;
this.xy = xy;
pepe = xyz;
}
public void produce() throws IOException, InterruptedException {
Files.walkFileTree(path, xy);
loka.lock();
ende = false;
loka.unlock();
closePool();
}
public void consume() throws InterruptedException {
while (ende) {
loka.lock();
System.out.println(xy.getQueue());
loka.unlock();
}
}
public void closePool() {
pepe.shutdown();
try {
if (!pepe.awaitTermination(60, TimeUnit.SECONDS)) {
pepe.shutdownNow();
if (!pepe.awaitTermination(60, TimeUnit.SECONDS)) {
System.err.println("Pool couldn't be terminated!");
}
}
} catch (InterruptedException e) {
pepe.shutdownNow();
}
}
}
public class Test {
public static void main(String[] args) {
Path startingDir = Paths.get("/usr/local/");
FindJavaVisitorp x = new FindJavaVisitorp();
ExecutorService exec = Executors.newCachedThreadPool();
final ProducerConsumer pp = new ProducerConsumer(startingDir, x, exec);
exec.submit(new Runnable() {
public void run() {
try {
pp.produce();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
// x.printQueue();
for (int j = 0; j < 5; j++) {
exec.submit(new Runnable() {
public void run() {
try {
pp.consume();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
exec.shutdown();
}
}
Yes, but only if thread goes to check it and not waiting for something else, which in your case is happening. In the while loop threads are waiting for BlockingQueue and if queue is empty then then will not get a chance to check the flag variable. Also you don't need locks as you're already using BlockingQueue. In your example there's no relation between the two critical sections.
.
Following code will try to guard loka=false from any concurrent access.
loka.lock();
ende = false;//critical section
loka.unlock();
Following code will be free from concurrent access and mutually exclusive from the above critical section.
while(ende){
loka.lock();
System.out.println(xy.getQueue());//critical section
loka.unlock();
}
As there's nothing common between these two critical sections, mutual exclusion is doing nothing. Since ende is volatile guarding it with locks doesn't do anything as primitive types already have atomic access.
Reads and writes are atomic for reference variables and for most primitive variables (all types except long and double).
Reads and writes are atomic for all variables declared volatile (including long and double variables).
Only code inside the guarded block by lock() andunlock()` will be locked from concurrent access. Object itself is free to do any concurrent simultaneous (to the locked block) task outside these blocks.
And finally follow proper naming conventions and give your variables meaningful names.
Main answer to your problem why your threads are still running is because they're waiting on the blockingQueue.takeItem() and they can not be released from it unless queue is filled again, however since Producer is finished there's no possibility of that happening.
How to avoid this behavior
There are no methods on BlockingQueue which allow immediate release of waiting threads
One thing we can do is make producer put a LAST_ITEM and have consumers check if the item they got is LAST_ITEM and thus they can release themselves.
Following is working code. I have made some modifications to the variable and method names to make them more meaningful.
JavaFileVisitor
package filevisitor;
import java.nio.file.FileVisitResult;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
public class JavaFileVisitor extends SimpleFileVisitor<Path> {
private BlockingQueue<String> blockingQueue = new ArrayBlockingQueue<String>(10);
public static String NO_MORE_ITEMS = "### NO MORE ITEMS ###";
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
if (file.toString().endsWith(".java")) {
try {
blockingQueue.put(file.toString());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
return FileVisitResult.CONTINUE;
}
public String getQueueItem() throws InterruptedException {
String item = blockingQueue.take();
if(NO_MORE_ITEMS.equals(item)) {
setNoMoreItems();
}
return item;
}
public void setNoMoreItems() {
try {
blockingQueue.put(NO_MORE_ITEMS);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
ProducerConsumer
package filevisitor;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
public class ProducerConsumer {
private Path path;
private JavaFileVisitor fileVisitor;
public ProducerConsumer(Path path, JavaFileVisitor visitor) {
this.path = path;
this.fileVisitor = visitor;
}
public void produce() throws IOException, InterruptedException {
Files.walkFileTree(path, fileVisitor);
fileVisitor.setNoMoreItems();
}
public void consume() throws InterruptedException {
while (true) {
String item = fileVisitor.getQueueItem();
if(JavaFileVisitor.NO_MORE_ITEMS.equals(item)) {
break;
}
System.out.println(item);
}
}
}
ProducerConsumerMain
package filevisitor;
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class ProducerConsumerMain {
public static void main(String[] args) {
Path startingDir = Paths.get("src/filevisitor");
JavaFileVisitor fileVisitor = new JavaFileVisitor();
ExecutorService executor = Executors.newCachedThreadPool();
final ProducerConsumer producerConsumer = new ProducerConsumer(startingDir, fileVisitor);
executor.submit(new Runnable() {
public void run() {
System.out.println("Producer started");
try {
producerConsumer.produce();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Producer finished");
}
});
for (int j = 0; j < 5; j++) {
executor.submit(new Runnable() {
public void run() {
String threadName = Thread.currentThread().getName();
System.out.println(threadName + " Consumer Started");
try {
producerConsumer.consume();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(threadName + " Consumer finished");
}
});
}
executor.shutdown();
System.out.println("Executor shutdown, waiting for threads to finish");
try {
executor.awaitTermination(60, TimeUnit.SECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Exiting main");
}
}
Output
Producer started
pool-1-thread-3 Consumer Started
pool-1-thread-2 Consumer Started
Executor shutdown, waiting for threads to finish
pool-1-thread-5 Consumer Started
pool-1-thread-6 Consumer Started
pool-1-thread-4 Consumer Started
src\filevisitor\JavaFileVisitor.java
src\filevisitor\ProducerConsumerMain.java
src\filevisitor\ProducerConsumer.java
pool-1-thread-6 Consumer finished
pool-1-thread-4 Consumer finished
pool-1-thread-3 Consumer finished
pool-1-thread-5 Consumer finished
Producer finished
pool-1-thread-2 Consumer finished
Exiting main
Im trying trying to stop a runnable thread from a Swing GUI. When I click on the button to stop the runnable thread it stops it but I am unable to start a new runnable thread afterwards.
Does anyone know why this is? Any help would be appreciated.
Thanks
Here's my GUI Code Listnere
if(button.getText().equals("Start Scraper")){
if(validate())
{
updateThread.running = true;
button.setText("Stop Scraper");
String searchType = comboBox.getSelectedItem().toString();
String email = emailTextField.getText();
String password = passwordTextField.getText();
String searchTerm = searchTermTextField.getText();
try{
thread = new updateThread(searchTerm, searchType, email, password );
thread.start();
}catch(Exception ex){
System.out.println("Something went wrong in the GUI");
ex.printStackTrace();
}
}else{
//not valid go again
}
}else{
button.setText("Start Crawler");
updateThread.running = false;
updateThread.terminate();
}
}
});
Here's my runnable thread class
package guiTool;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.SessionNotFoundException;
import linkedIncrawler.common.Utils;
import linkedin.actions.BaseClass;
import linkedin.actions.LinkedInActions;
public class updateThread extends Thread
{
private static WebDriver driver;
public String searchTerm, searchType, email, password;;
public volatile static Boolean running = true;
public updateThread(String searchTerm2, String searchType2, String email2, String password2)
{
email = email2;
password = password2;
searchTerm = searchTerm2;
searchType = searchType2;
}
public static void terminate() {
currentThread().interrupt();
//thread.stop();
driver.quit();
running = false;
}
#Override
public void run()
{
while(running)
{
try {
driver = Utils.OpenBrowser("SearchTerms");
new BaseClass(driver);
LinkedInActions.Execute(searchTerm, searchType, email, password);
driver.quit();
} catch (Exception e) {
System.out.println("2nd thread cant run linkedin");
e.printStackTrace();
}
}
}
}
Once a thread has died, it is dead... You need to create a new one. There are important reasons as to why you can't re-start a dead thread.
Rather than extending Thread, maybe implement Runnable/Callable?
public volatile static Boolean running = true;
this variable, being common for all instances, stops all updateThread's, current and future. Remove static modifier.
I'm trying to write a program that checks on the health of a database. One of the elements of the elements of the program is supposed to be that the program queries a database and then waits 5 minutes using wait. If there is no response it notifies and sends out some emails. My connection to the database/sending out emails all works, but i'm having trouble implementing wait and notify.
I read the api and its easy to understand in a simple program, but i'm really confused as to how to implement it in this case with all the additional complications because of errors where I can't call something dynamic from a static method.
I've been reading through lots of threads with wait and notify, but haven't figured out how do to it correctly in my program. If anyone could give me a few tips it would be a huge help. Thanks!
import com.fmr.ipgt.email.*;
import java.io.File;
import java.io.IOException;
import java.util.List;
import org.jdom.Document;
import org.jdom.Element;
import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
import javax.mail.MessagingException;
class MyResource {
synchronized void qQuery() throws Exception {
String query = ".z.k"; // The query that is used to query q; this can be changed here.
int version = 0;
c qConn = null;
qConn = new c(Main.host,Main.port); // Connect to the q database
while (Main.healthy) {
Object o = qConn.k(query); // Query q
version = c.t(o);
if(!(version==0)) {
break; // End the process if the database responds
}
}
}
synchronized void start() throws Exception {
Main.setHealth(false);
Main.sendMessages();
}
}
class MyThread implements Runnable {
MyResource myResource;
MyThread(String name, MyResource so) {
myResource = so;
new Thread(this, name).start();
}
public void run() {
try {
myResource.qQuery(); // Begin a method to query q.
} catch (Exception e) {
e.printStackTrace();
}
}
}
public class Main {
private static String[] recipients;
private static String subject = "Database Failure";
private static String message = "The database has failed or is in a hung state";
private static String from;
static String host;
static int port;
private static String emails;
private static int minutes;
static boolean healthy = true;
public static void main(String args[]) throws Exception {
// Import information from the configuration file
SAXBuilder builder = new SAXBuilder();
File xmlFile = new File("/export/home/jflt/file.xml"); // Note: The directory for the configuration file may need to be changed
try {
Document document = (Document) builder.build(xmlFile);
Element rootNode = document.getRootElement();
List list = rootNode.getChildren("parameters");
Element node = (Element) list.get(0);
host = node.getChildText("host");
port = Integer.parseInt(node.getChildText("port"));
emails = node.getChildText("emails");
String delims = "[ ]+";
recipients = emails.split(delims); // parse email list
minutes = Integer.parseInt(node.getChildText("time"));
from = node.getChildText("from");
} catch (IOException io) {
System.out.println(io.getMessage());
} catch (JDOMException jdomex) {
System.out.println(jdomex.getMessage());
}
MyResource unhealthy = new MyResource();
new MyThread("MyThread", unhealthy); // Create new Thread
new MyThread("WaitThread", unhealthy);
while(healthy) {
Thread.sleep(minutes*60000); // The wrong thread is sleeping here. The main method should probably be a different thread instead which will then need to wait and the second thread will notify.
}
unhealthy.start(); // The database has not responded for the given time. Report that it is unhealthy.
}
public static void setHealth(boolean health){
System.out.println("database unhealthy");
healthy = health;
}
public static void sendMessages() throws MessagingException {
System.out.println("sending emails");
FCAPMailSender.postMail(recipients,subject,message,from);
}
}
If wait and notify is to be used, i will advice you to use the Lock Interface and Reentrant Lock Class from java.util.concurrent package...
Schedule a task with an ExecutorService that sends email. When you get a response, cancel the task. If it's been more than 5 minutes, the mail has already been sent by the executor thread, and cancellation is a no-op. Otherwise, the email is aborted.
This isn't even a wait()/notify() problem; there's no data passing between threads. Here's a low level equivalent to the ExecutorService solution.
void test() {
Thread t = new Thread() {
#Override
public void run() {
try { Thread.sleep(TimeUnit.MINUTES.toMillis(5)); }
catch(InterruptedException abort) { return; }
email();
}
}
t.start();
query();
t.interrupt();
}