I'm pretty new to Multithreading in java but am totally stumped about why this isn't behaving as I want it to.
I have a Producer-Consumer wherein I have
private void produceConsume() {
try {
Thread producer = new Thread(new Runnable() {
public void run() {
try {
produce();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
completedProduce = true;
}
}
private void produce() throws InterruptedException {
synchronized (this) {
while (queue.size() == capacity) {
wait();
}
try(InputStream is = new FileInputStream(file)) {
queue.add("hello");
} catch (IOException e) {
LOG.error("Error doing stream stuff: " + e.getMessage(), e);
}
notify();
}
}
});
producer.start();
List<Thread> consumers = new ArrayList<>();
for (int i = 0; i < noOfThreads; i++) {
Thread consumer = new Thread(new Runnable() {
#Override
public void run() {
try {
consume();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private void consume() throws InterruptedException {
while (queue.size() > 0 || !completedProduce) {
synchronized (this) {
while (queue.size() == 0 && !completedProduce) {
wait();
}
String s = queue.poll();
System.out.println(s);
}
notify();
}
}
}
});
consumer.start();
consumers.add(consumer);
}
for (Thread t : consumers) {
t.join();
}
producer.join();
} catch (Exception e) {
LOG.error("InterruptedException e: " + e.getMessage(), e);
} finally {
LOG.info("We are done with this file!");
}
}
Now, I've noticed that all functionality changes based on where I put my producer.join() statement. For example, if I put producer.join() right after producer.start() then everything works - but the number of threads has no impact on runtime. This makes sense as I'm slowed down drastically by how long it takes to produce and so the longest task wins out.
However, if I put producer.join() where it is in the example provided (I do the join when I do the join for the consumers) then everything just stops running before the producer actually finishes. As in, the program stalls after the first thing is consumed, waiting for something, but the thread never dies.
How do I make it so that things run correctly and nothing stalls waiting for another process to finish?
Thanks in advance,
Related
I am starting multiple threads and the idea is to wait for only one of them(does not matter which one) to continue the main thread and ignoring the other threads. In other words the main thread starts several threads and then waits until the fastest thread joins. Any ideas how to implement such a thing?
There's several possibilities.
1. Use a CountDownLatch
The idea is to have a CountDownLatch set to 1. All tasks running on the relevant threads will end by calling countDown() on it.
The main thread will simply call await() on it, and as soon as the first task is finished, the latch will hit 0, and release the waiting main thread.
This approach works with raw Threads, as well as with an ExecutorService.
public class UseCountDownLatch {
public static void main(String[] args) {
int numberOfThreads = 5;
CountDownLatch countDownLatch = new CountDownLatch(1);
ExecutorService executorService = Executors.newFixedThreadPool(numberOfThreads);
for (int i = 0; i < numberOfThreads; i++) {
int print = i;
executorService.submit(() -> {
try {
TimeUnit.SECONDS.sleep(print * 3);
System.out.println(print);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
countDownLatch.countDown();
}
});
}
executorService.shutdown();
try {
countDownLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Continue");
try {
executorService.awaitTermination(1, TimeUnit.MINUTES);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Done");
}
}
2. Use a CompletionService
Wrap your ExecutionService in a CompletionService, then just wait for the first result to come in, and ignore further results.
public class UseCompletionService {
public static void main(String[] args) {
int numberOfThreads = 5;
ExecutorService executorService = Executors.newFixedThreadPool(numberOfThreads);
CompletionService<Void> completionService = new ExecutorCompletionService<Void>(executorService);
for (int i = 0; i < numberOfThreads; i++) {
int print = i;
completionService.submit(() -> {
try {
TimeUnit.SECONDS.sleep(print * 3);
System.out.println(print);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
return null;
});
}
executorService.shutdown();
try {
completionService.take();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Continue");
try {
executorService.awaitTermination(1, TimeUnit.MINUTES);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Done");
}
}
3. Use CompletableFutures
The CompletableFutures API is made to make tasks composable. The trick is simply to combine the CompletableFutures for each individual task in a CompletableFuture that completes as soon as any of its components completes. Then it's just a matter of getting the result from the composed CompletableFuture.
public class UseCompletableFuture {
public static void main(String[] args) {
int numberOfThreads = 5;
CompletableFuture<?>[] tasks = new CompletableFuture<?>[numberOfThreads];
for (int i = 0; i < numberOfThreads; i++) {
int print = i;
tasks[i] = CompletableFuture.runAsync(() -> {
try {
TimeUnit.SECONDS.sleep(print*3);
System.out.println(print);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
}
try {
CompletableFuture.anyOf(tasks).get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
System.out.println("Continue");
try {
CompletableFuture.allOf(tasks).get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
System.out.println("Done");
}
}
I'm using the ReadData class from https://github.com/iota-community/java-iota-workshop/blob/master/src/main/java/com/iota/ReadData.java to retrieve a message from the Iota Tangle (essentially a distributed Network) via a hash value (the bundlehash).
That's my method:
private String readMessageFromHash(String BundleHash) {
final String[] s = new String[]{""};
Thread thread = new Thread(new Runnable() {
#Override
public void run() {
try {
s[0] = ReadData.getTMessage(BundleHash);
} catch (Exception e) {
e.printStackTrace();
}
}
});
thread.start();
try {
thread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
return s[0];
}
I need the return value in my next line of code but without multithreading my program crashes.
With mutlithreading it sometimes works, but most of time it doesn't work (returns an empty String).
I tried using:
thread.start();
try {
while(s[0].length < 1){}
thread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
return s[0];
but it just loops infinitely.
I think the issue is my program not waiting long enough for a response from the network.
I have the following Thread example:
class Q
{
int num;
public synchronized void put(int num) {
System.out.println("Put :"+num);
this.num = num;
try {Thread.sleep(100);} catch (Exception e) {}
notify();
try {wait();} catch (Exception e) {}
}
public synchronized void get() {
try {wait();} catch (Exception e) {}
System.out.println("Get :"+num);
notify();
}
}
class Producer implements Runnable
{
Q q;
public Producer(Q q) {
this.q = q;
Thread t = new Thread(this,"Producer");
t.start();
}
public void run() {
int i = 0;
while(true) {
q.put(i++);
try {Thread.sleep(1000);} catch (Exception e) {}
}
}
}
class Consumer implements Runnable
{
Q q;
Thread t;
public Consumer(Q q) {
this.q = q;
t = new Thread(this,"Consumer");
t.start();
}
public void run() {
while(true) {
q.get();
try {Thread.sleep(500);} catch (Exception e) {}
}
}
}
public class InterThread {
public static void main(String[] args) {
Q q = new Q();
new Producer(q);
new Consumer(q);
}
}
I'm trying to run two threads, consumer and producer, in a loop.
sharing the same object q, one thread incrementing q.num and printing the value of it, and the other is consuming q.num by just printing its value.
The result I'm getting in a console is "Put :0" and stops there,
consumer thread not being called, even though I used Thread.sleep(100);
before calling notify() in the producer thread, why !!?
In this case, Producer thread is starting before Consumer. notify() is getting called, following by that wait() getting called. Producer thread goes in waiting state, releases acquired lock.
// producer
notify();
try {
System.out.println(Thread.currentThread().getName() + " Put :"+num);
this.wait(); // lock released
}
catch (Exception e) {
}
Now consumer thread acquires the lock, wait() is executed. Consumer goes in the waiting state.
// consumer
try {
System.out.println(Thread.currentThread().getName() + "Get :"+num);
this.wait(); // Both the threads are waiting
}
catch (Exception e) {
}
Now both the threads are waiting for notify call from the other thread
Note that Sleep() method doesn't release the lock , so there is no point in calling calling Thread.sleep before the producer's notify
difference-between-wait-and-sleep
I am trying to learn inter thread communication where I am using BlockingQueue.
I have written a producer which generate TaskId and insert it into BlockingQueue.
Now I have 2 consumers threads (name "1" and "0"). If taskId is odd, it is consumed by Thread "1" else "2".
#Override
public void run() {
while (true) {
while (queue.peek() != null && !name.equals(String.valueOf(queue.peek().intValue() % 2 ))) {
try {
System.out.println(name + ",consumed," + queue.take());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
How can i make that check also here?
One way I am thinking, there could be other better ways also:
#Override
public void run() {
String name = Thread.currentThread().getName();
while (true) {
while (queue.peek() == null) {
//some sleep time
}
synchronized (lock) {
while (queue.peek() != null && !name.equals(String.valueOf(queue.peek().intValue() % 2 ))) {
try {
lock.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
if(queue.peek() != null) {
try {
System.out.println(name + ",consumed," + queue.take());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
lock.notify();
}
}
}
Another Way: To have anotherLock that will be notified by producer thread whenever element is added to queue.
#Override
public void run() {
String name = Thread.currentThread().getName();
while (true) {
synchronized (anotherLock) {
while (queue.peek() == null) {
anotherLock.wait();
}
}
synchronized (lock) {
while (queue.peek() != null && !name.equals(String.valueOf(queue.peek().intValue() % 2 ))) {
try {
lock.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
if(queue.peek() != null) {
try {
System.out.println(name + ",consumed," + queue.take());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
lock.notify();
}
}
}
I have a Producer-Consumer problem to implement in Java, where I want the producer thread to run for a specific amount of time e.g. 1 day, putting objects in a BlockingQueue -specifically tweets, streamed from Twitter Streaming API via Twitter4j- and the consumer thread to consume these objects from the queue and write them to file. I've used the PC logic from Read the 30Million user id's one by one from the big file, where producer is the FileTask and consumer is the CPUTask (check first answer; my approach uses the same iterations/try-catch blocks with it). Of course I adapted the implementations accordingly.
My main function is:
public static void main(String[] args) {
....
final int threadCount = 2;
// BlockingQueue with a capacity of 200
BlockingQueue<Tweet> tweets = new ArrayBlockingQueue<>(200);
// create thread pool with given size
ExecutorService service = Executors.newFixedThreadPool(threadCount);
Future<?> f = service.submit(new GathererTask(tweets));
try {
f.get(1,TimeUnit.MINUTES); // Give specific time to the GathererTask
} catch (InterruptedException | ExecutionException | TimeoutException e) {
f.cancel(true); // Stop the Gatherer
}
try {
service.submit(new FileTask(tweets)).get(); // Wait til FileTask completes
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
service.shutdownNow();
try {
service.awaitTermination(7, TimeUnit.DAYS);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Now, the problem is that, although it does stream the tweets and writes them to file, it never terminates and never gets to the f.cancel(true) part. What should I change for it to work properly? Also, could you explain in your answer what went wrong here with the thread logic, so I learn from my mistake? Thank you in advance.
These are the run() functions of my PC classes:
Producer:
#Override
public void run() {
StatusListener listener = new StatusListener(){
public void onStatus(Status status) {
try {
tweets.put(new Tweet(status.getText(),status.getCreatedAt(),status.getUser().getName(),status.getHashtagEntities()));
} catch (InterruptedException e) {
e.printStackTrace();
Thread.currentTread.interrupt(); // Also tried this command
}
}
public void onException(Exception ex) {
ex.printStackTrace();
}
};
twitterStream.addListener(listener);
... // More Twitter4j commands
}
Consumer:
public void run() {
Tweet tweet;
try(PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter("out.csv", true)))) {
while(true) {
try {
// block if the queue is empty
tweet = tweets.take();
writeTweetToFile(tweet,out);
} catch (InterruptedException ex) {
break; // GathererTask has completed
}
}
// poll() returns null if the queue is empty
while((tweet = tweets.poll()) != null) {
writeTweetToFile(tweet,out);
}
} catch (IOException e) {
e.printStackTrace();
}
}
You should check if your Thread classes are handling the InterruptedException, if not, they will wait forever. This might help.