I'm implementing a program which contains different tasks and all have implemented Runnable. e.g. there is a task which works on a database and sends some of the tuples to a synchronized shared memory and subsequently, there is another thread which checks the shared memory and sends messages to a queue. Moreover, these two threads iterate over an infinite while loop.
Already, I have used the fixedThreadPool to execute these threads.
The problem is that sometimes program control remained in the first running thread and the second one never gets the chance to go to its running state.
Here is a similar sample code to mine:
public class A implements Runnable {
#Override
public void run() {
while(true) {
//do something
}
}
}
public class B implements Runnable {
#Override
public void run() {
while(true) {
//do something
}
}
}
public class Driver {
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(2);
A a = new A();
executorService.execute(a);
B b = new B();
executorService.execute(b);
}
}
I'd also done something tricky, make the first thread to sleep once for a second after a short period of running. As a result, it makes the second thread to find the chance for running. But is there any well-formed solution to this problem? where is the problem in your opinion?
This is a good example of Producer/Consumer pattern. There are many ways of implementing this. Here's one naive implementation using wait/notify pattern.
public class A implements Runnable {
private Queue<Integer> queue;
private int maxSize;
public A(Queue<Integer> queue, int maxSize) {
super();
this.queue = queue;
this.maxSize = maxSize;
}
#Override
public void run() {
while (true) {
synchronized (queue) {
while (queue.size() == maxSize) {
try {
System.out.println("Queue is full, " + "Producer thread waiting for "
+ "consumer to take something from queue");
queue.wait();
} catch (Exception ex) {
ex.printStackTrace();
}
}
Random random = new Random();
int i = random.nextInt();
System.out.println("Producing value : " + i);
queue.add(i);
queue.notifyAll();
}
}
}
}
public class B implements Runnable {
private Queue<Integer> queue;
public B(Queue<Integer> queue) {
super();
this.queue = queue;
}
#Override
public void run() {
while (true) {
synchronized (queue) {
while (queue.isEmpty()) {
System.out.println("Queue is empty," + "Consumer thread is waiting"
+ " for producer thread to put something in queue");
try {
queue.wait();
} catch (Exception ex) {
ex.printStackTrace();
}
}
System.out.println("Consuming value : " + queue.remove());
queue.notifyAll();
}
}
}
}
And here's hot we set things up.
public class ProducerConsumerTest {
public static void main(String[] args) {
Queue<Integer> buffer = new LinkedList<>();
int maxSize = 10;
Thread producer = new Thread(new A(buffer, maxSize));
Thread consumer = new Thread(new B(buffer));
ExecutorService executorService = Executors.newFixedThreadPool(2);
executorService.submit(producer);
executorService.submit(consumer);
}
}
In this case the Queue acts as the shared memory. You may substitute it with any other data structure that suits your needs. The trick here is that you have to coordinate between threads carefully. That's what your implementation above lacks.
I know it may sound radical, but non-framework parts of asynchonous code base should try avoiding while(true) hand-coded loops and instead model it as a (potentially self-rescheduling) callback into an executor
This allows more fair resources utilization and most importantly per-iteration monitoring instrumentation.
When the code is not latency critical (or just while prototyping) the easiest way is to do it with Executors and possibly CompletableFutures.
class Participant implements Runnable {
final Executor context;
#Override
public void run() {
final Item work = workSource.next();
if (workSource.hasNext()) {
context.execute(this::run);
}
}
}
Related
I want to create two threads that one adds elements into ArrayList (or vector) and the other removes elements from this list concurrently. For example, if thread1 adds 20 elements into the list, then thread2 starts removing at the same time until total elements are removed, but these two threads must be work at the same time.
I wrote a producer (adding to the list) thread. In this thread, when the number of elements added to the list is greater than 5 or any number, so new thread must be started but in here I am stuck. I will mark the point that I was stuck.
public class Test{
public static void main(String[] args) {
Data d = new Data();
Thread t = new Thread(new producer(d));
t.start();
}
}
class producer implements Runnable{
Data d;
Data d2;
Object lck;
public producer(Data dd)
{
d=dd;
}
#Override
public void run()
{
for (int i=0;i<100;++i ) {
synchronized (d){
d.a.add(i);
// if i is greater than 5,
// start consumer thread
// which remove elements from ArrayList.
// but how ??
Thread t = new Thread(new Runnable(){
#Override
public void run()
{
//if(d.a.isEmpty())
//wait the adder thread
}
});
t.start();
}
}
}
}
class Data{
ArrayList<Integer> a; // or vector
public Data()
{
a = new ArrayList<>();
}
}
How can I implement a remover thread that removes all elements in the list with the same time with adder thread and synchronize them?
You can try concurrent package of java .
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CopyOnWriteArrayList.html
You are using synchronized block in thread which will not help in this case. Method in collection or shared data should be synchronized as it will be accessed by multiple thread
In your code, you are creating 100 consumer thread in producer class within synchronized block. This is not efficient way to utilize parallelism using multi-threading. You are creating one thread for one data to be consumed. Once the data is consumed your thread will be in DEAD state and will not be useful to consume other incoming data, this is wastage of resource as well as requires more time to solve problem.
Take reference of below code to solve your consumer producer problem.
import java.util.*;
class Data {
final List<Integer> a;
public Data() {
a = new ArrayList<>();
}
}
public class Producer implements Runnable {
private final Data data;
public Producer(Data data) {
this.data = data;
}
#Override
public void run() {
for (int i = 0; i < 100; i++) {
synchronized (data) {
data.a.add(i);
}
}
}
}
public class Consumer implements Runnable {
private Data data;
private boolean isThreadEnabled = true;
public Consumer(Data data) {
this.data = data;
}
#Override
public void run() {
while (isThreadEnabled) {
synchronized (data) {
if (!data.a.isEmpty()) {
System.out.println(data.a.remove(0));
}
}
}
}
public void stopConsumer() {
isThreadEnabled = false;
}
}
public class ThreadsMain {
public static void main(String args[]) {
try {
Data data = new Data();
Consumer consumerRunnable = new Consumer(data);
Thread producer = new Thread(new Producer(data));
Thread consumer = new Thread(consumerRunnable);
producer.start();
consumer.start();
producer.join();
try {
//wait for consumer to consume data and then stop the thread
Thread.sleep(1000);
consumerRunnable.stopConsumer();
} catch (InterruptedException e) {
e.printStackTrace();
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
Thread A is a loop that performs calculations.
Thread B needs to read result produced by each iteration of loop enclosed in Thread A.
What would be the best approach to achieve this without blocking anything?
You need the Producer-Consumer pattern here. In Java, you can use BlockingQueues to implement it. Here's an example with an ArrayBlockingQueue that is used to deliver Double calculation results from a producer to a consumer:
Producer.java
class Producer implements Runnable {
private final BlockingQueue<Double> queue;
public Producer(BlockingQueue<Double> q) {
queue = q;
}
public void run() {
try {
while (true) {
Double result = calculateResult();
// This call will make the result available to the consumer:
queue.put(result);
}
} catch (InterruptedException ex) {
// Handle thread interruption here
}
}
}
Consumer.java
class Consumer implements Runnable {
private final BlockingQueue<Double> queue;
public Consumer(BlockingQueue<Double> q) {
queue = q;
}
public void run() {
try {
while (true) {
// This call Will wait until the next result is available:
Double result = queue.take();
// Process the result...
}
} catch (InterruptedException ex) {
// Handle thread interruption here
}
}
}
Program.java
class Program {
public static void main() {
BlockingQueue<Double> queue = new ArrayBlockingQueue<>();
Producer producer = new Producer(queue);
Consumer consumer = new Consumer(queue);
new Thread(producer).start();
new Thread(consumer).start();
}
}
If you need, you can create several consumers or several producers (or both) that share the same queue. That will allow you to balance the work between more than two threads.
Also have a look at the BlockingQueue's capabilities and on other implementations, there're plenty of them.
I am new to using threads. In another class an instance of the ConnectionMaster class is created and started (extends thread). A Client object is given to the ConnectionMaster object which adds it to the list. The overridden run() method of the Thread class essentially listens for a client to be added to the list. Indeed it does listen and "hears" when a Client object is added to the list. However, although .hasNext() returns true .Next() causes an exception. What am I doing wrong?
The following methods are from class ConnectionMaster which extends Thread:
Constructor
public ConnectionMaster(){
clients = new Vector<>();
listIterator = clients.listIterator();
}
Public method for adding client objects to the list
#Override
public synchronized void addClient(Client client) {
listIterator.add(client);
}
This is the overridden thread method of the class Thread. It consistently checks for elements added to the list.
#Override
public void run(){
while(true){
while(listIterator.hasNext()){
processClient(listIterator.next()); //this is where error occurs
listIterator.remove();
}
while(listIterator.hasPrevious()){
processClient(listIterator.previous());
listIterator.remove();
}
}
}
////////////////////////////////UPDATE////////////////////////////////////
Thank You OldCurmudgeon and Stephen C.
Based on your feedback, my code has been modified thus:
Constructor
public ConnectionMaster(){
clients = new ArrayBlockingQueue<Client>(1024);
}
Method for receiving client objects
#Override
public synchronized void addClient(Client client) {
try {
clients.put(client);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Listener
#Override
public void run(){
while(true){
try {
processClient((Client)clients.take());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
This is a very strange way to implement Producer/Consumer. The usual way is to use a BlockingQueue.
public class TwoThreads {
public static void main(String args[]) throws InterruptedException {
System.out.println("TwoThreads:Test");
new TwoThreads().test();
}
// The end of the list.
private static final Integer End = -1;
static class Producer implements Runnable {
final BlockingQueue<Integer> queue;
public Producer(BlockingQueue<Integer> queue) {
this.queue = queue;
}
#Override
public void run() {
try {
for (int i = 0; i < 1000; i++) {
queue.add(i);
Thread.sleep(1);
}
// Finish the queue.
queue.add(End);
} catch (InterruptedException ex) {
// Just exit.
}
}
}
static class Consumer implements Runnable {
final BlockingQueue<Integer> queue;
public Consumer(BlockingQueue<Integer> queue) {
this.queue = queue;
}
#Override
public void run() {
boolean ended = false;
while (!ended) {
try {
Integer i = queue.take();
ended = i == End;
System.out.println(i);
} catch (InterruptedException ex) {
ended = true;
}
}
}
}
public void test() throws InterruptedException {
BlockingQueue<Integer> queue = new LinkedBlockingQueue<>();
Thread pt = new Thread(new Producer(queue));
Thread ct = new Thread(new Consumer(queue));
// Start it all going.
pt.start();
ct.start();
// Wait for it to finish.
pt.join();
ct.join();
}
}
What am I doing wrong?
Quite a lot actually.
First thing you are doing wrong is (apparently) using an ListIterator object in multiple threads. The ListIterator and Iterator implementations for Vector are not thread-safe1, so what you are doing is potentially hazardous.
The second thing is that even if the iterators / list iterators were thread-safe, you are performing a sequence of operations (e.g. hasNext, next, remove) without doing anything to ensure that the sequence of operations is performed in a way that is threadsafe. There is a distinct possibility that two threads could be performing the same sequence simultaneously on the shared iterator, and that one could interfere with the other.
I'm not sure what to suggest in order to fix your code. Two threads sharing an iterator is not going to work.
It would probably be better to ditch it, and use some kind of Queue as suggested by #OldCurmugeon.
Either problem 1 or problem 2 (as outlined above) could give rise to NoSuchElement exceptions.
1 - This is apparent from examining the source code - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/java/util/Vector.java#Vector.ListItr .
Recently I have delved into the dark arts of Threads, I get how to create them and when to use them and when not to use them. But when I tried to learn how to communicate between them; I Found that Pipes are what you use to do it. I have a Object that is a Instance of one of my Class' that I created, but Pipes seem to be only able to send Byte Arrays or Integers. I wont to be able to use something like a Object Stream to send my object to the other Thread, but my surfing of the internet has gone terribly bad and I am lost. so I Guess the only thing to do is turn to Stack Overflow and see if any one can help. Thank you for the help in advance.
You should use one of the implementations of BlockingQueue.
I most commonly use ArrayBlockingQueue as it allows me to limit the memory footprint of the solution. A LinkedBlockingDeque can be used for an unlimited size but be certain you cannot overload memory.
Here are two threads communicating between themselves using an ArrayBlockingQueue.
public class TwoThreads {
public static void main(String args[]) throws InterruptedException {
System.out.println("TwoThreads:Test");
new TwoThreads().test();
}
// The end of the list.
private static final Integer End = -1;
static class Producer implements Runnable {
final BlockingQueue<Integer> queue;
public Producer(BlockingQueue<Integer> queue) {
this.queue = queue;
}
#Override
public void run() {
try {
for (int i = 0; i < 1000; i++) {
queue.add(i);
Thread.sleep(1);
}
// Finish the queue.
queue.add(End);
} catch (InterruptedException ex) {
// Just exit.
}
}
}
static class Consumer implements Runnable {
final BlockingQueue<Integer> queue;
public Consumer(BlockingQueue<Integer> queue) {
this.queue = queue;
}
#Override
public void run() {
boolean ended = false;
while (!ended) {
try {
Integer i = queue.take();
ended = i == End;
System.out.println(i);
} catch (InterruptedException ex) {
ended = true;
}
}
}
}
public void test() throws InterruptedException {
BlockingQueue<Integer> queue = new LinkedBlockingQueue<>();
Thread pt = new Thread(new Producer(queue));
Thread ct = new Thread(new Consumer(queue));
// Start it all going.
pt.start();
ct.start();
// Wait for it to finish.
pt.join();
ct.join();
}
}
I need to implement a consumer producer example. This is a simple program, I modified a bit but I'm not sure if there are potential problems with it. I would appreciate if someone can help me refine it. My main
issue right now is that I don't know how to stop the consumer when the producer is done.
I have tried the following code, but stop() is deprecated, and it also doesn't work:
if (!producer.isAlive()) {
consumer.stop();
}
ProducerConsumer.java:
import java.util.Vector;
public class ProducerConsumer {
public static void main(String[] args) {
int size = 5;
Vector<Integer> sQ = new Vector<Integer>(size);
Thread consumer = new Thread(new Consumer(sQ, size));
Thread producer = new Thread(new Producer(sQ, size));
consumer.start();
producer.start();
if (!producer.isAlive()) {
consumer.stop();
}
}
}
class Consumer implements Runnable {
Vector<Integer> sQ = new Vector<Integer>();
int size;
public Consumer(Vector<Integer> sQ, int size) {
this.sQ = sQ;
this.size = size;
}
#Override
public void run() {
while (true) {
try {
System.out.println("Consuming element: " + consume());;
Thread.sleep(50);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private int consume() throws InterruptedException {
synchronized (sQ) {
while (sQ.isEmpty()) {
System.out.println("The queue is empty and "
+ Thread.currentThread().getName() + " has to wait."
+ "size is: " + sQ.size());
sQ.wait();
}
sQ.notifyAll();
return sQ.remove(0);
}
}
}
class Producer implements Runnable {
Vector<Integer> sQ = new Vector<Integer>();
int size;
public Producer(Vector<Integer> sQ, int size) {
this.sQ = sQ;
this.size = size;
}
#Override
public void run() {
for (int i = 0; i < 12; ++i) {
try {
produce(i);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private void produce(int i) throws InterruptedException {
synchronized (sQ) {
while (sQ.size() == size) {
System.out.println("The queue is full and "
+ Thread.currentThread().getName() + " has to wait."
+ "size is: " + sQ.size());
sQ.wait();
}
sQ.add(i);
sQ.notify();
}
}
}
The recommended approach is generally to set a boolean flag (finished or similar) on threads that will need to be terminated and then loop while(!finished). (Note that the flag generally needs to be volatile so that the thread will see changes.) If the thread is expected to be blocking, then you can interrupt() it to restart its waiting loop.
The overall approach you're taking seems to be out of date, though. The BlockingQueue implementations were designed specifically to ease producer-consumer implementations, and many such problems can be more effectively handled by using an Executor and firing off tasks to it as they come in instead of manually queuing and polling.
Use a CountdownLatch. This allows you to wait for it to be lowered in one thread and actually lower it from another. It's thread safe and designed specifically for this usecase.
If you are going to use a boolean, as suggested in one of the commments, use an AtomicBoolean.
In general, avoid using language primitives such as synchronized or volatile and instead use the more higher level constructs provided by the java.concurrent package. If you are going to go low level, you'll need a firm understanding of the semantics.
If you want to reuse rather than reinvent, you might like to use my concurrent processing iterable: https://github.com/jillesvangurp/iterables-support/blob/master/src/main/java/com/jillesvangurp/iterables/ConcurrentProcessingIterable.java
You simply foreach over the input and it concurrently produces the output with as many threads as you need.