So I had this assingment to make a Producer Consumer model for homework, and I finished working on an extremely crude version (but the best I could do with my current java skills).
It seems to work but it runs into the Deadlocking problem http://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem
that the Wiki link describes, which is that basically for some reason eventually all Threads fall asleep and fail to wake each other entering eternal sleep cycles.
I'm not really sure what exactly in my code is causing this as I would've thought the way I wrote it this wouldn't occur, but then again I still don't 100% understand how Threads work.
Here's my code:
package boundedbuffer;
import java.util.LinkedList;
import java.util.Random;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.Queue;
public class BoundedBuffer {
public static int CapacityCheck = 0;
public static void main(String[] args){
MessageQueue queue = new MessageQueue(3); // <-- max capacity of queue is given here as 3
Thread t1 = new Thread(new Producer(queue));
Thread t2 = new Thread(new Producer(queue));
Thread t3 = new Thread(new Producer(queue));
Thread t4 = new Thread(new Consumer(queue));
Thread t5 = new Thread(new Consumer(queue));
Thread t6 = new Thread(new Consumer(queue));
t1.start();
t2.start();
t3.start();
t4.start();
t5.start();
t6.start();
}
}
public class Producer implements Runnable{
private MessageQueue queue;
private static String msgs[] = {
"some test message",
"long message",
"short message",
"yet another message"
};
public Producer(MessageQueue queue){
this.queue = queue;
}
#Override
public synchronized void run() {
while(true){
Random rand = new Random();
int wait = rand.nextInt(3000);
int index = rand.nextInt(4);
try {
Thread.sleep(wait);
} catch (InterruptedException ex) {
Logger.getLogger(Producer.class.getName()).log(Level.SEVERE,
null, ex);
}
if(BoundedBuffer.CapacityCheck < queue.capacity){
System.out.println("Puts into buffer: " + msgs[index]);
queue.put(msgs[index]);
BoundedBuffer.CapacityCheck++;
notifyAll();
}else{
try {
wait();
} catch (InterruptedException ex) {
Logger.getLogger(Producer.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
}
public class Consumer implements Runnable{
private MessageQueue queue;
public Consumer(MessageQueue queue){
this.queue = queue;
}
#Override
public synchronized void run() {
while(true){
Random rand = new Random();
int wait = rand.nextInt(3000);
try {
Thread.sleep(wait);
} catch (InterruptedException ex) {
Logger.getLogger(Consumer.class.getName()).log(Level.SEVERE, null, ex);
}
String msg = queue.get();
if(msg == null){
try {
wait();
} catch (InterruptedException ex) {
Logger.getLogger(Consumer.class.getName()).log(Level.SEVERE, null, ex);
}
}
queue.get();
BoundedBuffer.CapacityCheck--;
System.out.println("Takes out of buffer: " + msg);
notifyAll();
}
}
}
public class MessageQueue {
public final int capacity;
private final Queue<String> messages = new LinkedList<>();
public MessageQueue(int capacity) {
this.capacity = capacity;
}
public void put(String msg){
this.messages.add(msg);
}
public String get(){
if(messages.isEmpty()){
return null;
}else{
String msg = messages.element();
messages.remove();
return msg;
}
}
}
Another minor but interesting problem is that I either NEVER or maybe only once saw a case where "taking an item out" happened more than once after each other. Putting items in always happens either once, twice, or up to three times after one another (I made the buffer size 3 for this example so it can't happen 4 times) but taking out an item happens only maybe ONCE and then afterwards it always puts one back, takes one out, puts one back. I've never seen after 3 items are put in: takes one out, takes one out again for example.
This might be a problem or an error. Idk.
I also think that using Synchronized on the run methods feels a bit off but if I take it out then I get an IllegalMonitorState Exception.
I'm using multiple producers and multiple consumers because that's how my teacher asked us to do it.
All your thread stall's because you are obtaining mutex on different producers and consumers that you pass to threads.
You synchronize on run method meaning obtaining mutex on different object while calling wait method and entering in blocked state assuming someone would notify the thread to come back. Even though if other threads notify they notify on this (individual producers or consumers) instance rather than shared instance between producer and consumer.
Share common instance like you are doing MessageQueue and synchronize on Queue rather than on run method.
Related
I'm implementing a program which contains different tasks and all have implemented Runnable. e.g. there is a task which works on a database and sends some of the tuples to a synchronized shared memory and subsequently, there is another thread which checks the shared memory and sends messages to a queue. Moreover, these two threads iterate over an infinite while loop.
Already, I have used the fixedThreadPool to execute these threads.
The problem is that sometimes program control remained in the first running thread and the second one never gets the chance to go to its running state.
Here is a similar sample code to mine:
public class A implements Runnable {
#Override
public void run() {
while(true) {
//do something
}
}
}
public class B implements Runnable {
#Override
public void run() {
while(true) {
//do something
}
}
}
public class Driver {
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(2);
A a = new A();
executorService.execute(a);
B b = new B();
executorService.execute(b);
}
}
I'd also done something tricky, make the first thread to sleep once for a second after a short period of running. As a result, it makes the second thread to find the chance for running. But is there any well-formed solution to this problem? where is the problem in your opinion?
This is a good example of Producer/Consumer pattern. There are many ways of implementing this. Here's one naive implementation using wait/notify pattern.
public class A implements Runnable {
private Queue<Integer> queue;
private int maxSize;
public A(Queue<Integer> queue, int maxSize) {
super();
this.queue = queue;
this.maxSize = maxSize;
}
#Override
public void run() {
while (true) {
synchronized (queue) {
while (queue.size() == maxSize) {
try {
System.out.println("Queue is full, " + "Producer thread waiting for "
+ "consumer to take something from queue");
queue.wait();
} catch (Exception ex) {
ex.printStackTrace();
}
}
Random random = new Random();
int i = random.nextInt();
System.out.println("Producing value : " + i);
queue.add(i);
queue.notifyAll();
}
}
}
}
public class B implements Runnable {
private Queue<Integer> queue;
public B(Queue<Integer> queue) {
super();
this.queue = queue;
}
#Override
public void run() {
while (true) {
synchronized (queue) {
while (queue.isEmpty()) {
System.out.println("Queue is empty," + "Consumer thread is waiting"
+ " for producer thread to put something in queue");
try {
queue.wait();
} catch (Exception ex) {
ex.printStackTrace();
}
}
System.out.println("Consuming value : " + queue.remove());
queue.notifyAll();
}
}
}
}
And here's hot we set things up.
public class ProducerConsumerTest {
public static void main(String[] args) {
Queue<Integer> buffer = new LinkedList<>();
int maxSize = 10;
Thread producer = new Thread(new A(buffer, maxSize));
Thread consumer = new Thread(new B(buffer));
ExecutorService executorService = Executors.newFixedThreadPool(2);
executorService.submit(producer);
executorService.submit(consumer);
}
}
In this case the Queue acts as the shared memory. You may substitute it with any other data structure that suits your needs. The trick here is that you have to coordinate between threads carefully. That's what your implementation above lacks.
I know it may sound radical, but non-framework parts of asynchonous code base should try avoiding while(true) hand-coded loops and instead model it as a (potentially self-rescheduling) callback into an executor
This allows more fair resources utilization and most importantly per-iteration monitoring instrumentation.
When the code is not latency critical (or just while prototyping) the easiest way is to do it with Executors and possibly CompletableFutures.
class Participant implements Runnable {
final Executor context;
#Override
public void run() {
final Item work = workSource.next();
if (workSource.hasNext()) {
context.execute(this::run);
}
}
}
Thread A is a loop that performs calculations.
Thread B needs to read result produced by each iteration of loop enclosed in Thread A.
What would be the best approach to achieve this without blocking anything?
You need the Producer-Consumer pattern here. In Java, you can use BlockingQueues to implement it. Here's an example with an ArrayBlockingQueue that is used to deliver Double calculation results from a producer to a consumer:
Producer.java
class Producer implements Runnable {
private final BlockingQueue<Double> queue;
public Producer(BlockingQueue<Double> q) {
queue = q;
}
public void run() {
try {
while (true) {
Double result = calculateResult();
// This call will make the result available to the consumer:
queue.put(result);
}
} catch (InterruptedException ex) {
// Handle thread interruption here
}
}
}
Consumer.java
class Consumer implements Runnable {
private final BlockingQueue<Double> queue;
public Consumer(BlockingQueue<Double> q) {
queue = q;
}
public void run() {
try {
while (true) {
// This call Will wait until the next result is available:
Double result = queue.take();
// Process the result...
}
} catch (InterruptedException ex) {
// Handle thread interruption here
}
}
}
Program.java
class Program {
public static void main() {
BlockingQueue<Double> queue = new ArrayBlockingQueue<>();
Producer producer = new Producer(queue);
Consumer consumer = new Consumer(queue);
new Thread(producer).start();
new Thread(consumer).start();
}
}
If you need, you can create several consumers or several producers (or both) that share the same queue. That will allow you to balance the work between more than two threads.
Also have a look at the BlockingQueue's capabilities and on other implementations, there're plenty of them.
I am trying to manipulate this program to print ":---)))))" repeatedly.
I understand that a semaphore is a way of controlling threads, and acquire essentially acquires a permit (reads) and release returns a permit back to the semaphore. (writes)
I've tried manipulating the number of permits when initializing the semaphores, but I am not understanding how to sync them together because I can't figure out how exactly the semaphores operate with how they acquire and release.
I am looking for a helpful explanation that pertains to Java in the context of only using semaphores, acquire and release and how they work together to properly put the threads "in sync"
import java.lang.Thread;
import java.util.concurrent.*;
public class ThreadSync {
private static boolean runFlag = true;
private static Semaphore canPrintC = new Semaphore(1);
private static Semaphore canPrintD = new Semaphore(0);
private static Semaphore canPrintP = new Semaphore(0);
public static void main(String [] args) {
// Create and start each runnable
Runnable task1 = new TaskPrintC();
Runnable task2 = new TaskPrintD();
Runnable task3 = new TaskPrintP();
Thread thread1 = new Thread(task1);
Thread thread2 = new Thread(task2);
Thread thread3 = new Thread(task3);
thread1.start();
thread2.start();
thread3.start();
// Let them run for 500 ms
try {
Thread.sleep(500);
}
catch (InterruptedException e) {
e.printStackTrace();
}
runFlag = false;
thread3.interrupt();
thread2.interrupt();
thread1.interrupt();
}
public static class TaskPrintC implements Runnable {
public void run() {
while (runFlag) {
try {
canPrintC.acquire();
}
catch (InterruptedException ex) {
ex.printStackTrace();
}
System.out.printf("%s", ":");
canPrintD.release();
}
}
}
public static class TaskPrintD implements Runnable {
public void run() {
while (runFlag) {
try {
canPrintD.acquire();
}
catch (InterruptedException ex) {
ex.printStackTrace();
}
System.out.printf("%s", "-");
canPrintP.release();
}
}
}
public static class TaskPrintP implements Runnable {
public void run() {
while (runFlag) {
try {
canPrintP.acquire();
}
catch (InterruptedException ex) {
ex.printStackTrace();
}
System.out.printf("%s", ")");
canPrintC.release();
}
}
}
}
Threads execute tasks and semaphores can help you to let tasks (or runnable objects) know each other's state (e.g. task A waits for input from task B and task B can signal task A that input is available). The difference between a task and a thread is important.
To stress this point, I have taken your example and made one runnable class that performs the task of printing a character a number of times (configured via variables in the constructor). To mimic the serialized behavior (tasks run after each other), the runnable is also aware of the next runnable that should perform the print task.
To complete the example I also ensured that the thread that is executing the main-method is aware of when the tasks have completed, so that the program stops at the proper time. A CountDownLatch is used in this case (a CountDownLatch is a very simple variation of a Semaphore).
The example below might be a bit hard to understand, but it shows some good practices (re-use code, using a stop-flag instead of interrupt, use an executor to run tasks, cleanup and stop tasks in case of error). It also shows how Semaphores can orchestrate the execution of tasks.
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
public class ChainedSemaphoreTasks {
// amount of times chained tasks are executed.
static int MAX_CHAINED_LOOPS = 3;
// helper to let main-thread know when chained loops have been executed.
final static CountDownLatch MAX_LOOPS_REACHED = new CountDownLatch(1);
public static void main(String[] args) {
String printChars = ":-)";
int[] repeatChars = { 1, 3, 5};
List<ChainedTask> tasks = buildTasks(printChars, repeatChars);
ExecutorService executor = Executors.newCachedThreadPool();
for (ChainedTask task : tasks) {
executor.execute(task);
}
try {
// Trigger first task to start running.
tasks.get(0).triggerPrintTask();
// wait for loop to complete, but not too long.
if (!MAX_LOOPS_REACHED.await(5000L, TimeUnit.MILLISECONDS)) {
throw new RuntimeException("Chained tasks loop did not complete within timeout.");
}
long waitStart = System.currentTimeMillis();
executor.shutdown();
if (executor.awaitTermination(1000L, TimeUnit.MILLISECONDS)) {
System.out.println("All tasks stopped within " + (System.currentTimeMillis() - waitStart) + " ms.");
} else {
throw new RuntimeException("Not all chained tasks stopped within timeout.");
}
} catch (Exception e) {
e.printStackTrace();
// cleanup
try {
tasks.get(0).stop();
} catch (Exception e2) {
e2.printStackTrace();
}
executor.shutdownNow();
}
}
static List<ChainedTask> buildTasks(String printChars, int[] repeatChars) {
List<ChainedTask> tasks = new ArrayList<ChainedTask>();
int maxTasks = printChars.length();
if (maxTasks != repeatChars.length) {
throw new IllegalArgumentException("Amount of repeats per pritn character must match amount of characters.");
}
for (int i = 0; i < maxTasks; i++) {
ChainedTask task = new ChainedTask(printChars.charAt(i), repeatChars[i]);
tasks.add(task);
if (i > 0) {
tasks.get(i - 1).setNextTask(task);
}
}
// make last task trigger first task - creates an endless loop.
tasks.get(maxTasks - 1).setNextTask(tasks.get(0));
tasks.get(maxTasks - 1).setLastTask(true);
return tasks;
}
static AtomicInteger chainedLoopsCount = new AtomicInteger();
static class ChainedTask implements Runnable {
// Semaphore to trigger execution
Semaphore performTask = new Semaphore(0);
// If stop is true, task must finish.
// stop must be volatile to ensure updated value is always visible.
volatile boolean stop = false;
// The last task is responsible for stopping execution
boolean lastTask;
// The next task to run after this task.
ChainedTask nextTask;
char printChar;
int repeatAmount;
ChainedTask(char printChar, int repeatAmount) {
this.printChar = printChar;
this.repeatAmount = repeatAmount;
System.out.println("Created " + printChar + " / " + repeatAmount);
}
void triggerPrintTask() {
performTask.release(repeatAmount);
}
void stop() {
// first indicate to stop
stop = true;
// then release a permit to pickup the stop sign.
performTask.release();
// also stop next task, unless this is the last task
if (!isLastTask()) {
getNextTask().stop();
}
}
#Override
public void run() {
try {
while (!stop) {
runTask();
}
} catch (Exception e) {
e.printStackTrace();
}
System.out.println("Stopped " + printChar + " / " + repeatAmount);
}
void runTask() throws Exception {
// wait for our turn
performTask.acquire();
// must check 'stop' after getting permit, see the stop-method:
// first stop is set to true and then a permit is released.
if (stop) {
return;
}
// print text for loop-amount
do {
System.out.print(printChar);
} while (performTask.tryAcquire());
if (isLastTask()) {
System.out.println();
// check if we should stop
if (chainedLoopsCount.incrementAndGet() >= MAX_CHAINED_LOOPS) {
// since this is the last task, the next task is the first task.
// stopping the first task will call the stop-method on all tasks, including this one.
getNextTask().stop();
// signal main-thread we are done.
MAX_LOOPS_REACHED.countDown();
}
// Sleep for a long time to test what happens when last task hangs.
// Should trigger the "cleanup" code in the main method.
// Thread.sleep(10000);
}
// trigger next chained task to run
// this has no effect if next chained task was stopped
getNextTask().triggerPrintTask();
}
void setNextTask(ChainedTask nextTask) {
this.nextTask = nextTask;
}
ChainedTask getNextTask() {
return nextTask;
}
void setLastTask(boolean lastTask) {
this.lastTask = lastTask;
}
boolean isLastTask() {
return lastTask;
}
}
}
Semaphore – to solve Producer/Consumer problem
A high level explanation of semaphore.
A semaphore contains a count indicating whether a resource is locked or available. Semaphore is a signaling mechanism (“I am done, you can carry on.”). The resource itself may not be thread safe.
Producer
semObject.Post(); // Send the signal
Increase the semaphore count by 1. If a thread is waiting on the
specified semaphore, it is awakened.[1]
Consumer
semObject.Wait(); // Wait for the signal
When the semaphore count is zero, the thread calling this function
will wait for the semaphore. When the semaphore count is nonzero, the
count will be decremented by 1 and the thread calling this function
will continue.[1]
Reference
[1] Massa, Anthony J., Embedded software development with eCos, Pearson Education, Inc., 2002
I have an apllication where there are three folders. I am trying to implement the producer consumer concept using LinkedBlockingQueue.
Folder 1:
Contains a class which has a shared queue
public static BlockingQueue sharedQueue = new LinkedBlockingQueue();
From a method in this class I try to call the Producer thread and the Consumer thread both of which reside in separate files.
Thread updateThread = new Thread(new Producer(sharedQueue));
Thread takeThread = new Thread(new Consumer(sharedQueue));
updateThread.start();
takeThread.start();
Folder 2:Contains the producer thread class as follows:
public class Producer implements Runnable {
private final BlockingQueue Queue;
public Producer(BlockingQueue sharedQueue){
Queue = sharedQueue;
}
public void run()
{
while (Thread.currentThread() != null) {
Random random = new Random();
int pos = random.nextInt(productList.size());
String query = "insert into tab1 values("+pos+")";
Queue.put(query);
}
}
Folder 3: Contains the consumer class as follows:
public class Consumer implements Runnable{
private final BlockingQueue queue;
Collection<String> joblist;
public Consumer (BlockingQueue sharedQueue) {
queue = sharedQueue;
MonitoringForm.txtInforamtion.append("hi"+sharedQueue.size());
joblist = new ArrayList<String>();
}
#Override
public void run() {
while(true){
try {
for(int i = 0; i < queue.size(); i++)
{
joblist.add(queue.take().toString());
MonitoringForm.txtInforamtion.append("What we got "+queue.take().toString());
}
} catch (InterruptedException ex) {
Logger.getLogger(Consumer.class.getName()).log(Level.SEVERE, null, ex);
}
try {
Thread.sleep(60*1000);
} catch (Exception e) {
}
}
}
}
Can somebody tell me how to make sure that the data added to the shared queue by Producer class can be taken by Consumer class. The consumer consumes the data every minute. Not necessary that data is consumed as soon as it is produced. Calling the Consumer and Producer threads is not working as I have them in separate directories. Do I have to use remote method invocation? Is it possible for threads?
Here's a sample code I wrote which can help you understand the concept:
import java.util.Random;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
/**
* We want a Producer thread to create random values, and the Consumer thread to
* consume it. One caveat is that if the Producer has already created a random
* value, which the Consumer thread hasn't consumed yet, the Producer thread
* blocks or waits. On the flip side, the Consumer thread waits for the Producer
* thread to produce some value if the Producer thread hasn't already.
*
* Write a program to simulate such a situation.
*/
public class ProducerConsumerCommunication
{
private volatile boolean running = true;
private ArrayBlockingQueue<Integer> buffer = new ArrayBlockingQueue<>(1);
private Random random = new Random(System.currentTimeMillis());
private class ProducerTask implements Runnable
{
public void run()
{
while (running)
{
try
{
Thread.sleep(random.nextInt(2000));
Integer value = random.nextInt();
buffer.put(value); // Blocks if buffer is full.
System.out.println("Value Put: " + value);
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
}
private class ConsumerTask implements Runnable
{
public void run()
{
while (running)
{
try
{
Thread.sleep(random.nextInt(2000));
Integer value = buffer.take(); // Blocks if buffer is empty.
System.out.println("Value Taken: " + value);
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
}
public ProducerConsumerCommunication()
{
ExecutorService service = Executors.newCachedThreadPool();
service.execute(new ProducerTask());
service.execute(new ConsumerTask());
service.shutdown();
}
public static void main(String[] args)
{
new ProducerConsumerCommunication();
}
}
In the traditional Consumer/Producer concept the Consumer waits on a resource. And whenever the Producer pushes anything on the Queue it notifies the Consumer via notify() / notifyAll()
Producer :
queue.put(query) ;
obj.notifyAll();
Consumer :
while(true)
{
try {
obj.wait();
}catch (InterruptedException e) {
}
// get data from Queue
data = queue.take();
}
Refer to the following link for more information : example
To run the Producer and Consumer in the fashion like when the Producer produces something in the queue he should Notify the Consumer, and when the Consumer consumed from the queue he should notify to the Producer to produce something in the queue,
To implement your problem in that way you have to use
wait and Notify method if you have one producer and one consumer.
and if you have multiple consumer then you have to use NotifyAll method as well of Object Class,
And in your Consumer if you get this line printed to your console "What we got" then you are sure that Consumer has consumed something from the queue
I need to code a problem similar to producer-consumer, that must use semaphores. I tried a couple of solutions and none of them worked. First I tried a solution on Wikipedia and it didn't worked. My current code is something like that:
Method run of the consumer:
public void run() {
int i=0;
DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
String s = new String();
while (1!=2){
Date datainicio = new Date();
String inicio=dateFormat.format(datainicio);
try {
Thread.sleep(1000);///10000
} catch (InterruptedException e) {
System.out.println("Excecao InterruptedException lancada.");
}
//this.encheBuffer.down();
this.mutex.down();
// RC
i=0;
while (i<buffer.length) {
if (buffer[i] == null) {
i++;
} else {
break;
}
}
if (i<buffer.length) {
QuantidadeBuffer.quantidade--;
Date datafim = new Date();
String fim=dateFormat.format(datafim);
int identificador;
identificador=buffer[i].getIdentificador()[0];
s="Consumidor Thread: "+Thread.currentThread()+" Pedido: "+identificador+" Inicio: "+inicio+" Fim: "+fim+" posicao "+i;
//System.out.println("Consumidor Thread: "+Thread.currentThread()+" Pedido: "+identificador+" Inicio: "+inicio+" Fim: "+fim+" posicao "+i);
buffer[i]= null;
}
// RC
this.mutex.up();
//this.esvaziaBuffer.up();
System.out.println(s);
// lock.up();
}
}
Method run of the producer:
public void run() {
DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
int i=0;
while (1!=2){
Date datainicio = new Date();
String inicio=dateFormat.format(datainicio);
// Produz Item
try {
Thread.sleep(500);//50000
} catch (InterruptedException e) {
System.out.println("Excecao InterruptedException lancada.");
}
//this.esvaziaBuffer.down();
this.mutex.down();
// RC
i=0;
while (i<buffer.length) {
if (buffer[i]!=null) {
i++;
} else {
break;
}
}
if (i<buffer.length) {
int identificador[]=new int[Pedido.getTamanho_identificador()];
identificador[0]=i;
buffer[i]=new Pedido();
Produtor.buffer[i].setIdentificador(identificador);
Produtor.buffer[i].setTexto("pacote de dados");
QuantidadeBuffer.quantidade++;
Date datafim = new Date();
String fim=dateFormat.format(datafim);
System.out.println("Produtor Thread: "+Thread.currentThread()+" Pedido: "+identificador[0]+" Inicio: "+inicio+" Fim: "+fim+" posicao "+i);
i++;
}
// RC
this.mutex.up();
//this.encheBuffer.up();
}
//this.encheBuffer.up();
}
In the above code it happened of a consumer thread to read a position and then, another thread read the same position without a producer fill that position, something like this:
Consumidor Thread: Thread[Thread-17,5,main] Pedido: 1 Inicio: 2011/11/27 17:23:33 Fim: 2011/11/27 17:23:34 posicao 1
Consumidor Thread: Thread[Thread-19,5,main] Pedido: 1 Inicio: 2011/11/27 17:23:33 Fim: 2011/11/27 17:23:34 posicao 1
It seems that you are using a mutex not a semaphore?
In using a mutex you have only binary synchronisation - locking and unlocking one resource. Sempahores have a value that you can signal or acquire.
You are trying to lock/unlock the entire buffer but that is the wrong way to go because, as you are seeing, either the producer or consumer locks, and when the reader has locked it the producer can't fill the buffer (because it has to lock first).
You should instead create a Sempahore, then when the producer writes one packet or block of data it can signal the semaphore. The consumers can then be trying to acquire the semaphore so they will be waiting until the producer has signalled a packet has been written. Upon signalling a written packet, one of the consumers will be woken and it will know it can read one packet. It can read a packet, then go back to trying to acquire on the semaphore. If in that time the producer has written another packet it has signalled again and either of the consumers will then go on to read another packet. Etc...
For example:
(Producer)
- Write one packet
- Semaphore.release(1)
(Consumer xN)
- Semaphore.acquire(1)
- Read one packet
If you have multiple consumers then the consumers (not the producer) should lock the buffer when reading the packet (but not when acquiring the semaphore) to prevent race conditions. In the example below the producer also locks the list since everything is on the same JVM.
import java.util.LinkedList;
import java.util.concurrent.Semaphore;
public class Semaphores {
static Object LOCK = new Object();
static LinkedList list = new LinkedList();
static Semaphore sem = new Semaphore(0);
static Semaphore mutex = new Semaphore(1);
static class Consumer extends Thread {
String name;
public Consumer(String name) {
this.name = name;
}
public void run() {
try {
while (true) {
sem.acquire(1);
mutex.acquire();
System.out.println("Consumer \""+name+"\" read: "+list.removeFirst());
mutex.release();
}
} catch (Exception x) {
x.printStackTrace();
}
}
}
static class Producer extends Thread {
public void run() {
try {
int N = 0;
while (true) {
mutex.acquire();
list.add(new Integer(N++));
mutex.release();
sem.release(1);
Thread.sleep(500);
}
} catch (Exception x) {
x.printStackTrace();
}
}
}
public static void main(String [] args) {
new Producer().start();
new Consumer("Alice").start();
new Consumer("Bob").start();
}
}
One of the most common usage pattern of Multi threaded application is to create an asynchronous communication network. Several real world applications require this. There are 2 ways of achieving this :-
The producer and consumer are tightly coupled. This is not asynchronous and each producer waits for a consumer and vice versa. The throughput of the application also becomes the minimum of the 2 entities. This is generally never a good design.
The better (and more complicated) way of doing this is by introducing a shared buffer between the producer and consumer. This way, a faster producer or faster consumer are not throttled due to a slower counterpart. It also allows for multiple producers and multiple consumers to connect via the shared buffer.
There are several ways to create a Producer-Consumer pattern.
Using wait/notify/nofityAll which was covered in the earlier module on "Locking Fundamentals"
Using the API provided by Java - java.util.concurrent.BlockingQueue. We will cover more on this in a subsequent module.
Using Semaphores : This is a very convenient way of creating the producer-consumer pattern.
public class ProducerConsumerSemaphore {
private static final int BUFFER_SIZE = 10;
private static final int MAX_VALUE = 10000;
private final Stack<Integer> buffer = new Stack<Integer>();
private final Semaphore writePermits = new Semaphore(BUFFER_SIZE);
private final Semaphore readPermits = new Semaphore(0);
private final Random random = new Random();
class Producer implements Runnable {
#Override
public void run() {
while (true) {
writePermits.acquireUninterruptibly();
buffer.push(random.nextInt(MAX_VALUE));
readPermits.release();
}
}
}
class Consumer implements Runnable {
#Override
public void run() {
while (true) {
readPermits.acquireUninterruptibly();
System.out.println(buffer.pop());
writePermits.release();
}
}
}
public static void main(String[] args) {
ProducerConsumerSemaphore obj = new ProducerConsumerSemaphore();
Producer p1 = obj.new Producer();
Producer p2 = obj.new Producer();
Producer p3 = obj.new Producer();
Consumer c1 = obj.new Consumer();
Consumer c2 = obj.new Consumer();
Consumer c3 = obj.new Consumer();
Thread t1 = new Thread(p1);
Thread t2 = new Thread(p2);
Thread t3 = new Thread(p3);
Thread t4 = new Thread(c1);
Thread t5 = new Thread(c2);
Thread t6 = new Thread(c3);
t1.start();
t2.start();
t3.start();
t4.start();
t5.start();
t6.start();
}
We use 2 semaphores - 1 for consumers and 1 for producers.
The number of permits allowed for the producer are set to maximum buffer size.
Each producer consumes 1 write permit and releases 1 read permit on producing 1 message.
Each consumer consumes 1 read permit and releases 1 write permit for consumption of each message.
Imagine the permit to be piggy banked on the actual message. Write permit flows from the Producer to Consumer (and back to the Producer). Read permit flows from the Consumer to Producer (and back to the Consumer). Total messages in the buffer at any given point of time will be exactly equal to the number of read permits issued. If the rate of producing messages is greater than the rate of consuming messages, then at a certain point, number of write permits available would be exhausted and all the producer threads would be blocked until a consumer reads from the buffer and releases a write permit. The same logic exists the other way round as well.
Above is a more visual articulation of flow of messages and permits in the system.
By using Semaphores, we are only abstracting away the gory details and care required to write a piece of code using wait/notify/notifyAll.
The above code can be compared with the wait et. al approach :
When a thread is blocked for lack of permits, it is equivalent to a wait() call on that semaphore.
When a thread releases a permit, it is equivalent to a notifyAll() call on that particular semaphore.
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Semaphore;
import java.util.logging.Level;
import java.util.logging.Logger;
/*
* To change this license header, choose License Headers in Project Properties.
* To change this template file, choose Tools | Templates
* and open the template in the editor.
*/
/**
*
* #author sakshi
*/
public class SemaphoreDemo {
static Semaphore producer = new Semaphore(1);
static Semaphore consumer = new Semaphore(0);
static List<Integer> list = new ArrayList<Integer>();
static class Producer extends Thread {
List<Integer> list;
public Producer(List<Integer> list) {
this.list = list;
}
public void run() {
for (int i = 0; i < 10; i++) {
try {
producer.acquire();
} catch (InterruptedException ex) {
Logger.getLogger(SemaphoreDemo.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println("produce=" + i);
list.add(i);
consumer.release();
}
}
}
static class Consumer extends Thread {
List<Integer> list;
public Consumer(List<Integer> list) {
this.list = list;
}
public void run() {
for (int i = 0; i < 10; i++) {
try {
consumer.acquire();
} catch (InterruptedException ex) {
Logger.getLogger(SemaphoreDemo.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println("consume=" + list.get(i));
producer.release();
}
}
}
public static void main(String[] args) {
Producer produce = new Producer(list);
Consumer consume = new Consumer(list);
produce.start();
consume.start();
}
}
output:
produce=0
consume=0
produce=1
consume=1
produce=2
consume=2
produce=3
consume=3
produce=4
consume=4
produce=5
consume=5
produce=6
consume=6
produce=7
consume=7
produce=8
consume=8
produce=9
consume=9
import java.util.concurrent.Semaphore;
public class ConsumerProducer{
public static void main(String[] args) {
Semaphore semaphoreProducer=new Semaphore(1);
Semaphore semaphoreConsumer=new Semaphore(0);
System.out.println("semaphoreProducer permit=1 | semaphoreConsumer permit=0");
new Producer(semaphoreProducer,semaphoreConsumer).start();
new Consumer(semaphoreConsumer,semaphoreProducer).start();
}
/**
* Producer Class.
*/
static class Producer extends Thread{
Semaphore semaphoreProducer;
Semaphore semaphoreConsumer;
public Producer(Semaphore semaphoreProducer,Semaphore semaphoreConsumer) {
this.semaphoreProducer=semaphoreProducer;
this.semaphoreConsumer=semaphoreConsumer;
}
public void run() {
for(;;){
try {
semaphoreProducer.acquire();
System.out.println("Produced : "+Thread.currentThread().getName());
semaphoreConsumer.release();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
/**
* Consumer Class.
*/
static class Consumer extends Thread{
Semaphore semaphoreConsumer;
Semaphore semaphoreProducer;
public Consumer(Semaphore semaphoreConsumer,Semaphore semaphoreProducer) {
this.semaphoreConsumer=semaphoreConsumer;
this.semaphoreProducer=semaphoreProducer;
}
public void run() {
for(;;){
try {
semaphoreConsumer.acquire();
System.out.println("Consumed : "+Thread.currentThread().getName());
semaphoreProducer.release();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}