I need to implement a consumer producer example. This is a simple program, I modified a bit but I'm not sure if there are potential problems with it. I would appreciate if someone can help me refine it. My main
issue right now is that I don't know how to stop the consumer when the producer is done.
I have tried the following code, but stop() is deprecated, and it also doesn't work:
if (!producer.isAlive()) {
consumer.stop();
}
ProducerConsumer.java:
import java.util.Vector;
public class ProducerConsumer {
public static void main(String[] args) {
int size = 5;
Vector<Integer> sQ = new Vector<Integer>(size);
Thread consumer = new Thread(new Consumer(sQ, size));
Thread producer = new Thread(new Producer(sQ, size));
consumer.start();
producer.start();
if (!producer.isAlive()) {
consumer.stop();
}
}
}
class Consumer implements Runnable {
Vector<Integer> sQ = new Vector<Integer>();
int size;
public Consumer(Vector<Integer> sQ, int size) {
this.sQ = sQ;
this.size = size;
}
#Override
public void run() {
while (true) {
try {
System.out.println("Consuming element: " + consume());;
Thread.sleep(50);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private int consume() throws InterruptedException {
synchronized (sQ) {
while (sQ.isEmpty()) {
System.out.println("The queue is empty and "
+ Thread.currentThread().getName() + " has to wait."
+ "size is: " + sQ.size());
sQ.wait();
}
sQ.notifyAll();
return sQ.remove(0);
}
}
}
class Producer implements Runnable {
Vector<Integer> sQ = new Vector<Integer>();
int size;
public Producer(Vector<Integer> sQ, int size) {
this.sQ = sQ;
this.size = size;
}
#Override
public void run() {
for (int i = 0; i < 12; ++i) {
try {
produce(i);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private void produce(int i) throws InterruptedException {
synchronized (sQ) {
while (sQ.size() == size) {
System.out.println("The queue is full and "
+ Thread.currentThread().getName() + " has to wait."
+ "size is: " + sQ.size());
sQ.wait();
}
sQ.add(i);
sQ.notify();
}
}
}
The recommended approach is generally to set a boolean flag (finished or similar) on threads that will need to be terminated and then loop while(!finished). (Note that the flag generally needs to be volatile so that the thread will see changes.) If the thread is expected to be blocking, then you can interrupt() it to restart its waiting loop.
The overall approach you're taking seems to be out of date, though. The BlockingQueue implementations were designed specifically to ease producer-consumer implementations, and many such problems can be more effectively handled by using an Executor and firing off tasks to it as they come in instead of manually queuing and polling.
Use a CountdownLatch. This allows you to wait for it to be lowered in one thread and actually lower it from another. It's thread safe and designed specifically for this usecase.
If you are going to use a boolean, as suggested in one of the commments, use an AtomicBoolean.
In general, avoid using language primitives such as synchronized or volatile and instead use the more higher level constructs provided by the java.concurrent package. If you are going to go low level, you'll need a firm understanding of the semantics.
If you want to reuse rather than reinvent, you might like to use my concurrent processing iterable: https://github.com/jillesvangurp/iterables-support/blob/master/src/main/java/com/jillesvangurp/iterables/ConcurrentProcessingIterable.java
You simply foreach over the input and it concurrently produces the output with as many threads as you need.
Related
I'm implementing a program which contains different tasks and all have implemented Runnable. e.g. there is a task which works on a database and sends some of the tuples to a synchronized shared memory and subsequently, there is another thread which checks the shared memory and sends messages to a queue. Moreover, these two threads iterate over an infinite while loop.
Already, I have used the fixedThreadPool to execute these threads.
The problem is that sometimes program control remained in the first running thread and the second one never gets the chance to go to its running state.
Here is a similar sample code to mine:
public class A implements Runnable {
#Override
public void run() {
while(true) {
//do something
}
}
}
public class B implements Runnable {
#Override
public void run() {
while(true) {
//do something
}
}
}
public class Driver {
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(2);
A a = new A();
executorService.execute(a);
B b = new B();
executorService.execute(b);
}
}
I'd also done something tricky, make the first thread to sleep once for a second after a short period of running. As a result, it makes the second thread to find the chance for running. But is there any well-formed solution to this problem? where is the problem in your opinion?
This is a good example of Producer/Consumer pattern. There are many ways of implementing this. Here's one naive implementation using wait/notify pattern.
public class A implements Runnable {
private Queue<Integer> queue;
private int maxSize;
public A(Queue<Integer> queue, int maxSize) {
super();
this.queue = queue;
this.maxSize = maxSize;
}
#Override
public void run() {
while (true) {
synchronized (queue) {
while (queue.size() == maxSize) {
try {
System.out.println("Queue is full, " + "Producer thread waiting for "
+ "consumer to take something from queue");
queue.wait();
} catch (Exception ex) {
ex.printStackTrace();
}
}
Random random = new Random();
int i = random.nextInt();
System.out.println("Producing value : " + i);
queue.add(i);
queue.notifyAll();
}
}
}
}
public class B implements Runnable {
private Queue<Integer> queue;
public B(Queue<Integer> queue) {
super();
this.queue = queue;
}
#Override
public void run() {
while (true) {
synchronized (queue) {
while (queue.isEmpty()) {
System.out.println("Queue is empty," + "Consumer thread is waiting"
+ " for producer thread to put something in queue");
try {
queue.wait();
} catch (Exception ex) {
ex.printStackTrace();
}
}
System.out.println("Consuming value : " + queue.remove());
queue.notifyAll();
}
}
}
}
And here's hot we set things up.
public class ProducerConsumerTest {
public static void main(String[] args) {
Queue<Integer> buffer = new LinkedList<>();
int maxSize = 10;
Thread producer = new Thread(new A(buffer, maxSize));
Thread consumer = new Thread(new B(buffer));
ExecutorService executorService = Executors.newFixedThreadPool(2);
executorService.submit(producer);
executorService.submit(consumer);
}
}
In this case the Queue acts as the shared memory. You may substitute it with any other data structure that suits your needs. The trick here is that you have to coordinate between threads carefully. That's what your implementation above lacks.
I know it may sound radical, but non-framework parts of asynchonous code base should try avoiding while(true) hand-coded loops and instead model it as a (potentially self-rescheduling) callback into an executor
This allows more fair resources utilization and most importantly per-iteration monitoring instrumentation.
When the code is not latency critical (or just while prototyping) the easiest way is to do it with Executors and possibly CompletableFutures.
class Participant implements Runnable {
final Executor context;
#Override
public void run() {
final Item work = workSource.next();
if (workSource.hasNext()) {
context.execute(this::run);
}
}
}
I am trying to work around with threads in java. Though I understand that threads output are unpredictable, However was wondering if there is a way to do that.
I have to implement two threads, one prints alphabets(a,b,c...z) and other prints numbers(1,2,3....26). Have to implement it in such a way that the output should be a,1,b,2,c,3,d,4......z,26. Below is my code but it doesn't give the desired output.
public class ThreadsExample {
public static void main(String[] args) {
Runnable r = new Runnable1();
Thread t = new Thread(r);
Runnable r2 = new Runnable2();
Thread t2 = new Thread(r2);
t.start();
t2.start();
}
}
class Runnable2 implements Runnable{
public void run(){
for(char i='a';i<='z';i++) {
System.out.print(i+",");
}
}
}
class Runnable1 implements Runnable{
public void run(){
for(int i=1;i<=26;i++) {
System.out.print(i+",");
}
}
}
What tweak should I make in the code to get the desired output? How does synchronization helps here? Or is it really possible when working with Threads at all?
PS: This is not an assignment or some exercise. Its self learning.
It is possible. You need to synchronize it well.
Approach Pseudocode
query some (synchronized) state
state will tell whether nums or chars are allowed
if state allows char and caller will put chars, do it now and change state and wake up waiting threads
if not, wait
if state allows numbers and caller will put numbers, do it now and change state and wake up waiting threads
if not, wait
Java code
public class ThreadsExample {
public static ThreadsExample output = new ThreadsExample ();
public static void main(String[] args) {
Runnable r = new Runnable1();
Thread t = new Thread(r);
Runnable r2 = new Runnable2();
Thread t2 = new Thread(r2);
t.start();
t2.start();
}
private Object syncher = new Object (); // we use an explicit synch Object, you could use annotation on methods, too. like ABHISHEK did.
// explicit allows to deal with more complex situations, especially you could have more the one locking Object
private int state = 0; // 0 allows chars, 1 allows ints
public void print (char pChar) {
synchronized (syncher) { // prevent the other print to access state
while (true) {
if (state == 0) { // char are allowed
System.out.print(pChar + ","); // print it
state = 1; // now allow ints
syncher.notify(); // wake up all waiting threads
return;
} else { // not allowed for now
try {
syncher.wait(); // wait on wake up
} catch (InterruptedException e) {
}
}
}
}
}
public void print (int pInt) {
synchronized (syncher) {
while (true) {
if (state == 1) {
System.out.print(pInt + ",");
state = 0;
syncher.notify();
return;
} else {
try {
syncher.wait();
} catch (InterruptedException e) {
}
}
}
}
}
}
class Runnable2 implements Runnable{
public void run(){
for(char i='a';i<='z';i++) {
ThreadsExample.output.print(i);
}
}
}
class Runnable1 implements Runnable{
public void run(){
for(int i=1;i<=26;i++) {
ThreadsExample.output.print(i);
}
}
}
Output
a,1,b,2,c,3,d,4,e,5,f,6,g,7,h,8,i,9,j,10,k,11,l,12,m,13,n,14,o,15,p,16,q,17,r,18,s,19,t,20,u,21,v,22,w,23,x,24,y,25,z,26,
The whole idea of threads: it represents a "stream of activity" that executes code independent of other threads.
In your case, you want that these two threads go in "lockstep". Thread A does one step, then Thread B, then A, then B.
In order to get there, the two threads need something "synchronize" on - in other words: A sends a signal to B when it has done its steps - and B has to wait for that signal. Then B does its thing, signals to A, ...
For starters, a simple boolean value would do. One thread sets it to true, the other to false (to indicate when it has made its step). Then the thread waits for the boolean to toggle again.
As you intend to learn things, I would just start experimenting from there. In case you want to take detours, look here for example. This might help as well.
HERE IS THE CODE::
You need to create 2 threads and implement wait and notify methods correctly you can also refer "Create two threads, one display odd & other even numbers" for your answer.
public class ThreadClass {
volatile int i = 1;
volatile Character c = 'a';
volatile boolean state = true;
synchronized public void printAlphabet() {
try {
while (!state) {
wait();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(Thread.currentThread().getName() + " " +c);
state = false;
c++;
notifyAll();
}
synchronized public void printNumbers() {
try {
while (state) {
wait();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(Thread.currentThread().getName() + " " + i);
state = true;
i++;
notifyAll();
}
public static void main(String[] args) {
ThreadClass threadClass = new ThreadClass();
Thread t1 = new Thread() {
int k = 0;
#Override
public void run() {
while (k < 26) {
threadClass.printAlphabet();
k++;
}
}
};
t1.setName("Thread1");
Thread t2 = new Thread() {
int j = 0;
#Override
public void run() {
while (j < 26) {
threadClass.printNumbers();
j++;
}
}
};
t2.setName("Thread2");
t1.start();
t2.start();
}
}
Your threads are running at the same time. But not the way you want it, as mentioned above. You will see blocks of data from thread 1 and then a block of data from thread 2; and this is because of thread scheduling. Thread 1 is just queuing its output before thread 2.
To test this theory, increase your output to a 1000 records for example as the alphabet and 26 numbers are not as large to see this.
By doing so, you will see these 'blocks' of data. There is a way to do what you mentioned, but it is not advisable as this is not demonstrating how threads actually work but rather you forcing it to work that way.
With less Code:
class MyRunnable implements Runnable {
private static int n = 1;
private static char c = 'a';
public void run() {
for (int i = 1; i <= 26; i++) {
synchronized (this) {
try {
notifyAll();
if (Thread.currentThread().getName().equals("A")) {
System.out.print(c + ",");
c++;
} else {
System.out.print(n + ",");
n++;
}
if (i != 26) {
wait();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
public class PrintAlphabetNumberJob {
public static void main(String[] args) throws InterruptedException {
MyRunnable r = new MyRunnable();
Thread tAlphabet = new Thread(r, "A");
Thread tNumber = new Thread(r, "N");
tAlphabet.start();
Thread.sleep(100);
tNumber.start();
}
}
This question already has answers here:
Notify not getting the thread out of wait state
(3 answers)
Closed 7 years ago.
Basically I have to create 3 classes (2 threaded).
First one holds some cargo (has a minimum capacity (0) and a maximum (200))
Second one supplies the cargo every 500ms.
Third one takes away from cargo every 500ms.
Main program has one cargo class(1), 2 supplier classes(2) and 2 substraction classes(3). Problem I'm having is that one by one, they're falling into a wait(); state and never get out. Eventually all of them get stucked in the wait() state, with the program running, but without them actually doing anything.
First class:
public class Storage {
private int maxCapacity;
private int currentCapacity;
public Storage( int currentCapacity, int maxCapacity ) {
this.currentCapacity = currentCapacity;
this.maxCapacity = maxCapacity;
}
public int getCapacity(){ return this.currentCapacity; }
public void increase( int q ) {
this.currentCapacity += q;
System.out.println("increase" + q + ". Total: " + currentCapacity);
}
public int getMax() { return this.maxCapacity; }
public void decrease( int q ) {
this.currentCapacity -= q;
System.out.println("decrease - " + q + ". Total: " + currentCapacity);
}
}
2nd class (supplier):
public class Supplier implements Runnable {
private int capacity;
private Storage storage;
private volatile boolean run;
public Supplier( int capacity, Storage storage ) {
this.capacity = capacity;
this.storage = storage;
this.run = true;
}
public void kiss_kill() { run = !run; }
public synchronized void add() {
while(storage.getCapacity() + capacity > storage.getMax()) {
try {
System.out.println("wait - supplier");
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
storage.increase(capacity);
notifyAll();
}
public void run() {
synchronized (this) {
while(run) {
add();
Thread.yield(); //would be wait(500), but this just speeds it up
}
}
}
}
3rd class (taker/demander):
public class Taker implements Runnable {
private int capacity;
private Storage storage;
private volatile boolean run;
public Taker( int capacity, Storage storage ) {
this.capacity = capacity;
this.storage = storage;
this.run = true;
}
public void kiss_kill() { run = !run; }
public synchronized void take() {
while(storage.getCapacity() - capacity < 0) {
try {
System.out.println("wait - taker");
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
storage.decrease(capacity);
notifyAll();
}
public void run() {
synchronized (this) {
while(run) {
take();
Thread.yield(); //again, wait(500) should be instead
}
}
}
}
Main is something like this:
public class Main{
public static void main(String... args) {
Storage sk = new Storage(100, 200);
Supplier[] s = { new Supplier(10, sk), new Supplier(15, sk) };
Taker[] p = { new Taker(15, sk), new Taker(20, sk) };
Thread t[] = {
new Thread(s[0]),
new Thread(s[1]),
new Thread(p[0]),
new Thread(p[1]) };
for(Thread th : t) th.start();
try {
Thread.sleep(60000); //program should last for 60s.
} catch (InterruptedException e) {
e.printStackTrace();
}
s[0].kiss_kill(); s[1].kiss_kill(); p[0].kiss_kill(); p[1].kiss_kill();
}
}
Why doesn't notifyAll() release the wait() state of other object? What could I do to fix this?
Sorry, I know it's a long example, I hate posting too many classes like this. Thanks for reading!
I translated the code, so if you spot anything that you're unsure about that I've missed, please tell me and I'll fix it right away!
Doing concurrency is easy:
Anyone can slap synchronized on methods and synchronized () {} around blocks of code. It does not mean it is correct. And then they can continue to slap synchronized on everything until it works until it doesn't.
Doing concurrency correctly is Hard:
You should lock on the data that needs to be consistent not the methods making the changes. And you have to use the same lock instance for everything.
In this case that is the currentCapacity in Storage. That is the only thing that is shared and the only thing that needs to be consistent.
What you are doing now is having the classes lock on instances of themselves which means nothing shared is being protected because there is no shared lock.
Think about it, if you are not locking on the same exact instance which must be final of an object then what are you protecting?
Also what about code that has access to the object that needs to be consistent and does not request a lock on it. Well it just does what it wants. synchronized() {} in calling classes is not how you protect shared data from external manipulation.
Thread safe objects are NOT about the synchronized keyword:
Read up on the java.util.concurrent package it has all the things you need already. Use the correct data structure for your use case.
In this particular case if you use AtomicInteger for your counter, you do not need any error prone manual locking, no need for synchronized anywhere, it is already thread safe.
Immutable Data:
If you work with immutable data exclusively you do not need any of this silly locking semantics that are extremely error prone for even those that understand it and even more so for those that think they understand it.
Here is a working idiomatic example:
This is a good chance to learn what non-deterministic means and how to use the step debugger in your IDE to debug concurrent programs.
Q33700412.java
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
import com.vertigrated.FormattedRuntimeException;
public class Q33700412
{
public static void main(final String[] args)
{
final Storage s = new Storage(100);
final int ap = Runtime.getRuntime().availableProcessors();
final ExecutorService es = Executors.newFixedThreadPool(ap);
for (int i = 0; i < ap; i++)
{
es.execute(new Runnable()
{
final Random r = new Random();
#Override
public void run()
{
while (true)
{
/* this if/else block is NOT thread safe, I did this on purpose
the state can change between s.remainingCapacity() and
the call to s.increase/s.decrease.
This is ok, because the Storage object is internally consistent.
This thread might fail if this happens, this is the educational part.
*/
if (s.remainingCapacity() > 0)
{
if (r.nextBoolean()) { s.increase(r.nextInt(10)); }
else { s.decrease(10); }
System.out.format("Current Capacity is %d", s.getCurrentCapacity());
System.out.println();
}
else
{
System.out.format("Max Capacity %d Reached", s.getMaxCapacity());
System.out.println();
}
try { Thread.sleep(r.nextInt(5000)); }
catch (InterruptedException e) { throw new RuntimeException(e); }
}
}
});
}
es.shutdown();
try
{
Thread.sleep(TimeUnit.MINUTES.toMillis(1));
es.shutdown();
}
catch (InterruptedException e) { System.out.println("Done!"); }
}
public static final class Storage
{
/* AtomicInteger is used so that it can be mutable and final at the same time */
private final AtomicInteger currentCapacity;
private final int maxCapacity;
public Storage(final int maxCapacity) { this(0, maxCapacity); }
public Storage(final int currentCapacity, final int maxCapacity)
{
this.currentCapacity = new AtomicInteger(currentCapacity);
this.maxCapacity = maxCapacity;
}
public int remainingCapacity() { return this.maxCapacity - this.currentCapacity.get(); }
public int getCurrentCapacity() { return this.currentCapacity.get(); }
public void increase(final int q)
{
synchronized (this.currentCapacity)
{
if (this.currentCapacity.get() < this.maxCapacity)
{
this.currentCapacity.addAndGet(q);
}
else
{
throw new FormattedRuntimeException("Max Capacity %d Exceeded!", this.maxCapacity);
}
}
}
public int getMaxCapacity() { return this.maxCapacity; }
public void decrease(final int q)
{
synchronized (this.currentCapacity)
{
if (this.currentCapacity.get() - q >= 0)
{
this.currentCapacity.addAndGet(q * -1);
}
else
{
this.currentCapacity.set(0);
}
}
}
}
}
Notes:
Limit the scope of synchronized blocks to the minimum they need to protect and lock on the object that needs to stay consistent.
The lock object must be marked final or the reference can change and you will be locking on different instances.
The more final the more correct your programs are likely to be the first time.
Jarrod Roberson gave you the "how" half of the answer. Here's the other half--the "why".
Your Supplier object's add() method waits on itself (i.e., on the supplier object), and it notifies itself.
Your Taker object's take() method waits on its self (i.e., on the taker object), and it notifies its self.
The supplier never notifies the taker, and taker never notifies the supplier.
You should do all of your synchronization on the shared object (i.e., on the Storage object.
So I should convert storage into a thread?
No, you don't want Storage to be a thread, you want it to be the lock. Instead of having your Supplier objects and your Taker objects synchronize on themselves, they should all synchronize on the shared Storage object.
E.g., do this:
public void take() {
synchronized(storage) {
while(...) {
try {
storage.wait();
} catch ...
}
...
storage.notifyAll();
}
}
Instead of this:
public synchronized void take() {
while(...) {
try {
wait();
} catch ...
}
...
notifyAll();
}
And do the same for all of your other synchronized methods.
Questions:
Why do I get a NoSuchElementException when trying to remove the last
element?
How can I fix that?
I have 3 classes (see below) that add/remove Integers to a LinkedList.
Everything works fine until the removing Threads get to the last element.
It seems like both threads try to remove it. The first one succeeds, the second one can´t.
But I thought the synchronized-method/synchroniced-attribute + !sharedList.isEmpty() would handle that.
Class Producer:
This class is supposed to created random numbers, put them in the sharedList, write to console that it just added a number and stop once it gets interrupted. Only 1 thread of this class is expected.
import java.util.LinkedList;
public class Producer extends Thread
{
private LinkedList sharedList;
private String name;
public Producer(String name, LinkedList sharedList)
{
this.name = name;
this.sharedList = sharedList;
}
public void run()
{
while(!this.isInterrupted())
{
while(sharedList.size() < 100)
{
if(this.isInterrupted())
{
break;
} else
{
addElementToList();
}
}
}
}
private synchronized void addElementToList()
{
synchronized(sharedList)
{
sharedList.add((int)(Math.random()*100));
System.out.println("Thread " + this.name + ": " + sharedList.getLast() + " added");
}
try {
sleep(300);
} catch (InterruptedException e) {
this.interrupt();
}
}
}
Class Consumer: This class is supposed to remove the first element in the sharedList, if it exists. The execution should continue (after being interrupted) until sharedList is empty. Multiple (atleast 2) threads of this class are expected.
import java.util.LinkedList;
public class Consumer extends Thread
{
private String name;
private LinkedList sharedList;
public Consumer(String name, LinkedList sharedList)
{
this.name = name;
this.sharedList = sharedList;
}
public void run()
{
while(!this.isInterrupted())
{
while(!sharedList.isEmpty())
{
removeListElement();
}
}
}
private synchronized void removeListElement()
{
synchronized(sharedList)
{
int removedItem = (Integer) (sharedList.element());
sharedList.remove();
System.out.println("Thread " + this.name + ": " + removedItem + " removed");
}
try {
sleep(1000);
} catch (InterruptedException e) {
this.interrupt();
}
}
}
Class MainMethod: This class is supposed to start and interrupt the threads.
import java.util.LinkedList;
public class MainMethod
{
public static void main(String[] args) throws InterruptedException
{
LinkedList sharedList = new LinkedList();
Producer producer = new Producer("producer", sharedList);
producer.start();
Thread.sleep(1000);
Consumer consumer1 = new Consumer("consumer1", sharedList);
Consumer consumer2 = new Consumer("consumer2", sharedList);
consumer1.start();
consumer2.start();
Thread.sleep(10000);
producer.interrupt();
consumer1.interrupt();
consumer2.interrupt();
}
}
Exception: This is the exact exception I get.
Exception in thread "Thread-2" java.util.NoSuchElementException at
java.util.LinkedList.getFirst(LinkedList.java:126) at
java.util.LinkedList.element(LinkedList.java:476) at
Consumer.removeListElement(Consumer.java:29) at
Consumer.run(Consumer.java:20)
Your exception is rather simple to explain. In
while(!sharedList.isEmpty())
{
removeListElement();
}
sharedList.isEmpty() happens outside of synchronization and so one consumer can still see a list as empty while another consumer has already taken the last element.
The consumer that wrongfully believed it is empty will not try to remove an element that is no longer there which leads to your crash.
If you want to make it threadsafe using a LinkedList you'll have to do every read / write operation atomic. E.g.
while(!this.isInterrupted())
{
if (!removeListElementIfPossible())
{
break;
}
}
and
// method does not need to be synchronized - no thread besides this one is
// accessing it. Other threads have their "own" method. Would make a difference
// if this method was static, i.e. shared between threads.
private boolean removeListElementIfPossible()
{
synchronized(sharedList)
{
// within synchronized so we can be sure that checking emptyness + removal happens atomic
if (!sharedList.isEmpty())
{
int removedItem = (Integer) (sharedList.element());
sharedList.remove();
System.out.println("Thread " + this.name + ": " + removedItem + " removed");
} else {
// unable to remove an element because list was empty
return false;
}
}
try {
sleep(1000);
} catch (InterruptedException e) {
this.interrupt();
}
// an element was removed
return true;
}
The same problem exists within your producers. But they would just create a 110th element or something like that.
A good solution to your problem would be using a BlockingQueue. See the documentation for an example. The queue does all the blocking & synchronization for you so your code does not have to worry.
Edit: regarding 2 while loops: You don't have to use 2 loops, 1 loop loops enough but you'll run into another problem: consumers may see the queue as empty before the producers have filled it. So you either have to make sure that there is something in the queue before it can be consumed or you'll have to stop threads manually in other ways. You thread.sleep(1000) after starting the producer should be rather safe but threads are not guaranteed to be running even after 1 second. Use e.g. a CountDownLatch to make it actually safe.
I am wondering why you are not using the already existing classes that Java offers. I rewrote your program using those, and it becomes much shorter and easier to read. In addition the lack of synchronized, which blocks all threads except for the one who gets the lock (and you even do double synchronization), allows the program to actually run in parallel.
Here is the code:
Producer:
public class Producer implements Runnable {
protected final String name;
protected final LinkedBlockingQueue<Integer> sharedList;
protected final Random random = new Random();
public Producer(final String name, final LinkedBlockingQueue<Integer> sharedList) {
this.name = name;
this.sharedList = sharedList;
}
public void run() {
try {
while (Thread.interrupted() == false) {
final int number = random.nextInt(100);
sharedList.put(number);
System.out.println("Thread " + this.name + ": " + number);
Thread.sleep(100);
}
} catch (InterruptedException e) {
}
}
}
Consumer:
public class Consumer implements Runnable {
protected final String name;
protected final LinkedBlockingQueue<Integer> sharedList;
public Consumer(final String name, final LinkedBlockingQueue<Integer> sharedList) {
this.name = name;
this.sharedList = sharedList;
}
public void run() {
try {
while (Thread.interrupted() == false) {
final int number = sharedList.take();
System.out.println("Thread " + name + ": " + number + " taken.");
Thread.sleep(100);
}
} catch (InterruptedException e) {
}
}
}
Main:
public static void main(String[] args) throws InterruptedException {
final LinkedBlockingQueue<Integer> sharedList = new LinkedBlockingQueue<>(100);
final ExecutorService executor = Executors.newFixedThreadPool(4);
executor.execute(new Producer("producer", sharedList));
Thread.sleep(1000);
executor.execute(new Consumer("consumer1", sharedList));
executor.execute(new Consumer("consumer2", sharedList));
Thread.sleep(1000);
executor.shutdownNow();
}
There are several differences:
Since I use a concurrent list, I do not have to care (much) about synchronization, the list does that internally.
As this list uses atomic locking instead of true blocking via synchronized it will scale much better the more threads are used.
I do set the limit of the blocking queue to 100, so even while there is no check in the producer, there will never be more than 100 elements in the list, as put will block if the limit is reached.
I use random.nextInt(100) which is a convenience function for what you used and will produce a lot less typos as the usage is much clearer.
Producer and Consumer are both Runnables, as this is the preferred way for threading in Java. This allows to later on wrap any form of thread around them for execution, not just the primitive Thread class.
Instead of the Thread, I use an ExecutorService which allows easier control over multiple threads. Thread creation, scheduling and other handling is done internally, so all I need to do is to choose the most appropriate ExecutorService and call shutdownNow() when I am done.
Also note that there is no need to throw an InterruptedException into the void. If the consumer/producer is interrupted, that is a signal to stop execution gracefully as soon as possible. Unless I need to inform others "behind" that thread, there is no need to throw that Exception again (although no harm is done either).
I use the keyword final to note elements that won't change later on. For once this is a hint for the compiler that allows some optimizations, it as well helps me to prevent an accidental change of a variable that is not supposed to change. A lot of problems can be prevented by not allowing variables to change in a threaded environment, as thread-issues almost always require something to be read and written at the same time. Such things cannot happen if you cannot write.
Spending some time to search through the Java library for the class that fits your problem the best usually solves a lot of trouble and reduces the size of the code a lot.
Try to switch places of
while(!sharedList.isEmpty())
and
synchronized(sharedList)
I don't think you need synchronized on removeListElement().
I am trying to implement nodes talking to each other in Java. I am doing this by creating a new thread for every node that wants to talk to the server.
When the given number of nodes, i.e. that many threads have been created, have connected to the server I want each thread to execute their next bit of code after adding to the "sharedCounter".
I think I need to use 'locks' on the shared variable, and something like signalAll() or notifyAll() to get all the threads going, but I can't seem to make clear sense of exactly how this works or to implement it.
Any help explaining these Java concepts would be greatly appreciated :D
Below is roughly the structure of my code:
import java.net.*;
import java.io.*;
public class Node {
public static void main(String[] args) {
...
// Chooses server or client launchers depend on parameters.
...
}
}
class sharedResource {
private int sharedCounter;
public sharedResource(int i) {
sharedCounter = i;
}
public synchronized void incSharedCounter() {
sharedCounter--;
if (sharedCounter == 0)
// Get all threads to do something
}
}
class Server {
...
for (int i = 0; i < numberOfThreads; i++) {
new serverThread(serverSocket.accept()).start();
}
...
sharedResource threadCount = new sharedResource(numberOfThreads);
...
}
class serverThread extends Thread {
...
//some code
Server.threadCount.incSharedCounter();
// Some more code to run when sharedCounte == 0
...
}
class Client {
...
}
// Get all threads to do something
Threads (or rather Runnables, which you should implement rather than extending Thread) have a run method that contains the code they are expected to execute.
Once you call Thread#start (which in turn calls Runnable#run), the thread will start doing exactly that.
Since you seem to be new to multi-threading in Java, I recommend that you read an introduction to the Concurrency Utility package, that has been introduced in Java5 to make it easier to implement concurrent operations.
Specifically what you seem to be looking for is a way to "pause" the operation until a condition is met (in your case a counter having reached zero). For this, you should look at a CountDownLatch.
Indeed, the subject is broad, but I'll try to explain the basics. More details can be read from various blogs and articles. One of which is the Java trail.
It is best to see each thread as being runners (physical persons) that run alongside each other in a race. Each runner may perform any task while running. For example, take a cup of water from a table at a given moment in the race. Physically, they cannot both drink from the same cup at once, but in the virtual world, it is possible (this is where the line is drawn).
For example, take again two runners; each of them has to run back and forth a track, and push a button (shared by the runners) at each end for 1'000'000 times, the button is simply incrementing a counter by one each time. When they completed their run, what would be the value of the counter? In the physical world, it would be 2'000'000 because the runners cannot push the button at the same time, they would wait for the first one to leave first... that is unless they fight over it... Well, this is exactly what two threads would do. Consider this code :
public class ThreadTest extends Thread {
static public final int TOTAL_INC = 1000000;
static public int counter = 0;
#Override
public void run() {
for (int i=0; i<TOTAL_INC; i++) {
counter++;
}
System.out.println("Thread stopped incrementing counter " + TOTAL_INC + " times");
}
public static void main(String[] args) throws InterruptedException {
Thread t1 = new ThreadTest();
Thread t2 = new ThreadTest();
t1.start();
t2.start();
t1.join(); // wait for each thread to stop on their own...
t2.join(); //
System.out.println("Final counter is : " + counter + " which should be equal to " + TOTAL_INC * 2);
}
}
An output could be something like
Thread stopped incrementing counter 1000000 times
Thread stopped incrementing counter 1000000 times
Final counter is : 1143470 which should be equal to 2000000
Once in a while, the two thread would just increment the same value twice; this is called a race condition.
Synchronizing the run method will not work, and you'd have to use some locking mechanism to prevent this from happening. Consider the following changes in the run method :
static private Object lock = new Object();
#Override
public void run() {
for (int i=0; i<TOTAL_INC; i++) {
synchronized(lock) {
counter++;
}
}
System.out.println("Thread stopped incrementing counter " + TOTAL_INC + " times");
}
Now the expected output is
...
Final counter is : 2000000 which should be equal to 2000000
We have synchronized our counter with a shared object. This is like putting a queue line before only one runner can access the button at once.
NOTE : this locking mechanism is called a mutex. If a resource can be accessed by n threads at once, you might consider using a semaphore.
Multithreading is also associated with deadlocking. A deadlock is when two threads mutually waits for the other to free some synchronized resource to continue. For example :
Thread 1 starts
Thread 2 starts
Thread 1 acquire synchronized object1
Thread 2 acquire synchronized object2
Thread 2 needs to acquire object2 for continuing (locked by Thread 1)
Thread 1 needs to acquire object1 for continuing (locked by Thread 2)
Program hangs in deadlock
While there are many ways to prevent this from happening (it depends on what your threads are doing, and how they are implemented...) You should read about that particularly.
NOTE : the methods wait, notify and notifyAll can only be called when an object is synchronized. For example :
static public final int TOTAL_INC = 10;
static private int counter = 0;
static private Object lock = new Object();
static class Thread1 extends Thread {
#Override
public void run() {
synchronized (lock) {
for (int i=0; i<TOTAL_INC; i++) {
try {
lock.wait();
counter++;
lock.notify();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
static class Thread2 extends Thread {
#Override
public void run() {
synchronized (lock) {
for (int i=0; i<TOTAL_INC; i++) {
try {
lock.notify();
counter--;
lock.wait();
} catch (InterruptedException e) {
/* ignored */
}
}
}
}
}
Notice that both threads are running their for...loop blocks within the synchronized block. (The result of counter == 0 when both threads end.) This can be achieved because they "let each other" access the synchronized resource via the resource's wait and notify methods. Without using those two methods, both threads would simply run sequentially and not concurrently (or more precisely, alternately).
I hope this shed some light about threads (in Java).
** UPDATE **
Here is a little proof of concept of everything discussed above, using the CountDownLatch class suggested by Thilo earlier :
static class Server {
static public final int NODE_COUNT = 5;
private List<RunnableNode> nodes;
private CountDownLatch startSignal;
private Object lock = new Object();
public Server() {
nodes = Collections.synchronizedList(new ArrayList<RunnableNode>());
startSignal = new CountDownLatch(Server.NODE_COUNT);
}
public Object getLock() {
return lock;
}
public synchronized void connect(RunnableNode node) {
if (startSignal.getCount() > 0) {
startSignal.countDown();
nodes.add(node);
System.out.println("Received connection from node " + node.getId() + " (" + startSignal.getCount() + " remaining...)");
} else {
System.out.println("Client overflow! Refusing connection from node " + node.getId());
throw new IllegalStateException("Too many nodes connected");
}
}
public void shutdown() {
for (RunnableNode node : nodes) {
node.shutdown();
}
}
public void awaitAllConnections() {
try {
startSignal.await();
synchronized (lock) {
lock.notifyAll(); // awake all nodes
}
} catch (InterruptedException e) {
/* ignore */
shutdown(); // properly close any connected node now
}
}
}
static class RunnableNode implements Runnable {
private Server server;
private int id;
private boolean working;
public RunnableNode(int id, Server server) {
this.id = id;
this.server = server;
this.working = true;
}
public int getId() {
return id;
}
public void run() {
try {
Thread.sleep((long) (Math.random() * 5) * 1000); // just wait randomly from 0 to 5 seconds....
synchronized (server.getLock()) {
server.connect(this);
server.getLock().wait();
}
if (!Thread.currentThread().isAlive()) {
throw new InterruptedException();
} else {
System.out.println("Node " + id + " started successfully!");
while (working) {
Thread.yield();
}
}
} catch (InterruptedException e1) {
System.out.print("Ooop! ...");
} catch (IllegalStateException e2) {
System.out.print("Awwww! Too late! ...");
}
System.out.println("Node " + id + " is shutting down");
}
public void shutdown() {
working = false; // shutdown node here...
}
}
static public void main(String...args) throws InterruptedException {
Server server = new Server();
for (int i=0; i<Server.NODE_COUNT + 4; i++) { // create 4 more nodes than needed...
new Thread(new RunnableNode(i, server)).start();
}
server.awaitAllConnections();
System.out.println("All connection received! Server started!");
Thread.sleep(6000);
server.shutdown();
}
This is a broad topic. You might try reading through the official guides for concurrency (i.e. threading, more or less) in Java. This isn't something with cut-and-dried solutions; you have to design something.