I have a Situation where I wrote a simple Producer Consumer model for reading in chunks of data from Bluetooth then every 10k bytes I write that to file. I used a standard P-C Model using a Vector as my message holder. So how do I change this so that multiple Thread consumers can read the same messages, I think the term would be Multicaster? I am actually using this on an Android phone so JMS is probably not an option.
static final int MAXQUEUE = 50000;
private Vector<byte[]> messages = new Vector<byte[]>();
/**
* Put the message in the queue for the Consumer Thread
*/
private synchronized void putMessage(byte[] send) throws InterruptedException {
while ( messages.size() == MAXQUEUE )
wait();
messages.addElement( send );
notify();
}
/**
* This method is called by the consumer to see if any messages in the queue
*/
public synchronized byte[] getMessage()throws InterruptedException {
notify();
while ( messages.size() == 0 && !Thread.interrupted()) {
wait(1);
}
byte[] message = messages.firstElement();
messages.removeElement( message );
return message;
}
I am referencing code from an Oreilly book Message Parser section
Pub-sub mechanism is definitely the way to achieve what you want. I am not sure why developing for Android will restrict you from using JMS, which is as simple a spec as it gets. Check out
this thread on SO.
You should definitely use a queue instead of the Vector!
Give every thread its own queue and, when a new message is received, add() the new message to every thread's queue. For flexibility, a listener pattern may be useful, too.
Edit:
Ok, I feel I should add an example, too:
(Classical observer pattern)
This is the interface, all consumers must implement:
public interface MessageListener {
public void newMessage( byte[] message );
}
A producer might look like this:
public class Producer {
Collection<MessageListener> listeners = new ArrayList<MessageListener>();
// Allow interested parties to register for new messages
public void addListener( MessageListener listener ) {
this.listeners.add( listener );
}
public void removeListener( Object listener ) {
this.listeners.remove( listener );
}
protected void produceMessages() {
byte[] msg = new byte[10];
// Create message and put into msg
// Tell all registered listeners about the new message:
for ( MessageListener l : this.listeners ) {
l.newMessage( msg );
}
}
}
And a consumer class could be (using a blocking queue which does all that wait()ing and notify()ing for us):
public class Consumer implements MessageListener {
BlockingQueue< byte[] > queue = new LinkedBlockingQueue< byte[] >();
// This implements the MessageListener interface:
#Override
public void newMessage( byte[] message ) {
try {
queue.put( message );
} catch (InterruptedException e) {
// won't happen.
}
}
// Execute in another thread:
protected void handleMessages() throws InterruptedException {
while ( true ) {
byte[] newMessage = queue.take();
// handle the new message.
}
}
}
This is what I came up with as an example when digging through some code and modifiying some existing examples.
package test.messaging;
import java.util.ArrayList;
import java.util.concurrent.LinkedBlockingQueue;
public class TestProducerConsumers {
static Broker broker;
public TestProducerConsumers(int maxSize) {
broker = new Broker(maxSize);
Producer p = new Producer();
Consumer c1 = new Consumer("One");
broker.consumers.add(c1);
c1.start();
Consumer c2 = new Consumer("Two");
broker.consumers.add(c2);
c2.start();
p.start();
}
// Test Producer, use your own message producer on a thread to call up
// broker.insert() possibly passing it the message instead.
class Producer extends Thread {
#Override
public void run() {
while (true) {
try {
broker.insert();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
class Consumer extends Thread {
String myName;
LinkedBlockingQueue<String> queue;
Consumer(String m) {
this.myName = m;
queue = new LinkedBlockingQueue<String>();
}
#Override
public void run() {
while(!Thread.interrupted()) {
try {
while (queue.size() == 0 && !Thread.interrupted()) {
;
}
while (queue.peek() == null && !Thread.interrupted()) {
;
}
System.out.println("" + myName + " Consumer: " + queue.poll());
} catch (Exception e) { }
}
}
}
class Broker {
public ArrayList<Consumer> consumers = new ArrayList<Consumer>();
int n;
int maxSize;
public Broker(int maxSize) {
n = 0;
this.maxSize = maxSize;
}
synchronized void insert() throws InterruptedException {
// only here for testing don't want it to runaway and
//memory leak, only testing first 100 samples.
if (n == maxSize)
wait();
System.out.println("Producer: " + n++);
for (Consumer c : consumers) {
c.queue.add("Message " + n);
}
}
}
public static void main(String[] args) {
TestProducerConsumers pc = new TestProducerConsumers(100);
}
}
Related
I am trying to operate on the same source with two threads. I designed a typical producer and consumer problem for it. While setting the value in the resource class with the producer, I want to get setted values with the consumer one by one. The output I want should be like this:
Producer -> Setting data = 0
Consumer -> Getting data = 0
Producer -> Setting data = 1
Consumer -> Getting data = 1
Producer -> Setting data = 2
Consumer -> Getting data = 2
Producer -> Setting data = 3
Consumer -> Getting data = 3
Producer -> Setting data = 4
Consumer -> Getting data = 4
Here is my Resource class:
public class Resource{
private int value;
private boolean current = false;
public synchronized void setValue(int val) {
while(current == true) {
try {
wait();
}catch(Exception ex) {}}
value = val;
current = true;
notifyAll();
}
public synchronized int getValue() {
while(current == false) {
try {
wait();
}catch(Exception ex) {}}
current = false;
notifyAll();
return value;
}
}
And main method and Producer,Consumer class is here:
class Producer extends Thread{
private Resource rs;
public Producer(Resource rs1) {
rs = rs1;
}
public void run() {
for(int i = 0 ; i < 5 ; i++) {
rs.setValue(i);
System.out.println("Producer -> Setting data = " + i);
try {
sleep(100);
}catch(Exception ex){
ex.printStackTrace();
}
}
}
}
class Consumer extends Thread{
private Resource rs;
public Consumer(Resource rs1) {
rs = rs1;
}
public void run() {
int value = 0;
for(int i = 0 ; i < 5; i++) {
value = rs.getValue();
System.out.println("Consumer -> Getting data= " + i);
try {
sleep(100);
}catch(Exception ex) {
ex.printStackTrace();
}
}
}
}
public class Dependent {
public static void main(String[] args) throws IOException {
Resource res = new Resource();
Producer p1 = new Producer(res);
Consumer c1 = new Consumer(res);
p1.start();
c1.start();
}
}
Although I use synchronized, wait and notifyAll keywords in the methods in the resource class, the threads continue to work without waiting for each other. Where am I making a mistake? I've seen a code sample similar to this code sample in a java book, there doesn't seem to be a problem.
When I write without adding the current boolean variable, the code doesn't even work. That's why I had to add it by looking from the book. Don't the threads need to work synchronously without checking the Current value?
They do wait for each other, but the thread sync operations are much, much faster than Thread.sleep(100) so you can't tell. Your test code prints 'i' and not 'value', which is suspect. Get rid of Thread.sleep(100) in one of these threads (for example, in the consumer) and you'll find that the consumer nevertheless still requires about half a second to complete - as it will be waiting about 100 msec every time it invokes .getValue() on the resource, because that call will block (stuck in that wait() loop) until the producer calls .setValue which it only does about once every 100 msec.
Your Resource object 'works', for some value of 'works', but is very poorly designed, re-creating already existing and better implemented classes from the core library such as a java.util.concurrent.Latch, and which ignore interrupts and will blindly just keep waiting.
Their APIs are also a tad oddly named, in that a get call has considerably side effects. get is more of a get and clear operation: After a get operation, another get operation will freeze the thread forever, or at least, until some thread sets a value.
How do you think?
import java.io.IOException;
class Resource {
private volatile Integer value;
public synchronized void setValue(int val) {
while(value != null && !value.equals(val)) {
try {
wait();
}catch(Exception ex) {}}
value = val;
notifyAll();
}
public synchronized int getValue() {
while(value == null) {
try {
wait();
}catch(Exception ex) {}}
int answer = value;
value = null;
notifyAll();
return answer;
}
}
class Producer extends Thread{
private Resource rs;
public Producer(Resource rs1) {
rs = rs1;
}
public void run() {
for(int i = 0 ; i < 5 ; i++) {
rs.setValue(i);
System.out.println("Producer -> Setting data = " + i);
try {
sleep(100);
}catch(Exception ex){
ex.printStackTrace();
}
}
}
}
class Consumer extends Thread{
private Resource rs;
public Consumer(Resource rs1) {
rs = rs1;
}
public void run() {
for(int i = 0 ; i < 5; i++) {
int value = rs.getValue();
System.out.println("Consumer -> Getting data= " + value);
try {
sleep(100);
}catch(Exception ex) {
ex.printStackTrace();
}
}
}
}
public class Dependent {
public static void main(String[] args) throws IOException {
Resource res = new Resource();
Producer p1 = new Producer(res);
Consumer c1 = new Consumer(res);
p1.start();
c1.start();
}
}
or
class Resource {
private static final int WAIT_VALUE = -1;
private volatile int value = WAIT_VALUE;
public synchronized void setValue(int val) {
while(value > WAIT_VALUE && value != val) {
try {
wait();
}catch(Exception ex) {}}
value = val;
notifyAll();
}
public synchronized int getValue() {
while(value == WAIT_VALUE) {
try {
wait();
}catch(Exception ex) {}}
int answer = value;
value = WAIT_VALUE;
notifyAll();
return answer;
}
}
I have been reading zookeeper source code and found that when fetching ToSend message from the queue in FastLeaderElection.WorkerSender, they use poll() instead of take(). Why?
Is poll() better?
Searching the git history, I've found that it started using take(), so there must have been a reason for changing to poll().
public void run() {
while (!stop) {
try {
ToSend m = sendqueue.poll(3000, TimeUnit.MILLISECONDS);
if (m == null) {
continue;
}
process(m);
} catch (InterruptedException e) {
break;
}
}
LOG.info("WorkerSender is down");
}
TL;DR: To check the stop flag.
Change made for ZOOKEEPER-498: Diff of FastLeaderElection.java
Old code:
class WorkerReceiver implements Runnable {
QuorumCnxManager manager;
WorkerReceiver(QuorumCnxManager manager) {
this.manager = manager;
}
public void run() {
Message response;
while (true) {
// Sleeps on receive
try{
response = manager.recvQueue.take();
// Receive new message
LOG.debug("Receive new message.");
New code:
class WorkerReceiver implements Runnable {
volatile boolean stop;
QuorumCnxManager manager;
WorkerReceiver(QuorumCnxManager manager) {
this.stop = false;
this.manager = manager;
}
public void run() {
Message response;
while (!stop) {
// Sleeps on receive
try{
response = manager.recvQueue.poll(3000, TimeUnit.MILLISECONDS);
if(response == null) continue;
// Receive new message
LOG.debug("Receive new message.");
they use poll() instead of take(). Why ?
So they can check the new stop flag every 3 seconds.
I am working on my application which sends data to zeromq. Below is what my application does:
I have a class SendToZeroMQ that send data to zeromq.
Add same data to retryQueue in the same class so that it can be retried later on if acknowledgment is not received. It uses guava cache with maximumSize limit.
Have a separate thread which receives acknowledgement from the zeromq for the data that was sent earlier and if acknowledgement is not received, then SendToZeroMQ will retry sending that same piece of data. And if acknowledgement is received, then we will remove it from retryQueue so that it cannot be retried again.
Idea is very simple and I have to make sure my retry policy works fine so that I don't loose my data. This is very rare but in case if we don't receive acknolwedgements.
I am thinking of building two types of RetryPolicies but I am not able to understand how to build that here corresponding to my program:
RetryNTimes: In this it will retry N times with a particular sleep between each retry and after that, it will drop the record.
ExponentialBackoffRetry: In this it will exponentially keep retrying. We can set some max retry limit and after that it won't retry and will drop the record.
Below is my SendToZeroMQ class which sends data to zeromq, also retry every 30 seconds from a background thread and start ResponsePoller runnable which keeps running forever:
public class SendToZeroMQ {
private final ScheduledExecutorService executorService = Executors.newScheduledThreadPool(5);
private final Cache<Long, byte[]> retryQueue =
CacheBuilder
.newBuilder()
.maximumSize(10000000)
.concurrencyLevel(200)
.removalListener(
RemovalListeners.asynchronous(new CustomListener(), executorService)).build();
private static class Holder {
private static final SendToZeroMQ INSTANCE = new SendToZeroMQ();
}
public static SendToZeroMQ getInstance() {
return Holder.INSTANCE;
}
private SendToZeroMQ() {
executorService.submit(new ResponsePoller());
// retry every 30 seconds for now
executorService.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
for (Entry<Long, byte[]> entry : retryQueue.asMap().entrySet()) {
sendTo(entry.getKey(), entry.getValue());
}
}
}, 0, 30, TimeUnit.SECONDS);
}
public boolean sendTo(final long address, final byte[] encodedRecords) {
Optional<ZMQSocketInfo> liveSockets = PoolManager.getInstance().getNextSocket();
if (!liveSockets.isPresent()) {
return false;
}
return sendTo(address, encodedRecords, liveSockets.get().getSocket());
}
public boolean sendTo(final long address, final byte[] encodedByteArray, final Socket socket) {
ZMsg msg = new ZMsg();
msg.add(encodedByteArray);
boolean sent = msg.send(socket);
msg.destroy();
// adding to retry queue
retryQueue.put(address, encodedByteArray);
return sent;
}
public void removeFromRetryQueue(final long address) {
retryQueue.invalidate(address);
}
}
Below is my ResponsePoller class which polls all the acknowledgement from the zeromq. And if we get an acknowledgement back from the zeromq then we will remove that record from the retry queue so that it doesn't get retried otherwise it will get retried.
public class ResponsePoller implements Runnable {
private static final Random random = new Random();
#Override
public void run() {
ZContext ctx = new ZContext();
Socket client = ctx.createSocket(ZMQ.PULL);
String identity = String.format("%04X-%04X", random.nextInt(), random.nextInt());
client.setIdentity(identity.getBytes(ZMQ.CHARSET));
client.bind("tcp://" + TestUtils.getIpaddress() + ":8076");
PollItem[] items = new PollItem[] {new PollItem(client, Poller.POLLIN)};
while (!Thread.currentThread().isInterrupted()) {
// Tick once per second, pulling in arriving messages
for (int centitick = 0; centitick < 100; centitick++) {
ZMQ.poll(items, 10);
if (items[0].isReadable()) {
ZMsg msg = ZMsg.recvMsg(client);
Iterator<ZFrame> it = msg.iterator();
while (it.hasNext()) {
ZFrame frame = it.next();
try {
long address = TestUtils.getAddress(frame.getData());
// remove from retry queue since we got the acknowledgment for this record
SendToZeroMQ.getInstance().removeFromRetryQueue(address);
} catch (Exception ex) {
// log error
} finally {
frame.destroy();
}
}
msg.destroy();
}
}
}
ctx.destroy();
}
}
Question:
As you can see above, I am sending encodedRecords to zeromq using SendToZeroMQ class and then it gets retried every 30 seconds depending on whether we got an acknolwedgement back from ResponsePoller class or not.
For each encodedRecords there is a unique key called address and that's what we will get back from zeromq as an acknowledgement.
How can I go ahead and extend this example to build two retry policies that I mentioned above and then I can pick what retry policy I want to use while sending data. I came up with below interface but then I am not able understand how should I move forward to implement those retry policies and use it in my above code.
public interface RetryPolicy {
/**
* Called when an operation has failed for some reason. This method should return
* true to make another attempt.
*/
public boolean allowRetry(int retryCount, long elapsedTimeMs);
}
Can I use guava-retrying or failsafe here becuase these libraries already have many retry policies which I can use?
I am not able to work out all the details regarding how to use the relevant API-s, but as for algorithm, you could try:
the retry-policy needs to have some sort of state attached to each message (atleast the number of times the current message has been retried, possible what the current delay is). You need to decide whether the RetryPolicy should keep that itself or if you want to store it inside the message.
instead of allowRetry, you could have a method calculating when the next retry should occur (in absolute time or as a number of milliseconds in the future), which will be a function of the state mentioned above
the retry queue should contain information on when each message should be retried.
instead of using scheduleAtFixedRate, find the message in the retry queue which has the lowest when_is_next_retry (possibly by sorting on absolute retry-timestamp and picking the first), and let the executorService reschedule itself using schedule and the time_to_next_retry
for each retry, pull it from the retry queue, send the message, use the RetryPolicy for calculating when the next retry should be (if it is to be retried) and insert back into the retry queue with a new value for when_is_next_retry (if the RetryPolicy returns -1, it could mean that the message shall not be retried any more)
not a perfect way, but can be achieved by below way as well.
public interface RetryPolicy {
public boolean allowRetry();
public void decreaseRetryCount();
}
Create two implementation. For RetryNTimes
public class RetryNTimes implements RetryPolicy {
private int maxRetryCount;
public RetryNTimes(int maxRetryCount) {
this.maxRetryCount = maxRetryCount;
}
public boolean allowRetry() {
return maxRetryCount > 0;
}
public void decreaseRetryCount()
{
maxRetryCount = maxRetryCount-1;
}}
For ExponentialBackoffRetry
public class ExponentialBackoffRetry implements RetryPolicy {
private int maxRetryCount;
private final Date retryUpto;
public ExponentialBackoffRetry(int maxRetryCount, Date retryUpto) {
this.maxRetryCount = maxRetryCount;
this.retryUpto = retryUpto;
}
public boolean allowRetry() {
Date date = new Date();
if(maxRetryCount <= 0 || date.compareTo(retryUpto)>=0)
{
return false;
}
return true;
}
public void decreaseRetryCount() {
maxRetryCount = maxRetryCount-1;
}}
You need to make some changes in SendToZeroMQ class
public class SendToZeroMQ {
private final ScheduledExecutorService executorService = Executors.newScheduledThreadPool(5);
private final Cache<Long,RetryMessage> retryQueue =
CacheBuilder
.newBuilder()
.maximumSize(10000000)
.concurrencyLevel(200)
.removalListener(
RemovalListeners.asynchronous(new CustomListener(), executorService)).build();
private static class Holder {
private static final SendToZeroMQ INSTANCE = new SendToZeroMQ();
}
public static SendToZeroMQ getInstance() {
return Holder.INSTANCE;
}
private SendToZeroMQ() {
executorService.submit(new ResponsePoller());
// retry every 30 seconds for now
executorService.scheduleAtFixedRate(new Runnable() {
public void run() {
for (Map.Entry<Long, RetryMessage> entry : retryQueue.asMap().entrySet()) {
RetryMessage retryMessage = entry.getValue();
if(retryMessage.getRetryPolicy().allowRetry())
{
retryMessage.getRetryPolicy().decreaseRetryCount();
entry.setValue(retryMessage);
sendTo(entry.getKey(), retryMessage.getMessage(),retryMessage);
}else
{
retryQueue.asMap().remove(entry.getKey());
}
}
}
}, 0, 30, TimeUnit.SECONDS);
}
public boolean sendTo(final long address, final byte[] encodedRecords, RetryMessage retryMessage) {
Optional<ZMQSocketInfo> liveSockets = PoolManager.getInstance().getNextSocket();
if (!liveSockets.isPresent()) {
return false;
}
if(null==retryMessage)
{
RetryPolicy retryPolicy = new RetryNTimes(10);
retryMessage = new RetryMessage(retryPolicy,encodedRecords);
retryQueue.asMap().put(address,retryMessage);
}
return sendTo(address, encodedRecords, liveSockets.get().getSocket());
}
public boolean sendTo(final long address, final byte[] encodedByteArray, final ZMQ.Socket socket) {
ZMsg msg = new ZMsg();
msg.add(encodedByteArray);
boolean sent = msg.send(socket);
msg.destroy();
return sent;
}
public void removeFromRetryQueue(final long address) {
retryQueue.invalidate(address);
}}
Here is a working little simulation of your environment that shows how this can be done. Note the Guava cache is the wrong data structure here, since you aren't interested in eviction (I think). So I'm using a concurrent hashmap:
package experimental;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
import java.util.Arrays;
import java.util.Iterator;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.ScheduledExecutorService;
class Experimental {
/** Return the desired backoff delay in millis for the given retry number, which is 1-based. */
interface RetryStrategy {
long getDelayMs(int retry);
}
enum ConstantBackoff implements RetryStrategy {
INSTANCE;
#Override
public long getDelayMs(int retry) {
return 1000L;
}
}
enum ExponentialBackoff implements RetryStrategy {
INSTANCE;
#Override
public long getDelayMs(int retry) {
return 100 + (1L << retry);
}
}
static class Sender {
private final ScheduledExecutorService executorService = Executors.newScheduledThreadPool(4);
private final ConcurrentMap<Long, Retrier> pending = new ConcurrentHashMap<>();
/** Send the given data with given address on the given socket. */
void sendTo(long addr, byte[] data, int socket) {
System.err.println("Sending " + Arrays.toString(data) + "#" + addr + " on " + socket);
}
private class Retrier implements Runnable {
private final RetryStrategy retryStrategy;
private final long addr;
private final byte[] data;
private final int socket;
private int retry;
private Future<?> future;
Retrier(RetryStrategy retryStrategy, long addr, byte[] data, int socket) {
this.retryStrategy = retryStrategy;
this.addr = addr;
this.data = data;
this.socket = socket;
this.retry = 0;
}
synchronized void start() {
if (future == null) {
future = executorService.submit(this);
pending.put(addr, this);
}
}
synchronized void cancel() {
if (future != null) {
future.cancel(true);
future = null;
}
}
private synchronized void reschedule() {
if (future != null) {
future = executorService.schedule(this, retryStrategy.getDelayMs(++retry), MILLISECONDS);
}
}
#Override
synchronized public void run() {
sendTo(addr, data, socket);
reschedule();
}
}
long getVerifiedAddr() {
System.err.println("Pending messages: " + pending.size());
Iterator<Long> i = pending.keySet().iterator();
long addr = i.hasNext() ? i.next() : 0;
return addr;
}
class CancellationPoller implements Runnable {
#Override
public void run() {
while (!Thread.currentThread().isInterrupted()) {
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
long addr = getVerifiedAddr();
if (addr == 0) {
continue;
}
System.err.println("Verified message (to be cancelled) " + addr);
Retrier retrier = pending.remove(addr);
if (retrier != null) {
retrier.cancel();
}
}
}
}
Sender initialize() {
executorService.submit(new CancellationPoller());
return this;
}
void sendWithRetriesTo(RetryStrategy retryStrategy, long addr, byte[] data, int socket) {
new Retrier(retryStrategy, addr, data, socket).start();
}
}
public static void main(String[] args) {
Sender sender = new Sender().initialize();
for (long i = 1; i <= 10; i++) {
sender.sendWithRetriesTo(ConstantBackoff.INSTANCE, i, null, 42);
}
for (long i = -1; i >= -10; i--) {
sender.sendWithRetriesTo(ExponentialBackoff.INSTANCE, i, null, 37);
}
}
}
You can use apache camel. It provides a component for zeromq, and tools like errohandler, redeliverypolicy, deadletter channel and such things are natively provided.
I want to write program using multithreading wait and notify methods in Java.
This program has a stack (max-length = 5). Producer generate number forever and put it in the stack, and consumer pick it from stack.
When stack is full producer must wait and when stack is empty consumers must wait.
The problem is that it runs just once, I mean once it produce 5 number it stops but i put run methods in while(true) block to run nonstop able but it doesn't.
Here is what i tried so far.
Producer class:
package trail;
import java.util.Random;
import java.util.Stack;
public class Thread1 implements Runnable {
int result;
Random rand = new Random();
Stack<Integer> A = new Stack<>();
public Thread1(Stack<Integer> A) {
this.A = A;
}
public synchronized void produce()
{
while (A.size() >= 5) {
System.out.println("List is Full");
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
result = rand.nextInt(10);
System.out.println(result + " produced ");
A.push(result);
System.out.println(A);
this.notify();
}
#Override
public void run() {
System.out.println("Producer get started");
try {
Thread.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
while (true) {
produce();
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
And the consumer:
package trail;
import java.util.Stack;
public class Thread2 implements Runnable {
Stack<Integer> A = new Stack<>();
public Thread2(Stack<Integer> A) {
this.A = A;
}
public synchronized void consume() {
while (A.isEmpty()) {
System.err.println("List is empty" + A + A.size());
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.err.println(A.pop() + " Consumed " + A);
this.notify();
}
#Override
public void run() {
System.out.println("New consumer get started");
try {
Thread.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
while (true) {
consume();
}
}
}
and here is the main method:
public static void main(String[] args) {
Stack<Integer> stack = new Stack<>();
Thread1 thread1 = new Thread1(stack);// p
Thread2 thread2 = new Thread2(stack);// c
Thread A = new Thread(thread1);
Thread B = new Thread(thread2);
Thread C = new Thread(thread2);
A.start();
B.start();
C.start();
}
I think it will be better for understanding and dealing with synchronisation in general if you try to separate three things which are currently mixed:
Task which is going to do the actual job. Names for classes Thread1 & Thread2 are misleading. They are not Thread objects, but they are actually jobs or tasks implementing Runnable interface you are giving to Thread objects.
Thread object itself which you are creating in main
Shared object which encapsulates synchronised operations/logic on a queue, a stack etc. This object will be shared between tasks. And inside this shared object you will take care of add/remove operations (either with synchronized blocks or synchronized methods). Currently (as it was pointed out already), synchronization is done on a task itself (i.e. each task waits and notifies on its own lock and nothing happens). When you separate concerns, i.e. let one class do one thing properly it will eventually become clear where is the problem.
Your consumer and you producer are synchronized on different objects and do not block each other. If this works, I daresay it's accidental.
Read up on java.util.concurrent.BlockingQueue and java.util.concurrent.ArrayBlockingQueue. These provide you with more modern and easier way to implement this pattern.
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/BlockingQueue.html
You should synchronize on the stack instead of putting it at the method level try this code.
Also don't initalize the stack in your thread classes anyways you are passing them in the constructor from the main class, so no need of that.
Always try to avoid mark any method with synchronized keyword instead of that try to put critical section of code in the synchronized block because the more size of your synchronized area more it will impact on performance.
So, always put only that code into synchronized block that need thread safety.
Producer Code :
public void produce() {
synchronized (A) {
while (A.size() >= 5) {
System.out.println("List is Full");
try {
A.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
result = rand.nextInt(10);
System.out.println(result + " produced ");
A.push(result);
System.out.println("stack ---"+A);
A.notifyAll();
}
}
Consumer Code :
public void consume() {
synchronized (A) {
while (A.isEmpty()) {
System.err.println("List is empty" + A + A.size());
try {
System.err.println("wait");
A.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.err.println(A.pop() + " Consumed " + A);
A.notifyAll();
}
}
Try this:
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class CircularArrayQueue<T> {
private volatile Lock rwLock = new ReentrantLock();
private volatile Condition emptyCond = rwLock.newCondition();
private volatile Condition fullCond = rwLock.newCondition();
private final int size;
private final Object[] buffer;
private volatile int front;
private volatile int rare;
/**
* #param size
*/
public CircularArrayQueue(int size) {
this.size = size;
this.buffer = new Object[size];
this.front = -1;
this.rare = -1;
}
public boolean isEmpty(){
return front == -1;
}
public boolean isFull(){
return (front == 0 && rare == size-1) || (front == rare + 1);
}
public void enqueue(T item){
try {
// get a write lock
rwLock.lock();
// if the Q is full, wait the write lock
if(isFull())
fullCond.await();
if(rare == -1){
rare = 0;
front = 0;
} else if(rare == size - 1){
rare = 0;
} else {
rare ++;
}
buffer[rare] = item;
//System.out.println("Added\t: " + item);
// notify the reader
emptyCond.signal();
} catch(InterruptedException e){
e.printStackTrace();
} finally {
// unlock the write lock
rwLock.unlock();
}
}
public T dequeue(){
T item = null;
try{
// get the read lock
rwLock.lock();
// if the Q is empty, wait the read lock
if(isEmpty())
emptyCond.await();
item = (T)buffer[front];
//System.out.println("Deleted\t: " + item);
if(front == rare){
front = rare = -1;
} else if(front == size - 1){
front = 0;
} else {
front ++;
}
// notify the writer
fullCond.signal();
} catch (InterruptedException e){
e.printStackTrace();
} finally{
// unlock read lock
rwLock.unlock();
}
return item;
}
}
You can use Java's awesome java.util.concurrent package and its classes.
You can easily implement the producer consumer problem using the
BlockingQueue. A BlockingQueue already supports operations that wait
for the queue to become non-empty when retrieving an element, and wait
for space to become available in the queue when storing an element.
Without BlockingQueue, every time we put data to queue at the producer
side, we need to check if queue is full, and if full, wait for some
time, check again and continue. Similarly on the consumer side, we
would have to check if queue is empty, and if empty, wait for some
time, check again and continue. However with BlockingQueue we don’t
have to write any extra logic than to just add data from Producer and
poll data from Consumer.
Read more From:
http://javawithswaranga.blogspot.in/2012/05/solving-producer-consumer-problem-in.html
http://www.javajee.com/producer-consumer-problem-in-java-using-blockingqueue
use BlockingQueue,LinkedBlockingQueue this was really simple.
http://developer.android.com/reference/java/util/concurrent/BlockingQueue.html
package javaapplication;
import java.util.Stack;
import java.util.logging.Level;
import java.util.logging.Logger;
public class ProducerConsumer {
public static Object lock = new Object();
public static Stack stack = new Stack();
public static void main(String[] args) {
Thread producer = new Thread(new Runnable() {
int i = 0;
#Override
public void run() {
do {
synchronized (lock) {
while (stack.size() >= 5) {
try {
lock.wait();
} catch (InterruptedException e) {
}
}
stack.push(++i);
if (stack.size() >= 5) {
System.out.println("Released lock by producer");
lock.notify();
}
}
} while (true);
}
});
Thread consumer = new Thread(new Runnable() {
#Override
public void run() {
do {
synchronized (lock) {
while (stack.empty()) {
try {
lock.wait();
} catch (InterruptedException ex) {
Logger.getLogger(ProdCons1.class.getName()).log(Level.SEVERE, null, ex);
}
}
while(!stack.isEmpty()){
System.out.println("stack : " + stack.pop());
}
lock.notifyAll();
}
} while (true);
}
});
producer.start();
consumer.start();
}
}
Have a look at this code example:
import java.util.concurrent.*;
import java.util.Random;
public class ProducerConsumerMulti {
public static void main(String args[]){
BlockingQueue<Integer> sharedQueue = new LinkedBlockingQueue<Integer>();
Thread prodThread = new Thread(new Producer(sharedQueue,1));
Thread consThread1 = new Thread(new Consumer(sharedQueue,1));
Thread consThread2 = new Thread(new Consumer(sharedQueue,2));
prodThread.start();
consThread1.start();
consThread2.start();
}
}
class Producer implements Runnable {
private final BlockingQueue<Integer> sharedQueue;
private int threadNo;
private Random rng;
public Producer(BlockingQueue<Integer> sharedQueue,int threadNo) {
this.threadNo = threadNo;
this.sharedQueue = sharedQueue;
this.rng = new Random();
}
#Override
public void run() {
while(true){
try {
int number = rng.nextInt(100);
System.out.println("Produced:" + number + ":by thread:"+ threadNo);
sharedQueue.put(number);
Thread.sleep(100);
} catch (Exception err) {
err.printStackTrace();
}
}
}
}
class Consumer implements Runnable{
private final BlockingQueue<Integer> sharedQueue;
private int threadNo;
public Consumer (BlockingQueue<Integer> sharedQueue,int threadNo) {
this.sharedQueue = sharedQueue;
this.threadNo = threadNo;
}
#Override
public void run() {
while(true){
try {
int num = sharedQueue.take();
System.out.println("Consumed: "+ num + ":by thread:"+threadNo);
Thread.sleep(100);
} catch (Exception err) {
err.printStackTrace();
}
}
}
}
Notes:
Started one Producer and two Consumers as per your problem statement
Producer will produce random numbers between 0 to 100 in infinite loop
Consumer will consume these numbers in infinite loop
Both Producer and Consumer share lock free and Thread safe LinkedBlockingQueue which is Thread safe. You can remove wait() and notify() methods if you use these advanced concurrent constructs.
Seems like you skipped something about wait(), notify() and synchronized.
See this example, it should help you.
I have a problem where my class is performing the first run method after which it is not proceeding into a second, overidden run method.
The program execution beings in a controller class which has a main method and a thread pool:
public class RunnableController {
// Main method
public static void main(String[] args) throws InterruptedException {
try {
RunnableController controller = new RunnableController();
controller.initializeDb();
controller.initialiseThreads();
System.out.println("Polling");
} catch (Exception e) {
e.printStackTrace();
}
}
private void initialiseThreads() {
try {
threadExecutorRead = Executors.newFixedThreadPool(10);
PollingSynchronizer read = new PollingSynchronizer(incomingQueue, dbConncetion);
threadExecutorRead.submit(read);
} catch (Exception e){
e.printStackTrace();
}
}
}
My poller class which fetches new data and should do updating simulateously:
public class PollingSynchronizer implements Runnable {
public PollingSynchronizer(Collection<KamMessage> incomingQueue,
Connection dbConnection) {
super();
this.incomingQueue = incomingQueue;
this.dbConnection = dbConnection;
}
private int seqId;
public int getSeqId() {
return seqId;
}
public void setSeqId(int seqId) {
this.seqId = seqId;
}
// The method which runs Polling action and record the time at which it is done
public void run() {
int seqId = 0;
while (true) {
List<KamMessage> list = null;
try {
list = fullPoll(seqId);
if (!list.isEmpty()) {
seqId = list.get(0).getSequence();
incomingQueue.addAll(list);
this.outgoingQueue = incomingQueue;
System.out.println("waiting 3 seconds");
System.out.println("new incoming message");
Thread.sleep(3000);//at this wait I should execute run()
//when I debug my execution stops here and throws " Class not found Exception "
// its does not enters the message processor class
MessageProcessor processor = new MessageProcessor() {
//the run method which should fetch the message processor class.
final public void run() {
MessageProcessor(outgoingQueue).generate(outgoingQueue);
}
};
new Thread(processor).start();
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
}
}
My message processor class:
public abstract class MessageProcessor implements Runnable {
private Connection dbConnection;
Statement st = null;
ResultSet rs = null;
PreparedStatement pstmt = null;
private Collection<KamMessage> outgoingQueue;
public KamMsg804 MessageProcessor(Collection<KamMessage> outgoingQueue,
Connection dbConnection) {
this.outgoingQueue = outgoingQueue;
this.dbConnection = dbConnection;
return (KpiMsg804) fetchedMessages;
}
public Collection<KamMessage> generate(Collection<KamMessage> outgoingQueue) {
while (true) {
try {
while (rs.next()) {
KamMessage filedClass = convertRecordsetToPojo(rs);
outgoingQueue.add(filedClass);
}
for (KamMessage pojoClass : outgoingQueue) {
KamMsg804 updatedValue = createKamMsg804(pojoClass);
System.out.print(" " + pojoClass.getSequence());
System.out.print(" " + pojoClass.getTableName());
System.out.print(" " + pojoClass.getAction());
System.out.print(" " + updatedValue.getKeyInfo1());
System.out.print(" " + updatedValue.getKeyInfo2());
System.out.println(" " + pojoClass.getEntryTime());
}
return outgoingQueue;
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
My problem is exactly at the second run(9 method where I am getting exception in MessageProcessor class and it loops back to Polling.
How do I implement multithreading here, as when the thread sleeps for 3 seocnds in polling it should simultaneously update the database.
After which, how can the data be fed and updated back into the db.
My program flow - I have three classes:
1.Controller
2.PollerSynchro
3.Msgprocessor
I have database records, which are converted into POJO form and stored in a Collection. With these POJOs my classes try to do multiprocessing and updating in a single stretch.
Controller - has the thread pool, initiates poller class with poll method - done
Poller - should poll for new incoming messages and stores it in incoming queue - done
MsgProcessor - should look for new incoming messages and pass them from outgoing queue to incoming queue - also done
Problem:
Now my problem is
I have to implement this update while the poll thread sleeps for 3 sec,
In my code for the second void run() method in the Poller class, the outgoing queue is not passed and fed to the messageprocessor class for updating. My flow of execution only just loops back to first run method and am getting Class exception.
Please help me to solve these problems.
I can't sugar coat this, your code is a mess. However, as far as why your message processor code is not being executed, you never actually start the thread you created with this code:
MessageProcessor processor = new MessageProcessor() {
// the run method which should fetch the message processor class.
final public void run() {
MessageProcessor(outgoingQueue).generate(outgoingQueue);
}
};
Ignoring the confusingly named method being called, your code should look more like this:
Message processor = new MessageProcessor() {
// the run method which should fetch the message processor class.
final public void run() {
MessageProcessor(outgoingQueue).generate(outgoingQueue);
}
};
new Thread(processor).start();