I'm doing some courses on MultiThreading in Java. I've tried (following instructor) to synchronise two consumers reading from ArrayList with one producer filling it with some basic input. Yet producer gives first number and consumer gets in infinite loop because it gets empty List. No idea how to force it to get proper values.
Producer:
class MyProducer implements Runnable {
private List<String> buffer;
private String color;
public MyProducer(List<String> buffer, String color) {
this.buffer = buffer;
this.color = color;
}
public void run() {
Random random = new Random();
String[] nums = {"1", "2", "3", "4", "5"};
for(String num: nums) {
try {
System.out.println(color + "Adding..." + num);
synchronized (buffer) {
buffer.add(num);
System.out.println(buffer);
}
Thread.sleep(random.nextInt(1000));
} catch (InterruptedException e) {
System.out.println("Producer was interrupted");
}
}
System.out.println(color + "Adding EOF and exiting...");
synchronized (buffer) {
buffer.add(EOF);
}
}
}
Consumer:
class MyConsumer implements Runnable {
private List<String> buffer;
private String color;
public MyConsumer(List<String> buffer, String color) {
this.buffer = buffer;
this.color = color;
}
public void run() {
synchronized (buffer) {
while(true) {
if (buffer.isEmpty()) {
continue;
}
System.out.println("Przejście między warunkami działa");
if (buffer.get(0).equals(EOF)) {
System.out.println(color + "Exiting");
break;
} else {
System.out.println(color + "Removed" + buffer.remove(0));
}
}
}
}
}
Main:
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import static com.siecz-k-.Main.EOF;
public class Main {
public static final String EOF = "EOF";
public static void main(String[] args) {
List<String> buffer = new ArrayList<>();
MyProducer producer = new MyProducer(buffer, ThreadColor.ANSI_CYAN);
MyConsumer consumer1 = new MyConsumer(buffer, ThreadColor.ANSI_PURPLE);
MyConsumer consumer2 = new MyConsumer(buffer, ThreadColor.ANSI_RED);
new Thread(producer).start();
new Thread(consumer1).start();
new Thread(consumer2).start();
}
}
Related
This question already has answers here:
From Arraylist to Array
(11 answers)
Closed 6 months ago.
I should create Thread as ProducerThread and ConsumerThread those execute sequentially and put elements to queue. In this implementation I used ArrayList.
How might I convert an ArrayList object to integer Array in java.
This is my ConsumerThread class.
import java.util.ArrayList;
public class ConsumerThread implements Runnable {
ArrayList<Integer> queue = new ArrayList<Integer>();
public ConsumerThread(ArrayList<Integer> queue) {
this.queue = queue;
}
#Override
public void run() {
synchronized(queue) {
int value;
while(true) {
try {
queue.wait();
System.out.println("Consumer Started");
value = queue.remove(0);
System.out.println("Consumer thread consumes " + value);
System.out.println("Elements in Queue = " + queue);
Thread.sleep(1000);
queue.notify();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
This is my ProducerThread class.
import java.util.ArrayList;
public class ProducerThread implements Runnable {
ArrayList<Integer> queue = new ArrayList<Integer>();
public ProducerThread(ArrayList<Integer> queue) {
this.queue = queue;
}
#Override
public void run() {
synchronized(queue) {
int value = 10;
while(true) {
try {
queue.notify();
System.out.println("Producer Started");
System.out.println("Producer adding value = " + value + " to Queue");
queue.add(value);
value = value + 10;
//Thread.sleep(1000);
queue.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
This is my main class.
import java.util.ArrayList;
public class ThreadTest {
public static void main(String[] args) {
ArrayList<Integer> queue = new ArrayList<Integer>();
Thread producer = new Thread(new ProducerThread(queue));
Thread consumer = new Thread(new ConsumerThread(queue));
producer.start();
consumer.start();
System.out.println("Starting");
}
}
toArray() method convert any ArrayList to Array Objects -> Object[]
ArrayList<Integer> queue = new ArrayList<>();
queue.add(23);
queue.add(6765);
Object[] arrayQueue = queue.toArray();
System.out.println("arrayList = " + queue + "\n arrayNative = ");
Arrays.stream(arrayQueue).forEach(System.out::println);
package producerconsumer;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.Random;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.Semaphore;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Buffer {
int t;
private final Semaphore notEmptyy = new Semaphore(0); // prevent underflow
private final Semaphore notFulll = new Semaphore(10); // prevent overflow
private int itemn;
private int itemb;
int count=0;
int buffersize =10 ;
private final BlockingQueue<Integer> queue = new ArrayBlockingQueue<Integer>(buffersize);
private final Semaphore mutex = new Semaphore(1); // control buffer access
private final Semaphore notEmpty = new Semaphore(0); // prevent underflow
private final Semaphore notFull = new Semaphore(10); // prevent overflow
public Buffer(){
}
public void add( int x) throws InterruptedException{
while(count== buffersize )
notFulll.acquire();
System.out.println("user printer-request,: " +Thread.currentThread().getName() + " " + x);
queue.offer(x);
count++;
notEmptyy.release();
}
public int take() throws InterruptedException{
while(queue.isEmpty())
notEmptyy.acquire();
t=queue.take();
count--;
notFulll.release();
return t;
}
public void put( ){
while(true){
try {
Random random = new Random();
int data = random.nextInt(100);
notFull .acquire();
mutex .acquire();
add(data);
mutex .release();
notEmpty .release();
// itemb = data;
} catch (InterruptedException e){
System.out.println("InterruptedException caught ");
}
// System.out.println("user printer-request,: " +Thread.currentThread().getName() + " " + itemb);
// this.item = item;
}
}
public void get(){
while(true){
try{
notEmpty .acquire();
mutex .acquire();
itemn = take();
mutex .release();
notFull .release();
queue.remove(itemn);
System.out.println("print request, : "+Thread.currentThread().getName()+" " + itemn );
} catch (InterruptedException e){
System.out.println("InterruptedException caught ");
}
}
}
}
package producerconsumer;
public class producers implements Runnable{
private static final int DELAY = 500;
private Buffer osLabGroup;
public producers(Buffer buffer){
osLabGroup = buffer;
}
public void run(){
// while(true){
osLabGroup.put();
try{
Thread.sleep(DELAY);
}catch (InterruptedException exception) {}
}
}
package producerconsumer;
public class consumers implements Runnable{
private static final int DELAY = 1000;
private Buffer osLabGroup;
public consumers(Buffer buffer){
osLabGroup = buffer;
}
public void run(){
// while(true){
osLabGroup.get();
try{
Thread.sleep(DELAY);
}catch (InterruptedException exception) {}
}
}
//}
package producerconsumer;
public class ProducerConsumer {
public static void main(String[] args) {
Buffer buffer = new Buffer();
producers p1 = new producers(buffer);
consumers c1 = new consumers(buffer);
producers p2 = new producers(buffer);
consumers c2 = new consumers(buffer);
producers p3 = new producers(buffer);
consumers c3 = new consumers(buffer);
Thread pr1 = new Thread(p1);
Thread co1 = new Thread(c1);
Thread pr2 = new Thread(p2);
Thread co2 = new Thread(c2);
Thread pr3 = new Thread(p3);
Thread co3 = new Thread(c3);
pr1.setName("p1");
co1.setName("c1");
pr2.setName("p2");
co2.setName("c2");
pr3.setName("p3");
co3.setName("c3");
pr1.start();
co1.start();
pr2.start();
co2.start();
pr3.start();
co3.start();
}
}
I have a program that simulates the print jobs of a printer, one job at a time: either from Producer or Consumer and it shares an object called a Buffer.
in the buffer it has Set1 of threads, producers, put data with a ½ second delay.
also it does have a Set2 of threads, consumers, read data from the buffer with one second delay.
as it stands, I am trying to follow this guy https://www.youtube.com/watch?v=nxw2y27z0V4&t=1207s
using semaphore this is my code
I problem my output is not accurate enter image description here
this my edited code
package producerconsumer;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.Semaphore;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Buffer {
int t;
private final Semaphore notEmptyy = new Semaphore(0); // prevent underflow
private final Semaphore notFulll = new Semaphore(10); // prevent overflow
private int itemn;
private int itemb;
int count=0;
int buffersize =10 ;
private final ArrayList<Integer> list = new ArrayList<Integer>(buffersize);
private final LinkedList<Integer> queue = new LinkedList<Integer>(list);
private final Semaphore mutex = new Semaphore(1); // control buffer access
private final Semaphore notEmpty = new Semaphore(0); // prevent underflow
private final Semaphore notFull = new Semaphore(10); // prevent overflow
public Buffer(){
}
public void add( int x) throws InterruptedException{
while(count== buffersize )
notFulll.acquire();
System.out.println("user printer-request,: " +Thread.currentThread().getName() + " " + x);
queue.offer(x);
count++;
notEmptyy.release();
}
public int take() throws InterruptedException{
while(count ==0)
notEmptyy.acquire();
t=queue.pollFirst();
count--;
notFulll.release();
return t;
}
public void put( ){
while(true){
try {
Random random = new Random();
int data = random.nextInt(100);
notFull.acquire();
mutex.acquire();
add(data);
mutex.release();
notEmpty.release();
// itemb = data;
} catch (InterruptedException e){
System.out.println("InterruptedException caught ");
}
// System.out.println("user printer-request,: " +Thread.currentThread().getName() + " " + itemb);
// this.item = item;
}
}
public void get(){
while(true){
try{
notEmpty.acquire();
mutex.acquire();
itemn = take();
mutex.release();
notFull.release();
// queue.remove(itemn);
System.out.println("print request, : "+Thread.currentThread().getName()+" " + itemn );
} catch (InterruptedException e){
System.out.println("InterruptedException caught ");
}
}
}
}
i get this output enter image description here
and this output enter image description here
In Buffer#get, when you take an item from queue, you call queue#remove again. This is a wrong behavior. You should consume this item instead of operating the queue outside of the concurrency control code. This can lead to abnormal behaviors, such as deadlock.
try {
notEmpty .acquire();
mutex .acquire();
itemn = take();
mutex .release();
notFull .release();
// why remove from queue again? It is already taken from queue. Comment this line
// queue.remove(itemn);
System.out.println("print request, : "+Thread.currentThread().getName()+" " + itemn );
} catch (InterruptedException e){
System.out.println("InterruptedException caught ");
}
In addition, in your program, you should replace ArrayBlockingQueue with a collection class under java.util package, such as LinkedList.
ArrayBlockingQueue you use is actually a thread-safe blocking queue. You can directly use take&put to complete this program without other concurrency control.
EDIT:
As mentioned in the video introduction, there are two ways to solve the producer-consumer problem, semaphore or monitor. Your code seems to mix these two together. There is no need to do any concurrency control in your add and take method.
Code should be:
public class consumers implements Runnable{
private static final int DELAY = 1000;
private Buffer osLabGroup;
public consumers(Buffer buffer){
osLabGroup = buffer;
}
public void run(){
while (true) {
osLabGroup.get();
try{
Thread.sleep(DELAY);
}catch (InterruptedException exception) {}
}
}
}
public class producers implements Runnable{
private static final int DELAY = 500;
private Buffer osLabGroup;
public producers(Buffer buffer){
osLabGroup = buffer;
}
public void run(){
while(true) {
osLabGroup.put();
try {
Thread.sleep(DELAY);
} catch (InterruptedException exception) {
}
}
}
}
import java.util.LinkedList;
import java.util.Random;
import java.util.concurrent.Semaphore;
public class Buffer {
int buffersize =10 ;
private final LinkedList<Integer> queue = new LinkedList<Integer>();
private final Semaphore mutex = new Semaphore(1); // control buffer access
private final Semaphore notEmpty = new Semaphore(0); // prevent underflow
private final Semaphore notFull = new Semaphore(buffersize); // prevent overflow
public Buffer(){
}
public void add( int x) throws InterruptedException{
System.out.println("user printer-request,: " +Thread.currentThread().getName() + " " + x + "," + System.currentTimeMillis());
queue.add(x);
}
public int take() throws InterruptedException{
Integer first = queue.removeFirst();
System.out.println("print request, : "+Thread.currentThread().getName()+" " + first + "," + System.currentTimeMillis() );
return first;
}
public void put(){
try {
Random random = new Random();
int data = random.nextInt(100);
notFull.acquire();
mutex.acquire();
add(data);
mutex.release();
notEmpty.release();
} catch (InterruptedException e){
System.out.println("InterruptedException caught ");
}
}
public void get(){
try{
notEmpty.acquire();
mutex.acquire();
int take = take();
mutex.release();
notFull.release();
} catch (InterruptedException e){
System.out.println("InterruptedException caught ");
}
}
}
In the below code, restarting the tailer process is ok. However, restarting the appender process results in the tailer failing to receive any more messages. Is there a way to restart the appender and keep the channel open?
Edited: Below is a full class that I've used to recreate the issue consistently.
Environment:
Ubuntu 18
chronicle-queue-5.16.9.jar
1) java com.tradeplacer.util.IpcTest producer
2) java com.tradeplacer.util.IpcTest consumer
3) kill the producer
4) restart the producer
5) notice that the consumer is no longer reading anything
package com.tradeplacer.util;
import java.nio.ByteBuffer;
import net.openhft.chronicle.bytes.Bytes;
import net.openhft.chronicle.queue.ChronicleQueue;
import net.openhft.chronicle.queue.ChronicleQueueBuilder;
import net.openhft.chronicle.queue.ExcerptAppender;
import net.openhft.chronicle.queue.ExcerptTailer;
import net.openhft.chronicle.queue.RollCycles;
public class IpcTest {
private static final String DIR = "chronicle-test";
public static final void startProducer() {
new Thread() {
public void run() {
System.out.println("starting producer...");
ChronicleQueue queue = ChronicleQueueBuilder.single(DIR).blockSize(65536).rollCycle(RollCycles.MINUTELY).build();
ExcerptAppender appender = queue.acquireAppender();
ByteBuffer ipcBuffer = ByteBuffer.allocate(8192);
for (int i = 0; i < Integer.MAX_VALUE; i++) {
ipcBuffer.clear();
ipcBuffer.put(("data" + i).getBytes());
Bytes<ByteBuffer> bbb = Bytes.wrapForWrite(ipcBuffer);
appender.writeBytes(bbb);
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}.start();
}
public static final void startConsumer() {
new Thread() {
public void run() {
System.out.println("starting consumer...");
ChronicleQueue queue = ChronicleQueueBuilder.single(DIR).blockSize(65536).rollCycle(RollCycles.MINUTELY).build();
ExcerptTailer tailer = queue.createTailer().toEnd(); // skip to end, don't read old messages
Bytes bytes = Bytes.allocateDirect(8192);
while (true) {
try {
long ipcIndex = tailer.index();
boolean read = tailer.readBytes(bytes);
int len = bytes.length();
byte[] data = new byte[len];
bytes.read(data);
if (read) {
System.out.println("read " + data);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
}.start();
}
public static void main(final String[] args) {
if ("producer".equals(args[0]))
startProducer();
else
startConsumer();
}
}
I have modified the code a little to reduce object creation. On the latest version 5.17.1, I can restart the producer many times and the consumer keeps reading data.
NOTE: If you are going to write text, the writeText method might be a better choice.
If you want to write something more complex I suggest using Wire or each MethodReader/MethodWriters which allow you to make interface method calls.
package net.openhft.chronicle.queue;
import net.openhft.chronicle.bytes.Bytes;
import net.openhft.chronicle.core.Jvm;
import net.openhft.chronicle.queue.impl.single.SingleChronicleQueueBuilder;
import net.openhft.chronicle.threads.Pauser;
import java.nio.ByteBuffer;
public class IpcTest {
private static final String DIR = "chronicle-test";
public static final void startProducer() {
System.out.println("starting producer...");
ChronicleQueue queue = SingleChronicleQueueBuilder.single(DIR).blockSize(65536).rollCycle(RollCycles.MINUTELY).build();
ExcerptAppender appender = queue.acquireAppender();
Bytes<ByteBuffer> bytes = Bytes.elasticByteBuffer(8192);
ByteBuffer ipcBuffer = bytes.underlyingObject();
for (int i = 0; i < Integer.MAX_VALUE; i++) {
ipcBuffer.clear();
ipcBuffer.put(("data" + i).getBytes());
bytes.readPositionRemaining(0, ipcBuffer.position());
appender.writeBytes(bytes);
Jvm.pause(1);
}
}
public static final void startConsumer() {
System.out.println("starting consumer...");
ChronicleQueue queue = SingleChronicleQueueBuilder.single(DIR).blockSize(65536).rollCycle(RollCycles.MINUTELY).build();
ExcerptTailer tailer = queue.createTailer().toEnd(); // skip to end, don't read old messages
Bytes<ByteBuffer> bytes = Bytes.elasticHeapByteBuffer(8192);
Pauser pauser = Pauser.balanced();
while (true) {
try {
long ipcIndex = tailer.index();
bytes.clear();
boolean read = tailer.readBytes(bytes);
if (read) {
byte[] data = bytes.underlyingObject().array();
int len = (int) bytes.readRemaining();
System.out.println("read " + new String(data, 0, 0, len));
pauser.reset();
} else {
pauser.pause();
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
public static void main(final String[] args) {
if ("producer".equals(args[0]))
startProducer();
else
startConsumer();
}
}
Using MethodReader/MethodWriter
public class IpcTest {
interface Hello {
void hello(String text);
}
private static final String DIR = "chronicle-test";
public static final void startProducer() {
System.out.println("starting producer...");
ChronicleQueue queue = SingleChronicleQueueBuilder.single(DIR).blockSize(65536).rollCycle(RollCycles.MINUTELY).build();
Hello hello = queue.methodWriter(Hello.class);
for (int i = 0; i < Integer.MAX_VALUE; i++) {
hello.hello("data" + i);
Jvm.pause(1);
}
}
public static final void startConsumer() {
System.out.println("starting consumer...");
ChronicleQueue queue = SingleChronicleQueueBuilder.single(DIR).blockSize(65536).rollCycle(RollCycles.MINUTELY).build();
Hello hello = text -> System.out.println("read " + text);
MethodReader reader = queue.createTailer().methodReader(hello);
Pauser pauser = Pauser.balanced();
while (true) {
if (reader.readOne()) {
pauser.reset();
} else {
pauser.pause();
}
}
}
public static void main(final String[] args) {
if ("producer".equals(args[0]))
startProducer();
else
startConsumer();
}
}
You can use a DTO with is AbstractMarshallable to make it efficient to serialize and deserialize.
package net.openhft.chronicle.queue;
import net.openhft.chronicle.bytes.MethodReader;
import net.openhft.chronicle.core.Jvm;
import net.openhft.chronicle.core.pool.ClassAliasPool;
import net.openhft.chronicle.queue.impl.single.SingleChronicleQueueBuilder;
import net.openhft.chronicle.threads.Pauser;
import net.openhft.chronicle.wire.AbstractMarshallable;
public class IpcTest {
static class Hi extends AbstractMarshallable {
String text;
int value;
}
interface Hello {
void hi(Hi hi);
}
private static final String DIR = "chronicle-test";
public static final void startProducer() {
System.out.println("starting producer...");
ChronicleQueue queue = SingleChronicleQueueBuilder.single(DIR).blockSize(65536).rollCycle(RollCycles.MINUTELY).build();
Hello hello = queue.methodWriter(Hello.class);
Hi hi = new Hi();
for (int i = 0; i < Integer.MAX_VALUE; i++) {
hi.text = "data";
hi.value = i;
hello.hi(hi);
Jvm.pause(1);
}
}
public static final void startConsumer() {
System.out.println("starting consumer...");
ChronicleQueue queue = SingleChronicleQueueBuilder.single(DIR).blockSize(65536).rollCycle(RollCycles.MINUTELY).build();
Hello hello = text -> System.out.println("read " + text);
MethodReader reader = queue.createTailer().methodReader(hello);
Pauser pauser = Pauser.balanced();
while (true) {
if (reader.readOne()) {
pauser.reset();
} else {
pauser.pause();
}
}
}
public static void main(final String[] args) {
ClassAliasPool.CLASS_ALIASES.addAlias(Hi.class);
if ("producer".equals(args[0]))
startProducer();
else
startConsumer();
}
}
In this case, the consumer prints
....
read !Hi {
text: data,
value: 3862
}
read !Hi {
text: data,
value: 3863
}
read !Hi {
text: data,
value: 3864
}
....
I submitted some Runnables to an ExecutorService. Inside these Runnables, wait() and notify() are called. The code works with newFixedThreadPool as the ExecutorService. With newWorkStealingPool, the process exits unexpectedly without any error message.
import java.net.URL;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.IOException;
// For regular expressions
import java.util.regex.Matcher;
import java.util.regex.MatchResult;
import java.util.regex.Pattern;
import java.util.*;
import java.util.concurrent.*;
public class TestPipeline {
public static void main(String[] args) {
runAsThreads();
}
private static void runAsThreads() {
final BlockingQueue<String> urls = new OneItemQueue<String>();
final BlockingQueue<Webpage> pages = new OneItemQueue<Webpage>();
final BlockingQueue<Link> refPairs = new OneItemQueue<Link>();
final BlockingQueue<Link> uniqRefPairs = new OneItemQueue<Link>();
final ExecutorService executor = Executors.newWorkStealingPool(6);
// final ExecutorService executor = Executors.newFixedThreadPool(6);
executor.submit(new UrlProducer(urls));
executor.submit(new PageGetter(urls, pages));
executor.submit(new LinkScanner(pages,refPairs));
executor.submit(new Uniquifier<Link>(refPairs,uniqRefPairs));
executor.submit(new LinkPrinter(uniqRefPairs));
}
}
class UrlProducer implements Runnable {
private final BlockingQueue<String> output;
public UrlProducer(BlockingQueue<String> output) {
this.output = output;
}
public void run() {
System.out.println("in producer");
for (int i=0; i<urls.length; i++)
output.put(urls[i]);
}
private static final String[] urls =
{ "http://www.itu.dk", "http://www.di.ku.dk", "http://www.miele.de",
"http://www.microsoft.com", "http://www.cnn.com", "http://www.dr.dk",
"http://www.vg.no", "http://www.tv2.dk", "http://www.google.com",
"http://www.ing.dk", "http://www.dtu.dk", "http://www.bbc.co.uk"
};
}
class PageGetter implements Runnable {
private final BlockingQueue<String> input;
private final BlockingQueue<Webpage> output;
public PageGetter(BlockingQueue<String> input, BlockingQueue<Webpage> output) {
this.input = input;
this.output = output;
}
public void run() {
while (true) {
System.out.println("in pagegetter");
String url = input.take();
// System.out.println("PageGetter: " + url);
try {
String contents = getPage(url, 200);
output.put(new Webpage(url, contents));
} catch (IOException exn) { System.out.println(exn); }
}
}
public static String getPage(String url, int maxLines) throws IOException {
// This will close the streams after use (JLS 8 para 14.20.3):
try (BufferedReader in
= new BufferedReader(new InputStreamReader(new URL(url).openStream()))) {
StringBuilder sb = new StringBuilder();
for (int i=0; i<maxLines; i++) {
String inputLine = in.readLine();
if (inputLine == null)
break;
else
sb.append(inputLine).append("\n");
}
return sb.toString();
}
}
}
class Uniquifier<T> implements Runnable{
private final Set<T> set = new HashSet<T>();
private final BlockingQueue<T> input;
private final BlockingQueue<T> output;
public Uniquifier(BlockingQueue<T> input, BlockingQueue<T> output){
this.input = input;
this.output = output;
}
public void run(){
while(true){
System.out.println("in uniquifier");
T item = input.take();
if(!set.contains(item)){
set.add(item);
output.put(item);
}
}
}
}
class LinkScanner implements Runnable {
private final BlockingQueue<Webpage> input;
private final BlockingQueue<Link> output;
public LinkScanner(BlockingQueue<Webpage> input,
BlockingQueue<Link> output) {
this.input = input;
this.output = output;
}
private final static Pattern urlPattern
= Pattern.compile("a href=\"(\\p{Graph}*)\"");
public void run() {
while (true) {
System.out.println("in link scanner");
Webpage page = input.take();
// System.out.println("LinkScanner: " + page.url);
// Extract links from the page's <a href="..."> anchors
Matcher urlMatcher = urlPattern.matcher(page.contents);
while (urlMatcher.find()) {
String link = urlMatcher.group(1);
output.put(new Link(page.url, link));
}
}
}
}
class LinkPrinter implements Runnable {
private final BlockingQueue<Link> input;
public LinkPrinter(BlockingQueue<Link> input) {
this.input = input;
}
public void run() {
while (true) {
System.out.println("in link printer");
Link link = input.take();
// System.out.println("LinkPrinter: " + link.from);
System.out.printf("%s links to %s%n", link.from, link.to);
}
}
}
class Webpage {
public final String url, contents;
public Webpage(String url, String contents) {
this.url = url;
this.contents = contents;
}
}
class Link {
public final String from, to;
public Link(String from, String to) {
this.from = from;
this.to = to;
}
// Override hashCode and equals so can be used in HashSet<Link>
public int hashCode() {
return (from == null ? 0 : from.hashCode()) * 37
+ (to == null ? 0 : to.hashCode());
}
public boolean equals(Object obj) {
Link that = obj instanceof Link ? (Link)obj : null;
return that != null
&& (from == null ? that.from == null : from.equals(that.from))
&& (to == null ? that.to == null : to.equals(that.to));
}
}
// Different from java.util.concurrent.BlockingQueue: Allows null
// items, and methods do not throw InterruptedException.
interface BlockingQueue<T> {
void put(T item);
T take();
}
class OneItemQueue<T> implements BlockingQueue<T> {
private T item;
private boolean full = false;
public void put(T item) {
synchronized (this) {
while (full) {
try { this.wait(); }
catch (InterruptedException exn) { }
}
full = true;
this.item = item;
this.notifyAll();
}
}
public T take() {
synchronized (this) {
while (!full) {
try { this.wait(); }
catch (InterruptedException exn) { }
}
full = false;
this.notifyAll();
return item;
}
}
}
Because the Pool is allocating threads dynamically, there are no threads alive after runAsThreads exits because that's the end of the main thread. There needs to be at least on thread running to keep the application alive. Adding a call to awaitTermination is needed. It's not needed for the fixed pool because that will always have active threads until it is explicitly shut down as noted in the JavaDocs.
I have the problem regarding the implementation of One Publisher - Multiple Subscribers pattern. The Publisher uses the fixed-size buffer and queue the messages. The messages are send to all subscribers. The ordering of messages get by subscribers must be the same as the ordering of publishing messages.
I use BlockingQueue to hold publisher messages (publisherQueue) and pass them to each subscriber BlockingQueue (subscriberQueue).
The issue is that the buffer and subscribers are working correctly, but the buffer size (publisherQueue.size()) always returns 1.
System.out.println("Actual number of messages in buffer: " + publisherQueue.size());
Here is my full code:
PublisherSubscriberService.java
package program;
import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
public class PublisherSubscriberService {
private int buffer;
private int subscribersNumber;
static Set<subscriber> subscribers = new HashSet<subscriber>();
public PublisherSubscriberService(int buffer, int subscribersNumber) {
this.buffer = buffer;
this.subscribersNumber = subscribersNumber;
}
public void addsubscriber(subscriber subscriber) {
subscribers.add(subscriber);
}
public void start() {
publisher publisher = new publisher(buffer);
System.out.println("publisher started the job");
for (int i = 0; i < subscribersNumber; i++) {
subscriber subscriber = new subscriber(buffer);
subscriber.setName(Integer.toString(i + 1));
subscribers.add(subscriber);
new Thread(subscriber).start();
System.out.println("Subscriber " + subscriber.getName() + " started the job");
}
new Thread(publisher).start();
}
public class Publisher implements Runnable {
private int buffer;
final BlockingQueue<Message> publisherQueue;
public Publisher(int buffer) {
this.buffer = buffer;
publisherQueue = new LinkedBlockingQueue<>(buffer);
}
#Override
public void run() {
for (int i = 1; i < 100; i++) {
Message messageObject = new Message("" + i);
try {
Thread.sleep(50);
publisherQueue.put(messageObject);
System.out.println("Queued message no " + messageObject.getMessage());
System.out.println("Actual number of messages in buffer: " + publisherQueue.size());
for (subscriber subscriber : subscribers) {
subscriber.subscriberQueue.put(messageObject);
}
publisherQueue.take();
} catch (InterruptedException e) {
System.out.println("Some error");
e.printStackTrace();
}
}
}
}
class Subscriber implements Runnable {
private String name;
private int buffer;
final BlockingQueue<Message> subscriberQueue;
public Subscriber(int buffer) {
this.buffer = buffer;
subscriberQueue = new LinkedBlockingQueue<>(buffer);
}
public void setName(String name) {
this.name = name;
}
public String getName() {
return name;
}
#Override
public void run() {
try {
Message messageObject;
while (true) {
Thread.sleep(100);
messageObject = subscriberQueue.take();
System.out.println(this.getName() + " got message: " + messageObject.getMessage());
}
} catch (InterruptedException e) {
System.out.println("Some error");
e.printStackTrace();
}
}
}
class Message {
private String message;
public Message(String str) {
this.message = str;
}
public String getMessage() {
return message;
}
}
}
PublisherSubscriberProgram.java
package program;
public class ProducerConsumerProgram {
public static void main(String[] args) {
ProducerConsumerService service = new ProducerConsumerService(10, 3);
service.start();
}
}
Your publisher never has more than 1 item in the queue. Each time through your loop you put and take a single item:
**publisherQueue.put(messageObject);**
System.out.println("Queued message no " + messageObject.getMessage());
System.out.println("Actual number of messages in buffer: " + publisherQueue.size());
for (subscriber subscriber : subscribers) {
subscriber.subscriberQueue.put(messageObject);
}
**publisherQueue.take();**
With the code you have provided, there is point in even having the publisher queue.