We Keep Coding

Java Thread Synchronization problem in same resource

I am trying to operate on the same source with two threads. I designed a typical producer and consumer problem for it. While setting the value in the resource class with the producer, I want to get setted values with the consumer one by one. The output I want should be like this:
Producer -> Setting data = 0
Consumer -> Getting data = 0
Producer -> Setting data = 1
Consumer -> Getting data = 1
Producer -> Setting data = 2
Consumer -> Getting data = 2
Producer -> Setting data = 3
Consumer -> Getting data = 3
Producer -> Setting data = 4
Consumer -> Getting data = 4
Here is my Resource class:
public class Resource{
private int value;
private boolean current = false;
public synchronized void setValue(int val) {
while(current == true) {
try {
wait();
}catch(Exception ex) {}}
value = val;
current = true;
notifyAll();
}
public synchronized int getValue() {
while(current == false) {
try {
wait();
}catch(Exception ex) {}}
current = false;
notifyAll();
return value;
}
}
And main method and Producer,Consumer class is here:
class Producer extends Thread{
private Resource rs;
public Producer(Resource rs1) {
rs = rs1;
}
public void run() {
for(int i = 0 ; i < 5 ; i++) {
rs.setValue(i);
System.out.println("Producer -> Setting data = " + i);
try {
sleep(100);
}catch(Exception ex){
ex.printStackTrace();
}
}
}
}
class Consumer extends Thread{
private Resource rs;
public Consumer(Resource rs1) {
rs = rs1;
}
public void run() {
int value = 0;
for(int i = 0 ; i < 5; i++) {
value = rs.getValue();
System.out.println("Consumer -> Getting data= " + i);
try {
sleep(100);
}catch(Exception ex) {
ex.printStackTrace();
}
}
}
}
public class Dependent {
public static void main(String[] args) throws IOException {
Resource res = new Resource();
Producer p1 = new Producer(res);
Consumer c1 = new Consumer(res);
p1.start();
c1.start();
}
}
Although I use synchronized, wait and notifyAll keywords in the methods in the resource class, the threads continue to work without waiting for each other. Where am I making a mistake? I've seen a code sample similar to this code sample in a java book, there doesn't seem to be a problem.
When I write without adding the current boolean variable, the code doesn't even work. That's why I had to add it by looking from the book. Don't the threads need to work synchronously without checking the Current value?

They do wait for each other, but the thread sync operations are much, much faster than Thread.sleep(100) so you can't tell. Your test code prints 'i' and not 'value', which is suspect. Get rid of Thread.sleep(100) in one of these threads (for example, in the consumer) and you'll find that the consumer nevertheless still requires about half a second to complete - as it will be waiting about 100 msec every time it invokes .getValue() on the resource, because that call will block (stuck in that wait() loop) until the producer calls .setValue which it only does about once every 100 msec.
Your Resource object 'works', for some value of 'works', but is very poorly designed, re-creating already existing and better implemented classes from the core library such as a java.util.concurrent.Latch, and which ignore interrupts and will blindly just keep waiting.
Their APIs are also a tad oddly named, in that a get call has considerably side effects. get is more of a get and clear operation: After a get operation, another get operation will freeze the thread forever, or at least, until some thread sets a value.

How do you think?
import java.io.IOException;
class Resource {
private volatile Integer value;
public synchronized void setValue(int val) {
while(value != null && !value.equals(val)) {
try {
wait();
}catch(Exception ex) {}}
value = val;
notifyAll();
}
public synchronized int getValue() {
while(value == null) {
try {
wait();
}catch(Exception ex) {}}
int answer = value;
value = null;
notifyAll();
return answer;
}
}
class Producer extends Thread{
private Resource rs;
public Producer(Resource rs1) {
rs = rs1;
}
public void run() {
for(int i = 0 ; i < 5 ; i++) {
rs.setValue(i);
System.out.println("Producer -> Setting data = " + i);
try {
sleep(100);
}catch(Exception ex){
ex.printStackTrace();
}
}
}
}
class Consumer extends Thread{
private Resource rs;
public Consumer(Resource rs1) {
rs = rs1;
}
public void run() {
for(int i = 0 ; i < 5; i++) {
int value = rs.getValue();
System.out.println("Consumer -> Getting data= " + value);
try {
sleep(100);
}catch(Exception ex) {
ex.printStackTrace();
}
}
}
}
public class Dependent {
public static void main(String[] args) throws IOException {
Resource res = new Resource();
Producer p1 = new Producer(res);
Consumer c1 = new Consumer(res);
p1.start();
c1.start();
}
}
or
class Resource {
private static final int WAIT_VALUE = -1;
private volatile int value = WAIT_VALUE;
public synchronized void setValue(int val) {
while(value > WAIT_VALUE && value != val) {
try {
wait();
}catch(Exception ex) {}}
value = val;
notifyAll();
}
public synchronized int getValue() {
while(value == WAIT_VALUE) {
try {
wait();
}catch(Exception ex) {}}
int answer = value;
value = WAIT_VALUE;
notifyAll();
return answer;
}
}

Why does multithreaded version take the same amount of time as single threaded version?

I have the following work queue implementation, which I use to limit the number of threads in use. It works by me initially adding a number of Runnable objects to the queue, and when I am ready to begin, I run "begin()". At this point I do not add any more to the queue.
public class WorkQueue {
private final int nThreads;
private final PoolWorker[] threads;
private final LinkedList queue;
Integer runCounter;
boolean hasBegun;
public WorkQueue(int nThreads) {
runCounter = 0;
this.nThreads = nThreads;
queue = new LinkedList();
threads = new PoolWorker[nThreads];
hasBegun = false;
for (int i = 0; i < nThreads; i++) {
threads[i] = new PoolWorker();
threads[i].start();
}
}
public boolean isQueueEmpty() {
synchronized (queue) {
if (queue.isEmpty() && runCounter == 0) {
return true;
} else {
return false;
}
}
}
public void begin() {
hasBegun = true;
synchronized (queue) {
queue.notify();
}
}
public void add(Runnable r) {
if (!hasBegun) {
synchronized (queue) {
queue.addLast(r);
runCounter++;
}
} else {
System.out.println("has begun executing. Cannot add more jobs ");
}
}
private class PoolWorker extends Thread {
public void run() {
Runnable r;
while (true) {
synchronized (queue) {
while (queue.isEmpty()) {
try {
queue.wait();
} catch (InterruptedException ignored) {
}
}
r = (Runnable) queue.removeFirst();
}
// If we don't catch RuntimeException,
// the pool could leak threads
try {
r.run();
synchronized (runCounter) {
runCounter--;
}
} catch (RuntimeException e) {
// You might want to log something here
}
}
}
}
}
This is a runnable I use to keep track of when all the jobs on the work queue have finished:
public class QueueWatcher implements Runnable {
private Thread t;
private String threadName;
private WorkQueue wq;
public QueueWatcher(WorkQueue wq) {
this.threadName = "QueueWatcher";
this.wq = wq;
}
#Override
public void run() {
while (true) {
if (wq.isQueueEmpty()) {
java.util.Date date = new java.util.Date();
System.out.println("Finishing and quiting at:" + date.toString());
System.exit(0);
break;
} else {
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
Logger.getLogger(PlaneGenerator.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
public void start() {
wq.begin();
System.out.println("Starting " + threadName);
if (t == null) {
t = new Thread(this, threadName);
t.setDaemon(false);
t.start();
}
}
}
This is how I use them:
Workqueue wq = new WorkQueue(9); //Get same results regardless of 1,2,3,8,9
QueueWatcher qw = new QueueWatcher(wq);
SomeRunnable1 sm1 = new SomeRunnable1();
SomeRunnable2 sm2 = new SomeRunnable2();
SomeRunnable3 sm3 = new SomeRunnable3();
SomeRunnable4 sm4 = new SomeRunnable4();
SomeRunnable5 sm5 = new SomeRunnable5();
wq.add(sm1);
wq.add(sm2);
wq.add(sm3);
wq.add(sm4);
wq.add(sm5);
qw.start();
But regardless of how many threads I use, the result is always the same - it always takes about 1m 10seconds to complete. This is about the same as when I just did a single threaded version (when everything ran in main()).
If I set wq to (1,2,3--9) threads it is always between 1m8s-1m10s. What is the problem ? The jobs (someRunnable) have nothing to do with each other and cannot block each other.
EDIT: Each of the runnables just read some image files from the filesystems and create new files in a separate directory. The new directory eventually contains about 400 output files.
EDIT: It seems that only one thread is always doing work. I made the following changes:
I let the Woolworker store an Id
PoolWorker(int id){
this.threadId = id;
}
Before running I print the id of the worker.
System.out.println(this.threadId + " got new task");
r.run();
In WorkQueue constructor when creating the poolworkers I do:
for (int i = 0; i < nThreads; i++) {
threads[i] = new PoolWorker(i);
threads[i].start();
}
But it seems that that only thread 0 does any work, as the output is always:
0 got new task

Use queue.notifyAll() to start processing.
Currently you're using queue.notify(), which will only wake a single thread. (The big clue that pointed me to this was when you mentioned only a single thread was running.)
Also, synchronizing on Integer runCounter isn't doing what you think it's doing - runCounter++ is actually assigning a new value to the Integer each time, so you're synchronizing on a lot of different Integer objects.
On a side note, using raw threads and wait/notify paradigms is complicated and error-prone even for the best programmers - it's why Java introduced the java.util.concurrent package, which provide threadsafe BlockingQueue implementations and Executors for easily managing multithreaded apps.

How to generate PDF documents from rpt in a multi-threaded approach?

I have a rpt file, using which i will be generating multiple reports in pdf format. Using the Engine class from inet clear reports. The process takes very long as I have nearly 10000 reports to be generated. Can I use the Mutli-thread or some other approach to speed up the process?
Any help of how it can be done would be helpful
My partial code.
//Loops
Engine eng = new Engine(Engine.EXPORT_PDF);
eng.setReportFile(rpt); //rpt is the report name
if (cn.isClosed() || cn == null ) {
cn = ds.getConnection();
}
eng.setConnection(cn);
System.out.println(" After set connection");
eng.setPrompt(data[i], 0);
ReportProperties repprop = eng.getReportProperties();
repprop.setPaperOrient(ReportProperties.DEFAULT_PAPER_ORIENTATION, ReportProperties.PAPER_FANFOLD_US);
eng.execute();
System.out.println(" After excecute");
try {
PDFExportThread pdfExporter = new PDFExportThread(eng, sFileName, sFilePath);
pdfExporter.execute();
} catch (Exception e) {
e.printStackTrace();
}
PDFExportThread execute
public void execute() throws IOException {
FileOutputStream fos = null;
try {
String FileName = sFileName + "_" + (eng.getPageCount() - 1);
File file = new File(sFilePath + FileName + ".pdf");
if (!file.getParentFile().exists()) {
file.getParentFile().mkdirs();
}
if (!file.exists()) {
file.createNewFile();
}
fos = new FileOutputStream(file);
for (int k = 1; k <= eng.getPageCount(); k++) {
fos.write(eng.getPageData(k));
}
fos.flush();
fos.close();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (fos != null) {
fos.close();
fos = null;
}
}
}

This is a very basic code. A ThreadPoolExecutor with a fixed size threads in a pool is the backbone.
Some considerations:
The thread pool size should be equal or less than the DB connection pool size. And, it should be of an optimal number which is reasonable for parallel Engines.
The main thread should wait for sufficient time before killing all threads. I have put 1 hour as the wait time, but that's just an example.
You'll need to have proper Exception handling.
From the API doc, I saw stopAll and shutdown methods from the Engine class. So, I'm invoking that as soon as our work is done. That's again, just an example.
Hope this helps.
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.sql.Connection;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public class RunEngine {
public static void main(String[] args) throws Exception {
final String rpt = "/tmp/rpt/input/rpt-1.rpt";
final String sFilePath = "/tmp/rpt/output/";
final String sFileName = "pdfreport";
final Object[] data = new Object[10];
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(10);
for (int i = 0; i < data.length; i++) {
PDFExporterRunnable runnable = new PDFExporterRunnable(rpt, data[i], sFilePath, sFileName, i);
executor.execute(runnable);
}
executor.shutdown();
executor.awaitTermination(1L, TimeUnit.HOURS);
Engine.stopAll();
Engine.shutdown();
}
private static class PDFExporterRunnable implements Runnable {
private final String rpt;
private final Object data;
private final String sFilePath;
private final String sFileName;
private final int runIndex;
public PDFExporterRunnable(String rpt, Object data, String sFilePath,
String sFileName, int runIndex) {
this.rpt = rpt;
this.data = data;
this.sFilePath = sFilePath;
this.sFileName = sFileName;
this.runIndex = runIndex;
}
#Override
public void run() {
// Loops
Engine eng = new Engine(Engine.EXPORT_PDF);
eng.setReportFile(rpt); // rpt is the report name
Connection cn = null;
/*
* DB connection related code. Check and use.
*/
//if (cn.isClosed() || cn == null) {
//cn = ds.getConnection();
//}
eng.setConnection(cn);
System.out.println(" After set connection");
eng.setPrompt(data, 0);
ReportProperties repprop = eng.getReportProperties();
repprop.setPaperOrient(ReportProperties.DEFAULT_PAPER_ORIENTATION,
ReportProperties.PAPER_FANFOLD_US);
eng.execute();
System.out.println(" After excecute");
FileOutputStream fos = null;
try {
String FileName = sFileName + "_" + runIndex;
File file = new File(sFilePath + FileName + ".pdf");
if (!file.getParentFile().exists()) {
file.getParentFile().mkdirs();
}
if (!file.exists()) {
file.createNewFile();
}
fos = new FileOutputStream(file);
for (int k = 1; k <= eng.getPageCount(); k++) {
fos.write(eng.getPageData(k));
}
fos.flush();
fos.close();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
fos = null;
}
}
}
}
/*
* Dummy classes to avoid compilation errors.
*/
private static class ReportProperties {
public static final String PAPER_FANFOLD_US = null;
public static final String DEFAULT_PAPER_ORIENTATION = null;
public void setPaperOrient(String defaultPaperOrientation, String paperFanfoldUs) {
}
}
private static class Engine {
public static final int EXPORT_PDF = 1;
public Engine(int exportType) {
}
public static void shutdown() {
}
public static void stopAll() {
}
public void setPrompt(Object singleData, int i) {
}
public byte[] getPageData(int k) {
return null;
}
public int getPageCount() {
return 0;
}
public void execute() {
}
public ReportProperties getReportProperties() {
return null;
}
public void setConnection(Connection cn) {
}
public void setReportFile(String reportFile) {
}
}
}

I will offer this "answer" as a possible quick & dirty solution to get you started on a parallelization effort.
One way or another you're going to build a render farm.
I don't think there is a trivial way to do this in java; I would love to have someone post an answer that show how to parallelize your example in just a few lines of code. But until that happens this will hopefully help you make some progress.
You're going to have limited scaling in the same JVM instance.
But... let's see how far you get with that and see if it helps enough.
Design challenge #1: restarting.
You will probably want a place to keep the status for each of your reports e.g. "units of work".
You want this in case you need to re-start everything (maybe your server crashes) and you don't want to re-run all of the reports thus far.
Lots of ways you can do this; database, check to see if a "completed" file exists in your report folder (not sufficient for the *.pdf to exist, as that may be incomplete... for xyz_200.pdf you could maybe make an empty xyz_200.done or xyz_200.err file to help with re-running any problem children... and by the time you code up that file manipulation/checking/initialization logic, seems like it may have been easier to add a column to your database which holds the list of work to-be-done).
Design consideration #2: maximizing throughput (avoiding overload).
You don't want to saturate you system and run one thousand reports in parallel.
Maybe 10.
Maybe 100.
Probably not 5,000.
You will need to do some sizing research and see what gets you near 80 to 90% system utilization.
Design consideration #3: scaling across multiple servers
Overly complex, outside the scope of a Stack Exchange answer.
You'd have to spin up JVM's on multiple systems that are running something like the workers below, and a report-manager that can pull work items from a shared "queue" structure, again a database table is probably easier here than doing something file-based (or a network feed).
Sample Code
Caution: None of this code is well tested, it almost certainly has an abundance of typos, logic errors and poor design. Use at your own risk.
So anyway... I do want to give you the basic idea of a rudimentary task runner.
Replace your "// Loops" example in the question with code like the following:
main loop (original code example)
This is more or less doing what your example code did, modified to push most of the work into ReportWorker (new class, see below). Lots of stuff seems to be packed into your original question's example of "// Loop", so I'm not trying to reverse engineer that.
fwiw, it was unclear to me where "rpt" and "data[i]" are coming from so I hacked up some test data.
public class Main {
public static boolean complete( String data ) {
return false; // for testing nothing is complete.
}
public static void main(String args[] ) {
String data[] = new String[] {
"A",
"B",
"C",
"D",
"E" };
String rpt = "xyz";
// Loop
ReportManager reportMgr = new ReportManager(); // a new helper class (see below), it assigns/monitors work.
long startTime = System.currentTimeMillis();
for( int i = 0; i < data.length; ++i ) {
// complete is something you should write that knows if a report "unit of work"
// finished successfully.
if( !complete( data[i] ) ) {
reportMgr.assignWork( rpt, data[i] ); // so... where did values for your "rpt" variable come from?
}
}
reportMgr.waitForWorkToFinish(); // out of new work to assign, let's wait until everything in-flight complete.
long endTime = System.currentTimeMillis();
System.out.println("Done. Elapsed time = " + (endTime - startTime)/1000 +" seconds.");
}
}
ReportManager
This class is not thread safe, just have your original loop keep calling assignWork() until you're out of reports to assign then keep calling it until all work is done, e.g. waitForWorkToFinish(), as shown above. (fwiw, I don't think you could say any of the classes here are especially thread safe).
public class ReportManager {
public int polling_delay = 500; // wait 0.5 seconds for testing.
//public int polling_delay = 60 * 1000; // wait 1 minute.
// not high throughput millions of reports / second, we'll run at a slower tempo.
public int nWorkers = 3; // just 3 for testing.
public int assignedCnt = 0;
public ReportWorker workers[];
public ReportManager() {
// initialize our manager.
workers = new ReportWorker[ nWorkers ];
for( int i = 0; i < nWorkers; ++i ) {
workers[i] = new ReportWorker( i );
System.out.println("Created worker #"+i);
}
}
private ReportWorker handleWorkerError( int i ) {
// something went wrong, update our "report" status as one of the reports failed.
System.out.println("handlerWokerError(): failure in "+workers[i]+", resetting worker.");
workers[i].teardown();
workers[i] = new ReportWorker( i ); // just replace everything.
return workers[i]; // the new worker will, incidentally, be avaialble.
}
private ReportWorker handleWorkerComplete( int i ) {
// this unit of work was completed, update our "report" status tracker as success.
System.out.println("handleWorkerComplete(): success in "+workers[i]+", resetting worker.");
workers[i].teardown();
workers[i] = new ReportWorker( i ); // just replace everything.
return workers[i]; // the new worker will, incidentally, be avaialble.
}
private int activeWorkerCount() {
int activeCnt = 0;
for( int i = 0; i < nWorkers; ++i ) {
ReportWorker worker = workers[i];
System.out.println("activeWorkerCount() i="+i+", checking worker="+worker);
if( worker.hasError() ) {
worker = handleWorkerError( i );
}
if( worker.isComplete() ) {
worker = handleWorkerComplete( i );
}
if( worker.isInitialized() || worker.isRunning() ) {
++activeCnt;
}
}
System.out.println("activeWorkerCount() activeCnt="+activeCnt);
return activeCnt;
}
private ReportWorker getAvailableWorker() {
// check each worker to see if anybody recently completed...
// This (rather lazily) creates completely new ReportWorker instances.
// You might want to try pooling (salvaging and reinitializing them)
// to see if that helps your performance.
System.out.println("\n-----");
ReportWorker firstAvailable = null;
for( int i = 0; i < nWorkers; ++i ) {
ReportWorker worker = workers[i];
System.out.println("getAvailableWorker(): i="+i+" worker="+worker);
if( worker.hasError() ) {
worker = handleWorkerError( i );
}
if( worker.isComplete() ) {
worker = handleWorkerComplete( i );
}
if( worker.isAvailable() && firstAvailable==null ) {
System.out.println("Apparently worker "+worker+" is 'available'");
firstAvailable = worker;
System.out.println("getAvailableWorker(): i="+i+" now firstAvailable = "+firstAvailable);
}
}
return firstAvailable; // May (or may not) be null.
}
public void assignWork( String rpt, String data ) {
ReportWorker worker = getAvailableWorker();
while( worker == null ) {
System.out.println("assignWork: No workers available, sleeping for "+polling_delay);
try { Thread.sleep( polling_delay ); }
catch( InterruptedException e ) { System.out.println("assignWork: sleep interrupted, ignoring exception "+e); }
// any workers avaialble now?
worker = getAvailableWorker();
}
++assignedCnt;
worker.initialize( rpt, data ); // or whatever else you need.
System.out.println("assignment #"+assignedCnt+" given to "+worker);
Thread t = new Thread( worker );
t.start( ); // that is pretty much it, let it go.
}
public void waitForWorkToFinish() {
int active = activeWorkerCount();
while( active >= 1 ) {
System.out.println("waitForWorkToFinish(): #active workers="+active+", waiting...");
// wait a minute....
try { Thread.sleep( polling_delay ); }
catch( InterruptedException e ) { System.out.println("assignWork: sleep interrupted, ignoring exception "+e); }
active = activeWorkerCount();
}
}
}
ReportWorker
public class ReportWorker implements Runnable {
int test_delay = 10*1000; //sleep for 10 seconds.
// (actual code would be generating PDF output)
public enum StatusCodes { UNINITIALIZED,
INITIALIZED,
RUNNING,
COMPLETE,
ERROR };
int id = -1;
StatusCodes status = StatusCodes.UNINITIALIZED;
boolean initialized = false;
public String rpt = "";
public String data = "";
//Engine eng;
//PDFExportThread pdfExporter;
//DataSource_type cn;
public boolean isInitialized() { return initialized; }
public boolean isAvailable() { return status == StatusCodes.UNINITIALIZED; }
public boolean isRunning() { return status == StatusCodes.RUNNING; }
public boolean isComplete() { return status == StatusCodes.COMPLETE; }
public boolean hasError() { return status == StatusCodes.ERROR; }
public ReportWorker( int id ) {
this.id = id;
}
public String toString( ) {
return "ReportWorker."+id+"("+status+")/"+rpt+"/"+data;
}
// the example code doesn't make clear if there is a relationship between rpt & data[i].
public void initialize( String rpt, String data /* data[i] in original code */ ) {
try {
this.rpt = rpt;
this.data = data;
/* uncomment this part where you have the various classes availble.
* I have it commented out for testing.
cn = ds.getConnection();
Engine eng = new Engine(Engine.EXPORT_PDF);
eng.setReportFile(rpt); //rpt is the report name
eng.setConnection(cn);
eng.setPrompt(data, 0);
ReportProperties repprop = eng.getReportProperties();
repprop.setPaperOrient(ReportProperties.DEFAULT_PAPER_ORIENTATION, ReportProperties.PAPER_FANFOLD_US);
*/
status = StatusCodes.INITIALIZED;
initialized = true; // want this true even if we're running.
} catch( Exception e ) {
status = StatusCodes.ERROR;
throw new RuntimeException("initialze(rpt="+rpt+", data="+data+")", e);
}
}
public void run() {
status = StatusCodes.RUNNING;
System.out.println("run().BEGIN: "+this);
try {
// delay for testing.
try { Thread.sleep( test_delay ); }
catch( InterruptedException e ) { System.out.println(this+".run(): test interrupted, ignoring "+e); }
/* uncomment this part where you have the various classes availble.
* I have it commented out for testing.
eng.execute();
PDFExportThread pdfExporter = new PDFExportThread(eng, sFileName, sFilePath);
pdfExporter.execute();
*/
status = StatusCodes.COMPLETE;
System.out.println("run().END: "+this);
} catch( Exception e ) {
System.out.println("run().ERROR: "+this);
status = StatusCodes.ERROR;
throw new RuntimeException("run(rpt="+rpt+", data="+data+")", e);
}
}
public void teardown() {
if( ! isInitialized() || isRunning() ) {
System.out.println("Warning: ReportWorker.teardown() called but I am uninitailzied or running.");
// should never happen, fatal enough to throw an exception?
}
/* commented out for testing.
try { cn.close(); }
catch( Exception e ) { System.out.println("Warning: ReportWorker.teardown() ignoring error on connection close: "+e); }
cn = null;
*/
// any need to close things on eng?
// any need to close things on pdfExporter?
}
}

Producer–consumer pattern deisgn issue

I'm working on implements a tree that represent electric circus (without any circles, as in this picture)
I use this implementation:
Binary_Oprtator
public abstract class Binary_Oprtator {
abstract int calc(int x, int y);
#Override
public String toString() {
return super.toString().substring(0, super.toString().indexOf('#'));
}
}
And gate
public class and extends Binary_Oprtator {
public int calc(int x, int y){
return (x&y);
}
}
Or gate
public class or extends Binary_Oprtator {
public int calc(int x, int y){
return (x|y);
}
}
gate_node
public class gate_node {
gate_node father_c;
gate_node right_c, left_c;
Binary_Oprtator op;
int value;
int right_v, left_v;
int array_index;
int arr_size;
boolean leaf;
boolean isRightChild;
public gate_node(Binary_Oprtator op, int array_index, int arr_size, boolean right) {
this.array_index = array_index;
this.arr_size = arr_size;
this.left_c = null;
this.right_c = null;
this.op = op;
right_v = left_v = -1;
this.leaf = false;
this.isRightChild = right;
}
void set_left_son(Binary_Oprtator op) {
this.left_c = new gate_node(op, array_index, arr_size / 2,false);
this.left_c.father_c = this;
this.left_c.leaf = false;
this.left_c.isRightChild = false;
}
void set_right_son(Binary_Oprtator op) {
this.right_c = new gate_node(op, array_index + arr_size / 2,
arr_size / 2,true);
this.right_c.father_c = this;
this.right_c.leaf = false;
this.right_c.isRightChild = true;
}
void set_left_son_as_leaf(Binary_Oprtator op) throws InterruptedException {
this.left_c = new gate_node(op, array_index, arr_size / 2,false);
this.left_c.father_c = this;
this.left_c.leaf = true;
this.left_c.left_v = main_class.arr[array_index];
this.left_c.right_v = main_class.arr[array_index + 1];
this.left_c.isRightChild = false;
main_class.queue.put(this.left_c);
}
void set_right_son_as_leaf(Binary_Oprtator op) throws InterruptedException {
this.right_c = new gate_node(op, array_index + arr_size / 2,
arr_size / 2,true);
this.right_c.father_c = this;
this.right_c.left_v = main_class.arr[array_index + 2];
this.right_c.right_v = main_class.arr[array_index + 3];
this.right_c.leaf = true;
this.right_c.isRightChild = true;
main_class.queue.put(this.right_c);
}
gate_node get_left() {
return this.left_c;
}
gate_node get_right() {
return this.right_c;
}
int compute() {
/*
* The following use of a static sInputCounter assumes that the
* static/global input array is ordered from left to right, irrespective
* of "depth".
*/
final int left, right;
if (this.left_c.leaf != true) {
left = this.left_c.compute();
} else {
left = this.left_c.op.calc(this.left_c.left_v, this.left_c.right_v);
}
if (this.right_c.leaf != true) {
right = this.right_c.compute();
} else {
right = this.right_c.op.calc(this.right_c.left_v,
this.right_c.right_v);
}
return op.calc(left, right);
}
int compute_with_print() {
/*
* The following use of a static sInputCounter assumes that the
* static/global input array is ordered from left to right, irrespective
* of "depth".
*/
final int left, right;
System.out.print(this.op + "(");
if (null != this.left_c) {
left = this.left_c.compute_with_print();
System.out.print(",");
} else {
left = main_class.arr[array_index];
System.out.print(left + ",");
}
if (null != this.right_c) {
right = this.right_c.compute_with_print();
System.out.print(")");
} else {
right = main_class.arr[array_index + 1];
System.out.print(right + ")");
}
return op.calc(left, right);
}
}
tree
public class tree {
gate_node head;
public tree(Binary_Oprtator op,int array_index,int arr_size) {
this.head = new gate_node(op,array_index,arr_size,true);
head.father_c=null;
}
void calc_head_value(){
int t_value = head.op.calc(head.left_v,head.right_v);
/* System.out.println(head.left_v+" "+head.op.toString()+" "+head.right_v+" = "+head.op.calc(head.left_v,head.right_v));
*/ head.value = t_value;
}
int compute() {
return head.compute();
}
int compute_with_print(){
return head.compute_with_print();
}
void set_left_son(Binary_Oprtator op){
head.left_c = new gate_node(op,head.array_index,head.arr_size/2,false);
head.left_c.father_c=head;
}
void set_right_son(Binary_Oprtator op){
head.right_c = new gate_node(op,head.array_index + head.arr_size/2,head.arr_size/2,true);
head.right_c.father_c=head;
}
void set_right_son_as_leaf(Binary_Oprtator op) throws InterruptedException {
head.right_c = new gate_node(op,head.array_index,head.arr_size/2,false);
head.right_c.father_c=head;
head.right_c.father_c = head;
head.right_c.left_v = main_class.arr[head.array_index + 2];
head.right_c.right_v = main_class.arr[head.array_index + 3];
head.right_c.leaf = true;
head.right_c.isRightChild = true;
main_class.queue.put(head.right_c);
}
void set_left_son_as_leaf(Binary_Oprtator op) throws InterruptedException {
head.left_c = new gate_node(op, head.array_index, head.arr_size / 2,false);
head.left_c.father_c = head;
head.left_c.leaf = true;
head.left_c.left_v = main_class.arr[head.array_index];
head.left_c.right_v = main_class.arr[head.array_index + 1];
head.left_c.isRightChild = false;
main_class.queue.put(head.left_c);
}
gate_node get_left(){
return head.left_c;
}
gate_node get_right(){
return head.right_c;
}
}
main_class
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
public class main_class {
public static int arr[] = { 1, 0, 0, 0, 1, 0, 0, 1 };
static final BlockingQueue<gate_node> queue = new ArrayBlockingQueue<>(6);
public static void main(String[] args) throws InterruptedException {
/*************************************
* compute using multi threads
************************************/
System.out.println("compute using Multi threading");
//start a consumer... wait for nodes to be insert into the queue
Consumer consumer = new Consumer();
consumer.start();
tree t = new tree(new and(), 0, arr.length);
t.set_left_son(new or());
t.get_left().set_left_son_as_leaf(new and());
t.get_left().set_right_son_as_leaf(new or());
t.set_right_son(new and());
t.get_right().set_left_son_as_leaf(new or());
t.get_right().set_right_son_as_leaf(new or());
consumer.join();
t.calc_head_value(); //calc the head
System.out.println("The result is: " + t.head.value);
System.out.println();
/******************************
* compute with a single thread
********************************/
System.out.println("compute with a single thread");
int res = t.compute();
System.out.println("The result is: " + res);
/***********************************************
* printing a arithmatic expression of the tree
*************************************************/
System.out.println();
t.compute_with_print();
}
}
Consumer
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class Consumer extends Thread {
Consumer() {
}
#Override
public void run() {
gate_node temp;
// the threads pool parts
ExecutorService executor = Executors.newFixedThreadPool(4);
try {
while ((temp = main_class.queue.take()).father_c != null) {
Runnable worker = new computingThread(temp);
executor.execute(worker);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
executor.shutdown();
while (!executor.isTerminated()) {
}
}
}
computingThread
public class computingThread implements Runnable {
gate_node head;
int t_value;
public computingThread(gate_node head) {
this.head = head;
this.t_value = -1;
}
#Override
public void run() {
/* System.out.println("Start: "+this.hashCode()); */
t_value = head.op.calc(head.left_v,head.right_v);
/* System.out.println("thread: "+this.hashCode()+" is running ==> "+head.left_v+" "+head.op.toString()+" "+head.right_v+" = "+head.op.calc(head.left_v,head.right_v));
*/ head.value = this.t_value;
// update the father
if (head.isRightChild == true) { //update right fathers entire
head.father_c.right_v = t_value;
/*System.out.println("isRightChild");*/
} else { //update left fathers entire
head.father_c.left_v = t_value;
}
if ((head.father_c.right_v != -1) && (head.father_c.left_v != -1)){ //father is ready to compute-> to the queue!
try {
main_class.queue.put(head.father_c);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
/* try {
Thread.sleep(1);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}*/
/* System.out.println("thread: "+this.hashCode()+" is done!");
*/ return;
}
}
Here what I'm trying to do:
I'm trying to do a parllel comput that use multithreads to compute the finite value of the tree (each node gets two values, produce an outcome based on his opreator, pass it on the tree.. until the root is caculted). What I did is to set a queue of fixed number of spaces.
I insert the leafs to the queue as the tree is build. then I start a consumer that takes each leafs, caculate it, pass the result on the the right entrie of his father, and when both entreis are inserted into the father node, it also goes to the queue, and so on.. until the root is cacluted).
the only problem is that I cannot uses a queue that is smaller from the number of leafs in the tree, and I don't know why.
maybe becuase while I'm building the tree I'm inserting the leafs to the tree and if the queue is smaller then the leafs, I'm doing a: main_class.queue.put(this.right_c); when queue is already full, and that cause the progrem to wait until spaces on the queue will be freed which doesnt happen (cause I'm didn't start the threads yet).
Does anyone have any solution to that?
and another question? is that consider a parrlel computation? meaning if I set a queue with size 2, does that mean the I will do all the computation with only two threads (because I want to set is like the number of core CPU of a certain computer).
Thanks and sorry for my bad spelling.

I think you modelled it in a more complicated way than it was needed. I would not base my modelling on a tree. An electric circuit, is not always a tree. You could have more than one nodes acting as the circuits outputs, right?
I would base my modelling on the gate node. I would have a Gate class with two inputs and one output. Inputs and outputs, would be of type GateValue. Output would be calculated using a different way if the gate is an and or an or gate.
Then I would combine them building my circuit, like this:
gate1.Input1 = gate2.Output
gate1.Input2 = gate3.Output
etc.
Then, I would calculate the value of the last gate (output of the whole circuit) which would cause other gates to calculate their values. This way, you would not need a "parallel" calculation mechanism. As soon as you have no feedback loops in your circuit, this would work fine.
Hope I helped!

IllegalMonitorStateException when notifying threads

I have a program that simulates Gates to a ship. They run in threads. The idea is to let them run and pause during a random moment in the run method to simulate persons passing. This is done by all threads, meanwhile the main thread is waiting for notification and checking if the ship is getting full when notified by the threads that they added a person passing through the gate the main thread checks again if the ship is full. The program has three classes:
A counter:
public class Counter {
private int currentValue[];
private int maxValue;
public Counter(int[] nrOfPeople, int max) {
currentValue = nrOfPeople;
currentValue[0] = 0;
maxValue = max;
}
public synchronized void addPersons(int nr_p) {
currentValue[0] += nr_p;
}
public synchronized int getValue() {
return currentValue[0];
}
public synchronized boolean isFull() {
if(currentValue[0] < maxValue)
return false;
return true;
}
}
A Gate Class:
public abstract class Gate implements Runnable {
int nrOfPassengers;
int gatenr;
int gatesize;
Counter c;
private Thread t;
private Random r;
private boolean blocked; /* suspends people from passing */
public Gate(Counter c, int nr) {
this.c = c;
gatenr = nr;
this.open();
r = new Random();
t = new Thread(this);
t.start();
}
public void setGatesize(int size) {
gatesize = size;
}
public void close() {
blocked = true;
}
public void open() {
blocked = false;
}
public int getNoOfPassangers() {
return nrOfPassengers;
}
public int getId() {
return gatenr;
}
#Override
public void run() {
while(!blocked) {
int waitTime = (r.nextInt(5) + 1) * 1000; /* between 1-5 seconds */
System.out.println("Person-Gate " + gatenr + ": adding one to " + c.getValue());
try {
/* bigger throughput => amount can vary */
if(gatesize > 1) {
int persons = r.nextInt(gatesize)+1;
c.addPersons(persons);
nrOfPassengers += persons;
} else {
c.addPersons(1);
nrOfPassengers++;
}
Thread.sleep(waitTime);
} catch (InterruptedException e) {
System.out.println("Person-Gate " + gatenr + ": was interrupted adding person");
e.printStackTrace();
}
System.out.println("Person-Gate " + gatenr + ": added one to " + c.getValue());
t.notify();
}
}
public void join() {
try {
t.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
And a Simulator that runs the main method:
/*
* This class simulates cars and persons- entering a ferry.
*/
public class Simulator {
public static final int MAX = 30;
public static void main(String[] args) {
int nrOfPeople[] = new int[1]; /* array of size one for keeping count */
ArrayList<Gate> gates = new ArrayList<Gate>();
Counter counter = new Counter(nrOfPeople, MAX);
Thread mainThread = Thread.currentThread();
/* adding 3 person-gates */
for(int i=1; i<4; i++) {
gates.add(new PersonGate(counter, i));
}
/* let all gates work as long as passengers is under MAX */
while(!counter.isFull()) {
try {
mainThread.wait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.out.println("Announcement: Ship is full!");
/* wait for child threads to finish */
for(Gate g: gates) {
g.close();
try {
g.join();
} catch (Exception e) { /* InterruptedException */
e.printStackTrace();
}
System.out.println(g.getNoOfPassangers() + " passed through gate nr " + g.getId());
System.out.println(counter.getValue() + " has passed in total");
}
}
}
Im getting a error
Person-Gate 1: adding one to 0
Person-Gate 2: adding one to 1
Person-Gate 3: adding one to 2
Exception in thread "main" java.lang.IllegalMonitorStateException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at Simulator.main(Simulator.java:24)
Person-Gate 3: added one to 3Exception in thread "Thread-3"
Does anyone now whats going on?

You can only call wait and notify/notifyAll from within synchronized blocks.

t.notify();
You are notifying wrong monitor. This exception occurs, when you do not wrap monitor object with synchronize section. However, objects which you are using for notify and for wait methods are different. Create new Object() monitor and pass it to the constructor of Gate.
Also you can take a look at CountDownLatch, it does exactly what you are trying to achieve.

You must own the monitor of the object on which you call wait or notify. Meaning, you must be in a synchonize-Block, like
synchronized( objectUsedAsSynchronizer) {
while ( mustStillWait) {
objectUsedAsSynchronizer.wait();
}
}
This has been the subject of many other questions.