We Keep Coding

Why does multithreaded version take the same amount of time as single threaded version?

I have the following work queue implementation, which I use to limit the number of threads in use. It works by me initially adding a number of Runnable objects to the queue, and when I am ready to begin, I run "begin()". At this point I do not add any more to the queue.
public class WorkQueue {
private final int nThreads;
private final PoolWorker[] threads;
private final LinkedList queue;
Integer runCounter;
boolean hasBegun;
public WorkQueue(int nThreads) {
runCounter = 0;
this.nThreads = nThreads;
queue = new LinkedList();
threads = new PoolWorker[nThreads];
hasBegun = false;
for (int i = 0; i < nThreads; i++) {
threads[i] = new PoolWorker();
threads[i].start();
}
}
public boolean isQueueEmpty() {
synchronized (queue) {
if (queue.isEmpty() && runCounter == 0) {
return true;
} else {
return false;
}
}
}
public void begin() {
hasBegun = true;
synchronized (queue) {
queue.notify();
}
}
public void add(Runnable r) {
if (!hasBegun) {
synchronized (queue) {
queue.addLast(r);
runCounter++;
}
} else {
System.out.println("has begun executing. Cannot add more jobs ");
}
}
private class PoolWorker extends Thread {
public void run() {
Runnable r;
while (true) {
synchronized (queue) {
while (queue.isEmpty()) {
try {
queue.wait();
} catch (InterruptedException ignored) {
}
}
r = (Runnable) queue.removeFirst();
}
// If we don't catch RuntimeException,
// the pool could leak threads
try {
r.run();
synchronized (runCounter) {
runCounter--;
}
} catch (RuntimeException e) {
// You might want to log something here
}
}
}
}
}
This is a runnable I use to keep track of when all the jobs on the work queue have finished:
public class QueueWatcher implements Runnable {
private Thread t;
private String threadName;
private WorkQueue wq;
public QueueWatcher(WorkQueue wq) {
this.threadName = "QueueWatcher";
this.wq = wq;
}
#Override
public void run() {
while (true) {
if (wq.isQueueEmpty()) {
java.util.Date date = new java.util.Date();
System.out.println("Finishing and quiting at:" + date.toString());
System.exit(0);
break;
} else {
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
Logger.getLogger(PlaneGenerator.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
public void start() {
wq.begin();
System.out.println("Starting " + threadName);
if (t == null) {
t = new Thread(this, threadName);
t.setDaemon(false);
t.start();
}
}
}
This is how I use them:
Workqueue wq = new WorkQueue(9); //Get same results regardless of 1,2,3,8,9
QueueWatcher qw = new QueueWatcher(wq);
SomeRunnable1 sm1 = new SomeRunnable1();
SomeRunnable2 sm2 = new SomeRunnable2();
SomeRunnable3 sm3 = new SomeRunnable3();
SomeRunnable4 sm4 = new SomeRunnable4();
SomeRunnable5 sm5 = new SomeRunnable5();
wq.add(sm1);
wq.add(sm2);
wq.add(sm3);
wq.add(sm4);
wq.add(sm5);
qw.start();
But regardless of how many threads I use, the result is always the same - it always takes about 1m 10seconds to complete. This is about the same as when I just did a single threaded version (when everything ran in main()).
If I set wq to (1,2,3--9) threads it is always between 1m8s-1m10s. What is the problem ? The jobs (someRunnable) have nothing to do with each other and cannot block each other.
EDIT: Each of the runnables just read some image files from the filesystems and create new files in a separate directory. The new directory eventually contains about 400 output files.
EDIT: It seems that only one thread is always doing work. I made the following changes:
I let the Woolworker store an Id
PoolWorker(int id){
this.threadId = id;
}
Before running I print the id of the worker.
System.out.println(this.threadId + " got new task");
r.run();
In WorkQueue constructor when creating the poolworkers I do:
for (int i = 0; i < nThreads; i++) {
threads[i] = new PoolWorker(i);
threads[i].start();
}
But it seems that that only thread 0 does any work, as the output is always:
0 got new task

Use queue.notifyAll() to start processing.
Currently you're using queue.notify(), which will only wake a single thread. (The big clue that pointed me to this was when you mentioned only a single thread was running.)
Also, synchronizing on Integer runCounter isn't doing what you think it's doing - runCounter++ is actually assigning a new value to the Integer each time, so you're synchronizing on a lot of different Integer objects.
On a side note, using raw threads and wait/notify paradigms is complicated and error-prone even for the best programmers - it's why Java introduced the java.util.concurrent package, which provide threadsafe BlockingQueue implementations and Executors for easily managing multithreaded apps.

Can't stop ExecutorService in java

I have a problem with ExecutorService. I have a button in GUI that calls Runnable class that contains ExecutorService. I tried everything to stop ExecutorService (or main thread) but I didn't find a way to end it. Here is my code. Please post your answers and advises. Best regards.
public void actionPerformed(ActionEvent e) {
final FindGateWaysAndIps scanner = new FindGateWaysAndIps();
if (e.getActionCommand()=="Start Scan"){
scanner.start();
}
if (e.getActionCommand()=="Stop Scan"){
scanner.interrupt();
scanner.stopScans();
}
}
Class "FindGateWaysAndIps"
String ip = "192.168.";
String sql =" ";
static volatile boolean stop = false;
PingResult AllResaults = new PingResult();
int [] AllGateWays = new int [256];
final int NUM_THREADS = Runtime.getRuntime().availableProcessors();
ExecutorService exec = Executors.newFixedThreadPool(NUM_THREADS*5);
public void run() {
stop=true;
while(stop){
for (;GateWayKey<=GateWayKeyStop;GateWayKey++){
if (!stop){
exec.shutdownNow();
Thread.currentThread().interrupt();
break;
}
ip="192.168."+GateWayKey+".1";
AllSQLs.add(exec.submit((new PingTask(ip,GateWayKey,true))));
}
if (!stop) {
exec.shutdownNow();
Thread.currentThread().interrupt();
break;
}
AllGateWays=GetVectorData.GiveMeGateWays();
for (int j=0; j<= AllGateWays.length;j++){
System.out.println("stop je: "+stop);
if (!stop){
exec.shutdownNow();
Thread.currentThread().interrupt();
break;
}
removeDuplicateinVectors();
//System.out.println("Sada je j"+j);
for (;SubNetKey<=SubNetKeyStop;SubNetKey++){
if (!stop){
exec.shutdownNow();
Thread.currentThread().interrupt();
break;
}
ip="192.168."+AllGateWays[j]+"."+SubNetKey;
AllSQLs.add (exec.submit((new PingTask(ip,AllGateWays[j],false))));
}
// Process the result here (this is where you insert into the DB)
//WriteAllDataIntoDataBase();
}
exec.shutdown();
//WriteAllDataIntoDataBase();
}
public void stopScans(){
exec.shutdownNow();
stop=false;
}
Sorry here is PingTask class
public class PingTask implements Callable <String> {
String ips;
String sql;
PingResult PassDataToExternalClass = new PingResult();
//FindGateWaysAndIps DataProccesor = new FindGateWaysAndIps();
int GateWay;
ScanFrame MonitorData = new ScanFrame();
boolean GateWayORSubNet;
int [] AllGateWays = new int [256];
int i=0;
public int[] GiveMeGateWays(){
return AllGateWays;
}
public PingTask (){
}
public PingTask (String ip, int GateWayKey, boolean GateWayORSubNets){
ips=ip;
GateWay=GateWayKey;
GateWayORSubNet=GateWayORSubNets;
}
public String call(){
InetAddress address;
try {
address = InetAddress.getByName(ips);//ako nade gateway neka skoci u petlju u kojoj nade IP adrese pripadajuceg gatewaya
System.out.println("PINGAM: "+ips);
try {
if (address.isReachable(2000)) { //pinga gatewaya s 1000ms (jeli je moguce ovo smanjiti da se ubrza proces)?
System.out.println("Nasa sam IP: "+ips);
AllGateWays[i]=GateWay;
i++;
MonitorData.WriteMonitorData(ips,address.getHostName().toString(),"2000","da");
if (GateWayORSubNet){
sql="REPLACE INTO `gateways` (`ID_GATEWAY` , `GATEWAY_IP` , `GATEWAY_NAME`) VALUES ('"+GateWay+"', '"+ips+"', '"+address.getHostName().toString()+"');";
return sql;
}
else{
sql="REPLACE INTO `subnets` (`IP` , `COMPUTER_NAME` , `GATEWAY_KEY`) VALUES ('"+ips+"', '"+address.getHostName().toString()+"', '"+GateWay+"');";
return sql;
}
} else {
return ";";
}
} catch (IOException e) {
return ";";
}
} catch (UnknownHostException e) {
return ";";
}
}
}

Effectively, in order to get threads stopped, each thread within a pooler inialized by ExecutorService must defined a treatment when this one get interrupted.
This is why daemon starting by:
while(true){
}
isn't suitable and is one of the cause of the impossibility of shutdowning the thread pooler.
Prefer for instance:
while(!Thread.currentThread.isInterrupted){
}
//do here what to do in order to exit and clean safely your job and used resources like open filed.
But even with this, you make wonder what it may not work ......
Avoid to swallow InterruptedException!:
catch(InterruptedException e){
//do nothing
}
Indeed, the interrupted flag is cleared when exception is catched ! So don't forget to set it to true by reinpterrupting the thread:
catch(InterruptedException e){
Thread.currentThread.interrupt();
}
For a more detailed explanation, open this link:
http://www.ibm.com/developerworks/java/library/j-jtp05236/index.html

I guess your PingTask is making URL Connections and connect() call is non interruptible call.
ExecutorService#shutdownNow() offers a way of immediate shutdown by interrupting threads but since these threads can not be interrupted it provides not additional benefit.
It only provides benefit when thread has defined its interruption policy and it performs interruptible operations.

Your problem may be that address.isReachable(...) is not interruptible. Interrupting the thread, like #Mik378 mentioned, sets the interrupt bit on the thread and causes some methods (Thread.sleep(), Object.wait(), and others) to throw InterruptedException. The InetAddress.isReachable(...) will not get interrupted unfortunately.
If you are trying to have your print threads finish immediately so the application can exit, you could make them daemon threads. See here:
Making Callable Threads as Daemon

How to use SynchronousQueue appropriately in producer-consumer model? [closed]

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I wrote a test example of using SynchronousQueue in producer-consumer model. But it doesn't work well. Below are my codes:
public class QueueTest {
String input;
int pos;
BlockingQueue<String> queue;
volatile boolean exitFlag;
QueueTest()
{
for(int i=0; i<10000; i++)
input += "abcde";
input += "X";
pos = 0;
queue = new SynchronousQueue<String>();
exitFlag = false;
}
public static void main(String[] args) {
QueueTest qtest = new QueueTest();
qtest.runTest();
}
void runTest()
{
Thread producer = new Thread( new Producer());
Thread consumer = new Thread( new Consumer());
producer.start();
consumer.start();
try {
producer.join();
consumer.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
class Producer implements Runnable
{
public void run()
{
while(true)
{
String s = read();
if(s.equals("X"))
break;
try {
queue.put(s);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
exitFlag = true;
}
}
class Consumer implements Runnable
{
public void run()
{
while(exitFlag == false)
{
String s = null;
try {
s = queue.take();
} catch (InterruptedException e) {
e.printStackTrace();
}
process(s);
}
}
}
String read()
{
String str = input.substring(pos, pos+1);
pos++;
return str;
}
void process(String s)
{
long sum = 0;
for(long i=0; i<1000; i++)
sum = sum * i + i;
}
}
The problem is the running is stuck like a deadlock. Is there any bugs in these simple codes?

You are more then likely seeing a race condition. Imagine the scenario
Thread 1 put into queue
Thread 2 takes out of queue quickly processes and awaits another put from thread 1
Thread 1 finishes and sets exitFlag to true
In this case Thread 2 will sit the permanently since exitFlag was not set to false before Thread 2 read from it.
You may want to consider a poison pill. It's a message to the other thread that we have completed. For instance:
final String POISON_PILL = " POISON";
class Producer implements Runnable {
public void run() {
while (true) {
String s = read();
if (s.equals("X"))
break;
try {
queue.put(s);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
try {
queue.put(POISON_PILL);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
class Consumer implements Runnable {
public void run() {
String s = null;
try {
while ((s = queue.take()) != POISON_PILL) {
process(s);
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
So when the other thread is notified the other thread has completed both threads should end gracefully.

Since your exitFlag is shared between multiple threads, you must do something to make the update by the Producer visible to the Consumer (in terms of the java memory model). for this example, making the value volatile would be sufficient.
UPDATE:
You should generate a stack dump of your hung code. That will give you a clue as to what is happening. This code is a good example as to why you shouldn't use a flag for control along with a BlockingQueue.
UPDATE 2:
Bummer, #JohnVint let the cat out of the bag. Yes, the poison pill is the solution to this race condition.

Your program will be stuck in the following scenario:
Producer sets the exitFlag (without putting a new element) just after the Consumer checks whether existFlag is true. If there is no more elements in the queue (Consumer managed to process all the elements before) consumer will be blocked on queue.take().
You can use queue.poll(), which is no blocking method. It would require changing your program a little bit.

How to make a Java thread hang

I am trying to create a solution to treat hung threads due to memory leaks, locked resources in our applications. One of the main problems I am having is trying to simulate a hung thread to deal with it. Any sugestions?
This is what I tried, but it just doesn't seem to do the job. Any thoughts?
class KillerThread extends Thread{
public KillerThread() {
super();
}
public KillerThread(String threadName) {
super(threadName);
}
public void run (){
System.out.println("Start of KillerThread " + this.getName() );
if ( System.currentTimeMillis() % 2L == 0 ){
try {
sleep(Long.MAX_VALUE);
} catch (InterruptedException e) {
e.printStackTrace();
}
} else {
for(;;);
}
}
}

Joining on one's own thread works well for me:
Thread.currentThread().join();

try running sleep in a while loop like:
while(true) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}

running a thread then tell it to sleep in an unstoppable loop, is a good idea,.
but how if you are trying to make it waiting another thread,.? make more than one thread and make them wait one each other, a deadlock condition, is that a hung to,.?

I know what you need exactly, you are testing something through stopping the executor thread. Try something like this:
private void testKillingThread() {
Object kill = new Object();
try {
synchronized (kill) {
kill.wait();
}
} catch (Exception e) {
// Auto-generated catch block
}
}

Simply enough, just create a private member
private Object lock = new Object();
then use it to wait for a notification (that will never happen, unless you use reflection...)
while (true) {
try {
synchronized (lock) {
lock.wait();
}
} cath (InterruptedException e) {
/* ignore interruption */
}
}
and you thread will hang there, uninterruptable.

Here's a quick fix I'm using for testing. Just have the thread you want to lock up call new Hanger().hang().
Remove the logging if you're not interested in seeing it. You can add throws InterruptedException (although, in fact, it never does) to the hang method so you can just replace a Thread.sleep() with a new Hanger().hang() without otherwise modifying your code.
public class Hanger {
private final static Logger log = Logger.getLogger(Hanger.class);
private long started = 0;
private final int beat = 100; // ms
/**
* Hangs a thread for the indicated time
* #param millis the amount of time to hang the thread, in milliseconds
*/
public void hang(int millis) {
started = System.currentTimeMillis();
log.debug("Hanging this thread for " + millis + " ms");
while (hung() < millis) {
try {
Thread.sleep(beat);
} catch (InterruptedException e) {
log.debug("Still hanging, will release in " + (millis - hung()) + " ms.");
}
}
log.debug("Releasing thread again after " + hung() + " ms");
}
private int hung() {
return (int)(System.currentTimeMillis() - started);
}
}

Java - running jobs async using ReentrantLock?

The code below allows us to run a job while ensuring that only one job at a time can run by using ReentrantLock.
Is there any way to modify this code to run job.call() asynchronously and to return the MyConcurrentJobException to the client prior to starting the thread?
We tried wrapping the try/catch/finally block in a new Thread but the unlock and lock have to happen in the same thread so we get an IllegalMonitorException
??
final static Lock lock = new ReentrantLock();
public Object runJob(String desc, Callable job, boolean wait) {
logger.info("Acquiring lock");
if (!lock.tryLock()) {
throw new MyConcurrentJobException();
}
activeJob = new JobStatus(desc);
logger.info("Lock acquired");
try {
return job.call();
} catch (MarginServiceAssertionException e) {
throw e;
} catch (MarginServiceSystemException e) {
throw e;
} catch (Exception e) {
throw new MarginServiceSystemException(e);
} finally {
activeJob = null;
logger.info("Releasing lock");
lock.unlock();
logger.info("Lock released");
}
}

You can use Semaphore instead of ReentrantLock, its permits are not bound to thread.
Something like this (not sure what you want to do with the result of job.call() in the asynchronous case):
final static Semaphore lock = new Semaphore(1);
public void runJob(String desc, Callable job, boolean wait) {
logger.info("Acquiring lock");
if (!lock.tryAcquire()) {
throw new MyConcurrentJobException();
}
startThread(new Runnable() {
public void run() {
try {
job.call();
} finally {
lock.release();
}
}
});
}

I think I am misunderstanding completely because to block and wait while doing something asynchronously doesn't make too much sense to me unless some progress can be made on the invoking thread.
Could you do something like this:
final static Lock lock = new ReentrantLock();
final static ExecutorService service = Executors.newThreadPoolExecutor();
public Object runJob(String desc, Callable job, boolean wait) {
logger.info("Acquiring lock");
if (!lock.tryLock()) {
throw new MyConcurrentJobException();
}
activeJob = new JobStatus(desc);
logger.info("Lock acquired");
try {
Future<?> future = service.submit(job);
// This next line will block until the job is finished
// and also will hold onto the lock.
boolean finished = false;
Object o = null;
while(!finished) {
try {
o = future.get(300, TimeUnit.MILLISECONDS);
finished = true;
catch(TimeOutException e) {
// Do some periodic task while waiting
// foot.tapLots();
}
}
if (o instanceof MarginServiceAssertionException) {
throw ((MargineServiceAssertionException)o);
} else if (o instanceof MargineServiceSystemException) {
throw ((MarginServiceSystemException)o);
} else if (o instanceof Exception) {
throw new MarginServiceSystemException(e);
}
} catch (... InterruptedException e) { /// catch whatever exceptions throws as part of this
/// Whatever needs to be done.
} finally {
activeJob = null;
logger.info("Releasing lock");
lock.unlock();
logger.info("Lock released");
}
}