We Keep Coding

Thread.join() not behaving as expected

I'm pretty new to Multithreading in java but am totally stumped about why this isn't behaving as I want it to.
I have a Producer-Consumer wherein I have
private void produceConsume() {
try {
Thread producer = new Thread(new Runnable() {
public void run() {
try {
produce();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
completedProduce = true;
}
}
private void produce() throws InterruptedException {
synchronized (this) {
while (queue.size() == capacity) {
wait();
}
try(InputStream is = new FileInputStream(file)) {
queue.add("hello");
} catch (IOException e) {
LOG.error("Error doing stream stuff: " + e.getMessage(), e);
}
notify();
}
}
});
producer.start();
List<Thread> consumers = new ArrayList<>();
for (int i = 0; i < noOfThreads; i++) {
Thread consumer = new Thread(new Runnable() {
#Override
public void run() {
try {
consume();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private void consume() throws InterruptedException {
while (queue.size() > 0 || !completedProduce) {
synchronized (this) {
while (queue.size() == 0 && !completedProduce) {
wait();
}
String s = queue.poll();
System.out.println(s);
}
notify();
}
}
}
});
consumer.start();
consumers.add(consumer);
}
for (Thread t : consumers) {
t.join();
}
producer.join();
} catch (Exception e) {
LOG.error("InterruptedException e: " + e.getMessage(), e);
} finally {
LOG.info("We are done with this file!");
}
}
Now, I've noticed that all functionality changes based on where I put my producer.join() statement. For example, if I put producer.join() right after producer.start() then everything works - but the number of threads has no impact on runtime. This makes sense as I'm slowed down drastically by how long it takes to produce and so the longest task wins out.
However, if I put producer.join() where it is in the example provided (I do the join when I do the join for the consumers) then everything just stops running before the producer actually finishes. As in, the program stalls after the first thing is consumed, waiting for something, but the thread never dies.
How do I make it so that things run correctly and nothing stalls waiting for another process to finish?
Thanks in advance,

Android client-server communication throws exception on input.read

i am developing Android-Server communication using sockets and input,output streams (no wrappign classes like datainputstream, objectinput stream etc.)
Communiction is based on three threads on each side (server has additional thread which accepts new socket)
1st thread is director, which accepts messages from receiver via LinkedBlockingQueue, reacts on them and sends data via LinkedBlockingQueue to sender
2nd thread is receiver which periodically reads socket (via InputStream.read) and if there is message it pass it to director with LinkedBlockingQueue
when connection is lost, Client-Android device (which has input.read blocked) throws immediately connection time out exception
3rd thread is sender which periodically takes messages from LinkedBlockingQueue and sends that data to other side of connection
Problem is: Prevent Exception on ClientSide receiver from throws (this look alike some Android thing, because input.read alone should not throw any exception related to time out connection
here is code of receiver:
public class Receiver implements Runnable {
private boolean run = true;
BlockingQueue<MessageQueue> queueReceiverOut;
InputStream in;
////////////////////////////// CONSTRUCTOR ////////////////////////////////
public Receiver(BlockingQueue<MessageQueue> queueReceiverOut, InputStream in) {
this.queueReceiverOut = queueReceiverOut;
this.in = in;
}
// ////////////////////////////// METHODS ////////////////////////////////
/**
* Runs when thread starts.
*/
public void run() {
int[] message = new int[2];
byte[] data;
MessageQueue msg;
try {
while(true) {
msg = new MessageQueue();
message = receiveMessage();
System.out.println("receives message");
if(message[0] != -1) {
System.out.println("receives full message");
if(message[1] != 0) {
data = receiveData(message[1]);
msg.setMessageType(message[0]);
msg.setDataLength(message[1]);
msg.setData(data);
queueReceiverOut.put(msg);
} else {
msg.setMessageType(message[0]);
msg.setDataLength(message[1]);
queueReceiverOut.put(msg);
}
}
}
} catch (IOException e) {
System.out.println("----disconnected-----");
try {
MessageQueue msgReceiverOut = new MessageQueue();
msgReceiverOut.setMessageType(SocketMessages.STATUS_OFFLINE);
queueReceiverOut.put(msgReceiverOut);
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public int[] receiveMessage() throws IOException {
int[] messageHead = new int[2];
messageHead[0] = in.read();
if(messageHead[0] != -1) {
System.out.println("received message with type : " + messageHead[0]);
int length1 = in.read();
int length2 = in.read();
int length3 = in.read();
int length4 = in.read();
messageHead[1] = ((length1 << 24) + (length2 << 16) + (length3 << 8) + (length4 << 0));
System.out.println(" with length : " + messageHead[1]);
}
return messageHead;
}
public byte[] receiveData(int length) throws IOException {
byte[] buffer = new byte[length];
// Read in the bytes
int offset = 0;
int numRead = 0;
while (offset < length
&& (numRead = in.read(buffer,
offset, length - offset)) >= 0) {
offset += numRead;
}
// Ensure all the bytes have been read in
if (offset < length) {
throw new IOException("Could not completely read file ");
}
return buffer;
}
public boolean isRun() {
return run;
}
public void setRun(boolean run) {
this.run = run;
}
}
and sender:
public class Sender implements Runnable {
private boolean run = true;
BlockingQueue<MessageQueue> queueSenderIn;
BlockingQueue<MessageQueue> queueSenderOut;
OutputStream out;
////////////////////////////// CONSTRUCTOR ////////////////////////////////
public Sender(BlockingQueue<MessageQueue> queueSenderIn, BlockingQueue<MessageQueue> queueSenderOut, OutputStream out) {
this.queueSenderOut = queueSenderOut;
this.queueSenderIn = queueSenderIn;
this.out = out;
}
// ////////////////////////////// METHODS ////////////////////////////////
/**
* Runs when thread starts.
*/
#Override
public void run() {
MessageQueue msg;
try {
while(run) {
msg = queueSenderIn.poll(2, TimeUnit.SECONDS);
if(msg != null) {
sendMessage(msg.getMessageType(),msg.getDataLength());
if(msg.getDataLength()!=0) {
sendData(msg.getData());
}
}
}
Log.v(getClass().getName(),"sender destroyed");
} catch (IOException e) {
Log.v(getClass().getName(),"connection closed");
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void sendMessage(int messageType, int dataLength) throws IOException, InterruptedException {
MessageQueue msg = new MessageQueue();
Log.v(getClass().getName(), "sending message type : " + messageType);
out.write(messageType);
Log.v(getClass().getName(), "sending data with length : " +dataLength);
out.write((dataLength >>> 24) & 0xFF);
out.write((dataLength >>> 16) & 0xFF);
out.write((dataLength >>> 8) & 0xFF);
out.write((dataLength >>> 0) & 0xFF);
msg.setMessageType(messageType);
queueSenderOut.put(msg);
}
public void sendData(byte[] data) throws IOException {
String string = new String(data,"UTF-8");
Log.v(getClass().getName(), " with content : " + string);
out.write(data);
}
public boolean isRun() {
return run;
}
public void setRun(boolean run) {
this.run = run;
}
}
UPDATED : Because misinterpreting exception

Under abnormal conditions the underlying connection may be broken by
the remote host or the network software (for example a connection
reset in the case of TCP connections). When a broken connection is
detected by the network software the following applies to the returned
input stream :
The network software may discard bytes that are buffered by the
socket. Bytes that aren't discarded by the network software can be
read using read.
If there are no bytes buffered on the socket, or all buffered bytes
have been consumed by read, then all subsequent calls to read will
throw an IOException.
If there are no bytes buffered on the socket, and the socket has not
been closed using close, then available will return 0.
from: http://docs.oracle.com/javase/7/docs/api/java/net/Socket.html#getInputStream()
Especially mind the "subsequent calls" part. That means if you are already blocking in a read call, the above condition regarding read calls does not (yet) apply.
So far the explanation. Now to the solution:
You can (one of many possibilities) periodically send messages, even when communication is idle. So your sender will detect connection loss and can close the in-stream, too.
EDIT: To make it a little bit clearer ...
Connection loss
call read
IOException
whereas
call read (blocks!)
waiting for input: connection lost
-- no Exception!
I guess the point is that ( I assume ) you server enters a read and stays there for a long time, while your client is receiving data while connection is broken. Thus it will constantly call and return from read. It still is possible that the connection loss is happening while waiting for a read to unblock, but it is much less probable.

how to read from standard input non-blocking?

long end=System.currentTimeMillis()+60*10;
InputStreamReader fileInputStream=new InputStreamReader(System.in);
BufferedReader bufferedReader=new BufferedReader(fileInputStream);
try
{
while((System.currentTimeMillis()<end) && (bufferedReader.readLine()!=null))
{
}
bufferedReader.close();
}
catch(java.io.IOException e)
{
e.printStackTrace();
}
I actually tried doing the above for reading in 600 miliseconds time after which it should not allow reading but the readline of the bufferedreader is blocking.Please help

Using BufferedReader.available() as suggested by Sibbo isn't reliable. Documentation of available() states:
Returns an estimate of the number of bytes that can be read... It is never correct to use the return value of this method to allocate a buffer.
In other words, you cannot rely on this value, e.g., it can return 0 even if some characters are actually available.
I did some research and unless you are able to close the process input stream from outside, you need to resort to an asynchronous read from a different thread. You can find an example how to read without blocking line by line here.
Update: Here is a simplified version of the code from the link above:
public class NonblockingBufferedReader {
private final BlockingQueue<String> lines = new LinkedBlockingQueue<String>();
private volatile boolean closed = false;
private Thread backgroundReaderThread = null;
public NonblockingBufferedReader(final BufferedReader bufferedReader) {
backgroundReaderThread = new Thread(new Runnable() {
#Override
public void run() {
try {
while (!Thread.interrupted()) {
String line = bufferedReader.readLine();
if (line == null) {
break;
}
lines.add(line);
}
} catch (IOException e) {
throw new RuntimeException(e);
} finally {
closed = true;
}
}
});
backgroundReaderThread.setDaemon(true);
backgroundReaderThread.start();
}
public String readLine() throws IOException {
try {
return closed && lines.isEmpty() ? null : lines.poll(500L, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
throw new IOException("The BackgroundReaderThread was interrupted!", e);
}
}
public void close() {
if (backgroundReaderThread != null) {
backgroundReaderThread.interrupt();
backgroundReaderThread = null;
}
}
}

You could check with BufferedReader.available() > 0 if there are chars to read.
String s;
while((System.currentTimeMillis()<end))
{
if (bufferedReader.available() > 0)
s += bufferedReader.readLine();
}
bufferedReader.close();

long end=System.currentTimeMillis()+60*10;
InputStreamReader fileInputStream = new InputStreamReader(System.in);
BufferedReader bufferedReader = new BufferedReader(fileInputStream);
try {
while ((System.currentTimeMillis() < end)) {
if (bufferedReader.ready()) {
System.out.println(bufferedReader.readLine());
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (bufferedReader != null) {
bufferedReader.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}

The only reliable way would be to start a worker thread and do the actual reading inside it, while the caller thread would monitor the latency.
If the worker thread is waiting longer that allowed, the master thread would terminate it and throw an exception.

BufferReader.readLine() can block for a very long time if a line is extremely long like 1M chars.
Does your file contains such long lines?
If yes, you may have to break up the lines, or use per-char read methods like BufferReader.read().

Android BluetoothSocket - Timing out

I have written a Bluetooth API for connecting with an external accessory.
The way that the API is designed is that there are a bunch of blocking calls such as getTime, setTime, getVolume, setVolume, etc.
The way these work is that they create a payload to send and call a method called sendAndReceive() which does some prep work and eventually does the following:
byte[] retVal = null;
BluetoothSocket socket = getSocket();
// write
socket.getOutputStream().write(payload);
// read response
if(responseExpected){
byte[] buffer = new byte[1024]; // buffer store for the stream
int readbytes = socket.getInputStream().read(buffer);
retVal = new byte[readbytes];
System.arraycopy(buffer, 0, retVal, 0, readbytes);
}
return retVal;
The problem is that sometimes this device becomes slow or non-responsive so I would like to put a timeout on this call.
I have tried several methods of putting this code in a thread\future task and running it with a timeout, for example:
FutureTask<byte[]> theTask = null;
// create new task
theTask = new FutureTask<byte[]>(
new Callable<byte[]>() {
#Override
public byte[] call() {
byte[] retVal = null;
BluetoothSocket socket = getSocket();
// write
socket.getOutputStream().write(payload);
// read response
if(responseExpected){
byte[] buffer = new byte[1024]; // buffer store for the stream
int readbytes = socket.getInputStream().read(buffer);
retVal = new byte[readbytes];
System.arraycopy(buffer, 0, retVal, 0, readbytes);
}
return retVal;
}
});
// start task in a new thread
new Thread(theTask).start();
// wait for the execution to finish, timeout after 6 secs
byte[] response;
try {
response = theTask.get(6L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
throw new CbtException(e);
} catch (ExecutionException e) {
throw new CbtException(e);
} catch (TimeoutException e) {
throw new CbtCallTimedOutException(e);
}
return response;
}
The problem with this approach is that I can't re-throw exceptions in the call method and since some of the methods throw exceptions I want to forward back to the API's client I can't use this methodology.
Can you recommend some other alternative?
Thanks!

You're saving you can't use the Future<> method because you want to re-throw the exception but in fact this is possible.
Most examples online do implement Callable with the prototype public ? call() but just change it to public ? call() throws Exception and all will be fine: you'll get the exception in the theTask.get() call and you can rethrow it to callers.
I have personally used Executors exactly for bluetooth socket timeout handling on android:
protected static String readAnswer(...)
throws Exception {
String timeoutMessage = "timeout";
ExecutorService executor = Executors.newCachedThreadPool();
Callable<String> task = new Callable<String>() {
public String call() throws Exception {
return readAnswerNoTimeout(...);
}
};
Future<String> future = executor.submit(task);
try {
return future.get(SOCKET_TIMEOUT_MS, TimeUnit.MILLISECONDS);
} catch (TimeoutException ex) {
future.cancel(true);
throw new Exception(timeoutMessage);
}
}

Why not try something like
public class ReadTask extends Thread {
private byte[] mResultBuffer;
private Exception mCaught;
private Thread mWatcher;
public ReadTask(Thread watcher) {
mWatcher = watcher;
}
public void run() {
try {
mResultBuffer = sendAndReceive();
} catch (Exception e) {
mCaught = e;
}
mWatcher.interrupt();
}
public Exception getCaughtException() {
return mCaught;
}
public byte[] getResults() {
return mResultBuffer;
}
}
public byte[] wrappedSendAndReceive() {
byte[] data = new byte[1024];
ReadTask worker = new ReadTask(data, Thread.currentThread());
try {
worker.start();
Thread.sleep(6000);
} catch (InterruptedException e) {
// either the read completed, or we were interrupted for another reason
if (worker.getCaughtException() != null) {
throw worker.getCaughtException();
}
}
// try to interrupt the reader
worker.interrupt();
return worker.getResults;
}
There is an edge case here that the Thread calling wrappedSendAndReceive() may get interrupted for some reason other than the interrupt from the ReadTask. I suppose a done bit could be added to the ReadTask to allow the other thread to test if the read finished or the interrupt was caused by something else, but I'm not sure how necessary this is.
A further note is that this code does contain the possibility for data loss. If the 6 seconds expires and some amount of data has been read this will end up being discarded. If you wanted to work around this, you'd need to read one byte at a time in ReadTask.run() and then appropriately catch the InterruptedException. This obviously requires a little rework of the existing code to keep a counter and appropriately resize the read buffer when the interrupt is received.

How to copy input/output streams of the Process to their System counterparts?

This is a follow up to this question. The answer suggested there is
to copy the Process out, err, and input streams to the System versions
with IOUtils.copy as follows (after fixing various compilation errors):
import org.apache.commons.io.IOUtils;
import java.io.IOException;
public class Test {
public static void main(String[] args)
throws IOException, InterruptedException {
final Process process = Runtime.getRuntime().exec("/bin/sh -i");
new Thread(new Runnable() {public void run() {
try {
IOUtils.copy(process.getInputStream(), System.out);
} catch (IOException e) {}
} } ).start();
new Thread(new Runnable() {public void run() {
try {
IOUtils.copy(process.getErrorStream(), System.err);
} catch (IOException e) {}
} } ).start();
new Thread(new Runnable() {public void run() {
try {
IOUtils.copy(System.in, process.getOutputStream());
} catch (IOException e) {}
} } ).start();
process.waitFor();
}
}
However, the resulting code doesn't work for interactive processes like the one executing sh -i command. In the latter case there is no response to any of the sh commands.
So my question is: could you suggest an alternative to copy the streams that will work with interactive processes?

The problem is that IOUtil.copy() is running while there is data in the InputStream to be copied. Since your process only produces data from time to time, IOUtil.copy() exits as it thinks there is no data to be copied.
Just copy data by hand and use a boolean to stop the thread form outside:
byte[] buf = new byte[1024];
int len;
while (threadRunning) { // threadRunning is a boolean set outside of your thread
if((len = input.read(buf)) > 0){
output.write(buf, 0, len);
}
}
This reads in chunks as many bytes as there are available on inputStream and copies all of them to output. Internally InputStream puts thread so wait() and then wakes it when data is available.
So it's as efficient as you can have it in this situation.

Process.getOutputStream() returns a BufferedOutputStream, so if you want your input to actually get to the subprocess you have to call flush() after every write().
You can also rewrite your example to do everything on one thread (although it uses polling to read both System.in and the process' stdout at the same time):
import java.io.*;
public class TestProcessIO {
public static boolean isAlive(Process p) {
try {
p.exitValue();
return false;
}
catch (IllegalThreadStateException e) {
return true;
}
}
public static void main(String[] args) throws IOException {
ProcessBuilder builder = new ProcessBuilder("bash", "-i");
builder.redirectErrorStream(true); // so we can ignore the error stream
Process process = builder.start();
InputStream out = process.getInputStream();
OutputStream in = process.getOutputStream();
byte[] buffer = new byte[4000];
while (isAlive(process)) {
int no = out.available();
if (no > 0) {
int n = out.read(buffer, 0, Math.min(no, buffer.length));
System.out.println(new String(buffer, 0, n));
}
int ni = System.in.available();
if (ni > 0) {
int n = System.in.read(buffer, 0, Math.min(ni, buffer.length));
in.write(buffer, 0, n);
in.flush();
}
try {
Thread.sleep(10);
}
catch (InterruptedException e) {
}
}
System.out.println(process.exitValue());
}
}

You should instead use the ProcessBuilder.redirectOutput method & friends. Read more here