We Keep Coding

Java: Write to and read from same process multiple times

I've gone through so many related StackOverflow questions for this that I'm getting lost in them, and I've coded this multiple ways, but none seem to solve this problem in a way that works for me: How can I send output to the same command and process multiple times while at the same time receiving input from this same process?
(See Input various strings to same process in Java for a similar question, but this ended with only a theoretical answer.)
The command (command line, from a C++ executable) loads a large file, and then I want to send input to it very quickly, get back the answer, do other stuff in between, then send different input and get the corresponding answer. Multiply this by thousands or millions of times.
One implementation, with threads:
ProcessBuilder pb = new ProcessBuilder(command.split(" "));
kenLMProcess = pb.start();
KenLMInThread lmInput = new KenLMInThread(kenLMProcess.getInputStream());
KenLMInThread lmError = new KenLMInThread(kenLMProcess.getErrorStream());
KenLMOutThread lmOutput = new KenLMOutThread(kenLMProcess.getOutputStream());
lmOutput.inStr = "Test . \n";
lmInput.start();
lmOutput.start();
lmError.start();
lmOutput.join();
lmInput.join();
lmError.join();
outStr = lmInput.newStr;
But join waits until the thread ends. What if I don't want to wait for it to end? I can't seem to figure out how to use wait() for that purpose. For one I'd prefer to not have to keep opening and closing a new output stream and input stream every time I query the command. But at least that's better than starting a new ProcessBuilder every time.
Here's what run() looks like for KenLMOutThread:
public void run() {
try {
pw.write(inStr+"\n");
pw.write('\n');
} catch (Exception e) {
System.out.println("Error while inputting to KenLM.");
e.printStackTrace();
} finally {
pw.flush();
try {
pw.flush();
bw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Supposedly flush is supposed to let it move on, and "\n" at the end is supposed to help, but it just hangs unless I use close. And if I use close, I can't use the OutputStream anymore. I'm also then unable to make a new OutputStream from the Process.
If it helps, here's a more simple implementation with everything together (taken from How to send EOF to a process in Java?):
Note that close() is used, and using flush() without close() causes the program to hang.
public static String pipe(String str, String command2) throws IOException, InterruptedException {
Process p2 = Runtime.getRuntime().exec(command2);
OutputStream out = p2.getOutputStream();
out.write(str.getBytes());
out.close();
p2.waitFor();
BufferedReader reader
= new BufferedReader(new InputStreamReader(p2.getInputStream()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
return sb.toString();
}
Other things I've tried:
Using exec(): Process kenLMProcess=Runtime.getRuntime().exec(command);
Putting the command process in its own thread: KenLMProcessThread procThread = new KenLMProcessThread(pb.start());

If the target process is hanging unless you close the output stream, the problem is at that end: it is reading until end of stream before doing anything. Nothing you can do about that at the sending end.

Program freezes on bufferedreader close

Program freezes when closing buffered reader. Using forge modding API for minecraft, however I've had this issue before with standalone server side code. This particular code worked fine and then randomly started giving me this issue, not sure how to go about fixing this..
The close method:
public static void closeConnection() {
if (keepConnection) {
keepConnection = false;
try {
bufferedReader.close();
printWriter.close();
socket.close();
}
catch (IOException e) {
e.printStackTrace();
}
finally{
token = null;
}
}
}
I have checked to ensure that this is indeed where the freeze is occurring. Any ideas?

BufferedReader can block on close() because it contains a synchronized block on the lock instance:
synchronized (lock) {
if (in == null)
return;
in.close();
in = null;
cb = null;
}
This means there is another Thread in your program working with the BufferedReader (possibly blocked in a read()) which is holding the lock when you try to close. The solution is to have this other thread release the lock (interrupted if necessary) to allow the close to get the lock then complete.

Not possible. BufferedReader.close() doesn't do anything that blocks. You don't even need it. PrintWriter.close() will close everything. Remove it.
The only operation that can freeze here is closing the PrintWriter, which implies flushing its buffer, and the reason for that must be that the peer is a long way behind reading the output of this program, or isn't reading it at all.

How to end a thread handling socket connection?

I have a thread handling a socket connection:
BufferedReader socketInput = new BufferedReader(new InputStreamReader(mySocket.getInputStream()));
while (true)
{
String line = socketInput.readLine();
// do stuff
}
As I've read in a few answers on this site, the recommended solution is to use a flag which one thread sets and my (socket handling) thread checks and terminates itself when that flag changes state. Something like:
while (!done)
{
String line = socketInput.readLine();
// do stuff
}
But this can get stuck when readLine() is still waiting for input. I guess I could set a timeout:
mySocket.setSoTimeout(100);
while (!done)
{
String line = socketInput.readLine();
// do stuff
}
Which would probably work but I would still get a 100 ms delay before my thread "realizes" the flag's state changed.
Is there a way for the thread to "realize" right away that it should end? If not, is my solution (with timeout and flag done) correct?
Edit: I've clarified that the socketInput is of type BufferedReader (alternatively I'm considering Scanner).

The most common way to handle this is to close the socket from the other Thread. This will lead the reading side to unblock and exit with the (expected) error that the socket was closed. Depending on the socket API that you have available it might also be possible to shutdown only the reading side. From a short look at the JDK shutdownInput() might work.
If you however want to continue to read from the socket later on these obvisouly won't work. Your solution should work there, but is obvisouly worse for performance and reactivity since you basically poll the socket all 100ms.

Create a Selector
Configure your socket.getChannel() to non-blocking and register it to the Selector with SelectionKey.OP_READ
Call your Selector select() method that will return when there are some data to read so you can call readLine() (i.e. select() returns > 0)
Whenever you want to end your socket processing, set your done flag and call your Selector wakeup() method. That will make the select() return immediately (potentially 0, or 1 if there was activity). You can then check your done flag and end your thread gracefully.
Here is a quick implementation. Notice I pass the BufferedReader as an argument as if you're opening it in the thread you should also close it there, which would close the socket too, so it has to be done outside. There are two methods to signal the thread to gracefully stop processing input and one to send data:
public class SocketHandler extends Thread {
private Socket sok;
private BufferedReader socketInput;
private Selector sel;
private SocketChannel chan;
private boolean done;
public SocketHandler(Socket sok, BufferedReader socketInput) throws IOException {
this.sok = sok;
chan = sok.getChannel();
chan.configureBlocking(false);
sel = Selector.open();
chan.register(sel, SelectionKey.OP_READ);
this.socketInput = socketInput;
done = false;
}
#Override
public void run() {
while (!done) {
try {
if (sel.select() == 0)
continue;
} catch (IOException e) {
e.printStackTrace();
}
// Only one channel is registered on only one operation so we know exactly what happened.
sel.selectedKeys().clear();
doRead();
// Otherwise: loop through sel.selectedKeys(), check for readability and clear the set
}
try {
sel.close();
} catch (IOException e) {
e.printStackTrace();
}
}
private void doRead() {
try {
String line = socketInput.readLine();
// TODO: process 'line'
} catch (IOException e) {
e.printStackTrace();
}
}
public void signalStop() {
done = true;
if (sel != null)
sel.wakeup(); // Get out of sel.select()
}
public void doWrite(byte[] buffer) throws IOException { // Or "String message"
sok.getOutputStream().write(buffer); // Or anything else
}
}

The solution is correct, it will exit when done is set to true.
And yes, the readLine will always wait for 100ms, if you don't want to wait you may interrupt the thread by calling thread.interrupt() it but it's not very clean way.

The best way to know when finish a socket connection is to try to read something. If read method return -1 you can end threadling socket connection
byte[] data = new byte[2048];
while (!done) {
int count = input.read(data);
if (count <= 0) {
if (count < 0)
done = true;
continue;
}
String request = new String(data, 0, count);
//do stuff
}
We try to read something in input if count == -1, the socket client is disconnected now we can end the loop, by changing the value of done.

Can't detect disconnect without extra readLine() loop

I am developing a program that uses sockets and currently I have a function in my code that checks for a heartbeat from the client every second.
private void userLoop() { // checks for incoming data from client
Timer t = new Timer();
t.schedule(new TimerTask() {
#Override
public void run() {
try {
socketIn.read(); // check for heartbeat from client
String userInput;
while ((userInput = br.readLine()) != null) {
}
} catch (Exception e) {
ControlPanel.model.removeElement(getUsername());
ControlPanel.append(getUsername() + " has disconnected.");
}
}
}, 1000);
}
When a client closes the game via the X button, shutting off their computer, logging out, whatever it may be, I get the message "'username' has disconnected". This is exactly what I want, however, it only works with the while loop in the code. The while loop essentially does nothing and I have no idea why it doesn't work with out.
If I remove the while loop and I disconnect using my client nothing gets printed out server sided.
String userInput;
while ((userInput = br.readLine()) != null) {
}
The above is essentially the dead code that does nothing but without it my program doesn't work the way it should..
Why is the code needed and how can I remove it and still make my program work correctly?

In this case, your while loop is essentially stalling your program until you no longer receive an input string. It's not dead code; it is just your way of installing a wait.
Otherwise, based on my understanding in the Timer class, it only waits one second, which might be too short of a timespan for what you're waiting to capture.

I fixed my problem by changing everything in the try block with
br.readLine();

There's a saying I've heard about exception handling: "Exceptions should only be used for exceptional situations." A client disconnecting from a server is not exceptional.
Now that I have that off my chest, let's move on. According to this other question,
socket.getInputSteam.read() does not throw when I close the socket from the client
it sounds like the read call won't throw if you're closing things properly on the client side.

The problem is that when the remote socket is closed, read() does not throw an Exception, it just returns -1 to signal the end of the stream.
The following should work without needing to call readLine():
try {
int ret = socketIn.read(); // check for heartbeat from client
if (ret == -1) {
// Remote side closed gracefully
clientDisconnected();
}
} catch (SocketTimeoutException e) {
// Timeout -- handle as required
handleTimeout();
} catch (IOException e) {
// Connection lost due to I/O error
clientDisconnected()
}

Best practice for reading / writing to a java server socket

How do you design a read and write loop which operates on a single socket (which supports parallel read and write operations)? Do I have to use multiple threads? Is my (java) solution any good? What about that sleep command? How do you use that within such a loop?
I'm trying to use 2 Threads:
Read
public void run() {
InputStream clientInput;
ByteArrayOutputStream byteBuffer;
BufferedInputStream bufferedInputStream;
byte[] data;
String dataString;
int lastByte;
try {
clientInput = clientSocket.getInputStream();
byteBuffer = new ByteArrayOutputStream();
bufferedInputStream = new BufferedInputStream(clientInput);
while(isRunning) {
while ((lastByte = bufferedInputStream.read()) > 0) {
byteBuffer.write(lastByte);
}
data = byteBuffer.toByteArray();
dataString = new String(data);
byteBuffer.reset();
}
} catch (IOException e) {
e.printStackTrace();
}
}
Write
public void run() {
OutputStream clientOutput;
byte[] data;
String dataString;
try {
clientOutput = clientSocket.getOutputStream();
while(isOpen) {
if(!commandQueue.isEmpty()) {
dataString = commandQueue.poll();
data = dataString.getBytes();
clientOutput.write(data);
}
Thread.sleep(1000);
}
clientOutput.close();
}
catch (IOException e) {
e.printStackTrace();
}
catch (InterruptedException e) {
e.printStackTrace();
}
}
Read fails to deliver a proper result, since there is no -1 sent.
How do I solve this issue?
Is this sleep / write loop a good solution?

There are basically three ways to do network I/O:
Blocking. In this mode reads and writes will block until they can be fulfilled, so if you want to do both simultaneously you need separate threads for each.
Non-blocking. In this mode reads and writes will return zero (Java) or in some languages (C) a status indication (return == -1, errno=EAGAIN/EWOULDBLOCK) when they cannot be fulfilled, so you don't need separate threads, but you do need a third API that tells you when the operations can be fulfilled. This is the purpose of the select() API.
Asynchronous I/O, in which you schedule the transfer and are given back some kind of a handle via which you can interrogate the status of the transfer, or, in more advanced APIs, a callback.
You should certainly never use the while (in.available() > 0)/sleep() style you are using here. InputStream.available() has few correct uses and this isn't one of them, and the sleep is literally a waste of time. The data can arrive within the sleep time, and a normal read() would wake up immediately.

You should rather use a boolean variable instead of while(true) to properly close your thread when you will want to. Also yes, you should create multiple thread, one per client connected, as the thread will block itself until a new data is received (with DataInputStream().read() for example). And no, this is not really a design question, each library/Framework or languages have its own way to listen from a socket, for example to listen from a socket in Qt you should use what is called "signals and slots", not an infinite loop.