We Keep Coding

Interrupt BufferedReader#readLine() without closing InputStream

The InputStream of my Process should attach and detach whenever the user wants to see it or not. The attaching works fine, but the detach fails. Default answer to interrupt the readLine() method is always to close the stream, but I cant in this case or the Process will finish or at least not available for future attachments. This is how the stream is read:
BufferedReader reader = new BufferedReader(new InputStreamReader(getProcess().getInputStream()));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
To detach I tried some stuff:
Close any of the streams, failed: close method is blocking and waits for the readLine()
Implement another stream to send null / abortion value with SequenceInputStream, failed: when one InputStream was waiting for input, the other was not even called
Use reflections to unlock the read() method inside any of the streams, failed: not sure why, but did not work. Should we go on with this try? Here is the sourcecode:
try {
Field modifiers = Field.class.getDeclaredField("modifiers");
modifiers.setAccessible(true);
Field fdecoder = stream.getClass().getDeclaredField("sd");
fdecoder.setAccessible(true);
modifiers.setInt(fdecoder, 1);
StreamDecoder decoder = (StreamDecoder) fdecoder.get(stream);
Field flock = decoder.getClass().getSuperclass().getDeclaredField("lock");
flock.setAccessible(true);
modifiers.setInt(flock, 1);
Object lock = (Object) flock.get(decoder);
synchronized (lock) {
lock.notifyAll();
}
} catch (NoSuchFieldException | IllegalAccessException e) {
Wrapper.handleException(Thread.currentThread(), e);
}
Not sure how I can fix this. Could you help me interrupting the readLine() method without closing the stream, simple and performant? Thanks.
Edit:
What do I mean by "performant"? My application has not much users, but a lot of processes. The answer by #EJP is not wrong - but unperformant in the case of my application. I cannot have hundreds of threads for hundreds of processes, but I can have as many processes as I have users watching. That's why I try to interrupt the process gracefully. Fewer threads, less running/blocked threads.
Here is the application described (https://imgur.com/VUcYUfi.png)
The Thread that sends the information to the user is the same that reads the input.

I didn't expect it to work, but futures are actually cancelable (but why?).
After #Tarun Lalwani mentioned the TimeLimiter of Googles Guava library, I inspected the code, tried it in my examples (worked!) and rewrote it a bit - make it not time-based, but method-call-based?!
Here is what I got from my research: A wrapper for the BufferedReader:
public class CancelableReader extends BufferedReader {
private final ExecutorService executor;
private Future future;
public CancelableReader(Reader in) {
super(in);
executor = Executors.newSingleThreadExecutor();
}
#Override
public String readLine() {
future = executor.submit(super::readLine);
try {
return (String) future.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
} catch (CancellationException e) {
return null;
}
return null;
}
public void cancelRead() {
future.cancel(true);
}
}
This class allows you to use the BufferedReader#readLine() when you need it and cancel it when you want to continue / interrupt the Thread it is running in. Here is some example code of it in action:
public static void main(String[] args) {
System.out.println("START");
CancelableReader reader = new CancelableReader(new InputStreamReader(System.in));
String line;
new Thread(() -> {
try {
Thread.sleep(10000);
reader.cancelRead();
} catch (InterruptedException e) {
e.printStackTrace();
}
}).start();
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
System.out.println("END");
}
And the output of it:
START
> Hello World!
Hello World!
> What's up?
What's up?
END //Exactly after 5 seconds, when the cancel was called
> Hey, you still there?
//No output as expected
And the last thing I wanna say is why this and not closing InputStream or create one Thread per process?
In this case the InputStream is the stream of a Process, which means we cannot close it. One way would be to unblock readLine() and return null to finish the while-loop, but this is made with Reflection, which is not as beautiful as our solution now and didn't work for any reason. The application uses many processes but has a limited amount of users - thats why we decide for the amount of threads per user and not per process.
I hope you guys will find this Thread in the future and it is helpful for you. Would be awesome if you leave an upvote, so I can get back my rep of the bounty.
Dont forget to upvote the comments either! They helped me alot and brought me to the right solution:
Interrupt BufferedReader#readLine() without closing InputStream

You're going at this back to front.
You can't stop collecting the process's output, or you will stall the child process.
You want to stop displaying the output when the user doesn't want to see it. Look on it as a user interface issue only.

Thread to process multiple rest calls

I am trying to process around 1000 files using below code:
ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize);
Runnable worker = null;
for (File file : files) {
if (file.isFile()) {
worker = new FileProcessThread(file, connectionVo);
executor.execute(worker);
file.deleteOnExit();
}
}
while (!executor.isTerminated()) {
System.out.println("Still running");
}
executor.shutdown();
System.out.println("Finished all threads");
This code creates multiple threads. Each thread has multiple rest calls inside.
These rest apis are for processing input file. Each thread also logs each transaction event which occurs while processing.
But result of these threads execution is not consistent.
For few threads it works perfectly fine.Picks the file.Logs correct
transactions and move processed file to proper directory.
But for for some threads it shows some unpredictable behavior such as it logs file process event of one thread into other.
Steps in each thread :
Create transaction - rest call
Log event in transaction for process start - rest call
Gives file to other module for file conversion - rest call which internally
creates one more thread which is synchronized
Once file is processed it is moved to other - in the same code directory
I want consistent performance out of these threads. Any help will be appreciated.
Code inside run :
long transactionID = 0l;
long connectionId = connectionVo.getConnectionId();
try {
transactionID = beginTransaction.getTransactionId();
FileInputStream processedFileData;
processedFileData = new FileInputStream(file);
Response response = Service.postMessage(stream2file,
connectionId, 0, transactionID);
if (response.getStatus() != 200) {
writToDirectory(stream2file, userError, file.getName(), transactionID);
}
} else {
String userArchive = getUserArchive();
if (checkDirectory(userArchive, transactionID)) {
writToDirectory(stream2file, userArchive, file.getName(), transactionID);
}
}
file.delete();
} catch (FileNotFoundException e) {
}

I suggest you use Java 8 to do the multi-threading as it is much cleaner.
files.parallelStream()
.filter(File::isFile)
.forEach(f -> new FileProcessThread(file, connectionVo).run());
Your task deletes the file when finished successfully.
This will only pass each file to one task.
BTW Don't call your tasks xxxThread unless they are actually a Thread, and avoid ever sub-classing a Thread.

Can't detect disconnect without extra readLine() loop

I am developing a program that uses sockets and currently I have a function in my code that checks for a heartbeat from the client every second.
private void userLoop() { // checks for incoming data from client
Timer t = new Timer();
t.schedule(new TimerTask() {
#Override
public void run() {
try {
socketIn.read(); // check for heartbeat from client
String userInput;
while ((userInput = br.readLine()) != null) {
}
} catch (Exception e) {
ControlPanel.model.removeElement(getUsername());
ControlPanel.append(getUsername() + " has disconnected.");
}
}
}, 1000);
}
When a client closes the game via the X button, shutting off their computer, logging out, whatever it may be, I get the message "'username' has disconnected". This is exactly what I want, however, it only works with the while loop in the code. The while loop essentially does nothing and I have no idea why it doesn't work with out.
If I remove the while loop and I disconnect using my client nothing gets printed out server sided.
String userInput;
while ((userInput = br.readLine()) != null) {
}
The above is essentially the dead code that does nothing but without it my program doesn't work the way it should..
Why is the code needed and how can I remove it and still make my program work correctly?

In this case, your while loop is essentially stalling your program until you no longer receive an input string. It's not dead code; it is just your way of installing a wait.
Otherwise, based on my understanding in the Timer class, it only waits one second, which might be too short of a timespan for what you're waiting to capture.

I fixed my problem by changing everything in the try block with
br.readLine();

There's a saying I've heard about exception handling: "Exceptions should only be used for exceptional situations." A client disconnecting from a server is not exceptional.
Now that I have that off my chest, let's move on. According to this other question,
socket.getInputSteam.read() does not throw when I close the socket from the client
it sounds like the read call won't throw if you're closing things properly on the client side.

The problem is that when the remote socket is closed, read() does not throw an Exception, it just returns -1 to signal the end of the stream.
The following should work without needing to call readLine():
try {
int ret = socketIn.read(); // check for heartbeat from client
if (ret == -1) {
// Remote side closed gracefully
clientDisconnected();
}
} catch (SocketTimeoutException e) {
// Timeout -- handle as required
handleTimeout();
} catch (IOException e) {
// Connection lost due to I/O error
clientDisconnected()
}

How to I call a function to delay execute because of some conditions in java?

This is my run method, it opens a socket, and waiting for an accepted connection, if connection is accepted, will have a separate Thread open for execute it:
while (isKeepRun) {
socket = serverSocket.accept();
WorkTask worktask = new WorkTask();
worktask.setSocket(socket);
worktask.setIn(new ObjectInputStream(socket.getInputStream()));
worktask.setOut(new ObjectOutputStream(socket.getOutputStream()));
Thread wt = new Thread(worktask);
wt.start();
}
if (socket != null) {
socket.close();
}
if (serverSocket != null) {
serverSocket.close();
}
When the user call it to stop, they call this method, to change the while loop condition. in order to break the while loop:
public void stopWorking() {
isKeepRun = false;
}
Well, the WorkTask's run method is very simple like that:
try {
do {
objectOutputStream.flush();
receivedObj = objectInputStream.readObject();
if (receivedObj != null){
System.out.println(receivedObj.toString()+" " + receivedObj.hashCode());
}
} while (receivedObj != null
&& !receivedObj.equals(SharedConstant.SOCKET_EOF_STRING));
if (objectInputStream != null) {
objectInputStream.close();
}
if (objectOutputStream != null) {
objectOutputStream.close();
}
} catch (IOException e1) {
e1.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
The problem is assume that reading one message need 1 second per message, the user may give up to 100 messages, that's mean it requires 100 seconds to run, in the socket. When the isKeepRun is keep running, there is no problem. But when the user wanna to stop , and call stopWorking, the loop will be escaped, can the socket is closed, during the WorkTask is reading the message. How can I delay the execution of stopWorking if the socket is still reading, if the socket is finished reading, and the stopWorking will be call immediately, but if the socket don't have any thing to read, I can call the stopWorking in no delay?
Thanks.

If your worker thread is handling your client request then it should be his responsibility to close the socket connection. You should move the code that closes the accepted socket into your worker thread. Your server socket accept loop will be independent and will close as soon as the close request is made. But, the existing connections will still be valid and the worker thread can continue handling them.

There is a problem in your code:
while (isKeepRun) {
socket = serverSocket.accept();
....
}
....
if (socket != null) {
socket.close();
}
You use only one socket reference for all client socket. So if there are more than one client socket, then you will only close the last socket.
As above said, you can use wt.join() to wait for the worker thread to finish, then the main thread will finish. But you have only one reference for client socket. Even you wait, you can only wait for the last socket to finish. All the previous client socket will be closed if you set it to stop.

Try adding
wt.join()
after the wt.start(). This waits for thread to finish it's execution.

You can check if there are still active worker threads around by calling their isAlive method. To do so, you need to keep track of your workers (using some list or map).
You might also use a call back mechanism for the workers, through which they can report back, when they finished their task. Before stopping, you simply check if every worker is done.
In both cases, if there are still active workers, sleep for some time, and check again, until all threads have finished.
Edit:
Vikas Nalwar made a good point about letting the workers close the socket connection. It still might be a good idea to wait for the worker threads to finish, though.

java thread waiting for dead process to finish

I wrote a java class in order to perform multithreaded tasks, each task running an external process.
The process is in charge of converting ".chp" files into ".txt" files. It is written in C.
This process breaks at one point because it disappears when looking at a "top" in my terminal (probably due to a corrupted chp file). The problem is that the process in my java thread does not return. The "process.waitFor()" seems to go on forever (at least 'til the 12 hours I specified for the ExecutorService.
Am I doing something wrong (not catching an exception?)?
I tried setting a class variable of type String in MyThread and putting an error message in place of throwing a new RuntimeException, then print the String at the end of the main, but the thread code doesn't reach to this point. It still gets stuck at the waitFor().
Shouldn't the process terminate once the C program has failed?
The program prints on the terminal (cf: MyThread):
A
B
C
main:
String pathToBin = "/path/to/bin";
List<MyThread> threadList = new ArrayList<MyThread>();
for (File f : folderList) {
File[] chpFilesInFolder = f.listFiles(new FilenameFilter() {
#Override
public boolean accept(File dir, String name) {
if (name.endsWith(".chp")){
return true;
}else{
return false;
}
}
});
File chpFile = writeChpFiles(chpFilesInFolder);
String[] cmd = {pathToBin, "--arg1", chpFile, "--out-dir", outputFolder};
MyThread t = new MyThread(cmd, f, chpFilesInFolder);
threadList.add(t);
}
ExecutorService threadExecutor = Executors.newFixedThreadPool(4);
for(MyThread th : threadList){
threadExecutor.execute(th);
}
threadExecutor.shutdown();
try {
threadExecutor.awaitTermination(12, TimeUnit.HOURS);
} catch (InterruptedException e) {
e.printStackTrace();
}
MyThread:
class MyThread extends Thread{
private String[] cmd;
private File chpFolder;
private File[] chpFilesInFolder;
public MyThread(String[] cmd, File chpFolder, File[] chpFilesInFolder){
this.cmd = cmd;
this.chpFolder = chpFolder;
this.chpFilesInFolder = chpFilesInFolder;
}
#Override
public void run() {
Process process = null;
try{
System.err.println("A ");
ProcessBuilder procBuilder = new ProcessBuilder(cmd);
procBuilder.redirectErrorStream(true);
System.err.println("B");
process = procBuilder.start();
System.err.println("C");
process.waitFor();
System.err.println("D");
if(process.exitValue()!=0) System.err.println("ERROR !"+process.exitValue());
System.err.println("E");
}catch(IOException e){
e.printStackTrace();
}catch(InterruptedException e){
e.printStackTrace();
}catch(Throwable e){
e.printStackTrace();
}finally{
System.err.println("F");
if(process!=null) {try { process.destroy();} catch(Exception err) {err.printStackTrace();}}
}
File[] txtFilesInFolder = chpFolder.listFiles(new FilenameFilter() {
#Override
public boolean accept(File dir, String name) {
if (name.endsWith(".chp.txt")){
return true;
}else{
return false;
}
}
});
if (txtFilesInFolder.length==chpFilesInFolder.length){
for (File chp : chpFilesInFolder) {
chp.delete();
}
File logFile = new File(chpFolder, "apt-chp-to-txt.log");
if (logFile.exists()){
logFile.delete();
}
}else{
throw new RuntimeException("CHPs have not all been transformed to TXT in "+chpFolder.getAbsolutePath());
}

Is it possible that your C program is producing output on stdout? If so, you need to read Process.getOutputStream() before Process.waitFor() returns - see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4254231
Alternatively, call your C program that a shell script that redirects stdout.
You can use the jstack command to confirm that the thread is indeed blocked at Process.waitFor().

You could have the main thread wait for a reasonable amount of time and then call some method on the MyThread class to kill the started process, thus causing the thread to finish.

as often I would suggest to use a more robust and professional point of view while using a messsaging solution to make your C program interact with your Java application, it will be easy and clean to avoid those non daemon threads waiting for ever because of the crash of your C application... now all brokers have a STOMP interface which is pretty cool for any kind of application to invoke (just use any Http library), broker configuration will enable to restart non finished jobs, to put some timeouts and so one..Even if JMS does not support request and response it's quite easy to implement such paradigm....
HTH
Jerome

If I understad correctly, your Java threads remain waiting after the C program crashes.
Make the spawned C process send heart beats. You can do this even by printing sth to console (or inserting in a table) and have the Java thread every so often wake up and check the heartbeat. If it's not there, assume the C process died and terminate the thread.

Launching external processes in Java can get a little bit tricky. I usually try to avoid them as you'll have to deal with different error codes and some terminal madness. I recommend you use specialized libraries such as commons-exec (http://commons.apache.org/exec/)