We Keep Coding

How to deal with a Java Processes grandchilds when reading the output?

Working on a JavaFX mockup of a commandline I encountered the following problem:
If I run a process (for example a batchfile) that runs another process (for example opens the notepad with a simple start notepad) I cannot seem to properly determine when the batch-file is done executing:
Process#waitFor already returns when the batchfiles is started (I guess because I have to add cmd /c in front of the executable and cmd really is done after a short fraction of a second)
Reading the output using the Process#getInputStream only ends after I close the notepad not after the batchfile terminates.
Is there a method I keep missing? And more importantly: How can I determine the end of the process spawned through cmd /c if at all?
Reproducible example:
example.bat:
#echo off
start notepad
REM You can change the path to something else but it should be something where tree produces a longer output to reproduce the problem.
cd %USERPROFILE%\Desktop
tree
JavaCode:
import java.io.IOException;
import java.io.InputStream;
public class Main {
public static void main(String[] args) {
Process myProcess = null;
try {
myProcess = Runtime.getRuntime().exec("cmd /c C:\\Users\\geisterfurz007\\Desktop\\example.bat");
} catch (IOException e) {
e.printStackTrace();
}
startReadingThread(myProcess);
try {
myProcess.waitFor();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Process ended");
}
private static void startReadingThread(Process myProcess) {
InputStream stream = myProcess.getInputStream();
new Thread(() -> {
int character;
try {
while ((character = stream.read()) != -1) {
System.out.write(character);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
System.out.println("Reading ended");
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
}
}
starts printing the tree and mid printing "Process ended" is written to the console. Lastly "Reading ended" is printed after I close the notepad window.
My goal is to find the point where tree is done printing (ie the batchfile done processing) ignoring when the notepad was closed.
Based on Leviands's answer I tried to close the Processes streams once it was done executing to no avail.
The InputStream closes unfortunately mid content again and the ErrorStream (which I read in the actual application as well) won't close thus blocking the thread.

First of all a huge thanks to Leviand who mentioned InputStream#available in their answer which got me something that actually appears to work:
The idea is that at the point in time I am looking for, Process#isAlive should return false as the Stream takes longer to process than the Process processes (if that makes sense) while there should be no characters readable from the InputStream, so InputStream#available should return 0.
This leads to this piece of code:
import java.io.IOException;
import java.io.InputStream;
public class Main {
public static void main(String[] args) {
Process myProcess = null;
try {
myProcess = Runtime.getRuntime().exec("cmd /c C:\\Users\\geisterfurz007\\Desktop\\example.bat");
} catch (IOException e) {
e.printStackTrace();
}
startReadingThread(myProcess).start();
}
private static Thread startReadingThread(Process myProcess) {
InputStream stream = myProcess.getInputStream();
return new Thread(() -> {
int character;
try {
while (myProcess.isAlive() || stream.available() > 0) {
if ((character = stream.read()) == -1) {
break;
}
System.out.write(character);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
});
}
}
With the code above I am able to achieve reading from the Processes stream ignoring any grandchild processes.
After some time using this, there are a few things to address:
I moved to using an InputStreamReader now for one main reason: I can specify encoding. All occurences of stream.available() > 0 would then have to be replaced with reader.ready().
This eats a bunch of resources when idling! It makes sense to have the thread sleep for a few milliseconds if there is nothing to read before attempting to read again.
At least in my usecase where I send each character to my GUI one by one, this kills the GUI pretty quickly for longer outputs. Consider some kind of buffer for the output before further processing on your primary thread.

You are launching startReadingThread(myProcess);, then you are telling with myProcess.waitFor(); to wait that process myProcess is ended before printing System.out.println("Reading ended"); , which is the opposite that you are willing to do.
The process that should "block" the other from starting is the startReadingThread.
The problem also look in the while loop, which was not correct.
I would change in something like (not sure on stream.available() != 0, but it's working for testing):
private static Thread startReadingThread(Process myProcess) {
InputStream stream = myProcess.getInputStream();
return new Thread(() -> {
int character;
try {
while (stream.available() != 0) {
if((character = stream.read()) == -1) {
System.out.write(character);
stream.close();
break;
}
System.out.write(character);
}
stream.close();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
System.out.println("Reading ended");
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
});
}
Then edit the main with:
public static void main(String[] args) {
Process myProcess = null;
try {
myProcess = Runtime.getRuntime().exec("cmd /c C:\\Users\\geisterfurz007\\Desktop\\example.bat");
} catch (IOException e) {
e.printStackTrace();
}
try {
if(myProcess.waitFor() == 0){
startReadingThread(myProcess).start();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Process ended");
output (I'm from italy):
Connected to the target VM, address: '127.0.0.1:56607', transport:
'socket' Process ended Elenco del percorso delle cartelle per il
volume OSDisk Numero di serie del volume: 12DA-8173 C:. Non esistono
sottocartelle
Reading ended Disconnected from the target VM, address:
'127.0.0.1:56607', transport: 'socket'
Process finished with exit code 0
EDIT:
Here's the test on a folder with subfolders inside (I'm attaching screenshot cause of strange symbols)
And this is the folder content:

exception IOException is never thrown in body of corresponding try statement

I am not too stupid to realize that the code in the 'try catch' cannot throw an exception. But I am stuck here because I have copied the code exactly from the book so it must have been correct when published. It's left me wondering if the code can't compile anymore because it's out of date? I realize the nature of this question may offend some people here. If so please just don't be TOO harsh in your stern reprimands.
I am getting the error :
./StreamCopier.java:13: error: exception IOException is never thrown in body of corresponding try statement
} catch (IOException e) {System.err.println(e);}
//FileTyper.java
//Reads filenames from the command line and copies named files onto System.out.
import java.io.*;
public class FileTyper {
public static void main (String[] args) {
if (args.length == 0) {
System.err.print("usage: java FileTyper file1, file2");
return;
}
for (int i = 0 ; i < args.length ; i++) {
try {
typeFile(args[i]);
if (i+1 < args.length) { //more files waiting to be processed
System.out.println();
System.out.println("********************************************");
}
} catch (IOException e) {System.err.println(e);}
} //end for
} //end main()
public static void typeFile (String filename) throws IOException {
FileInputStream fin = new FileInputStream(filename);
StreamCopier.copy(fin, System.out);
fin.close();
}
}
//StreamCopier.java
//hard to know what this program does exactly
import java.io.*;
public class StreamCopier {
public static void main (String[] args) {
try {
} catch (IOException e) {System.err.println(e);}
}
public static void copy (InputStream in, OutputStream out) throws IOException {
//do not allow other threads to read from the input or
//write to the output while copying is taking place.
synchronized (in) {
synchronized (out) {
byte [] buffer = new byte [256];
while (true) {
int bytesRead = in.read(buffer);
if (bytesRead == -1) break;
out.write(buffer, 0, bytesRead);
}
}
}
} //end copy()
}

The error is because of the empty try {} catch {} block in the main method of StreamCopier.java. Remove the empty try{} catch{} block and then try. It should work.
public static void main (String[] args) {
try {
} catch (IOException e) {System.err.println(e);} // here is the problem
}

The try block is empty, apparently because you haven't written the code that goes in it yet. The Java compiler stupidly complains if there is a catch block and [it thinks] the exception cannot be thrown. This is one of the annoying qualities of Java's checked exceptions.
Simply continue to fill in the try block, and the error will go away. You can insert a dummy copy(null, null); inside it to make the compiler shut up.

JUnit and file write access

So I have a method to write a string to a file:
public static void saveStringToFile(String path, String string) {
File file = new File(path);
if (!file.exists()) {
try {
file.createNewFile();
} catch (IOException e) {
e.printStackTrace();
}
}
FileWriter out = null;
try {
out = new FileWriter(path);
out.write(string);
if (out != null) {
out.close();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
And my test class with the following setUp method which runs before each test (to delete the testfile before each one):
public static final String TEST_FILE = "somefile.xml";
//...
#Before
public void setUp() throws IOException {
if (MyCustomClass.fileExists(TEST_FILE)) {
new File(TEST_FILE).delete();
}
}
Each of my test tries to write something to the file using the method saveStringToFile(). It succeeds like for a couple of times, but a some random point I finally get the java.io.IOException: Access is denied. Got no idea why this happens - sometimes it occurs in test1, sometimes in test3...
It was working OK, when I was using Java7 FileIO, but I needed to migrate back to Java6...

Are you testing that you are able to create, write to and delete a file, or are you testing what is written to the file?
If the latter, then perhaps you should be mocking/overriding the saveStringToFile( ... ) method and instead focus on verifying that the code you're unit testing actually produces the correct output.
If the former, then I quite agree with #Omaha's suggestion that your test runner is likely running several tests in parallel.
Hope that helps.

There's some problems with the exception handling. The call to out.close() should be within a separate try-catch block inside a finally block. If an exception is thrown when writing to the file, the file is never closed.
I would recommend looking at something like Apache Commons IO which has many useful IO methods like FileUtils.writeStringToFile().

So, probably JUnit wasn't running it parrallel, cause as I suppose It doesn't do it by default.
The problem was in my readfile method:
private String readFile(String path) throws FileNotFoundException {
return (new Scanner(new File(path))).useDelimiter("\\Z").next();
}
To work fine I had to fix
private String readFile(String path) throws FileNotFoundException {
Scanner scanner = (new Scanner(new File(path)));
String s = scanner.useDelimiter("\\Z").next();
scanner.close();
return s;
}
The close() method for Scanner was the key...

Can't delete the file after writing some data to it, where is wrong?

I'm write some text a file then delete it, but the deletion is failed.
The code is very simple:
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
public class TestFile {
public static void main(String[] args) throws IOException {
File file = new File("c:\\abc.txt");
writeFile(file, "hello");
// delete the file
boolean deleted = file.delete();
System.out.println("Deleted? " + deleted);
}
public static void writeFile(File file, String content) throws IOException {
OutputStream out = null;
try {
out = new FileOutputStream(file);
out.write(content.getBytes("UTF-8"));
} catch (IOException e) {
try {
out.close();
} catch (IOException e1) {
// ignored
}
}
}
}
The output is:
Deleted? false
And there is a file abc.txt contains hello still there under c:.
Then I use FileUtils.writeStringToFile(...) from commons-io.jar instead, the file will be deleted.
But I don't know where is wrong with my code, please help me to find it out.

You are only closing the file if you get an IOException.
Change it to a finally block and you will be able to close and delete the file.
public static void writeFile(File file, String content) throws IOException {
OutputStream out = new FileOutputStream(file);
try {
out.write(content.getBytes("UTF-8"));
} finally {
try {
out.close();
} catch (IOException ignored) {
}
}
}

You need to close your OutputStream when you finished writing the file.
try {
out = new FileOutputStream(file);
out.write(content.getBytes("UTF-8"));
out.close();
} catch (IOException e) {
try {
out.close();
} catch (IOException e1) {
// ignored
}
}

In your main method,
public static void main(String[] args) throws IOException {
File file = new File("c:\\abc.txt");
writeFile(file, "hello");
// delete the file
boolean deleted = file.delete();
System.out.println("Deleted? " + deleted);
}
You open the file, write to it and then do not close it. Java keeps the file open for you, so if you wanted to add more information to it, you could. However, to be able to delete the file, you need to make sure no other reference is open to it. You can do this by using file.close() to close the file handle Java reserves for you.
It's best practice to always close a stream when you are done with it, especially if you added data to it. Otherwise, you might run into situations where you are keepings files open by accident, or, in extreme cases, lose data you thought was saved already.

Have a look at what FileUtils.writeStringToFile() does that you haven't.
public static void writeStringToFile(File file, String data, String encoding) throws IOException {
OutputStream out = new java.io.FileOutputStream(file);
try {
out.write(data.getBytes(encoding));
} finally {
IOUtils.closeQuietly(out);
}
}
You will note that the out stream is always closed, wheras in your example it only gets closed in your catch block if the write() throws an exception.
On Windows, files that are open by any program cannot be deleted.

You just delete your file if an exception occurs. You need to do that every time, after you opened the file.
You may want to put close into a finally block.
If you're using Java 7 I consider using a try-with-ressources block, which takes care of closing files for you.
try (BufferedReader br = new BufferedReader(new FileReader(path)))
{
return br.readLine();
}

FileOuputStream Write bytes for loop, but no data write?

Listing my program fragment as below
public class InStream {
static FileOutputStream file=null;
static {
try {
file = new FileOutputStream("deo.txt");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
try {
//when i try to replace below infinite loop,
//it is also not able to output my String
//while(ture)
or
//for(;;)
for(int i=0;i<100000;i++){
file.write("AB ".getBytes());
}
//file.flush();
file.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
Run this program -> open deo.txt -> there are no data within this file
but when i comment the for loop just only test below fragment code:
try {
file.write("AB ".getBytes());
file.close();
} catch (Exception e) {
e.printStackTrace();
}
Now i can see the "AB " string in the file. so strange....
Can any one do me a favor?

There is no error in your code. File "deo.txt" must be generate which contains AB AB...............

I tested your code. And it works. But for the deo.txt. You can check its size if it is about 293k. It displays nothing if you open it with Eclipse text editor. But you can view it with other system editor, such as notepad++.