We Keep Coding

How to check whether a PrintStream is open without printing to it

I have a GUI program written in Java which outputs data to the command line using System.out.println. The data is intended to be piped into another program. As an example, I'll pipe the program through head:
$ java MyProgram | head -n10
I want my program to exit when the pipe is broken: in this case, this should happen after MyProgram has printed out ten lines of text.
There is a similar question on this site, and the solution given there works fairly well; for example, I could have the following running in its own thread:
while(!System.out.checkError()) {
// sleep 100ms
}
System.exit(0);
The problem is that PrintStream.checkError() seems to return true only after you have tried to print to the stream and failed. For that reason, my program does not in fact exit until it has printed out eleven lines of text: the pipe is broken after the first ten, but System.out continues to return true until I try to pipe through the eleventh line.
Printing out extra 'junk' lines in order to trigger an error on the PrintStream is out of the question, since the program on the right hand side of the pipe may be very sensitive to the data it receives.
Calling System.out.flush() inside the loop has no effect on PrintStream.checkError(), even though the source code for PrintStream indicates that it ought to call the private ensureOpen method in that class.
How can I reliably test whether System.out is open without printing anything to it?
'Real world' example: suppose that I have some program consumer that takes in command line input and does something with it. Certain pieces of input will call consumer to fail silently. Since the input to consumer is sometimes long and abstruse, I write a GUI program InputProvider in Java where I can click buttons and have the corresponding commands printed out to stdout. If I pipe the output of InputProvider into consumer, then I am able to control consumer graphically.
Unfortunately, there seems to be no way for InputProvider to notify the user when consumer has shut down, except by attempting to write to consumer and getting an exception of some kind.

I don't think you can fix this in Java. There's nothing wrong with System.out until you write to it and it fails. Another process (head) ended but the Java process can't know about that.
So I think that you have two options.
don't use pipe to head but limit the output in your Java code - then you'll know when to stop
accept that the last line will fail and handle the exception appropriately

Try to set a new stream for the system that way you'll be able to check if that stream is closed or not: OutputStream output = new FileOutputStream("c:\\data\\system.out.txt"); and PrintStream printOut = new PrintStream(output);. Then set it here: System.setOut(printOut); and inside the if you can check if(printOut.checkError())

File cannot be deleted because the JVM holds it - a tricky one

My post got a little too long, sorry. Here is a summary:
File on disk cannot be deleted ("the JVM holds the file" error). both when deleting from the java code and when trying to manually delete the file from windows.
All streams to that file are closed and set to null. All file objects set to null.
The program does nothing at that point; but waiting 30 minutes allows me to deleted the file from windows. Weird. Is the file not used by java anymore? Plus, since nothing happens in the program, it indicates it cannot be some stream I forgot (plus, I triple checked nothing is open).
Invoking System.gc() seemed to work when files were small. Did not help when they got to about 20MB.
[EDIT2] - I tried writing some basic code to explain, but its tricky. I am sorry, I know it's difficult to answer like that. I can however write how I open and close streams, of course:
BufferedWriter bw = new BufferedWriter(new FileWriter(new File("C:\\folder\\myFile.txt")));
for(int i = 0; i < 10; i++)
{
bw.write("line " + i);
bw.newLine();
}
bw.close();
bw = null;
If I've used a file object:
File f = new File("C:\\folder\\myFile.txt");
// use it...
f = null;
Basic code, I know. But this is essentially what I do.
I know for a fact I've closed all streams in this exact way.
I know for a fact that nothing happens in the program in that 30-minutes interval in which I cannot delete the file, until I somehow magically can.
thank you for your input even without the coherent code.
I appreciate that.
Sorry for not providing any specific code here, since I can't pinpoint the problem (not exactly specific-code related). In any case, here is the thing:
I have written a program which reads, writes and modifies files on disk. For several reasons, the handling of the read/write is done in a different thread, which is constantly operating.
At some point, I am terminating the "read/write" thread, keeping only the main thread - it waits for input from a socket, totally unrelated to the file, and does nothing. Then, I try to delete the file (using either File.delete(), even tried nio.Files delete option).
The thing is - and it's very weird - sometimes it works, sometimes it doesn't. Even manually, going to the folder and trying to delete the file via windows, gives me the "The file is open by the JVM" message.
Now, I am well aware that keeping references from all kinds of streams to the file prevents me from deleting it. Well past that by now :)
I have made sure that all streams are closed. I even set their values to null, including any "File" objects I have used (even though it shouldn't make any difference). All set to null, all closed. And the thread which generates all of them - the "read/write" thread - well, it's terminated since it got the the end of its run() method.
Usually, if I wait about 30 minutes, while the JVM still operates, I can delete the file manually from windows. The error magically disappears. When the JVM is closed, I can always delete the file right away.
I am lost here. Tried specifically invoking System.gc() before trying to delete the file, even called it like 10 times (not that it should matter). Sometimes it helped, but on other occasions, for example, when the file got larger (say 20MB), that didn't help.
What am I missing here?
Obviously, this couldn't be my implicit fault (not closing some stream), since the read/write thread is dead, the main thread awaits something unrelated (so the program is at a "standstill"), I have explicitly closed all streams, even nullified the references (inStream = null), invoked the garbage collector.
What am I missing? Why is the file "deletable" after 30 minutes (nothing happens at that time - not something in my code). Am I missing some gentle reference/garbage collection thingy?

What you're doing just calls for problems. You say that "if an IOexception occurred, it is printed immediately" and it may be true, but given that something inexplicable happens, let's better doubt it.
I'd first ensure that everything gets always closed, and then I'd care about related logic (logging, exiting, ...).
Anyway, what you did is not how resources should be managed. The answer above is not exactly correct either. Anyway, try-with-resources is (besides #lombok.Cleanup) about the only way, clearly showing that nothing gets ever left open. Anything else is more complicated and more error-prone. I'd strongly recommend using it everywhere. This may be quite some work, but it also forces you to re-inspect all the critical code pieces.
Things like nullifying references and calling the GC should not help... and if they seem to do, it may be a chance.
Some ideas:
Are you using memory mapped files?
Are you sure System.exit is not disabled by a security manager?
Are you running an antivirus? They love to scan files just after they get written.
Btw., locking files is one reason why the WOW never started for me. Sometimes the locks persisted long after the culprit was gone, at least according to tools I could use.

Are you closing your streams in a try...finally or try(A a = new A()) block? If not the streams may not be closed.
I would strongly recommend using either Automatic Resource Block Management ( try(A a = new A()) ) or a try...finally block for all external resources.
try(BufferedWriter br = new BufferedWriter(new FileWriter(new File("C:\\folder\\myFile.txt")));
for(int i = 0; i < 10; i++)
{
br.write("line " + i);
br.newLine();
})

Is there an error/status log for java? My program just stops running without any output

I'm trying to benchmark execution time for a program I created. As I add more data for it to process it takes longer and I am trying to find out how long exactly and a few details about the program(number of loops, time for each loop,etc..) so I created a system.out.println... of the status messages I want. This program works with a small number of variables quickly, but to run it on a large amount I do: java my_program > results.txt &
I see the job running for about 2 hours than it vanishes, so assuming its finished I try to cat results.txt but its blank. If java was crashing, wouldn't the error message be in there also(I think I'm capturing all output)? If not, is there a place I can find this information?

You redirected stdout to a file, but any errors would have been written to strderr, which was not redirected. See http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-3.html for an example of how to redirect both stdout and stderr

You can do something like this
java my_program 1>results.txt 2>error_log.txt &
This would actually redirect your errors to error_log.txt and your sysouts to results.txt

How to Wait for windows process to finish before opening file in java

I have a implemented a listener that notifies if we receive a new file in a particular directory. This is implemented by polling and using a TimerTask.
Now the program is so set up that once it receives a new file it calls another java program that opens the file and validates whether it is the correct file. My problem is that since the polling happens a specified number of seconds later there can arise a case in which a file is being copied in that directory and hence is locked by windows.
This throws an IOException since the other java program that tries to open it for validation cannot ("File is being used by another process").
Is there a way I can know when windows has finished copying and then call the second program to do the validations from java?
I will be more than happy to post code snippets if someone needs them in order to help.
Thanks

Thanks a lot for all the help, I was having the same problem with WatchEvent.
Unfortunately, as you said, file.canRead() and file.canWrite() both return true, even if the file still locked by Windows. So I discovered that if I try to "rename" it with the same name, I know if Windows is working on it or not. So this is what I did:
while(!sourceFile.renameTo(sourceFile)) {
// Cannot read from file, windows still working on it.
Thread.sleep(10);
}

This one is a bit tricky. It would have been a piece of cake if you could control or at least communicate with the program copying the file but this won't be possible with Windows I guess. I had to deal with a similar problem a while ago with SFU software, I resolved it by looping on trying to open the file for writing until it becomes available.
To avoid high CPU usage while looping, checking the file can be done at an exponential distribution rate.
EDIT A possible solution:
File fileToCopy = File(String pathname);
int sleepTime = 1000; // Sleep 1 second
while(!fileToCopy .canWrite()){
// Cannot write to file, windows still working on it
Sleep(sleepTime);
sleepTime *= 2; // Multiply sleep time by 2 (not really exponential but will do the trick)
if(sleepTime > 30000){
// Set a maximum sleep time to ensure we are not sleeping forever :)
sleepTime = 30000;
}
}
// Here, we have access to the file, go process it
processFile(fileToCopy);

I think you can create the File object and then use canRead or canWrite to know whether file ready to be used by the other java program.
http://docs.oracle.com/javase/6/docs/api/java/io/File.html
Other option is to try to Open file on first program and if it throws the exception then dont call the other java program. But I ll recommend the above 'File option.

Why does my application stop running, while it has a while(true) loop?

I have an application that is meant to "run forever". To achieve that, my code is under a while(true) loop.
while(true){
//My code is here
//What it does is that it calls some Url,
//gets xml, parses it and stores the values to a db.
}
This application runs as a java application and runs on jdk 5 on Redhat Linux. After a day or two, I realize that the log has activities of even 5 days ago, meaning that it stopped working 5 days ago. But when I check the java processes in the machine, this application shows that it is running.
Question: Why is this so??

A probable cause is that an exception is being thrown while processing those urls which is cought and consumed by your code, so the processing is failing but the application continues running. Unfortunately you have shown only comments, not any real code, so this is only a hypothesis.

There are many possibilities, but without more code I can't precisely narrow it down to which one specifically it is. Here are some thoughts:
There could be a break statement somewhere in the loop that explicitly leaves the loop.
If your loop is in a function and something in the loop returns from the function, you would leave the loop.
If something in the loop throws an uncaught exception, the loop would terminate. This also includes things like OutOfMemoryErrors or StackOverflowErrors.
The JVM might have crashed overnight due to an unusual bug.
An external process killed the JVM.
The system rebooted to install an update or because someone logged in and ran sudo reboot now.
The laws of boolean arithmetic changed, and now true == false. :-)

Log the exception:
try{
while(true){
// ......
}
}catch(Throwable t){
//log the Throwable object
}