We Keep Coding

Is there any harm in failing to close a file when a Java program terminates?

When my program starts, it opens a file and writes to it periodically. (It's not a log file; it's one of the outputs of the program.) I need to have the file available for the length of the program, but I don't need to do anything in particular to end the file; just close it.
I gather that for file I/O in Java I'm supposed to implement AutoCloseable and wrap it in a try-with-resources block. However, because this file is long-lived, and it's one of a few outputs of the program, I'm finding it hard to organize things such that all the files I open are wrapped in try-with-resources blocks. Furthermore, the top-level classes (where my main() function lies) don't know about this file.
Here's my code; note the lack of writer.close():
public class WorkRecorder {
public WorkRecorder(String recorderFile) throws FileNotFoundException {
writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(recorderFile)));
}
private Writer writer;
public void record(Data data) throws Exception {
// format Data object to match expected file format
// ...
writer.write(event.toString());
writer.write(System.lineSeparator());
writer.flush();
}
}
tl;dr do I need to implement AutoCloseable and call writer.close() if the resource is an opened output file, and I never need to close it until the program is done? Can I assume the JVM and the OS (Linux) will clean things up for me automatically?
Bonus (?): I struggled with this in C#'s IDisposeable too. The using block, like Java's try-with-resources construct, is a nice feature when I have something that I'm going to open, do something with quickly, and close right away. But often that's not the case, particularly with files, when the access to that resource hangs around for a while, or when needing to manage multiple such resources. If the answer to my question is "always use try-with-resources blocks" I'm stuck again.

I have similar code that doesn't lend itself to being wrapped in a try-with-resources statement. I think that is fine, as long as you close it when the program is done.
Just make sure you account for any Exceptions that may happen. For example, in my program, there is a cleanup() method that gets called when the program is shut down. This calls writer.close(). This is also called if there is any abnormal behavior that would cause the program to shut down.
If this is just a simple program, and you're expecting the Writer to be open for its duration, I don't think it's really a big deal for it to not be closed when the program terminates...but it is good practice to make sure your resources are closed, so I would go ahead and add that to wherever your program may shut down.

You should always close resources or set them to null so it can be picked up by the garbage collector in Java. Using try-with-resource blocks is a great way to have Java automatically close resources when you're done with them. Even if you use it for the duration of the program, it is good programming practice to close it even at the end. Some might say you don't need to, I personally would say just go ahead and do it and here's why:
"When a stream is no longer needed, always close it using the close() method or automatically close it using a try-with-resource statement. Not closing streams may cause data corruption in the output file, or other programming errors."
-Introduction to Java Programming 10th Edition, Y. Daniel Liang
If possible, just run the .close() method on the resource at the very end of the program.

I (now) think a better answer is "It depends" :-). A detailed treatment is provided by Lukas Eder here. Also check out the Lambda EG group post.
But in general, it's a good idea to return the resource back to the operating system when you are done with it and use try-with-resources all the time (except when you know what you are doing).

Closing class IO resources in overridden finalize() method

If I have a class that utilizes an IO resource, such as a disk flat file, DB, or some other form of external resource, what are pros and cons of closing those streams/connections in an overridden finalize() method to be run by GC? I though this could leverage the existing JVM GC and reduce the exposure to the risk of relying on the client to invoke a class method called something like closeResources() as well as writing spaghetti-like try-catches (nested try-catches and ifs being my least favorite programming constructs).
As a concrete example, I have a simple file reading wrapper. The class is constructed with String filePath, it reads the file into a List<String[]> . I don't wan't to have to close the BufferedReader in multiple places like close it if there is a problem opening the file (catch clause) but also close it if the file reads fine etc. I want to put it in one place and make sure it is ALWAYS closed no matter what when the object gets GC.
Is this approach a good practice or am I trying to afford myself too high level a convenience within the scope of Java?

This is not a great idea as the finalize() method is not guaranteed to be called.
It's easier and better to just close the resources when your code is done with them.
If you hate writing the nested try-finally blocks to close the resources correctly, use something like commons-io's IOUtils to silently close the resources (or write your own simple util method to silently close them):
InputStream stream = ...;
try {
...
}
finally {
IOUtils.closeQuietly(stream);
}

When the IO resource is an instance variable, then you should close it in the finalize() method.
Why ?
Because beeing an instance variable, you need it in an open state because some method will be using it repeated times.
If you close it in a method other than finalize, then you are creating a temporal coupling, meaning the class user needs to know that he has to call certain methods in a certain temporal order, i.e, A before B etc.
EDIT:
Java documentation states that garbage collector is not guaranteed to run at any specific time, and will not run finalize() as long as there's any references to the object. If references linger, it's a memory leak, a programming error. finalize() is the best option when the resource is not local to a method. If the resource is local to a method, then close it in the finally end of a try/cath block.

Yes, finally block is always the best approach to release the resources such as connection, I/O Sreams etc.

Using flush() before close()

As per the java docs, invoking close() on any java.io Streams automatically invokes flush(). But I have seen in lot of examples, even in production codes, developers have explicitly used flush() just before close(). In what conditions we need to use flush() just before close()?

Developer get into a habit of calling flush() after writing something which must be sent.
IMHO Using flush() then close() is common when there has just been a write e.g.
// write a message
out.write(buffer, 0, size);
out.flush();
// finished
out.close();
As you can see the flush() is redundant, but means you are following a pattern.

I guess in many cases it's because they don't know close() also invokes flush(), so they want to be safe.
Anyway, using a buffered stream should make manual flushing almost redundant.

I want to point out an important concept that many previous comments have alluded to:
A stream's close() method does NOT necessarily invoke flush().
For example org.apache.axis.utils.ByteArray#close() does not invoke flush().
(click link to see source code)
The same is true more generally for any implementations of Flushable and Closeable. A prominent example being java.io.PrintWriter. Its close() method does NOT call flush().
(click link to see source code)
This might explain why developers are cautiously calling flush() before closing their streams. I personally have encountered production bugs in which close() was called on a PrintWriter instance without first calling flush().

The answers already provided give interesting insights that I will try to compile
here.
Closeable and Flushable being two independent traits, Closeable do not specify that close() should call flush(). This means that it is up to the implementation's documentation (or code) to specify whether flush() is called or not. In most cases it is the norm, but there is no guaranty.
Now regarding what #Fabian wrote: It is true that java.io.PrintWriter's close() method does not call flush(). However it calls out.close() (out being the underlying writer). Assuming out is a BufferedWriter, we are fine since BufferedWriter.close() is flushing (according to its doc). Had it be another writer, it may not have been the case...
So you have two choices:
either you ensure that at least one inner Writer/Stream flushes by itself (beware in case of code refactoring),
or you just call flush() and you're on the safe side all the time.
Solution 2, requiring less work, is my preferred one.

Java BufferedWriter close()

Assume that I have the following code fragment:
operation1();
bw.close();
operation2();
When I call BufferedReader.close() from my code, I am assuming my JVM makes a system call that ensures that the buffer has been flushed and written to disk. I want to know if close() waits for the system call to complete its operation or does it proceed to operation2() without waiting for close() to finish.
To rephrase my question, when I do operation2(), can I assume that bw.close() has completed successfully?

when I do operation2(), can I assume that bw.close() has completed successfully?
Yes

Close the stream, flushing it first. Once a stream has been closed, further write() or flush() invocations will cause an IOException to be thrown. Closing a previously-closed stream, however, has no effect.
Though the documentation does not say anything specifically, I would assume this call does block until finished. In fact, I'm pretty sure nothing in the java.io package is non-blocking.

The JavaDoc for the java.io.BufferedReader.close() is taken exactly from the contract if fulfills with the java.io.Reader.
The Doc says:
Closes the stream and releases any system resources associated with it. Once the stream has been closed, further read(), ready(), mark(), reset(), or skip() invocations will throw an IOException. Closing a previously closed stream has no effect.
While this makes no explicit claim of blocking until the file system is complete, with this same instance of BufferedReader all other operations will throw an exception if close() returns. Although the JavaDoc could be seen as ambiguous about when the operation completes, if the file system flush and close were not complete when this method returned it would violate the spirit of the contract and be a bug in Java (implementation or documentation).

NO! You cannot be sure for the following reason:
A BufferedWriter is a Wrapper for another Writer. A close() to the BufferedWriter just propagates to the underlying Writer.
IF this underlying Writer is an OutputStreamWriter, and IF the OutputStream is a FileOutputStream, THEN the close will issue a system call to close the file handle.
You are completely free to even have a Writer where close() is a noop, or where the close is implemented non-blocking, but when using only classes from java.io, this is never the case.

A Writer (or BufferedWriter) is a black box that writes a stream of characters somewhere, not necessarily to the disk. A call to close() must (by method contract) flush its buffered content before closing, and should (normally) block before all its "essential" work is done. But this would depend on the implementation and the environment (you cannot know about caches that are below the Java layer, for example). In what respects of the work to be done by the Java writer itself (eg: make the system call to write to disk, in the case of a FileWriter or akin, and close the filehandle) , yes, you can assume that when close() returns it has already done all its work.

In general with any i/o operation you can make no assumptions about what has happened after the write() operation completes, even after you close. The idea of delivery is a subjective concept relative to the medium.
For instance, what if the writer represents a TCP connection, and then the data is lost inbetween client and server? Or what if the kernel writes data to a disk, but the drive physically fails to write it? Or if the writer represents a carrier pigeon that gets shot en route?
Furthermore, imagine the case when the write has no way of confirming that the endpoint has received the data (read: udp/datagrams). What should the blocking policy be in that situation?

The buffer will have been flushed to the operating system and the file handle closed, so the Java operations required will have been completed.
BUT the operating system will have cached or queued the write to the actual disk, pipe, network, whatever - there is no guarantee that the physical write has completed. FileChannel.force() provides a way to do that for files on local disks: see the Javadoc.

Yes, IF you reach operation2();, the stream would've had to have been completely closed. However, close() throws IOException, so you may not even get to operation2();. This may or may not be the behavior that you expect.

flush in java.io.FileWriter

I have a question in my mind that, while writing into the file, before closing is done, should we include flush()??. If so what it will do exactly? dont streams auto flush??
EDIT:
So flush what it actually do?

Writers and streams usually buffer some of your output data in memory and try to write it in bigger blocks at a time. flushing will cause an immediate write to disk from the buffer, so if the program crashes that data won't be lost. Of course there's no guarantee, as the disk may not physically write the data immediately, so it could still be lost. But then it wouldn't be the Java program's fault :)
PrintWriters auto-flush (by default) when you write an end-of-line, and of course streams and buffers flush when you close them. Other than that, there's flushing only when the buffer is full.

I would highly recommend to call flush before close. Basically it writes remaining bufferized data into file.
If you call flush explicitly you may be sure that any IOException coming out of close is really catastrophic and related to releasing system resources.
When you flush yourself, you can handle its IOException in the same way as you handle your data write exceptions.

You don't need to do a flush because close() will do it for you.
From the javadoc:
"Close the stream, flushing it first. Once a stream has been closed, further write() or flush() invocations will cause an IOException to be thrown. Closing a previously-closed stream, however, has no effect."

To answer your question as to what flush actually does, it makes sure that anything you have written to the stream - a file in your case - does actually get written to the file there and then.
Java can perform buffering which means that it will hold onto data written in memory until it has a certain amount, and then write it all to the file in one go which is more efficient. The downside of this is that the file is not necessarily up-to-date at any given time. Flush is a way of saying "make the file up-to-date.
Close calls flush first to ensure that after closing the file has what you would expect to see in it, hence as others have pointed out, no need to flush before closing.

Close automatically flushes. You don't need to call it.

There's no point in calling flush() just before a close(), as others have said. The time to use flush() is if you are keeping the file open but want to ensure that previous writes have been fully completed.

As said, you don't usually need to flush.
It only makes sense if, for some reason, you want another process to see the complete contents of a file you're working with, without closing it. For example, it could be used for a file that is concurrently modified by multiple processes, although with a LOT of care :-)

FileWriter is an evil class as it picks up whatever character set happens to be there, rather than taking an explicit charset. Even if you do want the default, be explicit about it.
The usual solution is OutputStreamWriter and FileOutputStream. It is possible for the decorator to throw an exception. Therefore you need to be able to close the stream even if the writer was never constructed. If you are going to do that, you only need to flush the writer (in the happy case) and always close the stream. (Just to be confusing, some decorators, for instance for handling zips, have resources that do require closing.)

Another usecase for flushing in program is writing progress of longrunning job into file (so it can be stopped and restarted later. You want to be sure that data is safe on the drive.
while (true) {
computeStuff();
progresss += 1;
out.write(String.format("%d", progress));
out.flush();
}
out.close();