I have application (Java5) distributed among several JVMs that reads/writes files from a shared storage (the storage is managed by Windows). I wanted to use exclusive/shared locks for file write/read as following:
FileOutputStream fos = null;
FileLock lock = null;
ObjectOutputStream oos = null;
try {
fos = new FileOutputStream(new File("//share/test.dat")); // HERE IT MAY THROW FileNotFoundException...
lock = fos.getChannel().lock(); // ... and I won't acquire a lock.
oos = new ObjectOutputStream(fos);
oos.writeObject(value);
return true;
} catch (Exception e) {
// Log it.
} finally {
// Close locks and I/O streams.
}
The problem: if JVM1 is currently reading the file test.dat and JVM2 is trying to write to the same file then FileNotFoundException ("The process cannot access the file because it is being used by another process") will be thrown on JVM2 in "new FileOutputStream(new File("//share/test.dat"))" line.
It seems to be Catch22 case: on the one hand I want to acquire a lock to get I/O stream; on the other hand I need to have an I/O stream to acquire a lock.
As I could see, the same situation with RandomAccessFile.
Any ideas?..
As you can see from the error message, Windows just won't let you do what you're trying to do. You can treat the Windows error itself as the lock condition, if you think about it. You would have a cognate problem here anyway because of the timing window between the open and the lock. Windows in a way is doing you a favor here. For once.
Related
I have a bunch of files on a local file system. My server will serve those files. In some cases the server will receive an instruction to delete a file.
At the moment I'm using FileChannel.lock() to acquire a lock on the file, this is mostly to make sure that some other process isn't editing the file when I try to delete it.
If I successfully acquire the lock, can I delete the file straight away, or do I need to release the lock first?
like this:
FileOutputStream out = new FileOutputStream(file);
FileChannel channel = out.getChannel();
FileLock lock = channel.lock();
if(lock.isValid() && !lock.isShared()){
Path filePath = Paths.get(file.getPath());
Files.delete(filePath);
}
Do I need to release the lock after I've deleted the file?
Or should it be like this (lock.release() added):
FileOutputStream out = new FileOutputStream(file);
FileChannel channel = out.getChannel();
FileLock lock = channel.lock();
if(lock.isValid() && !lock.isShared()){
lock.release();
Path filePath = Paths.get(file.getPath());
Files.delete(filePath);
}
EDIT:
So it turns out the code above wouldn't work anyway, because you can't modify the file with a FileInputStream because, of course, it's read only. I've modified the code above to use FileOutputStream instead, but it still doesn't quite work, because even though I release the lock from the channel, the file object still has a lock. So I modified the code like so:
FileOutputStream out = new FileOutputStream(file);
FileChannel channel = out.getChannel();
FileLock lock = channel.lock();
if(lock.isValid() && !lock.isShared()){
channel.close();
boolean deleted = file.delete();
logger.info("{} #{} File {} deleted: {}", id, type, file.getName(), deleted);
}
This seems to work as expected. I'd still like to know if this is safe, or if there's a better way to do this?
File lock wont work while deleting the file. File lock will work only while some other process wants to edit the same file. Before deleting the file , you need to release the lock and close the channel
Do I need to close a FileOutputStream in the following example? And why?
FileOutputStream fos = new FileOutputStream("bytes.info");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(data);
oos.close();
If there were no exceptions thrown, then the FileOutputStream would be closed by ooo.close().
An exception thrown in writeObject would prevent any of the streams from being closed. So the close call should be in a finally block.
There's the additional problem that the ObjectOutputStream could throw an exception in its constructor. It writes the stream header in the constructor which can cause an exception. In this case, the FileOutputStream needs to be closed, but calling oos.close() is not possible because there's no reference to the ObjectOutputStream. So you really need two separate calls to close, one for each stream, both in finally blocks.
Using try-with-resources takes care of all of this for you:
try(
FileOutputStream fos = new FileOutputStream("bytes.info");
ObjectOutputStream oos = new ObjectOutputStream(fos)
) {
oos.writeObject(data);
}
Yes, you need to close the stream. Leaving FileOutputStream unclosed creates a possibility that some data that has been successfully written to the stream does not get saved to the file. If a program opens multiple file streams, not closing them creates a possibility of failures due to running out of native resources (too many files opened simultaneously).
FileOutputStream manages native resources, which are released by the close method. The class has a finalizer, too, which releases resources as well. As part of releasing native resources, the stream finishes out the writing of buffered data, if there is any. However, since JVM does not guarantee that a finalizer is going to be called on every object, failure to call close creates a risk of leaving buffered data unwritten.
Of course you have to close the FileOutputStream file. If not sometimes the data you save into the file might not get saved and you will end up with an empty file after executing the program. And you might wanna use try with resources so you won't have to close it manually and can do the exception handling part both at once.
try (FileOutputStream fos = new FileOutputStream("bytes.info");
ObjectOutputStream oos = new ObjectOutputStream(fos));{
}catch(){}
Just in case if you are not using try with resources close the file streams in the finally block manually.
FileOutputStream fos = null;
ObjectOutputStream oos = null;
try{
fos = new FileOutputStream("bytes.info");
oos = new ObjectOutputStream(fos));
oos.writeObject(data);
}catch(){
}finally{
if(fos != null){
fos.close();
}
if(oos != null){
oos.close();
}
}
It is a must to check whether those file streams are null or not. Because if they are null then there will be another error. Still it's better to use try with resources.
I'm using the following code to write files to the disk.
`try{
FileOutputStream fileOutputStream = null;
fileOutputStream = new FileOutputStream(filePath);
fileOutputStream.write(fileData);
fileOutputStream.flush();
}
finally{
fileOutputStream.close();
}
`
The problem is that I'm getting the following error intermittently:
Insufficient system resources exist to complete the requested service.
I have already checked a few cases when this problem can happen, like the lack of Paged Pool Memory, but none of them is my case. I'm using windows server 2003 Server R2 SP2. Architecture x86.
Should I try to write the file in smaller chunks? What is the best way to do that?
A few things.
First, you should consider using buffers. Try wrapping your FileOutputStream with a BufferedOutputStream.
try{
BufferedOutputStream outputBuffer = null;
outputBuffer = new BufferedOutputStream (new FileOutputStream(filePath));
outputBuffer.write(fileData);
outputBuffer.flush();
}
finally{
outputBuffer.close();
}
Second, try checking if you really are running out of handles. I left a comment with a link regarding this.
My application writes to Excel files. Sometimes the file can be used, in that case the FileNotFoundException thrown and then I do not know how to handle it better.
I am telling the user that the file is used and after that message I do not want to close the application, but to stop and wait while the file is available (assuming that it is opened by the same user). But I do not understand how to implement it. file.canWrite() doesn't work, it returns true even when the file is opened, to use FileLock and check that the lock is available I need to open a stream, but it throws FileNotFoundException (I've been thinking about checking the lock in a busy wait, I know that it is not a good solution, but I can't find another one).
This is a part of my code if it can help somehow to understand my problem:
File file = new File(filename);
FileOutputStream out = null;
try {
out = new FileOutputStream(file);
FileChannel channel = out.getChannel();
FileLock lock = channel.lock();
if (lock == null) {
new Message("lock not available");
// to stop the program here and wait when the file is available, then resume
}
// write here
lock.release();
}
catch (IOException e) {
new Message("Blocked");
// or to stop here and then create another stream when the file is available
}
What makes it more difficult for me is that it writes to different files, and if the first file is available, but the second is not, then it will update one file and then stop, and if I restart the program, it will update it again, so I can't allow the program to write into files until all of them are available.
I believe that there should be a common solution, since it must be a common issue in Windows to deal with such cases, but I can't find it.
To wait until a file exists you can make a simple loop:
File file = new File(filename);
while (!file.exists()) {
try {
Thread.sleep(100);
} catch (InterruptedException ie) { /* safe to ignore */ }
}
A better solution could be using WatchService but it's more code to implement.
The File.canWrite method only tells you if a path can be written to; if the path names a file that doesn't exist it will return false. You could use the canRead method instead of exists in a loop like above.
To use a file locks, the file has to exist first, so that wouldn't work either.
The only way to be sure you can write to a file is to try to open it. If the file doesn't exist, the java.io API will create it. To open a file for writing without creating you can use the java.nio.file.Files class:
try (OutputStream out = Files.newOutputStream(file.toPath(),
StandardOpenOption.WRITE))
{
// exists and is writable
} catch (IOException) {
// doesn't exist or can't be opened for writing
}
I'm trying to delete a file that another thread within my program has previously worked with.
I'm unable to delete the file but I'm not sure how to figure out which thread may be using the file.
So how do I find out which thread is locking the file in java?
I don't have a straight answer (and I don't think there's one either, this is controlled at OS-level (native), not at JVM-level) and I also don't really see the value of the answer (you still can't close the file programmatically once you found out which thread it is), but I think you don't know yet that the inability to delete is usually caused when the file is still open. This may happen when you do not explicitly call Closeable#close() on the InputStream, OutputStream, Reader or Writer which is constructed around the File in question.
Basic demo:
public static void main(String[] args) throws Exception {
File file = new File("c:/test.txt"); // Precreate this test file first.
FileOutputStream output = new FileOutputStream(file); // This opens the file!
System.out.println(file.delete()); // false
output.close(); // This explicitly closes the file!
System.out.println(file.delete()); // true
}
In other words, ensure that throughout your entire Java IO stuff the code is properly closing the resources after use. The normal idiom is to do this in the try-with-resources statement, so that you can be certain that the resources will be freed up anyway, even in case of an IOException. E.g.
try (OutputStream output = new FileOutputStream(file)) {
// ...
}
Do it for any InputStream, OutputStream, Reader and Writer, etc whatever implements AutoCloseable, which you're opening yourself (using the new keyword).
This is technically not needed on certain implementations, such as ByteArrayOutputStream, but for the sake of clarity, just adhere the close-in-finally idiom everywhere to avoid misconceptions and refactoring-bugs.
In case you're not on Java 7 or newer yet, then use the below try-finally idiom instead.
OutputStream output = null;
try {
output = new FileOutputStream(file);
// ...
} finally {
if (output != null) try { output.close(); } catch (IOException logOrIgnore) {}
}
Hope this helps to nail down the root cause of your particular problem.
About this question, I also try to find out this answer, and ask this question and find answer:
Every time when JVM thread lock a file exclusively, also JVM lock
some Jave object, for example, I find in my case:
sun.nio.fs.NativeBuffer
sun.nio.ch.Util$BufferCache
So you need just find this locked Java object and analyzed them and
you find what thread locked your file.
I not sure that it work if file just open (without locked exclusively), but I'm sure that is work if file be locked exclusively by Thread (using java.nio.channels.FileLock, java.nio.channels.FileChannel and so on)
More info see this question