I have 2 distinct tomcat7 servers, on each the same web service is deployed. Each web service request has to update a file on a shared flesystem (typically a NAS).
The file writing has to be synchronized to avoid conflict within the JVM of a tomcat server but also between JVM of the 2 servers.
I tried several methods :
=> A synchronized block for the file writing works only in the same JVM.
=> A FileLock works for each JVM but not within the JVM of a server (a file can be locked only once).
And how about using the Java7 nio file system ? can this help me ?
What is the best way to solve this problem ?
PS : sorry if my english is bad...
I can think of two approaches:
Use synchronized (or a Lock) and also FileLock; i.e. first obtain a local lock on a unique object that denotes the canonicalized absolute pathname, and then acquire a FileLock.
Use a database; e.g. lock a row in a database table which has the canonicalized absolute pathname as a unique key.
And how about using the Java7 nio file system ? can this help me ?
No. The problem is that file locking is OS functionality, and the behaviour (not locking against other threads in the same process) is fundamental file locking semantics ... across a range of different operating systems.
Related
I am using Java's file locking API on a Linux server machine and try to lock a file on a remote Linux NFS system. There are some issues that have popped and logs show that 2 different cluster nodes running the same Java webserver app are able to both acquire a lock on the same NFS file.
Reading online about how Java handles locks and Linux file locking (we usually deploy on Windows servers so there is very little Linux experience), what we do should work. We are essentially issuing advisory locks but since both cluster nodes run the same code they are cooperating processes and they check for lock acquisition before starting to do any read-write ops. However this does not seem to be the case and both systems are able to successfully acquire a lock on the file, concurrently.
Are those known issues? Many comments/articles online declare NFS file locking as unstable and should be avoided. Why is that? How would network connectivity issues (e.g. sudden communication drops) influence this behavior? Linux kernel version should be quite recent.
#StephenC so after some more testing, when calling RandomAccessFile.getChannel().tryLock() from a java main method it works fine over nfs4 but when the same code runs within Tomcat (8.5.68) multi-locking occurs.
OK. So I think I understand the root of your problem now. From what you have said, it sounds to me like you have are trying to use FileLock to stop one thread of your Tomcat JVM from locking a section of a file while another Tomcat thread has it locked.
That's not going to work.
The lock that you are using is a FileLock. A key paragraph of the javadoc states this:
File locks are held on behalf of the entire Java virtual machine. They are not suitable for controlling access to a file by multiple threads within the same virtual machine.
In this case, "not suitable" means "doesn't work".
If you drill down to the Linux manual page for flock (2) (which is used by Java to implement these locks), you will see that the semantics are defined in terms of multiple processes, not multiple threads. For example:
LOCK_EX Place an exclusive lock. Only one process may hold a shared lock for a given file at a given time.
and
A call to flock() may block if an incompatible lock is held by another process.
So, in summary, it is still not Java's fault. You are trying to use FileLock in a way that Java doesn't support ... and could not support, given how Linux (and indeed POSIX) flock is specified.
(IMO, all of the stuff about NFS is a red herring. The above problem is not caused by NFS. The reason that it shows up on an NFS file system, is that NFS operations take longer and therefore the time window for overlapping operations on the same file is much larger. And if your customer's use-case is hammering their NFS ...)
(But if I am wrong and NFS is implicated, then your "main vs Tomcat" observation is inexplicable. The JVM will not be doing file locking differently in those two cases: it will be using the same OpenJDK code in both cases. Furthermore, the JVM won't even be aware that it is talking to an NFS file system. You can take a look at the OpenJDK codebase if you don't believe me. It's not that hard ...)
See also:
How to lock file for another threads
Is FileLock in java safe across multiple threads within the same process or between different processes or both?
and so on.
I found the root cause of this issue. It seems that when two different threads of the same JVM create a RandomAccessFile object on the same file, calling RandomAccessFile.close from one thread, releases the lock the other thread has.
The documentation of RandomAccessFile.close says
Closes this random access file stream and releases any system resources associated with the stream.
I'm not sure if this means that the system resources are released on the JVM level. I would expect that each RandomAccessFile object would get its own stream and release only that one but it seems that is not the case (or at least the lock on that file gets released. Note that this behavior has not been observed on Windows systems.
Short version: Why should File.createNewFile() not be used for file locking? Or more specifically: Are there issues if it is used to lock an applications data directory?
Details:
I would like to protect my applications data directory using a lock file: If the file lock exists, the directory is locked and the application exits with an error message. If it does not exist it will be created and the application continues. On exit the file will be deleted.
The lock will not be created that often (i.e. performance is not an issue) and I have no problem with manually deleting the lock file in case of some error (i.e. failing to delete the file is not an issue).
The code looks something like this:
File lockFile = new File("lock");
boolean lockCreated = lockFile.createNewFile();
if (lockCreated)
{
// do stuff
lockFile.delete();
}
else
{
System.err.println("Lockfile exists => please retry later");
// alternative: Wait and retry e.g. 5 times
}
Now I'm a bit confused about the Javadoc of createNewFile():
Atomically creates a new, empty file named by this abstract pathname if and only if a file with this name does not yet exist. The check for the existence of the file and the creation of the file if it does not exist are a single operation that is atomic with respect to all other filesystem activities that might affect the file.
Note: this method should not be used for file-locking, as the resulting protocol cannot be made to work reliably. The FileLock facility should be used instead.
What are the potential problems mentioned in the note, considering the existence check and file creation are atomic?
This forum post from December 2007 indicates there are "significant platform differences" according to the Javadoc of File.delete() (although I cannot find such a statement since at least Java SE 1.4.2). But even if there would be such differences: Could they really cause the locking to fail (i.e. two processes think the data directory is usable at the same time)?
Note: I do not want any of the following:
Lock a file so that no other process can access and/or modify it (most information I found seems to discuss this issue).
Make sure no other process can remove the lock.
Synchronize multiple threads of the same JVM (although I think my solution should be able to handle that too).
The Javadoc of Files.createFile(…), part of java.nio.file available since Java 7, repeats the promise of atomicity but does not mention anything about file based locking.
My reasoning:
Either the newer method (from java.nio.file.Files) is affected by the same (or similar) problems as the older one (from java.io.File) and the Javadoc is simply missing this information…
… or the newer method actually behaves more predictably and correct.
Given the error handling and specification in java.nio.file has generally been improved compared to the File class (existing ever since JDK 1.2), I assume the second alternative is correct.
My conclusion: Using Files.createFile(…) is fine for this use case.
The short answer: reliable file based locking in Java is not practical.
The long answer: The issue with file based locking, in any OS, always comes down to what kind of storage system the file comes from. Almost all network accessed file systems (NFS, SAMBA, etc) have very unreliable (or at least unpredictable) synchronizations on file creates or deletes that make a general Java-ish approach inadvisable. In certain OSes, using local file systems, you can sometimes get what you desire. But you need to understand the underlying file system and its characteristics and proceed with care.
Short version: Why should File.createNewFile() not be used for file locking? Or more specifically: Are there issues if it is used to lock an applications data directory?
Details:
I would like to protect my applications data directory using a lock file: If the file lock exists, the directory is locked and the application exits with an error message. If it does not exist it will be created and the application continues. On exit the file will be deleted.
The lock will not be created that often (i.e. performance is not an issue) and I have no problem with manually deleting the lock file in case of some error (i.e. failing to delete the file is not an issue).
The code looks something like this:
File lockFile = new File("lock");
boolean lockCreated = lockFile.createNewFile();
if (lockCreated)
{
// do stuff
lockFile.delete();
}
else
{
System.err.println("Lockfile exists => please retry later");
// alternative: Wait and retry e.g. 5 times
}
Now I'm a bit confused about the Javadoc of createNewFile():
Atomically creates a new, empty file named by this abstract pathname if and only if a file with this name does not yet exist. The check for the existence of the file and the creation of the file if it does not exist are a single operation that is atomic with respect to all other filesystem activities that might affect the file.
Note: this method should not be used for file-locking, as the resulting protocol cannot be made to work reliably. The FileLock facility should be used instead.
What are the potential problems mentioned in the note, considering the existence check and file creation are atomic?
This forum post from December 2007 indicates there are "significant platform differences" according to the Javadoc of File.delete() (although I cannot find such a statement since at least Java SE 1.4.2). But even if there would be such differences: Could they really cause the locking to fail (i.e. two processes think the data directory is usable at the same time)?
Note: I do not want any of the following:
Lock a file so that no other process can access and/or modify it (most information I found seems to discuss this issue).
Make sure no other process can remove the lock.
Synchronize multiple threads of the same JVM (although I think my solution should be able to handle that too).
The Javadoc of Files.createFile(…), part of java.nio.file available since Java 7, repeats the promise of atomicity but does not mention anything about file based locking.
My reasoning:
Either the newer method (from java.nio.file.Files) is affected by the same (or similar) problems as the older one (from java.io.File) and the Javadoc is simply missing this information…
… or the newer method actually behaves more predictably and correct.
Given the error handling and specification in java.nio.file has generally been improved compared to the File class (existing ever since JDK 1.2), I assume the second alternative is correct.
My conclusion: Using Files.createFile(…) is fine for this use case.
The short answer: reliable file based locking in Java is not practical.
The long answer: The issue with file based locking, in any OS, always comes down to what kind of storage system the file comes from. Almost all network accessed file systems (NFS, SAMBA, etc) have very unreliable (or at least unpredictable) synchronizations on file creates or deletes that make a general Java-ish approach inadvisable. In certain OSes, using local file systems, you can sometimes get what you desire. But you need to understand the underlying file system and its characteristics and proceed with care.
The Java manual says:
The locks held on a particular file by a single Java virtual machine do not overlap. The overlaps method may be used to test whether a candidate lock range overlaps an existing lock.
I guess that if I lock a file in a tomcat web application I can't be sure that every call to this application is done by a different JVM, can I? So how can I lock files within my tomcat application in a reliable way?
I was actually having the exact opposite problem -- my servlet was causing files to get locked that wouldn't clear when the context was reloaded. Of course the reason was because I was opening up InputStreams/BufferedReaders and not closing them. For you, opening up the files in this fashion might be a pretty low-tech solution to your problem as it should result in a lock at the O/S level, which is probably what you want.
What happens when you concurrently open two (or more) FileOutputStreams on the same file?
The Java API says this:
Some platforms, in particular, allow a file to be opened for writing by only one FileOutputStream (or other file-writing object) at a time.
I'm guessing Windows isn't such a platform, because I have two threads that read some big file (each one a different one) then write it to the same output file. No exception is thrown, the file is created and seems to contain chunks from both input files.
Side questions:
Is this true for Unix, too?
And since I want the behaviour to be the same (actually I want one thread to write correctly and the other to be warned of the conflict), how can I determine that the file is already opened for writing?
There's not a reliable, cross-platform way to be passively notified when a file has another writer—i.e., raise an exception if a file is already open for writing. There are a couple of techniques that help you actively check for this, however.
If multiple processes (which can be a mix of Java and non-Java) might be using the file, use a FileLock. A key to using file locks successfully is to remember that they are only "advisory". The lock is guaranteed to be visible if you check for it, but it won't stop you from doing things to the file if you forget. All processes that access the file should be designed to use the locking protocol.
If a single Java process is working with the file, you can use the concurrency tools built into Java to do it safely. You need a map visible to all threads that associates each file name with its corresponding lock instance. The answers to a related question can be adapted easily to do this with File objects or canonical paths to files. The lock object could be a FileOutputStream, some wrapper around the stream, or a ReentrantReadWriteLock.
I would be wary of letting the OS determine file status for you (since this is OS-dependent). If you've got a shared resource I would restrict access to it using a Re-entrant lock
Using this lock means one thread can get the resource (file) and write to it. The next thread can check for this lock being held by another thread, and/or block indefinitely until the first thread releases it.
Windows (I think) would restrict two processes writing to the same file. I don't believe Unix would do the same.
If the 2 threads you are talking about are in the same JVM, then you could have a boolean variable somewhere that is accessed by both threads.
Unix allows concurrent writers to the same file.
You shouldn't be attempting to write to the same file more than once. If you are you have a design flaw.