I have a write method that is supposed to safely write data to a file.
// The current file I am writing to.
FileOutputStream file = null;
...
// Synchronized version.
private void write(byte[] bytes) {
if (file != null && file.getChannel() != null) {
try {
boolean written = false;
do {
try {
// Lock it!
FileLock lock = file.getChannel().lock();
try {
// Write the bytes.
file.write(bytes);
written = true;
} finally {
// Release the lock.
lock.release();
}
} catch (OverlappingFileLockException ofle) {
try {
// Wait a bit
Thread.sleep(0);
} catch (InterruptedException ex) {
throw new InterruptedIOException("Interrupted waiting for a file lock.");
}
}
} while (!written);
} catch (IOException ex) {
log.warn("Failed to lock " + fileName, ex);
}
} else {
log.warn("Failing - " + (file == null ? "file" : "channel") + " is null!!");
}
}
It has worked fine for me for a while now, although I know there are some wrinkles in it.
I have recently changed a project that uses this code to build and run under Java 5 (from Java 6) and now it looks like it is deadlocked awaiting a lock on the file. It is a multithreaded app and it is quite possible for several threads to attempt to write to the same file.
The debugger tells me that the hung threads are waiting for the FileLock lock = file.getChannel().lock() call to return.
Some research brought up this interesting little nugget which mentions:
File locks are held on behalf of the entire Java virtual machine. They are not suitable for controlling access to a file by multiple threads within the same virtual machine.
So am I doing it wrong? If so what is the right way? If I am doing it right how come I hit a deadlock?
Added: Forgot to mention - each thread holds its own copy of this object so there should not be any synchronisation issues within the code. I felt safe to rely on the FileChannel.lock() method to ensure writes do not interleave.
Added too: I have indeed solved the issue using various synchronized mechanisms. I do, however, have outstanding questions:
Why is FileLock lock = file.getChannel().lock(); not suitable ...?
Why did my issues only appear when switching back to Java-5 when everything worked fine with Java-6?
FileLock is only for interprocess locking, javadoc reads:
"File locks are held on behalf of the entire Java virtual machine.
They are not suitable for controlling access to a file by multiple
threads within the same virtual machine."
To lock between java threads (same JVM) you need to use some shared lock. I would suggest within the file writing class to use a synchronized block (which according to these articles is likely to perform best):
final Object lock = new Object();
public void write(...){
synchronized(lock){
// do writing
}
}
Another approach is to use a ReentrantLock and then use the proven idiom of
final ReentrantLock lock = new ReentrantLock();
public void write(...){
try {
lock.lock()
// do the writing
} finally {
// forget this and you're screwed
lock.unlock();
}
}
you may need to implement critical section concept on the actual code using the file, rather than the hashmap. You can either create a synchronized block or separate the file access code into a separate procedure and make that method synchronized.
Essentially, only one thread executes a synchronized block at a time. It gives you the exclusive access you need.
Another way of doing it is, to use a serial thread Executor, depending on your functional requirements.
You may want to look at this thread:
Howto synchronize file access in a shared folder using Java (OR: ReadWriteLock on network level)
Related
I need to prevent users from starting my Java application (WebStart Swing app) multiple times. So if the application is already running it shouldn't be possible to start it again or show a warning / be closed again.
Is there some convenient way to achieve this? I thought about blocking a port or write sth to a file. But hopefully you can access some system properties or the JVM?
btw. target platform is Windows XP with Java 1.5
I think your suggestion of opening a port to listen when you start your application is the best idea.
It's very easy to do and you don't need to worry about cleaning it up when you close your application. For example, if you write to a file but someone then kills the processes using Task Manager the file won't get deleted.
Also, if I remember correctly there is no easy way of getting the PID of a Java process from inside the JVM so don't try and formulate a solution using PIDs.
Something like this should do the trick:
private static final int PORT = 9999;
private static ServerSocket socket;
private static void checkIfRunning() {
try {
//Bind to localhost adapter with a zero connection queue
socket = new ServerSocket(PORT,0,InetAddress.getByAddress(new byte[] {127,0,0,1}));
}
catch (BindException e) {
System.err.println("Already running.");
System.exit(1);
}
catch (IOException e) {
System.err.println("Unexpected error.");
e.printStackTrace();
System.exit(2);
}
}
This sample code explicitly binds to 127.0.0.1 which should avoid any firewall warnings, as any traffic on this address must be from the local system.
When picking a port try to avoid one mentioned in the list of Well Known Ports. You should ideally make the port used configurable in a file or via a command line switch in case of conflicts.
As the question states that WebStart is being used, the obvious solution is to use javax.jnlp.SingleInstanceService.
This service is available in 1.5. Note that 1.5 is currently most of the way through its End Of Service Life period. Get with Java SE 6!
I think that the better idea would be to use file lock (quite an old idea :) ). Since Java 1.4 a new I/O library was introduced, that allows file locking.
Once the application starts it tries to acquire lock on a file (or create it if does not exist), when the application exits the lock is relased. If application cannot acquire a lock, it quits.
The example how to do file locking is for example in Java Developers Almanac.
If you want to use file locking in Java Web Start application or an applet you need to sing the application or the applet.
You can use JUnique library. It provides support for running single-instance java application and is open-source.
http://www.sauronsoftware.it/projects/junique/
See also my full answer at How to implement a single instance Java application?
We do the same in C++ by creating a kernal mutex object and looking for it at start up. The advantages are the same as using a socket, ie when the process dies/crashes/exits/is killed, the mutex object is cleaned up by the kernel.
I'm not a Java programmer, so I am not sure whether you can do the same kind of thing in Java?
I've create the cross platform AppLock class.
http://mixeddev.info/articles/2015/02/01/run-single-jvm-app-instance.html
It is using file lock technique.
Update. At 2016-10-14 I've created package compatible with maven/gradle https://github.com/jneat/jneat and explained it here http://mixeddev.info/articles/2015/06/01/synchronize-different-jvm-instances.html
You could use the registry, although this halfheartedly defeats the purpose of using a high-level language like java. At least your target platform is windows =D
Try JUnique:
String appId = "com.example.win.run.main";
boolean alreadyRunning;
try {
JUnique.acquireLock(appId);
alreadyRunning = false;
} catch (AlreadyLockedException e) {
alreadyRunning = true;
}
if (alreadyRunning) {
Sysout("An Instance of this app is already running");
System.exit(1);
}
I've seen so many of this questions and I was looking to solve the same problem in a platform independent way that doesn't take the chance to collide with firewalls or get into socket stuff.
So, here's what I did:
import java.io.File;
import java.io.IOException;
/**
* This static class is in charge of file-locking the program
* so no more than one instance can be run at the same time.
* #author nirei
*/
public class SingleInstanceLock {
private static final String LOCK_FILEPATH = System.getProperty("java.io.tmpdir") + File.separator + "lector.lock";
private static final File lock = new File(LOCK_FILEPATH);
private static boolean locked = false;
private SingleInstanceLock() {}
/**
* Creates the lock file if it's not present and requests its deletion on
* program termination or informs that the program is already running if
* that's the case.
* #return true - if the operation was succesful or if the program already has the lock.<br>
* false - if the program is already running
* #throws IOException if the lock file cannot be created.
*/
public static boolean lock() throws IOException {
if(locked) return true;
if(lock.exists()) return false;
lock.createNewFile();
lock.deleteOnExit();
locked = true;
return true;
}
}
Using System.getProperty("java.io.tmpdir") for the lockfile path makes sure that you will always create your lock on the same place.
Then, from your program you just call something like:
blah blah main(blah blah blah) {
try() {
if(!SingleInstanceLock.lock()) {
System.out.println("The program is already running");
System.exit(0);
}
} catch (IOException e) {
System.err.println("Couldn't create lock file or w/e");
System.exit(1);
}
}
And that does it for me. Now, if you kill the program it won't delete the lock file but you can solve this by writing the program's PID into the lockfile and making the lock() method check if that process is already running. This is left as an assingment for anyone interested. :)
I have an application where I want to ensure that a method is called at most once concurrently, say when updating user balance in a database.
I am thinking of using the following locking mechanism: (showing Scala code below, but should be similar with Java Lambdas):
object Foo{
val dbLocked = new java.util.concurrent.atomic.AtomicBoolean(false)
def usingAtoimcDB[T](f: => T):T = {
if (dbLocked.get) throw new Exception("db is locked")
dbLocked.set(true)
try f
finally dbLocked.set(false)
}
}
Is this safe to use when usingAtoimcDB may be called concurrently?
EDIT: The corrected code below, as pointed in this answer:
def usingAtoimcDB[T](f: => T):T = {
if(dbLocked.compareAndSet(false, true)) {
//db is now locked
try f
finally dbLocked.set(false)
} else {
//db is already locked
throw new Exception("db is locked")
}
}
EDIT 2:
Using a spinloop. Is this also ok?
def usingAtoimcDB[T](f: => T):T = {
while (!dbLocked.compareAndSet(false, true)) {Thread.sleep(1)}
try f
finally dbLocked.set(false)
}
EDIT3: Based on the answers and comments below, I am also considering using queues.
Inadvisable. You are requesting that the same pieco of code running in the same application instance on tha same server is the single point to do that transaction. There also is no provision to let this code stand-out. When you are retired, someone may start a second application instance or whatever.
Whereas a database commit/rollback is a quite simple and sure mechanism.
When you cannot write an integration (unit) test to ensure this sole point, then do not do it.
If you do it:
Revoke rights to the table modifications for the normal database user
Add a new database use who has sufficient right granted
And still: do not do it.
The code you posted above is not thread-safe, because you are not using an atomic check-and-set operation. Two threads can both be executing the if (dbLocked.get) statement at the same time and both get false as the answer, and then both will do dbLocked.set(true) and call f.
If you really want to use AtomicBoolean, then you must use compareAndSet as #leshkin already showed - this is an atomic operation that does the check and set in one go without the possibility of another thread doing the same thing at the same time, so that it is thread-safe.
You are using an AtomicBoolean as a lock here. There are classes in the standard Java library which are better suited (and specifically made) for this purpose; have a look at the package java.util.concurrent.locks.
You could for example use class ReentrantReadWriteLock, which combines two locks for reading and writing. The write lock is exclusive (when it's locked, nobody else can read or write); the read lock is shared (when it's locked, nobody can write, but others can read at the same time). This allows for there to be multiple readers concurrently, but only one writer at a time, possibly improving efficiency (it's not necessary to make reading an exclusive operation).
Example:
import java.util.concurrent.locks._
object Foo {
private val lock: ReadWriteLock = new ReentrantReadWriteLock
def doWriteOperation[T](f: => T): T = {
// Locks the write lock
lock.writeLock.lock()
try {
f
} finally {
lock.writeLock.unlock()
}
}
def doReadOperation[T](f: => T): T = {
// Locks the read lock
lock.readLock.lock()
try {
f
} finally {
lock.readLock.unlock()
}
}
}
Yes, it should work as espected. I would slightly modify your function using compareAndSet call.
compareAndSet method has the advantage to be an atomic operation - there are no race conditions and the value will be changed atomically.
def usingAtoimcDB[T](f: => T):T = {
if(dbLocked.compareAndSet(false, true)) {
//db is now locked
try f
finally dbLocked.set(false)
} else {
//db is already locked
throw new Exception("db is locked")
}
}
I'm trying to write to a file, using a Java FileLock, to prohibit all other processes and threads from reading from it or writing to it until I'm finished with it. Given this question and the answers to it, it seems to me like this is the perfect tool for what I want--a mutex for file access.
However, I am very concerned about this text from the JavaDocs:
File locks are held on behalf of the entire Java virtual machine. They
are not suitable for controlling access to a file by multiple threads
within the same virtual machine.
Can someone either alleviate my fears or point me in the right direction? It sounds like FileLock won't work at all to keep a different thread out of the file, even if another thread has already obtained it. If this is the case, is there another canonical Java method to do this that will protect from other threads?
The FileLock is a process level lock and will thus not protect the file from concurrent access from multiple threads within the process that has the lock.
You need to use a combination of the FileLock to protect from concurrent access from other processes and some other synchronization mechanism (like a synchronized method for accessing the file) within your process to protect from concurrent access by your own threads.
I would implement this as follows:
interface FileOperator {
public void operate(File file);
}
class FileProxy {
private static final ConcurrentHashMap<URI, FileProxy> map =
new ConcurrentHashMap<>();
private final Semaphore mutex = new Semaphore(1, true);
private final File file;
private final URI key;
private FileProxy(File file) {
this.file = file;
this.key = file.toURI();
}
public static void operate(URI uri, FileOperator operator) {
FileProxy curProxy = map.get(uri);
if(curProxy == null) {
FileProxy newProxy = new FileProxy(new File(uri));
FileProxy curProxy = map.putIfAbsent(newProxy.key, newProxy);
if(curProxy == null) {
curProxy = newProxy; // FileProxy was not in the map
}
}
try {
curProxy.mutex.acquire();
operator.operate(curProxy.file);
} finally {
curProxy.mutex.release();
}
}
}
The threads that are using a file implement FileOperator or something similar. Files are hidden behind a FileProxy that maintains a static ConcurrentHashMap of key (URI, or absolute path, or some other file invariant) value (FileProxy) pairs. Each FileProxy maintains a Semaphore that acts as a mutex - this is initialized with one permit. When the static operate method is called, a new FileProxy is created from the URI if none exists; the FileOperator is then added to the FileProxy queue; acquire is called on the mutex to ensure that only one thread can operate on the file at a time; and finally the FileOperator does its thing.
In this implementation, FileProxy objects are never removed from the ConcurrentHashMap - if this is a problem then a solution is to wrap the FileProxy objects in a WeakReference or SoftReference so that they can be garbage collected, and then call map.replace if reference.get() == null to ensure that only one thread replaces the GC'd reference.
I have a simple java program that creates a series of temporary files stored in a local tmp directory. I have added a simple shutdown hook that walks through all files and deletes them, then deletes the tmp directory, before exiting the program. here is the code:
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
File tmpDir = new File("tmp/");
for (File f : tmpDir.listFiles()) {
f.delete();
}
tmpDir.delete();
}
}));
My problem is that the thread that creates these files may not have terminated upon launch of the shutdown hook, and therefore, there may be a file created after listFiles() is called. this causes the tmp dir not to get deleted. I have come up with 2 hacks around this:
Hack # 1:
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
File tmpDir = new File("tmp/");
while (!tmp.delete()){
for (File f : tmpDir.listFiles()) {
f.delete();
}
}
}
}));
Hack # 2:
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
try{
Thread.sleep(1000);
} catch(InterruptedException e){
e.printStackTrace();
}
File tmpDir = new File("tmp/");
for (File f : tmpDir.listFiles()) {
f.delete();
}
tmpDir.delete();
}
}));
Neither is a particularly good solution. What would be ideal is to have the shutdown hook wait until all threads have terminated before continuing. Does anyone know if this can be done?
Just keep track of all your running threads and then.join() them before shutting down the program.
This is an answer to the question title as the ewok has said he can't use .deleteOnExit()
What Tyler said, but with a little more detail:
Keep references to the threads where the shutdown hook can access them.
Have the shutdown hook call interrupt on the threads.
Review the code of the threads to make sure they actually respond to interruption (instead of eating the InterruptedException and blundering on, which is typical of a lot of code). An interrupt should prompt the thread to stop looping or blocking, wrap up unfinished business, and terminate.
For each thread where you don't want to proceed until it finishes, check whether the thread is alive and if so call join on it, setting a timeout in case it doesn't finish in a reasonable time, in which case you can decide whether to delete the file or not.
UPDATE: Tyler Heiks accurately pointed out that deleteOnExit() isn't a valid solution since the OP tried it and it did not work. I am providing an alternate solution. It is again indirect, but mainly because the original design using threads and a ShutdownHook is fatally flawed.
Use finally blocks to delete the temp files.
Relying on ShutdownHooks for resource management is a very bad idea and makes the code very difficult to compose or reuse in a larger system. It's an even worse idea to hand resources from thread to thread. Resources like files and streams are among the most dangerous things to share between threads. There is likely very little to gain from this and it would make far more sense for each thread to independently obtain temp files using the library's createTempFile methods and manage their use and deletion using try/finally.
The convention for dealing with the temporary files on the system is to treat them as block boxes where:
location on disk is opaque (irrelevant to and not used directly by the program)
filename is irrelevant
filename is guaranteed to be mutually exclusive
The third above is very difficult to achieve if you hand-roll code to create and name temp files yourself. It is likely to be brittle and fail at the worst times (3AM pager anyone?).
The algorithm you present could delete files created by other processes that coincidentally share the same parent directory. That is unlikely to be a good thing for the stability of those other programs.
Here's the high-level process:
Get Path with Files.createTempFile() (or with legacy pre-Java 7 code File with File.createTempFile())
Use temp file however desired
Delete file
This is similar to InputStream or other resources that need to be manually managed.
That general pattern for explicit resource management (when AutoCloseable and try-with-resources aren't available) is as follows.
Resource r = allocateResource();
try {
useResource(r);
} finally {
releaseResource(r);
}
In the case of Path it looks like this:
Path tempDir = Paths.get("tmp/);
try {
Path p = Files.createTempFile(tempDir, "example", ".tmp");
try {
useTempFile(f);
} finally {
Files.delete(f);
}
} finally {
Files.delete(tempDir);
}
On pre-Java 7 legacy, the use with File looks like this:
File tempDir = new File("tmp/");
try {
File f = File.createTempFile(tempDir, "example", ".tmp");
try {
useTempFile(f);
} finally {
if (!f.delete()) {
handleFailureToDeleteTempFile(f);
}
}
} finally {
if (!tempDir.delete()) {
handleFailureToDeleteTempDir(tempDir);
}
}
A multi-threaded piece of code accesses a resource (eg: a filesystem) asynchronously.
To achieve this, I'll use condition variables. Suppose the FileSystem is an interface like:
class FileSystem {
// sends a read request to the fileSystem
read(String fileName) {
// ...
// upon completion, execute a callback
callback(returnCode, buffer);
}
}
I have now an application accessing the FileSystem. Suppose I can issue multiple reads through a readFile() method.
The operation should write data to the byte buffer passed to it.
// constructor
public Test() {
FileSystem disk = ...
boolean readReady = ...
Lock lock = ...
Condition responseReady = lock.newCondition();
}
// the read file method in quesiton
public void readFile(String file) {
try {
lock.lock(); // lets imagine this operation needs a lock
// this operation may take a while to complete;
// but the method should return immediately
disk.read(file);
while (!readReady) { // <<< THIS
responseReady.awaitUninterruptibly();
}
}
finally {
lock.unlock();
}
}
public void callback(int returnCode, byte[] buffer) {
// other code snipped...
readReady = true; // <<< AND THIS
responseReady.signal();
}
Is this the correct way to use condition variables? Will readFile() return immediately?
(I know there is some sillyness in using locks for reads, but writing to a file is also an option.)
There's a lot missing from your question (i.e. no specific mention of Threads) but I will try to answer anyway.
Neither the lock nor the conditional variables give you background capabilities -- they just are used for a thread to wait for signals from other threads. Although you don't mention it, the disk.read(file) method could spawn a thread to do the IO and then return immediately but the caller is going to sit in the readReady loop anyway which seems pointless. If the caller has to wait then it could perform the IO itself.
A better pattern could be to use something like the Java 5 Executors service:
ExecutorService pool = Executors.newFixedThreadPool(int numThreads);
You can then call pool.submit(Callable) which will submit the job to be performed in the background in another thread (when the pool next has one available). Submit returns a Future which the caller can use to investigate if the background task has finished. It can return a result object as well. The concurrent classes take care of the locking and conditional signal/wait logic for you.
Hope this helps.
p.s. Also, you should make readReady be volatile since it is not synchronized.