We Keep Coding

Prevent Text File From Being Deleted When Accessed From Multiple Threads

I'm trying to debug a problem that just surfaced in my program. Until now, I've been writing, reading and updating props file with no problem using the following code structure:
public void setAndReplacePropValue(String dir, String key, String value) throws FileNotFoundException, IOException {
if (value != null) {
File file = new File(dir);
if (!file.exists()) {
System.out.println("File: " + dir + " is not present. Attempting to create new file now..");
new FilesAndFolders().createTextFileWithDirsIfNotPresent(dir);
}
if (file.exists()) {
try {
FileInputStream fileInputStream = null;
fileInputStream = new FileInputStream(file);
if (fileInputStream != null) {
Properties properties = new Properties();
properties.load(fileInputStream);
fileInputStream.close();
if (properties != null) {
FileOutputStream fileOutputStream = new FileOutputStream(file);
properties.setProperty(key, value);
properties.store(fileOutputStream, null);
fileOutputStream.close();
}
}
}
catch (Exception e) {
e.printStackTrace();
}
} else {
System.out.println("File: " + dir + " does not exist and attempt to create new file failed");
}
}
}
However, recently I noticed that a specific file (let's call it: C:\\Users\\Admin\\Desktop\\props.txt) is being deleted after being updated from multiple threads. I'm not sure the exact source of this error, as it seems to happen randomly.
I thought that, perhaps, if two threads call setAndReplacePropValue() and the first thread calls FileOutputStream fileOutputStream = new FileOutputStream(file); before it has a chance to re-write data to file (via properties.store(fileOutputStream, null) ) then the second thread might call fileInputStream = new FileInputStream(file); on an empty file - causing the thread to delete previous data when writing 'empty' data back to file.
To test my hypothesis I tried calling setAndReplacePropValue() from multiple threads several hundred to thousand times in a row while making changes to setAndReplacePropValue() as needed. Here are my results:
If setAndReplace() is declared as static + synchronized the original props data is preserved. This remains true even when I add a random delay after calling FileOutputStream - as long as JVM exists normally. If JVM is killed/terminated (after FileOutputStream is called) then previous data will be deleted.
If I remove both static and synchronized modifiers from setAndReplace() and call setAndReplace() 5,000 times, the old data is still preserved (why?) - as long as JVM ends normally. This appears to be true even when I add random delay in setAndReplace() (after calling FileOutputStream).
When I try modifying props file using ExecutorService (I occasionally access setAndReplacePropValue() via ExecutorService in my program), file content is preserved as long as there's no delay after FileOutputStream. If I add delay and the delay is > 'timout' value set in future.get() (so interrupted exception is thrown) the data is NOT preserved. This remains true even if I add static + synchronized keywords to method.
In short, my question is what is the most likely explanation for why file is being deleted? (I thought point 3 might explain error but I'm not actually sleeping after calling new FileOutputStream() so presumably this would not prevent data from being written back to file after calling new FileOutputStream()). Is there another possibility I didn't think of?
Also, why is point 2 true? If method is not declared as static/synchronized shouldn't this cause one thread to create InputStream from empty file? Thanks.

Unfortunately it is very difficult to provide feedback on your code without a ton of additional information but hopefully, my comments will be helpful.
In general, having multiple threads reading and writing from the same file is a really bad idea. I can't agree more with #Hovercraft-Full-Of-Eels who recommends that you have 1 thread do the reading/writing and the other threads just add updates to a shared BlockingQueue.
But that said here are some comments.
If setAndReplace() is declared as static + synchronized the original props data is preserved.
Right, this stops the terrible race condition in your code where 2 threads could be trying to write to the output file at the same time. Or it could be that 1 thread starts to write and another thread reads an empty file causing data to be lost.
If JVM is killed/terminated (after FileOutputStream is called) then previous data will be deleted.
I don't quite understand this part but your code should have good try/finally clauses to make sure that the files are closed appropriately when the JVM terminates. If the JVM is hard-killed then the file may have been opened but not be written yet (depending on timing). In this case, I would recommend that you write to a temporary file and rename to your properties file which is atomic. Then you might miss the update if the JVM is killed but the file will never be overwritten and be empty.
If I remove both static and synchronized modifiers from setAndReplace() and call setAndReplace() 5,000 times, the old data is still preserved (why?)
No idea. Depends on race conditions. Maybe you are just getting lucky.
When I try modifying props file using ExecutorService (I occasionally access setAndReplacePropValue() via ExecutorService in my program), file content is preserved as long as there's no delay after FileOutputStream. If I add delay and the delay is > 'timeout' value set in future.get() (so interrupted exception is thrown) the data is NOT preserved. This remains true even if I add static + synchronized keywords to method.
I can't answer that without seeing the specific code.
This would be a good idea actually if you had a fixed thread pool with 1 thread, then each of the threads that want to update a value would just submit the field/value object to the thread-pool. This is approximately what #Hovercraft-Full-Of-Eels was talking about.
Hope this helps.

How to check that file is opened by another process in Java? [duplicate]

I need to write a custom batch File renamer. I've got the bulk of it done except I can't figure out how to check if a file is already open. I'm just using the java.io.File package and there is a canWrite() method but that doesn't seem to test if the file is in use by another program. Any ideas on how I can make this work?

Using the Apache Commons IO library...
boolean isFileUnlocked = false;
try {
org.apache.commons.io.FileUtils.touch(yourFile);
isFileUnlocked = true;
} catch (IOException e) {
isFileUnlocked = false;
}
if(isFileUnlocked){
// Do stuff you need to do with a file that is NOT locked.
} else {
// Do stuff you need to do with a file that IS locked
}

(The Q&A is about how to deal with Windows "open file" locks ... not how implement this kind of locking portably.)
This whole issue is fraught with portability issues and race conditions:
You could try to use FileLock, but it is not necessarily supported for your OS and/or filesystem.
It appears that on Windows you may be unable to use FileLock if another application has opened the file in a particular way.
Even if you did manage to use FileLock or something else, you've still got the problem that something may come in and open the file between you testing the file and doing the rename.
A simpler though non-portable solution is to just try the rename (or whatever it is you are trying to do) and diagnose the return value and / or any Java exceptions that arise due to opened files.
Notes:
If you use the Files API instead of the File API you will get more information in the event of a failure.
On systems (e.g. Linux) where you are allowed to rename a locked or open file, you won't get any failure result or exceptions. The operation will just succeed. However, on such systems you generally don't need to worry if a file is already open, since the OS doesn't lock files on open.

// TO CHECK WHETHER A FILE IS OPENED
// OR NOT (not for .txt files)
// the file we want to check
String fileName = "C:\\Text.xlsx";
File file = new File(fileName);
// try to rename the file with the same name
File sameFileName = new File(fileName);
if(file.renameTo(sameFileName)){
// if the file is renamed
System.out.println("file is closed");
}else{
// if the file didnt accept the renaming operation
System.out.println("file is opened");
}

On Windows I found the answer https://stackoverflow.com/a/13706972/3014879 using
fileIsLocked = !file.renameTo(file)
most useful, as it avoids false positives when processing write protected (or readonly) files.

org.apache.commons.io.FileUtils.touch(yourFile) doesn't check if your file is open or not. Instead, it changes the timestamp of the file to the current time.
I used IOException and it works just fine:
try
{
String filePath = "C:\sheet.xlsx";
FileWriter fw = new FileWriter(filePath );
}
catch (IOException e)
{
System.out.println("File is open");
}

I don't think you'll ever get a definitive solution for this, the operating system isn't necessarily going to tell you if the file is open or not.
You might get some mileage out of java.nio.channels.FileLock, although the javadoc is loaded with caveats.

Hi I really hope this helps.
I tried all the options before and none really work on Windows. The only think that helped me accomplish this was trying to move the file. Event to the same place under an ATOMIC_MOVE. If the file is being written by another program or Java thread, this definitely will produce an Exception.
try{
Files.move(Paths.get(currentFile.getPath()),
Paths.get(currentFile.getPath()), StandardCopyOption.ATOMIC_MOVE);
// DO YOUR STUFF HERE SINCE IT IS NOT BEING WRITTEN BY ANOTHER PROGRAM
} catch (Exception e){
// DO NOT WRITE THEN SINCE THE FILE IS BEING WRITTEN BY ANOTHER PROGRAM
}

If file is in use FileOutputStream fileOutputStream = new FileOutputStream(file); returns java.io.FileNotFoundException with 'The process cannot access the file because it is being used by another process' in the exception message.

Pause the execution of Java if files are used

My application writes to Excel files. Sometimes the file can be used, in that case the FileNotFoundException thrown and then I do not know how to handle it better.
I am telling the user that the file is used and after that message I do not want to close the application, but to stop and wait while the file is available (assuming that it is opened by the same user). But I do not understand how to implement it. file.canWrite() doesn't work, it returns true even when the file is opened, to use FileLock and check that the lock is available I need to open a stream, but it throws FileNotFoundException (I've been thinking about checking the lock in a busy wait, I know that it is not a good solution, but I can't find another one).
This is a part of my code if it can help somehow to understand my problem:
File file = new File(filename);
FileOutputStream out = null;
try {
out = new FileOutputStream(file);
FileChannel channel = out.getChannel();
FileLock lock = channel.lock();
if (lock == null) {
new Message("lock not available");
// to stop the program here and wait when the file is available, then resume
}
// write here
lock.release();
}
catch (IOException e) {
new Message("Blocked");
// or to stop here and then create another stream when the file is available
}
What makes it more difficult for me is that it writes to different files, and if the first file is available, but the second is not, then it will update one file and then stop, and if I restart the program, it will update it again, so I can't allow the program to write into files until all of them are available.
I believe that there should be a common solution, since it must be a common issue in Windows to deal with such cases, but I can't find it.

To wait until a file exists you can make a simple loop:
File file = new File(filename);
while (!file.exists()) {
try {
Thread.sleep(100);
} catch (InterruptedException ie) { /* safe to ignore */ }
}
A better solution could be using WatchService but it's more code to implement.
The File.canWrite method only tells you if a path can be written to; if the path names a file that doesn't exist it will return false. You could use the canRead method instead of exists in a loop like above.
To use a file locks, the file has to exist first, so that wouldn't work either.
The only way to be sure you can write to a file is to try to open it. If the file doesn't exist, the java.io API will create it. To open a file for writing without creating you can use the java.nio.file.Files class:
try (OutputStream out = Files.newOutputStream(file.toPath(),
StandardOpenOption.WRITE))
{
// exists and is writable
} catch (IOException) {
// doesn't exist or can't be opened for writing
}

FileOutputStream does not create file

I actually checked other posts that could be related to this and I couldn't find any answer to my question. So, had to create this newly:
The file does not get created in the given location with this code:
File as = new File ("C:\\Documents and Settings\\<user>\\Desktop\\demo1\\One.xls");
if (!as.exists()) {
as.createNewFile();
}
FileOutputStream fod = new FileOutputStream(as);
BufferedOutputStream dob = new BufferedOutputStream(fod);
byte[] asd = {65, 22, 123};
byte a1 = 87;
dob.write(asd);
dob.write(a1);
dob.flush();
if (dob!=null){
dob.close();
}
if(fod!=null){
fod.close();
The code runs fine and I don't get any FileNotFoundException!!
Is there anything that I'm missing out here?

You can rewrite your code like this:
BufferedOutputStream dob = null;
try {
File file = new File("C:\\Documents and Settings\\<user>\\Desktop\\demo1\\One.xls");
System.out.println("file created:" + file.exists());
FileOutputStream fod = new FileOutputStream(file);
System.out.println("file created:" + file.exists());
BufferedOutputStream dob = new BufferedOutputStream(fod);
byte[] asd = {65, 22, 123};
byte a1 = 87;
dob.write(asd);
dob.write(a1);
//dob.flush();
}
catch (Exception ex) {
ex.printStackTrace();
}
finally {
if (dob != null) {
dob.close();
}
}
In this case it is only necessary to call the topmost stream handler close() method - the BufferedOutputStream's one:
Closes this output stream and releases any system resources associated with the stream.
The close method of FilterOutputStream calls its flush method, and then calls the close method of its underlying output stream.
so, the dob.flush() in try block is commented out because the dob.close() line in the finally block flushes the stream. Also, it releases the system resources (e.g. "closes the file") as stated in the apidoc quote above. Using the finally block is a good practice:
The finally block always executes when the try block exits. This ensures that the finally block is executed even if an unexpected exception occurs. But finally is useful for more than just exception handling — it allows the programmer to avoid having cleanup code accidentally bypassed by a return, continue, or break. Putting cleanup code in a finally block is always a good practice, even when no exceptions are anticipated.
The FileOutputStream constructor creates an empty file on the disk:
Creates a file output stream to write to the file represented by the specified File object. A new FileDescriptor object is created to represent this file connection.
First, if there is a security manager, its checkWrite method is called with the path represented by the file argument as its argument.
If the file exists but is a directory rather than a regular file, does not exist but cannot be created, or cannot be opened for any other reason then a FileNotFoundException is thrown.
Where a FileDescriptor is:
Instances of the file descriptor class serve as an opaque handle to the underlying machine-specific structure representing an open file, an open socket, or another source or sink of bytes. The main practical use for a file descriptor is to create a FileInputStream or FileOutputStream to contain it.
Applications should not create their own file descriptors.
This code should either produce a file or throw an exception. You have even confirmed that no conditions for throwing exception are met, e.g. you are replacing the string and the demo1 directory exists. Please, rewrite this to a new empty file and run.
If it still behaving the same, unless I have missed something this might be a bug. In that case, add this line to the code and post output:
System.out.println(System.getProperty("java.vendor")+" "+System.getProperty("java.version"));
Judging from the path, I'd say you are using Win 7, am I right? What version?

Then it means there is a file already in your directory

Java file locking on a network

This is perhaps similar to previous posts, but I want to be specific about the use of locking on a network, rather than locally. I want to write a file to a shared location, so it may well go on a network (certainly a Windows network, maybe Mac). I want to prevent other people from reading any part of this file whilst it it being written. This will not be a highly concurrent process, and the files will be typically less than 10MB.
I've read the FileLock documentation and File documentation and am left somewhat confused, as to what is safe and what is not. I want to lock the entire file, rather than portions of it.
Can I use FileChannel.tryLock(), and it is safe on a network, or does it depend on the type of network? Will it work on a standard Windows network (if there is such a thing).
If this does not work, is the best thing to create a zero byte file or directory as a lock file, and then write out the main file. Why does that File.createNewFile() documentation say don't use this for file locking? I appreciate this is subject to race conditions, and is not ideal.

This can't be reliably done on a network file system. As long as your application is the only application that accesses the file, it's best to implement some kind of cooperative locking process (perhaps writing a lock file to the network filesystem when you open the file). The reason that is not recommended, however, is that if your process crashes or the network goes down or any other number of issues happen, your application gets into a nasty, dirty state.

You can have a empty file which is lying on the server you want to write to.
When you want to write to the server you can catch the token. Only when you have the token you should write to any file which is lying on the server.
When you are ready with you file operations or an exception was thrown you have to release the token.
The helper class can look like
private FileLock lock;
private File tokenFile;
public SLTokenLock(String serverDirectory) {
String tokenFilePath = serverDirectory + File.separator + TOKEN_FILE;
tokenFile = new File(tokenFilePath);
}
public void catchCommitToken() throws TokenException {
RandomAccessFile raf;
try {
raf = new RandomAccessFile(tokenFile, "rw"); //$NON-NLS-1$
FileChannel channel = raf.getChannel();
lock = channel.tryLock();
if (lock == null) {
throw new TokenException(CANT_CATCH_TOKEN);
}
} catch (Exception e) {
throw new TokenException(CANT_CATCH_TOKEN, e);
}
}
public void releaseCommitToken() throws TokenException {
try {
if (lock != null && lock.isValid()) {
lock.release();
}
} catch (Exception e) {
throw new TokenException(CANT_RELEASE_TOKEN, e);
}
}
Your operations then should look like
try {
token.catchCommitToken();
// WRITE or READ to files inside the directory
} finally {
token.releaseCommitToken();
}

I found this bug report which describes why the note about file locking was added to the File.createNewFile documentation.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4676183
It states:
If you mark the file as deleteOnExit before invoking createNewFile but the file already exists, you run the risk of deleting a file you didn't create and dropping someone elses lock! On the other hand, if you mark the file after creating it, you lose atomicity: if the program exits before the file is marked, it won't get deleted and the lock will be "wedged".
So it looks like the main reason locking is discouraged with File.createNewFile() is that you can end up with orphaned lock files if the JVM unexpectedly terminates before you have a chance to delete it. If you can deal with orphaned lock files then it could be used as a simple locking mechanism. However, I wouldn't recommend the method suggested in the comments of the bug report as it has race conditions around read/writing the timestamp value and reclaiming the expired lock.

Rather than implementing a locking strategy which will, in all likelihood, rely on readers to adhere to your convention but will not force them to, perhaps you can write the file out to a hidden or obscurely named file where it will be effectively invisible to readers. When the write operation is complete, rename the file to the expected public name.
The downside is that hiding and/or renaming without additional IO may require you to use native OS commands, but the procedure to do so should be fairly simple and deterministic.