I have a requirement where in I write a file to server. Another application has a scheduled job which reads the file at a specific interval. The file shouldn't be readable till my write is complete. I have tried using
File.isReadable(false)
But this is not working. And the scheduler is picking up the incomplete data from the file, if I am still writing to it.
Any Solutions?
Write to a different file name and then when the write is complete rename the file to the name the scheduler expects. If you're running on Linux or similar then file renames within the same file system are atomic.
File tempFile = new File("/path/to/file.tmp");
// write to tempFile
tempFile.renameTo(new File("/path/to/file"));
You can use another file with same name as marker. You will start writing into FileName.txt and when is finished, create file FileName.rdy
And your application will check only for *.rdy files, if found - read FileName.txt.
You can use the FileLock API.
I explained briefly how it works here.
the better option would be to synchronize the read and write procedures...
put your code to read file and write file in synchornized {} blocks....such that one can wait till other completes
Related
I am trying to consume (stream) a big zip file with Apache Camel. The streaming should begin as soon as the file is being written to. Below is the file consumer code.
rest("/api/request/{Id}/")
.get()
.produces(MediaType.APPLICATION_OCTET_STREAM_VALUE)
.process(new FindFileName)
.pollEnrich().simple("file:" + outputDir + "?fileName=${property.filnavn}&noop=false&readLock=none&delete=true").timeout(pollTimeout)
Claus Ibsen suggested using readLock=none to get the stream.
When I use the option the stream closes right away and I only get the 0 byte file with the correct filename.
How do I configure camel's file endpoint to use readLock=none and consume the file until it is completed?
A seperate route writes the file.
There is no safe way to know when a file is completed written by a 3rd party. What you do there, is that you get a hold of a java.io.File in the poll enrich to the file. Which Camel can convert to a FileInputStream to read from. But that stream has no way of knowing when the 3rd party if finished writing the file.
There its really a bad practice to read files that are currently in progress of being written.
To know when a file is complete written then 3rd parties may use a strategy to
write a 2nd dummy marker file to tell its finished
write a 2nd in-progress dummy file to tell the file is currently being written and delete this file when its finished
write the file using a temporary name and rename when done
write the file in another folder and move when done
monitor the file for modified timestamp and if the timestamp doesnt change after X period then assume its finished written
attempt to rename the file and assuming if the OS fails doing this then the 3rd party is still writing to the file
etc...
The JDK File Lock API does not work acrosss file systems and is generally not very useable to get file locks - it may work from within the same JVM, but not when its 2 different systems.
I need to make some ZIP- action on a freshly introduced file in a dir. There I subsribe for the CREATE event, which is fired. The copy operation into that directory takes some time.
So I get ACCESS_DENIED, "locked by another process" when accessing the file.
Does NIO provide something like "LOCK Released" or do I need to poll the file somehow for the lock to be released ? Like described here :http://stackoverflow.com/questions/750471/how-to-know-whether-a-file-copying-is-in-progress-complete-in-java-1-6
Thanks for any help.
Gerd
copy the file in a different directory; after it's done, move it to the watched directory.
java.nio.file.Files.move(srcFile, targetFile, StandardCopyOption.ATOMIC_MOVE);
the watcher will see one CREATE event
Modify the copy operation so that an additional, small file is written to the same directory, after the first, larger file is written. The filename could be the name of the original file, plus an extension.
Your watcher would look for files with this extension, chop it off to get the original filename, and then start the Zip-action.
Another option could be that your copy operation writes a file with an extension other than the one you look for. When done, it renames the file to give it the correct extension.
I have a zipped CSV file.
I have some quartz job schedular which reads the file. But sometimes the user can click and read the file also. Is it possible that if during the user operation file is open and then the quartz job schedular also starts and it also starts reading the file and as it's a zipped CSV file something may become corrupted.
Special Note : There is no write operation to the file.
You can read the same file in as many threads as you like.
It will only be corrupted if you write it in one thread and try to use it in another at the same time.
There is no problem to read the same file simultaneously from several threads. Just create separate FileInputStream for each thread.
If you have no write operation on your file then you will have no problem in reading it.
You can read it in all your threads.
Problem will come if you try to write it.
Even you can avoid this problem by making your File object Synchronized.
In File Monitoring process if one file came and it is processed immediately it does not check
if file is open and writing something..Then how to prevent moving of file without closing the file.
Do you have control over the program that's putting the files in the directory? Put something like ".partial" on the end of the filename while the file is still being written, and then rename it to remove the ".partial" when the writing is done. If you make the Java file-monitoring program ignore files whose names end in ".partial", it'll only see files after they've been fully written out.
I have this recurrent Java JAR program tasks that tries to modify a file every 60seconds.
Problem is that if user is viewing the file than Java program will not be able to modify the file. I get the typical IOException.
Anyone knows if there is a way in Java to modify a file currently in use? Or anyone knows what would be the best way to solve this problem?
I was thinking of using the File canRead(), canWrite() methods to check if file is in use. If file is in use then I'm thinking of making a backup copy of data that could not be written. Then after 60 seconds add some logic to check if backup file is empty or not. If backup file is not empty then add its contents to main file. If empty then just add new data to main file. Of course, the first thing I will always do is check if file is in use.
Thanks for all your ideas.
I was thinking of using the File
canRead(), canWrite() methods to check
if file is in use.
Not a good idea - you'll run into race conditions e.g. when your code has used those check methods, received true return values, but then the file is locked by a different application (possibly the user) just before you open it for writing.
Instead, try to get a FileLock on the file and use the "backup file" when that fails.
You can hold a lock on the file. This should guarantee you are able to write on the file.
See here on how to use the FileLock class.
If the user is viewing the file you should still be able to read it. In this case, make an exact copy of the file, and make changes to the new file.
Then after the next 60 seconds you can either:
1) Check if the file is being viewed and if not, delete it and replace it with the earlier file, then directly update this file.
2) If it is being viewed, continue making changes to the copy of the file.
EDIT: As Michael mentioned, when working with the main file, get a lock on it first.