I'm adding code to a large JSP web application, integrating functionality to convert CGM files to PDFs (or PDFs to CGMs) to display to the user.
It looks like I can create the converted files and store them in the directory designated by System.getProperty("java.io.tmpdir"). How do I manage their deletion, though? The program resides on a Linux-based server. Will the OS automatically delete from /tmp or will I need to come up with functionality myself? If it's the latter scenario, what are good ways to go about doing it?
EDIT: I see I can use deleteOnExit() (relevant answer elsewhere), but I think the JVM runs more or less continuously in the background so I'm not sure if the exits would be frequent enough.
I don't think I need to cache any converted files--just convert a file anew every time it's needed.
You can do this
File file = File.createTempFile("base_name", ".tmp", new File(temporaryFolderPath));
file.deleteOnExit();
the file will be deleted when the virtual machine terminates
Edit:
If you want to delete it after the job is done, just do it:
File file = null;
try{
file = File.createTempFile("webdav", ".tmp", new File(temporaryFolderPath));
// do sth with the file
}finally{
file.delete();
}
There are ways to have the JVM delete files when the JVM exits using deleteOnExit() but I think there are known memory leaks using that method. Here is a blog explaining the leak: http://www.pongasoft.com/blog/yan/java/2011/05/17/file-dot-deleteOnExit-is-evil/
A better solution would either be to delete old files using a cron or if you know you aren't going to use the file again, why not just delete it after processing?
From your comment :
Also, could I just create something that checks to see if the size of my files exceeds a certain amount, and then deletes the oldest ones if that's true? Or am I overthinking it?
You could create a class that keeps track of the created files with a size limit. When the size of the created files, after creating a new one, goes over the limit, it deletes the oldest one. Beware that this may delete a file that still needs to exist even if it is the oldest one. You might need a way to know which files still need to be kept and delete only those that are not needed anymore.
You could have a timer in the class to check periodically instead of after each creation. This solution is tied to your application while using a cron isn't.
Related
First of all I am not asking for difference between the two. I am wondering what will be a scenario where one will choose to use deleteOnExit() over delete().
Sometimes we may want to create temporary files to save some data that gets written by an application or to temporarily store some data for use in the near future by this same application, etc. etc. In these scenarios, we may do something like this:
File tempFile = File.createTempFile(...);
tempFile.deleteOnExit();
.... // the rest of the code
deleteOnExit would be appropriate for temporary files you'd like to be cleaned up on exit
I saw this nifty guide on how to do streaming file uploads via Apache Commons. This got me thinking where is the data stored? And is it necessary to "close" or "clean" that location?
Thanks!
where is the data stored?
I don't think it is stored.
The Streaming API doesn't use DiskFileItemFactory. But it does use a buffer for copying data as BalusC has posted.
Once you have the stream of the upload, you can use
long bytesCopied = Streams.copy(yourInputStream, yourOutputStream, true);
Look at the API
Here is the javadoc for DiskFileItemFactory.
The default FileItemFactory implementation. This implementation
creates FileItem instances which keep their content either in memory,
for smaller items, or in a temporary file on disk, for larger items.
The size threshold, above which content will be stored on disk, is
configurable, as is the directory in which temporary files will be
created.
If not otherwise configured, the default configuration values are as
follows:
Size threshold is 10KB.
Repository is the system default temp directory, as returned by System.getProperty("java.io.tmpdir").
Temporary files, which are created for file items, should be deleted
later on. The best way to do this is using a FileCleaningTracker,
which you can set on the DiskFileItemFactory. However, if you do use
such a tracker, then you must consider the following: Temporary files
are automatically deleted as soon as they are no longer needed. (More
precisely, when the corresponding instance of File is garbage
collected.) This is done by the so-called reaper thread, which is
started automatically when the class FileCleaner is loaded. It might
make sense to terminate that thread, for example, if your web
application ends. See the section on "Resource cleanup" in the users
guide of commons-fileupload.
So, yes close and cleanup are necessary, as FileItem may denote a real file on disk.
It's stored as a byte[] in the Java memory.
Tim Bray's article "Saving Data Safely" left me with open questions. Today, it's over a month old and I haven't seen any follow-up on it, so I decided to address the topic here.
One point of the article is that FileDescriptor.sync() should be called to be on the safe side when using FileOutputStream. At first, I was very irritated, because I never have seen any Java code doing a sync during the 12 years I do Java. Especially since coping with files is a pretty basic thing. Also, the standard JavaDoc of FileOutputStream never hinted at syncing (Java 1.0 - 6). After some research, I figured ext4 may actually be the first mainstream file system requiring syncing. (Are there other file systems where explicit syncing is advised?)
I appreciate some general thoughts on the matter, but I also have some specific questions:
When will Android do the sync to the file system? This could be periodic and additionally based on life cycle events (e.g. an app's process goes to the background).
Does FileDescriptor.sync() take care of syncing the meta data? That is syncing the directory of the changed file. Compare to FileChannel.force().
Usually, one does not directly write into the FileOutputStream. Here's my solution (do you agree?):
FileOutputStream fileOut = ctx.openFileOutput(file, Context.MODE_PRIVATE);
BufferedOutputStream out = new BufferedOutputStream(fileOut);
try {
out.write(something);
out.flush();
fileOut.getFD().sync();
} finally {
out.close();
}
Android will do the sync when it needs to -- such as when the screen turns off, shutting down the device, etc. If you are just looking at "normal" operation, explicit sync by applications is never needed.
The problem comes when the user pulls the battery out of their device (or does a hard reset of the kernel), and you want to ensure you don't lose any data.
So the first thing to realize: the issue is when power is suddenly lost, so a clean shutdown can not happen, and the question of what is going to happen in persistent storage at that point.
If you are just writing a single independent new file, it doesn't really matter what you do. The user could have pulled the battery while you were in the middle of writing, right before you started writing, etc. If you don't sync, it just means there is some longer time from when you are done writing during which pulling the battery will lose the data.
The big concern here is when you want to update a file. In that case, when you next read the file you want to have either the previous contents, or the new contents. You don't want to get something half-way written, or lose the data.
This is often done by writing the data in a new file, and then switching to that from the old file. Prior to ext4 you knew that, once you had finished writing a file, further operations on other files would not go on disk until the ones on that file, so you could safely delete the previous file or otherwise do operations that depend on your new file being fully written.
However now if you write the new file, then delete the old one, and the battery is pulled, when you next boot you may see that the old file is deleted and new file created but the contents of the new file is not complete. By doing the sync, you ensure that the new file is completely written at that point so can do further changes (such as deleting the old file) that depend on that state.
fileOut.getFD().sync(); should be on the finally clause, before the close().
sync() is way more important than close() considering durability.
So, everytime you want to 'finish' working on a file you should sync() it before close()ing it.
posix does not guarantee that pending writes will be written to disk when you issue a close().
I want to save a video file in C:\ by incrementing the file name e.g. video001.avi video002.avi video003.avi etc. i want to do this in java. The program is on
Problem in java programming on windows7 (working well in windows xp)
How do i increment the file name so that it saves without replacing the older file.
Using the File.createNewFile() you can atomically create a file and determine whether or not the current thread indeed created the file, the return value of that method is a boolean that will let you know whether or not a new file was created or not. Simply checking whether or not a file exists before you create it will help but will not guarantee that when you create and write to the file that the current thread created it.
You have two options:
just increment a counter, and rely on the fact that you're the only running process writing these files (and none exist already). So you don't need to check for clashes. This is (obviously) prone to error.
Use the File object (or Apache Commons FileUtils) to get the list of files, then increment a counter and determine if the corresponding file exists. If not, then write to it and exit. This is a brute force approach, but unless you're writing thousands of files, is quite acceptable performance-wise.
I need effective algorithm to keep only ten latest files on disk in particular folder to support some kind of publishing process. Only 10 files should present in this folder at any point of time. Please, give your advises what should be used here.
You can ask the File for the directory to listFiles, if there are more than 9 sort them by lastModified() and delete the files oldest (smallest number) to trim down to 9.
How about using a file system watcher like JNotify?
Register for events that you are interested in (for instance, Created event);
Mark your internal list for the number of files upon every created event.
As soon as you reach the 11th file, remove the file having oldest create date.
Or use Commons JCI FileAlterationMonitor (FAM) to monitor local filesystems and get notified about changes:
ReloadingClassLoader classloader = new ReloadingClassLoader(this.getClass().getClassLoader());
ReloadingListener listener = new ReloadingListener();
listener.addReloadNotificationListener(classloader);
FilesystemAlterationMonitor fam = new FilesystemAlterationMonitor();
fam.addListener(directory, listener);
fam.start();
This discussion may help you with file system watchers.
You'd have to poll the directory at regular intervals and delete everything that's older than the 10th oldest file in it.
Of course that leaves open to question what the "10th oldest file" actually is. The timestamp on the file might not indicate the date/time it was added to the folder after all.
So your system might actually need some independent way to keep track of files in the folder to determine when each was added, in order to delete files based on when they were
put there rather than how old the file actually is.
But that's a business requirement you don't provide (do you even know it yourself?).