Multi Thread zip4j - java

zip4j is a great library. But i run into a problem when using it in a class that uses a thread. The zip4j method is called from a class that implements thread and sometimes (not always) it leaves files uncompress and somtimes there are leftofer files with the extension *.zip345. Also the process returns net.lingala.zip4j.exception.ZipException: cannot rename modified zip file.
The method zip4jProcess is called from the class public method. Class name is: SZipInterface.class
The SZipInterface.class is initialized in the thread class ex: ThreadObj.class and instantiated per thread. No static method is used.
What is the cause of the problems? How do you fix it? Is zip4j thread safe?
Method:
private int zip4jProcess() {
int status = 0;
if (null != getInFiles() && getInFiles().length > 0) {
for (String file : getInFiles()) {
File sourceFile = new File(file);
ZipFile zipFile = null;
ZipParameters zipParams = new ZipParameters();
if (getPassword() != null
&& !getPassword().trim().equalsIgnoreCase("")) {
zipParams.setPassword(getPassword());
zipParams.setEncryptFiles(true);
zipParams
.setEncryptionMethod(Zip4jConstants.ENC_METHOD_STANDARD);
}
zipParams
.setCompressionLevel(Zip4jConstants.DEFLATE_LEVEL_NORMAL);
if (sourceFile.exists()) {
try {
zipFile = new ZipFile(getZipFileName());
if (zipFile.getFile().exists()) {
zipFile.addFile(sourceFile, zipParams);
if (log.isDebugEnabled()) {
log.debug("Adding: " + sourceFile.getName()
+ " to " + zipFile.getFile().getName()
+ " Pass: " + getPassword());
}
} else {
zipFile.createZipFile(sourceFile, zipParams);
if (log.isDebugEnabled()) {
log.debug("Creating: " + sourceFile.getName()
+ " to " + zipFile.getFile().getName()
+ " Pass: " + getPassword());
}
}
} catch (ZipException e) {
log.error(e);
status = 1;
}
}
}
}
return status;
}

I believe the times where you have leftovers or uncomprossed files may be when multiple threads try to use the same zip file (probably at zipFile.addFile(...)).
So try handling the addFile differently with concurrency in mind.
Their support forum said it's tricky and not currently supported - see the link for the limitations of doing it.
This can be quite tricky to implement, if not impossible to achieve,
especially when using encryption or when compressing the file (and not
just using the store method, which just copies the source file to the
zip without any compression). A current block of file being
compressed/decompressed depends on the previous block. So, if multiple
threads were to read or write, these threads cannot do this process
simultaneously, but have to wait until the block n-1 (if n is the
current block) is read/wrote. So, its as good as running the process
in the same thread.
Writing different files in different threads to a zip file (each
thread handling a unique file in the zip) can be tricky as well. For
example: AES encryption requires a unique number (as part of salt
calculation) for each file in the zip. And another example: if a zip
file is being created and multiple number of files being added (with
compression), then the second thread, which will start writing the
second file to the zip should know exactly at which location in the
zip file to start writing, and this cannot be determined until the
first thread is done writing.
Some compression algorithms, like LZMA/LZMA2, support multithreading.
Unfortunately, these compression methods are not supported by Zip4j at
the moment.
Full text of their response (in case the post gets removed).

Related

Java watch service appears to recreate deleted files. What is going on?

When a directory monitored by a WatchService gets deleted, its parent directory does not immediately reflect the deletion in its File's listFiles method and cannot be deleted. Until the entire service is explicitly stopped the consequences for the parent appear to be:
The recommended recursive solution for deleting a non-empty directory failing.
deleteOnExit not being carried out on normal termination
Calls to delete returning false and having no effect on the filesystem.
To demonstrate, this test code:
import java.io.*;
import java.nio.file.*;
class DirectoryTester {
static WatchService watcher;
static {
try{watcher = FileSystems.getDefault().newWatchService();}
catch (IOException e) {e.printStackTrace();}
}
public static void main(String[] args) throws IOException {
String SEPARATE = System.getProperty("file.separator");
String testDirName = System.getProperty("user.dir") + SEPARATE + "testDir";
String subDirName = testDirName + SEPARATE + "subDir";
String fileName = subDirName + SEPARATE +"aFile";
create(fileName);
Paths.get(subDirName).register(watcher, StandardWatchEventKinds.ENTRY_DELETE);
delete(new File(testDirName));
}
static void create(String nameOfFile) throws IOException {
new File(nameOfFile).getParentFile().mkdirs();
Files.createFile(Paths.get(nameOfFile));
System.out.println("Created " + nameOfFile);
}
static void delete(File toDelete) throws IOException {
if (toDelete.isDirectory())
for (File c : toDelete.listFiles())
delete(c);
int numContainedFiles = toDelete.listFiles() != null ? toDelete.listFiles().length : 0;
if (!toDelete.delete()) {
System.out.println("Failed to delete " + toDelete + " containing " + numContainedFiles);
}
else {
System.out.println("Deleted " + toDelete + " containing " + numContainedFiles);
}
}
}
gives the following output on windows, which corresponds with testDir not being deleted on the filesystem.
Created C:\Dropbox\CodeSpace\JavaTestbed\src\testDir\subDir\aFile
Deleted C:\Dropbox\CodeSpace\JavaTestbed\src\testDir\subDir\aFile containing 0
Deleted C:\Dropbox\CodeSpace\JavaTestbed\src\testDir\subDir containing 0
Failed to delete C:\Dropbox\CodeSpace\JavaTestbed\src\testDir containing 1
If I put a breakpoint after the subDir deletion I can see that it has actually been deleted on the filesystem. Resuming from the breakpoint causes the last deletion to suceed, suggesting that this might be an issue with the visibility of changes made by the watch service thread. Does anyone know what is going on here, and if it is a bug? What I am actually trying to do is to delete directories that are monitored without stopping the monitoring on other directories, given that there does not appear to be an unregister path method provided by the API what are other standard Java ways of accomplishing this?
possibly related:
http://bugs.sun.com/view_bug.do?bug_id=6972833
The WatchService has an open handle to each watched directory. If a a watch directory is deleted then the WatchService closes the handle so that the directory entry can be removed from the parent directory. A problem arises for utilities and application that expect to be able to delete the parent directory immediately as it can take a few milliseconds for the watch service to get the notificationa and close the handle. If during that time that the tool attempts to delete the parent directory then it will fail. We don't have a solution to this issue at this time.

Check if file exists without creating it

If I do this:
File f = new File("c:\\text.txt");
if (f.exists()) {
System.out.println("File exists");
} else {
System.out.println("File not found!");
}
Then the file gets created and always returns "File exists". Is it possible to check if a file exists without creating it?
EDIT:
I forgot to mention that it's in a for loop. So here's the real thing:
for (int i = 0; i < 10; i++) {
File file = new File("c:\\text" + i + ".txt");
System.out.println("New file created: " + file.getPath());
}
When you instantiate a File, you're not creating anything on disk but just building an object on which you can call some methods, like exists().
That's fine and cheap, don't try to avoid this instantiation.
The File instance has only two fields:
private String path;
private transient int prefixLength;
And here is the constructor :
public File(String pathname) {
if (pathname == null) {
throw new NullPointerException();
}
this.path = fs.normalize(pathname);
this.prefixLength = fs.prefixLength(this.path);
}
As you can see, the File instance is just an encapsulation of the path. Creating it in order to call exists() is the correct way to proceed. Don't try to optimize it away.
Starting from Java 7 you can use java.nio.file.Files.exists:
Path p = Paths.get("C:\\Users\\first.last");
boolean exists = Files.exists(p);
boolean notExists = Files.notExists(p);
if (exists) {
System.out.println("File exists!");
} else if (notExists) {
System.out.println("File doesn't exist!");
} else {
System.out.println("File's status is unknown!");
}
In the Oracle tutorial you can find some details about this:
The methods in the Path class are syntactic, meaning that they operate on the Path instance. But eventually you must access the file system to verify that a particular Path exists, or does not exist. You can do so with the exists(Path, LinkOption...) and the notExists(Path, LinkOption...) methods. Note that !Files.exists(path) is not equivalent to Files.notExists(path). When you are testing a file's existence, three results are possible:
The file is verified to exist.
The file is verified to not exist.
The file's status is unknown. This result can occur when the program does not have access to the file.
If both exists and notExists return false, the existence of the file cannot be verified.
Creating a File instance does not create a file on the file system, so the posted code will do what you require.
The Files.exists method has noticeably poor performance in JDK 8, and can slow an application significantly when used to check files that don't actually exist.
This can be applied too for Files.noExists, Files.isDirectory and Files.isRegularFile
According this you can use the following :
Paths.get("file_path").toFile().exists()

Synchronizing When Deleting Or Updating an IO File

Specifically I am using lucene to perform full text searching and in certain scenarios the index file might become corrupted or simply has not been created yet at which point I would delete the file and rewrite the index to said file. My question pertains to the actual act of deleting and re-writing to a file in a multi threaded Java program.
Will synchronizing protect the IO File while its being deleted and restored? In other words will it block access to another thread coming along and attempting to use the same method and begin rewriting while its already in the process?
The setDirectory method needs to be run before any other methods in the class will work (it will throw errors otherwise), so does the way I have the synchronization setup protect me from any multi threaded mishaps?
When another thread attempts to use the setDirectory method and the buildCompleteIndex method is already in progress, will the thread simply wait for that to finish and then its check on whether the path exists will pass and it will move on to opening the index?
In lucene do I have to synchronize when writing, deleting, or searching the index or can these tasks be done concurrently?
public void setDirectory(int organizationId) throws IOException {
this.organizationId = organizationId;
File path = new File(INDEX_PATH + "/" + String.valueOf(organizationId));
//If path does not exist, create it and create new index for organization
synchronized(this) {
if(!path.exists()) {
path.mkdirs();
buildCompleteIndex(organizationId, false);
}
}
this.directory = FSDirectory.open(path); //Open directory
}
private void buildCompleteIndex(int organizationId, boolean rebuildDir) {
if(rebuildDir) {
File path = new File(INDEX_PATH + "/" + String.valueOf(organizationId));
try {
Utils.deleteDirectory(path);
} catch (IOException e) {
throw new LuceneIndexException("Error rebuilding index directory.", e);
}
path.mkdirs();
}
List<Tag> tagList = tagDAO.findAll(organizationId);
for(Tag tag : tagList) {
add(tag);
}
}

Creating a Temp Dir in Java [duplicate]

Is there a standard and reliable way of creating a temporary directory inside a Java application? There's an entry in Java's issue database, which has a bit of code in the comments, but I wonder if there is a standard solution to be found in one of the usual libraries (Apache Commons etc.) ?
If you are using JDK 7 use the new Files.createTempDirectory class to create the temporary directory.
Path tempDirWithPrefix = Files.createTempDirectory(prefix);
Before JDK 7 this should do it:
public static File createTempDirectory()
throws IOException
{
final File temp;
temp = File.createTempFile("temp", Long.toString(System.nanoTime()));
if(!(temp.delete()))
{
throw new IOException("Could not delete temp file: " + temp.getAbsolutePath());
}
if(!(temp.mkdir()))
{
throw new IOException("Could not create temp directory: " + temp.getAbsolutePath());
}
return (temp);
}
You could make better exceptions (subclass IOException) if you want.
The Google Guava library has a ton of helpful utilities. One of note here is the Files class. It has a bunch of useful methods including:
File myTempDir = Files.createTempDir();
This does exactly what you asked for in one line. If you read the documentation here you'll see that the proposed adaptation of File.createTempFile("install", "dir") typically introduces security vulnerabilities.
If you need a temporary directory for testing and you are using jUnit, #Rule together with TemporaryFolder solves your problem:
#Rule
public TemporaryFolder folder = new TemporaryFolder();
From the documentation:
The TemporaryFolder Rule allows creation of files and folders that are guaranteed to be deleted when the test method finishes (whether it passes or fails)
Update:
If you are using JUnit Jupiter (version 5.1.1 or greater), you have the option to use JUnit Pioneer which is the JUnit 5 Extension Pack.
Copied from the project documentation:
For example, the following test registers the extension for a single test method, creates and writes a file to the temporary directory and checks its content.
#Test
#ExtendWith(TempDirectory.class)
void test(#TempDir Path tempDir) {
Path file = tempDir.resolve("test.txt");
writeFile(file);
assertExpectedFileContent(file);
}
More info in the JavaDoc and the JavaDoc of TempDirectory
Gradle:
dependencies {
testImplementation 'org.junit-pioneer:junit-pioneer:0.1.2'
}
Maven:
<dependency>
<groupId>org.junit-pioneer</groupId>
<artifactId>junit-pioneer</artifactId>
<version>0.1.2</version>
<scope>test</scope>
</dependency>
Update 2:
The #TempDir annotation was added to the JUnit Jupiter 5.4.0 release as an experimental feature. Example copied from the JUnit 5 User Guide:
#Test
void writeItemsToFile(#TempDir Path tempDir) throws IOException {
Path file = tempDir.resolve("test.txt");
new ListWriter(file).write("a", "b", "c");
assertEquals(singletonList("a,b,c"), Files.readAllLines(file));
}
Naively written code to solve this problem suffers from race conditions, including several of the answers here. Historically you could think carefully about race conditions and write it yourself, or you could use a third-party library like Google's Guava (as Spina's answer suggested.) Or you could write buggy code.
But as of JDK 7, there is good news! The Java standard library itself now provides a properly working (non-racy) solution to this problem. You want java.nio.file.Files#createTempDirectory(). From the documentation:
public static Path createTempDirectory(Path dir,
String prefix,
FileAttribute<?>... attrs)
throws IOException
Creates a new directory in the specified directory, using the given prefix to generate its name. The resulting Path is associated with the same FileSystem as the given directory.
The details as to how the name of the directory is constructed is implementation dependent and therefore not specified. Where possible the prefix is used to construct candidate names.
This effectively resolves the embarrassingly ancient bug report in the Sun bug tracker which asked for just such a function.
This is the source code to the Guava library's Files.createTempDir(). It's nowhere as complex as you might think:
public static File createTempDir() {
File baseDir = new File(System.getProperty("java.io.tmpdir"));
String baseName = System.currentTimeMillis() + "-";
for (int counter = 0; counter < TEMP_DIR_ATTEMPTS; counter++) {
File tempDir = new File(baseDir, baseName + counter);
if (tempDir.mkdir()) {
return tempDir;
}
}
throw new IllegalStateException("Failed to create directory within "
+ TEMP_DIR_ATTEMPTS + " attempts (tried "
+ baseName + "0 to " + baseName + (TEMP_DIR_ATTEMPTS - 1) + ')');
}
By default:
private static final int TEMP_DIR_ATTEMPTS = 10000;
See here
Do not use deleteOnExit() even if you explicitly delete it later.
Google 'deleteonexit is evil' for more info, but the gist of the problem is:
deleteOnExit() only deletes for normal JVM shutdowns, not crashes or killing the JVM process.
deleteOnExit() only deletes on JVM shutdown - not good for long running server processes because:
The most evil of all - deleteOnExit() consumes memory for each temp file entry. If your process is running for months, or creates a lot of temp files in a short time, you consume memory and never release it until the JVM shuts down.
As of Java 1.7 createTempDirectory(prefix, attrs) and createTempDirectory(dir, prefix, attrs) are included in java.nio.file.Files
Example:
File tempDir = Files.createTempDirectory("foobar").toFile();
This is what I decided to do for my own code:
/**
* Create a new temporary directory. Use something like
* {#link #recursiveDelete(File)} to clean this directory up since it isn't
* deleted automatically
* #return the new directory
* #throws IOException if there is an error creating the temporary directory
*/
public static File createTempDir() throws IOException
{
final File sysTempDir = new File(System.getProperty("java.io.tmpdir"));
File newTempDir;
final int maxAttempts = 9;
int attemptCount = 0;
do
{
attemptCount++;
if(attemptCount > maxAttempts)
{
throw new IOException(
"The highly improbable has occurred! Failed to " +
"create a unique temporary directory after " +
maxAttempts + " attempts.");
}
String dirName = UUID.randomUUID().toString();
newTempDir = new File(sysTempDir, dirName);
} while(newTempDir.exists());
if(newTempDir.mkdirs())
{
return newTempDir;
}
else
{
throw new IOException(
"Failed to create temp dir named " +
newTempDir.getAbsolutePath());
}
}
/**
* Recursively delete file or directory
* #param fileOrDir
* the file or dir to delete
* #return
* true iff all files are successfully deleted
*/
public static boolean recursiveDelete(File fileOrDir)
{
if(fileOrDir.isDirectory())
{
// recursively delete contents
for(File innerFile: fileOrDir.listFiles())
{
if(!FileUtilities.recursiveDelete(innerFile))
{
return false;
}
}
}
return fileOrDir.delete();
}
Well, "createTempFile" actually creates the file. So why not just delete it first, and then do the mkdir on it?
This code should work reasonably well:
public static File createTempDir() {
final String baseTempPath = System.getProperty("java.io.tmpdir");
Random rand = new Random();
int randomInt = 1 + rand.nextInt();
File tempDir = new File(baseTempPath + File.separator + "tempDir" + randomInt);
if (tempDir.exists() == false) {
tempDir.mkdir();
}
tempDir.deleteOnExit();
return tempDir;
}
As discussed in this RFE and its comments, you could call tempDir.delete() first. Or you could use System.getProperty("java.io.tmpdir") and create a directory there. Either way, you should remember to call tempDir.deleteOnExit(), or the file won't be deleted after you're done.
Just for completion, this is the code from google guava library. It is not my code, but I think it is valueable to show it here in this thread.
/** Maximum loop count when creating temp directories. */
private static final int TEMP_DIR_ATTEMPTS = 10000;
/**
* Atomically creates a new directory somewhere beneath the system's temporary directory (as
* defined by the {#code java.io.tmpdir} system property), and returns its name.
*
* <p>Use this method instead of {#link File#createTempFile(String, String)} when you wish to
* create a directory, not a regular file. A common pitfall is to call {#code createTempFile},
* delete the file and create a directory in its place, but this leads a race condition which can
* be exploited to create security vulnerabilities, especially when executable files are to be
* written into the directory.
*
* <p>This method assumes that the temporary volume is writable, has free inodes and free blocks,
* and that it will not be called thousands of times per second.
*
* #return the newly-created directory
* #throws IllegalStateException if the directory could not be created
*/
public static File createTempDir() {
File baseDir = new File(System.getProperty("java.io.tmpdir"));
String baseName = System.currentTimeMillis() + "-";
for (int counter = 0; counter < TEMP_DIR_ATTEMPTS; counter++) {
File tempDir = new File(baseDir, baseName + counter);
if (tempDir.mkdir()) {
return tempDir;
}
}
throw new IllegalStateException(
"Failed to create directory within "
+ TEMP_DIR_ATTEMPTS
+ " attempts (tried "
+ baseName
+ "0 to "
+ baseName
+ (TEMP_DIR_ATTEMPTS - 1)
+ ')');
}
I got the same problem so this is just another answer for those who are interested, and it's similar to one of the above:
public static final String tempDir = System.getProperty("java.io.tmpdir")+"tmp"+System.nanoTime();
static {
File f = new File(tempDir);
if(!f.exists())
f.mkdir();
}
And for my application, I decided that to add in a option to clear the temp on exit so I added in a shut-down hook:
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
//stackless deletion
String root = MainWindow.tempDir;
Stack<String> dirStack = new Stack<String>();
dirStack.push(root);
while(!dirStack.empty()) {
String dir = dirStack.pop();
File f = new File(dir);
if(f.listFiles().length==0)
f.delete();
else {
dirStack.push(dir);
for(File ff: f.listFiles()) {
if(ff.isFile())
ff.delete();
else if(ff.isDirectory())
dirStack.push(ff.getPath());
}
}
}
}
});
The method delete all subdirs and files before deleting the temp, without using the callstack (which is totally optional and you could do it with recursion at this point), but I want to be on the safe side.
As you can see in the other answers, no standard approach has arisen.
Hence you already mentioned Apache Commons, I propose the following approach using FileUtils from Apache Commons IO:
/**
* Creates a temporary subdirectory in the standard temporary directory.
* This will be automatically deleted upon exit.
*
* #param prefix
* the prefix used to create the directory, completed by a
* current timestamp. Use for instance your application's name
* #return the directory
*/
public static File createTempDirectory(String prefix) {
final File tmp = new File(FileUtils.getTempDirectory().getAbsolutePath()
+ "/" + prefix + System.currentTimeMillis());
tmp.mkdir();
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
try {
FileUtils.deleteDirectory(tmp);
} catch (IOException e) {
e.printStackTrace();
}
}
});
return tmp;
}
This is preferred since apache commons the library that comes as closest to the asked "standard" and works with both JDK 7 and older versions. This also returns an "old" File instance (which is stream based) and not a "new" Path instance (which is buffer based and would be the result of JDK7's getTemporaryDirectory() method) -> Therefore it returns what most people need when they want to create a temporary directory.
Try this small example:
Code:
try {
Path tmpDir = Files.createTempDirectory("tmpDir");
System.out.println(tmpDir.toString());
Files.delete(tmpDir);
} catch (IOException e) {
e.printStackTrace();
}
Imports:
java.io.IOException
java.nio.file.Files
java.nio.file.Path
Console output on Windows machine:
C:\Users\userName\AppData\Local\Temp\tmpDir2908538301081367877
Comment:
Files.createTempDirectory generates unique ID atomatically - 2908538301081367877.
Note:
Read the following for deleting directories recursively:
Delete directories recursively in Java
I like the multiple attempts at creating a unique name but even this solution does not rule out a race condition. Another process can slip in after the test for exists() and the if(newTempDir.mkdirs()) method invocation. I have no idea how to completely make this safe without resorting to native code, which I presume is what's buried inside File.createTempFile().
Before Java 7 you could also:
File folder = File.createTempFile("testFileUtils", ""); // no suffix
folder.delete();
folder.mkdirs();
folder.deleteOnExit();
Using File#createTempFile and delete to create a unique name for the directory seems ok. You should add a ShutdownHook to delete the directory (recursively) on JVM shutdown.

How to create a temporary directory/folder in Java?

Is there a standard and reliable way of creating a temporary directory inside a Java application? There's an entry in Java's issue database, which has a bit of code in the comments, but I wonder if there is a standard solution to be found in one of the usual libraries (Apache Commons etc.) ?
If you are using JDK 7 use the new Files.createTempDirectory class to create the temporary directory.
Path tempDirWithPrefix = Files.createTempDirectory(prefix);
Before JDK 7 this should do it:
public static File createTempDirectory()
throws IOException
{
final File temp;
temp = File.createTempFile("temp", Long.toString(System.nanoTime()));
if(!(temp.delete()))
{
throw new IOException("Could not delete temp file: " + temp.getAbsolutePath());
}
if(!(temp.mkdir()))
{
throw new IOException("Could not create temp directory: " + temp.getAbsolutePath());
}
return (temp);
}
You could make better exceptions (subclass IOException) if you want.
The Google Guava library has a ton of helpful utilities. One of note here is the Files class. It has a bunch of useful methods including:
File myTempDir = Files.createTempDir();
This does exactly what you asked for in one line. If you read the documentation here you'll see that the proposed adaptation of File.createTempFile("install", "dir") typically introduces security vulnerabilities.
If you need a temporary directory for testing and you are using jUnit, #Rule together with TemporaryFolder solves your problem:
#Rule
public TemporaryFolder folder = new TemporaryFolder();
From the documentation:
The TemporaryFolder Rule allows creation of files and folders that are guaranteed to be deleted when the test method finishes (whether it passes or fails)
Update:
If you are using JUnit Jupiter (version 5.1.1 or greater), you have the option to use JUnit Pioneer which is the JUnit 5 Extension Pack.
Copied from the project documentation:
For example, the following test registers the extension for a single test method, creates and writes a file to the temporary directory and checks its content.
#Test
#ExtendWith(TempDirectory.class)
void test(#TempDir Path tempDir) {
Path file = tempDir.resolve("test.txt");
writeFile(file);
assertExpectedFileContent(file);
}
More info in the JavaDoc and the JavaDoc of TempDirectory
Gradle:
dependencies {
testImplementation 'org.junit-pioneer:junit-pioneer:0.1.2'
}
Maven:
<dependency>
<groupId>org.junit-pioneer</groupId>
<artifactId>junit-pioneer</artifactId>
<version>0.1.2</version>
<scope>test</scope>
</dependency>
Update 2:
The #TempDir annotation was added to the JUnit Jupiter 5.4.0 release as an experimental feature. Example copied from the JUnit 5 User Guide:
#Test
void writeItemsToFile(#TempDir Path tempDir) throws IOException {
Path file = tempDir.resolve("test.txt");
new ListWriter(file).write("a", "b", "c");
assertEquals(singletonList("a,b,c"), Files.readAllLines(file));
}
Naively written code to solve this problem suffers from race conditions, including several of the answers here. Historically you could think carefully about race conditions and write it yourself, or you could use a third-party library like Google's Guava (as Spina's answer suggested.) Or you could write buggy code.
But as of JDK 7, there is good news! The Java standard library itself now provides a properly working (non-racy) solution to this problem. You want java.nio.file.Files#createTempDirectory(). From the documentation:
public static Path createTempDirectory(Path dir,
String prefix,
FileAttribute<?>... attrs)
throws IOException
Creates a new directory in the specified directory, using the given prefix to generate its name. The resulting Path is associated with the same FileSystem as the given directory.
The details as to how the name of the directory is constructed is implementation dependent and therefore not specified. Where possible the prefix is used to construct candidate names.
This effectively resolves the embarrassingly ancient bug report in the Sun bug tracker which asked for just such a function.
This is the source code to the Guava library's Files.createTempDir(). It's nowhere as complex as you might think:
public static File createTempDir() {
File baseDir = new File(System.getProperty("java.io.tmpdir"));
String baseName = System.currentTimeMillis() + "-";
for (int counter = 0; counter < TEMP_DIR_ATTEMPTS; counter++) {
File tempDir = new File(baseDir, baseName + counter);
if (tempDir.mkdir()) {
return tempDir;
}
}
throw new IllegalStateException("Failed to create directory within "
+ TEMP_DIR_ATTEMPTS + " attempts (tried "
+ baseName + "0 to " + baseName + (TEMP_DIR_ATTEMPTS - 1) + ')');
}
By default:
private static final int TEMP_DIR_ATTEMPTS = 10000;
See here
Do not use deleteOnExit() even if you explicitly delete it later.
Google 'deleteonexit is evil' for more info, but the gist of the problem is:
deleteOnExit() only deletes for normal JVM shutdowns, not crashes or killing the JVM process.
deleteOnExit() only deletes on JVM shutdown - not good for long running server processes because:
The most evil of all - deleteOnExit() consumes memory for each temp file entry. If your process is running for months, or creates a lot of temp files in a short time, you consume memory and never release it until the JVM shuts down.
As of Java 1.7 createTempDirectory(prefix, attrs) and createTempDirectory(dir, prefix, attrs) are included in java.nio.file.Files
Example:
File tempDir = Files.createTempDirectory("foobar").toFile();
This is what I decided to do for my own code:
/**
* Create a new temporary directory. Use something like
* {#link #recursiveDelete(File)} to clean this directory up since it isn't
* deleted automatically
* #return the new directory
* #throws IOException if there is an error creating the temporary directory
*/
public static File createTempDir() throws IOException
{
final File sysTempDir = new File(System.getProperty("java.io.tmpdir"));
File newTempDir;
final int maxAttempts = 9;
int attemptCount = 0;
do
{
attemptCount++;
if(attemptCount > maxAttempts)
{
throw new IOException(
"The highly improbable has occurred! Failed to " +
"create a unique temporary directory after " +
maxAttempts + " attempts.");
}
String dirName = UUID.randomUUID().toString();
newTempDir = new File(sysTempDir, dirName);
} while(newTempDir.exists());
if(newTempDir.mkdirs())
{
return newTempDir;
}
else
{
throw new IOException(
"Failed to create temp dir named " +
newTempDir.getAbsolutePath());
}
}
/**
* Recursively delete file or directory
* #param fileOrDir
* the file or dir to delete
* #return
* true iff all files are successfully deleted
*/
public static boolean recursiveDelete(File fileOrDir)
{
if(fileOrDir.isDirectory())
{
// recursively delete contents
for(File innerFile: fileOrDir.listFiles())
{
if(!FileUtilities.recursiveDelete(innerFile))
{
return false;
}
}
}
return fileOrDir.delete();
}
Well, "createTempFile" actually creates the file. So why not just delete it first, and then do the mkdir on it?
This code should work reasonably well:
public static File createTempDir() {
final String baseTempPath = System.getProperty("java.io.tmpdir");
Random rand = new Random();
int randomInt = 1 + rand.nextInt();
File tempDir = new File(baseTempPath + File.separator + "tempDir" + randomInt);
if (tempDir.exists() == false) {
tempDir.mkdir();
}
tempDir.deleteOnExit();
return tempDir;
}
As discussed in this RFE and its comments, you could call tempDir.delete() first. Or you could use System.getProperty("java.io.tmpdir") and create a directory there. Either way, you should remember to call tempDir.deleteOnExit(), or the file won't be deleted after you're done.
Just for completion, this is the code from google guava library. It is not my code, but I think it is valueable to show it here in this thread.
/** Maximum loop count when creating temp directories. */
private static final int TEMP_DIR_ATTEMPTS = 10000;
/**
* Atomically creates a new directory somewhere beneath the system's temporary directory (as
* defined by the {#code java.io.tmpdir} system property), and returns its name.
*
* <p>Use this method instead of {#link File#createTempFile(String, String)} when you wish to
* create a directory, not a regular file. A common pitfall is to call {#code createTempFile},
* delete the file and create a directory in its place, but this leads a race condition which can
* be exploited to create security vulnerabilities, especially when executable files are to be
* written into the directory.
*
* <p>This method assumes that the temporary volume is writable, has free inodes and free blocks,
* and that it will not be called thousands of times per second.
*
* #return the newly-created directory
* #throws IllegalStateException if the directory could not be created
*/
public static File createTempDir() {
File baseDir = new File(System.getProperty("java.io.tmpdir"));
String baseName = System.currentTimeMillis() + "-";
for (int counter = 0; counter < TEMP_DIR_ATTEMPTS; counter++) {
File tempDir = new File(baseDir, baseName + counter);
if (tempDir.mkdir()) {
return tempDir;
}
}
throw new IllegalStateException(
"Failed to create directory within "
+ TEMP_DIR_ATTEMPTS
+ " attempts (tried "
+ baseName
+ "0 to "
+ baseName
+ (TEMP_DIR_ATTEMPTS - 1)
+ ')');
}
I got the same problem so this is just another answer for those who are interested, and it's similar to one of the above:
public static final String tempDir = System.getProperty("java.io.tmpdir")+"tmp"+System.nanoTime();
static {
File f = new File(tempDir);
if(!f.exists())
f.mkdir();
}
And for my application, I decided that to add in a option to clear the temp on exit so I added in a shut-down hook:
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
//stackless deletion
String root = MainWindow.tempDir;
Stack<String> dirStack = new Stack<String>();
dirStack.push(root);
while(!dirStack.empty()) {
String dir = dirStack.pop();
File f = new File(dir);
if(f.listFiles().length==0)
f.delete();
else {
dirStack.push(dir);
for(File ff: f.listFiles()) {
if(ff.isFile())
ff.delete();
else if(ff.isDirectory())
dirStack.push(ff.getPath());
}
}
}
}
});
The method delete all subdirs and files before deleting the temp, without using the callstack (which is totally optional and you could do it with recursion at this point), but I want to be on the safe side.
As you can see in the other answers, no standard approach has arisen.
Hence you already mentioned Apache Commons, I propose the following approach using FileUtils from Apache Commons IO:
/**
* Creates a temporary subdirectory in the standard temporary directory.
* This will be automatically deleted upon exit.
*
* #param prefix
* the prefix used to create the directory, completed by a
* current timestamp. Use for instance your application's name
* #return the directory
*/
public static File createTempDirectory(String prefix) {
final File tmp = new File(FileUtils.getTempDirectory().getAbsolutePath()
+ "/" + prefix + System.currentTimeMillis());
tmp.mkdir();
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
try {
FileUtils.deleteDirectory(tmp);
} catch (IOException e) {
e.printStackTrace();
}
}
});
return tmp;
}
This is preferred since apache commons the library that comes as closest to the asked "standard" and works with both JDK 7 and older versions. This also returns an "old" File instance (which is stream based) and not a "new" Path instance (which is buffer based and would be the result of JDK7's getTemporaryDirectory() method) -> Therefore it returns what most people need when they want to create a temporary directory.
Try this small example:
Code:
try {
Path tmpDir = Files.createTempDirectory("tmpDir");
System.out.println(tmpDir.toString());
Files.delete(tmpDir);
} catch (IOException e) {
e.printStackTrace();
}
Imports:
java.io.IOException
java.nio.file.Files
java.nio.file.Path
Console output on Windows machine:
C:\Users\userName\AppData\Local\Temp\tmpDir2908538301081367877
Comment:
Files.createTempDirectory generates unique ID atomatically - 2908538301081367877.
Note:
Read the following for deleting directories recursively:
Delete directories recursively in Java
I like the multiple attempts at creating a unique name but even this solution does not rule out a race condition. Another process can slip in after the test for exists() and the if(newTempDir.mkdirs()) method invocation. I have no idea how to completely make this safe without resorting to native code, which I presume is what's buried inside File.createTempFile().
Before Java 7 you could also:
File folder = File.createTempFile("testFileUtils", ""); // no suffix
folder.delete();
folder.mkdirs();
folder.deleteOnExit();
Using File#createTempFile and delete to create a unique name for the directory seems ok. You should add a ShutdownHook to delete the directory (recursively) on JVM shutdown.

Categories