In Java, I'm working with code running under WinXP that creates a file like this:
public synchronized void store(Properties props, byte[] data) {
try {
File file = filenameBasedOnProperties(props);
if ( file.exists() ) {
return;
}
File temp = File.createTempFile("tempfile", null);
FileOutputStream out = new FileOutputStream(temp);
out.write(data);
out.flush();
out.close();
file.getParentFile().mkdirs();
temp.renameTo(file);
}
catch (IOException ex) {
// Complain and whine and stuff
}
}
Sometimes, when a file is created this way, it's just about totally inaccessible from outside the code (though the code responsible for opening and reading the file has no problem), even when the application isn't running. When accessed via Windows Explorer, I can't move, rename, delete, or even open the file. Under Cygwin, I get the following when I ls -l the directory:
ls: cannot access [big-honkin-filename]
total 0
?????????? ? ? ? ? ? [big-honkin-filename]
As implied, the filenames are big, but under the 260-character max for XP (though they are slightly over 200 characters).
To further add to the sense that my computer just wants me to feel stupid, sometimes the files created by this code are perfectly normal. The only pattern I've spotted is that once one file in the directory "locks", the rest are screwed.
Anybody ever run into something like this before, or have any insights into what's going on here?
Make sure you always close the stream in a finally block. In your case if an exception is thrown the stream might not get closed and will leak a file handle. You could use procexp from SysInternals to see which process holds the handle to the file.
Although, by definition, NTFS should handle path length up to 2^15-1, in practice the length of paths is limited to 255.
You can create files with a longer path name (filename including parent folder names), but you cannot access them afterwards. The error I get in these cases is that the file could not be found. To get rid of these files, I have to shorten the names of parent folders, until the path length is short enough.
Related
I need to write a custom batch File renamer. I've got the bulk of it done except I can't figure out how to check if a file is already open. I'm just using the java.io.File package and there is a canWrite() method but that doesn't seem to test if the file is in use by another program. Any ideas on how I can make this work?
Using the Apache Commons IO library...
boolean isFileUnlocked = false;
try {
org.apache.commons.io.FileUtils.touch(yourFile);
isFileUnlocked = true;
} catch (IOException e) {
isFileUnlocked = false;
}
if(isFileUnlocked){
// Do stuff you need to do with a file that is NOT locked.
} else {
// Do stuff you need to do with a file that IS locked
}
(The Q&A is about how to deal with Windows "open file" locks ... not how implement this kind of locking portably.)
This whole issue is fraught with portability issues and race conditions:
You could try to use FileLock, but it is not necessarily supported for your OS and/or filesystem.
It appears that on Windows you may be unable to use FileLock if another application has opened the file in a particular way.
Even if you did manage to use FileLock or something else, you've still got the problem that something may come in and open the file between you testing the file and doing the rename.
A simpler though non-portable solution is to just try the rename (or whatever it is you are trying to do) and diagnose the return value and / or any Java exceptions that arise due to opened files.
Notes:
If you use the Files API instead of the File API you will get more information in the event of a failure.
On systems (e.g. Linux) where you are allowed to rename a locked or open file, you won't get any failure result or exceptions. The operation will just succeed. However, on such systems you generally don't need to worry if a file is already open, since the OS doesn't lock files on open.
// TO CHECK WHETHER A FILE IS OPENED
// OR NOT (not for .txt files)
// the file we want to check
String fileName = "C:\\Text.xlsx";
File file = new File(fileName);
// try to rename the file with the same name
File sameFileName = new File(fileName);
if(file.renameTo(sameFileName)){
// if the file is renamed
System.out.println("file is closed");
}else{
// if the file didnt accept the renaming operation
System.out.println("file is opened");
}
On Windows I found the answer https://stackoverflow.com/a/13706972/3014879 using
fileIsLocked = !file.renameTo(file)
most useful, as it avoids false positives when processing write protected (or readonly) files.
org.apache.commons.io.FileUtils.touch(yourFile) doesn't check if your file is open or not. Instead, it changes the timestamp of the file to the current time.
I used IOException and it works just fine:
try
{
String filePath = "C:\sheet.xlsx";
FileWriter fw = new FileWriter(filePath );
}
catch (IOException e)
{
System.out.println("File is open");
}
I don't think you'll ever get a definitive solution for this, the operating system isn't necessarily going to tell you if the file is open or not.
You might get some mileage out of java.nio.channels.FileLock, although the javadoc is loaded with caveats.
Hi I really hope this helps.
I tried all the options before and none really work on Windows. The only think that helped me accomplish this was trying to move the file. Event to the same place under an ATOMIC_MOVE. If the file is being written by another program or Java thread, this definitely will produce an Exception.
try{
Files.move(Paths.get(currentFile.getPath()),
Paths.get(currentFile.getPath()), StandardCopyOption.ATOMIC_MOVE);
// DO YOUR STUFF HERE SINCE IT IS NOT BEING WRITTEN BY ANOTHER PROGRAM
} catch (Exception e){
// DO NOT WRITE THEN SINCE THE FILE IS BEING WRITTEN BY ANOTHER PROGRAM
}
If file is in use FileOutputStream fileOutputStream = new FileOutputStream(file); returns java.io.FileNotFoundException with 'The process cannot access the file because it is being used by another process' in the exception message.
I had some new code using the commons-io FileUtils.openOutputStream(File) method, for a file that doesn't exist at the point of the call. This was failing with a "FileNotFoundException". I first thought this was a bug in commons-io, but then I realized that it's just calling "new FileOutputStream(file, append)", which is also supposed to create the file if it doesn't exist.
I then added code right before my call to FileUtils.openOutputStream(File) like the following:
if (!file.exists()) {
logger.info("Parent file exists: " + file.getParentFile().exists());
try {
file.createNewFile();
}
catch (Exception ex) {
logger.error("Creating file failed", ex);
}
}
This prints "true" for the parent file, and then "java.io.IOException: The system cannot find the path specified". I googled for this situation, and some people were hitting this if they went past the supposed 260 character limit for a file path on Windows. I thought that might be relevant, but my file path is only 230 characters long.
I also tried an experiment of trying to "touch" the same file path in my Cygwin bash shell, and it had no trouble doing that.
Update:
So I took the partial advice of trying to use Paths & Files to do this instead of just "File". My incoming parameter is a "File", so I can't do anything about that. I added the following code:
try {
Path path = Paths.get(file.getAbsolutePath()).toAbsolutePath();
if (!Files.exists(path.getParent())) {
Files.createDirectories(path);
}
file = Files.createFile(path).toFile();
}
catch (Exception ex) {
logger.error("Failed to create file");
}
What's curious is that this doesn't give me a better error message. In fact, it doesn't give me any error message, because it doesn't fail. It appears that NIO is taking a very different path to creating the file than the regular File object.
Update:
What is now working fine is the following:
file = Paths.get(file.getAbsolutePath()).toAbsolutePath().toFile();
try {
Path path = file.toPath();
if (!Files.exists(path.getParent())) {
Files.createDirectories(path);
}
if (!file.exists()) {
file = Files.createFile(path).toFile();
}
}
catch (Exception ex) {
logger.error("Failed to create file");
}
What's curious is that I should be able to remove that first line, which is essentially converting a relative path to an absolute path. My test run creates 50 or so files in the process. I tried commenting out that line and then clearing out my output tree and running the test. It got the following exception attempting to create the first file:
java.nio.file.AccessDeniedException: build\gen1\org\opendaylight\yang\gen\v1\urn\opendaylight\params\xml\ns\yang\pcep\types\rev131005\vs\tlv\vs\tlv\VendorPayload.java
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
What the heck?
Also note that I never did remove the older code that uses "File.createNewFile()", I just put the "Files" code before that, and the old code checks for "!file.exists()", so theoretically the old code would only execute if the new code somehow didn't create the file. On this first file, since the NIO creation failed, the file still didn't exist, and it went through the old creation code, which SUCCEEDED.
And even stranger, I let the test case run to the next file, and that failed in the NEW code with:
java.nio.file.FileAlreadyExistsException: build\gen1\org\opendaylight\yang\gen\v1\urn\opendaylight\params\xml\ns\yang\pcep\types\rev131005\vs\tlv\VsTlv.java
Note that the only way that block could have gotten that exception is if it executed the "Files.createFile(path).toFile()" line, and the only way it could have gotten to that line is if "!file.exists()" was TRUE, which means that the file did not exist. my brain is starting to melt. Also note that while I'm sitting at this breakpoint, I examined the file system, and that file does not exist.
This is 2015 and you say that you use Java 7.
Don't use File. Use this instead:
final Path path = Paths.get("....").toAbsolutePath();
// use Files.exists(path.getParent()) to check for the existence;
// if it doesn't exist use Files.createDirectories() on it
Files.createFile(thePath);
If the operation fails, you will at least get a meaningful exception telling you why it fails.
This is 2015. Drop. File. Now.
I'm trying to run a exe file in path outside of the current package. My code.java file that runs it is in
%Workspace_path%\Project\src\main\java\com\util\code.java
However the directory of where the exe is
%Workspace_path%\Project\src\main\resources\program.exe
If possible, it seems like the best solution here would be to get the absolute path of the Project then append "src\main\resources\" to it. Is there a good way to do this or is there an alternative solution?
I'm using Eclipse, but it would great if it could be used in other IDEs too. Thanks for any help.
The de facto approach to solving this is to bundle the EXE as a classpath resource. It seems you have arranged for this already.
When working with classpath resources, a mature program should not assume that the resource is in the filesystem. The resources could be packaged in a JAR file, or even in a WAR file. The only thing you can trust at that point is the standard methods for accessing resources in Java, as hinted below.
The way to solve your problem, then, is to access the resource contents using the de facto standard of invoking Class.getResourceAsStream (or ClassLoader.getResourceAsStream), save the contents to a temporary file, and execute from that file. This will guarantee your program works correctly regardless of its packaging.
In other words:
Invoke getClass().getResourceAsStream("/program.exe"). From static methods, you can't call getClass, so use the name of your current class instead, as in MyClass.class.getResourceAsStream. This returns an InputStream.
Create a temporary file, preferably using File.createTempFile. This returns a File object identifying the newly created file.
Open an OutputStream to this temp file.
Use the two streams to copy the data from the resource into the temp file. You can use IOUtils.copy if you're into Apache Commons tools. Don't forget to close the two streams when done with this step.
Execute the program thus stored in the temporary file.
Clean up.
In other words (code snippet added later):
private void executeProgramFromClasspath() throws IOException {
// Open resource stream.
InputStream input = getClass().getResourceAsStream("/program.exe");
if (input == null) {
throw new IllegalStateException("Missing classpath resource.");
}
// Transfer.
OutputStream output = null;
try {
// Create temporary file. May throw IOException.
File temporaryFile = File.createTempFile(getClass().getName(), "");
output = new FileOutputStream(temporaryFile);
output = new BufferedOutputStream(output);
IOUtils.copy(input, output);
} finally {
// Close streams.
IOUtils.closeQuietly(input);
IOUtils.closeQuietly(output);
}
// Execute.
try {
String path = temporaryFile.getAbsolutePath();
ProcessBuilder processBuilder = new ProcessBuilder(path);
Process process = processBuilder.start();
process.waitFor();
} catch (InterruptedException e) {
// Optional catch. Keeps the method signature uncluttered.
throw new IOException(e);
} finally {
// Clean up
if (!temporaryFile.delete()) {
// Log this issue, or throw an error.
}
}
}
Well,in your context,the project root is happen to be the current path
.
,that is where the java.exe start to execute,so a easy way is:
String exePath="src\\main\\resources\\program.exe";
File exeFile=new File(".",exePath);
System.out.println(exeFile.getAbusolutePath());
...
I tested this code on Eclipse,It's ok. I think is should work on different ide.
Good Luck!
I know this has been probably answered a million times on here but everything I have looked at has not helped me. Here is my code:
for(File g: f.listFiles()){
for(File h : g.listFiles()){
try{
Scanner s = new Scanner(h);
String timestamp = s.next().split("[?]")[4];
File z = new File(h.getAbsolutePath().split("[.]")[0] + timestamp + h.getAbsolutePath().split("[.]")[1]);
boolean q = h.renameTo(z);
}catch(Exception e){
}
}
}
I have checked to see if File z exists and it doesnt. I have checked if File h exists and it does. I have doublechecked that h is an absolute path. If I print the absolute path of z, I get the correct path. None of the directories in f or files in g are open. The files denoted by h are not open. Could there be some flag set or something on the file where windows is not allowing my program to rename it?
My guess is that you are having a similar problem to one I had here File deletion/moving failing
Try using FileinputStreams for the Scanner
FileInputStream fin = new FileInputStream(h);
fin.open()
Scanner s = new Scanner(fin);
//do work
fin.close()
and closing the stream before renaming
The behavior of renameTo varies from platform to platform. Operations that succeed on one platform may fail on another. For example, on my local development workstation (OS X), everything worked as expected. On a production system (Solaris), renameTo failed consistently. I finally determined that it failed when the files were located on different partitions. Obviously that is not the case here, but it illustrates that the method can behave in unexpected ways.
To get consistent behavior, copy the data to a new file, then delete the original.
I had a almost same issue. Some of rename cases succeeded, some failed. For those failed cases, I found, the source file path and destination file path are not on in same file system. In my cases, the NTFS mounted another file system which the destination file would be moved to. Since the rename function's original purpose simply rename a name, not to move the data of the concerned file. If both source file path and destination file path are in different file system, some version of JVM will fail on certain platforms. Actually, it is a bug in java.io and Solaris has fixed this bug in new versions.
Good Luck!
HappyForever,
I need to create a java program which will create thread to search for a file in particular folder(source folder) and pick the file immediately for process work(convert it into csv file format) once it found the file in the source folder. Problem i am facing now is file which comes to source folder is big size(FTP tool is used to copy file from server to source folder), thread is picking that file immediately before it copies fully to source folder and throwing exception. How do i stop thread until the file copy into source folder completely?. It has to pick the file for processing only after the file is copied completely into source folder.
Tha safest way is to download the file to a different location and then move it to the target folder.
Another variation mentioned by Bombe is to change the file name to some other extension after downloading and look only for files with that extension.
I only read the file which is not in write mode. This is safest as this means no other process is writing in this file. You can check if file is not in write mode by using canWrite method of File class.
This solution works fine for me as I also have the exact same scenario you facing.
You could try different things:
Repeatedly check the last modification date and the size of the file until it doesn’t change anymore for a given amount of time, then process it. (As pointed out by qbeuek this is neither safe nor deterministic.)
Only process files with names that match certain criteria (e.g. *.dat). Change the FTP upload/download process to upload/download files with a different name (e.g. *.dat.temp) and rename the files once they are complete.
Download the files to a different location and move them to your processing directory once they’re complete.
As Vinegar said, if it doesn’t work the first time, try again later. :)
If you have some control on the process that does the FTP you could potentially have it create a "flag file" in the source directory immediately AFTER the ftp for the big file is finished.
Then your Java thread has to check the presence of this flag file, if it's present then there is a file ready to be processed in the source directory. Before processing the big file, the thread should remove the flag file.
Flag file can be anything (even an empty file).
Assuming you have no control over FTP process...
Let it be like this. When you get the exception, then try to process it again next time. Repeat it until the file gets processed. Its good to keep few attributes in case of exception to check it later, like; name, last-modified, size.
Check the exact exception before deciding to process it later, the exception might occur for some other reason.
If your OS is Linux, and your kernel > 2.6.13, you could use the filesystem event notification API named inotify.
There's a Java implementation here : https://bitbucket.org/nbargnesi/inotify-java.
Here's a sample code (heavily inspired from the website).
try {
Inotify i = new Inotify();
InotifyEventListener e = new InotifyEventListener() {
#Override
public void filesystemEventOccurred(InotifyEvent e) {
System.out.println("inotify event occurred!");
}
#Override
public void queueFull(EventQueueFull e) {
System.out.println("inotify event queue: " + e.getSource() +
" is full!");
}
};
i.addInotifyEventListener(e);
i.addWatch(System.getProperty("user.home"), Constants.IN_CLOSE_WRITE);
} catch (UnsatisfiedLinkError e) {
System.err.println("unsatisfied link error");
} catch (UserLimitException e) {
System.err.println("user limit exception");
} catch (SystemLimitException e) {
System.err.println("system limit exception");
} catch (InsufficientKernelMemoryException e) {
System.err.println("insufficient kernel memory exception");
}
This is in Grails and I am using FileUtils Library from the Apache commons fame. The sizeof function returns the size in bytes.
def fileModified = sourceFile.lastModified()
def fileSize = FileUtils.sizeOf(sourceFile)
Thread.sleep(3000) //sleep to calculate size difference if the file is currently getting copied
if((fileSize != FileUtils.sizeOf(sourceFile)) && (fileModified != sourceFile.lastModified())) //the file is still getting copied to return
{
if(log.infoEnabled)
log.info("File is getting copied!")
return
}
Thread.sleep(1000) //breather for picking up file just copied.
Please note that this also depends on what utility or OS you are using to transfer the files.
The safest bet is to copy the file which is been copied or has been copied to different file or directory. The copy process is robust one and it assure you that file is present after the copying process. The one I am using is from commons API.
FileUtils.copyFileToDirectory(File f, Directory D)
If you are copying a huge file which is in process of getting copied beware that this will take time and you might like to start this in parallel thread or best have a seperate application dedicated for transfer process.