The thing is this, I am creating a file, that an XML resource uses, right after creation. When the program is done executing, the file should be deleted. This is what happens:
I run the program, file does not yet exist...
File should be created using FileWriter:
File file = new File("src/main/resources/org/avalin/optaplanner/solver/employeeShiftsScoreRules.drl");
try (FileWriter fileWriter = new FileWriter(file))
{
fileWriter.write("Content...");
fileWriter.flush();
fileWriter.close();
}
catch(IOException e)
{
e.printStackTrace();
}
And then I have this code:
private static synchronized Solver buildSolver()
{
SolverFactory solverFactory =
SolverFactory.createFromXmlResource(SOLVER_CONFIG_XML);
return solverFactory.buildSolver();
}
The filewriting is also wrapped in a synchronized method, but I assume that since they're not directly accessing the same variable, they have no effect what so ever. The file is being read from the SOLVER_CONFIG_XML seen above.
When the program ends, it deletes the file on the path given, so that when it runs next time it will be created accordingly to parameters given to the program.
Now this is what happens...
The first time I run the program I get an error, saying the file isn't written.
Exception in thread "main" java.lang.IllegalArgumentException: The scoreDrl (org/avalin/optaplanner/solver/employeeShiftsScoreRules.drl) does not exist as a classpath resource in the classLoader
I can make prints right after filewriting, that concludes the method HAS run through the first time around, but for some reason, the file is not "created" anyway, before the program ends executing the first time around...
The second time, program runs fine, as the file was created before it gets to the exception?
Is there a way to make sure the file is "completely written" before the next part of my program executes? The file differs in length each time, as it is dynamically created from what the user inputs, so I can't check on that. I would assume it would be completely written as it did execute the prints I made after fileWriter.close() but apparently not so.
It looks like you are writing your file to the src/main/resources folder, which is a standard location for resource sources - that is, where your build system reads files from, and not where your running program does.
Although it's possible to add your source folders to the classpath of your running problem, it is bad practice - try to find out where your build system writes its output to (probably separate folders for class files and copied resources) and write your file there.
Related
I had some new code using the commons-io FileUtils.openOutputStream(File) method, for a file that doesn't exist at the point of the call. This was failing with a "FileNotFoundException". I first thought this was a bug in commons-io, but then I realized that it's just calling "new FileOutputStream(file, append)", which is also supposed to create the file if it doesn't exist.
I then added code right before my call to FileUtils.openOutputStream(File) like the following:
if (!file.exists()) {
logger.info("Parent file exists: " + file.getParentFile().exists());
try {
file.createNewFile();
}
catch (Exception ex) {
logger.error("Creating file failed", ex);
}
}
This prints "true" for the parent file, and then "java.io.IOException: The system cannot find the path specified". I googled for this situation, and some people were hitting this if they went past the supposed 260 character limit for a file path on Windows. I thought that might be relevant, but my file path is only 230 characters long.
I also tried an experiment of trying to "touch" the same file path in my Cygwin bash shell, and it had no trouble doing that.
Update:
So I took the partial advice of trying to use Paths & Files to do this instead of just "File". My incoming parameter is a "File", so I can't do anything about that. I added the following code:
try {
Path path = Paths.get(file.getAbsolutePath()).toAbsolutePath();
if (!Files.exists(path.getParent())) {
Files.createDirectories(path);
}
file = Files.createFile(path).toFile();
}
catch (Exception ex) {
logger.error("Failed to create file");
}
What's curious is that this doesn't give me a better error message. In fact, it doesn't give me any error message, because it doesn't fail. It appears that NIO is taking a very different path to creating the file than the regular File object.
Update:
What is now working fine is the following:
file = Paths.get(file.getAbsolutePath()).toAbsolutePath().toFile();
try {
Path path = file.toPath();
if (!Files.exists(path.getParent())) {
Files.createDirectories(path);
}
if (!file.exists()) {
file = Files.createFile(path).toFile();
}
}
catch (Exception ex) {
logger.error("Failed to create file");
}
What's curious is that I should be able to remove that first line, which is essentially converting a relative path to an absolute path. My test run creates 50 or so files in the process. I tried commenting out that line and then clearing out my output tree and running the test. It got the following exception attempting to create the first file:
java.nio.file.AccessDeniedException: build\gen1\org\opendaylight\yang\gen\v1\urn\opendaylight\params\xml\ns\yang\pcep\types\rev131005\vs\tlv\vs\tlv\VendorPayload.java
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
What the heck?
Also note that I never did remove the older code that uses "File.createNewFile()", I just put the "Files" code before that, and the old code checks for "!file.exists()", so theoretically the old code would only execute if the new code somehow didn't create the file. On this first file, since the NIO creation failed, the file still didn't exist, and it went through the old creation code, which SUCCEEDED.
And even stranger, I let the test case run to the next file, and that failed in the NEW code with:
java.nio.file.FileAlreadyExistsException: build\gen1\org\opendaylight\yang\gen\v1\urn\opendaylight\params\xml\ns\yang\pcep\types\rev131005\vs\tlv\VsTlv.java
Note that the only way that block could have gotten that exception is if it executed the "Files.createFile(path).toFile()" line, and the only way it could have gotten to that line is if "!file.exists()" was TRUE, which means that the file did not exist. my brain is starting to melt. Also note that while I'm sitting at this breakpoint, I examined the file system, and that file does not exist.
This is 2015 and you say that you use Java 7.
Don't use File. Use this instead:
final Path path = Paths.get("....").toAbsolutePath();
// use Files.exists(path.getParent()) to check for the existence;
// if it doesn't exist use Files.createDirectories() on it
Files.createFile(thePath);
If the operation fails, you will at least get a meaningful exception telling you why it fails.
This is 2015. Drop. File. Now.
Hi i have made a small program that reads a config file. This file is stored outside the actual jar file. On the same level as the jarfile actually.
When i start my program from a commandline in the actual directory (ie. D:\test\java -jar name.jar argument0 argument1) in runs perfectly.
But when i try to run the program from another location then the actual directory i get the filenotfound exception (ie. D:\java -jar D:\test\name.jar argument0 argument1).
The basic functionality does seem to work, what am i doing wrong?
As requested a part of the code:
public LoadConfig() {
Properties properties = new Properties();
try {
// load the properties file
properties.load(new FileInputStream("ibantools.config.properties"));
} catch (IOException ex) {
ex.printStackTrace();
} // end catch
// get the actual values, if the file can't be read it will use the default values.
this.environment = properties.getProperty("application.environment","tst");
this.cbc = properties.getProperty("check.bankcode","true");
this.bankcodefile = properties.getProperty("check.bankcodefile","bankcodes.txt");
} // end loadconfig
The folder looks like this:
This works:
This doesn't:
The jar doesn't contain the text file.
When reading a File using the String/path constructors of File, FileInpustream, etc.. a relative path is derived from the working directory - the directory where you started your program.
When reading a file from a Jar, the file being external to the jar, you have at least two options :
Provide an absolute path: D:/blah/foo/bar
Make the directory where your file is located part of the class path and use this.getClass().getClassLoader().getResourceAsStream("myfile")
The latter is probably more appropriate for reading configuration files stored in a path relative to the location of your application.
There could be one more possibility:
If one part of your code is writing the file and another one is reading, then it is good to consider that the reader is reading before the writer finishes writing the file.
You can cross check this case by putting your code on debug mode. If it works fine there and gives you FileNotFoundException, then surely this could be the potential reason of this exception.
Now, how to resolve:
You can use retry mechanism something similar to below code block
if(!file..exists()){
Thread.sleep(200);
}
in your code and change the sleep value according to your needs.
Hope that helps.!!
I am contacting an external services with my Java app.
The flow is as follow: ->I generate an XML file, and put it in an folder, then the service processes the file and return another file with the same name having an extension .out
Right now after I put the file in the folder I start with a loop, until I get that file back so I can read the result.
Here is the code:
fileName += ".out";
File f = new File(fileName);
do
{
f = new File(fileName);
} while (!f.exists());
response = readResponse(fileName); // got the response now read it
My question comes here, am I doing it in the right way, is there a better/more efficient way to wait for the file?
Some info: I run my app on WinXP, usually it takes the external service less than a second to respond with a file, I send around 200 request per day to this services. The path to the folder with the result file is always the same.
All suggestions are welcome.
Thank you for your time.
There's no reason to recreate the File object. It just represents the file location, whether the file exists or not. Also you probably don't want a loop without at least a short delay, otherwise it'll just max out a processor until the file exists. You probably want something like this instead:
File file = new File(filename);
while (!file.exists()) {
Thread.sleep(100);
}
Edit: Ingo makes a great point in the comments. The file might not be completely there just because it exists. One way to guarantee that it's ready is have the first process create a second file after the first is completely written. Then have the Java program detect that second file, delete it and then safely read the first one.
My guesses were wrong, and had nothing to do with the answer. This question is no longer valid. See my answer. Sorry about this poor question.
Tl;dr Version
Why can't a Java process find a certain file, until another process – the process that created that file – has finished executing. Can this be worked around?
Longer Version
I have an application that needs to restart itself (it just needs to, okay?). The first time, it creates a File and then serializes an object there. This is done with a FileOutputStream/ObjectOutputStream combo, as described below:
private static File serializeBootstrapInfoIntoFile(
final BootstrapInfo info) throws IOException {
final File tempFile = File.createTempFile("BobBootstrapInfo", null);
final FileOutputStream fos = new FileOutputStream(tempFile);
final ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(info);
// just being thorough
oos.flush();
oos.close();
fos.flush();
fos.close();
return tempFile;
}
After this, I create another java process with a System.exec()-call, to which I pass the the absolute path of the returned tempFile as a system property. The other java process should then open the file, and deserialize the containing Object. The first process remains alive until the spawned process exits, since it handles the new one's output/error streams.
The problem is, however, that the second process doesn't seem to find the file, and dies always in a FileNotFoundException during deserialization (I've confirmed this with a file.exists()).
When I check manually afterwards, the file does indeed exist. Also, if I manually run the exact same command line that is passed to the System.exec(), it runs fine. So, I'm guessing the first process somehow manages to hide the file from the new process, or fails to actually write the file to the file system, even the streams are flushed and closed. I also tried with Thread.sleep(10000) on the first thread, to let the IO operations to finish, but that didn't help a bit.
Am I doing something wrong? Or is this a Java thing, or maybe an OSX thing (which I'm running with atm)?
Answers to Comments
I'm running OS X 10.6.3, and the Java version is 1.6.0_20.
The System.exec() arguments are
java
-classpath
/var/folders/dr/drDlHsguGvq0zF2Jtgn4S++++TI/-Tmp-/bob7168396245507677201.tmp:/Users/wolfie/Documents/workspace/Bob/dist/lib/junit.jar:/Users/wolfie/Documents/workspace/Bob/dist/lib/bob.jar
-Dcache.location="/var/folders/dr/drDlHsguGvq0zF2Jtgn4S++++TI/-Tmp-/BobBootstrapInfo4944987280015634213.tmp"
com.github.wolfie.bob.Bob
where each line is an element in a String array. The /var/folders/dr/drDlHsguGvq0zF2Jtgn4S++++TI/-Tmp-/BobBootstrapInfo4606780571989099166.tmp-file is the one that was created, and being read by the other thread. The envp and dir arguments are null.
The whole stacktrace is:
Exception in thread "main" com.github.wolfie.bob.BootstrapError: java.io.FileNotFoundException: "/var/folders/dr/drDlHsguGvq0zF2Jtgn4S++++TI/-Tmp-/BobBootstrapInfo4606780571989099166.tmp" (No such file or directory)
at com.github.wolfie.bob.Bob.getBootstrapInfo(Bob.java:186)
at com.github.wolfie.bob.Bob.run(Bob.java:138)
at com.github.wolfie.bob.Bob.main(Bob.java:95)
Caused by: java.io.FileNotFoundException: "/var/folders/dr/drDlHsguGvq0zF2Jtgn4S++++TI/-Tmp-/BobBootstrapInfo4606780571989099166.tmp" (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:106)
at com.github.wolfie.bob.Bob.deserializeBootstrapInfoFromFile(Bob.java:265)
at com.github.wolfie.bob.Bob.getBootstrapInfo(Bob.java:184)
... 2 more
The answer isn't visible from the question: I defined the system property like so:
-Dcache.location="/foo/file.ext"
But the property should've been instead
-Dcache.location=/foo/file.ext
i.e. without the quotes. This, apparently, was ignored when the exact same arguments were passed from the command line, probably because Bash processed them in a different way to the JVM.
Sorry about the poor question.
I need to create a java program which will create thread to search for a file in particular folder(source folder) and pick the file immediately for process work(convert it into csv file format) once it found the file in the source folder. Problem i am facing now is file which comes to source folder is big size(FTP tool is used to copy file from server to source folder), thread is picking that file immediately before it copies fully to source folder and throwing exception. How do i stop thread until the file copy into source folder completely?. It has to pick the file for processing only after the file is copied completely into source folder.
Tha safest way is to download the file to a different location and then move it to the target folder.
Another variation mentioned by Bombe is to change the file name to some other extension after downloading and look only for files with that extension.
I only read the file which is not in write mode. This is safest as this means no other process is writing in this file. You can check if file is not in write mode by using canWrite method of File class.
This solution works fine for me as I also have the exact same scenario you facing.
You could try different things:
Repeatedly check the last modification date and the size of the file until it doesn’t change anymore for a given amount of time, then process it. (As pointed out by qbeuek this is neither safe nor deterministic.)
Only process files with names that match certain criteria (e.g. *.dat). Change the FTP upload/download process to upload/download files with a different name (e.g. *.dat.temp) and rename the files once they are complete.
Download the files to a different location and move them to your processing directory once they’re complete.
As Vinegar said, if it doesn’t work the first time, try again later. :)
If you have some control on the process that does the FTP you could potentially have it create a "flag file" in the source directory immediately AFTER the ftp for the big file is finished.
Then your Java thread has to check the presence of this flag file, if it's present then there is a file ready to be processed in the source directory. Before processing the big file, the thread should remove the flag file.
Flag file can be anything (even an empty file).
Assuming you have no control over FTP process...
Let it be like this. When you get the exception, then try to process it again next time. Repeat it until the file gets processed. Its good to keep few attributes in case of exception to check it later, like; name, last-modified, size.
Check the exact exception before deciding to process it later, the exception might occur for some other reason.
If your OS is Linux, and your kernel > 2.6.13, you could use the filesystem event notification API named inotify.
There's a Java implementation here : https://bitbucket.org/nbargnesi/inotify-java.
Here's a sample code (heavily inspired from the website).
try {
Inotify i = new Inotify();
InotifyEventListener e = new InotifyEventListener() {
#Override
public void filesystemEventOccurred(InotifyEvent e) {
System.out.println("inotify event occurred!");
}
#Override
public void queueFull(EventQueueFull e) {
System.out.println("inotify event queue: " + e.getSource() +
" is full!");
}
};
i.addInotifyEventListener(e);
i.addWatch(System.getProperty("user.home"), Constants.IN_CLOSE_WRITE);
} catch (UnsatisfiedLinkError e) {
System.err.println("unsatisfied link error");
} catch (UserLimitException e) {
System.err.println("user limit exception");
} catch (SystemLimitException e) {
System.err.println("system limit exception");
} catch (InsufficientKernelMemoryException e) {
System.err.println("insufficient kernel memory exception");
}
This is in Grails and I am using FileUtils Library from the Apache commons fame. The sizeof function returns the size in bytes.
def fileModified = sourceFile.lastModified()
def fileSize = FileUtils.sizeOf(sourceFile)
Thread.sleep(3000) //sleep to calculate size difference if the file is currently getting copied
if((fileSize != FileUtils.sizeOf(sourceFile)) && (fileModified != sourceFile.lastModified())) //the file is still getting copied to return
{
if(log.infoEnabled)
log.info("File is getting copied!")
return
}
Thread.sleep(1000) //breather for picking up file just copied.
Please note that this also depends on what utility or OS you are using to transfer the files.
The safest bet is to copy the file which is been copied or has been copied to different file or directory. The copy process is robust one and it assure you that file is present after the copying process. The one I am using is from commons API.
FileUtils.copyFileToDirectory(File f, Directory D)
If you are copying a huge file which is in process of getting copied beware that this will take time and you might like to start this in parallel thread or best have a seperate application dedicated for transfer process.