I'm writing a logger to my java program (in csv format).
The logger works fine, and I had one problem.
It sounds pretty logical that the program will crash when i tried to write to the file and at the same time open the file.
When i do that, I got that exception: "The process cannot access the file because it is being used by another process".
My question is if there is anyway to continue writing even if someone open the file?
Thanks.
UPDATE:
I think i solved the problem.
Every time after i write to the file (With bufferedWriter and FileWriter), I call to a close() function that closing the bufferedWriter and FileWriter.
I changed the close() function:
1. Added fileChannel and FileLock.
2. Igonore the line bw.close();
Its ok not to close the bufferWriter (bw)?, Can there be any problems later on?
private void close() throws IOException {
RandomAccessFile rf;
rf = new RandomAccessFile(file, "rw");
fileChannel = rf.getChannel();
lock = fileChannel.lock();
try {
if (bw != null) {
// bw.close(); The line i ignored.
bw = null;
}
if (fw != null) {
fw.close();
fw = null;
}
} catch (IOException ex) {
ex.printStackTrace();
}
lock.release();
}
UPDATE 2:
Now i found that if i change the function to that (close changed to flush), Its working:
private void close() {
try {
if (bw != null) {
bw.flush();
bw = null;
}
if (fw != null) {
fw.flush();
fw = null;
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
What is the best option ?
Reverse the problem: try to open while continuing writing:
if you want fixed datas, you can copy the file (by shell), and then read it;
if you want even future written datas, you must keep the same output: try to redirect the normal output, to something you can store and read.
Perhaps some library exists. It seems like tee and tpipe.
see for example:
Could I duplicate or intercept an output stream in Java?
for redirecting log4j to what you want, see this for example:
How do I redirect log4j output to my HttpServletResponse output stream?
Is there is anyway to continue writing even if something else has opened the file?
Not in Java.
To write a file, you must first open it. If you cannot open it because the OS won't permit it ... because something else has opened it ... then you cannot get to the point where you can write it.
In this scenario, you should consider opening a different log file.
Note that this scenario happens in Windows because Java is following normal Window practice and opening the file with an exclusive (mandatory) lock by default. Short of changing Java ... and every other Windows application that opens files like this ... you are stuck.
UPDATE
It turns out that there may be a way.
Read this Q&A: https://stackoverflow.com/a/22648514/139985
Use FileChannel.open as described, but use flags that allow you to write without forbidding other writers. For example
FileChannel.open(path, WRITE)
or
FileChannel.open(path, WRITE, APPEND)
The trick is that you don't want any of the NOSHARE_* options.
CAVEAT: I haven't tried this.
As #guillaume said, you can use a library like log4j.
But If you want to implements your solution in Java, you can use the observer pattern and write your logs async.
Related
UPDATE 2: Please close this question
After further debugging it is found that the problem is not in the inner try block, but a bug inside the 'while' loop. An exception was caused there and was not caught, which therefore skips the inner try block. Apologies for my mistake, please delete this thread.
UPDATE: added logging to capture errors during delete.
I am downloading 8000ish GZ files from a server, process its content locally, then delete the downloaded copy upon completion. I am running this over a number of threads, each process a disjoint batch of GZ files. But I do not understand for what reason that my code does not successfully delete the GZ files occasionally (not always). The code generally looks like this:
....
private static final Logger LOG = Logger.getLogger(....class.getName());
.....
for (String inputGZFile : gzFiles) { //gzFiles is a list of urls to be process by this thread
try {
File downloadTo = new
File(this.outFolder + "/" + new File(downloadFrom.getPath()).getName());
FileUtils.copyURLToFile(downloadFrom, downloadTo);
InputStream fileStream = new FileInputStream(downloadTo);
InputStream gzipStream = new GZIPInputStream(fileStream);
Reader decoder = new InputStreamReader(gzipStream, Charset.forName("utf8"));
Scanner inputScanner = new Scanner(decoder);
inputScanner.useDelimiter(" .");
while (inputScanner.hasNextLine() && (content = inputScanner.nextLine()) != null) {
//do something
}
try {
inputScanner.close();
FileUtils.forceDelete(downloadTo);
}catch (Exception e){
LOG.info("\t thread " + id + " deleting gz file error "+ inputGZFile);
LOG.info("\t thread " + id+ExceptionUtils.getFullStackTrace(e));
}
}catch(Exception e){
e.printStackTrace();
}
}
The only reason I can think of is that the scanner did not close the file or release the file handle. But that would be strange because I already call the close method to close the scanner.
Any suggestions highly appreiciated.
Without the ability to look into your log files, or debug your system first hand, it is close to impossible to tell you what is going wrong here.
But what you can definitely do: do that call to FileUtils.forceDelete(downloadTo); within a finally block for example.
The whole point of try/catch/finally is to enable you to enforce that specific actions always take place, no matter what happened in the try block!
Also note: if you are unable to tell what your code does, then add logging support to it. So that instead of printStackTrace(); you log the whole exception to a place where it does not get lost.
Meaning: the real answer here is that you step back and take the necessary actions to find out where your problems are coming from.
I need to write a custom batch File renamer. I've got the bulk of it done except I can't figure out how to check if a file is already open. I'm just using the java.io.File package and there is a canWrite() method but that doesn't seem to test if the file is in use by another program. Any ideas on how I can make this work?
Using the Apache Commons IO library...
boolean isFileUnlocked = false;
try {
org.apache.commons.io.FileUtils.touch(yourFile);
isFileUnlocked = true;
} catch (IOException e) {
isFileUnlocked = false;
}
if(isFileUnlocked){
// Do stuff you need to do with a file that is NOT locked.
} else {
// Do stuff you need to do with a file that IS locked
}
(The Q&A is about how to deal with Windows "open file" locks ... not how implement this kind of locking portably.)
This whole issue is fraught with portability issues and race conditions:
You could try to use FileLock, but it is not necessarily supported for your OS and/or filesystem.
It appears that on Windows you may be unable to use FileLock if another application has opened the file in a particular way.
Even if you did manage to use FileLock or something else, you've still got the problem that something may come in and open the file between you testing the file and doing the rename.
A simpler though non-portable solution is to just try the rename (or whatever it is you are trying to do) and diagnose the return value and / or any Java exceptions that arise due to opened files.
Notes:
If you use the Files API instead of the File API you will get more information in the event of a failure.
On systems (e.g. Linux) where you are allowed to rename a locked or open file, you won't get any failure result or exceptions. The operation will just succeed. However, on such systems you generally don't need to worry if a file is already open, since the OS doesn't lock files on open.
// TO CHECK WHETHER A FILE IS OPENED
// OR NOT (not for .txt files)
// the file we want to check
String fileName = "C:\\Text.xlsx";
File file = new File(fileName);
// try to rename the file with the same name
File sameFileName = new File(fileName);
if(file.renameTo(sameFileName)){
// if the file is renamed
System.out.println("file is closed");
}else{
// if the file didnt accept the renaming operation
System.out.println("file is opened");
}
On Windows I found the answer https://stackoverflow.com/a/13706972/3014879 using
fileIsLocked = !file.renameTo(file)
most useful, as it avoids false positives when processing write protected (or readonly) files.
org.apache.commons.io.FileUtils.touch(yourFile) doesn't check if your file is open or not. Instead, it changes the timestamp of the file to the current time.
I used IOException and it works just fine:
try
{
String filePath = "C:\sheet.xlsx";
FileWriter fw = new FileWriter(filePath );
}
catch (IOException e)
{
System.out.println("File is open");
}
I don't think you'll ever get a definitive solution for this, the operating system isn't necessarily going to tell you if the file is open or not.
You might get some mileage out of java.nio.channels.FileLock, although the javadoc is loaded with caveats.
Hi I really hope this helps.
I tried all the options before and none really work on Windows. The only think that helped me accomplish this was trying to move the file. Event to the same place under an ATOMIC_MOVE. If the file is being written by another program or Java thread, this definitely will produce an Exception.
try{
Files.move(Paths.get(currentFile.getPath()),
Paths.get(currentFile.getPath()), StandardCopyOption.ATOMIC_MOVE);
// DO YOUR STUFF HERE SINCE IT IS NOT BEING WRITTEN BY ANOTHER PROGRAM
} catch (Exception e){
// DO NOT WRITE THEN SINCE THE FILE IS BEING WRITTEN BY ANOTHER PROGRAM
}
If file is in use FileOutputStream fileOutputStream = new FileOutputStream(file); returns java.io.FileNotFoundException with 'The process cannot access the file because it is being used by another process' in the exception message.
My application writes to Excel files. Sometimes the file can be used, in that case the FileNotFoundException thrown and then I do not know how to handle it better.
I am telling the user that the file is used and after that message I do not want to close the application, but to stop and wait while the file is available (assuming that it is opened by the same user). But I do not understand how to implement it. file.canWrite() doesn't work, it returns true even when the file is opened, to use FileLock and check that the lock is available I need to open a stream, but it throws FileNotFoundException (I've been thinking about checking the lock in a busy wait, I know that it is not a good solution, but I can't find another one).
This is a part of my code if it can help somehow to understand my problem:
File file = new File(filename);
FileOutputStream out = null;
try {
out = new FileOutputStream(file);
FileChannel channel = out.getChannel();
FileLock lock = channel.lock();
if (lock == null) {
new Message("lock not available");
// to stop the program here and wait when the file is available, then resume
}
// write here
lock.release();
}
catch (IOException e) {
new Message("Blocked");
// or to stop here and then create another stream when the file is available
}
What makes it more difficult for me is that it writes to different files, and if the first file is available, but the second is not, then it will update one file and then stop, and if I restart the program, it will update it again, so I can't allow the program to write into files until all of them are available.
I believe that there should be a common solution, since it must be a common issue in Windows to deal with such cases, but I can't find it.
To wait until a file exists you can make a simple loop:
File file = new File(filename);
while (!file.exists()) {
try {
Thread.sleep(100);
} catch (InterruptedException ie) { /* safe to ignore */ }
}
A better solution could be using WatchService but it's more code to implement.
The File.canWrite method only tells you if a path can be written to; if the path names a file that doesn't exist it will return false. You could use the canRead method instead of exists in a loop like above.
To use a file locks, the file has to exist first, so that wouldn't work either.
The only way to be sure you can write to a file is to try to open it. If the file doesn't exist, the java.io API will create it. To open a file for writing without creating you can use the java.nio.file.Files class:
try (OutputStream out = Files.newOutputStream(file.toPath(),
StandardOpenOption.WRITE))
{
// exists and is writable
} catch (IOException) {
// doesn't exist or can't be opened for writing
}
Today, when I was working on some kind of servlet which was writing some information to some file present on my hard disk, I was using the following code to perform the write operation
File f=new File("c:/users/dell/desktop/ja/MyLOgs.txt");
PrintWriter out=new PrintWriter(new FileWriter(f,true));
out.println("the name of the user is "+name+"\n");
out.println("the email of the user is "+ email+"\n");
out.close(); //**my question is about this statement**
When I was not using the statement, the servlet was compiling well, but it was not writing anything to the file, but when I included it, then the write operation was successfully performed. My questions are:
Why was the data not being written to the file when I was not including that statement (even my servlet was compiling without any errors)?
Up to which extent the close operation is considerable for the streams?
Calling close() causes all the data to be flushed. You have constructed a PrintWriter without enabling auto-flush (a second argument to one of the constructors), which would mean you would have to manually call flush(), which close() does for you.
Closing also frees up any system resources used by having the file open. Although the VM and Operating System will eventually close the file, it is good practice to close it when you are finished with it to save memory on the computer.
You may also which to put the close() inside a finally block to ensure it always gets called. Such as:
PrintWriter out = null;
try {
File f = new File("c:/users/dell/desktop/ja/MyLOgs.txt");
out = new PrintWriter(new FileWriter(f,true));
out.println("the name of the user is "+name+"\n");
out.println("the email of the user is "+ email+"\n");
} finally {
out.close();
}
See: PrintWriter
Sanchit also makes a good point about getting the Java 7 VM to automatically close your streams the moment you don't need them automatically.
When you close a PrintWriter, it will flush all of its data out to wherever you want the data to go. It doesn't automatically do this because if it did every time you wrote to something, it would be very inefficient as writing is not an easy process.
You could achieve the same effect with flush();, but you should always close streams - see here: http://www.javapractices.com/topic/TopicAction.do?Id=8 and here: http://docs.oracle.com/javase/tutorial/jndi/ldap/close.html. Always call close(); on streams when you are done using them. Additionally, to make sure it is always closed regardless of exceptions, you could do this:
try {
//do stuff
} finally {
outputStream.close():
}
It is because the PrintWriter buffers your data in order for not making I/O operations repeatedly for every write operation (which is very expensive). When you call close() the Buffer flushes into the file. You can also call flush() for forcing the data to be written without closing the stream.
Streams automatically flush their data before closing. So you can either manually flush the data every once in a while using out.flush(); or you can just close the stream once you are done with it. When the program ends, streams close and your data gets flushed, this is why most of the time people do not close their streams!
Using Java 7 you can do something like this below which will auto close your streams in the order you open them.
public static void main(String[] args) {
String name = "";
String email = "";
File f = new File("c:/users/dell/desktop/ja/MyLOgs.txt");
try (FileWriter fw = new FileWriter(f, true); PrintWriter out = new PrintWriter(fw);) {
out.println("the name of the user is " + name + "\n");
out.println("the email of the user is " + email + "\n");
} catch (IOException e) {
e.printStackTrace();
}
}
PrintWriter buffers the data to be written so and will not write to disk until its buffer is full. Calling close() will ensure that any remaining data is flushed as well as closing the OutputStream.
close() statements typically appear in finally blocks.
Why the data was not being written to the file when I was not including that statement?
When the process terminates the unmanaged resources will be released. For InputStreams this is fine. For OutputStreams, you could lose an buffered data, so you should at least flush the stream before exiting the program.
I'm trying to delete a file that another thread within my program has previously worked with.
I'm unable to delete the file but I'm not sure how to figure out which thread may be using the file.
So how do I find out which thread is locking the file in java?
I don't have a straight answer (and I don't think there's one either, this is controlled at OS-level (native), not at JVM-level) and I also don't really see the value of the answer (you still can't close the file programmatically once you found out which thread it is), but I think you don't know yet that the inability to delete is usually caused when the file is still open. This may happen when you do not explicitly call Closeable#close() on the InputStream, OutputStream, Reader or Writer which is constructed around the File in question.
Basic demo:
public static void main(String[] args) throws Exception {
File file = new File("c:/test.txt"); // Precreate this test file first.
FileOutputStream output = new FileOutputStream(file); // This opens the file!
System.out.println(file.delete()); // false
output.close(); // This explicitly closes the file!
System.out.println(file.delete()); // true
}
In other words, ensure that throughout your entire Java IO stuff the code is properly closing the resources after use. The normal idiom is to do this in the try-with-resources statement, so that you can be certain that the resources will be freed up anyway, even in case of an IOException. E.g.
try (OutputStream output = new FileOutputStream(file)) {
// ...
}
Do it for any InputStream, OutputStream, Reader and Writer, etc whatever implements AutoCloseable, which you're opening yourself (using the new keyword).
This is technically not needed on certain implementations, such as ByteArrayOutputStream, but for the sake of clarity, just adhere the close-in-finally idiom everywhere to avoid misconceptions and refactoring-bugs.
In case you're not on Java 7 or newer yet, then use the below try-finally idiom instead.
OutputStream output = null;
try {
output = new FileOutputStream(file);
// ...
} finally {
if (output != null) try { output.close(); } catch (IOException logOrIgnore) {}
}
Hope this helps to nail down the root cause of your particular problem.
About this question, I also try to find out this answer, and ask this question and find answer:
Every time when JVM thread lock a file exclusively, also JVM lock
some Jave object, for example, I find in my case:
sun.nio.fs.NativeBuffer
sun.nio.ch.Util$BufferCache
So you need just find this locked Java object and analyzed them and
you find what thread locked your file.
I not sure that it work if file just open (without locked exclusively), but I'm sure that is work if file be locked exclusively by Thread (using java.nio.channels.FileLock, java.nio.channels.FileChannel and so on)
More info see this question