We Keep Coding

Why is StandardOpenOption.CREATE_NEW not creating a file to be read from in Java?

Here is my code:
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.SeekableByteChannel;
import java.nio.file.Files;
import java.nio.file.InvalidPathException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
public class ExplicitChannelRead {
public static void main(String[] args) {
int count;
Path filePath = null;
// First, obtain a path to a file.
try {
filePath = Paths.get("test1.txt");
}
catch(InvalidPathException e) {
System.out.println("Path error: "+e);
return;
}
// Next, obtain a channel to that file within a try-with-resources block.
try(SeekableByteChannel fChan =
Files.newByteChannel(filePath, StandardOpenOption.CREATE_NEW)) {
// Allocate a buffer.
ByteBuffer mBuf = ByteBuffer.allocate(128);
while((count=fChan.read(mBuf)) != -1) {
//Rewind the buffer so that it can be read.
mBuf.rewind();
for(int i=0; i<count; i++) System.out.print((char)mBuf.get());
}
System.out.println();
} catch (IOException e) {
e.printStackTrace();
// System.out.println("I/O error: "+e);
}
}
}
On running the above code I get this exception:
java.nio.file.NoSuchFileException: test1.txt
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:235)
at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
at java.base/java.nio.file.Files.newByteChannel(Files.java:426)
at fileNIO.ExplicitChannelRead.main(ExplicitChannelRead.java:31)
I don't understand why test1.txt file is not being created as it doesn't exist currently and I am using the StandardOpenOption.CREATE_NEW option?
When I use StandardOpenOption.WRITE option along with StandardOpenOption.CREATE_NEW then I see the file text1.txt being created and at that time I get the exception:
Exception in thread "main" java.nio.channels.NonReadableChannelException
This exception I understand its cause because I have opened the file in write mode and in the code I am performing read operation on the file.
It seems to me that a new file can't be created when the file is opened in read mode.

I have reproduced what you are seeing (on Linux with Java 17).
As I noted in the comments, the behavior seems to contradict what the javadocs say what should happen, but what I discovered is this:
With READ or neither READ or WRITE, a NoSuchFileException is thrown.
With WRITE (and no READ), the file is created but then NonReadableChannelException is thrown.
With both READ and WRITE, it works. At least ... it did for me.
I guess this sort of makes sense. You need READ to read the file and WRITE to create it. And the javadocs state that READ is the default if you don't specify READ, WRITE or APPEND.
But creating an empty file1 with CREATE_NEW and then immediately trying to read it is a use-case that borders on pointless. So it not entirely surprising that they didn't (clearly) document how to achieve this.
1 - As a comment noted, CREATE_NEW is specified to fail if the file already exists. If you want "create it if it doesn't exist", then you should use CREATE instead.

Write multiple files with same string without hanging the UI

I am working on an Android App that changes the CPU Frequency when a foreground app changes. The frequencies for the foreground app is defined in my application itself. But while changing the frequencies my app has to open multiple system files and replace the frequency with my text. This makes my UI slow and when I change apps continuously, it makes the systemUI crash. What can I do to write these multiple files all together at the same time?
I have tried using ASynctaskLoader but that too crashes the SystemUI later.
public static boolean setFreq(String max_freq, String min_freq) {
ByteArrayInputStream inputStream = new ByteArrayInputStream(max_freq.getBytes(Charset.forName("UTF-8")));
ByteArrayInputStream inputStream1 = new ByteArrayInputStream(min_freq.getBytes(Charset.forName("UTF-8")));
SuFileOutputStream outputStream;
SuFileOutputStream outputStream1;
try {
if (max_freq != null) {
int cpus = 0;
while (true) {
SuFile f = new SuFile(CPUActivity.MAX_FREQ_PATH.replace("cpu0", "cpu" + cpus));
SuFile f1 = new SuFile(CPUActivity.MIN_FREQ_PATH.replace("cpu0", "cpu" + cpus));
outputStream = new SuFileOutputStream(f);
outputStream1 = new SuFileOutputStream(f1);
ShellUtils.pump(inputStream, outputStream);
ShellUtils.pump(inputStream1, outputStream1);
if (!f.exists()) {
break;
}
cpus++;
}
}
} catch (Exception ex) {
}
return true;
}

I assume SuFile and SuFileOutputStream are your custom implementations extending Java File and FileOutputStream classes.
Couple of points need to be fixed first.
f.exists() check should be before initializing OutputStream, otherwise it will create the file before checking exists or not. This makes your while loop to become an infinite loop.
as #Daryll suggested, use the number of CPUs with while/for loop. I suggest using for loop.
close your streams after pump(..) method call.
If you want to keep the main thread free, then you can do something like this,
see this code segment:
public static void setFreq(final String max_freq, final String min_freq) {
new Thread(new Runnable() {
//Put all the stuff here
}).start();
}
This should solve your problem.

Determine the number of CPUs before hand and use that number in your loop rather than using a while (true) having to do SuFile.exists() every cycle.
I don't know what SuFileOutputStream is but you may need to close those file output streams or find a faster way to write the file if that implementation is too slow.

The optimal way to lock and write to a file in Scala on Linux

I'm having a hard time finding the correct way to do any advanced file-system operations on Linux using Scala.
The one which I really can't figure out if best described by the following pseudo-code:
with fd = open(path, append | create):
with flock(fd, exclusive_lock):
fd.write(string)
Basically open a file in append mode (create it if it's non existent), get an exclusive lock to it and write to it (with the implicit unlock and close afterwards).
Is there an easy, clean and efficient way of doing this if I know my program will be ran on linux only ? (preferably without glancing offer the exceptions that should be handled).
Edit:
The answer I got is, as far as I've seen and tested is correct. However it's quite verbose, so I'm marking it as ok but I'm leaving this snippet of code here, which is the one I ended up using (Not sure if it's correct, but as far as I see it does everything that I need):
val fc = FileChannel.open(Paths.get(file_path), StandardOpenOption.CREATE, StandardOpenOption.APPEND)
try {
fc.lock()
fc.write(ByteBuffer.wrap(message.getBytes(StandardCharsets.UTF_8)))
} finally { fc.close() }

You can use FileChannel.lock and FileLock to get what you wanted:
import java.nio.ByteBuffer
import java.nio.channels.FileChannel
import java.nio.charset.StandardCharsets
import java.nio.file.{Path, Paths, StandardOpenOption}
import scala.util.{Failure, Success, Try}
object ExclusiveFsWrite {
def main(args: Array[String]): Unit = {
val path = Paths.get("/tmp/file")
val buffer = ByteBuffer.wrap("Some text data here".getBytes(StandardCharsets.UTF_8))
val fc = getExclusiveFileChannel(path)
try {
fc.write(buffer)
}
finally {
// channel close will also release a lock
fc.close()
}
()
}
private def getExclusiveFileChannel(path: Path): FileChannel = {
// Append if exist or create new file (if does not exist)
val fc = FileChannel.open(path, StandardOpenOption.WRITE, StandardOpenOption.APPEND,
StandardOpenOption.CREATE)
if (fc.size > 0) {
// set position to the end
fc.position(fc.size - 1)
}
// get an exclusive lock
Try(fc.lock()) match {
case Success(lock) =>
println("Is shared lock: " + lock.isShared)
fc
case Failure(ex) =>
Try(fc.close())
throw ex
}
}
}

Is java.util.logging.FileHandler in Java 8 broken?

First, a simple test code:
package javaapplication23;
import java.io.IOException;
import java.util.logging.FileHandler;
public class JavaApplication23 {
public static void main(String[] args) throws IOException {
new FileHandler("./test_%u_%g.log", 10000, 100, true);
}
}
This test code creates with Java 7 only one File "test_0_0.log", no matter, how often I run the program. This is the expected behaviour because the append parameter in the constructor is set to true.
But if I run this sample in Java 8, every run creates a new File (test_0_0.log, test_0_1.log, test_0_2.log,...). I think this is a bug.
Imho, the related change in Java is this one:
## -413,18 +428,18 ##
// object. Try again.
continue;
}
- FileChannel fc;
+
try {
- lockStream = new FileOutputStream(lockFileName);
- fc = lockStream.getChannel();
- } catch (IOException ix) {
- // We got an IOException while trying to open the file.
- // Try the next file.
+ lockFileChannel = FileChannel.open(Paths.get(lockFileName),
+ CREATE_NEW, WRITE);
+ } catch (FileAlreadyExistsException ix) {
+ // try the next lock file name in the sequence
continue;
}
+
boolean available;
try {
- available = fc.tryLock() != null;
+ available = lockFileChannel.tryLock() != null;
// We got the lock OK.
} catch (IOException ix) {
// We got an IOException while trying to get the lock.
## -440,7 +455,7 ##
}
// We failed to get the lock. Try next file.
- fc.close();
+ lockFileChannel.close();
}
}
(In full: OpenJDK changeset 6123:ac22a52a732c)
I know that normally the FileHandler gets closed by the Logmanager, but this is not the case, if the system or the application crashes or the process gets killed. This is why I do not have a "close" statement in the above sample code.
Now I have two questions:
1) What is your opinion? Is this a bug? (Almost answered in the following comments and answers)
2) Do you know a workaround to get the old Java 7 behavior in Java 8? (The more important question...)
Thanks for your answers.

Closing of the FileHandler deletes the 'lck' file. If the lock file exists at all under a JDK8 version that is less than update 40 (java.util.logging), the FileHandler is going to rotate. From the OpenJDK discussion, the decision was made to always rotate if the lck file exists in addtion to if the current process can't lock it. The reason given is that it is always safer to rotate when the lock file exists. So this gets really nasty if you have rotating pattern in use with a mix of JDK versions because the JDK7 version will reuse the lock but the JDK8 version will leave it and rotate. Which is what you are doing with your test case.
Using JDK8 if I purge all log and lck files from the working directory and then run:
public static void main(String[] args) throws IOException {
System.out.println(System.getProperty("java.runtime.version"));
new FileHandler("./test_%u.log", 10000, 100, true).close();
}
I always see a file named 'test_0.log.0'. I get the same result using JDK7.
Bottom line is that is that you have to ensure your FileHandlers are closed. If it is never garbaged collected or removed from the logger tree then LogManager will close your FileHandler. Otherwise you have to close it. After that is fixed, purge all lock files before running your new patched code. Then be aware that if the JVM process crashed or is killed the lock file won't be deleted. If you have an I/O error on close your lock file won't be deleted. When the next process starts, the FileHandler will rotate.
As you point out, it is possible to use up all of the lock files on JDK8 if the above conditions occur over 100 runs. A simple test for this is to run the following code twice without deleting the log and lck files:
public static void main(String[] args) throws Exception {
System.out.println(System.getProperty("java.runtime.version"));
ReferenceQueue<FileHandler> q = new ReferenceQueue<>();
for (int i=0; i<100; i++) {
WeakReference<FileHandler> h = new WeakReference<>(
new FileHandler("./test_%u.log", 10000, 2, true), q);
while (q.poll() != h) {
System.runFinalization();
System.gc();
System.runFinalization();
Thread.yield();
}
}
}
However, the test case above won't work if JDK-6774110 is fixed correctly. The issue for this can be tracked on the OpenJDK site under RFR: 8048020 - Regression on java.util.logging.FileHandler and FileHandler webrev.

How to check FileLock without truncating file?

I recently added filelocks to my downloader asynctask:
FileOutputStream file = new FileOutputStream(_outFile);
file.getChannel().lock();
and after download completes, file.close() to release lock.
From a called BroadcastReceiver (different thread), I need to go through the files and see which are downloaded and which are still locked. I started with trylock:
for (int i=0; i<files.length; i++) {
try {
System.out.print((files[i]).getName());
test = new FileOutputStream(files[i]);
FileLock lock = test.getChannel().tryLock();
if (lock != null) {
lock.release();
//Not a partial download. Do stuff.
}
} catch (Exception e) {
e.printStackTrace();
} finally {
test.close();
}
}
Unfortunately I read the file is truncated (0 bytes) when the FileOutputStream is created.
I set it to append, but the lock doesn't seem to take effect, all appear to be un-locked (fully downloaded)
Is there another way to check if a write-lock is applied to the file currently, or am I using the wrong methods here? Also, is there a way to debug file-locks, from the ADB terminal or Eclipse?

None of this is going to work. Check the Javadoc. Locks are held on behalf of the entire process, i.e. the JVM, not by individual threads.

My first thought would be to open it for append per the javadocs
test = new FileOutputStream(files[i], true); // the true specifies for append