I tried to use FileChannel.transferFrom to move some content of a file to the begining.
try (RandomAccessFile rafNew = new RandomAccessFile(_fileName, "rw");
RandomAccessFile rafOld = new RandomAccessFile(_fileName, "r");)
{
rafOld.seek(pos);
rafOld.getChannel().transferTo(0, count, rafNew.getChannel());
} catch (IOException e) {
throw new RuntimeException(e.getMessage());
}
The result of this is a file with strange repetitions of data. The example works if I first transfer data to a buffer file and then from buffer file back to the origin file again.
The Java Docs say nothing about the case where source and destination are the same file.
You are transferring 'count' bytes starting from zero from 'rafOld' to 'rafNew', which hasn't had any seeks done on it, so is also at position zero. So at best your code doesn't do what you said it does. The seek() you did on 'rafOld' doesn't affect the transferTo() operation. You should have removed it and written
transferTo(pos, count, rafNew.getChannel());
But there are still two problems with this.
If count > pos you will be overwriting the source region.
transferTo() must be called in a loop, as it isn't guaranteed to compete the entire transfer in a single call. It returns the number of bytes actually transferred,
Related
I'm reading a file line by line, and I am trying to make it so that if I get to a line that fits my specific parameters (in my case if it begins with a certain word), that I can overwrite that line.
My current code:
try {
FileInputStream fis = new FileInputStream(myFile);
DataInputStream in = new DataInputStream(fis);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
if (line.startsWith("word")) {
// replace line code here
}
}
} catch (Exception ex) {
ex.printStackTrace();
}
...where myFile is a File object.
As always, any help, examples, or suggestions are much appreciated.
Thanks!
RandomAccessFile seems a good fit. Its javadoc says:
Instances of this class support both reading and writing to a random access file. A random access file behaves like a large array of bytes stored in the file system. There is a kind of cursor, or index into the implied array, called the file pointer; input operations read bytes starting at the file pointer and advance the file pointer past the bytes read. If the random access file is created in read/write mode, then output operations are also available; output operations write bytes starting at the file pointer and advance the file pointer past the bytes written. Output operations that write past the current end of the implied array cause the array to be extended. The file pointer can be read by the getFilePointer method and set by the seek method.
That said, since text files are a sequential file format, you can not replace a line with a line of a different length without moving all subsequent characters around, so to replace lines will in general amount to reading and writing the entire file. This may be easier to accomplish if you write to a separate file, and rename the output file once you are done. This is also more robust in case if something goes wrong, as one can simply retry with the contents of the initial file. The only advantage of RandomAccessFile is that you do not need the disk space for the temporary output file, and may get slight better performance out of the disk due to better access locality.
Your best bet here is likely going to be reading in the file into memory (Something like a StringBuilder) and writing what you want your output file to look like into the StringBuilder. After you're done reading in the file completely, you'll then want to write the contents of the StringBuilder to the file.
If the file is too large to accomplish this in memory you can always read in the contents of the file line by line and write them to a temporary file instead of a StringBuilder. After that is done you can delete the old file and move the temporary one in its place.
An old question, recently worked on this. Sharing the experience
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
public static void updateFile(Path file) {
// Get all the lines
try (Stream<String> stream = Files.lines(file,StandardCharsets.UTF_8)) {
// Do the replace operation
List<String> list = stream.map(line -> line.replaceAll("test", "new")).collect(Collectors.toList());
// Write the content back
Files.write(file, list, StandardCharsets.UTF_8);
} catch (IOException e) {
e.printStackTrace();
}
}
A custom file format being devised requires robust file corruption checking, currently implemented via a SHA-2 checksum appended at the end of the file. Given the large sizes of the given file, it takes a while to validate the SHA-2 checksums and thus it has been decided to place the final file size near the start of the file to quickly filter out files with mismatched file lengths.
Maybe I am just thinking too much, but is there anyway the following example will fail?
File outputFile = ... // Output file
try(FileOutputStream fOut = new FileOutputStream(outputFile);
FileChannel fChannel = fOut.getChannel()){
fOut.write(MAGIC); // byte array to magic header
fOut.write(new Byte[8]); // reserve space for eventual file size
//// [Bulk Writing to File] ////
// Ensure all writing is complete
fOut.flush();
fOut.getFD().sync(); // Is this necessary?
// Write final file size to file
ByteBuffer finalFileSize = ByteBuffer.allocate(8);
finalFileSize.order(ByteOrder.BIG_ENDIAN);
finalFileSize.putLong(outputFile.length()); // Will this statement return an inaccurate file length?
fChannel.position(MAGIC.length);
fChannel.write(finalFileSize);
}catch(IOException ex){
// Exception handle code... deletes current file and starts again.
}
I am particularly worried about outputFile.length() returning an invalid length due to the file stream being unclosed (and thus some bytes could persist in memory/metadata not updated on specific platforms).
In my particular case, having the filelength simply be unavailable (written as 0) is better than having it be invalid since the corruption detection code will ignore file lengths <= 0 and move on SHA-2 validation but will reject positive filelength mismatches.
Is my implementation sufficient or do I need to resort to writing a counting stream wrapper around the FIS to make sure the file length is correct?
I am always curious how a rolling file is implemented in logs.
How would one even start creating a file writing class in any language in order to ensure that the file size is not exceeded.
The only possible solution I can think of is this:
write method:
size = file size + size of string to write
if(size > limit)
close the file writer
open file reader
read the file
close file reader
open file writer (clears the whole file)
remove the size from the beginning to accommodate for new string to write
write the new truncated string
write the string we received
This seems like a terrible implementation, but I can not think up of anything better.
Specifically I would love to see a solution in java.
EDIT: By remove size from the beginning is, let's say I have 20 byte string (which is the limit), I want to write another 3 byte string, therefore I remove 3 bytes from the beginning, and am left with end 17 bytes, and by appending the new string I have 20 bytes.
Because your question made me look into it, here's an example from the logback logging framework. The RollingfileAppender#rollover() method looks like this:
public void rollover() {
synchronized (lock) {
// Note: This method needs to be synchronized because it needs exclusive
// access while it closes and then re-opens the target file.
//
// make sure to close the hereto active log file! Renaming under windows
// does not work for open files
this.closeOutputStream();
try {
rollingPolicy.rollover(); // this actually does the renaming of files
} catch (RolloverFailure rf) {
addWarn("RolloverFailure occurred. Deferring roll-over.");
// we failed to roll-over, let us not truncate and risk data loss
this.append = true;
}
try {
// update the currentlyActiveFile
currentlyActiveFile = new File(rollingPolicy.getActiveFileName());
// This will also close the file. This is OK since multiple
// close operations are safe.
// COMMENT MINE this also sets the new OutputStream for the new file
this.openFile(rollingPolicy.getActiveFileName());
} catch (IOException e) {
addError("setFile(" + fileName + ", false) call failed.", e);
}
}
}
As you can see, the logic is pretty similar to what you posted. They close the current OutputStream, perform the rollover, then open a new one (openFile()). Obviously, this is all done in a synchronized block since many threads are using the logger, but only one rollover should occur at a time.
A RollingPolicy is a policy on how to perform a rollover and a TriggeringPolicy is when to perform a rollover. With logback, you usually base these policies on file size or time.
I'm reading a file line by line, and I am trying to make it so that if I get to a line that fits my specific parameters (in my case if it begins with a certain word), that I can overwrite that line.
My current code:
try {
FileInputStream fis = new FileInputStream(myFile);
DataInputStream in = new DataInputStream(fis);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
if (line.startsWith("word")) {
// replace line code here
}
}
} catch (Exception ex) {
ex.printStackTrace();
}
...where myFile is a File object.
As always, any help, examples, or suggestions are much appreciated.
Thanks!
RandomAccessFile seems a good fit. Its javadoc says:
Instances of this class support both reading and writing to a random access file. A random access file behaves like a large array of bytes stored in the file system. There is a kind of cursor, or index into the implied array, called the file pointer; input operations read bytes starting at the file pointer and advance the file pointer past the bytes read. If the random access file is created in read/write mode, then output operations are also available; output operations write bytes starting at the file pointer and advance the file pointer past the bytes written. Output operations that write past the current end of the implied array cause the array to be extended. The file pointer can be read by the getFilePointer method and set by the seek method.
That said, since text files are a sequential file format, you can not replace a line with a line of a different length without moving all subsequent characters around, so to replace lines will in general amount to reading and writing the entire file. This may be easier to accomplish if you write to a separate file, and rename the output file once you are done. This is also more robust in case if something goes wrong, as one can simply retry with the contents of the initial file. The only advantage of RandomAccessFile is that you do not need the disk space for the temporary output file, and may get slight better performance out of the disk due to better access locality.
Your best bet here is likely going to be reading in the file into memory (Something like a StringBuilder) and writing what you want your output file to look like into the StringBuilder. After you're done reading in the file completely, you'll then want to write the contents of the StringBuilder to the file.
If the file is too large to accomplish this in memory you can always read in the contents of the file line by line and write them to a temporary file instead of a StringBuilder. After that is done you can delete the old file and move the temporary one in its place.
An old question, recently worked on this. Sharing the experience
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
public static void updateFile(Path file) {
// Get all the lines
try (Stream<String> stream = Files.lines(file,StandardCharsets.UTF_8)) {
// Do the replace operation
List<String> list = stream.map(line -> line.replaceAll("test", "new")).collect(Collectors.toList());
// Write the content back
Files.write(file, list, StandardCharsets.UTF_8);
} catch (IOException e) {
e.printStackTrace();
}
}
I actually checked other posts that could be related to this and I couldn't find any answer to my question. So, had to create this newly:
The file does not get created in the given location with this code:
File as = new File ("C:\\Documents and Settings\\<user>\\Desktop\\demo1\\One.xls");
if (!as.exists()) {
as.createNewFile();
}
FileOutputStream fod = new FileOutputStream(as);
BufferedOutputStream dob = new BufferedOutputStream(fod);
byte[] asd = {65, 22, 123};
byte a1 = 87;
dob.write(asd);
dob.write(a1);
dob.flush();
if (dob!=null){
dob.close();
}
if(fod!=null){
fod.close();
The code runs fine and I don't get any FileNotFoundException!!
Is there anything that I'm missing out here?
You can rewrite your code like this:
BufferedOutputStream dob = null;
try {
File file = new File("C:\\Documents and Settings\\<user>\\Desktop\\demo1\\One.xls");
System.out.println("file created:" + file.exists());
FileOutputStream fod = new FileOutputStream(file);
System.out.println("file created:" + file.exists());
BufferedOutputStream dob = new BufferedOutputStream(fod);
byte[] asd = {65, 22, 123};
byte a1 = 87;
dob.write(asd);
dob.write(a1);
//dob.flush();
}
catch (Exception ex) {
ex.printStackTrace();
}
finally {
if (dob != null) {
dob.close();
}
}
In this case it is only necessary to call the topmost stream handler close() method - the BufferedOutputStream's one:
Closes this output stream and releases any system resources associated with the stream.
The close method of FilterOutputStream calls its flush method, and then calls the close method of its underlying output stream.
so, the dob.flush() in try block is commented out because the dob.close() line in the finally block flushes the stream. Also, it releases the system resources (e.g. "closes the file") as stated in the apidoc quote above. Using the finally block is a good practice:
The finally block always executes when the try block exits. This ensures that the finally block is executed even if an unexpected exception occurs. But finally is useful for more than just exception handling — it allows the programmer to avoid having cleanup code accidentally bypassed by a return, continue, or break. Putting cleanup code in a finally block is always a good practice, even when no exceptions are anticipated.
The FileOutputStream constructor creates an empty file on the disk:
Creates a file output stream to write to the file represented by the specified File object. A new FileDescriptor object is created to represent this file connection.
First, if there is a security manager, its checkWrite method is called with the path represented by the file argument as its argument.
If the file exists but is a directory rather than a regular file, does not exist but cannot be created, or cannot be opened for any other reason then a FileNotFoundException is thrown.
Where a FileDescriptor is:
Instances of the file descriptor class serve as an opaque handle to the underlying machine-specific structure representing an open file, an open socket, or another source or sink of bytes. The main practical use for a file descriptor is to create a FileInputStream or FileOutputStream to contain it.
Applications should not create their own file descriptors.
This code should either produce a file or throw an exception. You have even confirmed that no conditions for throwing exception are met, e.g. you are replacing the string and the demo1 directory exists. Please, rewrite this to a new empty file and run.
If it still behaving the same, unless I have missed something this might be a bug. In that case, add this line to the code and post output:
System.out.println(System.getProperty("java.vendor")+" "+System.getProperty("java.version"));
Judging from the path, I'd say you are using Win 7, am I right? What version?
Then it means there is a file already in your directory