I have a delimited file which can contain around millions of records , now I want to delete the first line from the delimited file before processing it further.
The length of the first line is variable , it differs as per the file type.. now I have done a readup on the FileChannel and RandomAccessFile which have been suggested as the best ways to delete the first line.
But I am unable to figure it out , as to how to get the length of the first line and delete it.
Don't delete it, just read-and-ignore.
If you have to prepare the file because the file processing units can't handle a file with an incorrect first line, then you'll have to read and rewrite it. There is no I/O operation available that can delete contents from file in the filesystem.
use readLine() to read line by line , just ommit first line and consider others in processing
Thanks for the inputs. Depending on the same , I figured out a solution to remove the first line from the delimited pipe file.
Mentioned below is a code snippet
RandomAccessFile raf = new RandomAccessFile("path to ur delimited file", "rw");
FileChannel fileChannel = raf.getChannel();
raf.readLine();
raf.seek(raf.getFilePointer());
int len = (int) (raf.length() - raf.getFilePointer());
byte[] bytearr = new byte[len];
raf.readFully(bytearr, 0, len);
fileChannel.truncate(0);
raf.write(bytearr,0,len);
You could use a BufferedReader and use BufferedReader.readLine() to "delete" the first line before processing. From here you could continue to process the rest of the lines or store them into a file to process later. The latter option might not be the most efficient option available to you.
Related
I run a small experiment trying to read a file using BufferedReader, and I wanted to see what would happen if I call the delete method on the file before the read is complete, and given that BufferedReader will only read a chunk of the file at a time I expected the operation to fail, but to my surprise the read was successful.
Here is the code I used:
val file = File("test.txt")
val bufferedReader = BufferedReader(InputStreamReader(FileInputStream(file)), 1)
if (file.delete())
println("file deleted successfully")
println(bufferedReader.readLines().size)
I used a relatively big file for the test with around 300mb of size, and I also set the buffer size to the minimum value possible, and the execution returns this:
file deleted successfully
1303692
Did I misunderstand something here? and could someone please explain this behavior?
The motivation behind this experiment is that I have a method in my application that returns a sequence of all lines in a temporary file, and I wanted to remove the temporary file once all lines were read like this:
fun getTempFileLines(): Sequence<String> {
val file = File("temp.txt")
val bufferedReader = BufferedReader(InputStreamReader(FileInputStream(file)))
val sequenceOfLines = generateSequence {
bufferedReader.readLine()
}
file.delete()
return sequenceOfLines
}
From https://docs.oracle.com/javase/8/docs/api/java/io/BufferedReader.html
"Reads text from a character-input stream, buffering characters so as to provide for the efficient reading of characters, arrays, and lines."
So while the actual file may already be deleted, the bufferedReader still contains contents.
I have just relised that I have a file where only one line exists with a long string. This file (line) can be300MB heavy.
I would like stream some data from this string and save in another file
i.e the line from the file would look like:
String line = "{{[Metadata{"this, is my first, string"}]},{[Metadata{"this, is my second, string"}]},...,{[Metadata{"this, is my 5846 string"}]}}"
Now I would like to take 100 items from this string from one "Metadata" to another "Metadata", save it in the file and continue with the rest of the data.
So in the nutshell from one line I would like to get N files with i.e. 100 Metadata strings each
BufferedReader reader = new BufferedReader(new StringReader(line));
This is what I've got and I don't know what I can do with the reader.
Probably
reader.read(????)
but I don't know what to put inside :(
Can you please help
I was experimenting with BufferedReader to read 1st line file to a string. How do I do this? Also how can I read an entire file to a string? How to read a particular line like readline(int line) without iterating through the previous lines?
File namefile = new File(root, ".name");
FileReader namereader = new FileReader(namefile);
BufferedReader in = new BufferedReader(namereader);
You can use BufferedReader.readLine() to get the first line.
Note that the next call to readLine() will get you the 2nd line, and the next the 3rd line....
EDIT:
If you want to specify a specific line, as your comment suggest - you might want to use Apache Commons FileUtils, and use: FileUtils.readLines(). It will get you a List<String> which you can handle like any list, including getting a specific line. Note that it has more overhead because it reads the entire file, and populates a List<String> with its lines.
Um, what's wrong with BufferedReader.readLine()?
Reads a line of text. A line is considered to be terminated by any one of a line feed ('\n'), a carriage return ('\r'), or a carriage return followed immediately by a linefeed.
(I don't see any sign of a readFile() method though - what documentation were you looking at?)
Personally I prefer to use FileInputStream wrapped in InputStreamReader instead of FileReader by the way, as otherwise it will always use the platform default encoding - are you sure what's what you want?
final File namefile = new File(root, ".name");
final FileReader namereader = new FileReader(namefile);
final BufferedReader in = new BufferedReader(namereader);
in.readLine();
If you use the BufferedReader to read the File there should be a Method called
readLine()
wich reads exactly one Line.
http://developer.android.com/reference/java/io/BufferedReader.html
See the readline() method of the BufferedReader.
http://docs.oracle.com/javase/1.4.2/docs/api/java/io/BufferedReader.html#readLine%28%29
I want to read the last 2 lines in some files, and if the content of second last line matches a specific string, then delete the last line only.
Also, after the above operation, 2 lines of data have to be appended to the end of the modified file. I saw other questions on SO that deal with different parts of my problem, but is there an easy way of doing all of the above with minimal code, preferably in a single function? (I can combine the different functions available at SO but that would be messy...)
I would recommend you to do it "in memory". It's easy to read line by line into a List, check the last lines and update the lines and write it back to the file.
Example:
public static void main(String[] args) throws IOException {
String fileName = "test.txt";
List<String> lines = new ArrayList<String>();
// read the file into lines
BufferedReader r = new BufferedReader(new FileReader(fileName));
String in;
while ((in = r.readLine()) != null)
lines.add(in);
r.close();
// check your condition
String secondFromBottom = lines.get(lines.size() - 2);
if (secondFromBottom.matches("Hello World!")) {
lines.remove(lines.size() - 1);
lines.add("My fixed string");
}
// write it back
PrintWriter w = new PrintWriter(new FileWriter(fileName));
for (String line : lines)
w.println(line);
w.close();
}
Note: No exception handling is done in the example above... you need to handle cases where the file for example doesn't contain two lines and other problems!
If you have really big files and perfomance is an issue the way to go is to use a RandomAccessFile and read backwards looking for the line termination bytes to determine where the last two lines begin. Otherwise use dacwe's approach.
As Gandalf said you can:
take RandomAccessFile,
use method seek(long) to jump forward and read those lines. But you won't know exactly how big the jump should be.
to delete last lines you need the position of begin of last line so before reading each line store their file pointer position (method getFilePointer()). Deleting to that position you use setLength(long).
My example of reading and deleting last lines you have here:
Deleting the last line of a file with Java
Useful can be also:
Quickly read the last line of a text file?
I have a requirement that I need to append block of a file to other file.
let say I have 100 lines in source file and I need to append the 50 lines from the bottom to destination file.
To navigate to the 50th line I using .readLine() of the BufferedReader.
Once I reached to 50 th line I want to append remaining content to the destination file.
I don't want to append line by line as it is consuming much time.
Please help me how to do that..
Please provide a code snippet if possible.
read the file line by line and append in StringBuffer object
for eg
StringBuffer lineBuffer = new StringBuffer();
String line = "";
while((line=br.readLine())!=null){
///// any condition or anything you want to do
lineBuffer.append(line);
}
now after you get the lines into the stringbuffer object write this into another file