My app reads text file line by line and record offset of each line until the end of file. offset returns changed value when readLine is first executed. But it does not change any more after that. What is wrong with my code? I use RandomAccessFile instead of FileInputStream because seek() is faster than skip() when file is big.
String buffer;
long offset;
RandomAccessFile raf = new RandomAccessFile("data.txt", "r");
FileInputStream is = new FileInputStream(raf.getFD());
BufferedReader br = new BufferedReader(new InputStreamReader(is));
while (true) {
offset = raf.getFilePointer(); // offset remains the same after 1st readLine. why?
if ((buffer = br.readLine()) == null) // buffer has correct value.
return;
………………………………
}
Because BufferedReader is buffered. So it reads the data into it's buffer the first time and then just keeps it there until it needs more buffered data.
If you want to use a smaller buffer for testing purposes, try new BufferedReader(new InputStreamReader(is), 1000); or something. Your pointer should now increment by 1000 occasionally.
If you want your counter to work properly, you can do one of two things. Either you can count the characters you are receiving and then do some converting to byte lengths which you can use to make your own counter or you can use a FileReader with no buffering which will increment the counter the way you expect.
Update: It seems FileReader does something behind the scenes. I'd use something like new CountingInputStream(new BufferedInputStream(new FileInputStream(raf.getFD())) loop through the data in byte form, manually identify line endings while dumping the bytes into a String. Not the prettiest way, but the only way I can think of given Reader's internal buffering. I think CountingInputStream is provided by Apache freely and it contains a count method or something like that.
Related
Say we have a file like so:
one
two
three
(but this file got encrypted)
My crypto method returns the whole file in memory, as a byte[] type.
I know byte arrays don't have a concept of "lines", that's something a Scanner (for example) could have.
I would like to traverse each line, convert it to string and perform my operation on it but I don't know
how to:
Find lines in a byte array
Slice the original byte array to "lines" (I would convert those slices to String, to send to my other methods)
Correctly traverse a byte array, where each iteration is a new "line"
Also: do I need to consider the different OS the file might have been composed in? I know that there is some difference between new lines in Windows and Linux and I don't want my method to work only with one format.
Edit: Following some tips from answers here, I was able to write some code that gets the job done. I still wonder if this code is worthy of keeping or I am doing something that can fail in the future:
byte[] decryptedBytes = doMyCrypto(fileName, accessKey);
ByteArrayInputStream byteArrInStrm = new ByteArrayInputStream(decryptedBytes);
InputStreamReader inStrmReader = new InputStreamReader(byteArrInStrm);
BufferedReader buffReader = new BufferedReader(inStrmReader);
String delimRegex = ",";
String line;
String[] values = null;
while ((line = buffReader.readLine()) != null) {
values = line.split(delimRegex);
if (Objects.equals(values[0], tableKey)) {
return values;
}
}
System.out.println(String.format("No entry with key %s in %s", tableKey, fileName));
return values;
In particular, I was advised to explicitly set the encoding but I was unable to see exactly where?
If you want to stream this, I'd suggest:
Create a ByteArrayInputStream to wrap your array
Wrap that in an InputStreamReader to convert binary data to text - I suggest you explicitly specify the text encoding being used
Create a BufferedReader around that to read a line at a time
Then you can just use:
String line;
while ((line = bufferedReader.readLine()) != null)
{
// Do something with the line
}
BufferedReader handles line breaks from all operating systems.
So something like this:
byte[] data = ...;
ByteArrayInputStream stream = new ByteArrayInputStream(data);
InputStreamReader streamReader = new InputStreamReader(stream, StandardCharsets.UTF_8);
BufferedReader bufferedReader = new BufferedReader(streamReader);
String line;
while ((line = bufferedReader.readLine()) != null)
{
System.out.println(line);
}
Note that in general you'd want to use try-with-resources blocks for the streams and readers - but it doesn't matter in this case, because it's just in memory.
As Scott states i would like to see what you came up with so we can help you alter it to fit your needs.
Regarding your last comment about the OS; if you want to support multiple file types you should consider making several functions that support those different file extensions. As far as i know you do need to specify which file and what type of file you are reading with your code.
My app reads text file line by line and record offset of each line until the end of file.
But position() always returns 0.
What is wrong with my code?
String buffer;
long offset;
RandomAccessFile raf = new RandomAccessFile("data.txt", "r");
FileChannel channel = raf.getChannel();
BufferedReader br = new BufferedReader(new InputStreamReader(Channels.newInputStream(channel)));
while (true) {
offset = channel.position(); // offset is always 0. why?
if ((buffer = br.readLine()) == null) // buffer has correct value.
return;
………………………………
}
I cannot reproduce your error, that is, offset is not always 0 when I run your code. Still, it doesn't do what you expect it to do. You create a BufferedReader on top of your FileChannel. The BufferedReader will fill its buffer (and thus increase the offset in the channel) and then read from the buffer until its empty. So after calling br.readLine() once, the offset is not the length of the string you've read, it is the length of the buffer.
You can better use a BufferedReader and FileInputStream directly and count the characters by some other means.
I have a buffer reader working to read a text file then save the read data as a string. The buffer reader is in a continuous loop reading the text file every second. I would like to do a Thread.sleep if the buffer reader reads the same text two times in a row.
For example if the buffer reader reads "foo" then when it is ran again in the next second if it reads "foo" again then the thread sleeps for a certain amount of time.
I have the buffer reader within a try method but I do not know how to get it to do the above.
Could anyone help with this?
Thanks
Keep track of the previous line read in in a String variable, which you can initialize as an empty string. When you read in each line, check whether it's equals() to the previously stored line in that variable. If so, do what you want to do, like Thread.sleep(). Otherwise save the current line into that variable. In code:
BufferedReader reader = new BufferedReader(new FileReader("foo.txt"));
String foo = "";
String old = "";
while((foo = reader.readLine()) != null) {
if(foo.equals(old)) {
System.out.println("Sleeping...");
Thread.sleep(1000);
} else {
old = foo;
}
}
I'm writing an InputStream that supplies lines from a file in constant intervals. I used BufferedReader before, but ran into buffering issues with it (wasn't getting anything until the entire file was read), and speed isn't an issue anyways (the intervals are something like every second, or every half second - along those lines). Is there a class with a readLine method like in BufferedReader, except unbuffered?
(Edit: I just checked - my class seems to work, apparently the problem was with the output)
Here's the code where I used the stream (OnlineDataSimulator). I already checked, the stream does exactly what I want, so apparently I'm doing something wrong with the output. (The actual problem is, I want output to occur every X milliseconds - X being the second parameter to OnlineDataSimulator. What happens when I run this code is, that I first get an X*lines wait and then the entire output at once instead.)
System.out.println("Testing:");
PrintStream fout = new PrintStream(new FileOutputStream("testfile"));
for(int i=0; i<20; ++i) {
fout.println(i);
}
fout.close();
BufferedReader fin = new BufferedReader(new InputStreamReader(
new OnlineDataSimulator("testfile",250)));
String line;
while((line=fin.readLine())!= null){
System.out.println(line);
System.out.flush();
}
fin.close();
(new File("testfile")).delete();
Try it this way.... This worked for me..
File f = new File("path");
FileReader fr = new FileReader(f);
BufferedReader br = new BufferedReader(fr);
String s = null;
while ((s=br.readLine())!=null)
{
System.out.println(s);
}
No, there is no other non-buffered option.
A solution would be to write your own Reader which has a InputStreamReader as an underlying stream and in the readLine() method you should call read() of the underlying input stream reader until "\n" is found. Aggregate all these and return them as a string.
If you don't want to have a real buffer but want to use the functionality of BufferedReader you could initialize it with buffer size 1. As you commented that speed isn't an issue maybe is the most reliable solution.
new BufferedReader(reader, 1)
public BufferedReader(Reader in, int sz)
and you can check the readLine() method source code here, in case you want to implement your own.
I have 2 classes who must read an InputStream, the first one should only interpret the first line of the stream BUT the first line should be removed from the stream so that class B can interpret everything after the first line. Which doesn't work when I pass my InputStream to a BufferedReader and do a readLine().
I know I could do a read on the stream until I've encountered a \b but maybe a more proper solution exists to do the job?
// Reads the first line from the stream and everything else
public String retrieveFileNameFromTheFirstLineInInputStream(InputStream in) throws IOException {
InputStreamReader isReader = new InputStreamReader(in);
BufferedReader reader = new BufferedReader(isReader);
return reader.readLine();
}
You can't remove something from an InputStream, you just can read from it. Don't use the BufferedReader to read the line, because it surely will read much more than the first line from the InputStreamReader (to fill its buffer) which itself reads from the InputStream.
I'd suggest to read using the InputStreamReader until the end of the line is reached, then pass the InputStream instance to your code which should read it.
BTW, you always should specify the encoding used by the InputStreamReader, otherwise the system encoding will be used to convert the bytes from the InputStream to characters which can differ on different machines.
I believe even InputStreamReader can buffer input, so Mike L's answer can miss input.
It's awkward, but you could use ReaderInputStream from Apache commons-io. So:
BufferedReader reader = new BufferedReader(
new InputStreamReader(in));
String firstLine = reader.readLine();
InputStream in2 = new ReaderInputStream(reader);
// continue with in2 ..