I cannot really grasp what the purpose is for the FileReader and BufferedReader classes in Java.
At docs.oracle one is recommended to wrap a buffered reader around a FileReader object because it's not efficient to use the FileReader directly. Where does the cost or overhead come from?
Let say I have a text file that I want to read into my java program using these classes:
I use FileReader and BufferedReader
FileReader fileReader = new FileReader(new File("text.txt)"); // probably correct???
BufferedReader bufferedReader = new BufferedReader(fileReader);
1) What is the task of the FileReader object here? Is it responsible to make an I/O-request via the OS to the file, and thereafter read bytes? What is the cost with this?
Is it true that the FileReader makes several I/O-requests? Or is the cost when the FileReader object has to converting bytes to characters, character by character?
2) The task of the BufferedReader-object - to refer to the last sentence above. - Is the role for the BufferedReader-object to simply buffer arrays of incoming bytes and THEN convert those to character?
Very grateful for answers
Edit: first of all thanks for incoming answers. But I should have mentioned that its exactly this documentation I have studied. Call me stupid or something - but what is meant by "each read request". WHEN is each read request made? how often?
In general, each read request made of a Reader causes a corresponding read request to be made of the underlying character or byte stream. It is therefore advisable to wrap a BufferedReader around any Reader whose read() operations may be costly, such as FileReaders and InputStreamReaders. For example,
This is mostly why a launched this question - It sounds that the FileReader causes a lot of I/O-request which slows erverthing down.
From oracle docs :
In general, each read request made of a Reader causes a corresponding
read request to be made of the underlying character or byte stream. It
is therefore advisable to wrap a BufferedReader around any Reader
whose read() operations may be costly, such as FileReaders and
InputStreamReaders. For example,
BufferedReader in = new BufferedReader(new FileReader("foo.in"));
will buffer the input from the specified file. Without buffering, each
invocation of read() or readLine() could cause bytes to be read from
the file, converted into characters, and then returned, which can be
very inefficient.
So, as the document clearly suggests, Wrapping a BufferedReader around a FileReader prevents reading of data from the file over and over again. BufferedReader buffers the input.
Java has many I/O classes which may be combined. So you might see something like:
BufferedReader in = new BufferedReader(new InputStreamReader(
new FileInputStream(new File("..."), "UTF-8"));
...
in.close(); // Closes all.
This allows flexible combination. So an XML parser could use Reader, and not care where the text comes from: file, URL, memory. This as opposed to "simpler" languages, where there is no choice of implementation (Map with implementations HashMap, TreeMap).
Now, FileReader and FileWriter are old utility classes to read from and write to a file in the default operating system encoding. So for local files. Not portable (!) to other operating systems, or requiring a fixed encoding.
For this FileReader extends InputStreamReader which reads binary data (an InputStream) using a FileInputStream. It does just that.
However it makes sense to use a larger memory buffer to read, hence the advice; which also is long-standing. Remember performance was an issue in the early times of java.
The only advantage of FileReader-standalone would be for tight memory situations, maybe on a smart phone.
Related
I would like to know the specific difference between BufferedReader and FileReader.
I do know that BufferedReader is much more efficient as opposed to FileReader, but can someone please explain why (specifically and in detail)? Thanks.
First, You should understand "streaming" in Java because all "Readers" in Java are built upon this concept.
File Streaming
File streaming is carried out by the FileInputStream object in Java.
// it reads a byte at a time and stores into the 'byt' variable
int byt;
while((byt = fileInputStream.read()) != -1) {
fileOutputStream.write(byt);
}
This object reads a byte(8-bits) at a time and writes it to the given file.
A practical useful application of it would be to work with raw binary/data files, such as images or audio files (use AudioInputStream instead of FileInputStream for audio files).
On the other hand, it is very inconvenient and slower for text files, because of looping through a byte at a time, then do some processing and store the processed byte back is tedious and time-consuming.
You also need to provide the character set of the text file, i.e if the characters are in Latin or Chinese, etc. Otherwise, the program would decode and encode 8-bits at a time and you'd see weird chars printed on the screen or written in the output file (if a char is more than 1 byte long, i.e. non-ASCII characters).
File Reading
This is just a fancy way of saying "File streaming" with inclusive charset support (i.e no need to define the charset, like earlier).
The FileReader class is specifically designed to deal with the text files.
As you've seen earlier, the file streaming is best to deal with raw binary data, but for the sake of text, it is not so efficient.
So the Java-dudes added the FileReader class, to deal specifically with the text files. It reads 2 bytes (or 4 bytes, depends on the charset) at a time. A remarkably huge improvement over the preceding FileInputStream!!
so the streaming operation is like this,
int c;
while ( (c = fileReader.read()) != -1) { // some logic }
Please note, Both classes use an integer variable to store the value retrieved from the input file (so every char is converted into an integer while fetching and back to the char while storing).
The only advantage here is that this class deals only with text files, so you don't have to specify the charset and a few other properties. It provides an out-of-the-box solution, for most of the text files processing cases. It also supports internationalization and localization.
But again it's still very slow (Imaging reading 2 bytes at a time and looping through it!).
Buffering streams
To tackle the problem of continuous looping over a byte or 2. The Java-dudes added another spectacular functionality. "To create a buffer of data, before processing."
The concept is pretty much alike when a user streams a video on YouTube. A video is buffered before playing, to provide flawless video watching experience. (Tho, the browser keeps buffering until the whole video is buffered ahead of time.) The same technique is used by the BufferedReader class.
A BufferedReader object takes a FileReader object as an input which contains all the necessary information about the text file that needs to be read. (such as the file path and charset.)
BufferedReader br = new BufferedReader( new FileReader("example.txt") );
When the "read" instruction is given to the BufferedReader object, it uses the FileReader object to read the data from the file. When an instruction is given, the FileReader object reads 2 (or 4) bytes at a time and returns the data to the BufferedReader and the reader keeps doing that until it hits '\n' or '\r\n' (The end of the line symbol).
Once a line is buffered, the reader waits patiently, until the instruction to buffer the next line is given.
Meanwhile, The BufferReader object creates a special memory place (On the RAM), called "Buffer", and stores all the fetched data from the FileReader object.
// this variable points to the buffered line
String line;
// Keep buffering the lines and print it.
while ((line = br.readLine()) != null) {
printWriter.println(line);
}
Now here, instead of reading 2 bytes at a time, a whole line is fetched and stored in the RAM somewhere, and when you are done with processing the data, you can store the whole line back to the hard disk. So it makes the process run way faster than doing 2 bytes a time.
But again, why do we need to pass FileReader object to the BufferReader? Can't we just say "buffer this file" and the BufferReader would take care of the rest? wouldn't that be sweet?
Well, the BufferReader class is created in a way that it only knows how to create a buffer and to store incoming data. It is irrelevant to the object from where the data is coming. So the same object can be used for many other input streams than just text files.
So being said that, When you provide the FileReader object as an input, it buffers the file, the same way if you provide the InputStreamReader as an object, it buffers the Terminal/Console input data until it hits a newline symbol. such as,
// Object that reads console inputs
InputStreamReader console = new InputStreamReader(System.in);
BufferedReader br = new BufferedReader(console);
System.out.println(br.readLine());
This way, you can read (or buffer) multiple streams with the same BufferReader class, such as text files, consoles, printers, networking data etc, and all you have to remember is,
bufferedReader.readLine();
to print whatever you've buffered.
In simple manner:
A FileReader class is a general tool to read in characters from a File. The BufferedReader class can wrap around Readers, like FileReader, to buffer the input and improve efficiency. So you wouldn't use one over the other, but both at the same time by passing the FileReader object to the BufferedReader constructor.
Very Detail
FileReader is used for input of character data from a disk file. The input file can be an ordinary ASCII, one byte per character text file. A Reader stream automatically translates the characters from the disk file format into the internal char format. The characters in the input file might be from other alphabets supported by the UTF format, in which case there will be up to three bytes per character. In this case, also, characters from the file are translated into char format.
As with output, it is good practice to use a buffer to improve efficiency. Use BufferedReader for this. This is the same class we've been using for keyboard input. These lines should look familiar:
BufferedReader stdin =
new BufferedReader(new InputStreamReader( System.in ));
These lines create a BufferedReader, but connect it to an input stream from the keyboard, not to a file.
Source: http://www.oopweb.com/Java/Documents/JavaNotes/Volume/chap84/ch84_3.html
BufferedReader requires a Reader, of which FileReader is one - it descends from InputStreamReader, which descends from Reader.
FileReader - read character files
BufferedReader - "Read text from a character-input stream, buffering characters so as to provide for the efficient reading of characters, arrays, and lines."
http://docs.oracle.com/javase/7/docs/api/java/io/BufferedReader.html
http://docs.oracle.com/javase/7/docs/api/java/io/FileReader.html
Actually BufferedReader makes use of Readers like FileReader.
FileReader class helps in writing on file but its efficency is low since it has yo retrive one character at a time from file but BufferedReader takes chunks of data and store it in buffer so instead of retriving one character at atime from file retrival becomes easy using buffer.
Bufferedreader - method that you can use actually as a substitute for Scanner method, gets file, gets input.
FileReader - as the name suggests.
I'm having trouble understanding how do buffers work in Java IO.
Excuse me if I don't express myself as clearly as I would like, I'm not
strong on all these concepts.
As I undestand it, in Java there are readers/writers, for chars (meaning the
possibility of more than one byte per char and encoding options), and streams
for bytes.
Then there are some classes that use buffers.
I believe that a buffer is used mainly so that we can avoid unnecessary system
calls that would involve expensive operations, like accesing a slower device, by
storing more in memory and making the system call useful for more data.
The issue I have is that there seem to be buffering classes for both readers/writers and streams.
I would like to know if buffering characters is enough, or if, by the time those
bytes get to the streaming class, they would be flushed on for example newlines,
as some classes state.
The best I've found about this is this post
How do these different code snippets compare in regard to buffering?
Does autoflush thwart the intent of buffering?
Should there be only one buffer in play, and if so, where (writer or stream)
and why?:
// resolveSocket is some socket
PrintWriter out = new PrintWriter(
resolveSocket.getOutputStream(),
true);
// vs
PrintWriter out = new PrintWriter(
new OutputStreamWriter(
new BufferedOutputStream(resolveSocket.getOutputStream()),
StandardCharsets.UTF_8),
true)
My interest is first and foremost to understand buffering better, and practical only after that.
Thak you all in advance.
EDIT: Found this other stack overflow question interesting and related.
Also this other one talks about BufferedOutputStream.
It may help you to understand the difference between a writer and a stream. An OutputStream is a binary sink. A Writer has a character encoding and understands about newlines. The OutputStreamWriter allows you to send character encoded data, and have it correctly translated to binary for consumption by an OutputStream. What you probably want is:
PrintWriter out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(resolveSocket.getOutputStream())));
This will allow you to use your PrintWriter to output chars, have it buffered by the BufferedWriter, and then translated by the OutputStreamWriter for consumption by the socket output stream.
What i want to do is, when I used an initialised bufferedReader which i used to read the lines from a file using readLine(), when I reach the end of file i want to use the same bufferedReader object to read lines from the beginnig of the same file for an example.
what i attempted to achieve this matter is, I used mark() and reset(), after initialising the bufferedReader I set bufferReaderobject.mark(0) and I read the entire file and after that I used bufferReaderobject.reset() and agian read the entire file but then an exception was thrown. it seems that reset() only valid when the end of file has not reached.
is there any way to reset a previously initialised bufferedReader to the beginnig of the stream?
To be able to use reset() in such a way, the whole file must fit into the buffer of the BufferedReader.
That means you must either create a BufferedReader with a buffer that is large enough or create a new instance when you want to read the file again.
Try this:
FileInputStream fIn = ...;
BufferedReader bRead = new BufferedReader(new InputStreamReader(fIn));
// "reset" to beginning of file (discard old buffered reader)
fIn.getChannel().position(0);
bRead = new BufferedReader(new InputStreamReader(fIn));
The documentation of BufferedReader for mark method calls the parameter readAheadLimit and states that
An attempt to reset the stream after reading characters up to this limit or beyond may fail.
So it seems that mark only works forward and doesn't works for reset to the start.
Also take a look at this question.
I would like to know the specific difference between BufferedReader and FileReader.
I do know that BufferedReader is much more efficient as opposed to FileReader, but can someone please explain why (specifically and in detail)? Thanks.
First, You should understand "streaming" in Java because all "Readers" in Java are built upon this concept.
File Streaming
File streaming is carried out by the FileInputStream object in Java.
// it reads a byte at a time and stores into the 'byt' variable
int byt;
while((byt = fileInputStream.read()) != -1) {
fileOutputStream.write(byt);
}
This object reads a byte(8-bits) at a time and writes it to the given file.
A practical useful application of it would be to work with raw binary/data files, such as images or audio files (use AudioInputStream instead of FileInputStream for audio files).
On the other hand, it is very inconvenient and slower for text files, because of looping through a byte at a time, then do some processing and store the processed byte back is tedious and time-consuming.
You also need to provide the character set of the text file, i.e if the characters are in Latin or Chinese, etc. Otherwise, the program would decode and encode 8-bits at a time and you'd see weird chars printed on the screen or written in the output file (if a char is more than 1 byte long, i.e. non-ASCII characters).
File Reading
This is just a fancy way of saying "File streaming" with inclusive charset support (i.e no need to define the charset, like earlier).
The FileReader class is specifically designed to deal with the text files.
As you've seen earlier, the file streaming is best to deal with raw binary data, but for the sake of text, it is not so efficient.
So the Java-dudes added the FileReader class, to deal specifically with the text files. It reads 2 bytes (or 4 bytes, depends on the charset) at a time. A remarkably huge improvement over the preceding FileInputStream!!
so the streaming operation is like this,
int c;
while ( (c = fileReader.read()) != -1) { // some logic }
Please note, Both classes use an integer variable to store the value retrieved from the input file (so every char is converted into an integer while fetching and back to the char while storing).
The only advantage here is that this class deals only with text files, so you don't have to specify the charset and a few other properties. It provides an out-of-the-box solution, for most of the text files processing cases. It also supports internationalization and localization.
But again it's still very slow (Imaging reading 2 bytes at a time and looping through it!).
Buffering streams
To tackle the problem of continuous looping over a byte or 2. The Java-dudes added another spectacular functionality. "To create a buffer of data, before processing."
The concept is pretty much alike when a user streams a video on YouTube. A video is buffered before playing, to provide flawless video watching experience. (Tho, the browser keeps buffering until the whole video is buffered ahead of time.) The same technique is used by the BufferedReader class.
A BufferedReader object takes a FileReader object as an input which contains all the necessary information about the text file that needs to be read. (such as the file path and charset.)
BufferedReader br = new BufferedReader( new FileReader("example.txt") );
When the "read" instruction is given to the BufferedReader object, it uses the FileReader object to read the data from the file. When an instruction is given, the FileReader object reads 2 (or 4) bytes at a time and returns the data to the BufferedReader and the reader keeps doing that until it hits '\n' or '\r\n' (The end of the line symbol).
Once a line is buffered, the reader waits patiently, until the instruction to buffer the next line is given.
Meanwhile, The BufferReader object creates a special memory place (On the RAM), called "Buffer", and stores all the fetched data from the FileReader object.
// this variable points to the buffered line
String line;
// Keep buffering the lines and print it.
while ((line = br.readLine()) != null) {
printWriter.println(line);
}
Now here, instead of reading 2 bytes at a time, a whole line is fetched and stored in the RAM somewhere, and when you are done with processing the data, you can store the whole line back to the hard disk. So it makes the process run way faster than doing 2 bytes a time.
But again, why do we need to pass FileReader object to the BufferReader? Can't we just say "buffer this file" and the BufferReader would take care of the rest? wouldn't that be sweet?
Well, the BufferReader class is created in a way that it only knows how to create a buffer and to store incoming data. It is irrelevant to the object from where the data is coming. So the same object can be used for many other input streams than just text files.
So being said that, When you provide the FileReader object as an input, it buffers the file, the same way if you provide the InputStreamReader as an object, it buffers the Terminal/Console input data until it hits a newline symbol. such as,
// Object that reads console inputs
InputStreamReader console = new InputStreamReader(System.in);
BufferedReader br = new BufferedReader(console);
System.out.println(br.readLine());
This way, you can read (or buffer) multiple streams with the same BufferReader class, such as text files, consoles, printers, networking data etc, and all you have to remember is,
bufferedReader.readLine();
to print whatever you've buffered.
In simple manner:
A FileReader class is a general tool to read in characters from a File. The BufferedReader class can wrap around Readers, like FileReader, to buffer the input and improve efficiency. So you wouldn't use one over the other, but both at the same time by passing the FileReader object to the BufferedReader constructor.
Very Detail
FileReader is used for input of character data from a disk file. The input file can be an ordinary ASCII, one byte per character text file. A Reader stream automatically translates the characters from the disk file format into the internal char format. The characters in the input file might be from other alphabets supported by the UTF format, in which case there will be up to three bytes per character. In this case, also, characters from the file are translated into char format.
As with output, it is good practice to use a buffer to improve efficiency. Use BufferedReader for this. This is the same class we've been using for keyboard input. These lines should look familiar:
BufferedReader stdin =
new BufferedReader(new InputStreamReader( System.in ));
These lines create a BufferedReader, but connect it to an input stream from the keyboard, not to a file.
Source: http://www.oopweb.com/Java/Documents/JavaNotes/Volume/chap84/ch84_3.html
BufferedReader requires a Reader, of which FileReader is one - it descends from InputStreamReader, which descends from Reader.
FileReader - read character files
BufferedReader - "Read text from a character-input stream, buffering characters so as to provide for the efficient reading of characters, arrays, and lines."
http://docs.oracle.com/javase/7/docs/api/java/io/BufferedReader.html
http://docs.oracle.com/javase/7/docs/api/java/io/FileReader.html
Actually BufferedReader makes use of Readers like FileReader.
FileReader class helps in writing on file but its efficency is low since it has yo retrive one character at a time from file but BufferedReader takes chunks of data and store it in buffer so instead of retriving one character at atime from file retrival becomes easy using buffer.
Bufferedreader - method that you can use actually as a substitute for Scanner method, gets file, gets input.
FileReader - as the name suggests.
I want to read each line from a text file and store them in an ArrayList (each line being one entry in the ArrayList).
So far I understand that a BufferedInputStream writes to the buffer and only does another read once the buffer is empty which minimises or at least reduces the amount of operating system operations.
Am I correct - do I make sense?
If the above is the case in what situations would anyone want to use DataInputStream. And finally which of the two should I be using and why - or does it not matter.
Use a normal InputStream (e.g. FileInputStream) wrapped in an InputStreamReader and then wrapped in a BufferedReader - then call readLine on the BufferedReader.
DataInputStream is good for reading primitives, length-prefixed strings etc.
The two classes are not mutually exclusive - you can use both of them if your needs suit.
As you picked up, BufferedInputStream is about reading in blocks of data rather than a single byte at a time. It also provides the convenience method of readLine(). However, it's also used for peeking at data further in the stream then rolling back to a previous part of the stream if required (see the mark() and reset() methods).
DataInputStream/DataOutputStream provides convenience methods for reading/writing certain data types. For example, it has a method to write/read a UTF String. If you were to do this yourself, you'd have to decide on how to determine the end of the String (i.e. with a terminator byte or by specifying the length of the string).
This is different from BufferedInputStream's readLine() which, as the method sounds like, only returns a single line. writeUTF()/readUTF() deal with Strings - that string can have as many lines it it as it wants.
BufferedInputStream is suitable for most text processing purposes. If you're doing something special like trying to serialize the fields of a class to a file, you'd want to use DataInput/OutputStream as it offers greater control of the data at a binary level.
Hope that helps.
You can always use both:
final InputStream inputStream = ...;
final BufferedInputStream bufferedInputStream =
new BufferedInputStream(inputStream);
final DataInputStream dataInputStream =
new DataInputStream(bufferedInputStream);
InputStream: Base class to read byte from stream (network or file ), provide ability to read byte from the stream and delete the end of the stream.
DataInputStream: To read data directly as a primitive datatype.
BufferInputStream: Read data from the input stream and use buffer to optimize the speed to access the data.
You shoud use DataInputStream in cases when you need to interpret the primitive types in a file written by a language other Java in platform-independent manner.
I would advocate using Jakarta Commons IO and the readlines() method (of whatever variety).
It'll look after buffering/closing etc. and give you back a list of text lines. I'll happily roll my own input stream wrapping with buffering etc., but nine times out of ten the Commons IO stuff works fine and is sufficient/more concise/less error prone etc.
The differences are:
The DataInputStream works with the binary data, while the BufferedReader work with character data.
All primitive data types can be handled by using the corresponding methods in DataInputStream class, while only string data can be read from BufferedReader class and they need to be parsed into the respective primitives.
DataInputStream is a part of filtered streams, while BufferedReader is not.
DataInputStream consumes less amount of memory space being it is a binary stream, whereas BufferedReader consumes more memory space being it is character stream.
The data to be handled is limited in DataInputStream, whereas the number of characters to be handled has wide scope in BufferedReader.