how do you guys test for an empty InputStream? I know that InputStream is designed to work with remote resources, so you can't know if it's there until you actually read from it. I cannot use read() because current position would change and using mark() and resetting for that seems to be inappropriate.
The problem is, that sometimes one can't test if read() returns -1, because if you have a stream and some third party library uses it, you need to test if it is empty before you send it there.
By empty InputStreams I mean these new ByteArrayInputStream(new byte[0])
You can wrap your InputStream in a PushbackInputStream. This class will store the first few bytes from read() in an internal buffer. You can later unread() the bytes and pass the object to the 3rd party library.
I don't really like ByteArrayInputStream, because it keeps all the data from the stream in memory.
Also, in any case, you will be forced to read() to check for the empty stream, which means you'll hit the network, at least for a few bytes.
A couple of alternatives:
ByteArrayInputStreams and several other similar classes are by definition non-blocking, as the data is already in the VM memory. In those cases the available() from InputStream could be what you need. This will not work when reading from an input source external to the program, e.g. a network socket, the standard input or perhaps even a file.
If the markSupported() method returns true for a specific InputStream instance, you may be able to use the mark() and reset() methods to return to the start of the stream after attempting to use read() on it.
EDIT:
By the way, ByteArrayInputStreams support mark() and reset() quite nicely and they are by default marked at position 0. This code:
InputStream x = new ByteArrayInputStream(new String("1234567890").getBytes());
byte b[] = new byte[1];
x.read(b, 0 , 1);
System.out.println(b[0]);
x.read(b, 0 , 1);
System.out.println(b[0]);
x.reset();
x.read(b, 0 , 1);
System.out.println(b[0]);
has this output:
49
50
49
Related
I'm new to Java ByteBuffers and was wondering what the correct way to write to a ByteBuffer after it has been flipped.
In my use case, I am writing an outputBuffer to a socket:
outBuffer.flip();
//Non-blocking SocketChannel
int bytesWritten = getSocket().write(outBuffer);
After this, the output buffer has to be written to again. Also not all of the bytes in the outBuffer may have been written to the socket.
Since it is currently flipped, how can I make it writable again, without overriding any data if it is still in the buffer and wasn't written to the socket?
If I am right, outBuffer.position() == bytesWritten and limit should be at how much data there was to write.
So would using the following in order to reuse the output buffer be right? :
int limit = outBuffer.limit()
outBuffer.limit(outBuffer.capacity());
outBuffer.position(limit);
Again from the API spec.:
The following loop copies bytes from one channel to another via the buffer buf:
while (in.read(buf) >= 0 || buf.position != 0) {
buf.flip();
out.write(buf);
buf.compact(); // In case of partial write
}
since it is currently flipped
It will stay flipped. The write doesn't change that.
how can I make it writable again, without overriding any data if it is still in the buffer and wasn't written to the socket?
You don't have to do anything, but if you want to read before you write again you should do flip/write/compact. If you just want to repeat the write just call write() again, with the buffer still in its current state.
But I prefer to always keep these buffers ready for reading, so there is no possibility of a slip-up, and to flip/write/compact (or flip/get/compact) when those operations are necessary, atomically as it were.
Note that you should not use clear(), unless you are certain that the write was complete and the buffer is now empty. In that case compact and clear are equivalent. But it is simpler to just always compact.
If you're copying in blocking mode, use the loop quoted by #zlakad.
I simply use
IOUtils.copy(myInputStream, myOutputStream);
And I see that before calling IOUtils.copy the input stream is avaible to read and after not.
flux.available()
(int) 1368181 (before)
(int) 0 (after)
I saw some explanation on this post, and I see I can copy the bytes from my InputStream to a ByteArrayInputStream and then use mark(0) and read(), in order to read multiple times an input stream.
Here is the code resulted (which is working).
I find this code very verbose, and I'd like if there is a better solution to do that.
ByteArrayInputStream fluxResetable = new ByteArrayInputStream(IOUtils.toByteArray(myInputStream));
fluxResetable.mark(0);
IOUtils.copy(fluxResetable, myOutputStream);
fluxResetable.reset();
An InputStream, unless otherwise stated, is single shot: you consume it once and that's it.
If you want to read it many times, that isn't just a stream any more, it's a stream with a buffer. Your solution reflects that accurately, so it is acceptable. The one thing I would probably change is to store the byte array and always create a new ByteArrayInputStream from it when needed, rather than resetting the same one:
byte [] content = IOUtils.toByteArray(myInputStream);
IOUtils.copy(new ByteArrayInputStream(content), myOutputStream);
doSomethingElse(new ByteArrayInputStream(content));
The effect is more or less the same but it's slightly easier to see what you're trying to do.
Sorry if this question is a dulplicate but I didn't get an answer I was looking for.
Java docs says this
In general, each read request made of a Reader causes a corresponding read request to be made of the underlying character or byte stream. It is therefore advisable to wrap a BufferedReader around any Reader whose read() operations may be costly, such as FileReaders >and InputStreamReaders. For example,
BufferedReader in = new BufferedReader(new FileReader("foo.in"));
will buffer the input from the specified file. Without buffering, each invocation of read() or readLine() could cause bytes to be read from the file, converted into characters, and then returned, which can be very inefficient.
My first question is If bufferedReader can read bytes then why can't we work on images which are in bytes using bufferedreader.
My second question is Does Bufferedreader store characters in BUFFER and what is the meaning of this line
will buffer the input from the specified file.
My third question is what is the meaning of this line
In general, each read request made of a Reader causes a corresponding read request to be >made of the underlying character or byte stream.
There are two questions here.
1. Buffering
Imagine you lived a mile from your nearest water source, and you drink a cup of water every hour. Well, you wouldn't walk all the way to the water for every cup. Go once a day, and come home with a bucket full of enough water to fill the cup 24 times.
The bucket is a buffer.
Imagine your village is supplied water by a river. But sometimes the river runs dry for a month; other times the river brings so much water that the village floods. So you build a dam, and behind the dam there is a reservoir. The reservoir fills up in the rainy season and gradually empties in the dry season. The village gets a steady flow of water all year round.
The reservoir is a buffer.
Data streams in computing are similar to both those scenarios. For example, you can get several kilobytes of data from a filesystem in a single OS system call, but if you want to process one character at a time, you need something similar to a reservoir.
A BufferedReader contains within it another Reader (for example a FileReader), which is the river -- and an array of bytes, which is the reservoir. Every time you read from it, it does something like:
if there are not enough bytes in the "reservoir" to fulfil this request
top up the "reservoir" by reading from the underlying Reader
endif
return some bytes from the "reservoir".
However when you use a BufferedReader, you don't need to know how it works, only that it works.
2. Suitability for images
It's important to understand that BufferedReader and FileReader are examples of Readers. You might not have covered polymorphism in your programming education yet, so when you do, remember this. It means that if you have code which uses FileReader -- but only the aspects of it that conform to Reader -- then you can substitute a BufferedReader and it will work just the same.
It's a good habit to declare variables as the most general class that works:
Reader reader = new FileReader(file);
... because then this would be the only change you need to add buffering:
Reader reader = new BufferedReader(new FileReader(file));
I took that detour because it's all Readers that are less suitable for images.
Reader has two read methods:
int read(); // returns one character, cast to an int
int read(char[] block); // reads into block, returns how many chars it read
The second form is unsuitable for images because it definitely reads chars, not ints.
The first form looks as if it might be OK -- after all, it reads ints. And indeed, if you just use a FileReader, it might well work.
However, think about how a BufferedReader wrapped around a FileReader will work. The first time you call BufferedReader.read(), it will call FileReader.read(buffer) to fill its buffer. Then it will cast the first char of the buffer back to an int, and return that.
Especially when you bring multi-byte charsets into the picture, that can cause problems.
So if you want to read integers, use InputStream not Reader. InputStream has int read(byte[] buf, int offset, int length) -- bytes are much more reliably cast back and forth from int than chars.
Readers (and Writers) in java are specialized classes for dealing with text (character) streams - the concept of a line is meaningless in any other type of stream.
for the general IO equivalent, have a look at BufferedInputStream
so, to answer your questions:
while the reader does eventually read bytes, it converts them to characters. it is not intended to read anything else (like images) - use the InputStream family of classes for that
a buffered reader will read large blocks of data from the underlying stream (which may be a file, socket, or anything else) into a buffer in memory and will then serve read requests from this buffer until the buffer is emptied. this behaviour of reading large chunks instead of smaller chucks every time improves performance.
it means that if you dont wrap a reader in a buffered reader then every time you want to read a single character, it will access the disk.network to get just the single character you want. doing I/O in such small chunks is usually terrible for performance.
Default behaviour is it will convert to character, but when you have an image you cannot have a character data, instead you need pixel of bytes data. So you cannot use it.
It is buffereing, means , it is reading a certain chunk of data in an char array. You can see this behaviour in the code:
public BufferedReader(Reader in) {
this(in, defaultCharBufferSize);
}
and the defaultCharBufferSize is as mentioned below:
private static int defaultCharBufferSize = 8192;
3 Every time you do read operation, it will be reading only one character.
So in a nutshell, buffred means, it will read few chunk of character data first that will keep in a char array and that will be processed and again it will read same chunk of data until it reaches end of stream
You can refer the following to get to know more
BufferedReader
Is there a way to flush the input stream in Java, perhaps prior to closing it? In relation to
iteratively invoking the statements below, while reading several files on disk
InputStream fileStream = item.openStream();
fileStream.close;
InputStream cannot be flushed. Why do you want to do this?
OutputStream can be flushed as it implements the interface Flushable. Flushing makes IMHO only sense in scenarios where data is written (to force a write of buffered data). Please see the documentation of Flushable for all implementing classes.
This is an old question but appears to be the only one of its kind, and I think there is a valid use case for "flushing" an input stream.
In Java, if you are using a BufferedReader or BufferedInputStream (which I believe is a common case), "flushing" the stream can be considered to be equivalent to discarding all data currently in the buffer -- i.e., flushing the buffer.
For an example of when this might be useful, consider implementing a REPL for a programming language such as Python or similar.
In this case you might use a BufferedReader on System.in. The user enters a (possibly large) expression and hits enter. At this point, the expression (or some part of it) is stored in the buffer of your Reader.
Now, if a syntax error occurs somewhere within the expression, it will be caught and displayed to the user. However, the remainder of the expression still resides in the input buffer.
When the REPL loop continues, it will begin reading just beyond the point where the syntax error occurred, in the middle of some erroneous expression. This is likely not desirable. Rather, it would be better to simply discard the remainder of the buffer and continue with a "fresh start."
In this sense, we can use the BufferedReader API method ready() to discard any remaining buffered characters. The documentation reads:
"Tells whether this stream is ready to be read. A buffered character stream is ready if the buffer is not empty, or if the underlying character stream is ready."
Then we can define a method to "flush" a BufferedReader as:
void flush(BufferedReader input) throws IOException
{
while (input.ready())
input.read();
}
which simply discards all remaining data until the buffer is empty. We then call flush() after handling a syntax error (by displaying to the user). When the REPL loop resumes you have an empty buffer and thus do not get a pile of meaningless errors caused by the "junk" left in the buffer.
(Forgive me because I do not write in Java very often.)
I'm writing a client-side network application in Java and I'm having an interesting issue. Every call to readInt() throws an EOFException. The variable is of type DataInputStream (initialized as: DataInputStream din = new DataInputStream(new BufferedInputStream(sock.getInputStream())); where sock is of type Socket).
Now, sock.isInputShutdown() returns false and socket.isConnected() returns true. I'm assuming that this means that I have a valid connection to the other machine I'm connecting to. I've also performed other checks to ensure that I'm properly connected to the other machine.
Is it possible that the DataInputStream was not set up correctly? Are there any preconditions that I have missed?
Any help is greatly appreciated.
#tofubeer: I actually wrote 17 bytes to the socket. The socket is connected to another machine and I'm waiting on input from that machine (I'm sorry if this was unclear). I successfully read from the stream (to initiate a handshake) first and this worked just fine. I'm checking now to see if my sent-requests are malformed, but I don't think they are. Also, I tried reading a single byte from the stream (via read()) and it returned -1.
Are you writing 4 bytes to the socket? According to the JavaDoc it will throw an EOFException if this stream reaches the end before reading all the bytes.
Try calling readByte() 4 times in a row instead of readInt() and see what happens (likely not all of them will work).
Edit (given your edit).
Find out how many times you can call read() before you get the -1.
When read() returns -1 it means that it has hit the end of file.
Also find out what each read() returns to make sure what you are reading in is what you actually wrote out.
It sounds like a problem either with the read code reading more than you thing while doing the handshake or the other side not writing what you think it is writing.
Some things to check:
Did the handshake consume more than 13 bytes, leaving less than four for the readInt()?
Was the integer you want to read written via DataOutputStream.writeInt()?
Did you flush the stream from the sender?
Edit: I took a look at the Java sources (I have the 1.4 sources on my desktop, not sure which version you're using) and the problem might be in BufferedInputStream. DataInputStream.readInt() is just calling BufferedInputStream.read() four times. BufferedInputStream.read() is calling BufferedInputStream.fill() if its buffer is exhausted (e.g., if its first read only got 16 bytes). BufferedInputStream.fill() calls the underlying InputStream's read(byte[], int, int) method, which by contract might not actually read anything! If this happens, BufferedInputStream.read() will return an erroneous EOF.
This is all assuming that I'm reading all of this correctly, which might not be the case. I only took a quick peek at the sources.
I suspect that your BufferedInputStream is only getting the first 16 bytes of the stream in its first read. I'd be curious what your DataInputStream's available() returns right before the readInt. If you're not already, I'd suggest you flush your OutputStream after writing the int you can't read as a possible workaround.