I try this code :
byte arr[] = ByteBuffer.allocate(2).putInt(1).array()
But it fails with a BufferOverflowException.
Is 1 too big to be stored in 2 bytes ? Or is my problem somewhere else ?
Is 1 too big to be stored in 2 bytes ?
Well, an int is... putInt always writes 4 bytes. From the documentation for ByteBuffer.putInt
Throws:
BufferOverflowException - If there are fewer than four bytes remaining in this buffer
If you only want to put a two-byte integer, use putShort instead. If you want to store data in a variable-width encoding (where the space taken depends on the value), you'll probably need to write the code yourself.
From javadoc of method putInt:
Writes four bytes containing the given int value, in the current byte order, into this buffer at the current position, and then increments the position by four
You allocated only 2 bytes so the BufferOverflowException:
Unchecked exception thrown when a relative put operation reaches the target buffer's limit.
You can solve expading to 4 bytes the buffer or using putShort that use only 2 bytes to store the number 1.
ByteBuffer.putInt(1) puts an integer value into the buffer. An integer is 4 byte long (32 bit).
Related
I have a few questions about the way InputStream.read() works. I'm trying to listen on InputStream and whenever bytes are available, I need them to be copied to a byte array and then the byte array is handed off to another method for further processing and then return to listening to the InputStream for next set of bytes. Here's how far I've gotten:
while(true){
while((i = inputstream.read(recvBuffer, 0, recvBuffer.length)) != -1){
doSomething(recvBuffer);
}
}
I read that read(byte[] b, int off, int len) method returns an int that is supposed to represent the number of bytes read from the stream. Doesn't that mean whenever bytes are available, it sets the value of i to the respective number of bytes? Once it reads x number of bytes and reaches the end of the stream, wouldn't it return a positive integer representing the number of bytes, instead of -1? Then, when would the check against -1 happen for i? I know I'm interpreting this wrong but I can't say how. Any help understanding this would be appreciated
Also, I know the max amount of bytes that a sender can push onto a stream. In this case, is it sufficient to specify the size of recvBuffer as the max amount of bytes or is it prudent to allocate a bit more than that?
doSomething(recvBuffer);
That should be
doSomething(recvBuffer, i);
The method needs to know how many bytes were actually received.
I read that read(byte[] b, int off, int len) method returns an int that is supposed to represent the number of bytes read from the stream.
Correct.
Doesn't that mean whenever bytes are available, it sets the value of
i to the respective number of bytes?
Yes.
Once it reads x number of bytes
and reaches the end of the stream, wouldn't it return a positive
integer representing the number of bytes, instead of -1?
No, it transfers the bytes and returns the count, then the next time there are no bytes, only end of stream, so it returns -1.
Then, when would the check against -1 happen for i? I know I'm interpreting this
wrong but I can't say how.
See above.
Also, I know the max amount of bytes that a sender can push onto a stream. In this case, is it sufficient to specify the size of recvBuffer as the max amount of bytes or is it prudent to allocate a bit more than that?
Most people use 4096 or 8192 bytes. There's not a lot of point in specifying a buffer larger than the path MTU in truth, which is normally < 1500, unless you are slow at reading so that the kernel socket receive buffer fills up.
Consider we have a socket connection between two device (A and B). Now if I write only 16 bytes (size doesn't matter here) to the output stream (not BufferedOutputStream) of the socket in side A 3 times or in general more than once like this:
OutputStream outputStream = socket.getOutputStream();
byte[] buffer = new byte[16];
OutputStream.write(buffer);
OutputStream.write(buffer);
OutputStream.write(buffer);
I read the data in side B using the socket input stream (not BufferedInputStream) with a buffer larger than sending buffer for example 1024:
InputStream inputStream = socket.getInputStream();
byte[] buffer = new byte[1024];
int read = inputStream.read(buffer);
Now I wonder how the data is received on side B? May it get accumulated or it exactly read the data as A sends it? In another word may the read variable get more than 16?
InputStream makes very few guarantees about how much data will be read by any invocation of the multi-byte read() methods. There is a whole class of common programming errors that revolve around misunderstandings and wrong assumptions about that. For example,
if InputStream.read(byte[]) reads fewer bytes than the provided array can hold, that does not imply that the end of the stream has been reached, or even that another read will necessarily block.
the number of bytes read by any one invocation of InputStream.read(byte[]) does not necessarily correlate to any characteristic of the byte source on which the stream draws, except that it will always read at least one byte when not at the end of the stream, and that it will not read more bytes than are actually available by the time it returns.
the number of available bytes indicated by the available() method does not reliably indicate how many bytes a subsequent read should or will read. A nonzero return value reliably indicates only that the next read will not block; a zero return value tells you nothing at all.
Subclasses may make guarantees about some of those behaviors, but most do not, and anyway you often do not know which subclass you actually have.
In order to use InputStreams properly, you generally must be prepared to perform repeated reads until you get sufficient data to process. That can mean reading until you have accumulated a specific number of bytes, or reading until a delimiter is encountered. In some cases you can handle any number of bytes from any given read; generally these are cases where you are looping anyway, and feeding everything you read to a consumer that can accept variable length chunks (many compression and encryption interfaces are like that, for example).
Per the docs:
public int read(byte[] b) throws IOException
Reads some number of bytes from the input stream and stores them into the buffer array b. The number of bytes
actually read is returned as an integer. This method blocks until
input data is available, end of file is detected, or an exception is
thrown. If the length of b is zero, then no bytes are read and 0 is
returned; otherwise, there is an attempt to read at least one byte. If
no byte is available because the stream is at the end of the file, the
value -1 is returned; otherwise, at least one byte is read and stored
into b.
The first byte read is stored into element b[0], the next one into
b[1], and so on. The number of bytes read is, at most, equal to the
length of b. Let k be the number of bytes actually read; these bytes
will be stored in elements b[0] through b[k-1], leaving elements b[k]
through b[b.length-1] unaffected.
Read(...) tells you how many bytes it put into the array and yes, you can read further; you'll get whatever was already there.
I have whole byte buffer of message whose bytes have been word and byte swapped coming across sockets. I have methods that can do this swapping for 2 shorts or long value. But I was thinking I should really do this on the whole byte array first and then my parsing routines will work. Is there any byte buffere tools for swapping bytes/words for a whole buffer????????????
You can use ByteBuffer.order(endianess)
socket.getChannel().read(byteBuffer); // More or less
byteBuffer.order(ByteOrder.BIG_ENDIAN);
byteBuffer.order(ByteOrder.LITTLE_ENDIAN);
This will use LSB or MSB first (least/most significant byte first). There still are main frames where 4-bytes words have a still different order. But here it should be just what you wanted.
The javadoc says the following.
Parameters:
b - destination buffer.
off - offset at which to start storing bytes.
len - maximum number of bytes to read.
I would like to confirm my understanding of the "offset at which to start storing bytes". Does this mean that off is "the index at the destination buffer b at which to start storing bytes"? It does sound like off means it. I think the method is more usable if off is the "offset at the BufferedInputStream at which to start reading bytes", but I want to confirm. I tried looking at the source but it's hard to read.
A side question: When 1024 bytes of a stream is read, will the said 1024 bytes always be removed from the stream?
Does this mean that off is "the index at the destination buffer b at which to start storing bytes"?
It's documented: "The first byte read is stored into element b[off]".
When 1024 bytes of a stream is read, will the said 1024 bytes always be removed from the stream?
Of course, but you seem to be really asking whether 1024 bytes will always be read if you supply a buffer of 1024 bytes. Answer is no. It's documented: "there is an attempt to read at least one byte".
Yes. off is the index in b where the stream will start entering len bytes.
When 1024 bytes of a stream is read, will the said 1024 bytes always be removed from the stream?
Using an InputStream, you have no knowledge of what's going on underneath. All you know are the methods available to you and what they do (what their documentation says). Implementations can do whatever they want.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I was trying to find out efficient data types.
I know int is 4 bytes and char is one byte.
an object which contains five integers (4 * 5 = 20 bytes)
a String object which has ten characters. ( Suppose it has 10 characters 10 * 1 = 10 bytes)
Am I right?
Which one do you think it is better?
The objective answer first:
Primitive data types are documented here
Strings are more complicated because the JVM can intern them. See a good explanation here
The not so objective answer: pick the data structure that makes for the best design for your application.
If you have a specific constraint in your application, post more details about the data you need to handle and the constraints you have.
A String is not just an array of characters, it is an independent object, and has fields other than its backing char[]. For example, String has three int fields: offset, count and hash. The empty string, therefore, is generally 16 bytes (since we also need to take the char[] field into account) plus the normal 8 bytes of object overhead. Also note that a char[] is itself an object, and has the int field length and an associated object overhead. Once you have taken all this into account, then you can add the two (not one!) bytes per char.
So, for a 10-character string:
3 int fields: 12 bytes
char[] field: 8 bytes
int field: 4 bytes
object overhead: 8 bytes
10 characters: 20 bytes
object overhead: 8 bytes
This comes out to about 60 bytes. I say "about" because some of this is dependent on the VM.
You are incorrect about chars in Java: since they are designed to hold 16-bit UNICODE code points, they take two, not one byte each. In the end, both representations will take the same amount of memory.
You should pick the data type that makes the most sense to you, the designer of your classes, and to the readers of your code. Memory concerns should not be at the top of your design priorities unless the number of objects that you need threatens to overflow your available memory. Even then you should do careful memory profiling before you optimize.
Characters are 2 bytes in size. They are equivalent to an unsigned short, so a character's value can range between [0, 65535] inclusive.
The number of bytes a String occupies is actually:
string.length * 2
So for your example, a 10 character string occupies 20 bytes, not 10 bytes.
This would be just the string content. There are other variables within the String class which will occupy more bytes of course. And even an empty object occupies a certain number of bytes that will vary based on the JVM implementation.
However, just the character content will occupy 2 bytes per character.
But don't worry about this as its most assuredly premature optimization. Clean code is more important than lightning fast code usually. Pick appropriate data types, write code that's easy to follow and read. These things are more important.
If you are worried about holding large strings in memory consider changing your approach. The most common problem I see with large strings is when new programmers read an entire file into memory.
If you are doing this, try processing data line by line. Only hold the smallest unit you need in memory at a time, perform your processing, and move on.
I know int is 4 bytes
correct
and char is one byte.
A char is a 16-bit unsigned integer, so 2 bytes
an object which contains five integers (4 * 5 = 20 bytes)
A Object has a header which is 12 bytes on a 32-bit JVM and 16 bytes on a 64-bit JVM. Objects are 8 byte aligned, possibly 16 or 32 byte aligned if this is changed.
This means a new int[5] uses 16 + 20 + 4 (padding) = 40 bytes
a String object which has ten characters. ( Suppose it has 10 characters 10 * 1 = 10 bytes)
A String uses ~24 bytes with header and length fields etc, but it wraps a char[] which contains the actual chars, which is a further 16+20+4 = 40 bytes.
A simple way to check this is to use the following. Make sure you use -XX:-UseTLAB which improves memory accounting (but is slower for multi-threaded programming)
public static void main(String... ignored) {
char[] chars = new char[10];
long used = memoryUsed();
String s= new String(chars);
long diff = memoryUsed() - used;
if (diff == 0) throw new AssertionError("You must set -XX:-UseTLAB on the command line");
System.out.printf("Creating a String of 10 characters used %,d bytes of memory%n", diff);
}
private static long memoryUsed() {
return Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
}
prints
Creating a String of 10 characters used 64 bytes of memory