How can I concat two ByteBuffers to one ByteBuffer?
The following doesn't work:
ByteBuffer bb = ByteBuffer.allocate(100);
ByteBuffer bb2 = ByteBuffer.allocate(200);
bb.allocate(200).put(bb2);
System.out.println(bb.array().length);
The length of bb is still 100.
Something like
bb = ByteBuffer.allocate(300).put(bb).put(bb2);
should do the job: Create a buffer that is large enough to hold the contents of both buffers, and then use the relative put-methods to fill it with the first and the second buffer. (The put method returns the instance that the method was called on, by the way)
We'll be copying all data. Remember that this is why string concatenation is expensive!
public static ByteBuffer concat(final ByteBuffer... buffers) {
final ByteBuffer combined = ByteBuffer.allocate(Arrays.stream(buffers).mapToInt(Buffer::remaining).sum());
Arrays.stream(buffers).forEach(b -> combined.put(b.duplicate()));
return combined;
}
you can use the method here
https://github.com/ata4/ioutils/blob/047e401d73c866317af2e12f7803b3ee43eec80a/src/main/java/info/ata4/io/buffer/ByteBufferUtils.java#L289
and for example:
ByteBuffer concat() {
int length = 0;
for (ByteBuffer bb : buffers) {
bb.rewind();
length += bb.remaining();
}
ByteBuffer bbNew = ByteBuffer.allocateDirect((int) length);
// put all buffers from list
for (ByteBuffer bb : buffers) {
bb.rewind();
bbNew.put(bb);
}
bbNew.rewind();
return bbNew;
}
Probably because on line3 i.e. bb.allocate(200).put(bb2); ,
bb.allocate(200) is returning a new Byte buffer (See https://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html#allocate(int) ). That is not actually changing bb itself. So its still the bytebuffer of capacity 100 from line1.
Try the following code:
//store both ByteBuffer object as an array
byte[] array1 = ByteBuffer.allocate(100).array();
byte[] array2 = ByteBuffer.allocate(200).array();
//create a ByteBuffer which is big enough
ByteBuffer bigenough = ByteBuffer.allocate(array1.length + array2.length);
//put the two arrays in one ByteBuffer
ByteBuffer after1 = bigenough.put(array1, 0, array1.length);
ByteBuffer result = after1.put(array2, array1.length, array2.length);
//print the length of the combined array.
System.out.println(result.array().length);
Related
I have an array byte[] arr;
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] arr = out.toByteArray();
How can I measure the data size in arr (if it was written to disk or transferred via network)?
Are below approaches are correct - they suppose that sizeof(byte) = 1B
int byteCount = out.size();
int byteMsgCount = arr.length;
Yes, by definition the size of a variable of type byte is one byte. So the length of your array is indeed array.length bytes.
out.size() will give you the same value, i.e. the number of bytes that you wrote into the output stream.
[Edit] From cricket_007 comment: if you look at the implementation of size and toByteArray
public synchronized byte toByteArray()[] {
return Arrays.copyOf(buf, count);
}
public synchronized int size() {
return count;
}
... so toByteArray basically copies the current output buffer, up to count bytes. So using size is a better solution.
I need to serialize an array of doubles to base64 in Java. I have following method from C#
public static string DoubleArrayToBase64( double[] dValues ) {
byte[] bytes = new byte[dValues.Length * sizeof( double )];
Buffer.BlockCopy( dValues, 0, bytes, 0, bytes.Length );
return Convert.ToBase64String( bytes );
}
How do I do that in Java? I tried
Byte[] bytes = new Byte[abundaceArray.length * Double.SIZE];
System.arraycopy(abundaceArray, 0, bytes, 0, bytes.length);
abundanceValues = Base64.encodeBase64String(bytes);
however this leads to an IndexOutofBoundsException.
How can I achieve this in Java?
EDIT:
Buffer.BlockCopy copies on byte level, the last paramter is number of bytes. System.arraycopy last parameter is number of elements to copy. So yes it should be abundaceArray.length but then a ArrayStoreException is thrown.
EDIT2:
The base64 string must be the same as the ine created with the c# code!
You get an ArrayStoreException when the array types on the method are not the same primitive, so double to byte will not work. Here is a workaround i patched up that seems to work. I do not know of any method in the java core that does automatic conversion from primitive to byte block :
public class CUSTOM {
public static void main(String[] args) {
double[] arr = new double[]{1.1,1.3};
byte[] barr = toByteArray(arr);
for(byte b: barr){
System.out.println(b);
}
}
public static byte[] toByteArray(double[] from) {
byte[] output = new byte[from.length*Double.SIZE/8]; //this is reprezented in bits
int step = Double.SIZE/8;
int index = 0;
for(double d : from){
for(int i=0 ; i<step ; i++){
long bits = Double.doubleToLongBits(d); // first transform to a primitive that allows bit shifting
byte b = (byte)((bits>>>(i*8)) & 0xFF); // bit shift and keep adding
int currentIndex = i+(index*8);
output[currentIndex] = b;
}
index++;
}
return output;
}
}
The Double.SIZE get 64 which is number of bits I suggest to initialize the array like this
Byte[] bytes = new Byte[abundaceArray.length * 8];
Not sure what this C# function does, but I suspect you should replace this line
System.arraycopy(abundaceArray, 0, bytes, 0, bytes.length);
with this
System.arraycopy(abundaceArray, 0, bytes, 0, abundaceArray.length);
I'm guessing you're using the apache commons Base64 class. That only has methods accepting an array of bytes (the primitive type), not Bytes (object wrapper around primitive type).
It's not clear what type your 'abundaceArray' is - whether it's doubles or Doubles.
Either way, you can't use System.arraycopy to copy between arrays of difference primitive types.
I think your best bet is to serialise your array object to a byte array, then base64 encode that.
eg:
ByteArrayOutputStream b = new ByteArrayOutputStream(); // to store output from serialization in a byte array
ObjectOutputStream o = new ObjectOutputStream(b); // to do the serialization
o.writeObject(abundaceArray); // arrays of primitive types are serializable
String abundanceValues = Base64.encodeBase64String(b.toByteArray());
There is of course an ObjectInputStream for going in the other direction at the other end.
I have connected by TCP to a socket which is constantly sending a large amount of data, which I need to read in. What I have so far is a byte buffer that is reading byte by byte in a while loop. But the test case I am using right now is about 3 MB, which takes a while to read when reading in byte by byte.
Here is my code for this explanation:
ByteBuffer buff = ByteBuffer.allocate(3200000);
while(true)
{
int b = in.read();
if(b == -1 || buff.remaining() == 0)
{
break;
}
buff.put((byte)b);
}
I know that byte buffers are not thread safe and I'm not sure if this could be made faster by possibly reading in multiple bytes at a time and then storing it in the buffer? What would be a way for me to speed this process up?
Use a bulk read instead of a single byte read.
byte[] buf = new byte[3200000];
int pos = 0;
while (pos < buf.length) {
int n = in.read(buf, pos, buf.length - pos);
if (n < 0)
break;
pos += n;
}
ByteBuffer buff = ByteBuffer.wrap(buf, 0, pos);
Instead of getting an InputStream from the socket, and filling a byte array to be wrapped, you can get the SocketChannel and read() directly to the ByteBuffer.
There are several ways.
Use Channels.newChannel() to get a channel from the input stream and use ReadableByteChannel.read(buffer).
Get the byte[] array from the buffer with buffer.array() and read directly into that with in.read(array). Make sure the BB really does have an array of course. If it's a direct byte buffer it won't, but in that case you shouldn't be doing all this at all, you should be using a SocketChannel, otherwise there is zero benefit.
Read into your own largeish byte array and then use a bulk put into the ByteBuffer, taking care to use the length returned by the read() method.
Don't do it. Make up your mind as to whether you want InputStreams or ByteBuffers and don't mix your programming metaphors.
Okay, so I was looking up what the best way to convert from a byte array to it's numeric value in java was and I came across this link. And the second answer mentions the use of the ByteBuffer class. For those that do not wish to click on the link, originally the question asks if I have:
byte[] by = new byte[8];
How does one convert that to int? Well the answer goes...
ByteBuffer bb = ByteBuffer.wrap(new byte[] {0, 0, 0, 0, 0, 0, 0, 4});
long l = bb.getLong();
System.out.println(l);
Result
4
And that is awesome to learn, but I just want to confirm something before going that route.
Say I have a previously read in byte array that is 8 bytes long.
byte[] oldByte = new byte[8];
then I do...
ByteBuffer bb = ByteBuffer.wrap(new byte[] {oldByte[2], oldByte[3]});
int intValue = bb.getInt();
Will that work/be read in the same manner as the previous example?
The documentation for the ByteBuffer class specifies that the getInt() method reads the next four bytes, so if you are only passing two bytes to the call to wrap, you will get a BufferUnderflowException:
Throws: BufferUnderflowException - If there are fewer than four bytes remaining in this buffer
I am using ByteBuffer APIs to convert an object into bytes. The object's class is as follows
public class Obj{
int a; // size 1 byte
int b; // size 4 bytes
int c; // size 4 bytes
}
Using ByteBuffer API, I have allocated an object
ByteBuffer bbf = ByteBuffer.allocate(9);
bbf.put((byte) this.getA());
bbf.putInt(this.getB());
bbf.putInt(this.getC());
byte[] msg = bbf.array();
I set the value of B as 100 but when I convert the byte array from offset 1 till length 4, I get a different integer value.
Any idea where is the problem?
thanks!
The code works as it should, and if you indeed select bytes with index 1,2,3,4 they will yield the value 100:
ByteBuffer bbf = ByteBuffer.allocate(9);
bbf.put((byte) 10);
bbf.putInt(100);
bbf.putInt(55);
byte[] msg = bbf.array();
byte[] from4to8 = Arrays.copyOfRange(msg, 1, 5);
ByteBuffer buf2 = ByteBuffer.wrap(from4to8);
System.out.println(buf2.getInt()); // Prints 100
Some notes:
Keep in mind that the endian may differ from system to system (so check the endian on both hosts if this is part of a protocol)
The array you get from the bbf.array() call is a backing array, i.e.:
Modifications to this buffer's content will cause the returned array's content to be modified, and vice versa.
As far as I can see, B is at offset 1