As part of a program I'm writing I need to consolidate two bytes into a long from an array of bytes.
So assuming this:
byte a[] = new byte[2]
a[0] = 0b1000111
a[1] = 0b1111000
how can I consolidate them such that
long b = 0b10001111111000
EDIT: The program will attempt to consolidate anywhere between 2 bytes and 100, just for reference.
java.nio can do that:
ByteBuffer.wrap(a).getShort()
BigInteger can do this: http://docs.oracle.com/javase/6/docs/api/java/math/BigInteger.html
BigInteger buffer = new BigInteger(yourByteArray);
long result = buffer.longValue();
Note that if your byte array has a value larger than a long (eg, you mentioned that you might go up to 100 bytes), toLong() will only return the value of the lower value bytes. BigInteger, however, will handle any arbitrary number of bytes.
In order to combine those two bytes you will only need 16 bits. A short will suffice:
short result = (short)(((a[0] & 0xff) << 8) | (a[1] & 0xff))
If for some reason, you need the long data type, you need only replace the cast with a long.
Related
What I want to do is take a decimal integer, convert it into hexadecimal, and then separate the bytes.
It's my understanding that ByteBuffer is the best way to do this. The integer will not exceed 65535, so the hex number is guaranteed to be 2 bytes. For an example, I have an integer of 40000 (hex value 9C40).
int n1 = 40000;
ByteBuffer b = ByteBuffer.allocate(2);
b.putInt(n1);
However, I get the following error when I run the program:
Exception in thread "main" java.nio.BufferOverflowException
What am I doing wrong? Shouldn't 9C40 be written into b (with b[0] = 9C and b[1] = 40)?
Also, once I get past this, if I want to convert the value stored in b[0] (which is 9C) to decimal (which is 156), would I just use the following code?
int n2 = b.get(0);
As you are working with a ByteBuffer, it stores an amount of x allocated bytes. Now you allocated 2 bytes and you try to store a datatype that has the size of 4 bytes. So the buffer will run out of bounds as the message said. If you want to store this data in a two byte sized buffer, you either use a short (16 bit - 2 bytes) or you allocate 4 bytes for your ByteBuffer.
With short:
ByteBuffer bb = ByteBuffer.allocate(2);
short myShort = (short) 40000;
bb.putShort(myShort);
System.out.println(String.format("%02X, %02X", bb.get(0), bb.get(1)));
With int:
ByteBuffer bb = ByteBuffer.allocate(4);
int myInt = 40000;
bb.putInt(myInt);
System.out.println(String.format("%02X, %02X", bb.get(2), bb.get(3)));
Output: 9C, 40
The data type you used to store the number 40000 is int, which requires 4 bytes of space. Yes I know the number won't exceed 65535 but the computer doesn't. You have to change it to an appropriate data type that can be stored in 2 bytes.
That data type, is short.
But there's another problem if you used short, you can't really store 40000 in short in Java is signed, so its max value is 32767.
So to store your 40000, you have to store -25536 instead in a short, because of overflow.
short n1 = (short)40000; // this will cause n1 to store -25536
ByteBuffer b = ByteBuffer.allocate(2);
b.putShort(n1);
Now it's time to print out the bytes. Bytes in Java are signed as well. So if you print this:
System.out.println(b.get(0));
System.out.println(b.get(1));
You'd get
-100
64
64 is expected since 64 in hex is 40, but why -100? Since bytes are signed, 156 can't be represented as 156. 156 in a signed byte is -100.
Instead of ByteBuffer I prefer Integer class which can convert the integer value to hex string & you can get each byte by index of method.
Use following code it do that
int n = 4000;
String hex = Integer.toHexString(n);
In this way you can get the hex value of any integer for one one byte use indexOf() method of string clas
You can get return the hex value as integer using valueOf() method in Integer class which takes two arguments one is string and another is radix
What's a nice, readable way of getting the byte representation (i.e. a byte[]) of an int, but only using 3 bytes (instead of 4)? I'm using Hadoop/Hbase and their Bytes utility class has a toBytes function but that will always use 4 bytes.
Ideally, I'd also like a nice, readable way of encoding to as few bytes as possible, i.e. if the number fits in one byte then only use one.
Please note that I'm storing this in a byte[], so I know the length of the array and thus variable length encoding is not necessary. This is about finding an elegant way to do the cast.
A general solution for this is impossible.
If it were possible, you could apply the function iteratively to obtain unlimited compression of data.
Your domain might have some constraints on the integers that allow them to be compressed to 24-bits. If there are such constraints, please explain them in the question.
A common variable size encoding is to use 7 bits of each byte for data, and the high bit as a flag to indicate when the current byte is the last.
You can predict the number of bytes needed to encode an int with a utility method on Integer:
int n = 4 - Integer.numberOfLeadingZeros(x) / 8;
byte[] enc = new byte[n];
while (n-- > 0)
enc[n] = (byte) ((x >>> (n * 8)) & 0xFF);
Note that this will encode 0 as an empty array, and other values in little-endian format. These aspects are easily modified with a few more operations.
If you need to represent the whole 2^32 existing 4-byte integers, you need to chose between:
fixed-size representation, using 4 bytes always; or
variable-size representation, using at least 5 bytes for some numbers.
Take a look on how UTF-8 encodes the Unicode charactes, you might get some insights. (you use some short prefix to describe how many bytes must be read for that unicode character, then you read that many bytes and interpret them).
Try using ByteBuffer. You can even set little endian mode if required:
int exampleInt = 0x11FFFFFF;
ByteBuffer buf = ByteBuffer.allocate(Integer.SIZE / Byte.SIZE);
final byte[] threeByteBuffer = new byte[3];
buf.putInt(exampleInt);
buf.position(1);
buf.get(threeByteBuffer);
Or the shortest signed, Big Endian:
BigInteger bi = BigInteger.valueOf(exampleInt);
final byte[] shortestSigned = bi.toByteArray();
Convert your int to a 4 bytes array, and iterate it, if every high order byte is zero then remove it from array.
Something like:
byte[] bytes = toBytes(myInt);
int neededBytes = 4;
for (;neededBytes > 1; i--) {
if (bytes[neededBytes - 1] != 0) {
break;
}
}
byte[] result = new byte[neededBytes];
// then just use array copy to copy first neededBytes to result.
You can start with something like this:
byte[] Convert(int i)
{ // warning: untested
if (i == 0)
return new byte[0];
if (i > 0 && i < 256)
return new byte[]{(byte)i};
if (i > 0 && i < 256 * 256)
return new byte[]{(byte)i, (byte)(i >> 8)};
if (i > 0 && i < 256 * 256 * 256)
return new byte[]{(byte)i, (byte)(i >> 8), (byte)(i >> 16)};
return new byte[]{(byte)i, (byte)(i >> 8), (byte)(i >> 16), (byte)(i >> 24)};
}
You'll need to decide if you want to be little-endian or big-endian. Note that negative numbers are encoded in 4 bytes.
If i understand right that you really, desperately want to save space, even at expense of arcane bit shuffling: any array type is an unecessary luxury because you cannot use less than one whole byte for the length = addressing space 256 while you know that at most 4 will be needed. So i would reserve 4 bits for the length and sign flag and cram the rest aligned to that number of bytes. You might even save one more byte if your MSB is less than 128. The sign flag i see useful for ability to represent negative numbers in less than 4 bytes too. Better have the bit there every time (even for positive numbers) than overhead of 4 bytes for representing -1.
Anyway, this all is a thin water until you make some statistics on your data set, how many integers are actually compressible and whether the compression overhead is worth the effort.
To convert an int into a byte array, I'm using the following code:
int a = 128;
byte[] b = convertIntValueToByteArray(a);
private static byte[] convertIntValueToByteArray(int intValue){
BigInteger bigInteger = BigInteger.valueOf(intValue);
byte[] origByteArray = bigInteger.toByteArray();
byte[] noSignByteArray = new byte[bigInteger.bitLength()/8];
if(bigInteger.bitLength()%8!=0){
noSignByteArray = origByteArray;
}else{
System.arraycopy(origByteArray,1,noSignByteArray,0,noSignByteArray.length);
}
return noSignByteArray;
}
There are two things which I'm attempting to do.
1)I need to know the number of bytes (rounded up to the closes byte) of the original integer. However, I don't need the additional bit that is added for the sign bit when I call the toByteArray() method. This is the reason why I have the helper method. So in this example, if I don't have the helper method, when I convert 128 to a byte array I get the length to be 2 octets because of the sign bit but I'm only expecting it to be one octet.
2)I need the positive representation of the number. In this example, if I attempt to print the first element in array b, I get -128. However, the numbers I will be using will be positive numbers only so what I actually want is 128. I'm limited to using a byte array. Is there a way to accomplish this?
Updated Post
Thank you for the responses. I haven't found the exact answer I was looking for so I'll attempt to give more details. Ultimately, I want to write values of different types over a data output stream. In this post, I'd like to clarify what happens when ints are written to a data output stream. I've come across two scenarios.
1)
DataOutputStream os = new DataOutputStream(this.socket.getOutputStream());
byte[] b = BigInteger.valueOf(128).toByteArray();
os.write(b);
2)
DataOutputStream os = new DataOutputStream(this.socket.getOutputStream());
os.write(128);
In the first scenario, when the bytes are read from a data input stream, it seems that the first element in the byte array is a 0 to represent the msb and the second element in the array contains the number -128. However, since the msb is 0 we would be able to determine that it is intended to be a positive number. In the second scenario, there is no msb and the only element present in the byte array read from the input stream is -128. I was expecting the write() method of the data output stream to convert the int into the byte array in the same manner as the toByteArray() method does on a BigInteger object. However, this doesn't seem to be the case as the msb is not present. So my question is, how in the second scenario are we supposed to know that 128 is supposed to be a positive number and not a negative one if there is no msb.
As you probably already know
In an octet, the pattern 10000000 can be interpreted as either 128 or -128, depending on the, um, outside interpretation
Java's byte type interprets octects as values in -128...127 only.
If you are building an application in which the entire world consists of nonnegative integers only, then you could simply do all of your work under the assumption that the byte value -128 will mean 128 and -127 will mean 129 and ... and -1 will mean 255. This is certainly doable but it takes work.
Dealing with the notion of an "unsigned byte" like this is normally done by expanding the byte into a short or int with the higher order bits all set to zero and then performing arithmetic or displaying your values. You will need to decide whether such an approach is more to your liking than just representing 128 as two octets in your array.
I think the following code might be sufficient.
In java int is a twos-complements binary number:
-1 = 111...111
ones complement = 000...000; + 1 =
1 = 000...001
So that about the sign bit I do not understand. Be it, that you could do Math.abs(n).
A byte ranges from -128 to 127, but the interpretation is a matter of masking, as below.
public static void main(String[] args) {
int n = 128;
byte[] bytes = intToFlexBytes(n);
for (byte b: bytes)
System.out.println("byte " + (((int)b) & 0xFF));
}
public static byte[] intToFlexBytes(int n) {
// Convert int to byte[4], via a ByteBuffer:
byte[] bytes = new byte[4];
ByteBuffer bb = ByteBuffer.allocateDirect(4);
bb.asIntBuffer().put(n);
bb.position(0);
bb.get(bytes);
// Leading bytes with 0:
int i = 0;
while (i < 4 && bytes[i] == 0)
++i;
// Shorten bytes array if needed:
if (i != 0) {
byte[] shortenedBytes = new byte[4 - i];
for (int j = i; j < 4; ++j) {
shortenedBytes[j - i] = bytes[j]; // System.arrayCopy not needed.
}
bytes = shortenedBytes;
}
return bytes;
}
To answer your first question—how many bytes are required to represent a nonnegative integer using an unsigned representation—consider the following functions I wrote in Common Lisp.
(defconstant +bits-per-byte+ 8)
(defun bit-length (n)
(check-type n (integer 0) "a nonnegative integer")
(if (zerop n)
1
(1+ (floor (log n 2)))))
(defun bytes-for-bits (n)
(check-type n (integer 1) "a positive integer")
(values (ceiling n +bits-per-byte+)))
These highlight the mathematical underpinnings of the problem: namely, the logarithm tells you how many powers of two (as provided by bits) it takes to dominate a given nonnegative integer, adjusted to be a step function with floor, and the number of bytes it takes to hold that number of bits again as a step function, this time adjusted with ceiling.
Note that the number zero is intolerable as input to a logarithm function, so we avoid it explicitly. You may observe that the bit-length function could also be written with a slight transformation of the core expression:
(defun bit-length-alt (n)
(check-type n (integer 0) "a nonnegative integer")
(values (ceiling (log (1+ n) 2))))
Unfortunately, as the logarithm of one is always zero, regardless of the base, this version says that the integer zero can be represented by zero bits, which isn't the answer we want.
For your second goal, you can use the functions I've defined above to allocate the required number of bytes, and incrementally set the bits you need, ignoring sign. It's hard to tell if you're having trouble getting the proper bits set in the byte vector, or whether your problem is in interpreting the bits in way that avoids treating the high bit as a sign bit (that is, two's complement representation). Please elaborate what kind of push you need to get you moving again.
I am really short on time for doing the learning of bitwise operations.
I want to convert large integer(>127) values without doing '<<' or anything similar.
I need byte representation of integer values used to identify sequence numbers of packets in header sent across UDP. If there is no solution I will introduce two bytes..
Something like: 1, 1 ; 1,2 ; 1,3 ; packet lost ; 1,4 ; packet lost; 2,1 ,2,2
and then reset it upon reaching 127; 127
I can introduce third, but this is rather ugly.
It would be really useful to have black box that is part of java api doing all that byte conversion for me. Is there?
Thanks,
To pack an unsigned 8-bit value into a byte:
static byte toByte(int i) {
if ((i < 0) || (i > 255))
throw new IllegalArgumentException(String.valueOf(i));
return (byte) i;
}
To convert back:
static int toInt(byte b) {
return (b < 0) ? (b + 256) : b;
}
After reading your comments on other answers, it sounds like you might want something like this:
byte[] b = BigInteger.valueOf(counter).toByteArray();
and
long counter = new BigInteger(b).longValue();
Since the length of the array would vary as the counter grows, you'd need some way to indicate its length or delimit it. But this technique will convert any integer value to an array of bytes.
Is the problem that you want unsigned bytes, as in, numbers between 128 and 255 inclusive?
That's...tricky. The Java language won't let you directly treat bytes as unsigned...but with library support it gets a little easier. Guava provides an UnsignedBytes utility class for some of these needs. Addition, multiplication, and subtraction are all exactly the same on signed and unsigned bytes.
EDIT: Judging from your additional comments, you might be interested in Ints.toByteArray(int) and the like, which work on types between byte and BigInteger.
According to my understanding, you want to separate an int into 4 bytes. If so, then just copy paste this code:
int i = /* your int */
int[] b = { (i >> 24) & 0xff, (i >> 16) & 0xff, (i >> 8) & 0xff, i & 0xff };
Indices 0-3 are each of the 4 bytes in the int.
I have a byte[] that I've read from a file, and I want to get an int from two bytes in it. Here's an example:
byte[] bytes = new byte[] {(byte)0x00, (byte)0x2F, (byte)0x01, (byte)0x10, (byte)0x6F};
int value = bytes.getInt(2,4); //This method doesn't exist
This should make value equal to 0x0110, or 272 in decimal. But obviously, byte[].getInt() doesn't exist. How can I accomplish this task?
The above array is just an example. Actual values are unknown to me.
You should just opt for the simple:
int val = ((bytes[2] & 0xff) << 8) | (bytes[3] & 0xff);
You could even write your own helper function getBytesAsWord (byte[] bytes, int start) to give you the functionality if you didn't want the calculations peppering your code but I think that would probably be overkill.
Try:
public static int getInt(byte[] arr, int off) {
return arr[off]<<8 &0xFF00 | arr[off+1]&0xFF;
} // end of getInt
Your question didn't indicate what the two args (2,4) meant. 2 and 4 don't make sense in your example as indices in the array to find ox01 and 0x10, I guessed you wanted to take two consecutive element, a common thing to do, so I used off and off+1 in my method.
You can't extend the byte[] class in java, so you can't have a method bytes.getInt, so I made a static method that uses the byte[] as the first arg.
The 'trick' to the method is that you bytes are 8 bit signed integers and values over 0x80 are negative and would be sign extended (ie 0xFFFFFF80 when used as an int). That is why the '&0xFF' masking is needed. the '<<8' shifts the more significant byte 8 bits left.
The '|' combines the two values -- just as '+' would. The order of the operators is important because << has highest precedence, followed by & followed by | -- thus no parentheses are needed.
Here's a nice simple reliable way.
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(4);
// by choosing big endian, high order bytes must be put
// to the buffer before low order bytes
byteBuffer.order(ByteOrder.BIG_ENDIAN);
// since ints are 4 bytes (32 bit), you need to put all 4, so put 0
// for the high order bytes
byteBuffer.put((byte)0x00);
byteBuffer.put((byte)0x00);
byteBuffer.put((byte)0x01);
byteBuffer.put((byte)0x10);
byteBuffer.flip();
int result = byteBuffer.getInt();
Alternatively, you could use:
int val = (bytes[2] << 8) + bytes[3]
You can use ByteBuffer. It has the getInt method you are searching for and many other useful methods
The Google Base16 class is from Guava-14.0.1.
new BigInteger(com.google.common.io.BaseEncoding.base16().encode(bytesParam),16).longValue();