I have an integer called writePos that takes a value between [0,1023]. I need to store it in the last two bytes of a byte array called bucket. So, I figure I need to represent it as a concatenation of the array's last two bytes.
How would I go about breaking down writePos into two bytes that, when concatenated and cast into an int, produces writePos again?
How would I go about concatenating once I get it broken down into the bytes?
This would be covered high-level by a ByteBuffer.
short loc = (short) writeLocation;
byte[] bucket = ...
int idex = bucket.length - 2;
ByteBuffer buf = ByteBuffer.wrap(bucket);
buf.order(ByteOrder.LITTLE__ENDIAN); // Optional
buf.putShort(index, loc);
writeLocation = buf.getShort(index);
The order can be specified, or left to the default (BIG_ENDIAN).
The ByteBuffer wraps the original byte array, and changes to ByteBuffer effect on the byte array too.
One can use sequential writing and reading an positioning (seek), but here I use overloaded methods for immediate positioning with index.
putShort writes to the byte array, modifying two bytes, a short.
getShort reads a short from the byte array, which can be put in an int.
Explanation
A short in java is a two-byte (signed) integral number. And that is what is meant. The order is whether LITTLE_ENDIAN: least significant byte first (n % 256, n / 256) or big endian.
Bitwise operations.
To byte:
byte[] bytes = new byte[2];
// This uses a bitwise and (&) to take only the last 8 bits of i
byte[0] = (byte)(i & 0xff);
// This uses a bitwise and (&) to take the 9th to 16th bits of i
// It then uses a right shift (>>) then move them right 8 bits
byte[1] = (byte)((i & 0xff00) >> 8);from byte:
To go back the other way
// This just reverses the shift, no need for masking.
// The & here is used to handle complications coming from the sign bit that
// will otherwise be moved as the bytes are combined together and converted
// into an int
i = (byte[0] & 0xFF)+(byte[1] & 0xFF)<<8;
There is a working example here of some of the conversions that you can play around with:
http://ideone.com/eRzsun
You need to split the integer into two bytes. The high and the low byte. Following your description it's stored as bug endian in the array.
int writeLocation = 511;
byte[] bucket = new byte[10];
// range checks must be done before
// bitwise right rotation by 8 bits
bucket[8] = (byte) (writeLocation >> 8); // the high byte
bucket[9] = (byte) (writeLocation & 0xFF); // the low byte
System.out.println("bytes = " + Arrays.toString(bucket));
// convert back the integer value 511 from the two bytes
bucket[8] = 1;
bucket[9] = (byte) (0xFF);
// the high byte will bit bitwise left rotated
// the low byte will be converted into an int
// and only the last 8 bits will be added
writeLocation = (bucket[8] << 8) + (((int) bucket[9]) & 0xFF);
System.out.println("writeLocation = " + writeLocation);
Related
I have 2 bytes in a byte array, and I'd like to "merge" them together so as to get byte 2's binary appended to byte 1's binary, and then get the decimal value of that binary.
byte 1: 01110110
byte 2: 10010010
combined gives 16 bits: 0111011010010010.
desired output: 30354
I am doing it like this right now, but wanted to know if there's a way that doesn't involve strings?
StringBuilder combined = new StringBuilder();
byte[] buffer = new byte[2];
buffer[0] = input[i];
buffer[1] = input[i + 1];
combined.append(Integer.toBinaryString(buffer[0] & 255 | 256).substring(1));
combined.append(Integer.toBinaryString(buffer[1] & 255 | 256).substring(1));
int key = Integer.parseInt(combined.toString(), 2);
Thanks!
To concatenate two bytes together efficiently, some bitwise arithmetic is required. However, to achieve the result that you want, we need to operate on the bytes as int values, as the result may be represented by more than 8 bits.
Consequently, we need to ensure that we're always working with the least significant 8 bits of each value (keep in mind that negative integers are represented using leading 1s in binary instead of 0s).
This can be accounted for by performing a bitwise-and of each value with 0xFF:
byte higher = (byte) 0b01110110;
byte lower = (byte) 0b10010010;
int concatenated = ((higher & 0xFF) << 8) | (lower & 0xFF);
System.out.println(concatenated);
As expected, the output of this snippet is:
30354
Now I understand java doesn't have unsigned bytes, but I'm not sure how to solve this if not.
I'm trying to implement SHA256 hashing in java, and i'm in the processing of converting the message to 512-bit.
int l = bytes.length; //total amount of bytes in the original message
int k = 0;
while((l+1+k) % 512 != 448) {
k++;
}
//k is the total amount of 0's to be padded
int rest = k % 8; //get the amount of 0's to be added in the byte with the 1
byte tmp =(byte) Math.pow(2, rest);
So the key instruction is the last row, if rest = 7 the resulting int is 128, but the bytes are signed in java and so the byte becomes 0x80 instead of 0xF0.
How can I achieve this in Java?
If anyone has a idea on how to implement this part please let me know.
Starting from the assumption your message consists of bytes, the padding always works out as mupltiple of 8 bits, aka bytes. This ensures the most significant pad bit is always located in bit 7 of the first padding byte following the message, thus the padding, if any, is always started by 0x80, follwed by as many 0x00 as needed.
This can be implemented in a very simple manner:
public static byte[] padMsg(byte[] rawMsg) {
int rawLen = rawMsg.length;
int padLen = (64 - (rawLen & 0x3F)) & 0x3F;
if (padLen == 0)
return rawMsg;
// all extra bytes in padded msg are zeros.
byte[] paddedMsg = Arrays.copyOf(rawMsg, rawLen + padLen);
// ensure topmost pad bit is a one
paddedMsg[rawLen] = (byte) 0x80;
return paddedMsg;
}
This takes the message length and gets the remainder. The remainder of a power of two (in this case 64), is most effectively gotten by simply and-masking with (power - 1), and this is where the 0x3F in the code comes from (= 64 - 1).
The remainder is taken again after calculating (64 - remainder) as prelimary padding length, to catch the special case where remainder is 0, leading to a wrong padding length of 64 bytes (which should be 0 padding).
Once the padding length in bytes is known, the case padding = 0 is caught. In any other case the message length is increased (with 0x00 bytes, Arrays.copyOf does this automatically). Then the first padding byte is replaced with 0x80 and the padded message that is now guaranteed to be a multiple of 64 bytes long is returned.
I'm developing an Android 2.3.3 application with Java.
This app is an iOS code with unsigned data types port to Java.
On iOS it works with UInt16 and UInt8. In one case instead using byte[] I'm using char[].
But know I have to send that char[] as a byte[] using a DatagramPacket.
If one element of char[] is 128, how can I do to insert into byte[] and the receiver gets 128. I don't know what happens if I do this:
char c = (char)128;
byte b = (byte)c;
Which will be b value?
128 = 10000000. b = -1 or b = 127?
How can I convert char[] to byte[] without losing any bits?
In Java char is an unsigned 16-bit quantity. So you can directly convert your uint16 to char without doing anything else.
For unsigned 8-bit quantity you have 2 options:
Use byte. It also holds 8 bits. You don't lose any bits just because it is signed. However, if you do arithmetic with it you need to remember that Java will scale byte up automatically to an int and sign-extend it. To prevent this just always mask it like this:
byte b;
int foo = 5 * (b & 0xFF);
Use char. It is unsigned and can hold 16 bits so the 8 bits will fit in there quite nicely. To put a byte into a char just do this:
byte b;
char c = (char)(b & 0xFF); // Mask off the low-order 8 bits
To put a char into a byte just do:
char c;
byte b = (byte)c; // Truncates to 8 bits
Be aware that byte in Java is signed, so that whenever you do arithmetic with it you need to mask the low-order 8 bits only (to prevent sign-extension). Like this:
byte b;
int foo = (b & 0xFF);
You can do all the normal bitwise operations you want with a byte without having to mask:
byte b;
if (b & 0x80) ... // Test a flag
b |= 0x40; // Set a flag
b ^= 0x20; // Flip a flag from 0 to 1 or 1 to 0
b ^= ~0x10; // Clear a flag
byte x = b << 3; // Shift left 3 bits and assign
byte x = b >>> 4; // Shift right 4 bits and assign (WITHOUT sign extension)
I think you need to rethink your approach so you don't end up needing to convert char[] to byte[].
If your data really is characters, then you want to look at various serialization techniques, such as using new String(char[]) to create a string and then using getBytes(Charset) to get the bytes as encoded by a given Charset (because, of course, the same characters result in different bytes when encoded in ASCII or UTF-8 or UTF-16, etc.).
But from your question, it sounds like you're not really using characters, you're just using char as a 16-bit type. If so, doing the conversion isn't difficult, something along these lines:
byte[] toBytes(char[] chars) {
byte[] bytes = new byte[chars.length * 2];
int ci, bi;
char ch;
bi = 0;
for (ci = 0; ci < chars.length; ++ci) {
ch = chars[ci];
bytes[bi++] = (byte)((ch & 0xFF00) >> 8);
bytes[bi++] = (byte)(ch & 0x00FF);
}
return bytes;
}
Reverse the masks if you want the result to be small-endian instead.
But again, I would look at your overall approach and try to avoid this.
What's a nice, readable way of getting the byte representation (i.e. a byte[]) of an int, but only using 3 bytes (instead of 4)? I'm using Hadoop/Hbase and their Bytes utility class has a toBytes function but that will always use 4 bytes.
Ideally, I'd also like a nice, readable way of encoding to as few bytes as possible, i.e. if the number fits in one byte then only use one.
Please note that I'm storing this in a byte[], so I know the length of the array and thus variable length encoding is not necessary. This is about finding an elegant way to do the cast.
A general solution for this is impossible.
If it were possible, you could apply the function iteratively to obtain unlimited compression of data.
Your domain might have some constraints on the integers that allow them to be compressed to 24-bits. If there are such constraints, please explain them in the question.
A common variable size encoding is to use 7 bits of each byte for data, and the high bit as a flag to indicate when the current byte is the last.
You can predict the number of bytes needed to encode an int with a utility method on Integer:
int n = 4 - Integer.numberOfLeadingZeros(x) / 8;
byte[] enc = new byte[n];
while (n-- > 0)
enc[n] = (byte) ((x >>> (n * 8)) & 0xFF);
Note that this will encode 0 as an empty array, and other values in little-endian format. These aspects are easily modified with a few more operations.
If you need to represent the whole 2^32 existing 4-byte integers, you need to chose between:
fixed-size representation, using 4 bytes always; or
variable-size representation, using at least 5 bytes for some numbers.
Take a look on how UTF-8 encodes the Unicode charactes, you might get some insights. (you use some short prefix to describe how many bytes must be read for that unicode character, then you read that many bytes and interpret them).
Try using ByteBuffer. You can even set little endian mode if required:
int exampleInt = 0x11FFFFFF;
ByteBuffer buf = ByteBuffer.allocate(Integer.SIZE / Byte.SIZE);
final byte[] threeByteBuffer = new byte[3];
buf.putInt(exampleInt);
buf.position(1);
buf.get(threeByteBuffer);
Or the shortest signed, Big Endian:
BigInteger bi = BigInteger.valueOf(exampleInt);
final byte[] shortestSigned = bi.toByteArray();
Convert your int to a 4 bytes array, and iterate it, if every high order byte is zero then remove it from array.
Something like:
byte[] bytes = toBytes(myInt);
int neededBytes = 4;
for (;neededBytes > 1; i--) {
if (bytes[neededBytes - 1] != 0) {
break;
}
}
byte[] result = new byte[neededBytes];
// then just use array copy to copy first neededBytes to result.
You can start with something like this:
byte[] Convert(int i)
{ // warning: untested
if (i == 0)
return new byte[0];
if (i > 0 && i < 256)
return new byte[]{(byte)i};
if (i > 0 && i < 256 * 256)
return new byte[]{(byte)i, (byte)(i >> 8)};
if (i > 0 && i < 256 * 256 * 256)
return new byte[]{(byte)i, (byte)(i >> 8), (byte)(i >> 16)};
return new byte[]{(byte)i, (byte)(i >> 8), (byte)(i >> 16), (byte)(i >> 24)};
}
You'll need to decide if you want to be little-endian or big-endian. Note that negative numbers are encoded in 4 bytes.
If i understand right that you really, desperately want to save space, even at expense of arcane bit shuffling: any array type is an unecessary luxury because you cannot use less than one whole byte for the length = addressing space 256 while you know that at most 4 will be needed. So i would reserve 4 bits for the length and sign flag and cram the rest aligned to that number of bytes. You might even save one more byte if your MSB is less than 128. The sign flag i see useful for ability to represent negative numbers in less than 4 bytes too. Better have the bit there every time (even for positive numbers) than overhead of 4 bytes for representing -1.
Anyway, this all is a thin water until you make some statistics on your data set, how many integers are actually compressible and whether the compression overhead is worth the effort.
I need to extract two integer values from a byte stored within a ByteBuffer (little endian order)
ByteBuffer bb = ByteBuffer.wrap(inputBuffer);
bb.order(ByteOrder.LITTLE_ENDIAN);
The values I need to obtain from any byte within the ByteBuffer are:
length = integer value of low order nibble
frequency = integer value of high order nibble
At the moment I'm extracting the low order nybble with this code:
length = bb.getInt(index) & 0xf;
Which seems to work perfectly well. It is however the high order nybble that I seem to be having trouble interpreting correctly.
I get a bit confused with bit shifting or masking, which I think I need to perform, and any advice would be super helpful.
Thanks muchly!!
I need to extract two integer values from a byte
So you need to get a byte not an int, and the byte order doesn't matter.
int lowNibble = bb.get(index) & 0x0f; // the lowest 4 bits
int hiNibble = (bb.get(index) >> 4) & 0x0f; // the highest 4 bits.
To get the high order nibble, all you need to do is bit shift; the low order bits will simply fall off.
int val = 0xAB;
int lo = val & 0xF;
int hi = val >> 4;
System.out.println("hi is " + Integer.toString(hi, 16));
System.out.println("lo is " + Integer.toString(lo, 16));