I have this code
byte[] b = new byte[]{-33,-4,20,30};
System.err.println(Arrays.toString(b));
int x = (b[0] << 24) + (b[1] << 16) + (b[2] << 8) + b[3];
b = new byte[]{(byte)(x >> 24), (byte)(x >> 16), (byte)(x >> 8), (byte)(x)};
System.err.println(Arrays.toString(b));
Output:
[-33, -4, 20, 30]
[-34, -4, 20, 30]
I cannot figure out why this operations are not invers.
Your problem is unwanted sign extension.
Specifically, b[1] is -4, or 0xfc which is sign-extended to 0xfffffffc then left-shifted to 0xfffc0000. This has the effect of decrementing the most-significant byte by 1.
Try:
int x = ((b[0] & 0xff) << 24) +
((b[1] & 0xff) << 16) +
((b[2] & 0xff) << 8) +
(b[3] & 0xff);
Please disregard my previous answer; it's totally wrong.
I think that the problem here is that when you compose the bits in this manner:
(b[0] << 24) + (b[1] << 16) + (b[2] << 8) + b[3]
You are not doing what you think you're doing. In particular, suppose that b[1] is negative (which it is). When Java does bitshifts, it always promotes the value to an int before doing the shift. This means that b[1] will look like this when it gets promoted:
11111111 11111111 11111111 bbbbbbbb
Here, the leading 1s are from the signed two's-complement representation of integers, which makes negative numbers represented by a lot of leading zeros. When you shift this number up, you get
11111111 bbbbbbbb 00000000 00000000
If you then add these bits to (b[0] << 24), which has the form
aaaaaaaa 00000000 00000000 00000000
You do not get
aaaaaaaa bbbbbbbb 00000000 00000000
Because of the leading 1s in the representation. To fix this, you need to mask out the sign bits before doing the addition. Specifically, if you change
b[1] << 16
to
(b[1] << 16) & 0x00FFFFFF
Then you mask out the bits to get
00000000 bbbbbbbb 00000000 00000000
So now when you add the two values, you get
aaaaaaaa bbbbbbbb 00000000 00000000
As desired.
The correct expression for composing bits is thus formed by ANDing in the appropriate masks at the appropriate times:
(b[0] << 24) + ((b[1] << 16) & 0x00FFFFFF) + ((b[2] << 8) & 0x0000FFFF) + (b[3] & 0x000000FF)
I've tested this on my system and it seems to work just fine.
Hope this helps!
Related
I am attempting to convert 16 bit audio into 12 bit audio. However, I am quite inexperienced with such conversions and believe my approach is possibly incorrect or flawed.
The use case, as context for the code snippets below, is an Android app which the user can speak into and that audio is transmitted to an IoT device for immediate playback. The IoT device expects audio in mono 12 bit, 8k sample rate, little endian, unsigned, with the data stored in the first twelve bits (0-11) and final four bits (12-15) are zeroes. Audio data needs to be received in packets of 1000 bytes.
The audio is being created in the Android app through the use of AudioRecord. The instantiation of which is as follows:
int bufferSize = 1000;
this.audioRecord = new AudioRecord(
MediaRecorder.AudioSource.MIC,
8000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT,
bufferSize
);
In a while loop, the AudioRecord is being read from by 1000 byte packets and modified to the specifications in the use case. Not sure this part is relevant, but for completeness:
byte[] buffer = new byte[1000];
audioRecord.read(buffer, 0, buffer.length);
byte[] modifiedBytes = convert16BitTo12Bit(buffer);
Then the modifiedBytes are sent off to the device.
Here are the methods which modify the bytes. Basically, to conform to the specifications, I am shifting the bits in each 16 bit set (tossing the least significant 4) and adding zeroes to the final four spots. I do this through BitSet.
/**
* Takes a byte array presented as 16 bit audio and converts it to 12 bit audio through bit
* manipulation. Packets must be of 1000 bytes or no manipulation will occur and the input
* will be immediately returned.
*/
private byte[] convert16BitTo12Bit(byte[] input) {
if (input.length == 1000) {
for (int i = 0; i < input.length; i += 2) {
Log.d(TAG, "convert16BitTo12Bit: pass #" + (i / 2));
byte[] chunk = new byte[2];
System.arraycopy(input, i, chunk, 0, 2);
if (!isEmptyByteArray(chunk)) {
byte[] modifiedBytes = convertChunk(chunk);
System.arraycopy(
modifiedBytes,
0,
input,
i,
modifiedBytes.length
);
}
}
return input;
}
Log.d(TAG, "convert16BitTo12Bit: Failed - input is not 1000 in length; it is " + input.length);
return input;
}
/**
* Converts 2 bytes 16 bit audio into 12 bit audio. If the input is not 2 bytes, the input
* will be returned without manipulation.
*/
private byte[] convertChunk(byte[] chunk) {
if (chunk.length == 2) {
BitSet bitSet = BitSet.valueOf(chunk);
Log.d(TAG, "convertChunk: bitSet starts as " + bitSet.toString());
modifyBitSet(bitSet);
Log.d(TAG, "convertChunk: bitSet ends as " + bitSet.toString());
return bitSet.toByteArray();
}
Log.d(TAG, "convertChunk: Failed = chunk is not 2 in length; it is " + chunk.length);
return chunk;
}
/**
* Removes the first four bits and shifts the rest to leave the final four bits as 0.
*/
private void modifyBitSet(BitSet bitSet) {
for (int i = 4; i < bitSet.length(); i++) {
bitSet.set(i - 4, bitSet.get(i));
}
if (bitSet.length() > 8) {
bitSet.clear(12, 16);
} else {
bitSet.clear(4, 8);
}
}
/**
* Returns true if the byte array input contains all zero bits.
*/
private boolean isEmptyByteArray(byte[] input) {
BitSet bitSet = BitSet.valueOf(input);
return bitSet.isEmpty();
}
Unfortunately, this approach produces subpar results. The audio is quite noisy and it is difficult to make out what someone is saying (but you can hear that words are being spoken).
I also have been playing around with just saving the bytes to a file and playing it back on Android through AudioTrack. I noticed that if I just remove the first four bits and do not shift anything, the audio actually sounds good, as such:
private void modifyBitSet(BitSet bitSet) {
bitSet.clear(0, 4);
}
However, when played through the device, it sounds even worse, and I don't even think I can make out any words.
Clearly, my approach is not working here. Central question is how would one convert a 16 bit chunk into 12 bit audio and maintain audio quality given the requirement that the final four bits must be zero? Additionally, given my larger approach of using AudioRecord to obtain the audio, would such a solution for the prior question fit this use case?
Please let me know if there is anything more I can provide to clarify these questions and my intent.
Given that the audio is 16 bits but must be changed to 12 with four zeros at the end, four bits somewhere do have to be tossed.
Yes, of course and there is no other way, is there?
This is something quick that I can comeout with right now. Certainly not fully tested though. Only tested with input of 2 and 4 bytes. I'll leave it to you to test it.
//Reminder :: Convert as many as possible.
//Reminder :: To calculate the required size for store:
//if((bytes.length & 1) == 0) Math.round((bytes.length * 6) / 8F) : Math.round(((bytes.length - 1) * 6) / 8F).
//Return :: Amount of converted bytes.
public static final int convert16BitTo12Bit(final byte[] bytes, final byte[] store)
{
final int size = bytes.length;
int storeIndex = 0;
//Copy the first 2 bytes into store.
store[storeIndex++] = bytes[0]; store[storeIndex] = bytes[1];
if(size < 4) {
store[storeIndex] = (byte)(store[storeIndex] & 0xF0);
return 2;
}
final int result;
final byte tmp;
// 11111111 11110000 00000000 00000000
//+ 11111111 11110000 (<< 12)
//= 11111111 11111111 11111111 00000000 (1)
//-----------------------------------------
// 11111111 00000000 00000000 00000000 (1)
//+ 11111111 11110000 (<< 16)
//= 11111111 11111111 11110000 00000000 (2)
//-----------------------------------------
// 11110000 00000000 00000000 00000000 (2)
//+ 1111 11111111 0000 (<< 20)
//= 11111111 11111111 00000000 00000000 (3)
//-----------------------------------------
// 00000000 00000000 00000000 00000000 (3)
//+ 11111111 11110000 (<< 24)
//= 11111111 11110000 00000000 00000000
for(int i=2, shiftBits = 12; i < size; i += 2) {
if(shiftBits == 24) {
//Copy 2 bytes from bytes[] into store[] and move on.
store[storeIndex] = bytes[i];
//Never store byte 0 (Garbage).
tmp = (byte)(bytes[i + 1] & 0xF0); //Bit order: 11110000.
if(tmp != 0) store[++storeIndex] = tmp;
shiftBits = 12; //Reset
} else if(shiftBits == 20) {
result = ((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
| (((bytes[i] & 0xFF) << 20) | ((bytes[i + 1] & 0xFF) << 12));
store[storeIndex] = (byte)((result >> 24) & 0xFF);
tmp = (byte)((result >> 16) & 0xFF);
//Never store byte 0 (Garbage).
if(tmp != 0) store[++storeIndex] = tmp;
shiftBits = 24;
} else if(shiftBits == 16) {
result = ((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
| (((bytes[i] & 0xFF) << 16) | ((bytes[i + 1] & 0xFF) << 8));
store[storeIndex] = (byte)((result >> 16) & 0xFF);
tmp = (byte)((result >> 8) & 0xF0);
//Never store byte 0 (Garbage).
if(tmp != 0) store[++storeIndex] = tmp;
shiftBits = 20;
} else {
result = ((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
| (((bytes[i] & 0xFF) << 12) | ((bytes[i + 1] & 0xFF) << 4));
store[storeIndex] = (byte)((result >> 16) & 0xFF);
tmp = (byte)((result >> 8) & 0xFF);
//Never store byte 0 (Garbage).
if(tmp != 0) store[++storeIndex] = tmp;
shiftBits = 16;
}
}
return ++storeIndex;
}
Explanations
result = ((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
| (((bytes[i] & 0xFF) << 20) | ((bytes[i + 1] & 0xFF) << 12));
What this does is basically merge two integers into one.
((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
The first one is make an integer with same constant bit position.
(((bytes[i] & 0xFF) << 20) | ((bytes[i + 1] & 0xFF) << 12));
The latter is for 2 current bytes with different bit positions.
(...) | (...)
Pipe or vertical bar at the middle is to merge these two integers we've just created into one.
Usage
To use this method is pretty straight forward.
byte[] buffer = new byte[1000];
byte[] store;
if((buffer.length & 1) == 0) { //Even.
store = new byte[Math.round((bytes.length * 6) / 8F)];
} else { //Odd.
store = new byte[Math.round(((bytes.length - 1) * 6) / 8F)];
}
audioRecord.read(buffer, 0, buffer.length);
int convertedByteSize = convert16BitTo12Bit(buffer, store);
System.out.println("size: " + convertedByteSize);
I have discovered a solution that produces clear audio. First, it is important to recount the requirements for the use case, which is 12 bit unsigned mono audio which will be read in little endian by the device in packets of 1000 bytes.
The initialization and configuration of the AudioRecord as described in the question is fine.
Once the 1000 bytes of audio is read from AudioRecord, it can be put into a ByteBuffer and defined as little endian for modification, and then put into a ShortBuffer to do manipulation on the 16 bit level:
// Audio specifications of device is in little endian.
ByteBuffer byteBuffer = ByteBuffer.wrap(input).order(ByteOrder.LITTLE_ENDIAN);
// Turn into a ShortBuffer so bitwise manipulation can occur on the 16 bit level.
ShortBuffer shortBuffer = byteBuffer.asShortBuffer();
Next, in a loop, take each short and modify it to 12 bit unsigned:
for (int i = 0; i < shortBuffer.capacity(); i++) {
short currentShort = shortBuffer.get(i);
shortBuffer.put(i, convertShortTo12Bit(currentShort));
}
This can be accomplished by shifting the 16 bits four spaces to the right to turn it into 12 bit signed. Then, to convert to unsigned, add 2048. For our purposes as a safety step, we also mask the least significant four bits as required by device, but given the shifting and adding, it shouldn't be the case that any bits actually remain there:
private static short convertShortTo12Bit(short input) {
int inputAsInt = input;
inputAsInt >>>= 4;
inputAsInt += 2048;
input = (short) (inputAsInt & 0B0000111111111111);
return input;
}
If one wishes to return 12 bits to 16 bits, do the reverse for each short (subtract 2048 and shift four spaces to the left).
Java Integer class has the static method highestOneBit method which will return a value with a single one-bit, in the position of the highest-order one-bit in the specified value, or zero if the specified value is itself equal to zero.
For example input of int 17 will return 16; As 17 can be represented in binary as 10001 so it will return the furthest bit left which is equal to 16.
And in Integer class it has the following implementation in Java doc.
public static int highestOneBit(int i) {
// HD, Figure 3-1
i |= (i >> 1);
i |= (i >> 2);
i |= (i >> 4);
i |= (i >> 8);
i |= (i >> 16);
return i - (i >>> 1);
}
I just want to know the logic behind implementing it this way and the logic behind using shift operations
. Can any one put some light on it.
This algorithm calculates for a given i whose binary representation is:
0..01XXXXXXX...XXXX
the value
0..011111111...1111
That's what the 5 |= operators do.
Then, in the return statement, it subtracts from it that value shifted right by one bit
0..001111111...1111
to get the result
0..010000000...0000
How does it work:
The highest possible 1 bit the the 32nd (left most) bit. Suppose the input number has 1 in that bit:
1XXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
You or that value with the value shifted right by 1 (i >> 1) and get
11XXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
Then you or that new value with the value shifted right by 2 (i >> 2) and get
1111XXXX XXXXXXXX XXXXXXXX XXXXXXXX
Then you or that new value with the value shifted right by 4 (i >> 4) and get
11111111 XXXXXXXX XXXXXXXX XXXXXXXX
Then you or that new value with the value shifted right by 8 (i >> 8) and get
11111111 11111111 XXXXXXXX XXXXXXXX
Finally you or that new value with the value shifted right by 16 (i >> 16) and get
11111111 11111111 11111111 11111111
If the highest 1 bit is smaller than the 32nd bit, these operations still turn all the bits to the right of it to 1 and keep the remaining (higher bits) 0.
The i |= statements help to compute a sequence of ones that is the same length as i. For example, for 101011 it computes 111111. I've explained how it works in this answer (I can't retype it right now since I am on mobile).
So basically, once you have the string of ones, subtracting itself shifted right one bit gives the H.O. bit.
111111 - (111111 >>> 1) = 111111 - 011111 = 100000
The first five lines (i |= (i >> x)) will set every bit below the highest 1-bit to 1. Then, the final line will subtract every 1-bit below the highest one, so that only the highest 1-bit will remain.
For simplicity, let's pretend an int was 8 bits. The code would in that case be like this:
public static int highestOneBit(int i) {
i |= (i >> 1);
i |= (i >> 2);
i |= (i >> 4);
return i - (i >>> 1);
}
Now, let's say we have the value 128 (10000000). This is what would happen:
// i == 10000000
i |= (i >> 1); // i = 10000000 | 11000000 = 11000000
i |= (i >> 2); // i = 11000000 | 11110000 = 11110000
i |= (i >> 4); // i = 11110000 | 11111111 = 11111111
return i - (i >>> 1); // 11111111 - 01111111 = 10000000
The >> is an arithmetic right shift, so it will preserve the signed bit.
The last >>> is a logical right shift, which will not preserve the signed bit. It will always insert zeroes on the left side.
Now, let's try with 64 (01000000):
// i == 01000000
i |= (i >> 1); // i = 01000000 | 00100000 = 01100000
i |= (i >> 2); // i = 01100000 | 00011000 = 01111000
i |= (i >> 4); // i = 01111000 | 00000111 = 01111111
return i - (i >>> 1); // 01111111 - 00111111 = 01000000
I'm having trouble making sense of how Java promotes bytes to ints with bitwise operations. I'm attempting to implement AES, and while my output is correct as a 2d byte array, I ultimately need to store it in a 1d int array. However, the following code changes some of the expected values
ciphertexts[0] = ((state[0][0] & 0xFF) << 24) ^ ((state[1][0] & 0xFF) << 16)
^ ((state[2][0] & 0xFF) << 8) ^ state[3][0];
ciphertexts[1] = ((state[0][1] & 0xFF) << 24) ^ ((state[1][1] & 0xFF) << 16)
^ ((state[2][1] & 0xFF) << 8) ^ state[3][1];
ciphertexts[2] = ((state[0][2] & 0xFF) << 24) ^ ((state[1][2] & 0xFF) << 16)
^ ((state[2][2] & 0xFF) << 8) ^ state[3][2];
ciphertexts[3] = ((state[0][3] & 0xFF) << 24) ^ ((state[1][3] & 0xFF) << 16)
^ ((state[2][3] & 0xFF) << 8) ^ state[3][3];
I didn't particularly expect masking with 0xFF to help, since the mask should just return the original byte value, but then I tried this:
int zero = ((state[0][0] & 0xFF) << 24);
int one = ((state[0][1] & 0xFF) << 16);
int two = ((state[0][2] & 0xFF) << 8) ;
int three = (state[0][3] & 0xFF);
int total = zero ^ one ^ two ^ three;
printhex(zero);
printhex(one);
printhex(two);
printhex(three);
printhex(total);
Which gives the following output:
69000000
006A0000
0000D800
00000070
696AD870
Which is what I'm trying to do with the code above. Without the masking, the following code gives the following output:
int zero = (state[0][0] << 24);
int one = (state[0][1] << 16);
int two = (state[0][2] << 8);
int three = state[0][3];
int total = zero ^ one ^ two ^ three;
69000000
006A0000
FFFFD800
00000070
9695D870
I also tried what seemed to me more sensible, which is masking after shifting, and got similarly messed up output:
ciphertexts[0] = ((state[0][0] << 24) & 0xFFFFFFFF) ^
((state[1][0] << 16) & 0xFFFFFF) ^ ((state[2][0] << 8) & 0xFFFF)
^ state[3][0];
ciphertexts[1] = ((state[0][1] << 24) & 0xFFFFFFFF) ^
((state[1][1] << 16) & 0xFFFFFF) ^ ((state[2][1] << 8) & 0xFFFF)
^ state[3][1];
ciphertexts[2] = ((state[0][2] << 24) & 0xFFFFFFFF) ^
((state[1][2] << 16) & 0xFFFFFF) ^ ((state[2][2] << 8) & 0xFFFF)
^ state[3][2];
ciphertexts[3] = ((state[0][3] << 24) & 0xFFFFFFFF) ^
((state[1][3] << 16) & 0xFFFFFF) ^ ((state[2][3] << 8) & 0xFFFF)
^ state[3][3];
Where "messed up" means:
ciphertext at round 9 is 963b1fd86a7b04302732488070b4c55a
instead of:
69C4E0D86A7B0430D8CDB78070B4C55A
So my questions are how do I neatly or bytes together into an int, and what is actually going on with the masking and shifting. I looked at other answers and can't figure out why they're not working in this case. Thanks!
That´s the cruelty of a language lacking unsigned
(one can get the same result in C if he/she use a signed char, ie. signed byte)
Let´s ignore shift´s , only concentrate at the assignment and &.
Example value here 0xfe instead of 0xd8
(the problem will happen with each value between 0x80 and 0xff)
With problem, java:
byte a = 0xfe;
int i = a;
With problem, C:
signed char a = 0xfe;
int i = a;
What does happen: A byte can hold value between -128 and +127.
0xfe maps to a negative number (2-complement): -2
...and so, i get the value -2 in i, and i is not 8bit, but 32bit long.
According to the rules of the 2-complement, this gives 0xfffffffe
(http://en.wikipedia.org/wiki/Two%27s_complement)
So, what does & change, because masking 0xfe first with 0xff
shouldn´t change the value?
Yes, but: As & is a "calculation" like + - ...
the value gets expanded first to 32bit
(because more suited for the processor´s ALU)
That´s more likely to be known by C/Asm programmers,
but as you see, it´s relevant in Java too.
(if nessecary for an assignment to an smaller variable than 32bit,
it will be shortened again after calculation)
Ie. first, -2=0xfe becomes 32bit -2=0xfffffffe,
then masking results in a 0xfe again (already 32bit)...
which is assigned to i.
Your value of state[0][2] is a byte 0xD8. This has the most significant bit set to 1: in binary: 1101 1000. Before the shift operation << is applied, the byte is converted to an int. Java doesn't care that byte is unsigned, it is treated as a signed byte. So the byte's most significant bit is filled all the way to the int's most significant bit.
In short: With bytes you need the mask with 0xFF as this masks the filled in bits away in the already converted int.
The task is to fetch each byte from a given integer. This is the approach I saw somewhere:
byte[] bytes = new byte[4];
bytes[0] = (byte) ((id >> 24) & 0xff);
bytes[1] = (byte) ((id >> 16) & 0xff);
bytes[2] = (byte) ((id >> 8) & 0xff);
bytes[3] = (byte) (id & 0xff);
It would result in the same break-up as this:
bytes[0] = (byte) (id >>> 24);
bytes[1] = (byte) (id >>> 16);
bytes[2] = (byte) (id >>> 8);
bytes[3] = (byte) (id);
where, id is an integer value and will ALWAYS be unsigned. In fact, I don't see the need to AND with 0xff in the first approach (isn't it? since we're always using the least significant byte).
Is there any difference in the two approaches and which one is preferred?
You do not need the & 0xff in the upper example either, because your example always chops off the bits that are different in sign-extended vs. non-sign-extended numbers.
Here is why: when you shift a number right by n bits using >>, the upper n bits will get the same value as the most significant bit of the number being shifted. The behavior of >>> differs only in that >>> forces the upper n bits to zero. The lower (32-n) bits are the same regardless of the kind of the shift that you use.
None of your examples shifts by more 24 bits, so the lower eight bits would be the same if you replace >>> with >> in your bottom example.
Since it is entirely unnecessary to mask with 0xff, I would use your second snippet using >> or >>> for the operator, because the code is shorter.
I want to switch the two hexadecimals symbols in a byte, for example if
input = 0xEA
then
output = 0xAE
It has to be in java.
I already have this method I made, but it only works in some cases:
public static final byte convert(byte in){
byte hex1 = (byte) (in << 4);
byte hex2 = (byte) (in >>> 4);
return (byte) (hex1 | hex2);
}
A working example is:
input: 0x3A
hex1: 0xA0
hex2: 0x03
output: 0xA3
A not working example is:
input: 0xEA
hex1: 0xA0
hex2: 0xFE
output: 0xFE
Anyone can shed some lights on why this is not working?
I suspect the problem is the sign extension. Specifically, you probably need to do
byte hex2 = (byte) ((in >>> 4) & 0xF);
try
byte hex1 = (byte) (in << 4);
byte hex2 = (byte) ( in >>> 4);
return (byte) (hex1 | hex2 & 0x0F);
this is like in a known puzzle
byte x = (byte)0xFF;
x = (byte) (x >>> 1);
System.out.println(x);
prints -1 because before unsigned shift 0xFF is promoted to int -> 0xFFFFFFFF; after shift it is 0x7FFFFFFF; cast to byte -> 0xFF
but
byte x = (byte)0xFF;
x = (byte) ((x & 0xFF) >>> 1);
System.out.println(x);
prints 127 because we truncated 0xFFFFFFFF -> 0x000000FF, now shift produces 0x0000007F, cast to byte -> 0x7F
Actually, this promotion is done at compile time. JVM works only with 4 or 8 bytes operands (local variables on stack). Even boolean in bytecode is 0 or 1 int.