We Keep Coding

How to convert 16-bit audio created with Android's AudioRecord to 12-bit audio through bit shifting?

I am attempting to convert 16 bit audio into 12 bit audio. However, I am quite inexperienced with such conversions and believe my approach is possibly incorrect or flawed.
The use case, as context for the code snippets below, is an Android app which the user can speak into and that audio is transmitted to an IoT device for immediate playback. The IoT device expects audio in mono 12 bit, 8k sample rate, little endian, unsigned, with the data stored in the first twelve bits (0-11) and final four bits (12-15) are zeroes. Audio data needs to be received in packets of 1000 bytes.
The audio is being created in the Android app through the use of AudioRecord. The instantiation of which is as follows:
int bufferSize = 1000;
this.audioRecord = new AudioRecord(
MediaRecorder.AudioSource.MIC,
8000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT,
bufferSize
);
In a while loop, the AudioRecord is being read from by 1000 byte packets and modified to the specifications in the use case. Not sure this part is relevant, but for completeness:
byte[] buffer = new byte[1000];
audioRecord.read(buffer, 0, buffer.length);
byte[] modifiedBytes = convert16BitTo12Bit(buffer);
Then the modifiedBytes are sent off to the device.
Here are the methods which modify the bytes. Basically, to conform to the specifications, I am shifting the bits in each 16 bit set (tossing the least significant 4) and adding zeroes to the final four spots. I do this through BitSet.
/**
* Takes a byte array presented as 16 bit audio and converts it to 12 bit audio through bit
* manipulation. Packets must be of 1000 bytes or no manipulation will occur and the input
* will be immediately returned.
*/
private byte[] convert16BitTo12Bit(byte[] input) {
if (input.length == 1000) {
for (int i = 0; i < input.length; i += 2) {
Log.d(TAG, "convert16BitTo12Bit: pass #" + (i / 2));
byte[] chunk = new byte[2];
System.arraycopy(input, i, chunk, 0, 2);
if (!isEmptyByteArray(chunk)) {
byte[] modifiedBytes = convertChunk(chunk);
System.arraycopy(
modifiedBytes,
0,
input,
i,
modifiedBytes.length
);
}
}
return input;
}
Log.d(TAG, "convert16BitTo12Bit: Failed - input is not 1000 in length; it is " + input.length);
return input;
}
/**
* Converts 2 bytes 16 bit audio into 12 bit audio. If the input is not 2 bytes, the input
* will be returned without manipulation.
*/
private byte[] convertChunk(byte[] chunk) {
if (chunk.length == 2) {
BitSet bitSet = BitSet.valueOf(chunk);
Log.d(TAG, "convertChunk: bitSet starts as " + bitSet.toString());
modifyBitSet(bitSet);
Log.d(TAG, "convertChunk: bitSet ends as " + bitSet.toString());
return bitSet.toByteArray();
}
Log.d(TAG, "convertChunk: Failed = chunk is not 2 in length; it is " + chunk.length);
return chunk;
}
/**
* Removes the first four bits and shifts the rest to leave the final four bits as 0.
*/
private void modifyBitSet(BitSet bitSet) {
for (int i = 4; i < bitSet.length(); i++) {
bitSet.set(i - 4, bitSet.get(i));
}
if (bitSet.length() > 8) {
bitSet.clear(12, 16);
} else {
bitSet.clear(4, 8);
}
}
/**
* Returns true if the byte array input contains all zero bits.
*/
private boolean isEmptyByteArray(byte[] input) {
BitSet bitSet = BitSet.valueOf(input);
return bitSet.isEmpty();
}
Unfortunately, this approach produces subpar results. The audio is quite noisy and it is difficult to make out what someone is saying (but you can hear that words are being spoken).
I also have been playing around with just saving the bytes to a file and playing it back on Android through AudioTrack. I noticed that if I just remove the first four bits and do not shift anything, the audio actually sounds good, as such:
private void modifyBitSet(BitSet bitSet) {
bitSet.clear(0, 4);
}
However, when played through the device, it sounds even worse, and I don't even think I can make out any words.
Clearly, my approach is not working here. Central question is how would one convert a 16 bit chunk into 12 bit audio and maintain audio quality given the requirement that the final four bits must be zero? Additionally, given my larger approach of using AudioRecord to obtain the audio, would such a solution for the prior question fit this use case?
Please let me know if there is anything more I can provide to clarify these questions and my intent.

Given that the audio is 16 bits but must be changed to 12 with four zeros at the end, four bits somewhere do have to be tossed.
Yes, of course and there is no other way, is there?
This is something quick that I can comeout with right now. Certainly not fully tested though. Only tested with input of 2 and 4 bytes. I'll leave it to you to test it.
//Reminder :: Convert as many as possible.
//Reminder :: To calculate the required size for store:
//if((bytes.length & 1) == 0) Math.round((bytes.length * 6) / 8F) : Math.round(((bytes.length - 1) * 6) / 8F).
//Return :: Amount of converted bytes.
public static final int convert16BitTo12Bit(final byte[] bytes, final byte[] store)
{
final int size = bytes.length;
int storeIndex = 0;
//Copy the first 2 bytes into store.
store[storeIndex++] = bytes[0]; store[storeIndex] = bytes[1];
if(size < 4) {
store[storeIndex] = (byte)(store[storeIndex] & 0xF0);
return 2;
}
final int result;
final byte tmp;
// 11111111 11110000 00000000 00000000
//+ 11111111 11110000 (<< 12)
//= 11111111 11111111 11111111 00000000 (1)
//-----------------------------------------
// 11111111 00000000 00000000 00000000 (1)
//+ 11111111 11110000 (<< 16)
//= 11111111 11111111 11110000 00000000 (2)
//-----------------------------------------
// 11110000 00000000 00000000 00000000 (2)
//+ 1111 11111111 0000 (<< 20)
//= 11111111 11111111 00000000 00000000 (3)
//-----------------------------------------
// 00000000 00000000 00000000 00000000 (3)
//+ 11111111 11110000 (<< 24)
//= 11111111 11110000 00000000 00000000
for(int i=2, shiftBits = 12; i < size; i += 2) {
if(shiftBits == 24) {
//Copy 2 bytes from bytes[] into store[] and move on.
store[storeIndex] = bytes[i];
//Never store byte 0 (Garbage).
tmp = (byte)(bytes[i + 1] & 0xF0); //Bit order: 11110000.
if(tmp != 0) store[++storeIndex] = tmp;
shiftBits = 12; //Reset
} else if(shiftBits == 20) {
result = ((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
| (((bytes[i] & 0xFF) << 20) | ((bytes[i + 1] & 0xFF) << 12));
store[storeIndex] = (byte)((result >> 24) & 0xFF);
tmp = (byte)((result >> 16) & 0xFF);
//Never store byte 0 (Garbage).
if(tmp != 0) store[++storeIndex] = tmp;
shiftBits = 24;
} else if(shiftBits == 16) {
result = ((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
| (((bytes[i] & 0xFF) << 16) | ((bytes[i + 1] & 0xFF) << 8));
store[storeIndex] = (byte)((result >> 16) & 0xFF);
tmp = (byte)((result >> 8) & 0xF0);
//Never store byte 0 (Garbage).
if(tmp != 0) store[++storeIndex] = tmp;
shiftBits = 20;
} else {
result = ((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
| (((bytes[i] & 0xFF) << 12) | ((bytes[i + 1] & 0xFF) << 4));
store[storeIndex] = (byte)((result >> 16) & 0xFF);
tmp = (byte)((result >> 8) & 0xFF);
//Never store byte 0 (Garbage).
if(tmp != 0) store[++storeIndex] = tmp;
shiftBits = 16;
}
}
return ++storeIndex;
}
Explanations
result = ((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
| (((bytes[i] & 0xFF) << 20) | ((bytes[i + 1] & 0xFF) << 12));
What this does is basically merge two integers into one.
((store[storeIndex - 1] << 24) | ((store[storeIndex] & 0xFF) << 16))
The first one is make an integer with same constant bit position.
(((bytes[i] & 0xFF) << 20) | ((bytes[i + 1] & 0xFF) << 12));
The latter is for 2 current bytes with different bit positions.
(...) | (...)
Pipe or vertical bar at the middle is to merge these two integers we've just created into one.
Usage
To use this method is pretty straight forward.
byte[] buffer = new byte[1000];
byte[] store;
if((buffer.length & 1) == 0) { //Even.
store = new byte[Math.round((bytes.length * 6) / 8F)];
} else { //Odd.
store = new byte[Math.round(((bytes.length - 1) * 6) / 8F)];
}
audioRecord.read(buffer, 0, buffer.length);
int convertedByteSize = convert16BitTo12Bit(buffer, store);
System.out.println("size: " + convertedByteSize);

I have discovered a solution that produces clear audio. First, it is important to recount the requirements for the use case, which is 12 bit unsigned mono audio which will be read in little endian by the device in packets of 1000 bytes.
The initialization and configuration of the AudioRecord as described in the question is fine.
Once the 1000 bytes of audio is read from AudioRecord, it can be put into a ByteBuffer and defined as little endian for modification, and then put into a ShortBuffer to do manipulation on the 16 bit level:
// Audio specifications of device is in little endian.
ByteBuffer byteBuffer = ByteBuffer.wrap(input).order(ByteOrder.LITTLE_ENDIAN);
// Turn into a ShortBuffer so bitwise manipulation can occur on the 16 bit level.
ShortBuffer shortBuffer = byteBuffer.asShortBuffer();
Next, in a loop, take each short and modify it to 12 bit unsigned:
for (int i = 0; i < shortBuffer.capacity(); i++) {
short currentShort = shortBuffer.get(i);
shortBuffer.put(i, convertShortTo12Bit(currentShort));
}
This can be accomplished by shifting the 16 bits four spaces to the right to turn it into 12 bit signed. Then, to convert to unsigned, add 2048. For our purposes as a safety step, we also mask the least significant four bits as required by device, but given the shifting and adding, it shouldn't be the case that any bits actually remain there:
private static short convertShortTo12Bit(short input) {
int inputAsInt = input;
inputAsInt >>>= 4;
inputAsInt += 2048;
input = (short) (inputAsInt & 0B0000111111111111);
return input;
}
If one wishes to return 12 bits to 16 bits, do the reverse for each short (subtract 2048 and shift four spaces to the left).

Converting bytes to a long: Why do some implementations bitwise-AND each byte with 0xff?

I'm looking at the implementation of java.io's DataInputStream.readLong() in SE6:
private byte readBuffer[] = new byte[8];
public final long readLong() throws IOException {
readFully(readBuffer, 0, 8);
return (((long)readBuffer[0] << 56) +
((long)(readBuffer[1] & 255) << 48) +
((long)(readBuffer[2] & 255) << 40) +
((long)(readBuffer[3] & 255) << 32) +
((long)(readBuffer[4] & 255) << 24) +
((readBuffer[5] & 255) << 16) +
((readBuffer[6] & 255) << 8) +
((readBuffer[7] & 255) << 0));
Given that readBuffer[] is an array of bytes, why is it necessary to & each byte with 255?
When a single byte is cast to a long, shouldn't the remaining bits (9-64) bits of the long automatically be set to zero, rendering the & unnecessary?

java's byte type is signed, so 0xff (255) == -1, during extending from byte to int/long - signed value is preserved, so if you just have code:
final byte a = (byte)0xff;
final long b = a;
System.out.println(b); // output here is -1, not 255
so, here comes one trick:
final byte a = (byte)0xff;
final long b = a & 0xff; // binary and between byte A and int 0xff
System.out.println(b); // output here is 255
so, first byte variable a is promoted to int (and became 0xffffffff) because of sign extension, then we truncate it by doing bitwise AND

To prevent sign-extension of bytes with negative values.

BSD checksum and bits operations explanation

I try understand BSD checksum calulcation algorithm, writed in Java language.
Wiki writed:
byte checksum(byte[] input) {
byte checksum = 0;
for (byte cur_byte: input) {
checksum = (byte) (((checksum & 0xFF) >>> 1) + ((checksum & 0x1) << 7)); // Rotate the accumulator
checksum = (byte) ((checksum + cur_byte) & 0xFF); // Add the next chunk
}
return checksum;
}
And my questions:
Why we use bitwise & in this line checksum = (byte) ((checksum + cur_byte) & 0xFF); ? 0xFF is binary "11111111" and this operation not return always this same number?
What is a sense of this operation? checksum = (byte) (((checksum & 0xFF) >>> 1) + ((checksum & 0x1) << 7)); I understand binary operation and logical and arithmetical shifts, but dont understand what we doing.
Thanks for help :)

b & 0xFF is often used to cast signed byte to bit-identical int. In this case it is unnecessary - (byte)(b & 0xFF) identical to (b). For example ((byte)-1) & 0xFF = 255
12345678 >>>1 01234567
12345678 <<7 80000000 ADD -> 81234567
Thus it is cyclic rotation

Java: >>> operator vs 0xff

The task is to fetch each byte from a given integer. This is the approach I saw somewhere:
byte[] bytes = new byte[4];
bytes[0] = (byte) ((id >> 24) & 0xff);
bytes[1] = (byte) ((id >> 16) & 0xff);
bytes[2] = (byte) ((id >> 8) & 0xff);
bytes[3] = (byte) (id & 0xff);
It would result in the same break-up as this:
bytes[0] = (byte) (id >>> 24);
bytes[1] = (byte) (id >>> 16);
bytes[2] = (byte) (id >>> 8);
bytes[3] = (byte) (id);
where, id is an integer value and will ALWAYS be unsigned. In fact, I don't see the need to AND with 0xff in the first approach (isn't it? since we're always using the least significant byte).
Is there any difference in the two approaches and which one is preferred?

You do not need the & 0xff in the upper example either, because your example always chops off the bits that are different in sign-extended vs. non-sign-extended numbers.
Here is why: when you shift a number right by n bits using >>, the upper n bits will get the same value as the most significant bit of the number being shifted. The behavior of >>> differs only in that >>> forces the upper n bits to zero. The lower (32-n) bits are the same regardless of the kind of the shift that you use.
None of your examples shifts by more 24 bits, so the lower eight bits would be the same if you replace >>> with >> in your bottom example.
Since it is entirely unnecessary to mask with 0xff, I would use your second snippet using >> or >>> for the operator, because the code is shorter.

How to convert a byte array to its numeric value (Java)?

I have an 8 byte array and I want to convert it to its corresponding numeric value.
e.g.
byte[] by = new byte[8]; // the byte array is stored in 'by'
// CONVERSION OPERATION
// return the numeric value
I want a method that will perform the above conversion operation.

One could use the Buffers that are provided as part of the java.nio package to perform the conversion.
Here, the source byte[] array has a of length 8, which is the size that corresponds with a long value.
First, the byte[] array is wrapped in a ByteBuffer, and then the ByteBuffer.getLong method is called to obtain the long value:
ByteBuffer bb = ByteBuffer.wrap(new byte[] {0, 0, 0, 0, 0, 0, 0, 4});
long l = bb.getLong();
System.out.println(l);
Result
4
I'd like to thank dfa for pointing out the ByteBuffer.getLong method in the comments.
Although it may not be applicable in this situation, the beauty of the Buffers come with looking at an array with multiple values.
For example, if we had a 8 byte array, and we wanted to view it as two int values, we could wrap the byte[] array in an ByteBuffer, which is viewed as a IntBuffer and obtain the values by IntBuffer.get:
ByteBuffer bb = ByteBuffer.wrap(new byte[] {0, 0, 0, 1, 0, 0, 0, 4});
IntBuffer ib = bb.asIntBuffer();
int i0 = ib.get(0);
int i1 = ib.get(1);
System.out.println(i0);
System.out.println(i1);
Result:
1
4

Assuming the first byte is the least significant byte:
long value = 0;
for (int i = 0; i < by.length; i++)
{
value += ((long) by[i] & 0xffL) << (8 * i);
}
Is the first byte the most significant, then it is a little bit different:
long value = 0;
for (int i = 0; i < by.length; i++)
{
value = (value << 8) + (by[i] & 0xff);
}
Replace long with BigInteger, if you have more than 8 bytes.
Thanks to Aaron Digulla for the correction of my errors.

If this is an 8-bytes numeric value, you can try:
BigInteger n = new BigInteger(byteArray);
If this is an UTF-8 character buffer, then you can try:
BigInteger n = new BigInteger(new String(byteArray, "UTF-8"));

Simply, you could use or refer to guava lib provided by google, which offers utiliy methods for conversion between long and byte array. My client code:
long content = 212000607777l;
byte[] numberByte = Longs.toByteArray(content);
logger.info(Longs.fromByteArray(numberByte));

You can also use BigInteger for variable length bytes. You can convert it to Long, Integer or Short, whichever suits your needs.
new BigInteger(bytes).intValue();
or to denote polarity:
new BigInteger(1, bytes).intValue();

Complete java converter code for all primitive types to/from arrays
http://www.daniweb.com/code/snippet216874.html

Each cell in the array is treated as unsigned int:
private int unsignedIntFromByteArray(byte[] bytes) {
int res = 0;
if (bytes == null)
return res;
for (int i=0;i<bytes.length;i++){
res = res | ((bytes[i] & 0xff) << i*8);
}
return res;
}

public static long byteArrayToLong(byte[] bytes) {
return ((long) (bytes[0]) << 56)
+ (((long) bytes[1] & 0xFF) << 48)
+ ((long) (bytes[2] & 0xFF) << 40)
+ ((long) (bytes[3] & 0xFF) << 32)
+ ((long) (bytes[4] & 0xFF) << 24)
+ ((bytes[5] & 0xFF) << 16)
+ ((bytes[6] & 0xFF) << 8)
+ (bytes[7] & 0xFF);
}
convert bytes array (long is 8 bytes) to long

You can try use the code from this answer: https://stackoverflow.com/a/68393576/7918717
It parses bytes as a signed number of arbitrary length. A few examples:
bytesToSignedNumber(false, 0xF1, 0x01, 0x04) returns 15794436 (3 bytes as int)
bytesToSignedNumber(false, 0xF1, 0x01, 0x01, 0x04) returns -251592444 (4 bytes as int)
bytesToSignedNumber(false, 0xF1, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x04) returns -1080581331768770303 (8 of 9 bytes as long)