I am having a strange program with a GzipInputStream zero filling part of the buffer. I have the fortune of knowing what the bytes are supposed to look like in the stream and I can see the buffer is being filled with 8 correct bytes and 12 zeros (shouldn't be zero)
BYTES SHOULD LOOK LIKE THIS---->
0
20
82
22
-91
27
-96
65
66
65
88
32
32
32
32
81
32
0
0
0
100
78
BYTES ACTUALLY LOOK LIKE THIS--->
0
20
82
22
-91
27
-96
65
66
65
0
0
0
0
0
0
0
0
0
0
0
0
The first two bytes represent an integer that determine the size of the variable length (in bytes) payload after the first two. So in this example, the first bytes are 0 20 and in BIG_ENDIAN this gives us a subsequent payload size of 20 bytes.
Here is my code for reading
gzipInputStream = new GZIPInputStream(url.openStream());
byte[] payload = new byte[2];
gzipInputStream.read(payload);
for(int i=0;i<payload.length;i++){
System.out.println(payload[i]);
}
int payloadSize = ((payload[0] & 0xFF) << 8) | ((payload[1]) & 0xFF);
//read the next payloadSize bytes
byte[] messageBytes = new byte[payloadSize];
gzipInputStream.read(messageBytes);
So the first two bytes are those in the payload array and the second 20 bytes are those in messageBytes. Can't figure it out
Modified code thanks to NPE
byte[] payloadSizeBytes = new byte[2];
int payloadSizeBytesRead = 0;
while(payloadSizeBytesRead < 2){
int r = gzipInputStream.read(buffer);
if(r>0){
payloadSizeBytes[payloadSizeBytesRead] = buffer[0];
payloadSizeBytesRead++;
}
}
int payloadSize = ((payloadSizeBytes[0] & 0xFF) << 8) | ((payloadSizeBytes[1]) & 0xFF);
//read the next payloadSize bytes
byte[] messageBytes = new byte[payloadSize];
int messageBytesRead = 0;
while(messageBytesRead < payloadSize){
int r = gzipInputStream.read(buffer);
if(r>0){
messageBytes[messageBytesRead] = buffer[0];
messageBytesRead++;
}
}
for(int i=0;i<messageBytes.length;i++){
System.out.println(messageBytes[i]);
}
The contract on read(byte[]) is that it reads some data, and returns how many bytes have been read. As things stand, you are ignoring the return value. Instead, you should examine the return value of read() and keep calling read() until you've read payloadSize bytes.
An easy way to do this is by using read(b, off, len) in a loop:
int payloadSize = ((payload[0] & 0xFF) << 8) | ((payload[1]) & 0xFF);
byte[] messageBytes = new byte[payloadSize];
int bytesRead = 0;
while (bytesRead < payloadSize) {
bytesRead += gzipInputStream.read(messageBytes, bytesRead, payloadSize - bytesRead);
}
Related
I have a byte array from which I need to read specific bits and convert to int (see the byte array structure below). Even though the bits information I want to read is in 3 bytes, I tried reading 4 bytes (6-9) as integer and then read the bits from that integer value with bits or bitsValue method but somehow I am not able to see the right values from the bit manipulation. And with my expertise in bits I am pretty sure I am doing something wrong.
Can someone please suggest I am doing it correctly and why its not working. Thanks in Advance!!
Byte array is in Little Endian format.
0th byte - Some Value
1st Byte - Some Value
2nd - 5th Byte - Some Value
6th - 9th Byte - first 18 bits represent some value
- Next 5 bits represent some value
- Next 1 bit represent some value
- Last 8 bits represent some value
public class Test {
public static void main(String... dataProvider) {
String s = "46 00 ef 30 e9 08 cc a5 03 43";
byte[] bytes = new byte[s.length()];
bytes = hexStringToByteArray(s);
int bytePointer = 0;
int msgType = getIntFromSingleByte(bytes[bytePointer]); // 0th byte
int version = getIntFromSingleByte(bytes[++bytePointer]); // 1st byte
int tickInMS = getIntValue(bytes, ++bytePointer); // 2nd-5th bytes
bytePointer = bytePointer + 4;
int headr = getIntValue(bytes, bytePointer); // 6th-9th bytes
int utcTime = bits(headr, 0, 18); // 6th-9th bytes - 18 bits
int reserved = bits(headr, 18, 5); // 6th-9th bytes- 5 bits
int reportOrEvent = bits(headr, 23, 1); // 6th-9th bytes - 1 bits
int reportId = bitsValue(headr, 24, 32); // 6th-9th- 8 bits
}
public static int getIntFromSingleByte(byte data) {
return (data & 0xFF);
}
public static int getIntValue(byte[] bytes, int startPosition) {
byte[] dest = new byte[4];
System.arraycopy(bytes, startPosition, dest, 0, dest.length);
return toInt(dest);
}
// took from Stack overflow
static int bits(int n, int offset, int length) {
// shift the bits rightward, so that the desired chunk is at the right end
n = n >> (31 - offset - length);
// prepare a mask where only the rightmost `length` bits are 1's
int mask = ~(-1 << length);
// zero out all bits but the right chunk
return n & mask;
}
public static int bitsValue(int intNum, int startBitPos, int endBitPos) {
// parameters checking ignored for now
int tempValue = intNum << endBitPos;
return tempValue >> (startBitPos + endBitPos);
}
public static byte[] hexStringToByteArray(final String s) {
String[] splits = s.split(" ");
final byte[] data = new byte[splits.length];
for (int i = 0; i < splits.length; i++) {
char first = splits[i].length() < 2 ? '0' : splits[i].charAt(0);
char second = splits[i].length() < 2 ? splits[i].charAt(0) : splits[i].charAt(1);
data[i] = (byte) ((Character.digit(first, 16) << 4) + Character.digit(second, 16));
}
return data;
}
public static int toInt(byte[] data) {
if (data == null || data.length != 4)
return 0x0;
return (int) ((0xff & data[0]) << 24 | (0xff & data[1]) << 16 | (0xff & data[2]) << 8
| (0xff & data[3]) << 0);
}
}
Wrapping your input data in a ByteBuffer will simplify parsing and allow you to adjust endianness as necessary.
Your bits method is wrong. The constant 31 should be 32. Also, the method uses MSB 0 bit numbering, which is odd for little-endian data. You should confirm that your input is documented as using this bit numbering scheme.
Your bitsValue method is wrong too. May as well just use bits after fixing it.
This code is simpler and extracts the bit fields correctly:
public static void main(String... args) {
String s = "46 0 79 37 a8 3 9f 37 1 43 eb 7a f 3 3 fe c4 1 c5 4 c5 5e";
byte[] input = hexStringToByteArray(s);
// Wrap the input in a ByteBuffer for parsing. Adjust endianness if necessary.
ByteBuffer buffer = ByteBuffer.wrap(input).order(ByteOrder.BIG_ENDIAN);
int msgType = buffer.get() & 0xff;
int version = buffer.get() & 0xff;
int tickInMS = buffer.getInt();
int header = buffer.getInt();
int utcTime = bits(header, 0, 18); // 6th-9th bytes - 18 bits
int reserved = bits(header, 18, 5); // 6th-9th bytes - 5 bits
int reportOrEvent = bits(header, 23, 1); // 6th-9th bytes - 1 bit
int reportId = bits(header, 24, 8); // 6th-9th bytes - 8 bits
System.out.printf("utc: %d, report? %d, id: %d\n", utcTime, reportOrEvent, reportId);
}
/**
* Extract a bit field from an int. Bit numbering is MSB 0.
*/
public static int bits(int n, int offset, int length) {
return n >> (32 - offset - length) & ~(-1 << length);
}
Instead of error prone bit mangeling you should rather use the binary string representation of the numbers and do your "bit picking" as string operations:
String s = "46 00 ef 30 e9 08 cc a5 03 43";
String[] hexNumbers = s.split(" ");
for(String hexNumber : hexNumbers) {
String binaryNumber = String.format("%8s", new BigInteger(hexNumber,16).toString(2)).replace(' ','0');
System.out.print(String.format("value in hex : %s, value in binary: %s", hexNumber,binaryNumber));
}
I'm playing around with audio in Java. I discovered that a commonly used AudioFormat uses two bytes. However, I couldn't figure out how to put the bytes together into a single int. So I tried doing it in reverse:
public class SineWave {
public static void main(String[] args) throws LineUnavailableException {
int hz = 440;
int samplerate = 16384;
int amplitude = 127;
AudioFormat format = new AudioFormat((float) samplerate, 16, 1, true, true);
SourceDataLine sdl = AudioSystem.getSourceDataLine(format);
sdl.open(format, samplerate * 2);
sdl.start();
while (true) {
byte[] toWrite = new byte[samplerate * 2];
for (int x = 0; x < samplerate; x++) {
int y = (int) Math.round(amplitude * Math.sin(2 * Math.PI * x * hz / samplerate));
byte b1 = (byte) (y & 0xFF);
byte b2 = (byte) ((y >> 8) & 0xFF);
toWrite[2 * x] = b1;
toWrite[2 * x + 1] = b2;
// System.out.printf("%d %d%n", b1, b2);
}
sdl.write(toWrite, 0, toWrite.length);
}
}
}
However, this only works up to an amplitude of 127. When the System.out.printf is uncommented, it is clear that this amplitude uses only 1 byte. When I go up to 128, I get outputs like this (and ugly sounds):
0 0
21 0
42 0
62 0
80 0
96 0
109 0
118 0
125 0
-128 0
127 0
123 0
115 0
104 0
90 0
73 0
55 0
35 0
13 0
Negative values are similar, without the change in sign, and the second byte is always -1
I have deduced that this is because of signed bytes and two's complement, but I still can't figure out what I can do to fix this.
How does Java compose its audio?
You are on the right track, although you might have your byte ordering backwards (try swapping b1 and b2 in the toWrite assignments to see if that makes things sound better). That could explain why things sound bad. Also, an amplitude of 127 is very small so you should try increasing it to the max (32767).
The way you are printing the bytes out is probably adding to the confusion. Splitting a signed 16-bit number into 2 signed 8-bit numbers doesn't really make any sense. Consider when the 16-bit number is -1 (0xffff), you print out two signed bytes and you get -1 (0xff) and -1 (0xff). You'd be better off printing the bytes out as hex values and dealing with the sign in your head.
I have a byte array myByteArray[82]
I want to change exactly 37 bits in this array to the complement value. ie. if bit0 has a '1' I want to change it to '0'. I need to change first 37 such bits to introduce error in this byte array.
Please suggest how to do this
It's not entirely clear what you are trying to do. My best understanding is that you have an array of 82 bytes and you want to invert the lowest 37 bits of the array. Since a byte is 8 bits, so you can do this:
byte[] myByteArray = new byte[82];
// invert lowest 32 bits, 8 at a time
for (int i = 0; i < 4; ++i) {
myByteArray[i] = (byte)(~myByteArray[i]);
}
// invert next five bits
myByteArray[4] = (byte) (
(myByteArray[4] & 0xE0) // top 3 bits unchanged
|
((~myByteArray[4)) & 0x1F) // bottom 5 bits inverted
);
try
byte[] a82 = ...
Set<Integer> set = new HashSet<Integer>();
while (set.size() < 37) {
set.add((int) (Math.random() * 82));
}
for (int i : set) {
int ibyte = i / 8;
int ibit = i % 8;
int m = 1 << ibit;
a[ibyte] ^= m;
}
This works:
int nBits = 37;
int i = 0;
for (; i<nBits / 8; ++i)
myByteArray[i] = (byte)((byte) myByteArray[i] ^ 0xFF);
myByteArray[i] = (byte)(myByteArray[i] ^ ((0xFF >>> 5) ^ 0xFF));
Whether or not you do the final XOR with the 0xFF on the last line depends on whether you consider the most significant bit to be the first bit (then use it) or the last bit (then omit it);
To invert a specific 37 bits, randomly chosen:
// array of 82 bytes with 37 selected bits set to 1, all the rest zero
// you could generate this programmatically as well if you need a different
// set of bits each time, but your question implies you don't
byte[] mask = { 0x00, 0x01, 0x02, 0x80, .... 0x00 };
for (int i=0; i<myByteArray.length; i++)
{
myByteArray[i] ^= mask[i];
}
Use the bit-exclusive-or operator ^. The truth table for xor is
M a s k
| 0 | 1
D -+---+---
a 0| 0 | 1
t -+---+---
a 1| 1 | 0
Wherever there's a 1 in the mask the corresponding bit in the data will be flipped.
I'm trying to flip some bytes around in Java and the function I have is working correctly for some bytes and failing for others.
The function I am using is this:
public static int foldInByte(int m, int pos, byte b) {
int tempInt = (b << (pos * 8));
tempInt = tempInt & (0x000000ff << (pos * 8));
m = m | tempInt;
return m;
}
And the code that implements this is:
byte[] bitMaskArray = new byte[]{
byteBuffer.get(inputIndex),
byteBuffer.get(inputIndex + 1),
byteBuffer.get(inputIndex + 2),
byteBuffer.get(inputIndex + 3)};
int tempInt = 0;
tempInt = foldInByte(0, 3, bitMaskArray[3]);
tempInt = foldInByte(tempInt, 2, bitMaskArray[2]);
tempInt = foldInByte(tempInt, 1, bitMaskArray[1]);
tempInt = foldInByte(tempInt, 0, bitMaskArray[0]);
bitMask = tempInt;
The bytes are being read from a ByteBuffer with the byteOrder being Little Endian.
For example, the bytes 00 01 B6 02 set the bitMask to: 2B60100 - which works perfectly in my program.
However, if the bytes are A0 01 30 00, the bitMask is set to: 3001A0 - which has stipped the last zero from the bitmask.
Is there any way I can stop Java from stipping off trailing zeros?
I hope that makes sense.
Thanks
Tony
The zeros are not being stripped -- both examples cited are correct.
00 01 B6 02 is the 4-byte little-endian for 2B60100
A0 01 30 00 is the 4-byte little-endian for 3001A0
The zeros are there, but probably just not being printed. The System.out.print family of calls will not print leading zero digits.
I might mention that your method is needlessly complex. Here is a single method that computes the same value:
static int extractLittleEndian4(byte[] buf, int index)
{
int a = buf[index+0]&0xff, b = buf[index+1]&0xff, c = buf[index+2]&0xff, d = buf[index+3]&0xff;
return a | (b << 8) | (c << 16) | (d << 24);
}
It looks like you have a ByteBuffer filled with your bytes already. Why don't you let the ByteBuffer reverse the bytes for you? Just add the bytes to the buffer (BIG_ENDIAN is the default if you want to add an integer instead of bytes) and then change the order before reading the integer.
byteBuffer.order(ByteOrder.LITTLE_ENDIAN);
int output = byteBuffer.getInt(0);
If all you're doing is reversing the byte order, let the library do the work for you. If you happened to start with an integer value, you can even just do this:
int input = ...;
int output = Integer.reverseBytes(input);
alright, so my code to read bytes into a int is like so:
int offset = (byte << 16) | (byte2 << 8) | byte3;
And it's reading the bytes "00 00 be" as -66.
How do I read it as the 190 it's meant to be?
byte b = -66;
int i = b & 0xff;
byte b = -66;
int i = b < 0 ? b + 256 : b;
It might be useful declare helper function for this.