We Keep Coding

Java - why does readAllBytes return incorrect byte codes? [duplicate]

This question already has answers here:
Java byte array contains negative numbers
(5 answers)
Closed 6 years ago.
I have the following problem. I'm using Java to create a byte array from a file. So I do the following:
byte[] myByteArray = Files.readAllBytes(filename);
However, for many of the bytes it is returning incorrect/negative values.
For instance, if I test using javascript, to read every byte of a file e.g.
function readbytes(s){
var f = new File(s);
var i,a,c;
var d = [];
if (f.isopen) {
c = f.eof;
for(i=0;i<c ;i++){
a = f.readbytes(1);
d.push(a);
}
f.close();
return d;
} else {
post("could not open file: " + s + "n");
}
}
(readbytes is a function in the program Im using that gives the byte at a specific position).
This returns the correct bytes
So Im wondering, why does java return incorrect codes? Is this something to do with unsigned values?

Java doesn't know unsigned bytes. For instance the unsigned byte 255 would be printed as its signed version -1. In memory however, the actual value would be the same and represented as 255.
If you'd like to convert a byte to its unsigned representation, you may use the bitwise AND operator.
For instance:
bytes[x] & 0xff
Java doesn't know about bytes at runtime either for any operand that may be pushed onto the Java virtual machine's stack. In fact every operation you apply to an integral value results in an integer. That's why ((byte)-1) & 0xff) results in an integer and its value is 255. If you would like to store that value back into a byte, you'd have to cast it to byte again, which of course, is -1.
byte x = -1; // java is friendly enough to insert the implicit cast here
System.out.println(x); // -1
System.out.println(x & 0xff); // 255
byte y = (byte)(x & 0xff); // must add (byte) cast
System.out.println(y); // -1
Also keep in mind that technically the output you see is different but the content is still the same since you can map from Java's signed byte always to its unsigned representation. Ideally, you'd use something like DataInputStream which offers you int readUnsignedByte().

Java - converting int to byte array without considering sign

To convert an int into a byte array, I'm using the following code:
int a = 128;
byte[] b = convertIntValueToByteArray(a);
private static byte[] convertIntValueToByteArray(int intValue){
BigInteger bigInteger = BigInteger.valueOf(intValue);
byte[] origByteArray = bigInteger.toByteArray();
byte[] noSignByteArray = new byte[bigInteger.bitLength()/8];
if(bigInteger.bitLength()%8!=0){
noSignByteArray = origByteArray;
}else{
System.arraycopy(origByteArray,1,noSignByteArray,0,noSignByteArray.length);
}
return noSignByteArray;
}
There are two things which I'm attempting to do.
1)I need to know the number of bytes (rounded up to the closes byte) of the original integer. However, I don't need the additional bit that is added for the sign bit when I call the toByteArray() method. This is the reason why I have the helper method. So in this example, if I don't have the helper method, when I convert 128 to a byte array I get the length to be 2 octets because of the sign bit but I'm only expecting it to be one octet.
2)I need the positive representation of the number. In this example, if I attempt to print the first element in array b, I get -128. However, the numbers I will be using will be positive numbers only so what I actually want is 128. I'm limited to using a byte array. Is there a way to accomplish this?
Updated Post
Thank you for the responses. I haven't found the exact answer I was looking for so I'll attempt to give more details. Ultimately, I want to write values of different types over a data output stream. In this post, I'd like to clarify what happens when ints are written to a data output stream. I've come across two scenarios.
1)
DataOutputStream os = new DataOutputStream(this.socket.getOutputStream());
byte[] b = BigInteger.valueOf(128).toByteArray();
os.write(b);
2)
DataOutputStream os = new DataOutputStream(this.socket.getOutputStream());
os.write(128);
In the first scenario, when the bytes are read from a data input stream, it seems that the first element in the byte array is a 0 to represent the msb and the second element in the array contains the number -128. However, since the msb is 0 we would be able to determine that it is intended to be a positive number. In the second scenario, there is no msb and the only element present in the byte array read from the input stream is -128. I was expecting the write() method of the data output stream to convert the int into the byte array in the same manner as the toByteArray() method does on a BigInteger object. However, this doesn't seem to be the case as the msb is not present. So my question is, how in the second scenario are we supposed to know that 128 is supposed to be a positive number and not a negative one if there is no msb.

As you probably already know
In an octet, the pattern 10000000 can be interpreted as either 128 or -128, depending on the, um, outside interpretation
Java's byte type interprets octects as values in -128...127 only.
If you are building an application in which the entire world consists of nonnegative integers only, then you could simply do all of your work under the assumption that the byte value -128 will mean 128 and -127 will mean 129 and ... and -1 will mean 255. This is certainly doable but it takes work.
Dealing with the notion of an "unsigned byte" like this is normally done by expanding the byte into a short or int with the higher order bits all set to zero and then performing arithmetic or displaying your values. You will need to decide whether such an approach is more to your liking than just representing 128 as two octets in your array.

I think the following code might be sufficient.
In java int is a twos-complements binary number:
-1 = 111...111
ones complement = 000...000; + 1 =
1 = 000...001
So that about the sign bit I do not understand. Be it, that you could do Math.abs(n).
A byte ranges from -128 to 127, but the interpretation is a matter of masking, as below.
public static void main(String[] args) {
int n = 128;
byte[] bytes = intToFlexBytes(n);
for (byte b: bytes)
System.out.println("byte " + (((int)b) & 0xFF));
}
public static byte[] intToFlexBytes(int n) {
// Convert int to byte[4], via a ByteBuffer:
byte[] bytes = new byte[4];
ByteBuffer bb = ByteBuffer.allocateDirect(4);
bb.asIntBuffer().put(n);
bb.position(0);
bb.get(bytes);
// Leading bytes with 0:
int i = 0;
while (i < 4 && bytes[i] == 0)
++i;
// Shorten bytes array if needed:
if (i != 0) {
byte[] shortenedBytes = new byte[4 - i];
for (int j = i; j < 4; ++j) {
shortenedBytes[j - i] = bytes[j]; // System.arrayCopy not needed.
}
bytes = shortenedBytes;
}
return bytes;
}

To answer your first question—how many bytes are required to represent a nonnegative integer using an unsigned representation—consider the following functions I wrote in Common Lisp.
(defconstant +bits-per-byte+ 8)
(defun bit-length (n)
(check-type n (integer 0) "a nonnegative integer")
(if (zerop n)
1
(1+ (floor (log n 2)))))
(defun bytes-for-bits (n)
(check-type n (integer 1) "a positive integer")
(values (ceiling n +bits-per-byte+)))
These highlight the mathematical underpinnings of the problem: namely, the logarithm tells you how many powers of two (as provided by bits) it takes to dominate a given nonnegative integer, adjusted to be a step function with floor, and the number of bytes it takes to hold that number of bits again as a step function, this time adjusted with ceiling.
Note that the number zero is intolerable as input to a logarithm function, so we avoid it explicitly. You may observe that the bit-length function could also be written with a slight transformation of the core expression:
(defun bit-length-alt (n)
(check-type n (integer 0) "a nonnegative integer")
(values (ceiling (log (1+ n) 2))))
Unfortunately, as the logarithm of one is always zero, regardless of the base, this version says that the integer zero can be represented by zero bits, which isn't the answer we want.
For your second goal, you can use the functions I've defined above to allocate the required number of bytes, and incrementally set the bits you need, ignoring sign. It's hard to tell if you're having trouble getting the proper bits set in the byte vector, or whether your problem is in interpreting the bits in way that avoids treating the high bit as a sign bit (that is, two's complement representation). Please elaborate what kind of push you need to get you moving again.

Is there a way to represent int value in byte over 127 without doing bitwise operations?

I am really short on time for doing the learning of bitwise operations.
I want to convert large integer(>127) values without doing '<<' or anything similar.
I need byte representation of integer values used to identify sequence numbers of packets in header sent across UDP. If there is no solution I will introduce two bytes..
Something like: 1, 1 ; 1,2 ; 1,3 ; packet lost ; 1,4 ; packet lost; 2,1 ,2,2
and then reset it upon reaching 127; 127
I can introduce third, but this is rather ugly.
It would be really useful to have black box that is part of java api doing all that byte conversion for me. Is there?
Thanks,

To pack an unsigned 8-bit value into a byte:
static byte toByte(int i) {
if ((i < 0) || (i > 255))
throw new IllegalArgumentException(String.valueOf(i));
return (byte) i;
}
To convert back:
static int toInt(byte b) {
return (b < 0) ? (b + 256) : b;
}
After reading your comments on other answers, it sounds like you might want something like this:
byte[] b = BigInteger.valueOf(counter).toByteArray();
and
long counter = new BigInteger(b).longValue();
Since the length of the array would vary as the counter grows, you'd need some way to indicate its length or delimit it. But this technique will convert any integer value to an array of bytes.

Is the problem that you want unsigned bytes, as in, numbers between 128 and 255 inclusive?
That's...tricky. The Java language won't let you directly treat bytes as unsigned...but with library support it gets a little easier. Guava provides an UnsignedBytes utility class for some of these needs. Addition, multiplication, and subtraction are all exactly the same on signed and unsigned bytes.
EDIT: Judging from your additional comments, you might be interested in Ints.toByteArray(int) and the like, which work on types between byte and BigInteger.

According to my understanding, you want to separate an int into 4 bytes. If so, then just copy paste this code:
int i = /* your int */
int[] b = { (i >> 24) & 0xff, (i >> 16) & 0xff, (i >> 8) & 0xff, i & 0xff };
Indices 0-3 are each of the 4 bytes in the int.

Java: Convert byte to integer

I need to convert 2 byte array ( byte[2] ) to integer value in java. How can I do that?

You can use ByteBuffer for this:
ByteBuffer buffer = ByteBuffer.wrap(myArray);
buffer.order(ByteOrder.LITTLE_ENDIAN); // if you want little-endian
int result = buffer.getShort();
See also Convert 4 bytes to int.

In Java, Bytes are signed, which means a byte's value can be negative, and when that happens, #MattBall's original solution won't work.
For example, if the binary form of the bytes array are like this:
1000 1101 1000 1101
then myArray[0] is 1000 1101 and myArray[1] is 1000 1101, the decimal value of byte 1000 1101 is -115 instead of 141(= 2^7 + 2^3 + 2^2 + 2^0)
if we use
int result = (myArray[0] << 8) + myArray[1]
the value would be -16191 which is WRONG.
The reason why its wrong is that when we interpret a 2-byte array into integer, all the bytes are
unsigned, so when translating, we should map the signed bytes to unsigned integer:
((myArray[0] & 0xff) << 8) + (myArray[1] & 0xff)
the result is 36237, use a calculator or ByteBuffer to check if its correct(I have done it, and yes, it's correct).

Well, each byte is an integer in the range -128..127, so you need a way to map a pair of integers to a single integer. There are many ways of doing that, depending on what you have encoded in the pair of bytes. The most common will be storing a 16-bit signed integer as a pair of bytes. Converting that back to an integer depends on whether you store it big-endian form:
(byte_array[0]<<8) + (byte_array[1] & 0xff)
or little endian:
(byte_array[1]<<8) + (byte_array[0] & 0xff)

Also, if you can use the Guava library:
Ints.fromByteArray(0, 0, myArray[1], myArray[0]);
It was worth mentioning since a lot of projects use it anyway.

Simply do this:
return new BigInteger(byte[] yourByteArray).intValue();
Works great on Bluetooth command conversions etc. No need to worry about signed vs. unsigned conversion.

import java.io.*;
public class ByteArray {
public static void main(String[] args) throws IOException {
File f=new File("c:/users/sample.txt");
byte[]b={1,2,3,4,5};
ByteArrayInputStream is=new ByteArrayInputStream(b);
int i;
while((i=is.read())!=-1) {
System.out.println((int)i);
FileOutputStream f1=new FileOutputStream(f);
FileOutputStream f2=new FileOutputStream(f);
ByteArrayOutputStream b1=new ByteArrayOutputStream();
b1.write(6545);
b1.writeTo(f1);
b1.writeTo(f2);
b1.close();
}

Read two bytes into an integer?

I have a byte[] that I've read from a file, and I want to get an int from two bytes in it. Here's an example:
byte[] bytes = new byte[] {(byte)0x00, (byte)0x2F, (byte)0x01, (byte)0x10, (byte)0x6F};
int value = bytes.getInt(2,4); //This method doesn't exist
This should make value equal to 0x0110, or 272 in decimal. But obviously, byte[].getInt() doesn't exist. How can I accomplish this task?
The above array is just an example. Actual values are unknown to me.

You should just opt for the simple:
int val = ((bytes[2] & 0xff) << 8) | (bytes[3] & 0xff);
You could even write your own helper function getBytesAsWord (byte[] bytes, int start) to give you the functionality if you didn't want the calculations peppering your code but I think that would probably be overkill.

Try:
public static int getInt(byte[] arr, int off) {
return arr[off]<<8 &0xFF00 | arr[off+1]&0xFF;
} // end of getInt
Your question didn't indicate what the two args (2,4) meant. 2 and 4 don't make sense in your example as indices in the array to find ox01 and 0x10, I guessed you wanted to take two consecutive element, a common thing to do, so I used off and off+1 in my method.
You can't extend the byte[] class in java, so you can't have a method bytes.getInt, so I made a static method that uses the byte[] as the first arg.
The 'trick' to the method is that you bytes are 8 bit signed integers and values over 0x80 are negative and would be sign extended (ie 0xFFFFFF80 when used as an int). That is why the '&0xFF' masking is needed. the '<<8' shifts the more significant byte 8 bits left.
The '|' combines the two values -- just as '+' would. The order of the operators is important because << has highest precedence, followed by & followed by | -- thus no parentheses are needed.

Here's a nice simple reliable way.
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(4);
// by choosing big endian, high order bytes must be put
// to the buffer before low order bytes
byteBuffer.order(ByteOrder.BIG_ENDIAN);
// since ints are 4 bytes (32 bit), you need to put all 4, so put 0
// for the high order bytes
byteBuffer.put((byte)0x00);
byteBuffer.put((byte)0x00);
byteBuffer.put((byte)0x01);
byteBuffer.put((byte)0x10);
byteBuffer.flip();
int result = byteBuffer.getInt();

Alternatively, you could use:
int val = (bytes[2] << 8) + bytes[3]

You can use ByteBuffer. It has the getInt method you are searching for and many other useful methods

The Google Base16 class is from Guava-14.0.1.
new BigInteger(com.google.common.io.BaseEncoding.base16().encode(bytesParam),16).longValue();