We Keep Coding

JAVA Byte Manipulation

I want to read a binary file and do some manipulation on each byte. I want to test that I am manipulating the bytes correctly. I want to set a byte variable1 to "00000000" and then another byte variable2 set at "00001111" and OR them newvariable = variable1|variable2, shift the newvariable << 4 bits and then print out the int value.
byte a = 00000000;
//Convert first oneByte to 4 bits and then xor with a;
byte b = 00001111;
byte c = (byte)(a|b);
c = c << 4;
System.out.println("byte= " + c + "\n");
I am not sure why I keep getting "incompatiable types:possible lossy conversion from byte to int"

You need to put a '0b' in front of those numbers to express binary constants. The number 00001111 is interpreted as a literal in octal, which is 585 in decimal. The max byte is 127 (since it's signed). Try 0b00001111 instead.
As literals, those will still be int, so depending on where you do the assignment, you may also need to explicitly cast down to byte.

Difference in BigInteger values from a String and a byte array

Can someone please explain the difference between the below two initialisations of BigInteger.
Input:
BigInteger bi1 = new BigInteger("EF", 16);
byte[] ba = new byte[] {(byte)0xEF};
BigInteger bi2 = new BigInteger(ba);
Log.d("BIGINTEGER", "Big Integer1 = " + bi1.toString(16));
Log.d("BIGINTEGER", "Big Integer2 = " + bi2.toString(16));
Output:
Big Integer1 = ef
Big Integer2 = -11
How can I initialise a BigInteger with the value "EF" from a byte array?

From the BigInteger docs
Constructor and Description
BigInteger(byte[] val)
Translates a byte array containing the two's-complement binary
representation of a BigInteger into a BigInteger.
The Two's-complement is the real reason.
Lets see how...
(Byte)0xef in binary = 11101111
Now convert that back to Int and you get -17 (base 10) or -11 (base 16).
Now take a look at
byte[] ba = new byte[] {0, (byte)0xEF};
This has the (Byte)0xef but prepended by 0. Which means this array has 00000000 11101111, which when converted gives the correct result.
Why was the previous case different?
Check out 2's complement rules - SO Answer, Mandatory Wikipedia link
Another way of thinking about this
0xEF in Decimal = 239
Range of Byte = -127 to 128
We have Overflow.
239 - 128 = 111
Now count this 111 from back (Numeric data types have this circular behaviour, again due to 2's complement representation).
For example: 129.toByte = -127
(129 - 128 = 1, count from back the 1st value = -127)
Shortcut to counting from back if x>128 && x<256 then x.toByte = (x - 128) - 128
Here x = 239 so x.toByte = -17

Put a leading zero into the byte[]:
byte[] ba = new byte[] {0, (byte)0xEF};
Ideone demo

You need to add a zero into the byte[] array:
byte[] myByteArray = new byte[] {0, (byte)0xEF};
BigInteger bi2 = new BigInteger(ba);
Log.d("BIGINTEGER", "Big Integer1 = " + bi1.toString(16));
Log.d("BIGINTEGER", "Big Integer2 = " + bi2.toString(16));
why?
well the reason is related to the language specification:
Decimal literals have a particular property that is not shared by hexadecimal, i.e Decimal literals are all positive [JLS 3.10.1].
To write a negative decimal constant, you need to use the unary negation operator (-) in combination with a decimal literal.
In this way, you can write any int or long value, whether positive
or negative, in decimal form, and negative decimal constants are clearly identifiable by the presence of a minus sign.
Not so for hexadecimal nor octal literals.
They can take on both positive and negative values. Hex and octal literals are
negative if their high-order bit is set.
So after having said that, 0xFE is actually a negative number...

public BigInteger(byte[] val)
Translates a byte array containing the two's-complement binary representation of a BigInteger into a BigInteger. The input array is assumed to be in big-endian byte-order: the most significant byte is in the zeroth element.
public BigInteger(String val,
int radix)
Translates the String representation of a BigInteger in the specified radix into a BigInteger. [...]
Source: Oracle Java 7 Docs
Your Initialization from a bytearray does not behave as expected, because 0xEF casted to a bytearray returns {1, 1, 1, 0, 1, 1, 1, 1}.
Made to an integer according to the specs mentioned above is done as follows:
1*2^0 + 1*2^1 + 1*2^2 + 1*2^3 + 0*2^4 + 1*2^5 + 1*2^6 - 1*2^7 = -17 = -0x11
The two's-compliment causes the highest byte to be substracted, rather than being added. So adding a 0 to the beginningthe byte array should probably fix the problem:
byte[] ba = new byte[] {0, (byte)0xEF};

parseInt on a string of 8 bits returns a negative value when the first bit is 1

I've got a huge string of bits (with some \n in it too) that I pass as a parameter to a method, which should isolate the bits 8 by 8, and convert them all to bytes using parseInt().
Thing is, every time the substring of 8 bits starts with a 1, the resulting byte is a negative number. For example, the first substring is '10001101', and the resulting byte is -115. I can't seem to figure out why, can someone help? It works fine with other substrings.
Here's my code, if needed :
static String bitsToBytes(String geneString) {
String geneString_temp = "", sub;
for(int i = 0; i < geneString.length(); i = i+8) {
sub = geneString.substring(i, i+8);
if (sub.indexOf("\n") != -1) {
if (sub.indexOf("\n") != geneString.length())
sub = sub.substring(0, sub.indexOf("\n")) + sub.substring(sub.indexOf("\n")+1, sub.length()) + geneString.charAt(i+9);
}
byte octet = (byte) Integer.parseInt(sub, 2);
System.out.println(octet);
geneString_temp = geneString_temp + octet;
}
geneString = geneString_temp + "\n";
return geneString;
}

In Java, byte is a signed type, meaning that when the most significant bit it set to 1, the number is interpreted as negative.
This is precisely what happens when you print your byte here:
System.out.println(octet);
Since PrintStream does not have an overload of println that takes a single byte, the overload that takes an int gets called. Since octet's most significant bit is set to 1, the number gets sign-extended by replicating its sign bit into bits 9..32, resulting in printout of a negative number.

byte is a signed two's complement integer. So this is a normal behavior: the two's complement representation of a negative number has a 1 in the most-significant bit. You could think of it like a sign bit.
If you don't like this, you can use the following idiom:
System.out.println( octet & 0xFF );
This will pass the byte as an int while preventing sign extension. You'll get an output as if it were unsigned.
Java doesn't have unsigned types, so the only other thing you could do is store the numbers in a wider representation, e.g. short.

In Java, all integers are signed, and the most significant bit is the sign bit.

Because parseInt parse signed int that means it converts the binary if it begins with 0 its positive and if 1 its negative try to use parseUnsignedInt instead

Unable to parse 64 bit binary numbers to long

Given binary number in a string "0", I converted it to long to find its Bitwise Not/Complement.
long number = Long.parseLong("0",2);
number = ~number;
System.out.println(Long.toBinaryString(number));
which prints
1111111111111111111111111111111111111111111111111111111111111111
i.e., 64 1's. But I'm unable to find complement of this.
Long.parseLong("111111111111111111111111111111111111111111111111111111111111111",2); //fails
I get java.lang.NumberFormatException. What am I to do?

When you invert zero
number = ~number
you get negative one. The Long.parseLong(String, int) method expects negative numbers to be represented with a minus prefix. When you pass 64 1-s to the method, it thinks it's an overflow, and returns an error.
One way to fix this is to check that the length is less than 64 before you parse the value. If the length is exactly 64, chop off the first digit, and parse the rest of the number. Then check the initial digit. If it is zero, leave the parsed number as is; otherwise, use binary OR to set the most significant bit:
String s = "1111111111111111111111111111111111111111111111111111111111111111";
long res;
if (s.length() < 64) {
res = Long.parseLong(s, 2);
} else {
res = Long.parseLong(s.substring(1), 2);
if (s.charAt(0) == '1') {
res |= (1L << 63);
}
}

The complement of 0 is 64 1's, which is equivalent to -1, since Java uses two's complement.
Long.parseLong(String, int)
expects a signed long (aka if the number is negative, it expects a leading -), but you are passing it 64 1's, which are supposed to represent -1, but do not in this form.
Given that for negatives, it expects the a negative sign, passing it 64 1's causes the it to believe that the number is too large.
EDIT (explanation of dasblinkenlight's fix: couldn't properly format in comment):
So if String s =
"1111111111111111111111111111111111111111111111111111111111111111";
, and we have:
long res = Long.parseLong(s.substring(1), 2);
The binary form of res is:
0111111111111111111111111111111111111111111111111111111111111111
Now, if we know that the first char of s is '1', then we do the following:
res |= (1L << 63);
(1L << 63) produces:
1000000000000000000000000000000000000000000000000000000000000000
So, the bitwise-or assignment to res yields 64 1's, which in two's complement is -1, as desired.

This is because Long.parseLong (as well as Integer.parseInt etc) cannot parse two's complements and for it "111111111111111111111111111111111111111111111111111111111111111" is a positive number that exceeds Long.MAX_VALUE. But we can use BigInteger
long l = new BigInteger("1111111111111111111111111111111111111111111111111111111111111111", 2).longValue()
this produces expected result = -1

Java converting int to hex and back again

I have the following code...
int Val=-32768;
String Hex=Integer.toHexString(Val);
This equates to ffff8000
int FirstAttempt=Integer.parseInt(Hex,16); // Error "Invalid Int"
int SecondAttempt=Integer.decode("0x"+Hex); // Error "Invalid Int"
So, initially, it converts the value -32768 into a hex string ffff8000, but then it can't convert the hex string back into an Integer.
In .Net it works as I'd expect, and returns -32768.
I know that I could write my own little method to convert this myself, but I'm just wondering if I'm missing something, or if this is genuinely a bug?

int val = -32768;
String hex = Integer.toHexString(val);
int parsedResult = (int) Long.parseLong(hex, 16);
System.out.println(parsedResult);
That's how you can do it.
The reason why it doesn't work your way: Integer.parseInt takes a signed int, while toHexString produces an unsigned result. So if you insert something higher than 0x7FFFFFF, an error will be thrown automatically. If you parse it as long instead, it will still be signed. But when you cast it back to int, it will overflow to the correct value.

It overflows, because the number is negative.
Try this and it will work:
int n = (int) Long.parseLong("ffff8000", 16);

int to Hex :
Integer.toHexString(intValue);
Hex to int :
Integer.valueOf(hexString, 16).intValue();
You may also want to use long instead of int (if the value does not fit the int bounds):
Hex to long:
Long.valueOf(hexString, 16).longValue()
long to Hex
Long.toHexString(longValue)

It's worth mentioning that Java 8 has the methods Integer.parseUnsignedInt and Long.parseUnsignedLong that does what you wanted, specifically:
Integer.parseUnsignedInt("ffff8000",16) == -32768
The name is a bit confusing, as it parses a signed integer from a hex string, but it does the work.

Try using BigInteger class, it works.
int Val=-32768;
String Hex=Integer.toHexString(Val);
//int FirstAttempt=Integer.parseInt(Hex,16); // Error "Invalid Int"
//int SecondAttempt=Integer.decode("0x"+Hex); // Error "Invalid Int"
BigInteger i = new BigInteger(Hex,16);
System.out.println(i.intValue());

As Integer.toHexString(byte/integer) is not working when you are trying to convert signed bytes like UTF-16 decoded characters you have to use:
Integer.toString(byte/integer, 16);
or
String.format("%02X", byte/integer);
reverse you can use
Integer.parseInt(hexString, 16);

Java's parseInt method is actally a bunch of code eating "false" hex : if you want to translate -32768, you should convert the absolute value into hex, then prepend the string with '-'.
There is a sample of Integer.java file :
public static int parseInt(String s, int radix)
The description is quite explicit :
* Parses the string argument as a signed integer in the radix
* specified by the second argument. The characters in the string
...
...
* parseInt("0", 10) returns 0
* parseInt("473", 10) returns 473
* parseInt("-0", 10) returns 0
* parseInt("-FF", 16) returns -255

Using Integer.toHexString(...) is a good answer. But personally prefer to use String.format(...).
Try this sample as a test.
byte[] values = new byte[64];
Arrays.fill(values, (byte)8); //Fills array with 8 just for test
String valuesStr = "";
for(int i = 0; i < values.length; i++)
valuesStr += String.format("0x%02x", values[i] & 0xff) + " ";
valuesStr.trim();

Below code would work:
int a=-32768;
String a1=Integer.toHexString(a);
int parsedResult=(int)Long.parseLong(a1,16);
System.out.println("Parsed Value is " +parsedResult);

Hehe, curious. I think this is an "intentianal bug", so to speak.
The underlying reason is how the Integer class is written. Basically, parseInt is "optimized" for positive numbers. When it parses the string, it builds the result cumulatively, but negated. Then it flips the sign of the end-result.
Example:
66 = 0x42
parsed like:
4*(-1) = -4
-4 * 16 = -64 (hex 4 parsed)
-64 - 2 = -66 (hex 2 parsed)
return -66 * (-1) = 66
Now, let's look at your example
FFFF8000
16*(-1) = -16 (first F parsed)
-16*16 = -256
-256 - 16 = -272 (second F parsed)
-272 * 16 = -4352
-4352 - 16 = -4368 (third F parsed)
-4352 * 16 = -69888
-69888 - 16 = -69904 (forth F parsed)
-69904 * 16 = -1118464
-1118464 - 8 = -1118472 (8 parsed)
-1118464 * 16 = -17895552
-17895552 - 0 = -17895552 (first 0 parsed)
Here it blows up since -17895552 < -Integer.MAX_VALUE / 16 (-134217728).
Attempting to execute the next logical step in the chain (-17895552 * 16)
would cause an integer overflow error.
Edit (addition): in order for the parseInt() to work "consistently" for -Integer.MAX_VALUE <= n <= Integer.MAX_VALUE, they would have had to implement logic to "rotate" when reaching -Integer.MAX_VALUE in the cumulative result, starting over at the max-end of the integer range and continuing downwards from there. Why they did not do this, one would have to ask Josh Bloch or whoever implemented it in the first place. It might just be an optimization.
However,
Hex=Integer.toHexString(Integer.MAX_VALUE);
System.out.println(Hex);
System.out.println(Integer.parseInt(Hex.toUpperCase(), 16));
works just fine, for just this reason. In the sourcee for Integer you can find this comment.
// Accumulating negatively avoids surprises near MAX_VALUE