We Keep Coding

JAVA Byte Manipulation

I want to read a binary file and do some manipulation on each byte. I want to test that I am manipulating the bytes correctly. I want to set a byte variable1 to "00000000" and then another byte variable2 set at "00001111" and OR them newvariable = variable1|variable2, shift the newvariable << 4 bits and then print out the int value.
byte a = 00000000;
//Convert first oneByte to 4 bits and then xor with a;
byte b = 00001111;
byte c = (byte)(a|b);
c = c << 4;
System.out.println("byte= " + c + "\n");
I am not sure why I keep getting "incompatiable types:possible lossy conversion from byte to int"

You need to put a '0b' in front of those numbers to express binary constants. The number 00001111 is interpreted as a literal in octal, which is 585 in decimal. The max byte is 127 (since it's signed). Try 0b00001111 instead.
As literals, those will still be int, so depending on where you do the assignment, you may also need to explicitly cast down to byte.

parseInt on a string of 8 bits returns a negative value when the first bit is 1

I've got a huge string of bits (with some \n in it too) that I pass as a parameter to a method, which should isolate the bits 8 by 8, and convert them all to bytes using parseInt().
Thing is, every time the substring of 8 bits starts with a 1, the resulting byte is a negative number. For example, the first substring is '10001101', and the resulting byte is -115. I can't seem to figure out why, can someone help? It works fine with other substrings.
Here's my code, if needed :
static String bitsToBytes(String geneString) {
String geneString_temp = "", sub;
for(int i = 0; i < geneString.length(); i = i+8) {
sub = geneString.substring(i, i+8);
if (sub.indexOf("\n") != -1) {
if (sub.indexOf("\n") != geneString.length())
sub = sub.substring(0, sub.indexOf("\n")) + sub.substring(sub.indexOf("\n")+1, sub.length()) + geneString.charAt(i+9);
}
byte octet = (byte) Integer.parseInt(sub, 2);
System.out.println(octet);
geneString_temp = geneString_temp + octet;
}
geneString = geneString_temp + "\n";
return geneString;
}

In Java, byte is a signed type, meaning that when the most significant bit it set to 1, the number is interpreted as negative.
This is precisely what happens when you print your byte here:
System.out.println(octet);
Since PrintStream does not have an overload of println that takes a single byte, the overload that takes an int gets called. Since octet's most significant bit is set to 1, the number gets sign-extended by replicating its sign bit into bits 9..32, resulting in printout of a negative number.

byte is a signed two's complement integer. So this is a normal behavior: the two's complement representation of a negative number has a 1 in the most-significant bit. You could think of it like a sign bit.
If you don't like this, you can use the following idiom:
System.out.println( octet & 0xFF );
This will pass the byte as an int while preventing sign extension. You'll get an output as if it were unsigned.
Java doesn't have unsigned types, so the only other thing you could do is store the numbers in a wider representation, e.g. short.

In Java, all integers are signed, and the most significant bit is the sign bit.

Because parseInt parse signed int that means it converts the binary if it begins with 0 its positive and if 1 its negative try to use parseUnsignedInt instead

Standard way to create a hash in Java

The question is about the correct way of creating a hash in Java:
Lets assume I have a positive BigInteger value that I would like to create a hash from. Lets assume that below instance of the messageDigest is a valid instance of (SHA-256)
public static final BigInteger B = new BigInteger("BD0C61512C692C0CB6D041FA01BB152D4916A1E77AF46AE105393011BAF38964DC46A0670DD125B95A981652236F99D9B681CBF87837EC996C6DA04453728610D0C6DDB58B318885D7D82C7F8DEB75CE7BD4FBAA37089E6F9C6059F388838E7A00030B331EB76840910440B1B27AAEAEEB4012B7D7665238A8E3FB004B117B58", 16);
byte[] byteArrayBBigInt = B.toByteArray();
this.printArray(byteArrayBBigInt);
messageDigest.reset();
messageDigest.update(byteArrayBBigInt);
byte[] outputBBigInt = messageDigest.digest();
Now I only assume that the code below is correct, as according to the test the hashes I produce match with the one produced by:
http://www.fileformat.info/tool/hash.htm?hex=BD0C61512C692C0CB6D041FA01BB152D4916A1E77AF46AE105393011BAF38964DC46A0670DD125B95A981652236F99D9B681CBF87837EC996C6DA04453728610D0C6DDB58B318885D7D82C7F8DEB75CE7BD4FBAA37089E6F9C6059F388838E7A00030B331EB76840910440B1B27AAEAEEB4012B7D7665238A8E3FB004B117B58
However I am not sure why we are doing the step below i.e.
because the returned byte array after the digest() call is signed and in this case it is a negative, I suspect that we do need to convert it to a positive number i.e. we can use a function like that.
public static String byteArrayToHexString(byte[] b) {
String result = "";
for (int i=0; i < b.length; i++) {
result += Integer.toString((b[i] & 0xff) + 0x100, 16).substring(1);
}
return result;
}
thus:
String hex = byteArrayToHexString(outputBBigInt)
BigInteger unsignedBigInteger = new BigInteger(hex, 16);
When I construct a BigInteger from the new hex string and convert it back to byte array then I see that the sign bit, that is most significant bit i.e. the leftmost bit, is set to 0 which means that the number is positive, moreover the whole byte is constructed from zeros ( 00000000 ).
My question is: Is there any RFC that describes why do we need to convert the hash always to a "positive" unsigned byte array. I mean even if the number produced after the digest call is negative it is still a valid hash, right? thus why do we need that additional procedure. Basically, I am looking for a paper: standard or rfc describing that we need to do so.

A hash consists of an octet string (called a byte array in Java). How you convert it to or from a large number (a BigInteger in Java) is completely out of the scope for cryptographic hash algorithms. So no, there is no RFC to describe it as there is (usually) no reason to treat a hash as a number. In that sense a cryptographic hash is rather different from Object.hashCode().
That you can only treat hexadecimals as unsigned is a bit of an issue, but if you really want to then you can first convert it back to a byte array, and then perform new BigInteger(result). That constructor does threat the encoding within result as signed. Note that in protocols it is often not needed to convert back and forth to hexadecimals; hexadecimals are mainly for human consumption, a computer is fine with bytes.

Java: have integer, need it in byte in 0x00 format

I have an integer used to seed my for loop:
for(int i = 0; i < value; i++)
Within my for loop I am seeding a byte array with byte content values that increment +1. For instance new byte[]{0x00}; But the 0x00 needs to be 0x01 on the next iteration, how can I convert my value of integer i into a value of byte in the 0x00 format?
I tried things like Byte.valueOf(Integer.toHexString(i)) but this just gives me a value that looks like 0 instead of 0x00.

new byte[]{0x00}
is actually equivalent to
new byte[]{0}
The 0x00 notation is just an alternative way to write integer constants, and if the integer constant is in the range -128 to 127, then it can be used as a byte.
If you have an existing integer variable that you want to use, and its value is in the range -128 to 127, then you just have to cast it:
int i = 1;
new byte[]{(byte)i};

I think the real problem is that you are confused about number representations and text renderings of numbers. Here are some key facts that you need to understand:
The byte type is the set of integral values from -128 to +127.
All integral types use the same representation (2's complement). The difference between the different types is their ranges.
When you "see" a number, what you are seeing is a rendering of the number into a sequence of characters. There are MANY possible renderings; e.g. the number represented in memory as 00101010 (42) can be rendered as "42" or "+42" or "0x2a" or ... "forty two".
The default format for rendering a byte, short, int and long is the same; i.e. an optional minus sign followed by 1 or more decimal digits (with no zero padding). If you want to see your numbers formatted differently, then you need to do the formatting explicitly; e.g. using String.format(...).
So to pull this together, if you want the bytes to look like 0x00 and 0x01 when you output or display them, you need to format them appropriately as you output / display them. In your example code, I doubt that there is anything wrong with the numbers themselves, or with the loop you are using to populate the array.

You are confusing the string representation of the value with the value itself. A value can be represented as binary, decimal, or hex, but it is still the same value.
If you want to use your integer to initialise a byte array, you just need to cast your integer value to a byte value as follows:
arr[i] = (byte) i;

You want
new byte[]{(byte)i}
How you print this array is another matter. Look up printf in the API reference.

I would just like to note that 0 is NOT the same thing as 0x00. If i were to use:
ColorChooserOutputText.append(Integer.toHexString(list[i].getRed()));
ColorChooserOutputText.append(Integer.toHexString(list[i].getGreen()));
ColorChooserOutputText.append(Integer.toHexString(list[i].getBlue()));
and wanted to return the color, Purple it would return: ff0cc Which would be fine if I were just using Java. But if you are going between Java and something that had format specific needs ff0cc would not produce purple.. ff00cc is actually purple.
//Declare some variables.
Color HexColor = JButton1.getBackground();
String MyRValue = null;
String MyGValue = null;
String MyBValue = null;
//Get Hex Value
MyRValue = Integer.toHexString(HexColor.getRed());
MyGValue = Integer.toHexString(HexColor.getGreen());
MyBValue = Integer.toHexString(HexColor.getBlue());
//Make sure to keep both 0's
MyRValue = ("00"+MyRValue).substring(MyRValue.length());
MyGValue = ("00"+MyGValue).substring(MyGValue.length());
MyBValue = ("00"+MyBValue).substring(MyBValue.length());
//Format your HexColor to #00ff00, #000000, #ff00ee
JTextArea1.append("#");
JTextArea1.append(MyRValue+MyGValue+MyBValue);
JTextArea1.append(", ");

String.Format ("%02x") is your friend :)
http://download.oracle.com/javase/1.5.0/docs/api/java/util/Formatter.html
http://www.xinotes.org/notes/note/1195/

C /C++ long long to Java long

I have a file on disk which I'm reading which has been written by c/c++ code. I know I have two 64-bit unsigned integers to read, but Java doesn't support unsigned integers, so the value I get when I do DataInputStream.readLong() is incorrect. (Ignore byte-order for now I'm actually using a derivative of DIS called LEDataInputStream which I downloaded from the web)
A lot of posts on here talk about using BigInteger but the javadoc for reading a bytearray only talks about loading a bytearray respresentation, and the questions seem centered on the fact that some people are going outside the positive bounds of the java long type, which I will be nowhere near with the data I'm reading.
I have a MATLab/Octave script which reads these long long values as two 32-bit integers each, then does some multiplying and adding to get the answer it wants too.
I suppose the question is - how do i read a 64-bit unsigned integer either using BigInteger, or using [LE]DataInputStream.XXX?
Thanks in advance

I would suggest using a ByteBuffer and then using code such as this to get what you want.

You can use a long as a 64-bit value to store unsigned data. Here is a module showing that most Unsigned operations can be performed using the standard long type. It really depends on what you want to do with the value as whether this is problem or not.
EDIT: A common approach to handling unsigned numbers is to widen the data type. This simpler in many cases but not a requirement (and for long using BigInteger doesn't make things any simpler IMHO)
EDIT2: What is wrong with the following code?
long max_unsigned = 0xFFFFFFFFFFFFFFFFl;
long min_unsigned = 0;
System.out.println(Unsigned.asString(max_unsigned) + " > "
+ Unsigned.asString(min_unsigned) + " is "
+ Unsigned.gt(max_unsigned, min_unsigned));
prints
18446744073709551615 > 0 is true

first you check out this question
Also see this
Now use of BigInteger class
// Get a byte array
byte[] bytes = new byte[]{(byte)0x12, (byte)0x0F, (byte)0xF0};
// Create a BigInteger using the byte array
BigInteger bi = new BigInteger(bytes);
// Format to binary
String s = bi.toString(2); // 100100000111111110000
// Format to octal
s = bi.toString(8); // 4407760
// Format to decimal
s = bi.toString(); // 1183728
// Format to hexadecimal
s = bi.toString(16); // 120ff0
if (s.length() % 2 != 0) {
// Pad with 0
s = "0"+s;
}
// Parse binary string
bi = new BigInteger("100100000111111110000", 2);
// Parse octal string
bi = new BigInteger("4407760", 8);
// Parse decimal string
bi = new BigInteger("1183728");
// Parse hexadecimal string
bi = new BigInteger("120ff0", 16);
// Get byte array
bytes = bi.toByteArray();