We Keep Coding

Big Endian Hex Number conversation

I have the following 4 bytes of Hex String (38 01 02 00) and the expected output is (201.38) Decimal, Like the first input is reversed.
If (89 00 00 00) is given then the expected outcome would be (0.89)
I don't know the mathematical name of this conversation.
I have tried converting Big-endian to little-endian but the outcome has failed with (00020138).
I have tried writing a simple method but the outcome is still wrong (1492992770)
int htonl(final int value) {
return ByteBuffer
.allocate(4)
.putInt(value).order(ByteOrder.nativeOrder())
.getInt(0);
}

It seems to be BCD encoding (Binary Coded Decimal) where each 4-byte group represents one decimal place.
And the byte-order is little endian - it's beginning with the least significant byte.
All I see suggests that the decimal point is in a fixed position, but that's just a guess.
So, the result of your first try comes close. Just divide it by 100, and you're done.

Solved it with the help of #Ralf Kleberhoff
I reversed the hex input first
Divided the output with 100
public static String reverse(final String originalHex) {
final int lengthInBytes = originalHex.length() / 2;
final char[] chars = new char[lengthInBytes * 2];
for (int index = 0; index < lengthInBytes; index++) {
final int reversedIndex = lengthInBytes - 1 - index;
chars[reversedIndex * 2] = originalHex.charAt(index * 2);
chars[reversedIndex * 2 + 1] = originalHex.charAt(index * 2 + 1);
}
return new String(chars);
}
final float f = Float.parseFloat(reverse("89000000"));
System.out.println(f / 100);
Given input is 89000000 and result was 0.89 which was expected.

Inverting colours of a greyscale image (in java) stored as a byte array

I have an array of bytes, byte bytes[], representing a greyscale image.
I want to invert the colours of this image - so figured I would just flip the bits (bitwise not).
I have attempted this as seen below (have included an extra loop to populate a dummy byte array to allow quick and easy re-creation of my problem)
Random rand = new Random();
byte bytes[] = new byte[500];
// populate a dummy byte array
for (int i = 0; i < bytes.length; ++i)
{
new Random().nextBytes(bytes);
System.out.println(bytes[i]);
}
for (int i = 0; i < bytes.length; ++i)
{
System.out.print(bytes[i] + " ");
//bytes[i] = ~bytes[i]; This throws an error as apparantly java treats a byte array as an integer array?!
bytes[i] = (byte)~bytes[i]; // This compiles but output not a bitwise not, results displayed below
System.out.println(bytes[i]);
}
The results I am getting are:
116 -117
48 -49
70 -71
What I'm looking for is: (I have added the binary manually to fuly illustrate what my understanding of bitwise not (please correct if wrong)
116 (01110100) = (10001011) 139
48 (00110000) = (11001111) 207
70 (01000110) = (10111001) 185
Thanks in advance for any advice

You can XOR the value with 255, so the particular line should be
bytes[i] = (byte) (bytes[i] ^ 0xff);

10001011 actually represents -117 (in two's complement format: https://en.wikipedia.org/wiki/Two%27s_complement ); Java byte type is signed. You probably want unsigned byte values as output (which range from 0 to 255).
As Java byte type is signed, it can only have values from -128 to 127. If you want to have values between 0 and 255 (like 139), you have to use some other type, for example short:
byte b = 116;
short sh = (short) (((short) b) ^ 0xff);
System.out.println(sh);
This produces 139.

You can leave it as it is. It's already correct, just printed in a confusing way.
For example note that (byte)207 will (by default) be printed as -49. But it's still the same value, it's just printed with a different interpretation of the most significant bit.
In short, change nothing.

java
public static byte[] bitwiseNot(byte[] array) {
byte[] bytes = new byte[array.length];
for (int i = 0; i < array.length; i++) {
bytes[i] = (byte) (array[i] ^ 0xff);
}
return bytes;
}
kotlin xor syntax
import kotlin.experimental.xor
fun bitwiseNot(array: ByteArray) : ByteArray {
return array.map{ b -> b xor 0xFF.toByte() }.toByteArray()
}
kotlin with Byte.inv() syntax
import kotlin.experimental.inv
fun bitwiseNot(array: ByteArray) : ByteArray {
return array.map{ b -> b.inv() }.toByteArray()
}
If you want to convert to BufferedImage

First Byte's Bit Off-By-One after Steganography

Currently working on a Steganography project where, given a message in bytes and the number of bits to modify per byte, hide a message in an arbitrary byte array.
In the first decoded byte of the resulting message, the value has it's first (leftmost) bit set to '1' instead of '0'. For example, when using message "Foo".getBytes() and maxBits = 1 the result is "Æoo", not "Foo" (0b01000110 gets changed to 0b11000110). With message "Æoo".getBytes() and maxBits = 1 result is "Æoo", meaning the bit is not getting flipped as far as I can tell.
Only certain values of maxBits for certain message bytes cause this error, for example "Foo" encounters this problem at maxBits equal to 1, 5, and 6, whereas "Test" encounters this problem at maxBits equal to 1, 3, and 5. Only the resulting first character ends up with its first bit set, and this problem only occurs at the specified values of this.maxBits related to the initial data.
Why, for certain values of maxBits, is the first bit of the
resulting decoded message always 1?
Why do different inputs have different values for maxBits that
work fine, and others that do not?
What is the pattern with the value of maxBits and the
resulting erroneous results in relation to the original data?
Encode and Decode Methods:
public byte[] encodeMessage(byte[] data, byte[] message) {
byte[] encoded = data;
boolean[] messageBits = byteArrToBoolArr(message);
int index = 0;
for (int x = 0; x < messageBits.length; x++) {
encoded[index] = messageBits[x] ? setBit(encoded[index], x % this.maxBits) : unsetBit(encoded[index], x % this.maxBits);
if (x % this.maxBits == 0 && x != 0)
index++;
}
return encoded;
}
public byte[] decodeMessage(byte[] data) {
boolean[] messageBits = new boolean[data.length * this.maxBits];
int index = 0;
for (int x = 0; x < messageBits.length; x++) {
messageBits[x] = getBit(data[index], x % this.maxBits);
if (x % this.maxBits == 0 && x != 0)
index++;
}
return boolArrToByteArr(messageBits);
}
Unset, Set, and Get Methods:
public byte unsetBit(byte data, int pos) {
return (byte) (data & ~((1 << pos)));
}
public byte setBit(byte data, int pos) {
return (byte) (data | ((1 << pos)));
}
public boolean getBit(byte data, int pos) {
return ((data >>> pos) & 0x01) == 1;
}
Conversion Methods:
public boolean[] byteArrToBoolArr(byte[] b) {
boolean bool[] = new boolean[b.length * 8];
for (int x = 0; x < bool.length; x++) {
bool[x] = false;
if ((b[x / 8] & (1 << (7 - (x % 8)))) > 0)
bool[x] = true;
}
return bool;
}
public byte[] boolArrToByteArr(boolean[] bool) {
byte[] b = new byte[bool.length / 8];
for (int x = 0; x < b.length; x++) {
for (int y = 0; y < 8; y++) {
if (bool[x * 8 + y]) {
b[x] |= (128 >>> y);
}
}
}
return b;
}
Sample Code and Output:
test("Foo", 1);//Æoo
test("Foo", 2);//Foo
test("Foo", 3);//Foo
test("Foo", 4);//Foo
test("Foo", 5);//Æoo
test("Foo", 6);//Æoo
test("Foo", 7);//Foo
test("Foo", 8);//Foo
test("Test", 1);//Ôest
test("Test", 2);//Test
test("Test", 3);//Ôest
test("Test", 4);//Test
test("Test", 5);//Ôest
test("Test", 6);//Test
test("Test", 7);//Test
test("Test", 8);//Test
private static void test(String s, int x) {
BinaryModifier bm = null;
try {
bm = new BinaryModifier(x);//Takes maxBits as constructor param
} catch (BinaryException e) {
e.printStackTrace();
}
System.out.println(new String(bm.decodeMessage(bm.encodeMessage(new byte[1024], s.getBytes()))));
return;
}

Your logic of incrementing index has two flaws, which overwrite the first bit of the first letter. Obviously, the bug is expressed when the overwriting bit is different to the first bit.
if (x % this.maxBits == 0 && x != 0)
index++;
The first problem has to do with embedding only one bit per byte, i.e. maxBits = 1. After you have embedded the very first bit and reached the above conditional, x is still 0, since it will be incremented at the end of the loop. You should be incrementing index at this point, but x != 0 prevents you from doing so. Therefore, the second bit will also be embedded in the first byte, effectively overwriting the first bit. Since this logic also exists in the decode method, you read the first two bits from the first byte.
More specifically, if you embed a 00 or 11, it will be fine. But a 01 will be read as 11 and a 10 will be read as 00, i.e., whatever value is the second bit. If the first letter has an ascii code less or equal than 63 (00xxxxxx), or greater or equal than 192 (11xxxxxx), it will come out fine. For example:
# -> # : 00100011 (35) -> 00100011 (35)
F -> Æ : 01000110 (70) -> 11000110 (198)
The second problem has to do with the x % this.maxBits == 0 part. Consider the case where we embed 3 bits per byte. After the 3rd bit, when we reach the conditional we still have x = 2, so the modulo operation will return false. After we have embedded a 4th bit, we do have x = 3 and we're allowed to move on to the next byte. However, this extra 4th bit will be written at the 0th position of the first byte, since x % this.maxBits will be 3 % 3. So again, we have a bit overwriting our very first bit. However, after the first cycle the modulo operation will correctly write only 3 bits per byte, so the rest of our message will be unaffected.
Consider the binary for "F", which is 01000110. By embedding N bits per byte, we effectively embed the following groups in the first few bytes.
1 bit 01 0 0 0 1 1 0
2 bits 010 00 11 0x
3 bits 0100 011 0xx
4 bits 01000 110x
5 bits 010001 10xxxx
6 bits 0100011 0xxxxx
7 bits 01000110
8 bits 01000110x
As you can see, for groups of 5 and 6 bits, the last bit of the first group is 1, which will overwrite our initial 0 bit. For all other cases the overwrite doesn't affect anything. Note that for 8 bits, we end up using the first bit of the second letter. If that happened to have an ascii code greater or equal than 128, it would again overwrite the firstmost 0 bit.
To address all problems, use either
for (int x = 0; x < messageBits.length; x++) {
// code in the between
if ((x + 1) % this.maxBits == 0)
index++;
}
or
for (int x = 0; x < messageBits.length; ) {
// code in the between
x++;
if (x % this.maxBits == 0)
index++;
}
Your code has another potential problem which hasn't been expressed. If your data array has a size of 1024, but you only embed 3 letters, you will affect only the first few bytes, depending on the value of maxBits. However, for the extraction, you define your array to have a size of data.length * this.maxBits. So you end up reading bits from all of the bytes of the data array. This is currently no problem, because your array is populated by 0s, which are converted to empty strings. However, if your array had actual numbers, you'd end up reading a lot of garbage past the point of your embedded data.
There are two general ways of addressing this. You either
append a unique sequence of bits at the end of your message (marker), such that when you encounter that sequence you terminate the extraction, e.g. eight 0s, or
you add a few bits before embedding your actual data (header), which will tell you how to extract your data, e.g., how many bytes and how many bits per byte to read.

One thing you're probably going to run afoul of is the nature of character encoding.
When you call s.getBytes() you are turning the string to bytes using your JVM's default encoding. Then you modify the bytes and you create a new String from the modified bytes again using the default encoding.
So the question is what is that encoding and precisely how does it work. For example, the encoding may well in some cases only be looking at the lower 7 bits of a byte relating to the character, then your setting of the top bit won't have any effect on the string created from the modified bytes.
If you really want to tell if your code is working right, do your testing by directly examining the byte[] being produced by your encode and decode methods, not by turning the modified bytes into strings and looking at the strings.

Byte to "Bit"array

A byte is the smallest numeric datatype java offers but yesterday I came in contact with bytestreams for the first time and at the beginning of every package a marker byte is send which gives further instructions on how to handle the package. Every bit of the byte has a specific meaning so I am in need to entangle the byte into it's 8 bits.
You probably could convert the byte to a boolean array or create a switch for every case but that can't certainly be the best practice.
How is this possible in java why are there no bit datatypes in java?

Because there is no bit data type that exists on the physical computer. The smallest allotment you can allocate on most modern computers is a byte which is also known as an octet or 8 bits. When you display a single bit you are really just pulling that first bit out of the byte with arithmetic and adding it to a new byte which still is using an 8 bit space. If you want to put bit data inside of a byte you can but it will be stored as a at least a single byte no matter what programming language you use.

You could load the byte into a BitSet. This abstraction hides the gory details of manipulating single bits.
import java.util.BitSet;
public class Bits {
public static void main(String[] args) {
byte[] b = new byte[]{10};
BitSet bitset = BitSet.valueOf(b);
System.out.println("Length of bitset = " + bitset.length());
for (int i=0; i<bitset.length(); ++i) {
System.out.println("bit " + i + ": " + bitset.get(i));
}
}
}
$ java Bits
Length of bitset = 4
bit 0: false
bit 1: true
bit 2: false
bit 3: true
You can ask for any bit, but the length tells you that all the bits past length() - 1 are set to 0 (false):
System.out.println("bit 75: " + bitset.get(75));
bit 75: false

Have a look at java.util.BitSet.
You might use it to interpret the byte read and can use the get method to check whether a specific bit is set like this:
byte b = stream.read();
final BitSet bitSet = BitSet.valueOf(new byte[]{b});
if (bitSet.get(2)) {
state.activateComponentA();
} else {
state.deactivateComponentA();
}
state.setFeatureBTo(bitSet.get(1));
On the other hand, you can create your own bitmask easily and convert it to a byte array (or just byte) afterwards:
final BitSet output = BitSet.valueOf(ByteBuffer.allocate(1));
output.set(3, state.isComponentXActivated());
if (state.isY){
output.set(4);
}
final byte w = output.toByteArray()[0];

How is this possible in java why are there no bit datatypes in java?
There are no bit data types in most languages. And most CPU instruction sets have few (if any) instructions dedicated to adressing single bits. You can think of the lack of these as a trade-off between (language or CPU) complexity and need.
Manipulating a single bit can be though of as a special case of manipulating multiple bits; and languages as well as CPU's are equipped for the latter.
Very common operations like testing, setting, clearing, inverting as well as exclusive or are all supported on the integer primitive types (byte, short/char, int, long), operating on all bits of the type at once. By chosing the parameters appropiately you can select which bits to operate on.
If you think about it, a byte array is a bit array where the bits are grouped in packages of 8. Adressing a single bit in the array is relatively simple using logical operators (AND &, OR |, XOR ^ and NOT ~).
For example, testing if bit N is set in a byte can be done using a logical AND with a mask where only the bit to be tested is set:
public boolean testBit(byte b, int n) {
int mask = 1 << n; // equivalent of 2 to the nth power
return (b & mask) != 0;
}
Extending this to a byte array is no magic either, each byte consists of 8 bits, so the byte index is simply the bit number divided by 8, and the bit number inside that byte is the remainder (modulo 8):
public boolean testBit(byte[] array, int n) {
int index = n >>> 3; // divide by 8
int mask = 1 << (n & 7); // n modulo 8
return (array[index] & mask) != 0;
}

Here is a sample, I hope useful for you!
DatagramSocket socket = new DatagramSocket(6160, InetAddress.getByName("0.0.0.0"));
socket.setBroadcast(true);
while (true) {
byte[] recvBuf = new byte[26];
DatagramPacket packet = new DatagramPacket(recvBuf, recvBuf.length);
socket.receive(packet);
String bitArray = toBitArray(recvBuf);
System.out.println(Integer.parseInt(bitArray.substring(0, 8), 2)); // convert first byte binary to decimal
System.out.println(Integer.parseInt(bitArray.substring(8, 16), 2)); // convert second byte binary to decimal
}
public static String toBitArray(byte[] byteArray) {
StringBuilder sb = new StringBuilder();
for (int i = 0; i < byteArray.length; i++) {
sb.append(String.format("%8s", Integer.toBinaryString(byteArray[i] & 0xFF)).replace(' ', '0'));
}
return sb.toString();
}

Get "XOR Mask" and "AND mask" from Icon

I need to get the XOR Mask and the AND Mask from an Icon(.ICO) file.
If anyone can suggest a way how I can do this from Java, this would be brilliant.
If not, do you know of any application which can get these two masks and allow you
to dump them?

This article, Enhance Java GUIs with Windows Icons, has a good explanation of the format, and there's a link to some source code.

The wikipedia article on Windows Icon Image File format, https://en.m.wikipedia.org/wiki/ICO_(file_format), is very straight forward.
There are three sections, header, entries, and image data. The header is 6 bytes. each entry is 16 bytes. Offset 8 of of each entry structure is a Java integer for the offset to the image data
The image data of 32 bitDepth bitmap has no AND mask. If it is a 24 bitDepth then it needs XOR color mask and 1 bitDepth AND mask.
bytes[] openFile(String fname) throws Exception
{
java.io.InputStream file = new java.io.FileInputStream(fname);
bytes []bytes = new bytes[file.available()];
file.read(bytes);
file.close();
return bytes;
}
//assumes 24 bitDepth
bytes[] getXorMask(bytes []ico)
{
int i = 6 + 8;
i = bytes[i+0] | (bytes[i+1]<<1) | (bytes[i+2]<<2) | (bytes[i+3]<<3);
i += sizeof(BITMAPINFOHEADER); // WOW! NOT JAVA
int width = bytes[6] == 0 ? 256 : bytes[6];
int height = bytes[7] == 0 ? 256 : bytes[7];
int sz = width * height * 3; // very presumptious
byte []bytes = new byte[sz];
for(int e=0; e<sz; ) bytes[e++] = ico[i++];
return bytes;
}
bytes[] getAndMask(bytes []ico)
{
int i = 6 + 8;
i = bytes[i+0] | (bytes[i+1]<<1) | (bytes[i+2]<<2) | (bytes[i+3]<<3);
i += sizeof(BITMAPINFOHEADER); // WOW! NOT JAVA
int width = bytes[6] == 0 ? 256 : bytes[6];
int height = bytes[7] == 0 ? 256 : bytes[7];
int sz = width * height * 3; // very presumptious
byte []bytes = new byte[sz];
i += sz; // seek to monochrome mask
// only works if bounds is multiple of 4
sz = width/8 * height;
for(int e=0; e<sz; ) bytes[e++] = ico[i++];
return bytes;
}
The preceding example always gets the first image entry. Also I thought this was simple implementation but Java knows nothing about BITMAPINFOHEADER and this structure is a variable size structure.
Edit:
Firstly big ups to Raymond Chen. He has a course on Icons (part 1 - part 4) on MSDN.
Further reading at, https://learn.microsoft.com/en-us/previous-versions/ms997538(v=msdn.10)?redirectedfrom=MSDN, shows that the BITMAPINFOHEADER structure should be of fixed size. Check out win32 definition of struct BITMAPINFOHEADER. Size of it is 40 bytes.
Edit:
BITMAPINFOHEADER structure's first DWORD is the size of the structure in little endian.