The essence of the task is this, I encode the bytes of the file, 1 byte of the source file = 4 bytes of the encrypted one. For example, the encoded byte is 3125890409. In byte representation, this is [186, 81, 77, 105]. For decryption, I must present this array as a single number. How can I first convert these 4 numbers to binary, and then to decimal and assign BigIntger? I thought to do it like this:
for(int i = 0; i < fileData2.length; i+=4) {
BigInteger message = BigInteger.valueOf(fileData2[i]);
BigInteger message2 = BigInteger.valueOf(fileData2[i + 1]);
BigInteger message3 = BigInteger.valueOf(fileData2[i + 2]);
BigInteger message4 = BigInteger.valueOf(fileData2[i + 3]);
}
And then translate each into binary, but it looks too complicated, and what if you need to do not 4 bytes, but 8 bytes and higher. How can it be implemented faster?
Do not bother with BigInteger or BigDecimals. Instead think of your original byte as one byte (or 8 bits), and the encoded/encrypted values as 4 bytes (32 bits). If you just shove them into a byte[] you can choose an arbitrary size - be it 4, 8, or 13. Lots of flexibility there.
This approach makes it also easier to read/write the data, as you might simply serialize the bytes into a stream//read the data from a stream, as it feels quite natural to read and write the low indexes of the array first.
Once done with that, all you have to focus on is your function to turn 8 bits into 32 bits and vice versa.
4 bytes fit in an int. Though a negative one here. Use a long otherwise. The byte order seems to be big endian, most significant byte first - it is an odd number as is the last byte.
byte[] fileData = {(byte)186, 81, 77, 105};
int n = new BigInteger(fileData).intValue();
int n = ByteBuffer.wrap(fileData).byteOrder(ByteOrder.BIG_ENDIAN).getInt();
long nn = n & 0xFF_FF_FF_FFL;
BIG_ENDIAN is the default, so the byteOrder is not needed here.
Related
What I want to do is take a decimal integer, convert it into hexadecimal, and then separate the bytes.
It's my understanding that ByteBuffer is the best way to do this. The integer will not exceed 65535, so the hex number is guaranteed to be 2 bytes. For an example, I have an integer of 40000 (hex value 9C40).
int n1 = 40000;
ByteBuffer b = ByteBuffer.allocate(2);
b.putInt(n1);
However, I get the following error when I run the program:
Exception in thread "main" java.nio.BufferOverflowException
What am I doing wrong? Shouldn't 9C40 be written into b (with b[0] = 9C and b[1] = 40)?
Also, once I get past this, if I want to convert the value stored in b[0] (which is 9C) to decimal (which is 156), would I just use the following code?
int n2 = b.get(0);
As you are working with a ByteBuffer, it stores an amount of x allocated bytes. Now you allocated 2 bytes and you try to store a datatype that has the size of 4 bytes. So the buffer will run out of bounds as the message said. If you want to store this data in a two byte sized buffer, you either use a short (16 bit - 2 bytes) or you allocate 4 bytes for your ByteBuffer.
With short:
ByteBuffer bb = ByteBuffer.allocate(2);
short myShort = (short) 40000;
bb.putShort(myShort);
System.out.println(String.format("%02X, %02X", bb.get(0), bb.get(1)));
With int:
ByteBuffer bb = ByteBuffer.allocate(4);
int myInt = 40000;
bb.putInt(myInt);
System.out.println(String.format("%02X, %02X", bb.get(2), bb.get(3)));
Output: 9C, 40
The data type you used to store the number 40000 is int, which requires 4 bytes of space. Yes I know the number won't exceed 65535 but the computer doesn't. You have to change it to an appropriate data type that can be stored in 2 bytes.
That data type, is short.
But there's another problem if you used short, you can't really store 40000 in short in Java is signed, so its max value is 32767.
So to store your 40000, you have to store -25536 instead in a short, because of overflow.
short n1 = (short)40000; // this will cause n1 to store -25536
ByteBuffer b = ByteBuffer.allocate(2);
b.putShort(n1);
Now it's time to print out the bytes. Bytes in Java are signed as well. So if you print this:
System.out.println(b.get(0));
System.out.println(b.get(1));
You'd get
-100
64
64 is expected since 64 in hex is 40, but why -100? Since bytes are signed, 156 can't be represented as 156. 156 in a signed byte is -100.
Instead of ByteBuffer I prefer Integer class which can convert the integer value to hex string & you can get each byte by index of method.
Use following code it do that
int n = 4000;
String hex = Integer.toHexString(n);
In this way you can get the hex value of any integer for one one byte use indexOf() method of string clas
You can get return the hex value as integer using valueOf() method in Integer class which takes two arguments one is string and another is radix
I have an integer called writePos that takes a value between [0,1023]. I need to store it in the last two bytes of a byte array called bucket. So, I figure I need to represent it as a concatenation of the array's last two bytes.
How would I go about breaking down writePos into two bytes that, when concatenated and cast into an int, produces writePos again?
How would I go about concatenating once I get it broken down into the bytes?
This would be covered high-level by a ByteBuffer.
short loc = (short) writeLocation;
byte[] bucket = ...
int idex = bucket.length - 2;
ByteBuffer buf = ByteBuffer.wrap(bucket);
buf.order(ByteOrder.LITTLE__ENDIAN); // Optional
buf.putShort(index, loc);
writeLocation = buf.getShort(index);
The order can be specified, or left to the default (BIG_ENDIAN).
The ByteBuffer wraps the original byte array, and changes to ByteBuffer effect on the byte array too.
One can use sequential writing and reading an positioning (seek), but here I use overloaded methods for immediate positioning with index.
putShort writes to the byte array, modifying two bytes, a short.
getShort reads a short from the byte array, which can be put in an int.
Explanation
A short in java is a two-byte (signed) integral number. And that is what is meant. The order is whether LITTLE_ENDIAN: least significant byte first (n % 256, n / 256) or big endian.
Bitwise operations.
To byte:
byte[] bytes = new byte[2];
// This uses a bitwise and (&) to take only the last 8 bits of i
byte[0] = (byte)(i & 0xff);
// This uses a bitwise and (&) to take the 9th to 16th bits of i
// It then uses a right shift (>>) then move them right 8 bits
byte[1] = (byte)((i & 0xff00) >> 8);from byte:
To go back the other way
// This just reverses the shift, no need for masking.
// The & here is used to handle complications coming from the sign bit that
// will otherwise be moved as the bytes are combined together and converted
// into an int
i = (byte[0] & 0xFF)+(byte[1] & 0xFF)<<8;
There is a working example here of some of the conversions that you can play around with:
http://ideone.com/eRzsun
You need to split the integer into two bytes. The high and the low byte. Following your description it's stored as bug endian in the array.
int writeLocation = 511;
byte[] bucket = new byte[10];
// range checks must be done before
// bitwise right rotation by 8 bits
bucket[8] = (byte) (writeLocation >> 8); // the high byte
bucket[9] = (byte) (writeLocation & 0xFF); // the low byte
System.out.println("bytes = " + Arrays.toString(bucket));
// convert back the integer value 511 from the two bytes
bucket[8] = 1;
bucket[9] = (byte) (0xFF);
// the high byte will bit bitwise left rotated
// the low byte will be converted into an int
// and only the last 8 bits will be added
writeLocation = (bucket[8] << 8) + (((int) bucket[9]) & 0xFF);
System.out.println("writeLocation = " + writeLocation);
As part of a program I'm writing I need to consolidate two bytes into a long from an array of bytes.
So assuming this:
byte a[] = new byte[2]
a[0] = 0b1000111
a[1] = 0b1111000
how can I consolidate them such that
long b = 0b10001111111000
EDIT: The program will attempt to consolidate anywhere between 2 bytes and 100, just for reference.
java.nio can do that:
ByteBuffer.wrap(a).getShort()
BigInteger can do this: http://docs.oracle.com/javase/6/docs/api/java/math/BigInteger.html
BigInteger buffer = new BigInteger(yourByteArray);
long result = buffer.longValue();
Note that if your byte array has a value larger than a long (eg, you mentioned that you might go up to 100 bytes), toLong() will only return the value of the lower value bytes. BigInteger, however, will handle any arbitrary number of bytes.
In order to combine those two bytes you will only need 16 bits. A short will suffice:
short result = (short)(((a[0] & 0xff) << 8) | (a[1] & 0xff))
If for some reason, you need the long data type, you need only replace the cast with a long.
What's a nice, readable way of getting the byte representation (i.e. a byte[]) of an int, but only using 3 bytes (instead of 4)? I'm using Hadoop/Hbase and their Bytes utility class has a toBytes function but that will always use 4 bytes.
Ideally, I'd also like a nice, readable way of encoding to as few bytes as possible, i.e. if the number fits in one byte then only use one.
Please note that I'm storing this in a byte[], so I know the length of the array and thus variable length encoding is not necessary. This is about finding an elegant way to do the cast.
A general solution for this is impossible.
If it were possible, you could apply the function iteratively to obtain unlimited compression of data.
Your domain might have some constraints on the integers that allow them to be compressed to 24-bits. If there are such constraints, please explain them in the question.
A common variable size encoding is to use 7 bits of each byte for data, and the high bit as a flag to indicate when the current byte is the last.
You can predict the number of bytes needed to encode an int with a utility method on Integer:
int n = 4 - Integer.numberOfLeadingZeros(x) / 8;
byte[] enc = new byte[n];
while (n-- > 0)
enc[n] = (byte) ((x >>> (n * 8)) & 0xFF);
Note that this will encode 0 as an empty array, and other values in little-endian format. These aspects are easily modified with a few more operations.
If you need to represent the whole 2^32 existing 4-byte integers, you need to chose between:
fixed-size representation, using 4 bytes always; or
variable-size representation, using at least 5 bytes for some numbers.
Take a look on how UTF-8 encodes the Unicode charactes, you might get some insights. (you use some short prefix to describe how many bytes must be read for that unicode character, then you read that many bytes and interpret them).
Try using ByteBuffer. You can even set little endian mode if required:
int exampleInt = 0x11FFFFFF;
ByteBuffer buf = ByteBuffer.allocate(Integer.SIZE / Byte.SIZE);
final byte[] threeByteBuffer = new byte[3];
buf.putInt(exampleInt);
buf.position(1);
buf.get(threeByteBuffer);
Or the shortest signed, Big Endian:
BigInteger bi = BigInteger.valueOf(exampleInt);
final byte[] shortestSigned = bi.toByteArray();
Convert your int to a 4 bytes array, and iterate it, if every high order byte is zero then remove it from array.
Something like:
byte[] bytes = toBytes(myInt);
int neededBytes = 4;
for (;neededBytes > 1; i--) {
if (bytes[neededBytes - 1] != 0) {
break;
}
}
byte[] result = new byte[neededBytes];
// then just use array copy to copy first neededBytes to result.
You can start with something like this:
byte[] Convert(int i)
{ // warning: untested
if (i == 0)
return new byte[0];
if (i > 0 && i < 256)
return new byte[]{(byte)i};
if (i > 0 && i < 256 * 256)
return new byte[]{(byte)i, (byte)(i >> 8)};
if (i > 0 && i < 256 * 256 * 256)
return new byte[]{(byte)i, (byte)(i >> 8), (byte)(i >> 16)};
return new byte[]{(byte)i, (byte)(i >> 8), (byte)(i >> 16), (byte)(i >> 24)};
}
You'll need to decide if you want to be little-endian or big-endian. Note that negative numbers are encoded in 4 bytes.
If i understand right that you really, desperately want to save space, even at expense of arcane bit shuffling: any array type is an unecessary luxury because you cannot use less than one whole byte for the length = addressing space 256 while you know that at most 4 will be needed. So i would reserve 4 bits for the length and sign flag and cram the rest aligned to that number of bytes. You might even save one more byte if your MSB is less than 128. The sign flag i see useful for ability to represent negative numbers in less than 4 bytes too. Better have the bit there every time (even for positive numbers) than overhead of 4 bytes for representing -1.
Anyway, this all is a thin water until you make some statistics on your data set, how many integers are actually compressible and whether the compression overhead is worth the effort.
To convert an int into a byte array, I'm using the following code:
int a = 128;
byte[] b = convertIntValueToByteArray(a);
private static byte[] convertIntValueToByteArray(int intValue){
BigInteger bigInteger = BigInteger.valueOf(intValue);
byte[] origByteArray = bigInteger.toByteArray();
byte[] noSignByteArray = new byte[bigInteger.bitLength()/8];
if(bigInteger.bitLength()%8!=0){
noSignByteArray = origByteArray;
}else{
System.arraycopy(origByteArray,1,noSignByteArray,0,noSignByteArray.length);
}
return noSignByteArray;
}
There are two things which I'm attempting to do.
1)I need to know the number of bytes (rounded up to the closes byte) of the original integer. However, I don't need the additional bit that is added for the sign bit when I call the toByteArray() method. This is the reason why I have the helper method. So in this example, if I don't have the helper method, when I convert 128 to a byte array I get the length to be 2 octets because of the sign bit but I'm only expecting it to be one octet.
2)I need the positive representation of the number. In this example, if I attempt to print the first element in array b, I get -128. However, the numbers I will be using will be positive numbers only so what I actually want is 128. I'm limited to using a byte array. Is there a way to accomplish this?
Updated Post
Thank you for the responses. I haven't found the exact answer I was looking for so I'll attempt to give more details. Ultimately, I want to write values of different types over a data output stream. In this post, I'd like to clarify what happens when ints are written to a data output stream. I've come across two scenarios.
1)
DataOutputStream os = new DataOutputStream(this.socket.getOutputStream());
byte[] b = BigInteger.valueOf(128).toByteArray();
os.write(b);
2)
DataOutputStream os = new DataOutputStream(this.socket.getOutputStream());
os.write(128);
In the first scenario, when the bytes are read from a data input stream, it seems that the first element in the byte array is a 0 to represent the msb and the second element in the array contains the number -128. However, since the msb is 0 we would be able to determine that it is intended to be a positive number. In the second scenario, there is no msb and the only element present in the byte array read from the input stream is -128. I was expecting the write() method of the data output stream to convert the int into the byte array in the same manner as the toByteArray() method does on a BigInteger object. However, this doesn't seem to be the case as the msb is not present. So my question is, how in the second scenario are we supposed to know that 128 is supposed to be a positive number and not a negative one if there is no msb.
As you probably already know
In an octet, the pattern 10000000 can be interpreted as either 128 or -128, depending on the, um, outside interpretation
Java's byte type interprets octects as values in -128...127 only.
If you are building an application in which the entire world consists of nonnegative integers only, then you could simply do all of your work under the assumption that the byte value -128 will mean 128 and -127 will mean 129 and ... and -1 will mean 255. This is certainly doable but it takes work.
Dealing with the notion of an "unsigned byte" like this is normally done by expanding the byte into a short or int with the higher order bits all set to zero and then performing arithmetic or displaying your values. You will need to decide whether such an approach is more to your liking than just representing 128 as two octets in your array.
I think the following code might be sufficient.
In java int is a twos-complements binary number:
-1 = 111...111
ones complement = 000...000; + 1 =
1 = 000...001
So that about the sign bit I do not understand. Be it, that you could do Math.abs(n).
A byte ranges from -128 to 127, but the interpretation is a matter of masking, as below.
public static void main(String[] args) {
int n = 128;
byte[] bytes = intToFlexBytes(n);
for (byte b: bytes)
System.out.println("byte " + (((int)b) & 0xFF));
}
public static byte[] intToFlexBytes(int n) {
// Convert int to byte[4], via a ByteBuffer:
byte[] bytes = new byte[4];
ByteBuffer bb = ByteBuffer.allocateDirect(4);
bb.asIntBuffer().put(n);
bb.position(0);
bb.get(bytes);
// Leading bytes with 0:
int i = 0;
while (i < 4 && bytes[i] == 0)
++i;
// Shorten bytes array if needed:
if (i != 0) {
byte[] shortenedBytes = new byte[4 - i];
for (int j = i; j < 4; ++j) {
shortenedBytes[j - i] = bytes[j]; // System.arrayCopy not needed.
}
bytes = shortenedBytes;
}
return bytes;
}
To answer your first question—how many bytes are required to represent a nonnegative integer using an unsigned representation—consider the following functions I wrote in Common Lisp.
(defconstant +bits-per-byte+ 8)
(defun bit-length (n)
(check-type n (integer 0) "a nonnegative integer")
(if (zerop n)
1
(1+ (floor (log n 2)))))
(defun bytes-for-bits (n)
(check-type n (integer 1) "a positive integer")
(values (ceiling n +bits-per-byte+)))
These highlight the mathematical underpinnings of the problem: namely, the logarithm tells you how many powers of two (as provided by bits) it takes to dominate a given nonnegative integer, adjusted to be a step function with floor, and the number of bytes it takes to hold that number of bits again as a step function, this time adjusted with ceiling.
Note that the number zero is intolerable as input to a logarithm function, so we avoid it explicitly. You may observe that the bit-length function could also be written with a slight transformation of the core expression:
(defun bit-length-alt (n)
(check-type n (integer 0) "a nonnegative integer")
(values (ceiling (log (1+ n) 2))))
Unfortunately, as the logarithm of one is always zero, regardless of the base, this version says that the integer zero can be represented by zero bits, which isn't the answer we want.
For your second goal, you can use the functions I've defined above to allocate the required number of bytes, and incrementally set the bits you need, ignoring sign. It's hard to tell if you're having trouble getting the proper bits set in the byte vector, or whether your problem is in interpreting the bits in way that avoids treating the high bit as a sign bit (that is, two's complement representation). Please elaborate what kind of push you need to get you moving again.