I'm reading a file into a byte array in chunks and sending it over the network via a POST request to a webserver. It's not anything complicated, I've done it before using this exact same code. This time, I noticed that my images are looking really odd when they get to the server, so I decided to look at the byte array being sent and the one being received just to make sure it was the same. It's not. On the java sending side the byte array contains negative numbers. On the C# receiving side, there are no negative numbers.
The first 15 bytes on the receiving side (C#)
137
80
78
71
13
10
26
10
0
0
0
13
73
72
68
Those same bytes but on the sending side (java)
-119
80
78
71
13
10
26
10
0
0
0
13
73
72
68
All of the non-negative numbers are the same, and the -119 isn't the only negative number, they are all over. I did notice that -119 and 137 are 256 apart and wondered if that has something to do with it.
The code I'm using to read the image (java)
public static byte[] readPart(String fileName, long offset, int length) throws FileNotFoundException, Exception
{
byte[] data = new byte[length];
File file = new File(fileName);
InputStream is = new FileInputStream(file);
is.skip(offset);
is.read(data,0,data.length);
is.close();
return data;
}
The code I'm using to write the data (c#)
private void writeFile(string fileName, Stream contents)
{
using (FileStream fs = new FileStream(fileName, FileMode.Append, FileAccess.Write, FileShare.ReadWrite))
{
int bufferLen = 65000;
byte[] buffer = new byte[bufferLen];
int count = 0;
while ((count = contents.Read(buffer, 0, bufferLen)) > 0)
{
fs.Write(buffer, 0, count);
}
fs.Close();
}
contents.Close();
}
I don't know if that is something that always happens and I just never noticed it before or if it is something that decided to go horribly wrong. What I do know is that this code worked before for something very similar and that it's not working now.
If anyone has any suggestions or an explanation I would really appreciate it.
EDIT:
The reason my images were looking odd is how I was calling the readPart method.
byte[] data = FileUtilities.readPart(fileName,counter,maxFileSize);//counter is the current chunk number
How I should have been calling it
byte[] data = FileUtilities.readPart(fileName,counter*maxFileSize,maxFileSize);//the current chunk * cuhnksize for the offset...
Thanks everyone, I'm significantly less confused now :)
In Java, byte is a signed value (using two's complement to encode negative values), so what you see it correct if unexpected by most people.
To convert a byte to an unsigned int value, use b & 0xff
Java doesn't have unsigned bytes; all bytes are treated as signed. That's all.
All that really matters is how you think of the bytes, since you rarely ever actually need to do comparisons on bytes. The only significant difference is that they print out as signed, as you've discovered.
If you like, you can use e.g. Guava's UnsignedBytes utilities to view Java bytes as unsigned, but there's really not much practical difference.
As a further explanation, assume you have 137 as an unsigned byte. That is represented as:
1000 1001
This binary value, when expressed as a signed two's complement number, turns out to be -119. (-128 + 9)
Any unsigned byte values over 128 will be affected by the difference since the left-most bit is used in this way by the two's complement scheme.
Maybe it has something to do with the fact that Java's byte is signed (range -128 to 127) while C#'s is unsigned (0 to 255) :). The information is the same in binary, it's just interpreted differently.
The range of byte is from -128 to 127, so if you try to assign a byte 128, it will cycle around and the result will be -128.
System.out.println("Max val = " + Byte.MAX_VALUE); //prints: Max val = 127
System.out.println("Min val = " + Byte.MIN_VALUE); //prints: Min val = -128
System.out.println("(byte)137 = " + (byte)137); //prints: (byte)137 = -119
System.out.println("(byte)128 = " + (byte)128); //prints: (byte)128 = -128
System.out.println("(byte)-129 = " + (byte)-129); //prints: (byte)-129 = 127
Related
The essence of the task is this, I encode the bytes of the file, 1 byte of the source file = 4 bytes of the encrypted one. For example, the encoded byte is 3125890409. In byte representation, this is [186, 81, 77, 105]. For decryption, I must present this array as a single number. How can I first convert these 4 numbers to binary, and then to decimal and assign BigIntger? I thought to do it like this:
for(int i = 0; i < fileData2.length; i+=4) {
BigInteger message = BigInteger.valueOf(fileData2[i]);
BigInteger message2 = BigInteger.valueOf(fileData2[i + 1]);
BigInteger message3 = BigInteger.valueOf(fileData2[i + 2]);
BigInteger message4 = BigInteger.valueOf(fileData2[i + 3]);
}
And then translate each into binary, but it looks too complicated, and what if you need to do not 4 bytes, but 8 bytes and higher. How can it be implemented faster?
Do not bother with BigInteger or BigDecimals. Instead think of your original byte as one byte (or 8 bits), and the encoded/encrypted values as 4 bytes (32 bits). If you just shove them into a byte[] you can choose an arbitrary size - be it 4, 8, or 13. Lots of flexibility there.
This approach makes it also easier to read/write the data, as you might simply serialize the bytes into a stream//read the data from a stream, as it feels quite natural to read and write the low indexes of the array first.
Once done with that, all you have to focus on is your function to turn 8 bits into 32 bits and vice versa.
4 bytes fit in an int. Though a negative one here. Use a long otherwise. The byte order seems to be big endian, most significant byte first - it is an odd number as is the last byte.
byte[] fileData = {(byte)186, 81, 77, 105};
int n = new BigInteger(fileData).intValue();
int n = ByteBuffer.wrap(fileData).byteOrder(ByteOrder.BIG_ENDIAN).getInt();
long nn = n & 0xFF_FF_FF_FFL;
BIG_ENDIAN is the default, so the byteOrder is not needed here.
What I want to do is take a decimal integer, convert it into hexadecimal, and then separate the bytes.
It's my understanding that ByteBuffer is the best way to do this. The integer will not exceed 65535, so the hex number is guaranteed to be 2 bytes. For an example, I have an integer of 40000 (hex value 9C40).
int n1 = 40000;
ByteBuffer b = ByteBuffer.allocate(2);
b.putInt(n1);
However, I get the following error when I run the program:
Exception in thread "main" java.nio.BufferOverflowException
What am I doing wrong? Shouldn't 9C40 be written into b (with b[0] = 9C and b[1] = 40)?
Also, once I get past this, if I want to convert the value stored in b[0] (which is 9C) to decimal (which is 156), would I just use the following code?
int n2 = b.get(0);
As you are working with a ByteBuffer, it stores an amount of x allocated bytes. Now you allocated 2 bytes and you try to store a datatype that has the size of 4 bytes. So the buffer will run out of bounds as the message said. If you want to store this data in a two byte sized buffer, you either use a short (16 bit - 2 bytes) or you allocate 4 bytes for your ByteBuffer.
With short:
ByteBuffer bb = ByteBuffer.allocate(2);
short myShort = (short) 40000;
bb.putShort(myShort);
System.out.println(String.format("%02X, %02X", bb.get(0), bb.get(1)));
With int:
ByteBuffer bb = ByteBuffer.allocate(4);
int myInt = 40000;
bb.putInt(myInt);
System.out.println(String.format("%02X, %02X", bb.get(2), bb.get(3)));
Output: 9C, 40
The data type you used to store the number 40000 is int, which requires 4 bytes of space. Yes I know the number won't exceed 65535 but the computer doesn't. You have to change it to an appropriate data type that can be stored in 2 bytes.
That data type, is short.
But there's another problem if you used short, you can't really store 40000 in short in Java is signed, so its max value is 32767.
So to store your 40000, you have to store -25536 instead in a short, because of overflow.
short n1 = (short)40000; // this will cause n1 to store -25536
ByteBuffer b = ByteBuffer.allocate(2);
b.putShort(n1);
Now it's time to print out the bytes. Bytes in Java are signed as well. So if you print this:
System.out.println(b.get(0));
System.out.println(b.get(1));
You'd get
-100
64
64 is expected since 64 in hex is 40, but why -100? Since bytes are signed, 156 can't be represented as 156. 156 in a signed byte is -100.
Instead of ByteBuffer I prefer Integer class which can convert the integer value to hex string & you can get each byte by index of method.
Use following code it do that
int n = 4000;
String hex = Integer.toHexString(n);
In this way you can get the hex value of any integer for one one byte use indexOf() method of string clas
You can get return the hex value as integer using valueOf() method in Integer class which takes two arguments one is string and another is radix
EDITED & SOVLED (below)
I'm using Java for Android trying to send the byte 255 (0xFF in WriteSingleCoil function) to a ModBUS server device.
Device is not runnig, I don't know if because of not able to interpretate the signed byte -1 or because of I'm wrong calculating the CRC.
I don't know how to calculate CRC for negative bytes.
Summarizing: I don't know how to send function 05 Write Single Coil with 0xFF value for switch on the coil for from Java to ModBUS server.
Can anyone help me please?
SOLUTION:
" iIndex = ucCRCLo ^ b: operations like this must be written as iIndex
= (ucCRCLo ^ b)&0xff because the & will cast ucCRCLo, b and the result to int, which is 32 bits while short is 16 so you will have a lot of
extra bits set to 1 "
This answer helped me. Thanks a lot to TheDayOfcondor
But also my huge problem was the usual problem in Java with signed bytes. My CRC calculating function is doing it right for unsigned bytes, but it give errors if I pass inside signed bytes.
The trick for work with bytes for ModBUS comunication is work in the whole App with shorts as bytes, for have the range 0-255, even calculating trames and CRC. And only in the last step, when sending trame to ModBUS sever, cast them to bytes again. This is running.
Hope it will helps to someone in future.
EXPLAINING PROBLEM:
I'm trying to set ON a coil to ModBUS with function 05, this is explaining of function:
Request
I'm tryiing to set ON the coil on address 1:
This hex: 0A 05 00 01 ff 00 DC 81
This byte array: 10 5 0 1 255 0 220 129
10: The Slave Address (10 = 0A hex)
05: The Function Code (Force Single Coil)
0001: The Data Address of the coil. (coil# 1 = 01 hex)
FF00: The status to write ( FF00 = ON, 0000 = OFF )
DC81: The CRC (cyclic redundancy check) for error checking.
The thing is that Java is using signed bytes, so I can't put 255 on my byte array.
I understand I should put -1, but then I can't calculate CRC correctly, because of I have a couple of precalculated array of bytes for get the CRC but the function send a negative index.
So: I don't know if I'm doing right trying to send -1, if I have an alternative for sending 255, neither how to calculate CRC for -1.
This is function for calculate CRC:
public short[] GenerateCRC (byte[] pMsg) {
short ucCRCHi = 0xFF;
short ucCRCLo = 0xFF;
int iIndex;
for (byte b : pMsg)
{
iIndex = ucCRCLo ^ b;
try {
ucCRCLo = (short)(ucCRCHi ^ aucCRCHi[ ( iIndex ) ]);
ucCRCHi = aucCRCLo[ ( iIndex ) ];
} catch (Exception e) {
Log.e(LOGTAG, "GenerateCRC: " + e.toString(), e);
e.printStackTrace();
}
}
short[]result= new short[2];
result0]= ucCRCHi;
result1]= ucCRCLo;
return result;
}
The question is not very clear - however, the most common problem dealing with bytes is the fact Java does not have unsigned bytes, and boolean operation are always between int
The best way to deal with bytes is to use integers, and-ing every operation with 0xff. Also use >>> for the shift right (it is the unsigned version)
Example:
byte b= (byte)(255 & 0xff) // gives you the "unsigned byte"
byte b= (byte) ((b<<2)0xff ) // shift left must be truncated
If you post your code to calculate the CRC I can have a look into it
The best way to define a byte array without using negative numbers is like this:
byte[]={ (byte)0xff, (byte)0xff, (byte)0xff };
What's a nice, readable way of getting the byte representation (i.e. a byte[]) of an int, but only using 3 bytes (instead of 4)? I'm using Hadoop/Hbase and their Bytes utility class has a toBytes function but that will always use 4 bytes.
Ideally, I'd also like a nice, readable way of encoding to as few bytes as possible, i.e. if the number fits in one byte then only use one.
Please note that I'm storing this in a byte[], so I know the length of the array and thus variable length encoding is not necessary. This is about finding an elegant way to do the cast.
A general solution for this is impossible.
If it were possible, you could apply the function iteratively to obtain unlimited compression of data.
Your domain might have some constraints on the integers that allow them to be compressed to 24-bits. If there are such constraints, please explain them in the question.
A common variable size encoding is to use 7 bits of each byte for data, and the high bit as a flag to indicate when the current byte is the last.
You can predict the number of bytes needed to encode an int with a utility method on Integer:
int n = 4 - Integer.numberOfLeadingZeros(x) / 8;
byte[] enc = new byte[n];
while (n-- > 0)
enc[n] = (byte) ((x >>> (n * 8)) & 0xFF);
Note that this will encode 0 as an empty array, and other values in little-endian format. These aspects are easily modified with a few more operations.
If you need to represent the whole 2^32 existing 4-byte integers, you need to chose between:
fixed-size representation, using 4 bytes always; or
variable-size representation, using at least 5 bytes for some numbers.
Take a look on how UTF-8 encodes the Unicode charactes, you might get some insights. (you use some short prefix to describe how many bytes must be read for that unicode character, then you read that many bytes and interpret them).
Try using ByteBuffer. You can even set little endian mode if required:
int exampleInt = 0x11FFFFFF;
ByteBuffer buf = ByteBuffer.allocate(Integer.SIZE / Byte.SIZE);
final byte[] threeByteBuffer = new byte[3];
buf.putInt(exampleInt);
buf.position(1);
buf.get(threeByteBuffer);
Or the shortest signed, Big Endian:
BigInteger bi = BigInteger.valueOf(exampleInt);
final byte[] shortestSigned = bi.toByteArray();
Convert your int to a 4 bytes array, and iterate it, if every high order byte is zero then remove it from array.
Something like:
byte[] bytes = toBytes(myInt);
int neededBytes = 4;
for (;neededBytes > 1; i--) {
if (bytes[neededBytes - 1] != 0) {
break;
}
}
byte[] result = new byte[neededBytes];
// then just use array copy to copy first neededBytes to result.
You can start with something like this:
byte[] Convert(int i)
{ // warning: untested
if (i == 0)
return new byte[0];
if (i > 0 && i < 256)
return new byte[]{(byte)i};
if (i > 0 && i < 256 * 256)
return new byte[]{(byte)i, (byte)(i >> 8)};
if (i > 0 && i < 256 * 256 * 256)
return new byte[]{(byte)i, (byte)(i >> 8), (byte)(i >> 16)};
return new byte[]{(byte)i, (byte)(i >> 8), (byte)(i >> 16), (byte)(i >> 24)};
}
You'll need to decide if you want to be little-endian or big-endian. Note that negative numbers are encoded in 4 bytes.
If i understand right that you really, desperately want to save space, even at expense of arcane bit shuffling: any array type is an unecessary luxury because you cannot use less than one whole byte for the length = addressing space 256 while you know that at most 4 will be needed. So i would reserve 4 bits for the length and sign flag and cram the rest aligned to that number of bytes. You might even save one more byte if your MSB is less than 128. The sign flag i see useful for ability to represent negative numbers in less than 4 bytes too. Better have the bit there every time (even for positive numbers) than overhead of 4 bytes for representing -1.
Anyway, this all is a thin water until you make some statistics on your data set, how many integers are actually compressible and whether the compression overhead is worth the effort.
I need to convert 2 byte array ( byte[2] ) to integer value in java. How can I do that?
You can use ByteBuffer for this:
ByteBuffer buffer = ByteBuffer.wrap(myArray);
buffer.order(ByteOrder.LITTLE_ENDIAN); // if you want little-endian
int result = buffer.getShort();
See also Convert 4 bytes to int.
In Java, Bytes are signed, which means a byte's value can be negative, and when that happens, #MattBall's original solution won't work.
For example, if the binary form of the bytes array are like this:
1000 1101 1000 1101
then myArray[0] is 1000 1101 and myArray[1] is 1000 1101, the decimal value of byte 1000 1101 is -115 instead of 141(= 2^7 + 2^3 + 2^2 + 2^0)
if we use
int result = (myArray[0] << 8) + myArray[1]
the value would be -16191 which is WRONG.
The reason why its wrong is that when we interpret a 2-byte array into integer, all the bytes are
unsigned, so when translating, we should map the signed bytes to unsigned integer:
((myArray[0] & 0xff) << 8) + (myArray[1] & 0xff)
the result is 36237, use a calculator or ByteBuffer to check if its correct(I have done it, and yes, it's correct).
Well, each byte is an integer in the range -128..127, so you need a way to map a pair of integers to a single integer. There are many ways of doing that, depending on what you have encoded in the pair of bytes. The most common will be storing a 16-bit signed integer as a pair of bytes. Converting that back to an integer depends on whether you store it big-endian form:
(byte_array[0]<<8) + (byte_array[1] & 0xff)
or little endian:
(byte_array[1]<<8) + (byte_array[0] & 0xff)
Also, if you can use the Guava library:
Ints.fromByteArray(0, 0, myArray[1], myArray[0]);
It was worth mentioning since a lot of projects use it anyway.
Simply do this:
return new BigInteger(byte[] yourByteArray).intValue();
Works great on Bluetooth command conversions etc. No need to worry about signed vs. unsigned conversion.
import java.io.*;
public class ByteArray {
public static void main(String[] args) throws IOException {
File f=new File("c:/users/sample.txt");
byte[]b={1,2,3,4,5};
ByteArrayInputStream is=new ByteArrayInputStream(b);
int i;
while((i=is.read())!=-1) {
System.out.println((int)i);
FileOutputStream f1=new FileOutputStream(f);
FileOutputStream f2=new FileOutputStream(f);
ByteArrayOutputStream b1=new ByteArrayOutputStream();
b1.write(6545);
b1.writeTo(f1);
b1.writeTo(f2);
b1.close();
}