We Keep Coding

java.nio.BufferOverflowException constructing byte[]

I'm building a byte array to identify an M-Bus Master and i need to do it using the secondary address.
To do it i need to build a byte[] with the identification of the address:
Identification Number (4 bytes) – A number ranging from 00000000 to
99999999 to identify the meter.
Manufacturer ID (2 bytes) – Three letters that identify the
manufacturer.
Version (1 byte) – Specifies the version of the device. The version
is manufacturer specific.
Device type (1 byte) – This field codes the device type (e.g.
electricity meter, cold water meter)
If my math is not failing me this has a total of 8 bytes.
So here is my code to do it:
public static void main(String[] args) {
// TODO code application logic here
MBusSerialBuilder builder = MBusConnection.newSerialBuilder("COM4").setBaudrate(2400);
try (MBusConnection mBusConnection = builder.build()) {
// read/write
int primaryAddress = 253;
byte[] idNum = ByteBuffer.allocate(4).putInt(46152604).array();
byte version = 0xFF & 88; //only need a byte not a 4 byte int
byte deviceType = 0xFF & 13; //only need a byte not a 4 byte int
short manuID = createManuID("ZRI");
//builds the message without overflow now
byte[] data = ByteBuffer.allocate(8).put(idNum).putShort(manuID).put(version).put(deviceType).array();
mBusConnection.write(primaryAddress, data);
VariableDataStructure vds = mBusConnection.read(primaryAddress);
System.out.println(vds);
} catch (IOException ex) {
System.out.println(ex.getLocalizedMessage());
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
}
Note previouslly i had
byte[] manId = ByteBuffer.allocate(2).putChar('Z').putChar('R').putChar('I').array();
And it was returning me java.nio.BufferOverflowException.
With the recent changes the error is now on data declaration.
Even if i alloc 50 bytes
byte[] data = ByteBuffer.allocate(50).put(idNum).put(manId).putInt(88).putInt(13).array();
Diferent error
java.lang.IndexOutOfBoundsException
Here is some info i extracted from a log file of the seller's program.
MBus Tx_raw-><11><68><b><b><68><53><fd><52><4><26><15><46><ff><ff><ff><ff><23><16>
MBus Rx_raw-><0><aa><1><e5><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0><0>
MBus Tx_raw-><5><10><7b><fd><78><16>
MBus Rx_raw-><0><aa><b7><68><b1><b1><68><8><0><72><4><26><15><46><49><6a><88><d><29><0><0><0><4><6d><34><a><27><2c><82><4><6c><21><21><82><a><6c><21><2c><4><6><0><0><0><0><84><4><6><0><0><0><0><84><a><6><0><0><0><0><4><13><4a><30><0><0>
MBus consecutive Frame [183]-><68><b1><b1><68><8><0><72><4><26><15><46><49><6a><88><d><29><0><0><0><4><6d><34><a><27><2c><82><4><6c><21><21><82><a><6c><21><2c><4><6><0><0><0><0><84><4><6><0><0><0><0><84><a><6><0><0><0><0><4><13><4a><30><0><0><2><59><8a><7><2><5d><bc><7><2><61><ce><ff><4><3b><bf><2><0><0><4><2d><4><0><0><0><4><26><b><8><0><0><84><10><6><2><0><0><0><84><14><6><0><0><0><0><84><1a><6><0><0><0><0><84><40><14><c1><6><0><0><84><44><14><0><0><0><0><84><4a><14><a9><0><0><0><84><80><40><14><10><0><0><0><84><84><40><14><0><0><0><0><84><8a><40><14><0><0><0><0><84><c0><40><14><e3><0><0><0><84><c4><40><14><0><0><0><0><84><ca><40><14><0><0><0><0><1b><16>
Readout insert->INSERT INTO LETTURE_CONTATORI_TEMP VALUES(NULL,'OK','1','1','510','07/12/2017 10:16:23','','','1512641783','0','07/12/2017 10:52','01/01/2017','01/12/2017','0','0','0','12362','1930','1980','-50','703','4','2059','2','0','0','1729','0','169','16','0','0','227','0','0','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','','');

Ok, so with an offset of 65 (A) to every character you can create a mapping so-to-speak of the letters to convert them to smaller values which will fit in the 2 bytes (16 bits) whereby the values range from 0 to 25 (0=A, 1=B..., 25=Z). Since this range requires at most 5 bits, and you have a maximum of 3 characters to convert, you only need 15 bits and can squeeze these into the 2 bytes (16 bits) required for the manufacturer id. All you have to do is apply a bit shift of 5 (size of the values) * the index of the character in your manufacturer id string.
Here is the method
public static short createManuID(String id)
{
int bitMashedManuID = 0;
id = id.toUpperCase(); //force the chars to be within 65-90
if(id.length() == 3)
{
short offset = 65; //A = 0, B = 1 ... Z = 25
//number bits needed to fit 0-25 and so values won't overlap during the bit mashing
short bitShift = 5;
for(int i = 0; i < id.length(); i++)
{
short valueOfChar = (short)id.charAt(i);
valueOfChar -= offset; //apply the offset
bitMashedManuID += valueOfChar << bitShift * i; //pack the bits
}
}
return (short)bitMashedManuID;
}
Example
Z = 90, apply the offset of 65 and we get 25 (11001)
So a manufacturer id of ZZZ should look like (11001|11001|11001) which equals 26425.
System.out.println(createManuID("ZZZ")); //outputs 26425
Your manufacturer id
Z = 90 - 65 = 25 = 11001
R = 82 - 65 = 17 = 10001
I = 73 - 65 = 8 = 01000
ZRI = |01000|10001|11001| = 8761
System.out.println(createManuID("ZRI")); //8761
Therefore when all is said and done you can create your byte array like this without overflow and satisfying the 8 byte array length requirement.
public static void main(String[] args)
{
byte[] idNum = ByteBuffer.allocate(4).putInt(46152604).array();
byte version = 0xFF & 88; //only need a byte not a 4 byte int
byte deviceType = 0xFF & 13; //only need a byte not a 4 byte int
short manuID = createManuID("ZRI");
//builds the message without overflow now
byte[] data = ByteBuffer.allocate(8).put(idNum).putShort(manuID).put(version).put(deviceType).array();
}
All that's left is determine the order of the letters are going to be packed in. Currently I pack them from right to left but depending on the device you are talking to it may require left to right which means you have the loop start at for(int i = id.length() - 1; i >= 0; i--)

The BB isn't large enough.
You don't need all this. Allocate one ByteBuffer large enough for all the data, and then call all the puts you need. Or use a DataOutputStream.

Two 16 bit ints to One 32 bit float value

I am able to extract values from modbus as 16-bit shorts (unsigned) or as ints (should be treated as 16-bit words). I am tasked to combine two values to create a single 32 bit float value using java.
some example values I observed using a gui program:
int + int = float
0 + 16256 = 1
0 + 17096 = 100
0 + 17097 = 100.5
0 + 17530 = 1000
8192 + 17530 = 1000.5
I attempted bit wise operators but that didn't seem to do the trick.
leaves me scratching my head!

You can use Float.intBitsToFloat(int bits) to build a float from the bits of an int.
short high = ... // the high 16 bits
short low = ... // the low 16 bits
int combined = (high << 16) | low;
float num = Float.intBitsToFloat(combined);
for example:
short high = 17530;
short low = 8192;
produces the float 1000.5.

How to create 32 bit mask in Java

So far I have this code to create 16 bit mask. However I do not know how to create a 32 bit mask using this approach. Any ideas?
edit: I want to create 32 masks of 32 bits, each with with its respective bit being 1 and the rest of the bits being zero. For example: mask 1 has the leftmost bit being 1 while the rest of the bits are zero, mask 2 has the 2nd leftmost bit being 1 while the rest of the bits are zero. I dunno how to explain more succinctly but I hope you guys get the idea...
mask = new int[16];
mask[0] = 0x8000;
mask[1] = 0x4000;
mask[2] = 0x2000;
mask[3] = 0x1000;
mask[4] = 0x0800;
mask[5] = 0x0400;
mask[6] = 0x0200;
mask[7] = 0x0100;
mask[8] = 0x0080;
mask[9] = 0x0040;
mask[10] = 0x0020;
mask[11] = 0x0010;
mask[12] = 0x0008;
mask[13] = 0x0004;
mask[14] = 0x0002;
mask[15] = 0x0001

Here's how to create a 32 bit mask in Java.
int mask = 0x00010000; // for example.
And if you want to create a 32 bit mask with bit N set then
int mask = 1 << N; // Assumes the rightmost bit is numbered zero ...
And if you want to create an array of masks, then just do the above in a loop, in the obvious fashion.
int[] masks = new int[32];
for (int n = 0; n < 32; n++) {
masks[n] = 1 << n;
}
In reality, your supposed "16 bit masks" are also 32 bit masks, because int is a 32 bit type in Java.
And like #Matt Ball, I'm puzzled as to what you are really trying to do, and whether you are going about it is a sensible way to achieve it. Why do you need an array of masks when you can create a mask on the fly with less code?

I think you are doing well.
AMOUNT_OF_BITS = 32;
mask = new long[AMOUNT_OF_BITS];
for (long i = 0; i < AMOUNT_OF_BITS; i++)
{
mask[i] = Math.pow(2,i);
}
This should work, I think.

Android: How to get pixels of a single channel 16bit PNG Bitmap

I have a png file created with python PIL that contains an height map (positive values).
The format is : single channel (grey level) at 16bit, thus 16 bit per pixel.
I read the file on android with BitmapFactory.decodeStream(<...data input stream...>);
I correctly get the size of the image through getWidth() and getHeight().
However when I loop though the pixels calling the getPixel(i,j) I obtain negative values, something like: -16776192 -16250872 -16250872 -16250872 -16250872 -16249848 ....
Instead I expect a positive value between 0 and 65535.
I discover that the value -16250872 in binary is 1111111111111111111111111111111111111111000010000000100000001000
This shows that the information relies on the first 16 least significant bits.
I tryed with getPixel(i,j)&0xffff and I obtain a plausible values, however I'm not sure about the endianess: should I to flip the 2 extracted bytes?
Is there a way to do it in more elegant and portable way this conversion ?
NOTE: the file is not a color (RGBA) PNG but a grey level PNG image with a single 16 bit value for each pixel.

I found a solution by myself using the consideration made in this post:
Android: loading an alpha mask bitmap
Basically, If you load directly a 16 bit greylevel PNG from bitmap factory the pixel format will be not correct. You need to use a RGBA 32 bit color format setting the pixel format to ARGB_8888. Then you have to get all the pixels with getPixel(i,j) and mask the integer value with 0xffff. In this way you will obtain the expected 16bit values.
This is a part of the code that I've used:
BitmapFactory.Options opt = new BitmapFactory.Options();
opt.inPreferredConfig= Bitmap.Config.ARGB_8888;
Bitmap bmp=BitmapFactory.decodeStream(entity.getContent(),null,opt);
int W=bmp.getWidth();
int H=bmp.getHeight();
int px;
for (int i = 0; i < H; i++)
for (int j = 0; j < W; j++)
{
px= bmp.getPixel(j, i)&0x0000ffff;//in px you will find a value between 0 and 65535
...
}

I assume your trying to get the RGB for a pixel in an image. If that is correct then you will find the following helpful.
Returns the Color at the specified location. Throws an exception if x or y are out of bounds (negative or >= to the width or height respectively).
That is the quote for the Bitmap.getPixel();
What you need to do to print this off so that it is human readable. I have programed with android, but not done this with android. I used the following function for one my programs.
public static int[] getRGBValue(BufferedImage bi, int x, int y) {
int argb = bi.getRGB(x, y);
int rgb[] = new int[] {
(argb >> 16) & 0xff, // red
(argb >> 8) & 0xff, // green
(argb ) & 0xff, // blue
};
return rgb;
}
public static String getStringOfRGB(int[] value) {
return "Color: (" + value[0] + ", " + value[1] + ", " + value[2] + ")";
}
I don't kow what you trying to do exactly, so the code above won't answer your question... but should assist you in finding your answer on however you want to use the data of the pixel.
Hope this helps! :)

Get Bytes from an int to avoid bit shifting fun - Java (Median Filtering)

I'm trying to perform a Median filter on an image in Java but it's terribly slow. Firstly, if any of you know of a standalone implementation I could use it would be fantastic if you could let me know. I'm implementing on Android, trying to replicate a small part of the JAI.
In my method I take each pixel, extract the R,G & B values using
r = pixel >> 16 & 0xFF
Or similar, find the median for the kernel and finish with
pixel = a | r <<16 | g << 8 | b
Is there any way I can grab the bytes from an int in such a way that this would be faster?
Kind regards,
Gavin
EDIT: Full code to help diagnose my low performance upon request
For the actual source file please go here that's where my implementation of medianFilter can be found.
width and height variables are for the size of dest and are available as class member variables. The pixels are linearized into a one dimensional array.
private void medianFilterSquare(int[] source, int[] dest, int rWidth,
int rHeight, int radius) {
// Source has been reflected into a border of size radius
// This makes it radius * 2 pixels wider and taller than the dest
int r,g,b;
int destOffset, rOffset, kOffset;
// The first offset into the source to calculate a median for
// This corresponds to the first pixel in dest
int rFirst = radius + (rWidth*radius);
// We use a square kernel with the radius passed
int neighbours = (radius+radius+1)*(radius+radius+1);
int index;
// Arrays to accumulate the values for median calculation
int[] rs = new int[neighbours];
int[] gs = new int[neighbours];
int[] bs = new int[neighbours];
// Declaring outside the loop helps speed? I'm sure this is done for me
// by the compiler
int pixel;
// Iterate over the destination pixels
for(int x = 0; x < height; x++){
for(int y = 0; y < width; y++){
// Offset into destination
destOffset = x + (y * width);
// Offset into source with border size radius
rOffset = destOffset + rFirst + (y * (radius *2));
index = 0;
// Iterate over kernel
for(int xk = -radius; xk < radius ; xk ++){
for(int yk = -radius; yk < radius ; yk ++){
kOffset = rOffset + (xk + (rWidth*yk));
pixel = source[kOffset];
// Color.red is equivalent to (pixel>>16) & 0xFF
rs[index] = Color.red(pixel);
gs[index] = Color.green(pixel);
bs[index] = Color.blue(pixel);
index++;
}
}
r = medianFilter(rs);
g = medianFilter(gs);
b = medianFilter(bs);
dest[destOffset] = Color.rgb(r, g, b);
}
}
}

As others have said, it's possible that it's the bit in between which is causing the problem. One thing I would say (which may be obvious, but anyway) - don't just profile the application on a desktop VM and assume that the bottleneck will be in the same place. I wouldn't be at all surprised to find entirely different bottlenecks within Dalvik.
Is it possible for you to work with the values still shifted? For instance, if you were to just mask for different colours:
int r = pixel & 0xff0000;
int g = pixel & 0xff00;
int b = pixel & 0xff;
could you tweak your processing algorithm accordingly?
One final thought: I always find the precedence of shift operators confusing. I'd strongly recommend that from a readability point of view, you bracket them:
r = (pixel >> 16) & 0xFF;
pixel = a | (r <<16) | (g << 8) | b;
Irrelevant to performance, but if I were a maintainer I'd certainly appreciate it :)

The fastet way to get your r,g,b values should be
new byte[] {
(byte)(value >>> 24),
(byte)(value >>> 16),
(byte)(value >>> 8),
(byte)value
};

Concentrating on how to do the bit operations is a distraction. It only matters how you're doing these operations because you're needlessly processing the same pixel over and over.
You're calling median filter for every pixel three times, and you're getting multiple pixels around the pixel per pixel. Which means you're doing all that bit work for the same pixel multiple times. You have for loops nested four deep!
If your radius is 5, you're processing 121 pixels. Then you move down by one and process 121 pixels again, and you've already converted all but 11 of them! You do the same thing for every pixel down, then move to the right one pixel. For a radius of five, you're doing two orders of magnitude as many rgb conversions as you have to.
I'd suggest keeping your image in or converting your image to separate red, blue, and green arrays first.
If radius is large, you could keep the red, blue, and green sums as you move along, subtracting out the pixels from the top and adding in the pixels from the bottom as you crawl down the bitmap, but that would make the code a touch more complicated. Whether you add code to optimize further depends on your requirements.
Also, you have a bunch of little things that could be optimized. I'm not sure if the compiler is taking care of them or not. For instance, you could do some strength reduction. You don't need any of the multiplications you have in the lines that calculate neighbors, destOffset, rOffset, or kOffset. Addition is all you need for those if you refactor the code a bit.

You can occasionally get away with doing arithmetic on the red & blue components simultaneously in a single int:
int average(int rgb1, int rgb2)
{
int rb = (((rgb1 & 0xFF00FF) + (rgb2 & 0xFF00FF)) >> 1) & 0xFF00FF;
int g = (((rgb1 & 0xFF00) + (rgb2 & 0xFF00)) >> 1) & 0xFF00;
return (rb | g);
}
because the red and blue components are separated by 8 bits, they don't interfere with each other.
I've never seen a significant (more than 5-10%) speedup from this though.