We Keep Coding

How to map binary data in java class object?

Input data :hexadecimal 64 byte
String binaryData="01000076183003104000800180f5010100010100000063000000630000006300000063000000000000000000820000000200b8010307010700640005e1cbe180";
Question is to read this binary data and set in class object
Here is the model
public class Transaction_PLUSale {
public byte opcode;
public byte[] code=new byte[7];
public byte flag1;
public byte flag2;
public byte flag3;
public byte flag4;
public byte flag5;
public short deptnum;
public byte multi_sell_unit;
public byte return_type;
public byte tax_pointer;
public int qty;
public int price;
public int amount;
public int no_tax_price;
public int no_tax_amount;
public int return_surcharge_percent;
public byte product_code;
public byte flags;
public TransactionTail tail;
}
I am currently doing this way to set values in each fields.
String hexArray[]= binaryData.split("(?<=\\G..)");
public static void readPLUSalesData(String hexArray[]) {
Transaction_PLUSale pluSale=new Transaction_PLUSale();
pluSale.setOpcode(Byte.valueOf(hexArray[0]));
byte arr[]=new byte[7];
for(int i=1;i<=7;i++) {
arr[i-1]=Byte.valueOf(hexArray[i]);
}
pluSale.setCode(arr);
pluSale.setFlag1(Byte.valueOf(hexArray[8]));
pluSale.setFlag2(Byte.valueOf(hexArray[9]));
pluSale.setFlag3(Byte.valueOf(hexArray[10]));
pluSale.setFlag4(Byte.valueOf(hexArray[11]));
pluSale.setFlag5(Byte.valueOf(hexArray[12]));
pluSale.setDeptnum((short)Integer.parseInt((hexArray[14]+hexArray[13]),16));
pluSale.setMulti_sell_unit(Byte.valueOf(hexArray[15]));
pluSale.setReturn_type(Byte.valueOf(hexArray[16]));;
pluSale.setTax_pointer(Byte.valueOf(hexArray[17]));
pluSale.setQty(Integer.parseInt((hexArray[21]+hexArray[20]+hexArray[19]+hexArray[18]),16));
pluSale.setPrice(Integer.parseInt((hexArray[25]+hexArray[24]+hexArray[23]+hexArray[22]),16));
pluSale.setAmount(Integer.parseInt((hexArray[29]+hexArray[28]+hexArray[27]+hexArray[26]),16));
pluSale.setNo_tax_price(Integer.parseInt((hexArray[33]+hexArray[32]+hexArray[31]+hexArray[30]),16));
pluSale.setNo_tax_amount(Integer.parseInt((hexArray[37]+hexArray[36]+hexArray[35]+hexArray[34]),16));
pluSale.setReturn_surcharge_percent(Integer.parseInt((hexArray[41]+hexArray[40]+hexArray[39]+hexArray[38]),16));
pluSale.setProduct_code(Byte.valueOf(hexArray[42]));
pluSale.setFlags(Byte.valueOf(hexArray[43]));
}
It is working fine. But I want it to be generic. So instead of giving byte by byte value. I want to direct map it to class fields.
In .net we are doing the marshalling for same feature that I need.
Here is the example
foreach (KeyValuePair<string, byte[]> s in t)
{
//byte array consist of bytes of the above hexadecimal string.
Ticket ticket = new Ticket();
int count = Marshal.SizeOf(typeof(Transaction_Coupon));
MemoryStream ms = new MemoryStream(s.Value);
byte[] readBuffer = new byte[count];
BinaryReader br = new BinaryReader(ms);
readBuffer = br.ReadBytes(count);
GCHandle handle = GCHandle.Alloc(readBuffer, GCHandleType.Pinned);
//here we are mapping byte data to each field
Transaction_PLUSale t_plusale = (Transaction_PLUSale)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(Transaction_PLUSale));
}

To convert binary data, a byte[] to a class with fields, there is no memory template to shift the data in. A good solution is using a ByteBuffer, either on a byte array or InputStream.
public static void readPLUSalesData(String[] hexArray) {
byte[] bytes = new byte[hexArray.length];
for (int i = 0; i < bytes.length; ++i) {
bytes[i] = Byte.parseByte(hexArray[i], 16);
}
ByteBuffer buf = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN;
Transaction_PLUSale pluSale=new Transaction_PLUSale();
pluSale.setOpcode(buf.get());
byte[] arr[] = new byte[7];
buf.get(arr);
pluSale.setCode(arr);
pluSale.setFlag1(buf.get());
pluSale.setFlag2(buf.get());
pluSale.setFlag3(buf.get());
pluSale.setFlag4(buf.get());
pluSale.setFlag5(buf.get());
pluSale.setDeptnum(buf.getShort());
pluSale.setMulti_sell_unit(buf.get());
pluSale.setReturn_type(buf.get());
pluSale.setTax_pointer(buf.get());
pluSale.setQty(buf.getInt());
pluSale.setPrice(buf.getInt());
pluSale.setAmount(buf.getInt());
pluSale.setNo_tax_price(buf.getInt());
pluSale.setNo_tax_amount(buf.getInt());
pluSale.setReturn_surcharge_percent(buf.getInt());
pluSale.setProduct_code(buf.get());
pluSale.setFlags(buf.get());
}
There exist other solutions, like using reflection, which is inefficient.
I used little endian byte order here, default in java is big endian.
There is the ObjectOutputStream, Serializable, persistence using serialisation.
It stores class data too, so is not the language agnostic format you desire.
While developing with a ByteBuffer is makes sense to check the read position.
If you are interested in XML persistence, JAXB with annotations offers a nice reflection based way, without need of handling every field.
A remark: Type[] variable is the preferred notation; Type var[] was initially added to java to be compatible with C/C++.

Casting double[] to byte[] using stream in Java

Hey I'm working on an app that uses Paho mqtt
Now I'm trying to cast the contents of a couple of objects to byte arrays so I can send them to the broker. There are a couple of different objects that all adhere to a abstract class, but the one I started with contains a double[]
Here's the function I'm trying to implement:
#Override
public byte[] getBytes() {
return Arrays.stream(driveVector).map(d -> Double.valueOf(d).byteValue()).toArray();
}
I thought this would work, but I get an error that the return value is a double[]
I think I either don't understand the map method or I'm goin about this all wrong in general (I looked at the ByteBuffer class, but it seems like a pain to implement this with it)
Thanks in advance

You can't cast a double[] to a byte[] for the fundamental reason that they are unrelated types, and you can only cast between related types.
Casts in Java, unlike, say, C++, don't actually create a new object: they are merely a way to the compiler "I know more about the type of this object than you; trust me." For example, you might know that a variable of type Object actually holds a reference to a String, something which the compiler cannot know; in that case, you can cast the reference.
You can, however, construct a new array:
byte[] output = new byte[input.length];
for (int j = 0; j < input.length; j++) {
output[j] = (byte) input[j];
}
There is no way to do this with streams. Or rather, there is, in that you could crowbar this code into a stream operation on a Stream<double[]>, say; but involving streams like that clearly adds no benefit.

You can use ByteBuffer for it:
double[] doubles = new double[] {1,2,3,4,5};
ByteBuffer buffer = ByteBuffer.allocate(doubles.length * Double.BYTES);
Arrays.stream(doubles).forEach(buffer::putDouble);
buffer.array();

Java Streams is not the right tool here, especially not since there is no ByteStream in Java.
Your method can be implemented as a simple for loop.
#Override
public byte[] getBytes() {
byte[] arr = new byte[driveVector.length];
for (int i = 0; i < arr.length; i++)
arr[i] = (byte) driveVector[i];
return arr;
}

In my MQTT application I read a single double value and post that to the broker. However, there is no real difference between a single and an array of doubles. The client needs to know the array length, while with a single value it always knows there is one.
I'm confident that you can adapt my code to writing multiple values, adapt the toMessage to write multiple double values.
public abstract class SensorMonitor {
protected final MqttAsyncClient client;
protected final String topic;
protected final Logger logger = Logger.getLogger(getClass().getName());
private final ByteArrayOutputStream byteOut = new ByteArrayOutputStream(8);
private final DataOutputStream dataOut = new DataOutputStream(byteOut);
public SensorMonitor(MqttAsyncClient mqttClient, String topic) {
this.client = mqttClient;
this.topic = topic;
}
public void start(ScheduledExecutorService service) {
service.scheduleWithFixedDelay(this::publish, 0, 30, TimeUnit.SECONDS);
}
protected void publish() {
try {
MqttMessage message = toMessage(readNewValue());
client.publish(topic, message);
} catch (MqttException | IOException e) {
logger.log(Level.SEVERE, "Could not publish message", e);
}
}
private MqttMessage toMessage(double value) throws IOException {
byteOut.reset();
dataOut.writeDouble(value);
return new MqttMessage(byteOut.toByteArray());
}
protected abstract double readNewValue();
}
The DataOutputStream.writeDouble uses Double.doubleToLongBits to create a IEEE 754 floating-point "double format" bit layout.
In my case I could pre-alloc and reuse the byteOut output stream as I knew upfront the needed size of the byte[].

How to get the base pointer of Unicode character?

Currently I have "codePointAt" which returns the code point of the character from the string.
Is there any API or other way to get the base pointer of the current character?
public class Testclass {
public static void main(String[] args) {
String unicodeString = "कागज़";
int currentPoint = unicodeString.codePointAt(0);
// Now currentPoint = 0x0915
// I need currentPoint = 0x0900
}
}
Note# I cannot create the base pointer by addition/subtraction because different language's base point start from different One's/Ten's place values. For e.g.
Armenian - 0530-058F - Base pointer 0x0530(ten's place value)
Devanagari - 0900-097F - Base pointer 0x0900(hundred's place value)
Currently I'm using if-else blocks to get the base pointer which not dynamic and also lengthy approach. :-(
int basePointer;
if(currentPoint>0x600 && currentPoint<=0x6FF)//Means Arabic
{
basePointer = 0x0600;
}
if(currentPoint>0x900 && currentPoint<=0x97F)//Means Devnagri
{
basePointer = 0x0900;
}

OK, after thinking about this for a bit, here is a way to do it just using the Java API. It consists of three parts:
Regenerating the inaccessible block base table blockStarts in Character.UnicodeBlock into a Map
Using Character.UnicodeBlock.of(int) to look up the block name given the codepoint
Using the Map to lookup the Unicode block base given the block name
Note that regenerating the block base table is relatively slow at approx 10-15 ms on my machine, so it would probably be best to generate this once and reuse. I've left the rudimentary timing code in place.
private static final int SUPPLEMENTARY_PRIVATE_USE_AREA_A_BASE = 0x0F0000;
private static final int SUPPLEMENTARY_PRIVATE_USE_AREA_B_BASE = 0x100000;
private static final Character.UnicodeBlock SUPPLEMENTARY_PRIVATE_USE_AREA_A =
Character.UnicodeBlock.of(SUPPLEMENTARY_PRIVATE_USE_AREA_A_BASE);
private static final Character.UnicodeBlock SUPPLEMENTARY_PRIVATE_USE_AREA_B =
Character.UnicodeBlock.of(SUPPLEMENTARY_PRIVATE_USE_AREA_B_BASE);
public static Map<Character.UnicodeBlock, Integer> makeUnicodeBlockBaseMap() {
long startNanos = System.nanoTime();
Map<Character.UnicodeBlock, Integer> unicodeBases = new HashMap<>();
// Unicode blocks start on 16 (0x10) byte boundaries.
for (int cp = 0x00000; cp < SUPPLEMENTARY_PRIVATE_USE_AREA_A_BASE; cp += 0x10) {
Character.UnicodeBlock ucb = Character.UnicodeBlock.of(cp);
if (ucb != null) {
unicodeBases.putIfAbsent(ucb, cp);
}
}
// These blocks are huge, so add them manually.
unicodeBases.put(SUPPLEMENTARY_PRIVATE_USE_AREA_A, SUPPLEMENTARY_PRIVATE_USE_AREA_A_BASE);
unicodeBases.put(SUPPLEMENTARY_PRIVATE_USE_AREA_B, SUPPLEMENTARY_PRIVATE_USE_AREA_B_BASE);
long endNanos = System.nanoTime();
System.out.format("Total time = %.3f s%n", (endNanos - startNanos) / 1e9);
return unicodeBases;
}
public static void main(String[] args) {
Map<Character.UnicodeBlock, Integer> unicodeBlockBases = makeUnicodeBlockBaseMap();
String unicodeString = "कागज़";
int currentPoint = unicodeString.codePointAt(0);
Character.UnicodeBlock ucb = Character.UnicodeBlock.of(currentPoint);
System.out.println(ucb); // DEVANAGARI
System.out.format("0x%04X%n", unicodeBlockBases.get(ucb)); // 0x0900
}

You can put the start/end positions to SortedMaps for each language and check the codePoints:
private static final SortedSet<Integer, Integer> startToBase = new TreeMap<>();
private static final SortedSet<Integer, Integer> endToBase = TreeMap<>();
static {
// Fill the SortedMaps:
// latin
startToBase.put(0, 0);
endToBase.put(0x00ff, 0);
// ...
}
// Or load this from a web service, table or anything you find comfortable
public static final int baseCodePoint(int codePoint) {
// The codePoint should be inserted here (after)
int baseFromStart = startToBase.get(startToBase.headMap(codePoint + 1).lastKey());
// the code point should be inserted here (before).
int baseFromEnd = endToBase.get(endToBAse.tailMap(codePoint).firstKey());
if (baseFromStart == baseFromEnd) {
return baseFromStart;
}
throw new IllegalArgumentException(codePoint + " is unknown.");
}

This is what I have done, thanks to Gábor Bakos for inspiration:
TreeMap<Integer, Integer> languageCodePoints = new TreeMap<>();
languageCodePoints.put(0x0020, 0x007E);
languageCodePoints.put(0x00A0, 0x00FF);
languageCodePoints.put(0x0100, 0x017F);
languageCodePoints.put(0x0900, 0x097F); // Devanagri
// So on for all other languages, referred ISO/IEC 10646:2010
// for code points of present languages
In the function I used this only:
String unicodeString = "कागज़";
int currentPoint = unicodeString.codePointAt(0);
int startCodePoint = languageCodePoints.floorKey(currentPoint);
Now "startCodePoint = 0x900" which I really required. I think pretty simple way. :-P
Just one thing is that, I have to maintain "languageCodePoints" TreeMap for new language entries but far better than switch/if-else.
Thanks to all for such kind support. :-)

You can use bit manipulation to find the base pointers, something like this:
switch (codePoint & 0xffffff00) {
case 0x0600: // Arabic
case 0x0900: // Devnagri, though you might need to check it is below 0x97F
case 0x0000: // Latin
default: // Something else
}
Ah, sorry I think Armenian requires further processing, but hopefully the general idea is applicable for most of the languages.
public static int baseCodePoint(int codePoint) {
switch (codePoint & 0xffffff00) {
case 0x0900: if (codePoint < 0x0980) return 0x0900;
case 0x0500: if (codePoint >= 0x0530 && codePoint <= 0x058F) return 0x0530;
// case ...: other bases where it is not the real base
// Handling regular base pointers
default: return codePoint & 0xffffff00;
}
}

How to use over-ride the super class methods?

This was the question asked:
Develop a class Decrypt that derives from FileInputStream and overrides the read() method of FileInputStream such that overriding read method returns a decrypted integer. Use this class to decrypt the file information that is contained out.txt.
I wrote a code for encryption, it worked but for decryption it doesn't. For decryption, i have to XOR the value with 128.
The problem is that after running the program, it doesn't write anything on the output file.
Here's the link for the sample input:
https://www.dropbox.com/s/jb361cxmjc9yd8n/in.txt
This is how the sample output looks like:
How high he holds his haughty head.
The code is below:
//Decrypt class
import java.io.*;
public class Decrypt extends FileInputStream {
public Decrypt(String name) throws FileNotFoundException {
super(name);
}
public int read() throws IOException {
int value = super.read();
int xor = value^128; // exclusive OR to decrypt
return xor;
}
}
//main
import java.io.*;
public class LA4ex3b {
public static void main(String[] args) throws IOException {
Decrypt de=null;
FileOutputStream output=null;
try
{
de = new Decrypt("C:/Users/user/workspace/LA4ex3a/in.txt");
output = new FileOutputStream("C:/Users/user/workspace/LA4ex3a /out.txt");
int a;
while ((a = de.read()) != -1)
{
output.write(a);
}
}
finally
{
if (de!=null)
de.close();
if (output!=null)
output.close();
}
}
}

int value = super.read();
int xor = value^128; // exclusive OR to decrypt
return xor;
In the above you do not check for the special value of -1 returned from super.read(), which you must push through transparently. Without that you'll never receive -1 in your while loop and the program will not terminate normally. The code as below should fix that issue:
int value = super.read();
return value == -1? value : value^128;

Well, I think you should ask in your Overriden method, wether super.read() is -1, too. Because if super.read() is -1, and you xor it with 128, it will not be -1 any longer, so Decrypt.read() wont be -1.
Edit: Ok, I wasn't fast enough!

Two quick corrections:
The output filename should not have a space in it
You listed no restrictions on data in your input file, so I suggest you use the "read data into a byte array" method (just in case a "-1" byte value is legitimate data in the input. Your particular text input is probably ok, but think of problems like these and solve the most inclusive problem you can still simply.
byte[] dataBuffer = new byte[1000];
int bytesRead;
bytesRead = de.read(dataBuffer);
while (bytesRead != -1 ) [
// decrypt each byte
// write the decrypted bytes
bytesRead = de.read(dataBuffer);
}

class Overload
{
public int Add(int a, int b)
{
Console.WriteLine("Int method with Two params executed");
return a + b;
}
public int Add(int a, int b, int c)
{
Console.WriteLine("Int method with three params executed");
return a + b + c;
}
public double Add(double a, double b)
{
Console.WriteLine("double method with Two params executed");
return a + b;
}
}
//class Derived : Overload //over riding//
//{
// public int Add(int a, int b)
// {
// return a + b;
// }
//}
}

Error while passing array of structures conataining char array to native function using JNA

Here is my updated code after implementing suggestions. But still problems persist.
typedef struct S1{
char temp1[100];
char temp2[100];
}S1
...
int manipulateTemp(S1 s1Arr[] );
JNA interface looks like this
public interface Add extends Library
{
Add INSTANCE = (Add) Native.loadLibrary("add", Add.class);
public static class S1 extends Structure {
public byte[] temp1 = new byte[100];
public byte[] temp2 = new byte[100];
public static class ByReference extends S1 implements Structure.ByReference {
};
};
int manipulateTemp( S1[]);
}
//
public static byte[] toByteArray(char[] a ,Charset c){
CharBuffer cBuffer = CharBuffer.wrap(a);
ByteBuffer bBuffer = c.encode(cBuffer);
return bBuffer.array;
}
//in main method
Add lib = Add.INSTANCE;
Add.S1.ByReference s1Ref = new Add.S1.ByReference();
Add.S1[] s1Arr = (Add.S1[])s1Ref.toArray(10);
s1Ref.clear();
//initialize array
for(int i =0;i<s1Arr.lenth ;i++){
byte[] data = toByteArray("myString1".toCharArray,Charset.defaultCharSet
System.arrarycopy(data,0, s1Arr[i].temp1,0,data.length);
data = toByteArray("myString2".toCharArray,Charset.defaultCharSet
System.arrarycopy(data,0, s1Arr[i].temp2,0,data.length);
}
// calling native function
lib.manipulateTemp(s1Arr[]);
After execution
Exception in thread "main" java.lang.Error: Invalid memory access
at com.sun.jna.Function.invokeInt(Native Method)
at com.sun.jna.Function.invoke(Function.java:344)
at com.sun.jna.Function.invoke(Function.java:276)
at com.sun.jna.Library$Handler.invoke(Library.java:216)
at com.sun.proxy.$Proxy0.manipulateTemp((Unknown Source)
at LoanTest.newTestCalc.main(newTestCalc.java:288)
I even checked memory dump, structures are seems to be allocated stored correctly.Structure size is also correct = 200 bytes
Any clues about this error?

You need to copy values into the existing temp field, not overwrite it. When you overwrite it, you're actually changing its size, which JNA uses to determine the structure size. Following is how you should initialize your structure data:
class S1 extends Structure {
public byte[] temp = new byte[100];
...
}
S1 s = new S1();
S1[] array = (S1[])s.toArray(ARRAY_SIZE);
System.setProperty("jna.encoding", "utf-8"); // You probably want utf-8; utf-16 has 16-bit code units, so unless your native code is actually expecting a utf-16 encoding broken down into byte units, use utf-8
byte[] data = Native.toByteArray("myString"); // includes NUL terminator
System.arraycopy(data, 0, array[0].temp, 0, data.length);
// Repeat as needed for other members of the array
lib.manipulateTemp(array);
Note that the declarations manipulateTemp(S1 s) or manipulateTemp(S1[] array) will both work, although the latter is more accurate and conveys your intent explicitly.