We Keep Coding

Casting double[] to byte[] using stream in Java

Hey I'm working on an app that uses Paho mqtt
Now I'm trying to cast the contents of a couple of objects to byte arrays so I can send them to the broker. There are a couple of different objects that all adhere to a abstract class, but the one I started with contains a double[]
Here's the function I'm trying to implement:
#Override
public byte[] getBytes() {
return Arrays.stream(driveVector).map(d -> Double.valueOf(d).byteValue()).toArray();
}
I thought this would work, but I get an error that the return value is a double[]
I think I either don't understand the map method or I'm goin about this all wrong in general (I looked at the ByteBuffer class, but it seems like a pain to implement this with it)
Thanks in advance

You can't cast a double[] to a byte[] for the fundamental reason that they are unrelated types, and you can only cast between related types.
Casts in Java, unlike, say, C++, don't actually create a new object: they are merely a way to the compiler "I know more about the type of this object than you; trust me." For example, you might know that a variable of type Object actually holds a reference to a String, something which the compiler cannot know; in that case, you can cast the reference.
You can, however, construct a new array:
byte[] output = new byte[input.length];
for (int j = 0; j < input.length; j++) {
output[j] = (byte) input[j];
}
There is no way to do this with streams. Or rather, there is, in that you could crowbar this code into a stream operation on a Stream<double[]>, say; but involving streams like that clearly adds no benefit.

You can use ByteBuffer for it:
double[] doubles = new double[] {1,2,3,4,5};
ByteBuffer buffer = ByteBuffer.allocate(doubles.length * Double.BYTES);
Arrays.stream(doubles).forEach(buffer::putDouble);
buffer.array();

Java Streams is not the right tool here, especially not since there is no ByteStream in Java.
Your method can be implemented as a simple for loop.
#Override
public byte[] getBytes() {
byte[] arr = new byte[driveVector.length];
for (int i = 0; i < arr.length; i++)
arr[i] = (byte) driveVector[i];
return arr;
}

In my MQTT application I read a single double value and post that to the broker. However, there is no real difference between a single and an array of doubles. The client needs to know the array length, while with a single value it always knows there is one.
I'm confident that you can adapt my code to writing multiple values, adapt the toMessage to write multiple double values.
public abstract class SensorMonitor {
protected final MqttAsyncClient client;
protected final String topic;
protected final Logger logger = Logger.getLogger(getClass().getName());
private final ByteArrayOutputStream byteOut = new ByteArrayOutputStream(8);
private final DataOutputStream dataOut = new DataOutputStream(byteOut);
public SensorMonitor(MqttAsyncClient mqttClient, String topic) {
this.client = mqttClient;
this.topic = topic;
}
public void start(ScheduledExecutorService service) {
service.scheduleWithFixedDelay(this::publish, 0, 30, TimeUnit.SECONDS);
}
protected void publish() {
try {
MqttMessage message = toMessage(readNewValue());
client.publish(topic, message);
} catch (MqttException | IOException e) {
logger.log(Level.SEVERE, "Could not publish message", e);
}
}
private MqttMessage toMessage(double value) throws IOException {
byteOut.reset();
dataOut.writeDouble(value);
return new MqttMessage(byteOut.toByteArray());
}
protected abstract double readNewValue();
}
The DataOutputStream.writeDouble uses Double.doubleToLongBits to create a IEEE 754 floating-point "double format" bit layout.
In my case I could pre-alloc and reuse the byteOut output stream as I knew upfront the needed size of the byte[].

Need some help ensuring my ByteUtil class is accurate

I am not very familiar with exactly all of the implications of bytes or even close to charsets, simply because i have not used them often. However i am working on a project in which i need to convert every Java primitive type (and Strings) to AND from bytes. I want them all with the charset UTF-8, but i'm not sure if i am converting them properly.
Anyways, although i am pretty sure that all number to/from byte conversions are correct, but then again, i need to be 100% sure. If someone has really good experience with bytes with numbers and charsets, could you look over the class below, and point out any issues?
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
public class ByteUtil
{
//TO BYTES FROM PRIMITIVES & STRINGS
public static byte[] getBytes(short i)
{
return ByteBuffer.allocate(2).putInt(i).array();
}
public static byte[] getBytes(int i)
{
return ByteBuffer.allocate(4).putInt(i).array();
}
public static byte[] getBytes(long i)
{
return ByteBuffer.allocate(8).putLong(i).array();
}
public static byte getBytes(boolean i)
{
return (byte) (i ? 1 : 0);
}
public static byte[] getBytes(char i)
{
return getBytes(String.valueOf(i).trim());
}
public static byte[] getBytes(String i)
{
return i.getBytes(StandardCharsets.UTF_8);
}
public static byte[] getBytes(float i)
{
return getBytes(Float.floatToIntBits(i));
}
public static byte[] getBytes(double i)
{
return getBytes(Double.doubleToLongBits(i));
}
//TO PRIMITIVES & STRINGS FROM BYTES
public static short getShort(byte[] b)
{
ByteBuffer wrapped = ByteBuffer.wrap(b);
return wrapped.getShort();
}
public static int getInt(byte[] b)
{
ByteBuffer wrapped = ByteBuffer.wrap(b);
return wrapped.getInt();
}
public static long getLong(byte[] b)
{
ByteBuffer wrapped = ByteBuffer.wrap(b);
return wrapped.getLong();
}
public static boolean getBoolean(byte b)
{
return(b == 1 ? true : false);
}
public static char getChar(byte[] b)
{
return getString(b).trim().toCharArray()[0];
}
public static String getString(byte[] b)
{
return new String(b, StandardCharsets.UTF_8);
}
public static float getFloat(byte[] b)
{
return Float.intBitsToFloat(getInt(b));
}
public static double getDouble(byte[] b)
{
return Double.longBitsToDouble(getLong(b));
}
}
Additionally, all the data put in and returned is read by my source internally, for example the boolean conversion may or may not be the correct way to do something like such, but in the boolean case, it wont matter since i know what i am checking for.

You don't even need to do this. You can use a DataOutputStream to write your primitive types and Strings to a ByteArrayOutputStream. You can then use toByteArray() to get a byte[] that you put into a ByteArrayInputStream. You can wrap that InputStream in a DataInputStream to get back your primitives.
If you're doing a school assignment where you need to implement this yourself (which sounds like a dumb assignment), you can look up the implementations of ByteArrayOutputStream and ByteArrayInputStream on GrepCode. Copy/pasting is a bad idea, but it might give you some hints about considerations to take into account.

Convert byte[] to UnsignedByte[]

I want to optimize the following java code (a single method):
private static UnsignedByte[] getUnsignedBytes(byte[] bytes){
UnsignedByte[] usBytes = new UnsignedByte[bytes.length];
int f;
for(int i = 0; i< bytes.length;i++){
f = bytes[i] & 0xFF;
usBytes[i] = new UnsignedByte(f) ;
}
return usBytes;
}
This code basically converts a byte array(which is a file) to UnsignedByte array so that it can be send to a webservice that i am consuming through apache axis.
Is there any way i can avoid this for loop. Is there any direct method for this?
Thank you.

No, unfortunately there is not. Conversion of the array of bytes has to be done by element.

I would do it with Guava this way:
UnsignedByte[] usBytes = Lists.transform(Arrays.asList(bytes), new Function<UnsignedByte, Short>() {
#Override
public UnsignedByte apply(#Nullable Byte input) {
f = input & 0xFF;
return new UnsignedByte(f) ;
}
}).toArray(new UnsignedByte[bytes.length]);

Error while passing array of structures conataining char array to native function using JNA

Here is my updated code after implementing suggestions. But still problems persist.
typedef struct S1{
char temp1[100];
char temp2[100];
}S1
...
int manipulateTemp(S1 s1Arr[] );
JNA interface looks like this
public interface Add extends Library
{
Add INSTANCE = (Add) Native.loadLibrary("add", Add.class);
public static class S1 extends Structure {
public byte[] temp1 = new byte[100];
public byte[] temp2 = new byte[100];
public static class ByReference extends S1 implements Structure.ByReference {
};
};
int manipulateTemp( S1[]);
}
//
public static byte[] toByteArray(char[] a ,Charset c){
CharBuffer cBuffer = CharBuffer.wrap(a);
ByteBuffer bBuffer = c.encode(cBuffer);
return bBuffer.array;
}
//in main method
Add lib = Add.INSTANCE;
Add.S1.ByReference s1Ref = new Add.S1.ByReference();
Add.S1[] s1Arr = (Add.S1[])s1Ref.toArray(10);
s1Ref.clear();
//initialize array
for(int i =0;i<s1Arr.lenth ;i++){
byte[] data = toByteArray("myString1".toCharArray,Charset.defaultCharSet
System.arrarycopy(data,0, s1Arr[i].temp1,0,data.length);
data = toByteArray("myString2".toCharArray,Charset.defaultCharSet
System.arrarycopy(data,0, s1Arr[i].temp2,0,data.length);
}
// calling native function
lib.manipulateTemp(s1Arr[]);
After execution
Exception in thread "main" java.lang.Error: Invalid memory access
at com.sun.jna.Function.invokeInt(Native Method)
at com.sun.jna.Function.invoke(Function.java:344)
at com.sun.jna.Function.invoke(Function.java:276)
at com.sun.jna.Library$Handler.invoke(Library.java:216)
at com.sun.proxy.$Proxy0.manipulateTemp((Unknown Source)
at LoanTest.newTestCalc.main(newTestCalc.java:288)
I even checked memory dump, structures are seems to be allocated stored correctly.Structure size is also correct = 200 bytes
Any clues about this error?

You need to copy values into the existing temp field, not overwrite it. When you overwrite it, you're actually changing its size, which JNA uses to determine the structure size. Following is how you should initialize your structure data:
class S1 extends Structure {
public byte[] temp = new byte[100];
...
}
S1 s = new S1();
S1[] array = (S1[])s.toArray(ARRAY_SIZE);
System.setProperty("jna.encoding", "utf-8"); // You probably want utf-8; utf-16 has 16-bit code units, so unless your native code is actually expecting a utf-16 encoding broken down into byte units, use utf-8
byte[] data = Native.toByteArray("myString"); // includes NUL terminator
System.arraycopy(data, 0, array[0].temp, 0, data.length);
// Repeat as needed for other members of the array
lib.manipulateTemp(array);
Note that the declarations manipulateTemp(S1 s) or manipulateTemp(S1[] array) will both work, although the latter is more accurate and conveys your intent explicitly.

Store and retrieve a float[] to/from Cassandra using Hector

I have the following Cassandra schema:
ColumnFamily: FloatArrays {
SCKey: SuperColumn Key (Integer) {
Key: FloatArray (float[]) {
field (String): value (String)
}
}
}
In order to insert data that adheres to this schema I created the following template in Hector:
template = new ThriftSuperCfTemplate<Integer, FloatArray, String>(
keyspace, "FloatArrays", IntegerSerializer.get(),
FloatArraySerializer.get(), StringSerializer.get());
To (de-)serialize the FloatArray I created (and unit tested) a custom Serializer:
public class FloatArraySerializer extends AbstractSerializer<FloatArray> {
private static final FloatArraySerializer instance =
new FloatArraySerializer();
public static FloatArraySerializer get() {
return instance;
}
#Override
public FloatArray fromByteBuffer(ByteBuffer buffer) {
buffer.rewind();
FloatBuffer floatBuf = buffer.asFloatBuffer();
float[] floats = new float[floatBuf.limit()];
if (floatBuf.hasArray()) {
floats = floatBuf.array();
} else {
floatBuf.get(floats, 0, floatBuf.limit());
}
return new FloatArray(floats);
}
#Override
public ByteBuffer toByteBuffer(FloatArray theArray) {
float[] floats = theArray.getFloats();
ByteBuffer byteBuf = ByteBuffer.allocate(4 * descriptor.length);
FloatBuffer floatBuf = byteBuf.asFloatBuffer();
floatBuf.put(floats);
byteBuf.rewind();
return byteBuf;
}
}
Now comes the tricky bit. Storing and then retrieving an array of floats does not return the same result. In fact, the number of elements in the array isn't even the same. The code I use to retrieve the result is shown below:
SuperCfResult<Integer, FloatArray, String> result =
template.querySuperColumns(hash);
for (FloatArray floatArray: result.getSuperColumns()) {
// Do something with the FloatArrays
}
Do I make a conceptual mistake here since I'm quite new to Cassandra/Hector? Right now I don't even have a clue on where it goes wrong. The Serializer seems to be ok. Can you please provide me with some pointers to continue my search? Many thanks!

I think you're on the right track. When I work with ByteBuffers I find I sometimes need the statement:
import org.apache.thrift.TBaseHelper;
...
ByteBuffer aCorrectedByteBuffer = TBaseHelper.rightSize(theByteBufferIWasGiven);
The byte buffer sometimes has its value stored as an offset into its buffer but the Serializers seem to assume that the byte buffer's value starts at offset 0. The TBaseHelper corrects the offsets as best I can tell so the assumptions in the Serializer implementation are made valid.
The difference in lengths of the array in and array out are the result of starting at the wrong offset. The first byte or two of the serialized value contain the length of the array.

Thanks to Chris I solved the problem. The Serializer now looks like this:
public class FloatArraySerializer extends AbstractSerializer<FloatArray> {
private static final FloatArraySerializer instance =
new FloatArraySerializer();
public static FloatArraySerializer get() {
return instance;
}
#Override
public FloatArray fromByteBuffer(ByteBuffer buffer) {
ByteBuffer rightBuffer = TBaseHelper.rightSize(buffer); // This does the trick
FloatBuffer floatBuf = rightBuffer.asFloatBuffer();
float[] floats = new float[floatBuf.limit()];
if (floatBuf.hasArray()) {
floats = floatBuf.array();
} else {
floatBuf.get(floats, 0, floatBuf.limit());
}
return new FloatArray(floats);
}
#Override
public ByteBuffer toByteBuffer(FloatArray theArray) {
float[] floats = theArray.getDescriptor();
ByteBuffer byteBuf = ByteBuffer.allocate(4 * descriptor.length);
FloatBuffer floatBuf = byteBuf.asFloatBuffer();
floatBuf.put(floats);
byteBuf.rewind();
return byteBuf;
}
}