We Keep Coding

java.util.set fields serialization/deSerialization using Apache Avro

I am trying to serialize custom object which has java.util.set fields using Apache Avro using below code:
final Schema schemaItemImportSchema = ReflectData.get().getSchema(clazz);
final DatumWriter<T> writer = new ReflectDatumWriter<>(clazz);
byte[] data = new byte[0];
final ByteArrayOutputStream stream = new ByteArrayOutputStream();
try {
final Encoder encoder = EncoderFactory.get().jsonEncoder(schema, stream);
datumWriter.write(data, encoder);
encoder.flush();
data = stream.toByteArray();
} catch (final Exception excp) {
log.error(excp);
}
And deSerialization using below code,
final Schema schemaItemImportSchema = ReflectData.get().getSchema(clazz);
final DatumReader<T> reader = new ReflectDatumReader<>(clazz);
Object dataActual = new Object();
try {
final Decoder decoder = DecoderFactory.get().jsonDecoder(schema, new String(data));
dataActual = reader.read(null, decoder);
} catch (final IOException excp) {
log.error(excp);
}
Using above code I am able to serialize successfully with set fields but during de-serialization getting below error,
java.lang.RuntimeException: java.lang.NoSuchMethodException: java.util.Set.<init>()
If I use #AvroIgnore for set fields, both serialization and de-serialization works perfectly.
How can I serialize and deserialize java.util.set fields?

Resolved this issue by changing type to HashSet instead of set.
Referred https://blog.51cto.com/shadowisper/1947979

Java - How to do Gzip compression of java object

How to compress a Java pojo object using Gzip?
Below code compress a string -
public static String compress(String str, String inEncoding) {
if (str == null || str.length() == 0) {
return str;
}
try {
ByteArrayOutputStream out = new ByteArrayOutputStream();
GZIPOutputStream gzip = new GZIPOutputStream(out);
gzip.write(str.getBytes(inEncoding));
gzip.close();
return URLEncoder.encode(out.toString("ISO-8859-1"), "UTF-8");
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
Instead of String str as a parameter, how to use below pojo class object (Client cc) and compress?
Pojo class -
Class client {
Public string name;
Public string location;
//Getter and setter methods
}
How can i compress and decompress this client pojo class using gzip.?

You can compress your Client class which implements serializable using gzip by doing the following :
public static bytes[] compressThis(Client client){
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream gzipOut = new GZIPOutputStream(client);
ObjectOutputStream objectOut = new ObjectOutputStream(gzipOut);
objectOut.writeObject(client);
objectOut.close();
return baos.toByteArray();
}
Following which you can decompress it by doing the following :
public static getClientFrom(bytes[] bytes){
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
GZIPInputStream gzipIn = new GZIPInputStream(bais);
ObjectInputStream objectIn = new ObjectInputStream(gzipIn);
Client client = (Client) objectIn.readObject();
objectIn.close();
return client;
}

akka Serialization ByteBuffer - java.lang.UnsupportedOperationException

I need some help to serialize an Entity to send it by Akka Remote.
This is the serializer class:
#Override
public void toBinary(Object o, ByteBuffer buf) {
byte[] bytes = null;
ByteArrayOutputStream bos = null;
ObjectOutputStream oos = null;
try {
bos = new ByteArrayOutputStream();
oos = new ObjectOutputStream(bos);
oos.writeObject(o);
oos.flush();
bytes = bos.toByteArray();
}
catch(Exception e){
//System.out.println(e.getStackTrace());
e.printStackTrace();
}
buf.put(bytes);
}
#Override
public Object fromBinary(ByteBuffer buf, String manifest) {
Object obj = null;
ByteArrayInputStream bis = null;
ObjectInputStream ois = null;
try {
bis = new ByteArrayInputStream(buf.array());
ois = new ObjectInputStream(bis);
obj = ois.readObject();
}
catch(Exception e){
//System.out.println(e.getStackTrace());
e.printStackTrace();
}
return obj;
}
Im getting the following exception in line 5
java.lang.UnsupportedOperationException
at java.nio.ByteBuffer.array(ByteBuffer.java:994)
at serializers.ExampleByteBufSerializer.fromBinary(ExampleByteBufSerializer.java:67)
at akka.serialization.Serialization.deserializeByteBuffer(Serialization.scala:190)
at akka.remote.MessageSerializer$.deserializeForArtery(MessageSerializer.scala:91)
at akka.remote.artery.Deserializer$$anon$3.onPush(Codecs.scala:620)
at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:499)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:401)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:571)
at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:457)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:546)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:725)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:740)
at akka.actor.Actor$class.aroundReceive(Actor.scala:513)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:650)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
at akka.actor.ActorCell.invoke(ActorCell.scala:496)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
This is the message Im sending to the remote actor:
public class Message2Serialize implements Serializable {
String nombre;
public Message2Serialize(String nombre) {
this.nombre = nombre;
}
public Message2Serialize() {
}
public String getNombre() {
return nombre;
}
public void setNombre(String nombre) {
this.nombre = nombre;
}
}
The weird thing is that it works in one way, if we send the message using this it works fine in the receptor:
ActorSelection selection = getContext().system().actorSelection("akka://applicationremote#localhost:25521/user/actors.MessageReceiverActor");
selection.tell(message2Serialize, getSelf());
But when we replay to the sender actor using getSender().tell(m, getSelf()); then we got the exception.
We are using Java 1.8 and akka-remote_2.11:2.5.3
Thanks in advance!
Rodri

javadoc extract
#throws UnsupportedOperationException. If this buffer is not backed by an accessible array
the ByteBuffer seems not to be completely initialized...
furthermore the javadoc also says what to do
Invoke the {#link #hasArray hasArray} method before invoking this
method in order to ensure that this buffer has an accessible backing
array.

By changing this line:
bis = newByteArrayInputStream(arr); to
byte[] arr = new byte[buf.remaining()];
buf.get(arr);
bis = new ByteArrayInputStream(arr);
It works, but Im not sure why.

Java ObjectInputStream throws EOFException with bigger object

I have this method to deserialize:
public static Object deserialize(byte[] data) throws IOException, ClassNotFoundException {
ByteArrayInputStream in = new ByteArrayInputStream(data);
ObjectInputStream is = new ObjectInputStream(in);
Object res = is.readObject();
is.close();
in.close();
return res;
}
and this one to serialize:
public static byte[] serialize(Object obj) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
ObjectOutputStream os = new ObjectOutputStream(out);
os.writeObject(obj);
byte[] res = out.toByteArray();
out.close();
os.close();
return res;
}
I use these methods to serialize and deserialize a class object, that only has a string and an arrayList of another class, exchanged between 2 devices. Both the class of the object and the class of the arrayList implement serializable.
When I send an object with up to 3 elements in the arrayList these methods work perfectly. However, when the arrayList has 4 or more elements, the device receiving the object still detects that some data has "arrived" but the deserialize method generates an "EOFException" in the "Object res = is.readObject();" line.
Any ideas about what the problem could be ?
EDIT
This is the class of the arrayList:
import java.io.Serializable;
public class Info implements Serializable {
public Info() {
...
}
...
}
This is the class of the object:
import java.io.Serializable;
import java.util.ArrayList;
public class BluetoothDataContainer implements Serializable{
private ArrayList<Info> dataList;
private String originDevice;
public BluetoothDataContainer(String originDevice){
dataList= new ArrayList<Info>();
this.originDevice = originDevice;
}
...
}
This is the code I use to send the object:
BluetoothDataContainer data = new BluetoothDataContainer(mBluetoothAdapter.getName());
...
// add needed info to variable 'data'
...
s.write(data);
Where 's' is a thread with the method 'write':
private BluetoothSocket mmSocket = bluetoothDevice.createRfcommSocketToServiceRecord(ID_CONNECTION);
private OutputStream mmOutStream = mmSocket.getOutputStream();
...
public void write(BluetoothDataContainer m) {
try {
mmOutStream.write(serialize(m));
} catch (IOException e) {
this.mContext.showToast("IOException caught in thread ConnectedThread [Bluetooth connection handler] - write() !");
}
//cancel();
this.interrupt();
}
And this is how I handle the object when it is read:
private final Handler mHandler = new Handler() {
#Override
public void handleMessage(Message msg) {
switch (msg.what) {
case 1:
byte[] readBuf = (byte[]) msg.obj;
// construct a string from the valid bytes in the buffer
final BluetoothDataContainer data;
try {
data = (BluetoothDataContainer) deserialize(readBuf);
...
} catch (IOException | ClassNotFoundException e) {
e.printStackTrace();
}
break;
default:
break;
}
}
};
And this is how I read the object:
private final Handler mHandler; // value set in the constructor
...
public void run() {
mmInStream = bluetoothSocket.getInputStream();
byte[] buffer = new byte[1024]; // buffer store for the stream
int bytes;
try {
// Read from the InputStream
bytes = mmInStream.read(buffer);
this.mContext.showToast("ConnectedThread [Bluetooth connection handler] data received !");
// Send the obtained bytes to the UI activity
mHandler.obtainMessage(1, bytes, -1, buffer).sendToTarget();
} catch (IOException e) {
this.mContext.showToast("IOException caught in thread ConnectedThread [Bluetooth connection handler] - run() !");
}
}

Clearly you have not 'exchanged' the entire byte array in the case that fails.
bytes = mmInStream.read(buffer);
You can't possibly know from this alone whether you've read:
an entire message
less than one message
more than one message.
As you already have a socket with input and output streams, it beats me why you are creating byte arrays at all. Just wrap the ObjectOutputStream around the socket output stream and use writeObject(). At the receiver, wrap the ObjectInputStream around the socket input stream and use readObject().
NB you should use the same object streams for the life of the socket, and if you are sending objects both ways you must create the object output stream before the object input stream for the same socket: otherwise you can get a deadlock.

BIG Problems with reciving Class Object via Objectoutputstream and bytearrayobjectoutputstream ON TCP NIO

Here is some part of my code:
This I use to receive JSONObject if it is an JSONObject, if it fails to create a JSONObject of the incoming buffer, then it is an object of class PackageFile
that contains metadata and a byte[].
public JSONObject receiveJSONObject(){
JSONObject json =null;
byte[] headerBytes=null;
ByteBuffer buffer=null;
try{
buffer= ByteBuffer.allocateDirect(2000024);
socketChannel.read(buffer);
buffer.flip();
headerBytes = new byte[buffer.remaining()];
buffer.get(headerBytes);
String jsonString = new String(headerBytes);
json = new JSONObject(jsonString);
return json;
}catch(Exception ex){
if(ex.toString().equals("org.json.JSONException: A JSONObject")){
System.out.println("byte[] is a FilePackage");
getDownloadedFilePackage(buffer);
}else{
ex.printStackTrace();
}
json=null;
}
return json;
}
private boolean getDownloadedFilePackage(ByteBuffer buffer){
FilePackage filePackage=null;
ByteArrayInputStream in=null;
ObjectInputStream obIn=null;
try{
if(buffer!=null){
in = new ByteArrayInputStream(buffer.array()); //Fails,,see: PR1
obIn = new ObjectInputStream(Channels.newInputStream(socketChannel));
filePackage= (FilePackage) obIn.readObject();
fdm.addFilePackageToDownloadingProtocol(filePackage);
in.close();
obIn.close();
return true;
}
}catch(Exception ex){
ex.printStackTrace();
}
return false;
}
PR1: When it fails in = new ByteArrayInputStream(buffer.array());, it gives this exception:
java.lang.UnsupportedOperationException
at java.nio.ByteBuffer.array(ByteBuffer.java:959)
at Client.CommunicationProtocol.getDownloadedFilePackage(CommunicationProtocol.java:115)
at Client.CommunicationProtocol.receiveJSONObject(CommunicationProtocol.java:100)
at Client.OperationManagerNIO.run(OperationManagerNIO.java:19)
at java.lang.Thread.run(Thread.java:745)
This part of the code below send the FilePackage:
public void sendFilePackage(FilePackage filePackage,SocketChannel channel){
ByteArrayOutputStream baos =null;
ObjectOutputStream oos =null;
try{
baos = new ByteArrayOutputStream();
oos = new ObjectOutputStream(baos);
oos.writeObject(filePackage);
//baos.writeTo(oos);//not used
oos.flush();
channel.write(ByteBuffer.wrap(baos.toByteArray()));
}catch(Exception ex){
ex.printStackTrace();
}
}
Please help me! I can receive any message with JSONObject without any problems, but the problem is when I'm trying to send a class object, it does not work! I have been on this problem for 2 days now, and I can not see straight anymore.
As I said before, the problem is receiving the FilePackage(Class) object, it throws an exception on the first row that handles the incoming buffer from TCP NIO socketChannel.
I have tried to:
Just send FilePackage object - didn't succeed, the problem lays there, - how to send and retrieve a class object via Objectoutputstream and bytearrayoutputstream.