Is there any restriction on the minimum size of the data that should be present in ByteBuffer so that Jackson will be able to serialize it ? I get BufferUnderflowException while doing so.
But it works fine when the size of data present is large.
public class MyTest {
private static class Wrapper {
private ByteBuffer buffer;
public void setBuffer(ByteBuffer buffer) {
this.buffer = buffer;
}
public ByteBuffer getBuffer() {
return buffer;
}
}
#Test
public void fails() throws Exception {
// Fails
ByteBuffer smallBuffer = ByteBuffer.wrap("small".getBytes());
Wrapper wrapper1 = new Wrapper();
wrapper1.setBuffer(smallBuffer);
System.out.println(new ObjectMapper().writeValueAsBytes(wrapper1));
}
#Test
public void works() throws Exception {
// Works
ByteBuffer smallBuffer = ByteBuffer.wrap("larger string works, wonder why".getBytes());
Wrapper wrapper1 = new Wrapper();
wrapper1.setBuffer(smallBuffer);
System.out.println(new ObjectMapper().writeValueAsBytes(wrapper1));
}
}
Exception stack trace:
org.codehaus.jackson.map.JsonMappingException: (was java.nio.BufferUnderflowException) (through reference chain: com.test.Wrapper["buffer"]->java.nio.HeapByteBuffer["int"])
at org.codehaus.jackson.map.JsonMappingException.wrapWithPath(JsonMappingException.java:218)
at org.codehaus.jackson.map.JsonMappingException.wrapWithPath(JsonMappingException.java:183)
at org.codehaus.jackson.map.ser.std.SerializerBase.wrapAndThrow(SerializerBase.java:140)
at org.codehaus.jackson.map.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:158)
Jackson will normally run into problems when serializing complex objects with lots of internal state and/or non-standard getter/setter methods. In general, you should always stick to pure POJOs in the object hierarchy to be serialized.
In this particular case, you've created a wrapper that contains a ByteBuffer. Well, Jackson actually tries to serialize the entire byte buffer object (not just its byte contents), and if you look under the covers its actually trying to write out all these 'properties':
[
property 'short' (via method java.nio.HeapByteBuffer#getShort),
property 'char' (via method java.nio.HeapByteBuffer#getChar),
property 'int' (via method java.nio.HeapByteBuffer#getInt),
property 'long' (via method java.nio.HeapByteBuffer#getLong),
property 'float' (via method java.nio.HeapByteBuffer#getFloat),
property 'double' (via method java.nio.HeapByteBuffer#getDouble),
property 'direct' (via method java.nio.HeapByteBuffer#isDirect),
property 'readOnly' (via method java.nio.HeapByteBuffer#isReadOnly)
]
Its just by dumb luck that the second case works (because the buffer is long enough to get all the above shown methods invoked on it without underflowing). If you want to serialize the buffer as bytes, then either:
change the wrapper property to byte[]
keep the property as ByteBuffer, but mark it as #JsonIgnoreable and provide an alternate accessor method around it that converts to a byte[]
Example of first:
class Wrapper {
final byte[] buffer;
public Wrapper(final ByteBuffer buffer) {
super();
this.buffer = buffer != null ? buffer.array() : new byte[0];
}
}
Related
I have an instance of an object for which I need to create proxy to intercept one of the methods:
The object implements an interface, but I need to proxy the full type not just implement the interface.
I don't know the exact type of the object, only its interface class.
There are no accessible public constructors.
The object is Serializable.
I have full accessibility to read the library code, but no ability to change any of it.
So what I need to do is something like:
TheObject obj = library.getObject();
TheObject proxy = createProxyObject(obj);
library.doSomethingWith(proxy);
It seems to me that theoretically this should be possible as the object is Serializable, but I can't find any way of using that.
Note on the following: I've been trying using cglib but I'm not tied to that at all. If it is possible in asm, javaassist, or any other library that would be fine.
What I have so far with cglib is I can proxy a simple object with a public constructor:
public class SimpleObject {
private String name;
public void setName(String name) {
this.name = name;
}
public String getName() {
return name;
}
// return a random number
public int getRandom() {
return (int)(Math.random() * 100);
}
}
public void testCglibEnhancer() throws Exception {
SimpleObject object = new SimpleObject();
object.setName("object 1");
System.out.println(object.getName() + " -> " + object.getRandom());
Enhancer enhancer = new Enhancer();
enhancer.setSuperclass(object.getClass());
// intercept getRandom and always return 32
ProxyRefDispatcher passthrough = proxy -> object;
MethodInterceptor fixedRandom = (obj, method, args, proxy) -> 32;
enhancer.setCallbacks(new Callback[]{passthrough, fixedRandom});
enhancer.setCallbackFilter(method -> method.getName().equals("getRandom") ? 1 : 0);
SimpleObject proxy = (SimpleObject)enhancer.create();
System.out.println(proxy.getName() + " -> " + proxy.getRandom()); // always 32
}
But I've been unable to replicate this using an object with no public constructor:
public static class ComplexObject implements Serializable {
public static ComplexObject create() {
return new ComplexObject();
}
private String name;
private ComplexObject() {
}
public void setName(String name) {
this.name = name;
}
public String getName() {
return name;
}
public int getRandom() {
return (int)(Math.random() * 100);
}
}
ComplexObject proxy = (ComplexObject)enhancer.create();
// throws IllegalArgumentException: No visible constructors
As the object is Serializable, I can clone it:
public static <T extends Serializable> T cloneViaSerialization(T source) throws IOException, ClassNotFoundException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream out = new ObjectOutputStream(bos);
out.writeObject(source);
ByteArrayInputStream bis = new ByteArrayInputStream(bos.toByteArray());
ObjectInputStream in = new ObjectInputStream(bis);
return (T)in.readObject();
}
public void testClone() throws Exception {
ComplexObject object1 = ComplexObject.create();
object1.setName("object 1");
ComplexObject object2 = cloneViaSerialization(object1);
object2.setName("object 2");
System.out.println(object1.getName() + " -> " + object1.getRandom());
System.out.println(object2.getName() + " -> " + object2.getRandom());
}
So is there any way I can get cglib (or any library) to use this approach?
ComplexObject object = library.getObject();
ObjectInputStream in = ... // serialised version of object
Enhancer enhancer = new Enhancer();
enhancer.setSuperclass(object.getClass());
// add callbacks etc.
// note createFromSerialized is not a existing method of
// Enhancer - it is what I'm trying to synthesise somehow
ComplexObject proxy = (ComplexObject)enhancer.createFromSerialized(in);
Thanks
Got it working:
Created a derived class definition using ASM.
Added this class definition to the same class loader as the target
class.
Serialised the existing object to a byte array.
Injected the derived class name into the serialised byte array.
Deserialised the resulting bytes using standard
ObjectInputStream.readObject.
For (1), I couldn't get cglib or byte-buddy to create the class I needed, so I switched to ASM.
For (2), I used a custom class loader to load the whole jar file containing the target class.
This does mean I end up with a clone of the original object not a proxy as per the question, but that works fine for what I need.
For the serialisation hack:
I created a few example minimal classes, serialized them to disk and compared the resulting binary data.
To inject the class name:
Serialised the full parent object to a byte array.
Manually created a byte array for the derived class header (see below).
Both byte arrays contain the stream header data (magic, version and initial object type), so I took that from (2) for simplicity, although I did have to tweak a couple of bytes.
So then just created a stream of all of (2) followed by all of (1) except the first 6 bytes.
To create the byte array for the derived class, I just looked at the delivered Java source and came up with the following:
/*
* Returned array contains:
* - stream header
* - class header
* - end block / class desc markers for next class
*/
private byte[] derivedClass(Class<?> clss) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
ObjectStreamClass osc = ObjectStreamClass.lookup(clss);
oos.writeUTF(osc.getName());
oos.writeLong(osc.getSerialVersionUID());
oos.write(SC_SERIALIZABLE); // flags
oos.writeShort(0); // field count
oos.writeByte(TC_ENDBLOCKDATA);
oos.writeByte(TC_CLASSDESC);
oos.flush();
// header appears to write 0x77 (TC_BLOCKDATA) and 0x54 (???) for bytes 5 & 6
// samples streamed from other files use 0x73 (TC_OBJECT) and 0x72 (TC_CLASSDESC)
byte[] bytes = baos.toByteArray();
bytes[4] = TC_OBJECT;
bytes[5] = TC_CLASSDESC;
return bytes;
}
Have not tested this as a general approach, but appears to work fine for my case.
So for a homework assignment, I have a example of how to marshal data and unmarshal.
The structure they gave us was this:
Event is an interface.
Wireformat is a class that "inherits" an Event.
WireFormatWidget is a class with the actual code that has the marshal and unmarshal.
I have separate threads that handle the sending data in byte array using TCP.
What I have an issue is that when I create a Wireformat object. I run into issue with a thread trying to marshal the data.
Exception in thread "main" java.lang.NullPointerException
at myhw.WriteFormatWidget.getBytes(WriteFormatWidget.java:38)
The interface structure defines the data as a message, a type of message as an integer, a timestamp (of what I am assuming is Date and getTime of that date), and a tracker. I am not sure what the tracker is.
I am told this structure is the best method to sending data which is why I am trying to implement this code style.
The WriteFormatWidget consist of this:
private int type;
private long timestamp;
private String identifier;
private int tracker;
So for my wireformat, I created it as a class that extends WireFormatWidget and implements Event because that was the only way Eclipse did not spit an error or suggest changing WireFormatWidget or Event.
Now when I hardcode my specific wireformat, I instantiate it and it seems to not be able to call getBytes() with the hardcoded values I uses for the same variables.
public class MyWireFormat extends WireFormatWidget implements Event {
private String identifier = "here is my custom wireformat";
....
When I print out the identifier in the getBytes in WireFormatWidget, I get null and not the expected identifier I hardcoded. So I must not be "inheriting" appropriately. What am I doing wrong?
EDIT: WireFormatWidget (given)
public class WriteFormatWidget {
private int type;
private long timestamp;
private String identifier;
private int tracker;
public byte[] getBytes() throws IOException {
byte[] marshalledBytes = null;
ByteArrayOutputStream baOutputStream = new ByteArrayOutputStream();
DataOutputStream dout = new DataOutputStream(new BufferedOutputStream(baOutputStream));
dout.writeInt(type);
dout.writeLong(timestamp);
System.out.println("getBytes using identifier: " + identifier);
byte[] identifierBytes = identifier.getBytes();
int elementLength = identifierBytes.length;
dout.writeInt(elementLength);
dout.write(identifierBytes);
dout.writeInt(tracker);
dout.flush();
marshalledBytes = baOutputStream.toByteArray();
baOutputStream.close();
dout.close();
return marshalledBytes;
}
}
I'll save space by not posting the unmarshalling portion. But its the same thing just in reverse.
The issue I am having is printing the data from the Client-side as proof of what I am sending beforehand.
So I will perform a simple test like print the type or print the identifier. It fails and I have null.
You're not initializing WireFormatWidget#identifier. It's declared but never initialized. Add a constructor to WireFormatWidget and provide a String as the identifier.
You need to implement something that implements Serializable, or implement directly Serializable (I think is simpler).
You do not specify many things about your interface event, but probably will inherit from Serializable, at least if you are going to implement standard java serialization.
If Event implements Serializable so it is ok, otherwise if you use another serialization method you need to specify more about it.
Assuming that you implement Serializable you need to create a ByteBuffer and call to writeObject. To create the stream you can check for example Java Serializable Object to Byte Array, so joining all:
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream stream = new ObjectOutputStream(bos);
stream.writeObject(yourinstancetoserialize);
out.flush();
byte[] yourBytes = bos.toByteArray();
...
Probably you will need to implement the writeObject directly. In that case you use the ObjectOutputStream methods to serialize the properties, check them in https://docs.oracle.com/javase/7/docs/api/java/io/ObjectOutputStream.html for example.
private void writeObject(java.io.ObjectOutputStream stream) throws IOException {
stream.writeInt(this.type);
stream.writeLong(this.timestamp);
stream.writeBytes(this.identifier); or stream.writeChars(this.identifier);
stream.writeInt(this.tracker);
...
}
I just heard of AspectJ and it doesn't look too easy to understand, so I want to know beforehand if it (or anything else) will help me with my problem or not.
I have bunch of simple POJO classes and want to write binary serializers for them but without writing Write/Read methods by hand for each class. I could've done so with help of reflection but that will affect runtime performance. I believe I need something similar to Macroses in Scala with compile-time reflection and quasiquotes.
Update:
I'm unable to use any serialization available, because I have custom binary protocol which I can't modify (online game)
Update 2:
Example POJO with it's read, write and some helper methods. Not final version, there possibly could be some annotations, for example, but general structure should be the same. I also omitted inheritance for simplicity, in reality LoginPacket extends CommandPacket class which in turn extends Packet class.
public class LoginPacket {
public short length;
public int sessionId;
public short command;
public short error;
public String reason;
private String getString(ByteBuffer data) {
short length = data.getShort();
byte[] stringData = new byte[length];
data.get(stringData);
return new String(stringData, "UTF-8");
}
private void putString(ByteBuffer data, String someString) {
data.putShort(someString.length());
byte[] stringData = someString.getBytes("UTF-8");
data.put(stringData);
}
public static LoginPacket read(ByteBuffer data) {
LoginPacker loginPacket = new LoginPacket();
loginPacket.length = data.getShort();
loginPacket.sessionId = data.getInt();
loginPacket.command = data.getShort();
loginPacket.error = data.getShort();
loginPacket.reason = getString(data);
return loginPacket;
}
public void write(ByteBuffer data) {
data.putShort(this.length);
data.putInt(this.sessionId);
data.putShort(this.command);
data.putShort(this.error);
putString(data, this.reason);
}
}
I don't think you need to use AspectJ to modify your classes. I don't see what benefits using compile team weaving would add. I would suggest having your POJOs use implements Serializableand then serialize your objects using an ObjectOutputStream.
A simple example writing an object to a file:
outputStream = new ObjectOutputStream(new FileOutputStream(filePath));
outputStream.writeObject(yourObject);
...
// do whatever else and close stream
Similar questions:
Saving to binary/serialization java
Best way to store data for your game? (Images, maps, and such)
Is there a way to write data to an OutputStream object which is connected to a channel, and this channel will pass the data to a bytebuffer (preferably, direct bytebuffer)?
I have a situation where a third party function can write its output to an outputStream. I want to be able to write this data to a bytebuffer using channels.
Is it possible?
Thank you
You can easily create a class that extends OutputStream since this only requires one method to be implemented. Sample, untested code:
public final class ByteBufferOutputStream
extends OutputStream
{
private final ByteBuffer buf;
public ByteBufferOutputStream(final int size)
{
buf = ByteBuffer.allocateDirect(size);
}
#Override
public void write(final int b)
throws IOException
{
if (buf.remaining() == 0)
throw new IOException("buffer is full");
buf.put((byte) (b & 0xff));
}
}
Then just pass an instance of that class to your API. You'll also probably want to override the other write methods since ByteBuffer has dedicated methods to write byte arrays.
I have a Simple XML annotated class that I want to use for serialization/deserialization. I have a byte[] array which I am using with a custom converter but it fails at the read method..
Here is my Simple XML annotated object
#Root
public class Device implements Serializable
{
#Element
#Convert(ByteArrayConverter.class)
protected byte[] imageRef;
...
}
Here is my converter
import org.simpleframework.xml.convert.Converter;
import org.simpleframework.xml.stream.InputNode;
import org.simpleframework.xml.stream.OutputNode;
public class ByteArrayConverter implements Converter<byte[]>
{
#Override
public byte[] read(InputNode node) throws Exception
{
String value = node.getValue();
//return value.getBytes();
return new byte[]{1,2,3,4,5};
}
#Override
public void write(OutputNode node, byte[] byteArray) throws Exception
{
node.setValue("something");
}
}
Here is my XML called device.xml
<device>
<imageRef>AQIDBAU=</imageRef>
</device>
Here is my serialization code which is failing
Strategy strategy = new AnnotationStrategy();
Serializer serializer = new Persister(strategy);
File file = new File("device.xml");
Device device = serializer.read(Device.class, file);
I have put a break point in ByteArrayConverter.read() method and say that it executes correctly and returns a byte array. However after that somewhere in Simple XML I get this
Exception in thread "main"
org.simpleframework.xml.core.InstantiationException: Incompatible
class [B for field 'imageRef' protected byte[]
com.test.Device.imageRef at line 4 at
org.simpleframework.xml.core.Factory.getOverride(Factory.java:147) at
org.simpleframework.xml.core.PrimitiveFactory.getInstance(PrimitiveFactory.java:82)
at
org.simpleframework.xml.core.Primitive.readElement(Primitive.java:186)
at org.simpleframework.xml.core.Primitive.read(Primitive.java:124)
at
org.simpleframework.xml.core.Composite.readVariable(Composite.java:623)
at
org.simpleframework.xml.core.Composite.readInstance(Composite.java:573)
at
org.simpleframework.xml.core.Composite.readUnion(Composite.java:549)
at
org.simpleframework.xml.core.Composite.readElement(Composite.java:532)
at
org.simpleframework.xml.core.Composite.readElements(Composite.java:445)
at
org.simpleframework.xml.core.Composite.access$400(Composite.java:59)
at
org.simpleframework.xml.core.Composite$Builder.read(Composite.java:1383)
at org.simpleframework.xml.core.Composite.read(Composite.java:201)
at org.simpleframework.xml.core.Composite.read(Composite.java:148)
at org.simpleframework.xml.core.Traverser.read(Traverser.java:92) at
org.simpleframework.xml.core.Persister.read(Persister.java:625) at
org.simpleframework.xml.core.Persister.read(Persister.java:606) at
org.simpleframework.xml.core.Persister.read(Persister.java:584) at
org.simpleframework.xml.core.Persister.read(Persister.java:543) at
org.simpleframework.xml.core.Persister.read(Persister.java:521) at
org.simpleframework.xml.core.Persister.read(Persister.java:426)
Any pointers will be helpful..
You are using a primitive type array, this may cause the exception. Is your byte array encoded (e.g. base64)? If you use an encoding, you have to care about this in read() / write() too.
A Converter is not the best choice here, better take Transform. With those you can implemented the Object <-> String transformation by your own.
Here's a very simple example:
public class ByteArrayTransformer implements Transform<byte[]>
{
#Override
public byte[] read(String value) throws Exception
{
/*
* TODO: Implement proper encoding
*/
return value.getBytes();
}
#Override
public String write(byte[] value) throws Exception
{
/*
* TODO: Implement proper encoding
*/
return new String(value);
}
}
This allows you to remove the converter (#Convert annotation of the field and the AnnotationStrategy).
And finally here's how to bind a transformer:
RegistryMatcher m = new RegistryMatcher();
m.bind(byte[].class, new ByteArrayTransformer());
Serializer ser = new Persister(m);
Device d = ser.read(Device.class, f);