We Keep Coding

Kryo: Difference between readClassAndObject/ReadObject and WriteClassAndObject/WriteObject

I am trying to understand the following statement from the documentation:
If the concrete class of the object is not known and the object couldbe null:
kryo.writeClassAndObject(output, object);
Object object = kryo.readClassAndObject(input);
What does if the concrete class is not known exactly.
I am having the following code:
case class RawData(modelName: String,
sourceType: String,
deNormalizedVal: String,
normalVal: Map[String, String])
object KryoSpike extends App {
val kryo = new Kryo()
kryo.setRegistrationRequired(false)
kryo.addDefaultSerializer(classOf[scala.collection.Map[_,_]], classOf[ScalaImmutableAbstractMapSerializer])
kryo.addDefaultSerializer(classOf[scala.collection.generic.MapFactory[scala.collection.Map]], classOf[ScalaImmutableAbstractMapSerializer])
kryo.addDefaultSerializer(classOf[RawData], classOf[ScalaProductSerializer])
//val testin = Map("id" -> "objID", "field1" -> "field1Value")
val testin = RawData("model1", "Json", "", Map("field1" -> "value1", "field2" -> "value2") )
val outStream = new ByteArrayOutputStream()
val output = new Output(outStream, 20480)
kryo.writeClassAndObject(output, testin)
output.close()
val input = new Input(new ByteArrayInputStream(outStream.toByteArray), 4096)
val testout = kryo.readClassAndObject(input)
input.close()
println(testout.toString)
}
When I use readClassAndObject and writeClassAndObject is works. However if I use writeObject and readObject it does not.
Exception in thread "main" com.esotericsoftware.kryo.KryoException:
Class cannot be created (missing no-arg constructor):
com.romix.scala.serialization.kryo.ScalaProductSerializer
I just don't understand why.
earlier using the same code, Instead of using my class RawData, I used a Map and it worked like a charm with writeObject and ReadObject. Hence i am confused.
Can someone help understand it ?

The difference is as follows:
you use writeClassAndObject and readClassAndObject when you're using a serializer that:
serializes a base type: an interface, a class that has subclasses, or - in case of Scala - a trait like Product,
and needs the type (i.e. the Class object) of the deserialized object to construct this object (without this type, it doesn't know what to construct),
example: ScalaProductSerializer
you use writeObject and readObject when you're using a serializer that:
serializes exactly one type (i.e. a class that can be instantiated; example: EnumSetSerializer),
or serializes more than one type but the specific type can be somehow deduced from the serialized data (example: ScalaImmutableAbstractMapSerializer)
To sum this up for your specific case:
when you deserialize your RawData:
ScalaProductSerializer needs to find out the exact type of Product to create an instance,
so it uses the typ: Class[Product] parameter to do it,
as a result, only readClassAndObject works.
when you deserialze a Scala immutable map (scala.collection.immutable.Map imported as IMap):
ScalaImmutableAbstractMapSerializer doesn't need to find out the exact type - it uses IMap.empty to create an instance,
as a result, it doesn't use the typ: Class[IMap[_, _]] parameter,
as a result, both readObject and readClassAndObject work.

TypeReference<Map<String, String>>() { }

Since few days ago I started to work on a webservice project. This project is using Jackson to marshalling and unmarshalling JSON objects. So my question is:
Why always I have to put the {} when I am creating an instance of TypeReference? I know the constructor is protected, but why is protected? I think that it's like a hack to make visible the constructor creating an implementation of the constructor since TypeReference is abstract and you can do it. But what is the point of this?
String jsonString = "{\" firstName\":\"John\",\"lastName\":\"Chen\"}";
ObjectMapper objectMapper = new ObjectMapper();
// properties will store name and value pairs read from jsonString
Map<String, String> properties = objectMapper.readvalue(
jsonString, new TypeReference<Map<String, String>>()
{ //
});

TL;DR
Via subclassing it is possible for TypeReference to extract the actual generic type parameter. E.g:
TypeReference<String> ref = new TypeReference<String>(){};
System.out.println(ref.getType());
Prints:
class java.lang.String
This can be useful when you can't use normal classes. E.g when this doesn't work:
// doesn't work
Type type = ArrayList<String>.class;
You still can get that class by using a TypeReference:
// will yield Class<ArrayList<String>>>
Type type = new TypeReference<ArrayList<String>>(){}.getType();
Detailed
When looking at the source code of TypeReference (using Jackson 2.8.5) you can see that the constructor body contains the following lines:
Type superClass = getClass().getGenericSuperclass();
if (superClass instanceof Class<?>) { // sanity check, should never happen
throw new IllegalArgumentException("Internal error: TypeReference constructed without actual type information");
}
_type = ((ParameterizedType) superClass).getActualTypeArguments()[0];
The interesting lines are the first and last. Let's take a closer look at the first line:
Type superClass = getClass().getGenericSuperclass();
For example when you're creating a subclass, by using an anonymous class:
TypeReference<SomeStype> ref = new TypeReference<SomeType>(){};
Then getClass returns the current Class object (an anonymous class), and getGenericSuperclass() will return the Class object from the class the current implementation extends from, in our case, superClass will equal Class<TypeReference<?>>.
Now when looking at the last line from the constructor body:
_type = ((ParameterizedType) superClass).getActualTypeArguments()[0];
As we know that the superClass is the Class object for TypeReference<?> we know that it has a generic parameter. Hence the cast to ParameterizedType. This specified Type has the method getActualyTypeArguments() which returns an array of all generic parameters specified by that class. In our case it's just 1. So [0] will yield the first element. In the example we will get the actually specified type parameter SomeType.

How to dynamically deserialize generic class with multiple occurances of T

I am trying to serialize instances of a generic class while preserving the generic types, so I will be able to deserialize it later without having to specify the generic type manually.
I understand that the usual way of deserializing generic objects is by using type references or JavaType objects like this:
ObjectMapper om = new ObjectMapper();
ObjectReader or = om.reader();
ObjectWriter ow = om.writer();
String json = "[1, 2, 3]"
JavaType listType = or.getTypeFactory()
.constructParametricType(List.class, Integer.class);
List<Integer> integers = or.forType(listType).readValue(json)
But I do not know the generic type (Integer in this case) beforehand, therefore I can't do this.
I also understand that, because of erasure, I'll have to include the type information within the serialized JSON in some way or another. This can be done with the #JsonTypeInfo annotation:
class Pojo<T> {
#JsonTypeInfo(use = JsonTypeInfo.Id.CLASS)
public T value;
}
However, this quickly becomes bloated if the type T is used in various other places. Consider the following example:
class Pojo<T> {
#JsonTypeInfo(use = JsonTypeInfo.Id.CLASS)
public T value;
#JsonTypeInfo(use = JsonTypeInfo.Id.CLASS)
public List<T> otherStuff;
// constructor
}
// ...
Pojo<BigDecimal> pojo = new Pojo<>(
BigDecimal.valueOf(42),
Lists.newArrayList(BigDecimal.valueOf(14), BigDecimal.valueOf(23))
);
final String json = ow.writeValueAsString(pojo);
System.out.println(json);
produces the following result:
{
"value": ["java.math.BigDecimal", 42],
"otherStuff":[
["java.math.BigDecimal", 14],
["java.math.BigDecimal", 23]
]
}
which repeats the type, BigDecimal in this case, for every single object. This is unnecessary because the type is the same for all occurances of T anyway (except in some polymorphic cases I suppose).
If the #JsonTypeInfo annotation is omitted for otherStuff, Jackson is unable to defer the type of the contents of otherStuff from the type of value. In this example it will deserialize otherStuff as List<Integer>, even though value is of type BigDecimal.
How can I serialize instances of a generic class, so that I can safely deserialize them later and retain the generic argument?

The type information indeed needs to be included in the serialized json string, but only once. The easiest way I know of is to write a custom creator method using the #JsonCreator annotation to perform the deserialization in two steps:
Let Jackson deserialize all non-generic fields, and one generic field that includes the type information. Capture the other generic fields as raw JsonNode, so they can be manually deserialized.
Take that type information at runtime to deserialize the remaining fields.
For the example above this would look like this (exception handling code omitted):
class Pojo<T> {
#JsonTypeInfo(use = JsonTypeInfo.Id.CLASS)
public T value;
public List<T> otherStuff;
// constructor
#JsonCreator
public static <T> Pojo<T> jsonCreator(
#JsonProperty("value") T value,
#JsonProperty("otherStuff") JsonNode otherStuffRaw) {
JavaType listType = or.getTypeFactory()
.constructParametricType(List.class, value.getClass());
return new Pojo<T>(
value,
or.forType(listType).readValue(otherStuffRaw)
);
}
}
This, however, allows value to be a subclass of T, which might produce unexpected results. If this is a concern, another approach might be using Class<T> to retain the exact type of T. This would also shield against value possibly being null.

Can jackson determine root object type to deserialize to when json includes type property?

Assume serialization to json includes the class name of the actual object, using this annotation on the Class:
#JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "#type")
class MyClass {
String foo;
}
So json is for example:
{"#type": "com.example.MyClass", "foo": "bar"}
Can this be deserialized without specifying the type? And I mean not even the super type. Just something like:
objectMapper.readValue(value, Object.class);
which doesn't actually work, it brings back a Map.

Well, it is certainly possible to do that although I have personally never used Jackson that way. You can deserialize it to a JsonNode object and then convert it to the proper type.
final ObjectMapper objectMapper = new ObjectMapper();
final MyClass myClass = new MyClass();
myClass.foo = "bar";
// Serialize
final String json = objectMapper.writeValueAsString(myClass);
// Deserialize
final JsonNode jsonNode = objectMapper.readTree(json);
// Get the #type
final String type = jsonNode.get("#type").asText();
// Create a Class-object
final Class<?> cls = Class.forName(type);
// And convert it
final Object o = objectMapper.convertValue(jsonNode, cls);
System.out.println(o.getClass());
The output is:
MyClass

ObjectMapper mapper = new ObjectMapper();
mapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
MyClass original = new MyClass();
original.foo = "hello";
String json = mapper.writeValueAsString(original);
MyClass foo = (MyClass) mapper.readValue(json, MyClass.class);
This should work and is very convenient.

Yes, but there is a caveat: type you give MUST be something that includes #JsonTypeInfo you specify. Object.class will not have, unless you use "mix-in annotations" to associate it.
However, if you need to add type information for properties of (declared type of) java.lang.Object, you probably want to enable default typing: see ObjectMapper.enableDefaultTyping(...) for more information.
That will actually enable inclusion (and use) of type information for larger categories of classes, without need to add annotations.

What you want to use is Jackson Polymorphic Deserialization.

Jackson Polymorphic Deserialisation (where the class depends on the JSON key)

TL;DR
basically, my problem is that i have a list of wrapper objects
{"stuff": [
{"foobar" : {someObjectOfTypeA}},
{"barfoo" : {someObjectOfTypeB}},
{"foobar" : {someObjectOfTypeA}}
]}
and the type of someObjectOfTypeX depends on the value of the key "foobar" or "barfoo". how can i deserialize this? (for now) serializing is not a problem.
long version
i don't know enough jackson to solve the following problem. i've tried, but i'm stuck.
the json structure i want to parse looks like this:
{
"id": "foobar",
"responses": [
{
"responseType1": {
"code": 0,
"foo": "bar"
}
},
{
"responseType2": {
"code": 1,
"bar": {"foo": ...}
}
},
{
"responseType1": {
"code": 1,
"foo": "foobar"
}
}
]
}
i tried to deserialize it using jacksons full data binding. my pojos are:
// pseudocode
// the outermost object
#JsonCreator
ServiceResponse(
#JsonProperty("id") String id,
#JsonProperty("responses") ArrayList<ResponseWrapper> responses)
// every response has a wrapper. the wrapper is an object with just one key and one value. the value is an object of a certain class (ResponseTypeX extends AResponse), and the exact ResponseType is identified by the key (the key isn't the class name though).
#JsonCreator
ResponseWrapper(AResponse keyDependsOnTypeOfAResponse ???)
// base class for all responseTypeX classes
// all subclasses of AResponse have a code and a complex payload object
#JsonCreator
AResponse (
#JsonProperty("code") int code)
// one response type
// here, the payload is just a string, in reality it's a deep structure, so i dont want to parse this manually
#JsonCreator
ResponseType1 extends AResponse (
#JsonProperty("code") int code,
#JsonProperty("foo") String foo)
// one response type
#JsonCreator
ResponseType2 extends AResponse (
#JsonProperty("code") int code,
#JsonProperty("bar") SomeOtherObject foo)
as you can see, responses is an array of wrapper objects; the "payload" class of the wrapper object is identified by the key (but the keys aren't a 1:1 match to class names). my ResponseTypeX's are limited, there are about 20 of them, so if i have to do a manual key:value type identification, i'm happy.
but is it possible to write a manual deserializer for the WrapperResponse object and continue deserializing its children with full data binding? if so, how?
i tried to just make the Wrapper accept all possible ResponseTypes as properties, hoping it would just nullify the "unset" ones, e.g.
#JsonCreator
ResponseWrapper(
#JsonProperty("responseKey1") ResponseType1 response1,
#JsonProperty("responseKey2") ResponseType2 response2,
#JsonProperty("responseKey3") ResponseType3 response3,
...)
but this failed, probably because all ResponseTypes are subclasses of AResponse and thus jackson gets confused.

Some custom deserialization processing is necessary. I'd recommend including a simple registry (map) of foobar/barfoo-to-type entries in the solution, much like the sixth example in my old blog post from May 25, 2011, "Deserialize JSON with Jackson into Polymorphic Types - A Complete Example".