Well, Arongo DB Java driver has no problems to store Kotlin data classes but it cannot load them back.
Showcase:
import com.arangodb.ArangoCollection
import com.arangodb.ArangoDB
import com.arangodb.entity.DocumentCreateEntity
fun main(args: Array<String>) {
// Get or recreate collection: "some_collection" in DB "test_db"
val collection: ArangoCollection = with(ArangoDB.Builder().build()!!.db("test_db")) {
if (!exists()) create()
with(collection("some_colelction")) {
if (!exists()) create()
this
}
}
// POJO as Kotlin data class
data class Foo(
val topic: String,
val answer: Int
)
val result: DocumentCreateEntity<Foo> = collection.insertDocument(Foo("The ultimate answer", 42))
// reusult have a key of the new document
// And in ArangoDB Web Interface you can see this document: {"answer":42,"topic":"The ultimate answer"}
// http://localhost:8529/_db/test_db/_admin/aardvark/index.html#collection/some_colelction/documents/
// But it doesn't work backwards
val foo: Foo = collection.getDocument(result.key, Foo::class.java)
}
Stacktrace:
Exception in thread "main" com.arangodb.ArangoDBException: com.arangodb.velocypack.exception.VPackParserException: java.lang.InstantiationException: MainKt$main$Foo
at com.arangodb.internal.util.ArangoDeserializerImpl.deserialize(ArangoDeserializerImpl.java:59)
at com.arangodb.internal.util.ArangoUtilImpl.deserialize(ArangoUtilImpl.java:58)
at com.arangodb.internal.ArangoExecutor.createResult(ArangoExecutor.java:112)
at com.arangodb.internal.ArangoExecutorSync$1.deserialize(ArangoExecutorSync.java:56)
at com.arangodb.internal.ArangoExecutorSync.execute(ArangoExecutorSync.java:72)
at com.arangodb.internal.ArangoExecutorSync.execute(ArangoExecutorSync.java:53)
at com.arangodb.internal.ArangoExecutorSync.execute(ArangoExecutorSync.java:49)
at com.arangodb.internal.ArangoCollectionImpl.getDocument(ArangoCollectionImpl.java:134)
at com.arangodb.internal.ArangoCollectionImpl.getDocument(ArangoCollectionImpl.java:126)
at MainKt.main(main.kt:30)
Caused by: com.arangodb.velocypack.exception.VPackParserException: java.lang.InstantiationException: MainKt$main$Foo
at com.arangodb.velocypack.VPack.deserialize(VPack.java:398)
at com.arangodb.internal.util.ArangoDeserializerImpl.deserialize(ArangoDeserializerImpl.java:55)
... 9 more
Caused by: java.lang.InstantiationException: MainKt$main$Foo
at java.lang.Class.newInstance(Class.java:427)
at com.arangodb.velocypack.VPack.createInstance(VPack.java:488)
at com.arangodb.velocypack.VPack.deserializeObject(VPack.java:450)
at com.arangodb.velocypack.VPack.getValue(VPack.java:569)
at com.arangodb.velocypack.VPack.deserialize(VPack.java:396)
... 10 more
Caused by: java.lang.NoSuchMethodException: MainKt$main$Foo.<init>()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.newInstance(Class.java:412)
... 14 more
Kotlin’s data classes nicely serialized into expected JSON documents but seems like ArangoDB Java driver cannot load them back.
If I get the document as BaseDocument I have no problems to map it back to my data class using some JSON library but come on! There must be a better way or I definitely missed something.
The ArangoDB Java driver supports alternative serializer to de-/serialize documents, edges and query results. One implementation is VelocyJack which is based on Jackson working with jackson-dataformat-velocypack.
You should be able to configure it, that it is working together with the jackson-kotlin-module.
VelocyJack velocyJack = new VelocyJack();
velocyJack.configure((mapper) -> {
mapper.registerModule(new KotlinModule())
});
ArangoDB arango = new ArangoDB.Builder().serializer(velocyJack).build();
ArangoDB uses its own serialization framework - VelocyPack - to serialize and deserialize classes. As you can see in the code (and in the stacktrace you provided) that framework needs parameterless constructor to create an instance of deserialized class, which Kotlin's data classes do not have. As far as I know VelocyPack does not have a module for working with Kotlin data classes (like Jackson does), so your only option would be to create custom deserializer for your class and register it - it's possible with VelocyPack (see documentation), so I assume it's also possible in ArrangoDB.
Edit: Class ArrangoDB.Builder has method registerDeserializer(final Class<T> clazz, final VPackDeserializer<T> deserializer), which I assume can be used to register custom JSON deserializers for VelocyPack.
As a workaround for the vert.x project with com.fasterxml.jackson.module:jackson-module-kotlin dependency you can add a custom inlined extension function with a reified type so it will generically extract hashmap of the document then let Jackson’s Kotlin module to do the magic:
inline fun <reified T> ArangoCollection.getDoc(key: String): T =
JsonObject(getDocument(key, BaseDocument::class.java).properties).mapTo(T::class.java)!!
Then this line works with type inferring by the Kotlin compiler:
val for: Foo = collection.getDoc("document-key")
Known issues:
Does not consider ArangoDB document native properties like: _id, _key, _rev, _from, _to
Seems like Jackson still have issues with anonymous classes
Any ideas on how to improve it or how to reduce conversion overhead?
You have to make the fields in your data class mutable by using var instead of val. Val means the fields are final.
Next you need a parameterless constructor. You can achieve this by setting default values for the fields in your constructor or setting them null by default. If you decide to set them null you have to remove the Null safety from your fields by adding a '?' behind the data types.
More on removing Null Safety: https://kotlinlang.org/docs/reference/null-safety.html
Final data class with defaults:
data class Foo(
var topic: String = "",
var answer: Int = ""
)
Final data class with null:
data class Foo(
var topic: String? = null,
var answer: Int? = null
)
In your code you should use:
arango.setSerializer(VelocyJack...
instead of:
arango.serializer(VelocyJack...
otherwise you only use it for serializing and not for deserializing.
I created a pull request which you can use as workaround, using the kotlin no-arg maven plugin, here:
https://github.com/slavaatsig/arangodb-jackson-dataformat-velocypack-issue/pull/1/files
Even if the jackson KotlinModule works with data classes (as I verified here: https://github.com/rashtao/arangodb-jackson-dataformat-velocypack-issue/blob/dbg_serde/src/main/kotlin/com/example/issue/main.kt#L16-L21 ), somehow the driver tries accessing the no-arg constructor...
For further updates on this issue: https://github.com/arangodb/arangodb-java-driver/issues/202
fixed since java-velocypack 2.2.0: https://github.com/arangodb/java-velocypack#kotlin-data-classes
Related
The Collateral state class with the member variable of type java.sql.Timestamp throwing below exception on deserialization in version 3.1-corda :
java.io.NotSerializableException: net.corda.core.contracts.TransactionState -> data(net.corda.core.contracts.ContractState) -> No constructor for deserialization found for class java.sql.Timestamp. -> class java.sql.Timestamp -> class com.syne.cordapp.state.Collateral
The java.sql.Timestamp class already part of whitelist implemented as:
class CordaSerializationWhitelist : SerializationWhitelist {
override val whitelist: List<Class<*>>
get() = listOf(Timestamp::class.java, HashSet::class.java, Set::class.java)
}
And this class also added to file - META-INF/services/net.corda.core.serialization.SerializationWhitelist
So what could be the cause?
As pointed out by Andreas, if a Java class has several constructors, the Corda serialisation framework requires it to designate which constructor to use using the #ConstructorForDeserialization annotation.
java.sql.Timestamp does not do this. To use it as a field within a state, you would have to provide a custom serialiser: https://docs.corda.net/cordapp-custom-serializers.html. However, we recommend using the supported types where possible.
I have a simple TYPE_USE annotation:
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.TYPE_USE, ElementType.TYPE_PARAMETER})
public #interface Cool {
}
And the following sample Kotlin class:
class Item(
id: Long? = null,
var names: List<#Cool String> = emptyList())
Is there any way to extract the annotation using Java reflection?
Item.class.getMethod("getName").getAnnotatedReturnType() loses the annotations, same with getting the field.
Can I even get the annotation from Kotlin?
Item::class.memberProperties.elementAt(0).returnType return a KType which has the annotation, but I don't see a way to extract it. Nor to obtain an AnnotatedType from KType, even though I have JDK8 extensions.
All I see is KType#javaType but this returns Type, not AnnotatedType... so it looses the annotations again.
Edit: this is a bug and has been reported. There is not yet a target version, but its priority has been set to Major. This was fixed in Kotlin 1.3.
TL; DR: no...?
The item annotated with #Cool is the first type argument, so you need to retrieve it:
val type = Item::class.memberProperties.elementAt(0).returnType
val arg = type.arguments[0]
println(arg) // KTypeProjection(variance=INVARIANT, type=#Cool kotlin.String)
Unfortunately there doesn't seem to be a way to retrieve the annotations on a KType (as you have mentioned).
Strangely, this is a very internal process. Looking into the source for KTypeImpl shows that toString is implemented via ReflectionObjectRenderer.renderType(type), (where type is a KotlinType) which is delegated to DescriptorRenderer.FQ_NAMES_IN_TYPES, which we can see is a DescriptorRenderer with modifiers ALL.
The renderer checks if the type is a subclass of kotlin.reflect.jvm.internal.impl.descriptors.annotations.Annotated, and then accesses its annotations property.
I tried this:
val retType = Item::class.memberProperties.elementAt(0).returnType
val arg = retType.arguments[0]
println(arg) // KTypeProjection(variance=INVARIANT, type=#Cool kotlin.String)
val type = arg.type!!
println(type)
val field = type::class.memberProperties.first { it.name == "type" }
val kotlinType = field.call(type) as Annotated
println(kotlinType)
println(kotlinType.annotations)
Unfortunately, I get a ClassNotFoundException for org.jetbrains.kotlin.types.KotlinType, so that option is gone.
Equally strangely, KType is not a subtype of KAnnotatedElement (which is why it has no annotations property).
I suppose this may have been an oversight, since KTypeImpl wraps a KotlinType, which does contain annotations.
I have a generic getter trait
trait Getter[A] {
def get: A
}
and I would like to parse JSON into a List of objects implementing this trait. Two such implementations:
case class CoalesceGetter[A](getters: List[Getter[String]]) extends Getter[A] {
override def get: A = getters.map(_.get).find(_ != null).orNull
}
case class AsnGetter(ipGetter: Getter[String]) extends Getter[Long] {
override def get: Long = 99L // dummy function
}
I would like to parse JSON into the correct Getter class based upon a property called function which corresponds to the class and type which corresponds to the generic type in the case of getters which need a generic (both properties are strings in the json blob I'm parsing). I've looked at custom serializers for json4s but don't see how to work with generics. Any help is appreciated!
First of all, I don't think it is a good idea to jsonify classes with type argument. I think it is a better design to define non-typed (case) classes that are direct equivalent of your json object, and use standard read/write json as provided by many libraries.
But then, to answer your question, I'd like to return another question: how would you do it "manually"?
I.e. how would you write and read different CoalesceGetter[A] with different A?
Here is a proposition: put the type arg in a json field:
"ofInt": {"type-arg":"Int", "getters":[ ... list of getters in json ...]},
"ofDouble":{"type-arg":"Double", "getters":[ ... list of getters in json ...]}
Now, if you'd write the reader, how would you instantiate the 2 ofInt and ofDouble, knowing the type-arg "Int" and "Double" (which are string!).
I see 2 solutions:
1) Either you have a hard-coded map of arg-type string => actual scala type
argType match{
case "Int" => new CoalesceGetter[Int](...)
case "Double" => new CoalesceGetter[Double](...)
}
2) Or you store and read a generalized type as string value in the arg-type string, such as the java Class.forName (see [https://stackoverflow.com/a/7495850/1206998] for example). But this is a really really bad idea IMHO.
(note: if you want to serialize any object just to reload it later or on another computer, don't use json but dedicated serialization such as the Java Serialization or kryo that is used by spark)
My data on firebase uses many fields which have string type, but really are enum values (which I check in my validation rules). To download the data into my Android app, following the guide, the field must be a basic String. I know I can work around this with a second (excluded) field which is an enum, and set this basing on the string value. A short example:
class UserData : BaseModel() {
val email: String? = null
val id: String = ""
val created: Long = 0
// ... more fields omitted for clarity
#Exclude
var weightUnitEnum: WeightUnit = WeightUnit.KG
var weightUnit: String
get() = weightUnitEnum.toString()
set(value) { weightUnitEnum = WeightUnit.fromString(value) }
}
enum class WeightUnit(val str: String) {
KG("kg"), LB("lb");
override fun toString(): String = str
companion object {
#JvmStatic
fun fromString(s: String): WeightUnit = WeightUnit.valueOf(s.toUpperCase())
}
}
Now, while this works, it's not really clean:
The enum class itself is (1) kinda long for an
enum, (2) the insides are repeated for every enum. And I have more of them.
It's not only enums, the created field above is really a timestamp,
not a Long.
Each model uses these enum fields a lot of times, which bloats the model classes with repeatable code...
The helper field/functions are getting much worse/longer for fields with types such as Map<SomeEnum, Timestamp>...
So, is there any way to do this properly? Some library maybe? Or some way to write a magic "field wrapper" that would automatically convert strings to enums, or numbers to timestamps, and so on, but is still compatible with Firebase library for getting/setting data?
(Java solutions are welcome too :) )
If the conversion between a property with your enum value and another property of String type is enough, this can be easily done in a flexible way using Kotlin delegated properties.
To say it short, you can implement a delegate for String properties which performs the conversion and actually gets/sets the value of another property storing the enum values, and then delegate the String property to it.
One possible implementation would look like this:
class EnumStringDelegate<T : Enum<T>>(
private val enumClass: Class<T>,
private val otherProperty: KMutableProperty<T>,
private val enumNameToString: (String) -> String,
private val stringToEnumName: (String) -> String) {
operator fun getValue(thisRef: Any?, property: KProperty<*>): String {
return enumNameToString(otherProperty.call(thisRef).toString())
}
operator fun setValue(thisRef: Any?, property: KProperty<*>, value: String) {
val enumValue = java.lang.Enum.valueOf(enumClass, stringToEnumName(value))
otherProperty.setter.call(thisRef, enumValue)
}
}
Note: This code requires you to add the Kotlin reflection API, kotlin-reflect, as a dependency to your project. With Gradle, use compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version".
This will be explained below, but first let me add a convenience method to avoid creating the instances directly:
inline fun <reified T : Enum<T>> enumStringLowerCase(
property: KMutableProperty<T>) = EnumStringDelegate(
T::class.java,
property,
String::toLowerCase,
String::toUpperCase)
And a usage example for your class:
// if you don't need the `str` anywhere else, the enum class can be shortened to this:
enum class WeightUnit { KG, LB }
class UserData : BaseModel() {
// ... more fields omitted for clarity
#Exclude
var weightUnitEnum: WeightUnit = WeightUnit.KG
var weightUnit: String by enumStringLowerCase(UserData::weightUnitEnum)
}
Now, the explanation:
When you write var weightUnit: String by enumStringLowerCase(UserData::weightUnitEnum), you delegate the String property to the constructed delegate object. This means that when the property is accessed, the delegate methods are called instead. And the delegate object, in turn, works with the weightUnitEnum property under the hood.
The convenience function I added saves you from the necessity of writing UserData::class.java at the property declaration site (using a reified type parameter) and provides the conversion functions to EnumStringDelegate (you can create other functions with different conversions at any time, or even make a function that receives the conversion functions as lambdas).
Basically, this solution saves you from the boilerplate code that represents a property of enum type as a String property, given the conversion logic, and also allows you to get rid of the redundant code in your enum, if you don't use it anywhere else.
Using this technique, you can implement any other conversion between properties, like the number to timestamp you mentioned.
I am in similar situation & thus found your question, plus whole lot of other similar questions/answers.
Cant answer your question directly but this is what I ended up doing: I decided to change my app & not use enum data types at all - mainly because of the advice from Google dev portal which shows how bad the enum's are on app's performance. See the video below https://www.youtube.com/watch?v=Hzs6OBcvNQE
I created the following Kotlin data class:
#JsonInclude(JsonInclude.Include.NON_NULL)
public data class ITunesArtist(val artistName: String,
val artistId: Long, val artistLinkUrl: URL)
(a data class is a Kotlin class that auto-generates equals, hashcode, toString etc at compile time - saves time).
Now I've tried populating it using Spring RestTemplate:
#Test
fun loadArtist()
{
val restTemplate = RestTemplate()
val artist = restTemplate.getForObject(
"https://itunes.apple.com/search?term=howlin+wolf&entity=allArtist&limit=1", ITunesQueryResults::class.java);
println("Got artist: $artist")
}
It fails with:
Could not extract response: no suitable HttpMessageConverter found for response type
[class vampr.api.service.authorization.facebook.ITunesArtist]
and content type [text/javascript;charset=utf-8]
Fair enough - the JSON object mapper was probably expecting mime-type of text/json. Other than telling RestTemplate to map to String::class.java, and then instantiating an instance of JacksonObjectMapper by hand, is there a way to tell my RestTemplate to treat the returned mime type as JSON?
Instead of providing defaults for all properties in your data class you can also use this: https://github.com/FasterXML/jackson-module-kotlin
This Jackson module will allow you to serialize and deserialize Kotlin's data classes without having to worry about providing an empty constructor.
In a Spring Boot Application you can register the module with a #Configuration class like so:
#Configuration
class KotlinModuleConfiguration {
#Bean
fun kotlinModule(): KotlinModule {
return KotlinModule()
}
}
Other than that you can also use the extension functions mentioned in the documentation to register the module with Jackson.
Besides supporting data classes you will also get support for several classes from the Kotlin stdlib, like Pair for example.
Not sure about Spring, but Jackson needed me to specify that I worked with a Java Bean. You see, Kotlin data class is exactly the same as a standard Bean on the byte code level.
Do not forget that Java Bean specification implies an empty constructor (without parameters). A nice way to have it auto-generated is to provide default values for all parameters of your primary constructor.
To serialize an object from Jackson to String:
The 'get' portion of Java Beans specification is required.
To read a JSON string to object:
The 'set' portion of the spec is required.
Additionally the object requires an empty constructor.
Modify the class to include:
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonInclude(JsonInclude.Include.NON_NULL)
data public class ITunesArtist(var artistName: String? = null,
var artistId: Long = -1L, val amgArtistId: String = "id",
var artistLinkUrl: URL? = null)
Fields provide defaults in order for there to be an empty constructor.
Edit:
Uses the Kotlin Module from #mhlz's (now accepted) answer removes the need to provide a default constructor.