I'm writing a basic SDK for Java & Python (and potentially other languages in the future) consumers of an API (REST as well as message-queue orientated responses).
Instead of maintaining separate per-language descriptions, I was wondering if there was perhaps a way to define classes and enums in something like YAML which could be automatically be converted into appropriate objects in each language.
I imagine I could write the objects in C and then make per-language bindings - but this seems a bit hacky as a solution.
You can use JSON to represent your object model and translate it to objects in each of the language you are using.
Here are some relevant articles that will help you get started.
How to convert Java object to / from JSON (Jackson)
https://www.mkyong.com/java/how-to-convert-java-object-to-from-json-jackson/
https://realpython.com/python-json/#encoding-and-decoding-custom-python-objects
Working With JSON Data in Python
Another option is Protocol Buffers though it is not as favorite as JSON when working with REST apis.
Related
Is it possible to integrate the two worlds at least on the data transfer level?
Say i have Java objects which are provided through a Spring WebMVC REST endpoint and my Dart client access these resources with AJAX. It would be nice if the transfered type would be known by the Dart client aswell so i don't have to synchronize the Dart and Java version of the same type definition but the IDE could give me suggestion and errors if the data access on the client side is invalid.
EDIT
A little more explanation what i'm trying to do because it seems i was not clear enough:
On the Java side i define a bean which is converted to JSON by Spring WebMVC + Jackson. So the transfer unit IS already JSON. I can easily access data with Dart dynamically but that's not what i want to do.
I want to parse the retreived Data to a Dart class which as it turns out being a replicate of the original Java bean's class definition. Take a look at JsonObject's explanation on Dart's site, especially the Language abstract class. That's exactly what i'm doing right now. I'm defining an abstract class which defines the JSON data i'm retreiving from the server. This way Dart can give syntax errors if i'm accessing non existing fields or doing incompatible casts, etc. Of course this can still yield into a parse error but after that i can work with the data in a typed manner.
My problem is that to achieve this i have to manually synchronize the data bean's class definition on the Java side and the abstract class definition on the Dart side. I'm wondering if there's somebody working on creating something like a code generator which creates a Dart class definition from a Java class definition or so.
You are asking for Editor feature which would quitely lead to performance degradation of the editor. For any such feature the Editor might need to build/maintain Object Graph/Syntax Tree ( would this use AST in java world ) for java objects and then compare it with Dart's Syntax Tree.
Also different languages will put forward the same request example C#, Ruby, etc. There does not seem to be any sane way to validate the objects from different programming world within performance limits.
I can borrow some more points from below stackoverflow q/a on why its simpler to use JSON/XML rather than any other way to exchange data between java/c# world to dart world -\
How does Java's serialization work and when it should be used instead of some other persistence technique?
Read a BigInteger serialized from Java into C#
I have a Java Server that sending the java serializable object to my client, and receive java serializable object for execution. If my client is also java written, which is nice, that allow me to do communication within any problems.
But now, I would like to extend my programme to not only java client, the client may be written in C, objective C, python or php. So, I would like to do something to "convent" to client request to a java object, and send back to Server. The convent process, I can use the JSON to receive, and construct a Java object to the Server, but I also need a layer that convert back the Java object to JSON to the client.
My Question is except make a JSON-Java Translation layer, is there any other ways to do so? Also, we can afford to change some code in server side, but we must use Java as our primary language for that. Any suggestions? Thanks.
I use Netty API for designing my protocol and it is quite quick to do so if you can understand a NIO-like Byte and Buffer API.
It is design to work with a concept of Encoder and Decoder that could fit your need, there are a lot of default implementation of Encoder and Decoder for zipping, using ssl...
The problem you have seems to looks like this one:
JBoss Netty with JSON
I don't know JSON very well but most of the time is could also be quick and easy to design your own protocol.
Do you need a generic Serialization process for any kind of Object or do you simply need to serialize some String and primitive types (Integer, Short, Float..etc)?
In the case of simple objects it is easy and a lot faster to do the wrapper by yourself.
If objects are quite simple, and I would guess this is the case, your need it to design your own "protocol" specification meaning how to turn each Object into a sequence of primitive types, String and arrays. Than it should be quite easy to write both the Encoder and the Decoder in each language.
Good luck
There are other libraries designed for this, like protocol buffers and thrift.
http://thrift.apache.org/
http://code.google.com/p/protobuf/
I'd like to create a web API of some kind (I don't have a preference for the protocol), where the server uses Java and the client uses PHP.
I want the request and response to both be objects (instances of classes, not JSON-style hashes). The objects' fields can be primitive types or other objects. I would define all the necessary classes in both the client and server code. PHP and Java have similar object models, so it shouldn't be hard to write corresponding classes in both languages.
To make this work, there would need to be some automated way to serialize an object on one side, and unserialize it on the other. It would need to know which PHP class maps to which Java class, and how to convert the fields. I could write something, but is there an existing protocol for transferring objects like this? Can this be done with SOAP?
Java and PHP objects are not interchangeable. You will have to define the object types on both ends, and the transfer protocol could be anything you like. Serialization and deserialization makes the whole process transparent. The transport medium could be JSON, XML, YAML, or anything else for that matter.
For a record-like objects:
{"_type":"MyCoolObjectType", "a":1, "b":2, "c":3"}
If you're wanting to write once and use everywhere, I'd recommend using the same language on both ends, otherwise you'll have to have a compiler that can translate between your choice languages.
A SOAP web service can handle the basic abstraction as long as the request/response is not very complex. You can create the classes in java and then get the API to export a WSDL for them.
You need to have them both serialize to the same string. The PHP format and Java format for serialization are different, and therefore incompatible. You need a common exchange format, and I recommend that you DON'T use PHP's. However, the functions to serialize in PHP are fairly simple, are contained in ext/standard/var.c file in the PHP source if you choose to use it..
See the following:
Unserialize in Java a serialized php object - A similar question to yours.
http://en.wikipedia.org/wiki/Serialization#Serialization_formats
http://en.wikipedia.org/wiki/XML
XML, API, CSV, SOAP! Understanding the Alphabet Soup of Data Exchange
From http://en.wikipedia.org/wiki/XML (emphasis mine):
Although the design of XML focuses on documents, it is widely used for the representation of arbitrary data structures, for example in web services.
Does somebody know a Java library which serializes a Java object hierarchy into Java code which generates this object hierarchy? Like Object/XML serialization, only that the output format is not binary/XML but Java code.
Serialised data represents the internal data of objects. There isn't enough information to work out what methods you would need to call on the objects to reproduce the internal state.
There are two obvious approaches:
Encode the serialised data in a literal String and deserialise that.
Use java.beans XML persistence, which should be easy enough to process with your favourite XML->Java source technique.
I am not aware of any libraries that will do this out of the box but you should be able to take one of the many object to XML serialisation libraries and customise the backend code to generate Java. Would probably not be much code.
For example a quick google turned up XStream. I've never used it but is seems to support multiple backends other than XML - e.g. JSON. You can implement your own writer and just write out the Java code needed to recreate the hierarchy.
I'm sure you could do the same with other libraries, in particular if you can hook into a SAX event stream.
See:
HierarchicalStreamWriter
Great question. I was thinking about serializing objects into java code to make testing easier. The use case would be to load some data into a db, then generate the code creating an object and later use this code in test methods to initialize data without the need to access the DB.
It is somehow true that the object state doesn't contain enough info to know how it's been created and transformed, however, for simple java beans there is no reason why this shouldn't be possible.
Do you feel like writing a small library for this purpose? I'll start coding soon!
XStream is a serialization library I used for serialization to XML. It should be possible and rather easy to extend it so that it writes Java code.
The log4j network adapter sends events as a serialised java object. I would like to be able to capture this object and deserialise it in a different language (python). Is this possible?
NOTE The network capturing is easy; its just a TCP socket and reading in a stream. The difficulty is the deserialising part
Generally, no.
The stream format for Java serialization is defined in this document, but you need access to the original class definitions (and a Java runtime to load them into) to turn the stream data back into something approaching the original objects. For example, classes may define writeObject() and readObject() methods to customise their own serialized form.
(edit: lubos hasko suggests having a little java program to deserialize the objects in front of Python, but the problem is that for this to work, your "little java program" needs to load the same versions of all the same classes that it might deserialize. Which is tricky if you're receiving log messages from one app, and really tricky if you're multiplexing more than one log stream. Either way, it's not going to be a little program any more. edit2: I could be wrong here, I don't know what gets serialized. If it's just log4j classes you should be fine. On the other hand, it's possible to log arbitrary exceptions, and if they get put in the stream as well my point stands.)
It would be much easier to customise the log4j network adapter and replace the raw serialization with some more easily-deserialized form (for example you could use XStream to turn the object into an XML representation)
Theoretically, it's possible. The Java Serialization, like pretty much everything in Javaland, is standardized. So, you could implement a deserializer according to that standard in Python. However, the Java Serialization format is not designed for cross-language use, the serialization format is closely tied to the way objects are represented inside the JVM. While implementing a JVM in Python is surely a fun exercise, it's probably not what you're looking for (-:
There are other (data) serialization formats that are specifically designed to be language agnostic. They usually work by stripping the data formats down to the bare minimum (number, string, sequence, dictionary and that's it) and thus requiring a bit of work on both ends to represent a rich object as a graph of dumb data structures (and vice versa).
Two examples are JSON (JavaScript Object Notation) and YAML (YAML Ain't Markup Language).
ASN.1 (Abstract Syntax Notation One) is another data serialization format. Instead of dumbing the format down to a point where it can be easily understood, ASN.1 is self-describing, meaning all the information needed to decode a stream is encoded within the stream itself.
And, of course, XML (eXtensible Markup Language), will work too, provided that it is not just used to provide textual representation of a "memory dump" of a Java object, but an actual abstract, language-agnostic encoding.
So, to make a long story short: your best bet is to either try to coerce log4j into logging in one of the above-mentioned formats, replace log4j with something that does that or try to somehow intercept the objects before they are sent over the wire and convert them before leaving Javaland.
Libraries that implement JSON, YAML, ASN.1 and XML are available for both Java and Python (and pretty much every programming language known to man).
I would recommend moving to a third-party format (by creating your own log4j adapters etc) that both languages understand and can easily marshal / unmarshal, e.g. XML.
In theory it's possible. Now how difficult in practice it might be depends on whether Java serialization format is documented or not. I guess, it's not. edit: oops, I was wrong, thanks Charles.
Anyway, this is what I suggest you to do
capture from log4j & deserialize Java object in your own little Java program.
now when you have the object again, serialize it using your own custom formatter.
Tip: Maybe you don't even have to write your own custom formatter. for example, JSON (scroll down for libs) has libraries for Python and Java, so you could in theory use Java library to serialize your objects and Python equivalent library to deserialize it
send output stream to your python application and deserialize it
Charles wrote:
the problem is that for this
to work, your "little java program"
needs to load the same versions of all
the same classes that it might
deserialize. Which is tricky if you're
receiving log messages from one app,
and really tricky if you're
multiplexing more than one log stream.
Either way, it's not going to be a
little program any more.
Can't you just simply reference Java log4j libraries in your own java process? I'm just giving general advice here that is applicable to any pair of languages (name of the question is pretty language agnostic so I just provided one of the generic solutions). Anyway, I'm not familiar with log4j and don't know whether you can "inject" your own serializer into it. If you can, then of course your suggestion is much better and cleaner.
Well I am not Python expert so I can't comment on how to solve your problem but if you have program in .NET you may use IKVM.NET to deserialize Java objects easily. I have experimented this by creating .NET Client for Log4J log messages written to Socket appender and it worked really well.
I am sorry, if this answer does not make sense here.
If you can have a JVM on the receiving side and the class definitions for the serialized data, and you only want to use Python and no other language, then you may use Jython:
you would deserialize what you received using the correct Java methods
and then you process what you get with you Python code