I have an Object which is not serializable. I want to share it between more than two java processes.
How can I do that?
(I wouldn't like to use transient because I need non-serializable fields in my object.)
EDIT:
Java processes are local and run in one system.
There are a number of ways to serialize a Java object between two processes. One that is commonly used these days is JSON. Two popular frameworks are Jackson and GSON.
The nice thing about JSON is that it is well-understood, plays well with HTTP, maps well to other languages such as Javascript, Ruby, Python, etc, and it is fairly easy to ready which helps a lot with debugging. We have been using Jackson for a number of years and are pretty happy with its power and speed, although sometimes it can be overly complex. Others are quite happy with GSON.
If you are looking to optimize the payload on the network, and HTTP is not an issue, you can look at binary mechanisms like Google Protocol Buffers
Related
I have a client-server system implemented in C#, and the client and server exchange .Net objects via serialization / deserialization and communicating via TCP/IP. This runs on a local network, it is not web-based or Internet-based.
Now I want to include Android clients connected by wifi. Again, this is local network only, not via the Internet and not web-based. The Android programming will be in Java. (I am aware of Mono for Android, but prefer not to get into that now.)
Is there some fairly simple way to implement object to object interchange between Java and .Net objects, provided, of course, that they are compatible?
I've looked a bit at JSON (Jackson on the Java end and Json.Net on the .Net end), and I'm guessing it can probably be done, but only with major efforts on remapping things at each end as soon as the objects become fairly complicated.
Any other suggestions? JSON-based or otherwise?
PS. My question is somewhat related to this one Mapping tool for converting Java's JSON to/from C#, but it never got a suitable answer, perhaps due to insufficient info in the question. Also, I don't care whether I end up using a JSON-based transport or XML or something else.
I would suggest either JSON or XML (which is based on a .xsd file) because these are independent of their respective implementations (instead of something like an ObjectOutputStream in java).
The problem of having this format between the two components (client and server) is that they need to be at the same version. My best practice is to have one underlying definition of the format (i use xml with an xsd file which specifies how the xml has to look like), then use jaxb to generated java classes. That way you can (un)marshal from/to xml in the java part.
I am very sure a similar thing exists in the world of .NET.
JSON is smaller than xml in size, i find xml to be more readable.
SO user "default locale" should get the honor for this, but he/she has only answered via a comment. So just to make it very clear what my choice was I'll answer my own question.
I've decided to go with Google Protocol Buffers, which in my opinion has much better support for moving objects back and forth between Java and .Net than JSON. Because I have a lot of experience with C#, and a lot of existing C#-defined classes, I've selected Marc Gravell's protobuf-net program for the .Net end, and Google's own support for the Android end (no - see edit). This implies that I'm defining the objects in C#, not in .proto files - protobuf-net generates the .proto files from which I then generate the Java code.
Incidentally, as the transport mechanism I'm using a little-known program called naga on the Android end. http://code.google.com/p/naga/ Naga seems to work fine, and is well-documented and has sample programs, and should be better known in my opinion.
EDIT:
OK, I've got it working now to my satisfaction. Here's what I'm using:
Google Protocol buffers as the interchange format: https://developers.google.com/protocol-buffers/
Marc Gravell's protobuf-net at the C# end: http://code.google.com/p/protobuf-net/
A program called called protostuff at the Java end: http://code.google.com/p/protostuff/
(I prefer protostuff to the official Google Java implementation of protocol buffers due to Google's implementation being based on the Java objects being immutable.)
Actually, I'm not using pure protocol buffers as the interchange format - I prefix the data with the name of the (outermost) class being transmitted. This makes the data self-identifying for deserializing at the other end.
You can also try wox (https://github.com/codelion/wox), it is a cross platform serialization library for Java and C# based on XML.
I am doing a Software Engineering course in which different teams are building different prototype subsystems of a big system (different subsystem of F35 Lightning aircraft!).
The problem is that teams can use different programming languages (like C++ and Java) depending upon what they are most comfortable in. However, these subsystems need to be communicating with each other (like radar needs to provide object corodinates to navigation and control). Hence we need to come up with a solution in which different modules can interact in real time.
Someone suggested XML-RPC and hence I was reading about it. After reading it I think it is used in server client architecture. Is this a good way of doing interprocess kind of communication? What are my options?
Any help would be appreciated.
regards,
Newbie
There are a couple of options beside XML-RPC. For a short bullet-point comparison, take a look at:
http://michaeldehaan.net/2008/07/17/xmlrpc-vs-rest-vs-soap-vs-all-your-rpc-options/
If your exchange is more data-oriented, Protocol Buffers might be an alternative.
Protocol Buffers are a way of encoding structured data in an efficient yet extensible format.
Personally, I would go for lightweight exchange format or method first since the components are considered prototypes. Something like REST or some custom message passing might be simple enough, yet sufficient.
If you are already familiar with XML, it can be a reasonable answer. An advantage of XML is that you don't have to worry about how different machines represent numbers. A disadvantage is the time it takes to keep converting numbers to text and back to numbers.
I am pretty far into my first Android application, and I have the sneaking suspicion that I'm "Doing It Wrong". My app talks to a Ruby on Rails server and serializes objects back and forth via XML. Before I knew what was happening, I found myself knee deep in writing my own crappy ORM, a problem which is compounded by the fact that I haven't written any Java since high school.
My conflict here is that I want my client-side (android) app to be capable of serializing via a variety of methods, such as HTTP/XML, to a local database, or out to the local filesystem. I started out with the Strategy pattern, but I feel like my solution is badly lacking.
For one, should I re-implement all of Rails model validation on the client side, because I don't know if I'm always going to be working with Rails on the other side? The even bigger issue is that right now I can only represent flat objects as key-values, as my code can't handle nested objects like a true ORM.
I'm sure Android devs deal with this all the time, so I'm interested to hear what other people do to cope with these issues.
I wouldn't approach your Android application as an extension of a Ruby app - rather a consumer of an API. If you can try to expose your server application as JSON (or other format, but JSON is the most lightweight) and consume these APIs from your Android application you would most likely have less problems as JSON is already in K/V format.
I have not yet written Android objects to SQLite yet, but I have written them as both Parcelable objects and to the SharedPreferences. Both of these strategies are sufficient for small to mid-range apps. For data intensive apps, obviously you will have to take it a step further to SQLite.
There are some great articles for these approaches: Managing State.
It boils down to designing your objects in a way that can be serialized easily. That means no circular references or extremely complex objects. This shouldn't be a very large problem, especially if your data is in JSON format already. You simply need to extend some classes and add functions that return a Parcelable object representation or a string representation so your objects can be saved thus.
I would avoid cloning your server-side objects and validation in Android as it then requires modifying both sources if you make small changes. The server should handle all data and validation and you should simply be requesting, caching, and sending data from Android.
I'd be interested to hear if there any challenges to writing objects to SQLite, but I imagine it's not that much more of step from the details I've outlined above. Hope this helps in some capacity!
Hessian is great for RPC. You don't have to do any serialization yourself. It doesn't use XML, so it's more efficient and more appropriate for a mobile platform.
I haven't done much of persistence storage on Android but I think you need to use SQLiteDatabase and make your own Cusor that De/Serializes your object so that it can be added to the database. A possible solution would be to extend a SQLiteCursor or an AbstractCursor.
Otherwise I don't think there is other solution apart from, possibly, "hardcore" Serializabled (Which I suspect it may be too much for a phone, I may be wrong)
I think you might be going too heavy for a smart phone application. I would look at using RESTful style web services with JSON content.
Looking to your question I got the feeling that maybe you just over-complicating your requirements? Why can't you just use JSON format to represent your objects data in portable way? Then you will be able just to store it either on file system or in database in simple text field. You can leverage android-active-record library for transparent DB persistence (http://code.google.com/p/android-active-record)
I'm working on a .net system that will both expose and consume web-services with another system to pass data back and forth - the other system is java based.
Our proposed XSD contains complex types and some concern has been expressed about using complex types and how we'd be better sticking to simple types. I'd have thought .net would have been able to support complex types so was hoping someone can elaborate on what problems I'm likely to face. I've tried googling but not found anything specific.
The Exposing .NET WebService to Other Platform (Java) stackoverflow question has an answer that
"This should work out of the box, but
I would advise against returning
complex data structures or expecting
such as input arguments. If you need
complexity of that kind, I would
suggest returning/accepting XML
instead."
but doesn't really explain why, so any thoughts / explanation greatly appreciated
EDIT - note I'm not planning to transfer platform specific objects over these services, instead I want to model business entities in a shared XSD as complex types, built out of simple types, (so that they can be easily controlled and reused in other XSDs) and these are the element that concern has been raised about.
I plan to do some proof of concept on this to see if I can prove this working / surface any problems, but thought I'd get some views of SO users who have done this before, first.
There are many platform-specific types that can be used easily as long as both endpoints are homogeneous, but which don't map cleanly to xsd or to other platforms. For example DataTable in .NET is a royal PITA from anywhere else; and anything implementing IXmlSerializable in .NET is most-likely completely lying in the schema.
In the interop scenario, it would usually be worth starting from xsd, as that gives a common standard that all reasonable clients should expect.
We're looking into transport/protocol solutions and were about to do various performance tests, so I thought I'd check with the community if they've already done this:
Has anyone done server performance tests for simple echo services as well as serialization/deserialization for various messages sizes comparing EJB3, Thrift, and Protocol Buffers on Linux?
Primarily languages will be Java, C/C++, Python, and PHP.
Update: I'm still very interested in this, if anyone has done any further benchmarks please let me know. Also, very interesting benchmark showing compressed JSON performing similar / better than Thrift / Protocol Buffers, so I'm throwing JSON into this question as well.
Latest comparison available here at the thrift-protobuf-compare project wiki. It includes many other serialization libraries.
I'm in the process of writing some code in an open source project named thrift-protobuf-compare comparing between protobuf and thrift. For now it covers few serialization aspects, but I intend to cover more. The results (for Thrift and Protobuf) are discussed in my blog, I'll add more when I'll get to it.
You may look at the code to compare API, description language and generated code. I'll be happy to have contributions to achieve a more rounded comparison.
You may be interested in this question: "Biggest differences of Thrift vs Protocol Buffers?"
I did test performance of PB with number of other data formats (xml, json, default object serialization, hessian, one proprietary one) and libraries (jaxb, fast infoset, hand-written) for data binding task (both reading and writing), but thrift's format(s) was not included. Performance for formats with multiple converters (like xml) had very high variance, from very slow to pretty-darn-fast. Correlation between claims of authors and perceived performance was rather weak. Especially so for packages that made wildest claims.
For what it is worth, I found PB performance to be bit over hyped (usually not by its authors, but others who only know who wrote it). With default settings it did not beat fastest textual xml alternative. With optimized mode (why is this not default?), it was bit faster, comparable with the fastest JSON package. Hessian was rather fast, textual json also. Properietary binary format (no name here, it was company internal) was the slowest. Java object serialization was fast for larger messages, less so for small objects (i.e. high fixed per-operation noverhead).
With PB message size was compact, but given all trade-offs you have to do (data is not self-descriptive: if you lose the schema, you lose data; there are indexes of course, and value types, but from what you have reverse-engineer back to field names if you want), I personally would only choose it for specific use cases -- size-sensitive, closely coupled system where interface/format never (or very very rarely) changes.
My opinion in this is that (a) implementation often matters more than specification (of data format), (b) end-to-end, differences between best-of-breed (for different formats) are usually not big enough to dictate the choice.
That is, you may be better off choosing format+API/lib/framework you like using most (or has best tool support), find best implementation, and see if that works fast enough.
If (and only if!) not, consider next best alternative.
ps. Not sure what EJB3 here would be. Maybe just plain of Java serialization?
If the raw net performance is the target, then nothing beats IIOP (see RMI/IIOP).
Smallest possible footprint -- only binary data, no markup at all. Serialization/deserialization is very fast too.
Since it's IIOP (that is CORBA), almost all languages have bindings.
But I presume the performance is not the only requirement, right?
One of the things near the top of my "to-do" list for PBs is to port Google's internal Protocol Buffer performance benchmark - it's mostly a case of taking confidential message formats and turning them into entirely bland ones, and then doing the same for the data.
When that's been done, I'd imagine you could build the same messages in Thrift and then compare the performance.
In other words, I don't have the data for you yet - but hopefully in the next couple of weeks...
To back up Vladimir's point about IIOP, here's an interesting performance test, that should give some additional info over the google benchmarks, since it compares Thrift and CORBA. (Performance_TIDorb_vs_Thrift_morfeo.pdf // link no longer valid)
To quote from the study:
Thrift is very efficient with small
data (basic types as operation
arguments)
Thrifts transports are not so efficient as CORBA with medium and
large data (struct and >complex
types > 1 kilobytes).
Another odd limitation, not having to do with performance, is that Thrift is limited to returning only several values as a struct - although this, like performance, can surely be improved perhaps.
It is interesting that the Thrift IDL closely matches the CORBA IDL, nice. I haven't used Thrift, it looks interesting especially for smaller messages, and one of the design goals was for a less cumbersome install, so these are other advantages of Thrift. That said, CORBA has a bad rap, there are many excellent implementations out there like omniORB for example, which has bindings for Python, that are easy to install and use.
Edited: The Thrift and CORBA link is no longer valid, but I did find another useful paper from CERN. They evaluated replacements for their CORBA system, and, while they evaluated Thrift, they eventually went with ZeroMQ. While Thrift performed the fastest in their performance tests, at 9000 msg/sec vs. 8000 (ZeroMQ) and 7000+ RDA (CORBA-based), they chose not to test Thrift further because of other issues notably:
It is still an immature product with a buggy implementation
I have done a study for spring-boot, mappers (manual, Dozer and MapStruct), Thrift, REST, SOAP and Protocol Buffers integration for my job.
The server side: https://github.com/vlachenal/webservices-bench
The client side: https://github.com/vlachenal/webservices-bench-client
It is not finished and has been run on my personal computers (I have to ask for servers to complete the tests) ... but results can be consulted on:
Laptop: https://github.com/vlachenal/webservices-bench/blob/master/results.md
Desktop: https://github.com/vlachenal/webservices-bench/blob/master/results-desktop.md
As conclusion :
Thrift offers the best performance and is easy to use
RESTful webservice with JSON content type is pretty close to Thrift performance, is "browser ready to use" and is quite elegant (from my point of view)
SOAP has very poor performance but offers the best data control
Protocol Buffers has good performance ... until 3 simultaneous calls ... and I don't know why. It is very difficult to use: I give up (for now) to make for it work with MapStruct and I don't try with Dozer.
Projects can be completed through pull requests (either for fixes or other results).