This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
What is object serialization?
I know the details of how the JVM does the serialization of object graphs. What I am more interested in is the use of serialization? What was the necessity that drove the specification of serialization? More specifically, what practical uses does serialization have these days? Is is used as a means to store the data? Or is it used to send data across the network?
I will appreciate any links to this question if a full answer is not possible.
Thanks in advance.
Very simple. Suppose you have object graph and want to store it in file and then read it back. The luck of java programmer is that he/she does not have to implement gory details of field-by-field writing and reading the data. If whole graph is consists of serializable objects java does this work for you.
The same is relevant if 2 applications exchange the data.
Serialization is mainly used to send objects across the network. It's used in RMI, Spring's HttpInvoker, and other binary remote method invocation systems.
Using it for durable persistent storage is questionable, because it's impossible to query, binary and thus hard to analyze with a simple text editor, and hard to maintain when the classes change and their serialization format is thus modified. So a more open format is often chosen (XML, JSON, etc.)
Yes and yes! You can use it to send objects across the network, cache them, save them to disk, whatever you like. It is used for things like session replication between clustered JVM instances. Much of the time it is used under the covers by libraries that you use as well.
Related
This question already has answers here:
What's the easiest way to persist java objects?
(6 answers)
Closed 2 years ago.
I have a program that creates objects that should store their data (recipes) on my computer in a way that should allow me to store a couple thousand (and I would like to save storage space). But when looking at serialization I don't know what approach to take. I don't want the risk of losing data in the future which I heard is a problem and I would also like to be able to store thousands without loading up disk space. Any suggestions help. Thanks
Serialization protocols tend to either be error prone, or quite complicated, with you re-inventing either a database engine or journalled systems - at least, if you want them to not cause permanent corruption if your app crashes or you trip over a power cable at the wrong time.
So why not just.. bite the bullet, and use something like H2 (a database engine) together with something like JDBI (a library to talk to that database engine easily)?
Any options for writing to disk do not carry any storage risks.
Once a write has occurred, the file system (or database) is responsible for storing the file.
For greater data security, you can duplicate them.
And from the point of view of system security, you can organize raid-1 (or another raid). But most likely you don't need for this task
I have an Object which is not serializable. I want to share it between more than two java processes.
How can I do that?
(I wouldn't like to use transient because I need non-serializable fields in my object.)
EDIT:
Java processes are local and run in one system.
There are a number of ways to serialize a Java object between two processes. One that is commonly used these days is JSON. Two popular frameworks are Jackson and GSON.
The nice thing about JSON is that it is well-understood, plays well with HTTP, maps well to other languages such as Javascript, Ruby, Python, etc, and it is fairly easy to ready which helps a lot with debugging. We have been using Jackson for a number of years and are pretty happy with its power and speed, although sometimes it can be overly complex. Others are quite happy with GSON.
If you are looking to optimize the payload on the network, and HTTP is not an issue, you can look at binary mechanisms like Google Protocol Buffers
I am working on a game that has multiplayer support and I want to encrypt the server-client connection. I have done it before using a SecretKey object and an ObjectInput/OutputStream. However, I want to leave the ability open for other languages to connect to the server (if I ever take up another language and want to port my game.) Is there any way I could encrypt all the data without using Java objects so any language can use it?
You can create your own custom object serializer in Java with the Externalizable interface. The custom serializer can write out the state of the Java objects so that another language could read them. I've implemented this is a project where I needed serialization to work even if the objects changed and old state needed to be read back. The painful part of custom serialization is that you have to track the object fields carefully or your deserialize methods will create strange bugs.
Binary object serialization
One action you need to take is to serialize your objects to a binary format. You can do this using the standard serialization API, or you can create your own encoder/decoder. Be sure you describe your own protocol in detail - every bit should be described or you will run into trouble - if not directly then several years after.
There are standardized methods or creating your own protocol with binary encodings such as ASN.1, but if you take that route expect a rather steep learning curve. The general idea of using tag/length/value for values is a good one though, so maybe you can take a look at e.g. BER/DER encoding.
Encryption of your serialized objects
To encrypt, you can create your own cryptographic protocol. Most people on this forum go this route, and most fail. They manage to get their protocol working, but they also leave multiple security holes open.
One of the best ways of securing data in transit is TLS. So if TLS is applicable, by all means go this route. After initial setup, TLS has a relatively low overhead, so there is probably no need to try and implement a competing proprietary protocol.
You can also encrypt at application level instead of transport level. A solution to this is to rely on previous standards for cryptographic constainer formats. Well known formats are CMS (previously known as PKCS#7) or PGP. PGP and CMS formats are implemented in the Bouncy Castle libraries. They both are binary formats with many options present within them.
The way that I did it about six years ago, I serialized the object (so that it was a string) converted the string to a byte array, encrypted the bytes, and sent the data as bytes. The other end then reversed the process. I got this to work for encrypted communications between a Java server and an AS3 client. It won't work for languages that don't support byte arrays though. Do you need more details?
My universities peer to peer communication course uses an in house client/server program for demonstration and (i think) extending it is part of the assessment. The program we use is written in java and uses serialisation for the network communication.
To get a better grip I want to try reimplementing the protocol used in objective c, but googling around I cant find any information on using serialised data between languages. I would like to keep this as simple as possible, ideally be able to drop my replacement server/client onto a network and have it behave.
Edit Didnt actually ask a question there.
Is it possible to communicate between the two serialised formats, How can I make this work without reverse engineering the format java uses.
I would recommend avoiding writing (de)serialization support of java's native serialization in another language.
If you can change the existing Java server and clients, use a more language agnostic serialization format.
Assuming that you are not allowed to make that sort of change, I would define the new protocol, and implement a bridge in Java. The bridge (process) would establish a connection on behalf of each client that connects to it, and translate messages between the Java serialized and language agnostic form. This will provide a good migration strategy.
Java serialization protocol (if it's built-in default Java serialization) is documented, so you won't have to reverse engineer it - check this article and this link. However, if you can, use JSON, XML or XML-RPC; it will be much simpler than creating Java serializer/deserializer in another language.
I am pretty far into my first Android application, and I have the sneaking suspicion that I'm "Doing It Wrong". My app talks to a Ruby on Rails server and serializes objects back and forth via XML. Before I knew what was happening, I found myself knee deep in writing my own crappy ORM, a problem which is compounded by the fact that I haven't written any Java since high school.
My conflict here is that I want my client-side (android) app to be capable of serializing via a variety of methods, such as HTTP/XML, to a local database, or out to the local filesystem. I started out with the Strategy pattern, but I feel like my solution is badly lacking.
For one, should I re-implement all of Rails model validation on the client side, because I don't know if I'm always going to be working with Rails on the other side? The even bigger issue is that right now I can only represent flat objects as key-values, as my code can't handle nested objects like a true ORM.
I'm sure Android devs deal with this all the time, so I'm interested to hear what other people do to cope with these issues.
I wouldn't approach your Android application as an extension of a Ruby app - rather a consumer of an API. If you can try to expose your server application as JSON (or other format, but JSON is the most lightweight) and consume these APIs from your Android application you would most likely have less problems as JSON is already in K/V format.
I have not yet written Android objects to SQLite yet, but I have written them as both Parcelable objects and to the SharedPreferences. Both of these strategies are sufficient for small to mid-range apps. For data intensive apps, obviously you will have to take it a step further to SQLite.
There are some great articles for these approaches: Managing State.
It boils down to designing your objects in a way that can be serialized easily. That means no circular references or extremely complex objects. This shouldn't be a very large problem, especially if your data is in JSON format already. You simply need to extend some classes and add functions that return a Parcelable object representation or a string representation so your objects can be saved thus.
I would avoid cloning your server-side objects and validation in Android as it then requires modifying both sources if you make small changes. The server should handle all data and validation and you should simply be requesting, caching, and sending data from Android.
I'd be interested to hear if there any challenges to writing objects to SQLite, but I imagine it's not that much more of step from the details I've outlined above. Hope this helps in some capacity!
Hessian is great for RPC. You don't have to do any serialization yourself. It doesn't use XML, so it's more efficient and more appropriate for a mobile platform.
I haven't done much of persistence storage on Android but I think you need to use SQLiteDatabase and make your own Cusor that De/Serializes your object so that it can be added to the database. A possible solution would be to extend a SQLiteCursor or an AbstractCursor.
Otherwise I don't think there is other solution apart from, possibly, "hardcore" Serializabled (Which I suspect it may be too much for a phone, I may be wrong)
I think you might be going too heavy for a smart phone application. I would look at using RESTful style web services with JSON content.
Looking to your question I got the feeling that maybe you just over-complicating your requirements? Why can't you just use JSON format to represent your objects data in portable way? Then you will be able just to store it either on file system or in database in simple text field. You can leverage android-active-record library for transparent DB persistence (http://code.google.com/p/android-active-record)