I've had a look around but nothing seems to quite cover what I wish to do. Is it possible to save a Class<?> instance object at run-time? If so how would I go about doing it?
Have you gone through concept of serialization using java. this link will help you on your problem.
In short Java.lang.Serializable is your friend to do this.
This is a set of comments on Java Serialization rather than an answer. Just some info not (yet) in the other answers.
Serialization not only saves an object, it saves all the objects it references, directly and indirectly. This can be really cool, but you might write one little objectette and find you've unexpectedly created a 10MB file.
If there's a reference, however indirect, to an non-serializable object in the one you're writing, the write will throw an Exception.
If you're using a socket, reset the ObjectOutput stream regularly. Otherwise, every time an object is written other than the first time, all that gets sent is a reference to the original data. Send the same object with successive values of 1, 2, 3, 4, and 5, and the object read will have values of 1, 1, 1, 1, and 1. Also, without reset, memory usage will soar because both ObjectOutput and ObjectInput will keep pretty much everything sent in memory. (Though it will only keep one copy of each distinct object.)
Serialization doesn't work if a class changes between being written and being read. Clever work with Externalization can get around this, however. (And remember serialVersionUID if your IDE will let you forget.)
Externalization lets you write the code to serialize a class. This can be very useful. You can put in version numbers and check them and you can leave out data than isn't needed or can be recreated during the read. It takes more work than automatic serialization, though.
When doing an Externalization read, be aware that all references may refer to objects whose data has not yet arrived; you can't consistently sum amounts from a list of child objects, for instance. It might pay to call a method after readObject to set up values that need to be calculated. (It's often better to send redundant information than to recalculate it.)
I learned all this the hard way.
I learned about Serialization from this website. It teaches the concept well. I recommend starting there.
Related
Thinking about what is serializable and what is not, do I get it right that if no error messages pop up during de/serialization then everything has been perfectly serialized and deserialized? Or is it still possible while not getting any errors to have my object somehow damaged or changed during de/serialization?
My question may seem odd but it's rather difficult for a newbie like myself to keep track of every part of an object (which is fairly vast) whether this part can be serialized or not. So I'd rather fully rely on error indications if it's an adequate approach.
Actually not. Absence of error in writing/reading serializable object to/from DataStream means there is no exceptional situations. But that doesn't mean that you will get consistent data.
You could read a lot more in Effective Java by Bloch. There are several chapters concerning serialization.
Someone thought it would be a good idea to store Objects in the database in a blob column using Java's default serialization methods.
The structure of these objects is controlled by another group and they changed a field type from BigDecimal to a Long,
but the data in our database remains the same.
Now we can't read the objects back because it causes ClassCastExceptions.
I tried to override it by writing my own readObject method,
but that throws a StreamCorruptedException because what was written by the default writeObject method.
How do I make my readObject call behave like Java's default one?
Is there a certain number of bytes I can skip to get to my data?
Externalizable allows you to take full control of serialization/deserialization. But it means you're responsible for writing and reading every field,
When it gets difficult though is when something was written out using the default serialization and you want to read it via Externalizable. (Or rather, it's impossible. If you try to read an object serialized with the default method using Externalizable, it'll just throw an exception.)
If you've got absolutely no control on the output, your only option is to keep two versions of the class: use the default deserialization of the old version, then convert to the new. The upside of this solution is that it keeps the "dirty" code in one place, separate from your nice and clean objects.
Again, unless you want to do things really complicated, your best option is to keep the old class as the "transport" bean and rename the class your code really uses to something else.
If you want to read what's already in your database your only option is to get them to change the class back again, and to institute some awareness that you're relying on the class definition as it was when the class was serialized. Merely implementing your own readObject() call can't fix this, and if the class is under someone else's control you can't do that anyway.
If you're prepared to throw away the existing data you have many other choices starting with custom Serialization, writeReplace()/readResolve(), Externalizable, ... or a different mechanism such as XML.
But if you're going to have third parties changing things whenever they feel like it you're always going to have problems of one kind or another.
BigDecimal to Long sounds like a retrograde step anyway.
Implement the readObject and readObjectNoData methods in you class.
Read the appropriate type using ObjectInoutStream.readObject and convert it to the new type
See the Serializable interface API for details.
More Details
You can only fix this easily if you control the source of the class that was serialized into the blob.
If you do not control this class,
then you have only a few limited and difficult options:
Have the controlling party give you a version of the class that reads the old format and writes the new format.
Write you own form of serialization (as in you read the blob and convert the bytes to classes) that can read the old format and generate new versions of the classes.
Write you own version of the class in question (remove the other from the class path) which reads the old format and produces some intermediate form (perhaps JSON).
Next you have to do one of these
Convince the powers that be that the blob technique is shitty and should be done away with. use the current class change as evidance. Almost any technique is better that this. Writing JSON to the db in the blob is better.
Stop depending on shitty classes from other people. (shitty is a judgement which I can only suspect, not know, is true). Instead create a suite of classes that represent the data in the database and convert from the externally controlled classes to the new data classes before writing to the database.
When I first started using the Java Preferences API, the one glaring omission from the API was a putObject() method. I've always wondered why they did not include it.
So, I did some googling and I found this article from IBM which shows you how to do it: http://www.ibm.com/developerworks/library/j-prefapi/
The method they're using seems a bit hackish to me, because you have to break the Object up into byte matrices, store them, and reassemble them later.
My question is, has anyone tried this approach? Can you testify that it is a good way to store/retrieve objects?.
I'm also curious why the Java devs left putObject() out of the API. Does anyone have valuable insight?
I'm also curious why the Java devs left putObject() out of the API.
Does anyone have valuable insight?
From: http://docs.oracle.com/javase/7/docs/technotes/guides/preferences/designfaq.html
Why doesn't this API contain methods to read and write arbitrary
serializable objects?
Serialized objects are somewhat fragile: if the version of the program
that reads such a property differs from the version that wrote it, the
object may not deserialize properly (or at all). It is not impossible
to store serialized objects using this API, but we do not encourage
it, and have not provided a convenience method.
The article describes a reliable way to do it. I see there are a couple of things I may do differently (like I would store the count of the number of pieces as well as the pieces themselves so that I can figure things out easily when I retrieve them).
Your comment about Serialization is wrong though.... the object you want to store has to be Serializable.... that's how the ObjectOutputStream that the document uses does it's job.
So, Yes, it looks like a reliable mechanism, you need to have Serializable objects, and I imagine that the reason that putObject and getObject are not part of the API for two reasons:
it's not part of the way that is native to Windows registries
It risks people putting huge amounts of data in the registry.
Storing serialized objects in the registry strikes me as being somewhat concerning because they can be so big. I would only use it for occasions when there is no way to reconstruct the Object from constructors, and the serialized version is relatively small.
I have written a math game in Java, and have distributed some copies to a few beta-testers. The problem is that the version I have given them is saving the GameData via object serialization, which I found out is mainly for sending Objects, or in this case, ArrayLists of GameData, over a network. It is NOT persistance; that is what a relational database is for. Knowing this, I would like to know if it would be better to create a database on the beta-tester's machine (and rewrite the game), or continue with the Object serialization version of the game, and then retrieve the Objects when they are ready to send the data?
My guess would be to just move their data to a database that is created on their computer, and then give them the database version of the game. That way, the data can be persisted and be much easier to manipulate. What turns me away from that idea is the question of how am I going to write their database into mine (in the future)?
Although relatively rare, there are still lots of applications that use serialization for storage and retrieval of objects. It's not wrong to do this, just slightly unusual. If it's working for you, stick with it because DB's are a heavyweight solution. What you found out, about serialization, is only an opinion and an ill-formed one at that.
In terms of using an embedded database, two options to consider are SQLite and HyperSQL. However, serialization is also an option, and in my opinion it should be your default option if you've already implemented it. Some considerations:
With serialization you've generally got to retrieve the entire object, which is slow if you've got an object with several dozen fields and you only want to read one of them. If you're making queries like these, then use a database. I suspect that you're just reading in all of your serialized objects at startup and serializing them back out to disk at shutdown, in which case there's no reason to use a database instead of serialization.
Java's default serialization mechanism is fairly slow. You may want to consider another serialization mechanism, such as Kryo or Jackson, but only if you're not happy with your program's serialization performance.
It is difficult to advise on the best choice of technology without knowing what you are persisting and why.
If the state is simply a snapshot of your game state (i.e. a save file) or a "best scores" table, then you don't need a database. Serializing using JSON, XML or ... Java Object serialization is sufficient.
If the state needs to be read or updated incrementally or shared with other applications ... or users on other machines ... then a database is more appropriate.
Serialization mechanisms are problematic if the requirements include incremental changes, etcetera. You end up building a database-like layer over the top of the serialization.
As to whether you should stick with Java serialization ... or switch to JSON or XML or something like that:
Object serialization is simple, but it can be fragile if you change the classes that you are serializing. This fragility can be mitigated, but it is messy and you lose the simplicity. (You need to write custom readObject and writeObject methods that know how to read "old versions" of the serialized objects.)
JSON and XML are a bit more complicated, but still relatively simple if you use an object binding mechanism.
It is worth noting that changes to the persisted object classes (or the database schemas) are potentially problematic no matter what you do. There is no easy universal solution to this problem.
UPDATE
Given the additional information that you provided in your first comment (below), it seems like you don't need a database in the game itself. All you need is something that can read and analyse the session state save files that your beta testers provide for you. Indeed, it doesn't even seem like the actual app needs to be able read the files. (But that's unclear, because you've not said what the real purpose of these files is ... or at least, not what the entire purpose is.)
It is also worth noting that you are probably saving the wrong information if your aim is to tune the sets of questions. What you really need to do is record the length of time and whether the user got the right or wrong answer and the time ... for each individual question. And you probably need to know what the actual answer given was ... so that you can spot cases where the user's answer was actually right and you "marked" it as wrong ... or vice versa.
"What turns me away from that idea is the question of how am I going to write their database into mine (in the future)?"
Exactly. If you hadn't prematurely "analysed" the data, you wouldn't have this problem.
But ignoring that, it seems like that a simple state saving mechanism is sufficient to meet your (still hypothetical / inferred) requirement of keeping a personal score board for the end user. Your "tuning" stuff would be better implemented using a custom log file. I cannot see any value in incorporating a database as part of the app itself.
I presume you are doing java serialisation, If so there is nothing wrong with it. Just be aware of its limitations - Different versions of java might not be able to retrieve the file.
Also If you change the Class, previous saved data can not be retrieved.
If you decide to change you could look at Xml, JSon, Protocol Buffers, Thrift, Avro etc as well as a DB.
Note:
Xml is builtin in to java
Java Db (Derby) is also in Java
Other serialisation schema's require a seperate library.
I just had an idea that I wonder whether is possible in java. Let's say when doing debugging using eclipse or netbeans, you could record an object and save it. Then when going through the second round of debugging, save the object again. Now you could compare the first object recorded with the second object for all properties and find out any differences. Is this possible?
You can do this in plain Java code (assuming your objects are Serializable), but I don't think any debuggers have this feature built-in.
It would simply be a case of serialising the first object during the debugging run (which if you had a static method to do so, you could generally call from the debugger) and saving it somewhere. Then, during the second run, call another method to reconstitute the object from it's serialised form - and then compare the objects (either with their equals() methods, or some more bespoke comparison method).
In practice though I find that whenever I want to do this I just scribble down the relevant properties on a piece of paper and compare them manually. Rarely am I looking at thousands and thousands of properties that might change between a run; if you think about the symptoms you're seeing and the behaviour of your object, you can normally have a very good idea of what might be changing before you even fire up the debugger, and then use the latter to confirm your hypothesis and backtrack to see where the value "went wrong".
Give your object a useful toString() method and then use unit tests to compare the result with what you expect.
But I agree: The wire protocol for remote debugging can serialize any object, so it should be possible to write a program that does this automatically.
OTOH, objects which aren't meant to be serialized can be dangerous. If you accidentally use this on a classloader, you'll get all objects and classes and everything back as one big lump. So you need a way to stop the serialization to make sure it can't run havoc in a deep object tree.
On top of that, I'd like a feature to save the current state of the app and be able to go back in time.
I don't think any debugger can save object to compare them later. What you can do though is to create a watch variable on the variable, but wrap it with the ToStringBuilder() of the apache commons and dump it in the console, like so:
System.out.println(ToStringBuilder.reflectionToString(object));
Each time the breakpoint is reached, the content of the object will be shown in the console. You can even see the private data.
Therefore, you do not need to modify the toString() method of the object directly (this is useful for library object for example). You can then compare the output of your two passes.