I'm still working on the project I already needed a bit of help with:
JavaFX - TableView doesn't update items
Now I want to understand how this whole Serialization process in Java works, because unfortunately, I don't really get it now.
Before I go on, first of all, I'm a student, I'm not a professional. Second, I'm neither familiar with using DBs, nor XML or JSON, so I'd just like to find solution to my approach, no matter how inelegant it might be in the end, it just needs to work. So please don't feel offended if I just reject any advice in using other techniques.
So here's what I want:
Saving three different class objects to separate files BUT maintaining backward compatibility to each of it. The objects are Settings, Statistics and a "database" object, containing all words in a list added to it. In the future I may add more statistics or settings, means adding new variables, mostly type of IntegerProperty or DoubleProperty.
Now the question is: is it possible to load old version saved files and then during the process just initiate new variables not found in the old version with just null but keep the rest as it has been saved?
All I know is that the first thing to do so is not to alter the serialVersionUID.
Another thing would be saving the whole Model object (which contains the three objects mentioned before), so I just have to implement stuff for one class instead of three. But how would that work then concerning backward compatibility? I mean the class itself would not change but it's attributes in their own class structure.
Finally, what approach should I go for? And most of all, how do I do this and maintaning backward compatibilty at the same time? I do best with some concrete examples rather than plain theory.
Here are two example methods, if it's of any help. I already have methods for each class to write and read an object.
public static void saveModel(Model model, String destination) throws IOException
{
try
{
fileOutput = new FileOutputStream(destination);
objectOutput = new ObjectOutputStream(fileOutput);
objectOutput.writeObject(model);
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
if (objectOutput != null)
try
{
objectOutput.close();
}
catch (IOException e) {}
if (fileOutput != null)
try
{
fileOutput.close();
}
catch (IOException e) {}
}
}
public static Settings readSettings(String destination) throws IOException, FileNotFoundException
{
Settings s = null;
try
{
fileInput = new FileInputStream(destination);
objectInput = new ObjectInputStream(fileInput);
Object obj = objectInput.readObject();
if (obj instanceof Settings)
{
s = (Settings)obj;
}
}
catch (IOException e)
{
e.printStackTrace();
}
catch (ClassNotFoundException e)
{
e.printStackTrace();
}
finally
{
if (objectInput != null) try { objectInput.close(); } catch (IOException e) {}
if (fileInput != null) try { fileInput.close(); } catch (IOException e) {}
}
return s;
}
Tell me if you need more of my current code.
Thank you in advance!
... you must be this tall
Best advice for Serialisation is to avoid it for application persistence, especially if backwards compatibility is desired property in your application.
Answers
Is it possible to load old version saved files and then during the process just initiate new variables not found in the old version with just null but keep the rest as it has been saved?
Yes. Deserialising objects saved using previous versions of the class into a new version of this class will work only if:
fully qualified name of the class has not changed (same name and package)
previous and current class have exactly the same serialVersionUID; if one of the versions is missing it, it will be calculated as a 'hash' of all fields and methods and upon a mismatch deserialisation will fail.
inheritance hierarchy has not changed for that class (the same ancestors)
no fields have been removed in the new version of the class
no fields have become static
no fields have become transient
I just have to implement stuff for one class instead of three. But how would that work then concerning backward compatibility?
Yes. Providing that all classes of all fields of Model and Model class itself adhere to the rules above.
Finally, what approach should I go for? And most of all, how do I do this and maintaning backward compatibilty at the same time?
Yes, as long as you can guarantee all of the above rules forever, you will be backwards compatible.
I am sure you can appreciate that forever, or even for next year can be very hard to guarantee, especially in software.
This is why people do application persistence using more robust data exchange formats, than binary representation of serialised Java objects.
Raw data for the table, could be saved using anything from CSV file to JSON docs stored as files or as documents in NoSQL database.
For settings / config have a look at Java's Properties class which could store and load properties to and from *.properties or *.xml files or separately have a look at YAML.
Finally for backwards compatibility, have a look at FlatBuffers
The field of application persistence is very rich and ripe, so happy exploring.
Related
This question already has answers here:
What is object serialization?
(15 answers)
Closed 2 years ago.
I'm trying to make a Client/Server chat application using java. I'm pretty new to using sockets to communicate between applications. I've decided to use ObjectInput/ObjectOutput streams to send objects between the client and server.
I'm trying to send user data to the server when the client connects to the socket. Here is the code.
Server:
private void startServer() {
try {
this.server = new ServerSocket(port);
this.socket = server.accept();
ChatUtils.log("Accepted a new connection!");
this.output = new ObjectOutputStream(socket.getOutputStream());
this.input = new ObjectInputStream(socket.getInputStream());
try {
User user = (User) input.readObject();
ChatUtils.log(user.getDisplayName() + " (" + user.getUsername() + ") has connected!");
} catch (ClassNotFoundException e) {
}
} catch (IOException e) {
e.printStackTrace();
}
}
Client:
public void connectToServer(int port) {
try {
server = new Socket("127.0.0.1", port);
this.port = port;
this.objectOutput = new ObjectOutputStream(server.getOutputStream());
System.out.println("Connected to a server on port " + port + "!");
objectOutput.writeObject(user);
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
Everything works fine, but I'm looking for some clarification as to how the methods ObjectOutputStream#writeObject() and ObjectInputStream#readObject() work.
When I write the line User user = (User) input.readObject();, it reads the object as a User object. Would this only attempt to convert "User" objects that are send from the client's ObjectOutputStream?
As this method is only called once, can I cast the input stream to other objects if I send those objects to the server from the output stream? Ex: String message = (String) input.readObject();.
What would happen if I sent multiple objects to the server from the output stream at once?
4)In example one, I try to read the "user" object. What happens if there are two or more objects waiting to be read? How do I determine which object is which? Ex:
// Client
public void connectToServer() {
String message = "Hello server!"
User user = new User("John Doe", "jdoe123");
output.writeObject(user);
output.writeObject(message);
}
If someone could answer these questions, that'd be great. Thanks so much!
Every time you call .writeObject, java will take the object you specified and will serialize it.
This process is a hacky, not-recommended strategy.
Java will first attempt to break down the object you passed into its constituent parts. It will do this, hopefully, with some assistance from the class definition (the class that the object is, i.e. the one returned by theObjectWritten.getClass(). any class def that implements Serializable claims to be designed for this and gets some additional help, but the mechanism will try with reflection hacks if you don't.
Then, the constituent parts are sent along the wire (that is, take the object, and any fields that are primitives can just be sent; ObjectOutputStream knows how to send an int intrinsically, for example. Any other types are sent by, in turn, asking THAT object's class to do so). For each object, java also sends the so called 'serial version uid', which is a calculated number and changes any time any so-called signature changes anywhere in the class. It's a combination of the class's package, name, which class it extends, which interfaces it implements, and every name and type of every field (and possibly every name, return type, param types, and exception types thrown for every method).
So, now we have a bundle, consisting of:
The name of the class (e.g. com.foo.elliott.User)
The serialversionUID of the class
the actual data in User. If User contained any non-primitive fields, apply this process recursively.
Then this is all sent across the wire.
Then on receipt, the receiving code will take all that and pack it back into a User object. This will fail, unless the receiving end actually has com.foo.elliott.User on the classpath, and that def has the same serial version UID.
In other words, if you ever update this class, the transport fails unless the 'other side' also updates.
You can manually massage this stuff by explicitly declaring the serialVersionUID, but note that e.g. any created fields just end up being blank, even if the constructor ordinarily would ensure they could never be.
You can also fully manually manage all this by overriding some specific 'voodoo' methods (a method with a specific name. Java is ordinarily not structurally typed, but these relics of 25 years in the past, such as psv main and these methods, are the only structurally typed things in all of java).
In addition, the binary format of this data is more or less 'closed', it is not obvious, not easy to decode, and few libraries exist.
So, the upshot is:
It is a finicky, error ridden process.
Updating anything you serialize is a pain in the behind.
You stand no chance of ever reading this wire protocol with any programming language except java.
The format is neither easy to read, nor easy to work with, nor particularly compact.
This leads to the inevitable conclusion: Don't use ObjectOutputStream.
Instead, use other serialization frameworks that weren't designed 25 years ago, such as JSON or XML marshallers like google's GSON or Jackson.
NB: In addition your code is broken. Whenever you make a resource, you must also close it, and as code may exit before you get there, the only solution is a special construct. This is how to do it:
try (OutputStream out = socket.getOutputStream()) { .. do stuff here .. }
note that no matter how code 'escapes' from the braces, be it normally (run to the end of it), or because you return/break/continue out of it, or an exception is thrown, the resource is closed.
This also means assigning resources (anything that implements AutoClosable, like Socket, InputStream, and OutputStream, does so) to fields is broken, unless you make the class itself an AutoClosable, and whomever makes it, does so in one of these try-with blocks.
Finally, don't catch exceptions unless you can actually handle them, and 'printStackTrace' doesn't count. If you have no idea how to handle it, throw it onwards; declare your methods to 'throws IOException'. main can (and should!) generally be declared as throws Exception. If truly you can't, the 'stand in', forget-about-it correct way to handle this, and update your IDE to generate this instead of the rather problematic e.printStackTrace(), is this:
catch (ThingICantHandleException e) {
throw new RuntimeException("unhandled", e);
}
Not doing so means your code continues whilst the process is in an error state, and you don't want that.
I am using Serialization to get persistent storage for my library managing app (I know it is not the right way, but it's the way my professor wants it).
I am using the following code inside my main();
controlador.getBiblioteca().getGestorMaterial().setListaLibros((Modelo.ColeccionLibros) controlador.getSerializador().abrirArchivo("libros.dat"));
My Serializador class has the abrirArchivo("FileName.dat") function (openFile in English).
That function looks like this:
public Object abrirArchivo(String nombreDelArchivo) {
Object retorno = null;
try {
lectorArchivos = new ObjectInputStream(new FileInputStream(
nombreDelArchivo));
retorno = lectorArchivos.readObject();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
return retorno;
}
Now I am trying to check if the program gets a FileNotFoundException for any of the files. If it does, it should just not deserialize the file and go for the next one: that would just mean there is no book in my library yet.
The problem is, if the line runs, it seems to set my book list using setListaLibros() to null. And whenever I try to access that list, i get a NullPointerException. The list was already initialized as an empty list though, so I just need to leave it alone as long as the "libros.dat" file is not found.
What is the right way to get that done?
I don't know if I understand the problem well. However, as I can see in your code, when an exception FileNotFoundException happens, "retorno" will keep null. That's the reason why you get setListaLibros(null).
And then your list will became null. If you don't want that behavior, you should initialize "retorno" with an empty list instead of null.
You could add a line before this: controlador.getBiblioteca().getGestorMaterial().setListaLibros((Modelo.ColeccionLibros) controlador.getSerializador().abrirArchivo("libros.dat")); which checks if the file exists. If it does not, then it prompts the user. This way, the user knows something went wrong and can act accordingly.
Alternatively, you can make a change in your setListLibros method wherein, if the argument passed is null, then you do not do any assignment.
Personally, I would go with the first option.
As a side note, please break down your code, something like so: controlador.getBiblioteca().getGestorMaterial().setListaLibros((Modelo.ColeccionLibros) controlador.getSerializador().abrirArchivo("libros.dat")) can get hard to read and debug.
There is an aspect that the other answers are not mentioning: why are there no serialized objects when your library is empty?!
What I mean is: you could distinguish between "program runs the first time" (and obviously no serialized data exists) or "program ran before; and thus it fully configures itself from serialized data.
Meaning: "being empty" can be a valid state of a library, too. So another option would be to not use a "special value" (aka "no file with data") to represent that information ... but (de)serialize an empty list.
You could check if the file exist like:
String fileName;
File f1 = new File(fileName);
if (f1.exists()) {
//Do the work
}
So here is my problem:
for a project I had to create a custom linked list whereby I had to add nodes to it and save/load it to and from the disk using serialisation
here are some things about my system before I define the problem
I have a generic 'customer file' which acts as the node data
this is stored in a node object which acts as an element of the list
there is a customer file class where the information such as name and email address are stored as well as the various get and set methods for each - these work fine
there is a node class with get and set data and next methods for each whereby the next item is a node object and acts as the next item in the list
there is a singly linked class with add, remove, modify, sort, search etc... methods - IT IS A CUSTOM MADE CLASS AND SO DOES NOT IMPLEMENT ANY JAVA PREMADE LISTS.
a lot of testing has been on all classes separately and used together - these methods are foolproof - they work
there is a main class which is used for the main interface between the user and the system - it is a CLI system (command line)
it has a save list to file function and load list from file function (in the main class) whereby each function uses serialization or deserialization to save/load the list from the disk
all classes implement the serializable interface
there is a 'MAIN' method in the main class whereby a while loop operates which allows the user to modify the list in some way (eg add a record, remove a record etc...)
the list is loaded outside the loop so it is not cleared each time the loop iterates (a common problem amongst colleagues)
i have a password for the system whereby identical methods are used to save a string to another file location and that has worked for weeks - the password is saved at that location and can be accessed, changed and removed at will
these load/save methods have the appropriate try/catch methods to catch any exceptions
The problem is that each time i load up my programming environment and want to look at the list, I find that the list on file is 'empty' and contained no records from when i last added/removed stuff.
I add records and modify the list - easy peasy as the other classes work - once these are added, i call the print function which simply displays all items in the list and it is fine.
However, the minute i close the environment, they are lost and when i reopen the environment to look at the list again, it is empty!
Upon looking in the folder where these classes are saved, i have noticed each time i run the program that 'shells' are created and remain there until the program is closed/finished however the 'listData.ser' which should have the linked list saved does not have any data.
Likewise the password file contains the password which was saved fine - so i am a little confused as to why my code does not work.
Here is my save list method:
private static void saveListToFile(SinglyLinkedList lst) {
try {
ObjectOutputStream os = new ObjectOutputStream(new FileOutputStream("ListData.ser"));
os.writeObject(lst);
os.flush();
os.close();
}
catch (FileNotFoundException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
}
Likewise the load list method is similar but uses object input stream and file input stream.
Any suggestions?
P.S. My main while loop is over 400 lines of code long and therefore not feasible to post.
Update 1.
Deserialization code in load list method:
private static SinglyLinkedList loadListFromFile() {
SinglyLinkedList lst = null;
try {
ObjectInputStream is = new ObjectInputStream(new FileInputStream("ListData.ser"));
lst = (SinglyLinkedList) is.readObject();
is.close();
}
catch(FileNotFoundException e) {
e.printStackTrace();
}
catch(IOException e) {
e.printStackTrace();
}
catch(ClassNotFoundException e) {
e.printStackTrace();
}
return lst;
}
I dont think the singly linked list class itself is the problem (response to comment) and it is not feasible to copy as it is also over 300 lines of code (lots of methods).
Have you tried calling close() on the FileOutputStream when you are done writing the file/object?
I have solved it, instead of posting my code I tried to do it myself. I turns out there were a few static methods of the list class - these were changed to non-static and now the list saves as expected each time.
Thanks for the help
In my experience and after repeated tests I've done and deep web researches, I've found that major java libraries (either "Apache Commons" or Google.coomons or Jcifs) doesn't predict the case of “cyclic copy” of a file onto a destination differently mapped (denoted with different RootPath according with newer java.nio package Path Class) that,at last end of mapping cycle,resolves into the itself origin file.
That's a situation of data losing, because Outputsream method nor jnio's GetChannel method prevents itself this case:the origin file and the destination file are in reality "the same file" and the result of these methods is that the file become lost, better said the size o file become 0 length.
How can one avoid this without get off at a lower filesystem level or even surrender to a more safe Runtime.exec, delegating the stuff at the underlying S.O.
Should I have to lock the destination file (the above methods not allowing this), perhaps with the aid of the oldest RandomAccessFile Class ?
You can test using those cited major libraries with a common "CopyFile(File origin,File dest)" method after having done:
1) the origin folder of file c:\tmp\test.txt mapped to to x: virtual drive via a cmd's [SUBST x: c:\tmp] thus trying to copy onto x:\test.txt
2) Similar case if the local folder c:\tmp has been shared via Windows share mechanism and the destination is represented as a UNC path ending with the same file name
3) Other similar network situations ...
I think there must be another better solution, but my experience of java is fairly few and so I ask for this to you all. Thanks in advance if interested in this “real world” discussion.
Your question is interesting, never thought about that. Look at this question: Determine Symbolic Links. You should detect the cycle before copying.
Perhaps you can try to approach this problem slightly differently and try to detect that source and destination files are the same by comparing file's metadata (name, size, date, etc) and perhaps even calculate hash of the files content as well. This would of course slow processing down.
If you have enough permissions you could also write 'marker' file with random name in destination and try to read it at the source to detect that they're pointing to the same place. Or try to check that file already exist at destination before copying.
I agree that it is unusual situations, but you will agree that files are a critical base of every IT system. I disagree that manipulating files in java is unusual: in my case I have to attach image files of products through FileChooser and copy them in ordered way to a repository ... but real world users (call them customers who buy your product) may fall in such situations and if it happens, one can not 'blame the devil of bad luck if your product does something "less" than expected.
It is a good practice learning from experience and try to avoid what one of Murphy's Laws says, more' or less: "if something CAN go wrong, it WILL go wrong sooner or later.
Is perhaps also for one of those a reason I believe the Java team at Sun and Oracle has enhanced the old java.io package for to the newest java.nio. I'm analyzing a the new java.nio.Files Class which I had escaped to attention, and soon I believe I've found the solution I wanted and expected. See you later.
Thank for the address from other experienced members of the community,and thanks also to a young member of my team, Tindaro, who helped me in the research, I've found the real solution in Jdk 1.7, which is made by reliable, fast, simple and almost definitively will spawn a pity veil on older java.io solutions. Despite the web is still plenty full of examples of copying files in java using In/out Streams I'll warmely suggest everyone to use a simple method : java.nio.Files.copy(Path origin, Path destination) with optional parameters for replacing destination,migrate metadata file attributes and even try a transactional move of files (if permitted by the underlying O.S.).
That's a really good Job, waited for so long!
You can easily convert code from copy(File file1, File file2) by appending a ".toPath()" to the File instance (e.g. file1.toPath(), file2.toPath().
Note also that the boolean method "isSameFile(file1.toPath(), file2.toPath())", is already used inside the above copy method but easily usable in every case you want.
For every case you can't upgrade to 1.7 using community libraries from Apache or Google is still suggested, but for reliable purpose, permit me to suggest the temporary workaround I've found before:
public static boolean isTheSameFile(File f1, File f2) {//throws Exception{
// minimum prerequisites !
if(f1.length()!=f2.length()) return false;
if (!file1.exists() || !file2.exists()) { return false; }
if (file1.isDirectory() || file2.isDirectory()){ return false; }
//if (file1.getCanonicalFile().equals(file2.getCanonicalFile())); //don't rely in this ! can even still fail
//new FileInputStream(f2).getChannel().lock();//exception, can lock only on OutputStream
RandomAccessFile rf1=null,rf2=null; //the only practicable solution on my own ... better than parse entire files
try {
rf1 = new RandomAccessFile(f1, "r");
rf2=new RandomAccessFile(f2, "rw");
} catch (FileNotFoundException e) {
e.printStackTrace();
return false;
}
try {
rf2.getChannel().lock();
} catch (IOException e) {
return false;
}
try {
rf1.getChannel().read(ByteBuffer.allocate(1));//reads 1 only byte
} catch (IOException e) {
//e.printStackTrace(); // if and if only the same file, the O.S. will throw an IOException with reason "file already in use"
try {rf2.close();} catch (IOException e1) {}
return true;
}
//close the still opened resources ...
if (rf1.getChannel().isOpen())
try {rf1.getChannel().close();} catch (IOException e) {}
try {
rf2.close();
} catch (IOException e) {
return false;
}
// done, files differs
return false;
}
Has anyone ever persisted a training set for CI-Bayes? I have sample code from this site: http://www.theserverside.com/news/thread.tss?thread_id=49773
here is the code:
FisherClassifier fc=new FisherClassifierImpl();
fc.train("The quick brown fox jumps over the lazy dog's tail","good");
fc.train("Make money fast!", "bad");
String classification=fc.getClassification("money", "unknown"); // should be "bad"
so I need to be able to store the training set in a local file.
Has anyone ever done this before?
To persist a java Object in a local file, the Object must first implement the Serializable interface.
import java.io.Serializable;
public class MyClass implements Serializable {...
Then, the class from which you would like to persist this training set, should include a method like:
public void persistTrainingSet(FisherClassifier fc) {
String outputFile = <path/to/output/file>;
try {
FileOutputStream fos = new FileOutputStream(outputFile);
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(fc);
oos.close();
}
catch (IOException e) {
//handle exception
}
finally {
//do any cleaning up
}
}
I have. After doing a couple projects with CI-Bayes, I would recommend you look elsewhere (of course this was a long time ago). It is a very bad idea to use an inference engine that needs to be trained before each use and if you really consider the issue of state management, it's complicated (e.g. do you want to just store the training data? or perhaps the trained distributions? chains?).
CI-Bayes is also kind of a convoluted codebase. It was modeled off some Python code that appeared in a book about intelligence. The Java version is not very well designed. It also does not use TDD, does not really have JavaDoc to speak of.
That said, you can get a simple classifier going pretty quickly. The longer term goal is the one you asked about though.