For this specific project I need to serialize my entity layer (made of POJO's) to files. As I have the need for updating specific objects I would like to use one file per serialized object.
Example: Customer --ArrayList-> Order --ArrayList-> Product
When I edit a customer, and then serialize it using the java.io.Serializable interface, all fields, and their fields (please correct me if wrong), get serialized.
How would I apply serialization in such a way that only one object per file is used? I already have given each object a uniqe UUID which is used as filename when serializing.
If there are any frameworks that do File based ORM, that would be even better ;)
I'm not familiar with such framework.
What you can do is use other frameworks such as apache BeanUtils in order to perform the following recursive algorithm:
A. For each object gets its properties (assuming the object is a Java bean).
B. For each primitive field , write all primitives to file (you can use reflection to determine if a field is primitive or not).
C. For each non primitive file, write a special section in the file, pointing to the file name that will contain the object that is the value of the field.
D. Call recursively the algorithm for each non primitive field.
Similar approach can be done for collections -
HashMap, ArrayList and others.
The serializing code for the primitive elements can be the code provided by #Anshu
You can always read and write serializable objects using readObject and writeObject. Following is the example code:
import java.io.*;
import java.util.*;
import java.util.logging.*;
public class ExerciseSerializable {
public static void main(String... aArguments) {
//create a Serializable List
List<String> quarks = Arrays.asList(
"up", "down", "strange", "charm", "top", "bottom"
);
//serialize the List
//note the use of abstract base class references
try{
//use buffering
OutputStream file = new FileOutputStream( "quarks.ser" );
OutputStream buffer = new BufferedOutputStream( file );
ObjectOutput output = new ObjectOutputStream( buffer );
try{
output.writeObject(quarks);
}
finally{
output.close();
}
}
catch(IOException ex){
fLogger.log(Level.SEVERE, "Cannot perform output.", ex);
}
//deserialize the quarks.ser file
//note the use of abstract base class references
try{
//use buffering
InputStream file = new FileInputStream( "quarks.ser" );
InputStream buffer = new BufferedInputStream( file );
ObjectInput input = new ObjectInputStream ( buffer );
try{
//deserialize the List
List<String> recoveredQuarks = (List<String>)input.readObject();
//display its data
for(String quark: recoveredQuarks){
System.out.println("Recovered Quark: " + quark);
}
}
finally{
input.close();
}
}
catch(ClassNotFoundException ex){
fLogger.log(Level.SEVERE, "Cannot perform input. Class not found.", ex);
}
catch(IOException ex){
fLogger.log(Level.SEVERE, "Cannot perform input.", ex);
}
}
// PRIVATE //
//Use Java's logging facilities to record exceptions.
//The behavior of the logger can be configured through a
//text file, or programmatically through the logging API.
private static final Logger fLogger =
Logger.getLogger(ExerciseSerializable.class.getPackage().getName())
;
}
Related
Is there a way to read a serialized object from a .ser file and update or delete one of the objects that have been serialized?
The following is my code which read's in objects of type 'Driver':
public boolean checkPassword(String userName, String password, String depot) throws IOException {
FileInputStream fileIn = new FileInputStream("Drivers.ser");
ObjectInputStream in = new ObjectInputStream(fileIn);
try
{
while (true) {
Driver d = (Driver) in.readObject();
if (d.userName.equals(userName) && d.password.equals(password) && d.depot.equals(depot))
{
this.isManager = d.isManager;
validAccount = true;
}
}
}
catch (Exception e) {}
return validAccount;
}
You will need to read all objects from the original file and then write a new file containing only the objects you want to retain or update.
The Java serialized object stream format is not an archive file format like ZIP, JAR, TAR and so on. It is just a sequence of serialized objects. There is no "index" that would facilitate updating or deleting objects.
This is one reason why serialized objects are not a good way to implement data persistence. This is what databases are designed for.
I am given an assignment where we are not allowed to use a DB or libraries but only textfile for data storage.
But it has rather complex requirements, for e.g. many validations, because of that, we need to "access the db" (i.e. read the textfile) many times.
My question is: should I create a class like this:
class SomeRepository{
static ArrayList<Users> users = new ArrayList();
public SomeRepository(){
//instantiate this class on program load
//In constructor, we read the text file, instantiate and store everything inside the arraylist.
}
//public getOneUser(){ // for get methods, we don't read from text file at all }
/public save() { //text file saving code overhere }
}
Is this a good approach to solve the above problem? Currently, what we are doing is reading and writing to the text file every time we want to retrieve some data or write something new.
Wouldn't this be too expensive in terms of heap space memory? Or should I just read/write to the text file for every method?
public class IOManager {
public static void writeObjToTxtFile(String fileName, Object object) {
File file = new File(fileName + ".txt");//File will be created in the root directory where the program runs.
try (FileOutputStream fos = new FileOutputStream(file);
ObjectOutputStream oos = new ObjectOutputStream(fos);) {
oos.writeObject(object);
} catch (IOException e) {
e.printStackTrace();
}
}
public static Object readObjFromTxtFile(String fileName) {
Object obj = null;
File file = new File(fileName + ".txt");
FileInputStream fis = null;
try {
fis = new FileInputStream(file);
ObjectInputStream ois = new ObjectInputStream(fis);
obj = ois.readObject();
} catch (ClassNotFoundException | IOException e) {
e.printStackTrace();
}
return obj;
}
}
Add this class to your project. Since it's general for all Objects, you can pass and receive Objects like these as well: ArrayList<Users>. Play around and Tinker with it to fit whatever your specific purpose is. Hint: You can write other custom methods that calls these methods. eg:
public static void writeUsersToFile(ArrayList<Users> usersArrayList){
writeObjToTxtFile("users",usersArrayList);
}
Ps. Make sure your Objects implement Serializable. Eg:
public class Users implements Serializable {
}
I would suggest reading the contents of your file to a dynamic list such as an arraylist at the start of your program. Make the required queries/changes to your arraylist and then write that arraylist to your file when the program is set to close. This will save significant time over repeated file reads/writes.
This isn't without it's drawbacks, though. You don't want to hogg up memory in case of very large files - but considering this is an assignment, that may not be the case. Additionally, should your program terminate prior to the write at the end, all changes made to your database during the current execution will be lost.
I am writing data with types as int, char, double, byte, and boolean to and from files.
I have the methods that write my data to the files.
For the reading method, I am having the content of the files put into an ArrayList and then transferring them into a plain array. However in order to do that I have to know what the data type of the file's contents are.
So my question here is:
How do I check to see what the data type of the contents of a random file is?
You access files via Streams.
What you read out of a file depends on the stream-class you use to access the file.
There are two main groups in Java (up to Java 7):
1. The "...Reader" classes. Here the content of the file is read as sequence of chars.
2. The "...Stream" classes. Here the content of the file is read as a sequence of bytes.
However, you can write and read Java Objects to and from a file "directly" via the ObjectOutputStream and the ObjectInputStream classes. With them you ca read/write serialized Java-objects and primitive datatypes. With this you could check what you want in the following way:
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("filename"));
Object o = ois.readObject();
if (o instanceof)
//... checked
(This only works if the content of the file are serialized java objects.)
There's is no simple generic way to do this automatically.
If you want to retain additional information as the type, you will have to store it along with the data.
Here are some examples of how you can store the type info.
Using annotation. Always store the type explicitly with the related data, e.g. if the first token on a line is equal to "type3" then the following data represents a floating point number (for example).
type1 herecomesthedata1
type2 11011010111011010
type3 55.67
For more complex data with trees of variables use a well known data annotation standard as JSON or XML.
Using structure. Always have the variables written in the same order, e.g. the first token on a line is always an integer and the next is always floating point etc. Use this information when reading the data.
123 43.11
456 78.90
Use Java's built in serialization utilites as ObjectOutputStream and ObjectInputStream (only works on primitive types and objects that implement java.io.Serializable).
Example (using structure to determine type):
int i = 5;
double d = 7.3;
try (ObjectOutputStream out = new ObjectOutputStream(
new FileOutputStream("test.dat"))) {
out.writeInt(i);
out.writeDouble(d);
} catch (IOException e) {
System.err.println("An error occured writing to file");
}
i = 0;
d = 0.0;
try (ObjectInputStream in = new ObjectInputStream(
new FileInputStream("test.dat"))) {
i = in.readInt();
d = in.readDouble();
} catch (IOException e) {
System.err.println("An error occured reading from file");
}
System.out.println("i = " + i + " and d = " + d); // 5 and 7.3
Before I proceed to my question : please note that I am not working on any client-server application that would require serialization, but the program I am trying to customize stores one big instance of one big class in a .dat file. I have read about this issue (memory leak in ObjectOutputStream and ObjectInputStream)and the fact that I could probably need to :
use the ObjectOutputStream.reset() method after writing the class instance in the .dat file, so that it doesn't hold the reference anymore;
re-write the code without using serialization;
split the file and read it in chunks;
change the JVM memory parameter by using -Xmx;
So, I was provided with one class that generates a language model and saves it with a .dat extension; the code was probably optimized for small model files (there are 2 model files provided as examples, both around 10MB ), but I generated a much larger model class, and it is around 40MB. Then, there is another class in another folder, totally independent on the first one, that uses this model, and the model has to be loaded using ObjectInputStream. Here comes the problem : a classic "OutOfMemoryError : Java heap space".
Writing the object:
try {
// Create an output stream to the file.
FileOutputStream file_output = new FileOutputStream (file);
ObjectOutputStream o = new ObjectOutputStream( file_output );
o.writeObject(this);
file_output.close ();
}
catch (IOException e) {
System.err.println ("IO exception = " + e );
}
Reading the object:
InputStream model = null;
ModelGeneration oRead = null;
ObjectInputStream p = null;
try {
model = new FileInputStream(filename);
BufferedInputStream buf = new BufferedInputStream(model);
p = new ObjectInputStream(buf);
oRead = (ModelGeneration) p.readObject();
p.reset();
} catch (IOException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
} finally {
try {
model.close();
} catch (Exception e) {
e.printStackTrace();
}
}
I tried to use the reset() method, but it is useless because we load only one instance of one class at a time, nothing else needed. This is why I can't split the file, too: only one class instance is stored in the .dat file.
Changing the heap space seems like a worse solution than optimizing the code.
I would really appreciate your advice on what I can do.
Btw the code is here : http://svn.apache.org/repos/asf/uima/addons/trunk/Tagger/, I only implemented the required classes for a different language.
P.S. Works fine if I create a smaller model, but I would prefer the bigger one.
I'm doing an animation in Processing. I'm using random points and I need to execute the code twice for stereo vision.
I have lots of random variables in my code, so I should save it somewhere for the second run or re-generate the SAME string of "random" numbers any time I run the program. (as said here: http://www.coderanch.com/t/372076/java/java/save-random-numbers)
Is this approach possible? How? If I save the numbers in a txt file and then read it, will my program run slower? What's the best way to do this?
Thanks.
If you just need to be able to generate the same sequence for a limited time, seeding the random number generator with the same value to generate the same sequence is most likely the easiest and fastest way to go. Just make sure that any parallel threads always request their pseudo random numbers in the same sequence, or you'll be in trouble.
Note though that there afaik is nothing guaranteeing the same sequence if you update your Java VM or even run a patch, so if you want long time storage for your sequence, or want to be able to use it outside of your Java program, you need to save it to a file.
Here is a sample example:
public static void writeRandomDoublesToFile(String filePath, int numbersCount) throws IOException
{
FileOutputStream fos = new FileOutputStream(new File(filePath));
BufferedOutputStream bos = new BufferedOutputStream(fos);
DataOutputStream dos = new DataOutputStream(bos);
dos.writeInt(numbersCount);
for(int i = 0; i < numbersCount; i++) dos.writeDouble(Math.random());
}
public static double[] readRandomDoublesFromFile(String filePath) throws IOException
{
FileInputStream fis = new FileInputStream(new File(filePath));
BufferedInputStream bis = new BufferedInputStream(fis);
DataInputStream dis = new DataInputStream(bis);
int numbersCount = dis.readInt();
double[] result = new double[numbersCount];
for(int i = 0; i < numbersCount; i++) result[i] = dis.readDouble();
return result;
}
Well, there's a couple of ways that you can approach this problem. One of them would be to save the random variables as input into a file and pass that file name as a parameter to your program.
And you could do that in one of two ways, the first of which would be to use the args[] parameter:
import java.io.*;
import java.util.*;
public class bla {
public static void main(String[] args) {
// You'd need to put some verification code here to make
// sure that input was actually sent to the program.
Scanner in = new Scanner(new File(args[1]));
while(in.hasNextLine()) {
System.out.println(in.nextLine());
}
} }
Another way would be to use Scanner and read from the console input. It's all the same code as above, but instead of Scanner in = new Scanner(new File(args[1])); and all the verification code above that. You'd substitute Scanner in = new Scanner(System.in), but that's just to load the file.
The process of generating those points could be done in the following manner:
import java.util.*;
import java.io.*;
public class generator {
public static void main(String[] args) {
// You'd get some user input (or not) here
// that would ask for the file to save to,
// and that can be done by either using the
// scanner class like the input example above,
// or by using args, but in this case we'll
// just say:
String fileName = "somefile.txt";
FileWriter fstream = new FileWriter(fileName);
BufferedWriter out = new BufferedWriter(fstream);
out.write("Stuff");
out.close();
}
}
Both of those solutions are simple ways to read and write to and from a file in Java. However, if you deploy either of those solutions, you're still left with some kind of parsing of the data.
If it were me, I'd go for object serialization, and store a binary copy of the data structure I've already generated to disk rather than having to parse and reparse that information in an inefficient way. (Using text files, usually, takes up more disk space.)
And here's how you would do that (Here, I'm going to reuse code that has already been written, and comment on it along the way) Source
You declare some wrapper class that holds data (you don't always have to do this, by the way.)
public class Employee implements java.io.Serializable
{
public String name;
public String address;
public int transient SSN;
public int number;
public void mailCheck()
{
System.out.println("Mailing a check to " + name
+ " " + address);
}
}
And then, to serialize:
import java.io.*;
public class SerializeDemo
{
public static void main(String [] args)
{
Employee e = new Employee();
e.name = "Reyan Ali";
e.address = "Phokka Kuan, Ambehta Peer";
e.SSN = 11122333;
e.number = 101;
try
{
FileOutputStream fileOut =
new FileOutputStream("employee.ser");
ObjectOutputStream out =
new ObjectOutputStream(fileOut);
out.writeObject(e);
out.close();
fileOut.close();
}catch(IOException i)
{
i.printStackTrace();
}
}
}
And then, to deserialize:
import java.io.*;
public class DeserializeDemo
{
public static void main(String [] args)
{
Employee e = null;
try
{
FileInputStream fileIn =
new FileInputStream("employee.ser");
ObjectInputStream in = new ObjectInputStream(fileIn);
e = (Employee) in.readObject();
in.close();
fileIn.close();
}catch(IOException i)
{
i.printStackTrace();
return;
}catch(ClassNotFoundException c)
{
System.out.println(.Employee class not found.);
c.printStackTrace();
return;
}
System.out.println("Deserialized Employee...");
System.out.println("Name: " + e.name);
System.out.println("Address: " + e.address);
System.out.println("SSN: " + e.SSN);
System.out.println("Number: " + e.number);
}
}
Another alternative solution to your problem, that does not involve storing data, is to create a lazy generator for whatever function that provides you your random values, and provide the same seed each and every time. That way, you don't have to store any data at all.
However, that still is quite a bit slower (I think) than serializing the object to disk and loading it back up again. (Of course, that's a really subjective statement, but I'm not going to enumerate cases where that is not true). The advantage of doing that is so that it doesn't require any kind of storage at all.
Another way, that you may have not possibly thought of, is to create a wrapper around your generator function that memoizes the output -- meaning that data that has already been generated before will be retrieved from memory and will not have to be generated again if the same inputs are true. You can see some resources on that here: Memoization source
The idea behind memoizing your function calls is that you save time without persisting to disk. This is ideal if the same values are generated over and over and over again. Of course, for a set of random points, this isn't going to work very well if every point is unique, but keep that in the back of your mind.
The really interesting part comes when considering the ways that all the previous strategies I've described in this post can be combined together.
It'd be interesting to setup a Memoizer class, like described in the second page of 2 and then implement java.io.Serialization in that class. After that, you can add methods save(String fileName) and load(String fileName) in the memoizer class that make serialization and deserialization easier, so you can persist the cache used to memoize the function. Very useful.
Anyway, enough is enough. In short, just use the same seed value, and generate the same point pairs on the fly.