I have an application which stores information in a JList. However, of course, when the application is closed all of the information is deleted from memory.
I'm trying to build the app so that when re-launched, it will contain the same data. So is there a way to store this data in a database or similar and if so? Where and how do I go about this?
The simplest way to persist IMHO is in a File.
Try using Properties if you need a key-value map.
Or, if it you're binding more complex objects I recommend a Simple XML serialization package.
You need to connect your application to a database using JDBC. JDBC stands for Java Database Connectivity. As you can see from the name, it lets you to connect to a database. Hence, you can link your application to a database,and store your data persistenly.Here's a link to start off with. And here is something for further reading.
If the data is not complex and is not large (more than a few instances of a few objects) you could persist the list to a file using serialization. This will get you started. If you list is large or complex you might consider a database. Searching for JDBC will in your favorite search engine will get you started.
I think you want a plain flat file. It's simple; you can have one going in no time. (The learning curve is much less than with databases.) And it's fast; you can read a 1 GB file before you can even log on to a DB. Java serialization is a bit tricky, but it can be a very powerful way to save vast amounts of complicated data. (See here for things to watch out for, plus more helpful links.) If, for instance, you wanted to save a large, complex game between sessions, serializing it is the way to go. No need to convert an Object Oriented structure to a relational one.
Use a database:
if you want to add data to a large file, or read only part of the data from a large file. Or if other processes are going to read and modify it.
Consider a DB:
if you are already using one for other purposes. If the user might start on another machine and have trouble finding the file from the last session and the data is not too extensive. Or if the data is relational in nature anyway and someone else may be interested in looking at it.
So if you have a simple case where the user always starts in the same directory, just write and read a simple file. If you have a lot of complex, extensive OO data, use a flat file even if it is not easy to do--you'll need the speed. Otherwise, think about a DB.
Related
I am a beginner in programming, so I am trying to learn with projects. My newest project is to create an agenda/calendar that is accessible from different computers (like a family calendar) so mom or dad can put up their events and everyone can see everyone's plans.
To a program that can store the instance of a family's agenda so they can go back to it at any time, I assume some sort of database or server to store their information is needed. How could I do this?
I apologize if my question is vague. I am relatively new to programming, but am so eager to keep learning.
You have several options.
The easiest is Serialization. Serialization takes an object and writes it to a stream using an ObjectOutputStream. You can read it back with an ObjectInputStream.
It's trivial because without error checking, it's just a few lines of code.
FileOutputStream fos = new FileOutputStream("calendar.dat");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(yourCalendar);
oos.close();
Similarly:
FileInputStream fis = new FileInputStream("calendar.dat");
ObjectInputStream ois = new ObjectInputStream(fis);
YourCalendar yourCalendar = (YourCalendar)ois.readObject();
ois.close();
Where yourCalendar is the master object containing your entire calendar and the appointments, etc.
Since you're not dealing with large amounts of information, it's perfectly adequate.
Now, that said, it's also fraught with danger. The file format is opaque (you can't just open it up in an editor and look at it). It can also be quite brittle. If you change your underlying classes that you're serializing, you may not be able to read a file back in. There's also potential security implications (likely not germane in your case, but they're still there).
Much of those can be mitigated, at the cost of complexity.
Similarly, you can use one of the JSON or XML libraries to serialize your objects out in to one of those text based formats. These are human readable, and can be a bit less sensitive to change than the binary format.
Of course with all of these, they're "all or nothing". In this case, you're writing out the entire object and all of its embedded objects. That means you can can't individually access the data. Nor can you use a 3rd party tool to access the data (like an SQL toolset). But, again, you don't have much data, so having this kind of access is likely not a big deal.
You wouldn't want to use this in a multi-user scenario, as it can not be incrementally updated (again, all or nothing).
But, all that said, for getting up and running, for simple persistence and being cognizant of its limitations, it will do the job for you and let you check this box on your project as you strive to work the other aspects of it. It's easy to enough to start with this and then, later, make a more robust persistence mechanism.
Memory is volatile. For storing data persistently you need to write it either in files or in databases.
Since it is opinion based question, I am putting my opinion.
You may begin with learning to read and write to files (text as well as binary).
While Writing to and reading from files you need to decide in which format you need to store in which format JSON, Yaml, XML or comma-separated or serialize your objects and store them into a file. The choice is yours.
While reading you need to write your own logic to search into them. So while files are a good and easy way to store data, you need to write either your own search mechanism or use document search like Elastic search.
Another option is to use a database that provides the power of SQL (if using a relational database) to search into your database. In order to use a database, you should learn about databases, reading from and writing to databases, and making a connection to the database in java.
In my opinion,
You should begin with the database approach as you can easily query on a date to get all the events present on the given date. Since, you not only want to store the events but you also want to go to a particular date and list out the events planned on that date. So, you need to store your data in such a way in which it is easy to search and read for you.
Also, I advise you to use the Spring framework and Maven which can take care of all the dependency, database connection with minimal configuration.
You may use h2 database, it is SQLite version and easy to use. Use file-based database, you need not use a server as of now.
Edit
Also as suggested by #springe, you can use any ORM like Hibernate to deal with the database which is a secure and recommended way used even in industrial code. Basically it is good practice to use JPA/ Hibernate when performing CRUD operations.
However, since you are new to programming and stuff, get mastery over plain SQL as well as learn good practices like using ORM.
For references
You can refer Baeldung for references, just google how to do this and that in java Bealdung and you will get pretty cool and short guide how to do it.
You will get spring configuration to connect to h2database, maven dependency to for Spring, and database there at Baeldung. Everything is standard and you just need to copy-paste while also learning how things work.
Keep learning, I loved your spirit. :)
I'm starting to work on a new Java desktop app that should help me and my colleagues learn vocabulary. It will contain around 700 words, some texts (that point to the words contained in them) and maybe some images (not sure about that part yet). The data will never change and I want the program to be able to run offline.
The question is: Should I use database, text file or serialize the data into file? Or perhaps if there is any other option I don't know about? If you could explain your choice in detail I would be glad.
If the data never changes and is only 700 words it would probably be easiest to use a file.
If your data was a bit more complex and had many fields and was being constantly updated, a database would be more preferable but a csv file could still be used.
Since you want to access this data offline and data never changes, I think the best option would be to just use text file, which will be more efficient in terms of access and speed.
Keep all the data in memory as Serializable Java objects, and store them serialized when your application is not running. Evaluate airomem - really nice solution that would perfectly work for you.
I am relatively new to Java and have much more experience with Matlab. I was wondering what the best way is to store a relatively small amount of data, which has been calculated in one program, that should be used in another program.
Example: program A computes 100 values to be stored in an array. Now I would like to access this array in program B, as it needs these values. Of course, I could just write one program all together, which also implements the part of A. However, now every time I want to execute the total program, all the values have to be calculated again (in part A), which is a waste of resources. In Matlab, I was able to easily save the array in a .mat file and load it in a different script.
Looking around to find my answer I found the option of serializing (What is object serialization? ), which I think would be a suitable for doing what I want. My question: is serializing the easiest and quickest solution to store a small amount of data in Java, or is there a quicker, more user-friendly option (like .mat files in Matlab)?
I think you have several options to do this job. Java object serialization is one possible way. From my point of view there are other options to serialize the data:
Write and read a simple text file to store the computed values.
Using Java Architecture for XML Binding (JAXB) to write annotated Java classes to XML file. Same for JSON is also available.
Using a lightweight database like SQLite or HSQLDB (native Java database).
Using Apache Thrift or Protocol Buffer to de/serializing Java objects to files.
I want to store my blobs outside of the database in files, however they are just random blobs of data and aren't directly linked to a file.
So for example I have a table called Data with the following columns:
id
name
comments
...
I can't just include a column called fileLink or something like that because the blob is just raw data. I do however want to store it outside of the database. I would love to create a file called 3.dat where 3 is the id number for that row entry. The only thing with this setup is that the main folder will quickly start to have a large number of files as the id is a flat folder structure and there will be OS file issues. And no the data is not grouped or structured, it's one massive list.
Is there a Java framework or library that will allow me to store and manage the blobs so that I can just do something like MyBlobAPI.saveBlob(id, data); and then do MyBlobAPI.getBlob(id) and so on? In other words something where all the File IO is handled for me?
Simply use an appropriate database which implements blobs as you described, and use JDBC. You really are not looking for another API but a specific implementation. It's up to the DB to take care of effective storing of blobs.
I think a home rolled solution will include something like a fileLink column in your table and your api will create files on the first save and then write that file on update.
I don't know of any code base that will do this for you. There are a bunch that provide an in memory file system for java. But it's only a few lines of code to write something that writes and reads java objects to a file.
You'll have to handle any file system limitations yourself. Though I doubt you'll ever burn through the limitations of modern file systems like btrfs or zfs. FAT32 is limited to 65K files per directory. But even last generation file systems support something on the order of 4 billion files per directory.
So by all means, write a class with two functions. One to serialize an object to a file; given it a unique key as a name. And another to deserialize the object by that key. If you are using a modern file system, you'll never run out of resources.
As far as I can tell there is no framework for this. The closest I could find was Hadoop's HDFS.
That being said the advice of just putting the BLOB's into the database as per the answers below is not always advisable. Sometimes it's good and sometimes it's not, it really depends on your situation. Here are a few links to such discussions:
Storing Images in DB - Yea or Nay?
https://softwareengineering.stackexchange.com/questions/150669/is-it-a-bad-practice-to-store-large-files-10-mb-in-a-database
I did find some addition really good links but I can't remember them offhand. There was one in particular on StackOverFlow but I can't find it. If you believe you know the link please add it in the comments so that I can confirm it's the right one.
In my program, I am reading a series of text files from the disk. With each text file, I process out some data and store the results as JSON on the disk. In this design, each file has its own JSON file. In addition to this, I also store some of the data in a separate JSON file, which stores relevant data from multiple files. My problem is that the shared JSON grows larger and larger with every file parsed, and eventually uses too much memory. I am on a 32-bit machine and have 4 GB of RAM, and cannot increase the memory size of the Java VM anymore.
Another constraint to consider is that I often refer back to the old JSON. For instance, say I pull out ObjX from FileY. In pseudo code, the following happens (using Jackson for JSON serialization/deserialization):
// In the main method.
FileYJSON = parse(FileY);
ObjX = FileYJSON.get(some_key);
sharedJSON.add(ObjX);
// In sharedJSON object
List objList;
function add(obj)
if (!objList.contains(obj))
objList.add(obj);
The only thing I can think to do is use streaming JSON, but the problem is that I frequently need to access the JSON that came before, so I don't know that stream will work. Also my data types on not only strings, which prevents me from using Jackson's streaming capabilities (I believes). Does anyone know of a good solution?
If you're getting to the point where your data structures are so large that you're running out of memory, you'll have to start using something else. I would recommend that you use a database, which will significantly speed up data retrieval and storage. It will also make the limit of your data structure the size of your hard drive, instead of the size of your RAM.
Try this page for an introduction to Java and Databases.
I can't believe that you really need nearly 4GB RAM only for text files and JSON.
I see three possible solutions.
Switch to plain text if it's possible. That is not that memory hungry.
Just open and close the files as you need them. You can order the files to a specific naming convention, like the first two/three/... digits of their hashes, and open them as you need them.
If you have so many data, you could maybe switch to a database. That would save a lot of resources.
I would prefer option 3 if it's possible for you.
you can make api and get responce.body from it