I'm in the early stages of a note-taking application for android and I'm hoping that somebody can point me to a nice solution for storing the note data.
Ideally, I'm looking to have a solution where:
Each note document is a separate file (for dropbox syncing)
A note can be composed of multiple pages
Note pages can have binary data (such as images)
A single page can be loaded without having to parse the entire document into memory
Thread-safety: Multiple reads/writes can occur at the same time.
XML is out (at least for the entire file), since I don't have a good way to extract a single page at a time. I considered using zip files, but (especially when compressed) I think they'd be stuck loading the entire file as well.
It seems like there should be a Java library out there that does this, but my google-fu is failing me. The only other alternative I can think of is to make a separate sqlite database for every note.
Does anybody know of a good solution to this problem? Thanks!
Seems like a relational database would work here. You just need to play around with the schema a little.
Maybe make a Pages table with each page including, say, a field for the document it belongs to and a field for its order in the document. Pages could also have a field for binary data, which might be contained in another table. If the document itself has additional data, maybe you have a table for documents too.
I haven't used SQLite transactions on an Android device, but it seems like that would be a good way to address thread safety.
I would recommend using SQLite to store the documents. Ultimately, it'll be easier than trying to deal with file I/O every time you access the note. Then, when somebody wants to upload to dropbox, you generate the file on the fly and upload it. It would make sense to have a Notes table and a pages table, at least. That way you can load each page individually and a note is just a collection of pages anyway. Additionally, you can store images as BLOBS in the database for a particular page. Basically, if you only want one type of content per page, then you would have, in the pages table, something like an id column and a content column. Alternatively, if you wanted to support something that is more complex such as multiple types of content then you would need to make your pages a collection of something else, like "entities."
IMO, a relational database is going to be the easiest way to accomplish your requirement of reading from particular pages without having to load the entire file.
Related
I'm starting to work on a new Java desktop app that should help me and my colleagues learn vocabulary. It will contain around 700 words, some texts (that point to the words contained in them) and maybe some images (not sure about that part yet). The data will never change and I want the program to be able to run offline.
The question is: Should I use database, text file or serialize the data into file? Or perhaps if there is any other option I don't know about? If you could explain your choice in detail I would be glad.
If the data never changes and is only 700 words it would probably be easiest to use a file.
If your data was a bit more complex and had many fields and was being constantly updated, a database would be more preferable but a csv file could still be used.
Since you want to access this data offline and data never changes, I think the best option would be to just use text file, which will be more efficient in terms of access and speed.
Keep all the data in memory as Serializable Java objects, and store them serialized when your application is not running. Evaluate airomem - really nice solution that would perfectly work for you.
I want to store my blobs outside of the database in files, however they are just random blobs of data and aren't directly linked to a file.
So for example I have a table called Data with the following columns:
id
name
comments
...
I can't just include a column called fileLink or something like that because the blob is just raw data. I do however want to store it outside of the database. I would love to create a file called 3.dat where 3 is the id number for that row entry. The only thing with this setup is that the main folder will quickly start to have a large number of files as the id is a flat folder structure and there will be OS file issues. And no the data is not grouped or structured, it's one massive list.
Is there a Java framework or library that will allow me to store and manage the blobs so that I can just do something like MyBlobAPI.saveBlob(id, data); and then do MyBlobAPI.getBlob(id) and so on? In other words something where all the File IO is handled for me?
Simply use an appropriate database which implements blobs as you described, and use JDBC. You really are not looking for another API but a specific implementation. It's up to the DB to take care of effective storing of blobs.
I think a home rolled solution will include something like a fileLink column in your table and your api will create files on the first save and then write that file on update.
I don't know of any code base that will do this for you. There are a bunch that provide an in memory file system for java. But it's only a few lines of code to write something that writes and reads java objects to a file.
You'll have to handle any file system limitations yourself. Though I doubt you'll ever burn through the limitations of modern file systems like btrfs or zfs. FAT32 is limited to 65K files per directory. But even last generation file systems support something on the order of 4 billion files per directory.
So by all means, write a class with two functions. One to serialize an object to a file; given it a unique key as a name. And another to deserialize the object by that key. If you are using a modern file system, you'll never run out of resources.
As far as I can tell there is no framework for this. The closest I could find was Hadoop's HDFS.
That being said the advice of just putting the BLOB's into the database as per the answers below is not always advisable. Sometimes it's good and sometimes it's not, it really depends on your situation. Here are a few links to such discussions:
Storing Images in DB - Yea or Nay?
https://softwareengineering.stackexchange.com/questions/150669/is-it-a-bad-practice-to-store-large-files-10-mb-in-a-database
I did find some addition really good links but I can't remember them offhand. There was one in particular on StackOverFlow but I can't find it. If you believe you know the link please add it in the comments so that I can confirm it's the right one.
We are going to use iText to create a large report. There will be a lot of static text and images that will be the same for every report. We will then insert dynamic data from a database into the report. The static data and the dynamic data will be mixed together when the report is finished. What is the best way to handle the static data. We are going to use a java servlet in a web application for the creation of the report. We want the report to be sent to the web page immediately, so performance is critical.
These are some of the ideas I had. Not sure if any of these are good ideas.
1.) Create a PDF with the static content then insert the dynamic data. If I go this way how would I know where to insert the dynamic data? Is it possible to bookmark place to insert data?
2.) Get the static data from a database. This seems like it would be difficult trying to design a database with static content that would have to consider things like pages, paragraphs, headings, images. I would also think that this would not be great for performance if the website gets hit hard.
3.) Cache the static content in the servlet context. This would seem to help performance but would still have the same design issues as the database.
I would love some opinions on the best way to store large amounts of static text when creatig a PDF using iText.
Thanks for your help
Doug
I am not sure if the best way to do it, but this worked well for me in a similar situation, I created a template file that I read into code, and in the template I had hash tagged variable holders that I used to find a location and replace with dynamic text from whatever source. For example:
Hello #CUSTOMER_NAME#, Thank you for the purchase of #RECENT_PURCHASES# on
#RECEIPT_DATE#, for a total value of #RECEIPT_TOTAL#. ...
Like I said, it probably isn't ideal, but it really well for my needs.
This is more of an overall architecture question about Android, and I'm curious what the community thinks is best practice for this type of endeavor. I am developing an Android application which loads an xml file which is stored on the device. My first question is, when you are dealing with a formatted xml file in the scope of an Android application, and the main point of the application is to sort of "browse" through the nodes of the xml, is it smarter to "load the xml" (not really sure what the term is) into memory and do it that way? Or is it smarter to take the xml, write it to an internal database (still getting used to the whole SQL Lite concept), and then browse through the data that way? The latter seems like a roundabout way, but I'm trying to understand core concepts here.
This brings me to my second question. If I were to draw out how the data from this XML "flows", the immediate answer in my head as far as what I know about Android is, a bunch of ListViews. Node 1 has 2 choices. This loads two choices into a ListView. When you click on the first node, it goes to the corresponding subnode in the xml, which has, say, four choices. I create a ListView with 4 choices. So on and so forth.
Does this make logical sense? Am I looking at the approach wrong? Is there a better way to do it using a different object that makes more sense? Any references to things that have already been done for me to compare to would be helpful as well. Thanks!!
Don't convert the XML into a sqlite database. Just parse it in memory.
As far as your other questions, I'd have one activity that extends from ListActivity. Override onListItemClick() and make it start your activity again with some kind of pointer to the next element to browse.
Doing it this way will make the activity stack behave well as the user presses the back button.
A lot depends on the specific use case you have and the size of XML file. For most parts, I think you will have a heck of hard time placing your XML in the database unless you already have data model that is represented by XML and suitable for persistence. You surely don't want to do it with random XML.
If you have small XML you can always load it in memory using DOM. That will make it easy to navigate. But with large XML, you need to consider some streaming API (Stax) and read directly from file.
SQLite is relational database, so you need to store data from xml in DB only if you need to perform relational operation with data (e.g. selection/update/grouping so on). If you need just to go through DOM and do something (e.g. count specific nodes), I believe you should not parse xml to DB.
I am making a java program that has a collection of flash-card like objects. I store the objects in a jtree composed of defaultmutabletreenodes. Each node has a user object attached to it with has a few string/native data type parameters. However, i also want each of these objects to have an image (typical formats, jpg, png etc).
I would like to be able to store all of this information, including the images and the tree data to the disk in a single file so the file can be transferred between users and the entire tree, including the images and parameters for each object, can be reconstructed.
I had not approached a problem like this before so I was not sure what the best practices were. I found XLMEncoder (http://java.sun.com/j2se/1.4.2/docs/api/java/beans/XMLEncoder.html) to be a very effective way of storing my tree and the native data type information. However I couldn't figure out how to save the image data itself inside of the XML file, and I'm not sure it is possible since the data is binary (so restricted characters would be invalid). My next thought was to associate a hash string instead of an image within each user object, and then gzip together all of the images, with the hash strings as the names and the XMLencoded tree in the same compmressed file. That seemed really contrived though.
Does anyone know a good approach for this type of issue?
THanks!
Thanks!
Assuming this isn't just a serializable graph, consider bundling the files together in Jar format. If you already have your data structures working with XMLEncoder, you can reuse this code by saving the data as a jar entry.
If memory serves, the jar library has better support for Unicode name entries than the zip package, which is why I would favour it.
You might consider using an MS JET database (.mdb file) and storing all the stuff in there. That'll also make it easy to examine and edit the data in (for example) MS Access.
You can employ some virtual file system, which stores it's data in a single container. We develop and offer one of such files sytems, SolFS, however right now there's no Java binding for it. We will release Java JNI interface for SolFS within a month.