I am new to this SQLite. So long being used Oracle / SQL which are maintained centrally so DBA manages all these
I am planning to use SQLite DB in one of our Java/JSP application.
The data will be written and read to this DB
I store this SQLite DB file in the same Server as the application itself. There could be a possibility that the DB file getting deleted (for what ever reasons)
I am wondering what backup and restore strategy we could apply here in order to backup the DB incrementally and also restore in worst case.
Simply copying the file (Batch file to copy file from one location to another) every now and then won't work as the DB file may be used
How big files are you talking about?
The locking issue could possibly be solved by using LLVM snapshot like described here:
http://tldp.org/HOWTO/LVM-HOWTO/snapshots_backup.html
With normal databases, like MariaDB you can do like this:
Flush data and lock writes
Take LLVM snapshot
Release locks
Mount snapshot somewhere and make backup with tar, rsync, tarsnap, etc
Then again, to this be usable you probably need to lock the SQLite DB file somehow when creating the snapshot.
If you are sure that the database is not currently being written to, you can simply copy the file(s).
If there might be concurrent accesses, you must read the database file from within a database transaction.
SQLite has a backup API for this; the simplest way to use it is to run the sqlite3 command-line shell with the .backup command.
There is no mechanism to make incremental backups.
Related
I want to work very portable, so I don't install a monster of Sql Server which has 2.7Gb size on my computer. I want to find an easier solution. I have for example a form application in Java which must to access a database (maybe a server), I thought it's easier for me to work with a portable database (Access or SqlLite).
It can be an installer solution but not very big such as 1Gb size or more. Initially thought to use a text file or Excel for keep the data but this way is hard too because I can't easily simulate the constraints and relationships between lines and tables.
Of course it is possible to access an SQLite database from Java! As you can read here:
A database in SQLite is a single disk file¹.
So you just need to use the driver to read/write from/to this database (file).
The code in Java would look like the following (taken from here):
How to Specify Database Files
Here is an example to select a file C:\work\mydatabase.db (in Windows)
Connection connection = DriverManager.getConnection("jdbc:sqlite:C:/work/mydatabase.db");
A UNIX (Linux, Mac OS X, etc) file /home/leo/work/mydatabase.db
Connection connection = DriverManager.getConnection("jdbc:sqlite:/home/leo/work/mydatabase.db");
I am trying to do an eclipse java android app using internal storage.
It is a notepad. What it should be able to do is write and save notes, search for them an edit them.
the most similar application of what I need is this:
http://developer.android.com/training/notepad/notepad-ex3.html
but I dont know how to modify if to use internal storage and not external, i also found this file that allows you to read and save in internal storage
http://www.androidaspect.com/2013/09/android-internal-storage-tutorial.html
but now i dont know how change the database for the inernal storage
Okay let's break it down. You want to make something that will let you take notes. Fair enough. You need the kind of storage that will make retrieval and insertion easier.
Let's start with a single text file that you store somewhere (internal or external). If you use this approach, inserting and deleting data becomes difficult because any changes will require you to make changes to the actual text. Not to mention, you will need to format your notes in a way so that you can differentiate one note from another.
Okay, so will using multiple text files solve this? In a way, yes, but you can go for a better approach using SQLite.
SQLiteDatabase has methods to create, delete, execute SQL commands,
and perform other common database management tasks.
SQLite seems like the prime candidate for your storage because it makes CRUD (Create, Read, Update, Delete) easier. Instead of trying to modify the code, start from scratch.
Have a look at this tutorial https://www.youtube.com/watch?v=j-IV87qQ00M
in my java application I'm maintaining a users database and I have to store user account picture together with other details. i can store the photo in my database or I can store the image in my file system and store relevant path in the database. which one is more suitable in terms of memory and running time ?
Reference taken from this article:
You should take decision based on your system complexity and scalability. I suggest you to go with File System and Database Storage is not allowable, but still it is advisable. Management is always easy with database storage, but will create a performance issue in some circumstances. Management is quite complex with file system, but you will get more performance.
You should store file on file system. You can read about this on programmers stackexchange:
https://softwareengineering.stackexchange.com/a/150787
File system is obviously faster if you have the option.
You can use varbinary if you have to insert it to your db.
I am in the process of refreshing/re-writing a data analysis application I wrote a couple of years back. What I am trying to achieve is as follows:
I need to have a light-weight database to do queries to, I decided on using HSQLDB.
I will have 2 applications; one for creating the DB and one that will do the analysis (will be used by others). It is my intention that the analysis software (which is multithreaded) will use the DB in a read-only fashion
The DB will likely be distributed via FTP, and preferably with minimum hassle for the user of the analysis software (most of which are not very technically skilled).
I am not very well-read on SQL but I managed to get the info into tables and tried out simple queries. So in order to finish the "database creator" application I just need to figure out how to "package" the DB.
I have experimented both with mem and file "catalogs" as described in the HSQLDB user guide when I generate the DB. The way I see it, with mem catalogs I cannot write them to disk (to distribute later on) and with file catalogs I have several files that need to be taken care of:
A file: catalog consists of between 2 to 6 files, all named the same
but with different extensions, located in the same directory. For
example, the database named "test" consists of the following files:
• test.properties
• test.script
• test.log
• test.data
• test.backup
• test.lobs
The properties file contains a few settings about the database. The script file contains the definition of tables and other database objects, plus the data for non-cached tables. The log file contains
recent changes to the database. The data file contains the data for
cached tables and the backup file is a compressed backup of the last
known consistent state
of the data file. All these files are essential and should never be deleted. For some catalogs, the test.data and test.backup files
will not be present. In addition to those files, a HyperSQL database
may link to any formatted text files, such as CSV lists, anywhere on
the disk.
Question(s):
I think the *.script and *.properties files are the most important ones but the guide specifically says that all the files are essential and should not be deleted. Since there is no *.data file in my case, and all the data to generate my database is stored in *.script file (in clear text) it makes me think that when I "open" that file, the JVM recreates the entire DB all over again. Is this correct? Isn't this a very inefficient representation of data?
If my understanding in (1) is correct, why are the other files essential? Do I have to distribute them all?
If (1) and (2) are not off-track, then what options do I have to achieve my goal? Is it for instance feasible to gzip all the files and transport them that way? Then my analysis software would need to unpack them in a reasonable spot, and do a clean-up from time to time when it gets "crowded" in there...
If there is no *.data or *.lobs file then the .script file contains all the data as well as table definitions. In this case, the JVM recreates the database by reading the .script file.
The other files are essential if they exist. If you do not use LOBs, there will be no .lobs file. If you do not disk tables (CACHED tables), there will be no .data file.
You can distribute the .script file only. Each time this file is opened, a .properties file will be created if it does not exist.
You can use the res: option if your database does not change. For databases that change and need reloading use the file: option.
I cannot understand how distributing Java programs that use a database works.
Let's say I am using Derby as RDBMS and I want to store tasks and calendar entries in a database.
I want each user of the program to have a local database.
But I don't understand how in-memory databases are supposed to work. Should I write a script so that the first time my program is launched it creates a database and empty tables? Or will they be already created during the installation of the program?
If your program wants to store the user's tasks and calendar entries in a database, you probably don't want to use an in-memory database, because the in-memory database disappears when your program exits.
Rather, you want to use an ordinary persistent Derby database, which will store the user's data in files in a folder on the filesystem.
You do indeed have to create the database and issue the CREATE TABLE etc. statements to create the tables in that database. You could provide that as a separate script, or you could have your program issue those statements itself.
Tables are not automatically created, though; you have to issue the CREATE TABLE statements one way or another.