We Keep Coding

SQLite memory issue with singleton approach

I have a SQLite database powering all the content in an Android app.
I have a DatabaseHelper class that extends SQLiteAssetHelper.
I have been having problems with too many instances of my database and then getting a SQLiteCantOpenDatabaseException.
To counter this, I have changed my class to maintain a single instance of the DatabaseHelper object.
I have the following:
private static DatabaseHelper databaseHelper;
public static synchronized DatabaseHelper getInstance(Context context, boolean singleRow, boolean showLoader){
if(databaseHelper == null) {
databaseHelper = new DatabaseHelper(context, singleRow, showLoader);
}
return databaseHelper;
}
public DatabaseHelper(Context context, boolean singleRow, boolean showLoader){
super(context, (new File(DatabaseManager.getDatabasePath(context))).getName(), (new File(DatabaseManager.getDatabasePath(context))).getParentFile().getAbsolutePath(), null, DATABASE_VERSION);
this.context = context;
this.singleRow = singleRow;
this.showLoader = showLoader;
}
I then call the getInstance static method as follows:
DatabaseHelper databaseHelper = DatabaseHelper.getInstance(activity.getApplicationContext(), false, false);
After a certain amount of database activity, the app is still crashing on memory grounds.
I then get this error:
Error Code : 2062 (SQLITE_CANTOPEN_EMFILE)
Caused By : Application has opened two many files. Maximum of available file descriptors in one process is 1024 in default.
(unable to open database file (code 2062))
Having taken the singleton approach, I'm a bit lost for why this is still causing this memory leak.
Any help would be appreciated.

If you are getting a message indicating too many files open, a cause may well be that there are too many Cursor that are still open.
However, the message returned may not always be the same and is probably specific to the task/call being called.
In this case the message was (unable to open database file (code 2062)), yet in another case (from a SELECT the message was unable to open database file (code 14)). SQLite unable to open database file (code 14) on frequent “SELECT” query.
The above link also points to a post I made what quite clearly shows that creating a Cursor results in a file (or files) being opened.
The example was looping through about 500 rows and for each row it was creating/recreating 3 cursors for each row (so potentially 1500+ cursors even though only using 4 cursor objects).
Initially it was only closing the 3 cursors at the end (last row of the parent of all) resulting in the unable to open database File (code 14). Closing the 3 cursors for each iteration resolved the issue.
The code that failed was :-
SQLiteDatabase db = getWritableDatabase();
Cursor shoplistcursor = getAllRowsFromTable(SHOPLIST_TABLE_NAME);
Cursor productcsr;
Cursor aislecsr;
Cursor prdusecsr;
while(shoplistcursor.moveToNext()) {
productcsr = getProductFromProductId(shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_PRODUCTREF)));
aislecsr = getAisleFromAisleId(shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_AISLEREF)));
prdusecsr = getProductUsage(shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_AISLEREF)),
shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_PRODUCTREF)));
if (productcsr.getCount() < 1 | aislecsr.getCount() < 1 | prdusecsr.getCount() < 1) {
deleteShopListEntry(shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_ID)));
}
if(shoplistcursor.isLast()) {
prdusecsr.close();
aislecsr.close();
productcsr.close();
}
}
shoplistcursor.close();
db.close();
}
Whilst the fixed code was :-
SQLiteDatabase db = getWritableDatabase();
Cursor shoplistcursor = getAllRowsFromTable(SHOPLIST_TABLE_NAME);
Cursor productcsr;
Cursor aislecsr;
Cursor prdusecsr;
while(shoplistcursor.moveToNext()) {
productcsr = getProductFromProductId(shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_PRODUCTREF)));
aislecsr = getAisleFromAisleId(shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_AISLEREF)));
prdusecsr = getProductUsage(shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_AISLEREF)),
shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_PRODUCTREF)));
if (productcsr.getCount() < 1 | aislecsr.getCount() < 1 | prdusecsr.getCount() < 1) {
productcsr.close();
aislecsr.close();
prdusecsr.close();
deleteShopListEntry(shoplistcursor.getLong(shoplistcursor.getColumnIndex(SHOPLIST_COLUMN_ID)));
} else {
productcsr.close();
aislecsr.close();
prdusecsr.close();
}
}
shoplistcursor.close();
db.close();
}
I tend to now follow the following rule/practice :-
If just getting the result e.g. getting the number of rows, close the Cursor in the method.
If using the Cursor for a display e.g. a ListView, then close the cursor in the activity's onDestroy method.
If using the Cursor for what I'll call more complex processing e.g. deleting rows with underlying references then close the cursors as soon as they are done with, within the processing loop(s).

I really don't know why and really got curious about "Application has opened two many files" error and would like to know what causes it.
However, i use singleton with database without any issues over a year. I used this snippet to get database with singleton in 14 apps and never had any problems.
public class DatabaseManager {
private AtomicInteger mOpenCounter = new AtomicInteger();
private static DatabaseManager instance;
private static SQLiteOpenHelper mDatabaseHelper;
private SQLiteDatabase mDatabase;
private DatabaseManager() {
}
public static synchronized DatabaseManager getDatabaseManager(SQLiteOpenHelper helper) {
if (instance == null) {
instance = new DatabaseManager();
mDatabaseHelper = helper;
}
return instance;
}
public static synchronized DatabaseManager getDatabaseManager(Context context) {
if (instance == null) {
instance = new DatabaseManager();
mDatabaseHelper = new DatabaseOpenHelper(context.getApplicationContext());
}
return instance;
}
/**
* Get a writable database
*/
public synchronized SQLiteDatabase openDatabase() {
if (mOpenCounter.incrementAndGet() == 1) {
// Opening new database
mDatabase = mDatabaseHelper.getWritableDatabase();
// System.out.println("DataBaseManager: Database Opened");
} else {
// System.out.println("DataBaseManager: Database Already Open");
}
return mDatabase;
}
public synchronized void closeDatabase() {
if (mOpenCounter.decrementAndGet() == 0) {
// Closing database
mDatabase.close();
// System.out.println("DataBaseManager: Database Closed");
} else {
// System.out.println("DataBaseManager: Database is NOT Closed");
}
}
}
onCreate() i get instance with mDatabaseManager = DatabaseManager.getDatabaseManager(getActivity().getApplicationContext());, onStart() i open database with mDatabaseManager.openDatabase();, and onStop() close it with mDatabaseManager.closeDatabase();

I do the singleton approach as well. There are two ways to access your Data obviously.
You can use a Cursor to populate a list of objects and then close your cursor and then close your DB.
Unless you are returning the cursor for dynamic paging of content because of larger list.
Whether or not to close your connection is contingent upon frequency of usage and your app's specific needs.
However, if you are accessing from new context and sharing the previously created SQLHelper class you could be creating memory leak issues as the constructor requires the context.
It sounds to me like you have too many files open on that single connection though. Have you considered closing your db connection after each interaction. Example:
public static ArrayList<OrderModel> getOrders(Context context){
ArrayList<OrderModel> orderList = new ArrayList<OrderModel>();
SQLiteDatabase db = null;
try{
db = A35DBHelper.openDatabase(context);
String columns[] = {
"*"
};
Cursor cursor = db.query(OrdersTable.TABLE_NAME, columns, OrdersTable.COLUMN_PRIMARY_ID, null, null, null, null);
if (cursor != null) {
for(cursor.moveToFirst(); !cursor.isAfterLast(); cursor.moveToNext()){
OrderModel order = new OrderModel();
order.setLocalDatabaseId(cursor.getLong(cursor.getColumnIndex(OrdersTable.COLUMN_PRIMARY_ID)));
order.setID(cursor.getString(cursor.getColumnIndex(OrdersTable.COLUMN_REPAIR_ORDER_NUMBER)));
order.setOrderNumber(cursor.getString(cursor.getColumnIndex(OrdersTable.COLUMN_ORDER_NUMBER)));
order.setCreatedAtDate(cursor.getString(cursor.getColumnIndex(OrdersTable.COLUMN_CREATED_AT_DATE)));
order.setImageCount(MediaDataContext.getAllMediaForOrderId(context, order.getID()).size());
order.setDefaultThumbnailUrl(cursor.getString(cursor.getColumnIndex(OrdersTable.COLUMN_DEFAULT_THUMBNAIL_URL)));
orderList.add(order);
}
cursor.close();
}
} catch (Exception ex) {
A35Log.e(TAG, "Failed to get orders: " + ex.getMessage());
}
A35DBHelper.closeDatabase(db);
return orderList;
}
Then my singleton class has the open and close where if the context has changed I new up a new instance of the helper before opening.
Then I use the CloseUtil for try/catch closing each time.
This can still be the same if you are returning the Cursor Object instead of an ArrayList as you are possibly getting dynamic data that handles paging or is to big to fill a list.
But sounds like to me your connection is getting over worked so you may need to revisit your model.

SQLite database recreated on application restart

I am creating an android based Appointment management system. For storing appointments I'm using an SQLite database. The application works flawlessly but the only problem is that the Database gets recreated every time I restart the application it self. Following is my onCreate and onUpgrade methods. (PS. I used a video tutorial on creating and connecting the database. This worked fine on him. Only difference is I'm using a Mac and he was using Windows)
public DatabaseHelper(Context context) {
super(context, DATABASE_NAME, null, 1);
}
#Override
public void onCreate(SQLiteDatabase db) {
db.execSQL("create table " + TABLE_NAME + " (ID INTEGER PRIMARY KEY AUTOINCREMENT, DATE TEXT, TIME TEXT, TITLE TEXT, DETAIL TEXT, UNIQUE(DATE, TITLE))");
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
db.execSQL("DROP TABLE IF EXITS" + TABLE_NAME);
onCreate(db);
}
Just in case I have uploaded my full database connection code here, Database.java

Hard to tell without seeing the entire class, but you may be inadvertently deleting the database somewhere? Look for calls to deleteDatabase that may be buried somewhere, particularly if you based your code of a tutorial that may have left bits and pieces of code in place.

A SQLiteConnection object for database was leaked! Please fix your application

My application give me this warning
A SQLiteConnection object for database
'+data+data+com_example_test+database' was leaked! Please fix
your application to end transactions in progress properly and to close
the database when it is no longer needed.
But I close the db object and the cursor after every use.
try {
while (cursor.moveToNext()) {
...
}
} finally {
if (cursor != null && !cursor.isClosed())
cursor.close();
}
...
db.close();
Can you help me for understand what is the problem?
thanks!!!
UPDATE!
I try this solution from this post
SQLite Connection leaked although everything closed
and I don't have memory leak anymore, is it a good solution?

Possible Solutions:
You have not committed the transactions you have started (You should
always close the transaction once you started)
Check whether you have closed the cursors you have opened if you are
using Sqlite (Looks like you have done this step from the code you posted)
Also move the db.close to finally block
You have not called db.close on a database before deleting it with context.deleteDatabase(...) and then recreating it with dbHelper.getWritableDatabase()

Just drag that db.close up into the finally block.

//Inside your SQLite helper class
#Override
public synchronized void close () {
if (db != null) {
db.close();
super.close();
}
}
//Inside the activity that makes a connection to the helper class
#Override
protected void onDestroy () {
super.onDestroy();
//call close() of the helper class
dbHelper.close();
}

this code stops the leak and fixes cursor problems.
public class DatabaseHelper extends SQLiteOpenHelper {
private static DatabaseHelper sInstance;
private static final String DATABASE_NAME = "database_name";
private static final String DATABASE_TABLE = "table_name";
private static final int DATABASE_VERSION = 1;
public static DatabaseHelper getInstance(Context context) {
// Use the application context, which will ensure that you
// don't accidentally leak an Activity's context.
if (sInstance == null) {
sInstance = new DatabaseHelper(context.getApplicationContext());
}
return sInstance;
}
/**
* Constructor should be private to prevent direct instantiation.
* make call to static factory method "getInstance()" instead.
*/
private DatabaseHelper(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
}
}

In my case the error was caused when y try to download new data and database should be updated.
I solved it instantiating the database by calling a SELECT 0. That cause database to be updated, so after that I try to download the new data. And worked fine.

Probably you forgot to remove the break point of debugging
sample:

In my case, I was calling to getWritableDatabase or getReadableDatabase and not use it at all. for example if you use it with "execSQL" execSQL will call "releaseReference" "Releases a reference to the object, closing the object if the last reference was released."

Android SQLite onUpgrade not called

I have my database created in event onCreate, in which I have a lot of tables, but I need add 1 more table, and I can't lose any data, So I need to use the event onUpgrade, So I hope you guys help me because I don't know how to use it.
Example :
public void onCreate(SQLiteDatabase db) {
sql = "CREATE TABLE IF NOT EXISTS funcionarios"
+"(codigo INTEGER PRIMARY KEY, funcionario TEXT, apelido TEXT , functionTEXT, cartao TEXT , foto TEXT , tipo_foto TEXT);";
db.execSQL(sql);
}
what i need is
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
if(oldVersion < 2){
db.execSQL("CREATE TABLE IF NOT EXISTS calibrar_aceleracao"+
"(limiteMaximo INTEGER, limiteMinimo INTEGER);");
}
}
but it doesn't work.
thanks.

You do not need to change you applications version to update your database - not saying it is incorrect but there is a more efficient way of doing it. And that is through the use of the super's constructor of your helper it would look something like the following:
public class MyDatabaseHelper extends SQLiteOpenHelper {
public MyDatabaseHelper(Context context) {
super(context, "My.db", null, 1 /* This is the version of the database*/);
}
#Override
public void onCreate(SQLiteDatabase database) {
sql = "CREATE TABLE IF NOT EXISTS funcionarios (codigo INTEGER PRIMARY KEY, funcionario TEXT, apelido TEXT , functionTEXT, cartao TEXT , foto TEXT , tipo_foto TEXT);";
db.execSQL(sql);
}
#Override
public void onUpgrade(SQLiteDatabase database, int oldVersion, int newVersion) {
if(oldVersion < 2){
db.execSQL("CREATE TABLE IF NOT EXISTS calibrar_aceleracao (limiteMaximo INTEGER, limiteMinimo INTEGER);");
}
}
}

The method onUpgrade is called when your version database is incremented. Verify in your class where you define your database version and increment this value.
Run application. Your method onUpgrade is called.

For onUpgrade to get called you must increase the database version that you supply to the SqliteOpenHelper implementation constructor.
Use a field in your class to store the same and increment it when you change your database schema.

This is not the way onUpgrade works.This is a method which will be called when you release some new version of your application and make it available for download in google play(and which may be requiring some updations to the database of the application already installed on users' devices').For your problem's solution
Just add the query of CREATE TABLE IF NOT EXISTS in your onCreate() as you did for the creation of the other table in your onCreate() method already
public void onCreate(SQLiteDatabase db) {
sql = "CREATE TABLE IF NOT EXISTS funcionarios"
+"(codigo INTEGER PRIMARY KEY, funcionario TEXT, apelido TEXT , functionTEXT, cartao TEXT , foto TEXT , tipo_foto TEXT);";
///HERE YOUR Create Table QUERY and call db.execSQL
db.execSQL(sql);
}

The method is onUpgrade is not being called probably because there are errors in the sql code, for example in the onCreate:
functionTEXT
is missing a space before TEXT.
Also after,
calibrar_aceleracao"+ "(limiteMaximo
is missing a space before the bracket.
I had the same problem, I was caching the SQLiteException so I didn't see the error was there.
Put some logcat at the beginning and at the end of the method body and you'll see where the error is.
EDIT: another thing, why did you put that if?
if (oldVersion < 2)
It's not necessary at all.

Ormlite setup without using base activities

I'm using ORMLite in an android project, and I'm not wanting to use the extended activities because I'm inserting values into the database on an AsyncTask.
In the docs it says:
"If you do not want to extend the OrmLiteBaseActivity and other base classes then you will need to duplicate their functionality. You will need to call OpenHelperManager.getHelper(Context context, Class openHelperClass) at the start of your code, save the helper and use it as much as you want, and then call OpenHelperManager.release() when you are done with it."
It also says to add the database helper class in the strings.xml, which I have. So I'm not sure what I'm doing wrong.
I'm using a class called DataAccess for my data tier that looks like this:
public class DataAccess {
private Context context;
private DBHelper dbHelper;
public DataAccess(Context _context) {
this.context = _context;
dbHelper = getDBHelper(_context);
}
private DBHelper getDBHelper(Context context) {
if (dbHelper == null) {
dbHelper = (DBHelper) OpenHelperManager.getHelper(context, DBHelper.class);
}
return dbHelper;
}
}
And I'm using the extended helper class:
public class DBHelper extends OrmLiteSqliteOpenHelper {
private static final String DATABASE_NAME = "database.db";
private static final int DATABASE_VERSION = 1;
private Dao<SomeObject, Integer> someObjectTable = null;
private ConnectionSource connectionSource = null;
public DBHelper(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
}
#Override
public void onCreate(SQLiteDatabase db, ConnectionSource connectionSource) {
this.connectionSource = connectionSource;
try {
TableUtils.createTable(connectionSource, SomeObject.class);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
#Override
public void onUpgrade(SQLiteDatabase db, ConnectionSource connectionSource, int oldVersion, int newVersion) {
}
public Dao<SomeObject, Integer> getSomeObjectDao() throws SQLException {
if (someObjectTable == null) {
dateTable = getDao(SomeObject.class);
}
return someObjectTable;
}
The idea is to create the DataAccess class and have it create the DBHelper if it hasn't already.
Can someone tell me if this is right or wrong, or if I'm on the right path?
Thanks!

I'm using ORMLite in an android project, and I'm not wanting to use the extended activities because I'm inserting values into the database on an AsyncTask.
You are on the right track but a little off #Matt. Frankly I'd never done a project without extending our base classes. But it is a good exercise so I've created this ORMLite example project which uses an Activity and manages its own helper.
Your DBHelper class is fine but really you do not need your DataAccess class. In each of your activities (or services...) you will need to have something like the following:
private DBHelper dbHelper = null;
#Override
protected void onDestroy() {
super.onDestroy();
if (dbHelper != null) {
OpenHelperManager.releaseHelper();
dbHelper = null;
}
}
private DBHelper getHelper() {
if (dbHelper == null) {
dbHelper = (DBHelper)OpenHelperManager.getHelper(this, DBHelper.class);
}
return dbHelper;
}
You [obviously], then use this in your code by doing something like:
Dao<SomeObject, Integer> someObjectDao = getHelper().getSomeObjectDao();
So whenever you call getHelper() the first time, it will get the helper through the manager, establishing the connection to the database. Whenever your application gets destroyed by the OS, it will release the helper -- possibly closing the underlying database connection if it is the last release.
Notice that the OpenHelperManager.getHelper() needs the Context as the first argument in case you do this without even an Activity base class.
Edit:
If you do want to create a DataAccess type class to centralize the handling of the helper class then you will need to make the methods static and do your own usage counter. If there are multiple activities and background tasks calling getHelper() then the question is when do you call releaseHelper()? You'll have to increment a count for each get and only call release when the counter gets back to 0. But even then, I'm not 100% sure how many lines you'd save out of your activity class.

I could nitpick but essentially you are doing it correct.
The call
dbHelper = (DBHelper) OpenHelperManager.getHelper(context, DBHelper.class);
Looks up the DBHelper class and instantiates it for the context. If you have defined it in your strings.xml, you can leave off the DBHelper.class at the end.
onUpgrade in you DBHelper.java, you may want to consider dropping the table you create in onCreate and then calling onCreate (to make sure you don't have conversion issues from update to update). You could do a more complex update if you wanted.
Other than that, it looks good. If you end up wanting data accessory methods for your DB objects beyond the base DAO methods, you will eventually want to create more thorough implementations of your object DAOs, but this is a good start.