In my application, I need the ability to close all HSQL connections, delete the database files, and then open new connections that create a new database. I am using CACHED tables on disk.
What I am finding is that HSQL is remembering the contents of the old database - even though I've completed deleted those files from disk and established new connections! I've put breakpoints on the constructors for the org.hsqldb.jdbc.JDBCConnection class to verify that I'm getting new Connection objects (I am).
If I stop and start the process, then HSQL correctly 'forgets' the old database and starts fresh, but this is a server-side process that I can't stop and start easily.
The only solution I've found so far is to make some tweak to the path to the HSQL data files, so that it will not try to use the cached data from the last connection. I'm hoping that there is some method I can call in HSQL to clear out the cache, so that new connections will start with no data remembered from the old data.
You should shutdown the database. This closes any connections that are left open and clears up the caches.
The SQL statement, SHUTDOWN is used.
You can delete the files after shutdown.
Related
We are using H2 started as database server process and listening on standard TCP/IP port 9092.
Our application is deployed in a Tomcat using connection pooling. We do a purge during idle time which at the end results in closing all connections to H2. From time to time we observe errors when the application tries to open the connection to H2 again:
SCHEDULERSERVICE schedule: Exception: Database may be already in use: "Waited for database closing longer than 1 minute". Possible solutions: close all other connection(s); use the server mode [90020-199]
org.h2.jdbc.JdbcSQLNonTransientConnectionException: Database may be already in use: "Waited for database closing longer than 1 minute". Possible solutions: close all other connection(s); use the server mode [90020-199]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:617)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:427)
at org.h2.message.DbException.get(DbException.java:205)
at org.h2.message.DbException.get(DbException.java:181)
at org.h2.engine.Engine.openSession(Engine.java:209)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:178)
at org.h2.engine.Engine.createSession(Engine.java:161)
at org.h2.server.TcpServerThread.run(TcpServerThread.java:160)
at java.lang.Thread.run(Thread.java:748)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:617)
at org.h2.engine.SessionRemote.done(SessionRemote.java:607)
at org.h2.engine.SessionRemote.initTransfer(SessionRemote.java:143)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:431)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:317)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:169)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:148)
at org.h2.Driver.connect(Driver.java:69)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
The problem occurs when the Tomcat connection pool closes all idle connection (unused) and one connection still in use is closed afterwards.
The next attempt to open a new connection fails, a retry is successfully after some wait time.
Under which circumstances does this exception happen?
What does the exception mean?
Are there any recommendations to follow to avoid the problem?
It sounds to me that H2 does a database close after the last connection has been closed.
When does the database close occure?
How can database closures been controlled?
Thx in advance
Thorsten
Embedded database in web applications needs careful handling of its lifecycle.
You can add a javax.servlet.ServletContextListener implementation (marked with #WebListener annotation or included into web.xml) and add explicit database shutdown to its contextDestroyed() methods.
You can force database shutdown here with connection.createStatement().execute("SHUTDOWN"). If your application needs to write something to database during unload, it should do it before that command.
Without the explicit shutdown H2 closes the database when all connections are closed, if some other behavior wasn't configured explicitly (with parameters in JDBC URL, for example). For example, DB_CLOSE_DELAY sets the additional delay, maybe your application uses that setting and therefore H2 doesn't close the database immediately, or application doesn't close all connections immediately.
Anyway, when you're trying to update the web application of the fly, Tomcat tries to initialize the new version before its old version is unloaded. If H2 is in classpath of the web application itself, the new version will be unable to connect to the database during short period of time when the new version is already online but the old version isn't unloaded yet.
If you don't like it, you can run the standalone H2 Server process and use remote connections to it in your web applications.
Another option is to move H2 to the classpath of Tomcat itself and configure the connection pool as resource in the server.xml, in that case it shouldn't be affected by the lifecycle of your applications.
In both these cases you shouldn't use the SHUTDOWN command.
UPDATED
With client-server connections to a remote server such exception means that server decided to close the database because there are no active connection. This operation can't be interrupted and reverted in the middle. On attempt to open a new connection to the same database during this process it waits at most for 1 minute for completion of this process to re-open the database again. This timeout is not configurable.
There are two possible solutions.
DB_CLOSE_DELAY setting can be used with some large value in seconds. When all connections are closed, database will stay online for the specified number of seconds. -1 also can be used to set an infinite timeout.
You can try to speed up the shutdown process, but you have to figure out what takes so much time by yourself. The file compaction procedure is limited to 200 milliseconds by default, it may take a longer time, but I think it shouldn't be that long. Maybe you have a lot of temporary objects or uncommitted data. Maybe you have a very high fragmentation of database file. It's hard to say what's going wrong without further investigation.
I have read of a few ways to do this (namely, querying the pg_stat_activity table to get the pid to kill), but I have no data coming back in those queries.
In other DBMSs like MSSQL, I could at least see the uncommitted data and/or use profiler to see which connections are active. With postgres, I only know that this is the case because I am running a java application which occasionally closes unexpectedly which prevents the app from committing or rolling back its changes. When I try to run the app, I see errors for duplicate key data that isn't committed but is still active somehow. I have stopped my postgres instance but still ran into the problem once I opened it again. Is there another way to view anything which has a hanging connection to my db server and kill it?
The only way you get unique constraint errors from duplicate keys on uncommitted values is if both dups were inserted in that same transaction. Otherwise, the 2nd process to insert blocks until the first one either commits or rollsback.
If the process is bombing itself, then it is not surprising there is nothing to see in pg_stat_activity.
I really had a bad experience today. I applied some batch SQL scripts through Netbeans IDE to my H2 database (which is running in TCP mode). After 5 hours of work, the database connection in Netbeans IDE suddenly freezed... Subsequently I restarted the server (on which the H2 database is running) and then I realized that all changes of the last 5 hours were not applied or somehow rolled back...
My conclusion is, that the changes were only in the cache and not flushed to the database, since the results were at any one time visible when queried after each SQL script.
Therefore, what happens to the database cache in the case of a system failure ? Gone ... ?
Yes. The cache is gone in the event of a system failure. You must not have committed the transaction. The only guarantee then is that it must rollback (since it wasn't committed and the client has disconnected).
If it had been committed and subsequently crashed (before flushing) then it would have been possible for the server to still recover based on some combination of commit/transaction log and internal metadata.
I have a requirement to write a java application (web based) that will connect to an Oracle 11G databases (currently is connecting 10-12 different oracle databases), read some data from it (all are select queries).
After iterating this arraylist, I am connecting each database, fire select query (same query I am firing for all databases), get record , put it in one global collection list and close connection and continuing same process over this loop.
Currently I am "Executors" to connect multiple databases.
again using ExecutorService executor = Executors.newFixedThreadPool(20);
give one surprise to me. for creting first database connection, it shows immediate log, but for subsequent database connection, it prints logs after 60 seconds, so i am not getting why this is taking 60 secs or more for all connections?
logically, it should take time for all connections as like one.
Please suggest performance improvement for this application.
Opening a database connection is an expensive operation; if possible you should connect to each database once and reuse the connection for all queries made to that database (also known as connection pooling). It's not clear if this is what you're already doing.
Another thing that seems strange is that you have made the openConnection method synchronized. I see no need for doing that, and depending on how your code is structured it may mean that only one thread can be opening a connection at a given time. Since connecting takes a long time you'd want each thread to make its connection in parallel.
So, I'm trying to figure out how to log the SQL statements that would be run, without actually having an active mySQL server.
The problem I'm trying to solve is right now we're writing data to both a remote mySQL instance, as well as a local (archive data). We're logging to a local as a backup in case the remote becomes unreachable/goes down/etc.
So, what we'd like to do instead is log the SQL statements locally (we're going through Spring JDBC Template w/variable replacement), so not quite as easy as taking the SQL we piece together ourselves and write it to a file.
I did find log4jdbc which looks great, except we'd still need a local mySQL instance active, even if it's just using the blackhole engine. I did think maybe we could just use the log4jdbc on the remote server, but if the connection goes away, will the JDBCTemplate even try and run the queries on the underlying JDBC driver objects before getting the failure? Most pooling mechanisms will validate the connection before returning it, so it'd still just fail before the query had a chance to run & get logged.
So, who has any bright ideas?
Run the remote MySQL instance with bin logging enabled.
The binary logs can be backed up and if necessary converted into SQL using the mysqlbinlog command, to later restore a database.
Mark O'Connors solution is probably the best in general, but the way we solved this was to simply write out the data as CSV files formatted to be ready for import via a load data infile statement.