I have read of a few ways to do this (namely, querying the pg_stat_activity table to get the pid to kill), but I have no data coming back in those queries.
In other DBMSs like MSSQL, I could at least see the uncommitted data and/or use profiler to see which connections are active. With postgres, I only know that this is the case because I am running a java application which occasionally closes unexpectedly which prevents the app from committing or rolling back its changes. When I try to run the app, I see errors for duplicate key data that isn't committed but is still active somehow. I have stopped my postgres instance but still ran into the problem once I opened it again. Is there another way to view anything which has a hanging connection to my db server and kill it?
The only way you get unique constraint errors from duplicate keys on uncommitted values is if both dups were inserted in that same transaction. Otherwise, the 2nd process to insert blocks until the first one either commits or rollsback.
If the process is bombing itself, then it is not surprising there is nothing to see in pg_stat_activity.
Related
Our app written in Java sends long-running queries to Oracle through JDBC API. It's inevitable that sometimes the app could crash or could get killed abruptly for plethora of reasons without giving it the chance to terminate the queries it has sent. When the app gets killed or stops, it also loses connection to Oracle.
Does Oracle DB keep the query running in the background even if it already has lost connection with the app that had sent the query?
Please cite sources.
When a connection between the database and the app is lost, Oracle will stop the session's queries and kill the session. But there are two potential exceptions:
Rollback must finish. From the Database Concepts manual: "A transaction ends when ... A client process terminates abnormally, causing the transaction to be implicitly rolled back using metadata stored in the transaction table and the undo segment." That rollback process cannot be stopped regardless of what happens to the connection. Even if you kill the database instance, when the instance restarts it will resume the rollback. As a general rule of thumb, the time to rollback will be about the same as the time the database spent running the DML. You just have to wait while Oracle puts itself back into a consistent state.
Zombie sessions. Although I don't have a reproducible test case for this problem, I'm sure every DBA has a story about sessions running after the client process disappeared, or even after they killed the session. Before you dismiss this concern as an old myth, note that the SQLNET.EXPIRE_TIME parameter was created for this scenario. Setting the value greater than 0 will have Oracle periodically check and clear terminated sessions. But you don't need to set this parameter unless you're having specific problems.
I really had a bad experience today. I applied some batch SQL scripts through Netbeans IDE to my H2 database (which is running in TCP mode). After 5 hours of work, the database connection in Netbeans IDE suddenly freezed... Subsequently I restarted the server (on which the H2 database is running) and then I realized that all changes of the last 5 hours were not applied or somehow rolled back...
My conclusion is, that the changes were only in the cache and not flushed to the database, since the results were at any one time visible when queried after each SQL script.
Therefore, what happens to the database cache in the case of a system failure ? Gone ... ?
Yes. The cache is gone in the event of a system failure. You must not have committed the transaction. The only guarantee then is that it must rollback (since it wasn't committed and the client has disconnected).
If it had been committed and subsequently crashed (before flushing) then it would have been possible for the server to still recover based on some combination of commit/transaction log and internal metadata.
I am working in a multisharded database environment in which downtime of a shard is not a fatal occurrence. When the application starts, all the shard info is loaded in a cache and respective Session objects loaded, however, it is expected that during the application uptime some of the shards may go down, in which case the application fails over to one of the remaining shards. It is almost a standard use case scenario.
In some cases, I need to do a cross-shard scan and the way I do it is iterate through the map of sessions described above and retrieve data. The problem comes when I have a Hibernate session to a database that went down in the meantime since the session was open. I would like to have a preemptive way of checking that the DB is down without attempting a transaction and getting org.hibernate.exception.GenericJDBCException, which is the way I am finding it out now but that is bad cause it is a runtime exception, among all the other evils. I also tried Session.isOpen() and Session.isConnected()but both methods return true even when the database is down.
Is there a way to check connectivity of a Hibernate Session preemptively and gracefully? Something like send a ping to the DB. Also it may be worthwhile knowing that we are using c3p0 for connection pooling. Is there a way I can configure it to tell Hibernate that a connection is down and close/disconnect the associated session?
I think those only work is you explicity called the method session.close()
We have an older web-based application (Java with Spring 2.5.4 framework) running on a GlassFish 3.1 (build 43) server. This application was recently (a few weeks ago) re-directed to use an Oracle 11g (11.2.0.3.0) database and ojdbc6.jar/orai18n.jar (up from Oracle 10g 10.2.0.3.0 and ojdbc14.jar) -- using a JDBC Thin connection. The application is using org.apache.commons.dbcp.BasicDataSource version 1.2.2 for connections and the database requests are handled either through Spring jdbcTemplate (via the JdbcDaoSupport abstract class) or Spring's PlatformTransactionManager.
This morning we noticed that application users were able to enter information, modify it and later to retrieve and print that data through the application, but that there were no committed updates for the last 24 hours. This application currently has only a few users each day and they are apparently sharing the same connection which has been kept open by the connection pool during the last day and so their uncommitted updates were visible through the application, but not through other connections to the database. When the connection was closed, the uncommitted updates were lost.
Examining the server logs showed no errors from the time of the last committed changes to the database through the times of printed reports the next morning. In addition, even if some of the changes had been (somehow) made with the JDBC connection being set to Auto-Commit false, there were specific commits made for some of those updates that were part of a transaction which, as part of a try/catch block should have either executed one of the "transactionManager.commit(transactionStatus);" or "transactionManager.rollback(transactionStatus);" calls that must have been processed without error. It looks as though the commit was returning successfully, but no commit actually occurred.
Restarting the GlassFish domain and the application restored the normal operation with the various updates being committed as they are entered.
My question is has anyone seen or heard about anything like this occurring and, if so, what could have caused it?
Thank you for any ideas here -- we are at a loss.
Some new information:
Examination of our Oracle 11g Server showed that near the time that we believe that the commits appeared to stop, there were four operations blocked on some other operation that we were not able to fully resolve, but was probably an update.
Examination of the Glassfish Server logs showed that the appearance of the worker threads changed following this estimated start time and fewer threads were appearing in the log until only one thread continued to be used for several hours.
The problem occurred again about one week later and was caught after about 1/2 hour. At this time, there were two worker threads in operation.
The problem occurred due to a combination of two things. The first was a method that setup a Spring Transaction, but had an exit that bypassed both the TransactionManager.commit() and the TransactionManager.rollback() (as well as the several SQL requests making up the transaction). Although this was admittedly incorrect coding, in the past, this transaction was closed and therefore had no effect on subsequent usage.
The solution was to insure that the transaction was not started if there was nothing to be done; or, in general double check to make sure that all transactions, once started, are completed.
I am not certain of the exact how or why this problem began presenting itself, so the following is partly conjectured. Apparently, upgrading to Oracle 11g and/or switching to the ojdbc6.jar driver altered the earlier behavior of the incorrect code so that the transaction was not terminated and the connection auto-commit was left false. (It could also be due to some other change that we have not identified since the special case above happens rarely – but does happen.) The corresponding JDBC connection appears to be bound to a specific GlassFish worker thread (I will call this a 'bad' thread in the following as opposed to the normally acting 'good' threads). Whenever this 'bad' thread is used to handle an application request (for this particular application), changes are uncommitted and selects return dirty data. As time goes on, when a change is requested on a 'good' thread and JDBC connection that already has an uncommitted change made on the 'bad' thread, than the new request hangs and the worker thread also hangs. Eventually all but the 'bad' worker thread are hung and everything seems to work correctly from the application viewpoint, but nothing is ever committed.
Again, the solution was to correct the bad code.
In my application, I need the ability to close all HSQL connections, delete the database files, and then open new connections that create a new database. I am using CACHED tables on disk.
What I am finding is that HSQL is remembering the contents of the old database - even though I've completed deleted those files from disk and established new connections! I've put breakpoints on the constructors for the org.hsqldb.jdbc.JDBCConnection class to verify that I'm getting new Connection objects (I am).
If I stop and start the process, then HSQL correctly 'forgets' the old database and starts fresh, but this is a server-side process that I can't stop and start easily.
The only solution I've found so far is to make some tweak to the path to the HSQL data files, so that it will not try to use the cached data from the last connection. I'm hoping that there is some method I can call in HSQL to clear out the cache, so that new connections will start with no data remembered from the old data.
You should shutdown the database. This closes any connections that are left open and clears up the caches.
The SQL statement, SHUTDOWN is used.
You can delete the files after shutdown.