We Keep Coding

Does JDBC template cache connection?

I have an java-spring web application, which will read, write and delete information from a user upload SQLite DB. I am using JDBCtemplate to set connection, query the DB and update the information.
I observed one behavior during my tests:
Every time,after users uploaded a new SQLite db file(it will has the same name, place at the same directory as the old DB file), if they do not reboot/restart tomcat, jdbcquery will report the db was corrupted exception.
To me this looked like the JDBCtemplate somehow cached the connection and is trying to resume the connection with the old db?
If so, do you know anyway to refresh the connection without rebooting the application?
final SingleConnectionDataSource dataSource = new singleConnectionDataSource();
try {
Class.forName("org.sqlite.JDBC");
} catch (ClassNotFoundException e) {
throw new applicationException(MessageTemplateNames.GENERAL_UNKNOWN_ERROR, e);
}
createDirectoryForDbIfNotExists();
dataSource.setUrl(String.format("%s%s", JDBC.PREFIX, getDbFileLocation()));
dataSource.setAutoCommit(true);

JDBCTemplate does not handle connection.It obtains the connection from the datasource set to it.
From the reference documentation for SingleConnectionDataSource
Implementation of SmartDataSource that wraps a single JDBC Connection
which is not closed after use. .....
This is primarily intended for testing. For example, it enables easy
testing outside an application server, for code that expects to work
on a DataSource. In contrast to DriverManagerDataSource, it reuses the
same Connection all the time, avoiding excessive creation of physical
Connections.
A DriverManagerDataSource will suite your requirement to get a new connection without reboot
Simple implementation of the standard JDBC DataSource interface,
configuring the plain old JDBC DriverManager via bean properties, and
returning a new Connection from every getConnection call.
Update
My answer might not solve your original problem , but will only answer the question for new connection without reboot. Please go through #Thilo's comment on the same.

Detect invalid credentials being passed to c3p0 connection pool

I'm writing a program that does some stuff on the database. Users are allowed to configure db processes, by passing db host port, type and credentials. It all works fine when values are correct. But when user passes invalid credentials I would like to show an error. So here is the part where I create my connection pool
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setJdbcUrl( connectionUrl );
cpds.setUser(username);
cpds.setPassword(password);
And later to verify if all is ok with the connection I do
cpds.getConnection()
I would expect to get some SQLException with vendor specific error saying that credentials are invalid (which happens when you use typical DriverManager way of getting the connection), but instead the process waits until a connection checkout exception is thrown
java.sql.SQLException: An attempt by a client to checkout a Connection has timed out.
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:118)
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:77)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:690)
....
Caused by: com.mchange.v2.resourcepool.TimeoutException: A client timed out while waiting to acquire a resource from com.mchange.v2.resourcepool.BasicResourcePool#20014b8 -- timeout at awaitAvailable()
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1467)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:644)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:554)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutAndMarkConnectionInUse(C3P0PooledConnectionPool.java:758)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:685)
... 66 more
How can I identify that there is a invalid credential issue with c3p0?

Your best way to validate provided credentials/JDBC params is to avoid connection pool at all.
Open dedicated connection just for this purpose and try to execute simplest SQL against new connection (eg SELECT 1 or similar).
After success, you can pass them to C3P0 otherwise propagate error back to user.

JDBC providers are free to create whatever error/exception messages they want. So you need to be ready to parse the error message of each provider in order to make sense of what is happening.
You can also try to get information from exception types if the JDBC provider segregates errors in separate types.
As a side note, giving too much information regarding why the connection failed may be considered a security breach. So one should not expect the JDBC driver to give you such information. For instance, why would any database collaborate with invasion attempts by saying "the username is correct, but the password is not."?

How to differ between MSSQL JDBC error codes?

My server calls sometimes the MSSQL JDBC CallableStatement.execute() that works with a previously connected connection. Everything is ok while the connection is alive. But if the connection is disconnected somehow (e.g. somebody turn SQL server down) the execute() call throws SQLException. I need to differ between 'connection was dropped' error and any other JDBC error (like table I'm trying to use doesn't exist). Since if I hit the connection error - I need to reconnect. And in any other case I need just to give error message to the user.
SQLException.getSQLState() always returns null and getErrorCode() always returns 0.
Thanks

I am no Java expert but this seems to be mainly based on Java.
You'd need to catch the exception from the .execute() command and query the string to see what the error is, then make a decision on which method to execute next based on that information.

"Lazy initialization" of jdbc connections from jndi datasource/connection pool: feasibility

I have a main controller servlet in which i instantiate a datasource. The servlet opens and closes the connections. Mainly, the servlet instantiates a command from the application using the "factory pattern". here is some code to explain:
public void init() throws ServletException {
super.init();
try {
datasource =(DataSource) getServletContext().getAttribute("DBCPool");
}
catch (Exception e) {
}
}
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
//some code...
Connection connection = null;
if(cmd.mightNeedLazyLoadingAConnection)
{
connection = null;
}
else
connection = getConnection();//where getConnection is a method: datasource.getconnection();
//now a command (a java class) is instantied, to which the "null" CONNECTION obj is passed as parameter
cmdFactory.getInstance().getCommand(Cmd).execute(tsk,connection);
//some code
//Then wherever there is catch exception i close() the connection
// and it is always closed in finally
finally {
if(connection!=null)
connection.close()
}
}
Now , this has a problem for the first case, ie connection=null, as it does never close the connection in the "finally" part (explained why in Update below).
"connection=null" is for cases where commands might not need to open a db connection because the data it is seeking for are cached in an identity map.
I tried to pass the "Connection" obj as a "null" parameter in the .execute(tsk,connection); and then in the corresponding java class to open a connection if needed
--> it did open the connection inside the command, however when process goes back to servlet : "Connection" is null as thus not closed.
What can i do to make the "Connection" obj's value get updated so that when back in servlet it is not "Null" anymore and i'd be able to close it?
I generally prefer to use a controller servlet that opens/close db connections, so what would be the best way to deal this kind of scenario where you have to do some sort of "lazy loading" a db connection from the pool and at the same time keep the opens/close of db connection assigned to the servlet?
Update (to explain further):
say i have a command : X.java
this command might/might not need a db connection (depends if the data searched for are in the identity map or not)
The system that i would like to have is:
(1)"client request"
(2)---> "Servlet": command.execute(connection)//where connection = null for now
(3) ---> "Command X": Do i need to go to database or record is in identity map?
(3.a) Case where it is needed to go to the database:
(3.a.1)connection = datasource.getconnection
(3.a.2) go get the data
(4)--->back to servlet: close "connection" in "Servlet"
Right now it is working until (3.a.2), but once back in (4) it appears that connection is still "null" and thus the code:
finally {
if(connection!=null)
connection.close()
}
Does not work (doesn't close the connection) and thus the db pool get drained like that.
How could connection - which starts as "null" and changes inside command "X"- get "globaly" updated to the its new value, and not only "updated" inside the scope of command "X"?
SOLUTION(S)
In case you are encountering the same scenario, you can chose of these 2 solutions:
You can Either use LazyConnectionDataSourceProxy, as mentionned by #Ryan Stewart for a "clean abstraction" and more professional solution
Or if you'd like use my solution described below (Basically i implemented a class similar to "LazyConnectionDataSourceProxy" but it is not as clean, it has less abstraction of details than "LazyConnectionDataSourceProxy")
My personal solution, Details:
I created a "Helper" class, which constructor takes the "datasource" as parameter
This helper class has methods to: "Lazy get" connection from pool,"close" connection
This class is instantiated in the servlet, and it gets a connection from the pool Only if needed throughout the application.
This is the code i added/modified in the servlet:
Connection connection = null;
if(cmd.mightNeedLazyLoadingAConnection)
{
helper hp = new helper(datasource);
cmdFactory.getInstance().getCommand(Cmd).execute(tsk,hp);
}
else
{
connection = getConnection();
cmdFactory.getInstance().getCommand(Cmd).execute(tsk,connection);
}
Then say in a command "X" , a db connection is needed i do:
Connection connection = hp.LazyGet();//Now got a connection from the pool
And this way, when proccess flow is back to the servlet level, i can :
Close
rollback
commit
etc..
All on this hp object of the helper class.
What benefits do i get from this:
I limit all database open / close / commit / rollback in one place, ie the Servlet, which is responsible of executing commands.
Having 3 cases: never needs db / always needs db / might need db thus now i decreased calls to the database by 1/3 , which is quite a lot knowing that database call grows exponentially with new features and new users registrations.
It might not be the Cleanest workaround, but between this way and having an additional "unnecessary" 1/3 database calls, it surely is better. Or just use LazyConnectionDataSourceProxy if you want a Tested, abstract and clean method.

Use a LazyConnectionDataSourceProxy. Then just get a "connection" every time, but a real connection is opened only when you actually do something that requires one. Thus you obey the "create/destroy" wisdom which Hiro2k pointed out because the connection's lifecycle is completely managed by your servlet.

In your specific case the only way to do it would be to return the connection. Java doesn't have any pass by reference semantics that could help you, unlike C where you could pass in the reference to the connection and then set it within the method.
I don't recommend your method return the connection, instead remember this simple rule and everything will work as you expect:
The object that creates it, is responsible for destroying it.
If what you want to do is not instantiate a connection for commands that don't require one, then add a method to your command interface that simply returns if needs one.
Command command = cmdFactory.getInstance().getCommand(Cmd);
if(command.requiresConnections){
connection = getConnection();
}
command.execute(tsk,connection);

java.sql.Exception ClosedConnection

I am getting the following error:
java.sql.SQLException: Closed
Connection at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:208)
at
oracle.jdbc.driver.PhysicalConnection.getMetaData(PhysicalConnection.java:1508)
at
com.ibatis.sqlmap.engine.execution.SqlExecutor.moveToNextResultsSafely(SqlExecutor.java:348)
at
com.ibatis.sqlmap.engine.execution.SqlExecutor.handleMultipleResults(SqlExecutor.java:320)
at
com.ibatis.sqlmap.engine.execution.SqlExecutor.executeQueryProcedure(SqlExecutor.java:277)
at
com.ibatis.sqlmap.engine.mapping.statement.ProcedureStatement.sqlExecuteQuery(ProcedureStatement.java:34)
at
com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithCallback(GeneralStatement.java:173)
at
com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryForList(GeneralStatement.java:123)
at
com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:614)
at
com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:588)
at
com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForList(SqlMapSessionImpl.java:118)
at
org.springframework.orm.ibatis.SqlMapClientTemplate$3.doInSqlMapClient(SqlMapClientTemplate.java:268)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.execute(SqlMapClientTemplate.java:193)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.executeWithListResult(SqlMapClientTemplate.java:219)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.queryForList(SqlMapClientTemplate.java:266)
at
gov.hud.pih.eiv.web.authentication.AuthenticationUserDAO.isPihUserDAO(AuthenticationUserDAO.java:24)
at
gov.hud.pih.eiv.web.authorization.AuthorizationProxy.isAuthorized(AuthorizationProxy.java:125)
at
gov.hud.pih.eiv.web.authorization.AuthorizationFilter.doFilter(AuthorizationFilter.java:224)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
at
I am really stumped and can't figure out what could be causing this error. I am not able to reproduce the error on my machine but on production it is coming a lot of times. I am using iBatis in the whole application so there are no chances of my code not closing connections.
We do have stored procedures that run for a long time before they return results (around 15 seconds).
does anyone have any ideas on what could be causing this? I dont think raising the # of connections on the application server will fix this issue buecause if connections were running out then we'd see "Error on allocating connections"
Sample code snippet:
this.setSqlMapClientTemplate(getSqlTempl());
getSqlMapClientTemplate().queryForList("authentication.isUserDAO", parmMap);
this.setSqlMapClientTemplate(getSqlTemplDW());
List results = (List) parmMap.get("Result0");
I am using validate in my connection pool.

Based on the stack trace, the likely cause is that you are continuing to use a ResultSet after close() was called on the Connection that generated the ResultSet.

What is your DataSource framework? Apache Commons DBCP?
do you use poolPrepareStatement property in data source configuration?
Check the following:
Make sure testOnBorrow and testOnReturn are true and place a simple validationQuery like select 0 from dual.
Do you use au
do you use autoCommit? Are you using START TRANSACTION, COMMIT in your stored procedures? After several days of debugging we found out that you can't mix transaction management both in Java and in SQL - you have to decide on one place to do it. Where are you doing yours?
Edit your question with answers to this, an we'll continue from there.

When a db server reboots, or there are some problems with a network, all the connections in the connection pool are broken and this usuall requires a reboot of application server
And if broken connection detected, you shold create a new one to replace it in connection pool. It's common problem called deadly connections.