Can someone explain how exactly prepared connection pooling using dbcp can be used? (with some example code if possible). I've figured out how to turn it on - passing a KeyedObjectPoolFactory to the PoolableConnectionFactory.
But how should the specific prepared statements be defined after that?
Right now I'm only using a PoolingDataSource to get connections from the pool. How do I use the prepared statements from the pool?
Well talking about getting connection from the pool vs getting "not-pooled" connection, do you have any change in your code :)? I bet you do not. Same way with prepared statements. Your code should not change. So, there is no useful code example to this.
You should read docs for your JDBC Datasource implementation and see what developers have to say about pooling. There is no other source of reliable info on this.
From here:
This component has also the ability to pool PreparedStatements. When enabled a statement pool will be created for each Connection and PreparedStatements created by one of the following methods will be pooled:
* public PreparedStatement prepareStatement(String sql)
* public PreparedStatement prepareStatement(String sql, int resultSetType, int resultSetConcurrency)
So, you just keep using prepareStatement() call and your dbcp will in theory take care of pooling (i.e. if you are trying to create "select * from users u where u.name like :id", it will try to find this statement in the pool first)
Here's basic code I use.
GenericObjectPool connectionPool = new GenericObjectPool(null);
connectionPool.setMinEvictableIdleTimeMillis(1000 * 60 * 30);
connectionPool.setTimeBetweenEvictionRunsMillis(1000 * 60 * 30);
connectionPool.setNumTestsPerEvictionRun(3);
connectionPool.setTestOnBorrow(true);
connectionPool.setTestWhileIdle(false);
connectionPool.setTestOnReturn(false);
props = new Properties();
props.put("user", username);
props.put("password", password);
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(url, props);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(connectionFactory, connectionPool, null, "SELECT 1", false, true);
PoolingDataSource dataSource = new PoolingDataSource(connectionPool);
The thing is if you use a single Connection, it will cache PreparedStatements whether you want this or not, the only possible way to impact on this is to use DataSource properties or to use vendor-specific API. But these statements are not visible by other connections and if you prepare the same statement using another connection, it will recreate it again. So Connection Pools like DBCP under the hood allow reusing of PreparedStatements betwixt different connections (it uses PooledConnection interface instead of simple Connection), they keep track of all the statements prepared by all connections.
UPDATE: it seems I was wrong on this info, at least I couldn't find this functionality in C3P0.
Related
My server app uses prepared statements in almost all cases, to prevent sql injection. Nevertheless a possibility is needed providing special users executing raw SELECT queries.
How can I more or less securely make sure the query does not modify the database? Is it possible to execute a query read only, or is there any other 'secure' way making sure noone tries any sql injection?
(Using sqlite3, so I cannot use any privileges)
Thanks a lot!
JDBC supports read-only connections by calling Connection.setReadOnly(true). However the javadoc says:
Puts this connection in read-only mode as a hint to the driver to enable database optimizations.
Some JDBC drivers will enforce the read-only request, others will use it for optimizations only, or simply ignore it. I don't know how sqlite3 implements it. You'll have to test that.
Otherwise, you could do a "simple" parse of the SQL statement, to ensure that it's a single valid SELECT statement.
I'm not aware of a general JBDC configuration which specifies readonly. But Sqlite does have special database open modes and this can be leveraged in your connection to your sqlite database. Eg.
Properties config = new Properties();
config.setProperty("open_mode", "1"); //1 == readonly
Connection conn = DriverManager.getConnection("jdbc:sqlite:sample.db", config);
Credit: https://stackoverflow.com/a/18092761/62344
FWIW All supported open modes can be seen here.
If you use some sort of factory class to create or return connections to the database, you can individually set connections to be read-only:
public Connection getReadOnlyConnection() {
// Alternatively this could come from a connection pool:
final Connection conn = DriverManager.getConnection("jdbc:sqlite:sample.db");
conn.setReadOnly(true);
return conn;
}
If you're using a connection pool, then you may also want to provide a method for getting writeable connections too:
public Connection getWriteableConnection() {
final Connection conn = getPooledConnection(); // I'm assuming this method exists!
conn.setReadOnly(false);
return conn;
}
You could also provide just a single getConnection(boolean readOnly) method and simply pass the parameter through to the setReadOnly(boolean) call. I prefer the separate methods personally, as it makes your intent much clearer.
Alternatively, some databases like Oracle provide a read only mode that can be enabled. SQLite doesn't provide one, but you can emulate it by simply setting the actual database files (including directories) to read only on the filesystem itself.
Another way of doing it is as follows (credit goes to deadlock for the below code):
public Connection getReadOnlyConnection() {
SQLiteConfig config = new SQLiteConfig();
config.setReadOnly(true);
Connection conn = DriverManager.getConnection("jdbc:sqlite:sample.db",
config.toProperties());
}
I'm running a web application on Tomcat. I have a class that handles all DB queries.
This class contains the Connection object and methods that returns query results.
This is the connection object:
private static Connection conn = null;
It has only one instance (singleton).
In addition, I have methods that execute queries, such as search for a user in the db:
public static ResultSet searchUser(String user, String pass) throws SQLException
This method uses the static Connection object. My question is, is my use in static Connection object thread safe? Or can it cause problems when a lot of users will call the searchUser method?
is my use in static Connection object thread safe?
Absolutely not!
This way the connection going to be shared among all requests sent by all users and thus all queries will interfere with each other. But threadsafety is not your only problem, resource leaking is also your other problem. You're keeping a single connection open during the entire application's lifetime. The average database will reclaim the connection whenever it's been open for too long which is usually between 30 minutes and 8 hours, depending on DB's configuration. So if your web application runs longer than that, the connection is lost and you won't be able to execute queries anymore.
This problem also applies when those resources are held as a non-static instance variable of a class instance which is reused multiple times.
You should always acquire and close the connection, statement and resultset in the shortest possible scope, preferably inside the very same try-with-resources block as where you're executing the query according the following JDBC idiom:
public User find(String username, String password) throws SQLException {
User user = null;
try (
Connection connection = dataSource.getConnection();
PreparedStatement statement = connection.prepareStatement("SELECT id, username, email FROM user WHERE username=? AND password=md5(?)");
) {
statement.setString(1, username);
statement.setString(2, password);
try (ResultSet resultSet = statement.executeQuery()) {
if (resultSet.next()) {
user = new User();
user.setId(resultSet.getLong("id"));
user.setUsername(resultSet.getString("username"));
user.setEmail(resultSet.getString("email"));
}
}
}
return user;
}
Note that you should not return a ResultSet here. You should immediately read it and map it to a non-JDBC class and then return it, so that the ResultSet can safely be closed.
If you're not on Java 7 yet, then use a try-finally block wherein you manually close the closeable resources in the reverse order as you've acquired them. You can find an example here: How often should Connection, Statement and ResultSet be closed in JDBC?
If you worry about connecting performance, then you should be using connection pooling instead. This is built-in into many Java EE application servers and even barebones servletcontainers like Tomcat supports it. Just create a JNDI datasource in the server itself and let your webapp grab it as DataSource. It's transparently already a connection pool. You can find an example in the first link of the list below.
See also:
How should I connect to JDBC database / datasource in a servlet based application?
When my app loses connection, how should I recover it?
Am I Using JDBC Connection Pooling?
Show JDBC ResultSet in HTML in JSP page using MVC and DAO pattern
DAO tutorial with JDBC
If you are only running Select queries (searchUser sounds like only selecting data) there will be no issues, apart from thread contention.
As far as I know, a Connection can only handle one query at a time, so by using a single instance you will essentially serialize database access. But this does not necessarily mean, it is always safe to access a database like this in a multi threaded environment. There might still be issues if concurrent accesses are interleaving.
I have a use case where I use memcache to cache certain results from DB. I use the query itself as the key and value will be of type CachedRowSetImpl which serializes the result set. To form the query, I need to use PreparedStatement which in turn needs a connection object to the DB. This defeats the whole purpose of caching since more than half the time is being spent in establishing the connection. Is there any work around for this? Or do I have to use an alternate approach to cache results?
To avoid establishing a connection every time, you use a connection pool, like c3p0. You configure a connection pool to use Postgres, username swaldman, and passwordComboPooledDataSource
// in constructor
cpds = new ComboPooledDataSource();
cpds.setDriverClass( "org.postgresql.Driver" ); //loads the jdbc driver
cpds.setJdbcUrl( "jdbc:postgresql://localhost/testdb" );
cpds.setUser("swaldman");
cpds.setPassword("test-password");
When you need a JDBC Connection, just use:
Connection connection = cpds.getConnection();
There are other connection pools, like DBCP, which are set up in a similar way.
I have an application that connects to MySQL using JDBC. There are cases where the JDBC connection lies idle for hours (maybe even days) and its loosing its connection to MySQL and then excepts when it tries to execute a query. What is the best solution for this?
Keeping the connection open for an undertemined time is a bad practice. The DB will force a close when it's been open for a too long time. You should write your JDBC code so that it always closes the connection (and statement and resultset) in the finally block of the very same try block where you've acquired them in order to prevent resource leaking like this.
However, acquiring the connection on every hiccup is indeed a pretty expensive task, so you'd like to use a connection pool. Decent connection pools will manage the opening, testing, reusing and closing the connections themselves. This does however not imply that you can change your JDBC code to never close them. You still need to close them since that would actually release the underlying connection back to the pool for future reuse.
There are several connection pools, like Apache DBCP which is singlethreaded and thus poor in performance, C3P0 which is multithreaded and performs better, and Tomcat JDBC for the case that you're using Tomcat and wouldn't like to use the builtin DBCP due to bad performance.
You can create connection pools programmatically, here's an example with C3P0:
ComboPooledDataSource dataSource = new ComboPooledDataSource();
dataSource.setDriverClass("com.mysql.jdbc.Driver");
dataSource.setJdbcUrl("jdbc:mysql://localhost:3306/dbname");
dataSource.setUser("username");
dataSource.setPassword("password");
Do it once during application's startup, then you can use it as follows:
Connection connection = null;
// ...
try {
connection = dataSource.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
When you're running inside a JNDI-capable container like a servletcontainer (e.g. Tomcat), then you can also declare it as a java.sql.DataSource (Tomcat specific manual here). It will then use the servletcontainer-provided connection pooling facilities. You can then acquire the datasource as follows:
DataSource dataSource = (DataSource) new InitialContext().lookup("java:comp/env/jdbc/YourDataSourceName");
There are libraries, such as Apache's DBCP which can do connection pooling. A part of this is they can be setup to automatically test the connection when you go to use it (such as "SELECT NOW() FROM DUAL", or something else harmless) and automatically re-establish the connection transparently if necessary, allowing your application to pretend that the connection is everlasting.
Check here:
http://download.oracle.com/javase/tutorial/jdbc/basics/connecting.html
Basically, you should use DataSource and always do a getConnection() before using it. DataSource, unless there's something terribly wrong, will reconnect if necessary.
I just want to make sure that if I use the following, I am opening a separate DB session and not resuing the same one (for testing I need individual sessions).
Connection connection = DriverManager.getConnection(URL,USER,PASSWORD);
each time I do the above code, I run my query, then do a connection.close()
So for example:
while(some condition) {
Connection connection = DriverManager.getConnection(URL,USER,PASSWORD);
//now use the connection to generate a ResultSet of some query
connection.close();
}
So, each iteration of the loop (each query) needs its own session.
Is this properly opening separte sessions as I need (and if not, what would I need to add/change)? thanks
The javadoc says:
Attempts to establish a connection to
the given database URL
Slightly woolly language, and I suspect that this is up to the JDBC driver, but I'd be surprised if this did anything other than open a new connection.
I suppose it's possible for a JDBC driver to perform connection pooling under the hood, but I'd be surprised to see that.
In the case of the Oracle JDBC driver, this will open a new connection every time. This is a relatively slow process in Oracle, you may want to consider using a connection pool (e.g. Apache Commons DBCP, or c3p0) to improve performance.