SQL connections timing out with DataSource - java

I have a server that submits a query to a remote host to access account information from a database to log into a gameserver. The problem is the connections time out randomly however, I am using DataSource which should automatically re-establish any lost connections. Anyone have a clue how to resolve the issue?
public class Database {
private final PoolingDataSource source;
public Database(String driver, String url, String user, String password)
throws ClassNotFoundException {
Class.forName(driver);
source = createSource(url, user, password);
}
#SuppressWarnings({ "rawtypes", "unchecked", "unused" })
private PoolingDataSource createSource(String connectURI, String user,
String password) {
GenericObjectPool.Config config = new GenericObjectPool.Config();
config.maxActive = 150;
config.maxIdle = 100;
config.minIdle = 30;
config.maxWait = 1000;
config.testWhileIdle = true;
ObjectPool connectionPool = new GenericObjectPool(null, config);
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(
connectURI, user, password);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(
connectionFactory, connectionPool, null, null, false, true);
PoolingDataSource poolingDataSource = new PoolingDataSource(
connectionPool);
return poolingDataSource;
}
public Connection getConnection() throws SQLException {
return getSource().getConnection();
}
public PoolingDataSource getSource() {
return source;
}
}

"Back in the day...", in 2003, I was working with Tomcat 4.1 and was unpleasantly surprised to find that its implementation of DataSource required you to give it a validationQuery or else it would initialize the Connection only once and never verify that database server side would still respect the connection. Our Oracle server would simply eliminate connections that hadn't been used for, I think, 60 minutes. The effect of this was that as my server went to low volume, some number of my pooled connections would get killed on the database server side, but the application server didn't know, until it tried to use them at higher volumes. At this point they just stacktraced and that was the end of it. Adding the validationQuery had the effect of the DataSource itself keeping the connection alive and everything just worked. The gory details of this discovery process is here https://groups.yahoo.com/neo/groups/seajug/conversations/messages/4902 , if you are interested.
I recommend that you check if your GenericObjectPool implementation has a concept of validation query or heartbeat or something and figure out how to leverage it to keep your connections "fresh".

Related

Websphere 7.0.0.17 error with Pool Connection

Im having a problem with an application, I'm using EclipseLink 2.4.1 with oracle 11g, I'm getting a J2CA0045E error, because the pooling does not set free any connections pool from the jndi,
Have you have this problem?, apparently the connection does not get released and it reach the limit I've configured.
Any ideas, this is my Connection pool
Connection timeout: 180
Maximum connections: 10
Minimum connections: 1
Reap time: 60
Unused timeout: 180
Aged timeout: 120
Purge policy: Entire Pool
EDIT
I have a web service in the same service, and the behavior is the same, using same jndi, the connection increase and does not release any connection
initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/abc");
conn = ds.getConnection();
Based on the code provided, it looks like you are storing the Connection reference at a class scope. You should avoid storing references to Connection objects that could span multiple requests.
Instead, store a reference to the DataSource.
To illustrate. This approach is preferred:
public class GoodService {
DataSource ds;
public GoodService() throws NamingException {
// Only the DataSource reference is stored, this is allowed
ds = new InitialContext().lookup("java:comp/env/jdbc/abc");
}
public void doSomething() throws Exception {
// Get the connection in a request.
Connection conn = ds.getConnection();
try {
// do something...
} finally {
// Always close the connection in the finally block
// so it gets returned to the pool no matter what
conn.close();
}
}
}
The following approach is considered bad practice, and is likely to cause the J2CA0045E error you are seeing:
public class BadService {
Connection conn; // BAD: don't store a Connection across requests!
public BadService() throws NamingException, SQLException {
DataSource ds = new InitialContext().lookup("java:comp/env/jdbc/abc");
// BAD: don't leave connections open indefinitely like this
conn = ds.getConnection();
}
public void doSomething() throws Exception {
// do something using conn...
}
}

H2 connection pool

I want to make a connection pool for my h2 database. But I think my pool opens new connection every time I call getConnection(). I guess there should be a fixed amount of reusable connections, but if I run this code :
Connection conn = DataSource.getInstance().getConnection();
Statement stmt = conn.createStatement();
ResultSet rs;
rs = stmt.executeQuery("SELECT * FROM NODE_USERS;");
while (rs.next()) {
System.out.println(rs.getString("login"));
}
try {
// wait a bit
Thread.sleep(20000);
} catch (InterruptedException e) {
e.printStackTrace();
}
stmt.close();
rs.close();
conn.close();
DataSource:
public class DataSource {
private static volatile DataSource datasource;
private BasicDataSource ds;
private DataSource() throws IOException, SQLException, PropertyVetoException {
ds = new BasicDataSource();
ds.setUsername("sa");
ds.setPassword("sa");
ds.setUrl("jdbc:h2:tcp://localhost/~/test");
ds.setMinIdle(5);
ds.setMaxActive(10);
ds.setMaxIdle(20);
ds.setMaxOpenPreparedStatements(180);
}
public static DataSource getInstance() throws IOException, SQLException, PropertyVetoException {
if (datasource == null) {
synchronized (DataSource.class) {
if (datasource == null) {
datasource = new DataSource();
}
}
datasource = new DataSource();
}
return datasource;
}
public Connection getConnection() throws SQLException {
return this.ds.getConnection();
}
}
and then execute select * from information_schema.sessions;, there will be two rows. What is wrong? Also I was trying H2 tutorial example, but I've got the same result.
You are using a connection pool, namely BasicDataSource. It will create a configured number of connections initially and then when getConnection() is called, it will either reused a free pooled connection or create a new one, up to a configured limit (or no limit if configured so). When the connection obtained is "closed" using Connection.close(), it is actually returned to the pool instead of being closed right away.
You basically have no control over how many open connctions there will be at a given time, apart of configuring the minimum and maximum allowed open connections. You observing two open connections therefore doesn't prove anything. If you want to see whether there is a limit on open connections, configure the BasicDataSource to use at most 2 connections using BasicDataSource.maxActive(), then try to obtain three connections at the same time. And for another test, with the same configuration, try obtaining two connections, returning them using Connection.close() and then obtaining another two connections.

Single database connection for web application

I am developing a simple CRUD application, using JDBC to establish connection and perform basic CRUD operations. In that process, created a DatabaseListener to create a connection object at startup and storing it in the context attribute for reuse.
Below is the code.
import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import org.apache.log4j.Logger;
public class DatabaseInitListner implements ServletContextListener {
private static final Logger LOG = Logger.getLogger(DatabaseInitListner.class);
private DBUtil databaseUtil = null;
#Override
public void contextDestroyed(ServletContextEvent event) {
databaseUtil.closeConnection();
}
#Override
public void contextInitialized(ServletContextEvent contextinitEvent) {
ServletContext servletContext = contextinitEvent.getServletContext();
String database = servletContext.getInitParameter("db_name");
String url = servletContext.getInitParameter("db_url")
+ database;
String username = servletContext.getInitParameter("db_user");
String password = servletContext.getInitParameter("db_password");
String driverName = servletContext.getInitParameter("db_driver");
databaseUtil = new DBUtil(url, username, password,
driverName);
servletContext.setAttribute("databaseSingleConnectionObject",
databaseUtil.getConnection());
}
}
public class DBUtil {
private Connection connection = null;
private static final Logger LOG = Logger.getLogger(DatabaseUtil.class);
public DatabaseUtil(String url, String username, String password,
String driver) {
try {
Class.forName(driver);
this.connection = DriverManager.getConnection(url, username,
password);
LOG.debug("Connection Established... ");
} catch (ClassNotFoundException | SQLException e) {
LOG.error("Could not create connection... ", e);
}
}
public Connection getConnection() {
return connection;
}
public void closeConnection() {
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
LOG.error("Unable to close connection... ", e);
}
}
}
}
I am accessing the connection in servlets like this
Connection jdbcConnection = (Connection) getServletContext().getAttribute("databaseSingleConnectionObject");
I am not sure if this is right approach. What are the effects of single database connection?
When you use a single database connection like this you make your application slow and brittle.
Slow: because the connection implementation is synchronized, each user has to wait until the connection is free. If one user's query takes a while to come back that directly increases the time any other concurrent users spend waiting. If there were multiple connections available from a pool then the time spent by one user would not impact other users nearly as greatly (unless a query's results take all the JVM's memory or a big query bogs down the database server).
Brittle: The connection is a network connection, they tend to go down. Without a provision to create new connections any kind of timeout, network hiccup, or period of database non-availability (such as taking the database offline for maintenance) is going to require an application restart. Using a connection pool will mean your application will be able to survive these episodes and recover without outside intervention.
This will not be threadsafe, and if it were, performance would be really poor.
Look into using a Connection Pool, like DBCP or C3PO
You should let your application server manage database connection. Add a JNDI datasource in its configuration file and make a lookup from your application to get a connection when needed (for instance when you instantiate a class that must access your database).
You may configure the datasource to manage a connection pool so that each user session will get its own.
Depending on the AS you use run a search with keywords 'JNDI' and 'datasource' and you will get further details about the AS configuration and how to implement it in your application.

Spring JdbcTemplate with DBCP BasicDataSource still closes connections

I'm using postgres jdbc driver to connect to Amazon RedShift. Here are also BasicDataSource from DBCP 2.0.1 and JdbcTemplate from Spring 4. I use DataSourceTransactionManager with Transactional annotations.
It looks like DataSource still keeps on creating new connections!
// that is how dataSource is created
BasicDataSource dataSource = new BasicDataSource() {
public Connection getConnection() throws SQLException {
Connection c = super.getConnection();
System.out.println("New connection: " + c);
return c;
}
};
dataSource.setUsername(env.getProperty(USERNAME));
dataSource.setPassword(env.getProperty(PASSWORD));
dataSource.setDriverClassName(env.getProperty(DRIVER_CLASS));
dataSource.setUrl(env.getProperty(CONNECTION_URL));
and I see in console for each operation another Connection object (they have different hashcodes). If I switch to SingleConnectionDataSource all works as expected, with a single connection object.
Before call to jdbcTemplate#execute I use TransactionSynchronizationManager.isActualTransactionActive to see that transactions are working (they are)...
What could I miss then? Why transactions are closed? Or what more can I do to investigate the problem. The url also have tcpKeepAlive=true parameter...
UPD thanks to Evgeniy, I've changed the code to see when connections are really created:
BasicDataSource dataSource = new BasicDataSource() {
protected ConnectionFactory createConnectionFactory() throws SQLException {
final ConnectionFactory cf = super.createConnectionFactory();
return new ConnectionFactory() {
public Connection createConnection() throws SQLException {
Connection c = cf.createConnection();
System.out.println("New connection from factory: " + c);
return c;
}
};
}
};
//dataSource.setMaxIdle(0);
Now I really see that only two connections were created (and if I add setMaxIdle(0) they are instead recreated before each query).
So my suspicion was wrong and pool works as expected. Thanks a lot!
Different hash codes do not prove they are different physical connections. Try to watch sessions on the database and you will see that close on connection from BasicDataSource does not close a physical connection.

Basic DB connection pool with Java and Tomcat 7

I'm trying to create my first connection pool. I'm creating a Java web aplication with Tomcat 7 and a MySQL DB, and I'd like to create the simplest connection pool possible.
I've taken a look at several tutorials but it's not really clear for me, so I'd like you to confirm if I'm doing well.
I've written the following class as a connection pool manager:
package dao.mysql;
import java.sql.Connection;
import java.sql.SQLException;
import org.apache.tomcat.jdbc.pool.DataSource;
import org.apache.tomcat.jdbc.pool.PoolProperties;
public class MySQLConnectionPool {
private static DataSource datasource;
private static String driver = "com.mysql.jdbc.Driver";
private static String url = "jdbc:mysql://localhost:3306/mydb";
private static String username = "user";
private static String password = "password";
public MySQLConnectionPool() {
datasource = new DataSource(configurePoolProperties(driver, url, username, password));
}
private PoolProperties configurePoolProperties(String driver, String url, String username, String password) {
PoolProperties properties = new PoolProperties();
properties.setDriverClassName(driver);
properties.setUrl(url);
properties.setUsername(username);
properties.setPassword(password);
return properties;
}
public static synchronized Connection getConnection() {
Connection connection = null;
try {
connection = datasource.getConnection();
} catch (SQLException ex) {
System.out.println("Error while getting a connection from the pool! \nSQL state:" + ex.getSQLState() + "\nMESSAGE" + ex.getMessage());
}
return connection;
}
}
I'm not sure about the static properties nor the synchronized.
And I'm not sure about the "client" classes of the pool. I understand they have only to get a connection using
Connection con = MySQLConnectionPool.getConnection();
and finally close this connection using
con.close();
And that's it?
And also, is there any simpler or better way to do this?
Thanks very much!
This is the wrong way to do it.
Tomcat already has a connection pool and you can configure and setup without any code through the context.xml in the conf directory.
Once it is defined there, all you need to do is to lookup the JNDI DataSource in your code. Hardcoding all that (and re-inventing the wheel) is a very bad idea.
To learn how to configure a JNDI DataSource check out the manual: http://tomcat.apache.org/tomcat-7.0-doc/jndi-datasource-examples-howto.html
The Tomcat manual also has an example on how to obtain a connection from the pool:
InitialContext cxt = new InitialContext();
DataSource ds = (DataSource) cxt.lookup( "java:/comp/env/jdbc/dsname" );
where dsname is the name you provided in the context.xml
Check out the JNDI Datasource HOW-TO and the Tomcat JDBC Connection Pool Tomcat documentation. Letting Tomcat do it is preferable especially since it avoids class loader leaks.

Categories