Considering a simple thread pool. The main purpose of a thread pool is reusing(recycling) a thread in order to limit the maximum number of threads and preventing of remove and create threads, am I right?
Recently I have read(wikipedia connection pool) that in Java EE, database connection are pooled by server, but I cannot figure out why?
Questions:
What is the main purpose of pooling sql connection in java?
Is it okay to pool 128 connections and keep all of them open?
What is the difference if 128 concurrent select commands get run by one connection or 128 connection?
What is the correct way to connect to the database through a connection pool?
How do both server and database handle the connections?
How does server reuse a closed connection?
Is pool still necessary when whole server components use a shared connection?
Thanks in advance.
Database connection pooling solve is very similar to the thread pooling you have mentioned here. Here are the main reasons.
Main purpose of the db connection pool is to create some db
connections and keep them in the pool and reuse them whenever your
application code need to execute queries to db. Because creation of
connections is a very costly operation, Connection pooling allows us
to save that effort.
It is perfectly fine to create 128 connections and keep them in the pool.
No 128 concurrent selects on one connection may not work properly. With connection pooling you get the connection from the
pool and the use it to fire all the queries that are required for the
operation and then you return the connection back to the pool. So you
would need 128 connections to run the concurrent selects.
Generally you let the container handle the connection pooling. You can configure connection pooling in web.xml or in the container
specify configuration file and associate a JNDI name with data source
that is connected to the pool. Then you lookup that DataSource and
get the connections from there. Here is the example code for getting
connections.
public static Connection getConnection() {
Connection con = null;
try {
Context initContext = new InitialContext();
Context envContext = (Context) initContext.lookup("java:/comp/env");
DataSource dataSource = (DataSource) envContext.lookup("jdbc/db");
con = dataSource.getConnection();
} catch (NamingException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
return con;
}
Database will assume that the connection are still alive and its the
responsibility of the application server to keep the connections in
the pool alive (of course by the configuration by user) and avoid the
connection timeouts.
When your application code requests the connection to be closed by calling connection.close(), This connection is not closed and is actually returned to the pool so that further requests for connections from the pool will use this connection.
For all web applications its is strongly advised that you create the connection pool else you will run into problems related to
performance and too many connections opened.
Related
We are currently dealing with the function, that has to work partially with the database, and partially with a service, whice operations are time-consuming. So, generally speaking, here is a transactional method, that has a code like this:
Connection conn = null;
try {
conn = getConnection(); // This I get from the connection pool
Employee emp = queryDatabase(id);
// Point A - ??
Response resp = makeLongTimeServiceCall(emp);
// Point B - ??
anotherQueryDatabase(resp);
} catch (Exception e) {
throw e; // And this also rolls back the transaction
} finally {
// If conn is not closed, close it, so there would be no leaks
}
So the big question is - should I close the connection at point A and then get it again from the connection pool at point B, so that other servlets could use that connection while I interact with the service? My guess is that I should, but will this hurt my transaction?
In many circumstances: yes, closing and reopening the connection sounds good. However you need to understand the implication of doing them in two separate transactions (by closing and re-opening the connection you are inherently doing them in separate transaction).
If another user invokes an operation that changes the state of your data at Point B will the end result still be correct?
If you don't have a transaction you can close the connection and ask for a new connection when needed.
Remember that if you are using a connection pool closing a connection will not really close the connection, but only flag it as reusable by other threads.
Database connections shouldn't be left open. Open the connections just when you need to execute a query and close it as early as possible. So my solution is, YES.
if your application doesn't close connection properly may lead to some issues like the
connection pool maxing out.
Applications Not Properly Closing Connections:
When writing an application to use the WebSphere Application Server(WAS) datasource, the best way is fallowing get/use/close pattern.
1.get
-This is when the application makes a request to get a connection from the pool.
The application will first lookup the datasource name and then do a getConnection() call.
2.use
-This is when the application executes a query and waits for a response
3.close
-This is the final stage when the application has received the response from the database and is now done using that connection.
-The application should call close() in a finally block to close out the connection and return it to the free pool.
If your application does not follow this pattern, you may see connections staying open for long periods of time. This is especially seen when the application does not close the connection properly. After the response is received from the database, the application should be calling close in a finally block. If the application does not explicitly close the connection, that connection will stay open forever in WAS until that server is restarted. This is known as a connection leak. To resolve this type of issue, you will have to modify your application to close every connection
for further information: https://www.ibm.com/support/pages/common-reasons-why-connections-stay-open-long-period-time-websphere-application-server
We have BoneCP library managing our connection pools. I'd like to know if the following statement will return the connection back to the pool.
statement = conn.createStatement();
....
lots of code.
....
Connection conn = statement.getConnection();
statement.close();
conn.close();
Will the above code close the connection and put the connection back into the pool?
Update:
I'm asking this question because when I run the statistics on connectionpool, i still see the conPool.getTotalLeased() showing that 2 connections are being used. But, I've closed the connections using the above mechanism.
The whole sense of a pool is to hold already established connections to your database. When you retrieve a connection from your pool you save the time to connect to your database.
What you are seeing is the pool holding your connections so it is all fine.
When you close your connection, it is only returned to the pool and marked as available for your next retrieval.
Yes it does moves the connection back to the pool. There was a mistake when getting the connection from the pool, fixed it, now i'm not seeing the 2 connections in the totalLeased() method.
Mistake that I found
conPool.getConnection(); // loitering connection which we see in the getTotalLeased();
..
Statement st = conPool.getConnection().getStatement(); //we have handle to this connection.
return st;
The database connection is get like below
public Connection getDBConection(){
Context context = new InitialContext();
DataSource dataSource = (javax.sql.DataSource) context.lookup("java:myDataSource");
Connection conn = dataSource.getConnection();
}
For a userA, is each database request should call getDBConnection() once; but no need to control all request use the same connection?
That is, if userA has three database request, then userA should call getDBConnection() three times, and call Connection.closed() after used in each request?
If the userA call getDBConnection() three times (that is, call dataSource.getConnection() three times), is three connection created? Or it is unknown and controlled by weblogic?
I feel very chaos, is it true that there should be one new connection for one database request? or just call DataSource.getConnection() for each database request and the number of new connection created is controlled by web server, no need to think how many connection is actually created.
Every time you call DataSource.getConnection, the data source will retrieve a connection for you. It should be true that the returned connection is not being actively used by anyone else, but it is not necessarily a brand-new connection.
For example, if you use a connection pool, which is a very common practice, then when you call Connection.close, the connection is not actually closed, but instead returns to a pool of available connections. Then, when you call DataSource.getConnection, the connection pool will see if it has any spare connections lying around that it hasn't already handed out. If so, it will typically test that they haven't gone stale (usually by executing a very quick query against a dummy table). If not, it will return the existing connection to the caller. But if the connection is stale, then the connection pool will retrieve a truly new connection from the underlying database driver, and return that instead.
Typically, connection pools have a maximum number of real connections that they will keep at any one time (say, 50). If your application tries to request more than 50 simultaneous connections, DataSource.getConnection will throw an exception. Or in some implementations, it will block for a while until one becomes available, and then throw an exception after that time expires. For a sample implementation, have a look at Apache Commons DBCP.
Hopefully that answers your question!
I have an application that connects to MySQL using JDBC. There are cases where the JDBC connection lies idle for hours (maybe even days) and its loosing its connection to MySQL and then excepts when it tries to execute a query. What is the best solution for this?
Keeping the connection open for an undertemined time is a bad practice. The DB will force a close when it's been open for a too long time. You should write your JDBC code so that it always closes the connection (and statement and resultset) in the finally block of the very same try block where you've acquired them in order to prevent resource leaking like this.
However, acquiring the connection on every hiccup is indeed a pretty expensive task, so you'd like to use a connection pool. Decent connection pools will manage the opening, testing, reusing and closing the connections themselves. This does however not imply that you can change your JDBC code to never close them. You still need to close them since that would actually release the underlying connection back to the pool for future reuse.
There are several connection pools, like Apache DBCP which is singlethreaded and thus poor in performance, C3P0 which is multithreaded and performs better, and Tomcat JDBC for the case that you're using Tomcat and wouldn't like to use the builtin DBCP due to bad performance.
You can create connection pools programmatically, here's an example with C3P0:
ComboPooledDataSource dataSource = new ComboPooledDataSource();
dataSource.setDriverClass("com.mysql.jdbc.Driver");
dataSource.setJdbcUrl("jdbc:mysql://localhost:3306/dbname");
dataSource.setUser("username");
dataSource.setPassword("password");
Do it once during application's startup, then you can use it as follows:
Connection connection = null;
// ...
try {
connection = dataSource.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
When you're running inside a JNDI-capable container like a servletcontainer (e.g. Tomcat), then you can also declare it as a java.sql.DataSource (Tomcat specific manual here). It will then use the servletcontainer-provided connection pooling facilities. You can then acquire the datasource as follows:
DataSource dataSource = (DataSource) new InitialContext().lookup("java:comp/env/jdbc/YourDataSourceName");
There are libraries, such as Apache's DBCP which can do connection pooling. A part of this is they can be setup to automatically test the connection when you go to use it (such as "SELECT NOW() FROM DUAL", or something else harmless) and automatically re-establish the connection transparently if necessary, allowing your application to pretend that the connection is everlasting.
Check here:
http://download.oracle.com/javase/tutorial/jdbc/basics/connecting.html
Basically, you should use DataSource and always do a getConnection() before using it. DataSource, unless there's something terribly wrong, will reconnect if necessary.
I have a website, in which currently I am getting 1000 page views. I am expecting it will go around 30k per day in future. Now the problem for me to manage the DB connections.
At present I am just connecting to DB directly from java program. I know it is worst design in the world. But for time being I have written like that.
I have plan to manage connection pooling using JNDI. But the problem is my hosting provider is not supporting JNDI.
Can anyone suggest me how to manage DB connections without jndi?
Connection pooling does not per se require the connections to be obtained by JNDI. You can also just setup and use a connection pool independently from JNDI. Let's assume that you'd like to use C3P0, which is one of the better connection pools, then you can find "raw" JNDI-less setup details in this tutorial.
Here's an extract of the tutorial:
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setDriverClass( "org.postgresql.Driver" ); //loads the jdbc driver
cpds.setJdbcUrl( "jdbc:postgresql://localhost/testdb" );
cpds.setUser("swaldman");
cpds.setPassword("test-password");
Create the datasource once during application's startup and store it somewhere in the context. The connection can then be acquired and used as follows:
Connection connection = null;
// ...
try {
connection = cpds.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
Yes, closing in finally is still mandatory, else the connection pool won't be able to take the connection back in pool for future reuse and it'll run out of connections.