We Keep Coding

database connection pool purpose?

Considering a simple thread pool. The main purpose of a thread pool is reusing(recycling) a thread in order to limit the maximum number of threads and preventing of remove and create threads, am I right?
Recently I have read(wikipedia connection pool) that in Java EE, database connection are pooled by server, but I cannot figure out why?
Questions:
What is the main purpose of pooling sql connection in java?
Is it okay to pool 128 connections and keep all of them open?
What is the difference if 128 concurrent select commands get run by one connection or 128 connection?
What is the correct way to connect to the database through a connection pool?
How do both server and database handle the connections?
How does server reuse a closed connection?
Is pool still necessary when whole server components use a shared connection?
Thanks in advance.

Database connection pooling solve is very similar to the thread pooling you have mentioned here. Here are the main reasons.
Main purpose of the db connection pool is to create some db
connections and keep them in the pool and reuse them whenever your
application code need to execute queries to db. Because creation of
connections is a very costly operation, Connection pooling allows us
to save that effort.
It is perfectly fine to create 128 connections and keep them in the pool.
No 128 concurrent selects on one connection may not work properly. With connection pooling you get the connection from the
pool and the use it to fire all the queries that are required for the
operation and then you return the connection back to the pool. So you
would need 128 connections to run the concurrent selects.
Generally you let the container handle the connection pooling. You can configure connection pooling in web.xml or in the container
specify configuration file and associate a JNDI name with data source
that is connected to the pool. Then you lookup that DataSource and
get the connections from there. Here is the example code for getting
connections.
public static Connection getConnection() {
Connection con = null;
try {
Context initContext = new InitialContext();
Context envContext = (Context) initContext.lookup("java:/comp/env");
DataSource dataSource = (DataSource) envContext.lookup("jdbc/db");
con = dataSource.getConnection();
} catch (NamingException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
return con;
}
Database will assume that the connection are still alive and its the
responsibility of the application server to keep the connections in
the pool alive (of course by the configuration by user) and avoid the
connection timeouts.
When your application code requests the connection to be closed by calling connection.close(), This connection is not closed and is actually returned to the pool so that further requests for connections from the pool will use this connection.
For all web applications its is strongly advised that you create the connection pool else you will run into problems related to
performance and too many connections opened.

Should a database connection stay open all the time or only be opened when needed?

I have a bukkit plugin (minecraft) that requires a connection to the database.
Should a database connection stay open all the time, or be opened and closed when needed?

The database connection must be opened only when its needed and closed after doing all the necessary job with it. Code sample:
Prior to Java 7:
Connection con = null;
try {
con = ... //retrieve the database connection
//do your work...
} catch (SQLException e) {
//handle the exception
} finally {
try {
if (con != null) {
con.close();
}
} catch (SQLException shouldNotHandleMe) {
//...
}
}
Java 7:
try (Connection con = ...) {
} catch (SQLException e) {
}
//no need to call Connection#close since now Connection interface extends Autocloseable
But since manually opening a database connection is too expensive, it is highly recommended to use a database connection pool, represented in Java with DataSource interface. This will handle the physical database connections for you and when you close it (i.e. calling Connection#close), the physical database connection will just be in SLEEP mode and still be open.
Related Q/A:
Java Connection Pooling
Some tools to handle database connection pooling:
BoneCP
c3po
Apache Commons DBCP
HikariCP

Depends on what are your needs.
Creating a connection takes some time, so if you need to access database frequently it's better to keep the connection open. Also it's better to create a pool, so that many users can access database simultaneously(if it's needed).
If you need to use this connection only few times you may not keep it open, but you will have delay when you would like to access database. So i suggest you to make a timer that will keep connection open for some time(connection timeout).

You need to close your connections after each query executions.Sometimes you need to execute multiple queries at the same time because the queries are hanging from each other.Such as "first insert task then assign it to the employees".At this time execute your queries on the same transaction and commit it, if some errors occur then rollback.By default autocommit is disabled in JDBC. Example
Use connection pooling.If you are developing a webapplication then use App Server connection pooling.App server will use the same pooling for each of your applications so you can control the connection count from the one point.Highly recommend the Apache Tomcat Connection pooling.Example
As an additional info:
Connection, Statement and ResultSet.
1.If you close connection you don't need close statement or resultset.Both of them will be closed automatically
2.If you close Statement it will close ResultSet also
3.if you use try-with-resources like this:
try (Connection con = ...) {
} catch (SQLException e) {
}
it will close the connection automatically.Because try-with-resources require autoclosable objects and Connection is autocloseable.You can see the details about try-with-resources here

Actually, it's all matter on how you write your application! It's an art, but sadly everyone takes a tutorial for a good practice like Microsoft's tutorials.
If you know what you are coding, then you keep your connection open for the lifetime of the application. It's simple, not because you have to go at work in the morning that everyday we have to build a special route just for you! You take that single route or 2 or 4 like everyone does! You judge for the traffics and you build 2, 4 or 6 routes as needed. If there is traffic with these 4 or 6 routes, you wait!
Happy coding.

The Connection should be opened only when required. If it is open before the actual need, it reduces one active connection from the connection pool..so it ultimately effects the users of the application.
So,it is always a better practice to open connection only when required and closing it after completion of process.
Always try puttting you connection close logic inside the finally block that will ensure that your connection will be closed,even if any exception occurs in the application
finally
{
connection.close()
}

Problem with singleton database connection

I'm using singleton design-pattern for my database connection.
Here is my method.
private static Connection con=null;
public static Connection getConnection(){
try {
if(con==null){
Class.forName("com.mysql.jdbc.Driver").newInstance();
con = DriverManager.getConnection("jdbc:mysql://localhost/mydb", "root", "");
}else if(con.isClosed()){
Class.forName("com.mysql.jdbc.Driver").newInstance();
con = DriverManager.getConnection("jdbc:mysql://localhost/mydb", "root", "");
}
} catch (Exception ex) {
}
return con;
}
Whenever I restarted mysql service, it generates following error and have to restart the application also to run again.
Error:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure
Can anyone show what is the correct way of implementing this to avoid this error?
thank you.

You're reusing the same Connection object across... well, everything. Your singleton should be a ConnectionFactory that returns a new connection every time.
As someone already noted, rather than create/close new connections use a connection pool. You don't have to write it yourself—there are readily available open-source implementations such as C3PO or DBCP.
Even better, rather than reinventing this wheel all over again, use a DataSource. If you can use Spring, it has a DriverManagerDataSource implementation just for this. Otherwise, feel free to look at how it's written and write your own (I've done this at least once when Spring wasn't an 'allowed' technology).
Finally, if you're running in a container such as Tomcat, JBoss or Glassfish, then you can simply bind a DataSource to JNDI and use that. This frees up your application from having to handle pooling and configuration/deployment issues (stuff like database user name & password) itself.

You should go for connection-pooling.
Your mechanism won't work if mysql server drops your connection some how.

A persistent connection with JDBC to MySQL

I have an application that connects to MySQL using JDBC. There are cases where the JDBC connection lies idle for hours (maybe even days) and its loosing its connection to MySQL and then excepts when it tries to execute a query. What is the best solution for this?

Keeping the connection open for an undertemined time is a bad practice. The DB will force a close when it's been open for a too long time. You should write your JDBC code so that it always closes the connection (and statement and resultset) in the finally block of the very same try block where you've acquired them in order to prevent resource leaking like this.
However, acquiring the connection on every hiccup is indeed a pretty expensive task, so you'd like to use a connection pool. Decent connection pools will manage the opening, testing, reusing and closing the connections themselves. This does however not imply that you can change your JDBC code to never close them. You still need to close them since that would actually release the underlying connection back to the pool for future reuse.
There are several connection pools, like Apache DBCP which is singlethreaded and thus poor in performance, C3P0 which is multithreaded and performs better, and Tomcat JDBC for the case that you're using Tomcat and wouldn't like to use the builtin DBCP due to bad performance.
You can create connection pools programmatically, here's an example with C3P0:
ComboPooledDataSource dataSource = new ComboPooledDataSource();
dataSource.setDriverClass("com.mysql.jdbc.Driver");
dataSource.setJdbcUrl("jdbc:mysql://localhost:3306/dbname");
dataSource.setUser("username");
dataSource.setPassword("password");
Do it once during application's startup, then you can use it as follows:
Connection connection = null;
// ...
try {
connection = dataSource.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
When you're running inside a JNDI-capable container like a servletcontainer (e.g. Tomcat), then you can also declare it as a java.sql.DataSource (Tomcat specific manual here). It will then use the servletcontainer-provided connection pooling facilities. You can then acquire the datasource as follows:
DataSource dataSource = (DataSource) new InitialContext().lookup("java:comp/env/jdbc/YourDataSourceName");

There are libraries, such as Apache's DBCP which can do connection pooling. A part of this is they can be setup to automatically test the connection when you go to use it (such as "SELECT NOW() FROM DUAL", or something else harmless) and automatically re-establish the connection transparently if necessary, allowing your application to pretend that the connection is everlasting.

Check here:
http://download.oracle.com/javase/tutorial/jdbc/basics/connecting.html
Basically, you should use DataSource and always do a getConnection() before using it. DataSource, unless there's something terribly wrong, will reconnect if necessary.

Connection timeout for DriverManager getConnection

I am trying to connect to DB using the standard JDBC way
connection = DriverManager.getConnection(url, username, password);
Is there a maximum value of timeout on the connection, how long does a connection live, can I increase the value. I want in cases the connection to be open forever , is it a good idea.

You can set the Timeout on the DriverManager like this:
DriverManager.setLoginTimeout(10);
Connection c = DriverManager.getConnection(url, username, password);
Which would imply that if the connection cannot open within the given time that it times out.
In terms of keeping a connection open forever, it is possible if you do not close the connection but it may not be a good idea. Connections should be closed as soon as you are finished with them.
If you want to optimise the opening and closing of connections then you can use a connection pool.

The value is usually DB-controlled. You have no control over it using code. It depends on the DB server used. It is usually around 30 minutes up to one hour.
On the other hand, keeping a Connection open forever is a very bad idea. Best practice is to acquire and close Connection, Statement and ResultSet in the shortest possible scope to avoid resource leaks and potential application crashes caused by the leaks and timeouts.
True, connecting the DB is an expensive task. If your application is supposed to run a relatively long time and to connect the DB fairly often, then consider using a connection pool to improve connecting performance. If your application is a webapplication, then take a look in the appserver's documentation, it usually provides a connection pooling facility in flavor of a DataSource. If it is a client application, then look for 3rd party connection pooling libraries which have proven their robustness with years, such as Apache Commons DBCP (commonly used, used in lot appservers), C3P0 (known from Hibernate) and Proxool (if you want XA connections).
Keep in mind, when using a connection pool, you still have to write proper JDBC code, i.o.w. acquire and close all the resources in the shortest possible scope. The connection pool will on its turn worry about actually closing the connection or just releasing it back to pool for further reuse.
You may get some more insights out of this article how to do the JDBC basics the proper way.
Hope this helps and happy coding.

Here is how to do it with Connector/J MYSQL driver:
String qqq = "jdbc:mysql://localhost/Test?connectTimeout=TIME_IN_MILLIS";
conn = DriverManager.getConnection(qqq, db_user, db_pass);
It worked for me after setLoginTimeout() did nothing.

Just reposting a more complete repost of a comment from user flamming_python as answer because it worked for me:
dbConnectionString = "jdbc:mysql://"+dbHost+":"+dbPort+"/"+dbTable+"?user="+dbUser+"&password="+dbPassword;
Properties properties = new Properties();
properties.put("connectTimeout", "2000");
dbConnect = DriverManager.getConnection(dbConnectionString, properties);
Original comment:
"LoL # your threads - you can do it quite simply by creating a Properties object prop, then prop.put("connectTimeout", "2000") (where the "2000" is the timeout in ms), and then pass your prop object to the DriverManager.getConnection() method along with your url"

As bestro suggested, it's possible to use a future with an associated timeout. For example:
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<Connection> future = executor.submit(new Callable<Connection>() {
#Override
public Connection call() throws Exception {
Connection con = DriverManager.getConnection(url, username, password);
return con;
}
});
Connection con = null;
try {
con = future.get(5, TimeUnit.SECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException ex) {
Logger.getLogger(Scratch.class.getName()).log(Level.SEVERE, null, ex);
}
executor.shutdownNow();
if (con == null) {
System.out.println("Could not establish connection");
} else {
System.out.println("Connection established!");
}

You can use ExecutorService interface from Java. Below is a sample of what you need to do.
Future<Boolean> future = executor.submit(YOUR_METHOD);
future.get(TIMEOUT_YOU_NEED, TimeUnit.SECONDS);