I have this class (my database connection factory):
#ApplicationScoped
public class ConnectionFactory {
private ComboPooledDataSource datasource;
private Long open = 0l;
private Long close = 0l;
#PostConstruct
public void init() throws PropertyVetoException, SQLException {
datasource = new ComboPooledDataSource();
datasource.setDriverClass("org.postgresql.Driver");
datasource.setJdbcUrl("jdbc:postgresql:dbcampanha");
datasource.setUser("postgres");
datasource.setPassword("admin");
datasource.setMinPoolSize(1);
datasource.setMaxPoolSize(5);
datasource.setCheckoutTimeout(30000);
datasource.setUnreturnedConnectionTimeout(30);
datasource.setMaxIdleTime(30);
datasource.setDebugUnreturnedConnectionStackTraces(true);
datasource.setAcquireIncrement(1);
}
#Produces
#RequestScoped
public Connection getConnection() throws ClassNotFoundException {
open++;
try {
Connection connection = datasource.getConnection();
connection.setAutoCommit(false);
return connection;
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public void close(#Disposes Connection connection) {
close++;
try {
connection.commit();
connection.close();
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public Long getOpenedConnectionCounter() {
return open;
}
public Long getClosedConnectionCounter(){
return close;
}
public ComboPooledDataSource getDatasource(){
return datasource;
}
}
I use this class with an JAX-RS application. And for some tests using this route:
#RequestScoped
#Path("/test")
public class TesteService {
#Inject
private Connection connection;
#GET
#Produces(MyMediaType.JSON)
#Path("/yes")
public Response success() throws SQLException {
connection.getClientInfo("");
return Response.ok().build();
}
}
And this class for my Client:
public class TesteMain {
private static final String prefix = "http://localhost:8080/schoolwork/service/test/";
public static void main(String[] args) throws InterruptedException {
for (int i = 0; i < 10000; i++) {
Request request = new Request(prefix + "yes");
request.start();
if(i % 10 == 0)
Thread.sleep(1000l);
}
}
public static class Request extends Thread {
private String rota;
public Request(String rota){
this.rota = rota;
}
#Override
public void run() {
try {
HttpURLConnection url = (HttpURLConnection) (new URL(rota).openConnection());
url.connect();
System.out.println(url.getResponseCode() == 200 ? "SUCCESS" : "ERROR");
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
I receive this infos:
{
"opened-connection": 789,
"closed-connection": 867,
}
Yes, I have a number of closed database connection greater than opened. HOW? Any idea for this?
I use
Tomcat 7 + Java 7
P.S. I am sorry for my bad English :/
SOLVED
I change my counters for AtomicInteger objects, and works perfectly.
So, two quick comments:
Your counters are (boxed) longs read and updated willy-nilly by 1000 concurrent Threads. Their values will in general be unpredictable and nondeterministic. They certainly won't accurately count what you intend them to count. Consider using atomic operations on AtomicLongs instead.
Your commit() (or rollback()) should be attached to your database business logic, the part where you can tell a unit of work has either succeeded or failed. You shouldn't automatically commit on close.
Related
My apologies for throwing this random subject, but I did not come up with a better name,
class ReportSenderRunnable implements Runnable {
private final LPLogCompressor compressor;
public ReportSenderRunnable(final LPLogCompressor compressor) {
this.compressor = compressor;
}
#Override
public void run() {
executeTasks();
}
private void executeTasks() {
try {
// compressor.compress();
reportStatus = ReportStatus.COMPRESSING;
System.out.println("compressing for 10 seconds");
Thread.sleep(10000);
} catch (final IllegalStateException e) {
logCompressionError(e.getMessage());
} /*catch (final IOException e) {
logCompressionError(e.getMessage());
}*/ catch (InterruptedException e) {
logCompressionError(e.getMessage());
}
try {
reportStatus = ReportStatus.SENDING;
System.out.println("sending for 10 seconds");
Thread.sleep(10000);
} catch (final InterruptedException e) {
reportStatus = ReportStatus.EXCEPTION_IN_SENDING;
}
try {
reportStatus = ReportStatus.SUBMITTING_REPORT;
System.out.println("submitting report for 10 seconds");
Thread.sleep(10000);
} catch (final InterruptedException e) {
reportStatus = ReportStatus.EXCEPTION_IN_SUBMITTING_REPORT;
}
System.out.println("Report Sender completed");
reportStatus = ReportStatus.DONE;
}
private void logCompressionError(final String cause) {
logError(ReportStatus.COMPRESSING, cause);
reportStatus = ReportStatus.EXCEPTION_IN_COMPRESSION;
}
private void logError(final ReportStatus status, final String cause) {
LOGGER.error("{} - {}", status, cause);
}
}
Ideally, statements like
System.out.println("sending for 10 seconds");
Thread.sleep(10000);
will be replaced by actual tasks, but for now assuming this is the case, and they way it runs is
private void submitJob() {
final ExecutorService executorService = Executors.newSingleThreadExecutor();
try {
final LPLogCompressor lpLogCompressor = getLpLogCompressor();
executorService.execute(getReportSenderRunnable(lpLogCompressor));
} catch (final IOException e) {
reportStatus = ReportStatus.EXCEPTION_IN_COMPRESSION;
LOGGER.debug("Error in starting compression: {}", e.getMessage());
}
System.out.println("started Report Sender Job");
}
My question was how to effectively test this code? The one I wrote is
#Test
public void testJobAllStages() throws InterruptedException, IOException {
final ReportSender reportSender = spy(new ReportSender());
doReturn(compressor).when(reportSender).getLpLogCompressor();
when(compressor.compress()).thenReturn("nothing");
reportSender.sendAndReturnStatus();
Thread.sleep(10);
assertEquals(ReportStatus.COMPRESSING, reportSender.getCurrentStatus());
Thread.sleep(10000);
assertEquals(ReportStatus.SENDING, reportSender.getCurrentStatus());
Thread.sleep(10000);
assertEquals(ReportStatus.SUBMITTING_REPORT, reportSender.getCurrentStatus());
}
This runs well for above code.
To me this is crappy for following reasons
Not all tasks would take same time in ideal cases
Testing with Thread.sleep will take too much time and also adds non-determinism.
Question
How do I test this effectively?
You could add a class with a method (e.g., TimedAssertion.waitForCallable) that accepts a Callable, which then uses an ExecutorService to execute that Callable every second until it returns true. If it doesn't return true in a specific period of time, it fails.
You would then call that class from your test like this:
boolean result;
result = new TimedAssertion().waitForCallable(() ->
reportSender.getCurrentStatus() == ReportStatus.COMPRESSING);
assertTrue(result);
result = new TimedAssertion().waitForCallable(() ->
reportSender.getCurrentStatus() == ReportStatus.SENDING);
assertTrue(result);
...etc. This way, you can easily wait for a particular state in your code to be true, without waiting too long -- and you can reuse this new class anywhere that you need this sort of assertion.
Based on #Boris the Spider comment, I made use of mocks and here is what my tests look like
#Mock
private ReportSenderRunnable reportSenderRunnable;
#Mock
private LPLogCompressor compressor;
#Before
public void setUp() throws Exception {
MockitoAnnotations.initMocks(this);
}
#Test(timeout = 1000)
public void testJobNoException() throws InterruptedException, IOException {
final ReportSender reportSender = spy(new ReportSender());
doReturn(compressor).when(reportSender).getLpLogCompressor();
when(compressor.compress()).thenReturn("nothing");
reportSender.sendAndReturnStatus();
Thread.sleep(10);
assertEquals("Job must be completed successfully", ReportStatus.DONE,
reportSender.getCurrentStatus());
}
#Test(timeout = 1000)
public void testJobWithIllegalStateException() throws Exception {
final ReportSender reportSender = spy(new ReportSender());
doReturn(compressor).when(reportSender).getLpLogCompressor();
doThrow(IllegalStateException.class).when(compressor).compress();
reportSender.sendAndReturnStatus();
Thread.sleep(10);
assertEquals("Job must failed during compression", ReportStatus.EXCEPTION_IN_COMPRESSION,
reportSender.getCurrentStatus());
}
I'm trying to build a simple mailer daemon for a Tomcat 7, MySQL app (and Eclipse). This the first time I have tried to use a ServletContextListener.
Everything works perfectly. Except, if I change my mailer code, and Tomcat reloads the class. It then bombs with a JNDI exception where it can't find the database. I'm not comfortable using it as-is. I don't want a class reload to kill the task on the server.
Everything works fine after a restart and before the reload. So I must be missing something or doing things in the wrong order.
The database connection is done in the DAO. So after restarting, the DAO must be getting severed?
Any help would be most appreciated...
The error I am getting is:
Name [comp/env/jdbc/somedb] is not bound in this Context. Unable to find [comp].
javax.naming.NameNotFoundException: Name [comp/env/jdbc/somedb] is not bound in this Context. Unable to find [comp].
PooledConnection has already been closed.
at org.apache.naming.NamingContext.lookup(NamingContext.java:819)
at org.apache.naming.NamingContext.lookup(NamingContext.java:167)
at org.apache.naming.SelectorContext.lookup(SelectorContext.java:156)
at javax.naming.InitialContext.lookup(Unknown Source)
at util.DbUtil.getConnection(DbUtil.java:23)
at dao.NoticeDao.getNoticesByEvent(NoticeDao.java:49)
at dao.NoticeDao.getNoticesByStatus(NoticeDao.java:46)
at util.AppMailer.sendMailQueue(AppMailer.java:88)
at util.AppMailer.run(AppMailer.java:71)
at java.lang.Thread.run(Unknown Source)
java.sql.SQLException: PooledConnection has already been closed.
at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:86)
at com.sun.proxy.$Proxy7.prepareStatement(Unknown Source)
at dao.NoticeDao.getNoticesByEvent(NoticeDao.java:60)
at dao.NoticeDao.getNoticesByStatus(NoticeDao.java:46)
at util.AppMailer.sendMailQueue(AppMailer.java:88)
at util.AppMailer.run(AppMailer.java:71)
at java.lang.Thread.run(Unknown Source)
Update: For a second attempt, I simplified and separated the daemon stuff from the application logic. The app logic is now fully standalone. But I have the same problem.
public class AppMailerRunner implements ServletContextListener {
private ServletContext context = null;
private Thread mailerThread;
public AppMailerRunner() {}
#Override
public void contextInitialized(ServletContextEvent event) {
this.context = event.getServletContext();
System.out.printf("Starting: %s\n",this.getClass());
mailerThread = new Thread(new MailerDaemon());
mailerThread.setDaemon(true);
mailerThread.start();
}
#Override
public void contextDestroyed(ServletContextEvent event) {
System.out.printf("Stopping: %s\n",this.getClass());
mailerThread.interrupt();
this.context = null;
}
class MailerDaemon implements Runnable {
#Override
public void run() {
AppMailer appMailer = new AppMailer();
while(!Thread.currentThread().isInterrupted()){
try {
appMailer.sendMailQueue();
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
...
<listener>
<listener-class>util.AppMailerRunner</listener-class>
</listener>
...
public class AppMailer{
private NoticeDao noticeDao;
private Session mailSession;
private Boolean sending;
...
public AppMailer() {
super();
noticeDao = new NoticeDao();
sending = false;
}
do stuff...
...
public class NoticeDao {
public NoticeDao() {
}
...
public List<Notice> getNotices() {
Connection conn = DbUtil.getConnection();
List<Notice> notices = new ArrayList<Notice>();
try {
PreparedStatement ps = conn.prepareStatement("SELECT * FROM notices");
ResultSet rs = ps.executeQuery();
while (rs.next()) {
Notice notice = mapFields(rs);
notices.add(notice);
}
} catch (SQLException e) {
System.out.println(e.getMessage());
e.printStackTrace();
} finally {
DbUtil.close(conn);
}
return notices;
}
private static Notice mapFields(ResultSet rs) throws SQLException {
Notice notice = new Notice();
notice.setId( rs.getLong("id"));
notice.setItemid( rs.getLong("itemid"));
notice.setItemtype( rs.getString("itemtype"));
notice.setTestmode( rs.getBoolean("testmode"));
notice.setName( rs.getString("name"));
notice.setStatus( rs.getString("status"));
notice.setError( rs.getString("error"));
notice.setCreated( rs.getDate("created"));
notice.setModified( rs.getDate("modified"));
notice.setLog( rs.getString("log"));
return notice;
}
...
}
...
public class DbUtil {
private static Connection conn = null;
public DbUtil() {
}
public static Connection getConnection() {
InitialContext ctx;
try {
ctx = new InitialContext();
DataSource ds = (DataSource)ctx.lookup("java:comp/env/jdbc/somedb");
conn = ds.getConnection();
} catch (NamingException e) {
System.out.println(e.getMessage());
e.printStackTrace();
} catch (SQLException e) {
System.out.println(e.getMessage());
e.printStackTrace();
}
return conn;
}
public static void close(Connection conn){
if(conn!=null)
try {
conn.close();
} catch (SQLException e) {
System.out.println(e.getMessage());
e.printStackTrace();
}
}
}
Try adding break statement in the catch block in your MailerDaemon class.
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
break;
}
Note that interruption status is cleared when InterruptedException is thrown. So the thread created in the contextInitialized will never break out of the loop.
See the javadoc here.
Hope this helps.
I have this simple CDI bean which displays content into JSF page:
#Named("ZonesController")
#ViewScoped
public class Zones implements Serializable
{
#Resource(name = "jdbc/Oracle")
private DataSource ds;
...........
public int countDBRowNum() throws Exception
{
String SqlStatement = null;
if (ds == null)
{
throw new SQLException();
}
Connection conn = ds.getConnection();
if (conn == null)
{
throw new SQLException();
}
PreparedStatement ps = null;
ResultSet resultSet = null;
int count = 0;
try
{
conn.setAutoCommit(false);
boolean committed = false;
try
{
SqlStatement = "SELECT COUNT(1) FROM component x, componentstats y WHERE x.componentstatsid = y.componentstatsid AND y.componenttypeid = 1100";
ps = conn.prepareStatement(SqlStatement);
resultSet = ps.executeQuery();
if (resultSet.next())
{
count = resultSet.getInt(1);
}
conn.commit();
committed = true;
}
finally
{
if (!committed)
{
conn.rollback();
}
}
}
finally
{
ps.close();
conn.close();
}
// Returns total rows in table.
return count;
}
.............
}
I created this JUnit test case which calls the Java method:
public class ZonesTest
{
#BeforeClass
public static void setUpClass() throws Exception
{
try
{
// Create initial context
System.setProperty(Context.INITIAL_CONTEXT_FACTORY,
"org.apache.naming.java.javaURLContextFactory");
System.setProperty(Context.URL_PKG_PREFIXES,
"org.apache.naming");
InitialContext ic = new InitialContext();
ic.createSubcontext("java:");
ic.createSubcontext("java:/comp");
ic.createSubcontext("java:/comp/env");
ic.createSubcontext("java:/comp/env/jdbc");
// Construct DataSource
OracleConnectionPoolDataSource ds = new OracleConnectionPoolDataSource();
ds.setURL("jdbc:oracle:thin:#192.168.1.104:1521:oracle");
ds.setUser("admin");
ds.setPassword("qwerty");
ic.bind("java:/comp/env/jdbc/oracle", ds);
}
catch (NamingException ex)
{
//Logger.getLogger(MyDAOTest.class.getName()).log(Level.SEVERE, null, ex);
}
}
#Test
public void testCountDBRowNum() throws Exception
{
Zones instance = new Zones();
int rows = instance.countDBRowNum();
System.out.println(rows);
}
}
I get error at these lines:
if (ds == null)
{
throw new SQLException();
}
How I can solve this problem? I want to use the datasource from the JUnit test during the testing. Can I somehow use the JUnit datasource?
You can make DataSource ds a JavaBean property and set its value on your JUnit test. This way you hide the complexity of the JNDI binding and focus your tests only on the business logic.
Your controller:
#Named("ZonesController")
#ViewScoped
public class Zones implements Serializable
{
#Resource(name = "jdbc/Oracle")
private DataSource ds;
public void setDs(DataSource ds){this.ds=ds;}
public DataSource getDs(){return ds;}
...
}
And you test class:
public class ZonesTest
{
private static OracleConnectionPoolDataSource ds;
#BeforeClass
public static void setUpClass() throws Exception
{
try
{
// Construct DataSource
ds = new OracleConnectionPoolDataSource();
ds.setURL("jdbc:oracle:thin:#192.168.1.104:1521:oracle");
ds.setUser("admin");
ds.setPassword("qwerty");
}
catch (NamingException ex)
{
//Logger.getLogger(MyDAOTest.class.getName()).log(Level.SEVERE, null, ex);
}
}
#Test
public void testCountDBRowNum() throws Exception
{
Zones instance = new Zones();
instance.setDs(ds);
int rows = instance.countDBRowNum();
System.out.println(rows);
}
}
As a side note, following the MVC design pattern I would even decouple the controller business logic from the database connection, so no valid connection is needed to execute the unit tests and your focus is entirely on the behavior of your controller.
If you are using Spring and want to use all Spring beans within your JUnit class you can always use the #RunWith JUnit annotation.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration
public class ZonesTest
{
#Autowire
Zones zones;
// your BeforeClass method here
#Test
public void testCountDBRowNum() throws Exception
{
int rows = zones.countDBRowNum();
System.out.println(rows);
}
}
I have my own connection pool class
public class ConnectionPool {
private static final Logger log = Logger.getLogger(ConnectionPool.class);
public final static String PROPERTIES_FILENAME = "config";
public static final int DEFAULT_POOL_SIZE = 10;
//single instance
private static ConnectionPool instatance;
//queue of free connections
private BlockingQueue<Connection> connectionQueue;
public ConnectionPool(String driver, String url, String user,
String password, int poolSize)
throws ClassNotFoundException, DAOException{
try{
Class.forName(driver);
connectionQueue = new ArrayBlockingQueue<Connection>(poolSize);
for(int i = 0; i < poolSize ;i++){
Connection connection = DriverManager.getConnection(url, user, password);
connectionQueue.offer(connection);
}
}
catch (SQLException e) {
log.error(e);
throw new DAOException(e.getMessage());
}
}
public static void init() throws DAOException{
try {
if(instatance == null){
String driver = ConfigurationManager.
getInstance().getProperty("DATABASE_DRIVER_NAME");
String url = ConfigurationManager.
getInstance().getProperty("DATABASE_URL");
String user = ConfigurationManager.
getInstance().getProperty("DATABASE_USER");
String password = ConfigurationManager.
getInstance().getProperty("DATABASE_PASSWORD");
String poolSizeStr = ConfigurationManager.
getInstance().getProperty("DATABASE_POOLSIZE");
int poolSize = (poolSizeStr != null) ?
Integer.parseInt(poolSizeStr) : DEFAULT_POOL_SIZE;
log.info("Trying to create pool of connections...");
instatance = new ConnectionPool(driver,url,user,password,poolSize);
log.info("Connection pool initialized");
}
}catch (ClassNotFoundException e) {
log.error(e);
} catch (SQLException e) {
log.error(e);
throw new DAOException(e.getMessage());
}
}
public static void dispose() throws DAOException {
try {
if(instatance != null){
instatance.clearConnectionQueue();
instatance = null;
log.info("Connection queue is disposed");
}
} catch (DAOException e) {
log.info(e.getMessage());
throw new DAOException(e.getMessage());
}
}
public static ConnectionPool getInstance(){
return instatance;
}
public Connection takeConnection() {
Connection connection = null;
try{
connection = connectionQueue.take();
}catch (InterruptedException e) {
log.info("Free connection waiting interrupted.Returned null connection");
log.error(e);
}
return connection;
}
public void releaseConnection(Connection connection) throws DAOException {
try {
if(!connection.isClosed()){
if(!connectionQueue.offer(connection)){
log.info("Connections is not added.");
}
}
else{
log.info("Trying to release closed connection.");
}
} catch (SQLException e) {
log.info("SQLException at connection isClosed(). Connection is not added");
throw new DAOException(e.getMessage());
}
}
private void clearConnectionQueue() throws DAOException{
try {
Connection connection;
while((connection = connectionQueue.poll()) != null){
if(!connection.getAutoCommit()){
connection.commit();
connection.close();
}
}
} catch (SQLException e) {
log.info(e.getMessage());
throw new DAOException(e.getMessage());
}
}
and need it to be initialized in the begining of struts servlet's life cycle. If it were regular servlet I'd used init() method not thinking, and I'd done it:
public class LinkAction extends DispatchAction {
private static final String PARAM_NAME_LANGUAGE = "language";
/**
* This is the Struts action method called on
* http://.../actionPath?method=myAction1,
* where "method" is the value specified in <action> element :
* ( <action parameter="method" .../> )
*/
private static ConnectionPool connectionPool =
ConnectionPool.getInstance();
public void init(){
try {
if(connectionPool == null){
ConnectionPool.init();
connectionPool = ConnectionPool.getInstance();
}
} catch (DAOException e) {
log.error(e);
}
}
public ActionForward newsList(ActionMapping mapping, ActionForm form,
HttpServletRequest request, HttpServletResponse response)
throws Exception {
return mapping.findForward("newsList");
}
public ActionForward addNews(ActionMapping mapping, ActionForm form,
HttpServletRequest request, HttpServletResponse response)
throws Exception {
return mapping.findForward("addNews");
}
public ActionForward changeLocale(ActionMapping mapping, ActionForm form,
HttpServletRequest request, HttpServletResponse response)
throws Exception {
String localeValue = request.getParameter("localeValue");
request.getSession().setAttribute(PARAM_NAME_LANGUAGE, localeValue);
return mapping.findForward("newsList");
}
}
But this doesn't work with Struts Action, so I decided that it could be done in the struts-config.xml or web.xml. But how?
I think that you should use a ServletContextListener:
http://docs.oracle.com/javaee/6/api/javax/servlet/ServletContextListener.html
Even when using another framework it's a great place to put data source initialization code.
This listener is called every time your applications comes up and running so that you can put your datasource into a context attribute and recover it whenever necessary.
Need code to create the connection pool in java?
How does we make sure that connection pool doesn't return the same object which is already in use?
How happens if client closed the connection after taking it out from Connection pool?
Update 1:
I want to create this in Simple Java terms and want to see how it works in Multithreading Env. I mean which methods would be synchronized and which are not. Also will this class would be a public class? If yes then any one can access this class and reinitialize the connection pool?
Update 2:
I have some code as below. But i don't how "Closing a connection coming from a pool returns it to the pool, it doesn't close the connection physically."
Also i didn't understood this "Because if a connection has been borrowed from the pool and not returned yet, it's not "available" and can't be redistributed to another client of the pool."
import java.util.*;
import java.sql.*;
class ConnectionPoolManager
{
String databaseUrl = "jdbc:mysql://localhost:3306/myDatabase";
String userName = "userName";
String password = "userPass";
Vector connectionPool = new Vector();
public ConnectionPoolManager()
{
initialize();
}
public ConnectionPoolManager(
//String databaseName,
String databaseUrl,
String userName,
String password
)
{
this.databaseUrl = databaseUrl;
this.userName = userName;
this.password = password;
initialize();
}
private void initialize()
{
//Here we can initialize all the information that we need
initializeConnectionPool();
}
private void initializeConnectionPool()
{
while(!checkIfConnectionPoolIsFull())
{
System.out.println("Connection Pool is NOT full. Proceeding with adding new connections");
//Adding new connection instance until the pool is full
connectionPool.addElement(createNewConnectionForPool());
}
System.out.println("Connection Pool is full.");
}
private synchronized boolean checkIfConnectionPoolIsFull()
{
final int MAX_POOL_SIZE = 5;
//Check if the pool size
if(connectionPool.size() < 5)
{
return false;
}
return true;
}
//Creating a connection
private Connection createNewConnectionForPool()
{
Connection connection = null;
try
{
Class.forName("com.mysql.jdbc.Driver");
connection = DriverManager.getConnection(databaseUrl, userName, password);
System.out.println("Connection: "+connection);
}
catch(SQLException sqle)
{
System.err.println("SQLException: "+sqle);
return null;
}
catch(ClassNotFoundException cnfe)
{
System.err.println("ClassNotFoundException: "+cnfe);
return null;
}
return connection;
}
public synchronized Connection getConnectionFromPool()
{
Connection connection = null;
//Check if there is a connection available. There are times when all the connections in the pool may be used up
if(connectionPool.size() > 0)
{
connection = (Connection) connectionPool.firstElement();
connectionPool.removeElementAt(0);
}
//Giving away the connection from the connection pool
return connection;
}
public synchronized void returnConnectionToPool(Connection connection)
{
//Adding the connection from the client back to the connection pool
connectionPool.addElement(connection);
}
public static void main(String args[])
{
ConnectionPoolManager ConnectionPoolManager = new ConnectionPoolManager();
}
}
Need code to create the connection pool in java?
Not sure what the question is but don't create yet another connection pool, use an existing solution like C3P0, Apache DBCP, Proxool or BoneCP (a new player in that field). I would use C3P0.
How does we make sure that connection pool doesn't return the same object which is already in use?
Because if a connection has been borrowed from the pool and not returned yet, it's just not in the pool and can't be assigned to another client of the pool (resources are removed from the pool until they are returned).
How happens if client closed the connection after taking it out from Connection pool?
The connection a client gets from a pool is not really a java.sql.Connection, it's a wrapper (a proxy) for a java.sql.Connection that customizes the behavior of some methods. The close() method is one of them and does not close the Connection instance but returns it to the pool.
Don't write your own. There are plenty of librarires out there that will do this for you that are open source and easy to use and will have solved all the problems you'll encounter trying to make it yourself.
Here is a simple example that uses Apache's Commons DBCP and Commons Pool:
First set up a DataSource.
javax.sql.DataSource source = new org.apache.commons.dbcp.BasicDataSource();
source.setDriverClassName("com.mysql.jdbc.Driver");
source.setUsername("username");
source.setPassword("password");
source.setUrl("jdbc:mysql://localhost:3306/myDatabase");
Once you have a DataSource, it is easy to get a connection from the pool.
java.sql.Connection connection = source.getConnection();
closing the connection will return it to the pool for you.
connection.close();
I hope this source code helps
http://jagadeeshmanne.blogspot.com/2014/03/connection-pool-in-java-jdbc.html
Configuration.java
package com.jmanne.utils;
public class Configuration {
public String DB_USER_NAME ;
public String DB_PASSWORD ;
public String DB_URL;
public String DB_DRIVER;
public Integer DB_MAX_CONNECTIONS;
public Configuration(){
init();
}
private static Configuration configuration = new Configuration();
public static Configuration getInstance(){
return configuration;
}
private void init(){
DB_USER_NAME = "root"
DB_PASSWORD = "root"
DB_URL = "jdbc:mysql://localhost:3306/jmanne"
DB_DRIVER = "com.mysql.jdbc.Driver"
DB_MAX_CONNECTIONS = 5
}
}
JdbcConnectionPool.java
package com.jmanne.db;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.List;
import com.jmanne.utils.Configuration;
import com.mysql.jdbc.Connection;
public class JdbcConnectionPool {
List<connection> availableConnections = new ArrayList<connection>();
public JdbcConnectionPool()
{
initializeConnectionPool();
}
private void initializeConnectionPool()
{
while(!checkIfConnectionPoolIsFull())
{
availableConnections.add(createNewConnectionForPool());
}
}
private synchronized boolean checkIfConnectionPoolIsFull()
{
final int MAX_POOL_SIZE = Configuration.getInstance().DB_MAX_CONNECTIONS;
if(availableConnections.size() < MAX_POOL_SIZE)
{
return false;
}
return true;
}
//Creating a connection
private Connection createNewConnectionForPool()
{
Configuration config = Configuration.getInstance();
try {
Class.forName(config.DB_DRIVER);
Connection connection = (Connection) DriverManager.getConnection(
config.DB_URL, config.DB_USER_NAME, config.DB_PASSWORD);
return connection;
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
return null;
}
public synchronized Connection getConnectionFromPool()
{
Connection connection = null;
if(availableConnections.size() > 0)
{
connection = (Connection) availableConnections.get(0);
availableConnections.remove(0);
}
return connection;
}
public synchronized void returnConnectionToPool(Connection connection)
{
availableConnections.add(connection);
}
}
DataSource.java
package com.jmanne.db;
import java.sql.SQLException;
import com.mysql.jdbc.Connection;
public class DataSource {
static JdbcConnectionPool pool = new JdbcConnectionPool();
public static Connection getConnection() throws ClassNotFoundException, SQLException{
Connection connection = pool.getConnectionFromPool();
return connection;
}
public static void returnConnection(Connection connection) {
pool.returnConnectionToPool(connection);
}
}
Just use Semaphores. Ideally what you should do is use CP3O or DBCP as your connection pool. Now you can throttle you connection based on Semaphore.
Every time you do Get, you acquire and on every Release you release it from Semaphore. More over semaphores are thread safe.
Use one of the existing, e.g. Apache DBCP
The connections returned by the pool are often proxies which "ignore" the call to close() from the application. When the connections are returned to the pool, they can be reused. Pools also close and reopen automatically if necessary.
If your application runs on a server, then configure as Data Source, where server will take care of Pooling or else if a simple Java Client then use Apache DBCP(if to the database) or else use Apache Commons Pooling API
See here: Apache Commons
One argument for rolling your own connpool is the configuration and additional jars that are avoided. I agree you need to enable the 3rd party interfaces so you can swap in a mature connpool, but having your own tiny solution can have its place. Self cleaning Vector with synchronized block and a conn wrapper with close() marking the conn as available works very well for servlet apps.
I have one solution for the same to create a connection pool utility which can help you create a pool of default size 10.
#Component
public class ConnectionPool {
private static final Logger logger = LoggerFactory.getLogger(ConnectionPool.class);
private static final int MAX_POOL_SIZE_LIMIT = 10;
private BlockingQueue activeConnectinoQueue = new LinkedBlockingQueue<>();
private BlockingQueue usedConnectinoList = new LinkedBlockingQueue<>();
private int initialPoolSize = 5;
#Autowired
#Qualifier("dataSource")
private DataSource dataSource;
public void initConnectionPool() {
logger.info("ConnectionPool initialization started.");
if(activeConnectinoQueue.isEmpty() && usedConnectinoList.isEmpty()) {
for (int i=0; i<initialPoolSize; i++) {
createConnections();
}
}
logger.info("ConnectionPool initialization completed. ConnectionPool size : {}", activeConnectinoQueue.size());
}
private void createConnections() {
try {
Connection connection = dataSource.getConnection();
activeConnectinoQueue.add(connection);
}catch (SQLException e) {
logger.error("Error in getting connection from pool : ", e);
}
}
public Connection getConnection() {
if(activeConnectinoQueue.isEmpty()) {
initConnectionPool();
}
Connection connection = activeConnectinoQueue.remove();
try {
if(connection.isClosed()) {
connection = dataSource.getConnection();
}
}catch (SQLException e) {
logger.error("Error while getting connection from pool : ", e);
}
usedConnectinoList.add(connection);
return connection;
}
public void releaseConnection(Connection connection) {
if(connection != null) {
usedConnectinoList.remove(connection);
activeConnectinoQueue.add(connection);
}
}
public void setInitialPoolSize(int initialPoolSize) {
if(!(initialPoolSize < 0 || initialPoolSize > MAX_POOL_SIZE_LIMIT)) {
this.initialPoolSize = initialPoolSize;
}
}
public int getInitialPoolSize() {
return initialPoolSize;
}
public int getConnectionPoolSize() {
return activeConnectinoQueue.size() + usedConnectinoList.size();
}
public void setDataSource(AbstractDataSource dataSource) {
this.dataSource = dataSource;
}
public void closeConnectionPool() {
logger.info("Closing connectionPool started.");
close(usedConnectinoList);
close(activeConnectinoQueue);
logger.info("ConnectionPool Closed.");
}
private void close(BlockingQueue<Connection> connectinosQueue) {
for (int i=0; i<connectinosQueue.size(); i++) {
Connection connection = connectinosQueue.remove();
if(connection != null) {
try {
connection.close();
} catch (SQLException e) {
logger.error("Error in initializing connection pool : ", e);
}
}
}
}
}
now To make it safe we need to attach a factory object..
public enum ConnectionFactory {
CONNECTION;
private ConnectionPool connectionPool;
public void setConnectionPool(ConnectionPool connectionPool) {
this.connectionPool = connectionPool;
}
public Connection getConnection() {
return connectionPool.getConnection();
}
public void closeConnection() {
connectionPool.closeConnectionPool();
}
public void releaseConnection(Connection connection) {
connectionPool.releaseConnection(connection);
}
public int getConnectionPoolSize() {
return connectionPool.getConnectionPoolSize();
}
#Component
public static class ConnectionBuilder {
#Autowired
private ConnectionPool connectionPool;
public void setConnectionPool(ConnectionPool connectionPool) {
this.connectionPool = connectionPool;
}
#PostConstruct
public void postConstruct() {
for (ConnectionFactory cfactory : EnumSet.allOf(ConnectionFactory.class)) {
cfactory.setConnectionPool(connectionPool);
}
}
}
}
Java Connection Pool ?
There are three way to create JDBC Connection Pooling is very easy...
Apache Commons DBCP
public class DBCPDataSource {
private static BasicDataSource ds = new BasicDataSource();
static {
ds.setUrl("jdbc:h2:mem:test");
ds.setUsername("user");
ds.setPassword("password");
ds.setMinIdle(5);
ds.setMaxIdle(10);
ds.setMaxOpenPreparedStatements(100);
}
public static Connection getConnection() throws SQLException {
return ds.getConnection();
}
private DBCPDataSource(){ }
}
Now you can get connection
Connection con = DBCPDataSource.getConnection();
HikariCP
public class HikariCPDataSource {
private static HikariConfig config = new HikariConfig();
private static HikariDataSource ds;
static {
config.setJdbcUrl("jdbc:h2:mem:test");
config.setUsername("user");
config.setPassword("password");
config.addDataSourceProperty("cachePrepStmts", "true");
config.addDataSourceProperty("prepStmtCacheSize", "250");
config.addDataSourceProperty("prepStmtCacheSqlLimit", "2048");
ds = new HikariDataSource(config);
}
public static Connection getConnection() throws SQLException {
return ds.getConnection();
}
private HikariCPDataSource(){}
}
Now you can get connection
Connection con = HikariCPDataSource.getConnection();
C3PO
public class C3poDataSource {
private static ComboPooledDataSource cpds = new ComboPooledDataSource();
static {
try {
cpds.setDriverClass("org.h2.Driver");
cpds.setJdbcUrl("jdbc:h2:mem:test");
cpds.setUser("user");
cpds.setPassword("password");
} catch (PropertyVetoException e) {
// handle the exception
}
}
public static Connection getConnection() throws SQLException {
return cpds.getConnection();
}
private C3poDataSource(){}
}
Now you can get connection
Connection con = C3poDataSource.getConnection();
I have some model code in Java which has connection pool with multithreading.
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Iterator;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
abstract class ObjectPool<T> {
private ConcurrentLinkedQueue<T> pool;
ScheduledExecutorService executorService;
ObjectPool(int minObjects) {
pool = new ConcurrentLinkedQueue<T>();
for (int i = 0; i < minObjects; i++) {
pool.add(createObject());
}
}
ObjectPool(final int minObjects, final int maxSize, final long interval){
pool = new ConcurrentLinkedQueue<T>();
for (int i = 0; i < minObjects; i++) {
pool.add(createObject());
}
executorService = Executors.newSingleThreadScheduledExecutor();
executorService.scheduleWithFixedDelay(new Runnable(){
public void run() {
int size = pool.size();
while(size > maxSize){
pool.remove();
}
Iterator i = pool.iterator();
while(i.hasNext()){
T t = (T) i.next();
if(checkExpiry(t)){
System.out.println("Expiry existed...");
i.remove();
}
}
while(pool.size() < minObjects){
System.out.println("Adding more objects to pool");
pool.add(createObject());
}
}
}, interval, interval, TimeUnit.MILLISECONDS);
}
public T borrowObject() {
if (pool.peek() == null)
return createObject();
return pool.remove();
}
public void addObject(T obj) {
if (obj == null)
return;
pool.add(obj);
}
public abstract T createObject();
public abstract boolean checkExpiry(T t);
}
class MultithreadQuery extends Thread{
private ObjectPool<Connection> pool;
private int threadNo;
String query;
MultithreadQuery(ObjectPool<Connection> pool,int threadNo, String query){
this.pool = pool;
this.threadNo = threadNo;
this.query = query;
}
#Override
public void run(){
Connection con = pool.borrowObject();
Statement stmt;
try {
stmt = con.createStatement();
System.out.println("Query started for thread->"+ threadNo);
ResultSet rs=stmt.executeQuery(query);
while(rs.next())
System.out.println(rs.getInt(1)+" "+rs.getString(2)+" "+rs.getString(3));
System.out.println("closing connection....");
con.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
pool.addObject(con);
System.out.println("Query ended for thread->"+ threadNo);
}
}
public class ObjectPoolPatternDemo {
ObjectPool<Connection> pool;
public void setUp(){
pool = new ObjectPool<Connection>(4, 10, 1) {
#Override
public Connection createObject() {
Connection con;
try {
con = DriverManager.getConnection("URL","Username","Password");
return con;
} catch (SQLException e) {
e.printStackTrace();
}
return null;
}
#Override
public boolean checkExpiry(Connection conn) {
boolean expiryFlag = false;
try {
if(conn.isClosed())
expiryFlag = true;
} catch (SQLException e) {
e.printStackTrace();
}
return expiryFlag;
}
};
}
public static void main(String[] args) throws SQLException {
ObjectPoolPatternDemo oppd = new ObjectPoolPatternDemo();
oppd.setUp();
ExecutorService es = Executors.newFixedThreadPool(4);
String query = "select * from TABLE";
es.execute(new MultithreadQuery(oppd.pool,1,query));
es.execute(new MultithreadQuery(oppd.pool,2,query));
es.execute(new MultithreadQuery(oppd.pool,3,query));
es.execute(new MultithreadQuery(oppd.pool,4,query));
es.execute(new MultithreadQuery(oppd.pool,5,query));
es.execute(new MultithreadQuery(oppd.pool,6,query));
es.shutdown();
try {
es.awaitTermination(1, TimeUnit.DAYS);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("finally completed...");
}
}