We Keep Coding

Java - Tomcat GC won't free up.. always cause crash, can't find any memory leak

I have few issues with my tomcat application.
I'm using a linux server with 1024M memory.
I deploy my app on the tomcat, and everything working great.
Recently i notice that the 'Tenured Gen' heap memory not free up when it should...
It reach 99% and then crash tomcat..
I check my application with VisualVM, and the same result.
it fill up the memory and the 'Old Gen' never free up.
This is when the application run few minutes with no request:
IMG: Everything looks normal
And when I start to send requests with 200 Thread on a loop
this what happened:
IMG: all memories are full
So then I check data on the MAT, and this is my result:
IMG: look like a memory leak
IMG: Problem with the sql jdbc?
IMG: Can't understand what is wrong
this is my ConnectionPool class:
public class ConnectionPool {
private static ConnectionPool singleton = null;
private ArrayList<Connection> freeConnections;
private ArrayList<Connection> allConnections;
private int MAX_CONNECTIONS = 1;
private final String shema = "Topic";
private final String url = "jdbc:mysql://localhost:3306/"+shema+"? autoReconnect=true&useSSL=false";
private final String username = "root";
private final String password = "password";
public static ConnectionPool getInstance(){
if (singleton == null)
{
synchronized (ConnectionPool.class) {
if (singleton == null){
System.out.println("ConnectionPool get instance");
try {
singleton = new ConnectionPool();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
} //this will invoke the constructor
}
}
}
return singleton;
}
private ConnectionPool() throws Exception {
freeConnections = new ArrayList<Connection>();
allConnections = new ArrayList<Connection>();
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
for (int i = 0; i < MAX_CONNECTIONS; i++) {
try {
addNewConnection();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
private void addNewConnection() throws SQLException {
try {
Connection conn = DriverManager.getConnection(url, username, password);
freeConnections.add(conn);
allConnections.add(conn);
} catch (SQLException e) {
throw e;
}
}
public Connection getConnection()
{
while (true) {
synchronized (freeConnections) {
if (freeConnections.size() != 0) { // free connection is available
Connection conn = freeConnections.get(0);
freeConnections.remove(0);
try {
conn.setAutoCommit(true);
} catch (SQLException e) {
e.printStackTrace();
}
return conn;
}
try {
freeConnections.wait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
public void returnConnection(Connection conn)
{
if (null == conn) { // ignore invalid value
return;
}
if (!allConnections.contains(conn)) {
return;
}
synchronized (freeConnections) {
if (freeConnections.contains(conn)) {
return;
}
freeConnections.add(conn);
freeConnections.notifyAll();
return;
}
}
public void closeAllconnections()
{
for (Connection conn : allConnections) {
try {
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
System.out.println("ConnectionPool all connection closed");
deregisterDriver();
}
public void deregisterDriver() {
try {
java.sql.Driver driver = DriverManager.getDriver(url);
DriverManager.deregisterDriver(driver);
} catch (SQLException e) {
e.printStackTrace();
}
System.out.println("ConnectionPool deregister driver");
}
}
Please help me to understand what is wrong and explain me.
1.Why the GC won't free up or why he can't do his job?
2.Is something wrong with my ConnectionPool Class?
3.why tomcat not saying anything about OutOfMemoryException in my logs(just crashing)?

See the connection details, apparently you have 10 connections and each retains cca 66 MB if the heap summing up to 660 MB required RAM.
I don't know what data you select, however when returning a connection you may want to close all resultsets and statements (why are you creating your own pool? dbcp, c3p0 or commons-pool ain't good enough? for learning?) and seems it may be not enough. I really don't know what the pool implementations do to release all resources properly.
And seems it is not so straightforward to share open connections between multiple threads Java MySQL JDBC Memory Leak so I would suggest to use a working pool solution (dbcp)

How to get Thread Safe in DAO design?

To my understanding it is possible with this code that when changing a user role another user can change the same role and always wins the last. It would even be possible for us to store parts of one and parts of the other. This is possible due to the 3 queries in the DAO. I would like to get "ThreadSafe" that during a change not another user can make a change or it will be detected that someone changed it before.
My idea was to change the method in the RoleManager.
Idea:
public interface RoleManager {
static synchronized void EditRole(UserRoleBO editedObjet, UserRoleBO nonEditedObject);
This does not work with this type of design(with a interface).
My Question:
Is there an elegant way to solve the problem without changing the
design?
Addition Note:
Tell me if i have big mistakes in my code.
Manager:
public class RoleManagerImpl implements RoleManager {
#Override
public void editRole(UserRoleBO editedObjet, UserRoleBO nonEditedObject) {
EditUserRole editUserRole = EditUserRole.Factory.createEditUserRole(nonEditedObject);
boolean hasChangedBeforeInDB = editUserRole.detectChanges();
if (hasChangedBeforeInDB) {
throw new ManagerException(ManagerException.TYPE.HASCHANGEDBEFOREINDB, null);
}
RoleDAO roleDAO = new RoleDAOImpl();
roleDAO.editRole(editedObjet);
}
}
DAO:
#Override
public int editRole(UserRoleBO role) {
Connection conn = null;
int status;
try {
//Set up connection
conn = ConnectionPool.getInstance().acquire();
DSLContext create = DSL.using(conn, SQLDialect.MARIADB);
//sql processing and return
status = create.executeUpdate(role.getRole());
EditUserRole editUserRole = EditUserRole.Factory.createEditUserRole(role);
editUserRole.detectChanges();
addPermission(editUserRole.getAddlist(), role.getRole());
deletePermissions(editUserRole.getDeleteList(), role.getRole());
}
// Error handling sql
catch (MappingException e) {
throw new DAOException(DAOException.TYPE.MAPPINGEXCEPTION, e);
}
catch (DataAccessException e) {
throw new DAOException(DAOException.TYPE.DATAACCESSEXECPTION, e);
}
catch (Exception e) {
throw new DAOException(DAOException.TYPE.UNKOWNEXCEPTION, e);
} finally {
//Connection release handling
try{
if(conn != null) {
ConnectionPool.getInstance().release(conn);
}
}
// Error handling connection
catch (DataAccessException e) {
throw new DAOException(DAOException.TYPE.RELEASECONNECTIONEXCEPTION, e);
}
catch (Exception e) {
throw new DAOException(DAOException.TYPE.UNKOWNRELEASECONNECTIONEXCEPTION, e);
}
}
//Return result
return status;
}
Thanks for helping.

this is just a possible answer. In my case, i use jooq and a mariadb.
With the assumption that we only have one central database this solution works. In a cluster, there is always the problem of the split brain.
What happens is that I lock the rows. So if the next thread tries to lock this he must wait. If it is allowed to continue, the exception HASCHANGEDBEFOREINDB is thrown.
Take care u have to commit or rollback to end the lock.
EditRole:
#Override
public int editRole(UserRoleBO editedRole ,UserRoleBO nonEditedRole) throws SQLException {
Connection conn = null;
int status;
try {
//Set up connection
conn = ConnectionPool.getInstance().acquire();
conn.setAutoCommit(false);
DSLContext create = DSL.using(conn, SQLDialect.MARIADB);
//lock rows
lockRowsOf(editedRole, conn);
EditUserRole editUserRole = EditUserRole.Factory.createEditUserRole(nonEditedRole);
boolean hasChangedBeforeInDB = editUserRole.detectChanges();
if (hasChangedBeforeInDB) {
throw new DAOException(DAOException.TYPE.HASCHANGEDBEFOREINDB, null);
}
EditUserRole editUserRole2 = EditUserRole.Factory.createEditUserRole(editedRole);
editUserRole2.detectChanges();
//sql processing and return
status = create.executeUpdate(editedRole.getRole());
addPermission(editUserRole2.getAddlist(), editedRole.getRole().getId(), conn);
deletePermissions(editUserRole2.getDeleteList(), editedRole.getRole(), conn);
conn.commit();
}
// Error handling sql
catch (MappingException e) {
conn.rollback();
throw new DAOException(DAOException.TYPE.MAPPINGEXCEPTION, e);
}
catch (DataAccessException e) {
conn.rollback();
throw new DAOException(DAOException.TYPE.DATAACCESSEXECPTION, e);
}
catch (Exception e) {
conn.rollback();
throw new DAOException(DAOException.TYPE.UNKOWNEXCEPTION, e);
} finally {
//Connection release handling
try{
if(conn != null) {
conn.setAutoCommit(true);
ConnectionPool.getInstance().release(conn);
}
}
// Error handling connection
catch (DataAccessException e) {
throw new DAOException(DAOException.TYPE.RELEASECONNECTIONEXCEPTION, e);
}
catch (Exception e) {
throw new DAOException(DAOException.TYPE.UNKOWNRELEASECONNECTIONEXCEPTION, e);
}
}
//Return result
return status;
}
Lock:
#Override
public void lockRowsOf(UserRoleBO role, Connection conn) {
int status;
try {
DSLContext create = DSL.using(conn, SQLDialect.MARIADB);
//sql processing and return
status = create.select()
.from(AUTH_ROLE)
.where(AUTH_ROLE.ID.eq(role.getRole().getId()))
.forUpdate().execute();
status = create.select()
.from(AUTH_ROLE_PERMISSION)
.where(AUTH_ROLE_PERMISSION.ROLE_ID.eq(role.getRole().getId()))
.forUpdate().execute();
}
// Error handling sql
catch (MappingException e) {
throw new DAOException(DAOException.TYPE.MAPPINGEXCEPTION, e);
}
catch (DataAccessException e) {
throw new DAOException(DAOException.TYPE.DATAACCESSEXECPTION, e);
}
catch (Exception e) {
throw new DAOException(DAOException.TYPE.UNKOWNEXCEPTION, e);
} finally {
//Connection will still needed to buffer the delete and insert
}
}

JAVA JDBC reusing connections

I have a Java program in which I am doing some JDBC for select queries. Will it be advisable to call testDataBase() each time which inturns calls DBConnection() each time or I should reuse one connection for all the queries. Thanks in advance.
private void testDataBase(String query){
Connection con = DBConnection();
Statement st = null;
ResultSet rs = null;
try {
st = con.createStatement();
rs = st.executeQuery(query);
boolean flag = true;
while (rs.next()) {
String resultString = "";
for(int i = 1; i <=rs.getMetaData().getColumnCount();i++){
resultString=resultString+" "+ rs.getString(i);
}
System.out.println(resultString);
}
} catch (SQLException e) {
e.printStackTrace();
} finally {
if (st != null) {
try {
st.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (con != null) {
try {
con.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
private Connection DBConnection() {
final String method_name = "DBConnection";
Connection conn = null;
try{
Class.forName(driver).newInstance();
conn = java.sql.DriverManager.getConnection(url,userName,password);
}catch (ClassNotFoundException e) {
System.out.println(e.getMessage());
} catch (SQLException e) {
System.out.println(e.getMessage());
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return conn;
}

Opening a DB connection is an expensive operation in terms of perfofmance. You should use a ConnectionPool for sharing connections among different requests.

Connections are not thread safe, so sharing them across requests is not a good idea.
A better idea is to pool connections and keep their scope as narrow as possible: check the connection out of the pool, use it, close it in transaction scope.

Database connections are long-running and should be re-used, unless you have a very low query rate.

Getting a database connection is quite an expensive operation, so it is advisable to re-use a connection if possible. Consider also using connection pooling, which will maintain a number of connections for you, so you can just grab one from the pool when needed. The method shown above might not need to change, it depends on the DBConnection() method you call.

I completely agree with #Amir Kost, in terms of performances, opening a DB connection in one of the slowest operation that you can do, and if you have restrictive real time constraints it could be a big issue.
I do not know if you are using a framework or not, but a good practice is to publish a bean which wrap a pool of connection and every time that you need to interact directly with the db, you get the current open connection (which usually corresponds to a so called "session").
I suggest to you, (even if you are not using any framework) to reproduce this technicality.

If you want only one instance of Connection, you can make use of the Singleton pattern, you can consider :
public class Connector {
private static final String URL = "jdbc:mysql://localhost/";
private static final String LOGIN = "root";
private static final String PASSWORD = "azerty";
private static final String DBNAME = "videotheque";
private static Connector connector;
private static Connection connection;
private Connector() {
}
public synchronized static Connector getInstance() {
if (connector == null) {
connector = new Connector();
}
return connector;
}
public static Connection getConnection() {
if (connection == null) {
Connection c = null;
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
c = DriverManager.getConnection(URL + DBNAME, LOGIN, PASSWORD);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return c;
}
return connection;
}
}
And then, you can call : Connector.getInstance().getConnection()

can't get connection to database through java class

i'm trying to establish connection with mysql database through file properties and then run the information from servlet. my Connection class looks like this:
public class pageDao {
private Connection connection;
private Statement statement;
private pageDao() {
Properties prop = new Properties();
try {
//Class.forName("oracle.jdbc.driver.OracleDriver");
//Class.forName("org.gjt.mm.mysql.Driver");
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException cnfe) {
System.out.println("Error loading driver: " +cnfe);
}
try {
try {
//load a properties file
prop.load(new FileInputStream("config.properties"));
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
String db = prop.getProperty("database");
String dbuser = prop.getProperty("dbuser");
String dbpassword = prop.getProperty("dbpassword");
connection = DriverManager.getConnection(db,dbuser,dbpassword);
} catch (SQLException e) {
e.printStackTrace();
}
}
private static pageDao thisDao;
public static pageDao gedDao()
{
if(thisDao == null)
thisDao = new pageDao();
return thisDao;
}
public PageData getPage(String id)
{
PageData data = new PageData();
try {
statement = connection.createStatement();
ResultSet rs = statement.executeQuery("select * from pages where id='"+id+"'");
if(rs.next())
{
data.setId(rs.getString("id"));
data.setParentid(rs.getString("parentid"));
data.setTitle(rs.getString("title"));
data.setTitle4menu(rs.getString("title4menu"));
data.setKeywords(rs.getString("keywords"));
data.setDescription(rs.getString("description"));
data.setMaintext(rs.getString("maintext"));
}
else
return null;
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return data;
}
when i run it, it doesn't show the mistake that connection wasn't established, but when it gets to the
public PageData getPage(String id) {
PageData data = new PageData();
try {
statement = connection.createStatement();
it throws java.lang.NullPointerException.
can anybody help me out with that?

there is no issue with code.
check your passing parameter ...
check sample
private Connection getConnection() {
try {
String dbUrl = "jdbc:mysql://localhost:3306/projectmining";
Class.forName("com.mysql.jdbc.Driver");
return DriverManager.getConnection(dbUrl, "root", "admin");
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}

return statement not working with try catch in DAO java spring

I have this function in userDAO
public List<Country> findAllCountries() {
try {
Session session = sessionFactory.getCurrentSession();
Query query = session.createQuery("FROM Country");
return query.list();
} catch (Exception e) {
System.out.println(e);
}
}
The problem is that eclipse is giving error that telling me to either use void in return or add return statement , if i remove try catch then it works

I'd write it this way. I don't know what you should be doing to close your Session or clean up in a finally block. This will make Eclipse happy:
public List<Country> findAllCountries()
{
List<Country> countries = new ArrayList<Country>();
try
{
Session session = sessionFactory.getCurrentSession();
Query query = session.createQuery("FROM Country");
countries = query.list();
}
catch (Exception e)
{
e.printStackTrace();
}
return countries;
}

You need to return something (or throw an Exception) from your catch block as well:
catch (Exception e) {
System.out.println(e);
return null;
}
catch (Exception e) {
System.out.println(e);
throw new RuntimeException("DAO failed", e);
}
Otherwise there will be a code path through the method without a return statement.