We Keep Coding

Should EntityManagerFactory be closed at application shutdown?

I have a Java application that has a GUI made with Swing and that uses two databases interchangeably. One of the two databases is mongoDB and the other one is MySQL. Which database to use is chosen with a command line option. For the MySQL database I am also using Hibernate and JPA. The code I have looks like this:
import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Option;
import java.awt.EventQueue;
import java.util.concurrent.Callable;
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
#Command(mixinStandardHelpOptions = true)
public class App implements Callable<Void> {
private static final Logger LOGGER = LogManager.getLogger(App.class);
#Option(names = { "--database" }, description = "'mongo' or 'mysql'")
private String databaseType = "mysql";
public static void main(String[] args) {
new CommandLine(new App()).execute(args);
}
#Override
public Void call() throws Exception {
EventQueue.invokeLater(() -> {
switch (databaseType) {
case "mysql":
EntityManagerFactory emf;
EntityManager entityManager;
try {
emf = Persistence.createEntityManagerFactory("name");
entityManager = emf.createEntityManager();
// other stuff
} catch (Exception e) {
LOGGER.log(Level.ERROR, "MySQL Exception", e);
}
break;
case "mongo":
// mongo stuff, no EntityManagerFactory here
break;
default:
LOGGER.log(Level.ERROR, "--database must be either 'mysql' or 'mongo'");
System.exit(1);
}
//...
try {
View view = new View();
view.setVisible(true);
} catch (Exception e) {
LOGGER.log(Level.ERROR, "Exception", e);
}
});
return null;
}
In mysql case I am creating an EntityManagerFactory and an EntityManager. The entityManager created here is passed as argument to the constructor of the repositories and used throughout the whole life of the application.
I was wondering what is the best practice about closing the entityManager and the factory.
Searching in the documentation I found this:
Closing an EntityManagerFactory should not be taken lightly. It is
much better to keep a factory open for a long period of time than to
repeatedly create and close new factories. Thus, most applications
will never close the factory, or only close it when the application is
exiting.
So I was wondering, what is the difference between closing the factory and entity manager at application shutdown and not closing it? Also in my case I'm declaring emf and entityManager inside the mysql case since are not required for mongodb. In order to close them at application shutdown what should I do? I found something about Runtime.getRuntime().addShutdownHook(). I tried using it like the code below, but it seems like it is not working.
try {
emf = Persistence.createEntityManagerFactory("name");
entityManager = emf.createEntityManager();
Thread closeHook = new Thread(() -> {
if (emf != null) {
entityManager.close();
emf.close();
LOGGER.log(Level.INFO, "Close entity manager and entity manager factory");
}
});
Runtime.getRuntime().addShutdownHook(closeHook);
// other stuff
} catch (Exception e) {
LOGGER.log(Level.ERROR, "MySQL Exception", e);
}

Short answer, yes, it should be closed. And the reason can be found at this answer:
The JVM will release all active resources upon termination; however, this does not ensure that the other end will free the resource too, so explicitly closing resources is in every programmer's best interest.
So in my case, it is true that the EntityManager and factory are closed at application shutdown, but this does not ensure that they are properly dealt with on the other end.
I didn't mention it in my question, but in fact the same thing holds true for the Mongo Client as well (see this answer):
If you ever re-deploy your web application without first restarting your application server, you must ensure that the MongoClient is closed when your web application is shutdown.
About the implementation I made an interface that I called DBInitializer. I instantiated an object of type MongoInitializer or MySQLInitializer (both implementing DBInitializer) inside the main method. See code for more clarity.
DBInitializer:
public interface DBInitializer {
public void startDbConnection();
public void closeDbConnection();
}
MySQLInitializer:
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
public class MySQLInitializer implements DBInitializer {
private EntityManagerFactory emf;
private EntityManager entityManager;
private final Logger logger = LogManager.getLogger(MySQLInitializer.class);
#Override
public void startDbConnection() {
try {
emf = Persistence.createEntityManagerFactory("name");
entityManager = emf.createEntityManager();
// other stuff
} catch (Exception e) {
logger.log(Level.ERROR, "MySQL Exception", e);
}
}
#Override
public void closeDbConnection() {
if (emf != null) {
entityManager.close();
emf.close();
}
}
}
MongoInitializer:
import com.mongodb.MongoClient;
import com.mongodb.ServerAddress;
public class MongoInitializer implements DBInitializer {
private MongoClient client;
private final Logger logger = LogManager.getLogger(MongoInitializer.class);
#Override
public void startDbConnection() {
try {
client = new MongoClient(new ServerAddress("localhost", 27017));
// other stuff
} catch (Exception e) {
logger.log(Level.ERROR, "Mongo Exception", e);
}
}
#Override
public void closeDbConnection() {
client.close();
}
}
App:
import java.awt.EventQueue;
import java.util.concurrent.Callable;
import DBInitializer;
import MongoInitializer;
import MySQLInitializer;
import View;
import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Option;
#Command(mixinStandardHelpOptions = true)
public class App implements Callable<Void> {
private static final Logger LOGGER = LogManager.getLogger(App.class);
#Option(names = { "--database" }, description = "Either 'mongo' or 'mysql'")
private String databaseType = "mysql";
public static void main(String[] args) {
new CommandLine(new App()).execute(args);
}
DBInitializer dBInitializer;
#Override
public Void call() throws Exception {
EventQueue.invokeLater(() -> {
try {
switch (databaseType) {
case "mysql":
dBInitializer = new MySQLInitializer();
break;
case "mongo":
dBInitializer = new MongoInitializer();
break;
default:
LOGGER.log(Level.ERROR, "--database must be either 'mysql' or 'mongo'");
System.exit(1);
}
dBInitializer.startDbConnection();
// other stuff
View view = new View();
view.setVisible(true);
} catch (Exception e) {
LOGGER.log(Level.ERROR, "Exception", e);
}
});
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
dBInitializer.closeDbConnection();
}
});
return null;
}
}

what is the difference between closing the factory and entity manager at application shutdown and not closing it?
A potential resource leak (e.g. an unclosed connection pool) vs lack thereof. Also:
I tried using it like the code below, but it seems like it is not working.
Why not use a try-with-resources statement?

Creating SQLite database in intellij

I have to create a SQLite database that I will ultimately have to insert lines of data from a parsed .csv file. The issue that I'm having is creating the SQLite database in the intellij idea. Using the code below I continue to get the "No suitable driver" error. I believe that I've added the classpath correctly by using Project Structure-> Modules-> Add then selecting my downloaded SQLite .jar file, which in this case is sqlite-jdbc-3.30.1 (1).jar
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.SQLException;
public class mint {
public static void createNewDatabase(String fileName) {
String url = "C:\\SQLite\\sqlite-jdbc-3.30.1 (1).jar" + fileName;
try {
Connection conn = DriverManager.getConnection(url);
if (conn != null) {
DatabaseMetaData meta = conn.getMetaData();
System.out.println("The driver name is " + meta.getDriverName());
System.out.println("A new database has been created.");
}
} catch (SQLException e) {
System.out.println(e.getMessage());
System.out.println("An error occurred while connecting MySQL database");
}
}
public static void main(String[] args) {
createNewDatabase("input-filename.db");
}
}
Screenshot of output

You need to read this tutorial - URL should be like this
String url = "jdbc:sqlite:C:/sqlite/" + fileName;

Using neo4j-jdbc open-connection is bottleneck

I am using neo4-jdbc with pool lib BasicDataSource.
I had huge latency problems so we profiled the app and we found out that opening connection is the cause. I didnt understand why open-connection takes so long we using pool. this is screenshot from our profiles:
This is how the Neo4jDatasourceRemote looks like:
package com.comp.wm.common.repo;
import com.comp.wm.common.utils.Constants;
import org.apache.commons.dbcp.BasicDataSource;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Value;
import javax.annotation.PostConstruct;
import java.sql.Connection;
import java.sql.SQLException;
private final Logger logger = Logger.getLogger(Neo4jDataSourceRemote.class);
private BasicDataSource ds;
#Value("${neo4j.host:localhost}")
private String NEO4J_HOST;
#Value("${neo4j.port:7474}")
private String NEO4J_PORT;
#Value("${neo4j.username:nouser}")
private String NEO4J_USERNAME;
#Value("${neo4j.password:nopass}")
private String NEO4J_PASSWORD;
#Value("${neo4j.pool.size:200}")
private int NEO4J_POOL_SIZE;
private String GetUrl() {
return String.format(Constants.NEO4J_JDBC_CONNECTIVITY_STRING, NEO4J_HOST, NEO4J_PORT);
}
#PostConstruct
public void init(){
ds = new BasicDataSource();
ds.setInitialSize(300);
ds.setDriverClassName("org.neo4j.jdbc.Driver");
ds.setUrl(GetUrl());
ds.setUsername(NEO4J_USERNAME);
ds.setPassword(NEO4J_PASSWORD);
}
#Override
public Connection openConnection() throws SQLException {
return this.ds.getConnection();
}
#Override
public void closeConnection(Connection conn) {
try {
if (conn != null)
conn.close();
} catch (SQLException ex) {
logger.info("error closing connection", ex);
}
}
}
and this is a sample of how I execute query against the graph:
public List<NearbyItem> executeQuery(..) {
conn = neo4jDataSource.openConnection();
String getUsersStatement = "some query..";
try (PreparedStatement ps = conn.prepareStatement(getUsersStatement)) {
..
ResultSet rs = ps.executeQuery();
while (rs.next()) {
...
}
} catch (Exception e) {
throw new RuntimeException("Error returning userId=" + userIdInput, e);
} finally {
neo4jDataSource.closeConnection(conn);
}
return distItemDatas;
}
any ideas?

Based on comments above, I'll add this as a reply.
By default Neo4j runs in the http interface 10 threads for core. You can tweak the total number of threads in neo4j-server.properties
org.neo4j.server.webserver.maxthreads=200
However the more threads you have the more you're suffering from context switches and lock contention. If you increase the number of threads I won't expect a large increase of throughput, you just shift the point where you have to wait. From initialization (openCOnnection) to processing the query.

Connection using ConnectionPoolDataSource

To connect my required Database. I am planning to use ConnectionPoolDataSource class. But How can i set the details regarding the database name(to which i want it to be connected) on using this instance. Please Help on this occassion.

Try to read this documentation and example
EDIT
just modified example from above links
prepared steps:
- download MySQL Server
- download mySQL java driver
- download Apache Commons Pool
- download Commons DBCP
- Open MySQL Client like MySQL Workbench and create DB using next script
delimiter $$
CREATE DATABASE `test_stackoverflow` /*!40100 DEFAULT CHARACTER SET utf8 */$$
delimiter $$
CREATE TABLE `test_table` (
`idtest_table` int(11) NOT NULL,
`test_field` varchar(45) DEFAULT NULL,
PRIMARY KEY (`idtest_table`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8$$
INSERT INTO `test_stackoverflow`.`test_table` (`idtest_table`, `test_field`) VALUES (1, 'test1');
INSERT INTO `test_stackoverflow`.`test_table` (`idtest_table`, `test_field`) VALUES (2, 'test2');
create java project, add to class path , myscl connector, pool and dbcp (you just download all these jars)
add next classes
import org.apache.commons.dbcp.cpdsadapter.DriverAdapterCPDS;
import org.apache.commons.dbcp.datasources.SharedPoolDataSource;
import javax.sql.DataSource;
import java.sql.Connection;
import java.sql.SQLException;
/**
* #author Sergii.Zagriichuk
*/
public class Pool {
private static DataSource ds;
static {
DriverAdapterCPDS cpds = new DriverAdapterCPDS();
try {
cpds.setDriver("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
cpds.setUrl("jdbc:mysql://localhost:3306/test_stackoverflow");
cpds.setUser("root");
cpds.setPassword("root");
SharedPoolDataSource tds = new SharedPoolDataSource();
tds.setConnectionPoolDataSource(cpds);
tds.setMaxActive(10);
tds.setMaxWait(50);
ds = tds;
}
public static Connection getConnection() throws SQLException {
return ds.getConnection();
}
}
user name and pass should be changed to your db user/password
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class MainClass {
public static void main(String[] args) {
Connection connection = null;
Statement statement = null;
ResultSet resultSet = null;
try {
connection = Pool.getConnection();
// Do work with connection
statement = connection.createStatement();
String selectEmployeesSQL = "select * from test_table";
resultSet = statement.executeQuery(selectEmployeesSQL);
while (resultSet.next()) {
printTestTable(resultSet);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (resultSet != null) {
try {
resultSet.close();
} catch (SQLException e) {
} // nothing we can do
}
if (statement != null) {
try {
statement.close();
} catch (SQLException e) {
} // nothing we can do
}
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
} // nothing we can do
}
}
}
private static void printTestTable(ResultSet resultSet) throws SQLException {
System.out.print(resultSet.getInt("idtest_table")+", ");
System.out.print(resultSet.getString("test_field"));
}
}
Just run main method and you will see printed test values to console!!!

You could use an instance of DriverAdapterCPDS. This will need to add two libraries, Apache Commons Pool and Apache Commons DBCP. It is very useful when the driver you use does not include an implementation of connection pooling.
You can find a example in http://massapi.com/class/dr/DriverAdapterCPDS.html

SQLException: no such table

now I got some trouble connecting to my database. I know the tables i am looking for exist because when I access them with the command line they can be queried.
Probably some minor oversight but I would love some help.
This is where I make my connection to my database
package persistence;
import java.sql.DriverManager;
import java.sql.Connection;
import java.sql.SQLException;
public class DBRegistry {
private static DBRegistry db = null;
private static Connection connection = null;
private DBRegistry() {};
public static synchronized DBRegistry getUniqueInstance() {
if (db == null) {
db = new DBRegistry();
return db;
}
else return db;
}
public synchronized Connection getDBConnection() {
try {
Class.forName("org.sqlite.JDBC");
connection = DriverManager.getConnection("jdbc:sqlite:src/database/taskMan.db");
return connection;
}
catch (SQLException e) {e.printStackTrace();}
catch (ClassNotFoundException e) {e.printStackTrace();}
return null;
}
public synchronized void closeConnection() {
try {
connection.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
Here is how I query it
public void create(UUID oid, Object obj) {
Task t = (Task)obj;
String statement = "INSERT INTO `complexTask` (`oid`,`description`,`status`) VALUES (?, ?, ?)";
try {
PreparedStatement dbStatement = db.prepareStatement(statement);
dbStatement.setString(1, oid.toString());
dbStatement.setString(2, t.getDescription());
dbStatement.setBoolean(3, t.getStatus());
dbStatement.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
}
and finally a stack trace:
java.sql.SQLException: no such table: complexTask
at org.sqlite.DB.throwex(DB.java:288)
at org.sqlite.NativeDB.prepare(Native Method)
at org.sqlite.DB.prepare(DB.java:114)
at org.sqlite.PrepStmt.<init>(PrepStmt.java:37)
at org.sqlite.Conn.prepareStatement(Conn.java:231)
at org.sqlite.Conn.prepareStatement(Conn.java:224)
at org.sqlite.Conn.prepareStatement(Conn.java:213)
at persistence.framework.ComplexTaskRDBMapper.create(ComplexTaskRDBMapper.java:23)
at persistence.PersistanceFacade.create(PersistanceFacade.java:49)
at persistence.persistanceStates.NewState.commit(NewState.java:10)
at persistence.PersistentObject.commit(PersistentObject.java:23)
at domain.objects.Task.commitToDB(Task.java:89)
at domain.TaskRepository.commitToDB(TaskRepository.java:60)
at domain.TaskController.persistanceCommit(TaskController.java:97)
at presentation.TaskControlsJPanel$3.actionPerformed(TaskControlsJPanel.java:127)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2012)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2335)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:404)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:253)
at java.awt.Component.processMouseEvent(Component.java:6175)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
at java.awt.Component.processEvent(Component.java:5940)
at java.awt.Container.processEvent(Container.java:2105)
at java.awt.Component.dispatchEventImpl(Component.java:4536)
at java.awt.Container.dispatchEventImpl(Container.java:2163)
at java.awt.Component.dispatchEvent(Component.java:4362)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4461)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4125)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4055)
at java.awt.Container.dispatchEventImpl(Container.java:2149)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4362)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:604)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:275)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:200)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:190)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:185)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:177)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:138)
And some JUnit code for good measure, the first test passes and the second fails with a similar error to the one above
package test.persistence;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import persistence.DBRegistry;
import junit.framework.TestCase;
public class TestDBRegistry extends TestCase {
public void testDBRegistryConnection() {
Connection con = DBRegistry.getUniqueInstance().getDBConnection();
assertNotNull(con);
}
public void testTableQuery() throws SQLException {
Connection con = DBRegistry.getUniqueInstance().getDBConnection();
PreparedStatement dbStatement = con.prepareStatement("SELECT COUNT(*) FROM `singleTask`");
assertEquals("should be 1 for successful query", 1, dbStatement.executeQuery());
}

I notice that in both your unit test and your other code you are using back-ticks around the table name. In the latest version of sqlite, this is fine but in older versions it wasn't handled as well I believe. Can you try removing the ticks around the table name or maybe changing them to regular quotes rather than back-ticks?
If that doesn't solve it I would check to be absolutely sure that you are pointing to the correct db file. If you specify a filename that doesn't exist you won't get an error, it will simply create a new database there. I'm not sure what the "current directory" is under the context of your app or unit test but be sure it is pointing to where you think it is. To test this, you could change the db file name to foo.db, run the unit test, then search your machine for foo.db to see where it got created. That will tell you where your app is working off of.

I'm not sure your JDBC connection string is quite right. The connection string you use ends with taskMan.db, but your comment above implies that the name of the database file is taskManDb.db (note the extra Db).

It might not fix your problem with the SQL INSERT, but I don't care at all for your DBRegistry implementation. I'd write it like this:
package persistence;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class DBRegistry
{
public static Connection getConnection(String driver, String url, String username, String password) throws ClassNotFoundException, SQLException
{
Class.forName(driver);
return DriverManager.getConnection(url, username, password);
}
public static void close(Connection connection)
{
try
{
if (connection != null)
{
connection.close();
}
}
catch (SQLException e)
{
e.printStackTrace();
}
}
public static void close(Statement statement)
{
try
{
if (statement != null)
{
statement.close();
}
}
catch (SQLException e)
{
e.printStackTrace();
}
}
public static void close(ResultSet resultSet)
{
try
{
if (resultSet != null)
{
resultSet.close();
}
}
catch (SQLException e)
{
e.printStackTrace();
}
}
}

Maybe your table is in a package. so you might have do something like
select * from Tasks.ComplextTask
('Tasks' being the package)

Now just try to find a file with the same name in c:\windows\system32 and you will find it. It tells to us that your path is not correct. Have a nice day;

Please Commit your Connection after created table and after insertion.