Multithreaded write on MySQL DB with JDBC and c3p0 - java

I have written a DAO class which allows several threads invoked by ExecutorServices to write to MySQL DB.
EDIT: I am using c3p0 to create a JDBC ConnectionPool. So every new thread will get a new JDBC Connection by calling
DataBaseManager.getInstance().getConnection()
There seems to be random concurrency issue while executing, e.g:
java.sql.SQLException: No value specified for parameter 1
at com.eanurag.dao.DataBaseManager.writeData(DataBaseManager.java:102)
I am not able to understand all the issues with the code. Should I just synchronize entire writeData() ?
public class DataBaseManager {
private final static Logger logger = Logger.getLogger(DataBaseManager.class);
private static volatile DataBaseManager dbInstance = null;
private DataBaseManager() {
cpds = new ComboPooledDataSource();
try {
cpds.setDriverClass("com.mysql.jdbc.Driver");
} catch (PropertyVetoException e) {
logger.error("Error in Initializing DB Driver class", e);
}
cpds.setJdbcUrl("jdbc:mysql://" + DB_HOST + "/" + DB_NAME);
cpds.setUser(DB_USER);
cpds.setPassword(DB_PASS);
cpds.setMinPoolSize(MINIMUM_POOL_SIZE);
cpds.setAcquireIncrement(INCREMENT_SIZE);
cpds.setMaxPoolSize(MAXIMUM_POOL_SIZE);
cpds.setMaxStatements(MAX_STATEMENTS);
}
public static DataBaseManager getInstance() {
if (dbInstance == null) {
synchronized (WorkerManager.class) {
if (dbInstance == null) {
dbInstance = new DataBaseManager();
}
}
}
return dbInstance;
}
private ComboPooledDataSource cpds;
private static final Integer MINIMUM_POOL_SIZE = 10;
private static final Integer MAXIMUM_POOL_SIZE = 1000;
private static final Integer INCREMENT_SIZE = 5;
private static final Integer MAX_STATEMENTS = 200;
private volatile Connection connection = null;
private volatile Statement statement = null;
private volatile PreparedStatement preparedStatement = null;
private static final String DB_HOST = "localhost";
private static final String DB_PORT = "3306";
private static final String DB_USER = "root";
private static final String DB_PASS = "";
private static final String DB_NAME = "crawly";
private static final String URL_TABLE = "url";
public Connection getConnection() throws SQLException {
logger.info("Creating connection to DB!");
return this.cpds.getConnection();
}
public Boolean writeData(URL url) {
StringBuffer writeDBStatement = new StringBuffer();
writeDBStatement.append("insert into");
writeDBStatement.append(" ");
writeDBStatement.append(DB_NAME);
writeDBStatement.append(".");
writeDBStatement.append(URL_TABLE);
writeDBStatement.append(" ");
writeDBStatement.append("values (?,?,default)");
Boolean dbWriteResult = false;
try {
connection = DataBaseManager.getInstance().getConnection();
preparedStatement = connection.prepareStatement(writeDBStatement.toString());
preparedStatement.setString(1, url.getURL());
preparedStatement.setString(2, String.valueOf(url.hashCode()));
dbWriteResult = (preparedStatement.executeUpdate() == 1) ? true : false;
if(dbWriteResult){
logger.info("Successfully written to DB!");
}
} catch (SQLException e) {
logger.error("Error in writing to DB", e);
} finally {
try {
preparedStatement.close();
connection.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
return dbWriteResult;
}
}

The connection and preparedStatement variables must be local, not instance members.
No synchronization required.

What is happening here?
public Connection getConnection() throws SQLException {
logger.info("Creating connection to DB!");
return this.cpds.getConnection();
}
Namely, what does cpds.getConnection() do? When you call:
connection = DataBaseManager.getInstance().getConnection();
Your connection object is a member of what's supposed to be a singleton class here but every call to writeData() overwrites it it with a new getConnection() call. Is the getConnection() call thread unsafe as well?
Also, why is the connection object declared as a class member and then overwritten each time writeData() is called? In a multi-threaded environment, the code as it exists allows for the connection object to be overwritten by another getConnection() call immediately before prepareStatement() is called, since access to writeData() is not locked. Same for preparedStatement. Move those into the writeData() method.

Related

Connection closed error - MySQL , JDBC and Hikaricp

Have implemented a DAO pattern with Hikari connection pool
Made the best use of the DAO design pattern.
Have created singleton class which returns the object of a class with a public connection if the object of the singleton class is NULL> object is again created thereby getting the connection
queries to call the static method of the singleton class to obtain public connection and prepared statements are closed in try-with-resources.
application is live for more than 12hrs after few requests queries are not getting executed
"The connection is closed."
public class DatabaseConnection {
//Constants
String url = "jdbc:mysql://localhost:3306/";
String driver="com.mysql.jdbc.Driver";
String userName = "root";
String password = "#";
private static HikariDataSource dataSource;
public Connection conn;
public static DatabaseConnection db;
/**
* A static method which uses HikariDataSource Connection Pooling library to connect with the MySQL Database.
* Accepts and Sets few JDBC details like User Name,Password,URL,Driver Name and many more.
* #return This methods returns HikariDataSource
* #throws SQLException and CasbnException
*/
public DatabaseConnection() throws SQLException,CasbnExceptions {
try {
//create the object of HikariDataSource
dataSource=new HikariDataSource();
//set up the JDBC details (username,password,url,driver)
System.out.println("Inside DatabaseConnection constructor..");
dataSource.setDriverClassName(driver);
dataSource.setJdbcUrl(url);
dataSource.setUsername(userName);
dataSource.setPassword(password);
this.conn=dataSource.getConnection();
}
catch(Exception e)
{
e.printStackTrace();
}
}
public static DatabaseConnection getCon() throws CasbnExceptions, SQLException
{
if(db==null)
{
System.out.println("Inside getCon() and if block...");
db=new DatabaseConnection();
}
return db;
}
}
public class DAO_Implementation implements DAOFactory {
//Declare all the SQL queries as private static and final
//getCompany query
private static final String getCompanyName="SELECT company_name FROM #.company_register where email=?";
#Override
public String getCompanyName(String Email) throws SQLException {
try
(PreparedStatement pst = DatabaseConnection.getCon().conn
.prepareStatement(getCompanyName)){
pst.setString(1, Email);
ResultSet rs = pst.executeQuery();
rs.next();
Name=rs.getString(1);
}
catch(Exception e)
{
e.printStackTrace();
}
return Name;
}
#Override
public Pojos Individual_Company_detail(String CompanyName) throws SQLException{
try
(PreparedStatement pst = DatabaseConnection.getCon().conn
.prepareStatement(CompanyDetail);){
pst.setString(1, CompanyName);
ResultSet rs = pst.executeQuery();
if(rs.next()==true)
{
Pojos Po=new Pojos();
Po.setCompanyID(rs.getInt(1));
Po.setUserCount(rs.getInt(2));
Po.setPlan(rs.getString(3));
Po.setDateofSub(rs.getString(4));
Po.setSubscriptionID(rs.getString(5));
Po.setVaildTill(rs.getString(6));
return Po;
}
else
{
Pojos Po=new Pojos();
Po.setErrorMessage(CompanyName +" Has not registered to of our Plan and no Recent Transactions");
return Po;
}
}
}
Currently you are returning the same connection all the time using .conn
remove conn from DatabaseConnection and add a different method which getConnection from Hikari datasource every time
This way Hikari will handle connection pooling
public static Connection getConnection() {
return dataSource.getConnection();
}
And open Connection resource separately so it will be closed:
try
(Connection conn = DatabaseConnection.getConnection();
PreparedStatement pst = conn.prepareStatement(CompanyDetail))
public class DatabaseConnection {
String url = "jdbc:mysql://localhost:3306/";
//String dbName = ""
String driver="com.mysql.jdbc.Driver";
String userName = "root";
String password = "";
private static HikariDataSource dataSource;
public static DatabaseConnection db;
public DatabaseConnection() throws SQLException,CasbnExceptions {
try {
//create the object of HikariDataSource
dataSource=new HikariDataSource();
//set up the JDBC details (username,password,url,driver)
System.out.println("Inside DatabaseConnection constructor..");
dataSource.setDriverClassName(driver);
dataSource.setJdbcUrl(url);
dataSource.setUsername(userName);
dataSource.setPassword(password);
}
catch(Exception e)
{
e.printStackTrace();
}
}
public static Connection getConnection() throws SQLException
{
if(db==null)
{
db=new DatabaseConnection();
}
return dataSource.getConnection();
}
}

How can I get the values from the main method to be accessed by the getConnection method?

I am trying to get the values that the main method would have produced and use them in the getConnection method. However when I try to access the getConnection method, null values are being returned.
I want to use the ConnectionManager class to connect to the database.
Code below.
public class ConnectionManager {
public static String database;
public static String dbuser;
public static String dbpassword;
public static void main(String args[]) {
Properties prop = new Properties();
InputStream input = null;
try {
input = new FileInputStream("config.properties");
// load a properties file
prop.load(input);
database = prop.getProperty("database");
dbuser = prop.getProperty("dbuser");
dbpassword = prop.getProperty("dbpassword");
System.out.println(database);
} catch (IOException ex) {
ex.printStackTrace();
} finally {
if (input != null) {
try {
input.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
public static String url = "jdbc:mysql://localhost:3306/" + database;
private static String driverName = "com.mysql.jdbc.Driver";
private static String username = dbuser;
private static String password = dbpassword;
private static Connection con;
public static Connection getConnection() {
try {
Class.forName(driverName);
try {
con = DriverManager.getConnection(url, username, password);
} catch (SQLException ex) {
// log an exception. For example:
System.out.println("Failed to create the database connection.");
System.out.println(url + " " + username + " " + password);
}
} catch (ClassNotFoundException ex) {
System.out.println("Your driver has not been found.");
}
return con;
}
}
You just have to call the getConnection() method with parameters.
public static Connection getConnection(String url, String username, String password) {
/* Your code here */
}
And then, call this method.
Connection connection = getConnection(url, username, password);
Static c fields are initialized once when the class is loaded. The Connection fields are set to the ConnectionManager fields once, when they are still null.
To "fix" your problem, have your code in Connection use the fields in ConnectionManager:
con = DriverManager.getConnection(url, ConnectionManager.dbuser, ConnectionManager.dbpassword);
You're getting null argument values in DriverManager.getConnection(url, username, password) calling cause you have declared them as static fields.
So you initialize them as nulls before you read the particular values from the config.properties
Let's follow the flow:
Static initialization step:
private static String username = dbuser; //username==null; dbuser==null;
private static String password = dbpassword; //password==null; dbpassword==null
private static Connection con; //con==null;
Method main execution:
database = (prop.getProperty("database"));
dbuser = (prop.getProperty("dbuser")); //dbuser="smth"; username==null
dbpassword = (prop.getProperty("dbpassword")); //dbpassword ="smth"; password==null
Method getConnection execution:
con = DriverManager.getConnection(url, username, password); //username==null; //password==null;
P.S.: As it was previously said, it's better to use function with arguments like this:
public static Connection getConnection(String url, String username, String password) {
/* Your code here */
}

Proper way to connect to MySQL DB from Java

Which is the best way to handle a MySql connection with Java, via static context or Object one?
Static context:
public class MainClass {
public static void main(String[] args) {
Class.forName("com.mysql.jdbd.Driver");
Connection con = DBHandler.getConnection();
//Using connection
}
}
public class DBHandler {
public static Connection getConnection() {
final String URL = "";
final String USER = "";
final String PASS = "";
return DriverManager.getConnection(URL, USER, PASS);
}
}
Or creating an object:
public class MainClass {
public static void main(String[] args) {
Class.forName("com.mysql.jdbd.Driver");
Connection con = new DBHandler().getConnection();
//Using connection
}
}
public class DBHandler {
private final Connection con;
public DBHandler() {
final String URL = "";
final String USER = "";
final String PASS = "";
this.con = DriverManager.getConnection(URL, USER, PASS);
}
public Connection getConnection() {
return con;
}
}
I think creating an object in every class you need to use the connection is weird but maybe it's the best way to solve that.
Thank you.

c3p0 CombopooledDataSource not committing SQL update

I am using a pooled datasource (msaccess database) to update local database (client side using h2 database) via an application i made.
The problem i have is when submitting a request say "INSERT INTO USERS (NAME, CODE) VALUES (Me ,hfd5255fd4);" the app runs perfectly , nothing is reported in the error log BUT nothing has changed in the database either.
The code i am using is as follows
private static Connection getDatabase() throws Exception {
cpds.setDriverClass("net.ucanaccess.jdbc.UcanaccessDriver");
// loads the jdbc driver
cpds.setJdbcUrl("jdbc:ucanaccess://"
+ (new File("Ressources/filter.mdb").getAbsolutePath()));
cpds.setUser("admin");
cpds.setPassword("ibnsina");
cpds.setAutoCommitOnClose(false);
return cpds.getConnection(); //tried removing this , but no effect
}
----doing some other stuff---
private static updating() throws exception{
conn = getDatabase();
File fileUpload = new File(logPath + "UploadLog.txt");
BufferedReader readerUpload = new BufferedReader(new FileReader(
fileUpload));
String Uploadingline = "";
StringBuffer secondaryline = new StringBuffer();
if (readerUpload.ready()) {
System.out.println("Uploadtxt ready");
Statement stUpload = conn.createStatement();
System.out.println("Stupload ready");
while ((Uploadingline = readerUpload.readLine()) != null) {
if (Uploadingline.endsWith(";")) {
secondaryline.append(Uploadingline);
/*stUpload.executeUpdate(secondaryline.toString()); tried this to execute each line separatly*/
stUpload.addBatch(secondaryline.toString());
System.out.println("Reading line :" + secondaryline);
secondaryline.setLength(0);
} else {
secondaryline.append(Uploadingline);
}
}
stUpload.executeBatch();
stUpload.clearBatch();
conn.commit(); //i even tried adding this to make it commit even tho autocommit is by default ON
stUpload.close();}
You should not create a new DataSource for each connection, you need to create just one DataSource and use that to get Connections. Remember to close() them as that will return the connection to the pool.
You should do something like:
// There should only ever be one of these.
private static final DataSource ds = makeDataSource();
private static DataSource makeDataSource() {
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setDriverClass("net.ucanaccess.jdbc.UcanaccessDriver");
// loads the jdbc driver
cpds.setJdbcUrl("jdbc:ucanaccess://"
+ (new File("Ressources/filter.mdb").getAbsolutePath()));
cpds.setUser("admin");
cpds.setPassword("ibnsina");
cpds.setAutoCommitOnClose(false);
return cpds;
}
private static Connection getConnection () {
return ds.getConnection();
}
private static void releaseConnection (Connection conn) {
conn.commit();
conn.close();
}
private static void updating() {
Connection conn = getConnection();
try {
//...
} finally {
releaseConnection(conn);
}
}

If I use a singleton class for a database connection, can one user close the connection for everybody?

I wrote a singleton class for obtaining a database connection.
Now my question is this: assume that there are 100 users accessing the application. If one user closes the connection, for the other 99 users will the connection be closed or not?
This is my sample program which uses a singleton class for getting a database connection:
public class GetConnection {
private GetConnection() { }
public Connection getConnection() {
Context ctx = new InitialContext();
DataSource ds = ctx.lookup("jndifordbconc");
Connection con = ds.getConnection();
return con;
}
public static GetConnection getInstancetoGetConnection () {
// which gives GetConnection class instance to call getConnection() on this .
}
}
Please guide me.
As long as you don't return the same Connection instance on getConnection() call, then there's nothing to worry about. Every caller will then get its own instance. As far now you're creating a brand new connection on every getConnection() call and thus not returning some static or instance variable. So it's safe.
However, this approach is clumsy. It doesn't need to be a singleton. A helper/utility class is also perfectly fine. Or if you want a bit more abstraction, a connection manager returned by an abstract factory. I'd only change it to obtain the datasource just once during class initialization instead of everytime in getConnection(). It's the same instance everytime anyway. Keep it cheap. Here's a basic kickoff example:
public class Database {
private static DataSource dataSource;
static {
try {
dataSource = new InitialContext().lookup("jndifordbconc");
}
catch (NamingException e) {
throw new ExceptionInInitializerError("'jndifordbconc' not found in JNDI", e);
}
}
public static Connection getConnection() {
return dataSource.getConnection();
}
}
which is to be used as follows according the normal JDBC idiom.
public List<Entity> list() throws SQLException {
List<Entity> entities = new ArrayList<Entity>();
try (
Connection connection = Database.getConnection();
PreparedStatement statement = connection.prepareStatement("SELECT id, foo, bar FROM entity");
ResultSet resultSet = statement.executeQuery();
) {
while (resultSet.next()) {
Entity entity = new Entity();
entity.setId(resultSet.getLong("id"));
entity.setFoo(resultSet.getString("foo"));
entity.setBar(resultSet.getString("bar"));
entities.add(entity);
}
}
return entities;
}
See also:
Is it safe to use a static java.sql.Connection instance in a multithreaded system?
Below code is a working and tested Singleton Pattern for Java.
public class Database {
private static Database dbIsntance;
private static Connection con ;
private static Statement stmt;
private Database() {
// private constructor //
}
public static Database getInstance(){
if(dbIsntance==null){
dbIsntance= new Database();
}
return dbIsntance;
}
public Connection getConnection(){
if(con==null){
try {
String host = "jdbc:derby://localhost:1527/yourdatabasename";
String username = "yourusername";
String password = "yourpassword";
con = DriverManager.getConnection( host, username, password );
} catch (SQLException ex) {
Logger.getLogger(Database.class.getName()).log(Level.SEVERE, null, ex);
}
}
return con;
}
While getting Connection in any Class simply use below line
Connection con = Database.getInstance().getConnection();
Hope it may help :)
package es.sm2.conexion;
import java.sql.Connection;
import java.sql.DriverManager;
public class ConexionTest {
private static Connection conn = null;
static Connection getConnection() throws Exception {
if (conn == null) {
String url = "jdbc:mysql://localhost:3306/";
String dbName = "test";
String driver = "com.mysql.jdbc.Driver";
String userName = "userparatest";
String password = "userparatest";
Class.forName(driver).newInstance();
conn = DriverManager.getConnection(url + dbName, userName, password);
}
return conn;
}
}
To close Connection
public static void closeConnection(Connection conn) {
try {
conn.close();
} catch (SQLException e) {
}
}
To call to the connection:
package conexion.uno;
import java.sql.*;
import es.sm2.conexion.ConexionTest;
public class LLamadorConexion {
public void llamada() {
Connection conn = null;
PreparedStatement statement = null;
ResultSet resultado = null;
String query = "SELECT * FROM empleados";
try {
conn = ConexionTest.getConnection();
statement = conn.prepareStatement(query);
resultado = statement.executeQuery();
while (resultado.next()) {
System.out.println(resultado.getString(1) + "\t" + resultado.getString(2) + "\t" + resultado.getString(3) + "\t" );
}
}
catch (Exception e) {
System.err.println("El porque del cascar: " + e.getMessage());
}
finally {
ConexionTest.closeConnection(conn);
}
}
}
Great post, farhangdon! I, however, found it a little troublesome because once you close the connection, you have no other way to start a new one. A little trick will solve it though:
Replace if(con==null) with if(con==null || con.isClosed())
import java.sql.Connection;
import java.sql.DriverManager;
public class sql11 {
static Connection getConnection() throws Exception {
Class.forName("com.mysql.jdbc.Driver");
Connection c = DriverManager.getConnection("jdbc:mysql://localhost:3306/ics", "root", "077");
return c;
}
}

Categories