We Keep Coding

JDBC helper that reduces boilerplate code

I am using the c3p0 library as my datasource object.
I want to create a JDBC helper class that helps reduce the boilerplate code that JDBC has and I am wondering if my implementation is correct and are following best practices? Also, if there is an already existing library that provides these functionalities, like QueryRunner, maybe?
Most of my queries returns a list of results of a specified column. Will it be okay if I use the following helper method for all my queries?
public List<String> retrieveSQLQuery(String sqlQuery, String column) {
List<String> values = new ArrayList<>();
try (Connection conn = getConnection();
PreparedStatement statement = conn.prepareStatement(sqlQuery);
ResultSet rs = statement.executeQuery(sqlQuery)) {
while (rs.next()) {
values.add(rs.getString(column));
}
} catch (SQLException e) {
e.printStackTrace();
}
return values;
}
The getConnection() method lives in a JDBCUtil class which provides the connection to the datasource object. This helper class will be extending JDBCUtil thus why it has access to that method.
I also know that frameworks like spring and Hibernate provide utilities, however, those frameworks are too large for my project.

How to identify non thread-safe code in a multi-threaded environment?

I have designed and implemented a simple webstore based on traditional MVC Model 1 architecture using pure JSP and JavaBeans (Yes, I still use that legacy technology in my pet projects ;)).
I am using DAO design pattern to implement my persistence layer for a webstore. But I am not sure if I have implemented the classes correctly in my DAO layer. I am specifically concerned about the QueryExecutor.java and DataPopulator.java classes (mentioned below). All the methods in both these classes are defined as static which makes me think if this is the correct approach in multithreaded environment. Hence, I have following questions regarding the static methods.
Will there be synchronization issues when multiple users are trying to do a checkout with different products? If answer to the above question is yes, then how can I actually reproduce this synchronization issue?
Are there any testing/tracing tools available which will actually show that a specific piece of code will/might create synchronization issues in a multithreaded environment? Can I see that a User1 was trying to access Product-101 but was displayed Product-202 because of non thread-safe code?
Assuming there are synchronization issues; Should these methods be made non-static and classes instantitable so that we can create an instance using new operator OR Should a synchronized block be placed around the non thread-safe code?
Please guide.
MasterDao.java
public interface MasterDao {
Product getProduct(int productId) throws SQLException;
}
BaseDao.java
public abstract class BaseDao {
protected DataSource dataSource;
public BaseDao(DataSource dataSource) {
this.dataSource = dataSource;
}
}
MasterDaoImpl.java
public class MasterDaoImpl extends BaseDao implements MasterDao {
private static final Logger LOG = Logger.getLogger(MasterDaoImpl.class);
public MasterDaoImpl(DataSource dataSource) {
super(dataSource);
}
#Override
public Product getProduct(int productId) throws SQLException {
Product product = null;
String sql = "select * from products where product_id= " + productId;
//STATIC METHOD CALL HERE, COULD THIS POSE A SYNCHRONIZATION ISSUE ??????
List<Product> products = QueryExecutor.executeProductsQuery(dataSource.getConnection(), sql);
if (!GenericUtils.isListEmpty(products)) {
product = products.get(0);
}
return product;
}
}
QueryExecutor.java
public final class QueryExecutor {
private static final Logger LOG = Logger.getLogger(QueryExecutor.class);
//SO CANNOT NEW AN INSTANCE
private QueryExecutor() {
}
static List<Product> executeProductsQuery(Connection cn, String sql) {
Statement stmt = null;
ResultSet rs = null;
List<Product> al = new ArrayList<>();
LOG.debug(sql);
try {
stmt = cn.createStatement();
rs = stmt.executeQuery(sql);
while (rs != null && rs.next()) {
//STATIC METHOD CALL HERE, COULD THIS POSE A SYNCHRONIZATION ISSUE ???????
Product p = DataPopulator.populateProduct(rs);
al.add(p);
}
LOG.debug("al.size() = " + al.size());
return al;
} catch (Exception ex) {
LOG.error("Exception while executing products query....", ex);
return null;
} finally {
try {
if (rs != null) {
rs.close();
}
if (stmt != null) {
stmt.close();
}
if (cn != null) {
cn.close();
}
} catch (Exception ex) {
LOG.error("Exception while closing DB resources rs, stmt or cn.......", ex);
}
}
}
}
DataPopulator.java
public class DataPopulator {
private static final Logger LOG = Logger.getLogger(DataPopulator.class);
//SO CANNOT NEW AN INSTANCE
private DataPopulator() {
}
//STATIC METHOD DEFINED HERE, COULD THIS POSE A SYNCHRONIZATION ISSUE FOR THE CALLING METHODS ???????
public static Product populateProduct(ResultSet rs) throws SQLException {
String productId = GenericUtils.nullToEmptyString(rs.getString("PRODUCT_ID"));
String name = GenericUtils.nullToEmptyString(rs.getString("NAME"));
String image = GenericUtils.nullToEmptyString(rs.getString("IMAGE"));
String listPrice = GenericUtils.nullToEmptyString(rs.getString("LIST_PRICE"));
Product product = new Product(new Integer(productId), name, image, new BigDecimal(listPrice));
LOG.debug("product = " + product);
return product;
}
}

Your code is thread-safe.
The reason, and the key to thread-safety, is your (static) methods do not maintain state. ie your methods only use local variables (not fields).
It doesn't matter if the methods are static or not.

Assuming there are synchronization issues; Should these methods be made non-static and classes instantitable so that we can create an instance using new operator
This won't help. Multiple threads can do as they please with a single object just as they can with a static method, and you will run into synchronization issues.
OR Should a synchronized block be placed around the non thread-safe code?
Yes this is the safe way. Any code inside of a synchronized block is guaranteed to have at most one thread in it for any given time.
Looking through your code, I don't see many data structures that could possibly be shared amongst threads. Assuming you had something like
public final class QueryExecutor {
int numQueries = 0;
public void doQuery() {
numQueries++;
}
}
Then you run into trouble because 4 threads could have executed doQuery at the same moment, and so you have 4 threads modifying the value of numQueries - a big problem.
However with your code, the only shared class fields is the logging class, which will have it's own thread-safe synchronization built in - therefore the code you have provided looks good.

There is no state within your code (no mutable member variables or fields, for example), so Java synchronisation is irrelevant.
Also as far as I can tell there are no database creates, updates, or deletes, so there's no issue there either.
There's some questionable practice, for sure (e.g. the non-management of the database Connection object, the wide scope of some variables, not to mention the statics), but nothing wrong as such.
As for how you would test, or determine thread-safety, you could do worse than operate your site manually using two different browsers side-by-side. Or create a shell script that performs automated HTTP requests using curl. Or create a WebDriver test that runs multiple sessions across a variety of real browsers and checks that the expected products are visible under all scenarios...

Java and MySQL: More than 'max_user_connections' exception

For university, it is my excercise to develop a multiplayer game with Java. The communication between the clients shall not be handled with sockets or the like, but with the help of a MySQL database where the clients are adding their steps in the game. Because it is a game of dice, not a lot of queries are needed. (approximiately 30 queries per gaming session are needed).
I never used MySQL in connection with Java before, so this maybe is a beginner's fault. But actually, I often get an exception during the execution of my java project.
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: User my_username already has more than 'max_user_connections' active connections
My queries are executed in a DatabaseHelper.java class. The results are returned and evaluated in another class of the project. Since I use an MVC pattern, I evaluate the results in a controller or model class.
This for example is one of my quers in the DatabaseHelper.java class. The other queries are similar:
private static Connection conn;
private Connection getConn() {
return conn;
}
public void db_connect() throws ClassNotFoundException, SQLException{
// JDBC Klassen laden
Class.forName(dbClassName);
// Verbindungsversuch auf 5 Sekunden setzen
DriverManager.setLoginTimeout(5);
this.setConn(DriverManager.getConnection(CONNECTION,p)); // p contains the username and the database
}
public void db_close(){
try {
this.getConn().close();
} catch (SQLException e) {
if(GLOBALVARS.DEBUG)
e.printStackTrace();
}
}
public String[] query_myHighscores(int gameid, PlayerModel p) throws SQLException{
List<String> rowValues = new ArrayList<String>();
PreparedStatement stmnt;
if(gameid == GLOBALVARS.DRAGRACE)
stmnt = this.getConn().prepareStatement("SELECT score FROM highscore WHERE gid = ? and pname = ? ORDER BY score ASC LIMIT 0,3");
else
stmnt = this.getConn().prepareStatement("SELECT score FROM highscore WHERE gid = ? and pname = ? ORDER BY score DESC LIMIT 0,3");
stmnt.setInt(1, gameid);
stmnt.setString(2, p.getUname());
ResultSet rs = stmnt.executeQuery();
rs.beforeFirst();
while(rs.next()){
rowValues.add(rs.getString(1));
}
stmnt.close();
rs.close();
return (String[])rowValues.toArray(new String[rowValues.size()]);
}
The CONNECTION string is a string which looks like jdbc:mysql://my_server/my_database
In the HighscoreGUI.java class, I request the data like this:
private void actualizeHighscores(){
DatabaseHelper db = new DatabaseHelper();
try{
db.db_connect();
String[] myScoreDragrace = db.query_myHighscores(GLOBALVARS.GAME1); // id of the game as parameter
// using the string
} finally {
db.db_close();
}
So I tried:
Closing the statement and the ResultSet after each query
Used db_close() to close the connection to the dabase in the finally-block
Never returning a ResultSet (found out this may become a performance leak)
The stacktrace leads in the DatabaseHelper.java class to the line
this.setConn(DriverManager.getConnection(CONNECTION,p));
But I cannot find my mistake why I still get this exception.
I cannot change every settings for the database since this is a shared host. So I'd prefer a solution on Java side.

The problem is that you exceed your allowed set of connections to that database. Most likely this limit is exactly or very close to "1". So as soon as you request your second connection your program crashes.
You can solve this by using a connection pooling system like commons-dbcp.
That is the recommended way of doing it and the other solution below is only if you may not use external resources.
If you are prohibited in the external code that you might use with your solution you can do this:
Create a "Database" class. This class and only this class ever connects to the DB and it does so only once per program run. You set it up, it connects to the database and then all the queries are created and run through this class, in Java we call this construct a "singleton". It usually has a private constructor and a public static method that returns the one and only instance of itself. You keep this connection up through the entire livetime of your program and only reactivate it if it gets stall. Basically you implement a "Connection Pool" for the specific case of the pool size "1".
public class Database {
private static final Database INSTANCE = new Database();
private Database() {}
public static Database getInstance() {
return INSTANCE;
}
// add your methods here.
}
When the program terminates, close the Connection (using a shutdown hook).

Database Connection Management

I am in the process of designing a simple Java application which deals with insert/delete/update of records in a MySQL database using JDBC. I have a class Member which deals with the member's details.
class Member {
... // Private Members
... // Accessors
}
..and I have a handler to deal with the member records
class MemberHandler {
public MemberHandler(){...}
public void addMember(Member mem){...}
public void removeMember(Member mem){...}
public Member[] getMembers(){...}
}
All I am worried about is the method in which I establish connection to the database and disconnect. I can do it in two ways -
Method 1:
I can have a member in MemberHandler Connection conn, establish connection while instantiating the class and close the connection when the object is no more needed. Here I would have a connection per object and I need not establish a connection whenever I need to do any database related activity. In this case, the disadvantage seems to be - when there is a loss in network connection, conn could become invalid.
class MemberHandler {
private java.sql.Connection conn;
... // Other members
private void createConnection(){/*creates the connection*/}
private void closeConnection(){conn.close(); /*called when conn is no more needed*/}
}
Method 2:
I can establish a connection when needed and close it when I am done with the activity. Disadvantage: everytime, I need to establish a connection and close it. For eg.,
...
...
private void addMember() {
//establish connection
//update database
//close connection
}
...
...
Which of these two ways seems to be better? Or is there a third better way?
Thanks!

I would suggest you to go for connection-pooling
c3po

Java: Prepare a statement without a connection

I'm trying to generate some sql files in my java application.
The application will not execute any sql statements, just generate a file with sql statements and save it.
I'd like to use the java.sql.PreparedStatement to create my statements so that i don't have to validate every string etc. with my own methods.
Is there a way to use the PreparedStatement without the calling java.sql.Connection.prepareStatement(String) function, because I don't have a java.sql.Connection?

Take a look at this Java library: http://openhms.sourceforge.net/sqlbuilder/

I'm guessing that until you've got a sql connection, the parser won't know what rules to apply. I'm guessing that it's actually the SQL driver or even server that's compiling the sql statement.
Assuming your sql is simple enough, then how about using a cheap connection, like, say a sqlite connection.
SQLite will create a new database on the fly if the database you're attempting to connect to does not exist.
public Connection connectToDatabase() {
// connect to the database (creates new if not found)
try {
Class.forName("org.sqlite.JDBC");
conn = DriverManager.getConnection("jdbc:sqlite:mydatabase.db");
// initialise the tables if necessary
this.createDatabase(conn);
}
catch (java.lang.ClassNotFoundException e) {
System.out.println(e.getMessage());
}
catch (java.sql.SQLException e) {
System.out.println(e.getMessage());
}
return conn;
}

Not really. Preparing a statement in most cases means that it will be compiled by DBMS which is "hard" without connection.
http://java.sun.com/docs/books/tutorial/jdbc/basics/prepared.html

This is a dastardly devious problem, thankfully it's pretty easy to cope with:
public class PreparedStatementBuilder
{
private String sql; // the sql to be executed
public PreparedStatementBuilder(final String sql) { this.sql = sql; }
protected void preparePrepared(final PreparedStatement preparedStatement)
throws SQLException
{
// this virtual method lets us declare how, when we do generate our
// PreparedStatement, we want it to be setup.
// note that at the time this method is overridden, the
// PreparedStatement has not yet been created.
}
public PreparedStatement build(final Connection conn)
throws SQLException
{
// fetch the PreparedStatement
final PreparedStatement returnable = conn.prepareStatement(sql);
// perform our setup directives
preparePrepared(returnable);
return returnable;
}
}
To use, just write an anonymous class that overrides void preparePrepared(PreparedStatement):
final String sql = "SELECT * FROM FOO WHERE USER = ?";
PreparedStatementBuilder psBuilder = new PreparedStatementBuilder(sql){
#Override
protected void preparePrepared(PreparedStatement preparedStatement)
throws SQLException
{
preparedStatement.setString(1, "randal");
}};
return obtainResultSet(psBuilder);
Presto! You now have a way to work with a PreparedStatement without yet having built it. Here's an example showing the minimal boilerplate you'd otherwise have to copy paste to kingdom come, every time you wanted to write a different statement:
public ResultSet obtainResultSet(final PreparedStatementBuilder builder)
throws SQLException {
final Connection conn = this.connectionSource.getConnection();
try
{
// your "virtual" preparePrepared is called here, doing the work
// you've laid out for your PreparedStatement now that it's time
// to actually build it.
return builder.build(conn).executeQuery();
}
finally
{
try { conn.close(); }
catch (SQLException e) { log.error("f7u12!", e); }
}
}
You really really don't want to be copy pasting that everywhere, do you?

Try implementing PreparedStatement.
Example : class YourOwnClass implements PreparedStatement {
// 1. Do implement all the methods ,
2. Get the minimal logic to implement from OraclePreparedStatement(classes12.jar) or
sun.jdbc.odbc.JdbcOdbcCallableStatement
}