We Keep Coding

What is the right way to deal with the PreparedStatement in the Java program flow?

There are two methods in which the PreparedStatement is used.
The first method is called in the second method.
First method:
protected List<String> findResultsByMandantId(Long mandantId) {
List<String> resultIds = new ArrayList<>();
ResultSet rs;
String sql = "SELECT result_id FROM results WHERE mandant_id = ?";
PreparedStatement statement = getPreparedStatement(sql, false);
try {
statement.setLong(1, mandantId);
statement.execute();
rs = statement.getResultSet();
while (rs.next()) {
resultIds.add(rs.getString(1));
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
return resultIds;
}
Second method:
protected void findResultLineEntityToDelete(Long mandantId, String title, String context) {
List<String> resultIds = findResultsByMandantId(mandantId);
String [] resultIdsArr = resultIds.toArray(String[]::new);
ResultSet rs;
//String sql = "SELECT * FROM resultline WHERE result_id in (SELECT result_id FROM results WHERE mandant_id =" + mandantId + ")";
String sql = "SELECT * FROM resultline WHERE result_id in (" + String.join(", ", resultIdsArr)+ ")";
PreparedStatement statement = getPreparedStatement(sql, false);
try {
statement.execute();
rs = statement.getResultSet();
while (rs.next()) {
if (rs.getString(3).equals(title) && rs.getString(4).equals(context)) {
System.out.println("Titel: " + rs.getString(3) + " " + "Context: " + rs.getString(4));
}
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
The class in which both methods are located extends the JDBCBaseManager.
JDBCBaseManager:
private final String url = "jdbc:mysql://localhost:3306/database";
private final String userName = "root";
private final String password = "";
private Connection connection = null;
private PreparedStatement preparedStatement = null;
private int batchSize = 0;
public JDBCBaseManager() {
// Dotenv env = Dotenv.configure().directory("./serverless").load();
// url = env.get("DB_PROD_URL");
// userName = env.get("DB_USER");
// password = env.get("DB_PW");
}
public void getConnection() {
try {
if (connection == null) {
connection = DriverManager.getConnection(url, userName, password);
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public PreparedStatement getPreparedStatement(String sql, boolean returnGeneratedKeys) {
try {
if (connection == null) {
getConnection();
}
if (preparedStatement == null) {
if (!returnGeneratedKeys) {
preparedStatement = connection.prepareStatement(sql);
} else {
preparedStatement = connection.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS);
connection.setAutoCommit(false);
}
}
return preparedStatement;
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public void closeConnection() {
try {
if (connection != null && !connection.isClosed()) {
System.out.println("Closing Database Connection");
connection.close();
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public void startBatch(int batchSize) throws SQLException {
connection.setAutoCommit(false);
setBatchSize(batchSize);
}
public void commit() {
try {
if (connection != null && !connection.isClosed()) {
connection.commit();
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public int getBatchSize() {
return batchSize;
}
public void setBatchSize(int batchSize) {
this.batchSize = batchSize;
}
The ResultSet in the second method still contains the results from the first method.
I already tried to close the connection and open it again before the second method is executed, but then I get the errors:
java.sql.SQLException: No operations allowed after statement closed.
java.sql.SQLNonTransientConnectionException: No operations allowed
after connection closed.
Can you tell me how to deal with the statement correctly in this case? Is my BaseManager incorrectly structured?

Here lies the error
public JDBCBaseManager() {
private PreparedStatement preparedStatement = null;
public PreparedStatement getPreparedStatement(String sql, boolean returnGeneratedKeys) {
try {
......
if (preparedStatement == null) {
if (!returnGeneratedKeys) {
preparedStatement = connection.prepareStatement(sql);
} else {
preparedStatement = connection.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS);
connection.setAutoCommit(false);
}
}
return preparedStatement;
You build the prepare statement only the first time the method getPreparedStatement is called because only the first time the field preparedStatement is null. Every next time you call the method getPreparedStatement you receive the previous preparedStatement from the previous SQL and not the new one.
Remove the check for if (preparedStatement == null) {
You need to build a new preparedStatement every time you want to execute a new SQL.

Hive, JDBC, TTransportException: SASL authentication not complete

I connect to Hive and get id's of my data from row of table. Problems does not happens, when I connect to hive, send request and get response. But when i get id's from ResultSet i get an exception: org.apache.thrift.transport.TTransportException: SASL authentication not complete. Why does this exception arise and what needs to be done to avoid it? Sorry for my bad english.
It's my subsidiary class to create hive connection and send requests:
public class HiveDataSearcher implements AutoCloseable {
private static final String hiveDriverName = "org.apache.hive.jdbc.HiveDriver";
static {
try {
Class.forName(hiveDriverName);
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
}
}
private Connection hiveConnection;
private String tableName;
private String whereBody;
public HiveDataSearcher(String url, String login, String password) {
try {
hiveConnection = DriverManager.getConnection(url, login, password);
} catch (SQLException e) {
throw new RuntimeException(e);
}
this.tableName = "";
this.whereBody = "";
}
public HiveDataSearcher(Connection hiveConnection) {
Objects.requireNonNull(hiveConnection, "hiveConnection");
this.hiveConnection = hiveConnection;
this.tableName = "";
this.whereBody = "";
}
public String getTableName() {
return tableName;
}
public HiveDataSearcher setTableName(String tableName) {
Objects.requireNonNull(tableName, "tableName");
this.tableName = tableName;
return this;
}
public String getWhereBody() {
return whereBody;
}
public HiveDataSearcher setWhereBody(String whereBody) {
Objects.requireNonNull(whereBody, "whereBody");
this.whereBody = whereBody;
return this;
}
public ResultSet select(String ... selectParams) {
return select(Arrays.asList(selectParams));
}
public ResultSet select(Iterable<String> selectParams) {
String request = prepareRequest(selectParams);
ResultSet response;
try {
response = hiveConnection
.createStatement()
.executeQuery(request);
} catch (SQLException e) {
throw new RuntimeException(e);
}
return response;
}
private String prepareRequest(Iterable<String> selectParams) {
return new StringBuilder()
.append("select").append(' ').append(selectParamsToHiveFormat(selectParams)).append(' ')
.append("from").append(' ').append(tableName).append(' ')
.append("where").append(' ').append(whereBody)
.toString();
}
private String selectParamsToHiveFormat(Iterable<String> selectParams) {
StringBuilder formattedSelectParams = new StringBuilder();
for (String selectedParam : selectParams) {
formattedSelectParams.append('\'').append(selectedParam).append('\'').append(',');
}
if (formattedSelectParams.length() == 0) {
formattedSelectParams.append('*');
} else {
formattedSelectParams.deleteCharAt(formattedSelectParams.length() - 1);
}
return formattedSelectParams.toString();
}
public void close() {
if (hiveConnection != null) {
try {
hiveConnection.close();
} catch (SQLException e) {
//nothing to do, just close connection
} finally {
hiveConnection = null;
}
}
}
}
This is the code in which i connect to hive:
private static final String HIVE_URL = <hive url>;
private static final String HIVE_LOGIN = <hive login>;
private static final String HIVE_PASSWORD = <hive password>;
private static final String[] SEARCH_FIELDS = new String[] {"rowkey"};
private List<String> getIdsFromHive(String tableName, String whereBody) {
ResultSet hiveResponse;
try (HiveDataSearcher searcher = new HiveDataSearcher(HIVE_URL, HIVE_LOGIN, HIVE_PASSWORD)) {
hiveResponse = searcher
.setTableName(tableName)
.setWhereBody(whereBody)
.select(SEARCH_FIELDS);
}
List<String> ids = new ArrayList<>();
try {
while (hiveResponse.next()) { // in this place throw TTransportException
ids.add(hiveResponse.getString(SEARCH_FIELDS[0]));
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
return ids;
}

In my case, the reason for this exception is closed the connection before closed the statement. So I suggest you to check whether you has maintain the connection correctly.
Here is my code, wish it will inspire you something:
Wrong code, close connection before closing the statement:
Connection connection = null;
Statement statement = null;
try {
connection = HIVEUTILS.getConnection();
statement = connection.createStatement();
statement.execute("DROP TABLE IF EXISTS tbl1");
statement.execute("CREATE TABLE `tbl1` (`id` int)");
statement.execute("INSERT INTO tbl1 VALUES(1)");
}finally {
if (connection != null){
connection.close();
}
if (statement != null){
statement.close(); // exception occur here.
}
}
The correct order of closing is: close resultSet(if any) -> close statement -> close connection.
Connection connection = null;
Statement statement = null;
try {
connection = HIVEUTILS.getConnection();
statement = connection.createStatement();
statement.execute("DROP TABLE IF EXISTS tbl1");
statement.execute("CREATE TABLE `tbl1` (`id` int)");
statement.execute("INSERT INTO tbl1 VALUES(1)");
}finally {
if (statement != null){
statement.close(); // change the order
}
if (connection != null){
connection.close();
}
}

Type " ..." does not exist - JDBC error

I'm making some database tables using jdbc. On one of the tables I get this error message : TYPE 'MEET' does not exist.. Any ideas why?
The code is below:
public class MeetingDAO {
public static String getTableString(){
String meetingTable = "create table project ( " +
"meeting_id integer not null, " +
"timestamp meet, " +
"project_key varchar(10), " +
"primary key(meeting_id))";
return projectTable;
}
}
public class JDBCUtil {
private static final String dbURL= "jdbc:derby:MyDB;create=true;";
private static final String userID = "moon";
private static final String password = "moonmoon";
public static void init(){
makeTable(MeetingDAO.getTableString());
}
private static void makeTable(String tableSQL){
Connection conn = null;
try {
conn = JDBCUtil.getConnection();
Statement stmt = null;
try {
stmt = conn.createStatement();
stmt.executeUpdate(tableSQL);
}
finally {
JDBCUtil.closeStatement(stmt);
}
JDBCUtil.commit(conn);
}
catch (SQLException e) {
System.out.println(e.getMessage());
JDBCUtil.rollback(conn);
}
finally {
JDBCUtil.closeConnection(conn);
}
}
public static Connection getConnection() throws SQLException{
Driver derbyEmbededDriver = new EmbeddedDriver();
DriverManager.registerDriver(derbyEmbededDriver);
Connection conn = DriverManager.getConnection(dbURL, userID, password);
conn.setAutoCommit(false);
return conn;
}
public static void closeConnection(Connection conn){
try{
if(conn != null)
conn.close();
}
catch(SQLException e){
e.printStackTrace();
}
}
public static void closeStatement(Statement stmt){
try {
if(stmt != null)
stmt.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
public static void closeResultSet(ResultSet rs){
try {
if(rs != null)
rs.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
public static void commit(Connection conn){
try {
if(conn != null)
conn.commit();
} catch (SQLException e) {
e.printStackTrace();
}
}
public static void rollback (Connection conn){
try {
if(conn != null)
conn.rollback();
} catch (SQLException e) {
e.printStackTrace();
}
}
}

just swap these two words:
timestamp meet
to
meet timestamp
first there comes the column naming, after that the column datatyp.

You mixed up column name and data type for the column meet.
public class MeetingDAO {
public static String getTableString(){
String meetingTable = "create table project ( " +
"meeting_id integer not null, " +
"meet timestamp, " +
"project_key varchar(10), " +
"primary key(meeting_id))";
return projectTable;
}
}

Deferencing Null pointer

public class Model
{
public static Connection getConnection()
{
Connection conn = null;
try
{
Class.forName("oracle.jdbc.OracleDriver");
conn = DriverManager.getConnection("jdbc:oracle:thin:#localhost:1521:xe", "System", "system");
}
catch(ClassNotFoundException e)
{
e.printStackTrace();
}
catch(SQLException e)
{
e.printStackTrace();
}
return conn;
}
public static class Cart
{
public String itmName="";
public int howmany=0;
public static long itmQty=0, itmID=0;
public double itmPrice=0.0, itmCost=0.0, totalSum=0.0;
}
public static ArrayList<Cart> getCartDatabase(String user) throws Exception
{
Connection conn = getConnection();
String sql = "select * from userCarts where userID = '" + user + "'";
PreparedStatement pstmt = conn.prepareStatement(sql);
ResultSet rst = pstmt.executeQuery();
ArrayList<Cart> al = null;
Cart crt=null;
while(rst.next())
{
System.out.println("CPoint");
try
{
long p = rst.getLong("itemID");
crt.itmID = p; // This is the line thats creating the error
System.out.println(p + " is long! I guess...");
}
catch(NullPointerException e)
{
System.out.println("NPE Caught in Model");
}
System.out.println("CP 1 " + crt.itmID);
ArrayList<row> alr=null;
try
{
alr = Model.getStoreInventory();
}
catch(Exception e)
{
e.printStackTrace();
}
System.out.println("CP 2");
for(int i=0; i<alr.size(); i++)
{
crt.itmName = alr.get(i).itmName;
crt.itmPrice = alr.get(i).itmPrice;
crt.itmQty = alr.get(i).itmQty;
}
System.out.println("CP 3");
crt.howmany = rst.getInt("howmany");
crt.itmCost = crt.itmPrice*crt.howmany;
al.add(crt);
}
return al;
}
}
When I try to access this method of getCartFromDatabase, it gives a NullPointerException however I don't understand why it would do this. Moreover, I tried to make the class as a non static class too, but still it gave the same error:
"Possible deferencing Null Pointer"

Cart crt=null;
while(rst.next())
{
System.out.println("CPoint");
try
{
long p = rst.getLong("itemID");
crt.itmID = p; // This is the line thats creating the error
System.out.println(p + " is long! I guess...");
}
crt is null when you try to access crt.itemID. You have to assign it an instance first.
I think you may simply change the first line from the snippet to
Cart crt = new Cart();

Java DBCP keeps creating new connection

My DBCP configuration keeps creating new connections, so much that my MySQL server blocks it because of too many connections:
public class SQL {
private final static String DRIVER_CLASS_NAME = "com.mysql.jdbc.Driver";
private final static String USERNAME = "secret";
private final static String PASSWORD = "secret";
private final static String URL = "secret";
public static Connection getConnection() {
BasicDataSource basicDataSource = new BasicDataSource();
basicDataSource.setDriverClassName(DRIVER_CLASS_NAME);
basicDataSource.setUrl(URL);
basicDataSource.setUsername(USERNAME);
basicDataSource.setPassword(PASSWORD);
try {
return basicDataSource.getConnection();
} catch (SQLException ex) {
Logger.getLogger(SQL.class.getName()).log(Level.SEVERE, null, ex);
throw new IllegalStateException("bf4.sql.SQL.getConnection: No connection could be made: " + ex.getMessage());
}
}
}
My xxxManager.java:
public class PlayerkillManager extends Manager<PlayerkillBean, PlayerkillConstraint> {
public PlayerkillManager() {
super(SQL.getConnection());
}
#Override
protected PreparedStatement insertPS(final PlayerkillBean playerkill) throws SQLException {
PreparedStatement ps = connection.prepareStatement("INSERT INTO playerkills (`date`, `playerId`, `targetId`, `weaponId`, `headshot`) VALUES(?, ?, ?, ?, ?)", PreparedStatement.RETURN_GENERATED_KEYS);
ps.setObject(1, playerkill.getDate());
ps.setObject(2, playerkill.getPlayerId());
ps.setObject(3, playerkill.getTargetId());
ps.setObject(4, playerkill.getWeaponId());
ps.setObject(5, playerkill.getHeadshot());
return ps;
}
#Override
protected PreparedStatement updatePS(final PlayerkillBean playerkill) throws SQLException {
throw new UnsupportedOperationException("There are no non-key columns in this table.");
}
#Override
protected PreparedStatement deletePS(final PlayerkillBean playerkill) throws SQLException {
PreparedStatement ps = connection.prepareStatement("DELETE FROM playerkills WHERE `id` = ? AND `date` = ? AND `playerId` = ? AND `targetId` = ? AND `weaponId` = ? AND `headshot` = ?");
ps.setObject(1, playerkill.getId());
ps.setObject(2, playerkill.getDate());
ps.setObject(3, playerkill.getPlayerId());
ps.setObject(4, playerkill.getTargetId());
ps.setObject(5, playerkill.getWeaponId());
ps.setObject(6, playerkill.getHeadshot());
return ps;
}
#Override
protected String searchQuery() {
return "SELECT `playerkills`.`id`, `playerkills`.`date`, `playerkills`.`playerId`, `playerkills`.`targetId`, `playerkills`.`weaponId`, `playerkills`.`headshot` FROM playerkills";
}
#Override
protected String tableName() {
return "playerkills";
}
#Override
protected String[] columnNames() {
return new String[] {
"id",
"date",
"playerId",
"targetId",
"weaponId",
"headshot",
};
}
#Override
protected Map<TableField, List<List<TableField>>> getPaths() {
//Function not interesting and too much code
}
#Override
protected PlayerkillBean createBean(final ResultSet rs) throws SQLException {
return new PlayerkillBean(rs);
}
}
Manager.java class:
public abstract class Manager<B extends Bean, C extends AbstractConstraint> implements Closeable {
protected final Connection connection;
public Manager(final Connection con) {
this.connection = con;
}
public final int insert(final B b) throws InsertException {
try {
try (PreparedStatement ps = insertPS(b)) {
ps.executeUpdate();
try (ResultSet rs = ps.getGeneratedKeys()) {
rs.last();
if (rs.getRow() != 0) {
rs.beforeFirst();
rs.next();
return rs.getInt(1);
}
else {
return -1;
}
}
}
} catch (SQLException ex) {
Logger.getLogger(Manager.class.getName()).log(Level.SEVERE, null, ex);
throw new InsertException(ex);
}
}
public final boolean update(final B b) throws UpdateException {
try {
try (PreparedStatement ps = updatePS(b)) {
return ps.execute();
}
} catch (SQLException ex) {
Logger.getLogger(Manager.class.getName()).log(Level.SEVERE, null, ex);
throw new UpdateException(ex);
}
}
public final boolean delete(final B b) throws DeleteException {
try {
try (PreparedStatement ps = deletePS(b)) {
return ps.execute();
}
} catch (SQLException ex) {
Logger.getLogger(Manager.class.getName()).log(Level.SEVERE, null, ex);
throw new DeleteException(ex);
}
}
public final B get(final AbstractConstraint... c) throws SearchException {
List<B> beans = search(c);
if (beans.size() == 1) {
return beans.get(0);
}
throw new IllegalArgumentException("orm.Manager.get: beans.size() != 1: beans.size() = " + beans.size());
}
public final List<B> search(final AbstractConstraint... c) throws SearchException {
if (c.length == 0) {
throw new IllegalArgumentException("orm.Manager.search: c.length == 0");
}
try {
List<B> beans = new ArrayList<>();
for (AbstractConstraint constraint : c) {
try (PreparedStatement ps = new QueryBuilder(connection, tableName(), getPaths(), searchQuery()).add(constraint).build();
ResultSet rs = ps.executeQuery()) {
while (rs.next()) {
beans.add(createBean(rs));
}
}
}
if (c.length > 1) {
boolean sorting = true;
Field field = c[0].getField();
Order order = c[0].getOrder();
for (int i = 1; i < c.length; i++) {
Field currentField = c[i].getField();
Order currentOrder = c[i].getOrder();
if (!field.equals(currentField) || !order.equals(currentOrder)) {
sorting = false;
break;
}
}
if (sorting) {
//sort on field with comparator of supertype
}
}
return beans;
} catch (SQLException ex) {
Logger.getLogger(Manager.class.getName()).log(Level.SEVERE, null, ex);
throw new SearchException(ex);
}
}
public final List<B> getAll() throws SearchException {
return getAll(Order.NONE, null);
}
public final List<B> getAll(final Order order, final Field field) throws SearchException {
try {
List<B> beans = new ArrayList<>();
try (
PreparedStatement ps = connection.prepareStatement(searchQuery() + " " + orderQuery(order, field));
ResultSet rs = ps.executeQuery()) {
while (rs.next()) {
beans.add(createBean(rs));
}
}
return beans;
} catch (SQLException ex) {
Logger.getLogger(Manager.class.getName()).log(Level.SEVERE, null, ex);
throw new SearchException(ex);
}
}
public final int getRowCount(final AbstractConstraint... c) throws SearchException {
return search(c).size();
}
#Override
public void close() {
//was uncommented?
try {
connection.close();
} catch (SQLException ex) {
Logger.getLogger(Manager.class.getName()).log(Level.SEVERE, null, ex);
}
}
private String orderQuery(final Order order, final Field field) {
if (order == Order.NONE) {
return "";
}
return "ORDER BY " + field.getFieldName() + " " + order.getOrdername();
}
abstract protected PreparedStatement insertPS(B b) throws SQLException;
abstract protected PreparedStatement updatePS(B b) throws SQLException;
abstract protected PreparedStatement deletePS(B b) throws SQLException;
abstract protected String searchQuery();
abstract protected String tableName();
abstract protected String[] columnNames();
abstract protected Map<TableField, List<List<TableField>>> getPaths();
abstract protected B createBean(ResultSet rs) throws SQLException;
}
Some statistics I have gathered:
Max concurrent connections: 152
Failed connections: 12
Aborted connections: 375
Total connections: 844
Number of insert queries: 373
I would have expected that 1 connection would have been used though, what is going wrong?
EDIT: To clarify, my code calls the xxxManager for example like this: playerkillManager.insert(new PlayerkillBean(...));

You are creating a new connection pool every time you call SQL.getConnection() which is not how connection pools should be used.
You should share a single javax.sql.DataSource (doc) around your application, not individual connections.
So, maybe you could change your code to:
public class SQL {
private final static String DRIVER_CLASS_NAME = "com.mysql.jdbc.Driver";
private final static String USERNAME = "secret";
private final static String PASSWORD = "secret";
private final static String URL = "secret";
private final static DataSource dataSource;
static {
BasicDataSource basicDataSource = new BasicDataSource();
basicDataSource.setDriverClassName(DRIVER_CLASS_NAME);
basicDataSource.setUrl(URL);
basicDataSource.setUsername(USERNAME);
basicDataSource.setPassword(PASSWORD);
dataSource = basicDataSource;
}
public static DataSource getDataSource() {
return dataSource;
}
}
Then in the rest of your classes, you can use that data source. Important things to remember are that DataSource.getConnection() borrows a connection from the pool and Connection.close() does not actually close the connection; calling close() returns the connection to the pool. If you fail to call Connection.close() on a borrowed connection you have a connection leak.
Your current code will need editing to use try-with-resources when borrowing the connection e.g.
public void foo() {
try (Connection conn = datasource.getConnection()) {
//your code here
} catch (SQLException e) {
e.printStackTrace();
}
}