I have a program that trims items from a SQLite database based on a timestamp field. It seems when I run it on a large existing database that it takes quite a very long time and hogs up the resources on my already very low end embedded machine.
Can anyone suggest any ideas on how to improve my code to get it to run better/faster?
Call from main program:
query = SQLiteInterface.getInstance().delete(entry, "SampleInfo_source_timestamp < '" + timestamp + "'");
dbCall(query);
SQLiteInterface delete method:
/**
* This creates a SQLite formatted 'delete' statement that allows for where clauses
*
* #param table - The table that the items are being selected from
* #param filters - The list of where clauses
* #return - A SQLite formatted delete query
*/
public String delete(String table, String filters)
{
if (filters == null)
{
return "delete from " + table;
} else
{
return "delete from " + table + " where " + filters;
}
}
Code that actually sends the DB Command using JDBC
/**
*
*
* #param query - A SQLite formatted DB query
*/
public void dbCall(String query)
{
// Good practice to create a new statement and result set instead of reusing
Statement stmt = null;
Connection conn = null;
ResultSet rs = null;
try
{
// Fetch the query and the request out of the QueryRequest object
conn = cpds.getConnection();
conn.setAutoCommit(false);
// Make sure the query request isn't empty, if it is, there is no point in sending it to the DB
if (!query.isEmpty())
{
// Initialize the new statement
stmt = conn.createStatement();
stmt.executeUpdate(query);
}
conn.commit();
}
catch (SQLException e)
{
if (e.getMessage().contains("no such column"))
{
// This table is not one that needs to be cleaned up.
}
else
{
errorLog.error("", e);
// Table busy, try again.
dbCall(query);
}
}
catch (Exception e2)
{
errorLog.error("", e2);
}
finally
{
if (rs != null)
{
try
{
rs.close();
}
catch (SQLException e)
{
errorLog.error("Failed to close JDBC result set.");
}
}
if (stmt != null)
{
try
{
stmt.close();
}
catch (SQLException e)
{
errorLog.error("Failed to close JDBC statement.");
}
}
if (conn != null)
{
try
{
conn.close();
}
catch (SQLException e)
{
errorLog.error("Failed to close JDBC connection.");
}
}
}
Related
I a working with java and MYSQL. I want to insert 1000 elements into MYSQL database from java. I created a stored procedure with batch insertion .The problem is that if the insertion of first element or any other element failed due to some SQL exception, the execution will be stop. I want to insert other elements even if some insertion failed due to SQL exception. How is it possible?
public void insertXml(ArrayList<String> xmlCollection) throws SQLException {
Connection connection = null;
CallableStatement statement = null;
try {
connection = DbConnector.getConnection();
Application.getAppInstance().monitorCountProcedure("insertXmlToDB");
statement = connection.prepareCall("{call insertXmlToDB(?)}");
connection.setAutoCommit(false);
for (String xmlString : xmlCollection) {
//System.out.println("xmlstring="+xmlString);
statement.setString(1, xmlString);
statement.addBatch();
}
statement.executeBatch();
connection.commit();
System.out.println("insertXmlToDB stored procedure called successfully!" + statement);
} catch (Exception ex) {
ex.printStackTrace();
System.err.println("insertXmlToDB stored procedure error : " + ex.getMessage());
} finally {
if (statement != null) {
try {
statement.close();
} catch (SQLException e) {
e.printStackTrace();
// log.error("Could not close statement" + e.getMessage());
}
}
if (connection != null) {
try {
connection.setAutoCommit(true);
connection.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
This question already has answers here:
Why does my ArrayList contain N copies of the last item added to the list?
(5 answers)
Closed 4 years ago.
I tried to print the objects from hashSet collection. the console display only the last object(one object). when I used ArrayList with the same method, I'm able to print all the objects. i've used an iterator method in order to print the collection set, see the attached test.
public Set<Coupon> getAllCoupouns() throws Exception {
Coupon coupon = new Coupon();
Set<Coupon> coupons = new HashSet<Coupon>();
// Open a connection
conn = DriverManager.getConnection(Utils.getDBUrl());
// Define the Execute query
java.sql.Statement stmt = null;
try {
stmt = conn.createStatement();
// build The SQL query
String sql = "SELECT * FROM COUPON";
// Set the results from the database
ResultSet resultSet = stmt.executeQuery(sql);
// constructor the object, retrieve the attributes from the results
while (resultSet.next()) {
coupon.setId(resultSet.getLong(1));
coupon.setTitle(resultSet.getString(2));
coupon.setStartDate((Date) resultSet.getDate(3));
coupon.setEndDate((Date) resultSet.getDate(4));
coupon.setAmount(resultSet.getInt(5));
CouponType type = CouponType.valueOf(resultSet.getString(6)); // Convert String to Enum
coupon.setType(type);
coupon.setMessage(resultSet.getString(7));
coupon.setPrice(resultSet.getDouble(8));
coupon.setImage(resultSet.getString(9));
coupons.add(coupon);
}
} catch (SQLException e) {
throw new Exception("Retriev all the coupons failed");
} finally {
// finally block used to close resources
try {
if (stmt != null)
conn.close();
} catch (SQLException se) {
// do nothing
}
try {
if (conn != null)
conn.close();
} catch (SQLException se) {
se.printStackTrace();
}
}
return coupons;
}
As you are initialising Coupon from outside the while loop, it keeps adding same object every time and hence, overwriting results in only last result being displayed.
What you need to do is to instantiate Coupon from within the while loop, e.g.:
public Set<Coupon> getAllCoupouns() throws Exception {
Set<Coupon> coupons = new HashSet<Coupon>();
// Open a connection
conn = DriverManager.getConnection(Utils.getDBUrl());
// Define the Execute query
java.sql.Statement stmt = null;
try {
stmt = conn.createStatement();
// build The SQL query
String sql = "SELECT * FROM COUPON";
// Set the results from the database
ResultSet resultSet = stmt.executeQuery(sql);
// constructor the object, retrieve the attributes from the results
while (resultSet.next()) {
Coupon coupon = new Coupon();
coupon.setId(resultSet.getLong(1));
coupon.setTitle(resultSet.getString(2));
coupon.setStartDate((Date) resultSet.getDate(3));
coupon.setEndDate((Date) resultSet.getDate(4));
coupon.setAmount(resultSet.getInt(5));
CouponType type = CouponType.valueOf(resultSet.getString(6)); // Convert String to Enum
coupon.setType(type);
coupon.setMessage(resultSet.getString(7));
coupon.setPrice(resultSet.getDouble(8));
coupon.setImage(resultSet.getString(9));
coupons.add(coupon);
}
} catch (SQLException e) {
throw new Exception("Retriev all the coupons failed");
} finally {
// finally block used to close resources
try {
if (stmt != null)
conn.close();
} catch (SQLException se) {
// do nothing
}
try {
if (conn != null)
conn.close();
} catch (SQLException se) {
se.printStackTrace();
}
}
return coupons;
}
cupon is always the same object. You make only one object of class Cupon, so the set contains only one object (you add always the same object). You must make a new object in each iteration in the while loop.
I want to insert values for multiple columns of a table. I am doing an Eclipse Project and I want to feed the data from my project into the database. There are multiple columns in the database and I have values for each of these columns from my Eclipse Project. The JDBC driver and the connections are all done. I just need to figure out how to input these values from the project into the table.
public void insert(Double num1, Double num2, Double result) throws Exception {
Class.forName("com.mysql.jdbc.Driver");
Connection con = null;
PreparedStatement stmt = null;
try {
con = DriverManager.getConnection("jdbc:mysql://localhost:3306/Calculator", "root", "");
stmt = con.prepareStatement("INSERT INTO RESULT(ID,VALUE1,VALUE2,RESULT) VALUES (?,?,?,?))");
stmt.executeUpdate();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (stmt != null) {
try {
stmt.close();
} catch (SQLException ex) {
}
}
if (con != null) {
try {
con.close();
} catch (SQLException ex) {
}
}
}
}
You're close to an answer, your are missing the setXXX calls to assign values to the ? in your insert statement, you didn't provide a value for ID in your function parameter and you have an extra parenthesis in the prepareStatement.
stmt = con.prepareStatement("INSERT INTO RESULT(ID,VALUE1,VALUE2,RESULT) VALUES (?,?,?,?)");
// stmt.set___(1,___);
stmt.setDouble(2,num1);
stmt.setDouble(3,num2);
stmt.setDouble(4,result);
How do you handle error condition while writing stored procedure or accessing stored procedure from java?
stored procedure should return error code if some operation fails but if stored procedure itself fail than catching SQLException is only choice.
try {
CallableStatement stmt=con.prepareCall("{call insertR(?,?)}");
stmt.setInt(1,1011);
stmt.setString(2,"Amit");
stmt.execute();
} catch(SQLException e) {
e.printStack();
}
this piece of code is taken from oracle [docs][1] to answer.
public void
createProcedureShowSuppliers()
throws SQLException {
String createProcedure = null;
String queryDrop =
"DROP PROCEDURE IF EXISTS SHOW_SUPPLIERS";
createProcedure =
"create procedure SHOW_SUPPLIERS() " +
"begin " +
"select SUPPLIERS.SUP_NAME, " +
"COFFEES.COF_NAME " +
"from SUPPLIERS, COFFEES " +
"where SUPPLIERS.SUP_ID = " +
"COFFEES.SUP_ID " +
"order by SUP_NAME; " +
"end";
Statement stmt = null;
Statement stmtDrop = null;
try {
System.out.println("Calling DROP PROCEDURE");
stmtDrop = con.createStatement();
stmtDrop.execute(queryDrop);
} catch (SQLException e) {
JDBCTutorialUtilities.printSQLException(e);
} finally {
if (stmtDrop != null)
{
stmtDrop.close();
}
}
try {
stmt = con.createStatement();
stmt.executeUpdate(createProcedure);
} catch (SQLException e) {
JDBCTutorialUtilities.printSQLException(e);
} finally {
if (stmt != null) { stmt.close(); }
}
}
[1]: https://docs.oracle.com/javase/tutorial/jdbc/basics/storedprocedures.html
When you call a stored procedure, it will execute in database server so if there any exception occurs that can be handled in EXCEPTION block in the stored procedure. If that stored procedure itself fails, it throws a SQL exception which can be handled by try/catch block and wrap it to your project specific exception.
Example
try {
CallableStatement stmt = con.prepareCall("{call geNamebyId(?)}");
stmt.setInt(1);
stmt.execute();
} catch(SQLException e) {
e.printStack();
}
I get this exception while truncating all table in a schema.
I truncate 3 schema in my Java code and first method get list of table names from given schema name and second method executes "TRUNCATE TABLE table_name" query.
I confused about my code always succesful while truncating first and third schema. But while executing on second schema I get ORA-01000 error.
My truncate code is
private void truncateTable(Connection conn, String tableName) {
PreparedStatement ps = null;
try {
ps = conn.prepareStatement(Utility.TRUNCATE_TABLE + tableName);
ps.executeUpdate();
} catch (SQLException e) {
log.error("SQLException occured while getting table names from schema", e);
} finally {
Utility.free(ps, null, null);
}
}
private List<String> getAllTableNames(Connection conn) {
PreparedStatement ps = null;
ResultSet rs = null;
List<String> list = new ArrayList<String>();
try {
ps = conn.prepareStatement(Utility.SELECT_ALL_TABLE_NAMES);
rs = ps.executeQuery();
while (rs.next()) {
list.add(rs.getString("TABLE_NAME"));
}
} catch (SQLException e) {
log.error("SQLException occured while getting table names from schema", e);
} finally {
Utility.free(ps, rs, null);
}
return list;
}
public static void free(PreparedStatement ps, ResultSet rs, Connection conn) {
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {
log.error("Error occurred while closing ResultSet",e);
}
}
if (ps != null) {
try {
ps.close();
} catch (SQLException e) {
log.error("Error occurred while closing PreparedStatement",e);
}
}
if (conn != null) {
try {
conn.close();
} catch (SQLException e) {
log.error("Error occurred while closing Connection",e);
}
}
}
What is the wrong about code or is it about schema configuraiton in Oracle?
How can I solve this?
If are you iterating over the List generated by getAllTableNames and calling truncateTable in a tight loop, your free calls in the finally block might just be delayed and stacking up to an extent that they aren't clearing fast enough for the next iterations - since you only know the finally will be called at some point, not necessarily immediately and before control is returned to the caller.
The schema size would make a difference to that, so it might make sense that a small schema succeeds and a large one fails. If that is what's happening then you should call free inside the try, as well as in the finally:
private void truncateTable(Connection conn, String tableName) {
PreparedStatement ps = null;
try {
ps = conn.prepareStatement(Utility.TRUNCATE_TABLE + tableName);
ps.executeUpdate();
Utility.free(ps, null, null);
ps = null;
} catch (SQLException e) {
log.error("SQLException occured while getting table names from schema", e);
} finally {
if (ps != null) {
Utility.free(ps, null, null);
}
}
}
If Utility.free checks whether ps is null then that check in the finally block might be redundant, but without it, free would be called twice if there is no SQLException.
Check out the code and make sure you are closing the cursors after being used. If the problem still persists please set OPEN_CURSORS to some more value.