I'm writing a program that reads lines from a csv file, for each of these lines it checks against an different database for some additional data and it finally inserts the newly constructed data in a mysql DB.
BufferedReader br = new BufferedReader(new FileReader(file));
for(String line; (line = br.readLine()) != null; ) { //Read each file line
try{
processLine(line);
} catch(ProcessLineException e){
logger.warn("Something happened");
}
}
br.close();
processLine being
private void processLine(String line) throws ProcessLineException{
...
insertData(foo, data);
}
private void insertData(String foo, String data) {
Connection connection = null;
PreparedStatement pStatement = null;
try{
connection = dataSource.getConnection();
pStatement = connection.prepareStatement("INSERT INTO table VALUES(?,?)");
pStatement.setString(1, foo);
pStatement.setString(2, data);
} catch(SQLException e){
logger.error("Error when inserting data");
} finally {
try {
pStatement.close();
connection.close();
} catch (SQLException e) {
logger.warn("Couldn't close resources");
}
}
}
I've learnt some things about PreparedStatements when I was looking for a better way to handle SQLException (could also get some help on that, code above) and, the way i see it, I could benefit from using a PreparedStatement to store the mysql insert query and just modify parameters on each iteration of the loop. But shouldn't that imply that I should keep an open connection with the database during the whole process? Would that be negative in any way?
You are executing each query separately. This hits the database for each insert statement.Instead you should use addBatch() method of Statement, instead of directly executing the query one after another like above and after adding all the queries should execute them in one go using statement.executeBatch() method.e.g
import java.sql.Connection;
import java.sql.Statement;
//...
Connection connection = new getConnection();
Statement statement = connection.createStatement();
for (Employee employee: employees) {
String query = "insert into employee (name, city) values('"
+ employee.getName() + "','" + employee.getCity + "')";
statement.addBatch(query);
}
statement.executeBatch();
statement.close();
connection.close();
Related
Trying to first make sure this part of my project is working before implementing it into a GUI.
Trying to first create a new table in the data base called n012345_Accounts with 4 columns:
Account Number
Name
Balance
Lock
Then I want to populate the data of this table by reading the lines of the file I have created with in Accounts.txt which includes the following
Number Name Balance Locked
1001 Isabel_Newton 2000 yes
1002 Blake_Wool 1500 yes
1003 Martha_Curie 3000 no
1004 Nortom_Eef 1500 no
1009 Dan_Heckler 2000 yes
1010 Timothy_Wicket 4000 no
1011 Jane_Doe 5000 no
The purpose of this is to practice my understanding of using PreparedStatements and transactions. If anyone can see what the error is that is not allowing the creation of the table I would appreciate the input.
Currently when running my project the console returns
unable to create new table for accounts
//Create a GUI application for a bank
//it should manage fund transfers from one account to another
//1
//Start
//# the start up it should create a table name YourStudentNumber_Accounts ( n012345)
//it should also populate this table with the information stored in the file provided ("Accounts.txt")
//2
//Then the application will ask for
//account number the funds are to be transferred from
//amount to be transferred
//account number funds are to be transferred to
//3
//Upon exit the application will present the contents of the Accounts table in standard output
//USE PREPARED STATEMENTS and TRANSACTIONS wherever appropriate
//All exceptions must be handled
import oracle.jdbc.pool.OracleDataSource;
import java.io.*;
import java.sql.*;
public class Main {
public static void main(String[] args) throws SQLException{
OracleDataSource ods = new OracleDataSource();
ods.setURL("jdbc:oracle:thin:n012345/luckyone#calvin.humber.ca:1521:grok");
//try to connect to the database connection we have declared
try(Connection con = ods.getConnection()) {
//create a statement object
try (Statement stmt = con.createStatement()) {
try (ResultSet rs = stmt.executeQuery("CREATE TABLE n012345_Accounts (AccountNumber float(4) , Name varchar(25), Balance float(9), Lock varchar(25))")) {
try (BufferedReader reader = new BufferedReader(new FileReader("Accounts.txt"));) {
String line;
//do not automatically commit statements
con.setAutoCommit(false);
while ((line = reader.readLine()) != null) {
//inputting data into a String array splitting data by the space between the items in the file
String[] fields = line.split(" ");
String queryString = "INSERT INTO n012345_Accounts (AccountNumber, Name, Balance, Lock) VALUES(?,?,?,?)";
try (PreparedStatement statement = con.prepareStatement(queryString);) {
statement.setFloat(1, Float.parseFloat(fields[0]));
statement.setString(2, fields[1]);
statement.setFloat(3, Float.parseFloat(fields[2]));
statement.setString(4, fields[3]);
statement.executeUpdate();
} catch (Exception e) {
System.out.println("There was an error inserting into the database.");
}
System.out.println("Accounts.txt data was populated into the table n01494108_Accounts");
}
} catch (Exception e) {
System.out.println("unable to read the file.");
}
con.commit();
} catch (SQLException ex) {
System.out.println("unable to create new table for accounts");
}
//closes the statement
} catch (Exception e) {
//using rollback() to ensure no statements in a transaction are committed if an exception error occurs
con.rollback();
}
}catch (SQLException ex){
//closes connection
}
} //closes main method
} // closes main class
Use execute instead of executeQuery when you are trying to create a table.
Your code is printing unable to create new table for accounts because database table N012345_ACCOUNTS already exists. Once you create a database table, you can't re-create it. Hence the very first time you run your code – assuming that the database table does not exist – the database table will be successfully created however the next time you run your code, you will get unable to create new table for accounts – unless you drop the table before running your code again. By the way, I recommend printing the stack trace in catch blocks rather than just some error message alone.
You can use DatabaseMetaData to check whether the database table already exists and create it if it doesn't.
After creating the database table, your next task is to populate it. I recommend using batching.
You populate the database table with data that you read from a text file. You need to verify the data read from the text file. According to the sample text file contents in your question, you need to ignore the first two lines of the file.
The below code uses text blocks, NIO.2, try-with-resources and multi-catch.
import java.io.BufferedReader;
import java.io.IOException;
import java.math.BigDecimal;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class Main {
private static void createTable(Connection conn) throws SQLException {
String sql = """
create table N012345_ACCOUNTS (
ACCOUNT_NUMBER number(4)
,ACCOUNT_NAME varchar2(30)
,ACCOUNT_BALANCE number(14,2)
,ACCOUNT_LOCKED varchar2(3)
,constraint ACCT_PK primary key (ACCOUNT_NUMBER)
,constraint ACCT_LOCKS check (ACCOUNT_LOCKED in ('no','yes'))
)
""";
try (Statement s = conn.createStatement()) {
s.executeUpdate(sql);
System.out.println("Database table N012345_ACCOUNTS created.");
}
}
private static void populateTable(Connection conn) throws IOException, SQLException {
String sql = "insert into N012345_ACCOUNTS values (?, ?, ?, ?)";
Path path = Paths.get("accounts.txt");
try (BufferedReader br = Files.newBufferedReader(path);
PreparedStatement ps = conn.prepareStatement(sql)) {
String line = br.readLine();
conn.setAutoCommit(false);
while (line != null) {
String[] fields = line.split(" ");
if (fields.length == 4) {
try {
BigDecimal number = new BigDecimal(fields[0]);
String name = fields[1];
BigDecimal balance = new BigDecimal(fields[2]);
String locked = fields[3];
ps.setBigDecimal(1, number);
ps.setString(2, name);
ps.setBigDecimal(3, balance);
ps.setString(4, locked);
ps.addBatch();
}
catch (NumberFormatException xNumberFormat) {
// Ignore.
}
}
line = br.readLine();
}
int[] results = ps.executeBatch();
int success = 0;
for (int result : results) {
if (result == 1) {
success++;
}
}
System.out.printf("Inserted %d rows.%n", success);
if (success == results.length) {
conn.commit();
}
else {
conn.rollback();
}
}
}
public static void main(String[] args) {
String url = "jdbc:oracle:thin:n012345/luckyone#calvin.humber.ca:1521:grok";
try (Connection conn = DriverManager.getConnection(url)) {
DatabaseMetaData dbmd = conn.getMetaData();
ResultSet rs = dbmd.getTables(null, null, "N012345_ACCOUNTS", null);
if (!rs.next()) {
createTable(conn);
}
else {
System.out.println("Database table N012345_ACCOUNTS already exists.");
}
populateTable(conn);
}
catch (IOException | SQLException x) {
x.printStackTrace();
}
}
}
Refer to the following (in no particular order):
SQL Language Reference from Oracle database documentation.
JDBC Developer's Guide and Reference also from Oracle database documentation.
JDBC Database Access trail in Oracle's Java tutorials.
javadoc for method executeBatch in interface java.sql.Statement
I have create a java software which make use of sqlite database. The whole database works smoothly however after some time of running the application I am reveiving the following message (from a try catch block):
java.sql.SQLException: [SQLITE_BUSY] The database file is locked (database is locked)
I ve solved my problems by closing the software every time the exception rising. However is there a way to close my database instead so as not to close the software every time?
I ve got plenty of queries however my prob arises always in a specific point:
try {
String query = "select * from StudentsSession where userId=? and course=? and level=?";
PreparedStatement pst = connectionUsers.prepareStatement(query);
pst.setString(1, saveUser.getText());
pst.setString(2, textSubjectQTest);
st.setString(3, showCurrentLevelLabel.getText());
ResultSet rs = pst.executeQuery();
while (rs.next()) {
count = count + 1;
}
pst.close();
rs.close();
} catch (Exception a) {
System.out.println(a);
}
try {
String countS, tmpS;
countS = String.valueOf(count);
sessionId.setText(countS);
long unixTime = System.currentTimeMillis() / 1000L;
tmpS = String.valueOf(unixTime);
date.setText(tmpS);
course.setText(textSubjectQTest);
String query = "insert into StudentsSession (userId,date,course,level,trial,score) values (?,?,?,?,?,-1)";
PreparedStatement pst = connectionUsers.prepareStatement(query);
pst.setString(1, saveUser.getText());
pst.setString(2, tmpS);
pst.setString(3, textSubjectQTest);
pst.setString(4, showCurrentLevelLabel.getText());
pst.setString(5, countS);
pst.executeUpdate();
pst.close();
} catch (Exception a) {
System.out.println(a);
System.exit(0);
}
String file1 = "";
ResultSet ts4;
try {
sessionId3 = "";
String query3 = "select * from studentssession where userid = ? and course = ? and level = ?";
PreparedStatement pst__1 = connectionUsers.prepareStatement(query3);
pst__1.setString(1, saveUser.getText());
pst__1.setString(2, textSubjectQTest);
pst__1.setString(3, showCurrentLevelLabel.getText());
ts4 = pst__1.executeQuery();
while (ts4.next()) {
sessionId3 = ts4.getString("sessionId");
}
pst__1.close();
ts4.close();
obj = new CaptureVideoFromWebCamera();
file1 = "videos/" + userTextFieldS.getText();
file1 = file1 + "_" + sessionId3;
file1 = file1 + ".wmv";
obj.start(file1);
} catch (Exception e4) {
e4.getCause();
}
Sometimes this code rise the exception.
Every time you open a connection with the SQLite database, make sure to close the database connection after you process the results or etc. if you already have an open connection to the database and if you try to get the connection again and try some Update or Insert queries, the system will not give the permission and will raise an error.
Your try catch are wrong, you try to close the ResultSet and Statemnt in the try block instead of a finally block. This could lead to leaks.
You should do it like this, in the finally.
PreparedStatement pst = null;
ResultSet rs = null;
try {
pst = connectionUsers.prepareStatement(query);
...
rs = pst.executeQuery();
...
} catch (Exception a) {
a.printStackTrace();
} finally {
if(rs != null){
try{
rs.close();
} catch(Exception e){
e.printStackTrace();
}
}
if(pst != null){
try{
pst.close();
} catch(Exception e){
e.printStackTrace();
}
}
}
Or you could look to the Try-with-resource.
This could be a reason.
Sorta duplicate of existing question, but this is a good starting point. Because SQLite is just a library that reads and writes to a file on the file system, not a full SQL database, you should really only have one connection open to it at a time. Otherwise it is pretty easy to get into a race condition.
For local testing, it should be fine, but for a system of any complexity expecting multiple users, you should be using something like Postgre or MySQL.
Trying to use prepareStatement on a query from an XML file:
<sql><![CDATA[
select MY_COLUMN from MY_TABLE where OTHER_COLUMN = ?
]]>
</sql>
But I get this exception:
com.ibm.db2.jcc.am.SqlException: [jcc][t4][10234][10927][3.59.81] SQL passed with no tokens. ERRORCODE=-4462, SQLSTATE=null
at com.ibm.db2.jcc.am.dd.a(dd.java:660)
at com.ibm.db2.jcc.am.dd.a(dd.java:60)
at com.ibm.db2.jcc.am.dd.a(dd.java:120)
at com.ibm.db2.jcc.am.jb.v(jb.java:7334)
at com.ibm.db2.jcc.am.jb.a(jb.java:2124)
at com.ibm.db2.jcc.am.jb.prepareStatement(jb.java:754)
I read that this is because the SQL is not in a single line. Is that the cause?
I also read that in IBM Portal, you have to change db2_zos.DbDriverType from 2 to 4. But I think it's irrelevant to me as I don't use IBM Portal.
Nothing else useful turned up from Google. I would love to know the real cause of the error, and to find an easier fix than forcing all SQL to single line.
Code:
ArrayList arrList = new ArrayList();
String sSQL = queryManager.getSQL("QRY001");
Connection conn = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
conn = getConnection(); //a connection pool wrapper for java.sql.DriverManager.getConnection(...)
ps = conn.prepareStatement(sSQL); // **** EXCEPTION OCCURS HERE
ps.setInt(1, Integer.parseInt(otherColumn));
rs = ps.executeQuery();
while (rs.next())
{
arrList.add(rs.getString(1));
}
} catch (Exception e) {
e.printStackTrace();
throw new OEException(e);
} finally {
try {
if (rs != null) {
rs.close();
}
if (ps != null) {
ps.close();
}
releaseConnection(conn);
} catch (Exception e) {
throw new CustomException(e);
}
}
return arrList;
Because the SQL execution has comments;
Please refer to the official documentation
https://www.ibm.com/support/pages/apar/PH14323
Also try writing it like this
<sql>
select MY_COLUMN from MY_TABLE where OTHER_COLUMN = ?
</sql>
I've got a following problem: I'm trying to insert data (in this case a username) into a table using the following code:
void AddNewUser(String name, Connection conn){
if(ret == null){
ret = new DB_Retriever(conn);
}
if(!ret.UserExists(name, conn)){
try{
Statement stm = conn.createStatement();
stm.executeUpdate(DB_OperationalData.insert_new_user[0][0] + name + DB_OperationalData.insert_new_user[0][1]);
stm.executeUpdate(DB_OperationalData.insert_new_user[1][0] + name + DB_OperationalData.insert_new_user[1][1]);
stm.close();
}
catch(SQLException e){
e.printStackTrace();
}
}
}
By the way: It absolutely doesn't matter what I put in the catch clause, nothing that I put there is executed. Just to make everything clear, here is the content of the DB_OperationalData.insert_new_user String array:
public static final String[][] insert_new_user = {
{"INSERT INTO User (Username, Status) VALUES ('","','IN');"},
{"INSERT INTO Statistics (Player_ID) SELECT ID FROM User WHERE Username='","';"}};
The second statement is supposed to copy the ID of the user that is inserted and put it into Player_ID field of the Statistics table (Table User's ID is an autonumbered field).
The exception I get is:
Error while processing the query: java.sql.SQLException: ResultSet closed
What is interesting, is that it works and the data is added correctly but I simply do not want any exceptions thrown.
That's the console output I get:
This is 'data' Package Testing class
Connection to the database established.
The number of tables existing in the database is: 0
All the queries have been processed successfully
Adding new users:
Error while processing the query: java.sql.SQLException: ResultSet closed
All the lines above the Exception are my own printouts, so I know what has actually happened.
[EDIT]
I have changed the code to use the PreparedStatement instead of ordinary Statement and the current try clause looks as follows:
PreparedStatement pstm = conn.prepareStatement(DB_OperationalData.insert_new_user[0]);
pstm.setString(1, name);
pstm.addBatch();
conn.setAutoCommit(false);
pstm.executeBatch();
conn.setAutoCommit(true);
pstm.close();
And the output is (still regardless of the contents of the catch clause):
This is 'data' Package Testing class
Connection to the database established.
The number of tables existing in the database is: 0
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
at org.sqlite.PrepStmt.batch(PrepStmt.java:173)
at org.sqlite.PrepStmt.setString(PrepStmt.java:254)
at data.DB_Writer.AddNewUser(DB_Writer.java:28)
at data.DataHandler.AddNewUser(DataHandler.java:94)
at data.Tester.main(Tester.java:18)
All the queries have been processed successfully
Adding new users:
Error while processing the query: java.sql.SQLException: ResultSet closed
[EDIT 2]
With regards to the original version, when I remove the stm.close(); there is absolutely no difference and I still get the 'ResultSet closed' Exception.
[EDIT 3]
Here is the code of the method that is calling the above:
public void AddNewUser(String username)throws IllegalUsernameException{
if(username.length()==0 || username.length()>20){
throw new IllegalUsernameException();
}
writer.AddNewUser(username, conn);
}
The connection to the database is established by this class:
class DB_Connection {
public static Connection getConnection(){
Connection conn = null;
try{
Class.forName("org.sqlite.JDBC");
}
catch(ClassNotFoundException e){
log("Error while loading the database driver: " + e);
return null;
}
try{
conn = DriverManager.getConnection("jdbc:sqlite:database.db");
}
catch(SQLException e){
log("Unable to connect to the database: " + e);
return null;
}
return conn;
}
public static void log(String msg){
System.out.println(msg);
}
}
The DB_Retriever's method that is checking for the existing username:
boolean UserExists(String name, Connection conn){
String result = "";
try{
Statement stm = conn.createStatement();
ResultSet rs = stm.executeQuery(DB_OperationalData.user_exists[0] + name + DB_OperationalData.user_exists[1]);
result = rs.getString("Username");
}
catch(SQLException e){
System.out.println("Error while processing the query: " + e);
}
if(result.equals(name)){
return true;
}
return false;
}
The only location where Error while processing the query: java.sql.SQLException: ResultSet closed could be printed to the console is in UserExists(..), unless there is another method with a similar catch block. Indeed the ResultSet is not used correctly in UserExists, what may cause the error.
For a more complete description of how to work with JDBC look at this answer or the JDBC documentation. A possible alternative to the existing UserExists is:
boolean userExists(String name, Connection conn) {
PreparedStatement stmt = null;
try{
stmt = conn.prepareStatement("SELECT COUNT(Username) FROM User WHERE Username = ?");
stmt.setString(1, name);
ResultSet rs = stmt.executeQuery();
rs.next(); // set cursor to first row
int count = rs.getInt(1);
rs.close();
return count > 0;
} catch(SQLException e) {
// propagate error
throw new RuntimeException(e);
} finally {
// clean up resources
if (stmt != null) {
try {
stmt.close();
} catch (SQLException ignore) {
log("error on sql clean up", ignore);
}
}
}
}
How do I insert a list of values into a column in a MySQL table.
Here is my project:
public void settingAmount(List<String>lst)throws Exception{
// Accessing driver from jar files
Class.forName("com.mysql.jdbc.Driver");
// here we create a variable for the connection called con
Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/ozon","root","root");
// here we create our query
Statement stmt = (Statement) con.createStatement();
//performing insert operation :)
lst.toArray(ss);
for(String ins:ss){
double d=Double.parseDouble(ins);
String insert = "INSERT INTO storage_other(total) VALUES ("+ins+");";
//String insert = "INSERT INTO storage_other(amount) VALUES ("+ls+");";
// Here we are going to Execute the query
stmt.executeUpdate(insert);
System.out.print("Done Seccesfully :)");
}
}
What you want to do is use batches. Batches allow you to send a list of statements to be done at the same time.
Here is an example
connection.setAutoCommit(false);
PreparedStatement ps = connection.prepareStatement("INSERT INTO storage_other(total) VALUES (?)");
for (String ins:ss){
ps.setObject(1, d);
ps.addBatch();
}
ps.executeBatch();
connection.commit();
This will be significantly faster than individual inserts on any table with indexes.
This is a method I used in order to insert some data in an Oracle SQL database.
private boolean submit(Connection con, String query){
try {
PreparedStatement preStatement;
preStatement = con.prepareStatement(query);
preStatement.executeQuery();
preStatement.close();
return true;
}catch (Exception e) {
System.out.println("Exception cought, updating log.");
return false;
}
}
You can prepare your insert statement and call this function to perform the action. Call it using your connection object and the query. It shall return true on completion false in case something goes wrong. If you want to log any errors, use e.getMessage() to get the error message as a String in the exception block.
As mentioned in the comments, try to use the PreparedStatement object to avoid SQL Injection attacks and also try to trim any ' you might have in your data.
Here's how I'd recommend you do it. A few thoughts:
Give the Connection to the object. That method should do one thing: the INSERT.
Should be transactional.
Should clean up resources when done.
Tell users to provide a List of Doubles if that is what the amounts are. Don't parse; let clients do that.
Here is complete code:
public class Inserter {
private static final String INSERT_SQL = "INSERT INTO storage_other(total) VALUES(?))";
private Connection connection;
public Inserter(Connection connection) {
this.connection = connection;
}
public int settingAmount(List<Double> amounts)throws SQLException {
int numAmountsInserted = 0;
PreparedStatement ps = null;
this.connection.setAutoCommit(false);
try {
ps = this.connection.prepareStatement(INSERT_SQL);
for(Double amount : amounts) {
ps.setDouble(1, amount);
numAmountsInserted += ps.executeUpdate();
}
this.connection.commit();
} catch (SQLException e) {
DatabaseUtils.rollback(this.connection);
throw e;
} finally {
DatabaseUtils.close(ps);
this.connection.setAutoCommit(true);
}
return numAmountsInserted;
}
}