I am inserting 132,000 records in a database table using jdbc batch of prepated statement. The problem i am facing is all records are not inserted in to table only records 1444 are inserted.
try{
StringBuffer insert = new StringBuffer("INSERT INTO mytable (field1,field2,
,field3,field 4 ) VALUES ( ?, ?, ?, ? )");
pstmt = conn.prepareStatement(insert.toString());
Iterator cptIcd9Iterator = cptIcd9List.iterator();
while(cptIcd9Iterator.hasNext()){
cptIcd9VO = (CptIcd9VO)cptIcd9Iterator.next();
count++;
pstmt.setString(1, "field1");
pstmt.setString(2, "field2");
pstmt.setString(3, "field3");
pstmt.setInt(4, 4);
pstmt.addBatch();
}
updateCounts = pstmt.executeBatch();
}
catch (Exception e) {
logger.error(e);
}
Can anyone help me ?
You can't insert all of them in one go, you must do it step by step.
Read this SO post.
Related
I am using batchUpdate with jdbc template.In that I want to increment a table column based on table id.I am using below code but getting exception.
String sql = "INSERT INTO TEMP (Table_Id, STATUS, VERSION,Name) " +
"VALUES (?, ?, SELECT (VERSION+1) FROM (SELECT VERSION, Table_Id, MAX(VERSION) OVER(PARTITION BY Table_Id) MAX_V FROM TEMP ) WHERE Table_Id = ? AND VERSION = MAX_V, ?)";
getJdbcTemplate().batchUpdate(sql, new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i){
try {
TDetails customer = chngCustStus.get(i);
ps.setInt(1, customer.getTableId());
ps.setInt(1, customer.getStatus());
ps.setInt(1, customer.getTableId());
ps.setInt(1, customer.getName());
} catch (SQLException e) {
//TODO Auto-generated catch block
e.printStackTrace();
}
}
#Override
public int getBatchSize() {
return chngCustStus.size();
}
I think getJdbcTemplate().batchUpdate is not possible to increment sequence generation because Once total insert list prepared as batch file then after connect to data base,So before connection not possible to sequence increment.
I have an "Invitation" object that is modeled in a MySQL database. This object has one list ("treatmentPlanIDsToCopyf") and is maintained in the database with a second table. The method I have written to insert into the main table and then loop through the list and insert records for each item in the list into the second table is below. At the line ps = cn.prepareStatement(sql);Eclipse is giving me a warning that says "Resource leak: 'ps' is not closed at this location". I am closing the prepared statement in the finally clause, so I wanted to know if there really is a resource leak I need to fix. This is my first time using batches with prepared statements, so I wasn't really sure. Thanks.
public void invitationCreate(Connection cn, Invitation invitation) throws SQLException{
PreparedStatement ps = null;
try {
//first insert primary invitation data into the invitation table
String sql = "INSERT INTO invitiation (invitation_code, recipient_email, sender_user_id_fk, date_intived, date_accepted, accepted, recipient_first_name, recipient_last_name) VALUES (?, ?, ?, ?, ?, ?, ?, ?)";
ps = cn.prepareStatement(sql);
ps.setString(1, invitation.getInvitationCode());
ps.setString(2, invitation.getRecipientEmail());
ps.setInt(3, invitation.getSenderUserID());
ps.setTimestamp(4, convertLocalTimeDateToTimstamp(invitation.getDateInvited()));
ps.setTimestamp(5, convertLocalTimeDateToTimstamp(invitation.getDateAccepted()));
ps.setBoolean(6, invitation.isAccepted());
ps.setString(7, invitation.getRecipientFirstName());
ps.setString(8, invitation.getRecipientLastName());
int success = ps.executeUpdate();
//now loop through all the treatmentPlanIDs in the invitation that are to be copied into the invitees account when the register
sql = "INSERT INTO invitation_treatment_plans (invitation_code_fk, invitation_treatment_plan_id_fk) VALUES (?, ?)";
ps = cn.prepareStatement(sql);//TODO confirm this if this is actually a resource leak
for(int treatmentPlanID : invitation.getTreatmentPlanIDsToCopy()){
ps.setString(1, invitation.getInvitationCode());
ps.setInt(2, treatmentPlanID);
ps.addBatch();
}
ps.executeBatch();
} finally {
DbUtils.closeQuietly(ps);
}
}
I believe the leak is in the first prepared statement.
After int success = ps.executeUpdate(); you need to close that prepared statement before you assign the variable to a new prepared statement.
Now I have to update this into my database, the problem is I don't know how to update it multiple times in the database.
public void generate() {
KpiMsg804 upd = createKpiMsg804();
try {
st = conn.createStatement();
System.out.println("Updating Values");
//Updating Values into DB
String query = "insert into msg_new_to_bde("
+ "tablename,action,keyinfo1,keyinfo2) values(?, ?, ?, ?)";
pstmt = conn.prepareStatement(query); // create a statement
pstmt.setString(1,upd.getTableName());
pstmt.setInt(2,upd.getAction()); // set value of staus of action
pstmt.setString(3,upd.getKeyInfo1()); // set keyinfo value1
pstmt.setString(4,upd.getKeyInfo2()); // set keyinfo value2
int rows = pstmt.executeUpdate();
System.out.println("Number of Rows Updated" +rows);
// execute insert statement
} catch (SQLException e) {
e.printStackTrace();
}
You probably want to use the JDBC addBatch() and executeBatch() functionality: JDBC insert multiple rows . This will let you queue up your multiple rows, then insert them in one group.
The other interpretation of your question is: you need to turn the autocommit off, then use the JDBC begin() and commit() functionality to insert into several different tables, and they all go in at the same time.
pstmt.executeUpdate(Table1);
pstmt.executeUpdate(Table2);
conn.commit();
This will insert all the rows as one transaction, that is all at once.
i have a method which has to insert into two tables, i have used batch insert for that.
my problem is it takes the second query and inserts in to that table and fails to insert into first table that is fails to take first query.
this is what is my code :
public String saveVillageDetails(VillagesViewModel viewmodel, String functionType) {
int[] saveOrupdateStatus = null;
StringBuffer disctrictQuery = new StringBuffer();
String saveOrupdateStatusMessage = "";
Connection connection = getConnection();
PreparedStatement districtPS = null;
ResultSet rs = null;
if (connection != null) {
try {
connection.setAutoCommit(false);
if(functionType.equals("add")){
// to insert in villages table
disctrictQuery.append(" INSERT INTO m_villages(districtid,village,creation_date,last_created_by) ");
disctrictQuery.append(" VALUES (?, ?, ?,?) ");
districtPS = connection.prepareStatement(disctrictQuery.toString());
districtPS.setInt(1, viewmodel.getDistrictid());
districtPS.setString(2, viewmodel.getVillage());
districtPS.setTimestamp(3, getCurrentDate());
districtPS.setInt(4,viewmodel.getUserid());
districtPS.addBatch();
// to insert in regions table
disctrictQuery=new StringBuffer();
disctrictQuery.append(" INSERT INTO m_regions(villageid,creation_date,last_created_by) ");
disctrictQuery.append(" VALUES (?, ?, ?) ");
districtPS = connection.prepareStatement(disctrictQuery.toString());
districtPS.setInt(1, getLatestVilalgeID());
districtPS.setTimestamp(2, getCurrentDate());
districtPS.setInt(3,viewmodel.getUserid());
districtPS.addBatch();
}else if(functionType.equals("edit")){
// to update villages table
disctrictQuery.append("UPDATE m_villages SET districtid=? ,village=? ,updation_date=? ,last_updated_by=? ");
disctrictQuery.append(" WHERE villageid=? ");
districtPS = connection.prepareStatement(disctrictQuery.toString());
districtPS.setInt(1, viewmodel.getDistrictid());
districtPS.setString(2, viewmodel.getVillage());
districtPS.setTimestamp(3, getCurrentDate());
districtPS.setInt(4,viewmodel.getUserid());
districtPS.setInt(5,viewmodel.getVillageid());
districtPS.addBatch();
disctrictQuery=new StringBuffer();
// to update regiions table
disctrictQuery.append("UPDATE m_regions SET villageid=?,updation_date=? ,last_updated_by=? ");
disctrictQuery.append(" WHERE regionid=? ");
districtPS = connection.prepareStatement(disctrictQuery.toString());
districtPS.setInt(1, viewmodel.getVillageid());
districtPS.setTimestamp(2, getCurrentDate());
districtPS.setInt(3,viewmodel.getUserid());
districtPS.setInt(4,getRegionID(viewmodel.getVillageid()));
districtPS.addBatch();
}
saveOrupdateStatus = districtPS.executeBatch();
if (saveOrupdateStatus.length > 0) {
if(functionType.equals("add")){
saveOrupdateStatusMessage = "Village Details Added successfully";
}else if(functionType.equals("edit")){
saveOrupdateStatusMessage = "Village Details Modified successfully";
}
connection.commit();
connection.setAutoCommit(true);
} else {
if(functionType.equals("add")){
saveOrupdateStatusMessage = "Failed to Add Village Details";
}else if(functionType.equals("edit")){
saveOrupdateStatusMessage = "Failded to Modify Village Details";
}
}
} catch (Exception ex) {
ex.printStackTrace();
//use log to print the exception ex.printStackTrace();
} finally {
try {
closeConnection(connection, rs, districtPS);
} catch (Exception ex) {
ex.printStackTrace();
//use logger here
}
}
}
return saveOrupdateStatusMessage;
}
when i printed the statement i get like this :
com.mysql.jdbc.ServerPreparedStatement[2] - INSERT INTO m_regions(villageid,creation_date,last_created_by) VALUES (18, '2012-11-06 17:41:25', 1)
and the first query is missing in the batch.
what could be the problem.
Please help
Regards
Please refactor your code in the way as below:
if(functionType.equals("add")){
// to insert in villages table
districtPS = connection.prepareStatement("INSERT INTO m_villages(districtid,village,creation_date,last_created_by) VALUES (?, ?, ?,?) ",
Statement.RETURN_GENERATED_KEYS);
districtPS.setInt(1, viewmodel.getDistrictid());
districtPS.setString(2, viewmodel.getVillage());
districtPS.setTimestamp(3, getCurrentDate());
districtPS.setInt(4,viewmodel.getUserid());
districtPS.executeUpdate();
ResultSet keySet = districtPS.getGeneratedKeys();
int villageId = 0;
if (keySet.next()) {
villageId = keySet.getInt(1);
}
// to insert in regions table
districtPS = connection.prepareStatement(" INSERT INTO m_regions(villageid,creation_date,last_created_by) VALUES (?, ?, ?) ",
Statement.RETURN_GENERATED_KEYS);
districtPS.setInt(1, villageId);
districtPS.setTimestamp(2, getCurrentDate());
districtPS.setInt(3,viewmodel.getUserid());
districtPS.executeUpdate();
}
This question already has answers here:
java.sql.SQLException: Column count doesn't match value count at row 1
(3 answers)
Closed 6 years ago.
I have encountered an error, java.sql.SQLException:
Column count doesn't match value count at row 1
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3609)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3541)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2002)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2624)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2127)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2427)
It's super frustrating, because I have been making amendments to the code like changing in the arg to all string, but the error still appears.
public void readDataBase(int val, String d1, String d2, String d3, String d4 ) throws Exception {
try {
// This will load the MySQL driver, each DB has its own driver
Class.forName("com.mysql.jdbc.Driver");
// Setup the connection with the DB
connect = DriverManager
.getConnection("jdbc:mysql://localhost/MAXdb?"+ "user=root&password=");
// Statements allow to issue SQL queries to the database
statement = connect.createStatement();
// Result set get the result of the SQL query
resultSet = statement
.executeQuery("select * from MAXdb.emcsg");
writeResultSet(resultSet);
// PreparedStatements can use variables and are more efficient
preparedStatement = connect
.prepareStatement("insert into MAXdb.emcsg values (default,?, ?, ? , ?, ?)");
// Parameters start with 1
preparedStatement.setInt(1, val);
preparedStatement.setString(2, d1);
preparedStatement.setString(3, d2);
preparedStatement.setString(4, d3);
preparedStatement.setString(5, d4);
preparedStatement.executeUpdate();
preparedStatement = connect
.prepareStatement("SELECT id, Date, Time, Demand, SUPPLY from MAXdb.emcsg");
resultSet = preparedStatement.executeQuery();
writeResultSet(resultSet);
} catch (Exception e) {
throw e;
} finally {
close();
}
}
My second class:
public void Csvreader() throws IOException {
try {
// TODO code application logic here
CSVReader reader = new CSVReader(new FileReader("D:/TEST.csv"));
String nextLine[];
int i = 1;
Mysql sen = new Mysql();
while ((nextLine = reader.readNext()) != null) {
try {
sen.readDataBase( i, nextLine[0], nextLine[1], nextLine[2], nextLine[3] );
i = i+1;
} catch (Exception ex) {
Logger.getLogger(Opencsv.class.getName()).log(Level.SEVERE, null, ex);
}
}
} catch (FileNotFoundException ex) {
Logger.getLogger(Opencsv.class.getName()).log(Level.SEVERE, null, ex);
}
}
Database:
Field Type Collation Attributes Null Default Extra Action
id int(11) No None
Date text utf8_general_ci No None
Time text utf8_general_ci No None
Demand text utf8_general_ci No None
SUPPLY text utf8_general_ci
Well, I suspect this is the problem:
insert into MAXdb.emcsg values (default,?, ?, ? , ?, ?)
You haven't specified which column each of those parameters is meant to refer to - and I suspect you've not got 6 columns. Even if you do have 6 columns in the table, it would be a good idea to explicitly state in the SQL which column you mean to use for each parameter.
It seems that the following statement:
insert into MAXdb.emcsg values (default,?, ?, ? , ?, ?)
causes an error. Check if emcsg has 6 columns.
Had you tried changing this
preparedStatement = connect
.prepareStatement("insert into MAXdb.emcsg values (default,?, ?, ? , ?, ?)");
to this
preparedStatement = connect
.prepareStatement("insert into MAXdb.emcsg values (?, ?, ? , ?, ?)");
That might work for you.
Regards
It seems to me, that there is a problem with your database structure. I think you don't have all the fields you assume you have.
Do you definitely have six columns in the database table?
Always google an error message or error code. Quite often the first link returned will clearly explain the problem and provide a solution.
Your method signature
public void readDataBase(int val, String d1, String d2, String d3, String d4 )
has 5 variables, but your method call has 6:
preparedStatement = connect
.prepareStatement("insert into MAXdb.emcsg values (default,?, ?, ? , ?, ?)");