I am trying update a large set of rows (around 5M). I first came across the heap overflow issue of having so many rows fetched in a resultset. Since I don't want to raise my heap size on this machine I was wondering if there is an effective way of doing this.
I tried setting the setFetchSize(Integer.MIN_VALUE) but then when I call the update function I get this error:
Streaming result set com.mysql.jdbc.RowDataDynamic#2087c268 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
If I call close() on the result set I cannot update it of course. Here is my code
public void getRows()
{
Statement stmt = null;
ResultSet rs = null;
int countSpecialChars = 0;
int upper = 0, lower = 0, digits =0;
String pass = null;
int id = 0;
char thisChar;
String query = "select id,pass from datatable";
try {
this.conn.setAutoCommit(false);
stmt = this.conn.createStatement();
stmt.setFetchSize(Integer.MIN_VALUE);
rs = stmt.executeQuery(query);
while (rs.next()) {
pass = rs.getString(2).trim();
id = rs.getInt(1);
for (int i=0; i<=pass.length()-1; i++)
{
thisChar= pass.charAt(i);
if (thisChar >= 65 && thisChar <= 90) {
upper++;
} else if (thisChar >= 97 && thisChar <= 122) {
lower++;
} else if( thisChar >= 48 && thisChar <= 57) {
digits++;
}
else
{countSpecialChars++;}
}
Entropy entropy = new Entropy();
double naiveEntropy = entropy.naiveEntropy(pass);
NumberFormat formatter = new DecimalFormat("#0.00");
this.updateRow(id, pass.length(), digits, upper, lower, countSpecialChars, Double.parseDouble(formatter.format(naiveEntropy)));
countSpecialChars = 0;
upper=digits=0;
lower = 0;
}
rs.close();
}
catch (SQLException e)
...
}
public void updateRow(int id, int length, int numbers, int upper,
int lower, int specialChars, double naiveEntropy )
{
PreparedStatement updatePassEntry = null;
String updateString = "update cwlCompatiblePassUnique " +
"set length = ?, numbers = ?, upper = ?, lower = ?, specialChars = ?, ShannonEntropy = ? where id = ?";
try {
this.conn.setAutoCommit(false);
updatePassEntry = this.conn.prepareStatement(updateString);
updatePassEntry.setInt(1, length);
...
updatePassEntry.setInt(7, id);
updatePassEntry.executeUpdate();
this.conn.commit();
}
catch (SQLException e)
...
}
Any ideas on what can be done?
Thanks
you call updateRow() method inside the rs.next() loop; which tries to make a SQL update on a SQL field (id) that is currently being processed inside your while (rs.next()) loop. this will raise the error you get. i suggest you write a method for pulling rs and storing them in java objects vector as a first step. this method will close the rs after exiting. then write another method to do both processing and update data on your cached vector objects .
something like this:
private void Vector<DataSet> getDataSet(){
Vector<DataSet> data=new Vector<DataSet>();
String query = "select id,pass from datatable";
try {
this.conn.setAutoCommit(false);
stmt = this.conn.createStatement();
stmt.setFetchSize(Integer.MIN_VALUE);
rs = stmt.executeQuery(query);
while (rs.next()) {
pass = rs.getString(2).trim();
id = rs.getInt(1);
data.addElement(new DataSet(id,pass));
}
}catch(Exception e){
// here close connection and rs
}
}
private void udpateData(Vector<dataSet> data){
//process data and update her
}
static class DataSet{
int id;
String pass;
//constructor here
}
Connection object should not hold multiple resultset object at a time.
After creation of ResultSet and Statement Objects, each has to close explicitly like,
resultSet.close()
statement.close()
Related
I'm trying to take two random rowid from my database. Everything works but I have a scenario when there is only one rowid. I want to make a loop on my try/catch until there is second number in my database.
What I'm doing wrong? Thank you
public void Kaslaimejo() {
String sql = "SELECT rowid FROM Zaidejai WHERE Pirmas < 4 ORDER BY random() LIMIT 2";
Integer value1 = null, value2 = null;
Integer judesiukas1 = null, judesiukas2 = null;
int a = 0;
int k = 15; // kiek kartu? Reikia infinity padaryti
for (a = 0; a < 3; a++) {
try {
Connection conn = Serveris.connect();
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(sql)) {
if (rs.next()) {
value1 = rs.getInt("rowid");
if (rs.next()) {
value2 = rs.getInt("rowid");
PreparedStatement buvo = conn.prepareStatement("UPDATE Zaidejai SET Numeriukas = ? WHERE rowid = ?");
buvo.setInt(1, i);
buvo.setInt(2, value1);
int buvolala = buvo.executeUpdate ();
PreparedStatement buvo2 = conn.prepareStatement("UPDATE Zaidejai SET Numeriukas = ? WHERE rowid = ?");
buvo2.setInt(1, i);
buvo2.setInt(2, value2);
int buvolala2 = buvo2.executeUpdate ();//
i++;
}
System.out.println("Pirmas zaidejas" + value1); // atspausdina 1 random zaideja is duomenu bazes
System.out.println("Antras zaidejas" + value2); // atspausdina 2 random zaideja is duomenu bazes
}
} catch (SQLException e) {
a--;
//System.out.println(e.getMessage());
}
}
}
Right now my program loops two times and then gives me SQLException. How I can loop my program until there is no SQLException?
OK, I've tried to write what I think you're trying to do.
You wait for ever until someone puts at least two entries in the database.
You extract two values, process them, then wait some more.
Some points to watch out:
1. Object comparisons need to be made with .equals() not with ==
2. You might want to provide some way to break out of the infinite loop I've written (while(true)).
3. Careful with null values. They might produce NullPointerException.
4. Try to break up your code into methods. Each large block of code could go into each own method.
public void Kaslaimejo(){
String sql = "SELECT rowid FROM Zaidejai WHERE Pirmas < 4 ORDER BY random() LIMIT 2";
Integer judesiukas1 = null, judesiukas2 = null;
while(true) {
List<Integer> values = new ArrayList<>();
while (values.size() < 2) {
try (Connection conn = Serveris.connect();
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(sql)) {
if( rs.next() ){
Integer value = rs.getInt("rowid");
values.add(value);
}
} catch (SQLException e) {
e.printStackTrace();
}
}
try( Connection conn = Serveris.connect()) {
PreparedStatement buvo = conn.prepareStatement("UPDATE Zaidejai SET Numeriukas = ? WHERE rowid = ?");
buvo.setInt(1, i);
buvo.setInt(2, values.get(0));
int buvolala = buvo.executeUpdate ();
PreparedStatement buvo2 = conn.prepareStatement("UPDATE Zaidejai SET Numeriukas = ? WHERE rowid = ?");
buvo2.setInt(1, i);
buvo2.setInt(2, values.get(1));
int buvolala2 = buvo2.executeUpdate ();//
i++;
}catch (SQLException e) {
e.printStackTrace();
}
Connection conn = Serveris.connect();
try {
PreparedStatement pstmt = conn.prepareStatement("SELECT Pirmas FROM Zaidejai WHERE rowid = ?");
PreparedStatement pstmt2 = conn.prepareStatement("SELECT Pirmas FROM Zaidejai WHERE rowid = ?");
pstmt.setInt(1, values.get(0));
pstmt2.setInt(1, values.get(1));
ResultSet myrsv = pstmt.executeQuery();
ResultSet myrsv2 = pstmt2.executeQuery();
{
if (myrsv.next()) {
judesiukas1 = myrsv.getInt("Pirmas");
if (myrsv2.next()) {
judesiukas2 = myrsv2.getInt("Pirmas");
}
}
//System.out.println("Pirmo zaidejo veiksmas" + myrsv.getInt("Pirmas"));
//System.out.println("Antro zaidejo veiksmas" + myrsv2.getInt("Pirmas"));
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
if (judesiukas1.equals(judesiukas2)) // careful here. NullPointerException may happen.
{
try {
PreparedStatement laim = conn.prepareStatement("UPDATE Zaidejai SET Rezultatas = ? WHERE rowid = ?"); // ble ble update reikia naudoti , o ne insert into. Insert kai sukuriame nauja kazka tik
PreparedStatement laim2 = conn.prepareStatement("UPDATE Zaidejai SET Rezultatas = ? WHERE rowid = ?");
laim.setString(1, "Lygiosios");
laim.setInt(2, values.get(0));
laim2.setString(1, "Lygiosios");
laim2.setInt(2, values.get(1));
int irasyk = laim.executeUpdate (); // kodel executeupdate, o ne executequery????
int irasyk2 = laim2.executeUpdate (); // kodel executeupdate, o ne executequery????
{
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.print("Lygiosios");
} else {
// (1) - Rock
// (2) Scissors
// (3) - Paper
switch (values.get(0)){
case 1:
if (judesiukas2 == 2)
System.out.print("Zaidejas 1 wins!");
else
System.out.print("Zaidejas 2 wins!");
break;
case 2:
if (judesiukas2 == 3)
System.out.print("Zaidejas 1 wins!");
else
System.out.print("Zaidejas 2 wins!");
break;
case 3:
if (judesiukas2 == 1)
System.out.print("Zaidejas 1 wins!");
else
System.out.print("Zaidejas 2 wins!");
break;
}
}
try {
conn.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The logic becomes easier if you add the values to a list
var values = new ArrayList<Integer>();
while (values.Count < 2) {
try (Connection conn = Serveris.connect();
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(sql))
{
while (values.Count < 2 && rs.next()) {
Integer v = rs.getInt("rowid");
values.Add(v);
}
} catch (SQLException e) {
}
}
//TODO: process the values here
The advantage is, that you can retrieve one value at the first database query and the second at a later one or both in the same round and you don't have to keep track of which one of two variables to use.
(Bear with me with the syntax details, I'm not a Java programmer.)
How i can loop my program until there is no SQLException?
Change this (because, it will only allow to loop two times)
for (a = 0; a < 2; a++) {
to
while(true)
Put everything inside while(true), if exception occurred, then it will come out from the while loop. Something similar :
try
{
while(true)
{
...
...
}
...
}
catch(SQLException e)
{
// do somthing
}
I am having an OOME problem when trying to run a native SQL query through the jpa / hibernate EM.
The treatment serves to make millions of insertion per pack of 50.
Here is code of my algorithm:
private void createNewJetonsForIndividus(boolean isGlobal, List<String> entreprises, List<String> services,
String user, Timestamp dateDB) {
LocalDateTime timer = LocalDateTime.now();
List<Object[]> MinMaxId = getMinMaxIdDroitsIndividusActifsForCreation(
isGlobal, entreprises, services);
if (null != MinMaxId.get(0)[0]) {
int idStart = ((BigInteger) MinMaxId.get(0)[0]).intValue();
int idEnd = idStart + PAS;
int idMax = ((BigInteger) MinMaxId.get(0)[1]).intValue();
int nbRowsTotal = 0;
Logger.debug("Droits Individus : ID Min {} - ID Max {}", idStart, idMax);
do {
int finalIdStart = idStart;
int finalIdEnd = idEnd;
callTransaction(() -> create(false, true,isGlobal, entreprises, services, finalIdStart,
finalIdEnd, user, dateDB));
idStart = idEnd + 1;
idEnd = idEnd + PAS;
}
while (idMax > idEnd);
}
}
the method is used to calculate the id min and max of the records that interest my treatment. Subsequently, I use the create method whose code is below :
int nbRowsFind;
List<Object[]> listeDroitsIndividusActifsForCreation = getDroitsIndividusActifsForCreation(
isGlobal, entreprises, services, idStart, idEnd);
if (ValidationUtils.isNotEmpty(listeDroitsIndividusActifsForCreation)) {
nbRowsFind = listeDroitsIndividusActifsForCreation.size();
StringBuilder sbJeton = new StringBuilder();
sbJeton.append("INSERT INTO sigs_nv_jeton VALUES ");
StringBuilder sbDroitHasJeton = new StringBuilder();
if (isCreateForIndiv) {
sbDroitHasJeton.append("INSERT INTO sigs_droits_individu_has_nv_jeton VALUES ");
}
listeDroitsIndividusActifsForCreation.stream().forEach(object -> {
sbJeton.append("(");
sbDroitHasJeton.append("(");
BigInteger idDroit = (BigInteger) object[0];
String jetonGenerated = IdJetonGenerator.codeGenerator(idDroit.toString(), DateUtils.now());
sbJeton.append("'").append(jetonGenerated).append("', ");
appendDate(sbJeton, object[1]);
appendDate(sbJeton, object[2]);
sbJeton.append(0).append(", ");
sbJeton.append(0).append(", ");
sbJeton.append("'").append(dateDB).append("', ");
sbJeton.append("'").append(user).append("'");
sbDroitHasJeton.append(idDroit).append(",'").append(jetonGenerated).append("'");
sbJeton.append("),");
sbDroitHasJeton.append("),");
});
String requestJeton = sbJeton.toString();
sbJeton.delete(33, sbJeton.length());
requestJeton = requestJeton.substring(0, requestJeton.length() - 1);
jpaApi.em().createNativeQuery(requestJeton).executeUpdate();
String requestDroitHasJeton = sbDroitHasJeton.toString();
sbDroitHasJeton.delete(54, sbDroitHasJeton.length());
requestDroitHasJeton = requestDroitHasJeton.substring(0, requestDroitHasJeton.length() - 1);
**jpaApi.em().createNativeQuery(requestDroitHasJeton).executeUpdate();**
**jpaApi.em().flush();
jpaApi.em().clear();**
}
When I analyze the Heap Dump, I notice that despite the flush and clear, queries are still referenced in the SessionFactory, is this normal?
enter image description here
flush is executing changes made in the unit of work and clear removes entities from the persistence context.
Both have nothing to do with native SQL queries. I assume that the native queries are cached anywhere else (Hibernate, JDBC...)
I would suggest that you use a prepared statement instead of your dynamic insert statement.
This is my new implementation :
Statement statement = null;
ResultSet resultSet = null;
int nbRows = 0;
try {
statement = connection.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_READ_ONLY);
try {
resultSet = statement.executeQuery(queryGetDroitsEntreprisesActifsForCreation(isGlobal, entreprises, services));
resultSet.beforeFirst();
while (resultSet.next()) {
nbRows += 1;
queryInsertJetonsAndLinkDroitsJeton(isCreateForEntreprise, isCreateForIndiv, connection, resultSet,
user, dateDB);
}
System.out.println(nbRows);
} finally {
if (resultSet != null) {
resultSet.close();
}
}
} finally {
if (statement != null) {
statement.close();
}
}
The method "queryGetDroitsEntreprisesActifsForCreation" returns a SQL query in String.
The method "queryInsertJetonsAndLinkDroitsJeton" use the famous PreparedStatement :
PreparedStatement psJeton = null;
PreparedStatement psDroitJeton = null;
try {
String sbJeton = "INSERT INTO TABLE1 VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ";
psJeton = connection.prepareStatement(sbJeton);
String jetonGenerated;
if (isCreateForEntreprise) {
jetonGenerated = IdJetonGenerator.codeGenerator(String.valueOf(resultSet.getInt("id_e")), DateUtils.now());
} else {
jetonGenerated = IdJetonGenerator.codeGenerator(String.valueOf(resultSet.getInt("id_i")), DateUtils.now());
}
psJeton.setString(1, jetonGenerated);
psJeton.setString(2, ContextType.GD.name());
psJeton.setString(3, JetonType.ANONYME.name());
psJeton.setInt(4, 1);
psJeton.setTimestamp(5, resultSet.getTimestamp("dt_debut"));
psJeton.setTimestamp(6, resultSet.getTimestamp("dt_fin"));
psJeton.setInt(7, 0);
psJeton.setInt(8, 0);
psJeton.setInt(9, 0);
psJeton.setInt(10, 0);
psJeton.setTimestamp(11, dateDB);
psJeton.setString(12, user);
psJeton.setTimestamp(13, dateDB);
psJeton.setString(14, user);
psJeton.executeUpdate();
String sbDroitHasJeton = null;
if (isCreateForEntreprise) {
sbDroitHasJeton = "INSERT INTO sigs_dej VALUES (?, ?)";
}
if (isCreateForIndiv) {
sbDroitHasJeton = "INSERT INTO sigs_dij VALUES (?, ?)";
}
psDroitJeton = connection.prepareStatement(sbDroitHasJeton);
if(isCreateForEntreprise) {
psDroitJeton.setInt(1, resultSet.getInt("id_e"));
} else {
psDroitJeton.setInt(1, resultSet.getInt("id_i"));
}
psDroitJeton.setString(2, jetonGenerated);
psDroitJeton.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
} finally {
if (psJeton != null) {
psJeton.close();
}
if (psDroitJeton != null) {
psDroitJeton.close();
}
}
I hope this is the best implementation of PreparedStatement & Scrollable ResultSet
In here i want to obtain data from the MySQL DB & run the play method.but it execute at once.i wanted to get row one information pass that in to the ply method & get the second row information & pass it to the play method so on.
please help me to resolve this.
thank u in advance.
public class Player {
static Play PL = new Play();
public static void main(String[] args) {
try {
Statement stmt = null;
// connect to database radio
Connection conn = DriverManager.getConnection("jdbc:mysql://localhost/radio", "root", "");
stmt=conn.createStatement();
String sql = "SELECT Link FROM split";
ResultSet rs = stmt.executeQuery(sql);
//STEP 5: Extract data from result set
while(rs.next()){
//Retrieve by column name
int numColumns = rs.getMetaData().getColumnCount();
for ( int i = 1 ; i <= numColumns ; i++ ) {
System.out.println(numColumns);
String SongLocation = rs.getString(i);
System.out.println(SongLocation);
PL.play(SongLocation);
System.out.println("playing song");
}
}
} catch (SQLException e1) {
e1.printStackTrace();
}
}
}
I think you are looping through columns by using
int numColumns = rs.getMetaData().getColumnCount();
for ( int i = 1 ; i <= numColumns ; i++ ) {
...
instead try using a while loop
while (rs.next()) {
...
and because you need the index, inlude a local variable int i, that you just increment after each while loop
In the below code I am copying resultset content to arraylist. First part of the wile loop i.e while(RS.next()) is returing the results but when cursor moves to
Next while loop i.e while(SR.next()) I am getting "result set is closed". Please help me where I am doing mistake.
String SSQ = "select DISTINCT S_NUMBER from OTG.S_R_VAL" +
" WHERE R_TS = (SELECT MAX(R_TS) FROM OTG.S_R_VAL) order by S_NUMBER";
String SDS = "SELECT DISTINCT S_NUMBER FROM OTG.S_R_VAL AS STG WHERE S_NUMBER NOT IN" +
"(SELECT S_NO FROM OTG.R_VAL AS REV WHERE STG.S_NUMBER = REV.S_NO )";
String SSR = "SELECT DISTINCT S_NO FROM OTG.R_VAL where S_NO != 'NULL' order by S_NO";
String SSO = "Select O_UID from OTG.OPTY where C_S_NO IN" +
"( SELECT DISTINCT S_NUMBER FROM OTG.S_R_VAL AS STG WHERE S_NUMBER NOT IN(SELECT S_NO FROM OTG.R_VAL AS REV WHERE STG.S_NUMBER = REV.S_NO ))";
//Statement statement;
try {
connection = DatabaseConnection.getCon();
statement = connection.createStatement();
statement1 = connection.createStatement();
statement2 = connection.createStatement();
statement3 = connection.createStatement();
statement4 = connection.createStatement();
ResultSet RS = statement1.executeQuery(selectQuery);
ResultSet DS = statement2.executeQuery(Distinct_SiebelNo);
ResultSet SR = statement3.executeQuery(SiebelNo_Rev);
ResultSet SO = statement4.executeQuery(selected_OppId);
ArrayList<String> RSList = new ArrayList<String>();
ArrayList<String> SRList = new ArrayList<String>();
/* ResultSetMetaData resultSetMetaData = RS.getMetaData();
int count = resultSetMetaData.getColumnCount();*/
int count=1;
System.out.println("******count********"+count);
while(RS.next()) {
int i = 1;
count=1;
while(i < count)
{
RSList.add(RS.getString(i++));
}
System.out.println(RS.getString("SIEBEL_NUMBER"));
RSList.add( RS.getString("SIEBEL_NUMBER"));
}
/* ResultSetMetaData resultSetMetaData1 = SR.getMetaData();
int count1 = resultSetMetaData1.getColumnCount();*/
int count1=1;
while(SR.next()) {
int i = 1;
while(i < count1)
{
SRList.add(SR.getString(i++));
}
System.out.println(SR.getString("SIEBEL_NO"));
SRList.add( SR.getString("SIEBEL_NO"));
}SR.close();
connection.commit();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
The logic of each loop is flawed.
int count=1;//Count is being set to one
while(RS.next()) {
int i = 1;//i is being set to one
count=1;//count again set to one
while(i < count) //condition will always fail as one is never less than one
{
RSList.add(RS.getString(i++));//Code is never Reached
}
System.out.println(RS.getString("SIEBEL_NUMBER"));
RSList.add( RS.getString("SIEBEL_NUMBER"));
}
The second while is not needed. Just use this:
int count = 1;
while(RS.next()) {
RSList.add(RS.getString(count++));
System.out.println(RS.getString("SIEBEL_NUMBER"));
RSList.add( RS.getString("SIEBEL_NUMBER"));
}
EDIT
int count1=1;
while(SR.next()) {
SRList.add(SR.getString(count1++));
System.out.println(SR.getString("SIEBEL_NO"));
SRList.add( SR.getString("SIEBEL_NO"));
}
EDIT 2:
for (String s : RSList)
for(String s1 : SRList)
if (s.equals(s1))
//Do what you need
You are using the first resultset (RS) in the second loop (System.out.println line)
I used the following codes to update column Clob in oracle, it seems to be okay and work properly, after performance testing, it reported that need consumed more than 200ms while the length of string is more than 130000. Is it any good way to improve it?
private void updateClobDetailsField(Map<Integer, String> idToDetails){
long s1 = System.currentTimeMillis();
Connection conn = null;
PreparedStatement pStmt = null;
ResultSet rset = null;
Map<Integer, Clob> idToDetailsClob = new HashMap<Integer, Clob>();
int BATCH_SIZE = CMType.BATCH_UPDATE_MAXSIZE;
try
{
conn = getConnection();
ServerAdapter adapter = ServerAdapter.getServerAdapter();
List<Integer> IDList = new ArrayList<Integer>();
for(Integer id : idToDetails.keySet()){
IDList.add(id);
}
List<Integer> tempIDList = new ArrayList<Integer>(IDList);
while(!tempIDList.isEmpty()){
int size = tempIDList.size() < BATCH_SIZE ? tempIDList.size() : BATCH_SIZE;
List<Integer> currentBatch = tempIDList.subList(0, size);
String inClause = SQLHelper.prepareInClause("ID",currentBatch.size());
pStmt = conn.prepareStatement("SELECT ID, DETAILS FROM PROGRAM_HISTORY WHERE " + inClause);
for(int i = 0; i < currentBatch.size(); i++){
pStmt.setInt(i+1, (currentBatch.get(i)));
}
rset = pStmt.executeQuery();
while(rset.next()){
int id = rset.getInt(1);
Clob detailsClob = rset.getClob(2);
Writer writer = adapter.getCharacterOutputStream(detailsClob);
String details = idToDetails.get(id);
if (details != null) {
writer.write(details);
}
writer.flush();
writer.close();
idToDetailsClob.put(id, detailsClob);
}
currentBatch.clear();
BaseSQLHelper.close(pStmt, rset);
}
int counter = 0;
pStmt = conn.prepareStatement("UPDATE PROGRAM_HISTORY SET DETAILS = ? WHERE ID = ?");
for(int i=0; i<IDList.size(); i++){
int index = 1;
Clob detailsClob = (Clob) idToDetailsClob.get(IDList.get(i));
pStmt.setClob(index++, detailsClob);
pStmt.setInt(index++, IDList.get(i));
pStmt.addBatch();
counter++;
if(counter % BATCH_SIZE == 0) {
pStmt.executeBatch();
pStmt.clearBatch();
counter = 0;
}
}
if(IDList.size() % BATCH_SIZE > 0) {
pStmt.executeBatch();
}
}
catch (SQLException se)
{
se.printStackTrace();
}
catch (IOException se)
{
se.printStackTrace();
}
finally
{
cleanup(conn, pStmt, null);
}
System.out.println(System.currentTimeMillis()-s1);
}
If I understand your code correctly, you are appending text to your details clob column.
Doing it in PL/SQL would be faster since you wouldn't have to fetch the clob across the network. For example you could prepare this statement:
DECLARE
l_details CLOB;
BEGIN
SELECT details INTO l_details FROM program_history WHERE ID = ?;
dbms_lob.append(l_details, ?);
END;
and bind currentBatch.get(i) and idToDetails.get(id).
Notice that you don't need an additional update with PL/SQL.
Execute your query with an updatable ResultSet so that you can update the data as you scroll through without separate update statements being executed.
You need to create your prepared statement with the resultSetConcurrency set to ResultSet.CONCUR_UPDATABLE. Check out the oracle documentation on dealing with streams for the various ways you can handle the clob data.