I'm truncating the table and then insert new data. Somehow we got error like (size more in data). It gives error but truncate statement is not rolled back.
Could you please suggest what is issue and how to roll it back?
try {
utx.begin();
List<Company> company = CompanyMapper.MAPPER.entityListToDaoList(query);
logger.log(Level.INFO,"Truncating Table Company!!");
emTarget.createNativeQuery("TRUNCATE TABLE Company").executeUpdate();
logger.log(Level.INFO,"Table Companyrole Company!!");
logger.log(Level.INFO,"Populating Table Company!! - " + company.size());
for (Company row : company) {
logger.log(Level.INFO,"ROW:" + row.getCompanyid()
+ "| Address:" + row.getVisitaddress1()
+ "| Size:" + (row.getVisitaddress1()!=null? row.getVisitaddress1().length():"0"));
emTarget.persist(row);
}
logger.log(Level.INFO,"Populated Table Company!!");
utx.commit();
} catch (Exception e) {
e.printStackTrace();
utx.rollback();
logger.log(Level.SEVERE, "Persist transaction failed. Rollback activated", e.getMessage());
throw new PersistenceException("There was an error reading the source table");
}
I'm using Oracle.
As noted in the Oracle documentation of TRUNCATE TABLE, you cannot roll back:
Note: You cannot roll back a TRUNCATE TABLE statement, nor can you use a FLASHBACK TABLE statement to retrieve the contents of a
table that has been truncated.
If you want the ability to roll back, you will need to use DELETE instead (this may be a lot slower than truncate).
Try using DELETE instead of TRUNCATE
TRUNCATE is DDL while DELETE is DML.
There are several database implementations where DDL cant be rollbacked.
You cannot roll back TRUNCATE. But if you have to - use DELETE instead.
Related
I want to insert multiple rows in a table from a console/tool (e.g.: Data studio) I get the following error message
THE INSERT OR UPDATE VALUE OF FOREIGN KEY FK$MAR$S IS INVALID.
SQLCODE=-530, SQLSTATE=23503, DRIVER=4.13.111
This means I have some trouble with a FOREIGN KEY variable, but I solved that later and it works well.
My problem is that when I'm running the same query from a Java application using PreparedStatement.executeBatch() (batch because it could insert more than one row at a time), I get a different error message:
com.ibm.db2.jcc.am.wn: [jcc][t4][102][10040][3.57.82] Batch failure.
The batch was submitted, but at least one exception occurred on an
individual member of the batch. Use getNextException() to retrieve
the exceptions for specific batched elements. ERRORCODE=-4228,
SQLSTATE=null
When I used getNextException(), I get the following:
com.ibm.db2.jcc.am.co: A NON-ATOMIC INSERT STATEMENT ATTEMPTED TO
PROCESS MULTIPLE ROWS OF DATA, BUT ERRORS OCCURRED
And the error code is -4228.
Why this difference? I want the java application return the same error details as the console tool, so I can handle those exceptions in my java code.
For example, if the returned error code=-803 which means duplicate exception, I would handle my code to make update instead of insert, or if the returned message contains some words like " FOREIGN KEY ", I'll tell user to make sure about lookup tables and so on
I use DB2 version 10.5.3 on z/OS and the DB2 driver version is : 3.65.92
} catch (SQLException ex) {
while (ex != null) {
if (ex instanceof com.ibm.db2.jcc.DB2Diagnosable) {
com.ibm.db2.jcc.DB2Diagnosable db2ex = (com.ibm.db2.jcc.DB2Diagnosable) ex;
com.ibm.db2.jcc.DB2Sqlca sqlca = db2ex.getSqlca();
if (sqlca != null) {
System.out.println("SQLCODE: " + sqlca.getSqlCode());
System.out.println("MESSAGE: " + sqlca.getMessage());
} else {
System.out.println("Error code: " + ex.getErrorCode());
System.out.println("Error msg : " + ex.getMessage());
}
} else {
System.out.println("Error code (non-db2): " + ex.getErrorCode());
System.out.println("Error msg (non-db2): " + ex.getMessage());
}
ex = ex.getNextException();
}
...
}
Above is an example of handling db2 exceptions. The example of output when there are 2 violations simultaneously: unique key on the table MYSCHEMA.MYTABLE where batch inserts come, and a foreign key on a parent table. I split it intentionally into 2 parts:
Before getNextException():
Error code: -4229
Error msg : [jcc][t4][102][10040][4.19.66] ... getNextException().
ERRORCODE=-4229, SQLSTATE=null
After getNextException():
SQLCODE: -803
MESSAGE: One or more values in the INSERT statement,
UPDATE statement, or foreign key update caused by a DELETE statement
are not valid because the primary key, unique constraint or unique
index identified by "1" constrains table "MYSCHEMA.MYTABLE" from
having duplicate values for the index key.. SQLCODE=-803,
SQLSTATE=23505, DRIVER=4.19.66
SQLCODE: -530
MESSAGE: The insert or update value of the FOREIGN KEY
"MYSCHEMA.MYTABLE.MYTABLE_FK" is not equal to any value of the parent
key of the parent table.. SQLCODE=-530, SQLSTATE=23503, DRIVER=4.19.66
I think the batch exception message is pretty clear. Consider that different statements in a batch might fail or issue warnings for different reasons. The batch level error message is therefore generic and instructs you to use "getNextException() to retrieve the exceptions for specific" statements in the batch.
Though this is an old thread. I will share the code which worked for me
try{
preparedStatement.batchUpdate( new ClassName{
//code with setting values and batch size});
}catch (Exception e) {
if (e.getCause() instanceof BatchUpdateException) {
BatchUpdateException be = (BatchUpdateException) e.getCause();
SQLException current = be.getNextException();
do {
current.printStackTrace();
} while ((current = current.getNextException()) != null);
}
}
In here I'm trying to get the exception based on BatchUpdateException instance.
we are extracting data from various database types (Oracle, MySQL, SQL-Server, ...). Once it is successfully written to a file we want to mark it as transmitted, so we update a specific column.
Our problem is, that a user has the possibility to change the data in the meantime but might forget to commit. The record is blocked with a select for update statement. So it can happen, that we mark something as transmitted, which is not.
This is an excerpt from our code:
Statement stmt = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_UPDATABLE);
ResultSet extractedData = stmt.executeQuery(sql);
writeDataToFile(extractedData);
extractedData.beforeFirst();
while (extractedData.next()) {
if (!extractedData.rowUpdated()) {
extractedData.updateString("COLUMNNAME", "TRANSMITTED");
// code will stop here if user has changed data but did not commit
extractedData.updateRow();
// once committed the changed data is marked as transmitted
}
}
The method extractedData.rowUpdated() returns false, because technically the user didn't change anything yet.
Is there any way to not update the row and detect if data was changed at this late stage?
Unfortunately I cannot change the program the user is using to change the data.
So you want to
Run through all rows of the table that have not been exported
Export this data somewhere
Mark these rows exported so your next iteration will not export them again
As there might be pending changes on a row, you don't want to mess with that information
How about:
You iterate over all rows.
for every row
generate a hash value for the contents of the row
compare column "UPDATE_STATUS" with calulated hash
if no match
export row
store hash into "UPDATE_STATUS"
if store fails (row locked)
-> no worries, will be exported again next time
if store succeeds (on data already changed by user)
-> no worries, will be exported again as hash will not match
This might further slow your export as you'll have to iterate over everything instead of over everything WHERE UPDATE_STATUS IS NULL but you might be able to do two jobs - one (fast)
iterating over WHERE UPDATE_STATUS IS NULL and one slow and thorough WHERE UPDATE_STATUS IS NOT NULL (with the hash-rechecking in place)
If you want to avoid store-failures/waits, you might want to store the hash /updated information into a second table copying the primary key plus the hash field value - that way user
locks on the main table would not interfere with your updates at all (as those would be on another table)
"a user [...] might forget to commit" > A user either commits or he doesn't. "Forgetting" to commit is tantamount to a bug in his software.
To work around that you need to either:
Start a transaction with isolation level SERIALIZABLE, and within that transaction:
Read the data and export it. Data read this way is blocked from being updated.
Update the data you processed. Note: don't do that with an updateable ResultSet, do that with an UPDATE statement. That way you don't need an CONCUR_UPDATABLE + TYPE_SCROLL_SENSITIVE which is much slower than a CONCUR_READ_ONLY + TYPE_FORWARD_ONLY.
Commit the transaction.
That way the buggy software will be blocked from updating data you are processing.
Another way
Start a TRANSACTION at a lower isolation level (default READ COMMITTED) and within that transaction
Select the data with proper Table Hints Eg for SQL Server these: TABLOCKX + HOLDLOCK (large datasets), or ROWLOCK + XLOCK + HOLDLOCK (small datasets), or PAGLOCK + XLOCK + HOLDLOCK. Having HOLDLOCK as a table hint is practically equivalent to having a SERIALIZABLE transaction. Note that lock escalation may escalate the latter two to table locks if the number of locks becomes too high.
Update the data you processed; Note: use an UPDATE statement. Lose the updatable/scroll_sensitive resultset.
Commit the TRANSACTION.
Same deal, the buggy software will be blocked from updating data you are processing.
In the end we had to implement optimistic locking. In some tables we already have a column that stores the version number. Some other tables have a timestamp column that holds the time of the last change (changed by trigger).
While a timestamp might not always be a reliable source for optimistic locking we went with it anyway. Several changes during a single second are not very realistic in our environment.
Since we have to know the primary key without describing it before hand, we had to access the resultset metadata. Some of our databases do not support this (DB/2 legacy tables for example). We are still using the old system for these.
Note: The tableMetaData is an XML-config file where our description of the table is stored. This is not directly related to the metadata of the table in the database.
Statement stmt = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_UPDATABLE);
ResultSet extractedData = stmt.executeQuery(sql);
writeDataToFile(extractedData);
extractedData.beforeFirst();
while (extractedData.next()) {
if (tableMetaData.getVersion() != null) {
markDataAsExported(extractedData, tableMetaData);
} else {
markResultSetAsExported(extractedData, tableMetaData);
}
}
// new way with building of an update statement including the version column in the where clause
private void markDataAsExported(ResultSet extractedData, TableMetaData tableMetaData) throws SQLException {
ResultSet resultSetPrimaryKeys = null;
PreparedStatement versionedUpdateStatement = null;
try {
ResultSetMetaData extractedMetaData = extractedData.getMetaData();
resultSetPrimaryKeys = conn.getMetaData().getPrimaryKeys(null, null, tableMetaData.getTable());
ArrayList<String> primaryKeyList = new ArrayList<String>();
String sqlStatement = "update " + tableMetaData.getTable() + " set " + tableMetaData.getUpdateColumn()
+ " = ? where ";
if (resultSetPrimaryKeys.isBeforeFirst()) {
while (resultSetPrimaryKeys.next()) {
primaryKeyList.add(resultSetPrimaryKeys.getString(4));
sqlStatement += resultSetPrimaryKeys.getString(4) + " = ? and ";
}
sqlStatement += tableMetaData.getVersionColumn() + " = ?";
versionedUpdateStatement = conn.prepareStatement(sqlStatement);
while (extractedData.next()) {
versionedUpdateStatement.setString(1, tableMetaData.getUpdateValue());
for (int i = 0; i < primaryKeyList.size(); i++) {
versionedUpdateStatement.setObject(i + 2, extractedData.getObject(primaryKeyList.get(i)),
extractedMetaData.getColumnType(extractedData.findColumn(primaryKeyList.get(i))));
}
versionedUpdateStatement.setObject(primaryKeyList.size() + 2,
extractedData.getObject(tableMetaData.getVersionColumn()), tableMetaData.getVersionType());
if (versionedUpdateStatement.executeUpdate() == 0) {
logger.warn(Message.COLLECTOR_DATA_CHANGED, tableMetaData.getTable());
}
}
} else {
logger.warn(Message.COLLECTOR_PK_ERROR, tableMetaData.getTable());
markResultSetAsExported(extractedData, tableMetaData);
}
} finally {
if (resultSetPrimaryKeys != null) {
resultSetPrimaryKeys.close();
}
if (versionedUpdateStatement != null) {
versionedUpdateStatement.close();
}
}
}
//the old way as fallback
private void markResultSetAsExported(ResultSet extractedData, TableMetaData tableMetaData) throws SQLException {
while (extractedData.next()) {
extractedData.updateString(tableMetaData.getUpdateColumn(), tableMetaData.getUpdateValue());
extractedData.updateRow();
}
}
I want to know whether there is a way to acknowledge the deletion of a document (object) of an OrientDB database. I am using the following query
DELETE FROM User WHERE #rid=#1:1
to delete the record and I can't seem to find a way to acknowledge where the object was actually deleted or not.
When I check the response from
statement.executeQuery("DELETE FROM User WHERE #rid=#1:1")
for both the cases (the actual deletion of the object and the object not being present in the db) it returns null.
The DELETE SQL command returns, by default, the number of deleted records. Try this:
Integer deleted = db.command(new OCommandSQL("DELETE FROM User WHERE #rid=#1:1")).execute();
By the way this is faster:
Integer deleted = db.command(new OCommandSQL("DELETE FROM #1:1")).execute();
You can try this code
try{
ORecordId id = new ORecordId(9, 1);
boolean deleted=new OCommandExecutorSQLDelete().result(id);
System.out.println("Record Deleted " + deleted);
}
catch(NullPointerException e){
System.out.println("Record not present in the db");
}
I'm getting a weird SQLException on a function I run against a database using JDBC.
SQLException: Column 'Message' not found.
I have this in my function:
st = con.prepareStatement("SELECT NotificationID,UserIDFrom,UserIDTo,Message,Timestamp,isNotified FROM notification WHERE UserIDTo=? AND isNotified=?");
st.setInt(1, _UserID);
st.setBoolean(2, false);
System.out.println("st is: " + st);
rs = st.executeQuery();
And I got that error, so I added this after the st.executeQuery() :
ResultSetMetaData meta = rs.getMetaData();
for (int index = 1; index <= meta.getColumnCount(); index++) {
System.out.println("Column " + index + " is named " + meta.getColumnName(index));
}
And when I run my code again this is what I get as a result:
Column 1 is named NotificationID
Column 2 is named UserIDFrom
Column 3 is named UserIDTo
Column 4 is named Message
Column 5 is named TimeStamp
Exception in thread "main" java.sql.SQLException: Column 'Message' not found.
Column 6 is named isNotified
And here is a screenshot of my table's design, from MySQL Workbench
And the data in the table
I really can't figure out what's going one here.... Anyone can help out?
EDIT
I've replaced the * in the SELECT statement just to add something to the question that I just noticed.
If I remove the Message column from the select then I get the same error for the TimeStamp column. And if I remove both columns I get no errors then.
EDIT2
OK,this is the part i get the errors, i get both on Message and Timestamp:
while (rs.next()) {
NotificationID = rs.getInt("NotificationID");
System.out.println("NotificationID: " + NotificationID);
SenderID = rs.getInt("UserIDFrom");
System.out.println("SenderID: " + SenderID);
From = findUserName(SenderID);
try {
body = rs.getString("Message");
System.out.println("body: " + body);
} catch (Exception e) {
System.out.println("Message error: " + e);
e.printStackTrace();
}
try {
time = rs.getString("Timestamp");
System.out.println("time: " + time);
} catch (Exception e) {
System.out.println("Timestamp error: " + e);
e.printStackTrace();
}
}
I get the error on the getString() methods for each column
StackTrace for TimeStamp(the same for Message):
java.sql.SQLException: Column 'TimeStamp' not found.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1167)
at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5733)
at NotifyMe_Server.Database.getUnNotified(Database.java:444)
at tests.Tests.main(Tests.java:39)
If you observe your code
try {
time = rs.getString("Timestamp");
System.out.println("time: " + time);
} catch (Exception e) {
System.out.println("Timestamp error: " + e);
e.printStackTrace();
}
}
you have used "Timestamp" in this format but if you changed it to "TimeStamp" as specified in your database, hopefully it will work.
Change datatype of your isNotified column as TINYINT in database and retry to insert
isNotified TINYINT(1)
Bool, Boolean: These types are synonyms for TINYINT(1). A value of zero is considered false. Non-zero values are considered true.
Can you change
System.out.println("Column " + index + " is named " + meta.getColumnName(index));
to
System.out.println("Column " + index + " is named '" + meta.getColumnName(index) + "'");
so that we can see if there is whitespace in the "Message" column name?
The fact that the error message comes between column 5 and 6 is not important I think, because one is Standard Output and the other one Standard Error, these are not synchronized output streams.
(Also see the previous answer about Timestamp vs TimeStamp.)
It sounds like the table metadata is corrupt. You should be able to correct this by dropping and recreating the table, although if the metadata is really borked you may not be able to drop the table. If that's the case or you need to keep the data, backing up and restoring the whole database is the way to go, but check the SQL dump file before restoring and/or restore to another database name before dropping the broken database. Depending on exactly what's wrong, your problem columns may be missing from the dump.
If refreshing the database is not an option there are ways to perform targetted repairs, but I'm no expert so I can't advise you on that. Again, back up your database AND verify that the backup is complete (i.e. it has all your columns) before proceeding. If this is a production database, I would be very wary about taking advice from the internet on manipulating metadata. Minor differences in version, storage engine and environment can make or break you with this stuff, and given the nature of the problem you can't do a dry run.
Scenario: I came across some code that is mixing JPA with JDBC within a transaction. The JDBC is doing an INSERT into a table with basically a blank row, setting the Primary Key to (SELECT MAX(PK) + 1) and the middleName to a temp timestamp. The method is then selecting from that same table for max(PK) + that temp timestamp to check if there was a collision. If successful, it then nulls out the middleName and updates. The method returns the newly created Primary Key.
Question:
Is there a better way to insert an entity into the database, setting the PK to max(pk) + 1 and gaining access to that newly created PK (preferably using JPA)?
Environment:
Using EclipseLink and need to support several versions of both Oracle and MS SqlServer databases.
Bonus Background: The reason I'm asking this question is because I run into a java.sql.BatchUpdateException when calling this method as part of a chain when running integration tests. The upper part of the chain uses JPA EntityManager to persist some objects.
Method in question
#Override
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public int generateStudentIdKey() {
final long now = System.currentTimeMillis();
int id = 0;
try {
try (final Connection connection = dataSource.getConnection()) {
if (connection.getAutoCommit()) {
connection.setAutoCommit(false);
}
try (final Statement statement = connection.createStatement()) {
// insert a row into the generator table
statement.executeUpdate(
"insert into student_demo (student_id, middle_name) " +
"select (max(student_id) + 1) as student_id, '" + now +
"' as middle_name from student_demo");
try (final ResultSet rs = statement.executeQuery(
"select max(student_id) as student_id " +
"from student_demo where middle_name = '" + now + "'")) {
if (rs.next()) {
id = rs.getInt(1);
}
}
if (id == 0) {
connection.rollback();
throw new RuntimeException("Key was not generated");
}
statement.execute("update student_demo set middle_name = null " +
"where student_id = " + id);
} catch (SQLException statementException) {
connection.rollback();
throw statementException;
}
}
} catch (SQLException exception) {
throw new RuntimeException(
"Exception thrown while trying to generate new student_ID", exception);
}
return id;
}
First off: it hurts to answer this. But I know, sometimes you have to deal with the devil :(
So technically, it's not JPA, but if you are using Hibernate as JPA-Provider, you can go with
#org.hibernate.annotations.GenericGenerator(
name = “incrementGenerator”,
strategy = “org.hibernate.id.IncrementGenerator”)
#GeneratedValue(generator="incrementGenerator")
private Long primaryKey;
The Hibernate solution is "thread-safe", but not "cluster-safe", i.e. if you run your application on several hosts, this may fail. You may catch the appropriate exception and try again.
If you stick with your solution: close the ResultSet, Statement and the Connection. Sorry, didn't catch the try-with-resources initially.
The JDBC code is pathological, makes no sense, and will not work in a multi user environment.
I would strongly recommend fixing the code to use a sequence object, or sequence table.
In JPA you can just use sequencing.
See,
http://en.wikibooks.org/wiki/Java_Persistence/Identity_and_Sequencing#Sequencing
If you really want to do your own sequencing, you can either assign the Id yourself, use PrePersist to assign your own id, or in EclipseLink implement your own Sequence subclass that does whatever you desire. You will need to register this Sequence object using a SessionCustomizer.
See,
http://wiki.eclipse.org/EclipseLink/Examples/JPA/CustomSequencing