We have a Kafka consumer in a Java - Spring Boot service that has multiple instances in multiple clusters (all with same consumer group id). The consumer saves data to the database (SQL Server).
The code checks if a record exists in the ItemSet table before saving to the database. The actual data in the payload gets saved in a child table ItemValue and not the parent, ItemSet. The relational data table hierarchy is one to many in this order ItemSet -> ItemName -> ItemValue. ItemSet table has a unique constraint for department id, season combination to prevent multiple duplicate adds.
I need to do some processing after catching this exception to ensure that the incoming data still gets saved under the existing ItemSet and doesn't get lost. I am using Spring Data JPA and as soon as I catch the exception and try to retrieve the existing record I end up getting:
org.hibernate.AssertionFailure: null id in ItemSet entry (don't flush the Session after an exception occurs).
The getItemSet() in the catch block blows up.
What is the best way to overcome these race conditions?
ItemSet savedItemSet = null;
try
{
String seasonName = itemSet.getSeasonName();
Long seasonYear = itemSet.getSeasonYear();
Long departmentId = itemSet.getDepartment().getId();
List<ItemSet> itemSets = attributeSetRepository.findBySeasonNameAndSeasonYearAndDepartmentId(
seasonName, seasonYear, departmentId);
LOGGER.info("Found {} item sets corresponding to season name : {}, season year : {}, "
+ "department id : {}", itemSets.size(), seasonName, seasonYear, departmentId);
if(CollectionUtils.isEmpty(itemSets)) {
savedItemSet = itemSetRepository.save(itemSet);
}
else {
return new CreatedItemSet(itemSets.get(0).getId());
}
}
catch(PersistenceException | DataIntegrityViolationException e)
{
LOGGER.error("An exception occurred while saving itemSet set", e);
if (e.getCause() instanceof ConstraintViolationException)
{
String seasonName = itemSet.getSeasonName();
Long seasonYear = itemSet.getSeasonYear();
Long deptId = itemSet.getDepartment().getId();
LOGGER.info("A duplicate item set found in the database corresponding "
+ "to season name : {}, season year : {} and department : {}",
seasonName, seasonYear, deptId);
ExistingItemSet existingItemSet = getItemSet(seasonName,
seasonYear, deptId);
if(existingItemSet == null) {
LOGGER.info("No item set found");
return null;
}
return new CreatedItemSet(existingItemSet.getId());
}
}
You can't "continue". A transaction is marked for rollback and the persistence context is unusable after a constraint violation happens.
You can either try to avoid the constraint violation, by checking if the DB contains an entry before persisting/updating, or you run the rest of your code in a separate transaction if a constraint violation happens.
Related
I need to insert records in multiple tables, but these tables are belonging to different schemas so transaction values are also different. But I want to insert records in all the tables atomically ( all or nothing). Following code I have tried to handle this.
#Transactional(value = "First")
public void insert(String f1, String f2, String f3, String f4){
try {
Entity1 entity1 = createEntity1(f1);
FirstTable1.insert(entity1);
insertInSecondSchema(f2, f3, f4);
}
catch (Exception e){
String errorMessage = "Error occur while inserting in first schema";
logger.error(errorMessage, e);
throw new CustomException(errorMessage,e);
}
}
#Transactional(value = "second")
private void insertInSecondSchema(String f2, String f3, String f4) {
try {
Entity2 entity2 = createEntity2(f2);
SecondTable2.insert(entity2);
Entity3 entity3 = createEntity3(f3);
SecondTable3.insert(entity3);
Entity4 entity4 = createEntity4(f4);
SecondTable4.insert(entity4);
}
catch (Exception e){
String errorMessage = "Error occur while inserting in second schema";
logger.error(errorMessage, e);
throw new CustomException(errorMessage,e);
}
}
In this scenario if some error occurs while inserting in SecondTable3. SecondTable2 is not rolled back but FirstTable1 is rolled-back. I have tried many propagation level in secondMethod, but none of them working for me. Please help how can I fix this using same code or some other code structure.
In this kind of situation I think is better to think as if this was a distributed system. In this case, you cannot make these insertions transactional, instead of that, you should use compensating actions in case one of the transaction succeeds and the other fails.
In your case if insertInSecondSchema fails then you have to call delete of the same record for the first schema, so the result is like the first insertion never happens
I want to insert multiple rows in a table from a console/tool (e.g.: Data studio) I get the following error message
THE INSERT OR UPDATE VALUE OF FOREIGN KEY FK$MAR$S IS INVALID.
SQLCODE=-530, SQLSTATE=23503, DRIVER=4.13.111
This means I have some trouble with a FOREIGN KEY variable, but I solved that later and it works well.
My problem is that when I'm running the same query from a Java application using PreparedStatement.executeBatch() (batch because it could insert more than one row at a time), I get a different error message:
com.ibm.db2.jcc.am.wn: [jcc][t4][102][10040][3.57.82] Batch failure.
The batch was submitted, but at least one exception occurred on an
individual member of the batch. Use getNextException() to retrieve
the exceptions for specific batched elements. ERRORCODE=-4228,
SQLSTATE=null
When I used getNextException(), I get the following:
com.ibm.db2.jcc.am.co: A NON-ATOMIC INSERT STATEMENT ATTEMPTED TO
PROCESS MULTIPLE ROWS OF DATA, BUT ERRORS OCCURRED
And the error code is -4228.
Why this difference? I want the java application return the same error details as the console tool, so I can handle those exceptions in my java code.
For example, if the returned error code=-803 which means duplicate exception, I would handle my code to make update instead of insert, or if the returned message contains some words like " FOREIGN KEY ", I'll tell user to make sure about lookup tables and so on
I use DB2 version 10.5.3 on z/OS and the DB2 driver version is : 3.65.92
} catch (SQLException ex) {
while (ex != null) {
if (ex instanceof com.ibm.db2.jcc.DB2Diagnosable) {
com.ibm.db2.jcc.DB2Diagnosable db2ex = (com.ibm.db2.jcc.DB2Diagnosable) ex;
com.ibm.db2.jcc.DB2Sqlca sqlca = db2ex.getSqlca();
if (sqlca != null) {
System.out.println("SQLCODE: " + sqlca.getSqlCode());
System.out.println("MESSAGE: " + sqlca.getMessage());
} else {
System.out.println("Error code: " + ex.getErrorCode());
System.out.println("Error msg : " + ex.getMessage());
}
} else {
System.out.println("Error code (non-db2): " + ex.getErrorCode());
System.out.println("Error msg (non-db2): " + ex.getMessage());
}
ex = ex.getNextException();
}
...
}
Above is an example of handling db2 exceptions. The example of output when there are 2 violations simultaneously: unique key on the table MYSCHEMA.MYTABLE where batch inserts come, and a foreign key on a parent table. I split it intentionally into 2 parts:
Before getNextException():
Error code: -4229
Error msg : [jcc][t4][102][10040][4.19.66] ... getNextException().
ERRORCODE=-4229, SQLSTATE=null
After getNextException():
SQLCODE: -803
MESSAGE: One or more values in the INSERT statement,
UPDATE statement, or foreign key update caused by a DELETE statement
are not valid because the primary key, unique constraint or unique
index identified by "1" constrains table "MYSCHEMA.MYTABLE" from
having duplicate values for the index key.. SQLCODE=-803,
SQLSTATE=23505, DRIVER=4.19.66
SQLCODE: -530
MESSAGE: The insert or update value of the FOREIGN KEY
"MYSCHEMA.MYTABLE.MYTABLE_FK" is not equal to any value of the parent
key of the parent table.. SQLCODE=-530, SQLSTATE=23503, DRIVER=4.19.66
I think the batch exception message is pretty clear. Consider that different statements in a batch might fail or issue warnings for different reasons. The batch level error message is therefore generic and instructs you to use "getNextException() to retrieve the exceptions for specific" statements in the batch.
Though this is an old thread. I will share the code which worked for me
try{
preparedStatement.batchUpdate( new ClassName{
//code with setting values and batch size});
}catch (Exception e) {
if (e.getCause() instanceof BatchUpdateException) {
BatchUpdateException be = (BatchUpdateException) e.getCause();
SQLException current = be.getNextException();
do {
current.printStackTrace();
} while ((current = current.getNextException()) != null);
}
}
In here I'm trying to get the exception based on BatchUpdateException instance.
I'm using hibernate to retrieve data from a MySQL database.
#Transactional
public List<PendingDomainEntity> listPendingDomain(int count, int offset)
throws DaoException {
try {
Criteria listCriteria = sessionFactory.getCurrentSession().createCriteria(PendingDomainEntity.class);
listCriteria.addOrder(Order.asc("domaincreateddt"));
listCriteria.setMaxResults(count);
listCriteria.setFirstResult(offset);
Criterion domainstatus = Restrictions.eq("domainstatus", "Pending").ignoreCase();
listCriteria.add(domainstatus);
#SuppressWarnings("unchecked")
List<PendingDomainEntity> pendingDomainList = (List<PendingDomainEntity>) listCriteria.list();
LOGGER.debug("Regg-Service: The pending domain list is:={}", pendingDomainList);
return pendingDomainList;
} catch (Exception ex) {
LOGGER.error("Exception: Service: Error while retrieving the pending domains list");
throw new DaoException(ex.getMessage(), Response.Status.INTERNAL_SERVER_ERROR);
}
}
This is how domainCreatedDt is declared in PendingDomainEntity class
#Column(name = "DOMAINCREATEDDT")
private Timestamp domainCreatedDt;
I get following exception message
could not resolve property: domaincreateddt of: mypackage.DTO.PendingDomainEntity
But when I use a different column name for Order, it works fine. Are there any restrictions for order by on timestamp columns ?
Check this documentation chapter about HQL (the same applies to Criteria API):
16.1. Case Sensitivity
With the exception of names of Java classes and properties, queries are case-insensitive. So SeLeCT is the same as sELEct is the same as SELECT, but org.hibernate.eg.FOO is not org.hibernate.eg.Foo, and foo.barSet is not foo.BARSET....
This is the JAVA name:
private Timestamp domainCreatedDt;
and that means, that this is NOT the same:
listCriteria.addOrder(Order.asc("domaincreateddt"));
Simply it must be domainCreatedDt
Scenario: I came across some code that is mixing JPA with JDBC within a transaction. The JDBC is doing an INSERT into a table with basically a blank row, setting the Primary Key to (SELECT MAX(PK) + 1) and the middleName to a temp timestamp. The method is then selecting from that same table for max(PK) + that temp timestamp to check if there was a collision. If successful, it then nulls out the middleName and updates. The method returns the newly created Primary Key.
Question:
Is there a better way to insert an entity into the database, setting the PK to max(pk) + 1 and gaining access to that newly created PK (preferably using JPA)?
Environment:
Using EclipseLink and need to support several versions of both Oracle and MS SqlServer databases.
Bonus Background: The reason I'm asking this question is because I run into a java.sql.BatchUpdateException when calling this method as part of a chain when running integration tests. The upper part of the chain uses JPA EntityManager to persist some objects.
Method in question
#Override
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public int generateStudentIdKey() {
final long now = System.currentTimeMillis();
int id = 0;
try {
try (final Connection connection = dataSource.getConnection()) {
if (connection.getAutoCommit()) {
connection.setAutoCommit(false);
}
try (final Statement statement = connection.createStatement()) {
// insert a row into the generator table
statement.executeUpdate(
"insert into student_demo (student_id, middle_name) " +
"select (max(student_id) + 1) as student_id, '" + now +
"' as middle_name from student_demo");
try (final ResultSet rs = statement.executeQuery(
"select max(student_id) as student_id " +
"from student_demo where middle_name = '" + now + "'")) {
if (rs.next()) {
id = rs.getInt(1);
}
}
if (id == 0) {
connection.rollback();
throw new RuntimeException("Key was not generated");
}
statement.execute("update student_demo set middle_name = null " +
"where student_id = " + id);
} catch (SQLException statementException) {
connection.rollback();
throw statementException;
}
}
} catch (SQLException exception) {
throw new RuntimeException(
"Exception thrown while trying to generate new student_ID", exception);
}
return id;
}
First off: it hurts to answer this. But I know, sometimes you have to deal with the devil :(
So technically, it's not JPA, but if you are using Hibernate as JPA-Provider, you can go with
#org.hibernate.annotations.GenericGenerator(
name = “incrementGenerator”,
strategy = “org.hibernate.id.IncrementGenerator”)
#GeneratedValue(generator="incrementGenerator")
private Long primaryKey;
The Hibernate solution is "thread-safe", but not "cluster-safe", i.e. if you run your application on several hosts, this may fail. You may catch the appropriate exception and try again.
If you stick with your solution: close the ResultSet, Statement and the Connection. Sorry, didn't catch the try-with-resources initially.
The JDBC code is pathological, makes no sense, and will not work in a multi user environment.
I would strongly recommend fixing the code to use a sequence object, or sequence table.
In JPA you can just use sequencing.
See,
http://en.wikibooks.org/wiki/Java_Persistence/Identity_and_Sequencing#Sequencing
If you really want to do your own sequencing, you can either assign the Id yourself, use PrePersist to assign your own id, or in EclipseLink implement your own Sequence subclass that does whatever you desire. You will need to register this Sequence object using a SessionCustomizer.
See,
http://wiki.eclipse.org/EclipseLink/Examples/JPA/CustomSequencing
I am using Hibernate and MySql.
I have a 2 tables:
User: id, name, type
City: id, name, type
type: id, name
Where user.type has foreign key to user_type.id. and as well city.
I would like before deleting a row in user_type table, to check if any row from any table is related to it.
my columns are mapped for example:
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "type_id")
How can I do it?
You said
I have around 100 tables like User and City mapped to this value
ok. Hibernate with JPA book says
You may have removed all other references manually
Which implies you should query manually any related Table. But it says if other entity references Type, database constraints prevent any inconsistency and you see a foreign key constraint exception. I Think it is the best way you can check out what you want. Otherwise, you should query manually for any related Table.
try {
userType = (Type) session.load(Type.class, id);
session.delete(userType);
/**
* or JDBCException
* e.getCause()
* e.getErrorCode() - vendor-specific
*/
} catch (HibernateException e) {
// checkout Exception right here e.getCause();
}
All exceptions thrown by Hibernate are fatal. This means you have to roll back the database transaction and close the current Session. So you may want To open a new session.
use native SQL with Hibernate together:
boolean canDeleteType(ind type_id){
Session s = HibernateUtil.getSessionFactory();
s.beginTransaction();
Query q = s.createQuery("SELECT User.type_id From User");
List l = q.list();
if(l.contains(type_id){
return false;
}
return false;
}
and do the same for your City table too.