I'm seeing a weird behavior with Spring Boot 2.0.4 + Hibernate.
I have an entity including a randomly generated code. If the generated code is already set for another entity, a DataIntegrityViolationException is thrown as expected. This way the loop can try again with a new code which hopefully is not used. When this happens, the loop continues, a new code is generated and the call to saveAndFlush() throws the same exception again saying that the original code that caused the problem (previous iteration) is already used (duplicate). However, I'm setting a new code now, not the one the exception mentions.
The only thing I can think of is that Hibernate doesn't remove the operation from the "queue" so when the second call to saveAndFlush() happens, it still tries to perform the first save and then the new one. Obviously, the first save fails as during the first iteration. Maybe I'm wrong, but then what is going on here?
#Entity
public class Entity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(nullable = false, unique = true)
private int code;
public void setCode(int code) {
this.code = code;
}
//Other properties
}
#Transactional
public void myFunction() {
boolean saved = false;
do {
int code = /* Randomly generated code */;
if(entity == null) {
entity = new Entity(code, /* other properties */);
} else {
entity.setCode(code);
}
try {
entity = myRepository.saveAndFlush(entity);
saved = true;
} catch (DataIntegrityViolationException e) {
/* Ignore so that we can try again */
}
} while(!saved);
}
EDIT:
If I replace saveAndFlush() by save(), the issue disappears. I saw somewhere that doing a save after a previous save that failed may be problematic if flush() is also called. This is exactly my case. However, I don't understand why it is a problem. The only reason I call saveAndFlush() instead of save() is to catch the duplicate key exception. Using save(), if Hibernate doesn't perform the INSERT or UPDATE directly, the exception is thrown during the flush occurring just before the transaction is committed which is not really what I want.
You will need to restart your transaction in such a scenario, as the error is already bound to the transaction context/session.
So instead, put the retry logic outside of the transaction boundary, or if you need to maintain integrity (all or nothing saved), check first if it is present to avoid the exception being thrown.
If you debug the code and see what's the state of persistence context you may get your answer. You are right hibernate maintains a queue of sorts i.e. all the queries that are made during a transaction will run on commit/flush.
Please post the value of persistence context you get while debugging.
Related
My application receives arrays containing objects, which are mapped to entities and then persisted.
The object has an id property that is mapped to a column with a unique constraint placed on it. Objects with duplicate ids may be sent to the application.
I'm using the saveAll method, but it throws during insertion if any of the objects happens to violate the unique constraint.
Is there an easy way to make it insert all the non-duplicates and ignore the duplicates, or simply update them?
I've tried overriding hash and equals, but that didn't help.
This application receives dozens of requests per second and he amount of duplicates I expect to receive is low.
entity:
#Table(name = "table", uniqueConstraints = #UniqueConstraint(columnNames = "natural_id"))
public class Entity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "natural_id", unique = true)
private String naturalId;
// ...
}
repository:
public interface Repository extends JpaRepository<Entity, Long>, JpaSpecificationExecutor<Entity> {}
saving method:
void save(List<Entities> entities) {
try {
repository.saveAll(entities);
} catch (Exception e) {
log.error("...");
throw e;
}
}
If I were to change the saving method to:
#Transactional
void save(List<Entities> entities) {
for (Entities et: entities) {
try {
repository.save(et);
} catch (Exception e) {
log.error("...");
}
}
}
Would that make it all happen within a single transaction? I'm worried because the save method usually creates a new transaction every time it is called, and that would make it extremely slow for an application that receives a decent number of requests per second.
I might ultimately have to query the database before insertion to filter the duplicates out for insertion, and do that again every time a constraint violation exception is caught. It should work, but is quite ugly.
You have to save individual records because save all will not be resume/recover if faced problem with some record and then other record will also not saved as all are in same transaction.
As you suggested
query the database before insertion to filter the duplicates out instead of that
better try for loop with try catch as you mentioned ''dozens of requests per second'' so it wont give much overhead.
If you already implemented hashCode and equals then use Set instead of List to collect your items before saving them.
I'm getting a really strange NullPointerException when saving my Entity and I can't seem to figure out why.
java.lang.NullPointerException: null
at org.hibernate.type.AbstractType.getHashCode(AbstractType.java:129)
at java.base/java.util.stream.Collectors.lambda$groupingBy$53(Collectors.java:1127)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at org.hibernate.collection.internal.PersistentBag.groupByEqualityHash(PersistentBag.java:196)
at org.hibernate.collection.internal.PersistentBag.equalsSnapshot(PersistentBag.java:151)
at org.hibernate.engine.spi.CollectionEntry.dirty(CollectionEntry.java:158)
at org.hibernate.engine.spi.CollectionEntry.preFlush(CollectionEntry.java:182)
at org.hibernate.event.internal.AbstractFlushingEventListener.lambda$prepareCollectionFlushes$0(AbstractFlushingEventListener.java:195)
at org.hibernate.engine.internal.StatefulPersistenceContext.forEachCollectionEntry(StatefulPersistenceContext.java:1091)
at org.hibernate.event.internal.AbstractFlushingEventListener.prepareCollectionFlushes(AbstractFlushingEventListener.java:194)
at org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:86)
at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:50)
at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:108)
at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1323)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1403)
at org.hibernate.query.internal.AbstractProducedQuery.doList(AbstractProducedQuery.java:1558)
at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1526)
at org.hibernate.query.Query.getResultList(Query.java:165)
[...] the rest was cut for readability (if you think it's really important, I'll edit and add the whole exception, but I think it's the first line that says it all)
This is the higher exception that's thrown because of the one above:
org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Error while committing the transaction
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:542)
and this nested one:
Caused by: javax.persistence.RollbackException: Error while committing the transaction
at org.hibernate.internal.ExceptionConverterImpl.convertCommitException
I've been running around in circles all day trying to figure out why this was happening and the internet didn't help because most of similar questions remain unanswered. What makes matters worse is Spring's abstraction makes it impossible to debug!
Here's the scenario:
I have an EventEntity which I'm trying to update, along with its child entity EventAttachmentEntity. I've turned on logging to follow what's happening and pretty much the expected happens:
I've updated eventEntity (update SQL generated)
I've inserted new attachments for the updated event (insert SQL generated with right event_id)
And then the strange thing happens, my method reaches the end and then this exception occurs.
The method is abstracted for disclosure reasons, but the main flow is as follows:
#Transactional(rollbackFor = Exception.class)
public void updateEvent(EventEntity event) {
EventEntity updatedEvent = eventRepository.save(event);
EventAttachmentEntity newAttachment = new EventAttachmentEntity(updatedEvent);
// fill attachment [...]
newAttachment.setEvent(updatedEvent);
eventAttachmentRepository.save(newAttachment);
System.out.println("This line gets printed.");
}
So first I save the updated parent and then I create new child, set the parent, and then save it. All the SQLs confirm this is what happens. The last line prints as well, meaning nothing broke in the previous saving methods. Then it reaches the end and BOOM! exception.
Here's my entities:
#Entity
#Table(name = "event", schema = "xxx")
#DynamicUpdate
public class EventEntity {
private long id;
private Collection<EventAttachmentEntity> eventAttachments;
// other properties abstracted
#OneToMany(mappedBy = "event")
public Collection<EventAttachmentEntity> getEventAttachments() {
return this.eventAttachments;
}
public void setEventAttachments(Collection<EventAttachmentEntity> eventAttachments) {
this.eventAttachments = eventAttachments;
}
}
#Entity
#Table(name = "event_attachment", schema = "xxx")
public class EventAttachmentEntity {
private long id;
private String filename;
private String contentType;
private long filesize;
private EventEntity event;
// other properties abstracted
#ManyToOne
#JoinColumn(name = "event_id", referencedColumnName = "id", nullable = false)
public EventEntity getEvent() {
return event;
}
public void setEvent(EventEntity event) {
this.event = event;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
EventAttachmentEntity that = (EventAttachmentEntity) o;
return id == that.id &&
filesize == that.filesize &&
Objects.equals(filename, that.filename) &&
Objects.equals(contentType, that.contentType);
}
#Override
public int hashCode() {
return Objects.hash(id, filename, contentType, filesize);
}
}
I've included equals() and hashCode() methods for EventAttachmentEntity because the exception obviously happens at getHashCode(), but when I place the breakpoint there, it's actually the Object whose getHashCode is called that's null.
Important note: I've checked whether any variable in my code was null or hasn't been properly autowired - everything's fine. Like I said, the method itself runs smoothly until it reaches the end and tries to commit the transaction. That's when it breaks.
Also, when I try to update only the event (without attachments), everything works okay. Another case I've tried is to save a similar relationship of this event (not shown here) and it also works fine. So it's something to do with attachments and I can't figure out what it is!
What's also weird is if I remove #Transactional from this method, it throws the same exception. There is no #Transactional marked method that's calling updateEvent(), so I'm not sure how this is even happening.
Please, I need any kind of suggestion as to what might be the issue so I can try it, because right now I pretty much hit the wall and can't continue.
You haven't shown the whole code and we can only guess what you have there. According tp the shown code the member variable EventEntity.eventAttachments is not initialized. When you try to persist such entity, this can lead to NPE.
Solution
Initialize it, e.g. as follows:
private Collection<EventAttachmentEntity> eventAttachments = new HashSet<>();
Also in setEventAttachments make sure that eventAttachment remains not null no matter what argument is passed to this setter.
In updateEvent(EventEntity updatedEvent) method updatedEvent is persisted to database eventRepository.save(updatedEvent), but the documentation of S save(S entity) method says:
Saves a given entity. Use the returned instance for further operations as the save operation might have changed the entity instance completely.
From the code you posted so far it seems that you do not use the returned by save method entity instance as recommended:
eventRepository.save(updatedEvent);
EventAttachmentEntity newAttachment = new EventAttachmentEntity(updatedEvent);
// fill attachment [...]
newAttachment.setEvent(updatedEvent);
I'm not sure if that could be the reason for the error, but the proper way to handle the update would be something along these lines:
updatedEvent = eventRepository.save(updatedEvent); // update updatedEvent
EventAttachmentEntity newAttachment = new EventAttachmentEntity(updatedEvent);
// fill attachment [...]
newAttachment.setEvent(updatedEvent);
I have got a Springboot Application and a Oracle DB with lots of PL/SQL Procedures and these change the state of the DB all the Time.
So now I want to change a loaded entity an want to save it. If the entitystate of the entitymanager and the state of the db is equal everything works fine. But in some cases they are not equal. So if I load an entity and make some changes an druring this a PL/SQL Procedure changes the DB Table. If I save the Entity I will get an Execption of course. So I tried to catch the Exception and then in the catch block I want to refresh the Entity before saving it. But I still get an Exception. Is the Transaction not jet finished? How can I handle this Problem?
I hope the example code explains a little bit.
#RestController
#RequestMapping("/*")
public class FacadeController {
...
#ResponseStatus(HttpStatus.OK)
#RequestMapping( value= "/test4" , method=RequestMethod.GET)
public String test4(){
Unit unit = unitSvice.loadUnit(346497519L);
List<UnitEntry> entries = unit.getEntries();
for (UnitEntry g : entries) {
if (g.getUnitEntryId == 993610345L) {
g.setTag("AA");
g.setVersion(g.getVersion() + 1);
g.setstatus("SaveOrUpdate");
}
}
//<-- DB Table changed entity managed by entitymanger and DB Table
// are no langer equal.
try {
unitSvice.updateUnit(unit , false);
}catch(DataAccessException | IllegalArgumentException e) {
unitSvice.updateUnit(unit , true);
}
...
}
}
#Service("unitSvice")
public class UnitSvice {
#Autowired
private UnitDao repoUnit;
#Transactional
public Unit loadUnit(Long _id) {
Unit unit = repoUnit.findOne(_id);
return unit;
}
#Transactional
public void updateUnit(Unit unit, boolean _withrefrsh ) {
if(_withrefrsh) {
getEntityManager().refresh(unit.getId());
}
repoUnit.save(unit);
}
}
I hope, anyone can help me.
Thanks
yes the problem is ..when you call load all method which is transactional method where entities became detached from session/entitymanager when you are returning from that method.. so,next you are trying to persist detached object. That's why you get exception.
so probably you can use session.update() or session.merge() to save the new update into database.
I wonder if anyone has come across this error and can explain what's happening:
<openjpa-2.1.1-SNAPSHOT-r422266:1087028 nonfatal user error>
org.apache.openjpa.persistence.InvalidStateException:
Primary key field com.qbe.config.bean.QBEPropertyHistory.id of com.qbe.config.bean.QBEPropertyHistory#1c710ab has non-default value.
The instance life cycle is in PNewProvisionalState state and hence an
existing non-default value for the identity field is not permitted.
You either need to remove the #GeneratedValue annotation or modify the
code to remove the initializer processing.
I have two objects, Property and PropertyHistory. Property has OneToMany List of PropertyHistory:
#OneToMany(fetch = FetchType.LAZY, cascade=CascadeType.MERGE, orphanRemoval=false)
#JoinColumn(name="PROPERTY_NAME")
#OrderBy("updatedTime DESC")
private List<QBEPropertyHistory> history = new ArrayList<QBEPropertyHistory>();
And Property object is loaded and saved like this:
public T find(Object id) {
T t = null;
synchronized(this) {
EntityManager em = getEm();
t = em.find(type, id);
//em.close(); //If this is uncommented, fetch=LAZY doesn't work. And fetch=EAGER is too slow.
}
return t;
}
public T update(T t) {
synchronized(this) {
EntityManager em = getEm();
em.getTransaction().begin();
t = em.merge(t);
em.getTransaction().commit();
em.close();
return t;
}
}
In the service layer I load a property using find(id) method, instantiate a new PropertyHistory, add it into property prop.getHistory().add(propHist) then call update(prop) and get the above error.
The error disappears if I close EntityManager in find() but that breaks lazy loading and prop.getHistory() always returns null. If I set fetch=EAGER it becomes unacceptably slow as there are 10s of 1000s of records and I need to select thousands of property objects at a time and history is not needed 99.99% of the time.
I can't remove the #GeneratedValue as the error text suggests because it is generated (DB2, autoincrement). Now I wonder how would i "modify the code to remove the initializer processing" ?
Thanks!
The problem is that you are trying to share an Entity across persistence contexts(EntityManager). You could change your methods to take an EntityManager instance and use the same EM for the find and update operations.
I am not really sure where my problem lies, as I am experimenting in two areas that I don't have much experience with: JPA and Futures (using Play! Framework's Jobs and Promises).
I have the following bit of code, which I want to return a Meeting object, when one of the fields of this object has been given a value, by another thread from another HTTP request. Here is what I have:
Promise<Meeting> meetingPromise = new Job<Meeting> () {
#Override
public Meeting doJobWithResult() throws Exception {
Meeting meeting = Meeting.findById(id);
while (meeting.bbbMeetingId == null) {
Thread.sleep(1000);
meeting = meeting.refresh(); // I tried each of these
meeting = meeting.merge(); // lines but to no avail; I
meeting = Meeting.findById(id); // get the same result
}
return meeting;
}
}.now();
Meeting meeting = await(meetingPromise);
As I note in the comments, there are three lines in there, any one of which I think should allow me to refresh the contents of my object from the database. From the debugger, it seems that the many-to-one relationships are refreshed by these calls, but the single values are not.
My Meeting object extends Play! Framework's Model, and for convenience, here is the refresh method:
/**
* Refresh the entity state.
*/
public <T extends JPABase> T refresh() {
em().refresh(this);
return (T) this;
}
and the merge method:
/**
* Merge this object to obtain a managed entity (usefull when the object comes from the Cache).
*/
public <T extends JPABase> T merge() {
return (T) em().merge(this);
}
So, how can I refresh my model from the database?
So, I ended up cross-posting this question on the play-framework group, and I got an answer there. So, for the discussion, check out that thread.
In the interest of having the answer come up in a web search to anyone who has this problem in the future, here is what the code snippet that I pasted earlier looks like:
Promise<Meeting> meetingPromise = new Job<Meeting> () {
#Override
public Meeting doJobWithResult() throws Exception {
Meeting meeting = Meeting.findById(id);
while (meeting.bbbMeetingId == null) {
Thread.sleep(1000);
if (JPA.isInsideTransaction()) {
JPAPlugin.closeTx(false);
}
JPAPlugin.startTx(true);
meeting = Meeting.findById(id);
JPAPlugin.closeTx(false);
}
return meeting;
}
}.now();
Meeting meeting = await(meetingPromise);
I am not using the #NoTransaction annotation, because that messes up some other code that checks if the request is coming from a valid user.
I'm not sure about it but JPA transactions are managed automatically by Play in the request/controller context (the JPAPlugin opens a transaction before invocation and closes it after invocation).
But I'm not sure at all what happens within jobs and I don't think transactions are auto-managed (or it's a feature I don't know). So, is your entity attached to an entitymanager or still transient? Is there a transaction somewhere? I don't really know but it may explain some weird behavior if not...