My application receives arrays containing objects, which are mapped to entities and then persisted.
The object has an id property that is mapped to a column with a unique constraint placed on it. Objects with duplicate ids may be sent to the application.
I'm using the saveAll method, but it throws during insertion if any of the objects happens to violate the unique constraint.
Is there an easy way to make it insert all the non-duplicates and ignore the duplicates, or simply update them?
I've tried overriding hash and equals, but that didn't help.
This application receives dozens of requests per second and he amount of duplicates I expect to receive is low.
entity:
#Table(name = "table", uniqueConstraints = #UniqueConstraint(columnNames = "natural_id"))
public class Entity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "natural_id", unique = true)
private String naturalId;
// ...
}
repository:
public interface Repository extends JpaRepository<Entity, Long>, JpaSpecificationExecutor<Entity> {}
saving method:
void save(List<Entities> entities) {
try {
repository.saveAll(entities);
} catch (Exception e) {
log.error("...");
throw e;
}
}
If I were to change the saving method to:
#Transactional
void save(List<Entities> entities) {
for (Entities et: entities) {
try {
repository.save(et);
} catch (Exception e) {
log.error("...");
}
}
}
Would that make it all happen within a single transaction? I'm worried because the save method usually creates a new transaction every time it is called, and that would make it extremely slow for an application that receives a decent number of requests per second.
I might ultimately have to query the database before insertion to filter the duplicates out for insertion, and do that again every time a constraint violation exception is caught. It should work, but is quite ugly.
You have to save individual records because save all will not be resume/recover if faced problem with some record and then other record will also not saved as all are in same transaction.
As you suggested
query the database before insertion to filter the duplicates out instead of that
better try for loop with try catch as you mentioned ''dozens of requests per second'' so it wont give much overhead.
If you already implemented hashCode and equals then use Set instead of List to collect your items before saving them.
Related
I'm seeing a weird behavior with Spring Boot 2.0.4 + Hibernate.
I have an entity including a randomly generated code. If the generated code is already set for another entity, a DataIntegrityViolationException is thrown as expected. This way the loop can try again with a new code which hopefully is not used. When this happens, the loop continues, a new code is generated and the call to saveAndFlush() throws the same exception again saying that the original code that caused the problem (previous iteration) is already used (duplicate). However, I'm setting a new code now, not the one the exception mentions.
The only thing I can think of is that Hibernate doesn't remove the operation from the "queue" so when the second call to saveAndFlush() happens, it still tries to perform the first save and then the new one. Obviously, the first save fails as during the first iteration. Maybe I'm wrong, but then what is going on here?
#Entity
public class Entity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(nullable = false, unique = true)
private int code;
public void setCode(int code) {
this.code = code;
}
//Other properties
}
#Transactional
public void myFunction() {
boolean saved = false;
do {
int code = /* Randomly generated code */;
if(entity == null) {
entity = new Entity(code, /* other properties */);
} else {
entity.setCode(code);
}
try {
entity = myRepository.saveAndFlush(entity);
saved = true;
} catch (DataIntegrityViolationException e) {
/* Ignore so that we can try again */
}
} while(!saved);
}
EDIT:
If I replace saveAndFlush() by save(), the issue disappears. I saw somewhere that doing a save after a previous save that failed may be problematic if flush() is also called. This is exactly my case. However, I don't understand why it is a problem. The only reason I call saveAndFlush() instead of save() is to catch the duplicate key exception. Using save(), if Hibernate doesn't perform the INSERT or UPDATE directly, the exception is thrown during the flush occurring just before the transaction is committed which is not really what I want.
You will need to restart your transaction in such a scenario, as the error is already bound to the transaction context/session.
So instead, put the retry logic outside of the transaction boundary, or if you need to maintain integrity (all or nothing saved), check first if it is present to avoid the exception being thrown.
If you debug the code and see what's the state of persistence context you may get your answer. You are right hibernate maintains a queue of sorts i.e. all the queries that are made during a transaction will run on commit/flush.
Please post the value of persistence context you get while debugging.
I am new to spring data jpa. I have a scenario where I have to create an entity if not exists or update based on non primary key name.Below is the code i wrote to create new entity,it is working fine,but if an already exists record ,its creating duplicate.How to write a method to update if exists ,i usually get list of records from client.
#Override
#Transactional
public String createNewEntity(List<Transaction> transaction) {
List<Transaction> transaction= transactionRespository.saveAll(transaction);
}
Add in your Transaction Entity on variable called name this for naming as unique:
#Entity
public class Transaction {
...
#Column(name="name", unique=true)
private String name;
...
}
Then you won't be able to add duplicate values for name column.
First, this is from google composite key means
A composite key is a combination of two or more columns in a table that can be used to uniquely identify each row in the table when the columns are combined uniqueness is guaranteed, but when it taken individually it does not guarantee uniqueness.
A composite key with an unique key is a waste.
if you want to update an entity by jpa, you need to have an key to classify if the entity exist already.
#Transactional
public <S extends T> S save(S entity) {
if(this.entityInformation.isNew(entity)) {
this.em.persist(entity);
return entity;
} else {
return this.em.merge(entity);
}
}
There are two ways to handle your problem.
If you can not get id from client on updating, it means that id has lost its original function. Then remove your the annotation #Id on your id field,set name with #Id. And do not set auto generate for it.
I think what you want is an #Column(unique = true,nullable = false) on your name field.
And that is the order to update something.
Transaction t = transactionRepository.findByName(name);
t.set.... //your update
transactionRepository.save(t);
I've got two classes.
The first class has one field being a primary key column.
The second class references this type as a OneToOne relationship, and uses this foreign key column as its own Primary key.
The tables and classes look like this:
IdTypeDescription
itype_code PK FK (IdType.code)
category
name
IdType
code PK
The classes
#Entity
public class IdTypeDescription implements Serializable
{
public IdTypeDescription()
{
/**
* necessary default constructor for JPA
*/
}
public IdTypeDescription(IdCategory idCat, String idTypeCode, String idTypeName)
{
this.category = idCat;
this.itype = new IdType(idTypeCode);
this.name = idTypeName;
}
#Id
#OneToOne(cascade=CascadeType.ALL)
public IdType itype;
public IdCategory category;
public String name;
}
Id table class
#Entity
public class IdType implements Serializable
{
public IdType()
{
}
public IdType(String code)
{
this.code = code;
}
#Id
public String code;
}
I need to have the structure like this because I also want another class/table referring to the IdType table as well. However, I haven't got that far yet.
My problem is now that when attempting to merge a new object
(something which works when itype is a simple String and primary key)
instead of going through I get complaints from Hibernate about my primary key being null.
idTypeCode is: Visit_number
idTypeName is: visit id
idCategory is: PATIENT
[error] - org.hibernate.engine.jdbc.spi.SqlExceptionHelper - NULL not allowed for column "ITYPE_CODE"; SQL statement:
insert into IdTypeDescription (category, name, itype_code) values (?, ?, ?) [23502-187]
javax.persistence.RollbackException: Error while committing the transaction
at org.hibernate.jpa.internal.TransactionImpl.commit(TransactionImpl.java:86)
I have tried persisting the objects in two ways:
IdType itype = (IdType) em.merge(idDescr.itype);
idDescr.itype = itype;
em.merge(idDescr);
and of course also tried simply:
em.merge(idDescr);
In fact, I get a similar problem when instead of a merge I try a persist.
I guessed the reason (but haven't found documentation for this) being that I needed to persist the child object first separately in its own transaction. Like so:
JPA.withTransaction(() -> JPA.em().persist(idType.itype));
JPA.withTransaction(() -> JPA.em().persist(idType));
That worked! However, I want to be able to do it in a single transaction!
Additionally, I need to do a merge, as I don't know if the object exists or not already in the database.
My attempt to try a similar strategy with the merge failed:
try
{
JPA.withTransaction(() -> idType.itype = JPA.em().merge(idType.itype));
JPA.withTransaction(() -> JPA.em().merge(idType));
}
catch (Throwable e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
even in doing a persist on the IdType and then a merge on the IdTypeDescriptor fails (at the merge. The persist works)
try
{
JPA.withTransaction(() -> JPA.em().persist(idType.itype));
JPA.withTransaction(() -> JPA.em().merge(idType));
}
catch (Throwable e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
So, what do I have to do to get this working ?
Is it possible?
If not, how do other people deal with objects that use purely natural primary keys only, and have no handy artificial key's existence to check for (to decide if to persist or merge).
Oh, and, before someone suggests it, I can't have a reference to the IdTypeDescriptor in the IdType. Or at least, for reasons of the actual Object design, I was trying to avoid that.
Anyone?
I have a bidirectional one-to-many relationship.
0 or 1 client <-> List of 0 or more product orders.
That relationship should be set or unset on both entities:
On the client side, I want to set the List of product orders assigned to the client; the client should then be set / unset to the orders chosen automatically.
On the product order side, I want to set the client to which the oder is assigned; that product order should then be removed from its previously assiged client's list and added to the new assigned client's list.
I want to use pure JPA 2.0 annotations and one "merge" call to the entity manager only (with cascade options). I've tried with the following code pieces, but it doesn't work (I use EclipseLink 2.2.0 as persistence provider)
#Entity
public class Client implements Serializable {
#OneToMany(mappedBy = "client", cascade= CascadeType.ALL)
private List<ProductOrder> orders = new ArrayList<>();
public void setOrders(List<ProductOrder> orders) {
for (ProductOrder order : this.orders) {
order.unsetClient();
// don't use order.setClient(null);
// (ConcurrentModificationEx on array)
// TODO doesn't work!
}
for (ProductOrder order : orders) {
order.setClient(this);
}
this.orders = orders;
}
// other fields / getters / setters
}
#Entity
public class ProductOrder implements Serializable {
#ManyToOne(cascade= CascadeType.ALL)
private Client client;
public void setClient(Client client) {
// remove from previous client
if (this.client != null) {
this.client.getOrders().remove(this);
}
this.client = client;
// add to new client
if (client != null && !client.getOrders().contains(this)) {
client.getOrders().add(this);
}
}
public void unsetClient() {
client = null;
}
// other fields / getters / setters
}
Facade code for persisting client:
// call setters on entity by JSF frontend...
getEntityManager().merge(client)
Facade code for persisting product order:
// call setters on entity by JSF frontend...
getEntityManager().merge(productOrder)
When changing the client assignment on the order side, it works well: On the client side, the order gets removed from the previous client's list and is added to the new client's list (if re-assigned).
BUT when changing on the client side, I can only add orders (on the order side, assignment to the new client is performed), but it just ignores when I remove orders from the client's list (after saving and refreshing, they are still in the list on the client side, and on the order side, they are also still assigned to the previous client.
Just to clarify, I DO NOT want to use a "delete orphan" option: When removing an order from the list, it should not be deleted from the database, but its client assignment should be updated (that is, to null), as defined in the Client#setOrders method. How can this be archieved?
EDIT: Thanks to the help I received here, I was able to fix this problem. See my solution below:
The client ("One" / "owned" side) stores the orders that have been modified in a temporary field.
#Entity
public class Client implements Serializable, EntityContainer {
#OneToMany(mappedBy = "client", cascade= CascadeType.ALL)
private List<ProductOrder> orders = new ArrayList<>();
#Transient
private List<ProductOrder> modifiedOrders = new ArrayList<>();
public void setOrders(List<ProductOrder> orders) {
if (orders == null) {
orders = new ArrayList<>();
}
modifiedOrders = new ArrayList<>();
for (ProductOrder order : this.orders) {
order.unsetClient();
modifiedOrders.add(order);
// don't use order.setClient(null);
// (ConcurrentModificationEx on array)
}
for (ProductOrder order : orders) {
order.setClient(this);
modifiedOrders.add(order);
}
this.orders = orders;
}
#Override // defined by my EntityContainer interface
public List getContainedEntities() {
return modifiedOrders;
}
On the facade, when persisting, it checks if there are any entities that must be persisted, too. Note that I used an interface to encapsulate this logic as my facade is actually generic.
// call setters on entity by JSF frontend...
getEntityManager().merge(entity);
if (entity instanceof EntityContainer) {
EntityContainer entityContainer = (EntityContainer) entity;
for (Object childEntity : entityContainer.getContainedEntities()) {
getEntityManager().merge(childEntity);
}
}
JPA does not do this and as far as I know there is no JPA implementation that does this either. JPA requires you to manage both sides of the relationship. When only one side of the relationship is updated this is sometimes referred to as "object corruption"
JPA does define an "owning" side in a two-way relationship (for a OneToMany this is the side that does NOT have the mappedBy annotation) which it uses to resolve a conflict when persisting to the database (there is only one representation of this relationship in the database compared to the two in memory so a resolution must be made). This is why changes to the ProductOrder class are realized but not changes to the Client class.
Even with the "owning" relationship you should always update both sides. This often leads people to relying on only updating one side and they get in trouble when they turn on the second-level cache. In JPA the conflicts mentioned above are only resolved when an object is persisted and reloaded from the database. Once the 2nd level cache is turned on that may be several transactions down the road and in the meantime you'll be dealing with a corrupted object.
You have to also merge the Orders that you removed, just merging the Client is not enough.
The issue is that although you are changing the Orders that were removed, you are never sending these orders to the server, and never calling merge on them, so there is no way for you changes to be reflected.
You need to call merge on each Order that you remove. Or process your changes locally, so you don't need to serialize or merge any objects.
EclipseLink does have a bidirectional relationship maintenance feature which may work for you in this case, but it is not part of JPA.
Another possible solution is to add the new property on your ProductOrder, I named it detached in the following examples.
When you want to detach the order from the client you can use a callback on the order itself:
#Entity public class ProductOrder implements Serializable {
/*...*/
//in your case this could probably be #Transient
private boolean detached;
#PreUpdate
public void detachFromClient() {
if(this.detached){
client.getOrders().remove(this);
client=null;
}
}
}
Instead of deleting the orders you want to delete you set detached to true. When you will merge & flush the client, the entity manager will detect the modified order and execute the #PreUpdate callback effectively detaching the order from the client.
i've hit a block once again with hibernate.I've posted numerous times on different aspects of the user and contact management that i've been building.
The sad thing is that i didn't really have the time to play with it and understand it better before actually starting working with it. Sorry but English is not my native language, i rather speak french. And again i've started coding in java in an autodidact way.i'm doing all of this by reading books and haven't gone to school for it. with time constraints it's hard to read a book from beginning to the end.
I'm not sure i should put every of my codes dealing with an issue here and from what i've learned from other forum is to post just the necessary and being concise.
So in my User model i have UserAccount class, Profile that holds details like name, preferences etc , AccountSession and Phone.
my contact management model have Contact and Group.
UserAccount has one-to-one association with Profile, one-to-many with AccountSession,contact and group, all bidirectional.the one-to-many association with phone is unidirectional because contact also has and unidirectional with Phone.
Contact has a bidirectional many-o-many with group and one-to-many with phone that i said earlier.
Group also has a many-to-many bedirectional with contact.
here are the mappings
// UserAccount
......
#OneToOne(targetEntity=UserProfileImpl.class,cascade={CascadeType.ALL})
#org.hibernate.annotations.Cascade(value=org.hibernate.annotations.CascadeType.DELETE_ORPHAN)
#JoinColumn(name="USER_PROFILE_ID")
private UserProfile profile;
#OneToMany(targetEntity=ContactImpl.class, cascade={CascadeType.ALL}, mappedBy="userAccount")
#org.hibernate.annotations.Cascade(value=org.hibernate.annotations.CascadeType.DELETE_ORPHAN)
private Set<Contact> contacts = new HashSet<Contact>();
#OneToMany(targetEntity=GroupImpl.class, cascade={CascadeType.ALL}, mappedBy="userAccount")
#org.hibernate.annotations.Cascade(value=org.hibernate.annotations.CascadeType.DELETE_ORPHAN)
private Set<Group> groups = new HashSet<Group>();
.......
//Group
#ManyToOne(targetEntity=UserAccountImpl.class)
#JoinColumn(name="USER_ACCOUNT_ID",nullable=false)
private UserAccount userAccount;
#ManyToMany(targetEntity=ContactImpl.class,cascade={CascadeType.PERSIST, CascadeType.MERGE})
#JoinTable(name="GROUP_CONTACT_MAP", joinColumns={#JoinColumn(name="GROUP_ID")},
inverseJoinColumns={#JoinColumn(name="CONTACT_ID")})
private Set<Contact> contacts = new HashSet<Contact>();
//Contact
....
#ManyToOne(targetEntity=UserAccountImpl.class)
#JoinColumn(name="USER_ACCOUNT_ID",nullable=false)
private UserAccount userAccount;
#ManyToMany(targetEntity=GroupImpl.class, mappedBy="contacts")
private Set<Group> groups=new HashSet<Group>();
....
// helper methods from group
public void addContact(Contact contact) {
try{
this.getContacts().add(contact);
contact.getGroups().add(this);
}catch(Exception e) {
}
}
//helper method from group
public void removeContact(Contact contact) {
contact.getGroups().remove(contact);
this.getContacts().remove(contact);
}
//helper method from contact
public void addGroup(Group group) {
try{
this.getGroups().add(group);
group.getContacts().add(this);
} catch(Exception e) {
e.printStackTrace();
}
}
//Helper method from group
public void removeGroup(Group group){
try{
group.getContacts().remove(this);
this.getGroups().remove(group);
} catch(Exception e) {
e.printStackTrace();
}
}
//UserAccount setter from Contact.All the children with many-to-one have the same
/**
* #param userAccount the userAccount to set
*/
public void setUserAccount(UserAccount userAccount) {
this.userAccount = userAccount;
}
I'ld like to pull the UserAccount by its email field which is an unique field in the UserAccount table.
In the UserAccountDAO the method i call to get the UserAccount is getUserAccountByEmail here below.So i expect this method to load all the children collections of the UserAccount namely its Contact collection, group collection.I want it in such a way that when UserAccount is loaded with Contacts collection each of the contact object has its reference with its belonging groups collection if any etc and vice versa.
public UserAccount getUserAccountByEmail(String email) {
// try {
logger.info("inside getUserAccountByEmail");
logger.debug(email);
Session session = (Session) this.getDBSession().getSession();
UserAccount user = (UserAccount) session.createCriteria(this.getPersistentClass())
.setFetchMode("contacts", FetchMode.SELECT) //recently added
.setFetchMode("groups", FetchMode.SELECT) // recently added
.add(Restrictions.eq("email", email))
.uniqueResult();
logger.debug(user);
return user;
// } catch(NonUniqueResultException ne) {
// logger.debug("Exception Occured: getUserAccountByEmail returns more than one result ", ne);
// return null;
// } catch(HibernateException he){
// logger.debug("Exception Occured: Persistence or JDBC exception in method getUserAccountByEmail ",he);
// return null;
// }catch(Exception e) {
// logger.debug("Exception Occured: Exception in method getUserAccountByEmail", e);
// return null;
// }
Since there has to be an UserAccount before any contact and groups, in my unit test when testing the saving of a contact object for which there must be an existing group i do this in order
a create userAccount object ua.
b create group object g1;
c create contact object c1;
d ua.addGroup(g1);
e c1.setUserAccount(ua);
f c1.addGroup(g1);
g uaDao.save(ua); // which saves the group because of the cascade
h cDao.save(c1);
Most of the time i use the session.get() from hibernate to pull c1 by its it id generated by hibernate and do all the assertions which works actually.
but in Integration test when i call getUserAccountByEmail with and without the setFetchMode and it returns the right object but then all the children collections are empty. i've tried the JOIN and the SELECT.the query string changes but then the result set is still the same. So this arises some questions :
1. What should i do to fix this?
2. the helper method works fine but it's on the parent side(i do it in the test).What i've been wondering about is that doing c1.setUserAccount(ua); is enough to create a strong relationship between UserAccount and contact.most of the time there will not be cases where i save the userAccount with contact but yet the helper method that set the association in both side and which is in UserAccount will not been called before i save the contact for a particular userAccount.So i'm little confused about that and suspecting that setting of the association is part of the why something is not working properly.and then calling session.get(UserAccount.class, ua.getID()) i think goes what i want and i'ld like getUserAccountByEmail to do the same.
3. ChssPly76 thinks the mapping has to be rewrite.So i'm willing to let you guide me through this.I really need to know the proper way to do this because we can't lean everything from a good book.So i you think i should change the mapping just show me how.and probable i'm doing things the wrong way without even been aware of that so don't forget i'm still learning java itself.THanks for the advise and remarks and thanks for reading this
I agree with you that it seems likely that the associations between your parent objects and their child collections are not getting persisted properly. I always like to start out by looking at what is in the database to figure out what's going on. After you run your test what do you see in the actual database?
It seems likely that one of two things is happening (using UserAccount as an example):
The items in the child collection are not getting saved to the database at all, in which case you'll be able to see in the database that there are no records associated with your UserAccount. This could be caused by saving the UserAccount object before you've added the child object to the UserAccount's collection.
The items in the child collection are getting saved to the database, but without the needed association to the parent object, in which case you'll see rows for your child items but the join column (ie 'userAccount' will be null). This could be caused by not setting the userAccount() property on the child object.
These are the two scenarios that I've run into where I've seen the problem you describe. Start by taking a look at what goes into your database and see if that leads you farther.