I have following classes:
Company.class:
public class Company {
#JoinTable(name = "company_employee", joinColumns = #JoinColumn(name = "company_id") , inverseJoinColumns = #JoinColumn(name = "employee_id") )
#ManyToMany(fetch = FetchType.LAZY)
private Set<Employee> employees;
#Column(name = "score")
private BigDecimal score;
}
and Employee.class
public class Employee {
#ManyToMany(fetch = FetchType.EAGER, mappedBy="employees")
private Set<Company> companies;
}
The Score column of Company is always null in the db and never updated via dao, because there is other table containing score for each unique pair Company-Employee.
I need the value of Score, only for the case when I fetch Employee by id, so this case all Company instances in the Set should contain score, thus I will get Employee-Company score pairs where employee is fetched Employee.
I have following code to achieve that:
public Employee get(Long id) {
Employee emp = (Employee) dao.find(id);
List<Company> compList = compnanyService.getByEmpId(id);
Set<Company> compSet = new HashSet<Company>(compList);
emp.setCompanies(compSet);
return emp;
}
And Company Dao contains method:
public List<Company> getByEmpId(Long id) {
final Query query = this.entityManager.createNativeQuery("select company.comp_id, ...some other fields, score.score from company join score on company.company_id=score.company_id where score.employee_id=:employee_id",
Company.class);
query.setParameter("employee_id", id);
List<Company> comps = query.getResultList();
return comps;
}
The problem is that getByEmpId(id) gives a ResultList where company.score is null though executed in the db it is not null.
I suspected that there is some caching intervening, so I tried to remove some columns from the native query, and it should have invoked an exception with "no column found" (or alike) message while mapping, but this method still gives List<Company> with all fields on their places though Hibernate prints out my native query in the console with all changes I make.
What am I doing wrong here and how to achieve what I need? Thank you.
It might be associated with first level cache, which can be out of sync when using native SQL queries. From here:
If you bypass JPA and execute DML directly on the database, either
through native SQL queries, JDBC, or JPQL UPDATE or DELETE queries,
then the database can be out of synch with the 1st level cache. If you
had accessed objects before executing the DML, they will have the old
state and not include the changes. Depending on what you are doing
this may be ok, otherwise you may want to refresh the affected objects
from the database.
So you can try using refresh method from EntityManager.
So I ended up doing that:
Created view in db from the query:
CREATE VIEW companyscore AS select company.comp_id, score.emp_id ...some other fields, score.score from company join score on company.comp_id=score.comp_id;
Created corresponding entity CompanyScore with composite primary id as comp_id and emp_id and created view as table.
Changed Employee entity to:
public class Employee {
#OneToMany(fetch = FetchType.EAGER)
#JoinColumn(name = "emp_id")
private Set<CompanyScore> companies;
}
This way I not only have score field always consistent, but I can choose set of fields to show as the whole Company class is quite extensive and I don't need all the fields for this particular case.
Related
I'm using Spring boot JPA to get list of objects (Using Java 8 now). Each object has relationships and I use the related objects also to transform to a dto list.
Let's say I have below model classes.
public class Product {
#EmbeddedId
private ProductId id;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "userid", referencedColumnName = "USER_ID")
#MapsId("userId")
private User owner;
}
public class User {
#Id
#Column(name = "USER_ID")
private Long userId;
#OneToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "gp_code", referencedColumnName = "GP_CODE")
#JoinColumn(name = "userid", referencedColumnName = "USER_ID")
private UserGroup userGroup;
#OneToOne(fetch = FetchType.LAZY)
#JoinColumnsOrFormulas(value = {
#JoinColumnOrFormula(formula = #JoinFormula(value = "country_id", referencedColumnName = "COUNTRY_ID")),
#JoinColumnOrFormula(column = #JoinColumn(name = "region_code", referencedColumnName = "REGION_CODE")) })
private Country country;
}
I do query for List<Product> and using stream I'm converting it into a dto object. During which I call the related entity to get the data. I have the below code and works fine unless the list is too much. If I have 1000+ items in the list it takes around 30 seconds.
I believe because of lazy loading this is happening. What is the best way to optimize this?
One option is to do pagination, but I cannot do it. I need all results together.
Is there any way to parallelly execute this? I tried to call parellelStream() instead of stream(), but it's same result.
public List<ProductDTO> getProducts(String countryId) {
//List<Product> products = Query Result
List<ProductDTO> productsList = products.stream().filter(isOwnerFromCountryAndInAnyGroup(countryId))
.map(product -> getProductDTO(product)).collect(Collectors.toList());
}
private Predicate<? super Product> isOwnerFromCountryAndInAnyGroup(String countryId) {
return product -> {
User user = product.getOwner();
return null != user && null != user.getCountry()
&& user.getCountry().getCountryId().equals(countryId) && (null != user.getUserGroup());
};
}
private ProductDTO getProductDTO(Product product) {
ProductDTO productDTO = new ProductDTO();
productDTO.setProductNbr(product.getId().getProductNbr());
productDTO.setPrice(product.getPrice());
productDTO.setOwnerName(product.getOwner().getName());
return productDTO;
}
Edit
I missed to add the line productDTO.setOwnerName(product.getOwner().getName()); for the purpose of asking question here. With query or using filter I'm getting the correct number of results. And with lazy loading, query returns faster and then while calling getOwner() for each row, the process takes time (30 seconds).
And with FethType.EAGER, the query takes similar time(30 seconds) and then processes faster. Either way it is similar time.
To fasten the process, is there any way to execute the stream code block in parallel and collect all results together in list?
public List<ProductDTO> getProducts(String countryId) {
//List<Product> products = Query Result
List<ProductDTO> productsList = products.stream().filter(isOwnerFromCountryAndInAnyGroup(countryId))
.map(product -> getProductDTO(product)).collect(Collectors.toList());
}
From your use case here I am pretty confident that it is not the creation of DTO that takes time. It is that you retrieve a huge set from database (even the complete table of Products) and then you filter for a relation with a specific country just from java.
So Step1 optimization:
If you want to filter for products that are associated with a user from a specific country then this can go on JPA level and translated in optimal way in database. Then the allocation of resources (memory, cpu) would be much more optimal, instead of your java application trying to load a huge data set and filter it there.
#Query("SELECT p FROM Product p where p.owner IS NOT NULL AND p.owner.userGroup IS NOT NULL AND p.owner.country IS NOT NULL AND p.owner.country.id = :countryId")
List<Product> findProductRelatedWithUserFromCountry(#Param String countryId);
and remove the filtering from your method getProducts.
Step2 optimization:
In addition to the above, not only you can pass the java filtering in the database query by moving it to JPA layer but you can also optimize the query a bit more by defining in JPA that you want to load the associated Owner as well so that it doesn't hit later the database to retrieve it when you create the DTO. You can achieve this with join fetch, so your query should now become:
#Query("SELECT p FROM Product p JOIN FETCH p.owner own where p.owner IS NOT NULL AND own.userGroup IS NOT NULL AND own.country IS NOT NULL AND own.country.id = :countryId")
List<Product> findProductRelatedWithUserFromCountry(#Param String countryId);
Step3 optimization:
If we want to take it an extra step further it seems that most times using DTO projections would speed up the execution. This can happen as the query would define only specific information it needs to retrieve and convert into DTO instead of the complete entities.
So your query now would be:
#Query("SELECT new org.your.package.where.dto.is.ProductDTO(p.id.productNbr, p.price, own.name) FROM Product p JOIN FETCH p.owner own where p.owner IS NOT NULL AND own.userGroup IS NOT NULL AND own.country IS NOT NULL AND own.country.id = :countryId")
List<ProductDTO> findProductRelatedWithUserFromCountry(#Param String countryId);
Also remember to have the DTO constructor used in the JPA query available in your ProductDTO.class.
In my REST API project (Java 8, Spring Boot 2.3.1) I have a problem with some queries triggering massive query chains by loading lazy relations, even though the related objects are never accessed.
I have a UserEntity and a polymorphic CompanyEntity that are related with a ManyToMany relationship. I have an endpoint that returns all users and I include the IDs of the related companies in the JSON. I excpect a query to the user table and a query to the company table, however all related entities of one sub-entity of CompanyEntity are always loaded for each of those sub-entities resulting in large query chains.
Here are snippets of my classes:
User entity
#Entity(name = "USERS")
public class UserEntity {
#Id
#GeneratedValue
private UUID id;
#EqualsAndHashCode.Exclude
#Fetch(FetchMode.SUBSELECT)
#ManyToMany(fetch = FetchType.LAZY)
#JoinTable(
name = "users_company",
joinColumns = #JoinColumn(name = "USER_ID"),
inverseJoinColumns = #JoinColumn(name = "COMPANY_ID")
)
private Set<CompanyEntity> companies = new HashSet<>();
public List<UUID> getCompanyIds() {
return companies.stream()
.map(CompanyEntity::getId)
.collect(Collectors.toList());
}
}
Polymorphic company entity
#Entity(name = "COMPANY")
#Inheritance(strategy = InheritanceType.JOINED)
public abstract class CompanyEntity {
#Id
#GeneratedValue
private UUID id;
#Fetch(FetchMode.SUBSELECT)
#ManyToMany(mappedBy = "companies", fetch = FetchType.LAZY)
private Set<UserEntity> users = new HashSet<>();
}
Concrete company subclass that triggers the problem
#Entity(name = "CUSTOMER")
public class CustomerEntity extends CompanyEntity {
#NotNull
#OneToOne(cascade = {CascadeType.PERSIST, CascadeType.MERGE}, fetch = FetchType.LAZY)
private ContactPersonEntity contactPerson;
#Fetch(FetchMode.SUBSELECT)
#OneToMany(cascade = {CascadeType.PERSIST, CascadeType.MERGE}, fetch = FetchType.LAZY, mappedBy = "customer")
private Set<TransactionEntity> transactions = new HashSet<>();
public Set<UUID> getTransactionIds() {
return this.transactions.stream()
.map(TransactionEntity::getId)
.collect(Collectors.toSet());
}
}
In the REST controller I return the following mapping:
#GetMapping(value = "", produces = MediaType.APPLICATION_JSON_VALUE)
public List<UserReadModel> getUsers() {
return userRepository.findAll().stream()
.map(userEntity -> new UserReadModel(userEntity))
.collect(Collectors.toList());
}
Where the UserReadModel is a DTO:
#Data
public class UserReadModel {
private UUID id;
private List<UUID> companyIds;
}
Logging the database queries results in the following output:
// Expected
Hibernate: select userentity0_.id as id1_47_, ... from users userentity0_
Hibernate: select companies0_.user_id ... case when companyent1_1_.id is not null then 1 when companyent1_2_.id is not null then 2 when companyent1_.id is not null then 0 end as clazz_0_ from users_company companies0_ inner join company companyent1_ on companies0_.company_id=companyent1_.id left outer join customer companyent1_1_ on companyent1_.id=companyent1_1_.id left outer join external_editor companyent1_2_ on companyent1_.id=companyent1_2_.id where companies0_.user_id in (select userentity0_.id from users userentity0_)
// Unexpected as they are marked lazy and never accessed
Hibernate: select contactper0_.id ... from contact_person contactper0_ where contactper0_.id=?
Hibernate: select transactio0_.customer_id ... from transactions transactio0_ where transactio0_.customer_id=?
Hibernate: select contactper0_.id ... from contact_person contactper0_ where contactper0_.id=?
Hibernate: select transactio0_.customer_id ... from transactions transactio0_ where transactio0_.customer_id=?
...
I've read through loads of articles on entity mapping and lazy loading but I can't seem to find a reason why this behavior persists. Did anyone have this problem before?
You are accessing the collection, so Hibernate has to load the collection. Since you only need the ids and already have a DTO, I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(UserEntity.class)
public interface UserReadModel {
#IdMapping
UUID getId();
#Mapping("companies.id")
Set<UUID> getCompanyIds();
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
UserReadModel a = entityViewManager.find(entityManager, UserReadModel.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<UserReadModel> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary! In your case, a query like the following will be generated:
select u.id, uc.company_id
from users u
left join users_company uc on uc.user_id = u.id
left join company c on c.id = uc.company_id
Depending on the Hibernate version, the join for the company might even be omitted.
I eventually figured out the solution and want to post it here, in case anyone stumbles upon this question. This was purely a mistake on my side and is not reproducible from the examples I posted.
I used lombok annotations to generate equals and hashcode methods on the customer entity (and all other entities for that matter) and forgot to annotate the contactPerson and transactions fields with #EqualsAndHashcode.Exclude. As the equals method was called somewhere along the execution, it triggered the lazy loading of those fields. Implementing equals and hashcode manually and using the guidelines from this article for that solved the problem.
I have a parent entity Stock which has a child entity StockDetails in a OneToOne relation.
I can't figure out how to properly set and replace values for the Stock.details field.
Here are my entity classes (#Getter/#Setter from Lombok):
#Getter
#Setter
#NoArgsConstructor
#Entity
#Table(name = "stocks")
public class Stock
{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
private String isin;
private String symbol;
private String name;
#OneToMany(mappedBy = "stock", cascade = CascadeType.ALL, orphanRemoval = true, fetch = FetchType.LAZY)
private List<StockChartPoint> chart;
#OneToOne(mappedBy = "stock", cascade = CascadeType.ALL, orphanRemoval = true, fetch = FetchType.LAZY)
private StockDetails details;
public void setDetails(StockDetails d)
{
details = d;
details.setStock(this);
}
}
and
#Getter
#Setter
#NoArgsConstructor
#Entity
#Table(name = "stock_details")
public class StockDetails
{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
#OneToOne
#JoinColumn(name = "stock")
private Stock stock;
}
Inserting a new value (and corresponding row) into the DB works fine when the details table is empty and I can see that Hibernate logs exactly one intert statement. My code looks like this:
dbService.transactional(session ->
{
var stocks = getAllStocks();
var s = stocks.get(0);
StockDetails n = new StockDetails();
s.setDetails(n);
session.save(s);
});
The DbService.transactional method (TransactionExecutor is just a functional interface):
public void transactional(TransactionExecutor executor)
{
Transaction t = null;
try
{
t = session.beginTransaction();
executor.execute(session);
session.flush();
t.commit();
}
catch (HibernateException ex)
{
if (t != null)
t.rollback();
}
}
But when there is already an existing row in the details table the old row is deleted and a new row is inserted. The result is that the PK of the details table is increasing every time I update the values. Is there a pattern which I couldn't find to address this? I could also just update the existing Stock.details field but this would lead in just copying all the fields from object A to object B and I guess there is a smarter way doing this. I tried using an EntityManager, merge()/saveOrUpdate()/persist() instead of save() as well as manipulating the ID in the new object but this resulted in changing nothing or throwing exceptions.
Then there is another problem I encountered and I don't know if it's related to the first one:
When executing the dbService.transactional(...) block twice it behaves differently: Now two rows are added to the DB. The SQL log looks like this:
...
Hibernate: insert into stock_details (stock) values (?)
Hibernate: delete from stock_details where id=?
-> insert/delete produces by the first run
Hibernate: select stock0_.id as id1_3_, stock0_.isin as isin2_3_, stock0_.name as name3_3_,
stock0_.symbol as symbol4_3_ from stocks stock0_
Hibernate: insert into stock_details (stock) values (?)
-> Just insert, no delete
Please let me know if more information is needed.
mysql.mysql-connector-java > 8.0.22
org.hibernate.hibernate-core > 5.4.25.Final
org.hibernate.hibernate-validator > 5.4.3.Final
But when there is already an existing row in the details table the old row is deleted...
...which is to be expected with orphanRemoval = true
...and a new row is inserted
...which is obviously to be expected as well, since you're overwriting the existing StockDetails associated with s with a brand new instance of StockDetails.
If you wish to update the existing StockDetails, rather than create a new StockDetails entity, you need to, well, do just that in Java code.
I could also just update the existing Stock.details field but this would lead in just copying all the fields from object A to object B...
that would be the least error-prone approach
...but I guess there is a smarter way doing this
You could just do:
StockDetails n = new StockDetails();
n.setId(s.getStockDetails().getId());
... //configure the remaining properties
n.setStock(s);
entityManager.merge(n);
As you might have guessed you are actually creating a new StockDetails and replacing the old one. If you want to update the existing StockDetails you really need to fetch that and update it field by field (as you told you could do). So like:
StockDetails sd = s.getStockDetails();
sd.setSomeFiled(updateValue);
// ... copying field by field
To prevent copying field by field you can obtain StockDetails from the database and make the possible edits directly into it so there would not be a need to copy each field.
However if it is a case - for example - that you need to write values from some DTO to your entity there are libraries that you can use to ease the pain of copying.
Just to mention one ModelMapper. With it the copying goes like:
ModelMapper mm = new ModelMapper();
StockDetails sd = s.getStocDetails();
mm.map(stockDetailsDto, sd);
where stockDetailsDto is some DTO object that contains fields to update to entity.
If your StockDetails contains all the other fields also even those not changed the em.merge is most easy as told in answer from crizzis.
But if you get only the updated fields then the other fields would be set to null. Sometimes id is not settable and then merge is impossible.
I am working with 2 entities. Let's call the entities Company (database table companies) and Employee (database table employees). The 2 entities are associated in a many to many relationship. Thier associations are held in a database table - lets call it company_to_employee.
The Employee object contains no references to Company while the Company object contains the following for it's association to Employee.
// Inside Company.java
#ManyToMany
#JoinTable(name = "company_to_employee", joinColumns = { #JoinColumn(name = "company_id") }, inverseJoinColumns = { #JoinColumn(name = "employee_id") })
private Set<Employee> employees;
I have a database query that is run when a user logs in which grabs the active companies that they own.
public List<Company> getActiveCompanies(String userId) {
CriteriaBuilder builder = entityManager.getCriteriaBuilder();
CriteriaQuery<Company> query = builder.createQuery(Company.class);
Root<Company> company = query.from(Company.class);
Join<Company, CompanyUser> owner = company.join(Company_.owner);
Join<CompanyUser,User> user = owner.join(CompanyUser_.user);
Predicate predicate = builder.and(
builder.equal(user.get(User_.id), userId),
builder.equal(company.get(Company_.enabled), true),
builder.equal(owner.get(CompanyUser_.enabled), true));
query.where(predicate);
query.orderBy(builder.asc(builder.lower(company.get(Company_.name))));
return entityManager.createQuery(query).getResultList();
}
The returned list of companies is used by the front end to display companies along with their employees, etc.
I recently added a new enabled field onto Employee. However, the employees on the companies returned by the above query do not pick up changes to this field.
#Column(name = "ENABLED")
private boolean enabled;
I've tried replacing the query's last line with the following with no luck (caching hint).
return entityManager.createQuery(query).setHint("org.hibernate.cacheable", false).getResultList();
I've also tried updating the mapping to #ManyToMany(fetch = FetchType.EAGER) but this also did nothing. And the two attempts did nothing in combination with one another.
The only way I've been able to fix the issue is to add the following to the beginning of the query, but I'd rather not have to flush the whole cache like this.
entityManager.unwrap(Session.class).clear();
Am I missing something?
I'm writing a service with JPA and Postgres db. I have a class named Student:
public class Student {
#id
private String id;
private String firstName;
private String lastName;
#OneToMany(targetEntity = Phone.class, fetch = FetchType.LAZY, cascade = CascadeType.ALL, orphanRemoval = true)
#JoinColumn(name = "student_id", referencedColumnName = "id")
private Set<Phone> phones;
// Setter and Getter
}
And a Phone class:
public class Phone {
#id
private String id;
private String number;
// Setter and Getter
}
Now there will be two tables in my db, following are columns in them:
student: id, first_name, last_name
phone: id, number, student_id (generated by #JoinColumn)
Now every time when I query the student, JPA will join the phone table into student table and the student result contains the phone info of this student. This is exactly what I want.
But now I met a problem. When I query the list of the students, the phone info is not useful in this case, only the id, firstName and lastName are necessary. But JPA does the same "join" operation for me too. I know this consumes lots of time. In such case how could I just return the info in the student table? Without joining the phone table?
I tried something in the repository like
#Query(SELECT s.id, s.firstName, s.lastName FROM student s)
public List<Student> findAllStudentWithoutPhone();
but it returns me the list of the values, but not converted into a Student object. How could I achieve this feature?
In the one-to-many mapping (refer below) you have set the fetch type to be lazy fetch = FetchType.LAZY hence hibernate won't fetch set of phones corresponding to the student until you access the set of phones via getter method, so no need to worry about it.
#OneToMany(targetEntity = Phone.class, fetch = FetchType.LAZY, cascade = CascadeType.ALL, orphanRemoval = true)
#JoinColumn(name = "student_id", referencedColumnName = "id")
private Set<Phone>;
However in case you need to verify whether things are working fine or not you can set the property show_sql to true and check the sql generated via hibernate.
fetch = FetchType.LAZY provides you possibility don't query Phone table until of the first call from the code. Please see: http://docs.oracle.com/javaee/7/api/javax/persistence/FetchType.html#LAZY
In case if you want to retrieve the list of the students without phones query should be:
#Query(SELECT * FROM student s where phones IS NULL)
For automatically converting results into Student object please don't use s.id, s.firstName, s.lastName in your query.
Answer given by #Sahil is absolutely correct but to add that.
#Cong
You do not need to add FetchType.LAZY as by default its already LAZY.
& as a note the Student class property for phone missing variable name.