What is the good way to force initialization of Lazy Loaded field in each object of collection?
At this moment the only thing that comes to my mind is to use for each loop to iterate trough collection and call getter of that field but it's not very effiecient. Collection can have even 1k objects and in that case every iteration will fire to db.
I can't change the way I fetch objects from DB.
Example of code.
public class TransactionData{
#ManyToOne(fetch = FetchType.LAZY)
private CustomerData customer;
...
}
List<TransactionData> transactions = getTransactions();
You may define Entity Graphs to overrule the default fetch types, as they are defined in the Mapping.
See the example below
#Entity
#NamedEntityGraph(
name = "Person.addresses",
attributeNodes = #NamedAttributeNode("addresses")
)
public class Person {
...
#OneToMany(fetch = FetchType.LAZY) // default fetch type
private List<Address> addresses;
...
}
In the following query the adresses will now be loaded eagerly.
EntityGraph entityGraph = entityManager.getEntityGraph("Person.addresses");
TypedQuery<Person> query = entityManager.createNamedQuery("Person.findAll", Person.class);
query.setHint("javax.persistence.loadgraph", entityGraph);
List<Person> persons = query.getResultList();
In that way you are able to define specific fetch behaviour for each differet use-case.
See also:
http://www.thoughts-on-java.org/jpa-21-entity-graph-part-1-named-entity/
https://docs.oracle.com/javaee/7/tutorial/persistence-entitygraphs001.htm
By the way: afaik do most JPA provider perform eager loading of #XXXtoOne relations, even if you define the mapping as lazy. The JPA spec does allow this behaviour, as lazy loading is always just a hint that the data may or may not be loaded immediately. Eager Loading on other other hand has to be performed immediately.
What you can do is something like this:
//lazily loaded
List<Child> childList = parent.getChild();
// this will get all the child in memory of that particular Parent
Integer childListSize = childList.size();
But if you eager load then all the child will be loaded for each of the parents. This should be your best bet.
Related
I googled a lot and It is really bizarre that Spring Boot (latest version) may not have the lazy loading is not working. Below are pieces of my code:
My resource:
public ResponseEntity<Page<AirWaybill>> searchAirWaybill(CriteraDto criteriaDto, #PageableDefault(size = 10) Pageable pageable{
airWaybillService.searchAirWaybill(criteriaDto, pageable);
return ResponseEntity.ok().body(result);
}
My service:
#Service
#Transactional
public class AirWaybillService {
//Methods
public Page<AirWaybill> searchAirWaybill(AirWaybillCriteriaDto searchCriteria, Pageable pageable){
//Construct the specification
return airWaybillRepository.findAll(spec, pageable);
}
}
My Entity:
#Entity
#Table(name = "TRACKING_AIR_WAYBILL")
#JsonIdentityInfo(generator=ObjectIdGenerators.IntSequenceGenerator.class, property="#airWaybillId") //to fix Infinite recursion with LoadedAirWaybill class
public class AirWaybill{
//Some attributes
#NotNull
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "FK_TRACKING_CORPORATE_BRANCH_ID")
private CorporateBranch corporateBranch;
}
And when debugging, I still getting all lazy loaded attributed loaded. See image below.
One of my questions is could Jackson be involved in such behaviour?
Is there any way that I may have missed to activate the lazy loading?
EDIT
Another question, could the debugger be involved in ruining the lazy loading?
EDIT 2:
For specification build, I have :
public static Specification<AirWaybill> isBranchAirWayBill(long id){
return new Specification<AirWaybill>() {
#Override
public Predicate toPredicate(Root<AirWaybill> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
return cb.equal(root.join("corporateBranch",JoinType.LEFT).get("id"),id);
}
};
}
Hibernate Session exists within method with #Transactional.
Passing entity outside Service class is a bad practise because session is being closed after leaving your search method. On the other hand your entity contains lazy initialised collections, which cannot be pulled once session is closed.
The good practise is to map entity onto transport object and return those transport objects from service (not raw entities).
SpringBoot by default has enabled:
spring.jpa.open-in-view = true
That means transaction is always open. Try to disable it.
more information here
Most likely you are debugging while still being inside the service, thus while the transaction is still active and lazy loading can be triggered (any method called on a lazy element triggered the fetch from the database).
The problem is that lazy loading cannot occur while being outside of the transaction. And Jackson is parsing your entity definitely outside the boundaries of one.
You either should fetch all the required dependencies when building your specification or try with the #Transactional on the resource level (but try that as of last resort).
Just so that you know, LAZY fetching strategy is only a hint.. not a mandatory action. Eager is mandatory:
The LAZY strategy is a hint to the persistence provider runtime that
data should be fetched lazily when it is first accessed. The
implementation is permitted to eagerly fetch data for which the LAZY
strategy hint has been specified.
When using a debugger, you are trying to access the value of your variables. So, at the moment you click that little arrow on your screen, the value of the variable in question is (lazily) loaded.
I suppose you are using Hibernate as JPA.
From specification:
The EAGER strategy is a requirement on the persistence provider runtime that data must be eagerly fetched. The LAZY strategy is a hint to the persistence provider runtime that data should be fetched lazily when it is first accessed. The implementation is permitted to eagerly fetch data for which the LAZY strategy hint has been specified. https://docs.jboss.org/hibernate/jpa/2.2/api/javax/persistence/FetchType.html
Hibernate ignores fetch type specially in OneToOne and ManyToOne relationships from non owning side.
There are few options how to force Hibernate use fetch type LAZY if you really need it.
The simplest one is to fake one-to-many relationship. This will work because lazy loading of collection is much easier then lazy loading of single nullable property but generally this solution is very inconvenient if you use complex JPQL/HQL queries.
The other one is to use build time bytecode instrumentation. For more details please read Hibernate documentation: 19.1.7. Using lazy property fetching. Remember that in this case you have to add #LazyToOne(LazyToOneOption.NO_PROXY) annotation to one-to-one relationship to make it lazy. Setting fetch to LAZY is not enough.
The last solution is to use runtime bytecode instrumentation but it will work only for those who use Hibernate as JPA provider in full-blown JEE environment (in such case setting "hibernate.ejb.use_class_enhancer" to true should do the trick: Entity Manager Configuration) or use Hibernate with Spring configured to do runtime weaving (this might be hard to achieve on some older application servers). In this case #LazyToOne(LazyToOneOption.NO_PROXY) annotation is also required.
For more informations look at this:
http://justonjava.blogspot.com/2010/09/lazy-one-to-one-and-one-to-many.html
Just a guess: you are forcing a fetch while building your specification.
I expect something like
static Specification<AirWaybill> buildSpec() {
return (root, query, criteriaBuilder) -> {
Join<AirWaybill, CorporateBranch> br = (Join) root.fetch("corporateBranch");
return criteriaBuilder.equal(br.get("addressType"), 1);
};
}
If this is the case, try changing root.fetch to root.join
The retrieved data already lazy but you are using debug mode its return value when click to watch a data from a debugger.
You can solve this problem with wit 2 steps with jackson-datatype-hibernate:
kotlin example
Add In build.gradle.kts:
implementation("com.fasterxml.jackson.datatype:jackson-datatype-hibernate5:$jacksonHibernate")
Create #Bean
#Bean
fun hibernate5Module(): Module = Hibernate5Module()
Notice that Module is com.fasterxml.jackson.databind.Module, not java.util.Module
Another consideration is while using Lombok, #Data/#Getter annotation causes to load lazy items without need. So be careful when using Lombok.
This was my case.
I think I might have a solution. You can give this a try. This worked for me after 4 hours of hit and trial -
User Entity :
class User {
#Id
String id;
#JsonManagedReference
#OneToMany(mappedBy = "user", fetch = FetchType.LAZY)
private List<Address> addressDetailVOList = new ArrayList<Address>();
}
Address entity :
class Address {
#JsonBackReference
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "userId")
private User user;
}
Your parent class will use #JsonManagedReference, and child class will use #JsonBackReference. With this, you can avoid the infinite loop of entity objects as response and stack overflow error.
I also faced the same issue with Spring data JPA. I added the below annotation & able to get the customer records for a given ORDER ID
Customer to Order : one to Many
Order to customer is lazy load.
Order.java
#ManyToOne(cascade = CascadeType.ALL,targetEntity = CustomerEntity.class,fetch = FetchType.LAZY)
#Fetch(FetchMode. JOIN)
#JoinColumn(name = "CUSTOMER_ID",referencedColumnName = "CUSTOMER_ID",insertable = false,updatable = false)
#LazyToOne(LazyToOneOption.PROXY)
Private CustomerEntity customer
Customer.java
#Entity
#TabLe(name = "CUSTOMER" ,
uniqueConstraints = #UniqueConstraint(columnNames= {"mobile"}))
public class CustomerEntity {
#GeneratedVaLue(strategy = GenerationType.IDENTITY)
#CoLumn(name = "customer_id" )
private Integer customerld;
private String name;
private String address;
private String city;
private String state;
private Integer zipCode;
private Integer mobileNumber;
#OneToMany(mappedBy = " customer" )
#Fetch(FetchMode.JOIN)
#LazyToOne(LazyToOneOption.PROXY)
private List<OrderEntity> orders;
}
This is a simple Product entity that refers to a subgroup:
public class Product implements Comparable<Product> {
...
#ManyToOne(optional=false, fetch=FetchType.LAZY)
#NotNull
private ProductSubGroup productSubGroup;
...
}
I have a map containing Product's in another entity:
public class FinishedProduct {
...
#NotNull
#ManyToOne
private Product product;
#ElementCollection(fetch=FetchType.LAZY)
#MapKeyJoinColumn
#Column(name="amount")
#Sort(type=SortType.NATURAL)
#Fetch(FetchMode.SUBSELECT)
private SortedMap<Product, Double> byproducts = new TreeMap<>();
...
}
I can load the map with this code:
Root<FinishedProduct> root = q.from(FinishedProduct.class);
root.fetch("product", JoinType.LEFT);
root.fetch("byproducts", JoinType.LEFT);
This works, but I need the productSubGroup of the byproducts stored in the map without generating n+1 selects. How can I fetch them? Just adding the fetch to the end results in an exception:
root.fetch("byproducts", JoinType.LEFT).fetch("productSubGroup", JoinType.LEFT);
org.springframework.dao.InvalidDataAccessApiUsageException:
Collection of values [null] cannot be source of a fetch
Also tried to fool around with MapJoin, same exception:
MapJoin<FinishedProduct,Product,Double> map = root.joinMap("byproducts", JoinType.LEFT);
map.fetch("productSubGroup", JoinType.LEFT);
I guess I somehow need to refer to the map key, but no idea how.
These are a bit complex mappings that you have here and I am not sure if there is an easier way to accomplish this. Hopefully somebody will provide a better answer, but as an alternative there is always the ability to pre-load into the persistence context all the entity instances that you know will be fetched with n+1 selects.
So, before firing your query, just load all ProductSubGroups which are expected to be fetched:
select p.productSubGroup from Product p
where p in (select index(byproducts) from FinishedProduct)
Of course, repeat any other additional restrictions on FinishedProduct in the subquery which you have in your original query to avoid loading ProductSubGroups which you don't need.
As even better alternative (in my opinion), you may want to consider defining #BatchSize for Product.productSubGroup association. That way ProductSubGroups would be loaded in batches instead of one by one.
What is a best practice to store 'large' data, represented by List in Java, in database?
i'm considering 3 variants:
Use '#OneToMany' to store data in separate table.
Serialize data and store it in parent table.
Store data as files(naming conventions? same as id?).
To be more specific
'Large' data entities:
class SingleSleeper{
private Double startPositionOnLeft;
private Double endPositionOnLeft;
private Double startPositionOnRight;
private Double endPositionOnRight;
....
}
class RutEntry{
private Double width;
private Double position;
...
}
There are about 50 instances of SingleSleeper class and about 25000 instances of RutEntry class in one parent instance. Parent instances are generated about 40 times every day.
i'm using EclipseLink JPA 2.1, derby
Addition
Most of all i'm interested in best readability in Java. But i'm afraid that database speed will significantly decrease if i will store too much data into database. An overwhelming number of requests will be to select all instances of SingleSleeper or RutEntry classes of particular parent entity. I'm not interested for support to different database types, but i can move to other database, if needed.
I think I would do neither of your variants.
I would add a ManyToOne to the child entities (which is somehow the opposite of your first variant):
public class SingleSleeper {
#ManyToOne(optional = false, fetch = FetchType.LAZY)
private ParentEntity parent;
...
}
public class RutEntry {
#ManyToOne(optional = false, fetch = FetchType.LAZY)
private ParentEntity parent;
}
This ensures that you have a mapping and that you never load all 25000 entities for a parent object, if you don't need them (the lazy fetch ensures that you even don't need to load the parent entity).
You can create a OneToMany in the parent object with a mappedBy link, if you really want to. For example because you always need all child objects in the parent entity:
class ParentEntity {
#OneToMany(mappedBy = "parent", fetch = FetchType.LAZY)
Collection<SingleSleeper> singleSleepers;
#OneToMany(mappedBy = "parent", fetch = FetchType.LAZY)
Collection<RutEntry> rutEntries;
}
But I don't know how EclipseLink is working here - for Hibernate you need at least an additional BatchSize annotation to indicate that it should load as many child entities as possible at once. It can't fetch all together with the parent instance (e.g. by defining both as FetchType.EAGER), as only one is allowed to be fetched eagerly (and otherwise you would have 25000 * 50 result rows in the result set of the corresponding SQL select statement).
The best to load all child entities for a parent entity is to load them separate, either using JPQL (easier to read, faster to write) or the Criteria API (typesafe, but you need a metamodel):
ParentEntity parent = entityManager.find(ParentEntity.class, id);
// JPQL:
List<SingleSleeper> singleSleepers = entityManager.createQuery(
"SELECT s FROM SingleSleeper s WHERE s.parent = %parent"
).setParameter("parent", parent).getResultList();
// Or Criteria API:
CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
CriteriaQuery<SingleSleeper> query = criteriaBuilder.createQuery(SingleSleeper.class);
Root<SingleSleeper> s = query.from(SingleSleeper.class);
query.select(s).where(criteriaBuilder.equal(s.get(SingleSleeper_.parent), parent));
List<SingleSleeper> singleSleepers = entityManager.createQuery(query).getResultList();
You have three advantages of that approach:
Still easy to read - if you put the loading into its own method.
You are flexible to decide when to load the 25050 children.
You can load a subset of the children as well (by modifying the result of createQuery with Query.setFirstResult and Query.setMaxResults).
I have two entities, topics and posts respectively. Here's many posts for one topics.
So i had set up topics to OneToMany:
class topics implements Serializable {
...
#OneToMany(fetch = FetchType.EAGER, mappedBy = "topic")
#OrderBy("date DESC")
private List<posts> postsList;
public List<posts> getPostsList()
{
return this.postsList;
}
}
In posts entity are not any specific settings for topic field, just typical Integer.
So problem is i can't fetch this postsList with EAGER fetch type, it will stay LAZY anyway:
topics theTopic = (topics) session.load(topics.class, topic);
List<posts> postsOfTopic = theTopic.getPostsList();
... the List contains one object now, but it's full of nulls.
Any ideas how to fix?
The session.load method typically returns a proxy whereas the session.get method is guaranteed to return the initialized entity and would eagerly fetch collection if marked as such. Also there is option of using eager fetch HQL queries to eager load the associated collection.
I have to do bulk inserts, and need the ids of what's being added. This is a basic example that shows what I am doing (which is obviously horrible for performance). I am looking for a much better way to do this.
public void omgThisIsSlow(final Set<ObjectOne> objOneSet,
final Set<ObjectTwo> objTwoSet) {
for (final ObjectOne objOne : objOneSet) {
persist(objOne);
for (final ObjThree objThree : objOne.getObjThreeSet()) {
objThree.setObjOne(objOne);
persist(objThree);
}
for (final ObjectTwo objTwo : objTwoSet) {
final ObjectTwo objTwoCopy = new ObjTwo();
objTwoCopy.setFoo(objTwo.getFoo());
objTwoCopy.setBar(objTwo.getBar());
persist(objTwoCopy);
final ObjectFour objFour = new ObjectFour();
objFour.setObjOne(objOne);
objFour.setObjTwo(objTwoCopy);
persist(objFour);
}
}
}
In the case above persist is a method which internally calls
sessionFactory.getCurrentSession().saveOrUpdate();
Is there any optimized way of getting back the ids and bulk inserting based upon that?
Thanks!
Update: Got it working with the following additions and help from JustinKSU
import javax.persistence.*;
#Entity
public class ObjectFour{
#ManyToOne(cascade = CascadeType.ALL)
private ObjectOne objOne;
#ManyToOne(cascade = CascadeType.ALL)
private ObjectTwo objTwo;
}
// And similar for other classes and their objects that need to be persisted
If you define the relationships using annotations and define appropriate cascading, you should be able set the object relationships in the objects in java and persist it all in one call. Hibernate will handle setting the foreign keys for you.
Documentation -
http://docs.jboss.org/hibernate/annotations/3.5/reference/en/html/entity.html#entity-mapping-association
An example annotation on a parent object would be
#OneToMany(mappedBy = "foo", fetch = FetchType.LAZY, cascade=CascadeType.ALL)
On the child object you would do the following
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "COLUMN_NAME", nullable = false)
I'm not sure but Hibernate makes bulk inserts/updates. The problem I understand is you need to persist the parent object in order to assign the reference to the child object.
I would try to persist all the "one" objects. And then, iterate over all their "three" objects and persist them in a second bulk insertion.
If your tree has three levels you can achieve all the insertions in 3 batchs. Pretty decent I think.
Assuming that you are just looking at getting a large amount of data persisted in one go and your problem is that you don't know what the IDs are going to be as the various related objects are persisted, one possible solution for this is to run all your inserts (as bulk inserts) into ancillary tables (one per real table) with temporary IDs (and some session ID) and have a stored procedure perform the inserts into the real tables whilst resolving the IDs.