I am trying to write a JPA NamedQuery. I have a working SQL query as follows:
SELECT MIN(t1.Id + 1) AS nextID
FROM
MyTable t1 LEFT JOIN MyTable t2
ON
t1.Id + 1 = t2.Id
WHERE
t2.Id IS NULL
I am not able to translate this query to JPQL syntax.
I think doing such a custom join is not possible with standard JPQL. I was looking for a possibility to do it some time ago and found that Hibernate offers a proprietary extension #JoinFormula to achieve this, cf. Hibernate #JoinFormula. But I couldn't find an equivalent for EclipseLink.
You might be able use a #NamedNativeQuery together with an #SqlResultSetMapping to map your SQL statement to a JPA entity, something like this:
#NamedNativeQuery( name = "customJoin",
query = "SELECT MIN(t1.Id + 1) AS nextID FROM MyTable t1 LEFT JOIN MyTable t2 ON t1.Id + 1 = t2.Id WHERE t2.Id IS NULL",
resultSetMapping = "customJoinMapping" )
#SqlResultSetMapping( name = "customJoinMapping",
entities = #EntityResult( entityClass = MyTable.class, fields = #FieldResult( name = "id", column = "nextID" ) ) )
#Entity
#Access( AccessType.Field )
public class MyTable {
private int id;
public int getId() {
return this.id;
}
public void setId( int id ) {
this.id = id;
}
}
UPDATE
Much later, I found a way to customize relationship joins in EclipseLink here. The feature to use is a DescriptorCustomizer, which can be placed on a JPA entity with #Customizer.
In the customize() method, one can express additional join criteria using the EclipseLink Expression API. For example, the following code fragment would produce the additional join criteria [...] AND TargetEntity.someAttribute IN ('V', 'W', 'U') (where someRelationship points to a TargetEntity).
OneToManyMapping mapping = (OneToManyMapping) descriptor.getMappingForAttributeName( "someRelationship" );
Expression origExp = mapping.buildSelectionCriteria();
ExpressionBuilder expBuilder = origExp.getBuilder();
Expression constantExp = expBuilder.get( "someAttribute" ).in( new String[] { "V", "W", "U" } );
Expression newExp = origExp.and( constantExp );
mapping.setSelectionCriteria( newExp );
Related
I'm getting a warning in the Server log "firstResult/maxResults specified with collection fetch; applying in memory!". However everything working fine. But I don't want this warning.
My code is
public employee find(int id) {
return (employee) getEntityManager().createQuery(QUERY).setParameter("id", id).getSingleResult();
}
My query is
QUERY = "from employee as emp left join fetch emp.salary left join fetch emp.department where emp.id = :id"
Although you are getting valid results, the SQL query fetches all data and it's not as efficient as it should.
So, you have two options.
Fixing the issue with two SQL queries that can fetch entities in read-write mode
The easiest way to fix this issue is to execute two queries:
. The first query will fetch the root entity identifiers matching the provided filtering criteria.
. The second query will use the previously extracted root entity identifiers to fetch the parent and the child entities.
This approach is very easy to implement and looks as follows:
List<Long> postIds = entityManager
.createQuery(
"select p.id " +
"from Post p " +
"where p.title like :titlePattern " +
"order by p.createdOn", Long.class)
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setMaxResults(5)
.getResultList();
List<Post> posts = entityManager
.createQuery(
"select distinct p " +
"from Post p " +
"left join fetch p.comments " +
"where p.id in (:postIds) " +
"order by p.createdOn", Post.class)
.setParameter("postIds", postIds)
.setHint(
"hibernate.query.passDistinctThrough",
false
)
.getResultList();
Fixing the issue with one SQL query that can only fetch entities in read-only mode
The second approach is to use SDENSE_RANK over the result set of parent and child entities that match our filtering criteria and restrict the output for the first N post entries only.
The SQL query can look as follows:
#NamedNativeQuery(
name = "PostWithCommentByRank",
query =
"SELECT * " +
"FROM ( " +
" SELECT *, dense_rank() OVER (ORDER BY \"p.created_on\", \"p.id\") rank " +
" FROM ( " +
" SELECT p.id AS \"p.id\", " +
" p.created_on AS \"p.created_on\", " +
" p.title AS \"p.title\", " +
" pc.id as \"pc.id\", " +
" pc.created_on AS \"pc.created_on\", " +
" pc.review AS \"pc.review\", " +
" pc.post_id AS \"pc.post_id\" " +
" FROM post p " +
" LEFT JOIN post_comment pc ON p.id = pc.post_id " +
" WHERE p.title LIKE :titlePattern " +
" ORDER BY p.created_on " +
" ) p_pc " +
") p_pc_r " +
"WHERE p_pc_r.rank <= :rank ",
resultSetMapping = "PostWithCommentByRankMapping"
)
#SqlResultSetMapping(
name = "PostWithCommentByRankMapping",
entities = {
#EntityResult(
entityClass = Post.class,
fields = {
#FieldResult(name = "id", column = "p.id"),
#FieldResult(name = "createdOn", column = "p.created_on"),
#FieldResult(name = "title", column = "p.title"),
}
),
#EntityResult(
entityClass = PostComment.class,
fields = {
#FieldResult(name = "id", column = "pc.id"),
#FieldResult(name = "createdOn", column = "pc.created_on"),
#FieldResult(name = "review", column = "pc.review"),
#FieldResult(name = "post", column = "pc.post_id"),
}
)
}
)
The #NamedNativeQuery fetches all Post entities matching the provided title along with their associated PostComment child entities. The DENSE_RANK Window Function is used to assign the rank for each Post and PostComment joined record so that we can later filter just the amount of Post records we are interested in fetching.
The SqlResultSetMapping provides the mapping between the SQL-level column aliases and the JPA entity properties that need to be populated.
Now, we can execute the PostWithCommentByRank #NamedNativeQuery like this:
List<Post> posts = entityManager
.createNamedQuery("PostWithCommentByRank")
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setParameter(
"rank",
5
)
.unwrap(NativeQuery.class)
.setResultTransformer(
new DistinctPostResultTransformer(entityManager)
)
.getResultList();
Now, by default, a native SQL query like the PostWithCommentByRank one would fetch the Post and the PostComment in the same JDBC row, so we will end up with an Object[] containing both entities.
However, we want to transform the tabular Object[] array into a tree of parent-child entities, and for this reason, we need to use the Hibernate ResultTransformer.
The DistinctPostResultTransformer looks as follows:
public class DistinctPostResultTransformer
extends BasicTransformerAdapter {
private final EntityManager entityManager;
public DistinctPostResultTransformer(
EntityManager entityManager) {
this.entityManager = entityManager;
}
#Override
public List transformList(
List list) {
Map<Serializable, Identifiable> identifiableMap =
new LinkedHashMap<>(list.size());
for (Object entityArray : list) {
if (Object[].class.isAssignableFrom(entityArray.getClass())) {
Post post = null;
PostComment comment = null;
Object[] tuples = (Object[]) entityArray;
for (Object tuple : tuples) {
if(tuple instanceof Identifiable) {
entityManager.detach(tuple);
if (tuple instanceof Post) {
post = (Post) tuple;
}
else if (tuple instanceof PostComment) {
comment = (PostComment) tuple;
}
else {
throw new UnsupportedOperationException(
"Tuple " + tuple.getClass() + " is not supported!"
);
}
}
}
if (post != null) {
if (!identifiableMap.containsKey(post.getId())) {
identifiableMap.put(post.getId(), post);
post.setComments(new ArrayList<>());
}
if (comment != null) {
post.addComment(comment);
}
}
}
}
return new ArrayList<>(identifiableMap.values());
}
}
The DistinctPostResultTransformer must detach the entities being fetched because we are overwriting the child collection and we don’t want that to be propagated as an entity state transition:
post.setComments(new ArrayList<>());
Reason for this warning is that when fetch join is used, order in result sets is defined only by ID of selected entity (and not by join fetched).
If this sorting in memory is causing problems, do not use firsResult/maxResults with JOIN FETCH.
To avoid this WARNING you have to change the call getSingleResult to
getResultList().get(0)
This warning tells you Hibernate is performing in memory java pagination. This can cause high JVM memory consumption.
Since a developer can miss this warning, I contributed to Hibernate by adding a flag allowing to throw an exception instead of logging the warning (https://hibernate.atlassian.net/browse/HHH-9965).
The flag is hibernate.query.fail_on_pagination_over_collection_fetch.
I recommend everyone to enable it.
The flag is defined in org.hibernate.cfg.AvailableSettings :
/**
* Raises an exception when in-memory pagination over collection fetch is about to be performed.
* Disabled by default. Set to true to enable.
*
* #since 5.2.13
*/
String FAIL_ON_PAGINATION_OVER_COLLECTION_FETCH = "hibernate.query.fail_on_pagination_over_collection_fetch";
the problem is you will get cartesian product doing JOIN. The offset will cut your recordset without looking if you are still on same root identity class
I guess the emp has many departments which is a One to Many relationship. Hibernate will fetch many rows for this query with fetched department records. So the order of result set can not be decided until it has really fetch the results to the memory. So the pagination will be done in memory.
If you do not want to fetch the departments with emp, but still want to do some query based on the department, you can achieve the result with out warning (without doing ordering in the memory). For that simply you have to remove the "fetch" clause. So something like as follows:
QUERY = "from employee as emp left join emp.salary sal left join emp.department dep where emp.id = :id and dep.name = 'testing' and sal.salary > 5000 "
As others pointed out, you should generally avoid using "JOIN FETCH" and firstResult/maxResults together.
If your query requires it, you can use .stream() to eliminate warning and avoid potential OOM exception.
try (Stream<ENTITY> stream = em.createQuery(QUERY).stream()) {
ENTITY first = stream.findFirst().orElse(null); // equivalents .getSingleResult()
}
// Stream returned is an IO stream that needs to be closed manually.
I'm trying to write a CriteriaQuery which will query latest observation for each city. City is defined by city_code field, while latest record is defined by observation_time field.
I can easily write it in a plain SQL, but I cant understand how to do it with jpa criteria api.
select distinct m.* from
(select city_code cc, max(observation_time) mo
from observations group by city_code) mx, observations m
where m.city_code = mx.cc and m.observation_time = mx.mo`
It is possible when You are open for loose efficiency.
So first let's transform our query to logical equivalent one:
select distinct m.* from observations m where
m.observation_time = (select max(inn. observation_time) from observations inn
where inn.city_code = m.city_code);
then let's translate it to JPA CriteriaQuery:
public List<Observation> maxForEveryWithSubquery() {
CriteriaBuilder builder = entityManager.getCriteriaBuilder();
CriteriaQuery<Observation> query = builder.createQuery(Observation.class);
Root<Observation> observation = query.from(Observation.class);
query.select(observation);
Subquery<LocalDateTime> subQuery = query.subquery(LocalDateTime.class);
Root<Observation> observationInner = subQuery.from(Observation.class);
subQuery.where(
builder.equal(
observation.get(Observation_.cityCode),
observationInner.get(Observation_.cityCode)
)
);
Subquery<LocalDateTime> subSelect = subQuery.select(builder.greatest(observationInner.get(Observation_.observationTime)));
query.where(
builder.equal(subSelect.getSelection(), observation.get(Observation_.observationTime))
);
TypedQuery<Observation> typedQuery = entityManager.createQuery(query);
return typedQuery.getResultList();
}
Unfortunately JPA does not support sub queries in FROM clause. You need to write a native query or use framework like FluentJPA.
When attempting to generate dynamic queries using CriteriaBuilder, Hibernate is not creating the proper SQL with regards to an Entities member variable associated with #ElementCollection.
Sample Entity:
#Entity
#Table(name = "myclass")
public class MyClass {
...
#ElementCollection
#CollectionTable(
name = "myclass_mysubclass",
joinColumns = #JoinColumn(name = "myclass_id")
)
#Column(name = "mysubclass_id")
private List<Integer> mySubClassIDs;
...
}
CriteriaBuilder code:
CriteriaBuilder criteriaBuilder = getEntityManager().getCriteriaBuilder();
CriteriaQuery criteriaQuery = criteriaBuilder.createQuery(MyClass.class);
Root<T> root = criteriaQuery.from(MyClass.class);
Expression<Object> e = root.get("mySubClassIDs");
List<Object> o = (List<Object>) entry.getValue();
criteriaQuery.where(e.in(o));
where entry.getValue() will return an ArrayList<Integer> of [1]
Produces:
SELECT distinct count(myclass0_.id) as col_0_0_
FROM hotel myclass0_
cross join myclass_mysubclass mySubClassids1_
where myclass0_.id=mySubClassids1_.myclass_id and (. in (1))
Why is Hibernate not generating the "in" clause properly? the "." should be mySubClassids1_.mysubclass_id
Am I missing something in the annotation of the member variable? Doesn't seem so, as it is enough to generate the cross join.
The env is Jboss AS 7 with Hibernate 4.2.7.SP1-redhat-3 on jdk-6
Your schema is creating two separate tables:
create table myclass (
id int8 not null,
primary key (id)
);
create table myclass_mysubclass (
myclass_id int8 not null,
mysubclass_id int4
);
So, it seems you need to do a join instead of a get:
Expression<Object> e = root.join("mySubClassIDs");
Worked for me at any rate.
I have problem with hibernate query performance which I can't figure out. In code snippet below I need select entities with at least one mapping and filtered mapping. I'm using FETCH JOIN for this to load only filtered mappings.
But in that case I have performance problems with query. Hibernate says warning log :
org.hibernate.hql.ast.QueryTranslatorImpl
- firstResult/maxResults specified with collection fetch; applying in
memory!
When I omit FETCH JOIN and left only JOIN query is nice fast. But in result I have all mappings loaded to entity which is not acceptable state for me. Is there a way to boost query performance? There are a lot rows in mapping table.
HQL query :
select distinct e from Entity
join fetch e.mappings as mapping
where e.deleted = 0 and e.mappings is not empty
and e = mapping.e and mapping.approval in (:approvals)
Entities :
#Entity
#Table(name="entity")
class Entity {
...
#OneToMany(mappedBy="entity", cascade=CascadeType.REMOVE, fetch=FetchType.LAZY)
#OrderBy("created")
private List<Mapping> mappings = new ArrayList<Mapping>();
...
}
#Entity
#Table(name="mapping")
class Mapping {
public static enum MappingApproval {
WAITING, // mapping is waiting for approval
APPROVED, // mapping was approved
DECLINED; // mapping was declined
}
...
#ManyToOne(fetch=FetchType.EAGER)
#JoinColumn(name="entity_id", nullable=false)
private Entity entity;
#Enumerated(EnumType.STRING)
#Column(name="approval", length=20)
private MappingApproval approval;
...
}
Thanks
From the JPA-Specifications
The effect of applying setMaxResults or setFirstResult to a query
involving fetch joins over collections is undefined. (JPA "Enterprise
JavaBeans 3.0, Final Release", Kapitel 3.6.1 Query Interface)
Hibernate does the right thing, but executes a part of the query in memory, which is tremendously slower. In my case the difference is between 3-5 ms to 400-500 ms.
My solution was to implement the paging within the query itself. Works fast with the JOIN FETCH.
If you need a firstResult/maxResults with "fetch" you can split your query in 2 queries:
Query your entity ids with firstResult/maxResults but without the "fetch" on sub-tables:
select entity.id from entity (without fetch) where .... (with firstResult/maxResults)
Query your entities with the "fetch" on the ids returned by your first query:
select entity from entity fetch ... where id in <previous ids>
The reason is slow is because Hibernate executes the SQL query with no pagination at all and the restriction is done in memory.
However, if the join has to scan and fetch 100k records, while you are interested in just 100 results, then 99.9% of the work being done by the Extractor and all the I/O done over networking is just waste.
You can easily turn a JPQL query that uses both JOIN FETCH and pagination:
List<Post> posts = entityManager.createQuery("""
select p
from Post p
left join fetch p.comments
where p.title like :title
order by p.id
""", Post.class)
.setParameter("title", titlePattern)
.setMaxResults(maxResults)
.getResultList();
into an SQL query that limits the result using DENSE_RANK by the parent identifier:
#NamedNativeQuery(
name = "PostWithCommentByRank",
query =
"SELECT * " +
"FROM ( " +
" SELECT *, dense_rank() OVER (ORDER BY \"p.created_on\", \"p.id\") rank " +
" FROM ( " +
" SELECT p.id AS \"p.id\", " +
" p.created_on AS \"p.created_on\", " +
" p.title AS \"p.title\", " +
" pc.id as \"pc.id\", " +
" pc.created_on AS \"pc.created_on\", " +
" pc.review AS \"pc.review\", " +
" pc.post_id AS \"pc.post_id\" " +
" FROM post p " +
" LEFT JOIN post_comment pc ON p.id = pc.post_id " +
" WHERE p.title LIKE :titlePattern " +
" ORDER BY p.created_on " +
" ) p_pc " +
") p_pc_r " +
"WHERE p_pc_r.rank <= :rank ",
resultSetMapping = "PostWithCommentByRankMapping"
)
#SqlResultSetMapping(
name = "PostWithCommentByRankMapping",
entities = {
#EntityResult(
entityClass = Post.class,
fields = {
#FieldResult(name = "id", column = "p.id"),
#FieldResult(name = "createdOn", column = "p.created_on"),
#FieldResult(name = "title", column = "p.title"),
}
),
#EntityResult(
entityClass = PostComment.class,
fields = {
#FieldResult(name = "id", column = "pc.id"),
#FieldResult(name = "createdOn", column = "pc.created_on"),
#FieldResult(name = "review", column = "pc.review"),
#FieldResult(name = "post", column = "pc.post_id"),
}
)
}
)
The query can be executed like this:
List<Post> posts = entityManager
.createNamedQuery("PostWithCommentByRank")
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setParameter(
"rank",
5
)
.unwrap(NativeQuery.class)
.setResultTransformer(
new DistinctPostResultTransformer(entityManager)
)
.getResultList();
To transform the tabular result set back into an entity graph, you need a ResultTransformer which looks as follows:
public class DistinctPostResultTransformer
extends BasicTransformerAdapter {
private final EntityManager entityManager;
public DistinctPostResultTransformer(
EntityManager entityManager) {
this.entityManager = entityManager;
}
#Override
public List transformList(
List list) {
Map<Serializable, Identifiable> identifiableMap =
new LinkedHashMap<>(list.size());
for (Object entityArray : list) {
if (Object[].class.isAssignableFrom(entityArray.getClass())) {
Post post = null;
PostComment comment = null;
Object[] tuples = (Object[]) entityArray;
for (Object tuple : tuples) {
if(tuple instanceof Identifiable) {
entityManager.detach(tuple);
if (tuple instanceof Post) {
post = (Post) tuple;
}
else if (tuple instanceof PostComment) {
comment = (PostComment) tuple;
}
else {
throw new UnsupportedOperationException(
"Tuple " + tuple.getClass() + " is not supported!"
);
}
}
}
if (post != null) {
if (!identifiableMap.containsKey(post.getId())) {
identifiableMap.put(post.getId(), post);
post.setComments(new ArrayList<>());
}
if (comment != null) {
post.addComment(comment);
}
}
}
}
return new ArrayList<>(identifiableMap.values());
}
}
That's it!
after increasing memory for JVM things goes much better. After all I end with not using FETCH in queries.
How to express the exists clause with JPA?
First (we are on Oracle and have a DUAL Table):
#Entity()
#Table(name = "DUAL")
#ReadOnly
public class Dual {
#Id
String dummy;
public String getDummy() {
return dummy;
}
}
Then:
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Dual> cq = cb.createQuery(Dual.class);
Subquery<ProtokollSatz> sq = cq.subquery(ProtokollSatz.class);
Root<ProtokollSatz> root1 = sq.from(ProtokollSatz.class);
sq.where(
cb.and(
cb.equal(root1.<Integer> get("field1"), Integer.valueOf(field1)),
cb.equal(root1.<Integer> get("field2"), Integer.valueOf(field2))));
cq.where(cb.exists(sq));
TypedQuery<Dual> query = em.createQuery(cq);
boolean ifExists = query.getResultList().size() > 0;
You get:
SELECT t0.DUMMY FROM DUAL t0 WHERE EXISTS
(SELECT ? FROM PROTOKOLL_SAETZE t1 WHERE ((t1.FIELD1 = ?) AND (t1.FIELD2 = ?)))
Tested with eclipselink.
Exists is perfectly legal in JPQL, just use it. Perhaps I don't understand the question though? It's a bit terse :)
SELECT user
FROM SOUsers user
WHERE EXISTS (SELECT user0
FROM SOUsers user0
WHERE user0 = user.bestFriendWhoAnswersTheirQuestions
and user0.name = 'Roman')