Our in-house framework built with Java 11, Spring Boot, Hibernate 5 and QueryDSL does a lot of auto-generation of queries. I try to keep everything efficient and load associations only when needed.
When loading full entities, the programmer can declare a NamedEntityGraph to be used. Now there is one case where a query like this is generated:
select user.groups
from User user
where user.id = ?1
Where the Entities in question look like this:
#Entity
#NamedEntityGraph(name = User.ENTITY_GRAPH,
attributeNodes = {
#NamedAttributeNode(User.Fields.permissions),
#NamedAttributeNode(value = User.Fields.groups, subgraph = "user-groups-subgraph")
},
subgraphs = #NamedSubgraph(
name = "user-groups-subgraph",
attributeNodes = {
#NamedAttributeNode(Group.Fields.permissions)
}
))
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Enumerated(EnumType.STRING)
#ElementCollection(targetClass = Permission.class)
#CollectionTable(name = "USERS_PERMISSIONS", joinColumns = #JoinColumn(name = "uid"))
private Set<Permission> permissions = EnumSet.of(Permission.ROLE_USER);
#ManyToMany(fetch = LAZY)
private Set<Group> groups = new HashSet<>();
}
#Entity
#NamedEntityGraph(name = Group.ENTITY_GRAPH,
attributeNodes = {
#NamedAttributeNode(value = Group.Fields.permissions)
})
public class Group {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE)
private Long id;
#Enumerated(EnumType.STRING)
#ElementCollection(targetClass = Permission.class)
#CollectionTable(
name = "GROUPS_PERMISSIONS",
joinColumns = #JoinColumn(name = "gid")
)
#NonNull
private Set<Permission> permissions = EnumSet.noneOf(Permission.class);
}
When selecting either User or Group directly, the generated query simply applies the provided NamedEntityGraphs. But for the above query the exception is:
org.hibernate.QueryException:
query specified join fetching, but the owner of the fetched association was not present in the select list
[FromElement{explicit,collection join,fetch join,fetch non-lazy properties,classAlias=user,role=foo.bar.User.permissions,tableName={none},tableAlias=permission3_,origin=null,columns={,className=null}}]
I first tried the User graph, but since we are fetching Groups, I tried the Group graph. Same Exception.
Problem is, there is no easy way to add a FETCH JOIN to the generated query, since I don't know which properties of the association should be joined in anyway. I would have to load the Entitygraph, walk it and any subgraph and generated the right join clauses.
Some more details on Query generation:
// QueryDsl 4.3.x Expressions, where propType=Group.class, entityPath=User, assocProperty=groups
final Path<?> expression = Expressions.path(propType, entityPath, assocProperty);
// user.id = ?1
final BooleanExpression predicate = Expressions.predicate(Ops.EQ, idPath, Expressions.constant(rootId));
// QuerydslJpaPredicateExecutor#createQuery from Spring Data JPA
final JPQLQuery<P> query = createQuery(predicate).select(expression).from(path);
// Add Fetch Graph
((AbstractJPAQuery<?, ?>) query).setHint(GraphSemantic.FETCH.getJpaHintName(), entityManager.getEntityGraph(fetchGraph));
EDIT:
I can reproduce this with a simple JPQL Query. It's very strange, if I try to make a typed query, it will select a List of Sets of Group and untyped just a List of Group.
Maybe there is something conceptually wrong - I'm selecting a Collection and I'm trying to apply a fetch join on it. But JPQL doesn't allow a SELECT from a subquery, so I'm not sure what to change..
// em is EntityManager
List gs = em
.createQuery("SELECT u.groups FROM User u WHERE u.id = ?1")
.setParameter(1, user.getId())
.setHint(GraphSemantic.FETCH.getJpaHintName(), em.getEntityGraph(Group.ENTITY_GRAPH))
.getResultList();
Same Exception:
org.hibernate.QueryException: query specified join fetching, but the owner of the fetched association was not present in the select list
So the problem can be distilled down to a resolution problem of the Entit Graphs attributes:
select user.groups
from User user
where user.id = ?1
With the Entity Graph
EntityGraph<Group> eg = em.createEntityGraph(Group.class);
eg.addAttributeNodes(Group.Fields.permissions);
Gives an Exception that shows that Hibernate tries to fetch User.permissions instead of Group.permissions. This is the bug report.
And there is another bug regarding the use of #ElementCollection here.
Related
I have two entities in MySQL as below. The primary key of nnm_tran is a composite of id and source. The primary key of bargains is actually a foreign key link to the nnm_tran table
I'm trying to use JPA inheritance to represent these.
nnm_tran entity
#Entity
#Table(name = "nnm_tran")
#IdClass(CommonTransactionKey.class)
#Inheritance(strategy = InheritanceType.JOINED)
#DiscriminatorColumn(name = "bargain_flag", discriminatorType = DiscriminatorType.CHAR)
#DiscriminatorValue("N")
public class CommonTransaction {
#Id
#Column(name = "id", nullable = false)
private String transactionId;
#Column(name = "plan_number", nullable = false)
private String planNumber;
#Column(name = "tran_date")
private LocalDateTime transactionDatetime;
#Column(name = "bargain_flag")
private String bargainFlag;
...
}
bargains entity
#Data
#EqualsAndHashCode(callSuper = true)
#Entity
#Table(name = "bargains")
#DiscriminatorValue("B")
#PrimaryKeyJoinColumns({ #PrimaryKeyJoinColumn(name = "nnm_tran_id", referencedColumnName = "id"), #PrimaryKeyJoinColumn(name = "nnm_tran_source", referencedColumnName = "source") })
public class Bargain extends CommonTransaction implements Serializable {
#Column(name = "unit_price")
private BigDecimal unitPrice;
#Column(name = "client_price")
private BigDecimal clientPrice;
...
}
I think so far this is all hooked up correctly. My problem comes when I attach a spring-data repository with a custom query.
Repository
public interface CommonTransactionRepository extends CrudRepository<CommonTransaction, CommonTransactionKey> {
#Query("select t from CommonTransaction t left join IoPlan p ON t.planNumber = p.planNumber "
+ "where (p.planNumber is NULL or p.planNumber = '') "
+ "and t.transactionDatetime between ?1 and ?2 "
+ "and t.cancelled = false")
public Iterable<CommonTransaction> findOrphanedTransactionsByTranDate(LocalDateTime fromDate, LocalDateTime toDate);
...
}
When this gets proxied and the method is executed it generates the SQL statement
SELECT DISTINCT nnm_tran.bargain_flag FROM nnm_tran t1 LEFT OUTER JOIN io_plan t0 ON (t1.plan_number = t0.plan_number) WHERE ((((t0.plan_number IS NULL) OR (t0.plan_number = ?)) AND (t1.tran_date BETWEEN ? AND ?)) AND (t1.CANCELLED = ?))
The issue with this is that the nnm_tran table is aliased to t1 but the discriminator column is referencing the full table name nnm_tran.bargain_flag The result is a lovely
UnitOfWork(17171249)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'nnm_tran.bargain_flag' in 'field list'
Question here is, am I doing something wrong or is this a bug in spring-data and/or eclipselink?
Versions: spring-data 1.7.2, Eclipselink 2.5.2, MySQL 5.6.28
Using #manish's sample app as a starting point I started layering back on the complexity that was missing and quickly stumbled across the thing causing the rogue SQL. It was down to the join I had performed in the JPQL
NOTE: If you've come here from the future then ignore the remainder of this answer and instead use #Chris's comment instead.
Most of the time I don't need to look at or even think about the IoPlan table that can be seen in the #Query
#Query("select t from CommonTransaction t left join IoPlan p ON t.planNumber = p.planNumber "
+ "where (p.planNumber is NULL or p.planNumber = '') "
+ "and t.transactionDatetime between ?1 and ?2 "
+ "and t.cancelled = false")
and so this table is not a part of the CommonTransaction entity as a field. Even the result of this query doesn't really care because it's looking only as a one off for CommonTransaction with no associated join in the IoPlan table.
When I added the join back in to the sample app from #manish it all broke in the same way my app has in EclipseLink, but broke in a different way for Hibernate. Hibernate requires a field for you to join with, which if you ask me defeats the purpose of writing the join in the #Query. In fact in Hibernate you have to define the join purely in JPA so you might as well then use dot notation to access it in the JPQL.
Anyway, going along with this idea I tried adding a dummy field to hold an IoPlan in my CommonTransaction entity and it almost worked. It defaulted some of the join logic but it was closer
SELECT DISTINCT t1.bargain_flag FROM nnm_tran t1 LEFT OUTER JOIN io_plan t0 ON ((t0.ID = t1.IOPLAN_ID) AND (t1.plan_number = t0.plan_number)) WHERE ((((t0.plan_number IS NULL) OR (t0.plan_number = ?)) AND (t1.tran_date BETWEEN ? AND ?)) AND (t1.CANCELLED = ?))
In this case t1.IOPLAN_ID and t0.ID don't exist. So I ended up defining the entire join in my CommonTransaction entity
#OneToOne
#JoinColumn(insertable = false, updatable = false, name = "plan_number", referencedColumnName = "plan_number")
private IoPlan ioPlan;
and voila, it started working. It's not pretty and now I have a redundant join condition
LEFT OUTER JOIN io_plan t1
ON ((t1.plan_number = t0.plan_number) AND (t0.plan_number = t1.plan_number))
but I can fix that. It's still annoying that I have to define a field for it whatsoever, I don't actually want or need it there, not to mention that the result from this query is returning CommonTransaction entities that have no IoPlan so the field will be permanently null.
I have 3 classes, I am trying to get a list of all the events of an eventhost that a user is subscribed to. I am probably thinking way too complicated but I have very little experience with JPA/HQL.
User class
#ManyToMany
#JoinTable(name = "Subscriptions", joinColumns = #JoinColumn(name = "user_id", referencedColumnName = "id") , inverseJoinColumns = #JoinColumn(name = "event_host_id", referencedColumnName = "id") )
private List<EventHost> subscriptions;
EventHost class
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY)
#JoinColumn(name = "event_host_id", referencedColumnName = "id", updatable = true)
private List<Event> events;
I tried using this query, but it tells me that subscriptions is not mapped, which it is not since it's not a java class.
String hql = "SELECT o FROM Event WHERE event_host_id IN (SELECT a FROM EventHost WHERE id IN(SELECT b FROM User WHERE = + " + userid + "))";
I know injecting the userid like this is bad practice, I'm just doing it for testing purposes.
Please ask if you need something more, I would really like to understand how to write a query for this.
This question should really be HQL with two join tables, but I'll let you change it. Since its HQL, or JPA, it's database independent.
Anyway, any time you see a OneToMany or ManyToMany relationship you have a join table and so you should be thinking joins. It's always a good idea to look at the sql create table statements to see what's going on. In this case your user_subscriptions join table is:
create table user_subscriptions (user_id integer not null, subscriptions_id integer not null)
and your event_host_events join table is this:
create table event_host_events (event_host_id integer not null, events_id integer not null)
Nothing new there. When you're trying to get something new working that you don't intuitively understand, break it down into things you can do. For example, you can execute two queries, getting a Users subscriptions first, and then getting the Events for those subscriptions:
Query query = session.createQuery("select u.subscriptions from User u where name = :name");
query.setParameter("name", name);
List<EventHost> subscriptions = query.list();
List<Event> events = new ArrayList<Event>();
Query query2 = session.createQuery("select s.events from EventHost s where id = :id");
for (EventHost s: subscriptions ) {
query2.setParameter("id", s.getId());
events.addAll( query2.list());
}
Not elegant, but it works. Then, keeping join in mind, figure out how to make one statement out of the two of them.
Query query = session.createQuery("select s.events from User u join u.subscriptions s where u.name = :name)");
query.setParameter("name", name);
return query.list();
The join will use an inner join by default, so you're ok there. The JPA provider will auto-magically join your three Entity tables and two Join Tables for you:
select
event4_.id as id1_2_
from user user0_
inner join user_subscriptions subscripti1_ on user0_.id=subscripti1_.user_id
inner join event_host eventhost2_ on subscripti1_.subscriptions_id=eventhost2_.id
inner join event_host_events events3_ on eventhost2_.id=events3_.event_host_id
inner join event event4_ on events3_.events_id=event4_.id
where user0_.name=?
Aren't you glad you don't have to write that query?
I have built a list of taggable documents, with a many-to-many relationship between the tags and the documents. I would now like to use the hibernate criteria mechanism to query a "summary" of each tag, which includes a count of how often a particular tag has been used, with an additional restriction on whether or not the document has been published.
The entities I'm using roughly look like this (You'll note an SQL join table in the middle there):
#Entity
public class DocumentTag {
... various things ...
#ManyToMany(fetch = FetchType.LAZY, mappedBy = "tags")
private List<Document> documents = new ArrayList<>();
}
#Entity
public class Document {
... various things ...
#Basic
#Column(name = "published", columnDefinition = "BIT", length = 1)
protected boolean published = false;
#ManyToMany(fetch = FetchType.LAZY)
#JoinTable(name = "document_tag_joins",
uniqueConstraints = #UniqueConstraint(
columnNames = {"document", "tag"}
),
joinColumns = {#JoinColumn(name = "document")},
inverseJoinColumns = {#JoinColumn(name = "tag")})
private List<DocumentTag> tags = new ArrayList<>();
}
Given the above, I've managed to figure out that building the query should work more or less as follows:
Criteria c = session.createCriteria(DocumentTag.class);
c.createAlias("documents", "docs",
JoinType.LEFT_OUTER_JOIN,
Restrictions.eq("published", true)
);
c.setProjection(
Projections.projectionList()
.add(Projections.alias(Projections.groupProperty("id"), "id"))
.add(Projections.alias(Projections.property("createdDate"), "createdDate"))
.add(Projections.alias(Projections.property("modifiedDate"), "modifiedDate"))
.add(Projections.alias(Projections.property("name"), "name"))
.add(Projections.countDistinct("docs.id"), "documentCount"));
// Custom response entity mapping
c.setResultTransformer(
Transformers.aliasToBean(DocumentTagSummary.class)
);
List<DocumentTagSummary> results = c.list();
Given the above, the hibernate generated SQL query looks as follows:
SELECT
this_.id AS y0_,
this_.createdDate AS y1_,
this_.modifiedDate AS y2_,
this_.name AS y3_,
count(DISTINCT doc1_.id) AS y5_
FROM tags this_
LEFT OUTER JOIN tag_joins documents3_
ON this_.id = documents3_.tag AND (doc1_.published = ?)
LEFT OUTER JOIN documents doc1_
ON documents3_.document = doc1_.id AND (doc1_.published = ?)
GROUP BY this_.id
As you can see above, the publishing constraint is applied to both of the left outer joins. I'm not certain whether that is by design, however what I need is for the published constraint to be applied ONLY to the second left outer join.
Any ideas?
I was able to circumvent this problem by coming at it sideways. First, I had to change the "published" column to use an integer rather than a bit. Then I was able to slightly modify the projection of the result as follows:
// Start building the projections
ProjectionList projections =
Projections.projectionList()
.add(Projections.alias(
Projections.groupProperty("id"), "id"))
.add(Projections.alias(
Projections.property("createdDate"),
"createdDate"))
.add(Projections.alias(
Projections.property("modifiedDate"),
"modifiedDate"))
.add(Projections.alias(
Projections.property("name"), "name"));
if (isAdmin()) {
// Give the raw count.
projections.add(Projections.countDistinct("docs.id"), "documentCount");
} else {
// Use the sum of the "published" field.
projections.add(Projections.sum("docs.published"), "documentCount");
}
I acknowledge that this doesn't actually answer the question about why hibernate criteria constraints on many-to-many tables get applied to all tables, but it solved my problem.
I need a jpql query for my Spring repository interface method, to retrieve all Posts for a given Semester.
#LazyCollection(LazyCollectionOption.FALSE)
#OneToMany(cascade = CascadeType.MERGE)
#JoinTable
(
name = "semester_post",
joinColumns = {#JoinColumn(name = "semester_id", referencedColumnName = "id")},
inverseJoinColumns = {#JoinColumn(name = "post_id", referencedColumnName = "id", unique = true)}
)
private List<PostEntity<?>> posts = new ArrayList<>();
PostEntity doesn't have a reference to Semester, and I do not want to add one, because I plan to use this PostEntity for other things than Semester. Maybe I'll have another class (let's say Group) which will also have a OneToMany of PostEntity (like the one in Semester)
So, how do I write this SQL query as a JPQL one ?
select * from posts join semester_post on semester_post.post_id = posts.id where semester_post.semester_id = 1;
My repository
public interface PostRepository extends JpaRepository<PostEntity, Long> {
String QUERY = "SELECT p FROM PostEntity p ... where semester = :semesterId";
#Query(MY_QUERY)
public List<PostEntity> findBySemesterOrderByModifiedDateDesc(#Param("semesterId") Long semesterId);
A query which will get you the result that you need is:
SELECT p FROM SemesterEntity s JOIN s.posts p WHERE s.id = :semesterId
This query uses the JOIN operator to join the SemesterEntity to the PostEntity across the posts relationship. By joining the two entities together, this query returns all of the PostEntity instances associated with the relevant SemesterEntity.
I have two entities in a #ManyToMany relationship.
// Output has 4 other #ManyToOne relationships if that matters
#Entity #Table public class Output {
#Id public String address;
#ManyToMany(targetEntity = Interval.class,
cascade = CascadeType.ALL,
fetch = FetchType.LAZY)
#JoinTable(name = "output_has_interval",
joinColumns = {#JoinColumn(name = "output_address",
referencedColumnName = "address")},
inverseJoinColumns = {#JoinColumn(name = "interval_start",
referencedColumnName = "start"),
#JoinColumn(name = "interval_end",
referencedColumnName = "end")})
Collection<Interval> intervals;
#IdClass(IntervalPK.class) // I'll omit this one.
#Entity #Table public class Interval {
#Id public Calendar start;
#Id public Calendar start;
#ManyToMany(targetEntity = Output.class,
mappedBy = "intervals",
cascade = CascadeType.ALL,
fetch = FetchType.LAZY)
public Collection<Output> outputs;
The join table is called output_has_interval between output and interval.
How do I do CriteriaQuery like this?
SELECT `output`.`address`
FROM `output`, `output_has_interval`, `interval`
WHERE `output`.`address` = `output_has_interval`.`output_address`
AND `interval`.`start` = `output_has_interval`.`interval_start`
AND `interval`.`end` = `output_has_interval`.`interval_end`
AND `interval`.`start` >= '2011-04-30'
This works as expected if I issue it in MySQL.
(I have the corresponding static meta model classes as well, on request I'll could post them - nothing fancy tho'.)
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Output> cq = cb.createQuery(Output.class);
Root<Output> root= cq.from(Output.class);
CollectionJoin<Output, Interval> join = root.join(Output_.intervals);
Expression<Calendar> start = join.get(Interval_.start);
Predicate pred = cb.greaterThanOrEqualTo(start, /* calendar for '2011-04-30' */);
cq.where(pred);
TypedQuery<Output> tq = em.createQuery(cq);
However tq.getResultList returns every output row from my database. Any idea?
(On a side note: Hibernate (the provider I'm using) generates many select statements when I issue this query, one for every relationship Output has, sometimes more.)
Edit.: I wrote:
tq.getResultList returns every
output row from my database
To clarify it: it returns more than just every output row from my database. It actually does a join using output and interval however the predicate:
`interval`.`start` >= '2011-04-30'
doesn't get satisfied.
Ok, I'll managed to solve my riddle on my own.
First of all: the whole problem originated from the fact that I'm a lousy programmer. I iterated over TypedQuery<Output>.getResultList() and accessed every Interval in Output.intervals in a recursive manner, thus Hiberate loaded lazily the requested objects generating a handful of select statements.
However I had to get a hold of those Interval instaces somehow. The following change to my CriteriaQuery did the trick.
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Tuple> cq = cb.createTupleQuery(); // or createQuery(Tuple.class)
Root<Output> root= cq.from(Output.class); // from clause
CollectionJoin<Output, Interval> join = root.join(Output_.intervals);
Path<String> addressPath = root.get(Output_.address); // mind these Path objects
Path<Calendar> startPath = join.get(Interval_.start); // these are the key to success!
cq.multiselect(addressPath, startPath); // select clause
Expression<Calendar> start = join.get(Interval_.start);
Predicate pred = cb.greaterThanOrEqualTo(start, /* calendar for '2011-04-30' */);
cq.where(pred); // where clause
TypedQuery<Tuple> tq = em.createQuery(cq); // holds Tuples
for (Tuple tuple : tq.getResultsList()) {
String address = tuple.get(addressPath);
Calendar start = tuple.get(startPath);
...
Edit
I've just realized that I could've used Path<T> objects instead Expression<T> objects (or vice versa) as Path<T> extends Expression<T>. Oh well...