I have this domain model:
final class Project {
private final #Id
#With
long projectId;
private final String projectType;
#With
private final ProjectRecipient projectRecipient;
#With
private final Set<PreCalculationCosts> preCalculationCosts;
#With
private final Set<PostCalculationCosts> postCalculationCosts;
}
The problem:
When i call the findById method from the CrudRepository the property projectRecipient gets materialized.
I even see all the issued sql statements being necessary for that in the logs.
When i use my own Query only the One to Many properties are getting materialized (there is no select statement issued for the one to one related projectrecipient):
select p.* from project p
inner join projectrecipient pr on pr.project = p.projectid
where p.projectid = :projectId
EDIT
When i debug the findById method and use this generated SQL as Query value, it gets materialized the right way. Problem with that is, that my Project table has tons of columns, so the Query value string is kind of 5 lines in my IDE(A) ...
On the other hand, i cant use the findById method because i need some postgres specific similar to clause ...
Currently there is no alternative to spelling out the full SQL statement including the columns for the referenced entity prefixed with the relationship name and an _.
If you don't include these columns the referenced entity will be considered to be null
The Spring Data team is thinking about enabling a way to provide everything but the select clause of a query, but it will take quite some time until these thoughts turn into code since the need some other substantial other changes.
Note that you can extract the query or parts thereof into static final values, which might make it easier to digest.
Also note that you can combine * notation with explicit columns (at least in the databases I worked with, so something like this would be fine:
select p.*, pr.id as recipient_id
Of course, the question if this is a good idea might cause some debate.
Related
Can jOOQ automatically add an alias prefix for all columns of a table in the select clause?
Can jOOQ also help with the 128 byte/character name length limitation of databases?
The reason for the questions are that SimpleFlatMapper is used for the mapping.
SimpleFlatMapper requires the fetched database column names to map to the model.
Model example (really getter/setter are used):
class Head {
public Integer id;
public List<Position> positions;
...
}
class Position {
public Integer id;
public Integer headId;
...
}
The naming can be done individually:
ResultSet resultSet = dsl.select(..., POSITION.ID.as("positions_id"), ...)
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
List<Head> headers = JdbcMapperFactory.newInstance()
.ignorePropertyNotFound()
.newMapper(Head.class)
.stream(resultSet)
.collect(Collectors.toList())
However if it is a complicated model with several joins/columns it is a bit tedious.
The only solution I found was to program a function manually. Have I however maybe overlooked something and jOOQ can do it by itself or help?
Something the likes of this would be very nice (probably not the best naming, I couldn't come up with something better on the spot):
ResultSet resultSet = dsl.select()
.from(HEAD)
.join(POSITION.as("positions").prefixColumns()).onKey()
.fetchResultSet();
Or:
ResultSet resultSet = dsl.selectWithTableAlias()
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
Resulting in the following SQL:
SELECT head.id, head.***, positions.id AS positions_id, positions.headid AS positions_headid, positions.***
FROM head JOIN position AS positions ON head.id = positions.headid
Furthermore databases like Oracle and MSSQL have a limitation of 128 bytes/characters for names.
In very rare complex scenarios the alias names might reach that limit because of the needed nesting.
jOOQ cannot offer a workaround for this in some form or can it? So basically define a name that is used in the SQL and a name for the actual resulting object.
I know, very niche requirements. But they would help a lot with the mapping.
Can jOOQ automatically add an alias prefix for all columns of a table in the select clause?
There's no such feature, because the possible sets of desired auto-prefix algorithms is quite big, and jOOQ ultimately wouldn't do exactly what you want. Maybe, there's room for an SPI to help you do this, in the future. But there isn't one available yet: https://github.com/jOOQ/jOOQ/issues/11545
However, you can easily do this yourself, because every jOOQ query is just a dynamically constructed expression tree, even if you're not always using jOOQ for dynamic SQL.
You can easily write a utility and use that everywhere:
public static List<Field<?>> autoPrefix(Field<?>... fields) {
return autoprefix(Arrays.asList(fields));
}
public static List<Field<?>> autoPrefix(Field<?>... fields) {
Stream.of(fields).map(f -> f.as(myPrefixLogic(f))).collect(toList());
}
This can now be used explicitly on all DSLContext.select() calls, e.g.
dsl.select(autoPrefix(..., POSITION.ID.as("positions_id"), ...))
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
Alternatively, you can wrap your query in a derived table:
public ResultSet fetchResultSetWithPrefix(Select<?> select) {
Table<?> table = select.asTable("t");
return
dsl.select(autoPrefix(table.fields()))
.from(table)
.fetchResultSet();
}
And now, instead of calling fetchResultSet(), you call your auxiliary function:
try (ResultSet rs = fetchResultSetWithPrefix(
dsl.select(..., POSITION.ID.as("positions_id"), ...)
.from(HEAD)
.join(POSITION.as("positions")).onKey()
)) {
...
}
Another, more complex option would be to do this with a VisitListener.
Remember, irrespective of what your query looks like (because jOOQ's DSL mimicks SQL, syntactically), all your jOOQ queries are dynamic SQL queries, so you can relatively easily transform them to whatever you want, automatically.
Can jOOQ also help with the 128 byte/character name length limitation of databases?
I'll have the same reservations about this automatism as before. Where would you see jOOQ provide automatic help here? Given your naming scheme (table_column) would you like a fair distribution of 63 characters per object type? Or is the table less important than the column, and you'll truncate table names at 42 characters, leaving 85 characters for the column?
What if you wanted to fully qualify identifiers, including the schema name, as in schema_table_column? What if you project UDTs, meaning you'll get schema_table_column_attribute1_attribute2
I don't think any automation can be provided by jOOQ which would suit all needs. However, as shown above, it is very simple to implement this only once for your entire application, making sure the naming is always applied correctly. And if an SPI were available, you could implement your abbreviation logic there.
I'm setting up a JPA Specification based repository implementation that utilizes jpa specifications(constructed based on RSQL filter strings) to filter the results, define result ordering and remove any duplicates via "distinct" that would otherwise be returned due to joined tables. The JPA Specification builder method joins several tables and sets the "distinct" flag:
final Join<Object, Object> rootJoinedTags = root.join("tags", JoinType.LEFT);
final Join<Object, Object> rootJoinedLocations = root.join("location", JoinType.LEFT);
...
query.distinct(true);
To allow sorting by joined table columns, I've applied the "HINT_PASS_DISTINCT_THROUGH" hint to the relevant repository method(otherwise, sorting by joined table columns returns an error along the lines of "sort column must be included in the SELECT DISTINCT query").
#QueryHints(value = {
#QueryHint(name = org.hibernate.jpa.QueryHints.HINT_PASS_DISTINCT_THROUGH, value = "false")
})
Page<SomeEntity> findAll(#Nullable Specification<SomeEntity> spec, Pageable pageable);
The arguments for said repository method are constructed as such:
final Sort sort = getSort(searchFilter);
final Specification spec = getSpecificationIfPresent(searchFilter);
final PageRequest pageRequest = PageRequest.of(searchFilter.getPageNumber(), searchFilter.getLimit(), sort);
return eventRepository.findAll(spec, pageRequest);
After those changes, filtering and sorting seem to work as expected. However, the hint seems to cause "distinct" filtering to be applied after the result page is already constructed, thus reducing the number of returned entities in the page from the configured "size" PageRequest argument, to whatever is left after the duplicates are filtered out. For example, if we'd make a PageRequest with "page=0" and "pageSize=10", then the resulting Page may return only 5 "SomeEntity" instances, although the database contains way more entries(177 entities to be exact in this case). If I remove the hint, then the returned entities number is correct again.
Question: is there a way to make the same Specification query setup work with correctly sized Pages(some other hints that might be added to have duplicate filtering performed before the Page object is constructed)? If not, then is there another approach I could use to achieve the required Specification-based filtering, with joined-column sorting and duplicate removal as with "distinct"?
PS: PostgreSQL is the database behind the application in question
The problem you are experimenting have to do with the way you are using the HINT_PASS_DISTINCT_THROUGH hint.
This hint allows you to indicate Hibernate that the DISTINCT keyword should not be used in the SELECT statement issued against the database.
You are taking advantage of this fact to allow your queries to be sorted by a field that is not included in the DISTINCT column list.
But that is not how this hint should be used.
This hint only must be used when you are sure that there will be no difference between applying or not a DISTINCT keyword to the SQL SELECT statement, because the SELECT statement already will fetch all the distinct values per se. The idea is improve the performance of the query avoiding the use of an unnecessary DISTINCT statement.
This is usually what will happen when you use the query.distinct method in you criteria queries, and you are join fetching child relationships. This great article of #VladMihalcea explain how the hint works in detail.
On the other hand, when you use paging, it will set OFFSET and LIMIT - or something similar, depending on the underlying database - in the SQL SELECT statement issued against the database, limiting to a maximum number of results your query.
As stated, if you use the HINT_PASS_DISTINCT_THROUGH hint, the SELECT statement will not contain the DISTINCT keyword and, because of your joins, it could potentially give duplicate records of your main entity. This records will be processed by Hibernate to differentiate duplicates, because you are using query.distinct, and it will in fact remove duplicates if needed. I think this is the reason why you may get less records than requested in your Pageable.
If you remove the hint, as the DISTINCT keyword is passed in the SQL statement which is sent to the database, as far as you only project information of the main entity, it will fetch all the records indicated by LIMIT and this is why it will give you always the requested number of records.
You can try and fetch join your child entities (instead of only join with them). It will eliminate the problem of not being able to use the field you need to sort by in the columns of the DISTINCT keyword and, in addition, you will be able to apply, now legitimately, the hint.
But if you do so it will you another problem: if you use join fetch and pagination, to return the main entities and its collections, Hibernate will no longer apply pagination at database level - it will no include OFFSET or LIMIT keywords in the SQL statement, and it will try to paginate the results in memory. This is the famous Hibernate HHH000104 warning:
HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!
#VladMihalcea explain that in great detail in the last part of this article.
He also proposed one possible solution to your problem, Window Functions.
In you use case, instead of using Specifications, the idea is that you implement your own DAO. This DAO only need to have access to the EntityManager, which is not a great deal as you can inject your #PersistenceContext:
#PersistenceContext
protected EntityManager em;
Once you have this EntityManager, you can create native queries and use window functions to build, based on the provided Pageable information, the right SQL statement that will be issued against the database. This will give you a lot of more freedom about what fields use for sorting or whatever you need.
As the last cited article indicates, Window Functions is a feature supported by all mayor databases.
In the case of PostgreSQL, you can easily come across them in the official documentation.
Finally, one more option, suggested in fact by #nickshoe, and explained in great detail in the article he cited, is to perform the sorting and paging process in two phases: in the first phase, you need to create a query that will reference your child entities and in which you will apply paging and sorting. This query will allow you to identify the ids of the main entities that will be used, in the second phase of the process, to obtain the main entities themselves.
You can take advantage of the aforementioned custom DAO to accomplish this process.
It may be an off-topic answer, but it may help you.
You could try to tackle this problem (pagination of parent-child entities) by separating the query in two parts:
a query for retrieving the ids that match the given criteria
a query for retrieving the actual entities by the resulting ids of the previous query
I came across this solution in this blog post: https://vladmihalcea.com/fix-hibernate-hhh000104-entity-fetch-pagination-warning-message/
I am attempting to create a domain/entity class based on a complex query. The query unions a bunch of tables together and unfortunately I am not able to create a view on the database for this query. I have been trying to set up the entity object but I am unsure of how to ensure that the marshaling works properly (and ensure the entity acts as read-only object).
As an example of the query, I am doing something like this:
Select
U_T.a,
U_T.b,
U_T.c,
C_T.a
FROM
(select
A_T.a,
null as b,
A_T.c,
1 as ind
from A_T
UNION
select
B_T.a,
B_T.b,
null,
0 as ind
FROM B_T
) U_T
left outer join C_T on C_T.fk_a = U_T.a;
The other issues are that this union can result in instances where there is no unique key column. This is fine as this data is for viewing only, and never editing. However the #Entity annotation wants a column to be listed with the #ID annotation. Another issue is that I do not believe I can use the other entity classes as the goal is to reduce the number of DB transactions from this query (as the actual one can result in hundreds of recursive queries being performed).
If I need to give any more information please let me know.
I'm working with DataNucleus as part of a Google App Engine project and I'm having a bit of trouble with columns in persistence.
#PrimaryKey(column = "user_id")
#Column(name = "user_id")
#Persistent(name = "user_id", column = "user_id", valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key m_id;
#Column(name = "user_name")
#Persistent(name = "user_name", column = "user_name")
private String m_userName;
If you can't tell, I'm trying to name the column something different than the name of the variable because I have two naming conventions (one works better in Java, one works better in SQL). Anyway, I've tried all variations of those marker annotations but the DataNucleus enhancer refuses to honor any of them, so when I run a query like this:
Query q = pm.newQuery(User.class,
"user_name == _username");
I always get an error like this:
org.datanucleus.store.appengine.FatalNucleusUserException: Unexpected expression type while parsing query. Are you certain that a field named user_name exists on your object?
Of course, when a run a query like this:
Query q = pm.newQuery(User.class,
"m_userName == _username");
...everything just works great. So, there would be a field named user_name if any of those annotations were honored, but they're clearly not.
SO my question is: Is there any way to decouple the tokens I use in the query from the name of the field? I'm looking for the ability to change the names of the fields without having to edit the queries by hand.
NOTE: I would sooner just use my SQL naming conventions in the Java classes than write the hideous amounts of XML by hand, so this has to be done with the annotations.
No idea of the talk of SQL, you're using GAE/J hence BigTable and not an RDBMS so SQL just won't work. #Column likely does nothing since it is for ORM. Here you're using JDOQLas the query language, so you use field names ... since it is an Object-Oriented query language. This is NOT SQL. You detest "this" ? JDOQL uses Java syntax, hence "this" makes lots of sense.
If you really want to have a type-safe query extension that allows refactoring then QueryDSL provides JDOQL for use with DataNucleus.
PS The DataNucleus enhancer has nothing to do with column names. It simply adds on extra methods for detecting updates to fields, as per the JDO spec.
Not 100% sure I get what is your problem. If you use m_userName in your query does it get translated as user_name in the SQL query?
You express you query according to the java class names and variables and they get translated to work according to the the SQL schema table and column names. That's most of the time what people want.
BTW, m_id and m_userName is a terrible naming convention for Java code. I would strongly advise you to follow the usual convention.
I must be really stupid, but I'm at my wits' end with a JPA issue, using MyEclipse 7.5.
I am accessing a DB2 database (on an AS400) via JPA. I have reverse-engineered a simple table to provide a DAO with some precision "find" methods. So far so good.
If I run a SELECT statement over the table thus, I get 4 rows:
SELECT * FROM MyTable WHERE MyValue = '1234'
However, if I try to access these same 4 records via JPA, I get a list that's the right size (4), but which contains 4 objects which are all the same, all copies of the first object found:
List <MyTableObject> objects = dao.findByMyValue("1234");
It's almost as if the internal Query object that the DAO class creates can't iterate through the rows of data. I've tweaked the reveng.xml file myriad ways, and I've tinkered with the generated DAO, but I'm getting nowhere. Am I missing something really obvious here? I just want to get a list of objects in the same way that the conventional SELECT statement returns a resultset!
(This is MyEclipse 7.5, using Hibernate 3.2 and its associated JPA library).
UPDATE: here's the generated code that findByMyValue() passes over to (loggin / try-catch removed for clarity):
#SuppressWarnings("unchecked")
public List<PolicyStatFile> findByProperty(String propertyName, final Object value)
{
final String queryString = "select model from MyTableObject model where model." + propertyName + "= :propertyValue";
Query query = getEntityManager().createQuery(queryString);
query.setParameter("propertyValue", value);
return query.getResultList();
}
FINAL UPDATE
It was all about the model: see comments to this post. Essentially, the model generated from the reverse engineering file was invalid because I didn't have a truly unique key. Once I resolved this (spurred by comments here), all was well.
Method you've posted looks correct (although it seems rather pointless to generate this for all properties). Couple things to check:
Is MyValue property you've mentioned mapped directly on your entity (e.g. to the column on the same table; no associations are involved)?
Can you enable Hibernate SQL debug (set 'hibernate.show_sql' property to true in your configuration) and check what the generated query looks like?
Are 4 objects returned actually the same (e.g. are '==' to each other) or are they copies of each other (e.g. have the same property values)?
Can you post your mapping for the entity in question and generated SQL from #2 above?
I suspect you haven't overridden hashCode() and equals() in your JPA entity (e.g. MyTableObject). So Hibernate can't distinguish the returning rows. That a look here.