We Keep Coding

Hint HINT_PASS_DISTINCT_THROUGH reduces the amount of Entities returned per page for a PageRequest down to below the configured page size (PostgreSQL)

I'm setting up a JPA Specification based repository implementation that utilizes jpa specifications(constructed based on RSQL filter strings) to filter the results, define result ordering and remove any duplicates via "distinct" that would otherwise be returned due to joined tables. The JPA Specification builder method joins several tables and sets the "distinct" flag:
final Join<Object, Object> rootJoinedTags = root.join("tags", JoinType.LEFT);
final Join<Object, Object> rootJoinedLocations = root.join("location", JoinType.LEFT);
...
query.distinct(true);
To allow sorting by joined table columns, I've applied the "HINT_PASS_DISTINCT_THROUGH" hint to the relevant repository method(otherwise, sorting by joined table columns returns an error along the lines of "sort column must be included in the SELECT DISTINCT query").
#QueryHints(value = {
#QueryHint(name = org.hibernate.jpa.QueryHints.HINT_PASS_DISTINCT_THROUGH, value = "false")
})
Page<SomeEntity> findAll(#Nullable Specification<SomeEntity> spec, Pageable pageable);
The arguments for said repository method are constructed as such:
final Sort sort = getSort(searchFilter);
final Specification spec = getSpecificationIfPresent(searchFilter);
final PageRequest pageRequest = PageRequest.of(searchFilter.getPageNumber(), searchFilter.getLimit(), sort);
return eventRepository.findAll(spec, pageRequest);
After those changes, filtering and sorting seem to work as expected. However, the hint seems to cause "distinct" filtering to be applied after the result page is already constructed, thus reducing the number of returned entities in the page from the configured "size" PageRequest argument, to whatever is left after the duplicates are filtered out. For example, if we'd make a PageRequest with "page=0" and "pageSize=10", then the resulting Page may return only 5 "SomeEntity" instances, although the database contains way more entries(177 entities to be exact in this case). If I remove the hint, then the returned entities number is correct again.
Question: is there a way to make the same Specification query setup work with correctly sized Pages(some other hints that might be added to have duplicate filtering performed before the Page object is constructed)? If not, then is there another approach I could use to achieve the required Specification-based filtering, with joined-column sorting and duplicate removal as with "distinct"?
PS: PostgreSQL is the database behind the application in question

The problem you are experimenting have to do with the way you are using the HINT_PASS_DISTINCT_THROUGH hint.
This hint allows you to indicate Hibernate that the DISTINCT keyword should not be used in the SELECT statement issued against the database.
You are taking advantage of this fact to allow your queries to be sorted by a field that is not included in the DISTINCT column list.
But that is not how this hint should be used.
This hint only must be used when you are sure that there will be no difference between applying or not a DISTINCT keyword to the SQL SELECT statement, because the SELECT statement already will fetch all the distinct values per se. The idea is improve the performance of the query avoiding the use of an unnecessary DISTINCT statement.
This is usually what will happen when you use the query.distinct method in you criteria queries, and you are join fetching child relationships. This great article of #VladMihalcea explain how the hint works in detail.
On the other hand, when you use paging, it will set OFFSET and LIMIT - or something similar, depending on the underlying database - in the SQL SELECT statement issued against the database, limiting to a maximum number of results your query.
As stated, if you use the HINT_PASS_DISTINCT_THROUGH hint, the SELECT statement will not contain the DISTINCT keyword and, because of your joins, it could potentially give duplicate records of your main entity. This records will be processed by Hibernate to differentiate duplicates, because you are using query.distinct, and it will in fact remove duplicates if needed. I think this is the reason why you may get less records than requested in your Pageable.
If you remove the hint, as the DISTINCT keyword is passed in the SQL statement which is sent to the database, as far as you only project information of the main entity, it will fetch all the records indicated by LIMIT and this is why it will give you always the requested number of records.
You can try and fetch join your child entities (instead of only join with them). It will eliminate the problem of not being able to use the field you need to sort by in the columns of the DISTINCT keyword and, in addition, you will be able to apply, now legitimately, the hint.
But if you do so it will you another problem: if you use join fetch and pagination, to return the main entities and its collections, Hibernate will no longer apply pagination at database level - it will no include OFFSET or LIMIT keywords in the SQL statement, and it will try to paginate the results in memory. This is the famous Hibernate HHH000104 warning:
HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!
#VladMihalcea explain that in great detail in the last part of this article.
He also proposed one possible solution to your problem, Window Functions.
In you use case, instead of using Specifications, the idea is that you implement your own DAO. This DAO only need to have access to the EntityManager, which is not a great deal as you can inject your #PersistenceContext:
#PersistenceContext
protected EntityManager em;
Once you have this EntityManager, you can create native queries and use window functions to build, based on the provided Pageable information, the right SQL statement that will be issued against the database. This will give you a lot of more freedom about what fields use for sorting or whatever you need.
As the last cited article indicates, Window Functions is a feature supported by all mayor databases.
In the case of PostgreSQL, you can easily come across them in the official documentation.
Finally, one more option, suggested in fact by #nickshoe, and explained in great detail in the article he cited, is to perform the sorting and paging process in two phases: in the first phase, you need to create a query that will reference your child entities and in which you will apply paging and sorting. This query will allow you to identify the ids of the main entities that will be used, in the second phase of the process, to obtain the main entities themselves.
You can take advantage of the aforementioned custom DAO to accomplish this process.

It may be an off-topic answer, but it may help you.
You could try to tackle this problem (pagination of parent-child entities) by separating the query in two parts:
a query for retrieving the ids that match the given criteria
a query for retrieving the actual entities by the resulting ids of the previous query
I came across this solution in this blog post: https://vladmihalcea.com/fix-hibernate-hhh000104-entity-fetch-pagination-warning-message/

Possible to make a query-able counter using Postgres sequence and Hibernate mapping?

I'm getting the "No data type for node" error when I run this query:
session.createQuery("select nextval( 'next_num_seq' )")
which I know means that I need to make it a property of a class, but I haven't been able to find a way to add a sequence to a class, only how to make a sequence generate IDs for a class.
Is there a way to include a sequence in a Hibernate mapping that isn't an ID generator?

As such, this question is valid, yet the path to the solution is headed in the wrong direction. Mapping a Sequence into a managed domain entity is not a "good" idea, as these are two separate concepts.
Sequences, such as the one you are trying to query from a PostgreSQL backend, are a central concept for the generation of unique IDs for primary key values of tuples or - from an ORM application perspective - Java objects. Thus, it is not clever to map their current state to a domain entity. Instead, one sets a single, particular value drawn from such a sequence - e.g. next_num_seq - into one particular object to be persisted in a relational database. Therefore, the related class of such an domain object is linked to this sequence by, for instance, dedicated ORM annotations (or via similar approaches).
In the JavaDoc of the Session interface we find the method createNativeQuery(String sql) which is inherited from the EntityManager interface, see also here.
It is described as follows:
Query createNativeQuery(java.lang.String sqlString)
Create an instance of Query for executing a native SQL statement, e.g., for update or delete.
Parameters:
sqlString - a native SQL query string
Returns:
the new query instance
Thus, you could modify your code to execute the native query against your PostgreSQL database as follows:
Query q = session.createNativeQuery("select nextval( 'next_num_seq' )");
This gives you the option to read the next valid sequence value as a long or Number instance for your programming purposes.
Note well: Be careful not to reuse this value multiple times (for several objects), as this might cause consistency trouble in your backend when used, for instance, in the context of separate threads.
Hope this helps.

how to search similar entities in database using Example class from hibernate

i know that there are an Hibernate class called Example that we can use to get similar entities in order to do a search, but is it possible that this class permit to get entities searching in a generic way.
I explain, I build an example entity having a property called name with value = "myname", is Hibernate capable to return an entity which has property having value = "mname" ?

Yes that's possible but to enable text-level similarity you need a Lucene index to speed-up the query, as it would otherwise be extremely inefficient to run on a relational database.
This is provided by Hibernate Search, the extension of Hibernate to integrate with Lucene and manage the indexes transparently.

About the use of #ForceDiscriminator/#DiscriminatorOptions(force=true)

Why is #ForceDiscriminator or its equivalent #DiscriminatorOptions(force=true) necessary in some cases of inheritance and polymorphic associations? It seems to be the only way to get the job done. Are there any reasons not to use it?

As I'm running over this again and again, I think it might help to clarify:
First, it is true that Hibernate does not require discrimination when using JOINED_TABLE mapping. However, it does require it when using SINGLE_TABLE. Even more importantly, other JPA providers mostly do require it.
What Hibernate actually does when performing a polymorphic JOINED_TABLE query is to create a discriminator named clazz on the fly, using a case-switch that checks for the presence of fields unique for concrete subclasses after outer-joining all tables involved in the inheritance-tree. You can clearly see this when including the "hibernate.show_sql" property in your persistence.xml. In my view this is probably the perfect solution for JOINED_TABLE queries, so the Hibernate folks are right to brag about it.
The matter is somewhat different when performing updates and deletes; here hibernate first queries your root-table for any keys that match the statement's where clause, and creates a virtual pkTable from the result. Then it performs a "DELETE FROM / UPDATE table WHERE pk IN pkTable" for any concrete class withing your inheritance tree; the IN operator causes an O(log(N)) subquery per table entry scanned, but it is likely in-memory, so it's not too bad from a performance perspective.
To answer your specific question, Hibernate simply doesn't see a problem here, and from a certain perspective they are correct. It would be incredibly easy for them to simply honour the #DiscriminatorValue annotations by injecting the discriminator values during entityManager.persist(), even if they do not actually use them. However, not honoring the discriminator column in JOINED_TABLE has the advantage (for Hibernate) to create a mild case of vendor lockin, and it is even defensible by pointing to superior technology.
#ForceDiscriminator or #DiscriminatorOptions(force=true) sure help to mitigate the pain a little, but you have to use them before the first entities are created, or be forced to manually add the missing discriminator values using SQL statements. If you dare to move away from Hibernate it at least costs you some code change to remove these Hibernate specific annotations, creating resistance against the migration. And that is obviously all that Hibernate cares about in this case.
In my experience, vendor lockin is the paradise every market leader's wildest dreams are about, because it is the machiavellian magic wand that protects market share without effort; it is therefore done whenever customers do not fight back and force a price upon the vendor that is higher than the benefits reaped. Who said that an Open Source world would be any different?
p.s, just to avoid any confusion: I am in no way affiliated to any JPA implementor.
p.p.s: What I usually do is ignore the problem until migration time; you can then formulate an SQL UPDATE ... FROM statement using the same case-switch-with-outer-joins trick Hibernate uses to fill in the missing discriminator values. It's actually quite easy once you have understood the basic principle.

Guys let me try to explain about #DiscriminatorOptions(Force=true).
Well , it is used in single table inheritence, i have recently used this in one of the scenario.
i have two entities which was mapped to single table. when i was trying to fetch the record for one entity i was getting list of result containg records from both the entities and this was my problem. To solve this problem i have used #DiscriminatorOptions(Force=true) which will create the predicate using Discriminator column with the specified value mapped to the corresponding entity.
so the query will be look like this after i used #DiscriminatorOptions(Force=true)
select *
from TABLE
where YOUR PREDICATE AND DiscriminatorColumn = DiscriminatorValue

I think this is more of my opinion but I think some will agree with me. I prefer the fact that Hibernate enables you to not use a discriminator. In several cases the discriminator isn't necessary.
For example, I have a Person entity which contains stuff like a name, a date of birth, etc. This entity can be used by several other entities like Employee or Customer. When I don't reference Person from other entities, but reference Employee or Customer instead, the discriminator isn't used as Hibernate is instructed to fetch either one.

#yannisf ForceDiscriminator is not the only solution to solve this issue.
You can do instanceof tests for each child class. Though this will be like hardcoding your classes in your code but is a cleaner way to solve the problem if the discriminator column is not populated.
This also helps your code avoid mixing jpa and hibernate annotations.
As pointed out by yannisf, instanceOf is kind of an antipattern in the OO world.
Another solution could be changing your entity mapping. Suppose an entity A has a refernce to a superclass B and B has child classes of type C1 and C2, the instead of A pointing to B, you can have C1 and C2 have a foreign key pointing to A. It all comes down to changing the entity design so as not to mix annotations.
Thanks
Vaibhav

Does Hibernate's Criteria API still not support nested relations

I'd like to use Hibernate's Criteria API for precisely what everybody says is probably its most likely use case, applying complex search criteria. Problem is, the table that I want to query against is not composed entirely of primitive values, but partially from other objects, and I need to query against those object's id's.
I found this article from 2 years ago that suggests it's not possible. Here's how I tried it to no avail, there are other aspect of Hibernate where I know of where this sort of dot notation is supported within string literals to indicate object nesting.
if (!lookupBean.getCompanyInput().equals("")) {
criteria.add(Restrictions.like("company.company", lookupBean.getCompanyInput() + "%"));
}
EDIT:
Here's my correctly factored code for accomplishing what I was trying above, using the suggestion from the first answer below; note that I am even using an additional createCriteria call to order on an attribute in yet another associated object/table:
if (!lookupBean.getCompanyValue().equals("")) {
criteria.createCriteria("company").add(
Restrictions.like("company", lookupBean.getCompanyValue() + "%"));
}
List<TrailerDetail> tdList =
criteria.createCriteria("location").addOrder(Order.asc("location")).list();

Not entirely sure I follow your example, but it's certainly possible to specify filter conditions on an associated entity, simply by nesting Criteria objects to form a tree. For example, if I have an entity called Order with a many-to-one relationship to a User entity, I can find all orders for a user named Fred with a query like this:
List<Order> orders = session.createCriteria(Order.class)
.createCriteria("user")
.add(eq("name", "fred"))
.list();
If you're talking about an entity that has a relationship to itself, that should work as well. You can also replace "name" with "id" if you need to filter on the ID of an associated object.