We Keep Coding

How to know the missing items from Spring Data JPA's findAllById method in an efficient way?

Consider this code snippet below:
List<String> usersList = Arrays.asList("john", "jack", "jill", "xxxx", "yyyy");
List<User> userEntities = userRepo.findAllById(usersList);
User class is a simple Entity object annotated with #Entity and has an #Id field which is of String datatype.
Assume that in db I have rows corresponding to "john", "jack" and "jill". Even though I passed 5 items in usersList(along with "xxxx" and "yyyy"), findAllById method would only return 3 items/entities corresponding to "john","jack",and "jill".
Now after the call to findAllById method, what's the best, easy and efficient(better than O(n^2) perhaps) way to find out the missing items which findAllById method did not return?(In this case, it would be "xxxx" and "yyyy").

Using Java Sets
You could use a set as the source of filtering:
Set<String> usersSet = new HashSet<>(Arrays.asList("john", "jack", "jill", "xxxx", "yyyy"));
And now you could create a predicate to filter those not present:
Set<String> foundIds = userRepo.findAllById(usersSet)
.stream()
.map(User::getId)
.collect(Collectors.toSet());
I assume the filter should be O(n) to go over the entire results.
Or you could change your repository to return a set of users ideally using a form of distinct clause:
Set<String> foundIds = userRepo.findDistinctById(usersSet)
.stream()
.map(User::getId)
.collect(Collectors.toSet());;
And then you can just apply a set operator:
usersSet.removeAll(foundIds);
And now usersSet contains the users not found in your result.
And a set has a O(1) complexity to find an item. So, I assume this should be O(sizeOf(userSet)) to remove them all.
Alternatively, you could iterate over the foundIds and gradually remove items from the userSet. Then you could short-circuit the loop algorithm in the event you realize that there are no more userSet items to remove (i.e. the set is empty).
Filtering Directly from Database
Now to avoid all this, you can probably define a native query and run it in your JPA repository to retrieve only users from your list which didn't exist in the database. The query would be somewhat as follows that I did in PostgreSQL:
WITH my_users AS(
SELECT 'john' AS id UNION SELECT 'jack' UNION SELECT 'jill'
)
SELECT id FROM my_users mu
WHERE NOT EXISTS(SELECT 1 FROM users u WHERE u.id = mu.id);
Spring Data: JDBC Example
Since the query is dynamic (i.e. the filtering set could be of different sizes every time), we need to build the query dynamically. And I don't believe JPA has a way to do this, but a native query might do the trick.
You could either pack a JdbcTemplate query directly into your repository or use JPA native queries manually.
#Repository
public class UserRepository {
private final JdbcTemplate jdbcTemplate;
public UserRepository(JdbcTemplate jdbcTemplate) {this.jdbcTemplate = jdbcTemplate;}
public Set<String> getUserIdNotFound(Set<String> userIds) {
StringBuilder sql = new StringBuilder();
for(String userId : userIds) {
if(sql.length() > 0) {
sql.append(" UNION ");
}
sql.append("SELECT ? AS id");
}
String query = String.format("WITH my_users AS (%sql)", sql) +
"SELECT id FROM my_users mu WHERE NOT EXISTS(SELECT 1 FROM users u WHERE u.id = mu.id)";
List<String> myUsers = jdbcTemplate.queryForList(query, userIds.toArray(), String.class);
return new HashSet<>(myUsers);
}
}
Then we just do:
Set<String> usersIds = Set.of("john", "jack", "jill", "xxxx", "yyyy");
Set<String> notFoundIds = userRepo.getUserIdNotFound(usersIds);
There is probably a way to do it with JPA native queries. Let me see if I can do one of those and put it in the answer later on.

You can write your own algorithm that finds missing users. For example:
List<String> missing = new ArrayList<>(usersList);
for (User user : userEntities){
String userId = user.getId();
missing.remove(userId);
}
In the result you will have a list of user-ids that are missing:
"xxxx" and "yyyy"

You can just add a method to your repo:
findByIdNotIn(Collection<String> ids) and Spring will make the query:
See here:
https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods
Note (from the docs):
In and NotIn also take any subclass of Collection as aparameter as well as arrays or varargs.

get all the values in where clause for empty stirng using hibernate

i am building a shopping cart using jsp and hibernate.
i am filtering the content by brand size and price using checkboxes
the checked checkboxes are returned to the class where hql query exists.
so i want i single hql query that can handle this.
as like if one of the parameter like size is empty (means user doesnt uses it to filter the content ) than an empty string is passed to the hql query which returns any value...
so is there anything possible that all values can be retrived in where clause for empty string or some other alternative except coding different methods for different parameter...

I typically use the Criteria api for things like this... if the user does not specify a size, do not add it to the criteria query.
Criteria criteria = session.createCriteria(MyClass.class);
if(size != null && !size.isEmpty()){
criteria.add(Restrictions.eq("size", size);
}
To have multiple restrictions via an OR statement, you use Disjunction. For an AND, you use Conjunction.
Criteria criteria = session.createCriteria(MyClass.class);
Disjunction sizeDisjunction = Restrictions.disjunction();
String[] sizes = { "small", "medium", "large" };
for(int i = 0; i < sizes.length; i++){
sizeDisjunction.add(Restrictions.eq("size", sizes[i]);
}
criteria.add(sizeDisjunction );

First, good practices say that instead of passing and empty String to the query, you should pass null instead. That said, this hql should help you:
from Product p
where p.brand = coalesce(:brand, p.brand)
and p.size = coalesce(:size, p.size)
and p.price = coalesce (:price, p.price)

JPA Criteria Query API and order by two columns

I'm stuck with a simple problem; struggling how to invoke order by on a joined entity. Essentially I am trying to achieve the following with JPA Criteria:
select distinct d from Department d
left join fetch d.children c
left join fetch c.appointments a
where d.parent is null
order by d.name, c.name
I have the following:
CriteriaBuilder cb = getEntityManager().getCriteriaBuilder();
CriteriaQuery<Department> c = cb.createQuery(Department.class);
Root<Department> root = c.from(Department.class);
Fetch<Department, Department> childrenFetch = root.fetch(
Department_.children, JoinType.LEFT);
childrenFetch.fetch(Department_.appointments, JoinType.LEFT);
c.orderBy(cb.asc(root.get(Department_.name)));
c.distinct(true);
c.select(root);
c.where(cb.isNull(root.get(Department_.parent)));
How to achieve order by d.name, c.name with Criteria API? I tried with Expression, Path but didn't work.
Any pointers will be greatly appreciated.

If you need to add couple of orders you can make something like (but for your query and different root objects)
CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
CriteriaQuery<Route> query = criteriaBuilder.createQuery(Route.class);
Root<Route> routeRoot = query.from(Route.class);
query.select(routeRoot);
List<Order> orderList = new ArrayList();
query.where(routeRoot.get("owner").in(user));
orderList.add(criteriaBuilder.desc(routeRoot.get("date")));
orderList.add(criteriaBuilder.desc(routeRoot.get("rating")));
query.orderBy(orderList);

I have the same problem with order by using Criteria API. I found this solution:
CriteriaQuery<Test> q = cb.createQuery(Test.class);
Root<Test> c = q.from(Test.class);
q.select(c);
q.orderBy(cb.asc(c.get("name")), cb.desc(c.get("prenom")));

I was having trouble doing the same, and I have found a solution on this page:
http://www.objectdb.com/api/java/jpa/criteria/CriteriaQuery/orderBy_Order_
//javax.persistence.criteria.CriteriaQuery
//CriteriaQuery<T> orderBy(Order... o)
Specify the ordering expressions that are used to order the query results. Replaces the previous ordering expressions, if any. If no ordering expressions are specified, the previous ordering, if any, is simply removed, and results will be returned in no particular order. The left-to-right sequence of the ordering expressions determines the precedence, whereby the leftmost has highest precedence.
Parameters:
o - zero or more ordering expressions
Returns:
the modified query
Since:
JPA 2.0

The solucion that work for me is the following
session=HibernateUtil.getSessionJavaConfigFactory_report().openSession();
CriteriaBuilder builder = session.getCriteriaBuilder();
CriteriaQuery<Object[]> criteriaQuery = builder.createQuery(Object[].class);
List<Order> orderList = new ArrayList();
orderList.add(builder.desc(ejeRoot.get("name")));
criteriaQuery.orderBy(orderList);
Note: ejeRoot is the class object

categoryRepository.findAll(predicates, new Sort(Direction.ASC, "sortOrder", "name"))
.forEach(categoryDtoList::add);

How to avoid hibernate default order by id?

I make this query:
String query = FROM Account acc WHERE acc.id = ? OR acc.id = ? or acc.id = ?...
I have array of ids:
long[] accountIds= {327913,327652,327910,330511,330643};
Then I make
getHibernateTemplate().find(query, accountIds);
I see that the list of accounts I get back from this query is:
327652,327910,327913,330511,330643, obviously , ordered by id.
Any chance I get it back in the order I wrote the ids?
Will appreciate all the help

You may want to use Criteria and its addOrder.
Something like this:
DetachedCriteria cr = DetachedCriteria.forClass(entityClass);
//Add expressions or restrictions to your citeria
//And add your ordering
cr.addOrder(Order.asc("yourID"));
List<T> ls = getHibernateTemplate().findByCriteria(cr);
return ls;

You can't do it on query level.
You can sort after loading from db, something like this
long[] accountIds= {327913,327652,327910,330511,330643};
List<Account> afterHql = getHibernateTemplate().find(query, accountIds);
List<Account> sortedList = new ArrayList<Acount>();
for (long id : accountIds)
{
for (Account account : afterHql)
{
if (account.getId() == id)
{
sortedList.add(account);
}
}
}

It is not possible to fetch results by providing any entity in OR Query or Restrictions.in(). As by deafult when you fire this kind of query it will search for the results id wise. So it will give you results id wise only. You can change the order by using Criteria either in asc or desc. And if you want to have results as per you enter id, then second answer is the only option.

You can only order by column values returned by the query, in a sequence defined by the data type . Wouldn't it be better to pre-order the IDs you supply, and order the query result by ID, so they come out in the same order?

How do I find a value in a column that just have unique values with EclipseLink?

You have the EntityManager.find(Class entityClass, Object primaryKey) method to find a specific row with a primary key.
But how do I find a value in a column that just have unique values and is not a primary key?

You can use appropriate JPQL with TypedQuery.
try {
TypedQuery<Bean> tq = em.createQuery("from Bean WHERE column=?", Bean.class);
Bean result = tq.setParameter(1, "uniqueKey").getSingleResult();
} catch(NoResultException noresult) {
// if there is no result
} catch(NonUniqueResultException notUnique) {
// if more than one result
}

For example, like this:
List<T> results = em.createQuery("SELECT t FROM TABLE t", T.class)
.getResultList();
With parameters:
List<T> results = em.createQuery("SELECT t FROM TABLE t where t.value = :value1")
.setParameter("value1", "some value").getResultList();
For single result replace getResultList() with getSingleResult():
T entity = em.createQuery("SELECT t FROM TABLE t where t.uniqueKey = :value1")
.setParameter("value1", "KEY1").getSingleResult();
One other way is to use Criteria API.

You can use a Query, either JPQL, Criteria, or SQL.
Not sure if your concern is in obtaining cache hits similar to find(). In EclipseLink 2.4 cache indexes were added to allow you to index non-primary key fields and obtain cache hits from JPQL or Criteria.
See,
http://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Caching/Indexes
Prior to 2.4 you could use in-memory queries to query the cache on non-id fields.

TL;DR
With in DSL level - JPA no practice mentioned in previous answers
How do I find a value in a column that just have unique values and is not a primary key?
There isn't specification for query with custom field with in root interface of javax.persistence.EntityManager, you need to have criteria base query.
CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
CriteriaQuery<R> criteriaQuery = criteriaBuilder.createQuery(EntityType.class)
Root<R> root = criteriaQuery.from(type);
criteriaBuilder.and(criteriaBuilder.equal(root.get(your_field), value));
You can also group your predicates together and pass them all together.
andPredicates.add(criteriaBuilder.and(root.get(field).in(child)));
criteriaBuilder.and(andPredicates.toArray(new Predicate[]{});
And calling result(rather single entity or a list of entities) with
entityManager.createQuery(suitable_criteria_query).getSingleResult();
entityManager.createQuery(suitable_criteria_query).getResultList();