We Keep Coding

How to know the missing items from Spring Data JPA's findAllById method in an efficient way?

Consider this code snippet below:
List<String> usersList = Arrays.asList("john", "jack", "jill", "xxxx", "yyyy");
List<User> userEntities = userRepo.findAllById(usersList);
User class is a simple Entity object annotated with #Entity and has an #Id field which is of String datatype.
Assume that in db I have rows corresponding to "john", "jack" and "jill". Even though I passed 5 items in usersList(along with "xxxx" and "yyyy"), findAllById method would only return 3 items/entities corresponding to "john","jack",and "jill".
Now after the call to findAllById method, what's the best, easy and efficient(better than O(n^2) perhaps) way to find out the missing items which findAllById method did not return?(In this case, it would be "xxxx" and "yyyy").

Using Java Sets
You could use a set as the source of filtering:
Set<String> usersSet = new HashSet<>(Arrays.asList("john", "jack", "jill", "xxxx", "yyyy"));
And now you could create a predicate to filter those not present:
Set<String> foundIds = userRepo.findAllById(usersSet)
.stream()
.map(User::getId)
.collect(Collectors.toSet());
I assume the filter should be O(n) to go over the entire results.
Or you could change your repository to return a set of users ideally using a form of distinct clause:
Set<String> foundIds = userRepo.findDistinctById(usersSet)
.stream()
.map(User::getId)
.collect(Collectors.toSet());;
And then you can just apply a set operator:
usersSet.removeAll(foundIds);
And now usersSet contains the users not found in your result.
And a set has a O(1) complexity to find an item. So, I assume this should be O(sizeOf(userSet)) to remove them all.
Alternatively, you could iterate over the foundIds and gradually remove items from the userSet. Then you could short-circuit the loop algorithm in the event you realize that there are no more userSet items to remove (i.e. the set is empty).
Filtering Directly from Database
Now to avoid all this, you can probably define a native query and run it in your JPA repository to retrieve only users from your list which didn't exist in the database. The query would be somewhat as follows that I did in PostgreSQL:
WITH my_users AS(
SELECT 'john' AS id UNION SELECT 'jack' UNION SELECT 'jill'
)
SELECT id FROM my_users mu
WHERE NOT EXISTS(SELECT 1 FROM users u WHERE u.id = mu.id);
Spring Data: JDBC Example
Since the query is dynamic (i.e. the filtering set could be of different sizes every time), we need to build the query dynamically. And I don't believe JPA has a way to do this, but a native query might do the trick.
You could either pack a JdbcTemplate query directly into your repository or use JPA native queries manually.
#Repository
public class UserRepository {
private final JdbcTemplate jdbcTemplate;
public UserRepository(JdbcTemplate jdbcTemplate) {this.jdbcTemplate = jdbcTemplate;}
public Set<String> getUserIdNotFound(Set<String> userIds) {
StringBuilder sql = new StringBuilder();
for(String userId : userIds) {
if(sql.length() > 0) {
sql.append(" UNION ");
}
sql.append("SELECT ? AS id");
}
String query = String.format("WITH my_users AS (%sql)", sql) +
"SELECT id FROM my_users mu WHERE NOT EXISTS(SELECT 1 FROM users u WHERE u.id = mu.id)";
List<String> myUsers = jdbcTemplate.queryForList(query, userIds.toArray(), String.class);
return new HashSet<>(myUsers);
}
}
Then we just do:
Set<String> usersIds = Set.of("john", "jack", "jill", "xxxx", "yyyy");
Set<String> notFoundIds = userRepo.getUserIdNotFound(usersIds);
There is probably a way to do it with JPA native queries. Let me see if I can do one of those and put it in the answer later on.

You can write your own algorithm that finds missing users. For example:
List<String> missing = new ArrayList<>(usersList);
for (User user : userEntities){
String userId = user.getId();
missing.remove(userId);
}
In the result you will have a list of user-ids that are missing:
"xxxx" and "yyyy"

You can just add a method to your repo:
findByIdNotIn(Collection<String> ids) and Spring will make the query:
See here:
https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods
Note (from the docs):
In and NotIn also take any subclass of Collection as aparameter as well as arrays or varargs.

How to order result of hibernate based on a specific order

I need to send a query to retrieve values that has a specific group of characters as following:
Lets say I am interested in 'XX' so it should search for any field that its value starts with 'XX' or has ' XX' (space XX). For example XXCDEF, PD XXRF and CMKJIEK XX are valid results.
I have following query that
returns the correct results but I need to sort them
in a way that it first return those with XX at the beginning then other results. As following:
XXABCD
XXPLER
XXRFKF
AB XXAB
CD XXCD
ZZ XXOI
POLO XX
Code
Criteria criteria = session.createCriteria(Name.class, "name")
.add(Restrictions.disjunction()
.add(Restrictions.ilike("name.fname", fname + "%"))
.add(Restrictions.ilike("name.fname", "%" + " " + fname + "%"))
)
.setProjection(Projections.property("name.fname").as("fname"));
List<String> names = (List<String>) criteria.list();

With JPQL (HQL):
select fname from Name
where upper(fname) like :fnameStart or upper(fname) like :fnameMiddle
order by (case when upper(fname) like :fnameStart then 1 else 2 end), fname
query.setParameter("fnameStart", "XX%");
query.setParameter("fnameMiddle", "% XX%");
With Criteria
With Criteria it's much trickier. Firstly, you have to resort to native SQL in the order clause. Secondly, you have to bind the variable.
public class FirstNameOrder extends Order {
public FirstNameOrder() {
super("", true);
}
#Override
public String toSqlString(Criteria criteria, CriteriaQuery criteriaQuery) throws HibernateException {
return "case when upper(FIRST_NAME) like ? then 1 else 2 end";
}
}
The case expression syntax and the upper function name should be changed in accordance with your database (and the column name if it's different, of course).
It is easy to add this to the Criteria, but there is no API to bind the parameter.
I tried to trick Hibernate by passing in an unused variable to the custom sql restriction so that it is effectively used for the variable in the order by clause:
Criteria criteria = session.createCriteria(Name.class, "name")
.add(Restrictions.disjunction()
.add(Restrictions.ilike("name.fname", fname + "%"))
.add(Restrictions.ilike("name.fname", "%" + " " + fname + "%")))
.setProjection(Projections.property("name.fname").as("fname"))
.add(Restrictions.sqlRestriction("1 = 1", fname + "%", StringType.INSTANCE))
.addOrder(new FirstNameOrder())
.addOrder(Order.asc("fname"));
and it works fine.
Obviously, this solution is not recommended and I suggest using JPQL for this query.

Hibernate supports Order: http://docs.jboss.org/hibernate/orm/4.2/devguide/en-US/html/ch11.html#ql-ordering
Because of the special criteria, I think you have to custom the Order in Hibernate. This link may help:
http://blog.tremend.ro/2008/06/10/how-to-order-by-a-custom-sql-formulaexpression-when-using-hibernate-criteria-api/

Run two selects, one filtered for all the strings starting with 'XX', the second filtered for the others.

You can use Predicates in criteria... something like this:
public List<Name> findByParameter(String key, String value, String orderKey)
CriteriaBuilder builder = this.entityManager.getCriteriaBuilder();
CriteriaQuery<Name> criteria = builder.createQuery(this.getClazz());
Root<Name> root = criteria.from(Name.getClass());
criteria.select(root);
List<Predicate> predicates = new ArrayList<Predicate>();
predicates.add(builder.equal(root.get(key), value));
criteria.where(predicates.toArray(new Predicate[predicates.size()]));
if (orderKey!=null && !orderKey.isEmpty()) {
criteria.orderBy(builder.asc(root.get(orderKey)));
}
result = this.entityManager.createQuery(criteria).getResultList();
return result;
}

Stupid but it may work for your case.
Since you got your correct result you can just reconstruct your results as follows:
pick up all results starting with XX you put them into a list L1 and do the normal sort like Collections.sort(L1);
for all other results, do the same like Collections.sort(L2)as list of L2
At last , put them together
List newList = new ArrayList(L1);
newList.addAll(L2);
Please note. Collections.sort are following the natural ordering of its elements.

If you don't want to sort the result in memory,you can modify your criteria,I'll show you the SQL
select * from table where fname like 'XX%' order by fname
union all
select * from table where fname like '% XX%' order by fname
union all
select * from table where fname like '% XX' order by fname
the result will be your order and alphabetical and then apply your filter.

Get individual values from an array created from a resultset in java

I have an array that was created from an ArrayList which was in turn created from a ResultSet. This array contains rows of database table and each row (with several columns based on my query) exists as a single element in the array. So far so good. My problem is how to get individual values (columns) from each row which, I said earlier, now exists as an element. I can get each element (row, of course) but that is not what I want. Each element is a composite of several values and how to get those? I am a beginner and really stuck here. I think this all make sense. Here's the code how I created the array.
List resultsetRowValues = new ArrayList();
while (resultSet.next()){
for (int i = 1; i <= columnCount; i++) {
resultsetRowValues.add(resultSet.getString(i));
}
}
String[] databaseRows = (String[]) resultsetRowValues.toArray(new String[resultsetRowValues.size()]);
EDIT: More explanation
My MySQL query is as follows:
String query = "SELECT FIRSTNAME, LASTNAME, ADDRESS FROM SOMETABLE WHERE CITY='SOMECITY'";
This returns several rows in a ResultSet. And according to the sample query each element of an array will cotain three values (columns) i.e FIRSTNAME, LASTNAME and ADDRESS. But these three values exist in the array as a single element. While I want each column separately from each element (which is actually a row of the database table). When I iterate through the aarray using for loop and print the values to the console, I get output similar to the following:
Doe
Jhon
Some Street (End of First element)
Smith
Jhon
Some Apartment (End of Second element and so on)
As it is evident from the output, each element of the contains three values which are printed on separate lines.
How to get these individual values.

You probably want something like that:
List<Map<String, String>> data = new ArrayList<>();
while (resultSet.next()){
Map<String, String> map = new HashMap<>();
for (int i = 1; i <= columnCount; i++) {
map.put("column" + i, resultSet.getString(i));
}
data.add(map)
}
// usage: data.get(2).get("column12") returns line 3 / column 12
Note that there are other possible options (2D-array, guava Table, ...)

get all the values in where clause for empty stirng using hibernate

i am building a shopping cart using jsp and hibernate.
i am filtering the content by brand size and price using checkboxes
the checked checkboxes are returned to the class where hql query exists.
so i want i single hql query that can handle this.
as like if one of the parameter like size is empty (means user doesnt uses it to filter the content ) than an empty string is passed to the hql query which returns any value...
so is there anything possible that all values can be retrived in where clause for empty string or some other alternative except coding different methods for different parameter...

I typically use the Criteria api for things like this... if the user does not specify a size, do not add it to the criteria query.
Criteria criteria = session.createCriteria(MyClass.class);
if(size != null && !size.isEmpty()){
criteria.add(Restrictions.eq("size", size);
}
To have multiple restrictions via an OR statement, you use Disjunction. For an AND, you use Conjunction.
Criteria criteria = session.createCriteria(MyClass.class);
Disjunction sizeDisjunction = Restrictions.disjunction();
String[] sizes = { "small", "medium", "large" };
for(int i = 0; i < sizes.length; i++){
sizeDisjunction.add(Restrictions.eq("size", sizes[i]);
}
criteria.add(sizeDisjunction );

First, good practices say that instead of passing and empty String to the query, you should pass null instead. That said, this hql should help you:
from Product p
where p.brand = coalesce(:brand, p.brand)
and p.size = coalesce(:size, p.size)
and p.price = coalesce (:price, p.price)

Using HibernateTemplate.find(...) with a String[]

I'm having troubles with HQL again :(
My desired SQL is this:
select employee.idemployee as id, employee.age as age,
employee.birthday as birthday, employee.firstName as firstName,
employee.gender as gender, employee.lastName as lastName from employee
employee
inner join employee_skillgroups skillgroup1 on
employee.idemployee=skillgroup1.idemployee
inner join employee_skillgroups skillgroup2 on
employee.idemployee=skillgroup.idemployee
where skillgroup1.idskillgroup = 'Sprachen'
and skillgroup.idskillgroup = 'SoftSkills'
But i just can't get HQL to generate me this ...
"Sprachen" and "SoftSkills" are two string coming out of a String[] I'm giving the method as a parameter. The method currently looks like this:
public List<Employee> findEmployeeWithTwoSkillGroups(final String[] skillGroups) {
return template.find("from Employee e join e.skillGroups as s where s in ?", Arrays.asList(skillGroups).toString
().substring(1, Arrays.asList(skillGroups).toString().length()-1));
}
I "cast" the array to a list, execute toString() on it (so i get "[Sprachen, SoftSkills]") and cut off the first and the last char (so i get "Sprachen, SoftSkills").
I guess the problem is that HQL generates "[...].idskillgroup in ('Sprachen, SoftSkills')", like it treats the two strings like ONE string ...
And i just can't get it to work like i want it to :/
Can someone please help me and give me a hint what to try/do next? :-)
Greetz
gilaras
Simmilar question:
HQL to get elements that possess all items in a set

The proper HQL statement should go more or less like this:
from Employee e join e.skillGroups as s where s in ( 'Foo', 'Bar', 'Baz' )
so you are missing the ' ' for each word.
EDIT: Now that I got what you want to achieve here :-)
I would recommend you doing this:
Query query = session.createQuery(
"from Employee e join e.skillGroups as s where s in (:skillGroups)");
// skillGroups must be a Collection type
query.setParameterList("skillGroups", skillGroups);
List list = query.list();
In case you need your result to be the AND of all elements of your String[] array, you can do the following:
Query query = session.createQuery(
"from Employee e join e.skillGroups as s where s in (:skillGroups) group by e having count(s) = :skillGroupsLength");
query.setParameterList("skillGroups", skillGroups);
query.setParameter("skillGroupsLength", skillGroups.length);
List list = query.list();

It's a prepared statement, not string concatenation. It handles ('Sprachen, SoftSkills') like a single string because it is a single string. You want to insert a collection in your query. I'm not sure how to do this in hibernateTemplate, but hibernate itself supports adding a collection. see the hibernate documentation.