Consider this code snippet below:
List<String> usersList = Arrays.asList("john", "jack", "jill", "xxxx", "yyyy");
List<User> userEntities = userRepo.findAllById(usersList);
User class is a simple Entity object annotated with #Entity and has an #Id field which is of String datatype.
Assume that in db I have rows corresponding to "john", "jack" and "jill". Even though I passed 5 items in usersList(along with "xxxx" and "yyyy"), findAllById method would only return 3 items/entities corresponding to "john","jack",and "jill".
Now after the call to findAllById method, what's the best, easy and efficient(better than O(n^2) perhaps) way to find out the missing items which findAllById method did not return?(In this case, it would be "xxxx" and "yyyy").
Using Java Sets
You could use a set as the source of filtering:
Set<String> usersSet = new HashSet<>(Arrays.asList("john", "jack", "jill", "xxxx", "yyyy"));
And now you could create a predicate to filter those not present:
Set<String> foundIds = userRepo.findAllById(usersSet)
.stream()
.map(User::getId)
.collect(Collectors.toSet());
I assume the filter should be O(n) to go over the entire results.
Or you could change your repository to return a set of users ideally using a form of distinct clause:
Set<String> foundIds = userRepo.findDistinctById(usersSet)
.stream()
.map(User::getId)
.collect(Collectors.toSet());;
And then you can just apply a set operator:
usersSet.removeAll(foundIds);
And now usersSet contains the users not found in your result.
And a set has a O(1) complexity to find an item. So, I assume this should be O(sizeOf(userSet)) to remove them all.
Alternatively, you could iterate over the foundIds and gradually remove items from the userSet. Then you could short-circuit the loop algorithm in the event you realize that there are no more userSet items to remove (i.e. the set is empty).
Filtering Directly from Database
Now to avoid all this, you can probably define a native query and run it in your JPA repository to retrieve only users from your list which didn't exist in the database. The query would be somewhat as follows that I did in PostgreSQL:
WITH my_users AS(
SELECT 'john' AS id UNION SELECT 'jack' UNION SELECT 'jill'
)
SELECT id FROM my_users mu
WHERE NOT EXISTS(SELECT 1 FROM users u WHERE u.id = mu.id);
Spring Data: JDBC Example
Since the query is dynamic (i.e. the filtering set could be of different sizes every time), we need to build the query dynamically. And I don't believe JPA has a way to do this, but a native query might do the trick.
You could either pack a JdbcTemplate query directly into your repository or use JPA native queries manually.
#Repository
public class UserRepository {
private final JdbcTemplate jdbcTemplate;
public UserRepository(JdbcTemplate jdbcTemplate) {this.jdbcTemplate = jdbcTemplate;}
public Set<String> getUserIdNotFound(Set<String> userIds) {
StringBuilder sql = new StringBuilder();
for(String userId : userIds) {
if(sql.length() > 0) {
sql.append(" UNION ");
}
sql.append("SELECT ? AS id");
}
String query = String.format("WITH my_users AS (%sql)", sql) +
"SELECT id FROM my_users mu WHERE NOT EXISTS(SELECT 1 FROM users u WHERE u.id = mu.id)";
List<String> myUsers = jdbcTemplate.queryForList(query, userIds.toArray(), String.class);
return new HashSet<>(myUsers);
}
}
Then we just do:
Set<String> usersIds = Set.of("john", "jack", "jill", "xxxx", "yyyy");
Set<String> notFoundIds = userRepo.getUserIdNotFound(usersIds);
There is probably a way to do it with JPA native queries. Let me see if I can do one of those and put it in the answer later on.
You can write your own algorithm that finds missing users. For example:
List<String> missing = new ArrayList<>(usersList);
for (User user : userEntities){
String userId = user.getId();
missing.remove(userId);
}
In the result you will have a list of user-ids that are missing:
"xxxx" and "yyyy"
You can just add a method to your repo:
findByIdNotIn(Collection<String> ids) and Spring will make the query:
See here:
https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods
Note (from the docs):
In and NotIn also take any subclass of Collection as aparameter as well as arrays or varargs.
I am working on spring boot and Hibernate and
I need data from different table in mysql database so I ran custom query that to get data from different tables.And my query is,
#Query(value = "Select i.item_id,i.item_name,s.size_name,sp.Prices from mt_item i join mt_sizeprice sp on i.item_id= sp.item_id join mt_size s on sp.size_id=s.size_id where i.merchant_fk= ?1 and is_featured=1",nativeQuery = true)
List<Object> get(long merchantId);
The result of the query is,
As the data is from different table I am unable to map List of data of Object into the List of my model class as,
List<Object> get(long merchantId); to
List<FavoriteItem> list = new ArrayList<>();
It seems the result of your query will be List of Object[] instead of Object List. You can pass result to method and create List of FavoriteItem by iterating over results.
I am returning only a few columns from a table in DB:
List<MyClass> l = (List<MyClass>) session.createQuery("Select p.one, p.two FROM MyClass p WHERE p.id IN :id")
.setParameter("userId", id)
.list();
However, the query returns List of arrays, e.g
l.get(0) // [0] is object representing p.one in query, [1] p.two
Is there a hibernate some effective way how to map it to MyClass object? So the query would actually return list of MyClass objects where selected properties would have values, others would be set to null?
I have read about new MyClass(arg1,arg2) way in the query with construtor, however i have also read it is ineffective.
Thanks for help!
You can use try catch block and use query.getSingleResult() to return Object.
Query query = em.createNativeQuery("FROM TipoUsuario WHERE NAME = :name;", TipoUsuario.class)
.setParameter("name", name);
TipoUsuario tipoUsuario = null;
try
{
tipoUsuario = (TipoUsuario) query.getSingleResult();
}
catch ( Exception e )
{
return null;
}
createNativeQuery is just a sample. You can use createNamedQuery instead and put HQL in Entity with annotation #NamedQueries.
If you want to create MyClass you just need to use the fully-qualified name of the class in your query, e.g.
SELECT NEW my.pack.MyClass(p.one, p.two) FROM MyClass p WHERE p.id IN :id
I haven't heard of this being ineffective, and I can't image why it would be. It's just calling the constructor on the result instead of returning it as an array or object. Overall, it's probably a very good way of fetching, since you only select the fields that you need.
This and other options are also explained in more detail here.
I have a case like this:
Class Foo with two children (A and B), each being Objects.
In Hibernate, if I want to return only a list of the children, I would use projections on my criteria:
criteria.setProjection(Projections.property("A"));
This gives me a list of A objects, but they are all lazy loaded. As soon as I try to access anything other than the id, obviously things go wrong.
My SQL query indeed shows it:
select A from Foo ...
Logically, only my id is filled in, and not the rest of my properties. How do I solve this problem so I get a list of A objects that have everything filled in?
I tried this:
criteria.setResultTransformer(Transformers.aliasToBean(A.class));
but without success.
if you use hql it would be more efficient:
String hql = "SELECT f.A FROM Foo f";
Query query = session.createQuery(hql);
List results = query.list();
Using criteria I would have use this code
Criteria crit = session.createCriteria(Foo.class);
ProjectionList proList = Projections.projectionList();
proList.add(Projections.property("A"));
crit.setProjection(proList);
List As= crit.list();
or this block
Criteria crit = session.createCriteria(Foo.class);
crit.createAlias("A","a").setProjection(Projections.property("a"));
List As= crit.list();
I know I can pass a list to named query in JPA, but how about NamedNativeQuery? I have tried many ways but still can't just pass the list to a NamedNativeQuery. Anyone know how to pass a list to the in clause in NamedNativeQuery? Thank you very much!
The NamedNativeQuery is as below:
#NamedNativeQuery(
name="User.findByUserIdList",
query="select u.user_id, u.dob, u.name, u.sex, u.address from user u "+
"where u.user_id in (?userIdList)"
)
and it is called like this:
List<Object[]> userList = em.createNamedQuery("User.findByUserIdList").setParameter("userIdList", list).getResultList();
However the result is not as I expected.
System.out.println(userList.size()); //output 1
Object[] user = userList.get(0);
System.out.println(user.length); //expected 5 but result is 3
System.out.println(user[0]); //output MDAVERSION which is not a user_id
System.out.println(user[1]); //output 5
System.out.println(user[2]); //output 7
The above accepted answer is not correct and led me off track for many days !!
JPA and Hibernate both accept collections in native query using Query.
You just need to do
String nativeQuery = "Select * from A where name in :names"; //use (:names) for older versions of hibernate
Query q = em.createNativeQuery(nativeQuery);
q.setParameter("names", l);
Also refer the answers here which suggest the same (I picked the above example from one of them)
Reference 1
Reference 2 which mentioned which cases paranthesis works which giving the list as a parameter
*note that these references are about jpql queries, nevertheless the usage of collections is working with native queries too.
A list is not a valid parameter for a native SQL query, as it cannot be bound in JDBC. You need to have a parameter for each argument in the list.
where u.user_id in (?id1, ?id2)
This is supported through JPQL, but not SQL, so you could use JPQL instead of a native query.
Some JPA providers may support this, so you may want to log a bug with your provider.
Depending on your database/provider/driver/etc., you can, in fact, pass a list in as a bound parameter to a JPA native query.
For example, with Postgres and EclipseLink, the following works (returning true), demonstrating multidimensional arrays and how to get an array of double precision. (Do SELECT pg_type.* FROM pg_catalog.pg_type for other types; probably the ones with _, but strip it off before using it.)
Array test = entityManager.unwrap(Connection.class).createArrayOf("float8", new Double[][] { { 1.0, 2.5 }, { 4.1, 5.0 } });
Object result = entityManager.createNativeQuery("SELECT ARRAY[[CAST(1.0 as double precision), 2.5],[4.1, 5.0]] = ?").setParameter(1, test).getSingleResult();
The cast is there so the literal array is of doubles rather than numeric.
More to the point of the question - I don't know how or if you can do named queries; I think it depends, maybe. But I think following would work for the Array stuff.
Array list = entityManager.unwrap(Connection.class).createArrayOf("int8", arrayOfUserIds);
List<Object[]> userList = entityManager.createNativeQuery("select u.* from user u "+
"where u.user_id = ANY(?)")
.setParameter(1, list)
.getResultList();
I don't have the same schema as OP, so I haven't checked this exactly, but I think it should work - again, at least on Postgres & EclipseLink.
Also, the key was found in: http://tonaconsulting.com/postgres-and-multi-dimensions-arrays-in-jdbc/
Using hibernate, JPA 2.1 and deltaspike data I could pass a list as parameter in query that contains IN clause. my query is below.
#Query(value = "SELECT DISTINCT r.* FROM EVENT AS r JOIN EVENT AS t on r.COR_UUID = t.COR_UUID where " +
"r.eventType='Creation' and t.eventType = 'Reception' and r.EVENT_UUID in ?1", isNative = true)
public List<EventT> findDeliveredCreatedEvents(List<String> eventIds);
can be as simple as:
#Query(nativeQuery =true,value = "SELECT * FROM Employee as e WHERE e.employeeName IN (:names)")
List<Employee> findByEmployeeName(#Param("names") List<String> names);
currently I use JPA 2.1 with Hibernate
I also use IN condition with native query. Example of my query
SELECT ... WHERE table_name.id IN (?1)
I noticed that it's impossible to pass String like "id_1, id_2, id_3" because of limitations described by James
But when you use jpa 2.1 + hibernate it's possible to pass List of string values. For my case next code is valid:
List<String> idList = new ArrayList<>();
idList.add("344710");
idList.add("574477");
idList.add("508290");
query.setParameter(1, idList);
In my case ( EclipseLink , PostGreSQL ) this works :
ServerSession serverSession = this.entityManager.unwrap(ServerSession.class);
Accessor accessor = serverSession.getAccessor();
accessor.reestablishConnection(serverSession);
BigDecimal result;
try {
Array jiraIssues = accessor.getConnection().createArrayOf("numeric", mandayWorkLogQueryModel.getJiraIssues().toArray());
Query nativeQuery = this.entityManager.createNativeQuery(projectMandayWorkLogQueryProvider.provide(mandayWorkLogQueryModel));
nativeQuery.setParameter(1,mandayWorkLogQueryModel.getPsymbol());
nativeQuery.setParameter(2,jiraIssues);
nativeQuery.setParameter(3,mandayWorkLogQueryModel.getFrom());
nativeQuery.setParameter(4,mandayWorkLogQueryModel.getTo());
result = (BigDecimal) nativeQuery.getSingleResult();
} catch (Exception e) {
throw new DataAccessException(e);
}
return result;
Also in query cannot use IN(?) because you will get error like :
Caused by: org.postgresql.util.PSQLException: ERROR: operator does not exist: numeric = numeric[]
'IN(?)' must be swapped to '= ANY(?)'
My solution was based on Erhannis concept.
In jpa, it worked for me
#Query(nativeQuery =true,value = "SELECT * FROM Employee as e WHERE e.employeeName IN (:names)")
List<Employee> findByEmployeeName(#Param("names") List<String> names);
Tried in JPA2 with Hibernate as provider and it seems hibernate does support taking in a list for "IN" and it works. (At least for named queries and I believe it will be similar with named NATIVE queries)
What hibernate does internally is generate dynamic parameters, inside the IN same as the number of elements in the passed in list.
So in you example above
List<Object[]> userList = em.createNamedQuery("User.findByUserIdList").setParameter("userIdList", list).getResultList();
If list has 2 elements the query will look like
select u.user_id, u.dob, u.name, u.sex, u.address from user u "+
"where u.user_id in (?, ?)
and if it has 3 elements it looks like
select u.user_id, u.dob, u.name, u.sex, u.address from user u "+
"where u.user_id in (?, ?, ?)
you should do this:
String userIds ="1,2,3,4,5";
List<String> userIdList= Stream.of(userIds.split(",")).collect(Collectors.toList());
Then, passes like parameter inside your query, like this:
#NamedNativeQuery(name="User.findByUserIdList", query="select u.user_id, u.dob, u.name, u.sex, u.address from user u where u.user_id in (?userIdList)")
It's not possible with standard JPA. Hibernate offers the proprietary method setParameterList(), but it only works with Hibernate sessions and is not available in JPA's EntityManager.
I came up with the following workaround for Hibernate, which is not ideal but almost standard JPA code and has some nice properties to it.
For starters you can keep the named native query nicely separated in a orm.xml file:
<named-native-query name="Item.FIND_BY_COLORS" result-class="com.example.Item">
<query>
SELECT i.*
FROM item i
WHERE i.color IN ('blue',':colors')
AND i.shape = :shape
</query>
</named-native-query>
The placeholder is wrapped in single quotes, so it's a valid native JPA query. It runs without setting a parameter list and would still return correct results when other matching color parameters are set around it.
Set the parameter list in your DAO or repository class:
#SuppressWarnings("unchecked")
public List<Item> findByColors(List<String> colors) {
String sql = getQueryString(Item.FIND_BY_COLORS, Item.class);
sql = setParameterList(sql, "colors", colors);
return entityManager
.createNativeQuery(sql, Item.class)
.setParameter("shape", 'BOX')
.getResultList();
}
No manual construction of query strings. You can set any other parameter as you normally would.
Helper methods:
String setParameterList(String sql, String name, Collection<String> values) {
return sql.replaceFirst(":" + name, String.join("','", values));
}
String getQueryString(String queryName, Class<?> resultClass) {
return entityManager
.createNamedQuery(queryName, resultClass)
.unwrap(org.hibernate.query.Query.class) // Provider specific
.getQueryString();
}
So basically we're reading a query string from orm.xml, manually set a parameter list and then create the native JPA query. Unfortunately, createNativeQuery().getResultList() returns an untyped query and untyped list even though we passed a result class to it. Hence the #SuppressWarnings("unchecked").
Downside: Unwrapping a query without executing it may be more complicated or impossible for JPA providers other than Hibernate. For example, the following might work for EclipseLink (untested, taken from Can I get the SQL string from a JPA query object?):
Session session = em.unwrap(JpaEntityManager.class).getActiveSession();
DatabaseQuery databaseQuery = query.unwrap(EJBQueryImpl.class).getDatabaseQuery();
databaseQuery.prepareCall(session, new DatabaseRecord());
Record r = databaseQuery.getTranslationRow();
String bound = databaseQuery.getTranslatedSQLString(session, r);
String sqlString = databaseQuery.getSQLString();
An alternative might be to store the query in a text file and add code to read it from there.
You can pass in a list as a parameter, but:
if you create a #NamedNativeQuery and use .createNamedQuery(), you don't use named param, you used ?1(positional parameter). It starts with 1, not 0.
if you use .createNativeQuery(String), you can use named param.
You can try this :userIdList instead of (?userIdList)
#NamedNativeQuery(
name="User.findByUserIdList",
query="select u.user_id, u.dob, u.name, u.sex, u.address from user u "+
"where u.user_id in :userIdList"
)