We Keep Coding

Using arbitrary query as projection in spring data rest project

How it is possible to use arbitrary sql query (I mean native sql query) in some repository? My actual problem is this:
#Data //lombok thing
#Entity
public class A extends AuditModel {
private long id;
private String name;
#OneToMany(mappedBy="a") //Comments.a is owning side of association, i.e. comments table does have column called a_id as foreign key
#ToString.Exclude
private Set<Comments> comments = new HashSet();
#OneToMany(mappedBy="a") //SimpleFile.a is owning side of association
private Set<SimpleFile> comments = new HashSet();
}
Than I have my repository, which exposes nice crud interface using HAL+json representation. I am trying to enrich it with some projection/view particularly due to web UI to load one page data in single request. I am aware of excerps and projections, but they seems not to be enough powerful.
#Repository
#RepositoryRestResource
#Transactional(readOnly = true)
public interface ARepository extends PagingAndSortingRepository<A, Long> {
Page<A> findByNameContaining(String namePart, Pageable pageable);
#Query(
value = "SELECT a.name,\n" +
"(SELECT CAST(count(ac.id) AS int) FROM COMMENTS ac WHERE ac.a_id = a.id),\n" +
"(SELECT listagg(asf.id) FROM SIMPLE_FILES asf WHERE asf.a_id = a.id)\n" +
"FROM AS a\n" +
"WHERE a.id = :id",
nativeQuery = true
)
Optional<ACustomPage42DTO> getByIdProjectedForScreen42(Long id);
}
I have also tried to use JPQL, but there I had problem with fetch join (as I am not familiar with JPQL). My last evaluation query was something like this:
#Query("SELECT new sk.qpp.qqq.documents.projections.ACustomPage42DTO(" +
"a " +
"(SELECT CAST(count(ac) AS int) FROM COMMENTS ac WHERE ac.a = a)" +
")\n" +
"FROM A a\n" +
"LEFT JOIN FETCH a.simpleFiles\n" +
"WHERE a.id = :id"
)
I would like to get some general advice about what approach is best to implement custom and complex query to be returned in DTO (ideally with some specific links to actions when needed).
PS: Implementing interface and returning simple (primitive) data works. Also using JPQL to create custom DAO instance works (with simple types and with single instance of type A for example). Method for using given query method does appear in search methods of given entity endpoint. I would like to have something more reasonable, so I would like to have projection as defined in spring data rest project.
I have my DTO object fully under my control. I prefer it to use #Value or #Data annotation from project lombok, but it is not a need. I have tried also these versions of DTO definition (using interface works for simple data and similarly class works for simple data).
interface ACustomPage42DTO {
String getName();
long getCommentsCount();
Object getAsdf();
}
Or using equivalent class with some bonus, like custom toString() method possible, or some custom getter for computed data:
#Value //lombok thing, imutable "POJO"
public class ACustomPage42DTO {
String name;
long commentsCount;
Set<SimpleFile> simpleFiles;
public ACustomPage42DTO(A a, long count) {
// constructor used by JPQL, if it works
name = a.getName();
this.commentsCount = count;
this.simpleFiles = a.getSimpleFiles(); // should be already fetched, due to fetch join in JPQL
}
}
Both working approaches can be called using "search" url, instead of projection. I see my method getByIdProjectedForScreen42 on url http://localhost:9091/api/a/search listing. I would like to use it like (I think that is the "right" way) http://localhost:8080/api/a?projection=ACustomPage42DTOProjection .

Question is quite broad and touches couple of aspects:
custom JPA repository method using #Query
selecting results in your #Query
mapping #Query results to an interface
exposing new repository method through #RepositoryRestResource
TLDR: wrote an example of what is talked about with couple of basic tests https://github.com/ivarprudnikov/test-spring-jpa-repository-query-exposed-through-http
custom JPA repository method using #Query
As you have mentioned it is quite straightforward, just annotate a method with #Query and make sure your return type corresponds to what is being returned from the query, eg:
public interface FooRepository extends JpaRepository<FooEntity, Long> {
#Query(nativeQuery = true, value = "select f from foo f where f.name = :myParam")
Optional<FooEntity> getInSomeAnotherWay(String myParam);
}
selecting results in your #Query
You have given an example already but I'll simplify to make it easier and shorter.
Given entities FooEntity.java and BarEntity.java:
#Entity
#Table(name = "foo")
public class FooEntity {
#Id
#Column(name = "id", unique = true, nullable = false)
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "name", nullable = false)
private String name;
#OneToMany(mappedBy = "foo")
private Set<BarEntity> bars = new HashSet<>();
// getter setters excluded for brevity
}
#Entity
#Table(name = "bar")
public class BarEntity {
#Id
#Column(name = "id", unique = true, nullable = false)
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "name", nullable = false)
private String name;
#ManyToOne(targetEntity = FooEntity.class)
#JoinColumn(name = "foo_id", nullable = false, foreignKey = #ForeignKey(name = "fk_bar_foo"))
private FooEntity foo;
// getter setters excluded for brevity
}
We want now to return custom result set which contains FooEntity.name and count of FooEntity.bars:
SELECT f.name as name, count(b.id) as barCount FROM foo f, bar b WHERE f.id = :id AND b.foo_id = :id
+-----------------+----------+
| name | barCount |
+-----------------+----------+
| Jonny tables | 1 |
+-----------------+----------+
mapping #Query results to an interface
To map above result set we need an interface where getters nicely reflect what is being selected:
public interface ProjectedFooResult {
String getName();
Long getBarCount();
}
Now we can rewrite our repository method to:
#Query(nativeQuery = true,
value = "SELECT f.name as name, count(b.id) as barCount FROM foo f, bar b WHERE f.id = :id AND b.foo_id = :id")
Optional<ProjectedFooResult> getByIdToProjected(Long id);
exposing new repository method through #RepositoryRestResource
I am not very familiar with this but after adding org.springframework.data:spring-data-rest-hal-browser dependency I got this nice interface that exposed available methods after repository was annotated with #RepositoryRestResource. For a given repository which contains above mentioned details:
#RepositoryRestResource(path = "foo")
public interface FooRepository extends JpaRepository<FooEntity, Long> {
#Query(nativeQuery = true, value = "SELECT f.name as name, count(b.id) as barCount FROM foo f, bar b WHERE f.id = :id AND b.foo_id = :id")
Optional<ProjectedFooResult> getByIdToProjected(Long id);
}
the method will be exposed through http://localhost:8080/foo/search/getByIdToProjected?id=1 when running locally.
As mentioned above the reference implementation is on Github https://github.com/ivarprudnikov/test-spring-jpa-repository-query-exposed-through-http
Additional helpful documentation for 'Custom Implementations for Spring Data Repositories'

Filter entities by a condition within a collection

The simplified question would be:
Having in a “A” entity, a collection of “B” entities (from a OneToMany relationship) and using CriteriaBuilder, how to get the elements from "A" that have in this collection at least one element "B" that meets a condition? (for example B has a property equal to 10, or as another example B points to another entity whose property is equal to 10).
To put in context:
In my program I have the following entities
#Entity
#SequenceGenerator(name = "sequence", sequenceName = "S_EXPEDIENT")
public class Expedient {
private Long idExpedient
private Integer numberExpedient
private Integer yearExpedient
#ManyToOne
#JoinColumn(name = "id_chamber")
private Chamber chamber
private Integer status
#OneToMany(cascade = CascadeType.ALL, mappedBy = "expedient")
private List<HistoricalAssignment> assignments = new ArrayList<HistoricalAssignment>();
//more fields
}
#SequenceGenerator(name = "sequence", sequenceName = "S_HISTORICAL_ASSIGNMENT ")
#Entity
public class HistoricalAssignment {
private Long idHistoricalAssignment
#ManyToOne
#JoinColumn(name = "id_expedient")
private Expedient expedient;
#ManyToOne
#JoinColumn(name = "id_office")
private Office office;
//more fields
}
#SequenceGenerator(name = "sequence", sequenceName = "S_OFFICE")
#Entity
public class Office extends {
private Long idOffice
#ManyToOne
#JoinColumn(name = "id_chamber")
private Chamber chamber
//more fields
}
#SequenceGenerator(name = "sequence", sequenceName = "S_CHAMBER")
#Entity
public class Chamber{
private Long id_chamber
private String description;;
//more fields
}
I am also using Spring Data Specifications to perform searches with filters.
The problem I have is when trying to get all the "Expedients" that were assigned at least once to a "Chamber", since I would have to look for those who have in the collection "assignments" at least one "Office" that belongs to a "Chamber."
I edit by adding more code. Keep in mind that the original code has several levels of inheritance (here simplified) and also the original model is in Spanish:
public class ExpedientSpecification implements Specification<Expedient> {
private final Integer STATUS_DELETED = -1;
protected ExpedientFilter criteria;
protected List<Predicate> predicates = new ArrayList<Predicate>();
public ExpedientSpecification(ExpedientFilter criteria) {
this.criteria=criteria;
}
public Predicate toPredicate(Root<Expedient> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
//variable filters
if(criteria.getId() != null){
predicates.add(cb.equal(root.get("id"), criteria.getId()));
}
if(criteria.getNumberExpedient() != null){
predicates.add(criteriaBuilder.equal(root.get("numberExpedient"), criteria.getNumberExpedient()));
}
if(criteria.getYearExpedient() != null){
predicates.add(criteriaBuilder.equal(root.get("yearExpedient"), criteria.getYearExpedient()));
}
//Fixed Filters
predicates.add(criteriaBuilder.notEqual(root.get("status"), STATUS_DELETED));
// This is where I need to add the filter. I need to get only those in the "assignments" List have at least one
// "Office" that belongs to a "Chamber"
Predicate returnPredicates = cb.and(predicates.toArray(new Predicate[this.predicates.size()]));
predicates.clear();
return returnPredicates;
}
}

How are you doing the criteria? Can you post the code?
Anyway you can join oneToMany normally with a criteria alias.
Example: criteria.createAlias("assignments","assignments"); and then add a restriction on the atributte from assignments that you wish to filter.

What is the most efficient way to retrieve specific data from DB without doing loop for all records?

I want to ask about what is the most efficient way to search about specific data from a database without doing a for loop in all of the records?
I have a project on java spring and I have this Entity:
#Entity
#Table(name = "USERS") public class USERS {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "UID")
private Integer id;
#Column(name = "FName")
private String firstName;
#Column(name = "SName")
private String secondName;
#Column(name = "TName")
private String thirdName;
#Column(name = "LName")
private String fourthName;
#Column(name = "Email")
private String email;
#Column(name = "PW")
private String password;
#Column(name = "MNumber")
private String mobileNumber;
#Column(name = "ISDeleted")
private boolean isUserDeleted;
//---------------------- Getters and Setters ----------------------
and I made this service:
public List<USERS> findAllActive() {
List<USERS> usersList = new ArrayList<USERS>();
for (USERS users: usersRepository.findAll()){
if (!users.isUserDeleted()){
usersList.add(users);
}
}
return usersList;
}
For example; I have one property for User, if he is active or not.
So, my question; what is the most efficient way to do get specific data like retrieving all of the active users from the DB without doing a for loop like in the code above? Because if the list of users is a 1 Million or more, it could have performance issues.

Assuming that you are using JpaRepository then you can create custom query.
#Query("SELECT u FROM USERS u WHERE u.userDeleted = false")
List<USERS> findNotDeletedUsers();
and then call usersRepository.findNotDeletedUsers();

First of all, use an index on the field you want to search on (this won't help you much if the column has only two distinct values, but will make a huge difference if the value has high sparsity).
#Entity
#Table(name = "USERS",
indexes = {
// not a huge performance gain, since the column values are true/false
#Index(name = "index_by_active", columnList="ISDeleted", unique = false),
// possible huge performance gain, since only the relevant records are scanned
#Index(name = "index_by_first_name", columnList="FName", unique = false)})
public class USERS {...}
Then, define a query method that uses the indexed field (if you are using spring data it would look as follows).
public interface UsersRepository extends CrudRepository<USERS, Long> {
List<USERS> findUsersByISDeleted(boolean deleted);
List<USERS> findUsersByFName(String name);
List<USERS> findUsersByFNameAndISDeleted(String name, boolean deleted);
}
Queries on indexed fields will leverage the underlying index and provide an efficient access plan (so you won't end up scanning the whole table in order to extract a subset of entities matching a given criteria).

The solution from #Madis is okay. But if you always want to get users which are not deleted in all queries, you can specify it on Entity:
#Entity
#Table(name = "USERS")
#Where("ISDeleted = false")
public class USERS {
So now the condition "ISDeleted = false" is automatically append to all queries from the UserRepository. You can use usersRepository.findAll() instead of.

You don't need to specify any sql query or where clause. CrudRepository will do it for you automatically. Just use below code and pass true/false on need basis
List<Users> findIsUserDeleted(boolean isDeleted)

JPA Criteria API Specification for Many to Many

I have got three classes as mentioned below. I am trying to create a specification to filter data where there is a match in the linked table.
public class Album {
private Long id;
private List<AlbumTag> albumTags;
}
public class Tag {
private Long id;
private String category;
}
public class AlbumTag{
private Long id;
private Album album;
private Tag tag;
}
In the schema given above what I am trying to find is a list of all albums from Album table with the link in AlbumTag. The SQL that I want to achieve, doesn't have to be same, is below
select *
from Album A
where (A.Id in (select [AT].AlbumId
from AlbumTag [AT]))
What I have tried so far which is not working, of course, is below
public class AlbumWithTagSpecification implements Specification<Album> {
#Override
public Predicate toPredicate(Root<Album> root, CriteriaQuery<?> cq, CriteriaBuilder cb) {
final Subquery<Long> personQuery = cq.subquery(Long.class);
final Root<Album> album = personQuery.from(Album.class);
final Join<Album, AlbumTag> albumTags = album.join("albumTags");
personQuery.select((albumTags.get("album")).get("id"));
personQuery.where(cb.equal(album.get("id"), (albumTags.get("album")).get("id")));
return cb.in(root.get("id")).value(personQuery);
}
}

Using spring boot and spring data JPA, you can prefer entity relationship to fetch the data.
1.Annotate the domain class with the entity relationship which given below:
#Entity
#Table(name="Album")
public class Album {
#Id
#Column(name="id")
private Long id;
#OneToMany(targetEntity = AlbumTag.class, mappedBy = "album")
private List<AlbumTag> albumTags;
//getter and setter
}
#Entity
#Table(name="Tag")
public class Tag {
#Id
#Column(name="id")
private Long id;
#Column(name="category")
private String category;
//getter and setter
}
#Entity
#Table(name="AlbumTag")
public class AlbumTag{
#Id
#Column(name="id")
private Long id;
#ManyToOne(optional = false, targetEntity = Album.class)
#JoinColumn(name = "id", referencedColumnName="id", insertable = false, updatable = false)
private Album album;
#ManyToOne(optional = false, targetEntity = Tag.class)
#JoinColumn(name = "id", referencedColumnName="id", insertable = false, updatable = false)
private Tag tag;
//getter and setter
}
2.use the spring data to fetch the details using the below:
Album album = ablumRepository.findOne(1); // get the complete details about individual album.
List<AlbumTag> albumTags = ablum.getAlbumTags(); // get the all related albumTags details for particular album.
I hope this will help you to solve it.

Subqueries in JPA only really work with CriteriaBuilder.exists() so i would try:
public Predicate toPredicate(Root<Album> root, CriteriaQuery<?> cq, CriteriaBuilder cb) {
final Subquery<Long> subQuery = cq.subquery(Long.class);
final Root<AlbumTag> albumTag = subQuery.from(AlbumTag.class);
// it doesn't really matter what we select
subQuery.select(cb.literal(1));
subQuery.where(cb.equal(root.get("id"), (albumTag.get("album")).get("id")));
return cb.exists(subQuery);
}
which is equivalent to
select *
from Album A
where exists(
select 1 from AlbumTag AT
where AT.AlbumId = A.Id
)

Well, I wouldn't go for in operation in this case - it just complicates the query and the specification. The problem you described is actually matter of joining records from Table A with related records from Table B so the query in your case would be like:
SELECT a from Album a join AlbumTag at on a.id = at.albumId - as you needed it will return all albums that have album tags. Inner join explained
So in your case I would create this "factory" method that would create for you this specification.
public static Specification<Album> withTags() {
return new Specification<Album>() {
#Override
public Predicate toPredicate(Root<Album> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
return root.join("albumTags").getOn();
}
};
}
Also I would suggest you to have a look at static metamodel library from hibernate - link to introduction. It generates for you static model from your entity classes that helps you avoid creating queries/specifications using hardcoded strings.

creteria query for join tables
CriteriaQuery<Album> query = cb.createQuery(Album.class);
Root<Album> album = query.from(Teacher.class);
Join<Album, AlbumTag> tag = teacher.join("id");
query.select(tag).where(cb.equal(album.get("album")));
List<Album> results = em.createQuery(query).getResultList();
for (Album al : results) {
System.out.println("album-->+al.get(name));
}

This looks like a classic many to many example. The three classes you have map directly to the tables you would expect in the database. JPA is an Object Relational Mapping (ORM) library which means we can structure the classes in a more OO style and map to the underlying relational database.
The AlbumTag class can be omitted and the #ManyToMany relationship added to both Album and Tag.
public class Album {
private Long id;
#ManyToMany
#JoinTable(name="AlbumTag",
joinColumns=
#JoinColumn(name="album", referencedColumnName="id"),
inverseJoinColumns=
#JoinColumn(name="tag", referencedColumnName="id"))
private List<Tag> tags;
}
public class Tag {
private Long id;
private String category;
#ManyToMany(mappedBy="tags")
private List<Album> albums;
}
To find albums by Tag you would first retrieve the Tag from the repository using something like findById(1l); or findByCategory("Rock"); and then simply call getAlbums() on the Tag object.
Note: One slight difference here is that the AlbumTag table would have only two columns (album and tag). The extra id column on AlbumTag is unnecessary since the combination of album and tag would be a unique id and you would never need to find by id in this table anyway.

Since you are using spring-data-jpa you should really take advantage of the features it provides.
My first question is related to your entity classes. I do not understand why is it necesary to store a list of album tags in the album class. Since you have a join table this information is reduntant.
Secondly you should adnotate your entity clases:
#Entity
public class Album {
#Id
#Column
private Long id;
}
#Entity
public class Tag {
#Id
#Column
private Long id;
#Column
private String category;
}
#Entity
#Table
public class AlbumTag{
#Id
#Column
private Long id;
#ManyToOne
#JoinColumn
private Album album;
#ManyToOne
#JoinColumn
private Tag tag;
}
Next you should create repositories for your entity classes.
interface AlbumRepository extends JpaRepository<Album, Long>{
#Query
("select DISTINCT(a) from AlbumTag at "+
"join at.album a "
"where at.tag is not null")
List<Album> findAlbumWithTag();
}
Then simply call the repository function which will return a list of albums which have at least one tag.

JPA : How to define #NamedEntityGraph for 3 levels?

I have 3 entities. Branch,Subject,Topic.
Branch has list of subjects and Subject has list of topics. Also
subjectList and topicList both are lazy. I want to fetch all branch
including its subjects and topics in single query.
1.
#Entity
public class Branch implements Serializable
{
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Integer id;
private String name;
#OneToMany(mappedBy = "branch")
private List<Subject> subjectList;
//Getters and Setters
}
2.
#Entity
public class Subject implements Serializable
{
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Integer id;
private String name;
#ManyToOne()
private Branch branch;
#OneToMany(mappedBy = "subject")
private List<Topic> topicList;
//Getters and Setters
}
3.
#Entity
public class Topic implements Serializable
{
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Integer id;
private String name;
#ManyToOne()
private Subject subject;
//Getters and Setters
}
I tried the method below but it didn't work.
#NamedEntityGraph(name="branch_subject",
attributeNodes = {
#NamedAttributeNode(value="name"),
#NamedAttributeNode(value="subjectList", subgraph = "subjectListGraph")
},
subgraphs = {
#NamedSubgraph(name="subjectListGraph",
attributeNodes = {
#NamedAttributeNode(value="name"),
#NamedAttributeNode(value = "topicList", subgraph = "topicListGraph")
}
),
#NamedSubgraph(name="topicListGraph",
attributeNodes = {
#NamedAttributeNode("name")
}
)
}
)
Also following code is used to fetch data from
database, I am using JPQL as follows
EntityGraph branchEntityGraph = entityManager
.getEntityGraph("branch_subject");
Branch branch = entityManager
.createQuery("SELECT b from Branch b WHERE b.id=:ID",
Branch.class)
.setHint("javax.persistence.loadgraph", branchEntityGraph)
.setParameter("ID", branch1.getId()).getResultList().get(0);
This gives below exception
org.hibernate.loader.MultipleBagFetchException: cannot simultaneously fetch multiple bags

Hibernate doesn't allow you to fetch multiple Bags because it would end up fetching a Cartesian Product.
M → N → P one-to-many or many-to-many relations
For a multi-nested hierarchy, you can use JOIN FETCH on multiple collections as long as your lists are mapped as Set.
M → N and M → P one-to-many or many-to-many relations
For sibling collections, like M → N and M → P, don't switch to using Set instead of List.
Using a Set instead of a List to avoid the MultipleBagFetchException is a very bad idea since you will still end up with a Cartesian Product, and that's going to cause performance issues because you are going to fetch M x N x P records.
In this case, a much better approach is to fetch one collection with the first query and use additional queries for the remaining collections:
List<Post> _posts = entityManager.createQuery("""
select distinct p
from Post p
left join fetch p.comments
where p.id between :minId and :maxId
""", Post.class)
.setParameter("minId", 1L)
.setParameter("maxId", 50L)
.setHint(QueryHints.PASS_DISTINCT_THROUGH, false)
.getResultList();
_posts = entityManager.createQuery("""
select distinct p
from Post p
left join fetch p.tags t
where p in :posts
""", Post.class)
.setParameter("posts", _posts)
.setHint(QueryHints.PASS_DISTINCT_THROUGH, false)
.getResultList();
This strategy allows you to avoid the M x N x P result set by fetching M x (N + P) records instead.
Fetching from the child-side to the parent
If you have to use INNER JOIN when fetching the child collection, then you can simply [fetch from the inner-most Child up to the root][3] and reassemble the structure afterward. This is much more efficient since the query goes like this:
select t
from Topic t
join t.subject s
join s.branch b

The solution is fairly simple use JPA 2.1 where you need add entiry grapgh at JPA repository only
#Repository
public interface BranchRepo extends JpaRepository< Branch, Long> {
#EntityGraph(attributePaths = {"subjectList" ,
"subjectList.topicList"})
List<Product> findAll();
}
This will create Join between the bi-directional mapping of Branch-> SubjectList-> TopicList
Blockquote you need to override equals method in each of these entity classes with ID else this will not work. You can get rid of complex multilevel NamedEntityGraph at each entity class which is not needed

I guess I had some similar issue. All my Device entities have a Transaction object which in my case stores the datetime and user and computer information for that particular object. Also have a DeviceType entity that has transaction object and has a Many to one relation with my Device entity. So Actually I had both 1 level and 2 levels of nested relations with this Transaction Entity. I got some nested exception because of that. Using Subgraph fixed that issue for me. Here is my NamedEntityGraphs definition code. Hope it helps:
#NamedEntityGraphs(value = {
#NamedEntityGraph(name = "Device.AllRelations",
attributeNodes = {
#NamedAttributeNode("transaction"),
#NamedAttributeNode(value = "deviceType", subgraph = "DeviceType.Transaction")
},
subgraphs = {
#NamedSubgraph(name = "DeviceType.Transaction", attributeNodes = {
#NamedAttributeNode("transaction")
})
}
),
#NamedEntityGraph(name = "Device.TransactionOnly", attributeNodes = {
#NamedAttributeNode("transaction")
}),
})