We Keep Coding

How to use Mongo Auditing and a UUID as id with Spring Boot 2.2.x?

I would like to have Documents stored with an UUID id and createdAt / updatedAt fields. My solution was working with Spring Boot 2.1.x. After I upgraded from Spring Boot 2.1.11.RELEASE to 2.2.0.RELEASE my test for MongoAuditing failed with createdAt = null. What do I need to do to get the createdAt field filled again?
This is not just a testproblem. I ran the application and it has the same behaviour as my test. All auditing fields stay null.
I have a Configuration to enable MongoAuditing and UUID generation:
#Configuration
#EnableMongoAuditing
public class MongoConfiguration {
#Bean
public GenerateUUIDListener generateUUIDListener() {
return new GenerateUUIDListener();
}
}
The listner hooks into the onBeforeConvert - I guess thats where the trouble starts.
public class GenerateUUIDListener extends AbstractMongoEventListener<IdentifiableEntity> {
#Override
public void onBeforeConvert(BeforeConvertEvent<IdentifiableEntity> event) {
IdentifiableEntity entity = event.getSource();
if (entity.isNew()) {
entity.setId(UUID.randomUUID());
}
}
}
The document itself (I dropped the getter and setters):
#Document
public class MyDocument extends InsertableEntity {
private String name;
}
public abstract class InsertableEntity extends IdentifiableEntity {
#CreatedDate
#JsonIgnore
private Instant createdAt;
}
public abstract class IdentifiableEntity implements Persistable<UUID> {
#Id
private UUID id;
#JsonIgnore
public boolean isNew() {
return getId() == null;
}
}
A complete minimal example can be find here (including a test) https://github.com/mab/auditable
With 2.1.11.RELEASE the test succeeds with 2.2.0.RELEASE it fails.

For me the best solution was to switch from event UUID generation to a callback based one. With the implementation of Ordered we can set the new callback to be executed after the AuditingEntityCallback.
public class IdEntityCallback implements BeforeConvertCallback<IdentifiableEntity>, Ordered {
#Override
public IdentifiableEntity onBeforeConvert(IdentifiableEntity entity, String collection) {
if (entity.isNew()) {
entity.setId(UUID.randomUUID());
}
return entity;
}
#Override
public int getOrder() {
return 101;
}
}
I registered the callback with the MongoConfiguration. For a more general solution you might want to take a look at the registration of the AuditingEntityCallback with the `MongoAuditingBeanDefinitionParser.
#Configuration
#EnableMongoAuditing
public class MongoConfiguration {
#Bean
public IdEntityCallback registerCallback() {
return new IdEntityCallback();
}
}

MongoTemplate works in the following way on doInsert()
this.maybeEmitEvent - emit an event (onBeforeConvert, onBeforeSave and such) so any AbstractMappingEventListener can catch and act upon like you did with GenerateUUIDListener
this.maybeCallBeforeConvert - call before convert callbacks like mongo auditing
like you can see in source code of MongoTemplate.class src (831-832)
protected <T> T doInsert(String collectionName, T objectToSave, MongoWriter<T> writer) {
BeforeConvertEvent<T> event = new BeforeConvertEvent(objectToSave, collectionName);
T toConvert = ((BeforeConvertEvent)this.maybeEmitEvent(event)).getSource(); //emit event
toConvert = this.maybeCallBeforeConvert(toConvert, collectionName); //call some before convert handlers
...
}
MongoAudit marks createdAt only to new entities by checking if entity.isNew() == true
because your code (UUID) already set the Id the createdAt is not populated (the entity is not considered new)
you can do the following (order by best to worst):
forget about the UUID and use String for your id, let the mongo itself create and manage it's entities ids (this how MongoTemplate actually works lines 811-812)
keep the UUID at the code level, convert from/to String when inserting and retrieving from the db
create a custom repository like in this post
stay with 2.1.11.RELEASE
set the updateAt by GenerateUUIDListener as well as id (rename it NewEntityListener or smth), basically implement the audit
implement a new isNew() logic that don't depends only on the entity id
in version 2.1.11.RELEASE the order of the methods was flipped (MongoTemplate.class 804-805) so your code worked fine
as an abstract approach, the nature of event is to be sort of send-and-forget (async compatible), so it's a very bad practice to change the object itself, there is NO grantee for order of computation, if any
this is why the audit build on callbacks and not events, and that's why Pivotal don't (need to) keep order between versions

Spring Data: default 'not deleted' logic for automatic method-based queries when using soft-delete policy

Let's say we use soft-delete policy: nothing gets deleted from the storage; instead, a 'deleted' attribute/column is set to true on a record/document/whatever to make it 'deleted'. Later, only non-deleted entries should be returned by query methods.
Let's take MongoDB as an example (alghough JPA is also interesting).
For standard methods defined by MongoRepository, we can extend the default implementation (SimpleMongoRepository), override the methods of interest and make them ignore 'deleted' documents.
But, of course, we'd also like to use custom query methods like
List<Person> findByFirstName(String firstName)
In a soft-delete environment, we are forced to do something iike
List<person> findByFirstNameAndDeletedIsFalse(String firstName)
or write queries manually with #Query (adding the same boilerplate condition about 'not deleted' all the time).
Here comes the question: is it possible to add this 'non-deleted' condition to any generated query automatically? I did not find anything in the documentation.
I'm looking at Spring Data (Mongo and JPA) 2.1.6.
Similar questions
Query interceptor for spring-data-mongodb for soft deletions here they suggest Hibernate's #Where annotation which only works for JPA+Hibernate, and it is not clear how to override it if you still need to access deleted items in some queries
Handling soft-deletes with Spring JPA here people either suggest the same #Where-based approach, or the solution applicability is limited with the already-defined standard methods, not the custom ones.

It turns out that for Mongo (at least, for spring-data-mongo 2.1.6) we can hack into standard QueryLookupStrategy implementation to add the desired 'soft-deleted documents are not visible by finders' behavior:
public class SoftDeleteMongoQueryLookupStrategy implements QueryLookupStrategy {
private final QueryLookupStrategy strategy;
private final MongoOperations mongoOperations;
public SoftDeleteMongoQueryLookupStrategy(QueryLookupStrategy strategy,
MongoOperations mongoOperations) {
this.strategy = strategy;
this.mongoOperations = mongoOperations;
}
#Override
public RepositoryQuery resolveQuery(Method method, RepositoryMetadata metadata, ProjectionFactory factory,
NamedQueries namedQueries) {
RepositoryQuery repositoryQuery = strategy.resolveQuery(method, metadata, factory, namedQueries);
// revert to the standard behavior if requested
if (method.getAnnotation(SeesSoftlyDeletedRecords.class) != null) {
return repositoryQuery;
}
if (!(repositoryQuery instanceof PartTreeMongoQuery)) {
return repositoryQuery;
}
PartTreeMongoQuery partTreeQuery = (PartTreeMongoQuery) repositoryQuery;
return new SoftDeletePartTreeMongoQuery(partTreeQuery);
}
private Criteria notDeleted() {
return new Criteria().orOperator(
where("deleted").exists(false),
where("deleted").is(false)
);
}
private class SoftDeletePartTreeMongoQuery extends PartTreeMongoQuery {
SoftDeletePartTreeMongoQuery(PartTreeMongoQuery partTreeQuery) {
super(partTreeQuery.getQueryMethod(), mongoOperations);
}
#Override
protected Query createQuery(ConvertingParameterAccessor accessor) {
Query query = super.createQuery(accessor);
return withNotDeleted(query);
}
#Override
protected Query createCountQuery(ConvertingParameterAccessor accessor) {
Query query = super.createCountQuery(accessor);
return withNotDeleted(query);
}
private Query withNotDeleted(Query query) {
return query.addCriteria(notDeleted());
}
}
}
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.METHOD)
public #interface SeesSoftlyDeletedRecords {
}
We just add 'and not deleted' condition to all the queries unless #SeesSoftlyDeletedRecords asks as to avoid it.
Then, we need the following infrastructure to plug our QueryLiikupStrategy implementation:
public class SoftDeleteMongoRepositoryFactory extends MongoRepositoryFactory {
private final MongoOperations mongoOperations;
public SoftDeleteMongoRepositoryFactory(MongoOperations mongoOperations) {
super(mongoOperations);
this.mongoOperations = mongoOperations;
}
#Override
protected Optional<QueryLookupStrategy> getQueryLookupStrategy(QueryLookupStrategy.Key key,
QueryMethodEvaluationContextProvider evaluationContextProvider) {
Optional<QueryLookupStrategy> optStrategy = super.getQueryLookupStrategy(key,
evaluationContextProvider);
return optStrategy.map(this::createSoftDeleteQueryLookupStrategy);
}
private SoftDeleteMongoQueryLookupStrategy createSoftDeleteQueryLookupStrategy(QueryLookupStrategy strategy) {
return new SoftDeleteMongoQueryLookupStrategy(strategy, mongoOperations);
}
}
public class SoftDeleteMongoRepositoryFactoryBean<T extends Repository<S, ID>, S, ID extends Serializable>
extends MongoRepositoryFactoryBean<T, S, ID> {
public SoftDeleteMongoRepositoryFactoryBean(Class<? extends T> repositoryInterface) {
super(repositoryInterface);
}
#Override
protected RepositoryFactorySupport getFactoryInstance(MongoOperations operations) {
return new SoftDeleteMongoRepositoryFactory(operations);
}
}
Then we just need to reference the factory bean in a #EnableMongoRepositories annotation like this:
#EnableMongoRepositories(repositoryFactoryBeanClass = SoftDeleteMongoRepositoryFactoryBean.class)
If it is required to determine dynamically whether a particular repository needs to be 'soft-delete' or a regular 'hard-delete' repository, we can introspect the repository interface (or the domain class) and decide whether we need to change the QueryLookupStrategy or not.
As for JPA, this approach does not work without rewriting (possibly duplicating) a substantial part of the code in PartTreeJpaQuery.

Using transactions when combining a RepositoryRestResource with another repository

I have a simple JpaRepository annotated with #RepositoryRestResource:
#RepositoryRestResource
public interface ItemRepository extends JpaRepository<Item, UUID> { }
Whenever something is changed in the database, I want to update a file. I do this using a RepositoryEventHandler:
#Component
#RepositoryEventHandler
public class ItemRepositoryEventHandler {
#HandleAfterCreate
#HandleAfterSave
#HandleAfterDelete
public void itemChanged(Item item) {
writeToFile();
}
}
What I want to do is if there is an error while writing the contents to file, then the database should be rolled back.
I've tried by adding the #Transactional annotation to the ItemRepository but it didn't work. Debugging revealed that the RepositoryRestResource does three steps: emitting the BeforeXXX events, persisting to the database, then emitting the AfterXXX events. It only uses a transaction during the persistence step, not one across all three.
So I see no way to use a transaction across the whole operation and the only alternative I see is to not use #RepositoryRestResource, but to implement the web layer manually then use a service which employs a transaction across both repositories. Is there an easier way?

One approach is to implement your business logic with a custom controller and a service. But this way is neutralizing the 'advantages' of Spring Data REST.
Another option (in my opinion it's more natural for SDR) is to use published events from aggregate roots. In this case you should extend your entities from AbstractAggregateRoot and implement a method that will be publish some 'event'. Then you will be able to handle this event (with help of #EventListener) in the same transaction during the process of saving your entity. For example:
#Entity
public class Order extends AbstractAggregateRoot {
//...
public void registerItems(List<Item> items) {
this.registerEvent(new RegisterItemsEvent(this, items));
}
}
#Getter
#RequiredArgsConstructor
public class RegisterItemsEvent {
private final Order order;
private final List<Item> items;
}
#RequiredArgsConstructor
#Component
public class EventHandler {
private final ItemRepo itemRepo;
#EventListener
#Transactional(propagation = MANDATORY)
public void handleRegisterItemsEvent(RegisterItemsEvent e) {
Order order = e.getOrder();
List<Item> items = e.getItems();
// update items with order - skipped...
itemRepo.saveAll(items);
}
}
Usage example:
#Component
#RepositoryEventHandler
public class OrderEventHandler {
#BeforeCreate
public void handleOrderCreate(Order order) {
// prepare a List of items - skipped...
order.registerItems(items);
}
}
When SDR saves the Order then it emits RegisterItemsEvent, which is handled in handleRegisterItemsEvent method of your EventHandler that saves prepared items in the same transaction (we use propagation = MANDATORY parameter of #Transaction annotation to make sure that transaction is present).
Additional info: Domain event publication from aggregate roots
UPDATED
Regarding your particular task, you can create class ItemChangedEvent:
#Getter
#RequiredArgsConstructor
public class ItemChangedEvent {
private final Item item;
}
Implement method markAsChanged in the Item entity:
#Entity
public class Item extends AbstractAggregateRoot {
//...
public void markAsChanged() {
this.registerEvent(new ItemChangedEvent(this));
}
}
When the item changes, you will mark it as "changed":
#Component
#RepositoryEventHandler
public class ItemRepositoryEventHandler {
#BeforeCreate
#BeforeSave
#BeforeDelete
public void itemChanged(Item item) {
item.markAsChanged();
}
}
And write it to a file in the ItemChangedEvent handler in the same transaction:
#Component
public class EventHandler {
#EventListener
#Transactional(propagation = MANDATORY)
public void handleItemChangedEvent(ItemChangedEvent e) {
Item item = e.getItem();
writeToFile(item);
}
}

How to add global where clause for all find methods of Spring data JPA with Hibernate?

We are working on web application using Spring data JPA with hibernate.
In the application there is a field of compid in each entity.
Which means in every DB call (Spring Data methods) will have to be checked with the compid.
I need a way that, this "where compid = ?" check to be injected automatically for every find method.
So that we won't have to specifically bother about compid checks.
Is this possible to achieve from Spring Data JPA framework?

Maybe Hibernate‘s annotation #Where will help you. It adds the passed condition to any JPA queries related to the entity. For example
#Entity
#Where(clause = "isDeleted='false'")
public class Customer {
//...
#Column
private Boolean isDeleted;
}
More info: 1, 2

Agree with Abhijit Sarkar.
You can achieve your goal hibernate listeners and aspects. I can suggest the following :
create an annotation #Compable (or whatever you call it) to mark service methods
create CompAspect which should be a bean and #Aspect. It should have something like this
#Around("#annotation(compable)")`
public Object enableClientFilter(ProceedingJoinPoint pjp, Compable compable) throws Throwable {
Session session = (Session) em.getDelegate();
try {
if (session.isOpen()) {
session.enableFilter("compid_filter_name")
.setParameter("comp_id", your_comp_id);
}
return pjp.proceed();
} finally {
if (session.isOpen()) {
session.disableFilter("filter_name");
}
}
}
em - EntityManager
3)Also you need to provide hibernate filters. If you use annotation this can look like this:
#FilterDef(name="compid_filter_name", parameters=#ParamDef(name="comp_id", type="java.util.Long"))
#Filters(#Filter(name="compid_filter_name", condition="comp_id=:comp_id"))
So your condition where compid = ? will be #Service method below
#Compable
someServicweMethod(){
List<YourEntity> l = someRepository.findAllWithNamesLike("test");
}
That's basically it for Selects,
For updates/deletes this scheme requires an EntityListener.

Like other people have said there is no set method for this
One option is to look at Query by example - from the spring data documentation -
Person person = new Person();
person.setFirstname("Dave");
Example<Person> example = Example.of(person);
So you could default compid in the object, or parent JPA object
Another option is a custom repository

I can contribute a 50% solution. 50% because it seems to be not easy to wrap Query Methods. Also custom JPA queries are an issue for this global approach. If the standard finders are sufficient it is possible to extend an own SimpleJpaRepository:
public class CustomJpaRepositoryIml<T, ID extends Serializable> extends
SimpleJpaRepository<T, ID> {
private JpaEntityInformation<T, ?> entityInformation;
#Autowired
public CustomJpaRepositoryIml(JpaEntityInformation<T, ?> entityInformation,
EntityManager entityManager) {
super(entityInformation, entityManager);
this.entityInformation = entityInformation;
}
private Sort applyDefaultOrder(Sort sort) {
if (sort == null) {
return null;
}
if (sort.isUnsorted()) {
return Sort.by("insert whatever is a default").ascending();
}
return sort;
}
private Pageable applyDefaultOrder(Pageable pageable) {
if (pageable.getSort().isUnsorted()) {
Sort defaultSort = Sort.by("insert whatever is a default").ascending();
pageable = PageRequest.of(pageable.getPageNumber(), pageable.getPageSize(), defaultSort);
}
return pageable;
}
#Override
public Optional<T> findById(ID id) {
Specification<T> filterSpec = filterOperatorUserAccess();
if (filterSpec == null) {
return super.findById(id);
}
return findOne(filterSpec.and((Specification<T>) (root, query, criteriaBuilder) -> {
Path<?> path = root.get(entityInformation.getIdAttribute());
return criteriaBuilder.equal(path, id);
}));
}
#Override
protected <S extends T> TypedQuery<S> getQuery(Specification<S> spec, Class<S> domainClass, Sort sort) {
sort = applyDefaultOrder(sort);
Specification<T> filterSpec = filterOperatorUserAccess();
if (filterSpec != null) {
spec = (Specification<S>) filterSpec.and((Specification<T>) spec);
}
return super.getQuery(spec, domainClass, sort);
}
}
This implementation is picked up e.g. by adding it to the Spring Boot:
#SpringBootApplication
#EnableJpaRepositories(repositoryBaseClass = CustomJpaRepositoryIml.class)
public class ServerStart {
...
If you need this kind of filtering also for Querydsl it is also possible to implement and register a QuerydslPredicateExecutor.

Java Spring REST API Handling Many Optional Parameters

I'm currently messing around with a Spring Boot REST API project for instructional purposes. I have a rather large table with 22 columns loaded into a MySQL database and am trying to give the user the ability to filter the results by multiple columns (let's say 6 for the purposes of this example).
I am currently extending a Repository and have initialized methods such as findByParam1 and findByParam2 and findByParam1OrderByParam2Desc and etc. and have verified that they are working as intended. My question to you guys is the best way to approach allowing the user the ability to leverage all 6 optional RequestParams without writing a ridiculous amount of conditionals/repository method variants. For example, I want to give the user the ability to hit url home/get-data/ to get all results, home/get-data?param1=xx to filter based on param1, and potentially, home/get-data?param1=xx&param2=yy...&param6=zz to filter on all the optional parameters.
For reference, here is what the relevant chunk of my controller looks like (roughly).
#RequestMapping(value = "/get-data", method = RequestMethod.GET)
public List<SomeEntity> getData(#RequestParam Map<String, String> params) {
String p1 = params.get("param1");
if(p1 != null) {
return this.someRepository.findByParam1(p1);
}
return this.someRepository.findAll();
}
My issue so far is that the way I am proceeding about this means that I will basically need n! amount of methods in my repository to support this functionality with n equalling the amount of fields/columns I want to filter on. Is there a better way to approach handling this, perhaps where I am filtering the repository 'in-place' so I can simply filter 'in-place' as I check the Map to see what filters the user did indeed populate?
EDIT: So I'm currently implementing a 'hacky' solution that might be related to J. West's comment below. I assume that the user will be specifying all n parameters in the request URL and if they do not (for example, they specify p1-p4 but not p5 and p6) I generate SQL that just matches the statement to LIKE '%' for the non-included params. It would look something like...
#Query("select u from User u where u.p1 = :p1 and u.p2 = :p2 ... and u.p6 = :p6")
List<User> findWithComplicatedQueryAndSuch;
and in the Controller, I would detect if p5 and p6 were null in the Map and if so, simply change them to the String '%'. I'm sure there is a more precise and intuitive way to do this, although I haven't been able to find anything of the sort yet.

You can do this easily with a JpaSpecificationExecutor and a custom Specification: https://spring.io/blog/2011/04/26/advanced-spring-data-jpa-specifications-and-querydsl/
I would replace the HashMap with a DTO containing all optional get params, then build the specifications based on that DTO, obviously you can also keep the HashMap and build the specification based on it.
Basically:
public class VehicleFilter implements Specification<Vehicle>
{
private String art;
private String userId;
private String vehicle;
private String identifier;
#Override
public Predicate toPredicate(Root<Vehicle> root, CriteriaQuery<?> query, CriteriaBuilder cb)
{
ArrayList<Predicate> predicates = new ArrayList<>();
if (StringUtils.isNotBlank(art))
{
predicates.add(cb.equal(root.get("art"), art));
}
if (StringUtils.isNotBlank(userId))
{
predicates.add(cb.equal(root.get("userId"), userId));
}
if (StringUtils.isNotBlank(vehicle))
{
predicates.add(cb.equal(root.get("vehicle"), vehicle));
}
if (StringUtils.isNotBlank(identifier))
{
predicates.add(cb.equal(root.get("identifier"), fab));
}
return predicates.size() <= 0 ? null : cb.and(predicates.toArray(new Predicate[predicates.size()]));
}
// getter & setter
}
And the controller:
#RequestMapping(value = "/{ticket}/count", method = RequestMethod.GET)
public long getItemsCount(
#PathVariable String ticket,
VehicleFilter filter,
HttpServletRequest request
) throws Exception
{
return vehicleService.getCount(filter);
}
Service:
#Override
public long getCount(VehicleFilter filter)
{
return vehicleRepository.count(filter);
}
Repository:
#Repository
public interface VehicleRepository extends JpaRepository<Vehicle, Integer>, JpaSpecificationExecutor<Vehicle>
{
}
Just a quick example adapted from company code, you get the idea!

Another solution with less coding would be to use QueryDsl integration with Spring MVC.
By using this approach all your request parameters will be automatically resolved to one of your domain properties and appended to your query.
For reference check the documentation https://spring.io/blog/2015/09/04/what-s-new-in-spring-data-release-gosling#querydsl-web-support and the example project https://github.com/spring-projects/spring-data-examples/tree/master/web/querydsl

You can do it even more easily using Query By Example (QBE) technique if your repository class implements JpaRepository interface as that interface implements QueryByExampleExecutor interface which provides findAll method that takes object of Example<T> as an argument.
Using this approach is really applicable for your scenario as your entity has a lot of fields and you want user to be able to get those which are matching filter represented as subset of entity's fields with their corresponding values that have to be matched.
Let's say the entity is User (like in your example) and you want to create endpoint for fetching users whose attribute values are equal to the ones which are specified. That could be accomplished with the following code:
Entity class:
#Entity
public class User implements Serializable {
private Long id;
private String firstName;
private String lastName;
private Integer age;
private String city;
private String state;
private String zipCode;
}
Controller class:
#Controller
public class UserController {
private UserRepository repository;
private UserController(UserRepository repository) {
this.repository = repository;
}
#GetMapping
public List<User> getMatchingUsers(#RequestBody User userFilter) {
return repository.findAll(Example.of(userFilter));
}
}
Repository class:
#Repository
public class UserRepository implements JpaRepository<User, Integer> {
}