We Keep Coding

Spring Data findById returns cached value instead of database one

I am preparing notification system for API which I've build before.
Basically I have an aspect which listens on projectRepository.save method. What I want to achieve is check project status in an entity which is a parameter for save method with original status from database record. What I have notice is that when I search for the DB record by id it returns cached value so it is always the same as the object which is in save method even if database still have old value. Can I force Spring Data Jpa to return database record instead of cached entity?
#Aspect
#Component
#RequiredArgsConstructor
public class NotificationAspect {
private final UserService userService;
private final ProjectRepository projectRepository;
private final NotificationService notificationService;
#Pointcut("execution(* *com.stars.domain.project.ProjectRepository.save(..))")
public void projectSavePointcut() {}
#Before("projectSavePointcut()")
public void sendNotificationOnStatusChange(JoinPoint joinPoint) {
if(joinPoint.getArgs().length > 0 && joinPoint.getArgs()[0] instanceof Project) {
Project projectToUpdate = (Project) joinPoint.getArgs()[0];
Optional<Project> oldProject = projectRepository.findById(projectToUpdate.getProjectId());
if(oldProject.isPresent() && !oldProject.get().getStatus().equals(projectToUpdate.getStatus())) {
notificationService.saveNotification(
MessageFormat.format("Project: {} status has been changed from: {} to: {}",
projectToUpdate.getName(),
oldProject.get().getStatus(),
projectToUpdate.getStatus()),
List.of(userService.getUser(projectToUpdate.getCreatedBy())));
}
}
}
}
This line always returns true even if database record has different value.
oldProject.get().getStatus().equals(projectToUpdate.getStatus())

I can think of two ways.
First, if you're interested only in status field, you can create a custom native query in a repository, which will bypass EntityManager, for example like this:
#Query("SELECT p.status FROM projects p WHERE p.id = :id", nativeQuery = true)
String getProjectStatusById(#Param("id") String projectId);
Second looks like a bad idea, but it should work - you can make the entity manager's cache detach all managed entities, so it will be forced to make a DB call again.
For this inject EntityManager in your aspect bean and call its .clear() method right before calling projectRepository.findById method.

CrudRepository native query is giving the old value when i do select * after doing update

I am trying to update the RUN_STATUS using native query as below:
#Modifying
#Transactional
#Query(value = "update rerun_scheduler set RUN_STATUS=:runStatus where SCHED_NAME = :scheduleName and STEP_NAME = :stepName and MODEL_ID = :modelId and SUBMITTED_TIME = :submittedTime and START_TIME = :startTime", nativeQuery = true)
int updateManualRun(#Param("scheduleName") String scheduleName, #Param("stepName") String stepName, #Param("modelId") String modelId, #Param("submittedTime") long submittedTime, #Param("startTime") long startTime, #Param("runStatus") String runStatus);
I am able to see the value being updated in the table using Mysqlworkbench. But in my code, when i try to read the status of the job as below:
#Query(value = "Select * from rerun_scheduler where SCHED_NAME = :scheduleName and MODEL_ID=:modelId and SUBMITTED_TIME=:submittedTime AND RUN_NUMBER = :runNumber", nativeQuery = true)
RerunDTO getJobStatus(#Param("scheduleName") String scheduleName, #Param("modelId") String modelId, #Param("submittedTime") Long submittedTime, #Param("runNumber") int runNumber);
Old value in RUN_STATUS was "TO_DO" when I am updating it, I am changing it to "COMPLETE". but when I am making the Select * query, I am still getting the RUN_STATUS as "TO_DO". Do I have to do any save operation after the update? if so, what is the command to save?

You may have two different transactions:
Some method marked with #Transactional is run. This code will call getJobStatus later.
After that updateManualRun is called from another thread, so another transaction is created and committed (note, that method above is still run and another transaction is still opened)
Since transaction for getJobStatus is still opened - it can't see updates which happened after it's start.
If it's the case - you need to call getJobStatus in separate transaction each time.
If you have service
#Service
class MyService {
#Autowired
JobStatusDao dao;
#Transactional
public void checkStatus() {
Boolean isFound=false;
while (!isFound) {
isFound=dao.getJobStatus() != null;
}
}
}
You may want to create another service (note, that there is temptation to just create another method with #Transactional in the same service - but this won't work, check this https://docs.spring.io/spring/docs/3.2.4.RELEASE/spring-framework-reference/html/aop.html#aop-proxying ).
#Service
class JobStatusService{
#Autowired
JobStatusDao dao;
#Transactional
public boolean checkStatus() {
return dao.getJobStatus() != null;
}
And then you need to rewrite your original service:
#Service
class MyService {
#Autowired
JobStatusService service;
public void checkStatus() {
Boolean isFound=false;
while (!isFound) {
isFound=service.checkStatus();
}
}
}
Btw, note that querying DB in while loop will create a decent load.
You should add some delay between checking.

Spring boot how to edit entity

I am using spring boot with spring data jpa and postgre. I have "item" entity that has price, quantity, auto generated int id and order that it belongs to.
I've searched how to edit that entity changing its price and quantity only, without making new entity and the only answer I got is to get the entity from the db and set each property to the new one then save it. But if i have 6 other properties except price and quantity that means in the update method i will set a property 8 times and this seems to me like way too much boilerplate code for spring. My question is: Is there better/default way to do that?

You can provide a copy constructor:
public Item(Item item) {
this(item.price, item.quantity);
}
or use org.springframework.beans.BeanUtils method:
BeanUtils.copyProperties(sourceItem, targetItem, "id");
Then in controller:
#RestController
#RequestMapping("/items")
public class ItemController {
#Autoware
private ItemRepo repo;
#PutMapping("/{id}")
public ResponseEntity<?> update(#PathVariable("id") Item targetItem, #RequestBody Item sourceItem) {
BeanUtils.copyProperties(sourceItem, targetItem, "id");
return ResponseEntity.ok(repo.save(targetItem));
}
}

No, you don't need to set anything for 8 times. If you want to change price and quantity only, just change those two. Put it in a #Transactional method:
#Transactional
public void updateItem(Item item){
// ....
// EntityManager em;
// ....
// Get 'item' into 'managed' state
if(!em.contains(item)){
item = em.merge(item);
}
item.price = newPrice;
item.quantity = newQuantity;
// You don't even need to call save(), JPA provider/Hibernate will do it automatically.
}
This example will generate a SELECT and a UPDATE query. And that's all.

Try using #Query annotation and define your update statement
#Modifying
#Transactional
#Query("update Site site set site.name=:name where site.id=:id")
void updateJustNameById(#Param("id")Long id, #Param("name")String name);

You should use spring data rest which handles all of this by itself. you just have to call a patch request at the specified URL and provide the changed entity properties. if you have some knowledge of spring data rest have a look at https://github.com/ArslanAnjum/angularSpringApi.

Just use this #DynamicUpdate in your Entity class
#DynamicUpdate
public class Item{
}

filtering api with spring data jpa [duplicate]

I have a spring-mvc project that is using spring-data-jpa for data access. I have a domain object called Travel which I want to allow the end-user to apply a number of filters to it.
For that, I've implemented the following controller:
#Autowired
private TravelRepository travelRep;
#RequestMapping("/search")
public ModelAndView search(
#RequestParam(required= false, defaultValue="") String lastName,
Pageable pageable) {
ModelAndView mav = new ModelAndView("travels/list");
Page<Travel> travels = travelRep.findByLastNameLike("%"+lastName+"%", pageable);
PageWrapper<Travel> page = new PageWrapper<Travel>(travels, "/search");
mav.addObject("page", page);
mav.addObject("lastName", lastName);
return mav;
}
This works fine: The user has a form with a lastName input box which can be used to filter the Travels.
Beyond lastName, my Travel domain object has a lot more attributes by which I'd like to filter. I think that if these attributes were all strings then I could add them as #RequestParams and add a spring-data-jpa method to query by these. For instance I'd add a method findByLastNameLikeAndFirstNameLikeAndShipNameLike.
However, I don't know how should I do it when I need to filter for foreign keys. So my Travel has a period attribute that is a foreign key to the Period domain object, which I need to have it as a dropdown for the user to select the Period.
What I want to do is when the period is null I want to retrieve all travels filtered by the lastName and when the period is not null I want to retrieve all travels for this period filtered by the lastName.
I know that this can be done if I implement two methods in my repository and use an if to my controller:
public ModelAndView search(
#RequestParam(required= false, defaultValue="") String lastName,
#RequestParam(required= false, defaultValue=null) Period period,
Pageable pageable) {
ModelAndView mav = new ModelAndView("travels/list");
Page travels = null;
if(period==null) {
travels = travelRep.findByLastNameLike("%"+lastName+"%", pageable);
} else {
travels = travelRep.findByPeriodAndLastNameLike(period,"%"+lastName+"%", pageable);
}
mav.addObject("page", page);
mav.addObject("period", period);
mav.addObject("lastName", lastName);
return mav;
}
Is there a way to do this without using the if ? My Travel has not only the period but also other attributes that need to be filtered using dropdowns !! As you can understand, the complexity would be exponentially increased when I need to use more dropdowns because all the combinations'd need to be considered :(
Update 03/12/13: Continuing from M. Deinum's excelent answer, and after actually implementing it, I'd like to provide some comments for completeness of the question/asnwer:
Instead of implementing JpaSpecificationExecutor you should implement JpaSpecificationExecutor<Travel> to avoid type check warnings.
Please take a look at kostja's excellent answer to this question
Really dynamic JPA CriteriaBuilder
since you will need to implement this if you want to have correct filters.
The best documentation I was able to find for the Criteria API was http://www.ibm.com/developerworks/library/j-typesafejpa/. This is a rather long read but I totally recommend it - after reading it most of my questions for Root and CriteriaBuilder were answered :)
Reusing the Travel object was not possible because it contained various other objects (who also contained other objects) which I needed to search for using Like - instead I used a TravelSearch object that contained the fields I needed to search for.
Update 10/05/15: As per #priyank's request, here's how I implemented the TravelSearch object:
public class TravelSearch {
private String lastName;
private School school;
private Period period;
private String companyName;
private TravelTypeEnum travelType;
private TravelStatusEnum travelStatus;
// Setters + Getters
}
This object was used by TravelSpecification (most of the code is domain specific but I'm leaving it there as an example):
public class TravelSpecification implements Specification<Travel> {
private TravelSearch criteria;
public TravelSpecification(TravelSearch ts) {
criteria= ts;
}
#Override
public Predicate toPredicate(Root<Travel> root, CriteriaQuery<?> query,
CriteriaBuilder cb) {
Join<Travel, Candidacy> o = root.join(Travel_.candidacy);
Path<Candidacy> candidacy = root.get(Travel_.candidacy);
Path<Student> student = candidacy.get(Candidacy_.student);
Path<String> lastName = student.get(Student_.lastName);
Path<School> school = student.get(Student_.school);
Path<Period> period = candidacy.get(Candidacy_.period);
Path<TravelStatusEnum> travelStatus = root.get(Travel_.travelStatus);
Path<TravelTypeEnum> travelType = root.get(Travel_.travelType);
Path<Company> company = root.get(Travel_.company);
Path<String> companyName = company.get(Company_.name);
final List<Predicate> predicates = new ArrayList<Predicate>();
if(criteria.getSchool()!=null) {
predicates.add(cb.equal(school, criteria.getSchool()));
}
if(criteria.getCompanyName()!=null) {
predicates.add(cb.like(companyName, "%"+criteria.getCompanyName()+"%"));
}
if(criteria.getPeriod()!=null) {
predicates.add(cb.equal(period, criteria.getPeriod()));
}
if(criteria.getTravelStatus()!=null) {
predicates.add(cb.equal(travelStatus, criteria.getTravelStatus()));
}
if(criteria.getTravelType()!=null) {
predicates.add(cb.equal(travelType, criteria.getTravelType()));
}
if(criteria.getLastName()!=null ) {
predicates.add(cb.like(lastName, "%"+criteria.getLastName()+"%"));
}
return cb.and(predicates.toArray(new Predicate[predicates.size()]));
}
}
Finally, here's my search method:
#RequestMapping("/search")
public ModelAndView search(
#ModelAttribute TravelSearch travelSearch,
Pageable pageable) {
ModelAndView mav = new ModelAndView("travels/list");
TravelSpecification tspec = new TravelSpecification(travelSearch);
Page<Travel> travels = travelRep.findAll(tspec, pageable);
PageWrapper<Travel> page = new PageWrapper<Travel>(travels, "/search");
mav.addObject(travelSearch);
mav.addObject("page", page);
mav.addObject("schools", schoolRep.findAll() );
mav.addObject("periods", periodRep.findAll() );
mav.addObject("travelTypes", TravelTypeEnum.values());
mav.addObject("travelStatuses", TravelStatusEnum.values());
return mav;
}
Hope I helped!

For starters you should stop using #RequestParam and put all your search fields in an object (maybe reuse the Travel object for that). Then you have 2 options which you could use to dynamically build a query
Use the JpaSpecificationExecutor and write a Specification
Use the QueryDslPredicateExecutor and use QueryDSL to write a predicate.
Using JpaSpecificationExecutor
First add the JpaSpecificationExecutor to your TravelRepository this will give you a findAll(Specification) method and you can remove your custom finder methods.
public interface TravelRepository extends JpaRepository<Travel, Long>, JpaSpecificationExecutor<Travel> {}
Then you can create a method in your repository which uses a Specification which basically builds the query. See the Spring Data JPA documentation for this.
The only thing you need to do is create a class which implements Specification and which builds the query based on the fields which are available. The query is build using the JPA Criteria API link.
public class TravelSpecification implements Specification<Travel> {
private final Travel criteria;
public TravelSpecification(Travel criteria) {
this.criteria=criteria;
}
public Predicate toPredicate(Root<T> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
// create query/predicate here.
}
}
And finally you need to modify your controller to use the new findAll method (I took the liberty to clean it up a little).
#RequestMapping("/search")
public String search(#ModelAttribute Travel search, Pageable pageable, Model model) {
Specification<Travel> spec = new TravelSpecification(search);
Page<Travel> travels = travelRep.findAll(spec, pageable);
model.addObject("page", new PageWrapper(travels, "/search"));
return "travels/list";
}
Using QueryDslPredicateExecutor
First add the QueryDslPredicateExecutor to your TravelRepository this will give you a findAll(Predicate) method and you can remove your custom finder methods.
public interface TravelRepository extends JpaRepository<Travel, Long>, QueryDslPredicateExecutor<Travel> {}
Next you would implement a service method which would use the Travel object to build a predicate using QueryDSL.
#Service
#Transactional
public class TravelService {
private final TravelRepository travels;
public TravelService(TravelRepository travels) {
this.travels=travels;
}
public Iterable<Travel> search(Travel criteria) {
BooleanExpression predicate = QTravel.travel...
return travels.findAll(predicate);
}
}
See also this bog post.

Spring data jpa get old result under multi threading

Under multi threading, I keep getting old result from repository.
#Transactional(isolation = Isolation.REPEATABLE_READ)
public void updateScore(int score, Long userId) {
logger.info(RegularLock.getInstance().getLock().toString());
synchronized (RegularLock.getInstance().getLock()) {
Customer customer = customerDao.findOne(userId);
System.out.println("start:": customer.getScore());
customer.setScore(customer.getScore().subtract(score));
customerDao.saveAndFlush(customer);
}
}
And CustomerDao looks like
#Transactional
public T saveAndFlush(T model, Long id) {
T res = repository.saveAndFlush(model);
EntityManager manager = jpaContext.getEntityManagerByManagedType(model.getClass());
manager.refresh(manager.find(model.getClass(), id));
return res;
}
saveAndFlush() from JpaRepository is used in order to save the change instantly and the entire code is locked. But I still keep getting old result.
java.util.concurrent.locks.ReentrantLock#10a9598d[Unlocked]
start:710
java.util.concurrent.locks.ReentrantLock#10a9598d[Unlocked]
start:710
I'm using springboot with spring data jpa.
I put all code in a test controller, and the problem remains
#RestController
#RequestMapping(value = "/test", produces = "application/json")
public class TestController {
private static Long testId;
private final CustomerBalanceRepository repository;
#Autowired
public TestController(CustomerBalanceRepository repository) {
this.repository = repository;
}
#PostConstruct
public void init() {
// CustomerBalance customer = new CustomerBalance();
// repository.save(customer);
// testId = customer.getId();
}
#SystemControllerLog(description = "updateScore")
#RequestMapping(method = RequestMethod.GET)
#Transactional(isolation = Isolation.REPEATABLE_READ)
public CustomerBalance updateScore() {
CustomerBalance customerBalance = repository.findOne(70L);
System.out.println("start:" + customerBalance.getInvestFreezen());
customerBalance.setInvestFreezen(customerBalance.getInvestFreezen().subtract(new BigDecimal(5)));
saveAndFlush(customerBalance);
System.out.println("end:" + customerBalance.getInvestFreezen());
return customerBalance;
}
#Transactional
public CustomerBalance saveAndFlush(CustomerBalance customerBalance) {
return repository.saveAndFlush(customerBalance);
}
}
and the results are
start:-110.00
end:-115.00
start:-110.00
end:-115.00
start:-115.00
end:-120.00
start:-120.00
end:-125.00
start:-125.00
end:-130.00
start:-130.00
end:-135.00
start:-130.00
end:-135.00
start:-135.00
end:-140.00
start:-140.00
end:-145.00
start:-145.00
end:-150.00

I tried to reproduce the problem and failed. I put your code, with very little changes into a Controller and executed it, by requestion localhost:8080/test and could see in the logs, that the score gets reduced as expected. Note: it actually produces an exception because I don't have a view resulution configured, but that should be irrelevant.
I therefore recommend the following course of action:
Take my controller from below, add it to your code with as little changes as possible. Verify that it actually works. Then modify it step by step until it is identical with your current code. Note the change that starts producing your current behavor. This will probably make the cause really obvious. If not update the question with what you have found.
#Controller
public class CustomerController {
private static String testId;
private final CustomerRepository repository;
private final JpaContext context;
public CustomerController(CustomerRepository repository, JpaContext context) {
this.repository = repository;
this.context = context;
}
#PostConstruct
public void init() {
Customer customer = new Customer();
repository.save(customer);
testId = customer.id;
}
#RequestMapping(path = "/test")
#Transactional(isolation = Isolation.REPEATABLE_READ)
public Customer updateScore() {
Customer customer = repository.findOne(testId);
System.out.println("start:" + customer.getScore());
customer.setScore(customer.getScore() - 23);
saveAndFlush(customer);
System.out.println("end:" + customer.getScore());
return customer;
}
#Transactional
public Customer saveAndFlush(Customer customer) {
return repository.saveAndFlush(customer);
}
}
After update from OP and a little discussion we seemed to have it pinned down:
The problem occurs ONLY with multiple threads (OP used JMeter to do this thing 10times/second).
Also Transaction level serializable seemed to fix the problem.
Diagnosis
It seems to be a lost update problem, which causes effects like the following:
Thread 1: reads the customer score=10
Thread 2: reads the customer score= 10
Thread 1: updates the customer to score 10-4 =6
Thread 2: updates the customer to score 10-3 =7 // update from Thread 1 is gone.
Why isn't this prevented by the synchronization?
The problem here is most likely that the read happens before the code shown in the question, since the EntityManager is a first level cache.
How to fix it
This should get caught by optimistic locking of JPA, for this one needs a column annotated with #Version.
Transaction Level Serializable might be the better choice if this happens often.

This looks like the case for pessimistic locking. Create in your repository method findOneWithLock lke this:
import org.springframework.data.jpa.repository.Lock;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.JpaRepository;
import org.springframework.data.repository.query.Param;
import javax.persistence.LockModeType;
public interface CustomerRepository extends JpaRepository<Customer, Long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
#Query("select c from Customer c where c.id = :id")
Customer findOneWithLock(#Param("id") long id);
}
and use it to obtain db level lock which will be held till the end of transaction:
#Transactional
public void updateScore(int score, Long userId) {
Customer customer = customerDao.findOneWithLock(userId);
customer.setScore(customer.getScore().subtract(score));
}
There is no need to use application level locks like RegularLock in your code.

The problem seems to be eventhough you call,
customerDao.saveAndFlush(customer);
The Commit will not take place until the end of the method is reached and a commit is made because your code inside of a
#Transactional(isolation = Isolation.REPEATABLE_READ)
What you can do is either change the propagation of the Transactional to Propagation.REQUIRES_NEW like below.
#Transactional(isolation = Isolation.REPEATABLE_READ, propagation=Propagation.REQUIRES_NEW)
This will result in a New Transaction being created and committed at the end of the method. And at the end of Transaction the changes will be committed.

With this line
manager.refresh(manager.find(model.getClass(), id));
you are telling JPA to undo all your changes. From the documentation of the refresh method
Refresh the state of the instance from the database, overwriting changes made to the entity, if any.
Remove it and your code should run as expected.