Spring JPA could not handle multiple transactions in one thread properly - java

I have the following code fragment. After I invoke BatchSettleService.batchSettleWork(Array.asList([1,2,3])). I found that the account balance only reduce 1 in DB. The debug result is every time accountRepository.findByIdForUpdate(2) returns the original Account without any change operated in the past loops. I have tried Isolation.SERIALIZABLE level but the result is the same. The data base I am using is MySQL 5.7.20 InnoDb Engine. The JPA implemention is Hibernate. I am expecting the account balance to be reduced 3. Is there something wrong with my understanding of transaction? Thank you in advance!
#Service
public class BatchSettleService {
private Logger logger = LoggerFactory.getLogger(getClass());
#Autowired
private WorkSettleService workSettleService;
public List<WorkSettleResponse> batchSettleWork(List<Long> workIds) {
List<WorkSettleResponse> results= new ArrayList<>();
for(Long workId:workIds) {
try {
results.add(workSettleService.settleWork(new WorkSettleRequest(workId)));
} catch (WrappedException e) {
results.add(new WorkSettleResponse(workId,e.getErrCode(),e.getMessage()));
logger.error("Settle work failed for {}",workId,e);
}
}
return results;
}
}
public class WorkSettleService{
#Autowired
private AccountRepository accountRepository;
#Transactional(rollbackFor= {WorkSettleException.class,RuntimeException.class},propagation=Propagation.REQUIRES_NEW)
public WorkSettleResponse settleWork(WorkSettleRequest req){
Account account = accountRepository.findByIdForUpdate(2);
Integer balance = account .getBalance();
accountRepository.updateBalanceById(account.getId(),balance-1)
}
}
public interface AccountRepository extends Repository<Account, Integer> {
public Account findById(Integer id);
#Lock(LockModeType.PESSIMISTIC_WRITE)
#Query("select ac from Account a where a.id = ?1")
public Account findByIdForUpdate(Integer id);
#Modifying
#Query("update Account a set a.balance = ?2 where a.id = ?1")
public int updateBalanceById(Integer id,Integer balance);
}

This case has been solved.
Using account.setBalance(balance-1) instead of accountRepository.updateBalanceById(account.getId(),balance-1).
I don't know why #Modifying annoation doesn't update the EM cache while #Query fetch data from EM cache, a bit strage for Spring JPA implemention.

Related

check if entity exist and then delete it in one transaction in spring app

i have a services layer and a repository layer in my spring boot application (i use also spring data, mvc etc)
before deleting an entity from the database, I want to check if such an entity exists and if not, then throw an EntityNotFoundException
for example my repository:
public interface RoomRepository extends CrudRepository<Room, Long> {
#Query("from Room r left join fetch r.messages where r.id = :rId")
Optional<Room> findByIdWithMessages(#Param("rId") long id);
#Override
List<Room> findAll();
}
and service:
#Service
#Loggable
public class RoomService implements GenericService<Room> {
private final RoomRepository roomRepository;
private final RoomDtoMapper roomMapper;
public RoomService(RoomRepository roomRepository, RoomDtoMapper roomMapper) {
this.roomRepository = roomRepository;
this.roomMapper = roomMapper;
}
#Override
public Room getById(long id) {
return roomRepository.findById(id).orElseThrow(
() -> new EntityNotFoundException(String.format("room with id = %d wasn't found", id)));
}
#Override
public void delete(Room room) {
getById(room.getId());
roomRepository.delete(room);
}
}
In this example in the delete method, I call the
getById(room.getId())
(so that it throws an EntityNotFoundException if the entity does not exist.)
before
roomRepository.delete(room);
it seems to me that such code is not thread-safe and the operation is not atomic
(because at the moment when in this thread at the moment of checking another request from another thread may already delete the same entity)
and I don't know if I'm doing the right thing
maybe i should add the #Transactional annotation?
would it allow me to make the method atomic?
like this:
#Override
#Transactional
public void delete(Room room) {
getById(room.getId());
roomRepository.delete(room);
}
maybe i should set some kind of isolation level?
you can test if your object needed, exist or not by autowiring the repository injected (in your case is RoomRepository e.g) and (insted User in my exmaple you can use Room): for example:
public ResponseEntity<Object> deletUserById(Long id) {
if (userrRepository.findById(id).isPresent()) {
userrRepository.deleteById(id);
return ResponseEntity.ok().body("User deleted with success");
} else {
return ResponseEntity.unprocessableEntity().body("user to be deleted not exist");
}
}

CrudRepository native query is giving the old value when i do select * after doing update

I am trying to update the RUN_STATUS using native query as below:
#Modifying
#Transactional
#Query(value = "update rerun_scheduler set RUN_STATUS=:runStatus where SCHED_NAME = :scheduleName and STEP_NAME = :stepName and MODEL_ID = :modelId and SUBMITTED_TIME = :submittedTime and START_TIME = :startTime", nativeQuery = true)
int updateManualRun(#Param("scheduleName") String scheduleName, #Param("stepName") String stepName, #Param("modelId") String modelId, #Param("submittedTime") long submittedTime, #Param("startTime") long startTime, #Param("runStatus") String runStatus);
I am able to see the value being updated in the table using Mysqlworkbench. But in my code, when i try to read the status of the job as below:
#Query(value = "Select * from rerun_scheduler where SCHED_NAME = :scheduleName and MODEL_ID=:modelId and SUBMITTED_TIME=:submittedTime AND RUN_NUMBER = :runNumber", nativeQuery = true)
RerunDTO getJobStatus(#Param("scheduleName") String scheduleName, #Param("modelId") String modelId, #Param("submittedTime") Long submittedTime, #Param("runNumber") int runNumber);
Old value in RUN_STATUS was "TO_DO" when I am updating it, I am changing it to "COMPLETE". but when I am making the Select * query, I am still getting the RUN_STATUS as "TO_DO". Do I have to do any save operation after the update? if so, what is the command to save?
You may have two different transactions:
Some method marked with #Transactional is run. This code will call getJobStatus later.
After that updateManualRun is called from another thread, so another transaction is created and committed (note, that method above is still run and another transaction is still opened)
Since transaction for getJobStatus is still opened - it can't see updates which happened after it's start.
If it's the case - you need to call getJobStatus in separate transaction each time.
If you have service
#Service
class MyService {
#Autowired
JobStatusDao dao;
#Transactional
public void checkStatus() {
Boolean isFound=false;
while (!isFound) {
isFound=dao.getJobStatus() != null;
}
}
}
You may want to create another service (note, that there is temptation to just create another method with #Transactional in the same service - but this won't work, check this https://docs.spring.io/spring/docs/3.2.4.RELEASE/spring-framework-reference/html/aop.html#aop-proxying ).
#Service
class JobStatusService{
#Autowired
JobStatusDao dao;
#Transactional
public boolean checkStatus() {
return dao.getJobStatus() != null;
}
And then you need to rewrite your original service:
#Service
class MyService {
#Autowired
JobStatusService service;
public void checkStatus() {
Boolean isFound=false;
while (!isFound) {
isFound=service.checkStatus();
}
}
}
Btw, note that querying DB in while loop will create a decent load.
You should add some delay between checking.

How to add global where clause for all find methods of Spring data JPA with Hibernate?

We are working on web application using Spring data JPA with hibernate.
In the application there is a field of compid in each entity.
Which means in every DB call (Spring Data methods) will have to be checked with the compid.
I need a way that, this "where compid = ?" check to be injected automatically for every find method.
So that we won't have to specifically bother about compid checks.
Is this possible to achieve from Spring Data JPA framework?
Maybe Hibernateā€˜s annotation #Where will help you. It adds the passed condition to any JPA queries related to the entity. For example
#Entity
#Where(clause = "isDeleted='false'")
public class Customer {
//...
#Column
private Boolean isDeleted;
}
More info: 1, 2
Agree with Abhijit Sarkar.
You can achieve your goal hibernate listeners and aspects. I can suggest the following :
create an annotation #Compable (or whatever you call it) to mark service methods
create CompAspect which should be a bean and #Aspect. It should have something like this
#Around("#annotation(compable)")`
public Object enableClientFilter(ProceedingJoinPoint pjp, Compable compable) throws Throwable {
Session session = (Session) em.getDelegate();
try {
if (session.isOpen()) {
session.enableFilter("compid_filter_name")
.setParameter("comp_id", your_comp_id);
}
return pjp.proceed();
} finally {
if (session.isOpen()) {
session.disableFilter("filter_name");
}
}
}
em - EntityManager
3)Also you need to provide hibernate filters. If you use annotation this can look like this:
#FilterDef(name="compid_filter_name", parameters=#ParamDef(name="comp_id", type="java.util.Long"))
#Filters(#Filter(name="compid_filter_name", condition="comp_id=:comp_id"))
So your condition where compid = ? will be #Service method below
#Compable
someServicweMethod(){
List<YourEntity> l = someRepository.findAllWithNamesLike("test");
}
That's basically it for Selects,
For updates/deletes this scheme requires an EntityListener.
Like other people have said there is no set method for this
One option is to look at Query by example - from the spring data documentation -
Person person = new Person();
person.setFirstname("Dave");
Example<Person> example = Example.of(person);
So you could default compid in the object, or parent JPA object
Another option is a custom repository
I can contribute a 50% solution. 50% because it seems to be not easy to wrap Query Methods. Also custom JPA queries are an issue for this global approach. If the standard finders are sufficient it is possible to extend an own SimpleJpaRepository:
public class CustomJpaRepositoryIml<T, ID extends Serializable> extends
SimpleJpaRepository<T, ID> {
private JpaEntityInformation<T, ?> entityInformation;
#Autowired
public CustomJpaRepositoryIml(JpaEntityInformation<T, ?> entityInformation,
EntityManager entityManager) {
super(entityInformation, entityManager);
this.entityInformation = entityInformation;
}
private Sort applyDefaultOrder(Sort sort) {
if (sort == null) {
return null;
}
if (sort.isUnsorted()) {
return Sort.by("insert whatever is a default").ascending();
}
return sort;
}
private Pageable applyDefaultOrder(Pageable pageable) {
if (pageable.getSort().isUnsorted()) {
Sort defaultSort = Sort.by("insert whatever is a default").ascending();
pageable = PageRequest.of(pageable.getPageNumber(), pageable.getPageSize(), defaultSort);
}
return pageable;
}
#Override
public Optional<T> findById(ID id) {
Specification<T> filterSpec = filterOperatorUserAccess();
if (filterSpec == null) {
return super.findById(id);
}
return findOne(filterSpec.and((Specification<T>) (root, query, criteriaBuilder) -> {
Path<?> path = root.get(entityInformation.getIdAttribute());
return criteriaBuilder.equal(path, id);
}));
}
#Override
protected <S extends T> TypedQuery<S> getQuery(Specification<S> spec, Class<S> domainClass, Sort sort) {
sort = applyDefaultOrder(sort);
Specification<T> filterSpec = filterOperatorUserAccess();
if (filterSpec != null) {
spec = (Specification<S>) filterSpec.and((Specification<T>) spec);
}
return super.getQuery(spec, domainClass, sort);
}
}
This implementation is picked up e.g. by adding it to the Spring Boot:
#SpringBootApplication
#EnableJpaRepositories(repositoryBaseClass = CustomJpaRepositoryIml.class)
public class ServerStart {
...
If you need this kind of filtering also for Querydsl it is also possible to implement and register a QuerydslPredicateExecutor.

Spring Boot - html table shows expired data

I do have Spring boot project, and I am using JPA.
One of my controllers returns a page 'myclass.html', with a table on it.
#Secured("ROLE_USER")
#Controller
public class UserController {
#RequestMapping("/myClass")
public String myClass(Model model) {
model.addAttribute("students", studentService.findCurrentStudents());
return "user/myclass";
}
}
The studentService does 2 things (by calling 2 separate methods on the UserRepository):
It updates the STUDENTS table to make some changes depending on the actual date.
It returns back the STUDENTS table
Even the update is defined earlier, and then the list return, when I test it, the table contains the old data (the one before the update), and if I refresh the window, after the refresh it will then show the modified data finally.
Do I need to encapsulate them into a single transaction to keep the order of the execution? If I must, how can I do it in the most simplest way?
I copy the actual part of the mentioned service:
#Override
public List<Student> findCurrentStudents() {
studentRepository.updateByUserAndDate(currentUserId(), new Date());
return studentRepository.findCurrentStudentsByUser(currentUserId());
}
And the repository:
#Repository
public interface StudentRepository extends CrudRepository<Student, Long> {
#Query(value="SELECT * FROM Students WHERE user_id = :currentUserId, nativeQuery = true)
List<Student> findCurrentStudentsByUser(#Param("userId") Integer currentUserId);
#Transactional
#Modifying
#Query(value="UPDATE Students SET last_review = :newDate WHERE user_id = :userId", nativeQuery = true)
void updateByUserAndDate(#Param("userId")Long id, #Param("newDate") Date date);

Spring data jpa get old result under multi threading

Under multi threading, I keep getting old result from repository.
#Transactional(isolation = Isolation.REPEATABLE_READ)
public void updateScore(int score, Long userId) {
logger.info(RegularLock.getInstance().getLock().toString());
synchronized (RegularLock.getInstance().getLock()) {
Customer customer = customerDao.findOne(userId);
System.out.println("start:": customer.getScore());
customer.setScore(customer.getScore().subtract(score));
customerDao.saveAndFlush(customer);
}
}
And CustomerDao looks like
#Transactional
public T saveAndFlush(T model, Long id) {
T res = repository.saveAndFlush(model);
EntityManager manager = jpaContext.getEntityManagerByManagedType(model.getClass());
manager.refresh(manager.find(model.getClass(), id));
return res;
}
saveAndFlush() from JpaRepository is used in order to save the change instantly and the entire code is locked. But I still keep getting old result.
java.util.concurrent.locks.ReentrantLock#10a9598d[Unlocked]
start:710
java.util.concurrent.locks.ReentrantLock#10a9598d[Unlocked]
start:710
I'm using springboot with spring data jpa.
I put all code in a test controller, and the problem remains
#RestController
#RequestMapping(value = "/test", produces = "application/json")
public class TestController {
private static Long testId;
private final CustomerBalanceRepository repository;
#Autowired
public TestController(CustomerBalanceRepository repository) {
this.repository = repository;
}
#PostConstruct
public void init() {
// CustomerBalance customer = new CustomerBalance();
// repository.save(customer);
// testId = customer.getId();
}
#SystemControllerLog(description = "updateScore")
#RequestMapping(method = RequestMethod.GET)
#Transactional(isolation = Isolation.REPEATABLE_READ)
public CustomerBalance updateScore() {
CustomerBalance customerBalance = repository.findOne(70L);
System.out.println("start:" + customerBalance.getInvestFreezen());
customerBalance.setInvestFreezen(customerBalance.getInvestFreezen().subtract(new BigDecimal(5)));
saveAndFlush(customerBalance);
System.out.println("end:" + customerBalance.getInvestFreezen());
return customerBalance;
}
#Transactional
public CustomerBalance saveAndFlush(CustomerBalance customerBalance) {
return repository.saveAndFlush(customerBalance);
}
}
and the results are
start:-110.00
end:-115.00
start:-110.00
end:-115.00
start:-115.00
end:-120.00
start:-120.00
end:-125.00
start:-125.00
end:-130.00
start:-130.00
end:-135.00
start:-130.00
end:-135.00
start:-135.00
end:-140.00
start:-140.00
end:-145.00
start:-145.00
end:-150.00
I tried to reproduce the problem and failed. I put your code, with very little changes into a Controller and executed it, by requestion localhost:8080/test and could see in the logs, that the score gets reduced as expected. Note: it actually produces an exception because I don't have a view resulution configured, but that should be irrelevant.
I therefore recommend the following course of action:
Take my controller from below, add it to your code with as little changes as possible. Verify that it actually works. Then modify it step by step until it is identical with your current code. Note the change that starts producing your current behavor. This will probably make the cause really obvious. If not update the question with what you have found.
#Controller
public class CustomerController {
private static String testId;
private final CustomerRepository repository;
private final JpaContext context;
public CustomerController(CustomerRepository repository, JpaContext context) {
this.repository = repository;
this.context = context;
}
#PostConstruct
public void init() {
Customer customer = new Customer();
repository.save(customer);
testId = customer.id;
}
#RequestMapping(path = "/test")
#Transactional(isolation = Isolation.REPEATABLE_READ)
public Customer updateScore() {
Customer customer = repository.findOne(testId);
System.out.println("start:" + customer.getScore());
customer.setScore(customer.getScore() - 23);
saveAndFlush(customer);
System.out.println("end:" + customer.getScore());
return customer;
}
#Transactional
public Customer saveAndFlush(Customer customer) {
return repository.saveAndFlush(customer);
}
}
After update from OP and a little discussion we seemed to have it pinned down:
The problem occurs ONLY with multiple threads (OP used JMeter to do this thing 10times/second).
Also Transaction level serializable seemed to fix the problem.
Diagnosis
It seems to be a lost update problem, which causes effects like the following:
Thread 1: reads the customer score=10
Thread 2: reads the customer score= 10
Thread 1: updates the customer to score 10-4 =6
Thread 2: updates the customer to score 10-3 =7 // update from Thread 1 is gone.
Why isn't this prevented by the synchronization?
The problem here is most likely that the read happens before the code shown in the question, since the EntityManager is a first level cache.
How to fix it
This should get caught by optimistic locking of JPA, for this one needs a column annotated with #Version.
Transaction Level Serializable might be the better choice if this happens often.
This looks like the case for pessimistic locking. Create in your repository method findOneWithLock lke this:
import org.springframework.data.jpa.repository.Lock;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.JpaRepository;
import org.springframework.data.repository.query.Param;
import javax.persistence.LockModeType;
public interface CustomerRepository extends JpaRepository<Customer, Long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
#Query("select c from Customer c where c.id = :id")
Customer findOneWithLock(#Param("id") long id);
}
and use it to obtain db level lock which will be held till the end of transaction:
#Transactional
public void updateScore(int score, Long userId) {
Customer customer = customerDao.findOneWithLock(userId);
customer.setScore(customer.getScore().subtract(score));
}
There is no need to use application level locks like RegularLock in your code.
The problem seems to be eventhough you call,
customerDao.saveAndFlush(customer);
The Commit will not take place until the end of the method is reached and a commit is made because your code inside of a
#Transactional(isolation = Isolation.REPEATABLE_READ)
What you can do is either change the propagation of the Transactional to Propagation.REQUIRES_NEW like below.
#Transactional(isolation = Isolation.REPEATABLE_READ, propagation=Propagation.REQUIRES_NEW)
This will result in a New Transaction being created and committed at the end of the method. And at the end of Transaction the changes will be committed.
With this line
manager.refresh(manager.find(model.getClass(), id));
you are telling JPA to undo all your changes. From the documentation of the refresh method
Refresh the state of the instance from the database, overwriting changes made to the entity, if any.
Remove it and your code should run as expected.

Categories