public class A {
String name = "foo";
}
#Service
public class B {
private final Repository repo;
public void someMethod(){
A a = repo.findByName("foo");
a.name = Thread.currentThread().getName();
repo.save(a);
}
}
Imagine that 2 thread executed someMethod at the same time. Especially in jpa hibernate implementation.
My opinion is there is a race condition.
First and second threads are obtained same object with name foo. If i am not wrong, without optimistic lock there will be error in this scenario.
Also optimistic lock throws exception so do i need to use pessimistic lock to work correctly?
Also if i open second level cache with distributed in memory cache(Redis, hazelcast), what will happen?
It is a IOT project and million devices are calling this api and service.
Do i need to approach with eventual consistency?
Related
Summarize
Goal
I have an application that is written in Java using the Spring framework. There is a service that is being used as the handler for grabbing and releasing locks in the database (InnoDB). My goal is to be able to log the grabbing and releasing of the locks to create a lock history. For each lock interaction, I would like to know not only the name of the lock involved, but also where this request is coming from in the code (if possible, class name, method name, and line number).
My expected database entry will look something like this:
id
lock_name
clazz
method
line
lock_date
unlock_date
unlock_type
0
tb_member
MemberTools
createMember
123
2021-12-23 10:16:00
2021-12-23 10:16:01
COMMIT
1
tb_member
MemberTools
editMember
234
2021-12-23 10:16:01
2021-12-23 10:16:02
COMMIT
I would like to know if there is an easy way to obtain this given that I am using the Spring framework.
Describe
So far, I have tried two things:
Forcing the caller to pass a reference to itself or its current StackTraceElement (using Thread.currentThread().getStackTrace()[1]). This is not only extremely repetitive, but it also is prone to human error, as a developer might not realize that they need to pass in some reference to themselves.
Inside of the lock service, use the getStackTrace method and walk through the elements to find the "correct" one. This is made very hard by Spring and the fact that before a call actually reaches the inside of a class with the #Service annotation, the call stack is muddled by numbers of calls between proxies and generated classes and such. Unless there is a deterministic way to find the number of calls in between the Service and the caller, then this doesn't seem like a good way either.
I have referenced this stack overflow question while working, but these do not take into account the usage of the Spring framework.
Show
A reproducible example will look something like this. First, the structure:
root\
LockService.java
getLock()
MemberTools.java
createMember()
LockService.java:
#Service
public class LockService {
#Transactional
public Lock getLock(String key) {
Lock searchLock = new Lock();
searchLock.setKey(key);
lockMapper.getLock(searchLock);
LockHistory lockHistory = new LockHistory();
// Fill out lockHistory object...
lockMapper.markAsLocked(lockHistory);
attachTransactionCompletedListener(lockHistory);
}
private void attachTransactionCompletedListener(LockHistory lockHistory) {
/* Attach a listener onto the current spring transaction so that we
* can update the database entry when the transaction finishes and
* the lock is released.
*/
}
}
MemberTools.java:
public class MemberTools {
#Autowired
LockService lockService;
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void createMember() {
lockService.getLock("tb_member");
/* Do create member stuff...
* When this returns, the lock will be released
* (either from COMMIT, ROLLBACK, or UNKNOWN Spring error)
*/
}
}
By the time the getLock() method is reached, the stack trace is muddled with many calls that Spring inserts (proxies, reflections, etc.). Putting a breakpoint in this function and examining Thread.currentThread().getStackTrace() will show this.
I have two servers connected to the same database. Both have scheduled jobs I don't really care which one runs the scheduled jobs as long as only one does. So the idea was to keep a key-value pair in DB and whichever reads the value as 0 first gets to run the scheduled job.
Ideally this would work as so:
App A and App B run the scheduled job at the same time.
App A access the DB first, locks the table for reading & writing.
App A sets value to 1 and releases the lock.
App A starts working on the scheduled job.
App B reads the value 1 from it's DB request and does not run the scheduled job.
I have a config table where I keep status on my locks.
config:
name: VARCHAR(55)
value: VARCHAR(55)
The repository:
#Repository
public interface ConfigRepository extends CrudRepository<Config, Long> {
#Lock(LockModeType.PESSIMISTIC_READ)
Config findOneByName(String name);
#Lock(LockModeType.PESSIMISTIC_WRITE)
<S extends Config> S save(S entity);
}
The service:
#Service
public class ConfigService {
#Transactional
public void unlock(ConfigEnum lockable) {
Config lock = configRepository.findOneByName(lockable.getSetting());
lock.setValue("0");
configRepository.save(lock);
}
#Transactional
public void lock(ConfigEnum lockable) {
Config lock = configRepository.findOneByName(lockable.getSetting());
lock.setValue("1");
configRepository.save(lock);
}
#Transactional
public boolean isLocked(ConfigEnum lockable) {
Config lock = configRepository.findOneByName(lockable.getSetting());
return lock.getValue().equals("1");
}
}
The Scheduler:
#Component
public class JobScheduler {
#Async
#Scheduled("0 0 1 * * *")
#Transactional
public void run() {
if (!configService.isLocked(ConfigEnum.CNF_JOB.getJobName())) {
configService.lock(ConfigEnum.CNF_JOB.getJobName());
jobService.run();
configService.unlock(ConfigEnum.CNF_JOB.getJobName());
}
}
}
However I have noticed that the scheduled jobs still run at the same time on both apps. At times one will throw a deadlock but it appears that Spring retries the transaction if it hits a deadlock. At which time it appears that the one app has finished so this one begins the same job again (not sure).
The tasks are not that short that a lock could be established, table updated, task run and lock released. I would like to keep this really simple without involving additional libraries like Quartz or ShedLock.
I think your transactions are too short. You don't start a transaction in the run method, but each ConfigService method is transactional. Most likely each method gets a new transaction and commits when done. A commit will release the lock, so there is a race condition between isLocked and lock.
Combine isLocked and lock:
#Transactional
public boolean tryLock(ConfigEnum lockable) {
Config lock = configRepository.findOneByName(lockable.getSetting());
if("1".equals(lock.getValue()) {
return false;
}
lock.setValue("1");
configRepository.save(lock);
return true;
}
This checks and writes in the same transaction and should work.
As a side note it is a dangerous method. What happens if the node that has the lock dies? There are many possible solutions. One is to lock a specific record and keep that lock throughout the job. The other node cannot proceed and if the first one dies the lock will be released. Another is to use a timestamp instead of 1 and require the timestamp to be updated on a regular basis by the owner. Or you could introduce something like Zookeeper.
I am looking for a rule engine for my web application and I found Easy Rules. However, in the FAQ section, it states that the limitation on thread safety.
Is a Web Container considered as a multi-threaded environment? For HTTP request is probably processed by a worker thread created by the application server.
How does thread safety comes into place?
How to deal with thread safety?
If you run Easy Rules in a multi-threaded environment, you should take into account the following considerations:
Easy Rules engine holds a set of rules, it is not thread safe.
By design, rules in Easy Rules encapsulate the business object model they operate on, so they are not thread safe neither.
Do not try to make everything synchronized or locked down!
Easy Rules engine is a very lightweight object and you can create an instance per thread, this is by far the easiest way to avoid thread safety problems
http://www.easyrules.org/get-involved/faq.html
http://www.easyrules.org/tutorials/shop-tutorial.html
Based on this example, how will multi-threading affects the rule engine?
public class AgeRule extends BasicRule {
private static final int ADULT_AGE = 18;
private Person person;
public AgeRule(Person person) {
super("AgeRule",
"Check if person's age is > 18 and
marks the person as adult", 1);
this.person = person;
}
#Override
public boolean evaluate() {
return person.getAge() > ADULT_AGE;
}
#Override
public void execute() {
person.setAdult(true);
System.out.printf("Person %s has been marked as adult",
person.getName());
}
}
public class AlcoholRule extends BasicRule {
private Person person;
public AlcoholRule(Person person) {
super("AlcoholRule",
"Children are not allowed to buy alcohol",
2);
this.person = person;
}
#Condition
public boolean evaluate() {
return !person.isAdult();
}
#Action
public void execute(){
System.out.printf("Shop: Sorry %s,
you are not allowed to buy alcohol",
person.getName());
}
}
public class Launcher {
public void someMethod() {
//create a person instance
Person tom = new Person("Tom", 14);
System.out.println("Tom:
Hi! can I have some Vodka please?");
//create a rules engine
RulesEngine rulesEngine = aNewRulesEngine()
.named("shop rules engine")
.build();
//register rules
rulesEngine.registerRule(new AgeRule(tom));
rulesEngine.registerRule(new AlcoholRule(tom));
//fire rules
rulesEngine.fireRules();
}
}
Yes, a web application is multithreaded. As you expect, there is a pool of threads maintained by the server. When the serversocket gets an incoming request on the port it's listening to, it delegates the request to a thread from the pool.Typically the request is executed on that thread until the response is completed.
If you try to create a single rules engine and let multiple threads access it, then either
the rules engine data is corrupted as a result of being manipulated by multiple threads (because data structures not meant to be threadsafe can perform operations in multiple steps that can be interfered with by other threads as they're accessing and changing the same data), or
you use locking to make sure only one thread at a time can use the rules engine, avoiding having your shared object get corrupted, but in the process creating a bottleneck. All of your requests will need to wait for the rules engine to be available and only one thread at a time can make progress.
It's much better to give each request its own copy of the rules engine, so it doesn't get corrupted and there is no need for locking. The ideal situation for threads is for each to be able to execute independently without needing to contend for shared resources.
In my service code, I am trying to create or update a Person domain object:
#Transactional
def someServiceMethod(some params....) {
try{
def person = Person.findByEmail(nperson.email.toLowerCase())
if (!person) {
person = new Person()
person.properties = nperson.properties
} else {
// update the person parameters (first/last name)
person.firstName = nperson.firstName
person.lastName = nperson.lastName
person.phone = nperson.phone
}
if (person.validate()) {
person.save(flush: true)
//... rest of code
}
// rest of other code....
} catch(e) {
log.error("Unknown error: ${e.getMessage()}", e)
e.printStackTrace()
return(null)
}
Now above code OCCASIONALLY when trying to save a Person object with already existing email throws following exception:
Hibernate operation: could not execute statement; SQL [n/a]; Duplicate entry 'someemail#gmail.com' for key 'email_UNIQUE'; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry 'someemail#gmail.com' for key 'email_UNIQUE'
This is very strange because I am already finding the person by email and hence the save() should try to update the record instead of creating the new one.
I was wondering why is this happening!
EDIT:
I am on grails 2.4.5 and Hibernate plugin in BuildConfig is:
runtime ':hibernate4:4.3.8.1'
EDIT2:
My application is on multiple servers hence synchronized block won't work
If this is concurrency issue, here is what we do in such case. We have a lot of concurrent background processes which work on the same tables. If there is such operation it indeed is in synchronized block, so code may look like:
class SomeService {
static transactional = false //service cannot be transactional
private Object someLock = new Object() //synchronized block on some object must be used
def someConcurrentSafeMethod(){
synchronized(someLock){
def person = Person.findByEmail(nperson.email.toLowerCase())
...
person.save(flush: true) // flush is very important, must be done in synchronized block
}
}
}
There are few important points to make this working (from our experience, not official):
Service cannot be transactional - if service is transactional, transaction is commited after the method returns value and synchronization inside method will not be enough. Programmatic transactions may be another way
synchronized method is not enough synchronized def someConcurrentSafeMethod() will not work - probably because service is wrapped in proxy
Session MUST be flushed inside synchronized block
every object which will be saved, should be read in synchronized block, if you pass it from external method, you may run into optimistic locking failed exception
UPDATED
Because application is deployed on distributed system, above will not solve the issue here (still may help others). After discussion we had on Slack, I just summarize potential ways to do that:
pessimistic locking of updated objects and lock of whole table for inserts (if possible)
moving 'dangerous' database related methods to single server with some API like REST and calling it from other deployments (and using synchronized approach from above)
using multiple save approach - if operation fails, catch exception and try again. This is supported by integration libraries like Spring Integration or Apache Camel and is one of enterprise patterns. See request-handler-advice-chain for Spring Integration as an example
use something to queue operations, for example JMS server
If anyone has more ideas please share them.
I have a couple of questions about Transactions in Spring if you may.
Let's suppose i have this DAO class :
public class MyDAO {
/**
* verifies if a certain record in DB contains 'True' in a certain Column named publishFlag
*/
#Transactional
public bloolean isBeingPublished(Long recordID){
...
}
/**
* sets the record's publishFlag column to true indicating that it's being published
*/
#Transactional
public boolean setBeingPublished(Long recordID){
...
}
}
And the following class using it :
public class MyClass {
#Autowired
MyDAO dao;
public void publishRecords(List<Long> ids){
for(Long id : ids){
if(!dao.isBeingPublished(id)){
dao.setBeingPublished(id);
//do something to publish the record
}
}
}
}
My questions are :
First of all, will the !dao.isBeingPublished(id) and dao.setBeingPublished(id) be executed in the same transaction or in separate ones?
Second question's about concurrency, Multiple MyClass instances can be created and concurrent calls to the publishRecord method can occur, so two concurrent calls to !dao.isBeingPublished(id) might both give the same result and thus making the record published twice!
I would consider making the publishRecords synchronized but the application may be deployed on multiple servers which renders the synchronized declaration useless, hence my question about transactions since the database is the only shared resource between the apps deployed on those servers.
What would be the solution to my problem exactly? I read about spring's transaction propagation and found out that REQUIRES_NEW would create a new transaction even if one is currently being executed, but still, I just can't see how that's going to be a solution to my problem.
Thank you in advance for your help.
Few things need consider, DAO is focus on operation on single entity, and service is focus on operation of one or more entities, so the transaction should put on service layer, so you can reuse DAO's operation without any transaction, but let service to decide when start and end transaction
It is not in single transaction, but two separate transaction.
That is the problem concurrency issue with your current design, see the following suggestion.
Interface
public interface MyClass {
public void publishRecords(List<Long> ids);
}
Implementation
#Service
#Transactional(readOnly = false)
class DefaultMyClass implements MyClass {
#Autowired
MyDAO dao;
// single transaction
#Override
public void publishRecords(List<Long> ids) {
for(Long id : ids){
if(!dao.isBeingPublished(id)){
dao.setBeingPublished(id);
//do something to publish the record
}
}
}
}
DAO
class MyDAO {
public bloolean isBeingPublished(Long recordID){
// bigbang
}
public boolean setBeingPublished(Long recordID){
// bigbang
}
}
Using the above design, both problems are being resolved.
First of all, will the !dao.isBeingPublished(id) and
dao.setBeingPublished(id) be executed in the same transaction or in
seperate ones?
Unless there's a method annotated with #Transactional further up the stack, they will be occurring in separate transactions, so yes you will have a potential for a race condition.
If I were you, I would toss the isBeingPublished and setBeingPublished in favor of a single #Transactional publishIfPossible method that returns a boolean value of whether it was able to acquire the database row lock and do the publish operation.