I have a springboot application that implements a user referral system. One use case is that when a user signs up using a valid referral code from another user, the referrer user gets one reward point, and for every five points they get 10$ in credit. According to this, I have implemented a use case in my application that honors these business rules, and to test proper behavior under high concurrency, I've created an integration test using #DataJpaTest and spring data repositories and H2 DB as storage system. In my test, I create a first user, and then I create a certain amount of users using the first user referral code, every one of those users is created on a different thread using a thread pool executor to spawn those threads. My problem is that the users created through thread pool spawned threads don't see the first user created in the main thread, even though I'm using JpaRepository.saveAndFlush() method to save them.
Could someone give me an explanation about what's happening here? Is it because Hibernate's session is not thread-safe?
You can see my code below, the first test has been simplified to just check the amount of user's in the repository.
#DataJpaTest(includeFilters = #ComponentScan.Filter(type = FilterType.ANNOTATION, classes = Repository.class))
public class JpaTest {
#Autowired
private TestEntityManager entityManager;
#Autowired
#Qualifier("JpaUserRepository")
private JpaUserRepository userRepository;
#Autowired
#Qualifier("JpaReferralRepository")
private ReferralRepository referralRepository;
private RegisterReferredUser registerReferredUser;
private CreateUser createUser;
private GetUser getUser;
#BeforeEach
void setUp() {
registerReferredUser = new RegisterReferredUser(referralRepository, userRepository);
createUser = new CreateUser(userRepository, referralRepository, registerReferredUser);
getUser = new GetUser(userRepository);
}
#Test
void createUser_shouldWorksProperlyUnderConcurrentExecution() throws InterruptedException {
ExecutorService executor = Executors.newFixedThreadPool(10);
EmailAddress referrerUserEmail = EmailAddress.of("john.doe#acme.inc");
User referrerUser = createUser.execute(new CreateUserCommand(referrerUserEmail.getValue(), null));
String referralCode = referrerUser.getReferralCode().getValue();
int maxIterations = 10;
for (int i = 0; i < maxIterations; i++) {
int emailSeed = i;
executor.submit(() -> {
createUser.execute(new CreateUserCommand(anEmailAddress(emailSeed), referralCode));
});
}
executor.shutdown();
if (!executor.awaitTermination(20, TimeUnit.SECONDS)) {
fail("Executor didn't finish in time");
}
assertThat(entityManager.getEntityManager().createQuery("from JpaUser").getResultList().size()).isEqualTo(maxIterations + 1);
// This fails: just 1 user in the repository, however, if I register users without referral (without checking the existence of the first user), users are created and this pass
}
#Test
void just_a_POC() throws InterruptedException {
ExecutorService executor = Executors.newFixedThreadPool(10);
userRepository.save(UserMother.aUserWithEmail("john.doe#acme.inc"));
int maxIterations = 10;
for (int i = 0; i < maxIterations; i++) {
int emailSeed = i;
executor.submit(() -> userRepository.save(UserMother.aUserWithEmail(anEmailAddress(emailSeed))));
}
executor.shutdown();
if (!executor.awaitTermination(20, TimeUnit.SECONDS)) {
fail("Executor didn't finish in time");
}
assertThat(entityManager.getEntityManager().createQuery("from JpaUser").getResultList().size()).isEqualTo(maxIterations + 1);
// This pass
}
}
In the CreateUser I have the following code:
private void assertReferralCodeIsValid(ReferralCode referralCode, EmailAddress email) {
if (!userRepository.findByReferralCode(referralCode).isPresent()) {
throw new NonExistentReferralCode(referralCode);
}
if (referralRepository.findByEmailAndCode(email, referralCode).isPresent()) {
throw new ReferralCodeAlreadyUsed(email, referralCode);
}
}
And this is the JpaUserRepository.save() method:
#Repository("JpaUserRepository")
public class JpaUserRepository implements UserRepository {
private JpaUserCrudRepository crudRepository;
public JpaUserRepository(JpaUserCrudRepository crudRepository) {
this.crudRepository = crudRepository;
}
#Override
public void save(User user) {
crudRepository.saveAndFlush(JpaUser.fromDomain(user));
}
}
Look at the isolation level configured for your transactions. Database engines usually try to serve data as fast as possible without blocking (when possible). So if all your threads read a table at the same time, they may get an "uncommited" version of the records.
If you need synchronization, you can change the isolation level, or lock the table before working on it.
More on this topic:
Spring #Transactional - isolation, propagation
https://www.baeldung.com/java-jpa-transaction-locks
Related
I am experimenting with the integration of MongoDB on Android using Java as the language.
I followed the guide provided by MongoDB to configure the Atlas account and the Realm to communicate with.
After that I tried implementing CRUD methods, for insertions I did not encounter any problems, while for queries I did.
In particular to get all the objects of a certain class in a certain collection.
I used this method, as suggested by the wiki (https://www.mongodb.com/docs/realm/sdk/java/quick-start-local/)
RealmResults<Contact> contacts = backgroundThreadRealm.where(Contact.class).findAll();
inserted in a class made to handle background tasks:
public class BackgroundTasks implements Runnable {
#Override
public void run() {
String realmName = "MyApp";
RealmConfiguration config = new RealmConfiguration.Builder().name(realmName).build();
Realm backgroundThreadRealm = Realm.getInstance(config);
// all Tasks in the realm
RealmResults<Contact> contacts = backgroundThreadRealm.where(Contact.class).findAll();
Log.v("Contacts", String.valueOf(contacts.size()));
backgroundThreadRealm.close();
}
}
While in the MainActivity I inserted this (looks like a battlefield, maybe I inserted stuff I won't even need, but I was experimenting):
// initialize mongodb realm
realm.init(this);
// open realm
String realmName = "MyApp";
RealmConfiguration config = new RealmConfiguration.Builder().name(realmName).build();
backgroundThreadRealm = Realm.getInstance(config);
app = new App(new AppConfiguration.Builder(appId).build());
User user = app.currentUser();
mongoClient = user.getMongoClient("mongodb-atlas");
mongoDatabase = mongoClient.getDatabase("MyApp");
MongoCollection<Document> mongoCollection = mongoDatabase.getCollection("Contacts");
FutureTask<String> task = new FutureTask(new BackgroundTasks(), "test");
ExecutorService executorService = Executors.newFixedThreadPool(2);
executorService.execute(task);
#Override
protected void onDestroy() {
super.onDestroy();
// the ui thread realm uses asynchronous transactions, so we can only safely close the realm
// when the activity ends and we can safely assume that those transactions have completed
backgroundThreadRealm.close();
}
I get no exceptions but the Log:
Log.v("Contacts", String.valueOf(contacts.size()));
results in 0.
Yet I have these contacts in the DB (they have different IDs):
And the related model in java:
#RealmClass
public class Contact extends RealmObject implements Serializable {
#PrimaryKey
private String nameSurname;
private int age;
// Drawable resource ID
private int imageResourceId;
public Contact() {
}
public Contact(String name, String surname, int age, int imageResourceId) {
this.nameSurname = name+" "+surname;
this.age = age;
this.imageResourceId = imageResourceId;
}
// In addition all the getters and setters
Can you help me?
It would also help to understand when it's appropriate to make synchronous and asynchronous calls, because I guess I've confused the implementations in general.
I'd like to use synchronous calls to get all the objects in the DB and then display them on the app, but it seems ill-advised online so I tried asynchronous, although I'm sure I did something wrong..
Thanks
In my app I have a "Manager" class that has a reference to a DAO class which loads data from DB and populate a HashMap as a cache solution.
Here is a snippet
class UserManager {
private final UserDAO userDAO;
private final Map<Long, User> users;
private final StampedLock lock = new StampedLock();
public UserManager() {
loadUsersFromDB();
}
// 99% of the times this method is called
public getUserById(long id) {
long stamp = lock.readLock();
try {
return users.get(id);
} finally {
lock.unlock(stamp);
}
}
// This is done once when CTOR calls it and from time to time
// by explicitly calling it from outside.
public void loadUsersFromDB() {
Map<Long, User> loadedUsers = userDAO.loadUsers();
long stamp = lock.writeLock();
try {
this.users = loadedUsers;
} finally {
lock.unlock(stamp);
}
}
}
This code work in a multithreaded environment, and the concern here is the use of StampedLock in this situation is an OVERKILL since most of the time it does READ operations and once in a while a simple load from DB and assignment to a class member.
I'm thinking to remove the StampedLock and instead use a simple AtomicReference<Map<Long, User>>, this way most of the time it's going to be a simple get and once in a while a set.
What do you think??
Suppose I have a test case like -
*Scenario: Facebook login test
GIVEN I am a Facebook user
WHEN I enter my user name & password
THEN login should be successful*
How could I get the scenario name from the step definition methods corresponding to "I am a Facebook user" or "I enter my user name & password" or "login should be successful" ?
Step definitions methods are -
#Given("^I am a Facebook user$")
public void method1() {
//some coding
//I want to get the scenario name here
}
#When("^I enter my user name & password$")
public void method2() {
//some coding
//I want to get the scenario name here
}
#Then("^login should be successful$")
public void method3() {
//some coding
//I want to get the scenario name here
}
You can use the #Before hook to get the current executing Scenario object.
#Before
public void beforeHook(Scenario scenario) {
this.sce = scenario
System......(scenario.getName())
System......(scenario.getId())
}
You can access the stored scenario object in your step definitions.
No #Bappa, it's possible, though your stepdefinition class is singleton and your tests are in parallel, see it be attacked with below approach by enhancing it with thread-safe static hash map variable used for storage:
public class StepDefinitions{
private static HashMap<Integer,String> scenarios;
public StepDefinitions(){ //or even inside of your singleton's getInstance();
if(scenarios == null)
scenarios = new HashMap<Integer,String();
}
#Before
public void beforeHook(Scenario scenario) {
addScenario(scenario.getName());
}
#When("your step definition")
public void stepDefinition1(){
String scenario = getScenario(); //problem-o-solved here...
}
private void addScenario(String scenario){
Thread currentThread = Thread.currentThread();
int threadID = currentThread.hashCode();
scenarios.put(threadID,scenario);
}
private synchronized String getScenario(){
Thread currentThread = Thread.currentThread();
int threadID = currentThread.hashCode();
return scenarios.get(threadID);
}
Inside the step definition, you can use CucumberHelper.scenario.getName().
Based on this API you can use getID, getSourceTagNames, getStatus and getClass methods.
I have a requirement where i have to send 100 mails/minute/user and it is time specific.(i.e All the mails i.e 100*1000 = 10000 mails/min should go on same time)
Currently there are almost 1000 users.
As well as for each email i am saving it first then sending it.
What things i need to implement for better performance and achieve goal optimally.
[Note: All the emails are sent via different accounts, so limit wont increase]
Any suggestion will be very helpful.
I am currently using Spring Boot for the project.
You can use Windows Service for this, if project run on windows server.
//Remember to set #EnableScheduling
//in the class containing your main method
#SpringBootApplication
#EnableScheduling
#EnableAsync
public class Application {
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class);
}
}
#Component
public class ScheduledTasks {
List<Email> listOfEmails;
int nextBatch = 50;
int curBatch = 0;
//This method will run every 15 second.
#Scheduled(fixedDelay = 15000)
public void yourMethodName() {
//This will process all of your objects all at once using treads
for(int i = curBatch; i < maxBatchSize(); i++){
listOfEmails.get(i).process();
}
nextBatch+=50;
curBatch+=50;
}
private int maxBatchSize(){
if(nextBatch < listOfEmails.size()){
return nextBatch;
} else {
return listOfEmails.size();
}
}
}
public class YourObject {
Integer someTest = 0;
#Async
public void process(Email e) {
e.send();
}
}
I would like to be able to report a certain method's progress in Spring Boot. I used a seperate class which I store the current status in and return as the current view:
It looks like this:
public class SearchTableReloadState {
//STATIC STORAGE
public static long TABLE_ROW_COUNT = 0;
public static long CURRENT_OFFSET = 0;
public static long CURRENT_LIMIT = 0;
public static long DEFAULT_LIMIT = 20000;
public static void reset() {
TABLE_ROW_COUNT = 0;
CURRENT_OFFSET = 0;
CURRENT_LIMIT = DEFAULT_LIMIT;
}
public static void setDefaultLimit(long defaultLimit) {
DEFAULT_LIMIT = defaultLimit;
}
// VIEWMODEL
public long tableRowCount = 0;
public long currentOffset = 0;
public long currentLimit = 0;
public static SearchTableReloadState getState() {
SearchTableReloadState reloadState = new SearchTableReloadState();
reloadState.tableRowCount = TABLE_ROW_COUNT;
reloadState.currentOffset = CURRENT_OFFSET;
reloadState.currentLimit = CURRENT_LIMIT;
return reloadState;
}
}
And the methods:
#RequestMapping(value = {"/manage/searchtable/reload/state"}, method = RequestMethod.GET)
public #ResponseBody SearchTableReloadState searchTableReloadState() {
return SearchTableReloadState.getState();
}
#ResponseStatus(HttpStatus.OK)
#RequestMapping(value = {"/manage/searchtable/reload"}, method = RequestMethod.GET)
public void searchTableReload() throws ResourceAlreadyExistsException, ParameterMissingIdException {
SearchTableReloadState.reset();
SearchTableReloadState.TABLE_ROW_COUNT = productDataReferenceDao.countJobReferences();
productDataReferenceDao.truncateSearchTable();
while (SearchTableReloadState.CURRENT_OFFSET < SearchTableReloadState.TABLE_ROW_COUNT) {
... long running task
....
SearchTableReloadState.CURRENT_OFFSET += SearchTableReloadState.CURRENT_LIMIT;
}
}
The method with the /state would report the current state, so I could call these with Ajax on a site. Problem is, If I start the long running one, the state report request won't complete until the long running did not complete. I thought Spring uses separate threads for each request. Do I need to implement threading in Spring?
If I use the #Async annotation for the long running process, it works like I expected, but I still don't understand, why could two separate HTTP requests for a different method block each other!
If I use the #Async annotation on the method that is supposed to take a long time, the HTTP Request calling it will get a response immediately and it will run in the background and I can call the state method as I expected. Even though it is working, I still don't know why it won't work without the asynchronous execution.
If you want to use the #Async annotation, you have to put the #EnableAsync annotation on the class you used the #SpringBootApplication and/or #EnableAutoConfiguration.
I hope someone can provide a better answer later.