We Keep Coding

Rollback changes done to a MariaDB database by a spring test without #Transactional

I have a Spring service that does something like that :
#Service
public class MyService {
#Transactional(propagation = Propagation.NEVER)
public void doStuff(UUID id) {
// call an external service, via http for example, can be long
// update the database, with a transactionTemplate for example
}
}
The Propagation.NEVER indicates we must not have an active transaction when the method is called because we don't want to block a connection to the database while waiting for an answer from the external service.
Now, how could I properly test this and then rollback the database ? #Transactional on the test won't work, there will be an exception because of Propagation.NEVER.
#SpringBootTest
#Transactional
public class MyServiceTest {
#Autowired
private MyService myService;
public void testDoStuff() {
putMyTestDataInDb();
myService.doStuff(); // <- fails no transaction should be active
assertThat(myData).isTheWayIExpectedItToBe();
}
}
I can remove the #Transactional but then my database is not in a consistent state for the next test.
For now my solution is to truncate all tables of my database after each test in a #AfterEach junit callback, but this is a bit clunky and gets quite slow when the database has more than a few tables.
Here comes my question : how could I rollback the changes done to my database without truncating/using #Transactional ?
The database I'm testing against is mariadb with testcontainers, so a solution that would work only with mariadb/mysql would be enough for me. But something more general would be great !
(another exemple where I would like to be able to not use #Transactional on the test : sometimes I want to test that transaction boundaries are correctly put in the code, and not hit some lazy loading exceptions at runtime because I forgot a #Transactional somewhere in the production code).
Some other precisions, if that helps :
I use JPA with Hibernate
The database is create with liquibase when the application context starts
Others ideas I've played with :
#DirtiesContext : this is a lot slower, creating a new context is a lot more expensive than just truncating all tables in my database
MariaDB SAVEPOINT : dead end, it's just a way to go back to a state of the database INSIDE a transaction. This would be the ideal solution IMO if i could work globally
Trying to fiddle with connections, issuing START TRANSACTION statements natively on the datasource before the test and ROLLBACK after the tests : really dirty, could not make it work

Personal opinion: #Transactional + #SpringBootTest is (in a way) the same anti-pattern as spring.jpa.open-in-view. Yes, it's easy to get things working at first and having the automatic rollback is nice, but it loses you a lot of flexibility and control over your transactions. Anything that requires manual transaction management becomes very hard to test that way.
We recently had a very similar case and in the end we decided to bite the bullet and use #DirtiesContext instead. Yeah, tests take 30 more minutes to run, but as an added benefit the tested services behave the exact same way as in production and the tests are more likely to catch any transaction issues.
But before we did the switch, we considered using the following workaround:
Create an interface and a service similar to the following:
interface TransactionService
{
void runWithoutTransaction(Runnable runnable);
}
#Service
public class RealTransactionService implements TransactionService
{
#Transactional(propagation = Propagation.NEVER)
public void runWithoutTransaction(Runnable runnable)
{
runnable.run();
}
}
In your other service wrap the external http calls with the #runWithoutTransaction-Method, e.g.:
#Service
public class MyService
{
#Autowired
private TransactionService transactionService;
public void doStuff(UUID id)
{
transactionService.runWithoutTransaction(() -> {
// call an external service
})
}
}
That way your production code will peform the Propagation.NEVER check, and for the tests you can replace the TransactionService with a different implemention that doesn't have the #Transactional annotations, e.g.:
#Service
#Primary
public class FakeTransactionService implements TransactionService
{
// No annotation here
public void runWithoutTransaction(Runnable runnable)
{
runnable.run();
}
}
This is not limited to Propagation.NEVER. Other propagation types can be implemented in the same way:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void runWithNewTransaction(Runnable runnable)
{
runnable.run();
}
And finally - the Runnable parameter can be replaced with a Function/Consumer/Supplier if the method needs to return and/or accept a value.

This is bit of wild idea, but if you are using mysql database, then maybe switch to dolt for tests?
Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
You can wrap it as testcontainers container, load necessary data on start and then, on start of each test run dolt reset.

Why transactional does not rollback when RuntimeException occur?

I want to test a non-transactional method in the service layer when inner methods are transactional separately. I want to know what will happen if some method throws an exception. Does it properly roll back or not? I also tried rollbackFor but it did not help. Any suggestions?
Spring v-2.5.1
app.properties
spring.datasource.url=jdbc:mysql://localhost:3306/test
spring.datasource.username=root
spring.datasource.password=root
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.hibernate.ddl-auto=create-drop
server.error.include-message=always
server.error.include-binding-errors=always
Controller
#PostMapping("/test")
public void test(){
service.test();
}
Service
#Service
public class UserService {
#Autowired
private UserRepository userRepository;
#Autowired
private PasswordEncoder passwordEncoder;
public void test() {
User user1 = new User();
String encodedPassword = passwordEncoder.encode("123");
user1.setUsername("username");
user1.setPassword(encodedPassword);
user1.setFirst_name("name1");
user1.setLast_name("family1");
save(user1);
User update = update(user1.getUsername());
throw new RuntimeException();// I expect Spring rollback all data for save and update methods
//but it seems data has been committed before an exception occur.
}
#Transactional
public void save(User user) {
if (userExists(user.getUsername()))
throw new ApiRequestException("Username has already taken!");
else {
User user1 = new User();
String encodedPassword = passwordEncoder.encode(user.getPassword());
user1.setUsername(user.getUsername());
user1.setPassword(encodedPassword);
user1.setFirst_name(user.getFirst_name());
user1.setLast_name(user.getLast_name());
user1.setCreate_date(user.getCreate_date());
user1.setModified_date(user.getModified_date());
user1.setCompany(user.getCompany());
user1.setAddresses(user.getAddresses());
userRepository.save(user1);
// throw new RuntimeException(); Similarly I expect rollback here.
}
}
#Transactional
public User update(String username) {
User userFromDb = userRepository.findByUsername(username);
userFromDb.setFirst_name("new username");
userFromDb.setLast_name("new lastname");
return userRepository.save(userFromDb);
}
}

Due to way spring proxying works by default, you can not call a "proxied" method from within the instance.
Consider that when you put #Transactional annotation on a method, Spring makes a proxy of that class and the proxy is where the transaction begin/commit/rollback gets handled. The proxy calls the actual class instance. Spring hides this from you.
But given that, if you have a method on the class instance (test()), that calls another method on itself (this.save()), that call doesn't goes through the proxy, and so there is no "#Transactional" proxy. There is nothing to do the rollback when the RuntimeException occurs.
There are ways to change the how Spring does the proxying which would allow this to work, but it has changed over the years. There are various alternatives. One way is to create create separate classes. Perhaps UserServiceHelper. The UserServiceHelper contains #Transactional methods that get called by UserService. This same issue occurs when different #Transactional isolations and propagations are needed.
Related answers and info:
Spring #Transaction method call by the method within the same class, does not work?
Does Spring #Transactional attribute work on a private method?
Spring Transaction Doesn't Rollback
Your example code is not very clear as to what you are trying to do. Often, #Transactional would be put on the service class and apply to all public methods (like test()).

To begin with, I'm not sure if you understand what a transaction represents. If there's a unit of work that contains several functionalities to be executed, and you either want them to completely fail or completely pass - that's when you use a transaction.
So in the case of test() method - why should underlying mechanism (in this case Hibernate) care about rolling back something that happened within save() and update() when the test() itself isn't a transaction?
Sorry for asking more than answering, but as far as I can see - you need to annotate test() with the annotation, and not the individual methods. Or you can annotate them all.

In Spring, how to flag transaction info?

I need to somehow flag a transaction. Need some method like: TransactionAspectSupport.setData(someObject); Then till the transaction is alive I would like to be able to read those data.
I need it to check in Aspect class that some operation on current transaction has been already proceeded.
EDIT:
To illustrate what I mean. Let's have two classes.
Service class:
class Service {
#Transactional
public void serviceA(){
// do something
serviceB();
}
#Transactional
public void serviceB(){
// do something
}
}
Aspect class:
#Aspect
class ServiceAspect {
#Pointcut("execution(public * service*)")
private void checkTransaction(){};
#Around("checkTransaction()")
public Object _checkTransaction(ProceedingJoinPoint joinPoint) throws Throwable {
if (!isTransactionFlagged())
doTheCheckingJob()
transactionInfo.setData(Boolean.True);
}
private boolean isTransactionFlagged() {
if (transactionInfo.getData != null)
return true;
return false;
}
}
As you can see I need to flag the transaction somehow, so the aspect do not fire multiple times in one transaction. But still need the aspect to fire for each service method. Just not fire on the inner call of service method.

You can achieve what you want without manual bookkeeping using cflow() or cflowbelow() pointcuts. Unfortunately they are unavailable in Spring AOP, but if you configure Spring to use AspectJ with LTW (load-time weaving) instead, you can use them.
If you want to stick with Spring AOP, though, you indeed need to do manual bookkeeping, e.g. using a ThreadLocal member (assuming that each transaction is executed in a separate thread) or whatever else is appropriate. But maybe there are on-board means in the Spring framework to enable the behaviour you need. I am not a Spring user, so I cannot help you there, though. I only know about AOP.
Feel free to ask follow-up questions and/or provide more code, then I can get more concrete.

AbstractRoutingDataSource doesn't change connection

I use AbstractRoutingDataSource to change data source dynamically and ThreadLocal to set up currentLookupKey. It works nice when I use only one data source per http request. I use JpaRepository
#Component
#Primary
public class RoutingDataSource extends AbstractRoutingDataSource {
#Autowired
private DatabaseMap databaseMap;
#Override
public void afterPropertiesSet() {
setTargetDataSources(databaseMap.getSourcesMap());
setDefaultTargetDataSource(databaseMap.getSourcesMap().get("DEFAULT"));
super.afterPropertiesSet();
}
#Override
protected Object determineCurrentLookupKey() {
return DatabaseContextHolder.getDatabaseType();
}
}
public class DatabaseContextHolder {
private static final ThreadLocal<String> contextHolder = new ThreadLocal<String>();
public static void setDatabaseType(String string) {
contextHolder.set(string);
}
public static String getDatabaseType() {
return (String) contextHolder.get();
}
public static void clearDatabaseType() {
contextHolder.remove();
}
}
When I try to get data in my REST controller I get data only from one database.
Some code in my REST controller
DatabaseContextHolder.setDatabaseType("db1");
//here I get data from db1 as expected
//I use JpaRepository
DatabaseContextHolder.clearDatabaseType();
DatabaseContextHolder.setDatabaseType("db2");
//here I should get data from db2 but get from db1
I tried to debug and it looks like Spring obtains data source only once in http request.
This method is called only once.
#Override
public Connection getConnection() throws SQLException {
return determineTargetDataSource().getConnection();
}
Is there any way to force Spring to change data source.

Your problem could be related with transaction delimitation.
When you define a #Transactional annotation in your code, Spring will create on your behalf all the stuff necessary to begin and end, and commiting or rollback if required, a transaction.
As you can see in the doBegin method in the source code of the DataSourceTransactionManager class - the same applies to other transaction managers - , Spring obtains a Connection when the transaction is initialized - this is why the method getConnection is invoked only once - , and it will reuse that connection across all the underlying operations against the database within that transaction (it makes sense for ACID preservation).
So, if you need to connect to several data sources within the same request processing, you can define different methods in you service code, every one annotated with a #Transactional annotation, and change the underlying data source as you require before invoke them:
DatabaseContextHolder.setDatabaseType("db1");
// Invoke a service method annotated with #Transactional
// It can use the underlying JpaRepositories that you need
DatabaseContextHolder.clearDatabaseType();
DatabaseContextHolder.setDatabaseType("db2");
// Invoke again another (or the same, what you need) service method
// annotated with #Transactional, You should get data from db2 this time

My suspicion here is you have a method annotated with #Transactional annotation. Before calling that transactional method, you first specify one datasource key and you call the transactional method.Inside the transactional method, you first call repository and it works as expected with datasource look up key you set. However then you set different key, inside the transactional method, and call another repository and it still uses the key you set first time.
DataSource will be chosen by the framework when the transaction starts so if you are using #Transactional annotation, whatever switching you do inside the method is useless. Because the datasource would have been chosen by proxy created for #Transactional annotation. Best option is to have the branching logic in non transactional service or use TransactionTemplate instead of #Transactional
For example, make sure YourRestController does not have class level #Transactional as well as no #Transactional annotation in this yourRestControllerMethod, you will keep them to your service.
#RestController
public class YourRestController {
#Autowired
TransactionalService transactional
public void yourRestControllerMethod(){
//set the datasource look up key to A1 and then
transactional.methodA1();
//change datasource look up key to A2 and then
transactional.methodA2();
}
}
#Service
public class TransactionalService {
#Transactional
public void methodA1(){
}
#Transactional
public void methodA2() {
}
}

I had the same issue, none of the above solution could fix it.. but making my Service method final (in my REST Controller)
public final Response

Set spring.jpa.open-in-view to false.

Spring transaction propagation - Service vs DAO

I have a Service class like below:
#Service("MyService")
public class MyService {
#Autowired
MyDao dao;
public void process() {
getFromDao();
// getMoreFromDao();
// process();
// if all good, then
doStuff();
}
public void getFromDao() {
// do some stuff
dao.getData();
}
#Transactional(transactionManager="simpleDatasourceTxMgr", propagation=Propagation.REQUIRED)
public void doStuff() {
dao.saveData(1);
dao.saveData(2);
dao.saveData(3);
}
}
The DAO called is:
#Repository
public class MyDao {
#Autowired
#Qualifier("myjdbcTemplate")
NamedParameterJdbcTemplate jdbcTemplate;
public void saveData(obj a) {
jdbcTemplate.execute("Query", ...);
}
}
I want my doStuff() method in the service class to run within a transaction and rollback everything if there is an exception in the saveData() method. But this is not running in transaction.
If I add #Transaction to a DAO method looks like it runs in separate transaction. Is this correct?
Update: I have added a process() method to my Service and I call getFromDao() and doStuff() from process(). process() is called from the controller. So looks like if I make the service class #Transactional, then everything executes within a transaction. But I don't want getFromDao() to execute in transaction.
We use just JDBC and no Hibernate.

You can place the #Transactional annotation before an interface
definition, a method on an interface, a class definition, or a public
method on a class. However, the mere presence of the #Transactional
annotation is not enough to activate the transactional behavior. The
#Transactional annotation is simply metadata that can be consumed by
some runtime infrastructure that is #Transactional-aware and that can
use the metadata to configure the appropriate beans with transactional
behavior. In the preceding example, the
element switches on the transactional behavior.
Or if you want annotations you can enable it with
It is not sufficient to tell you simply to annotate your classes with
the #Transactional annotation, add #EnableTransactionManagement to
your configuration, and then expect you to understand how it all
works. This section explains the inner workings of the Spring
Framework’s declarative transaction infrastructure in the event of
transaction-related issues.
http://docs.spring.io/spring/docs/current/spring-framework-reference/html/transaction.html