We Keep Coding

Rollback changes done to a MariaDB database by a spring test without #Transactional

I have a Spring service that does something like that :
#Service
public class MyService {
#Transactional(propagation = Propagation.NEVER)
public void doStuff(UUID id) {
// call an external service, via http for example, can be long
// update the database, with a transactionTemplate for example
}
}
The Propagation.NEVER indicates we must not have an active transaction when the method is called because we don't want to block a connection to the database while waiting for an answer from the external service.
Now, how could I properly test this and then rollback the database ? #Transactional on the test won't work, there will be an exception because of Propagation.NEVER.
#SpringBootTest
#Transactional
public class MyServiceTest {
#Autowired
private MyService myService;
public void testDoStuff() {
putMyTestDataInDb();
myService.doStuff(); // <- fails no transaction should be active
assertThat(myData).isTheWayIExpectedItToBe();
}
}
I can remove the #Transactional but then my database is not in a consistent state for the next test.
For now my solution is to truncate all tables of my database after each test in a #AfterEach junit callback, but this is a bit clunky and gets quite slow when the database has more than a few tables.
Here comes my question : how could I rollback the changes done to my database without truncating/using #Transactional ?
The database I'm testing against is mariadb with testcontainers, so a solution that would work only with mariadb/mysql would be enough for me. But something more general would be great !
(another exemple where I would like to be able to not use #Transactional on the test : sometimes I want to test that transaction boundaries are correctly put in the code, and not hit some lazy loading exceptions at runtime because I forgot a #Transactional somewhere in the production code).
Some other precisions, if that helps :
I use JPA with Hibernate
The database is create with liquibase when the application context starts
Others ideas I've played with :
#DirtiesContext : this is a lot slower, creating a new context is a lot more expensive than just truncating all tables in my database
MariaDB SAVEPOINT : dead end, it's just a way to go back to a state of the database INSIDE a transaction. This would be the ideal solution IMO if i could work globally
Trying to fiddle with connections, issuing START TRANSACTION statements natively on the datasource before the test and ROLLBACK after the tests : really dirty, could not make it work

Personal opinion: #Transactional + #SpringBootTest is (in a way) the same anti-pattern as spring.jpa.open-in-view. Yes, it's easy to get things working at first and having the automatic rollback is nice, but it loses you a lot of flexibility and control over your transactions. Anything that requires manual transaction management becomes very hard to test that way.
We recently had a very similar case and in the end we decided to bite the bullet and use #DirtiesContext instead. Yeah, tests take 30 more minutes to run, but as an added benefit the tested services behave the exact same way as in production and the tests are more likely to catch any transaction issues.
But before we did the switch, we considered using the following workaround:
Create an interface and a service similar to the following:
interface TransactionService
{
void runWithoutTransaction(Runnable runnable);
}
#Service
public class RealTransactionService implements TransactionService
{
#Transactional(propagation = Propagation.NEVER)
public void runWithoutTransaction(Runnable runnable)
{
runnable.run();
}
}
In your other service wrap the external http calls with the #runWithoutTransaction-Method, e.g.:
#Service
public class MyService
{
#Autowired
private TransactionService transactionService;
public void doStuff(UUID id)
{
transactionService.runWithoutTransaction(() -> {
// call an external service
})
}
}
That way your production code will peform the Propagation.NEVER check, and for the tests you can replace the TransactionService with a different implemention that doesn't have the #Transactional annotations, e.g.:
#Service
#Primary
public class FakeTransactionService implements TransactionService
{
// No annotation here
public void runWithoutTransaction(Runnable runnable)
{
runnable.run();
}
}
This is not limited to Propagation.NEVER. Other propagation types can be implemented in the same way:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void runWithNewTransaction(Runnable runnable)
{
runnable.run();
}
And finally - the Runnable parameter can be replaced with a Function/Consumer/Supplier if the method needs to return and/or accept a value.

This is bit of wild idea, but if you are using mysql database, then maybe switch to dolt for tests?
Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
You can wrap it as testcontainers container, load necessary data on start and then, on start of each test run dolt reset.

How can I persist entities during a SpringBootTest integration test

I am writing an integration test for a SpringBoot 2 RestController. I want to test 404 behaviour and creation of entities. However, when I try to create entities and persist them before or during a test, they are not persisted in the SpringBoot context. By that I mean they are visible in the test context (during debugging of the test) but not for the Controller (ie it does not find them and my tests fail). What am I doing wrong?
How can I persist entities and flush the context during a test so that code that is called during an integration test sees them? I don't want to use a #before annotation to populate a database because I want to do it in my #test methods.
Here is my code. Thanks
#RunWith(SpringRunner.class)
#Transactional
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class InvoiceControlllerIT extends GenericControllerIT {
#Autowired
EntityManager entityManager;
#Test
#Transactional
public void cascadesChildEntityAssociationOnCreate() throws IOException {
assertThat(invoicerRepository.count(), equalTo(0L));
assertThat(invoiceRepository.count(), equalTo(0L));
assertThat(invoiceeRepository.count(), equalTo(0L));
// create an invoicee
Invoicee savedInvoicee = invoiceeRepository.save(new Invoicee());
assertThat(invoiceeRepository.count(), equalTo(1L));
// create an invoicer
Invoicer savedInvoicer = invoicerRepository.save(new Invoicer());
assertThat(invoicerRepository.count(), equalTo(1L));
// THIS IS THE PROBLEM, FLUSHING DURING THE TEST DOES NOT EFFECT THE CONTROLLERS ABILITY TO SEE THE NEWLY CREATED ENTITIES
entityManager.flush();
// create input
InvoiceInputDto inputDto = InvoiceInputDto
.builder()
.invoicee(savedInvoicee.getId())
.invoicer(savedInvoicer.getId())
.name("test-name")
.build();
// make call
ResponseEntity<InvoiceDto> response = template.postForEntity(url("/invoices", TOKEN), inputDto, InvoiceDto.class);
assertThat(response.getStatusCode(), equalTo(HttpStatus.CREATED));
assertThat(response.getBody().getName(), equalTo(inputDto.getName()));
// check associations
assertThat(invoiceeRepository.findById(savedInvoicee.getId()).get().getInvoices(), hasSize(1));
}
}

According to the docs:
If your test is #Transactional, it rolls back the transaction at the
end of each test method by default. However, as using this arrangement
with either RANDOM_PORT or DEFINED_PORT implicitly provides a real
servlet environment, the HTTP client and server run in separate
threads and, thus, in separate transactions. Any transaction initiated
on the server does not roll back in this case
(source: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-testing.html#boot-features-testing-spring-boot-applications)
Since the test transaction is separate from the HTTP server transaction, the controller won't see changes made from within the test method until the test transaction is actually committed. Conversely, you won't be able to roll back changes made as a result to the server call.
You will seriously make your life easier by providing a mock implementation for whatever service/repository your controller uses. Alternatively, you could use a tool like DBUnit to setup and tear down the database around each test case.

This worked for me:
#Inject
private EntityManagerFactory entityManagerFactory;
#BeforeEach
void setUp() {
EntityManager entityManager = entityManagerFactory.createEntityManager();
entityManager.getTransaction().begin();
entityManager.persist(someEntity());
entityManager.getTransaction().commit();
}

How to flush data into db inside active spring transaction?

I want to test hibernate session's save() method using spring testing framework.
#Test method is :
#Test
#Transactional
public void testSave() {
User expected = createUser();
getGenericDao().currentSession().save(expected);
User actual = getUser(generatedId);
assertUsersEqual(expected,actual);
}
I want to flush user into database. I want my user to be in database after this method
getGenericDao().currentSession().save(expected);
Then I want to go to database using spring data framework and fetch this saved user by next line:
User actual = getUser(generatedId);
I tried to use hibernate flush method like:
currentSession().setFlushMode(MANUAL);
//do saving here
currentSession().flush();
It does not flush my user into database!
However if I don't make use of #Transactional spring annotation and save my user in programmatic spring transaction I achieve what I want. Unfortunately then user saved into db
is not rollbacked as there is no spring #Transactional. Therefore my test method changes the db and behaviour of subsequent test methods.
So I need to flush my user into db inside test method(not at the end) and at the end of test method rollback all changes to db.
UPDATE suggestion to prepare method as follows:
#Transactional
public void doSave(User user){
getGenericDao().currentSession().save(user);
}
And call doSave inside testSave is doing nothing. Still I have no user in db after executing this method. I set breakpoint and check my db from command-line.
UPDATE Thanks very much for response. The problem is that method flush() does not put my user into database.I tried Isolation.READ_UNCOMMITTED and it does not put my user into database. I can achieve what I want but only if I turn off spring transaction on #Test method and do saving in programmatic transaction. BUT then #Test method is not rolled back leaving saved user for subsequent #Test methods. Here #Test method to save user is not as dangerous as #Test method to delete user, because it is not rolled back. So there must spring transactional support for #Test method with which I can't anyway put my user(or delete) into db. Actually user is put(or deleted) into db only after #Test method ends and transaction for #Test method is comitted. So I want to save my User into db in the middle of #Test method and roll back it at the end of #Test method
Thank you!

Finally I stuck to the following solution:
First, my #Test methods are not running within spring #Transactional support. See this article to know how dangerous it may be. Next, instead of using #Repository beans inside #Test methods I autowire #Service beans which use #Transactional annotation.
The miracle is that #Test method like this
#Test
#Transactional(propagation = Propagation.NOT_SUPPORTED)
public void testSave() {
Answer created = createAnswer();
Long generatedId = answerService.save(created);
//at this moment answer is already in db
Answer actual=getAnswerById(generatedId);
... }
puts my Answer object into database (just after answerService.save(created);) and method getAnswerById goes to DB and extracts it to check if save was correct.
To eliminate changes made to database in #Test method I recreate database by JdbcTestUtils.executeSqlScript

Have a look here with warning about #Transactional tests (Spring Pitfalls: Transactional tests considered harmful). I've used #org.springframework.test.context.jdbc.Sql to re-populate DB in my service tests and #Transactional for controllers.
ConstraintViolationException for controller update test with invalid data have been thrown only when transaction is committed. So I've found 3 options:
2.1 Annotate test with #Commit or with #Transactional(propagation = Propagation.NEVER). Be aware of DB change.
2.2 Use TestTransaction
Code:
TestTransaction.flagForCommit();
TestTransaction.end();
2.3 Use TransactionTemplate
Code:
#Autowired
private PlatformTransactionManager platformTransactionManager;
#Test(expected = Exception.class)
public void testUpdate() throws Exception {
TransactionTemplate transactionTemplate = new TransactionTemplate(platformTransactionManager);
transactionTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
String json = ...
transactionTemplate.execute(ts -> {
try {
mockMvc.perform(put(REST_URL + USER_ID)
.contentType(MediaType.APPLICATION_JSON)
.content(json))
.andExpect(status().isOk());
...
} catch (Exception e) {
e.printStackTrace();
}
return null;
});

You should also take care of the importedpackage :
in my case I imported
import javax.transaction.Transactional;
instead of
import org.springframework.transaction.annotation.Transactional;

If flush is not working then it very much depend on your database isolation level.
Isolation is one of the ACID properties of database, that defines how/when the changes made by one operation become visible to other concurrent operations.
I believe your isolation level is set to Read Committed or Repeatable Read.

JUnit tests always rollback the transactions

I'm running a simple JUnit test agains an application DAO. The problem is that I always get:
javax.persistence.RollbackException: Transaction marked as rollbackOnly
The JUnit test is:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {"classpath:com/my/app/context.xml"}
#TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = false)
#Transactional
public class PerformanceTest {
#Test
#Transactional(propagation= Propagation.REQUIRES_NEW)
#Rollback(false)
public void testMsisdnCreationPerformance() {
// Create a JPA entity
// Persist JPA entity
}
}
As you can see I'm declaring clearly not to rollback this method.
Does Spring JUnit support always sets rollback to true?

It should work, like you expect it, but may be you open another transaction within your class under test or you have an other feature/or bug somewhere.
Btw this annotations should be enougth:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {"classpath:com/my/app/context.xml"}
#Transactional
public class PerformanceTest {
#Test
#Rollback(false)
public void testMsisdnCreationPerformance() {
// Create a JPA entity
// Persist JPA entity
}
}
#See Spring Reference Chapter 9.3.5.4 Transaction management (or current version)

Just add annotation Rollback and set the flag to false.
#Test
#Rollback(false)

It is strange to desire a test that changes your database and keep the modification.
Tests are supposed to be orthogonal : no test depends on an other.
Moreover, tests are supposed to be independent of tests order, and even idempotent.
So either you want to change you data base in your setUp() method and rollback the change in your tearDown() method, either you want to setup a test database with some good values in it for tests.
Maybe I am missing something here but usually you should not want that.

From official Documentation:
By default, test transactions will be automatically rolled back after
completion of the test; however, transactional commit and rollback
behavior can be configured declaratively via the #Commit and #Rollback
annotations
https://docs.spring.io/spring/docs/current/spring-framework-reference/html/integration-testing.html#integration-testing-annotations
#Commit indicates that the transaction for a transactional test method
should be committed after the test method has completed. #Commit can
be used as a direct replacement for #Rollback(false) in order to more
explicitly convey the intent of the code.

I use Junit5, both commit and rollback(false) works with me.
#ExtendWith(SpringExtension.class)
#SpringBootTest
#Transactional
public class MyIntegrationTest {
#Test
#DisplayName("Spring Boot Will Rollback Data, " +
"Can Disable it By Add #Commit Or #Rollback(false) Annotation")
//#Commit
//#Rollback(false)
public void test() throws Exception {
//your test codes here...
}

I agree the Ralph's answer.
The Propagation.REQUIRES_NEW creates a new transaction and this probably does not match with the main transactional route in which the test is running.
In my simple experience the annotation #Transactional will properly work to define the transactional context in which every single test should run, delegating to this one the specific current Rollback clause (as shown by Ralph).
The Ralph's answer is useful and in the same time the Snicolas's answer concerns a particular case of managing context of tests.
The idempotence is fundamental for integration and automatic tests, but should be different ways to implements them.
The question is, which kind of methods do you have? And what behavior do theese methods have?
[...]
#Transactional
public class Test {
#Test
#Rollback(false)
public void test() {
[...]
Is the simple, question-coherent way :)

add #Rollback on your Test class-level
add #Transactional(value = "your_ManagerName",rollbackFor = Exception.class) on your test method

Spring unit test case is not rolling back insertion of a record

The following test cases are functionally working properly, but one of the test methods having to create a new article in the database doesn't rollback at the end of the test case execution.
I expect it to work that way. For a test case that update article actually rollbacks the update at the end of test case execution.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(value = "/applicationContext-test.xml")
#TransactionConfiguration(transactionManager = "txManager", defaultRollback = true)
#Transactional
public class PriceRepositoryTest {
#Resource(name ="repository")
private PriceRepository repository;
#Test
public void testGetAll() throws Exception {
Assert.assertEquals(8, repository.getAll().size());
}
#Test
#Rollback
public void shouldSaveNewArticle(){
Article article = new Article();
article.setName("Article");
article.setPrice(33);
repository.save(article);
Assert.assertEquals(9, repository.getAll().size());
}
#Test
#Rollback
public void shouldUpdateArticle(){
Article article = repository.getArticle(4);
article.setPrice(33);
repository.update(article);
Assert.assertEquals(String.valueOf(33.0), String.valueOf(repository.getArticle(4).getPrice()));
}
}

Is your DAO perhaps also marked with #Transactional? If it is, that's the problem: the transactionality on a different layer doesn't know about your local transaction configuration.
If repository.update(article) is #Transactional, it may or may not start a new transaction (depending on the value of the propagation attribute), but it will commit the transaction after execution, and there's nothing your test method can do to intercept that.
That's one of the reasons why transactions should be started on the service level, not the DAO level.
(If that's not the case, I humbly apologize)

I also encountered this issue and spent some hours trying to find the root cause. The issue was a variant of the problems described here. In my case, the application calls stored procedures through Spring, and one of these procedures contained a COMMIT statement.
Commits should always be controlled by the app, if there is a stray commit somewhere in a stored procedure, Spring can't control the transaction and tests won't roll back.

I just encountered this with my unit tests all set to rollback and my record was still showing up in the database after the test finished. The reason was there was that in the DAO a call to the entitymanager to flush() was in the method - which forced the transaction to commit.
em.persist(jpaServer);
em.flush(); //commits the record no matter what spring is setup to do
taking the flush out and confirmed no record. Tested the same code with the test annotated to #Rollback(false) and confirmed the record (proving the flush was not needed)
Ben
www.nimbits.com