We Keep Coding

Rollback changes done to a MariaDB database by a spring test without #Transactional

I have a Spring service that does something like that :
#Service
public class MyService {
#Transactional(propagation = Propagation.NEVER)
public void doStuff(UUID id) {
// call an external service, via http for example, can be long
// update the database, with a transactionTemplate for example
}
}
The Propagation.NEVER indicates we must not have an active transaction when the method is called because we don't want to block a connection to the database while waiting for an answer from the external service.
Now, how could I properly test this and then rollback the database ? #Transactional on the test won't work, there will be an exception because of Propagation.NEVER.
#SpringBootTest
#Transactional
public class MyServiceTest {
#Autowired
private MyService myService;
public void testDoStuff() {
putMyTestDataInDb();
myService.doStuff(); // <- fails no transaction should be active
assertThat(myData).isTheWayIExpectedItToBe();
}
}
I can remove the #Transactional but then my database is not in a consistent state for the next test.
For now my solution is to truncate all tables of my database after each test in a #AfterEach junit callback, but this is a bit clunky and gets quite slow when the database has more than a few tables.
Here comes my question : how could I rollback the changes done to my database without truncating/using #Transactional ?
The database I'm testing against is mariadb with testcontainers, so a solution that would work only with mariadb/mysql would be enough for me. But something more general would be great !
(another exemple where I would like to be able to not use #Transactional on the test : sometimes I want to test that transaction boundaries are correctly put in the code, and not hit some lazy loading exceptions at runtime because I forgot a #Transactional somewhere in the production code).
Some other precisions, if that helps :
I use JPA with Hibernate
The database is create with liquibase when the application context starts
Others ideas I've played with :
#DirtiesContext : this is a lot slower, creating a new context is a lot more expensive than just truncating all tables in my database
MariaDB SAVEPOINT : dead end, it's just a way to go back to a state of the database INSIDE a transaction. This would be the ideal solution IMO if i could work globally
Trying to fiddle with connections, issuing START TRANSACTION statements natively on the datasource before the test and ROLLBACK after the tests : really dirty, could not make it work

Personal opinion: #Transactional + #SpringBootTest is (in a way) the same anti-pattern as spring.jpa.open-in-view. Yes, it's easy to get things working at first and having the automatic rollback is nice, but it loses you a lot of flexibility and control over your transactions. Anything that requires manual transaction management becomes very hard to test that way.
We recently had a very similar case and in the end we decided to bite the bullet and use #DirtiesContext instead. Yeah, tests take 30 more minutes to run, but as an added benefit the tested services behave the exact same way as in production and the tests are more likely to catch any transaction issues.
But before we did the switch, we considered using the following workaround:
Create an interface and a service similar to the following:
interface TransactionService
{
void runWithoutTransaction(Runnable runnable);
}
#Service
public class RealTransactionService implements TransactionService
{
#Transactional(propagation = Propagation.NEVER)
public void runWithoutTransaction(Runnable runnable)
{
runnable.run();
}
}
In your other service wrap the external http calls with the #runWithoutTransaction-Method, e.g.:
#Service
public class MyService
{
#Autowired
private TransactionService transactionService;
public void doStuff(UUID id)
{
transactionService.runWithoutTransaction(() -> {
// call an external service
})
}
}
That way your production code will peform the Propagation.NEVER check, and for the tests you can replace the TransactionService with a different implemention that doesn't have the #Transactional annotations, e.g.:
#Service
#Primary
public class FakeTransactionService implements TransactionService
{
// No annotation here
public void runWithoutTransaction(Runnable runnable)
{
runnable.run();
}
}
This is not limited to Propagation.NEVER. Other propagation types can be implemented in the same way:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void runWithNewTransaction(Runnable runnable)
{
runnable.run();
}
And finally - the Runnable parameter can be replaced with a Function/Consumer/Supplier if the method needs to return and/or accept a value.

This is bit of wild idea, but if you are using mysql database, then maybe switch to dolt for tests?
Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
You can wrap it as testcontainers container, load necessary data on start and then, on start of each test run dolt reset.

Spring testing DB rollback

I am #new to spring and faced some problem while running some tests. I have a few test-classes with the following code which should rollback my (in memory h2) database:
#Autowired
PlatformTransactionManager txm;
TransactionStatus txstatus;
#BeforeEach
public void setupDB() {
DefaultTransactionDefinition def = new DefaultTransactionDefinition();
def.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
txstatus = txm.getTransaction(def);
assumeTrue(txstatus.isNewTransaction());
txstatus.setRollbackOnly();
}
#AfterEach
public void rollback() {
txm.rollback(txstatus);
}
My problem is, that if one test class has finished, I get a JdbcSQLIntegrityConstraintViolationException (Unique index or primary key violation:..), because my Database is not rollbacked accordingly and Insert statements are executed again, because the database didnt get cleared. Does anyone has a tip how to fix that? Is there a way to rollback the inserts or not to do the inserts after they have been done ones?

You can just annotate your test class with
#Transactional
and Spring will handle everything (which means each test will run in its own transaction which will be rolled back after).
You can also use
#DirtiesContext(classMode = ClassMode.BEFORE_EACH_TEST_METHOD)
but this is heavy because the whole Spring context must be recreated.

A simple solution is to use the DirtiesContext annotation. This annotation has several options. You can use the below line in your test class:
#DirtiesContext(classMode = ClassMode.BEFORE_CLASS)
The context will be removed and recreated before the test class execution.

Should I use #Transactional or #Rollback for DAO test classes?

I have a dao layer for my database. Now, I am writing some integration tests for it. I wonder if #Transactional or #Rollback should be used in a test class, as they both revert the changes to the database. Which one would be a good practice and in what conditions?
I tried using both of them and they both work in my case. I have a #Before annotated method in my class.
#RunWith(SpringRunner.class)
#AutoConfigureTestDatabase(replace = NONE)
#DataJpaTest
// #Transactional or #Rollback?
public class TestDao {
#Autowired
private ConcreteDao concreteDao;
#Before
public void cleanUp(){ . . . }
#Test
public void testSaveAllEntries(){ . . . }
// and other tests
}

agree with #michael Don't make your tests #transactional at all (but your service layer)This means that all your service layer/persistence layer methods invoked through the tests would start their own transactions (you made them transactional, did you?) and flush the changes upon commit. So you are guaranteed to notice if something blows up on flush, and quite possible that after a while, a database full of junk test data

Running tests with databases usually will be done with integration tests. To keep it simple you could setup a h2 with the dialect you need. Prepare the database structures and run the service calls you need. assert the expecting results and mark the test method as dirties context (as annotation), or resetup the database with each test, otherwise saved results of a test could have impact of another test. In this way you can also test the transaction handling of your services.
Adding transactional to your tests will change your behaviour of your business logic. keep those out of your tests.

How can I persist entities during a SpringBootTest integration test

I am writing an integration test for a SpringBoot 2 RestController. I want to test 404 behaviour and creation of entities. However, when I try to create entities and persist them before or during a test, they are not persisted in the SpringBoot context. By that I mean they are visible in the test context (during debugging of the test) but not for the Controller (ie it does not find them and my tests fail). What am I doing wrong?
How can I persist entities and flush the context during a test so that code that is called during an integration test sees them? I don't want to use a #before annotation to populate a database because I want to do it in my #test methods.
Here is my code. Thanks
#RunWith(SpringRunner.class)
#Transactional
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class InvoiceControlllerIT extends GenericControllerIT {
#Autowired
EntityManager entityManager;
#Test
#Transactional
public void cascadesChildEntityAssociationOnCreate() throws IOException {
assertThat(invoicerRepository.count(), equalTo(0L));
assertThat(invoiceRepository.count(), equalTo(0L));
assertThat(invoiceeRepository.count(), equalTo(0L));
// create an invoicee
Invoicee savedInvoicee = invoiceeRepository.save(new Invoicee());
assertThat(invoiceeRepository.count(), equalTo(1L));
// create an invoicer
Invoicer savedInvoicer = invoicerRepository.save(new Invoicer());
assertThat(invoicerRepository.count(), equalTo(1L));
// THIS IS THE PROBLEM, FLUSHING DURING THE TEST DOES NOT EFFECT THE CONTROLLERS ABILITY TO SEE THE NEWLY CREATED ENTITIES
entityManager.flush();
// create input
InvoiceInputDto inputDto = InvoiceInputDto
.builder()
.invoicee(savedInvoicee.getId())
.invoicer(savedInvoicer.getId())
.name("test-name")
.build();
// make call
ResponseEntity<InvoiceDto> response = template.postForEntity(url("/invoices", TOKEN), inputDto, InvoiceDto.class);
assertThat(response.getStatusCode(), equalTo(HttpStatus.CREATED));
assertThat(response.getBody().getName(), equalTo(inputDto.getName()));
// check associations
assertThat(invoiceeRepository.findById(savedInvoicee.getId()).get().getInvoices(), hasSize(1));
}
}

According to the docs:
If your test is #Transactional, it rolls back the transaction at the
end of each test method by default. However, as using this arrangement
with either RANDOM_PORT or DEFINED_PORT implicitly provides a real
servlet environment, the HTTP client and server run in separate
threads and, thus, in separate transactions. Any transaction initiated
on the server does not roll back in this case
(source: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-testing.html#boot-features-testing-spring-boot-applications)
Since the test transaction is separate from the HTTP server transaction, the controller won't see changes made from within the test method until the test transaction is actually committed. Conversely, you won't be able to roll back changes made as a result to the server call.
You will seriously make your life easier by providing a mock implementation for whatever service/repository your controller uses. Alternatively, you could use a tool like DBUnit to setup and tear down the database around each test case.

This worked for me:
#Inject
private EntityManagerFactory entityManagerFactory;
#BeforeEach
void setUp() {
EntityManager entityManager = entityManagerFactory.createEntityManager();
entityManager.getTransaction().begin();
entityManager.persist(someEntity());
entityManager.getTransaction().commit();
}

Spring unit test case is not rolling back insertion of a record

The following test cases are functionally working properly, but one of the test methods having to create a new article in the database doesn't rollback at the end of the test case execution.
I expect it to work that way. For a test case that update article actually rollbacks the update at the end of test case execution.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(value = "/applicationContext-test.xml")
#TransactionConfiguration(transactionManager = "txManager", defaultRollback = true)
#Transactional
public class PriceRepositoryTest {
#Resource(name ="repository")
private PriceRepository repository;
#Test
public void testGetAll() throws Exception {
Assert.assertEquals(8, repository.getAll().size());
}
#Test
#Rollback
public void shouldSaveNewArticle(){
Article article = new Article();
article.setName("Article");
article.setPrice(33);
repository.save(article);
Assert.assertEquals(9, repository.getAll().size());
}
#Test
#Rollback
public void shouldUpdateArticle(){
Article article = repository.getArticle(4);
article.setPrice(33);
repository.update(article);
Assert.assertEquals(String.valueOf(33.0), String.valueOf(repository.getArticle(4).getPrice()));
}
}

Is your DAO perhaps also marked with #Transactional? If it is, that's the problem: the transactionality on a different layer doesn't know about your local transaction configuration.
If repository.update(article) is #Transactional, it may or may not start a new transaction (depending on the value of the propagation attribute), but it will commit the transaction after execution, and there's nothing your test method can do to intercept that.
That's one of the reasons why transactions should be started on the service level, not the DAO level.
(If that's not the case, I humbly apologize)

I also encountered this issue and spent some hours trying to find the root cause. The issue was a variant of the problems described here. In my case, the application calls stored procedures through Spring, and one of these procedures contained a COMMIT statement.
Commits should always be controlled by the app, if there is a stray commit somewhere in a stored procedure, Spring can't control the transaction and tests won't roll back.

I just encountered this with my unit tests all set to rollback and my record was still showing up in the database after the test finished. The reason was there was that in the DAO a call to the entitymanager to flush() was in the method - which forced the transaction to commit.
em.persist(jpaServer);
em.flush(); //commits the record no matter what spring is setup to do
taking the flush out and confirmed no record. Tested the same code with the test annotated to #Rollback(false) and confirmed the record (proving the flush was not needed)
Ben
www.nimbits.com