I am creating an integration test with Kafka and Postgres test containers, such as:
#Slf4j
#SpringBootTest
#Testcontainers
#EnableKafka
#ContextConfiguration(
initializers = {MyContainersInitializer.class} //test containers are initialized here
)
class MyIntegrationTest {
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
#Autowired
private MyRepository myRepository;
#MethodSource("testCases")
#ParameterizedTest(name = "#{index} {0}")
public void myTest(MyTestcase myTestcase) {
kafkaTemplate.send(
env.getProperty(KEY_TOPIC),
myTestcase.input()
);
//... backoff ...
assertEquals(myTestcase.expected(), myRepository.findById(myTestcase.input().getId());
}
I've confirmed that everything is processed correctly, i.e. the request is received by Kafka and is processed asynchronously by the application and inserted in the database. However the test is unable to see those changes in its final step, even if I add a backoff period.
I've noticed that if a #Transactional annotation is added to the application service it does the trick, unfortunately I am not allowed to do it (I don't have ownership), hence I was wondering if there is another way?
Thank you for your attention
I would advise against using #Transactional to solve this issue.
As the kafka data will be received in a different thread (possibly), you are better off testing that the message is received in one test and then testing if the receiving method itself saves the data in another test.
I would bet that the async nature of this will make the use of #Transactional a flaky test if you plan on doing just one test.
You can also use a Mockito verify to see if the desired function has been called.
Related
I have a Spring service that does something like that :
#Service
public class MyService {
#Transactional(propagation = Propagation.NEVER)
public void doStuff(UUID id) {
// call an external service, via http for example, can be long
// update the database, with a transactionTemplate for example
}
}
The Propagation.NEVER indicates we must not have an active transaction when the method is called because we don't want to block a connection to the database while waiting for an answer from the external service.
Now, how could I properly test this and then rollback the database ? #Transactional on the test won't work, there will be an exception because of Propagation.NEVER.
#SpringBootTest
#Transactional
public class MyServiceTest {
#Autowired
private MyService myService;
public void testDoStuff() {
putMyTestDataInDb();
myService.doStuff(); // <- fails no transaction should be active
assertThat(myData).isTheWayIExpectedItToBe();
}
}
I can remove the #Transactional but then my database is not in a consistent state for the next test.
For now my solution is to truncate all tables of my database after each test in a #AfterEach junit callback, but this is a bit clunky and gets quite slow when the database has more than a few tables.
Here comes my question : how could I rollback the changes done to my database without truncating/using #Transactional ?
The database I'm testing against is mariadb with testcontainers, so a solution that would work only with mariadb/mysql would be enough for me. But something more general would be great !
(another exemple where I would like to be able to not use #Transactional on the test : sometimes I want to test that transaction boundaries are correctly put in the code, and not hit some lazy loading exceptions at runtime because I forgot a #Transactional somewhere in the production code).
Some other precisions, if that helps :
I use JPA with Hibernate
The database is create with liquibase when the application context starts
Others ideas I've played with :
#DirtiesContext : this is a lot slower, creating a new context is a lot more expensive than just truncating all tables in my database
MariaDB SAVEPOINT : dead end, it's just a way to go back to a state of the database INSIDE a transaction. This would be the ideal solution IMO if i could work globally
Trying to fiddle with connections, issuing START TRANSACTION statements natively on the datasource before the test and ROLLBACK after the tests : really dirty, could not make it work
Personal opinion: #Transactional + #SpringBootTest is (in a way) the same anti-pattern as spring.jpa.open-in-view. Yes, it's easy to get things working at first and having the automatic rollback is nice, but it loses you a lot of flexibility and control over your transactions. Anything that requires manual transaction management becomes very hard to test that way.
We recently had a very similar case and in the end we decided to bite the bullet and use #DirtiesContext instead. Yeah, tests take 30 more minutes to run, but as an added benefit the tested services behave the exact same way as in production and the tests are more likely to catch any transaction issues.
But before we did the switch, we considered using the following workaround:
Create an interface and a service similar to the following:
interface TransactionService
{
void runWithoutTransaction(Runnable runnable);
}
#Service
public class RealTransactionService implements TransactionService
{
#Transactional(propagation = Propagation.NEVER)
public void runWithoutTransaction(Runnable runnable)
{
runnable.run();
}
}
In your other service wrap the external http calls with the #runWithoutTransaction-Method, e.g.:
#Service
public class MyService
{
#Autowired
private TransactionService transactionService;
public void doStuff(UUID id)
{
transactionService.runWithoutTransaction(() -> {
// call an external service
})
}
}
That way your production code will peform the Propagation.NEVER check, and for the tests you can replace the TransactionService with a different implemention that doesn't have the #Transactional annotations, e.g.:
#Service
#Primary
public class FakeTransactionService implements TransactionService
{
// No annotation here
public void runWithoutTransaction(Runnable runnable)
{
runnable.run();
}
}
This is not limited to Propagation.NEVER. Other propagation types can be implemented in the same way:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void runWithNewTransaction(Runnable runnable)
{
runnable.run();
}
And finally - the Runnable parameter can be replaced with a Function/Consumer/Supplier if the method needs to return and/or accept a value.
This is bit of wild idea, but if you are using mysql database, then maybe switch to dolt for tests?
Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
You can wrap it as testcontainers container, load necessary data on start and then, on start of each test run dolt reset.
I have a spring boot application, I want to write a junit test cases for the void method, So I just want test whether the log contains the particular messages.
Following is my business logic class:
public void processAdjustmentFeed(){
............
if(!CollectionUtils.isEmpty(adjustmentList)){
log.info("ADJUSTMENT FEED SIZE TO PUBLISH: {}", adjustmentList.size());
// SOME BUSINESS LOGIC
} else {
log.warn("NO ADJUSTMENT FEED FOUND TO PROCESS FROM : {}", lastSuccessfulQueriedTimeStamp);
}
}
And in my test class:
#RunWith(SpringRunner.class)
#SpringBootTest
public class AdjustmentSchedulerTest {
//
#Autowired
private AdjustmentScheduler adjustmentScheduler;
#Autowired
private AdjustmentRepository adjustmentRepository;
#Autowired
private AdjustmentService adjustmentService;
#Test
public void processAdjustmentFeed_ValidAdjustmentList()
{
....
adjustmentScheduler.processAdjustmentFeed();
//HERE I NEED TO CHECK FOR THE LOG MESSAGE
}
}
How can I do it without using mockito, because am using H2 database ,I want to have a test with actual method. Can anyone please provide me some code samples to do it.
You should focus on testing // SOME BUSINESS LOGIC instead of logger. Testing logs makes your test tightly coupled and you end up testing already tested logger code. This doesn't complies with unit testing principles.
Anyway if you really don't want to use mockito, you can always write your own appender where log messages go through.
See this for example.
There are articles on how to test Spring cloud stream applications without connecting to a messaging system with spring-cloud-stream-test-support. But I want to really connect to RabbitMQ from my integration test, and cannot do that. Here is test class:
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#EnableBinding(Source.class)
public class StreamIT {
#Autowired
private Source source;
#Test
public void testMessageSending() throws InterruptedException {
source.output().send(MessageBuilder.withPayload("test").build());
System.out.println("Message sent.");
}
}
Everything is the same as in #SpringBootApplication, they use the same properties from application.yml.
But there is no log line that message is sent (o.s.a.r.c.CachingConnectionFactory : Created new connection: SpringAMQP#22e79d25:0/SimpleConnection#5cce3ab6 [delegate=amqp://guest#127.0.1.1:5672/, localPort= 60934]
),
and even if broker is not started, there is no java.net.ConnectException: Connection refused (Connection refused).
Am I doing something wrong? What is needed to create real connection to broker and send message from test?
Since you are using #SpringBootTest annotation in your test, Spring Boot will evaluate all available auto-configurations.
If you have spring-cloud-stream-test-support dependency in your test classpath then following auto-configurations will be also evaluated:
org.springframework.cloud.stream.test.binder.TestSupportBinderAutoConfiguration
org.springframework.cloud.stream.test.binder.MessageCollectorAutoConfiguration
As a result, you have only one binder in the application context - org.springframework.cloud.stream.test.binder.TestSupportBinder. By its name, you can understand that it does nothing about real binding.
Excluding/removing of spring-cloud-stream-test-support dependency from test classpath - is a dubious solution. Since it forces you to create two separate modules for unit and integration tests.
If you want to exclude previously mentioned auto-configurations in your test. You can do it as follows:
#RunWith(SpringRunner.class)
#SpringBootTest
#EnableAutoConfiguration(exclude = {TestSupportBinderAutoConfiguration.class, MessageCollectorAutoConfiguration.class})
public class StreamIT {
EDIT
You need to remove the test-support jar from the pom. It's presence (in test scope) is what triggers replacing the real binder with a test binder.
After removing the test binder support, this works fine for me...
#RunWith(SpringRunner.class)
#SpringBootTest
public class So49816044ApplicationTests {
#Autowired
private Source source;
#Autowired
private AmqpAdmin admin;
#Autowired
private RabbitTemplate template;
#Test
public void test() {
// bind an autodelete queue to the destination exchange
Queue queue = this.admin.declareQueue();
this.admin.declareBinding(new Binding(queue.getName(), DestinationType.QUEUE, "mydest", "#", null));
this.source.output().send(new GenericMessage<>("foo"));
this.template.setReceiveTimeout(10_000);
Message received = template.receive(queue.getName());
assertThat(received.getBody()).isEqualTo("foo".getBytes());
}
}
Although there is not a rabbit sample; there is a kafka sample that uses a real (embedded) kafka binder for testing, although the test jar is excluded, it doesn't explicitly say that's needed.
I'm working on a Spring-Boot web application. The usual way of writing integration tests is:
#Test
#Transactional
#Rollback(true)
public void myTest() {
// ...
}
This works nicely as long as only one thread does the work. #Rollback cannot work if there are multiple threads.
However, when testing a #RestController class using the Spring REST templates, there are always multiple threads (by design):
the test thread which acts as the client and runs the REST template
the server thread which receives and processes the request
So you can't use #Rollback in a REST test. The question is: what do you use instead to make tests repeatable and have them play nicely in a test suite?
#DirtiesContext works, but is a bad option because restarting the Spring application context after each REST test method makes the suite really slow to execute; each test takes a couple of milliseconds to run, but restarting the context takes several seconds.
First of all, testing a controller using a Spring context is no unit test. You should consider writing a unit test for the controller by using mocks for the dependencies and creating a standalone mock MVC:
public class MyControllerTest {
#InjectMocks
private MyController tested;
// add #Mock annotated members for all dependencies used by the controller here
private MockMvc mvc;
// add your tests here using mvc.perform()
#Test
public void getHealthReturnsStatusAsJson() throws Exception {
mvc.perform(get("/health"))
.andExpect(status().isOk())
.andExpect(content().contentTypeCompatibleWith(MediaType.APPLICATION_JSON))
.andExpect(jsonPath("$.status", is("OK")));
}
#Before
public void createControllerWithMocks() {
MockitoAnnotations.initMocks(this);
MockMvcBuilders.standaloneSetup(controller).build()
}
}
This even works if you use an external #ControllerAdvice for error handling etc by simply calling setControllerAdvice() on the MVC builder.
Such a test has no problems running in parallel and is much faster by no need to setup a Spring context at all.
The partial integration test you described is also useful to make sure the right wiring is used and all tested units work together as expected. But I would more go for a more general integration test including multiple/all endpoints checking if they work in general (not checking the edge cases) and mocking only services reaching out to external (like internal REST clients, replacing the database by one in memory, ...). With this setup you start with a fresh database and maybe will not even need to rollback any transaction. This is of course most comfortable using a database migration framework like Liquibase which would setup your in memory db on the fly.
Below is my implement followed by #4
private MockMvc mockMvc;
#Mock
private LoginService loginService;
#Mock
private PersonalService personalService;
#InjectMocks
private LoginController loginController;
#BeforeEach
public void setUp() {
MockitoAnnotations.openMocks(this);
mockMvc = MockMvcBuilders.standaloneSetup(loginController).build();
}
#Test
void simple_login() throws Exception {
Mockito.when(loginService.login(Mockito.anyString(), Mockito.anyString()))
.thenReturn(UserAccessData.builder()
.accessToken("access_token_content")
.refreshToken("refresh_token_count")
.build());
mockMvc.perform(
MockMvcRequestBuilders.post("/login/simple")
.contentType(MediaType.APPLICATION_JSON_VALUE)
.param("accountName", "1234561")
.param("password", "e10adc3949ba59abbe56e057f20f883e")
)
.andDo(print())
.andExpect(MockMvcResultMatchers.status().isOk())
.andExpect(jsonPath("$.tokens.accessToken", is("access_token_content")))
.andExpect(jsonPath("$.tokens.refreshToken", is("refresh_token_count")))
;
}
I'm trying to test a spring batch job that performs a read (get data from another application) process (simple calculation) and write (into the mongodb)
the reader is #StepScope
here is the postConstruct of the read task.
#PostConstruct
public void init(){
employees.addAll(getListOfEmployeesBy(affectationMotifService.findAllRegistrationNumbers()));
}
public List<EmployeeForSalaryDTO> getListOfEmployeesBy(List<String> registrationNumbers){
LOG.debug("request to get all the employees by registration numbers {}" , registrationNumbers);
return coreResourceFeign.getAllEmployeesForSalaryByRegistrationNumbers(registrationNumbers).getBody();
}
When I try to launch the test of the job or what ever test in the application. spring always runs the init() of the read task .. which will fail the test because I need to mock the coreResourceFeign.getAllEmployeesForSalaryByRegistrationNumbers(registrationNumbers) .
I can't mock the method because it runs before the test begin.
here is the test
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {SalaryApp.class, SecurityBeanOverrideConfiguration.class})
public class SalaryJobServiceTest {
#Autowired
#InjectMocks
private SalaryJobService salaryJobService;
#Test
public void startJob() throws Exception {
SalaryJobDTO SalaryJobDTO = salaryJobService.start(Collections.emptyList());
Assert.assertNotNull(salaryJobDTO.getId());
}
}
I have no idea how to deal with spring batch tests. Any recommendation or help will be welcomed.
#PostConstruct will make sure your method is called immediately after the object is created. Spring application creates all the beans as per the configs while application start. This is expected behavior. If you do not want to call your method during application start up remove #PostConstruct and you can run your test mocking the dependent objects.
Rather you should use readers read method to load your data to the reader.