We Keep Coding

Validate repository state after producing request to Kafka test container

I am creating an integration test with Kafka and Postgres test containers, such as:
#Slf4j
#SpringBootTest
#Testcontainers
#EnableKafka
#ContextConfiguration(
initializers = {MyContainersInitializer.class} //test containers are initialized here
)
class MyIntegrationTest {
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
#Autowired
private MyRepository myRepository;
#MethodSource("testCases")
#ParameterizedTest(name = "#{index} {0}")
public void myTest(MyTestcase myTestcase) {
kafkaTemplate.send(
env.getProperty(KEY_TOPIC),
myTestcase.input()
);
//... backoff ...
assertEquals(myTestcase.expected(), myRepository.findById(myTestcase.input().getId());
}
I've confirmed that everything is processed correctly, i.e. the request is received by Kafka and is processed asynchronously by the application and inserted in the database. However the test is unable to see those changes in its final step, even if I add a backoff period.
I've noticed that if a #Transactional annotation is added to the application service it does the trick, unfortunately I am not allowed to do it (I don't have ownership), hence I was wondering if there is another way?
Thank you for your attention

I would advise against using #Transactional to solve this issue.
As the kafka data will be received in a different thread (possibly), you are better off testing that the message is received in one test and then testing if the receiving method itself saves the data in another test.
I would bet that the async nature of this will make the use of #Transactional a flaky test if you plan on doing just one test.
You can also use a Mockito verify to see if the desired function has been called.

How to use MockIntegration to build mocks for Spring Integration beans

I met a problem in test of my application,I dont understand what i need to do if i want to replace ImapIdleChannelAdapter as it is
written in the spring documentation
public class ImapConfiguration{
#Bean
ImapMailReceiver getReceiver() {
ImapMailReceiver receiver = new ImapMailReceiver(ImapConfig.getUri());
return receiver;
}
#Bean
ImapIdleChannelAdapter getAdapter(ImapMailReceiver receiver) {
ImapIdleChannelAdapter adapter = new InternalImapIdleChannelAdapter(receiver);
adapter.setAutoStartup(true);
return adapter;
}
#Bean
StandardIntegrationFlow getFlow(ImapIdleChannelAdapter adapter, GenericHandler handler) {
return IntegrationFlows.from(adapter)
.handle(handler)
.get();
}
}
In the spring integration documentation in the MockIntegration section says that "The MockIntegration factory provides an API to build mocks for Spring Integration beans that are parts of the integration flow (MessageSource, MessageProducer, MessageHandler, and MessageChannel).You can use the target mocks during the configuration phase as well as in the target test method to replace the real endpoints before performing verifications and assertions". I haven't found any examples using MessageProducer in the spring integration documentation and the Spring Integration Samples repository on github. I wrote test to try replace ImapIdleChannelAdapter
#SpringBootTest(classes = ImapConfiguration.class)
#Import({ReceiverTestConf.class})
#SpringIntegrationTest(noAutoStartup = "inboundChannelAdapter")
public class ImapMailReceiverTest {
#Captor
ArgumentCaptor<ReceivedMail> emailCaptor = ArgumentCaptor.forClass(ReceivedMail.class);
#MockBean
TestEmailHandler emailHandlerTestImpl;
#Autowired
TestImapReceiver imapReceiver;
#Autowired
MockIntegrationContext mockIntegrationContext;
#Test
#SneakyThrows
void receive() throws MessagingException {
Mockito.doNothing().when(emailHandlerTestImpl).handle(Mockito.any());
MessageSource<MimeMessage> message = () -> {
return new GenericMessage<>("testMessage");
};
this.mockIntegrationContext.substituteMessageSourceFor("imapIdleChannelAdapter", MockIntegration.mockMessageSource(message));
idleChannelAdapter.start();
await().atMost(1, TimeUnit.SECONDS).untilAsserted(() -> {
Mockito.verify(emailHandlerTestImpl, Mockito.times(1)).handle(emailCaptor.capture());
List<ReceivedMail> result = emailCaptor.getAllValues();
Assertions.assertEquals(1, result.size());
}
);
}
When I run the test, I am getting the exception.
Bean named 'imapIdleChannelAdapter' is expected to be of type 'org.springframework.integration.endpoint.SourcePollingChannelAdapter' but was actually of type 'com.test.emailadapter.imap.InternalImapIdleChannelAdapter'
org.springframework.beans.factory.BeanNotOfRequiredTypeException: Bean named 'imapIdleChannelAdapter' is expected to be of type 'org.springframework.integration.endpoint.SourcePollingChannelAdapter' but was actually of type 'com.test.emailadapter.imap.InternalImapIdleChannelAdapter'
at app//org.springframework.beans.factory.support.AbstractBeanFactory.adaptBeanInstance(AbstractBeanFactory.java:417)
at app//org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:398)
at app//org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:213)
at app//org.springframework.integration.test.context.MockIntegrationContext.substituteMessageSourceFor(MockIntegrationContext.java:217)
at app//org.springframework.integration.test.context.MockIntegrationContext.substituteMessageSourceFor(MockIntegrationContext.java:157)
at app//org.springframework.integration.test.context.MockIntegrationContext.substituteMessageSourceFor(MockIntegrationContext.java:142)

I believe the sentence in the doc needs some improvements. I definitely remember that there were some ambitions to be able mock everything in the flow. Therefore we mention over there a MessageProducer and MessageChannel as well. However in practice it turns out that we don't need to mock message channels since they can be supplied with ChannelInterceptor to verify various interaction with the channel in the flow.
The MessageProducer is also pointless to mock since you simply can emit a test message into the channel this producer is going to produce in the production. So, what you need so far is just stop this MessageProducer before the test and deal with its channel in the test already.
I see you already do a proper noAutoStartup = "inboundChannelAdapter" for your test class.
Since you don't have channel declared in your flow, the channel is auto-created by the framework with the pattern for name: [IntegrationFlow.beanName].channel#[channelNameIndex]. So, the output channel for your IntegrationFlows.from(adapter) is a DirectChannel with a getFlow.channel#0 bean name.
Please, consider to raise a GH issue, so we will improve the doc for that MockIntegration.

#SpringBootTest requiring database connection?

I have some integration tests, for which I am using Testcontainers. But I have suddenly realized that when my docker container with database for my application is down, all the other tests (excluding the integration tests using Testcontainers) are failing (even the contextLoads() test generated by Spring Boot initializr)
I get:
java.lang.IllegalStateException: Failed to load ApplicationContext at
org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:132)
Caused by: org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'liquibase' defined in class path
resource
[org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]:
Invocation of init method failed; nested exception is
liquibase.exception.DatabaseException:
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications
link failure
It is obvious that the application wants to connect to the database, and the database container is down.
I've been investigating, but I don't remember ever needing to start a container just for the test/build process of an application, so this problem is new for me. But if there is something done wrong, it could be here, in my AbstractDatabaseIT class:
#ActiveProfiles("test")
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
#ContextConfiguration(initializers = AbstractDatabaseIT.DockerMySqlDataSourceInitializer.class)
#Testcontainers
public abstract class AbstractDatabaseIT {
private static final String MYSQL_IMAGE_NAME = "mysql:5.7.24";
public static final MySQLContainer<?> mySQLContainer = new MySQLContainer<>(MYSQL_IMAGE_NAME);
static {
mySQLContainer.start();
}
public static class DockerMySqlDataSourceInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(#NotNull ConfigurableApplicationContext applicationContext) {
Map<String, String> parameters = new HashMap<>();
parameters.put("command", "--character-set-server=utf8");
TestPropertySourceUtils.addInlinedPropertiesToEnvironment(
applicationContext,
"spring.datasource.url=" + mySQLContainer.getJdbcUrl(),
"spring.datasource.username=" + mySQLContainer.getUsername(),
"spring.datasource.password=" + mySQLContainer.getPassword()
);
mySQLContainer.setParameters(parameters);
}
}
}
The integration test extend this class:
public class ChallengeIT extends AbstractDatabaseIT {
#Autowired
private ChallengeRepository repository;
// tests here
All the other, non-integration classes have #SpringBootTest annotation, and the dependencies injected using #Autowired (maybe this is a problem here?)
#SpringBootTest
class EthMessageVerifierTest {
#Autowired
private EthMessageVerifier ethMessageVerifier;
// tests here
What am I missing here? I remember seeing the H2 database dependency all around many projects. Should I drop the testcontainers in favour of H2? Or can I somehow create a single testcontainer instance for all the other tests?

Tests that you annotate with #SpringBootTest try to populate the entire Spring context. This includes all your beans: your web layer, your business logic, your database setup, etc.
Hence all the infrastructure (e.g. messaging queues, remote systems, databases) that you need otherwise to run your entire application also needs to be present for such tests.
So #SpringBootTest also indicates an integration test and you need to provide your database setup as on application start, Spring Boot's auto-configuration tries to configure your DataSource.
For more information, consider this article on #SpringBootTest and this general overview about unit & integration testing with Spring Boot. You don't always have to use #SpringBootTest and can also use one of Spring Boots many test slice annotations to test something in isolation.

Resolving port already in use in a Spring boot test DEFINED PORT

I have a spring boot application that fires up and executes a class that listens to Application Ready event to call an external service to fetch some data and then use that data to push some rules to the classpath for execution. For local testing we have mocked the external service within our application which works fine during the application startup.
The issue is while testing the application by running it with spring boot test annotation and embedded jetty container either on :
RANDOM PORT
DEFINED PORT
In case of RANDOM PORT, at the application startup, it picks up the url for the mock service from the properties file at a defined port and has no clue where the embedded container is running since it is randomly picked up, hence failing to give response.
In case of DEFINED PORT, for the first test case file it runs successfully, but the moment next file is picked up, it fails saying the port is already in use.
The test cases are partitioned logically in multiple files and need
the external service to be called before the container starts to load
the rules.
How can I either share the embedded container between test files in case of using defined port or refactor my application code instead to get hold of the random port while starting up during the test case execution.
Any help would be appreciated.
Application Startup code :
#Component
public class ApplicationStartup implements ApplicationListener<ApplicationReadyEvent> {
#Autowired
private SomeService someService;
#Override
public void onApplicationEvent(ApplicationReadyEvent arg0) {
try {
someService.callExternalServiceAndLoadData();
}
catch (Execption e) {}
}
}
Test Code Annotations: Test1
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
#TestPropertySource("classpath:test-application.properties")
public class Test1 {
#Autowired
private TestRestTemplate restTemplate;
#Test
public void tc1() throws IOException {.....}
Test Code Annotations: Test2
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
#TestPropertySource("classpath:test-application.properties")
public class Test2 {
#Autowired
private TestRestTemplate restTemplate;
#Test
public void tc1() throws IOException {.....}

If you insist on using the same port on multiple test, you can prevent spring from caching the context for further tests by annotating your testclass with: #DirtiesContext
In your case:
#RunWith(SpringRunner.class)
#DirtiesContext
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
#TestPropertySource("classpath:test-application.properties")
Here is a quote from Andy Wilkinson from his answer on this discussion
This is working as designed. Spring Framework's test framework will, by default, cache contexts for possible reuse by multiple test classes. You have two tests with different configuration (due to #TestPropertySource) so they will use different application contexts. The context for the first test will be cached and kept open while the second test is running. Both tests are configured to use the same port for Tomcat's connector. As a result, when the second test is run, the context fails to start due to a port clash with the connector from the first test. You have a few options:
Use RANDOM_PORT
Remove #TestPropertySource from Test2 so that the contexts have identical configuration and the context from the first test can be reused for the second test.
Use #DirtiesContext so that the context isn't cached

I ran across the same issue. I know this question is a little old, but this may be of assistance:
Tests that use #SpringBootTest(webEnvironment=WebEnvironment.RANDOM_PORT) can also inject the actual port into a field by using the #LocalServerPort annotation, as shown in the following example:
Source: https://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#howto-user-a-random-unassigned-http-port
The code example given is:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(webEnvironment=WebEnvironment.RANDOM_PORT)
public class MyWebIntegrationTests {
#Autowired
ServletWebServerApplicationContext server;
#LocalServerPort
int port;
// ...
}

in application.properties
server.port=0
will run the application in random ports

Unit testing of Spring Boot Actuator endpoints not working when specifying a port

recently I changed my spring boot properties to define a management port.
In doing so, my unit tests started to fail :(
I wrote a unit test that tested the /metrics endpoint as follows:
#RunWith (SpringRunner.class)
#DirtiesContext
#SpringBootTest
public class MetricsTest {
#Autowired
private WebApplicationContext context;
private MockMvc mvc;
/**
* Called before each test.
*/
#Before
public void setUp() {
this.context.getBean(MetricsEndpoint.class).setEnabled(true);
this.mvc = MockMvcBuilders.webAppContextSetup(this.context).build();
}
/**
* Test for home page.
*
* #throws Exception On failure.
*/
#Test
public void home()
throws Exception {
this.mvc.perform(MockMvcRequestBuilders.get("/metrics"))
.andExpect(MockMvcResultMatchers.status().isOk());
}
}
Previously this was passing. After adding:
management.port=9001
The tests started failing with:
home Failed: java.lang.AssertionError: Status expected: <200> but was: <404>
I tried changing the #SpringBootTest annotation with:
#SpringBootTest (properties = {"management.port=<server.port>"})
Where is the number used for the server.port. This didn't seem to make any difference.
So then changed the management.port value in the property file to be the same as the server.port. Same result.
The only way to get the test to work is remove the management.port from the property file.
Any suggestions/thoughts ?
Thanks

For Spring Boot 2.x the integration tests configuration could be simplified.
For example simple custom heartbeat endpoint
#Component
#Endpoint(id = "heartbeat")
public class HeartbeatEndpoint {
#ReadOperation
public String heartbeat() {
return "";
}
}
Where integration test for this endpoint
#SpringBootTest(
classes = HeartbeatEndpointTest.Config.class,
properties = {
"management.endpoint.heartbeat.enabled=true",
"management.endpoints.web.exposure.include=heartbeat"
})
#AutoConfigureMockMvc
#EnableAutoConfiguration
class HeartbeatEndpointTest {
private static final String ENDPOINT_PATH = "/actuator/heartbeat";
#Autowired
private MockMvc mockMvc;
#Test
void testHeartbeat() throws Exception {
mockMvc
.perform(get(ENDPOINT_PATH))
.andExpect(status().isOk())
.andExpect(content().string(""));
}
#Configuration
#Import(ProcessorTestConfig.class)
static class Config {
#Bean
public HeartbeatEndpoint heartbeatEndpoint() {
return new HeartbeatEndpoint();
}
}
}

For Spring boot test we need to specify the port it needs to connect to.
By default, it connects to server.port which in case of actuators is different.
This can be done by
#SpringBootTest(properties = "server.port=8090")
in application.properties we specify the management port as below
...
management.server.port=8090
...

Did you try adding the following annotation to your test class?
#TestPropertySource(properties = {"management.port=0"})
Check the following link for reference.

Isn't there an error in the property name?
Shouldn't be
#TestPropertySource(properties = {"management.server.port=..."}) instead of #TestPropertySource(properties = {"management.port=.."})

The guide stated that this can be achieved with #AutoConfigureMetrics.
And I moved with this.
Regardless of your classpath, meter registries, except the in-memory backed, are not auto-configured when using #SpringBootTest.
If you need to export metrics to a different backend as part of an integration test, annotate it with #AutoConfigureMetrics.
https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.testing.spring-boot-applications.metrics

Had the same issue, you just have to make the management.port null by adding this in your application-test.properties (set it to empty value)
management.port=
Make sure you use the test profile in your JUnit by annotating the class with
#ActiveProfiles("test")

Try using
#SpringBootTest(properties = {"management.port="})
Properties defined in the #SpringBootTest annotation have a higher precedence than those in application properties. "management.port=" will "unset" the management.port property.
This way you don't have to worry about configuring the port in your tests.

I was facing the same issue and tried several things but this is how I was able to solve mine without making any change in the application.yaml
Sample actuator endpoint
#Component
#RestControllerEndpoint(id = "endpoint")
public class SampleEndpoint
{
#GetMapping
public String sampleEndpoint(){
return ""
}
}
Unit test case
#RunWith(SpringRunner.class)
#SpringBootTest(
classes = {SampleEndpointTest.Config.class},
properties = {"management.server.port="}
)
#AutoConfigureMockMvc
public class SampleEndpointTest
{
#Autowired
private MockMvc mockMvc;
#SpringBootApplication(scanBasePackageClasses = {SampleEndpoint.class})
public static class Config
{
}
#Test
public void testSampleEndpoint() throws Exception
{
mockMvc.perform(
MockMvcRequestBuilders.get("/actuator/enpoint").accept(APPLICATION_JSON)
).andExpect(status().isOk());
}

Since now info endpoint must be enabled manually make sure the SpringBootTest tag includes this in properties, like this:
#SpringBootTest(
properties = {
"management.info.env.enabled=true" ,
"management.endpoints.web.exposure.include=info, health"
})

I had this problem recently, and as none of the above answers made any sense to me, I decided to do a bit more reading. In my case, I had already defined both server.port and management.server.port as 8091 in my test application-test.yaml file, and could not understand why my test was getting a connection refused error message.
It turns out that instead of using the annotation #SpringBootTest() I needed to use #SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT) - which causes the port numbers in the yaml file to be used. This is briefly discussed in the manual. Quoting the relevant section:
DEFINED_PORT — Loads an EmbeddedWebApplicationContext and provides a real servlet environment. Embedded servlet containers are started and listening on a defined port (i.e from your application.properties or on the default port 8080).
It seems in SpringBootTest the default is to avoid starting a real servlet environment, and if no WebEnvironment is explicitly specified then SpringBootTest.WebEnvironment.MOCK is used as a default.

After a long search: There is this nice Springboot annotation called #LocalManagementPort!
It works similar to #LocalServerPort but for actuator endpoins.
An example config would look as follows
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class MetricsIT {
#Autowired
RestTemplateBuilder restTemplateBuilder;
#LocalManagementPort
int managementPort;
#Test
public void testMetrics(){
ResponseEntity<String> response = restTemplateBuilder
.rootUri("http://localhost:" + managementPort + "/actuator")
.build().exchange("/metrics", HttpMethod.GET, new HttpEntity<>(null), String.class);
}
}