We Keep Coding

#SpringBootTest requiring database connection?

I have some integration tests, for which I am using Testcontainers. But I have suddenly realized that when my docker container with database for my application is down, all the other tests (excluding the integration tests using Testcontainers) are failing (even the contextLoads() test generated by Spring Boot initializr)
I get:
java.lang.IllegalStateException: Failed to load ApplicationContext at
org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:132)
Caused by: org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'liquibase' defined in class path
resource
[org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]:
Invocation of init method failed; nested exception is
liquibase.exception.DatabaseException:
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications
link failure
It is obvious that the application wants to connect to the database, and the database container is down.
I've been investigating, but I don't remember ever needing to start a container just for the test/build process of an application, so this problem is new for me. But if there is something done wrong, it could be here, in my AbstractDatabaseIT class:
#ActiveProfiles("test")
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
#ContextConfiguration(initializers = AbstractDatabaseIT.DockerMySqlDataSourceInitializer.class)
#Testcontainers
public abstract class AbstractDatabaseIT {
private static final String MYSQL_IMAGE_NAME = "mysql:5.7.24";
public static final MySQLContainer<?> mySQLContainer = new MySQLContainer<>(MYSQL_IMAGE_NAME);
static {
mySQLContainer.start();
}
public static class DockerMySqlDataSourceInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(#NotNull ConfigurableApplicationContext applicationContext) {
Map<String, String> parameters = new HashMap<>();
parameters.put("command", "--character-set-server=utf8");
TestPropertySourceUtils.addInlinedPropertiesToEnvironment(
applicationContext,
"spring.datasource.url=" + mySQLContainer.getJdbcUrl(),
"spring.datasource.username=" + mySQLContainer.getUsername(),
"spring.datasource.password=" + mySQLContainer.getPassword()
);
mySQLContainer.setParameters(parameters);
}
}
}
The integration test extend this class:
public class ChallengeIT extends AbstractDatabaseIT {
#Autowired
private ChallengeRepository repository;
// tests here
All the other, non-integration classes have #SpringBootTest annotation, and the dependencies injected using #Autowired (maybe this is a problem here?)
#SpringBootTest
class EthMessageVerifierTest {
#Autowired
private EthMessageVerifier ethMessageVerifier;
// tests here
What am I missing here? I remember seeing the H2 database dependency all around many projects. Should I drop the testcontainers in favour of H2? Or can I somehow create a single testcontainer instance for all the other tests?

Tests that you annotate with #SpringBootTest try to populate the entire Spring context. This includes all your beans: your web layer, your business logic, your database setup, etc.
Hence all the infrastructure (e.g. messaging queues, remote systems, databases) that you need otherwise to run your entire application also needs to be present for such tests.
So #SpringBootTest also indicates an integration test and you need to provide your database setup as on application start, Spring Boot's auto-configuration tries to configure your DataSource.
For more information, consider this article on #SpringBootTest and this general overview about unit & integration testing with Spring Boot. You don't always have to use #SpringBootTest and can also use one of Spring Boots many test slice annotations to test something in isolation.

Running SpringBootTest Context Without Testcontainers Launch

I have 2 parent test classes:
#SpringBootTest(properties = {
"spring.datasource.url=jdbc:tc:mysql:8.0.25:///my_test_db?TC_INITSCRIPT=db/init_mysql.sql",
"spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver"
})
#TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
public abstract class UserApplicationIntegrationTest {
}
and
#SpringBootTest
#TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
public abstract class UserApplicationTest {
}
The idea is for various test classes to extend these classes. The ones which require a mocked MySQL DB will extend UserApplicationIntegrationTest. Ones which don't need a DB connection but that do require a Spring context will extend UserApplicationTest.
In the absence of UserApplicationIntegrationTest, all the test classes extending UserApplicationTest work well, including using the Mockito framework. Unfortunately, when I introduce UserApplicationIntegrationTest and its sub-tests (which work perfectly with the dockerised db instance), these tests begin to fail as they suddenly demand a datasource.
Caused by: org.springframework.boot.autoconfigure.jdbc.DataSourceProperties$DataSourceBeanCreationException: Failed to determine a suitable driver class
If I try excluding datasource auto-configuration either in app properties or in annotations of the parent class, the testcontainers tests (those extending UserApplicationIntegrationTest) start failing because of a problem with the Spring context and not being able to autowire beans any longer in those tests.
Before I know it, I'm down a rabbit hole of attempting messy exclusions/additions that I've been down before in previous projects and it only leads to problems further down the line.
Essentially I want 3 types of tests coexisting in my project:
Unit tests with no Spring context
Unit tests with a Spring context (including lots of mocking but still autowiring/constructor injection support)
Integration tests with a Spring context that spin up testcontainers and allow me to test DB interactions (and potentially end to end tests to come)
The original reason that I wanted to avoid launching testcontainers for all Spring context tests (which would 'work' perfectly well and only include 1 docker delay in the build process) was because it was irritating me to have to wait for the mysql connection to the dockerised instance every time I ran individual Spring context tests locally during development.
Is there a tidy way to achieve this or an altogether better way of navigating the requirement?
Thanks in advance.

Hopefully I understand you right, what I did was implementing an abstract TestContainer test class:
package de.dwosch.it;
#ActiveProfiles("test")
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
#ContextConfiguration(initializers = AbstractPostgreSQLTestContainerIT.Initializer.class)
#Testcontainers
public abstract class AbstractPostgreSQLTestContainerIT {
private static final String POSTGRES_VERSION = "postgres:11.1";
public static PostgreSQLContainer database;
static {
database = new PostgreSQLContainer(POSTGRES_VERSION);
database.start();
}
static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(ConfigurableApplicationContext configurableApplicationContext) {
TestPropertySourceUtils.addInlinedPropertiesToEnvironment(
configurableApplicationContext,
"spring.datasource.url=" + database.getJdbcUrl(),
"spring.datasource.username=" + database.getUsername(),
"spring.datasource.password=" + database.getPassword()
);
}
}
}
Then I just extend my test classes by this abstract class which will fire up a test container and the whole spring context for better separation
class MyAwesomeControllerIT extends AbstractPostgreSQLTestContainerIT { }

How to manage spring hierarchy during unit/integration tests?

I am working on a Spring-enabled embedded Tomcat application using annotation-based configuration. The application uses Spring MVC Controllers for its REST endpoints. As a separation of concerns, and to avoid having duplicate beans in separate contexts, the parent context contains all beans that are not REST endpoints, and the Spring Web MVC context contains all beans that are REST endpoints.
I want to write new and refactor old integration tests for these endpoints that are representative of the structure of the app. There are existing test classes like so:
import com.stuff.web.MyEndpoint;
#Configuration
#ComponentScan(basePackages = {"com.stuff"})
public class SpringConfig { ... }
#WebAppConfiguration
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = {SpringConfig.class})
public class TestMyEndpoint {
#Autowired
private MyEndpoint myEndpoint;
private MockMvc mockMvc;
#Before
public void setUp() {
mockMvc = MockMvcBuilders
.standaloneSetup(myEndpoint)
.build();
}
#Test
public void testMyEndpoint() throws Exception {
mockMvc.perform(get("/myendpoint")
.accept(MediaType.APPLICATION_JSON_VALUE))
.andExpect(status().isOk())
.andReturn();
}
}
The problem is that the context that I am using for this test now has every bean loaded, whereas I would like to ensure that there are not non-REST beans loaded that call into RestController beans during the execution of the tests.
Adding something like
#Configuration
#ComponentScan(basePackages = {"com.stuff"},
excludeFilters = {
#Filter(type = FilterType.REGEX, pattern = "com.stuff.web.*")})
public class SpringConfig { ... }
Would ensure the kind of separation I'm going for, but then I don't have access to the com.stuff.web.MyEndpoint class that I'm trying to test.
Am I missing something easy? Let me know if I'm explaining the situation clearly.

The kind of separation you're describing (mvc vs non-mvc) made sense 10 years ago, not anymore. Separate your code by functionality/design patterns (web/service/repository etc), and have #Configuration classes specific to that layer. The Spring stereotype annotations are good enough hint how your app should be broken up. Then, put your tests in the same package as your target code, and mock/override any dependencies.
It doesn't appear you're using Spring Boot (you really should) but they have a great section in the docs for testing "slices" of your application.
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-testing.html#boot-features-testing-spring-boot-applications-testing-autoconfigured-tests

Resolving port already in use in a Spring boot test DEFINED PORT

I have a spring boot application that fires up and executes a class that listens to Application Ready event to call an external service to fetch some data and then use that data to push some rules to the classpath for execution. For local testing we have mocked the external service within our application which works fine during the application startup.
The issue is while testing the application by running it with spring boot test annotation and embedded jetty container either on :
RANDOM PORT
DEFINED PORT
In case of RANDOM PORT, at the application startup, it picks up the url for the mock service from the properties file at a defined port and has no clue where the embedded container is running since it is randomly picked up, hence failing to give response.
In case of DEFINED PORT, for the first test case file it runs successfully, but the moment next file is picked up, it fails saying the port is already in use.
The test cases are partitioned logically in multiple files and need
the external service to be called before the container starts to load
the rules.
How can I either share the embedded container between test files in case of using defined port or refactor my application code instead to get hold of the random port while starting up during the test case execution.
Any help would be appreciated.
Application Startup code :
#Component
public class ApplicationStartup implements ApplicationListener<ApplicationReadyEvent> {
#Autowired
private SomeService someService;
#Override
public void onApplicationEvent(ApplicationReadyEvent arg0) {
try {
someService.callExternalServiceAndLoadData();
}
catch (Execption e) {}
}
}
Test Code Annotations: Test1
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
#TestPropertySource("classpath:test-application.properties")
public class Test1 {
#Autowired
private TestRestTemplate restTemplate;
#Test
public void tc1() throws IOException {.....}
Test Code Annotations: Test2
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
#TestPropertySource("classpath:test-application.properties")
public class Test2 {
#Autowired
private TestRestTemplate restTemplate;
#Test
public void tc1() throws IOException {.....}

If you insist on using the same port on multiple test, you can prevent spring from caching the context for further tests by annotating your testclass with: #DirtiesContext
In your case:
#RunWith(SpringRunner.class)
#DirtiesContext
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
#TestPropertySource("classpath:test-application.properties")
Here is a quote from Andy Wilkinson from his answer on this discussion
This is working as designed. Spring Framework's test framework will, by default, cache contexts for possible reuse by multiple test classes. You have two tests with different configuration (due to #TestPropertySource) so they will use different application contexts. The context for the first test will be cached and kept open while the second test is running. Both tests are configured to use the same port for Tomcat's connector. As a result, when the second test is run, the context fails to start due to a port clash with the connector from the first test. You have a few options:
Use RANDOM_PORT
Remove #TestPropertySource from Test2 so that the contexts have identical configuration and the context from the first test can be reused for the second test.
Use #DirtiesContext so that the context isn't cached

I ran across the same issue. I know this question is a little old, but this may be of assistance:
Tests that use #SpringBootTest(webEnvironment=WebEnvironment.RANDOM_PORT) can also inject the actual port into a field by using the #LocalServerPort annotation, as shown in the following example:
Source: https://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#howto-user-a-random-unassigned-http-port
The code example given is:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(webEnvironment=WebEnvironment.RANDOM_PORT)
public class MyWebIntegrationTests {
#Autowired
ServletWebServerApplicationContext server;
#LocalServerPort
int port;
// ...
}

in application.properties
server.port=0
will run the application in random ports

How can I make a unit test that tests my Spring/OSGi configuration?

I have a bundle-context-osgi.xml file that lists services my project takes and services it publishes. I also have a bundle-context.xml that defines the beans defined in my application. How can I write a unit test that ensures that everything is wired correctly and I can product the services that my application is supposed to provide when it is on the server?
Note: it might be wrong to call this a "unit test" but I think the idea is clear.

For OSGi Integration Tests I usually use Pax Exam. It will start a osgi container publish all configured bundles and will create a tiny-bundle from the testsources to be deployed as a OSGi bundle.
With it you are able to access all registered services even by #Inject means.
Best to take a look at the Pax Exam documentation.
In short you create a new integration test module, where you configure your dependencies in the Test case:
#RunWith(PaxExam.class)
#ExamReactorStrategy(PerMethod.class)
public class MyTest {
#Inject
protected BundleContext bundleContext;
#Inject
MyService service;
...
#Configuration
public Option[] configure() {
return options(
workingDirectory("target/paxexam/"),
cleanCaches(true),
junitBundles(),
mavenBundle().groupId("my.group.id")
.artifactId("my-artifact")
.version("1.0.0")
...
);
}
...
#Test
public void test() throws Exception {
assertNotNull(service);
}
}