We Keep Coding

Strict #MockBean in a Spring Boot Test

I am developing a Spring Boot application. For my regular service class unit tests, I am able to extend my test class with MockitoExtension, and the mocks are strict, which is what I want.
interface MyDependency {
Integer execute(String param);
}
class MyService {
#Autowired MyDependency myDependency;
Integer execute(String param) {
return myDependency.execute(param);
}
}
#ExtendWith(MockitoExtension.class)
class MyServiceTest {
#Mock
MyDependency myDependency;
#InjectMocks
MyService myService;
#Test
void execute() {
given(myDependency.execute("arg0")).willReturn(4);
myService.execute("arg1"); //will throw exception
}
}
In this case, the an exception gets thrown with the following message (redacted):
org.mockito.exceptions.misusing.PotentialStubbingProblem:
Strict stubbing argument mismatch. Please check:
- this invocation of 'execute' method:
myDependency.execute(arg1);
- has following stubbing(s) with different arguments:
1. myDependency.execute(arg0);
In addition, if the stubbing was never used there would be the following (redacted):
org.mockito.exceptions.misusing.UnnecessaryStubbingException:
Unnecessary stubbings detected.
Clean & maintainable test code requires zero unnecessary code.
Following stubbings are unnecessary (click to navigate to relevant line of code):
1. -> at MyServiceTest.execute()
However, when I use #MockBean in an integration test, then none of the strict behavior is present. Instead, the stubbed method returns null because the stubbing "fails" silently. This is behavior that I do not want. It is much better to fail immediately when unexpected arguments are used.
#SpringBootTest
class MyServiceTest {
#MockBean
MyDependency myDependency;
#Autowired
MyService myService;
#Test
void execute() {
given(myDependency.execute("arg0")).willReturn(4);
myService.execute("arg1"); //will return null
}
}
Is there any workaround for this?

Yes there are some workarounds but it is quite involved.
It may be better to just wait for Mockito 4 where the default will be strict mocks.
The first option:
Replace #MockBean with #Autowired with a test configuration with #Primary ( this should give the same effect as #MockBean, inserting it into the application as well as into the test )
Create a default answer that throws an exception for any unstubbed function
Then override that answer with some stubbing - but you have to use doReturn instead of when thenReturn
// this is the component to mock
#Component
class ExtService {
int f1(String a) {
return 777;
}
}
// this is the test class
#SpringBootTest
#RunWith(SpringRunner.class)
public class ApplicationTests {
static class RuntimeExceptionAnswer implements Answer<Object> {
#Override
public Object answer(InvocationOnMock invocation) throws Throwable {
throw new RuntimeException(
invocation.getMethod().getName() + " was not stubbed with the received arguments");
}
}
#TestConfiguration
public static class TestConfig {
#Bean
#Primary
public ExtService mockExtService() {
ExtService std = Mockito.mock(ExtService.class, new RuntimeExceptionAnswer());
return std;
}
}
// #MockBean ExtService extService;
#Autowired
ExtService extService; // replace mockBean
#Test
public void contextLoads() {
Mockito.doReturn(1).when(extService).f1("abc"); // stubbing has to be in this format
System.out.println(extService.f1("abc")); // returns 1
System.out.println(extService.f1("abcd")); // throws exception
}
}
Another possible but far from ideal option: instead of using a default answer is to stub all your function calls first with an any() matcher, then later with the values you actually expect.
This will work because the stubbing order matters, and the last match wins.
But again: you will have to use the doXXX() family of stubbing calls, and worse you will have to stub every possible function to come close to a strict mock.
// this is the service we want to test
#Component
class ExtService {
int f1(String a) {
return 777;
}
}
// this is the test class
#SpringBootTest
#RunWith(SpringRunner.class)
public class ApplicationTests {
#MockBean ExtService extService;
#Test
public void contextLoads() {
Mockito.doThrow(new RuntimeException("unstubbed call")).when(extService).f1(Mockito.any()); // stubbing has to be in this format
Mockito.doReturn(1).when(extService).f1("abc"); // stubbing has to be in this format
System.out.println(extService.f1("abc")); // returns 1
System.out.println(extService.f1("abcd")); // throws exception
}
}
Yet another option is to wait until after the test finishes using the mock, and then use
verifyNoMoreInteractins();

As mentioned in this comment, this GitHub issue in the spring-boot project addresses this same problem and has remained open since 2019, so it's unlikely that an option for "strict stubs" will be available in #SpringBootTest classes anytime soon.
One way that Mockito recommends to enable "strict stubs" is to start a MockitoSession with Strictness.STRICT_STUBS before each test, and close the MockitoSession after each test. Mockito mocks for #MockBean properties in #SpringBootTest classes are generated by Spring Boot's MockitoPostProcessor, so a workaround would need to create the MockitoSession before the MockitoPostProcessor runs. A custom TestExecutionListener can be implemented to handle this, but only its beforeTestClass method would run before the MockitoPostProcessor. The following is such an implementation:
public class MyMockitoTestExecutionListener implements TestExecutionListener, Ordered {
// Only one MockitoSession can be active per thread, so ensure that multiple instances of this listener on the
// same thread use the same instance
private static ThreadLocal<MockitoSession> mockitoSession = ThreadLocal.withInitial { null };
// Count the "depth" of processing test classes. A parent class is not done processing until all #Nested inner
// classes are done processing, so all #Nested inner classes must share the same MockitoSession as the parent class
private static ThreadLocal<Integer> depth = ThreadLocal.withInitial { 0 };
#Override
public void beforeTestClass(TestContext testContext) {
depth.set(depth.get() + 1);
if (depth.get() > 1)
return; // #Nested classes share the MockitoSession of the parent class
mockitoSession.set(
Mockito.mockitoSession()
.strictness(Strictness.STRICT_STUBS)
.startMocking()
);
}
#Override
public void afterTestClass(TestContext testContext) {
depth.set(depth.get() - 1);
if (depth.get() > 0)
return; // #Nested classes should let the parent class end the MockitoSession
MockitoSession session = mockitoSession.get();
if (session != null)
session.finishMocking();
mockitoSession.remove();
}
#Override
public int getOrder() {
return Ordered.LOWEST_PRECEDENCE;
}
}
Then, MyMockitoTestExecutionListener can be added as a listener in test classes:
#SpringBootTest
#TestExecutionListeners(
listeners = {MyMockitoTestExecutionListener.class},
mergeMode = MergeMode.MERGE_WITH_DEFAULTS
)
public class MySpringBootTests {
#MockBean
Foo mockFoo;
// Tests using mockFoo...
}

How to write non-encapsulated unit tests?

I have an autowired variable
#Autowired
private DocumentConfig documentConfig;
I want to make tests for the DocumentService with various states of this configuration object. What are my options? What is the best option?
The first idea is this:
#Test
public void save_failure() {
documentConfig.setNameRequired(true);
/*
testing code goes here
*/
documentConfig.setNameRequired(false);
}
But I want to be somewhat more sure that the variable is reset after the test to not interfere with the other tests, to make sure only this test gets an error if it's the source of a problem.
My new idea was this:
#Before
public void after() { documentConfig.setNameRequired(true); }
#Test
public void save_failure() {
/*
testing code goes here
*/
}
#After
public void after() { documentConfig.setNameRequired(false); }
However, this doesn't work at all because Before and After execute for the whole file and not this single test. I would prefer not to make a new file just for one test.
I've now settled on a compromise:
#Test
public void save_failure() {
documentConfig.setNameRequired(true);
/*
testing code goes here
*/
}
#After
public void after() { documentConfig.setNameRequired(false); }
It seems to do everything I want but I have a few questions.
Assuming nameRequired starts as false, is this guaranteed not to interfere with the other tests?
Is there any way I can make this more clear? Both for my future self and for others.

You can create it before each test. Smth like
private DocumentConfig documentConfig;
#Before
public void createConfig() {
documentConfig = new DocumentConfig(mockedParams);
}

An often used approach is to set up a dummy DocumentConfig and inject it within the setUp() method (annotated with #Before) so that the entire context is reset within each test, for example:
#Before
public void setUp() {
this.documentConfig = new DocumentConfig();
this.documentConfig.setNameRequired(false);
this.service = new DocumentService(this.documentConfig);
}
In this case, I've set up a simple object with nameRequired being false. I could probably delete that statement, because a boolean field defaults to false anyways.
If you don't use constructor injection, and you don't have a setter for documentConfig, you'll have to use reflection to inject the field, for example:
ReflectionTestUtils.setField(this.service, "documentConfig", this.documentConfig);
Within your test you could now write something like this:
#Test
public void save_failure() {
this.documentConfig.setNameRequired(true);
// TODO: Implement test
}
Alternatively, you could mock DocumentConfig, so that you don't rely on its implementation to test DocumentService. I assume that you're calling isNameRequired() somewhere in the code of DocumentService, so you could mock it like this:
#Before
public void setUp() {
// Use a static import for Mockito.mock()
this.documentConfig = mock(DocumentConfig.class);
this.service = new DocumentService(this.documentConfig);
}
#Test
public void save_failure() {
// Use a static import for Mockito.when()
when(this.documentConfig.isNameRequired()).thenReturn(true);
// TODO: Implement test
}
Since this mocking/injection setup happens quite often, Mockito also has its own runner that allows you to get rid of the setUp() method, for example:
#RunWith(MockitoJUnitRunner.class)
public class DocumentServiceTest {
#InjectMocks
private DocumentService documentService;
#Mock
private DocumentConfig documentConfig;
#Test
public void save_failure() {
when(this.documentConfig.isNameRequired()).thenReturn(true);
// TODO: Implement test
}
}

It is not yet clear, which testing framework you use. For plain unit tests, make the value injectable by either a setter or constructor injection. Whatever suits your specific situation best.
If there's a lot (more than three ;-) ) of such values to be injected, you may consider introducing a configuration class to inject all those values as a single parameter.

Injecting mock before Spring's post-construct phase

Basically, the question is in the title.
I faced a problem that in post-construct phase my bean (that is autowired in the bean that is going through post-construct phase right now) is already mocked, but all the behavior described by Mockito.when() doesn't work, all the calls return null.
While searching I found this solution.
But is it possible to make it work without using any 3rd party libraries?
Test class:
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
#ContextConfiguration(classes = TestApplicationConfiguration.class)
public class ServiceTest {
#Autowired
#Qualifier("test")
private PCLPortType pclPortType;
#MockBean
private ClearingHelper сlearingHelper;
#MockBean
private OrganizationCacheRepository organizationCacheRepository;
#Before
public void setup() throws Exception{
OperationResultWithOrganizationSystemIdMappingList res = new OperationResultWithOrganizationSystemIdMappingList();
when(clearingHelper.getOrgIdSystemIdMapping(any(Keycloak.class))).thenReturn(res);
}
#Test
public void test() throws Exception{
pclPortType.call("123");
}
}
Test config:
#TestConfiguration
public class TestApplicationConfiguration {
#Bean(name = "test")
public PCLPortType pclPortTypeForTest() throws JAXBException {
...
}
#Bean
public Keycloak keycloak() {
return Mockito.mock(Keycloak.class);
}
}
Component where I want to get mocked beans:
#Component
public class OrganizationCacheJob {
private static final Logger logger =
LogManager.getLogger(OrganizationCacheJob.class);
private final ObjectFactory<Keycloak> factory;
private final ClearingHelper clearingHelper;
private final OrganizationCacheRepository organizationCacheRepository;
#Autowired
public OrganizationCacheJob(ObjectFactory<Keycloak> factory,
ClearingHelper clearingHelper,
OrganizationCacheRepository organizationCacheRepository) {
this.factory = factory;
this.clearingHelper = ClearingHelper;
this.organizationCacheRepository = organizationCacheRepository;
}
#PostConstruct
public void updateCacheRepository() {
doUpdateCacheRepository();
}
#Scheduled(cron = "${organization.cache.schedule}")
public void start() {
logger.info("Starting update organization cache.");
doUpdateCacheRepository();
logger.info("Job finished.");
}
private void doUpdateCacheRepository() {
try {
Keycloak keycloak = factory.getObject();
OperationResultWithOrganizationSystemIdMappingList orgIdSystemIdMapping = clearingHelper.getOrgIdSystemIdMapping(keycloak);
if (orgIdSystemIdMapping != null) {
orgIdSystemIdMapping.getContent().forEach(o -> organizationCacheRepository.saveOrgIdsSystemsIdsMappings(o.getOrgId(), o.getId()));
logger.debug("Was saved {} orgIds", orgIdSystemIdMapping.getContent().size());
}
} catch (Exception e) {
logger.error("Error fetching whole mapping for org and systems ids. Exception: {}", e);
}
}
}
So, in post-construct phase of OrganizationCacheJob I want to get res when calling clearingHelper, but instead I get null.
ClearingHelper is a regular Spring bean marked as a #Component with public methods.

Ahh ok I just realized - when you start your test case, whole env is up and running first, then you advance to testing phase. So, translating to your case - first you got injection and post-constructs called, then #Before method is done, thus the result.
So as you can see, code says more than all the words you could put in your original post.
If it is possible for you, use spies insteed of mocks. If it is not possible to construct that, you will have to redesign your tests to not rely on post construct.
In this case, since you want the same post-construct behavior for every test case, provide own factory method for given mock (like you did with keycloak) and move when-doReturn there. It will be guaranteed that it will happen before post construct.

Problems using JMockit with log4j 2.9.0

I'm trying to use JMockit to test that a certain logging operation takes place:
public class LogClass1 {
public void doLog() {
Logger logger = LogManager.getLogger(LogClass1.class);
logger.info("This is a log message for {}", "arg1");
}
}
public class LogClass1Test {
#Mocked
private Logger logger;
#Tested
private LogClass1 x;
#Before
public void setup() {
x = new LogClass1();
}
#Test
public void testDoLog() {
new Expectations() {
{
logger.info("This is a log message for {}", "arg1");
}
};
x.doLog();
}
}
But this results in a "missing 1 invocation to org.apache.logging.log4j.Logger#info" error.
I've done similar mocking with log4j 1.x in the past, and I haven't had this problem. I'm wondering if there's some issue because log4j 2.x seems to have many more overloads of its info() methods.
I tried changing "arg1" to (Object)"arg1" in the unit test to see if I could get it to match the signature. This didn't help.
Any thoughts on how I can get this to work?

Note that Logger is an interface, and that LogClass1 obtains an instance of it through the LogManager.getLogger static factory method. So, obviously, it creates an instance of some Logger implementation class. And said class is not being mocked in the test.
What the test needs to do is to mock LogManager, so it returns the #Mocked Logger instance. That is, add a #Mocked LogManager field to the test class.
(Also, no need for that setup method since #Tested creates an instance automatically.)

Junit - run set up method once

I set up a class with a couple of tests and rather than using #Before I would like to have a setup method that executes only once before all tests. Is that possible with Junit 4.8?

Although I agree with #assylias that using #BeforeClass is a classic solution it is not always convenient. The method annotated with #BeforeClass must be static. It is very inconvenient for some tests that need instance of test case. For example Spring based tests that use #Autowired to work with services defined in spring context.
In this case I personally use regular setUp() method annotated with #Before annotation and manage my custom static(!) boolean flag:
private static boolean setUpIsDone = false;
.....
#Before
public void setUp() {
if (setUpIsDone) {
return;
}
// do the setup
setUpIsDone = true;
}

You can use the BeforeClass annotation:
#BeforeClass
public static void setUpClass() {
//executed only once, before the first test
}

JUnit 5 now has a #BeforeAll annotation:
Denotes that the annotated method should be executed before all #Test
methods in the current class or class hierarchy; analogous to JUnit
4’s #BeforeClass. Such methods must be static.
The lifecycle annotations of JUnit 5 seem to have finally gotten it right! You can guess which annotations available without even looking (e.g. #BeforeEach #AfterAll)

When setUp() is in a superclass of the test class (e.g. AbstractTestBase below), the accepted answer can be modified as follows:
public abstract class AbstractTestBase {
private static Class<? extends AbstractTestBase> testClass;
.....
public void setUp() {
if (this.getClass().equals(testClass)) {
return;
}
// do the setup - once per concrete test class
.....
testClass = this.getClass();
}
}
This should work for a single non-static setUp() method but I'm unable to produce an equivalent for tearDown() without straying into a world of complex reflection... Bounty points to anyone who can!

JUnit 5 #BeforeAll can be non static provided the lifecycle of the test class is per class, i.e., annotate the test class with a #TestInstance(Lifecycle.PER_CLASS) and you are good to go

Edit:
I just found out while debugging that the class is instantiated before every test too.
I guess the #BeforeClass annotation is the best here.
You can set up on the constructor too, the test class is a class after all.
I'm not sure if it's a bad practice because almost all other methods are annotated, but it works. You could create a constructor like that:
public UT () {
// initialize once here
}
#Test
// Some test here...
The ctor will be called before the tests because they are not static.

Use Spring's #PostConstruct method to do all initialization work and this method runs before any of the #Test is executed

Try this solution:
https://stackoverflow.com/a/46274919/907576 :
with #BeforeAllMethods/#AfterAllMethods annotation you could execute any method in Test class in an instance context, where all injected values are available.

My dirty solution is:
public class TestCaseExtended extends TestCase {
private boolean isInitialized = false;
private int serId;
#Override
public void setUp() throws Exception {
super.setUp();
if(!isInitialized) {
loadSaveNewSerId();
emptyTestResultsDirectory();
isInitialized = true;
}
}
...
}
I use it as a base base to all my testCases.

If you don't want to force a declaration of a variable that is set and checked on each subtest, then adding this to a SuperTest could do:
public abstract class SuperTest {
private static final ConcurrentHashMap<Class, Boolean> INITIALIZED = new ConcurrentHashMap<>();
protected final boolean initialized() {
final boolean[] absent = {false};
INITIALIZED.computeIfAbsent(this.getClass(), (klass)-> {
return absent[0] = true;
});
return !absent[0];
}
}
public class SubTest extends SuperTest {
#Before
public void before() {
if ( super.initialized() ) return;
... magic ...
}
}

I solved this problem like this:
Add to your Base abstract class (I mean abstract class where you initialize your driver in setUpDriver() method) this part of code:
private static boolean started = false;
static{
if (!started) {
started = true;
try {
setUpDriver(); //method where you initialize your driver
} catch (MalformedURLException e) {
}
}
}
And now, if your test classes will extends from Base abstract class -> setUpDriver() method will be executed before first #Test only ONE time per run.

Here is one alternative suggestion:
What I do to get this working is
Create a method named _warmup or just _
Annotate the test class with #FixMethodOrder(MethodSorters.NAME_ASCENDING)
This is applicable only if you run all tests in the class
It has a downside of having additional test included, which will also run one additional #Before and #After
It is usually advised for your test methods to be order independent, this breaks that rule, but why someone would like tests ordered randomly in the reports I have no clue so NAME_ASCENDING is what I always use
But the upsides to this is simple setup with minimal code and without the need to extend classes/runners etc...
Test run lengths are more accurate since all setup time is reported on method _warmup

After experimenting for some time this is my solution. I needed this for spring boot test. I tried using #PostConstruct, unfortunately it is executed for every test.
public class TestClass {
private static TestClass testClass = null;
#Before
public void setUp() {
if (testClass == null) {
// set up once
...
testClass = this;
}
}
#AfterClass
public static void cleanUpAfterAll() {
testClass.cleanUpAfterAllTests();
}
private void cleanUpAfterAllTests() {
// final cleanup after all tests
...
}
#Test
public void test1() {
// test 1
...
}
#Test
public void test2() {
// test 2
...
}
}

The problem with #BeforeClass is that it fires before the constructor.
So if you rely on an #Autowired constructor to provide data, it simply will not work: wrong execution order.
Similarly #PostConstruct fires after the constructor has been called. And the constructor fires with every #Test, therefore your setup function will fire with every test, if you use this.
This has the exact same effect as calling a function from the constructor.
The only solution, I found that works, is to use a flag to indicate if the setUp() method has already been executed. While its not ideal, it will drastically reduce the amount of processing before each test.
private static boolean initialized = false;
#Autowired
public CacheTest( MyBean myBean ){
this.myBean = myBean;
}
#PostConstruct
public static void setUp(){
if( initialized ) { return };
initialized = true;
//do suff with myBean
}