We Keep Coding

How to test main class of Spring-boot application

I have a spring-boot application where my #SpringBootApplication starter class looks like a standard one. So I created many tests for all my functionalities and send the summary to sonarqube to see my coverage.
For my starter class Sonarqube tells me that I just have 60% coverage. So the average coverage is not good as expected.
My Test class is just the default one.
#RunWith(SpringRunner.class)
#SpringBootTest(classes = ElectronicGiftcardServiceApplication.class)
public class ElectronicGiftcardServiceApplicationTests {
#Test
public void contextLoads() {
}
}
So how can I test my main class in the starter class of my application?

All these answers seem overkill.
You don't add tests to make a metric tool happy.
Loading a Spring context of the application takes time. Don't add it in each developer build just to win about 0.1% of coverage in your application.
Here you don't cover only 1 statement from 1 public method. It represents nothing in terms of coverage in an application where thousands of statements are generally written.
First workaround : make your Spring Boot application class with no bean declared inside. If you have them, move them in a configuration class (for make them still cover by unit test). And then ignore your Spring Boot application class in the test coverage configuration.
Second workaround : if you really need to to cover the main() invocation (for organizational reasons for example), create a test for it but an integration test (executed by an continuous integration tool and not in each developer build) and document clearly the test class purpose :
import org.junit.Test;
// Test class added ONLY to cover main() invocation not covered by application tests.
public class MyApplicationIT {
#Test
public void main() {
MyApplication.main(new String[] {});
}
}

You can do something like this
#Test
public void applicationContextLoaded() {
}
#Test
public void applicationContextTest() {
mainApp.main(new String[] {});
}

I solved in a different way here. Since this method is there only as a bridge to Spring's run, I annotated the method with #lombok.Generated and now sonar ignores it when calculating the test coverage.
Other #Generated annotations, like javax.annotation.processing.Generated or javax.annotation.Generated might also work but I can't test now because my issue ticket was closed.
package com.stackoverflow;
import lombok.Generated;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
#SpringBootApplication
public class Application {
#Generated
public static void main(String... args) {
SpringApplication.run(Application.class, args);
}
}

I had the same goal (having a test that runs the main() method) and I noticed that simply adding a test method like #fg78nc said will in fact "start" the application twice : once by spring boot test framework, once via the explicit invocation of mainApp.main(new String[] {}), which I don't find elegant.
I ended up writing two test classes : one with #SpringBootTest annotation and the empty test method applicationContextLoaded(), another one without #SpringBootTest (only RunWith(SpringRunner.class)) that calls the main method.
SpringBootApplicationTest
package example;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.boot.test.context.SpringBootTest;
#RunWith(SpringRunner.class)
#SpringBootTest
public class SpringBootApplicationTest {
#Test
public void contextLoads() {
}
}
ApplicationStartTest
package example;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.test.context.junit4.SpringRunner;
#RunWith(SpringRunner.class)
public class ApplicationStartTest {
#Test
public void applicationStarts() {
ExampleApplication.main(new String[] {});
}
}
Overall, the application is still started two times, but because there is now two test classes. Of course, with only these two tests methods, it seems overkill, but usually more tests will be added to the class SpringBootApplicationTest taking advantage of #SpringBootTest setup.

In addition to the answers above, here is a unit test of a SpringBoot application's main method for if you are using JUnit 5 and Mockito 3.4+:
try (MockedStatic<SpringApplication> mocked = mockStatic(SpringApplication.class)) {
mocked.when(() -> { SpringApplication.run(ElectronicGiftCardServiceApplication.class,
new String[] { "foo", "bar" }); })
.thenReturn(Mockito.mock(ConfigurableApplicationContext.class));
ElectronicGiftCardServiceApplication.main(new String[] { "foo", "bar" });
mocked.verify(() -> { SpringApplication.run(ElectronicGiftCardServiceApplication.class,
new String[] { "foo", "bar" }); });
}
It verifies that the static method run() on the SpringApplication class is called with the expected String array when we call ElectronicGiftCardServiceApplication.main().
Same idea as awgtek and Ramji Sridaran, but their solutions are for JUnit 4.

You can Mock SpringApplication since that is a dependency of the method under test. See how here.
I.e.
import org.junit.Test;
import org.junit.runner.RunWith;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;
import org.springframework.boot.SpringApplication;
import static org.powermock.api.mockito.PowerMockito.mockStatic;
import static org.powermock.api.mockito.PowerMockito.verifyStatic;
#RunWith(PowerMockRunner.class)
public class ElectronicGiftcardServiceApplicationTest {
#Test
#PrepareForTest(SpringApplication.class)
public void main() {
mockStatic(SpringApplication.class);
ElectronicGiftcardServiceApplication.main(new String[]{"Hello", "World"});
verifyStatic(SpringApplication.class);
SpringApplication.run(ElectronicGiftcardServiceApplication.class, new String[]{"Hello", "World"});
}
}

Using junit
import org.junit.jupiter.api.Test;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import org.springframework.boot.SpringApplication;
import static org.assertj.core.api.Assertions.*;
class WebsiteApplicationTests {
#Test
void testApplication() {
MockedStatic<SpringApplication> utilities = Mockito.mockStatic(SpringApplication.class);
utilities.when((MockedStatic.Verification) SpringApplication.run(WebsiteApplication.class, new String[]{})).thenReturn(null);
WebsiteApplication.main(new String[]{});
assertThat(SpringApplication.run(WebsiteApplication.class, new String[]{})).isEqualTo(null);
}
}
Add these dependencies in pom.xml
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
<version>${mockito.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>3.8.0</version>
<scope>test</scope>
</dependency>

<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<mainClass>your.awesome.package.Application</mainClass>
</configuration>
</plugin>
If you aim for 100% coverage, one thing you can do is simply not having a main method at all. You still require a class annotated with #SpringBootApplication but it can be empty.
Be warned though as it has its drawbacks and other tools that rely on main can break.

This simple mock test for SpringApplication does not invoke any methods but just tests the starter app. [uses PowerMockRunner.class]
import org.junit.Test;
import org.junit.runner.RunWith;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PowerMockIgnore;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;
import org.springframework.boot.SpringApplication;
#RunWith(PowerMockRunner.class)
#PowerMockIgnore({"com.sun.org.apache.xerces.*", "javax.xml.*", "org.xml.*", "javax.management.*"})
public class JobsAppStarterTest {
#Test
#PrepareForTest(SpringApplication.class)
public void testSpringStartUp() {
PowerMockito.mockStatic(SpringApplication.class);
SpringApplication.run(JobsAppStarter.class, new String[] {"args"});
JobsAppStarter.main(new String[] {"args"});
}
}

If the idea is to exclude the SpringApplication class from sonar scan (which is the recommended way of doing it), you can exclude it with the following configuration in the build.gradle
plugins {
id 'org.sonarqube' version '3.4.0.2513'
}
sonarqube {
properties {
property "sonar.exclusions", "**/*Application.java"
}
}

Even though this question has been answered extensively I had a use case that is not covered here that is perhaps interesting to share. I am validating some properties at startup and I wanted to assert that the application would fail to start if these properties were configured wrong. In JUnit4 I could have done something like this:
#ActiveProfiles("incorrect")
#SpringBoot
public class NetworkProbeApplicationTest {
#Test(expected=ConfigurationPropertiesBindException.class)
public void contextShouldNotLoadWhenPropertiesIncorrect() {
}
}
But in JUnit5 you can no longer add the "expected" value to your #Test annotation and you have to do it differently. And since I wanted to start the application with an incorrect set of properties I needed to pass in which profile to use as a main() argument. I could not really find this documented anywhere, but passing in arguments through the main() method requires you to prefix your arguments with a double hyphen and separate the key and value with an equals sign. A complete test would look like this:
import org.junit.jupiter.api.Test;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.context.properties.ConfigurationPropertiesBindException;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
public class NetworkProbeApplicationTest {
#Test
public void contextShouldNotLoadWhenPropertiesIncorrect() {
Exception exception = assertThrows(ConfigurationPropertiesBindException.class, () -> {
SpringApplication.run(NetworkProbeApplication.class, "--spring.profiles.active=incorrect");
});
String expectedMessage = "Error creating bean with name 'dnsConfiguration': Could not bind properties to 'DnsConfiguration' : prefix=dns";
assertTrue(exception.getMessage().contains(expectedMessage));
}
}

How to run PowerMock on dynamically created TestCase

I was trying to mock my test suites. My test framework creates test cases by scanning test files on disk. So each time the test cases are dynamically created.
I was trying to use PowerMock. Below is the thing I tried first.
public class GroupTestcase_T extends TestSuite {
static void run() {
scan();
junit.textui.TestRunner.run(g);
}
static void scan() {
// scan disk
for (MyTestCase t : tests) { addTest(t); }
}
}
#RunWith(PowerMockRunner.class)
#PrepareForTest(ClassToStub.class)
public class MyTestCase extends TestCase {
public MyTestCase(TestInfo info) {...}
#Override
protected void setUp() throws Exception {
PowerMockito.mockStatic(ClassToStub.class);
when(ClassToStub.methodToStub())
.thenReturn(new FakeProxy());
}
#Test
public void test() throws Exception {
// Test!
}
}
Above code seems not working:
Also, this error might show up because:
1. you stub either of: final/private/equals()/hashCode() methods.
Those methods cannot be stubbed/verified.
2. inside when() you don't call method on mock but on some other object.
3. the parent of the mocked class is not public.
It is a limitation of the mock engine.
I traced the code and found that PowerMockRunner are not called at all.
Also I tried manually force Junit to run it with PowerMockRunner:
Result result = junit.run(new PowerMockRunner(MyTestCase.class));
PowerMockRunner has only one constructor that takes the test class as parameter. My test cases are different each time but all share the same class.
Any idea how to use PowerMock if TestCase are dynamically created?
I was using Junit 4 / PowerMock 1.5

You can generate your tests with the parameterized tests feature and apply the #PowerMockRule.
import static org.junit.Assert.assertTrue;
import java.util.Arrays;
import java.util.Collection;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.rule.PowerMockRule;
#RunWith(Parameterized.class)
#PrepareForTest(ClassToStub.class)
public class MyTestCase{
#Parameters
public static Collection<Object[]> scan() {
return Arrays.asList(new Object[][] {
{ new TestInfo() }, { new TestInfo() } });
}
#Rule
public PowerMockRule rule = new PowerMockRule();
public MyTestCase(TestInfo info) {
// ...
}
#Test
public void test() throws Exception {
PowerMockito.mockStatic(ClassToStub.class);
PowerMockito.when(ClassToStub.methodToStub()).thenReturn(new FakeProxy());
assertTrue(ClassToStub.methodToStub() instanceof FakeProxy);
}
}
Beware, in your example, you are mixing junit 3 (extends TestSuite, protected setUp) and junit 4 (#Test) test definitions.

How to parameterize junit Test Suite

Is it possible to parameterize a TestSuite in junit 4 ?
For declaring a class as a test suite I need the annotation #RunWith(Suite.class), but the same annotation is also needed to declare the test as parameterized: #RunWith(Parameterized.class) so I cannot add both to the same class.
I found a similar question in this site that did not help much. So far, all the examples I have found explain how to parameterize simple unit tests, not a complete test tuite.

I believe the basic answer is No, because as you said, the #RunsWith only take one parameter. I found a blog posting that got a bit creative in how to handle this situation.
We don't use the parameterized tests, but may you could create a separate suite like we do that only lists the test classes and the parameterized test could be part of that. I modified our test suite to include a parameterized test class to part of the suite and it ran fine. We create our suite like below where PrimeNumberCheckerTest was a simple I pulled from the web.
package com.jda.portfolio.api.rest.server;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
#RunWith(Suite.class)
#SuiteClasses({ com.mycompany.api.rest.server.resource.TestCartResourceJava.class,
com.mycompany.api.rest.server.resource.TestCustomerResource.class,
com.mycompany.api.rest.server.resource.TestWizardProfileResource.class,
com.mycompany.api.rest.server.interceptor.TestBaseSearchInterceptor.class,
com.mycompany.api.rest.server.resource.TestQueryParameters.class,
com.mycompany.api.rest.server.expression.TestCartExpressionGenerator.class,
com.mycompany.api.rest.server.expression.TestPreferenceExpressionGenerator.class,
com.mycompany.api.rest.server.PrimeNumberCheckerTest.class,
})
public class AllTests {}
Here's the source for the parameterized test case;
package com.jda.portfolio.api.rest.server:
import static org.junit.Assert.*;
import java.util.Arrays;
import java.util.Collection;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Suite.SuiteClasses;
#RunWith(Parameterized.class)
#SuiteClasses({PrimeNumberCheckerTest.class})
public class PrimeNumberCheckerTest {
private Integer inputNumber;
private Boolean expectedResult;
private PrimeNumberChecker primeNumberChecker;
#Before
public void initialize() {
primeNumberChecker = new PrimeNumberChecker();
}
// Each parameter should be placed as an argument here
// Every time runner triggers, it will pass the arguments
// from parameters we defined in primeNumbers() method
public PrimeNumberCheckerTest(Integer inputNumber,
Boolean expectedResult) {
this.inputNumber = inputNumber;
this.expectedResult = expectedResult;
}
#Parameterized.Parameters
public static Collection primeNumbers() {
return Arrays.asList(new Object[][] {
{ 2, true },
{ 6, false },
{ 19, true },
{ 22, false },
{ 23, true }
});
}
// This test will run five times since we have as many parameters defined
#Test
public void testPrimeNumberChecker() {
System.out.println("Parameterized Number is : " + inputNumber);
assertEquals(expectedResult,
primeNumberChecker.validate(inputNumber));
}

I was able to parameterize a test suite and use its data in a test class member of the suite as follows:
In JUTsuite:
#RunWith(Suite.class)
#Suite.SuiteClasses({
JUT_test1.class,
})
public class JUTSuite{
// Declare all variables/objects you want to share with the test classes, e.g.
protected static List<Fx> globalFxs;
// This is the data list we'll use as parameters
protected static List<Dx> globalDxs;
#Parameters
public static Collection<Object[]> data(){
// Instantiate object list for parameters.
// Note: you must do it here and not in, say, #BeforeClass setup()
// e.g.
globalDxs=new ArrayList<Dx>(serverObj.values());
Collection<Object[]> rows=new ArrayList<Object[]>();
for(Dx d:globalDxs) {
rows.add(new Object[]{d});
}
return rows;
}
#BeforeClass
public static void setUp() throws Exception {
// Instantiate/initialize all suite variables/objects to be shares with test classes
// e.g. globalFxs=new ArrayList<Fx>();
}
#AfterClass
public static void tearDown() throws Exception {
// Clean up....
}
}
Next, in test class:
#RunWith(Parameterized.class)
public class JUT_test1 {
// declare local names (if desired) for suite-wide variable/objects
// e.g.
private static List<Fx> globalFxs;
// This is the test parameter:
private Dx d;
public JUT_test1(Dx d){
this.d=d;
}
#Parameters
public static Collection<Object[]> data(){
// Note: we're calling the suite's data() method which has already executed.
return JUTSuite.data();
}
#BeforeClass
public static void setUpBeforeClass() throws Exception {
// (If desired)initialize local variables by referencing suite variables.
// e.g.globalFxs=JUTSuite.globalFxs;
}
}

I agree, it's not possible with the provided classes, but there are workarounds that will get you most of the way there, like #mikemil's.
I've spent some time extending Suite and delegating to Parameterized, with partial success; it is possible to build runner that does what you want, and the code is more-or-less written for you in those two classes. The way those classes interact (in particular, the definition of Parameterized#getChildren()) makes it difficult to extend or delegate to those classes to accomplish what you need, but creating a whole new class than extends ParentRunner and lifts code from the other two would be fairly easy.
I'll try to get more time to come back to this later. If you do build a new runner before I get around to it, please post it as an answer, I'd love to use it myself.

the best solution will be, keep suit classes separately in a blank class.
For example, I am testing logins as Parameterized tests and putting in a suit (for navigation performance measurement)
#RunWith(Suite.class)
#Suite.SuiteClasses({
LoginPageTest.class,
HomePageTests.class})
public class PerformanceTests {
}
and LoginPageTest is actually Parameterizedtests
#RunWith(Parameterized.class)
public class LoginPageTest
{...}

As already stated multiple times, it's not possible to parameterize a test suite with the runners provided by JUnit 4.
Anyway, I wouldn't recommend to make your testclasses dependent from some externally provided state. What if you want to run a single testclass?
I would recommend to make your separate test classes #Parameterized and use a utility class to provide the parameters:
#RunWith(Suite.class)
#SuiteClasses({ Test1.class, Test2.class })
public class TestSuite {
// suite
}
#RunWith(Parameterized.class}
public class Test1 {
public Test1(Object param1) { /* ... */ }
#Parameters
public static Collection<Object[]> data() {
return TestParameters.provideTestData()
}
#Test
public void someTest() { /* ... */ }
}
#RunWith(Parameterized.class}
public class Test2 {
public Test2(Object param1) { /* ... */ }
#Parameters
public static Collection<Object[]> data() {
return TestParameters.provideTestData()
}
#Test
public void someOtherTest() { /* ... */ }
}
class TestParameters {
public static Collection<Object[]> provideTestData() {
Collection<Object[]> data = new ...;
// build testdata
return data;
}

You're right: Both Suite and Parameterized are Runners and only one Runner may be used to run a test at a time. Standard JUnit 4 doesn't provide a combined Runner.
You can either implement your own Runner or have a look at this ready-to-use library which provides a ParameterizedSuite Runner: https://github.com/PeterWippermann/parameterized-suite
A parameterized test suite looks like this:
#RunWith(ParameterizedSuite.class)
#SuiteClasses({OneTest.class, TwoTest.class})
public class MyParameterizedTestSuite {
#Parameters(name = "Parameters are {0} and {1}")
public static Object[] params() {
return new Object[][] {{'A',1}, {'B',2}, {'C',3}};
}

Maybe this answer helps: Parameterized unit test suites
It uses #RunWith(Enclosed.class) and seems to solve the problem.

Do Mock objects get reset for each test?

I'm using the Mockito framework to create Mock objects in my JUnit tests. Each mock knows what methods have been called on it, so during my tests I can write
verify(myMock, atLeastOnce()).myMethod();
I am wondering if this internal mock knowledge of what it has called will persist across my tests? If it does persist, then I could be getting false positives when using the same verify method in two tests.
A code example
#RunWith(MockitoJUnitRunner.class)
public class EmrActivitiesImplTest {
#Mock private MyClass myMock;
#Before
public void setup() {
when(myMock.myMethod()).thenReturn("hello");
}
#Test
public void test1() {
// ..some logic
verify(myMock, atLeastOnce()).myMethod();
}
#Test
public void test2() {
// ..some other logic
verify(myMock, atLeastOnce()).myMethod();
}
}
Mock state is persisted - test2 will pass regardless, since test1's verify method passed
Mock state is reset - test2 will fail if myMock.myMethod() isn't called

JUnit creates a new instance of test class each time it runs a new test method and runs #Before method each time it creates a new test class. You can easily test it:
#Before
public void setup() {
System.out.println("setup");
when(myMock.myMethod()).thenReturn("hello");
}
And MockitoJUnitRunner will create a new MyMock mock instance for every test method.

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.runners.MockitoJUnitRunner;
import static org.mockito.Mockito.*;
#RunWith(MockitoJUnitRunner.class)
public class sandbox {
#Mock
private MyClass myMock;
#Before
public void setup() {
when(myMock.myMethod()).thenReturn("hello");
}
#Test
public void test1() {
myMock.myMethod();
verify(myMock, times(1)).myMethod();
}
#Test
public void test2() {
myMock.myMethod();
verify(myMock, times(1)).myMethod();
}
}
This passes. If the state persisted then the second test would fail. If you debug it you would see that you get a new instance of the mocked object for each test.

If you just initialize the Mock objects in your setup, then inside each test you can provide different functionality. You can initialize it once and change how they act/expect per test after that.

JUnit Application Suite Testing

I am trying to create a junit test suite that will run all of the test suites within the application... - this is what I have and as far as I can find it should work but it keeps telling me that no tests are found.
import static org.junit.Assert.*;
import org.junit.Test;
/**
* #author Jason
*
*/
#Test
public class applicationTest extends TestCase {
public applicationTestSuite(String name) {
super(name);
}
public static Test suite() {
TestSuite suite = new TestSuite("ApplicationTestSuite");
suite.addTest(domain.AllDomainTests.suite());
suite.addTest(services.AllServicesTests.suite());
suite.addTest(business.AllBusinessTests.suite());
return suite;
}
}
An example of one of the test suites it should be running -
package business;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
#RunWith(Suite.class)
#SuiteClasses({ ItemMgrTest.class })
public class AllBusinessTests {
}

You need to mark #Test annotation on test method
API Document
The Test annotation tells JUnit that the public void method to which it is attached can be run as a test case. To run the method, JUnit first constructs a fresh instance of the class then invokes the annotated method. Any exceptions thrown by the test will be reported by JUnit as a failure. If no exceptions are thrown, the test is assumed to have succeeded [...]

You can specify Suite classes in your #SuiteClasses, for instance:
#RunWith(Suite.class)
#SuiteClasses({ AllBusinessTests.class })
public class AllTests {
}
The suite() method is a JUnit 3 thing, and won't find any methods or tests in your suite, because you're using JUnit 4.