We Keep Coding

Parameterized testcontainer image tag with Junit5

In the Testcontainers documentation, there is an example for having the docker image to be parameterized with #ParameterizedTest.
This was a junit4 example.
https://github.com/testcontainers/testcontainers-java/blob/main/core/src/test/java/org/testcontainers/junit/ParameterizedDockerfileContainerTest.java
#RunWith(Parameterized.class)
public class ParameterizedDockerfileContainerTest {
private final String expectedVersion;
#Rule
public GenericContainer container;
public ParameterizedDockerfileContainerTest(String baseImage, String expectedVersion) {
container =
new GenericContainer(
new ImageFromDockerfile()
.withDockerfileFromBuilder(builder -> {
builder
.from(baseImage)
// Could potentially customise the image here, e.g. adding files, running
// commands, etc.
.build();
})
)
.withCommand("top");
this.expectedVersion = expectedVersion;
}
#Parameterized.Parameters(name = "{0}")
public static Object[][] data() {
return new Object[][] { //
{ "alpine:3.12", "3.12" },
{ "alpine:3.13", "3.13" },
{ "alpine:3.14", "3.14" },
{ "alpine:3.15", "3.15" },
{ "alpine:3.16", "3.16" },
};
}
#Test
public void simpleTest() throws Exception {
final String release = container.execInContainer("cat", "/etc/alpine-release").getStdout();
assertThat(release).as("/etc/alpine-release starts with " + expectedVersion).startsWith(expectedVersion);
}
}
I couldn't find a way to do something similar with junit5, basically :
having the container only started once for all the #ParameterizedTest methods in the class
Ofc, with a lot of if/else, playing with beforeEach, TestInfo, ... is possible but I feel like something is wrong and I'm sure the following question has probably should be answered with junit5
How to use parameterized tests for testing with multiple database versions

So, it seems the equivalent to junit5 is not possible, related to this opened issue https://github.com/junit-team/junit5/issues/878#issuecomment-546459081 as mentioned by Eddù above
With the great help of Michael Simons, I could manage to do something that works
Base class :
#Testcontainers(disabledWithoutDocker = true)
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
public abstract class MultipleNeo4jVersionsTest {
protected static String HEAP_SIZE = "256M";
public static Stream<String> neo4jVersions() {
return Stream.of("4.4.14", "5.2.0");
}
protected static String heapSizeSetting(Neo4jVersion version) {
return version.equals(Neo4jVersion.V4_4)
? "NEO4J_dbms_memory_heap_max__size"
: "NEO4J_server_memory_heap_max__size"
;
}
protected Neo4jContainer<?> getNeo4j(String version) {
var imageName = String.format("neo4j:%s-enterprise", version);
Neo4jVersion neo4jVersion = Neo4jVersion.of(version);
Neo4jContainer<?> container = new Neo4jContainer<>(imageName)
.withoutAuthentication()
.withEnv("NEO4J_ACCEPT_LICENSE_AGREEMENT", "yes")
.withEnv(heapSizeSetting(neo4jVersion), HEAP_SIZE)
.withReuse(true);
container.start();
return container;
}
}
And the actual test class
#ParameterizedTest
#MethodSource("neo4jVersions")
void loading_config(String version) throws Exception {
Neo4jContainer<?> neo4j = getNeo4j(version);
// do something here
}
Couple of useful links
https://foojay.io/today/faster-integration-tests-with-reusable-testcontainers/
https://github.com/michael-simons/neo4j-migrations/blob/main/neo4j-migrations-core/src/test/java/ac/simons/neo4j/migrations/core/CatalogBasedMigrationIT.java
https://github.com/michael-simons/neo4j-migrations/blob/main/neo4j-migrations-core/src/test/java/ac/simons/neo4j/migrations/core/TestBase.java#L49

Devopology Test Engine (Junit5 based) supports parameterized class testing. (I'm the author)
https://github.com/devopology/test-engine
Your example...
package test.neo4j;
import org.devopology.test.engine.api.AfterAll;
import org.devopology.test.engine.api.BeforeAll;
import org.devopology.test.engine.api.Parameter;
import org.devopology.test.engine.api.Test;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import java.util.ArrayList;
import java.util.Collection;
public class Neo4jTest {
#Parameter
public String dockerImageName;
#Parameter.Supplier
public static Collection<String> dockerImageNames() {
Collection<String> dockerImageNames = new ArrayList<>();
dockerImageNames.add("neo4j:4.4.14-enterprise");
dockerImageNames.add("neo4j:5.2.0-enterprise");
return dockerImageNames;
}
private Neo4jContainer<?> neo4jContainer;
#BeforeAll
public void beforeAll() {
neo4jContainer = new Neo4jContainer<>(dockerImageName)
.waitingFor(Wait.forLogMessage(".*Started..*", 1))
.withEnv("NEO4J_ACCEPT_LICENSE_AGREEMENT", "yes")
.withEnv(heapSizeSetting(dockerImageName), "256M")
.withLogConsumer(outputFrame -> System.out.print(outputFrame.getUtf8String()))
.withoutAuthentication();
neo4jContainer.start();
}
#Test
public void test1() {
System.out.println("test1 : dockerImageName = [" + dockerImageName + "]");
// do something here
}
#Test
public void test2() {
System.out.println("test2 : dockerImageName = [" + dockerImageName + "]");
// do something here
}
#AfterAll
public void afterAll() {
if (neo4jContainer != null) {
try {
neo4jContainer.close();
} catch (Throwable t) {
// DO NOTHING
}
neo4jContainer = null;
}
}
protected static String heapSizeSetting(String dockerImageName) {
return dockerImageName.contains("4.4")
? "NEO4J_dbms_memory_heap_max__size" : "NEO4J_server_memory_heap_max__size";
}
}

How to run a block of code after each #test, no matter if the test is #Disabled or not?

Basically I would like to add an entry to my Extent report for "Skipped Tests". I understand I can use #AfterEach, however I see that the #AfterEach code block does not get executed for #Disabled Test in Junit5.
I tried using TestWatcher interface and overrided the method testDisabled like below:
package utils;
import java.util.Optional;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.TestWatcher;
public class MyTestWatcher implements TestWatcher {
public static String testStatus;
public static String testCaseName;
#Override
public void testAborted(ExtensionContext Context, Throwable throwable) {
testStatus = "Aborted";
testCaseName = Context.getDisplayName();
System.out.println("Test Aborted: " + Context.getDisplayName());
}
#Override
public void testDisabled(ExtensionContext Context, Optional<String> optional) {
testStatus = "Skipped";
testCaseName = Context.getDisplayName();
System.out.println("Test Skipped: " + Context.getDisplayName());
}
#Override
public void testFailed(ExtensionContext Context, Throwable throwable) {
testStatus = "Failed";
testCaseName = Context.getDisplayName();
System.out.println("Test Failed: " + Context.getDisplayName());
}
#Override
public void testSuccessful(ExtensionContext Context) {
testStatus = "Passed";
testCaseName = Context.getDisplayName();
System.out.println("Test Passed: " + Context.getDisplayName());
}
}
Based on the 'testStatus' and 'testCaseName' variables, which are set in the testDisabled() method above, I would like to run a block of code from my test class, to add an entry to my Extent report saying which test case was skipped.

If I understood your question correctly you just need to apply TestWatcher to your test to solve your problem. With JUnit 5 you can use #ExtendsWith annotation. Complete example will be:
#ExtendWith(MyTestWatcher.class)
class MyTest {
#Test
void test() {
System.out.println("inside active test");
}
#Test
#Disabled
void testDisabled() {
System.out.println("inside disabled test");
}
}
With output:
inside active test
Test Passed: test()
Test Skipped: testDisabled()
Tested with JUnit 5.4.2

I was able to add reporting for Skipped Tests by initializing the Extent Report from the MyTestWatcher class. Even though the ExtentReport object was already initialized in my test class, when I again try to initialize (which would end up creating another HTML file report), I added the below code to my ExtentManager class:
public static ExtentReports GetExtent(String env){
**if (extent != null)
return extent; //avoid creating new instance of html file**
extent = new ExtentReports();
extent.attachReporter(getHtmlReporter());
extent.setSystemInfo("Environment",env);
extent.setAnalysisStrategy(AnalysisStrategy.SUITE);
return extent;
And my updated code in MyTestWatcher class looks like below:
package utils;
import java.util.Optional;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.TestWatcher;
import com.aventstack.extentreports.ExtentReports;
import com.aventstack.extentreports.ExtentTest;
import com.aventstack.extentreports.Status;
public class MyTestWatcher implements TestWatcher {
public static String testCaseName;
public static ExtentReports extent;
public static ExtentTest report;
static
{
extent = utils.ExtentReportManager.GetExtent(System.getProperty("env"));
report = extent.createTest("Skipped Tests", "Tests which are skipped are displayed here.");
}
#Override
public void testAborted(ExtensionContext Context, Throwable throwable) {
testCaseName = Context.getDisplayName();
System.out.println("Test Aborted: " + Context.getDisplayName());
}
#Override
public void testDisabled(ExtensionContext Context, Optional<String> optional) {
testCaseName = Context.getDisplayName();
System.out.println("Test Skipped: " + Context.getDisplayName());
report.createNode(testCaseName).log(Status.SKIP, "This test is marked as #DISABLED");
}
#Override
public void testFailed(ExtensionContext Context, Throwable throwable) {
testCaseName = Context.getDisplayName();
System.out.println("Test Failed: " + Context.getDisplayName());
}
#Override
public void testSuccessful(ExtensionContext Context) {
testCaseName = Context.getDisplayName();
System.out.println("Test Passed: " + Context.getDisplayName());
}
}

junit4 creating test suite with specific test methods

In junit4 I want to execute specific test methods from different classes i.e want create a test suite with specific test methods from different classes.
Lets say I have 2 classes:
public class Test_Login {
#Test
public void test_Login_001(){
System.out.println("test_Login_001");
}
#Test
public void test_Login_002(){
System.out.println("test_Login_002");
}
#Test
public void test_Login_003(){
System.out.println("test_Login_003");
}
}
public class Logout {
#Test
public void test_Logout_001(){
System.out.println("test_Logout_001");
}
#Test
public void test_Logout_002(){
System.out.println("test_Logout_002");
}
#Test
public void test_Logout_003(){
System.out.println("test_Logout_003");
}
}
From the above classes I want to execute test methods test_Login_001 , test_Login_003 , test_Logout_002 only.
How this can be achieved in junit4 ?

Since JUnit 4.8 introduced Categories there exists a clean solution, create a TestSuite:
#RunWith(Categories.class)
#IncludeCategory(MustHaveTests.class)
#SuiteClasses( { Test_Login.class, Test_Logout.class })
public class MustHaveTestsTestSuite {
public interface MustHaveTests { /* category marker */ }
}
And add the #Category(MustHaveTests.class) above every test you would like to run with the TestSuite, e.g.:
#Category(MustHaveTests.class)
#Test
public void test_Login_001(){
System.out.println("test_Login_001");
}
When running the TestSuite only the MustHaveTests-"tagged" tests will be executed. More Details on #Category: https://github.com/junit-team/junit4/wiki/categories

You need to create an org.junit.runner.Request and pass it to the JunitCore runner, or actually to any Runner.
JUnitCore junitRunner = new JUnitCore();
Request request = Request.method(Logout.class, "test_Logout_002");
Result result = junitRunner.run(request);
I actually created an Annotation and can search for methods with those annotations and dynamically create Request and run them
public class TestsSuite {
public static void main(String[] args) throws Exception {
Class annotation = MyTestAnnotation.class;
JUnitCore junitRunner = new JUnitCore();
Class testClass = Test_Login.class;
Method[] methods = testClass.getMethods();
for (Method method : methods) {
if (method.isAnnotationPresent(annotation)) {
if (method.isAnnotationPresent(org.junit.Test.class)) {
Request request = Request.method(testClass, method.getName());
Result result = junitRunner.run(request);
System.out.println(result.wasSuccessful());
}
}
}
}
}

This might not be the slickest implementation, but I solved a similar problem by created a new #SuiteMethods annotation as follows:
SuiteMethods.java
#Retention(RUNTIME)
#Target(TYPE)
public #interface SuiteMethods {
String[] value() default {""};
}
FilteredSuite.java
public class FilteredSuite extends Categories {
private static String[] TEST_METHODS_TO_RUN = {""}; // default behavior is to run all methods
private static Class<?> extractMethodNamesFromAnnotation(Class<?> clazz) {
SuiteMethods methodsAnnotation = clazz.getAnnotation(SuiteMethods.class);
if (methodsAnnotation != null) {
// if our MethodsAnnotation was specified, use it's value as our methods filter
TEST_METHODS_TO_RUN = methodsAnnotation.value();
}
return clazz;
}
public static Filter getCustomFilter() {
Filter f = new Filter() {
#Override
public boolean shouldRun(Description desc) {
String methodName = desc.getMethodName();
for (String subString : TEST_METHODS_TO_RUN) {
if (methodName == null || methodName.contains(subString)) {
return true;
}
}
return false;
}
#Override
public String describe() {
return null;
}
};
return f;
}
public FilteredSuite(Class<?> arg0, RunnerBuilder arg1) throws InitializationError {
super(extractMethodNamesFromAnnotation(arg0), arg1);
}
#Override
public void filter(Filter arg0) throws NoTestsRemainException {
// At test suite startup, JUnit framework calls this method to install CategoryFilter.
// Throw away the given filter and install our own method name filter
super.filter(getCustomFilter());
}
}
A Usage Example
#RunWith(FilteredSuite.class)
#SuiteClasses({
GroupRestTest.class,
ScenarioRestTest.class
})
#SuiteMethods({
"testReadOnlyFlag",
"testSheetWriteData",
"testAddScenarioMeta"
})
public class SubsetTestSuite {
}

How to obtain test case name in JUnit 4 at runtime? [duplicate]

This question already has answers here:
Get name of currently executing test in JUnit 4
(17 answers)
Closed 9 years ago.
I want to do some logging while executing my JUnit test. In JUnit 3.x it was always easy to obtain the name of the currently running test case, no matter how the test case was instantiated:
public void testFoo() throws Exception() {
String testName = this.getName();
// [...] do some stuff
}
In JUnit 4 things seem to be not so easy. Does anyone know a solution to this? Is there any option to reflect into the current Runner instance?

In JUnit 4.7, you can also get the name of the currently executed test method. May be nice when logging.
Taken from JUnit 4.7 Release Notes (read them here at github) :
public class NameRuleTest {
#Rule public TestName name = new TestName();
#Test public void testA() {
assertEquals("testA", name.getMethodName());
}
#Test public void testB() {
assertEquals("testB", name.getMethodName());
}
}

OK. I've found another approach [somewhere on the Internet](http://www.nabble.com/What-happened-to-getName()--td23456371.html):
#RunWith(Interceptors.class)
public class NameTest {
#Interceptor public TestName name = new TestName();
#Test public void funnyName() {
assertEquals("funnyName", name.getMethodName());
}
}

public class FooTest {
#Rule
final public TestRule traceTestWatcher = new TestWatcher() {
#Override
protected void starting(Description d) {
System.out.println(d);
}
};
#Test
public void testBar() {
...
}
#Test
public void testBaz() {
...
}
}

What's wrong with:
#Test
public void foo() throws Exception() {
String testName = this.getName();
// [...] do some stuff
}
?

I know this is old, but here is a useful (non-junit) method that I put at the top of all my tests.
public static void printTestName(){
final StackTraceElement[] ste = new Throwable().getStackTrace();
int buffer = 35 - ste[1].getMethodName().length();
System.out.println("*******************************************************");
System.out.println("* TEST: " + ste[1].getMethodName() + getBuffer(buffer) + "*");
System.out.println("*******************************************************");
}
private static String getBuffer(int offset){
StringBuilder buffer = new StringBuilder("");
for(int i = 1; i < offset; i++){
buffer.append(" ");
}
return buffer.toString();
}

Parameterized test case classes in JUnit 3.x

I have a JUnit 3.x TestCase which I would like to be able to parameterize. I'd like to parametrize the entire TestCase (including the fixture). However, the TestSuite.addTestSuite() method does not allow be to pass a TestCase object, just a class:
TestSuite suite = new TestSuite("suite");
suite.addTestSuite(MyTestCase.class);
I would like to be able to pass a parameter (a string) to the MyTestCase instance which is created when the test runs. As it is now, I have to have a separate class for each parameter value.
I tried passing it an anynomous subclass:
MyTestCase testCase = new MyTestCase() {
String getOption() {
return "some value";
}
}
suite.addTestSuite(testCase.getClass());
However, this fails with the assertion:
... MyTestSuite$1 has no public constructor TestCase(String name) or TestCase()`
Any ideas? Am I attacking the problem the wrong way?

If this is Java 5 or higher, you might want to consider switching to JUnit 4, which has support for parameterized test cases built in.

Rather than create a parameterized test case for the multiple/different backends you want to test against, I would look into making my test cases abstract. Each new implementation of your API would need to supply an implementing TestCase class.
If you currently have a test method that looks something like
public void testSomething() {
API myAPI = new BlahAPI();
assertNotNull(myAPI.something());
}
just add an abstract method to the TestCase that returns the specific API object to use.
public abstract class AbstractTestCase extends TestCase {
public abstract API getAPIToTest();
public void testSomething() {
API myAPI = getAPIToTest();
assertNotNull(myAPI.something());
}
public void testSomethingElse() {
API myAPI = getAPIToTest();
assertNotNull(myAPI.somethingElse());
}
}
Then the TestCase for the new implementation you want to test only has to implement your AbstractTestCase and supply the concrete implementation of the API class:
public class ImplementationXTestCase extends AbstractTestCase{
public API getAPIToTest() {
return new ImplementationX();
}
}
Then all of the test methods that test the API in the abstract class are run automatically.

Ok, here is a quick mock-up of how JUnit 4 runs parameterized tests, but done in JUnit 3.8.2.
Basically I'm subclassing and badly hijacking the TestSuite class to populate the list of tests according to the cross-product of testMethods and parameters.
Unfortunately I've had to copy a couple of helper methods from TestSuite itself, and a few details are not perfect, such as the names of the tests in the IDE being the same across parameter sets (JUnit 4.x appends [0], [1], ...).
Nevertheless, this seems to run fine in the text and AWT TestRunners that ship with JUnit as well as in Eclipse.
Here is the ParameterizedTestSuite, and further down a (silly) example of a parameterized test using it.
(final note : I've written this with Java 5 in mind, it should be trivial to adapt to 1.4 if needed)
ParameterizedTestSuite.java:
package junit.parameterized;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Collection;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
public class ParameterizedTestSuite extends TestSuite {
public ParameterizedTestSuite(
final Class<? extends TestCase> testCaseClass,
final Collection<Object[]> parameters) {
setName(testCaseClass.getName());
final Constructor<?>[] constructors = testCaseClass.getConstructors();
if (constructors.length != 1) {
addTest(warning(testCaseClass.getName()
+ " must have a single public constructor."));
return;
}
final Collection<String> names = getTestMethods(testCaseClass);
final Constructor<?> constructor = constructors[0];
final Collection<TestCase> testCaseInstances = new ArrayList<TestCase>();
try {
for (final Object[] objects : parameters) {
for (final String name : names) {
TestCase testCase = (TestCase) constructor.newInstance(objects);
testCase.setName(name);
testCaseInstances.add(testCase);
}
}
} catch (IllegalArgumentException e) {
addConstructionException(e);
return;
} catch (InstantiationException e) {
addConstructionException(e);
return;
} catch (IllegalAccessException e) {
addConstructionException(e);
return;
} catch (InvocationTargetException e) {
addConstructionException(e);
return;
}
for (final TestCase testCase : testCaseInstances) {
addTest(testCase);
}
}
private Collection<String> getTestMethods(
final Class<? extends TestCase> testCaseClass) {
Class<?> superClass= testCaseClass;
final Collection<String> names= new ArrayList<String>();
while (Test.class.isAssignableFrom(superClass)) {
Method[] methods= superClass.getDeclaredMethods();
for (int i= 0; i < methods.length; i++) {
addTestMethod(methods[i], names, testCaseClass);
}
superClass = superClass.getSuperclass();
}
return names;
}
private void addTestMethod(Method m, Collection<String> names, Class<?> theClass) {
String name= m.getName();
if (names.contains(name))
return;
if (! isPublicTestMethod(m)) {
if (isTestMethod(m))
addTest(warning("Test method isn't public: "+m.getName()));
return;
}
names.add(name);
}
private boolean isPublicTestMethod(Method m) {
return isTestMethod(m) && Modifier.isPublic(m.getModifiers());
}
private boolean isTestMethod(Method m) {
String name= m.getName();
Class<?>[] parameters= m.getParameterTypes();
Class<?> returnType= m.getReturnType();
return parameters.length == 0 && name.startsWith("test") && returnType.equals(Void.TYPE);
}
private void addConstructionException(Exception e) {
addTest(warning("Instantiation of a testCase failed "
+ e.getClass().getName() + " " + e.getMessage()));
}
}
ParameterizedTest.java:
package junit.parameterized;
import java.util.Arrays;
import java.util.Collection;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.parameterized.ParameterizedTestSuite;
public class ParameterizedTest extends TestCase {
private final int value;
private int evilState;
public static Collection<Object[]> parameters() {
return Arrays.asList(
new Object[] { 1 },
new Object[] { 2 },
new Object[] { -2 }
);
}
public ParameterizedTest(final int value) {
this.value = value;
}
public void testMathPow() {
final int square = value * value;
final int powSquare = (int) Math.pow(value, 2) + evilState;
assertEquals(square, powSquare);
evilState++;
}
public void testIntDiv() {
final int div = value / value;
assertEquals(1, div);
}
public static Test suite() {
return new ParameterizedTestSuite(ParameterizedTest.class, parameters());
}
}
Note: the evilState variable is just here to show that all test instances are different as they should be, and that there is no shared state between them.

a few details are not perfect, such as the names of the tests in the IDE being the same across parameter sets (JUnit 4.x appends [0], [1], ...).
To solve this you just need to overwrite getName() and change the constructor in your test case class:
private String displayName;
public ParameterizedTest(final int value) {
this.value = value;
this.displayName = Integer.toString(value);
}
#Override
public String getName() {
return super.getName() + "[" + displayName + "]";
}

For Android projects, we wrote a library called Burst for test parameterization. For example
public class ParameterizedTest extends TestCase {
enum Drink { COKE, PEPSI, RC_COLA }
private final Drink drink;
// Nullary constructor required by Android test framework
public ConstructorTest() {
this(null);
}
public ConstructorTest(Drink drink) {
this.drink = drink;
}
public void testSomething() {
assertNotNull(drink);
}
}
Not really an answer to your question since you're not using Android, but a lot of projects which still use JUnit 3 do so because Android's test framework requires it, so I hope some other readers will find this helpful.