We Keep Coding

Running JUnitCore with instances instead of classes

I'm looking to run JUnit 4.12+ programmatically, and a cursory search for doing so yielded (amongst many other similar posts) this answer, which prescribes the following basic solution:
#RunWith(Suite)
#Suite.SuiteClasses ({
MyTestClass1.class,
MyTestClass2.class
})
public class MyTestSuite {
}
Result testResults = JUnitCore.runClasses(MyTestSuite.class);
...and I was able to get this working, no sweat. So far so good!
Problem is: I have some pretty sophisticated test classes that need to be instantiated/injected with very specific properties at runtime...not something that can be done from inside a no-arg constructor. But the above method (specifying to just run any old instance of a set of classes) doesn't allow you to instantiate your test classes, configure them, and then run them.
Is there a way to do this? I couldn't find anything looking at the JUnit API. I am looking for something like:
MyTestClass1 mtc1 = new MyTestClass1(...);
MyTestClass2 mtc2 = new MyTestClass2(...);
Result testResults = JUnitCore.run(mtc1, mtc2);

You probably need custom runner to achieve that. Junit 4/5 comes with third party runner that can perform dependency Injection for Constructors and Methods. Few runner which are pretty popular are Mockito(MockitoJUnitRunner) and SpringJUnit4ClassRunner in case you are using Spring. You can check out custom runner and implementation details at:
https://github.com/junit-team/junit4/wiki/Custom-runners

I got this working with a custom Runner with sample (Groovy pseudo-code) as follows:
class MyRunner extends Runner {
#Override
Description getDescription() {
return null
}
#Override
void run(RunNotifier notifier) {
// LoginTests.class is a test class I want to run
LoginTests loginTests = new LoginTests(<my args here>)
Description description = Description.createSuiteDescription(LoginTests)
notifier.fireTestStarted(description)
try {
log.info("About to doSomething()...")
loginTests.doSomething()
log.info("Did it...")
notifier.fireTestFinished(description)
} catch(Throwable throwable) {
log.info("doSomething() failed...")
notifier.fireTestAssumptionFailed(new Failure(description, throwable))
}
}
}
Result testResults = new JUnitCore().run(Request.runner(new MyRunner()))

JUnit5 - How to get test result in AfterTestExecutionCallback

I write JUnit5 Extension. But I cannot find way how to obtain test result.
Extension looks like this:
import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
import org.junit.jupiter.api.extension.TestExtensionContext;
public class TestResultExtension implements AfterTestExecutionCallback {
#Override
public void afterTestExecution(TestExtensionContext context) throws Exception {
//How to get test result? SUCCESS/FAILED
}
}
Any hints how to obtain test result?

This work for me:
public class RunnerExtension implements AfterTestExecutionCallback {
#Override
public void afterTestExecution(ExtensionContext context) throws Exception {
Boolean testResult = context.getExecutionException().isPresent();
System.out.println(testResult); //false - SUCCESS, true - FAILED
}
}
#ExtendWith(RunnerExtension.class)
public abstract class Tests {
}

As other answers point out, JUnit communicates failed tests with exceptions, so an AfterTestExecutionCallback can be used to gleam what happened. Note that this is error prone as extension running later might still fail the test.
Another way to do that is to register a custom TestExecutionListener. Both of these approaches are a little roundabout, though. There is an issue that tracks a specific extension point for reacting to test results, which would likely be the most straight-forward answer to your question. If you can provide a specific use case, it would be great if you could head over to #542 and leave a comment describing it.

You can use SummaryGeneratingListener from org.junit.platform.launcher.listeners
It contains MutableTestExecutionSummary field, which implements TestExecutionSummary interface, and this way you can obtain info about containers, tests, time, failures etc.
You can create custom listener, for example:
Create class that extends SummaryGeneratingListener
public class ResultAnalyzer extends SummaryGeneratingListener {
#Override
public void testPlanExecutionFinished(TestPlan testPlan) {
//This method is invoked after all tests in all containers is finished
super.testPlanExecutionFinished(testPlan);
analyzeResult();
}
private void analyzeResult() {
var summary = getSummary();
var failures = summary.getFailures();
//Do something
}
}
Register listener by creating file
src\main\resources\META-INF\services\org.junit.platform.launcher.TestExecutionListener
and specify your implementation in it
path.to.class.ResultAnalyzer
Enable auto-detection of extensions, set parameter
-Djunit.jupiter.extensions.autodetection.enabled=true
And that's it!
Docs
https://junit.org/junit5/docs/5.0.0/api/org/junit/platform/launcher/listeners/SummaryGeneratingListener.html
https://junit.org/junit5/docs/5.0.0/api/org/junit/platform/launcher/listeners/TestExecutionSummary.html
https://junit.org/junit5/docs/current/user-guide/#extensions-registration-automatic

I have only this solution:
String testResult = context.getTestException().isPresent() ? "FAILED" : "OK";
It seems that it works well. But I am not sure if it will work correctly in all situations.

Fails in JUnit are propagated with exceptions. There are several exceptions, which indicate various types of errors.
So an exception in TestExtensionContext#getTestException() indicates an error. The method can't manipulate actual test results, so depending on your use case you might want to implement TestExecutionExceptionHandler, which allows you to swallow exceptions, thus changing whether a test succeeded or not.

You're almost there.
To implement a test execution callback and get the test result for logging (or generating a report) you can do the following:
import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
import org.junit.jupiter.api.extension.ExtensionContext;
public class TestResultExtension implements AfterTestExecutionCallback
{
#Override
public void afterTestExecution(ExtensionContext context) throws Exception
{
// check the context for an exception
Boolean passed = context.getExecutionException().isEmpty();
// if there isn't, the test passed
String result = passed ? "PASSED" : "FAILED";
// now that you have the result, you can do whatever you want
System.out.println("Test Result: " + context.getDisplayName() + " " + result);
}
}
And then you just add the TestResultExtension using the #ExtendWith() annotation for your test cases:
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import static org.junit.jupiter.api.Assertions.assertTrue;
#ExtendWith(TestResultExtension.class)
public class SanityTest
{
#Test
public void testSanity()
{
assertTrue(true);
}
#Test
public void testInsanity()
{
assertTrue(false);
}
}
It's a good idea to extend a base test that includes the extension
import org.junit.jupiter.api.extension.ExtendWith;
#ExtendWith(TestResultExtension.class)
public class BaseTest
{}
And then you don't need to include the annotation in every test:
public class SanityTest extends BaseTest
{ //... }

junit assume command - can i put it inside setup method to ignore all tests

I have a set of test cases and i would like the entire file ignored if some condition is met. Can i use
Assume.assumeTrue(precondition); in the setup method to ensure that if a precondition is false that the test will not run in the entire file?
so if i had a setup method like this:
#Before
public void setUp() throws Exception {
super.setUp();
Assume.assumeTrue(1==3);//entire test file should be ignored is my hope
//some setup stuff ...
}
can i hope that none of my test will run ? this is my end goal that on some condition being met i can ignore all tests in a file. I have tried it and it seems to ignore them but want a expert opinion and i dont want the Assume method to affect any other tests besides the one in the file its called in.

Even I thinking it is not a good practice, I can understand the #Ignore annotation to put some Tests in quarantine. But I am not sure about conditioning it to a flag.
That said, implement this:
https://gist.github.com/yinzara/9980184
Then use this #ConditionalIgnore annotation.
public class SomeTest {
#Rule
public ConditionalIgnoreRule rule = new ConditionalIgnoreRule();
#Test
#ConditionalIgnore( condition = IgnoredByTeamA.class )
public void testIgnoredByTeamA() {
...
}
}
public class IgnoredByTeamA implements IgnoreCondition {
public boolean isSatisfied() {
return true;
}
}
More details here

Using Mockito / PowerMock to verify a log has been written

I have a class which writes messages to some logs. The class is a utility which doesn't do anything else, it runs in the background, checks a few things and logs them. I'm wondering if it's possible for me to verify in a unit test that the log has been written to without caring about what it is actually writing. Here's my class being tested:
//imports...
public class MyClass {
private static Log log = LogFactory.getLog(MyClass.class);
public MyClass() {
log.info("MyClass is being created.");
}
public void doThing() {
if( everything_is_fine ) {
log.info("This is a message to say everything is fine.");
} else {
log.error("Uh oh...");
}
}
}
And my tester class:
// imports ...
#RunWith(PowerMockRunner.class)
#PrepareForTest({MyClass.class,LogFactory.class})
public class MyClassTest {
Log mockLog;
#Before
public void setup() {
PowerMockito.mockStatic(LogFactory.class);
mockLog = mock(Log.class);
PowerMockito.when(LogFactory.getLog(MyClass.class)).thenReturn(mockLog);
}
#Test
public void test_everything_is_ok() {
MyClass mything = new MyClass(); // should write to log.info
mything.doThing(); // should write to log.info
verify(mockLog, atLeastOnce()).info(anyString());
verify(mockLog, never()).error(anyString());
}
#Test
public void test_everything_is_not_ok() {
MyClass mything = new MyClass(); // should write to log.info
// do something which makes things not ok
mything.doThing(); // should write to log.error
verify(mockLog, atLeastOnce()).info(anyString());
verify(mockLog, atLeastOnce()).error(anyString());
}
}
When I run the tests, I expect that the log.info() is invoked for both tests, and the log.error() is invoked only for the second. However I'm getting a "Wanted but not invoked" for the log.info for both tests. and for log.error on the second. So either:
1) My code is broken and not writing to the log, or
2) My test is broken.
I'm thinking that I've messed up something in my test, probably something really obvious, so has anyone had experience testing something like this who could help me out? Any and all help will be appreciated.
UPDATE:
Thanks to those who helped out, I've got a solution now.
After playing around with the code for a bit, I discovered that there seems to be an issue with the initialization of the log. Doing private static Log log = LogFactory.getLog(MyClass.class); didn't seem to use the mock correctly, so if I move it to the constructor it seemed to be mocked OK and my tests all work as expected:
public class MyClass {
private static Log log;
public MyClass() {
MyClass.log = LogFactory.getLog(MyClass.class);
log.info("MyClass is being created.");
}
// etc ...
}
I've got it working now, but can anyone explain why initializing the log the first way didn't work? Or perhaps point me to somewhere which explains it? I'm not sure if there's a gap in my understanding of how Java initializes objects or if it's a limitation of mocking frameworks.

You can make an abstraction over the Logger like we did here in our project:
https://github.com/4finance/uptodate-gradle-plugin/blob/master/src/main/groovy/com/ofg/uptodate/LoggerProxy.groovy
Then you have to constructors
https://github.com/4finance/uptodate-gradle-plugin/blob/master/src/main/groovy/com/ofg/uptodate/UptodatePlugin.groovy
One with LoggerProxy initialized and one for tests where you can mock it out and verify if proper text was passed. No need for dirty hacks ;)

Since I had the same problem (private static Log log = LogFactory.getLog(MyClass.class);), I'm posting what I did to solve it.
Instead of modifying logger initialization (I was limited by coding rules) I simply split test in different classes...

Extending Junit ParentRunner Class to filter test methods

I have a requirement of reading a text file which contains list of all the testmethods in yes/no value and to pick the "yes" marked testmethods only for a TestCase Class,and to execute in Junit.
So I have written a script to read the file and to group it in a map< TestCaseName,ArrayList_ofEnabledTestMethods > . To run that I found one option is to use Assume.assumeTrue().
But I wanted to try some otherway... instead of writting extra lines before each test methods , So I tried to write a custom runner (ABCSuite which extends ParentRunner) and planned to use it in my TestSuite file like below :
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
#RunWith(ABCSuite.class)
#Suite.SuiteClasses({TestCalc.class})
public class BatTest{
}
Here TestCalc.class contains all the test methods some of which is marked "yes" in the earlier mentioned text file .
Please let me know how I can use of extending the ParentRunner class/Junit Libraries to achieve this . If any good tutorial is there or any link which addressed this before please.. share

You can do this by extending BlockJUnit4ClassRunner:
public class FilterRunner extends BlockJUnit4ClassRunner {
private List<String> testsToRun = Arrays.asList(new String[] { "test1" });
public FilterRunner(Class<?> klass) throws InitializationError {
super(klass);
}
#Override
protected void runChild(final FrameworkMethod method, RunNotifier notifier) {
Description description= describeChild(method);
if (method.getAnnotation(Ignore.class) != null || !testsToRun.contains(method.getName())) {
notifier.fireTestIgnored(description);
} else {
runLeaf(methodBlock(method), description, notifier);
}
}
}
You can fill in testsToRun as you like. The above will mark the other tests as Ignored. You use this like:
#RunWith(Suite.class)
#SuiteClasses({Class1Test.class})
public class TestSuite {
}
#RunWith(FilterRunner.class)
public class Class1Test {
#Test
public void test1() {
System.out.println("test1");
}
#Test
public void test2() {
System.out.println("test2");
}
}
This produces the following output:
test1
If you don't want to add the #FilterRunner to each test class, look at my answer to How to define JUnit method rule in a suite?.

The JUnit way of implementing this would be an implementation of a Filter. It must be instantiated by the Runner that implements Filterable. Filters are applied recursively through the tree of tests. So you only need to apply that filter once in your base suite.
You need to extend a runner and in the constructor apply the filter. To make things more flexible, you could configure the filters that should be applied with annotations.
I had the same requirement and that worked out well.