We Keep Coding

JUnit5 - How to get test result in AfterTestExecutionCallback

I write JUnit5 Extension. But I cannot find way how to obtain test result.
Extension looks like this:
import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
import org.junit.jupiter.api.extension.TestExtensionContext;
public class TestResultExtension implements AfterTestExecutionCallback {
#Override
public void afterTestExecution(TestExtensionContext context) throws Exception {
//How to get test result? SUCCESS/FAILED
}
}
Any hints how to obtain test result?

This work for me:
public class RunnerExtension implements AfterTestExecutionCallback {
#Override
public void afterTestExecution(ExtensionContext context) throws Exception {
Boolean testResult = context.getExecutionException().isPresent();
System.out.println(testResult); //false - SUCCESS, true - FAILED
}
}
#ExtendWith(RunnerExtension.class)
public abstract class Tests {
}

As other answers point out, JUnit communicates failed tests with exceptions, so an AfterTestExecutionCallback can be used to gleam what happened. Note that this is error prone as extension running later might still fail the test.
Another way to do that is to register a custom TestExecutionListener. Both of these approaches are a little roundabout, though. There is an issue that tracks a specific extension point for reacting to test results, which would likely be the most straight-forward answer to your question. If you can provide a specific use case, it would be great if you could head over to #542 and leave a comment describing it.

You can use SummaryGeneratingListener from org.junit.platform.launcher.listeners
It contains MutableTestExecutionSummary field, which implements TestExecutionSummary interface, and this way you can obtain info about containers, tests, time, failures etc.
You can create custom listener, for example:
Create class that extends SummaryGeneratingListener
public class ResultAnalyzer extends SummaryGeneratingListener {
#Override
public void testPlanExecutionFinished(TestPlan testPlan) {
//This method is invoked after all tests in all containers is finished
super.testPlanExecutionFinished(testPlan);
analyzeResult();
}
private void analyzeResult() {
var summary = getSummary();
var failures = summary.getFailures();
//Do something
}
}
Register listener by creating file
src\main\resources\META-INF\services\org.junit.platform.launcher.TestExecutionListener
and specify your implementation in it
path.to.class.ResultAnalyzer
Enable auto-detection of extensions, set parameter
-Djunit.jupiter.extensions.autodetection.enabled=true
And that's it!
Docs
https://junit.org/junit5/docs/5.0.0/api/org/junit/platform/launcher/listeners/SummaryGeneratingListener.html
https://junit.org/junit5/docs/5.0.0/api/org/junit/platform/launcher/listeners/TestExecutionSummary.html
https://junit.org/junit5/docs/current/user-guide/#extensions-registration-automatic

I have only this solution:
String testResult = context.getTestException().isPresent() ? "FAILED" : "OK";
It seems that it works well. But I am not sure if it will work correctly in all situations.

Fails in JUnit are propagated with exceptions. There are several exceptions, which indicate various types of errors.
So an exception in TestExtensionContext#getTestException() indicates an error. The method can't manipulate actual test results, so depending on your use case you might want to implement TestExecutionExceptionHandler, which allows you to swallow exceptions, thus changing whether a test succeeded or not.

You're almost there.
To implement a test execution callback and get the test result for logging (or generating a report) you can do the following:
import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
import org.junit.jupiter.api.extension.ExtensionContext;
public class TestResultExtension implements AfterTestExecutionCallback
{
#Override
public void afterTestExecution(ExtensionContext context) throws Exception
{
// check the context for an exception
Boolean passed = context.getExecutionException().isEmpty();
// if there isn't, the test passed
String result = passed ? "PASSED" : "FAILED";
// now that you have the result, you can do whatever you want
System.out.println("Test Result: " + context.getDisplayName() + " " + result);
}
}
And then you just add the TestResultExtension using the #ExtendWith() annotation for your test cases:
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import static org.junit.jupiter.api.Assertions.assertTrue;
#ExtendWith(TestResultExtension.class)
public class SanityTest
{
#Test
public void testSanity()
{
assertTrue(true);
}
#Test
public void testInsanity()
{
assertTrue(false);
}
}
It's a good idea to extend a base test that includes the extension
import org.junit.jupiter.api.extension.ExtendWith;
#ExtendWith(TestResultExtension.class)
public class BaseTest
{}
And then you don't need to include the annotation in every test:
public class SanityTest extends BaseTest
{ //... }

set number of withdraws to 3 and test it in Junit Testing.

In a Saving account for the withdraw variables i need to make sure it can't withdraw more than 3 in a month. I checked the criteria in the setter but I'm not sure how to do the testing in the JUintTesting. As both of them are void method and the testing method returns a boolean i am getting a error of void below is my code.
public void setNumberWithdrawals(int w)
{
if (getNumberWithdrawals() > 3)
{
System.out.println("You already have more than 3 withdraw!!");
}
else
{
numberWithdrawals = w;
}
}
For JUNit testing
SavingsAccount sa1;
#Test
public void testsetNumberWithdrawals()
{
assertEquals(true, sa1.setTest(4));
}
I am expecting it to return false since i pass 4 but i keep on getting the void and return type expected error, i even tried assigning the value to another int variable for number and tested it but i still got the error.

Assumption: You're using JUnit 4 (seems like a safe assumption given that the code you provided is using the #Test annotation).
You would want something like this for your test class:
import static org.junit.Assert.*;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
public class SavingsAccountTest {
private final ByteArrayOutputStream outContent = new ByteArrayOutputStream();
private final ByteArrayOutputStream errContent = new ByteArrayOutputStream();
#Before
public void setUpStreams() {
System.setOut(new PrintStream(outContent));
System.setErr(new PrintStream(errContent));
}
#After
public void cleanUpStreams() {
System.setOut(null);
System.setErr(null);
}
#Test
public void test() {
SavingsAccount savingsAcct = new SavingsAccount(...);
savingsAcct.setNumberWithdrawals(...);
assertEquals("You already have more than 3 withdraw!!\r\n", outContent.toString());
}
}
Other thoughts:
Given the code you posted in your question and code on github some things seem a little strange. It might just be that some of the code is still missing, but I want to point out a few things that stand out:
You posted this code:
assertEquals(true, sa1.setTest(4));
Now according to what I read on github, sa1 is an instance of SavingsAccount yet I don't see a definition for this setTest(...) method anywhere. I do see this SavingsAccount extends Account so I suppose setTest(...) could be defined in the Account class. The reason this stands out is that you said you were trying to test SavingsAccount.setNumberWithdrawals(...) but you're not invoking setNumberWithdrawals in your test. Again, I suppose that this setTest method could indirectly invoke setNumberWithdrawals but that's not clear from everything you posted.
You said in your question:
make sure it can't withdraw more than 3
The logic you have is if (getNumberWithdrawals() > 3) which means the number of withdrawals will have to be more than 3 to trigger this logic. This seems to violate your requirement.

how to unit test code that should cause compile error

I have a class that take a ArrayList as parameter:
public class Foo {
private ArrayList<Bar> bars;
public Foo(ArrayList barList) {
bars = barList;
}
}
there is a bug that I can pass any ArrayList into the constructor:
// should compile error with this line
Foo foo = new Foo(new ArrayList<String>());
the problem is if I add this case to test suite, when the bug fixed, I can not compile it.
is there anyway to test this case?

I don't know of any language/unit test framework that will let you "test" for code that should not compile. If you can't compile it, there is nothing to test. You can however turn on all compiler warnings when building. I'm pretty sure passing an unparameterized collection is a big warning in any JVM after JDK5.

I feel it is bad practise and I don't really see a use for it, but see this example for how to test for compilations errors:
import static org.junit.Assert.assertTrue;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
#RunWith(JUnit4.class)
public class CompileTest {
//A very naive way checking for any error
#Test(expected=java.lang.Error.class)
public void testForError() throws Exception {
this.is.not.a.statement;
}
//An example which check that we only find errors for regarding compilation errors.
#Test
public void expectNotCompilableMethod() {
try {
uncompilableMethod();
fail("No compile error detected");
} catch (Error e) {
assertTrue("Check for compile error message", e.getMessage().startsWith("Unresolved compilation problems"));
}
}
private void uncompilableMethod() {
do.bad.things;
}
}
Edit:
1) I am not sure how this may behave together with build-tools like maven. As far as I know maven breaks the build at compile-errors, so probably the tests will not even be executed.

Fix your method signature:
public Foo(ArrayList<Bar> barList) {
bars = barList;
}
Problem solved. (You probably also want to check for null.)

Misplaced argument matcher detected here. You cannot use argument matchers outside of verification or stubbing in Mockito

Out of the following two test cases in BundleProcessorTest.java, i am getting below exception, although, my first test case passes successfully.
org.mockito.exceptions.misusing.InvalidUseOfMatchersException:
Misplaced argument matcher detected here:
-> at bundle.test.BundleProcessorTest.bundlePluginShouldNotBeNull(BundleProcessorTest.java:22)
You cannot use argument matchers outside of verification or stubbing.
Examples of correct usage of argument matchers:
when(mock.get(anyInt())).thenReturn(null);
doThrow(new RuntimeException()).when(mock).someVoidMethod(anyObject());
verify(mock).someMethod(contains("foo"))
Also, this error might show up because you use argument matchers with
methods that cannot be mocked. Following methods cannot be
stubbed/verified: final/private/equals()/hashCode().
at
bundle.test.BundleProcessorTest.bundlePluginCollectionShouldNotBeNull(BundleProcessorTest.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
Please find below simplified code listing :-
BundlePlugin.java
package bundle;
import java.util.List;
public class BundlePlugin {
private final String pluginName ;
private final List<String> featureContent ;
public BundlePlugin(String pluginName, List<String> featureContent) {
super();
this.pluginName = pluginName;
this.featureContent = featureContent;
}
public String getPluginName() {
return pluginName;
}
public List<String> getFeatureContent() {
return featureContent;
}
}
BundleProcessor.java
package bundle;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
public class BundleProcessor {
public BundlePlugin getBundlePlugin(String pluginName, Iterator<String> artifactIterator) {
List<String> featureContent = new ArrayList<String>() ;
return new BundlePlugin(pluginName, featureContent);
}
}
BundleProcessorTest.java
package bundle.test;
import static org.junit.Assert.assertNotNull;
import static org.mockito.Matchers.anyString;
import static org.mockito.Mockito.mock;
import java.util.Iterator;
import java.util.List;
import org.junit.Test;
import bundle.BundleProcessor;
public class BundleProcessorTest {
BundleProcessor bundleProcessor = new BundleProcessor() ;
#Test
public void bundlePluginShouldNotBeNull() {
Iterator<String> artifactIterator = mock(Iterator.class) ;
bundle.BundlePlugin bundlePlugin = bundleProcessor.getBundlePlugin(anyString(), artifactIterator) ;
assertNotNull( bundlePlugin );
}
#Test
public void bundlePluginContentShouldNotBeNull() {
Iterator<String> artifactIterator = mock(Iterator.class) ;
bundle.BundlePlugin bundlePlugin = bundleProcessor.getBundlePlugin(anyString(), artifactIterator) ;
List<String> featureContent = bundlePlugin.getFeatureContent() ;
assertNotNull( featureContent );
}
}
How to execute this test without problem.
Edit 1:
But if i mark the bundlePluginCollectionShouldNotBeNull test with #Ignore annotation, then first test case passes without any exception.

You are using mockito anyString() while calling the test method, it should be used only for verifying a mock object to ensure a certain method is called with any string parameter inside the test, but not to invoke the test itself. For your test use empty string "" instead to anyString().

Ideally anyString() should not be used outside the mock or verify block. I was facing the same issue.Changing the anyString() to some string ("xyz") value works fine.
Note : Make a note that you might use anyString() to some other methods that leads in failure of some other method. It wasted my one hour to figure it out. My actual test method was getting passes individually but when i was trying to run that in a hole it was getting failed because of the reason that some other test case was using anyString() outside to mock or verify block.

We need to add a text file to the project's src/test/resources/mockito-extensions directory named org.mockito.plugins.MockMaker and add a single line of text:
mock-maker-inline
please refer article https://www.baeldung.com/mockito-final

Simple, Should use #Spy annotation with #InjectMocks annotation.

junit - Expected and actual values are same but the test result is failure

Expected and actual values are same but the test result is failure in JUnit test. I don't know why this happens. Codes and result image are attached. One more weird thing is that there are error marks in the packages even though the source codes do not have any errors. An image about this is also attached). I guess two things are related each other. How can I solve this problem? Thanks.If you need more information, please ask me.
package tests;
import junit.framework.TestCase;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import net.n3.nanoxml.*;
public class NanoSetAttr3_wy_v1Tests extends TestCase {
public void test0() throws Exception {
//setattr3.out
String result;
ByteArrayOutputStream byteBuffer;
byteBuffer = new ByteArrayOutputStream();
System.setOut(new PrintStream(byteBuffer));
try{
SetAttr3_wy_v1.main(new String[] {});
}catch (Throwable t) {
t.printStackTrace(System.out);
}
result = new String(byteBuffer.toByteArray());
assertEquals(result, "<FOO Weight=\"80\"/>");
}
}

If you look closely at the Expected and Actual windows, you'll see that Expected has two lines and Actual has one. This means that the Expected output has a newline but the Actual output does not.
Regarding the error marks, open up Problems window for details (if you're not using Eclipse, there should be something similar).