We Keep Coding

JUnit 5 #Tag does not work as expected with #BeforeEach/#AfterEach

Have a simple math operations test suite developed with JUnit 5.
I use two tags #Tag("add") and #Tag("minus").
With maven and surefire plugin I select the tag for the tests.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration combine.children="append">
<excludedGroups></excludedGroups>
<groups>add</groups>
</configuration>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.5.2</version>
</dependency>
</dependencies>
</plugin>
When I do the mvn test on the terminal it will execute only the tests marked by the #Tag("add") but for some reason will execute the #BeforeEach and #AfterEach tagged exclusively with #Tag("minus")
#Tag("minus")
#BeforeEach
void beforeEach(TestInfo test) { /* code */ }
#Tag("minus")
#AfterEach
void afterEach(TestInfo test) { /* code */ }
Since the expected behaviour was not working, I even forced with <excludedGroups>, changed from empty value to <excludedGroups>minus</excludedGroups> on pom.xml, still not worked.
There is something wrong on pom.xml that I'm missing?
A conflict between #Tag and #BeforeEach plus #AfterEach?

#Tag is meant for test classes and methods. It has nothing to do with setup/teardown of said tests.
You could either split those tests into separate test classes or you could leverage Nested Test. Nested tests could look something like this:
#DisplayName("A stack")
class MathTest {
#Nested
#DisplayName("when new")
#Tag("add")
class Add {
#BeforeEach
void setupAdding() {
// ...
}
#Test
#DisplayName("can add")
void canAdd() {
// ...
}
}
#Nested
#DisplayName("minus")
#Tag("minus")
class Minus {
#BeforeEach
void setupMinus() {
// ...
}
#Test
#DisplayName("can subtract")
void testMinus() {
// ...
}
}
}

I think that the #Tag annotation was conceived just to clasify your tests following a common characteristic. If you need different setUp() and teardown() I guess the righ way to go is to separate them in different test classes, not with the tag annotation.
In the documentation it is describes it can be used in a test class or a test method:
https://junit.org/junit5/docs/5.0.1/api/org/junit/jupiter/api/Tag.html

TestNG parallel classesAndMethods without XML

I'm trying to have both classes and methods running in parallel.
For example:
Method Tests
public class MethodTests(){
#Test(groups = "testMe")
public void methodTestOne(){
...
}
#Test(groups = "testMe")
public void methodTestTwo(){
...
}
}
Class Tests
-> Hoping the Test annotation on the class level would do it
#Test
public class ClassTests(){
#Test(groups = "testMe")
public void classTestOne(){
...
}
#Test(groups = "testMe")
public void classTestTwo(){
...
}
}
I've included the surefire-plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<parallel>all</parallel>
<threadCount>${threads}</threadCount>
...
</plugin>
Note: I've also tried classesAndMethods
I'm running from the command line like this:
mvn clean verify -Dgroups=testMe -Dthreads=3
I'm trying to accomplish the classTests run on one thread, and the method tests use a different thread after being complete.
Purpose:
In the beforeClass, I'm setting up the test, and then quickly asserting multiple things on a page, If one fails, I want it to still test the other things on the page.
Where as, if the method tests fail, I need it to completely stop.
I need both scenarios, and I do not want to use any XML.

I am not sure I quite understand the use case. But the only combinations of parallelism supported by TestNG are as below
tests - causes #Test methods inside tags to run in parallel.
instances - Causes #Test methods inside test class instances to run in parallel.
classes - causes test classes to run in parallel
methods - causes multiple #Test methods to run in parallel
I dont remember seeing something called all. I think that's applicable only to the JUnit provider for Maven surefire plugin. Please see here.
For your scenario as long as you have the #BeforeClass(alwaysRun=true) it would be executed for all groups and it would prevent the test methods that are dependent on the before class to be skipped.
The value that you should be using is classes, like this : <parallel>classes</parallel>

How to mock riak java client?

I'm trying to unit test code that uses com.basho.riak:riak-client:2.0.0. I mocked all riak client classes and was hoping to get a useless but working test. However, this fails with a null pointer:
java.lang.NullPointerException
at com.basho.riak.client.api.commands.kv.KvResponseBase.convertValues(KvResponseBase.java:243)
at com.basho.riak.client.api.commands.kv.KvResponseBase.getValue(KvResponseBase.java:150)
at com.basho.riak.client.api.commands.kv.FetchValue$Response.getValue(FetchValue.java:171)
My test looks like this:
#Test public void test() {
RiakClient riakClient = mock(RiakClient.class);
#SuppressWarnings("unchecked")
RiakCommand<FetchValue.Response, Location> riakCommand = (RiakCommand<FetchValue.Response, Location>) mock(RiakCommand.class);
Response response = mock(Response.class);
when(riakClient.execute(riakCommand)).thenReturn(response);
Response returnedResponse = riakClient.execute(riakCommand);
when(response.getValue(Object.class)).thenReturn(new Object());
MyPojo myData = returnedResponse.getValue(MyPojo.class);
// Make assertions
}
How do you unit test code that uses the riak client? Eventually I would like to ensure that the expected type/bucket/key combination is used and that the expected RiakCommand is run.
EDIT: I dug more into the FetchValue class and found this structure:
FetchValue
- is public final
FetchValue.Response
- is public static,
- has a package-private constructor Response(Init<?> builder)
FetchValue.Response.Init<T> is:
- protected static abstract class Init<T extends Init<T>> extends KvResponseBase.Init<T>
And there is FetchValue.Response.Builder:
static class Builder extends Init<Builder>
- with build() that: return new Response(this);
I assume that Mockito gets lost somewhere among the inner classes and my call ends up in KvResponseBase.convertValues, where the NP is thrown. KvResponseBase.convertValues assumes a List<RiakObject> of values and I see no sane way of assigning it.

I have investigate a bit your case. I have reduce your example to this simple SSCCE:
import static org.mockito.BDDMockito.given;
import static org.mockito.Mockito.mock;
import org.junit.Test;
import com.basho.riak.client.api.commands.kv.FetchValue.Response;
public class RiakTest {
#Test
public void test() throws Exception {
Response response = mock(Response.class);
given(response.getValue(Object.class)).willReturn(new Object());
}
}
which throws this error:
java.lang.NullPointerException
at com.basho.riak.client.api.commands.kv.KvResponseBase.convertValues(KvResponseBase.java:243)
at com.basho.riak.client.api.commands.kv.KvResponseBase.getValue(KvResponseBase.java:150)
at com.basho.riak.client.api.commands.kv.FetchValue$Response.getValue(FetchValue.java:171)
at RiakTest.test(RiakTest.java:12)
After some digging, i think i have identified the problem. It is that you are trying to stub a public method which is inherited from a package (visibility) class:
abstract class KvResponseBase {
public <T> T getValue(Class<T> clazz) {
}
}
It seems that Mockito fails to stub this method so the real one is invoked and a NullPointerException is thrown (due to an access of a null member: values).
One important thing to note is that if this function invocation not fails, Mockito would show a proper error:
org.mockito.exceptions.misusing.MissingMethodInvocationException:
when() requires an argument which has to be 'a method call on a mock'.
For example:
when(mock.getArticles()).thenReturn(articles);
Also, this error might show up because:
1. you stub either of: final/private/equals()/hashCode() methods.
Those methods *cannot* be stubbed/verified.
Mocking methods declared on non-public parent classes is not supported.
2. inside when() you don't call method on mock but on some other object.
I guess it is a Mockito bug or limitation so i have open an issue in the Mockito tracker where i have reproduce your case with simple classes.
UPDATE
The issue i opened is in fact a duplicate of an existing one. This issue will not be fixed but a workaround exists. You may use the Bytebuddy mockmaker instead of the cglib one. Explanations could be found here.

You can not mock final classes and final and/or static methods with mockito. Note that static nested classes are fine. This is because mockito subclasses (I'm not 100% sure this is the exact operation, it uses CGLIB to generate classes) objects, but isn't allowed to override final methods or extend the final classes. For static methods no overriding is ever possible.
In your code you probably are trying to call a final class or method. It is hard to tell which class causes the problem, from your NullPointer stackstrace you should suspect the first object on it that you have mocked (going up starting from the testcase method). The method on the mock should not be calling any other methods (expect internal to mockito), so probably that is final, because you is does not seem to call a 'mocked' method.
In your case the stacktrace is not complete (as your testcase is not on it). In a quick look on the riak framework I couldn't find the method take a look at FetchValue$Response.getValue.
Also note the following. From the snippet you posted, I can not tell what you are testing in your testcase. All objects you create are mocks. Normally you have 1 (or a few) real classes that you are testing. The other classes (that interact with your classes under test) you mock, to be able to simulate complex behavior.

Follow up:
Thanks #gontard I was able to find this:
<dependency>
<!-- We need this fix: https://github.com/mockito/mockito/pull/171 to use mockito with Riak -->
<!--http://stackoverflow.com/questions/28442495/how-to-mock-riak-java-client#28474106-->
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.0.52-beta</version>
<scope>test</scope>
</dependency>
That has the fixes included.
Unfortunately, if you use both Fetch and MultiFetch (likely), you're up a creek.
MultiFetch.Response is a final class (so you can use mockito, you need to use PowerMock)
FetchValue.Response has the issues you outlined, and can only be fixed with the beta mockito, not available with powermock yet...
Update, I figured out how to use both mockito & powermock together (until powermock upgrades):
<!-- We need this to mock Multi-Fetch responses from Riak, which are final -->
<!-- However, we need the beta version of mockito due to bugs (see below),
so we _cannot_ use the mockito api provided by powermock, do _not_ include _powermock-api-mockito, it'll mess stuff up -->
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<version>1.6.4</version>
<scope>test</scope>
</dependency>
<!--If we don't include this, we get: -->
<!--java.lang.IllegalStateException:
Extension API internal error: org.powermock.api.extension.proxyframework.ProxyFrameworkImpl could not be located in classpath.-->
<!-- it looks like this is due to some discrepancy in packaging with mockito 2, this may be fixed in Fall 2016:
https://groups.google.com/forum/#!topic/powermock/cE4T40Xa_wc -->
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-easymock</artifactId>
<version>1.6.4</version>
</dependency>
<!-- We need this fix: https://github.com/mockito/mockito/pull/171 to use mockito with Riak -->
<!--http://stackoverflow.com/questions/28442495/how-to-mock-riak-java-client#28474106-->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.0.52-beta</version>
<scope>test</scope>
</dependency>

Using JUnit RunListener in IntelliJ IDEA

I'm working on project where I need to perform some action before running each JUnit test. This problem was solved using RunListener that could be added to the JUnit core. The project assembly is done using Maven, so I have this lines in my pom file:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<properties>
<property>
<name>listener</name>
<value>cc.redberry.core.GlobalRunListener</value>
</property>
</properties>
</configuration>
</plugin>
So, everything works using:
mvn clean test
But when tests are started using IntelliJ (using its internal test runner) the actions coded in our RunListener are not executed, so it is impossible to perform testing using IntelliJ infrastructure.
As I see, IntelliJ does not parse this configuration from pom file, so is there a way to explicitly tell IntelliJ to add RunListener to JUnit core? May be using some VM options in configuration?
It is much more convenient to use beautiful IntelliJ testing environment instead of reading maven output.
P.S. The action I need to perform is basically a reset of static environment (some static fields in my classes).

I didn't see a way to specify a RunListener in Intellij, but another solution would be to write your own customer Runner and annotate #RunWith() on your tests.
public class MyRunner extends BlockJUnit4ClassRunner {
public MyRunner(Class<?> klass) throws InitializationError {
super(klass);
}
#Override
protected void runChild(final FrameworkMethod method, RunNotifier notifier) {
// run your code here. example:
Runner.value = true;
super.runChild(method, notifier);
}
}
Sample static variable:
public class Runner {
public static boolean value = false;
}
Then run your tests like this:
#RunWith(MyRunner.class)
public class MyRunnerTest {
#Test
public void testRunChild() {
Assert.assertTrue(Runner.value);
}
}
This will allow you to do your static initialization without a RunListener.

Maven Codehaus findbugs plugin "onlyAnalyze" option not working as expected

Update for the impatient: it's simple, use package.- for sub-package scanning instead of package.*, as-per martoe's answer below!
I cannot seem to get onlyAnalyze working for my multi-module project: regardless of what package (or pattern) I set, maven-findbugs-plugin doesn't evaluate sub-packages as I'd expect from passing it packagename.*.
To prove either myself or the plugin at fault (though I always assume it's the former!), I setup a small Maven project with the following structure:
pom.xml
src/
main/java/acme/App.java
main/java/acme/moo/App.java
main/java/no_detect/App.java
which is very simple!
The POM has the following findbugs configuration:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>2.4.0</version>
<executions>
<execution>
<phase>verify</phase>
<goals><goal>findbugs</goal><goal>check</goal></goals>
</execution>
</executions>
<configuration>
<debug>true</debug>
<effort>Max</effort>
<threshold>Low</threshold>
<onlyAnalyze>acme.*</onlyAnalyze>
</configuration>
</plugin>
</plugins>
</build>
and every App.java has the following code with two obvious violations:
package acme;
import java.io.Serializable;
public class App implements Serializable
{
private static final class NotSer {
private String meh = "meh";
}
private static final NotSer ns = new NotSer();// Violation: not serializable field
public static void main( String[] args )
{
ns.meh = "hehehe";// Vilation: unused
System.out.println( "Hello World!" );
}
}
Note that no_detect.App has the same content as above, but my expectation is that it wouldn't be evaluated by findbugs because I have the "onlyAnalyze" option set to acme.* which I assume would evaluate acme.App and acme.moo.App and nothing else.
I now execute a mvn clean install to clean, build, test, run findbugs, package, install, which produces the following findbugs report (snipped for brevity) and results in a build failure which is expected because acme.App and acme.moo.App:
<BugInstance category='BAD_PRACTICE' type='SE_NO_SERIALVERSIONID' instanceOccurrenceMax='0'>
<ShortMessage>Class is Serializable, but doesn't define serialVersionUID</ShortMessage>
<LongMessage>acme.App is Serializable; consider declaring a serialVersionUID</LongMessage>
<Details>
<p> This field is never read.&nbsp; Consider removing it from the class.</p>
</Details>
<BugPattern category='BAD_PRACTICE' abbrev='SnVI' type='SE_NO_SERIALVERSIONID'><ShortDescription>Class is Serializable, but doesn't define serialVersionUID</ShortDescription><Details>
<BugCode abbrev='UrF'><Description>Unread field</Description></BugCode><BugCode abbrev='SnVI'><Description>Serializable class with no Version ID</Description></BugCode>
To summarise: only acme.App is analysed, acme.moo.App isn't (bad) and neither is no_detect.App (good).
I tried with two wildcards in the onlyAnalyze option but that produces a successful build but with a findbugs error (Dangling meta character '*' etc).
I tried with onlyAnalyze set to acme.*,acme.moo.* which analyzes all the expected classes (acme.App and acme.moo.App) which means it "works" but not as I expect; i.e. I have to explicitly declare all parent-packages for the classes I want to analyze: that could get large and difficult to maintain on a multi-module project!
Do I have to define every package I want analyzed, or can I declare a wildcard/regex pattern that will do what I want?
I'd rather not use the inclusion/exclusion XML because that requires far more setup and reasoning that I don't currently have time for...

To cite the Findbugs manual: "Replace .* with .- to also analyze all subpackages"