I have a maven project (multimodule, JDK 11) that has a lot of modules and tests within it.
I suddenly noticed that one of my tests is NOT invoked as part of the surefire plugin.
Other tests in the same module do get to be invoked successfully.
I am able to run this test successfully via Intellij. The test is not disabled nor ignored. it contains a set of #Test methods. In addition, I am able to run it specifically using mvn test -Dtest=... argument.
The test class gets to be compiled successfully and appears in target/test-classes.
(I have already run with -X and could not find relevant additional data there)
I am working with surefire plugin 2.22.1 ; junit-vintage-engine:5.2.0 ; junit-platform-runner:1.2.0 ; Spring Boot 2.5.3 (with BOM) ; My test is spring based and invoked using #ExtendWith(SpringExtension.class)
(I do have other spring based tests that are invoked as expected though).
Following this question, I have also tried to work with 2.22.2. It did not solve the problem
Following this question, I have verified and indeed all tests in the test class that does not invoke are prefixed with the 'test' word
EDIT:
The problem happens in a test that belongs to a module that is part of a larger project in my company, which inherits various defaults across the POM inheritance chain. Unfortunately, I can not isolate only my POM out of it and present it.
I have tried to reproduce the problem in a new isolated project but did not succeed (that is, all the tests were invoked as expected).
What am I missing? Is there a hidden switch of the surefire plugin that reveals what's going on under the hood and where do surefire pull the list of tests he is meant to execute?
The surefire plugin (by default) will run only tests with the following name syntax ...
"**/Test*.java" - includes all of its subdirectories and all Java filenames that start with "Test".
"**/*Test.java" - includes all of its subdirectories and all Java filenames that end with "Test".
"**/*Tests.java" - includes all of its subdirectories and all Java filenames that end with "Tests".
"**/*TestCase.java" - includes all of its subdirectories and all Java filenames that end with "TestCase".
More details can be found here
Problem solved!
Following #Kathrin Geilmann comment:
...By default surefire only executes: ...Test.java, Test....java,
...Tests.java and ...TestCase.java
The name of my file ended with ...Tester (!##$$%^)
Thanks.
There are a bunch of questions related to the issue that mvn test doesn't execute test classes where the answer is always, that maven-surefire looks for certain prefixes or postfixes of filenames. I already considered this and it didn't help me as my filenames where already valid.
When I execute mvn test, the log contains the following section:
[INFO] --- maven-compiler-plugin:3.8.1:testCompile
(default-testCompile) # compiler --- [INFO] Changes detected -
recompiling the module! [INFO] Compiling 5 source files to
/project_path/target/test-classes
... so mvn test does actually find my test cases, and indeed, I can find all the compiled test classes in the directory /project_path/target/test-classes/
But still, only one of these 5 test files is actually executed and its tests are run whereas the other 4 are ignored by mvn test.
Any ideas why that could be the case? I am not really sure which additional information could be helpful to answer this question. Just ping me if there is something that I need to add to the question.
I just found a way to fix it..
Honestly, I don't understand why this makes the difference, but I have used #Test annotations on my test cases from org.junit.jupiter.api.Test in the file that was executed correctly by mvn test and used #Test annotations from org.junit.Test in the other files (which were not executed).
I've changed all files to import the #Test annotation from the jupiter package and then all tests were recognized.
Let's say I have a Java project using Maven 3 and junit. There are src/main/java and src/test/java directories which contain main sources and test sources, respectively (everything is standard).
Now I want to migrate the project to Java 9. src/main/java content represents Java 9 module; there is com/acme/project/module-info.java looking approximately like this:
module com.acme.project {
require module1;
require module2;
...
}
What if test code needs module-info.java of its own? For example, to add a dependence on some module that is only needed for tests, not for production code. In such a case, I have to put module-info.java to src/test/java/com/acme/project/ giving the module a different name. This way Maven seems to treat main sources and test sources as different modules, so I have to export packages from the main module to the test module, and require packages in the test module, something like this:
main module (in src/main/java/com/acme/project):
module prod.module {
exports com.acme.project to test.module;
}
test module (in src/test/java/com/acme/project):
module test.module {
requires junit;
requires prod.module;
}
This produces
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:testCompile (default-testCompile) on project test-java9-modules-junit: Compilation failure: Compilation failure:
[ERROR] /home/rpuch/git/my/test-java9-modules-junit/src/test/java/com/acme/project/GreeterTest.java:[1,1] package exists in another module: prod.module
because one package is defined in two modules. So now I have to have different projects in main module and test module, which is not convenient.
I feel I follow wrong path, it all starts looking very ugly. How can I have module-info.java of its own in test code, or how do I achieve the same effects (require, etc) without it?
The module system does not distinguish between production code and test code, so if you choose to modularize test code, the prod.module and the test.module cannot share the same package com.acme.project, as described in the specs:
Non-interference — The Java compiler, virtual machine, and run-time system must ensure that modules that contain packages of the same name do not interfere with each other. If two distinct modules contain packages of the same name then, from the perspective of each module, all of the types and members in that package are defined only by that module. Code in that package in one module must not be able to access package-private types or members in that package in the other module.
As indicated by Alan Bateman, the Maven compiler plugin uses --patch-module and other options provided by the module system when compiling code in the src/test/java tree, so that the module under test is augmented with the test classes. And this is also done by the Surefire plugin when running the test classes (see Support running unit tests in named Java 9 modules). This means you don't need to place your test code in a module.
You might want to rethink the project design you're trying to implement. Since you are implementing a module and its test into a project, you shall refrain from using different modules for each of them individually.
There should just be one single module-info.java for a module and its corresponding tests.
Your relevant project structure might look like this:-
Project/
|-- pom.xml/
|
|-- src/
| |-- test/
| | |-- com.acme.project
| | | |-- com/acme/project
| | | | |-- SomeTest.java
| |
| |-- main/
| | |-- com.acme.project
| | | |-- module-info.java
| | | |-- com/acme/project
| | | | |-- Main.java
where the module-info.java could further be:-
module com.acme.project {
requires module1;
requires module2;
// requires junit; not required using Maven
}
Just to sum all of the above as per your questions --
I feel I follow wrong path, it all starts looking very ugly. How can I
have module-info.java of its own in test code, or how do I achieve the
same effects (require, etc) without it?
Yes, you should not consider managing different modules for test code making it complex.
You can achieve similar effect by treating junit as a compile-time dependency using the directives as follows-
requires static junit;
Using Maven you can achieve this following the above-stated structure and using maven-surefire-plugin which would take care of patching the tests to the module by itself.
Adding some details.
In Java since 9, a jar file (or a directory with classes) may be put on classpath (as earlier), or on module path. If it is added to classpath, its module-info is ignored, and no module-related restrictions (what reads what, what exports what, etc) are applied. If, however, a jar is added to module path, it is treated as a module, so its module-info is processed, and additional module-related restrictions will be enforced.
Currently (version 2.20.1), maven-surefire-plugin can only work in the old way, so it puts the classes being tested on classpath, and module-path is ignored. So, right now, adding module-info to a Maven project should not change anything with tests being run using Maven (with surefire plugin).
In my case, the command line is like the following:
/bin/sh -c cd /home/rpuch/git/my/test-java9-modules-junit && /home/rpuch/soft/jdk-9/bin/java --add-modules java.se.ee -jar /home/rpuch/git/my/test-java9-modules-junit/target/surefire/surefirebooter852849097737067355.jar /home/rpuch/git/my/test-java9-modules-junit/target/surefire 2017-10-12T23-09-21_577-jvmRun1 surefire8407763413259855828tmp surefire_05575863484264768860tmp
The classes under test is not added as a module, so they are on classpath.
Currently, a work is under way in https://issues.apache.org/jira/browse/SUREFIRE-1262 (SUREFIRE-1420 is marked as a duplicate of SUREFIRE-1262) to teach surefire plugin to put code under test on module path. When it is finished and released, a module-info will be considered. But if they will make the module under test to read junit module automatically (as SUREFIRE-1420 suggests), module-info (which is a main module descriptor) will not have to include a reference to junit (which is only needed for tests).
A resume:
module-info just needs to be added to the main sources
for the time being, surefire ignores new module-related logic (but this will be changed in the future)
(when modules will work under surefire tests) junit will probably not need to be added to the module-info
(when modules will work under surefire tests) if some module is required by tests (and only by them), it may be added as a compile-only dependence (using require static), as suggested by #nullpointer. In this case, the Maven module will have to depend on an artifact supplying that test-only module using compile (not test) scope which I don't like much.
I just want to add my 0.02$ here on the general testing approach, since it seems no one is addressing gradle and we use it.
First thing first, one needs to tell gradle about modules. It is fairly trivial, via (this will be "on" since gradle-7):
plugins.withType(JavaPlugin).configureEach {
java {
modularity.inferModulePath = true
}
}
Once you need to test your code, gradle says this:
If you don’t have a module-info.java file in your test source set (src/test/java) this source set will be considered as traditional Java library during compilation and test runtime.
In plain english, if you do not define a module-info.java for testing purposes - things "will just work" and in the majority of cases this is exactly what we want.
But, that is not the end of story. What if I do want to define an JUnit5 Extension, via ServiceLocator. That means I need to go into module-info.java, from tests; one that I yet do not have.
And gradle has that solved again:
Another approach for whitebox testing is to stay in the module world by patching the tests into the module under test. This way, module boundaries stay in place, but the tests themselves become part of the module under test and can then access the module’s internals.
So we define a module-info.java in src/test/java, where I can put :
provides org.junit.jupiter.api.extension.Extension with zero.x.extensions.ForAllExtension;
we also need to do --patch-module, just like maven plugins do it. It looks like this:
def moduleName = "zero.x"
def patchArgs = ["--patch-module", "$moduleName=${tasks.compileJava.destinationDirectory.asFile.get().path}"]
tasks.compileTestJava {
options.compilerArgs += patchArgs
}
tasks.test {
jvmArgs += patchArgs
}
The only problem is that intellij does not "see" this patch and thinks that we also need a requires directive (requires zero.x.services), but that's not really the case. All the tests run just fine from command line and intellij.
The example is here
Also note that maven-surefire-plugin now has useModulePath false as a configuration option.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M6</version>
<configuration>
<useModulePath>false</useModulePath> <!-- tests use classpath -->
</configuration>
</plugin>
This is an option where a project uses module-path for main but classpath for tests and testing. It is probably not a bad option for people to go to this approach if "patching" the module-path gets painful.
Edit: We can also set this via property - surefire.useModulePath e.g.
<properties>
<surefire.useModulePath>false</surefire.useModulePath>
</properties>
I was not able to make it work also with the latest Maven surefire plugin version (3.0.0-M5). It seems if the main sources are using a module, the compiler plugin when using Java 11 is also expecting referenced packages to be in a module.
My solution was to place an own module-info.java inside the test sources (src/test/java in Maven) for the test module with the below contents.
I my case I had to use the keyword open (See Allowing runtime-only access to all packages in a module) because I'm using Mockito in my test, which requires reflective access.
// the same module name like for the main module can be used, so the main module has also the name "com.foo.bar"
open module com.foo.bar {
// I use junit4
requires junit;
// require Mockito here
requires org.mockito;
// very important, Mockito needs it
requires net.bytebuddy;
// add here your stuff
requires org.bouncycastle.provider;
}
I've been using Gradle + TestNG + Java + Selenium for my web UI tests for quite a while now and I've only recently run in to this issue. For some reason when I try to run a single test class using -DtaskName.single=ExampleTestClass where ExampleTestClass would be ExampleTestClass.java it only works on some of my test classes.
I'm getting the error: Could not find a matching test for pattern: ExampleTestClass
I've seen this error in the past due to typos or missing #Test annotations etc, so I'm familiar with the "normal" cause, but this is quite bizarre as it appears to work on some test classes and not others. I've inspected the code and all annotations and groups are in place for the test methods, they run fine from my IDE (IntelliJ), and they are all located in the same directory / package path. Is there something I'm missing here? I don't know if I'm seeing things but I did notice that it didn't work with a test class that did not have Test as the last four characters of the Java class name but upon renaming it, still no dice. I've read the documentation and can't find anything wrong. Is there anything else that may be causing this to fail? It's quite odd since these tests are all so similar in every way. I even checked character encoding etc - no discrepancies between any of them.
Any advice or ideas on where to look next would be great.
Cheers,
Darwin
I ran into the exact same thing with gradle 1.6 (haven't had time to upgrade), and TestNG. A single test in a project with multiple tests get skipped and also gradle complains about not finding it if you try running the single test alone. Debug run shows the missing test .class file being found by gradle.
I worked around it by adding an #Test annotation to the test class in addition to the test method. That seems to make gradle find it.
I recently ran into a similar issue. I tried to run a test with -DTest.single and --tests but it wasn't found. After a bit of frustration I realized that the test was in a test group that was excluded in my test task configuration. I had incorrectly assumed that running with -Dtest.single would overrule exclusions, including that test's group allowed it to run as a single test.
Having a very basic issue with running tests in a Java Play 2.0 app. It won't pick up my tests.
I've tried to put a test folder in the root of the project and use Junit as the docs seem to suggest. This results in:
$ play test
[info] Loading project definition from /Users/.../play20TestTest/project
[info] Set current project to play20TestTest (in build file:/Users/.../testapps/play20TestTest/)
[info] No tests to run for test:test
[success] Total time: 1 s, completed May 22, 2012 11:16:52 AM
I've also tried putting test under app. This at least picks up my code, but the test packages aren't available on my classpath so I assume it's incorrect.
I made a simple example and copied SimpleTest from the docs exactly: https://github.com/jsimone/Play2UnitTest
You placed your test in test/test/SimpleTest.java, move it to test/SimpleTest.java and it will work with play test command.