I have some JUnit tests that contained in a .jar that is intended to be used as a library. The library contains some tests that should be run whenever the library is used in another project.
However when I create a new project using the library and run JUnit on it in Eclipse then the tests in the dependency .jar don't run / don't get detected by the JUnit test runner. I get the message:
No tests found with test runner 'JUnit 4'.
Is there a way I can configure the dependency .jar so that the tests will run alongside any tests that might be contained in the main project?
Basically I want the dependency .jar to "export" the tests to whatever projects it is used in.
I'm using Eclipse Juno, JUnit 4.10, and Maven for the dependency management.
EDIT:
The point of this library is to be able to help test projects that use it - i.e. it runs some specialised tests. This is why I want to be able to import the library .jar and have it contribute the extra tests to the importing project.
You can try Maven Surefire.
In some cases it would be useful to have a set of tests that run with various dependency configurations. One way to accomplish this would be to have a single project that contains the unit tests and generates a test jar. Several test configuration projects could then consume the unit tests and run them with different dependency sets. The problem is that there is no easy way to run tests in a dependency jar. The Surefire plugin should have a configuration to allow me to run all or a set of unit tests contained in a dependency jar.
This can be done as follows (Junit 3):
Ensure test jar contains a class which has a static suite() method
import junit.framework.Test;
import junit.framework.TestSuite;
public class AllTests {
public static Test suite()
{
TestSuite suite = new TestSuite( "All Tests");
suite.addTestSuite(TestOne.class);
suite.addTestSuite(TestTwo.class);
return suite;
}
}
Then in the project using the test-jar dependency:
create a TestCase:
package org.melati.example.contacts;
import org.melati.poem.AllExportedTests;
import junit.framework.Test;
import junit.framework.TestCase;
public class PoemTest extends TestCase {
public static Test suite()
{
return AllExportedTests.suite();
}
}
Now the tests will be found.
I think that making a library of unit tests (#Test annotated methods) is a bad idea. However, making a library of reusable test components is a good one. We've done this in a few open source projects, and you can take a look how it works.
One Maven module exports test components (we call them "mocks"), from src/mock/java directory. Exported artifact has -mock classifier. See rexsl/pom.xml (pay attention to highlighted lines).
Mock artifacts are being deployed to Maven Central, together with usual artifacts: http://repo1.maven.org/maven2/com/rexsl/rexsl-core/0.3.8/ (pay attention to ...-mock.jar files)
Modules that need that mocks can include them as usual artifacts, for example rexsl-core/pom.xml (see highlighted lines):
Then, in your unit tests just use the classes from that mock libraries, like regular builders of mocks, for example: BulkHttpFeederTest
That's how you can make your test artifacts reusable, in an elegant way. Hope it helps.
#Mikera,
I find that this may help you. Just extend the Testcase Class to one of your java classes in project and you can run that particular class to run it as a JUnit Test.
I am not sure that this is desirable - On the one hand, if you use a jar, its behaviour might be influenced by the external context, e.g. other libraries in the classpath. From inside the jar, there is no simple way to analyse this context and to adjust the tests accordingly. On the other hand, if you write and compile a library, you should test it before packaging it as a jar. You might even want to not include your tests.
If it is really important to you to run the tests again, I would be interested in what could make them fail without changing the jar. In that case, however, you might want to extend the testrunner. As far as I know it uses reflection. You can quite easily load jars in a classloader and go through all their classes. By reflection you can identify the test classes and assemble testsuites. You could look into the testrunner for an example. Still, you would need to start this process from outside, e.g. from inside one of your test classes in the client project. Here, QATest's approach might be helpful: By providing an overriden version of testsuite or testrunner, you could automate this - if the client uses your overridden API.
Let me know if this rather costly approach seems to be applicable in your scenario and I can provide code examples.
Why should the user of the jar run the test cases inside the jar!!! When the jar is packaged and delivered, it means that the unit tests are run successfully.
Typically, the jar itself should be either treated as a separate project or as one of the modules. In both the cases, unit test cases are run before its delivered.
Related
I have a mock class with a trivial implementation of a service I provide from a module. I'm using OpenJDK 11.03, gradle 5.2.1 and IntelliJ 2019.2.
In /main/code/myPackage/myService.java I have:
package myPackage;
class myService {
public abstract void someFunction();
}
And in my test/code/somePackage/myMockService I have:
package myPackage;
// no import, they're in the same package.
class myMockService extends myService {
#Override
public void someFunction() { System.out.prinln("Hello World"); }
}
In my main/code/module-info.java I have:
module myModule {
exports somePackage;
}
I've tried several variations on a test/code/module-info.java, without success. For example:
// "open module" lets anyone use reflection within (mostly JUnit 5 in my case)
import myPackage.myService;
import myPackage.myMockService;
open module myTestModule {
exports myPackage;
provides myService with myMockService
}
The above module-info.java spews errors about how "module name myTestModule does not match expected name myModule", "package 'myPackage' is not visible" (from myMockModule.java), explaining "package myPackage is declared in module myModule but module myTestModule does not read it"
On the other hand, with the following module-info.java, I get a different batch of errors (below the code)
import myPackage.myService;
import myPackage.myMockService;
open module myModule {
provides myService with myMockService;
}
Without a requires myModule;, every reference to the main code branch from my test code gives an "error: cannot find symbol". With a requires myModule;, I get an "error: cyclic dependence involving myModule".
So... my tests can't be in a different module. AND they can't be the same module! [long string of expletives deleted]
How do I introduce a mock version of a service in test code rather than creating an entirely different module/gradle sub-project?
Or is this simply a case where that's not possible, and while you can have a separate test module-info, you can't do much with it?
Or is there some way to dynamically load things at runtime such that I don't have to put every little mock service in any module-info, test or otherwise? Such that ServiceLoader.load() will find them. Hmm... perhaps extend ServiceLoader and wrap its usage in main code such that it'll use the right one either in production code or test code...
a) Welcome to "Testing in the Modular World"!
TL;DR https://sormuras.github.io/blog/2018-09-11-testing-in-the-modular-world.html
Having one or more dedicated test modules is good. With all bells-and-whistles, read module-info.java declarations. Those test modules are your main modules' first clients. Just make sure, that your build tool packages all main modules before compiling and running the test modules. Otherwise you don't test your main modules as close as possible to reality — others will consume your main modules as JAR files. So should you. This solves all issues with services and multi-release JARs as well.
Now the interesting part: in-module testing, also named white box testing. Or how do test types residing non-exported packages or package-private types in exported packages? Either use a build that knows how to patch test modules into main modules (or vice versa) at test compile and/or test runtime. Like pro or Bach.java (which I maintain), or in your case of using Gradle, see b)elow part of this answer.
b) Gradle and Java main, test, … modules are not friends out-of-the-box, yet
Best plugin-based solution: https://github.com/java9-modularity/gradle-modules-plugin -- which honors the pass theses java command line options at test runtime module-info.test configuration file (which I invented). Here you basically desribe your test module requirements via verbose command line options, although a perfect DSL already exists: module-info-java ... loop back to a) and the module-aware build tools.
c) IntelliJ IDEA and Java test modules are ... improving!
https://youtrack.jetbrains.com/issue/IDEA-171419 module-info.java support in 2019.3
https://youtrack.jetbrains.com/issue/IDEA-222831 module-info.test support, soon?
This is my base module which needs implementations of interfaces defined in myspi package. Various providers can offer MyProvider implementations. Base module uses them via myspi.MyProvider interface implementation.
module base {
exports myspi;
uses myspi.MyProvider;
}
This is my sample implementation module which provides the MyProvider implementation with MyProviderImpl
module myspi.provider {
provides myspi.MyProvider with myspi.provider.MyProviderImpl;
}
All these work fine when I load the implementations in base module, with
public static List<MyProvider> getMyProviders() {
var myProviders = new ArrayList<MyProvider>();
for (MyProvider myProvider : ServiceLoader.<MyProvider>load(MyProvider.class)) {
myProviders.add(myProvider);
}
return myProviders;
}
But same code returns empty list in Junit 5 test code (ServiceLoader returns null). How can I test the service provider modules with Junit 5. Or is there any alternative to Junit that allows us to create test modules (modularized test API) that declares "uses myspi.MyProvider" in the module-info and works fine with getMyProviders()?
Basically you're on the right track. You need to convince the Java module system that your test modules are the single source of thruth when it comes to resolve modules are test runtime.
Black-box testing is easy.
White-box testing in the modular world, meaning testing protected and package private members within a module, is tricky. There are at least two ways to achieve this: a) use java command line options to configure the Java module system at test startup or b) blend main sources into the test sources at compile time and maintain a dedicated module-info.java in your test sources.
Please visit the links to the blogs and examples posted over at How to make a modular build with jdk > 1.8
Here is an excerpt for convenience:
Examples
Work-in-progress blueprint https://github.com/sormuras/sandbox/tree/master/sors-modular-testing-blueprint
Integration tests starting with "modular-world-" at https://github.com/sormuras/junit-platform-maven-plugin/tree/master/src/it
Background and other resources
https://github.com/junit-team/junit5-samples/tree/master/junit5-modular-world
https://github.com/forax/pro
https://blog.codefx.org/java/five-command-line-options-to-hack-the-java-9-module-system/
And expect most IDE to not support you either. For now.
SOLVED!
I've removed the Junit from class-path to module-path and also removed all Junit 4 compatibility stuff such as RunWith() etc, and made my test pure Junit 5 test.
I've added a module-info.java (Junit 5 doesn't require an open module although the books tell the opposite)
After I've modularized the tests I found that it still doesn't execute the ServiceLoader stuff. Then I've started looking for the fault myself.
And I found it! Running the ServiceLoader stuff in base module was possible, because the base module refers to the exported myProvider.jar, which in turns access a myProvider-config.properties file in the same directory. Without this config file myProvider cannot work properly.
The problematic test module on the other hand, refered the eclipse project of the myProvider instead of its exported .jar file and hence could not find its config file and exits. I'd moved this config file from Netbeans to Eclipse simply copying it into the same directory. Thus missing config file was the problem.
Changing the project settings I could run the tests without any failure.
I would like to thank all the contributors who responded.
This is quite an old post but if anyone gets here trying to test java modules with junit 5 with gradle, especially the consumer/provider as presented in this post , Sormuras solution is the easy way, to patch the consumer module with the tests classes.
it is supported by gradle-modules-plugin that does that out of the box:
https://github.com/java9-modularity/gradle-modules-plugin
I have a project that itself is a set of test cases for an application. The test cases are tested itself using a mockup of the application to ensure the tests itself is correct. This is important to ensure the test cases justify some sort of specifications that are hard to follow and easy to mess up.
Now I want the final tests (those who are tested before) being in src in Maven. As expected mvn test just executes the test cases of the test cases not the test cases themself.
So basically how do I execute test cases that reside inside the 'src' folder using a maven goal?
Try this :
<testSourceDirectory>src</testSourceDirectory>
I have a package structure containing Unit Tests that corresponds the production code. When I do the refactoring, I often move classes to different packages. In that case I also have to manually move the corresponding Unit Test class to the new package and sometimes I forget to do so. Is there any solution, how to move the corresponding JUnit test class to a corresponding package automatically?
I use Eclipse Luna.
There is a plugin called "MoreUnit" which is very helpful for everything concerning working with JUnit tests. It supports what you are looking for. Have a look at http://moreunit.sourceforge.net/
I want to be able to run all tests in a project programmatically. I know Eclipse has a "Run as JUnit test" configuration which somehow grabs all the tests in a project and run them. Is there any way for me to also grab the list of tests programmatically and run them? Or is there some good way to construct a test suite containing all the test cases without manually listing out every one (all 700+) of them?
I've tried the "New... -> Test Suite" option in Eclipse, but that seems to work only for JUnit 3, identifying tests by their extending from TestCase
The test classes are JUnit 4, so their only distinguishing characteristic is the annotation, no naming convention, no subclassing from TestCase.
Thanks in advance!
Though it does not really solve your immediate problem, I find it a very useful general practice to create suites and suites of suites, e.g. for a package something like PackageFooSuite etc. and assemble these suites in one or more suites again, like ModuleFooSuite and have one top-level suite, like AllTestsSuite. That way it's easy to run both all tests in one step as well as submodule tests for the package I'm currently working on (and have the tests run quicker than if I would always run all of them):
#RunWith(Suite.class)
#Suite.SuiteClasses({ PackageFooSuite.class, PackageBarSuite.class} )
public final class AllTestsSuite {} // or ModuleFooSuite, and that in AllTests
None of the other answers did it for me. I had 40k tests I needed to run, so manually listing every class was not an option.
I did it with ClasspathSuite. A test suite that runs all Junit4 and Junit3 test cases in the class path is as follows:
import org.junit.extensions.cpsuite.ClasspathSuite;
import org.junit.extensions.cpsuite.ClasspathSuite.*;
import org.junit.runner.RunWith;
import org.junit.runner.JUnitCore;
import static org.junit.extensions.cpsuite.SuiteType.*;
#RunWith(ClasspathSuite.class)
#SuiteTypes({ JUNIT38_TEST_CLASSES, TEST_CLASSES })
public class RunAllSuite {
/* main method not needed, but I use it to run the tests */
public static void main(String args[]) {
JUnitCore.runClasses(RunAllSuite.class);
}
}
I needed to run it from command line, so this is what I did:
Downloaded cp-1.2.6.jar
Create the previously mentioned RunAllSuite
Compile the class, javac RunAllSuite.java -cp cpsuite-1.2.6.jar;junit-4.8.1.jar
run it with target tests in the class path, java -cp cpsuite-1.2.6.jar;junit-4.8.1.jar;path/to/runallsuite/folder;target/classes;target/test-classes RunAllSuite
And that's it. With the RunAllSuite above, anywhere in your code you can just do JUnitCore.runClasses(RunAllSuite.class), which runs all tests in class path. There are other config options as well which are explained in the ClasspathSuite home page.
Note also that the class given above does not print anything. If that is needed, you can do
import org.junit.extensions.cpsuite.ClasspathSuite;
import org.junit.extensions.cpsuite.ClasspathSuite.*;
import org.junit.runner.RunWith;
import org.junit.runner.JUnitCore;
import org.junit.internal.TextListener;
import static org.junit.extensions.cpsuite.SuiteType.*;
#RunWith(ClasspathSuite.class)
#SuiteTypes({ JUNIT38_TEST_CLASSES, TEST_CLASSES })
public class RunAllSuite {
public static void main(String args[]) {
JUnitCore junit = new JUnitCore();
junit.addListener(new TextListener(System.out));
junit.run(RunAllSuite.class);
}
}
You can do this fairly easily from within maven using the surefire plugin: I usually clean/compile/install my projects from the command line before comparing them for eclipse usage (mvn eclipse:clean eclipse:eclipse) and you can define a test suite in your pom which lists all the tests you want to run en masse every time you run mvn install. You're not calling them programatically, exactly, but you can certainly call them en masse.
In Eclipse (I'm using 4.6.1) - Right click the project folder, select "Run As", choose "JUnit Test"
It will run all tests in that project. Same for a package.
Of the top of my head using Spring:
Implement a TypeFilter that matches classes with methods annotated with #Test (don't forget to consider the superclasses)
Invoke classpath scanning on your top-most test package
Invoke the JUnitRunner with the scan results
More info on classpath scanning and custom type filters here
With Eclipse Indigo (possibly Helios as well) in the Run Configurations dialog box, you now have the ability to Run all tests in a selected project, package or source folder.
Also a good reference from Eclipse is the article Java Unit testing with JUnit 4.x in Eclipse.
I also recommend using the JUnit Suite annotations. Follow the link for more detail.