Is it no option to exclude some tests in IntelliJ IDEA Ultimate? I want to run unit tests in IntelliJ but exclude the integration tests. I name the integration tests with *IT.java so the Maven failsafe plugin can run them seperatly from the unit tests.
In the JUnit Run configuration set the Test kind to Pattern, specify the following regular expression as the pattern:
^(?!.*IT$).*$
It matches against the class name, so you don't need to match .java extension. Regular expression will not match if the class name ends with IT using the negative lookahead.
With JUnit5, you can now tag your tests, e.g: #Tag("integration-test").
Also, given IntelliJ supports now JUnit5 as well, you can then create a JUnit Test Configuration and select Test Kind: Tags (JUnit5).
To exclude let's say "integration-test", you just need to specify as tags:
!integration-test, and IntelliJ will run all your JUnit5 tests except the ones tagged with integration-test.
I would split them to that they are in different packages. They are doing different things after all. You can then run your tests per package. This link details how to do this.
Related
I'm migrating to Junit5 from 4. Can run all tests fine, single tests fine (both via the green play buttons). When I try to run tests tagged with #Tag (#Tag("foo")), it tells me that no tests are found. In the run configuration, I'm putting foo in the "Tag expressions" box, using Test kind: Tags. I've used Categories with Junit4 in the past, and Tags seem like a 1-to-1 replacement, right? I do still have some Category tags in my codebase, as I'm switching over. Several tests tagged with #Tag as well. Whether the #Category tag is present or not makes no difference in the behavior.
IntelliJ result:
Running categories [interface org.junit.Test]
Process finished with exit code 0
Use ˋorg.junit.jupiter.api.Testˋ instead of ˋorg.junit.Testˋ to run JUnit5 tests. Only those can handle the ˋTagˋ annotation.
I have a project that itself is a set of test cases for an application. The test cases are tested itself using a mockup of the application to ensure the tests itself is correct. This is important to ensure the test cases justify some sort of specifications that are hard to follow and easy to mess up.
Now I want the final tests (those who are tested before) being in src in Maven. As expected mvn test just executes the test cases of the test cases not the test cases themself.
So basically how do I execute test cases that reside inside the 'src' folder using a maven goal?
Try this :
<testSourceDirectory>src</testSourceDirectory>
I am currently working on a Maven Project, using JUnit for defining tests and Jenkins for CI and am looking into how I can group my tests.
Say I had a test class with 20 tests, but I don't want to run all 20 tests, I want to be able to configure which tests to run. For Example, in another standalone project using TestNG and Selenium you can create a test method with the following annotation:
#Test (groups = { "AllTest" })
public void myTestMethod()
{
.. do something
.. assert something
}
... and then I am able to call which group to run based on an XML configuration.
Is it possible to define such type of groupings using Jenkins? I have researched into this and came across the plugin "Tests Selector Plugin" however can't understand how to get started once I've installed the plugin. There is a Wiki Page for it but I can't understand what to do after installing.
I have copy pasted the example property file, and didn't really understand what I needed to manipulate in it. When building, I simply get that the property file cannot be found or Jenkins doesn't have permission; can't find a way around this either :(
It's possible via maven + maven-surefire-plugin
http://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html
You can run a single test, set of tests or tests by regexp.
I have some JUnit tests that contained in a .jar that is intended to be used as a library. The library contains some tests that should be run whenever the library is used in another project.
However when I create a new project using the library and run JUnit on it in Eclipse then the tests in the dependency .jar don't run / don't get detected by the JUnit test runner. I get the message:
No tests found with test runner 'JUnit 4'.
Is there a way I can configure the dependency .jar so that the tests will run alongside any tests that might be contained in the main project?
Basically I want the dependency .jar to "export" the tests to whatever projects it is used in.
I'm using Eclipse Juno, JUnit 4.10, and Maven for the dependency management.
EDIT:
The point of this library is to be able to help test projects that use it - i.e. it runs some specialised tests. This is why I want to be able to import the library .jar and have it contribute the extra tests to the importing project.
You can try Maven Surefire.
In some cases it would be useful to have a set of tests that run with various dependency configurations. One way to accomplish this would be to have a single project that contains the unit tests and generates a test jar. Several test configuration projects could then consume the unit tests and run them with different dependency sets. The problem is that there is no easy way to run tests in a dependency jar. The Surefire plugin should have a configuration to allow me to run all or a set of unit tests contained in a dependency jar.
This can be done as follows (Junit 3):
Ensure test jar contains a class which has a static suite() method
import junit.framework.Test;
import junit.framework.TestSuite;
public class AllTests {
public static Test suite()
{
TestSuite suite = new TestSuite( "All Tests");
suite.addTestSuite(TestOne.class);
suite.addTestSuite(TestTwo.class);
return suite;
}
}
Then in the project using the test-jar dependency:
create a TestCase:
package org.melati.example.contacts;
import org.melati.poem.AllExportedTests;
import junit.framework.Test;
import junit.framework.TestCase;
public class PoemTest extends TestCase {
public static Test suite()
{
return AllExportedTests.suite();
}
}
Now the tests will be found.
I think that making a library of unit tests (#Test annotated methods) is a bad idea. However, making a library of reusable test components is a good one. We've done this in a few open source projects, and you can take a look how it works.
One Maven module exports test components (we call them "mocks"), from src/mock/java directory. Exported artifact has -mock classifier. See rexsl/pom.xml (pay attention to highlighted lines).
Mock artifacts are being deployed to Maven Central, together with usual artifacts: http://repo1.maven.org/maven2/com/rexsl/rexsl-core/0.3.8/ (pay attention to ...-mock.jar files)
Modules that need that mocks can include them as usual artifacts, for example rexsl-core/pom.xml (see highlighted lines):
Then, in your unit tests just use the classes from that mock libraries, like regular builders of mocks, for example: BulkHttpFeederTest
That's how you can make your test artifacts reusable, in an elegant way. Hope it helps.
#Mikera,
I find that this may help you. Just extend the Testcase Class to one of your java classes in project and you can run that particular class to run it as a JUnit Test.
I am not sure that this is desirable - On the one hand, if you use a jar, its behaviour might be influenced by the external context, e.g. other libraries in the classpath. From inside the jar, there is no simple way to analyse this context and to adjust the tests accordingly. On the other hand, if you write and compile a library, you should test it before packaging it as a jar. You might even want to not include your tests.
If it is really important to you to run the tests again, I would be interested in what could make them fail without changing the jar. In that case, however, you might want to extend the testrunner. As far as I know it uses reflection. You can quite easily load jars in a classloader and go through all their classes. By reflection you can identify the test classes and assemble testsuites. You could look into the testrunner for an example. Still, you would need to start this process from outside, e.g. from inside one of your test classes in the client project. Here, QATest's approach might be helpful: By providing an overriden version of testsuite or testrunner, you could automate this - if the client uses your overridden API.
Let me know if this rather costly approach seems to be applicable in your scenario and I can provide code examples.
Why should the user of the jar run the test cases inside the jar!!! When the jar is packaged and delivered, it means that the unit tests are run successfully.
Typically, the jar itself should be either treated as a separate project or as one of the modules. In both the cases, unit test cases are run before its delivered.
i am trying to use below command. I have
Mytest1_test.java,
Mytest2_test.java,
Mytest3_test.java,
Mytest4_test.java,
Mytest5_test.java,
I want to run first 4 at once.
mvn test -Dtest=Mytest[1-4]_test
but it is giving me no test to run.
Can some one plz advice...?
Use:
mvn -Dtest=Mytest*_test test
For the regex to match, you may need to add the package name - just prefixing with .* might be enough (you can try with and without the .* suffix - I used it to match '.class' under JUnit4):
mvn test "-Dtest=%regex[.*Mytest[1-4]_test.*]"
Note that I've used quotes around the whole -Dtest argument to make it more readable when we add more test combinations.
Adding a Problematic Test
When we have a problematic test that fails only in certain combinations then we can add a comma, and a space and another test inside the quotes (") but after the ].
Let's say Mytest5_test fails with tests 1, 2 and 3. We can use regex for the first 3, followed by a comma and standard matching for Mytest5_test, with a * if we need to catch any packages:
mvn test "-Dtest=%regex[.*Mytest[1-3]_test.*], *Mytest5_test"
Strictly speaking we don't need the quotes, but without them we'd need to remove the space between the different tests which would make the command less readable.
Find a Conflicting Test
When our test fails when run in combination with others (static mocks are often the culprit) we also can use regex to help find conflicting tests.
To run all the tests in packages beginning with my.co.[a-m], and Mytest5_test we can use:
mvn test "-Dtest=%regex[.*my.co.[a-m].*], *Mytest5_test"
If I recall correctly, JUnit 4 needs slash separators, JUnit 5 needs dot separators and with Spock just has class name without the package. Using .* and the wildcard single character . as in the regex above will work whether test names have slashes or dots for package names.
Then change the regex to run other tests (e.g. [n-z]) and keep narrowing down until you have the combination you need.
The tricks is to use * character and to exclude other tests with ! character as below.
mvn test -Dtest=Mytest*_test,!Mytest5_test
According to the official documentation of Maven surefire plugin, the command below should work with a regular expression but it don't seem to work (no test executed).
mvn test -Dtest=%regex[Mytest[1-4]_test]
Tested on Windows 7 with Maven Surefire plugin 3.x.
If You have got tests in directories, you can launch all tests from one of them by writing in regexp, eg. for all tests from subdirectory '/doc' in qa/src/test/java/com/frax/doc/:
mvn -Dtest=*/doc/* test