run only fail test but after all build work
This is my fail test case
ExempleIT extends BaseIT {
#Test
public void testfailure(){
Assertions.assertTrue(false);
}
and this is my base test class
#ExtendWith({SpringExtension.class})
#SpringBootTest()
public abstract class BaseIT { ...
I tried
maven -Dfailsafe.skipAfterFailureCount=1 -Dfailsafe.rerunFailingTestsCount=2
and surefire but these does not support junit5
I rerunner-jupiter but this supports only method base but I want to use base class for all tests and after all build, Which other way can I follow?
Check this answer.
The maven-surefire-plugin, as of today, does not have full support of JUnit 5. There is an open issue about adding this support in SUREFIRE-1206.
As such, you need to use a custom provider.(...)
This info is from an old post.
The existing limitation with maven-surefire is related only with some features on JUNIT5.
I am trying to run a JUnit4 test using categories against an AllTests suite of tests. In example 1 running Suite3 works and example 2 running Suite2 produces the following exception.
java.lang.Exception: Category annotations on Parameterized classes
are not supported on individual methods.
I need to generate the TestSuite as the test is executed.
Any suggestions for how to correct the problem?
Thanks
example 1
#RunWith(Categories.class)
#IncludeCategory(SlowTest.class)
#SuiteClasses(AllTests3.class)
public class Suite3 {
}
#RunWith(Suite.class)
#SuiteClasses({
MathUtilTest.class, MathUtil2Test.class
})
public class AllTests3 {
}
example 2
#RunWith(Categories.class)
#IncludeCategory(SlowTest.class)
#SuiteClasses(AllTests2.class)
public class Suite2 {
}
public final class AllTests2 {
public static TestSuite suite() {
final TestSuite result = new TestSuite();
result.addTest(new JUnit4TestAdapter(MathUtilTest.class));
result.addTest(new JUnit4TestAdapter(MathUtil2Test.class));
return result;
}
private AllTests2() {
}
There are various related bugs in the latest stable version of JUnit 4 (4.12):
https://github.com/junit-team/junit4/issues/1203
Somebody committed a fix for the unreleased 4.13-SNAPSHOT version, which you can build and check out yourself by cloning their GitHub repo over at https://github.com/junit-team/junit4
However, I ran a couple of tests myself and have to conclude that the #Category annotation does work when used on your MathUtilTest class, but only if the class is annotated, the annotation is ignored when used on individual test methods.
The solution I am using to create a JUnit test suite dynamically can be found in this similar question here: How do I Dynamically create a Test Suite in JUnit 4?
The solution I am trying to adapt looks like such:
#RunWith(AllTests.class)
public class SomeTests
{
public static TestSuite suite()
{
TestSuite suite = new TestSuite();
suite.addTest(new JUnit4TestAdapter(Test1.class));
suite.addTest(new JUnit4TestAdapter(Test2.class));
return suite;
}
}
However, I would not only like to be able to dynamically create a test suite, but also be able to allow for the user running my program to specify which tests they would like to run using a properties file.
Is there a way I can annotate my classes with a String such that I can get the actual class type given the annotation String? Are there any viable solutions for this or is it just bad practice in general?
As I understand you would like to mark your test classes as belonging to one or more groups, then user defines the group of test cases to execute. Is it correct?
Bad or good this practice is, it is already implemented in TestNG. There is no such feature in JUnit. But you could easily add annotation scanner to your code and select proper test classes dynamically (e.g. by using https://github.com/ronmamo/reflections to collect all your annotated tests in class path with just single line of code).
I was looking for a verification test for custom implemented collections, and stumbled upon this: http://www.gamlor.info/wordpress/2012/09/google-guava-collection-test-suite/
I haven't used junit before (so I'm a total noob with junit). I added junit4 to my test project and... got stuck on how to actually run the test suite created by the Google Guava Collection Test Suite. I run annotated test from my test class just fine, but not the test suite from guava.
The junit docs say that suites are created by annotating a suite class with the Suite annotation, listing the cases they should include, but obviously I can't list a dynamically generated class that way. I would be happy to just create a simple test and run the entire suite as a single test, only... how do I get junit to run the suite instance?
If you have a factory method that returns a Test object (let's call it TestMaker.foo(), then you can write something like the following to take the Test and add it to your own TestSuite class:
import junit.framework.JUnit4TestAdapter;
import junit.framework.TestSuite;
import org.junit.runner.RunWith;
import org.junit.runners.AllTests;
#RunWith(AllTests.class)
public class TestRunner {
public static TestSuite suite() {
TestSuite ts = new TestSuite();
ts.addTest(TestMaker.foo());
return ts;
}
}
Now, when you run the TestRunner tests, it will also run the Test returned by TestMaker.foo() along with any other tests that might be defined directly in TestRunner.
For related discussion (for example, how to refer to a Test Class rather than pass a Test object), check out: How do I Dynamically create a Test Suite in JUnit 4?
I need to do some per-suite initialisation (starting a web-server). It is working fine except that when I run all tests in my project in eclipse my tests run twice. My test suite looks a bit like this:
#RunWith(Suite.class)
#Suite.SuiteClasses({
SubtestOne.class,
SubtestTwo.class
})
public class TestSuite
{
[...]
}
public class SubtestOne
{
#Test public void testOne() { [...] }
}
public class SubtestTwo
{
#Test public void testTwo() { [...] }
}
When I run all test in project in eclipse this causes the junit plugin to run the tests twice like this:
SubtestOne
SubtestTwo
TestSuite
SubtestOne
SubtestTwo
Is it possible to make "run all test in project" not run the sub-tests twice? I want my sub tests to be only ever run as part of the suite.
No, the test class will always be started directly and then through the "link" in the suite. This is as expected.
One workaround might to set in the run configuration to only run tests from the package which contains your suites. Open the run configuration and select Run all tests in the selected project, package or source folder then click Search... and select the package.
I realize that this has been asked over 5 years ago, but as quite a few folks up-voted the question I thought I'd still chime in with a solution. Skip right to the end if you just want the solution; read the whole text if you also want to understand it ;-)
First of all, it is indeed possible to ensure that a particular JUnit test class gets only run inside a test suite. Also, it is irrelevant whether you want to run that test suite inside Eclipse (as asked here) or any other tool or environment; this is really a pure JUnit issue for the most part.
Before I sketch out the solution, it might be a good idea to revisit what the exact problem is here. All JUnit tests need to be visible and instantiable to be picked up by the JUnit framework and its various runners. This also applies to test suites and the individual tests that are part of a test suite. As a consequence, if JUnit picks up the test suite it will also pick up the individual tests, and all tests in the suite will be executed twice, once individually and once as part of the suite.
So, the trick, if you will, is to prevent JUnit from picking up the individual tests while still being able to instantiate and execute them as part of the suite.
One thing that comes to mind is to make the test classes static inner classes nested inside the test suite. However, the nested classes still need to be public (otherwise they can't be run in the suite either), and if they are public classes they will also be picked up individually, despite being nested inside the suite's public class. JUnit will not try to run test classes that are not considered visible, though. So, nesting the test classes inside a non-public class would presumably be sufficient to hide them, but we can't make the suite class non-public because then JUnit would not execute it. What we can do, however, is to nest the individual tests inside another non-public class that's nested inside the test suite, which leads us to the solution of this conundrum:
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
#RunWith(Suite.class)
#SuiteClasses({AllTests.InSuiteOnly.Test1.class, AllTests.InSuiteOnly.Test2.class})
public class AllTests
{
static class InSuiteOnly
{
public static class Test1
{
#Test
public void test1()
{
//...
}
}
public static class Test2
{
#Test
public void test2()
{
//...
}
}
}
}
A lot of folks will probably object to all tests needing to be inside a single source file now. What if I want to maintain separate JUnit test classes that don't get executed by themselves but still get executed inside the test suite? A simple solution is to make the individual test classes abstract (public/non-public doesn't matter) so that JUnit won't execute them, and inside the test suite we simply use concrete subclasses of the original abstract test classes:
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
#RunWith(Suite.class)
#SuiteClasses({AllTests.InSuiteOnly.SuiteTest1.class, AllTests.InSuiteOnly.SuiteTest2.class})
public class AllTests
{
static class InSuiteOnly
{
public static class SuiteTest1 extends Test1 {}
public static class SuiteTest2 extends Test2 {}
}
}
abstract class Test1
{
#Test
public void test1()
{
//...
}
}
abstract class Test2
{
#Test
public void test2()
{
//...
}
}
This scheme works with Maven, Eclipse, and all other environments that either directly leverage JUnit's runners or implement their own runners that closely follow JUnit's original behavior and semantics.
I have an idea for you. Actually you do not want to run these test case as stand-alone test cases. You can do the following.
Mark the test cases with annotation #RunWith(DoNothingRunner.class)
Implment DoNothingRunner as following:
public class DoNothingRunner extends Runner {
public Description getDescription() {
return "do nothing";
}
public void run(RunNotifier notifier) {
// indeed do nothing
}
}
I have not tried this personally but I hope this will work.
do you need the suite in the first place ? depending on when you click for run all (class, package, or src/test/java), all underlying tests will be executed. So what's the point of having a suite ?
There is a solution, it's a bit tricky, but it may easily resolve your problem: create one suite class, and include all your suite classes in it. Then you can use this suite class to run all your tests.
#RunWith(Suite.class)
#Suite.SuiteClasses({
AXXSuite.class,
BXXSuite.class,
CXXSuite.class
})
public class AllSuites {
}