I need to run certain tests depending using JUnitCore and Categories but I can't find a way to make it work, can you please take a look and let me know if this is valid?
I have the following TestSuite called:
#RunWith(Categories.class)
#IncludeCategory(FeatureA.class) //this is the interface required as per categories documenation
#SuiteClasses( { AllTests.class } ) //imagine that AllTests contains all my tests
public class FeatureASuite {
} //if I'm not mistaken this configuration
// will go over all my tests and
// pick up only the ones with category FeatureA
And then I have a main class that will handle the execution as follows:
public static void main(String[] args) {
List<Class<?>> classes = new ArrayList<Class<?>>(); //classes collection
boolean featureA= true; //as this is an example featureA is always enabled
if(featureA) { //if feature A enabled then..
classes.add(FeatureASuite.class); //...add the feature A suite.
}
JUnitCore jUnitCore = new JUnitCore(); //create the facade
jUnitCore.runClasses(classes.toArray(new Class[classes.size()])); //run the classes specified
}
After executing the code the tests are not run. I have tried this with a different runner (instead of using Categories.class I have tried Suite.class) and tests are executed, however I need to specify categories per test method and Suite.class is not hitting that mark.
I have found why my approach was not working, the implementation above is actually correct, the issue (what I consider a junit bug) is in how Junit reacts to RunWith, if any of the classes under SuiteClasses contains RunWith annotation for any reason the execution will stop before even starting to run a first test.
Related
I am trying to create Test cases during runtime.
Background:
I'm calling the test like this:
public class XQTest {
XQueryTest buildTest = new XQueryTest();
#Test
public void test() throws Exception {
buildTest.test();
}
}
Afterwards it searches the FileDirectory for matching Files and build tests from it.
XQueryTest.java
tester = new XQueryTester(a, b);
tester.testHeader(c, d);
XQueryTester.java performs the actual assertion.
Is it possible to "outsource" these actual Testcases, so it's easier to Identify which test failed on jenkins, because at the moment I only have One Test (XQTest.java) which generate serveral tests.
Another problem is, if one test fails, the whole Test failed and skips the rest, even though it's just a part of the whole.
Junit5 supports a runtime tests via the TestFactory and DynamicTest concepts.
See
https://dzone.com/articles/junit-5-dynamic-tests-generate-tests-at-run-time
https://www.baeldung.com/junit5-dynamic-tests
I have two test classes each containing a number of tests. I'd like to run both of these together without having to have the #BeforeClass setup method being ran both times. I am calling the classes like this:
public static void main(String[] args) {
TestListenerAdapter tla = new TestListenerAdapter();
TestNG testng = new TestNG();
testng.setTestClasses(new Class[] { TestClass1.class, TestClass2.class });
testng.addListener(tla);
testng.run();
}
The reason for this is because I have both of them calling in a pop up menu and only want to select the option one time. If this is unclear I will try to further explain.
I have a collection of individual tests across 5 classes. I want each class to be able to run separately, but I also want to make them run collectively should I desire. In the #BeforeClass I have each of them calling another class that will select what URL I want to use (I am testing with TestNG and using Selenium WebDriver).
When this code runs it will execute the #BeforeClass in each class I list, and I would like to, if possible, ignore the #BeforeClass in all the tests if I execute the tests using the code above.
I would recommend passing a transformer in to your TestNG test case that implements, IAnnotationTransformer2. That transformer can allow you to control the behavior of the non #Test Annotations at runtime.
IAnnotationTransformer2
You can use a #BeforeTest in a common class of your 2 test classes.
I understand you want to run the stuff inside #Before only once for your 2 test classes that will be executed at same time together.
If you are using maven + junit 4.x, there is an option for setup things before and after test suit start and complete.
Or you can simply create a #ClassRule at suite level, please see the doc
I have a requirement to write many tests. I have extended Suite Runner of JUnit in order to be able to add new annotations where I can mention several Prerequisite classes which will be executed before any of the tests or setups get executed. My Typical test looks like this.
#RunWith(CustomSuiteRunner.class)
#BeforeSuite(Prerequisite.class)
#AfterSuite(CleanupOperations.class)
#Suite.SuiteClasses({
SimpleTests.class,
WeatherTests.class
})
public class SimpleSuite {
}
I have overridden public void run(final RunNotifier notifier) to add code the required code to trigger prerequisites and cleanup operations mentioned in BeforeSuite and AfterSuite annotation.
Now, I'm trying to find out how I can achieve the same by extending BlockJUnit4Runner? I can't find any method equivalent to run that starts the execution to override the behaviour. There is runChild which gets triggered before a child gets executed.
The reason I'm looking for this is I'm trying created several rules in an Interface and make my tests implement that so that they will be available, however as Interface elements are static and final JUnit is ignoring these. In another Question I asked today I got answer that I can make JUnit consider rules mentioned in an Interface by extending BlockJUnit4Runner and overriding getTestRules().
So, Here is what I'm trying find out.
Is it possible to extend BlockJUnit4Runner to make it take a list of tests and run them as suite and also run some code before any tests get execute and after all tests are executed?
How can I extend Suite Runner to consider TestRules defined in an implemented interface?
It is pretty much possible to extend BlockJUnit4Runner and make it take a list of tests and run them as suite with required test dependencies handled within the extended runChild() method
public class CustomRunner extends BlockJUnit4ClassRunner {
private List<String> testsToRun = Arrays.asList(new String[] { “sample1” });
public CustomRunner(Class<?> klass) throws InitializationError {
super(klass);
}
public void runChild(FrameworkMethod method, RunNotifier notifier) {
//Handle any dependency logic by creating a customlistener registering notifier
super.runChild(method,notifier);
}
}
In JUnit 3, I could get all of the tests within a test suite with the following code:
TestSuite allTestsSuite = (TestSuite) AllTests.suite()
Enumeration enumeration = allTestsSuite.tests();
ArrayList listOfTests = Collection.list(enumeration);
However, I can't find an equivalent way of doing this in JUnit 4. Classes no longer have a .suite() method; they simply use the #Suite annotation. This wouldn't be a problem except that the Suite class no longer has a tests() method. There is a children() method, but that returns a list of Runners, which seem to be something different than why I'm looking for.
So how can I get the tests within a test suite in JUnit 4, like I could with JUnit 3?
The simplest way to perform any kind of filering is to create your own JUnit Categories.
See this Junit Category tutorial for more details but basically, you create your own categories named whatever you want
public interface GuiTest{ }
public interface DbTest { }
And now you can annotate either entire test classes or individual tests with that category:
#Category(GuiTest.class)
public void myJPanelTest{
#Test
public void testFoo(){
...
}
//look we can have other categories too
#Test
#Category(DbTest.class)
public void accidentalDbTest(){
}
}
Then in your test suite, you can specify to include or exclude tests that match the given category
#RunWith(Categories.class)
#IncludeCategory(GuiTest.class)
#ExcludeCategory(DbTest.class) //not sure if we need both but can't hurt
#SuiteClasses( {
...
})
public class GuiTestsOnlySuite{}
Using Categories is much better than having to come up with manually filtering tests based on ad-hoc naming conventions because it that is hard to remember to do (and to make sure everyone in your group adheres to the naming conventions) and since the categories are classes, you can use your IDE to search/refactor/ compile time check your category names.
The only downside I've seen is at least in my IDE, the tests take a little longer to run because there is extra reflection work to do to make sure the test matches your category filter criteria before it runs.
After a bit of experimentation, I discovered the following solution:
SuiteClasses suiteClassesAnnotation = AllTests.class.getAnnotation(SuiteClasses.class);
if (suiteClassesAnnotation == null)
throw new NullPointerException("This class isn't annotated with #SuiteClasses");
Class<?>[] classesInSuite = suiteClassesAnnotation.value();
Basically, it gets the classes the same way that JUnit itself gets them: by looking into the annotation and determining which values are included within it.
The category solution provided by dkatzel is also a good option if you're ultimately wanting to filter these classes, but if you need a list of classes in a suite for some other purpose such as code analysis, this is the simplest and most direct way to do it.
My scenario is:
public class ExampleTest extends AbstractExampleTest {
#Test(dependsOnMethods={"someMethodFromAbstractExampleTest"}
public void firstTest() {
// Assert
}
// here I would like to call CommonTests
}
public class CommonTests {
#Test
public void sharedTest() {
// Assert
}
}
The reason something like CommonTests exists, is that it will contain a repeated test sequence. The way I currently communicate information from ExampleTest to CommonTests is done via statics which seems to work, but probably not the best.
This works fine if I call CommonTests programmatically according to the TestNG documentation. The issue I have with that is the results aren't logged within the runner of ExampleTest.
#Test
public void actionBasedTest(ITestContext context) {
TestListenerAdapter tla = new TestListenerAdapter();
TestNG testng = new TestNG();
testng.setTestClasses(new Class[] { ExampleAction.class });
testng.addListener(tla);
context.getSuite().addListener(tla);
testng.run();
}
The above is slightly better, but the reporting back is limited to something like "org.testng.TestRunner#####" and doesn't expose the test methods run.
So my question is: can I run tests from another class(es) (not via inheritance) and get the results logged to the same listener?
EDIT: I want to avoid testng.xml.
Answering your last question , you can run tests of any classes using a testng.xml which allows you to structure your tests any way you like. You can specify your listener in the suite tag and that would be the listener used for all your classes. Refer this for examples.