I am experimenting with testng. My goal is to have test methods in several classes and "supporting" methods for preparation and wrap-up of a bunch of tests in a separate class.
Another requirement is that in a test suite the supporting methods have to be called for multiple test parts. E.g. a first part containing testA and testB, and a second part containing testC and testD. This would result in the following steps:
support1, testA, testB, support2, support1, testC, testD, support2
My first approach that (partly) worked was to annotate all methods with #Test, use groups and define dependencies between groups, e.g. the test methods depend on a group "setUp", which is a group of one supporting method "support1" in the above example.
The problem with this approach is that the supporting methods count as tests, so the generated report shows the wrong number of "real" tests.
The next idea was to use #BeforeGroups and #AfterGroups, put the supporting methods in a group, and use group dependencies. The supporting methods should not be counted as tests any more. But I am stuck at the very beginning.
For example I tried
#BeforeGroups (groups = {"setUp"})
for a setup method in class Support, and
#Test(groups = { "TestA" }, dependsOnGroups = { "setUp" })
in a "real" test class. This results in the following (simplyfied) error:
[testng] DependencyMap::Method "TestClass.testSomething()[...]" depends on nonexistent group "setUp"
Why is group "setUp" nonexistent? Did I overlook something?
Or is there another approach which works?
Thanks for your help!
Edit:
The tests are started with Ant and I use a testng.xml like this:
<test name="TestA">
<groups>
<run>
<include name="setUp" />
<include name="TestA"/>
<include name="tearDown"/>
</run>
</groups>
<classes>
<class name="seleniumtest.test.technical.Support"/>
<class name="seleniumtest.test.business.TestClassA"/>
</classes>
</test>
<test name="TestB">
<groups>
<run>
<include name="setUp" />
<include name="TestB"/>
<include name="tearDown"/>
</run>
</groups>
<classes>
<class name="seleniumtest.test.technical.Support"/>
<class name="seleniumtest.test.business.TestClassB"/>
</classes>
</test>
I got the glitch!!
The problem is with the annotation
#Test(groups = { "TestA" }, dependsOnGroups = { "setUp" })
Basically your error message is trying to say that there is no #Test method with groupname as setUp!! Coming to your question, the solution is to modify annotation for the test method as below
#Test(groups = { "TestA" })
And in the support method modify the annotation
#BeforeGroups (groups = {"TestA"})
I ran a sample example with this set up
public class TestSupport {
#BeforeGroups(groups = { "sample","sample1" })
public void beforeTest() {
System.out.println("Before test");
}
#AfterGroups(groups = { "sample","sample1" })
public void afterTest() {
System.out.println("after test");
}
}
and with my Test class as
public class TestClassA {
#Test(groups = { "sample" })
public void superTestA() {
System.out.println("This is the actual test");
}
#Test(groups = { "sample" })
public void superTestB() {
System.out.println("This is the another test under sample group");
}
#Test(groups = { "sample1" })
public void superTest() {
System.out.println("This is another test");
}
}
and my testng.xml as shown below
<test name="sampletest" >
<groups>
<run>
<include name="sample" />
<include name="sample1" />
</run>
</groups>
<classes>
<class name="test.global.testng.TestClassA"/>
<class name="test.global.testng.TestSupport"/>
</classes>
</test>
Now this is how the test runs: beforeGroups-> superTestA/superTestB ->afterGroups and beforeGroups-> superTest -> afterGroups and closes off
I think I have come up with the solution I wanted.
What I need to use is #BeforeTest and #AfterTest instead of #BeforeGroups and #AfterGroups, respectively, in the support class:
#BeforeTest(groups = {"setUp"})
public void beforeTest() {[...]}
#AfterTest( groups = {"tearDown"})
public void afterTest() {[...]}
In the test class:
#Test(groups = { "TestA" })
public void testSomething() {[...]}
The dependsOnGroups is gone, as in Patton's approach.
The testng.xml is unchanged compared to my question. I.e. the tests can be configured in the testng.xml file, without having to change java code.
Moreover, this solution also gets rid of another problem of the BeforeGroups approach, at least as supposed by Patton (#Patton I do not mean to offend you).
With the latter a test using several test groups does not run as intended, because the beforeTest() method would be run before any of the groups. E.g. if you have the following test (extract of testng.xml):
<groups>
<run>
<include name="TestA"/>
<include name="TestB"/>
</run>
</groups>
... the resulting steps of execution are:
beforeTest(), TestA, beforeTest(), TestB, afterTest().
Using the solution with BeforeTest, you would have the following test:
<groups>
<run>
<include name="setUp" />
<include name="TestA"/>
<include name="TestB"/>
<include name="tearDown"/>
</run>
</groups>
... the resulting steps of execution are:
setUp = beforeTest(), TestA, TestB, tearDown = afterTest().
package com.test.MySample;
import org.testng.annotations.*;
public class TestNGTest1 {
#BeforeTest
public void BeforeTest() {
System.out.println("#BeforeTest");
}
#BeforeClass
public void BeforeClass() {
System.out.println("#BeforeClass");
}
#BeforeGroups (groups = {"My group"})
public void BeforeGroups() {
System.out.println("#BeforeGroups");
}
#BeforeGroups (groups = {"My group1"})
public void BeforeGroups1() {
System.out.println("#BeforeGroups1");
}
#AfterGroups (groups = {"My group1"})
public void AfterGroups1() {
System.out.println("#AfterGroups1");
}
#BeforeMethod
public void BeforeMethod() {
System.out.println("#BeforeMethod");
}
#Test(groups = {"My group"})
public void test1() {
System.out.println("test1");
}
#Test (groups = {"My group", "My group1"})
public void test2() {
System.out.println("test2");
}
#AfterMethod
public void AfterMethod() {
System.out.println("#AfterMethod");
}
#AfterGroups (groups = {"My group"})
public void AfterGroups() {
System.out.println("#AfterGroups");
}
#AfterClass
public void AfterClass() {
System.out.println("#AfterClass");
}
#AfterTest
public void AfterTest() {
System.out.println("#AfterTest");
}
}
Related
I have two separate packages in my project, one for integration tests and one for unit tests, my testng.xml looks as follows:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="All test cases" verbose="1" parallel="classes">
<test name="Integration Tests">
<classes>
<class name="com.sample.integration.ClassC"/>
<class name="com.sample.integration.ClassD"/>
</classes>
</test>
<test name="Unit tests">
<classes>
<class name="com.sample.unit.ClassA"/>
<class name="com.sample.unit.ClassB"/>
</classes>
</test>
</suite>
Class C:
public class ClassC {
#BeforeTest
public void beforeIntegrationTests() {
System.out.println("Before Integration tests");
}
#Test
public void classCMethod() {
System.out.println("Executing class C method");
}
}
Class D:
public class ClassD {
#Test
public void classDMethod() {
System.out.println("Executing class D method");
}
}
Class A:
public class ClassA {
#BeforeTest
public void beforeUnitTests() {
System.out.println("Before unit tests");
}
#Test
public void classAMethod() {
System.out.println("Executing class A method");
}
}
Class B:
public class ClassB {
#Test
public void classBMethod() {
System.out.println("Executing class B method");
}
}
If I run the entire test suite it works as expected as follows:
Before Integration tests
Executing class C method
Executing class D method
Before unit tests
Executing class A method
Executing class B method
However, if I try to either run/debug just classAMethod() from ClassA, it runs beforeUnitTests() [expected] and classAMethod() [expected], however it also runs beforeIntegrationTests() which is not expected. As per the official documentation: #BeforeTest: The annotated method will be run before any test method belonging to the classes inside the <test> tag is run.
How do I configure TestNG and/or IntelliJ to run this correctly?
Side Note: Although I can see that the beforeIntegrationTests() is getting run either by adding a breakpoint in the debug mode or by adding a Thread.sleep in the run mode, the output from this method does not get printed in final console output.
Firstly, is the expectation valid, as in I expect only beforeUnitTests() and classAMethod() to run if I run just the classAMethod().
No, all methods with #BeforeTest annotation will run before execution of method with #Test annotation.
Ideal way to handle this scenario is with groups.
Class A:
public class ClassA {
#BeforeTest(groups="unitTest")
public void beforeUnitTests() {
System.out.println("Before unit tests");
}
#Test(groups="unitTest")
public void classAMethod() {
System.out.println("Executing class A method");
}
}
Class B:
public class ClassB {
#Test(groups="unitTest")
public void classBMethod() {
System.out.println("Executing class B method");
}
}
Class C:
public class ClassC {
#BeforeTest(groups="integrationTest")
public void beforeIntegrationTests() {
System.out.println("Before Integration tests");
}
#Test(groups="integrationTest")
public void classCMethod() {
System.out.println("Executing class C method");
}
}
Class D:
public class ClassD {
#Test(groups="integrationTest")
public void classDMethod() {
System.out.println("Executing class D method");
}
}
testNG XML:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="All test cases" verbose="1" parallel="classes">
<groups>
<run>
<include name="unitTest"></include>
</run>
</groups>
<test name="Test Suite">
<classes>
<class name="com.sample.integration.ClassC"/>
<class name="com.sample.integration.ClassD"/>
<class name="com.sample.unit.ClassA"/>
<class name="com.sample.unit.ClassB"/>
</classes>
</test>
</suite>
You can make further configuration to this testNg XML as per your needs.
I want to repeat performing of some subsequence of the test methods in my TestNG project. Let's say I have the following test class:
#Test
class Abc
{
public void a() {}
public void b() {}
public void c() {}
public void d() {}
public void e() {}
}
And the following testng.xml:
<suite name="My suite" verbose="1">
<test name="simple test">
<classes>
<class name="Abc">
<methods>
<include name="e"/>
<include name="d"/>
<include name="c"/>
<include name="b"/>
<include name="a"/>
</methods>
</class>
</classes>
</test>
</suite>
How I can repeat performing of subset of test that begins from the "c" method, so that after the last method in test sequence is completed the methods "c", "b", "a" will be called ?
You may use:
groups and dependsOnGroups;
dependsOnMethods;
or priority;
use Factory;
Run TestNG programitically.
All of them will help you to order you tests if you really needs it.
See code and xml example.
Say I have the following XML:
<suite name="MATS">
<test name="mats_test">
<groups>
<run>
<include name="mats" />
</run>
</groups>
<packages>
<package name="com.tests" />
</packages>
</test>
</suite>
And each test class in the com.tests package has only one test method with varying group annotations. Will the beforeClass() and afterClass() methods of classes not in the "mats" group be executed?
Before/After methods not in the specified group(s) will not run unless they set alwaysRun to true.
alwaysRun
For before methods (beforeSuite, beforeTest, beforeTestClass and
beforeTestMethod, but not beforeGroups): If set to true, this
configuration method will be run regardless of what groups it belongs
to.
For after methods (afterSuite, afterClass, ...): If set to true, this
configuration method will be run even if one or more methods invoked
previously failed or was skipped.
e.g. Given the following classes:
public class AMatsTest {
#BeforeSuite(groups = {"mats"})
public void beforeSuite() {}
}
public class NotAMatsTest {
#BeforeSuite
public void beforeSuite() {}
}
#Test(groups = {"mats"})
public class AnotherMatsTest {
#BeforeSuite public void beforeSuite() {}
}
public class AlwaysTest {
#BeforeSuite(alwaysRun = true)
public void beforeSuite() {}
}
AMatsTest.beforeSuite(), AnotherMatsTest.beforeSuite(), and AlwaysTest.beforeSuite() will be executed.
NotAMatsTest.beforeSuite() will not be executed.
there are some flows that i want to test through automation. i am using selenium , maven, java and testNG. i have 2 different class. Let say class A and Class B.
public class A (){
#Test(groups="flow1",priority=0)
public void method a1()
{
}
#Test(groups="flow1".priority=2)
public void method a2()
{
}
#Test
public void method a3()
{
}
and 2nd class is class B
public class b (){
#Test(groups="flow1", priority=1)
public void method b1()
{
}
#Test
public void method b2()
{
}
#Test
public void method b3()
{
}
now i want to achieve flow like below
method a1()
method b1()
method a2()
i had try in this way through testng.xml
<test name="test1">
<groups>
<run>
<include name="flow1" />
</run>
</groups>
<classes>
<class name="a" />
<class name="b" />
</classes>
</test>
but i am not getting that output. it will run only one test cases and then it is skipping others.
i had try also some different way but i am not getting my goal.
can anybody help me
thanks
in your testng.xml just add group-by-instances="true"
<suite thread-count="2" verbose="10" name="testSuite" parallel="tests">
<test verbose="2" name="MytestCase" group-by-instances="true">
<classes>
<class name="com.A.classA" />
<class name="com.A.classB" />
</classes>
</test>
</suite>
i have a class Calc which implements two methods add(int a, int b) and div(int a, int b) and a test class of this class:
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
public class CalcTest {
Calc c;
#BeforeClass
public void init() {
c = new Calc();
}
#Test(groups = "t1")
public void addTest() {
System.out.println("Testing add() method");
Assert.assertEquals(c.add(10, 5), 15);
}
#Test
public void divTest() {
System.out.println("Testing div() method");
Assert.assertEquals(c.div(10, 5), 2, 0);
}
#AfterClass
public void free() {
c = null;
}
}
and i have a testing.xml file to suite tests:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="first tests">
<test name="first test">
<groups>
<run>
<include name="t1" />
</run>
</groups>
<classes>
<class name="CalcTest" />
</classes>
</test>
</suite>
I just had a first look at the groups in testng so i would like to try it, butif i run testing.xml file i'm getting nullPointerException at line:
Assert.assertEquals(c.add(10, 5), 15);
-if i remove the "groups" annotation from the test method it works fine, thanks
You need to keep your #BeforeClass annotation in the group. Add (groups = "t1") to your beforeclass annotation.
pretty solution, as there might be more groups in the future, would be:
#BeforeClass(alwaysRun = true)
public void init() {
c = new Calc();
}
This causes your BeforeClass to run always, no matter what group you are running.