I am trying to run Selenium tests with testNG and have run into a problem when using dependsOnMethods and class factory.
I have a series of test methods in a class which depend on each other in one chain.
public class MyTest {
private String factoryParam1;
private String factoryParam2;
public MyTest(String factoryParam1, String factoryParam2){...do stuff}
#Test
public void test1(){...do Stuff}
#Test(dependsOnMethods={"test1"})
public void test2(){...do stuff}
#Test(dependsOnMethods={"test2"})
public void test3(){...do stuff}
}
I have a factory class like this.
public class MyTestFactory{
#Factory(dataProvider = "argList")
public Object[] createInstances(String arg1, String arg2){
return new Objcect[] {new MyTest(arg1, arg2)};
}
#DataProvider(name="argList")
static public Object[][] argList(){
return new Object[][] {
{"instance1arg1", "instance1arg2"},
{"instance2arg1", "instance2arg2"},
}
}
}
And, finally an xml like this.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0dtd">
<suite name="My test suite">
<test name="My test" group-by-instances="true">
<classes>
<class name ="test.testfactory.MyTestFactory"></class>
</classes>
</test>
</suite>
The first time the class is run with first set of data, everything runs in the correct order. On the second run however, with the second set of data, the test methods run out of order. I assume this is because the methods have already been marked as 'successful' in the first run, so all method dependencies are satisfied for each subsequent run. For me, the desired behavior is for the order to maintained on each run of the test class. How can I achieve this?
I have searched for this but I think I'm not using the correct terminology or something because I haven't found anything.
Related
I have two separate packages in my project, one for integration tests and one for unit tests, my testng.xml looks as follows:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="All test cases" verbose="1" parallel="classes">
<test name="Integration Tests">
<classes>
<class name="com.sample.integration.ClassC"/>
<class name="com.sample.integration.ClassD"/>
</classes>
</test>
<test name="Unit tests">
<classes>
<class name="com.sample.unit.ClassA"/>
<class name="com.sample.unit.ClassB"/>
</classes>
</test>
</suite>
Class C:
public class ClassC {
#BeforeTest
public void beforeIntegrationTests() {
System.out.println("Before Integration tests");
}
#Test
public void classCMethod() {
System.out.println("Executing class C method");
}
}
Class D:
public class ClassD {
#Test
public void classDMethod() {
System.out.println("Executing class D method");
}
}
Class A:
public class ClassA {
#BeforeTest
public void beforeUnitTests() {
System.out.println("Before unit tests");
}
#Test
public void classAMethod() {
System.out.println("Executing class A method");
}
}
Class B:
public class ClassB {
#Test
public void classBMethod() {
System.out.println("Executing class B method");
}
}
If I run the entire test suite it works as expected as follows:
Before Integration tests
Executing class C method
Executing class D method
Before unit tests
Executing class A method
Executing class B method
However, if I try to either run/debug just classAMethod() from ClassA, it runs beforeUnitTests() [expected] and classAMethod() [expected], however it also runs beforeIntegrationTests() which is not expected. As per the official documentation: #BeforeTest: The annotated method will be run before any test method belonging to the classes inside the <test> tag is run.
How do I configure TestNG and/or IntelliJ to run this correctly?
Side Note: Although I can see that the beforeIntegrationTests() is getting run either by adding a breakpoint in the debug mode or by adding a Thread.sleep in the run mode, the output from this method does not get printed in final console output.
Firstly, is the expectation valid, as in I expect only beforeUnitTests() and classAMethod() to run if I run just the classAMethod().
No, all methods with #BeforeTest annotation will run before execution of method with #Test annotation.
Ideal way to handle this scenario is with groups.
Class A:
public class ClassA {
#BeforeTest(groups="unitTest")
public void beforeUnitTests() {
System.out.println("Before unit tests");
}
#Test(groups="unitTest")
public void classAMethod() {
System.out.println("Executing class A method");
}
}
Class B:
public class ClassB {
#Test(groups="unitTest")
public void classBMethod() {
System.out.println("Executing class B method");
}
}
Class C:
public class ClassC {
#BeforeTest(groups="integrationTest")
public void beforeIntegrationTests() {
System.out.println("Before Integration tests");
}
#Test(groups="integrationTest")
public void classCMethod() {
System.out.println("Executing class C method");
}
}
Class D:
public class ClassD {
#Test(groups="integrationTest")
public void classDMethod() {
System.out.println("Executing class D method");
}
}
testNG XML:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="All test cases" verbose="1" parallel="classes">
<groups>
<run>
<include name="unitTest"></include>
</run>
</groups>
<test name="Test Suite">
<classes>
<class name="com.sample.integration.ClassC"/>
<class name="com.sample.integration.ClassD"/>
<class name="com.sample.unit.ClassA"/>
<class name="com.sample.unit.ClassB"/>
</classes>
</test>
</suite>
You can make further configuration to this testNg XML as per your needs.
I have the below #AfterSuit method. If I remove alwaysRun=true, TestNG is not executing it.
#AfterSuite(alwaysRun=true)
public void finishsuit()
{
System.out.println("completed suit");
}
I checked the documentation for alwaysRun and it says:
For after methods (afterSuite, afterClass, ...): If set to true, this
configuration method will be run even if one or more methods invoked
previously failed or was skipped.
This is what i understood : If any of the #Test method defined inside any of the classes specified in any of the <test> belonging to the current suit fails, then #AfterSuit wont be executed unless we specify alwaysRun=true.
But in my case none of the #Test are failing. And still my #AfterSuite is not executing without alwaysRun=true
You are probably running your test methods in groups and AfterSuite is not part of any group, that's why you need the alwaysRun=true.
But because I see you are a little bit confused about alwaysRun behavior, I'll proceed to explain...
For #BeforeXXXX methods, alwaysRun needs to be assigned if you want to run the method despite it doesn't belong to the group you are running.
Eg.
public class TestConfig {
#BeforeSuite
public void beforeSuite() {
doSomething();
}
}
public class TestClassA extends TestConfig {
#Test(groups = { "groupA" })
public void testA() {
doTest();
}
}
In this case, if we run group 'groupA', the #BeforeSuite method is not part of the group we are executing, unless we specify alwaysRun = true.
The same applies for #AfterXXXX methods, plus they need alwaysRun=true if you want to run them despite their #Before counterpart has failed or any depending methods. For example:
public class TestClassA {
#BeforeSuite( groups = { "testA" })
public void beforeSuite() {
doSomethingBefore();
throw new RuntimeException();
}
#Test( groups = { "testA" })
public void testA() {
doTest();
}
#AfterSuite( groups = { "testA" })
public void afterSuite() {
doSomethingAfter();
}
}
Now, #AfterSuite method won't run because it's counterpart #BeforeSuite has failed. the same will apply if #AfterSuite is on a class where a #BeforeClass had failed. But if we add alwaysRun=true testng will ignore any previous failure and proceed anyway.
NONE OF THIS APPLIES TO #TEST METHODS. #BeforeXXXX and #AfterXXXX methods will run ignoring any #Test results UNLESS you specify a test dependency.
Finally, alwaysRun in #Test methods will act in similar way as ignoreMissingDependencies=true. For example:
#Test
public void testA() {
doTestA();
}
#Test(dependsOnMethods = { testA }))
public void testB() {
doTestB();
}
Again, in this case testB won't run because it depends on #TestA, unless we assign alwaysRun=true (or ignoreMissingDependencies=true).
If you run it as a TestNG class, only #BeforeSuite method get picked even though #AfterSuite(alwaysRun=true)
However if you run it as a Testsuite via testplan.xml ,#AfterSuite method get picked directly and no need to have alwaysRun=true
Both BeforeSuite and AfterSuite, are work in optimal way if you run it as a test suite via xml rather than run it directly as a TestNG class. Since those are meant to be suite level annotations
Example - If I run below code as a class , the output will don't have the afterSuite method.
public class TestngExamples {
#BeforeSuite
public void initation() {
System.out.println("Before suite is ran ");
}
#Test
public void hello() {
System.out.println("Hello world method");
}
#Test
public void demo() {
System.out.println("demo method");
}
#AfterSuite
public void tearDown() {
System.out.println("After suite is ran ");
}
}
However If I run it as a Test suite xml it will print the Aftersuite method
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test thread-count="5" name="Test">
<classes>
<class name="com.webdriver.blogs.TestngExamples"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Sample output-
Before suite is ran
demo method
Hello world method
After suite is ran
For after methods (afterSuite, afterClass, ...): If set to true, this configuration method will be run even if one or more methods invoked previously failed or was skipped.
Methods here means all the #test methods and the all the other annotated methods aswell. in my case i was using invalid arguments to my #AfterTest function, causing it to fail. That was why my #BeforeTest was not working without alwaysRun
Say I have the following XML:
<suite name="MATS">
<test name="mats_test">
<groups>
<run>
<include name="mats" />
</run>
</groups>
<packages>
<package name="com.tests" />
</packages>
</test>
</suite>
And each test class in the com.tests package has only one test method with varying group annotations. Will the beforeClass() and afterClass() methods of classes not in the "mats" group be executed?
Before/After methods not in the specified group(s) will not run unless they set alwaysRun to true.
alwaysRun
For before methods (beforeSuite, beforeTest, beforeTestClass and
beforeTestMethod, but not beforeGroups): If set to true, this
configuration method will be run regardless of what groups it belongs
to.
For after methods (afterSuite, afterClass, ...): If set to true, this
configuration method will be run even if one or more methods invoked
previously failed or was skipped.
e.g. Given the following classes:
public class AMatsTest {
#BeforeSuite(groups = {"mats"})
public void beforeSuite() {}
}
public class NotAMatsTest {
#BeforeSuite
public void beforeSuite() {}
}
#Test(groups = {"mats"})
public class AnotherMatsTest {
#BeforeSuite public void beforeSuite() {}
}
public class AlwaysTest {
#BeforeSuite(alwaysRun = true)
public void beforeSuite() {}
}
AMatsTest.beforeSuite(), AnotherMatsTest.beforeSuite(), and AlwaysTest.beforeSuite() will be executed.
NotAMatsTest.beforeSuite() will not be executed.
Hi I have three class files. In first class, only one test case is used for login Gmail account. rest of the classes having only one test case and both are used for navigating Inbox and sent items. I have created testng.xml to run these multiple classes. Now the problem is when first class is being executed, I could be log in Gmail account. But when it comes to second class, I am unable to navigate to Inbox as I did not perform login operation in class 2 and class 3.
So it thrown me an error Null Pointer Exception. I know the reason why I am getting this exception. But I want to perform login action only one time but it should be available to all classes which means When I am going to execute class 2 and class 3, it should not throw error. It should continue from the class 1. How I can achieve this task.
I googled a lot. None of them helped me. Please suggest me any idea if you have. I have tried to extend my class also. I got no result. Please guide me where am I struggling?
Following is my testng.xml
<suite name="MynaTestCase">
<test name="order">
<classes>
<class name="myPackage.Login" />
<class name="myPackage.Inbox" />
<class name="myPackage.SentItems" />
</classes>
</test>
</suite>
Here is the Login Code
#BeforeSuite(alwaysRun = true)
public void setup(){
System.setProperty("webdriver.chrome.driver", "/home/vadmin/workspace/samplepjt/lib/chromedriver");
driver = new ChromeDriver();
baseUrl = "http://www.google.com";
driver.get(baseUrl+"/");
driver.manage().window().maximize();
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
System.out.println("Before suite executed");
}
#Test
public void LoginTest(){
driver.findElement(By.cssSelector("i.fa.fa-user-secret")).click();
driver.findElement(By.id("user_username")).clear();
driver.findElement(By.id("user_username")).sendKeys(username);
driver.findElement(By.id("user_password")).clear();
driver.findElement(By.id("user_password")).sendKeys(password);
driver.findElement(By.id("signInButton")).click();
}
Here is the Inbox.Java
#Test
public void testcase1()
{
driver.findElement(By.xpath("html/body/div[1]/div[2]/div/div/div/div[1]/div[4]/ul/li[8]/a/div/p")).click();
Assert.assertEquals(driver.findElement(By.xpath("html/body/div[1]/div[2]/div/div/div/div[2]/div/h4")).getText(), "Sections");
driver.findElement(By.name("commit")).click();
driver.findElement(By.xpath(".//*[#id='new_section']/div[3]/input")).submit();
}
The problem is that you defined your WebDriver in the Login class, so the Inbox class won't see it.
To solve it, create an abstract class, which include the BeforeSuite and AfterSuite classes and extend every test class with it.
Abstract class
public class MyAbstractClass
{
public WebDriver driver;
#BeforeSuite
public void beforeSuite() {
//...
this.driver = new ChromeDriver();
//...
}
#AfterSuite
public void afterSuite() {
//...
this.driver.quit();
//...
}
}
A Test Class
public class MyTestClass extends MyAbstractClass
{
#Test
public void myTest() {
this.driver.get("http://google.com");
}
}
You'll want to set #Test(priority = 1) for every test.
I have a scenario like two different test classes, second test class result depends on the first one
public class class1
{
#test
public void sometest1()
{
// some functionality
}
}
public class class2
{
#test
public void sometest2()
{
// some different functionality
}
}
Here class2 need to be executed if class1 result is pass else class2 needs to be failed.(if no provision for the same i need to execute the test1 in class1 to be executed always first in selenium grid parallel run)
As said avoid please refer :http://testng.org/doc/documentation-main.html#annotations.
As per above comments, i hope you are trying use depends on for method which is in another class. In such a case you need to provide complete path ie package.class.method.
class1:
public class testingcode {
#Test
public void test1(){
System.out.println("test1");
}
}
Class2:
public class testcase1{
#Test(dependsOnMethods={"com.test.testingcode.test1"})
public void testcase1result() throws InterruptedException{
System.out.println("test");
}
}
Run from TestNG.xml file
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="none" preserve-order="true">
<test name="Test">
<classes>
<class name="com.test.testingcode"/>
<class name="com.test.testcase1"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Let me know, if it is not working..
Thank You,
Murali
There are some annotations in testng that allows you the flow that you want.
I have never worked on testng but I think you can find the suitable annotation here :http://testng.org/doc/documentation-main.html#annotations
might be dependsOnGroups help you
You can use groups and dependency in testng to run a test after some test have executed. There is no way(atleast to my knowledge) to fail the Class2 test if Class1 test failed.
public class Class1Test {
#Test(groups="RunFirst")
public void sometest1() {
Assert.fail("just to skip the other test");
}
}
public class Class2Test {
#Test(dependsOnGroups="RunFirst")
public void sometest2() {
System.out.println("I am running");
}
}
Here the Class2Test will run after Class1Test has run and will be skipped if Class1Test failed.
You can use dependsOnMethods like below
#Test
public void test1() {
// Instantiate another case class
// Call your testcase
}
#Test(dependsOnMethods="test1")
public void test2() {
}