We Keep Coding

Can a TestNG cross-browser test be executed with Cucumber runner?

I am working with Selenium Webdriver with Cucumber. My tests work as expected with that combination. In order to achieve cross-browser testing, I added TestNG framework. To verify that my cross-browser test was working good, I ran it with TestNG alone, without Cucumber. It ran perfectly in both Chrome and Firefox browsers.
public class WebTest {
WebDriver driver = null;
BasePageWeb basePage;
public String browser;
#Parameters({ "Browser" })
public WebTest(String browser) {
this.browser = browser;
}
#BeforeClass
public void navigateToUrl() {
switch (browser) {
case "CHROME":
WebDriverManager.chromedriver().setup();
driver = new ChromeDriver();
break;
case "FF":
WebDriverManager.firefoxdriver().setup();
driver = new FirefoxDriver();
break;
default:
driver = null;
break;
}
driver.get("https://demosite.executeautomation.com/Login.html");
}
#Test
public void loginToWebApp() {
basePage = new BasePageWeb(driver);
basePage.enterUsername("admin")
.enterPassword("admin")
.clickLoginButton();
driver.quit();
}
}
The testng.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests" thread-count="5">
<test name="Chrome Test">
<parameter name="Browser" value="CHROME"/>
<classes>
<class name="tests.web.WebTest"/>
</classes>
</test>
<test name="Firefox Test">
<parameter name="Browser" value="FF"/>
<classes>
<class name="tests.web.WebTest"/>
</classes>
</test>
</suite>
I needed to integrate the TestNG test with my Cucumber set-up so that I can run the whole test with Cucumber. To do this, I added cucumber-testng dependency to POM and created a Cucumber runner extending the AbstractCucumberTestNG class. I specified the location of my feature file and step definition. The step definition is mapped to the TestNG test.
Cucumber runner:
#CucumberOptions(
plugin = {"pretty", "html:target/surefire-reports/cucumber",
"json:target/surefire-reports/cucumberOriginal.json"},
glue = {"stepdefinitions"},
tags = "#web-1",
features = {"src/test/resources/features/web.feature"})
public class RunCucumberNGTest extends AbstractTestNGCucumberTests {
}
Step definition:
public class WebAppStepDefinitions {
private final WebTest webTest = new WebTest("CHROME"); //create an object of the class holding the testng test. If I change the argument to FF, the test will run only on Firefox
static boolean prevScenarioFailed = false;
#Before
public void setUp() {
if (prevScenarioFailed) {
throw new IllegalStateException("Previous scenario failed!");
}
}
#After()
public void stopExecutionAfterFailure(Scenario scenario) throws Exception {
prevScenarioFailed = scenario.isFailed();
}
#Given("^I have navigated to the web url \"([^\"]*)\"$")
public void navigateToUrl(String url) { test
webTest.navigateToUrl(url); //calling the first method holding the testng
}
#When("^I log into my web account with valid credentials as specicified in (.*) and (.*)$")
public void logintoWebApp(String username, String password) {
webTest.loginToWebApp(username, password); //calling the second method holding the testng
}
}
On running the class, the test got executed only in one browser (Chrome). Somehow, Firefox got lost in the build-up. I suspect that I am calling the parameterised TestNG method wrongly from another class. How do I do the call successfully?

For running TestNG tests with Cucumber you have to define Test Runner classes in testng.xml.
your Test Runner class is RunCucumberNGTest.
So the xml should look like:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests" thread-count="5">
<test name="Chrome Test">
<parameter name="Browser" value="CHROME"/>
<classes>
<class name="some.package.name.RunCucumberNGTest"/>
</classes>
</test>
<test name="Firefox Test">
<parameter name="Browser" value="FF"/>
<classes>
<class name="some.package.name.RunCucumberNGTest"/>
</classes>
</test>
</suite>
From this xml I see the next requirements:
Run the same set of tests but with different parameter value.
This should work in parallel, so this should be thread-safe.
1 Introduce TestNG Parameter for Test Runner class
#CucumberOptions(
plugin = {"pretty", "html:target/surefire-reports/cucumber",
"json:target/surefire-reports/cucumberOriginal.json"},
glue = {"stepdefinitions"},
tags = "#web-1",
features = {"src/test/resources/features/web.feature"})
public class RunCucumberNGTest extends AbstractTestNGCucumberTests {
// static thread-safe container to keep the browser value
public final static ThreadLocal<String> BROWSER = new ThreadLocal<>();
#BeforeTest
#Parameters({"Browser"})
public void defineBrowser(String browser) {
//put browser value to thread-safe container
RunCucumberNGTest.BROWSER.set(browser);
System.out.println(browser);
}
}
2 Use the value in Step Definition class
public class WebAppStepDefinitions {
private WebTest webTest;
static boolean prevScenarioFailed = false;
#Before
public void setUp() {
if (prevScenarioFailed) {
throw new IllegalStateException("Previous scenario failed!");
}
//get the browser value for current thread
String browser = RunCucumberNGTest.BROWSER.get();
System.out.println("WebAppStepDefinitions: " + browser);
//create an object of the class holding the testng test. If I change the argument to FF, the test will run only on Firefox
webTest = new WebTest(browser);
}
#After
public void stopExecutionAfterFailure(Scenario scenario) throws Exception {
prevScenarioFailed = scenario.isFailed();
}
#Given("^I have navigated to the web url \"([^\"]*)\"$")
public void navigateToUrl(String url) {
webTest.navigateToUrl(url); //calling the first method holding the testng
}
#When("^I log into my web account with valid credentials as specicified in (.*) and (.*)$")
public void logintoWebApp(String username, String password) {
webTest.loginToWebApp(username, password); //calling the second method holding the testng
}
}
NOTE: All TestNG annotations should be removed from WebTest class, they won't work and not required. WebTest used explicitly by WebAppStepDefinitions class, all the methods invoked explicitly and not by TestNG.
So, based on your initial requirements:
public class WebTest {
WebDriver driver = null;
BasePageWeb basePage;
public String browser;
public WebTest(String browser) {
this.browser = browser;
}
public void navigateToUrl(String url) {
switch (browser) {
case "CHROME":
WebDriverManager.chromedriver().setup();
driver = new ChromeDriver();
break;
case "FF":
WebDriverManager.firefoxdriver().setup();
driver = new FirefoxDriver();
break;
default:
driver = null;
break;
}
driver.get(url);
}
public void loginToWebApp(String username, String password) {
basePage = new BasePageWeb(driver);
basePage.enterUsername(username)
.enterPassword(password)
.clickLoginButton();
driver.quit();
}

How to avoid executing the method written in #AfterSuite if a single test is executed as 'TestNG Test' or using the 'Run' command provided by plugin

BaseTest Class
public class BaseTest extends Base {
LoginPage login;
Logger logger = Logger.getLogger(Base.class);
#BeforeMethod()
public void setUp() throws IOException, IOException {
logger.info("starting initialisation method from base class");
initialisation();
logger.info("completed initialisation method from base class");
login = new LoginPage();
logger.info("initialising login page and profile page object");
}
#AfterMethod
public void tearDown() {
logger.info("starting tear down method");
Base.getDriver().close();
logger.info("ending tear down method");
}
#AfterSuite
public void sendEmailWithExtentReport() throws IOException {
logger.info("starting sendEmailWithExtentReport");
Utilities.sendJavaMailAfterExecution();
logger.info("ending sendEmailWithExtentReport");
}
LoginTest class
public class LoginTest extends BaseTest {
#Test(priority = 0,description = "Test case to verify login button is present or not")
public void loginPageLoadingTest() throws IOException, InterruptedException {
logger.info("starting loginPageLoadingTest");
Thread.sleep(2000);
Assert.assertEquals(login.verifyLoginButton(), true);
logger.info("ending loginPageLoadingTest");
}
#Test(priority = 1, description = "Test case to verify login page title")
public void loginPageTitleTest() throws IOException, InterruptedException {
logger.info("starting loginPageTitleTest");
Thread.sleep(2000);
Assert.assertEquals(login.getLoginPageTitle(), "Console");
logger.info("ending loginPageTitleTest");
}
}
testng-console.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<listeners>
<listener class-name="com.analyzer.MyTransformer" />
<listener class-name="com.utilities.TestListener"></listener>
</listeners>
<test thread-count="5" name="Test">
<classes>
<class name="com.ui.tests.BaseTest" />
<class name="com.ui.tests.LoginTest" />
<class name="com.ui.tests.FilterTest" /> -->
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
I need to skip the method in #AfterSuite if i am only executing the single test case from multiple test cases in LoginTest class without using testng.xml file, so that I can verify each test case separately. Right now when individual test is executed the method in #AfterSuite is also getting executed. I want to execute the method in #AfterSuite only when test cases are executed using testng.xml file. How can i add a condition in #AfterSuite to achieve this. Also i am using this command '''mvn clean test -Dbrowsername=chrome''' from command prompt. My testng file name is testng-console.xml .The method in #AfterSuite should only get executed when using the tesng-console.xml. Please help.

You can test your suite name for equality to Default Suite. Say you have testng.xml like:
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="Suite1" verbose="1" >
<test name="MyTest" >
<classes>
<class name="click.webelement.Suites" />
</classes>
</test>
</suite>
where you define suite name explicitly. Then you can have the logic like this:
package click.webelement;
import org.testng.ITestContext;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.Test;
public class Suites {
#Test
public void doTest(){
System.out.println("Doing test..");
}
#AfterSuite
public void doingAfterSuite(ITestContext testContext){
if(!testContext.getSuite().getName().equalsIgnoreCase("Default Suite")){
System.out.println("Doing after suite");
}
}
}
Since when you run a single test, Default Suite is created you can just test if default suite is set in test context.
UPDATE: Eclipse plugin sets default suite name as Default suite so that I changed equals to EqualsIgnoreCase to cover both Idea and Eclipse

How to configure TestNG to run just a single test case in IntelliJ?

I have two separate packages in my project, one for integration tests and one for unit tests, my testng.xml looks as follows:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="All test cases" verbose="1" parallel="classes">
<test name="Integration Tests">
<classes>
<class name="com.sample.integration.ClassC"/>
<class name="com.sample.integration.ClassD"/>
</classes>
</test>
<test name="Unit tests">
<classes>
<class name="com.sample.unit.ClassA"/>
<class name="com.sample.unit.ClassB"/>
</classes>
</test>
</suite>
Class C:
public class ClassC {
#BeforeTest
public void beforeIntegrationTests() {
System.out.println("Before Integration tests");
}
#Test
public void classCMethod() {
System.out.println("Executing class C method");
}
}
Class D:
public class ClassD {
#Test
public void classDMethod() {
System.out.println("Executing class D method");
}
}
Class A:
public class ClassA {
#BeforeTest
public void beforeUnitTests() {
System.out.println("Before unit tests");
}
#Test
public void classAMethod() {
System.out.println("Executing class A method");
}
}
Class B:
public class ClassB {
#Test
public void classBMethod() {
System.out.println("Executing class B method");
}
}
If I run the entire test suite it works as expected as follows:
Before Integration tests
Executing class C method
Executing class D method
Before unit tests
Executing class A method
Executing class B method
However, if I try to either run/debug just classAMethod() from ClassA, it runs beforeUnitTests() [expected] and classAMethod() [expected], however it also runs beforeIntegrationTests() which is not expected. As per the official documentation: #BeforeTest: The annotated method will be run before any test method belonging to the classes inside the <test> tag is run.
How do I configure TestNG and/or IntelliJ to run this correctly?
Side Note: Although I can see that the beforeIntegrationTests() is getting run either by adding a breakpoint in the debug mode or by adding a Thread.sleep in the run mode, the output from this method does not get printed in final console output.

Firstly, is the expectation valid, as in I expect only beforeUnitTests() and classAMethod() to run if I run just the classAMethod().
No, all methods with #BeforeTest annotation will run before execution of method with #Test annotation.
Ideal way to handle this scenario is with groups.
Class A:
public class ClassA {
#BeforeTest(groups="unitTest")
public void beforeUnitTests() {
System.out.println("Before unit tests");
}
#Test(groups="unitTest")
public void classAMethod() {
System.out.println("Executing class A method");
}
}
Class B:
public class ClassB {
#Test(groups="unitTest")
public void classBMethod() {
System.out.println("Executing class B method");
}
}
Class C:
public class ClassC {
#BeforeTest(groups="integrationTest")
public void beforeIntegrationTests() {
System.out.println("Before Integration tests");
}
#Test(groups="integrationTest")
public void classCMethod() {
System.out.println("Executing class C method");
}
}
Class D:
public class ClassD {
#Test(groups="integrationTest")
public void classDMethod() {
System.out.println("Executing class D method");
}
}
testNG XML:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="All test cases" verbose="1" parallel="classes">
<groups>
<run>
<include name="unitTest"></include>
</run>
</groups>
<test name="Test Suite">
<classes>
<class name="com.sample.integration.ClassC"/>
<class name="com.sample.integration.ClassD"/>
<class name="com.sample.unit.ClassA"/>
<class name="com.sample.unit.ClassB"/>
</classes>
</test>
</suite>
You can make further configuration to this testNg XML as per your needs.

Configuration of Parallel running at Class level in TestNG

I am trying to find out how to run all my TestNG tests in one class first, then all in the second class second and so forth. I need to use parallel running to speed execution though. These are selenium tests so can be slow running.
Given the following TestNG suite file:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Suite1" verbose="1" parallel="classes" thread-count="2">
<test name="parallel-running-test">
<classes>
<class name="com.mycompany.myproject.mypackage.MyFirstClassTest"/>
<class name="com.mycompany.myproject.mypackage.MySecondClassTest"/>
</classes>
</test>
..and the following 2 classes:
Class one:
public class MyFirstClassTest {
#Test
public void myFirstClassTestOne(){
System.out.println("myFirstClassTestOne");
}
#Test (dependsOnMethods = {"myFirstClassTestOne"})
public void myFirstClassTestTwo(){
System.out.println("myFirstClassTestTwo");
}
#Test (dependsOnMethods = {"myFirstClassTestTwo"})
public void myFirstClassTestThree(){
System.out.println("myFirstClassTestThree");
}
#Test(dependsOnMethods = {"myFirstClassTestThree"})
public void myFirstClassTestFour(){
System.out.println("myFirstClassTestFour");
}
#Test(dependsOnMethods = {"myFirstClassTestFour"})
public void myFirstClassTestFive(){
System.out.println("myFirstClassTestFive");
}
}
Class two:
public class MySecondClassTest {
#Test
public void mySecondClassTestOne(){
System.out.println("mySecondClassTestOne");
}
#Test(dependsOnMethods = {"mySecondClassTestOne"})
public void mySecondClassTestTwo(){
System.out.println("mySecondClassTestTwo");
}
#Test(dependsOnMethods = {"mySecondClassTestTwo"})
public void mySecondClassTestThree(){
System.out.println("mySecondClassTestThree");
}
#Test(dependsOnMethods = {"mySecondClassTestThree"})
public void mySecondClassTestFour(){
System.out.println("mySecondClassTestFour");
}
#Test(dependsOnMethods = {"mySecondClassTestFour"})
public void mySecondClassTestFive(){
System.out.println("mySecondClassTestFive");
}
}
Then the output is as follows:
...How can I have the report as:
MyFirstClassTest
myfirstClassTestOne
myfirstClassTestTwo
myfirstClassTestThree
myfirstClassTestFour
myfirstClassTestFive
MySecondClassTest
mySecondClassTestOne
mySecondClassTestTwo
mySecondClassTestThree
mySecondClassTestFour
mySecondClassTestFive
NB - I need dependsOnMethods and have obvs removed all the browser stuff and actual selenium stuff

Your issue is just the way how IntelliJ is displaying the result of tests.
By default, it displays tests by the order they finish.
You can change it and sort them by alphabetical order if you prefer:
Sadly, IntelliJ is not grouping tests by class name (or I didn't find the way to do it).
Maybe you should ask for the feature on https://youtrack.jetbrains.com

Steerpike, you can try running your tests in parallel (instead of classes) so slight changes to your XML should give you what you are after:
!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Suite1" verbose="1" parallel="tests" thread-count="2">
<test name="parallel-running-test1">
<classes>
<class name="com.mycompany.myproject.mypackage.MyFirstClassTest"/>
</classes>
</test>
<test name="parallel-running-test2">
<classes>
<class name="com.mycompany.myproject.mypackage.MySecondClassTest"/>
</classes>
</test>
</suite>
Best of luck!

NullPointerException in Selenium WebDriver Instance When Using in TestNG

I have created a Selenium test suite using TestNG for my website.
The name of my project is Test My Website. In order to execute my Selenium test script and create a test report, I execute the TestNG.xml file directly from the command prompt. For different modules of my website, I have created different Java classes for different modules and have kept them in one package. The source code of my files is given as follows:
TestNG.xml
<?xml version="1.0" encoding="UTF-8"?>
<suite name="Test My Website">
<test name="My Website Test">
<packages>
<package name="testmywebsite" />
</packages>
</test>
</suite>
TestLogin.java
public class TestLogin {
public static WebDriver driver;
#BeforeTest
public void setup() {
driver = new ChromeDriver();
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
driver.manage().window().maximize();
}
#Test
void loginTest() {
//Code to perform the login test
}
#AfterClass
public void setupWorkHistory() {
TestModule1.driver = driver;
}
}
TestModule1.java
public class TestModule1 {
public static WebDriver driver;
#Test
void module1Test() {
//Code to perform the module 1 test
driver.getTitle();
}
#AfterClass
public void setupModule2() {
TestModule2.driver = driver;
}
}
TestModule2.java
public class TestModule2 {
public static WebDriver driver;
#Test
void module2Test() {
//Code to perform the module 2 test
driver.getTitle();
}
#AfterTest
public void tearDown() {
driver.close();
driver.quit();
}
}
Note that in the first two Java classes I have added the setup<Next Class's name>() method to pass my driver instance.
The problem here is that the driver instance gets successfully passed from TestLogin.java to TestModule1.java. However it throws a NullPointerException in the module2Test() method and hence it shows a failed TestNG report for module2Test() despite creating the setupModule2() method in TestModule1.java and adding the #AfterClass annotation to it.
Can anyone tell me why exactly is this happening here? Replies at the earliest will be highly appreciated. Thank you.

Okay I found a solution to the problem myself.
For that I edited the source code of my TestNG.xml file and my Java class files. They are given as follows:
TestNG.xml
<?xml version="1.0" encoding="UTF-8"?>
<suite name="Test RockON">
<test name="Rockon Test" allow-return-values="true">
<classes>
<class name="testmywebsite.TestLogin" />
<class name="testmywebsite.TestModule1" />
<class name="testmywebsite.TestModule2" />
</classes>
</test>
</suite>
TestLogin.java
public class TestLogin {
public static WebDriver driver;
#BeforeTest
public void setup() {
driver = new ChromeDriver();
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
driver.manage().window().maximize();
}
#Test
void loginTest() {
//Code to perform the login test
}
public static WebDriver getDriver() {
return driver;
}
}
TestModule1.java
public class TestModule1 {
public static WebDriver driver;
#BeforeClass
public void setup() {
driver = TestLogin.getDriver();
}
#Test
void module1Test() {
//Code to perform the module 1 test
}
public static WebDriver getDriver() {
return driver;
}
}
Module2.java
public class TestModule2 {
public static WebDriver driver;
#BeforeClass
public void setup() {
driver = TestModule1.getDriver();
}
#Test
void module2Test() {
//Code to perform the module 2 test
}
public static WebDriver getDriver() {
return driver;
}
}
What I did is called the classes individually from my XML file and added the allow-return-values="true" attribute to my tests, which allows the classes to allow return types. In my Java classes I instantiated my WebDriver instance in every class in a #BeforeClass method. Hence I was able to prevent my WebDriver instance from throwing a NullPointerException.