Whenever we run our test suite in parallel as "tests" with the TestNG xml file, it opens both instances of a chrome driver, but intermediately executes both cucumber features in the same window of chrome.
Gives us some result like this:
Searches two times in the search bar
This are Maven dependencies we have:
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-java</artifactId>
<version>1.2.5</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-jvm-deps</artifactId>
<version>1.0.5</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-testng</artifactId>
<version>1.2.5</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
We use a test runner for each test. All test runners are basically the same. Here is a test runner used:
package bdxReport.biAdsDashboard.AdvertisingPerformance.Content;
import cucumber.api.CucumberOptions;
import org.testng.annotations.Test;
#CucumberOptions(
features = "src/test/resources/FeaturesAdsDashboard/FeaturesAdvertisingPerformance/Content/CheckContentAdvertisingByProduct.feature",
glue = {"stepDefinitions"},
format = {
"pretty",
"html:target/cucumber-reports/AdsDashboard/TestRunnerCheckContentAdvertisingByProduct-Reports",
"json:target/cucumber-reports/AdsDashboard/TestRunnerCheckContentAdvertisingByProductReport.json",
"rerun:target/cucumber-reports/AdsDashboard/TestRunnerCheckContentAdvertisingByProduct-Reports/rerun.txt"
})
#Test
public class TestRunnerCheckContentAdvertisingByProduct {
private TestNGCucumberRunner testNGCucumberRunner;
#BeforeClass(alwaysRun = true)
public void setUpClass() throws Exception {
testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());
}
#Test(groups = "cucumber", description = "Runs Cucumber Feature", dataProvider = "features")
public void feature(CucumberFeatureWrapper cucumberFeature) {
testNGCucumberRunner.runCucumber(cucumberFeature.getCucumberFeature());
}
#DataProvider
public Object[][] features() {
return testNGCucumberRunner.provideFeatures();
}
#AfterClass(alwaysRun = true)
public void tearDownClass() throws Exception {
testNGCucumberRunner.finish();
}
}
And this is the TestNG xml suite:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="BDX Executive Summary Advertising Performance" parallel="tests" thread-count="20" preserve-order="true">
<listeners>
<listener class-name="common.testcases.TestCaseListener" />
<listener class-name="common.testcases.CaptureScreenshotOnFailureListenerBDX"/>
</listeners>
<test name="01: Check Advertising Performance Section Data">
<classes>
<class name="bdxReport.biExecutiveSummary.AdvertisingPerformance.Data.TestRunnerAdvertisingSectionData" />
</classes>
</test>
<test name="02: Check Advertising Performance Section Content">
<classes>
<class name="bdxReport.biExecutiveSummary.AdvertisingPerformance.Content.TestRunnerAdvertisingSectionContent" />
</classes>
</test>
</suite>
We have done a lot of research on what could be causing this behavior but until now we haven't been able to determine whats causing this behavior
Creating a separate runner file for every feature does not make sense to me. Have you tried "cucumber-jvm-parallel-plugin" to run the features. Please check following answer:
How to execute cucumber feature file parallel
Further, as per my experience this is the issue with the driver you are instantiating either it is static or it is not being managed properly. Firstly, try above link, in the meantime let me implement parallel execution in a fresh automation framework and I will paste the code here
To take maximum advantage of TestNG you should use Testng's QAF framework. It supports multiple bdd syntax including gherkin using GherkinFactory.
QAF considers each scenario as TestNG test and Scenario Outline as TestNG data-driven test. As qaf provides driver management and resource management in-built, you don't need to write single line of code for driver management or resource management. All you need to do is create TestNG xml configuration file as per your requirement either to run parallel methods (scenarios) or groups or xml test on one or more browser.
Below example will run scenarios in parallel.
<test name="Gherkin-QAF-Test" parallel="methods">
<parameter name="step.provider.pkg" value="com.qmetry.qaf.automation.impl.step.qaf" />
<parameter name="scenario.file.loc" value="resources/features" />
<classes>
<class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
</classes>
</test>
It enables different possible configuration combinations. Here is another example where it will run scenarios in two browsers and in parallel. You can configure number of threads for each browser as standard TestNG xml configuration.
<suite name="AUT Test Automation" verbose="0" parallel="tests">
<test name="Test-on-chrome" parallel="methods">
<parameter name="step.provider.pkg" value="com.qmetry.qaf.automation.impl.step.qaf" />
<parameter name="scenario.file.loc" value="resources/features" />
<parameter name="driver.name" value="chromeDriver" />
<classes>
<class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
</classes>
</test>
<test name="Test FF" parallel="methods">
<parameter name="step.provider.pkg" value="com.qmetry.qaf.automation.impl.step.qaf" />
<parameter name="scenario.file.loc" value="resources/features" />
<parameter name="driver.name" value="firefoxDriver" />
<classes>
<class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
</classes>
</test>
</suite>
Related
I have this TestNG part :
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
<listeners>
<listener class-name="ShufersalMiron.Sprod.Listeners"/>
</listeners>
<test name="Test1">
<classes>
<class name="ShufersalMiron.Sprod.PA_Coupons"/>
</classes>
</test> <!-- Test -->
<test name="Test2">
<classes>
<class name="ShufersalMiron.Sprod.PA_SupermeCoins"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
When I am trying to run it as a suite I get this error :
Can someone please tell me what could be wrong and how to solve it please ?
Thanks.
The error might be due to conflicting dependencies.
extentreports-java version 5.0.9 (latest for now)
https://github.com/extent-framework/extentreports-java/blob/master/pom.xml
uses io.reactivex.rxjava3:rxjava-3.0.4 and org.testng:testng-7.3.0.
I can suggest 2 options:
Try to add io.reactivex.rxjava3 dependency, since
NoClassDefFound error references this.
Maven:
<dependency>
<groupId>io.reactivex.rxjava3</groupId>
<artifactId>rxjava</artifactId>
<version>3.0.4</version>
</dependency>
Gradle:
implementation 'io.reactivex.rxjava3:rxjava:3.0.4'
Try to use testNG 7.3.0 instead of 7.4.0.
I have a testng xml file that is setup to run a specific class that has methods containing the following group tags or are grouped as one of the following:
parallel-test
sequential-test
smoke-test
Although some of my tests are designated as either parallel or sequential, some of the test may contain an additional tag 'smoke-test.' In this scenario, I want to be able to just run those that are grouped as 'smoke-test.' When I run the tests, it either cannot find any tests or it just runs all tests grouped as 'sequential-test.' I cannot seem to restrict the test run from this xml file to just those tests grouped as 'smoke-test.' In the same class, I have multiple testNG methods that have either the 'parallel-test' group or the 'sequential-test' group. What it looks like the TestNG xml file is doing is just ignoring the 'parallel-test' group. Thats great, but what if there is Testng method that contains the group 'parallel-test' and 'smoke-test?' I want to run every test regardless of whether it is for parallel testing or sequential, but ONLY those that have the additional group tag 'smoke-test.' My environment is the following:
IntelliJ
Maven
TestNG
Java
Please help. See sample test method in the WebTest.Test class:
#Test(groups = {"sequential-test","smoke-test"},description = "Test will validate something")
public void RunTest()
{
//do something
}
Here is the sample xml file below
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!--This is a template to be modified for QA environment tests-->
<suite name="Smoke Test">
<listeners>
<listener class-name="Listeners.Listeners"/>
<listener class-name="Listeners.ExtentListeners"/>
</listeners>
<parameter name="environment" value="QA"></parameter>
<test verbose="2" name="Sequential tests" parallel="methods" thread-count="1">
<!--<groups>
<run>
<exclude name="parallel-test"/>
</run>
</groups>-->
<classes>
<class name="WebTests.Test">
<methods>
<include name="smoke-test"/>
</methods>
</class>
</classes>
</test>
</suite>
Below XML Suite should run the all test method having group 'smoke-test'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!--This is a template to be modified for QA environment tests-->
<suite name="Smoke Test">
<listeners>
<listener class-name="Listeners.Listeners"/>
<listener class-name="Listeners.ExtentListeners"/>
</listeners>
<parameter name="environment" value="QA"></parameter>
<test verbose="2" name="Sequential tests" parallel="methods" thread-count="1">
<groups>
<run>
<include name="smoke-test"/>
</run>
</groups>
<classes>
<class name="WebTests.Test"/>
</classes>
</test>
</suite>
i've setup my xml so it will run groups which depends-on different group in order to run.
when the different group fails, i would expect that the groups won't run, but it is still running.
here is my code:
<test name="Test1">
<parameter name="testedApp" value="someValue"/>
<groups>
<run>
**<include name="Test1"/>**
<exclude name="TestX"/>
<exclude name="TextY"/>
</run>
<dependencies>
<!-- Service Health Check -->
**<group name="Test1" depends-on="ServiceHealthCheck"/>**
</dependencies>
</groups>
<packages>
<package name="project.auto.tests.api...*"/>(package)
</packages>
<classes>
<!-- Service Health Check -->
<class name=".....PbServiceHealthCheckTest"/>(class)
</classes>
</test>
Answer: TestNG 6.14.3 has a bug which prevenr the "depends-on" to work correctly.
Upgrade to TestNG 7.1.0 solved the issue.
I am using Extent Report Version 4.0.9 to generate report for two of my tests. I am working on selenium, java, testng, cucumber, maven project. I have two TestRunner files in my cucumberOptions which I run one after the other using POM.xml.
When I run only one TestRunner, the Extent Report gets generated as expected. But when I run both TestRunners one after the other, the Extent Report which is generated by the second test overwrites the one generated by the first one.
How do I append the ExtentReport generated by the second test to that generated by the first test?
Here is my ExtentReporterListener code:
public class ExtentReporterListener {
private static Logger log = LogManager.getLogger(ExceptionHandler.class);
private static final String TEST_OUTPUT = TestConfiguration.USER_DIR + "\\test-output\\Screenshots";
public static ExtentHtmlReporter report = null;
public static ExtentReports extent = null;
public static ExtentTest feature = null;
public static ExtentTest scenario = null;
public static ExtentReports setUp() {
String reportLocation = "./Reports/Extent_Report.html";
report = new ExtentHtmlReporter(reportLocation);
report.config().setDocumentTitle("Word press Automation Test Report");
report.config().setReportName("Word Press Automation Test Report");
report.config().setTheme(Theme.STANDARD);
log.info("Extent Report location initialized...");
report.start();
extent = new ExtentReports();
extent.attachReporter(report);
extent.setSystemInfo("Application", "Word Press");
return extent;
}
}
This is my testng.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
<listeners>
<listener class-name = "listener-class" />
</listeners>
<test thread-count="5" name="frontEnd">
<parameter name="URL" value="front-end-url" />
<classes>
<class name="frontendTestRunner"/>
</classes>
</test>
<test thread-count="5" name="Backend">
<parameter name="URL" value="back-end-url" />
<classes>
<class name="backendtestrunner"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
I am using Extent Report version 4. Somehow setAppendExisting(true); method which was in older version doesnot seem to exist in version 4.
After the below line of code please make the append existing as true.
report = new ExtentHtmlReporter(reportLocation);
report.setAppendExisting(true);
This will append the tests after execution.
I could solve the problem of ExtentReport getting overwritting when I changed my testng.xml like the following:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
<listeners>
<listener class-name = "listener-class" />
</listeners>
<test thread-count="5" name="frontEnd">
<classes>
<class name="frontendTestRunner">
<parameter name="URL" value="front-end-url" />
</class>
<class name="backendtestrunner">
<parameter name="URL" value="back-end-url" />
</class>
</classes>
</test>
</suite> <!-- Suite -->
Currently I have a GitHub Repo that contains 4 Selenium/Maven Tests. I am able to run each test individually on my local machine via mvn test
I have Jenkins running also on my local machine and created a maven project to pull my GitHub Repository, select the pom.xml in one of the test (P_ProfileChangeMavenTest) and then execute a test as my maven goal.
When Jenkins runs my test, it begins to go though my maven project but is not pulling up Firefox browser to interact with my test.
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.test.profilechange</groupId>
<artifactId>Profile</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Profile</name>
<description>PRofile</description>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.39.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire</artifactId>
<version>2.16</version>
<type>pom</type>
</dependency>
</dependencies>
</project>
I am fairly new to Automation and am trying to verify that everything works correctly on my local machine first before sending these tests to run on a server with a selenium grid.
All the help would be greatly appreciated since I need to get this working ASAP.
Thanks in advanced!
What I have found works for us is using testNg to run the tests. We create an XML file with the different tests that we want run and we use
mvn test -D testng=testname.xml
TestNG is a great framework for integration style tests whereas the J Unit testing framework is more designed for unit tests.
A sample of the xml file would be something like this.
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name=" Report - Create New Report and Add Fields By Double Click and Filter by Right Clicking" parallel="none" verbose="1">
<test verbose="1" name="is site available" preserve-order="true">
<classes>
<class name="com.somecompany.qa.seleniumautomation.PucDD">
<methods>
<include name="siteAvailable" />
</methods>
</class>
</classes>
</test>
<test verbose="1" name="Login as 'admin' 'password'" preserve-order="true">
<classes>
<class name="com.somecompany.qa.seleniumautomation.PucDD">
<methods>
<parameter name="login.user" value="admin" />
<parameter name="login.password" value="password" />
<include name="login" />
</methods>
</class>
</classes>
</test>
<test verbose="1" name="Create a new Report based on the provided data source" preserve-order="true">
<classes>
<class name="com.somecompany.qa.seleniumautomation.Reports">
<methods>
<parameter name="data.source.name" value="Orders" />
<include name="createNewReport"/>
</methods>
</class>
</classes>
</test>
<test verbose="1" name="Add the passed fields to the report by double clicking and add the passed filters by right clicking the field." preserve-order="true">
<classes>
<class name="com.company.qa.seleniumautomation.Reports">
<methods>
<parameter name="login.user" value="admin" />
<parameter name="login.password" value="password" />
<parameter name="fields" value="Customer Number,Customer Name,Order Date,Status,Shipped Date" />
<parameter name="filterCriteria" value="Customer Number:lte=300,Customer Number:gte=150,Product Lines:contains=a" />
<include name="addFilters"/>
</methods>
</class>
</classes>
</test>
<test verbose="1" name="Close the report" preserve-order="true">
<classes>
<class name="com.company.qa.seleniumautomation.Reports">
<methods>
<include name="close"/>
</methods>
</class>
</classes>
</test>
</suite>
the jenkins job is running on it's own client. it don't have a visible UI, that's why you test failed.
you should change you test code to use Remote Webdriver or Selenium Grid
//need to start a selenium standalone server
DesiredCapabilities capability = DesiredCapabilities.firefox();
WebDriver driver = new RemoteWebDriver(new URL("http://SERVER_ADDRESS:4444/wd/hub"), capability);
refer to Selenium Grid Wiki for more detaisl
If you start Jenkins using command line and if you run tests then you can see running tests on browser.
Command to start Jenkins using command line :
java -jar jenkins.war
Hope this helps.
Try to run it as admin from the command prompt, worked for me.
If you are on Windows :
1).Go to command prompt:
type "cmd"
2).Press Ctl+Shft+Enter(to open as admin)
3).Type "Services.msc"
4).It will open the Windows Services,select "Jenkins" Service , right click & open properties and go to "Log On" tab and select "Local System Account" and select "Allow service to interact with the Desktop".
5).Press OK.
6).A dialog box will open .Click "View The Message" on the window "A program running on this computer is trying to display a message".
Now you can see the tests running on the same machine...enjoy!