I have a testng xml file that is setup to run a specific class that has methods containing the following group tags or are grouped as one of the following:
parallel-test
sequential-test
smoke-test
Although some of my tests are designated as either parallel or sequential, some of the test may contain an additional tag 'smoke-test.' In this scenario, I want to be able to just run those that are grouped as 'smoke-test.' When I run the tests, it either cannot find any tests or it just runs all tests grouped as 'sequential-test.' I cannot seem to restrict the test run from this xml file to just those tests grouped as 'smoke-test.' In the same class, I have multiple testNG methods that have either the 'parallel-test' group or the 'sequential-test' group. What it looks like the TestNG xml file is doing is just ignoring the 'parallel-test' group. Thats great, but what if there is Testng method that contains the group 'parallel-test' and 'smoke-test?' I want to run every test regardless of whether it is for parallel testing or sequential, but ONLY those that have the additional group tag 'smoke-test.' My environment is the following:
IntelliJ
Maven
TestNG
Java
Please help. See sample test method in the WebTest.Test class:
#Test(groups = {"sequential-test","smoke-test"},description = "Test will validate something")
public void RunTest()
{
//do something
}
Here is the sample xml file below
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!--This is a template to be modified for QA environment tests-->
<suite name="Smoke Test">
<listeners>
<listener class-name="Listeners.Listeners"/>
<listener class-name="Listeners.ExtentListeners"/>
</listeners>
<parameter name="environment" value="QA"></parameter>
<test verbose="2" name="Sequential tests" parallel="methods" thread-count="1">
<!--<groups>
<run>
<exclude name="parallel-test"/>
</run>
</groups>-->
<classes>
<class name="WebTests.Test">
<methods>
<include name="smoke-test"/>
</methods>
</class>
</classes>
</test>
</suite>
Below XML Suite should run the all test method having group 'smoke-test'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!--This is a template to be modified for QA environment tests-->
<suite name="Smoke Test">
<listeners>
<listener class-name="Listeners.Listeners"/>
<listener class-name="Listeners.ExtentListeners"/>
</listeners>
<parameter name="environment" value="QA"></parameter>
<test verbose="2" name="Sequential tests" parallel="methods" thread-count="1">
<groups>
<run>
<include name="smoke-test"/>
</run>
</groups>
<classes>
<class name="WebTests.Test"/>
</classes>
</test>
</suite>
Related
I would like all my classes running sequentially. Below is my testng xml file:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="instances" thread-count="1">
<test name="Test">
<classes>
<class name="atk.tests.Printer"/>
<class name="atk.tests.Rejecteur"/>
<class name="atk.tests.Specfeatures"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
I've tried in suite name tag to do :
<suite name="Suite" parallel="instances" thread-count="1"> then
<suite name="Suite" parallel="false" >
And then in classes:
<classes parallel="methods">
No solution is working to make my tests running sequentially. For the moment, they are running in parallel and i would like to avoid that.
Is anyone able to provide a solution for that ? Thanks
By default, TestNG will run your tests in the order they are found in the XML file. If you want the classes and methods listed in this file to be run in an unpredictable order, set the preserve-order attribute to false
For this ordering, you can use the XML attribute group-by-instances. This attribute is valid either on <suite> or <test>
<suite name="Suite" group-by-instances="true">
or
<test name="Test" group-by-instances="true">
try once with preserver order true and parallel none
<suite name="Suite" parallel="none" preserve-order="true">
I have below testng.xml file where I've mentioned a group name to execute all testcases which has the same group name in my test suite
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestDive" verbose="0" parallel="methods" thread-count="2">
<test name="HereWeGo" enabled="true">
<groups>
<run>
<include name="SANITY_PROD"/>
</run>
</groups>
<classes>
<class name="com.xyz.Classname"/>
</classes>
</test>
</suite>
Now if i pass the group name from command line e.g. mvn clean test -DtestSuiteFile="config/sanity_suite.xml" -DincludeGroups=SANITY_STAGE
The problem is it still executes the all test case with group SANITY_PROD as mentioned in xml file.
Am i doing something wrong here ?
I am not sure about includeGroups attribute but try like this:
mvn clean test -Dgroups=SANITY_STAGE
I have a requirement to create custom testng.xml file based on the inputs. I was able to create one successfully but the groups in the suite level are misplaced i.e below the parameters and the listeners instead of the top of the parameters i.e groups should be placed first and then the others as specified in the below error.
The content of element type "suite" must match "(groups?,(listeners|packages|test|parameter|method-selectors|
suite-files)*)".
Below is piece of code to create the xml file.
XmlSuite xmlSuite = new XmlSuite();
xmlSuite.setName("CustomSuiteFile");
xmlSuite.setParallel(ParallelMode.TESTS);
xmlSuite.setThreadCount(1);
List<String> group = new ArrayList<String>();
group.add("Regression");
group.add("Smoke");
xmlSuite.setIncludedGroups(group);
xmlSuite.setListeners(Arrays.asList("Utils.TestListener"));
parameters.put("reportPath", "C://ExtentReports//ExtentReportResult");
xmlSuite.setParameters(parameters);
xml file created for the above code but this throws the above specified error.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite thread-count="1" name="CustomSuiteFile" parallel="tests">
<parameter name="reportPath" value="C://ExtentReports//ExtentReportResult"/>
<listeners>
<listener class-name="Utils.TestListener"/>
</listeners>
<groups>
<run>
<include name="Regression"/>
<include name="Smoke"/>
</run>
</groups>
<test thread-count="1" name="Chrome" parallel="tests">
<parameter name="browser" value="Chrome"/>
</test> <!-- Chrome -->
</suite> <!-- CustomSuiteFile -->
But below is the expected (the group is placed first and then the others in the suite level):
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite thread-count="1" name="CustomSuiteFile" parallel="tests">
<groups>
<run>
<include name="Regression"/>
<include name="Smoke"/>
</run>
</groups>
<parameter name="reportPath" value="C://ExtentReports//ExtentReportResult"/>
<listeners>
<listener class-name="Utils.TestListener"/>
</listeners>
<test thread-count="1" name="Chrome" parallel="tests">
<parameter name="browser" value="Chrome"/>
</test> <!-- Chrome -->
</suite> <!-- CustomSuiteFile -->
Please advise if i'm wrong somewhere and how to achieve the expected one. Thanks in advance.
Below is the testng.xml code. here I have test name Test1 and I want to include all the scenarios starting with regression tag (tag is present in cucumber feature file).
I want to parameterise "regression.*" value and want to pass as an argument at run time (Runas testng).
<test name="Test1">
<groups>
<run>
<include name="regression.*"/>
</run>
</groups>
Can somebody help me with this scenario.
Thanks in advance.....
You should be able to build the dynamic ability to pick and choose a group at runtime using a beanshell as a method selector.
Here's a sample beanshell powered suite xml. In the below sample, we are making use of a JVM argument -DgroupToRun to pass in the name of the groups which we want to be executed.
For more details on how to work with beanshells in TestNG take a look at
The official documentation here
my blog post here
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="false">
<test name="Test">
<method-selectors>
<method-selector>
<script language="beanshell">
<![CDATA[
whatGroup = System.getProperty("groupToRun");
groups.containsKey(whatGroup);
]]>
</script>
</method-selector>
</method-selectors>
<classes>
<class name="organized.chaos.GroupsPlayGround" />
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Actually I'm testing multiple website for my company. All the website have the same structure and i'm testing various things on each.
For my test i have a Selenium Java Maven project using TestNG. For each web site, i've created a sub folder and generate a specific testing.xml file to specify each test i'm doing for the website.
Here the structure : http://imgur.com/g0VHfMw
My testing.xml files are like this :
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test suite website 1" parallel="tests" thread-count="2">
<test name="Test1">
<classes>
<class name="package.for.mytest1"></class>
</classes>
</test>
<test name="Test2">
<classes>
<class name="package.for.mytest2"></class>
</classes>
</test>
<test name="Test3">
<classes>
<class name="package.for.mytest3"></class>
</classes>
</test>
</suite>
How can I merge this 4 test suite in one testNG report ? And if it's possible, is there a way to only show the unstable or failed test on it ?
Any ideas ? Thanks for your help
Try Allure it could easily generate combined report for several suites and show only failed/broken tests by default.
Either you can Use Suite-XML tag in TestNG.xml file to report all your test in single report or implement a reporter of your own to collect all the results and then at the end flush everything into report of your own format with whatever information you need.
You can create a separate testng.xml file in which you can give the path of other testsuites files.
eg:-
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name= "Allsuites" verbose="1" preserve-order="true" >
<listeners>
<listener class-name="org.uncommons.reportng.HTMLReporter"/>
<listener class-name="org.uncommons.reportng.JUnitXMLReporter"/>
</listeners>
<suite-files preserve-order="true">
<suite-file path="testng1.xml" />
<suite-file path="testng2.xml" />
<suite-file path="testng3.xml" />
</suite-files>
</suite>