Is there any way to re-run failed tests using TestNG when same set of tests are run in different threads? I see there people use Retry listeners [https://stackoverflow.com/a/36320794/2020491 ] as in this link. But this does not seem to be working when i run same set of tests in multiple threads.
Below is my testng.xml where in com.example.SampleTest i have 2 tests in which 1 fails. i get the below result when run
===============================================
SampleTest
Total tests run: 5, Failures: 2, Skips: 1
===============================================
The one with test name="ONE", failed test runs for the second time but failed test is not run the second time for test with name="TWO" (when the class names are different for both the tests then this works as expected)
<suite name="SampleTest" verbose="1" parallel="tests" thread-count="1"><test name="ONE">
<classes>
<class name="com.example.SampleTest" />
</classes>
</test>
<test name="TWO">
<classes>
<class name="com.example.SampleTest" />
</classes>
</test>
<listeners>
<listener class-name="com.example.TestListener"/>
<listener class-name="com.example.RetryListener"/>
</listeners> </suite>
Below is my class file
#Test
public void testOne(){
Assert.assertTrue(false);
}
#Test
public void testTwo(){
Assert.assertTrue(true);
}
Related
In summary, I have 2 tests declared in a test suite as below however the testng runner executes the beforesuite method of the last test always. This results in the test reports shows there are 3 tests executed which is not expected. By trailing the code, I found the privateRun() overrides the beforesuite method for the last test because it iterates itselft for the 2 testRunners.
Please advise how I can run the beforesuite always for the first test instance, not the last one of the suite.
Here is the test structure.
Test.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="suiteA">
<test name="T1">
<classes>
<class name="C1"/>
<class name="C2"/>
</classes>
</test>
<test name="T2">
<classes>
<class name="C3"/>
<class name="C4"/>
</classes>
</test>
</suite>
Test Run
Suite A
T2 *(this doesn't run any tests other than Beforesuite)*
>T1
>T2
When I upgraded my existing package in POM.xml (Maven) from QAF 3.0.0 to 3.0.1b my automation test stopped picking up the tests mentioned in testng.
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS
All my tests are standard BDD (Gerkin) written and managed according to https://github.com/qmetry/qaf-blank-project-maven
Note: When I checked QAF latest repo all tests are written inside /test
My project structure
/scenarios
/config/
/src/...
..etc
Below is my TestNg config
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="test suite 1" verbose="0" parallel="none">
<listeners>
<listener class-name="com.proj.listener.proj_listener" />
</listeners>
<test name="Config-1" enabled="true">
<parameter name="env.resources" value="resources;executions/exec_1"/>
<groups>
<run>
<include name="#sit1" />
</run>
</groups>
<classes>
<class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory"/>
</classes>
</test>
</suite>
Try using meta-data filter. For example:
<test name="Config-1" enabled="true">
<parameter name="env.resources" value="resources;executions/exec_1"/>
<parameter name="include" value="{'groups':['#sit1']}"></parameter>
<classes>
<class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory"/>
</classes>
</test>
I am not able to invoke the browsers in parallel, which are currently invoking one after another. Need a way to invoke the browsers in parallel tests.
NOTE: In my configuration xml file I have kept the thread count as 2.
Below is my configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "testng.org/testng-1.0.dtd"; >
<suite name="Parallel" parallel="tests" thread-count="4" >
<test verbose="3" name="<name>">
<parameter name="platform" value ="win8"/>
<parameter name="browsername" value ="internet explorer"/>
<classes>
<class name="com.parallel.execution.ParallelExecution">
<methods>
<include name="testmethod1"/>
</methods>
</class>
</classes>
</test>
</suite>
We have to define two attributes 'parallel' and 'thread-count' in simple testng.xml file. See below:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Parallel Execution suite" parallel="methods" thread-count="2">
<test name="Regression 2">
<classes>
<class name="com.parallel.TestParallelExecution"/>
</classes>
</test>
</suite>
In above, as we want the test methods to be executed parallel, we have set the parallel attribute as 'methods' and thread-count attribute will control the max number of threads to be created.
Re-installing TestNG solved the above problem.
You have to explicitly write code to invoke the browser as part of a #Before configuration to invoke the browser every time a #Test is run . I will specify one of the many approaches so that you get an idea.
<suite name="Parallel" parallel="tests" thread-count="4" >
<test verbose="3" name="test1">
<classes>
<class name="com.parallel.execution.ParallelExecution1"/>
</classes>
</test>
<test verbose="3" name="test2">
<classes>
<class name="com.parallel.execution.ParallelExecution2"/>
</classes>
</test>
</suite>
Consider a suite file with 2 tests set to run in parallel. What we expect is #Test methods in ParallelExecution1 runs in first browser and #Test methods in ParallelExecution2 runs in second browser. So you need a mechanism by which you can invoke browser sessions and run your test methods. Enter BaseTest class.
public abstract class BaseTest {
protected WebDriver driver;
#BeforeTest
#Parameters({"browser"})
public void init(String browser) {
// Initialize your browser here. Code below is dummy
driver = new FF();
}
#AfterTest
public void end() {
driver.close();
driver.quit();
}
}
Now inherit this 'BaseTest' in both your test classes.
public class ParallelExecution1 extends BaseTest {
#Test
public void test1() {
}
}
public class ParallelExecution2 extends BaseTest {
#Test
public void test2() {
}
}
Now both the tests have #BeforeTest and #AfterTest methods which will invoke the browsers.
I installed NetBeans 8.0.1 on my computer and used it to create a test suite :
<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="testing">
<test name="suite">
<packages>
<package name="myPackage.tests"/>
</packages>
</test>
</suite>
In myPackage.tests there is a test class, a very simple one :
[package, imports]
public class Tests001 {
public Tests001 () {
}
#Test
public void Fail() {
assertFalse(true);
}
In NetBeans, I launched the test by right-clicking the.xml file and selecting 'test file'. Everything works perfectly - or rather, it correctly fails - intended result for now.
But I need to launch it from console, so I run :
> java org.testng.TestNG testing.xml
And get :
===============================================
testing
Total tests run: 0, Failures: 0, Skips: 0
===============================================
No test is run. I can't find out why : test class and methods are public ; java is configured correctly (JDK 1.8.0_25) and I set CLASSPATH to
C:\Program Files\NetBeans 8.0.1\platform\modules\ext\testng-6.8.1-dist.jar
I must have forgotten something...
Or is there a way, from console, to ask NetBeans to proceed to test ?
In my experience, the suite.xml file needs this structure:
<suite name="mySuite" allow-return-values="true" parallel="tests"
verbose="1" thread-count="2">
<test name="TestStuff">
<groups>
<run>
<include name="Smoke.mytests"/>
</run>
</groups>
<packages>
<package name="com.myco.mytests"/>
</packages>
</test>
</suite>
I have six classes (for example sake).
Want three of them to run parallely and the other 3 sequentially. How to design the TestNG.xml ?
Here's what I came up with -
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite thread-count="20" verbose="1" name="My Test Suite"
skipfailedinvocationcounts="false" junit="false" parallel="none" preserve-order="true"
data-provider-thread-count="10" annotations="JDK">
<test name = "my test" preserve-order="false">
<classes>
<class name = "com.mypackage.ClassA"/>
<class name = "com.mypackage.ClassB"/>
<class name = "com.mypackage.ClassC"/>
</classes>
</test>
<test name = "test2" preserve-order="true">
<classes>
<class name = "com.mypackage.ClassD"/>
<class name = "com.mypackage.ClassE"/>
<class name = "com.mypackage.ClassF"/>
</classes>
</test>
</suite>
I am not sure if the suite file is correct.
Also, in each of the classes there is one method printing the class name and the threadId.
The output I get for the above testng.xml is
class A 1
class B 1
class C 1
Class D 1
class E 1
class F 1
===============================================
My Test Suite
Total tests run: 6, Failures: 0, Skips: 0
===============================================