I have a testNG method just like this:
#Test(dataProvider="takeMyProvider")
public void myTest(String param1, String param2){
System.out.println(param1 + " " + param2);
}
My dataprovider returns 10 elements. My method will be executed 10 times in one thread. How is it possible to parallel this? For example
I want to have 5 methods in parallel. The webdriver should open 5 browsers at the same time. After these 5 tests in parallel the other 5 test should be executed
or
the webdriver should open 10 browsers and do all 10 elements parallel
Does anybody have an idea?
You can define the parallelism via a suite file in TestNG. Example following runs methods in parallel with 10 threads:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="MySuiteNameHere" parallel="methods" thread-count="10">
<test name="Selenium Tests">
<classes>
<class name="foo.bar.FooTest"/>
</classes>
</test>
</suite>
You also need to note that your data provider can is thread safe to allow it to not force the method to run sequentially.
// data providers force single threaded by default
#DataProvider(name = "takeMyProvider", parallel = true)
Be careful, though. TestNG does not create new instances of the class object when running with parallel methods. That means that if you save values on the test class object you can run into threading issues.
Also note, if you set the thread count to 5, it does not wait for the first 5 to all be finished and then start up the next 5. It basically puts all the test methods into a queue and then starts up x threads. Each thread then simply polls the next element from the queue when it is available.
TestNG's #Test annotation already has what you want... To some degree:
// Execute 10 times with a pool of 5 threads
#Test(invocationCount = 10, threadPoolSize = 5)
What this won't do is fit your first scenario exactly, that is, run the first 5, wait for them to finish, run the other 5.
many thx for your feedback and useful tipps.
My tests ran - maybe - in any parallel way but only in one browser instance.
Lets jump in in detail:
My dataprovider returns an object[][]
#Dataprovider(name = "takeMyProvider", parallel = true)
public object[][] myProvider(){
return new object[][]{{"1", "name1"}, {"2", "name2"} {"3", "name3"}}
}
This test method is executed three times
#Test(dataProvider="takeMyProvider")
public void myTest(String param1, String param2){
System.out.println(param1 + " " + param2);
}
but just in one browser instance. Thats not what I want.
I want testNG to start 3 chrome instances and doing the 3 tests in parallel.
Btw I am running the tests on a selenium grid. Maybe with 100 nodes.
It would be perfect when 100 nodes doing this test in parallel. Or even 1.000, depends on the dataprovider.
Does anybody have an idea?
Best regards
Related
If we build and execute selenium (TestNG) suite in Jenkins and if some tests fails; after fixing is there any way to execute only those failed test cases in Jenkins?
Yes, from the docs:
Every time tests fail in a suite, TestNG creates a file called testng-failed.xml in the output directory. This XML file contains the necessary information to rerun only these methods that failed, allowing you to quickly reproduce the failures without having to run the entirety of your tests.
If I understand your use case, you are looking to save the list of failed tests, make changes to the code and re-execute that list, is that correct ? In that case you can store that testng-failed.xml file and use it for the next execution in Jenkins, possibly adding a checkbox to the job that lets you choose wether to use this test suite or the default one.
Please try below to run specific tests (example: testcases you have fixed) with mnetioning below include tag in testNG.xml
<classes>
<class name="test.IndividualMethodsTest">
<methods>
<include name="testMethod" />
</methods>
</class>
</classes>
From my QA and automation perspective, it's always better to run the entire suite from jenkins, if you want to check why they fail you can do it locally. the other option is to parameterize the xml but it is a lot of work, it could be by maven arguments and writing in the xml
Why in Jenkins?, you can build one Testng Retry Listener where it keeps polling the entire test execution and re-executes only Failed testcases.
Refer this below.
public class RetryFailedTestCases implements IRetryAnalyzer {
private int retryCnt = 0;
// Mentioned maxRetryCnt (Maximiun Retry Count) as per your requirement. Here I
// took 3, If any failed testcases then it runs two times
private int maxRetryCnt = 1;
// This method will be called everytime a test fails. It will return TRUE if a
// test fails and need to be retried, else it returns FALSE
public boolean retry(ITestResult result) {
if (retryCnt < maxRetryCnt) {
System.out.println("Retrying " + result.getName() + " again and the count is " + (retryCnt + 1));
retryCnt++;
return true;
}
return false;
}
}
Below is my XML piece.
<?xml version="1.0" encoding="UTF-8"?>
<suite name='Automation' threadCount="5" parallel="methods">
<tests>
<parameter name='clientName' value='Five' />
<test name='PA'>
<classes>
<class name='TC_INC_1'>
</class>
</classes>
</test>
So I am loading the required data from excel through DATA PROVIDER in TestNg.
What I wanted to achieve is to run each row in different threads.
Lets say I had 5 rows of data
1- Go to Google.com
2- Go to Facebook.com
3- Go to Dollarama.com
4- Go to Walmart.com
5- Go to KegSteak.com
And say I am running two thread means two browsers.
I want both browsers run parallelly executing each of the row in any sequence.
Thread 1 - 1- Go to Google.com
Thread 2- 2- Go to Facebook.com
First test done - browser closed and opens again.
Now it should pick the 3 and fourth row.
Thread 1 - 3- Go to Dollarama.com
Thread 2- 4- Go to Walmart.com
browser closed and opens again.
Thread 1 - 5- Go to KegSteak.com
[![testdata][1]][1]
What I actually see is two browsers open and one of the browser runs the url and the other just becomes static after launching chrome.
Any fixes ?
With Local WebDriver variable
Make sure, you launch and tear-down your WebDriver within a test method:
#Test(dataProvider = "provideUrls")
public void navigateByUrlTest(String url){
WebDriver driver = ...
driver.get(url);
// do something
driver.quit();
}
//I know this implemented to get data from Excel, but just for example..
#DataProvider(parallel = true)
public Object[][] provideUrls() {
return new Object [][] {
{"https://google.com"},
{"https://facebook.com"},
{"https://dollarama.com"},
{"https://walmart.com"},
{"https://kegSteak.com"}
};
}
With Global Thread-Safe WebDriver variable
WebDriver configuration can be moved to #BeforeMethod/#AfterMethod methods.
NOTE: ThreadLocal wrapper should be used for WebDriver instance in this case. This ensures we will keep separate WebDriver instances per each thread.
protected ThreadLocal<WebDriver> driverThreadSafe = new ThreadLocal<WebDriver>();
#BeforeMethod
public void launchDriver() {
driverThreadSafe.set(new ChromeDriver());
}
#AfterMethod
public void quitDriver() {
driverThreadSafe.get().quit();
}
#Test(dataProvider = "provideUrls")
public void test(String url){
WebDriver driver = driverThreadSafe.get();
driver.get(url);
// do something, but do not quit the driver
}
Configure Threads Count
<suite name='Automation' threadCount="5" - this will not work for DaraProvider parallelism.
You have to pass dataproviderthreadcount testNG argument with thread count for data-provider.
e.g. programmatically, add this method to the current class (or parent base test class)
#BeforeSuite
public void setDataProviderThreadCount(ITestContext context) {
context.getCurrentXmlTest().getSuite().setDataProviderThreadCount(5);
}
References
TestNG parallel Execution with DataProvider
https://testng.org/doc/documentation-main.html#running-testng
https://www.baeldung.com/java-threadlocal
I have a TestNG test Suit for a JAVA Project and, In there I have a
#Test(DataProvider="ListOfObjects") annotated Method. Which provides method with around 20 rows of data.( Hence the method runs 20 times.)
Now, I want to run this class for 2hrs (part of SOAK related test.) On average the Class takes around 10 mins for single run. So I am thinking or running the whole class for 12 times, and thus thinking of using #Test(invocationCount = 20) on the Class itself.
Any Better Ideas ?
Found an Embarrassingly simple solution:
Repeating the whole Test Suit as follows
#Test
public void RepeatTestSuite() {
long startTime = new Date().getTime();
while(!isTestFinished(startTime)) {
List<String> suites = new ArrayList<String>();
suites.add("./SOAK_all41.xml"); //path of .xml file to be run-provide complete path
TestNG tng = new TestNG();
tng.setTestSuites(suites);
tng.run(); //run test suite
}
You could extract the test in a method, then create a test method with a reasonably high invocation count. In that test method, compare the time with a variable holding the timestamp of the first run and if it has run for more than 20 minutes skip the test.
Currently I am using Selenium Webdriver, and for annotation I am using testNG, In my code I have 100 test scripts with #test annotation ..I just want to run only one of my test case 100 times ..how I can do it any suggestion with proper example is much appreciated eg
run only script number 5 hundred times
In the #Test annotation - you can add a invocationCount attribute with the number of times you want to run it.
#Test(invocationCount = 100)
public void testMethod() {
}
I have a java file which has 7 junit tests to run. If I run all the tests at once all but 1 passes. If I comment out certain tests and that one test always passes.
Can anybody offer any suggestions as to what could be causing this?
My first thought was something in the test Setup or cleanup but I am not sure what it could be. All I do in the clean up is exit the driver and output the time taken to run the test.
In the setup I set up the driver, the time started, create a firefox profile and read in some data from a properties file to use in the tests.
If it was the setup / cleanup surely the other 6 tests would also be effected? The test that fails is a simple test to check that entering an invalid card type displays an error message on the page.
UPDATE:
I've renamed the test so it runs first and now all 7 pass each time. What could be causing this? Do I need to set something in my test cleanup to get it back to a default state?
My test cleanup:
#After
public void testCleanup() throws IOException {
driver.quit();
endTime = System.currentTimeMillis();
long totalTime = ((endTime - startTime)/1000)/60;
System.out.println();
System.out.println("Test Suite Took: " + totalTime + " Minutes.");
}