I am executing TestNG test with Jenkins build and Allure plugin. My problem is that if I am executing them from IDEA all test results are displayed.
If I am using same configuration from Jenkins some tests are missing.
XML Suite file
<suite name="All in one" verbose="1" thread-count="2" parallel="methods" >
<test name="Chrome:MainPage">
<parameter name="browser" value="chrome"/>
<classes>
<class name="mainPage.MainPageTest" />
<class name="patients.PatientsPageTest"/>
</classes>
</test>
<test name="Firefox:MainPage">
<parameter name="browser" value="firefox"/>
<classes>
<class name="mainPage.MainPageTest" />
<class name="patients.PatientsPageTest"/>
</classes>
</test>
Result from Allure when directly "called" on results is good.
Allure plugin from Jenkins ignores some tests , and i have no idea why.
I found solution after 2 days , i am not sure which of them worked , but maybe someone will run into same issue someday...
Check Jenkins version of Allure plugin
Check Gradle version of Allure plugin
Check TestNG for validity i missed this
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
Related
When I upgraded my existing package in POM.xml (Maven) from QAF 3.0.0 to 3.0.1b my automation test stopped picking up the tests mentioned in testng.
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS
All my tests are standard BDD (Gerkin) written and managed according to https://github.com/qmetry/qaf-blank-project-maven
Note: When I checked QAF latest repo all tests are written inside /test
My project structure
/scenarios
/config/
/src/...
..etc
Below is my TestNg config
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="test suite 1" verbose="0" parallel="none">
<listeners>
<listener class-name="com.proj.listener.proj_listener" />
</listeners>
<test name="Config-1" enabled="true">
<parameter name="env.resources" value="resources;executions/exec_1"/>
<groups>
<run>
<include name="#sit1" />
</run>
</groups>
<classes>
<class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory"/>
</classes>
</test>
</suite>
Try using meta-data filter. For example:
<test name="Config-1" enabled="true">
<parameter name="env.resources" value="resources;executions/exec_1"/>
<parameter name="include" value="{'groups':['#sit1']}"></parameter>
<classes>
<class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory"/>
</classes>
</test>
I have testNG file which has multiple methods.
In this testNG file I added the parameters to pass the parameter on code from xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="LocalIosTestSuite" parallel="tests" thread-count="1" >
<test name="iphonetest">
<parameter name="platformName" value="iphonetest" />
<classes>
<class name="tests.TestKotlin">
</class>
</classes>
</test>
<test name="androidtest">
<parameter name="platformName" value="androidtest" />
<classes>
<class name="tests.TestKotlin">
</class>
</classes>
</test>
</suite>
Kotlin Class
import org.testng.annotations.Parameters
import org.testng.annotations.Test
class TestKotlin {
#Parameters("platformName")
#Test()
fun Testmethod(platformName:String)
{
println("TestKotlin")
println(platformName)
}
}
Command to run a single test from testng file
./gradlew test -Psuite5 -testnames iphonetest
This is not working for me please help me any solution to run a specific test on gradle
I have a testng.xml like below. I am trying to run test by names which are in testng.xml (myfirstcase or mysecondcase) by using maven command line.
command line:
mvn -Dtest=myfirstcase -Dsurefire.suiteXmlFiles=/Users/huseyintumer/IdeaProjects/experitestandroid/testng.xml test -f /Users/huseyintumer/IdeaProjects/experitestandroid/pom.xml
When i command line below, it works. But parameters in testng.xml does not pass.
mvn -Dtest=firstTest -Dsurefire.suiteXmlFiles=/Users/huseyintumer/IdeaProjects/experitestandroid/testng.xml test -f /Users/huseyintumer/IdeaProjects/experitestandroid/pom.xml
Actually i dont want to run test by class name. I need to run test by test names in testng.xml with parameters.
testng.xml:
<suite name="Suite" parallel="tests">
<test name="myfirstcase">
<classes>
<class name="firstTest">
<parameter name="udid" value="CQ300002LG" />
</class>
</classes>
</test>
<test name="mysecondcase">
<classes>
<class name="firstTest">
<parameter name="udid" value="BH90018AAX" />
</class>
</classes>
</test>
I have a project which have Grid + Selenium + Java + Testng + maven.
Is there is a way to set up, that the person who will run test he will be able to pass his own configurations which will overwrite default configs?
Here is my testng.xml file which i`m running:
<suite name="TestSuite" parallel="tests">
<test name="firefox test">
<parameters>
<parameter name="platform" value="MAC" />
<parameter name="browser" value="firefox" />
<parameter name="version" value="50.1.0" />
<parameter name="url" value="google.com" />
</parameters>
<classes>
<class name="com.ParallelTest.CreateRandomProfileTest"/>
<class name="com.ParallelTest.LogInTest">
</class>
</classes>
</test>
</suite>
For example if someone will want to run 5 Tests against Chrome on Win? How to pass it without changing the code and where it should be overwritten. Can someone drop me a link for resources where i can check it? Thanks!
It can be done easily by file parameter plugin of Jenkins.
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build
Ie, Lets assume this a workspace with default folder structure
data
pageobjects
tests
suite.xml
now an user uploads his own version of suite.xml and run the job.
Jenkins will pull the project from source control/github and replaces the suite.xml with the one uploaded by the user and runs the project.
You could try this way as well.
Pass variable from jenkins to testng.xml
Actually I'm testing multiple website for my company. All the website have the same structure and i'm testing various things on each.
For my test i have a Selenium Java Maven project using TestNG. For each web site, i've created a sub folder and generate a specific testing.xml file to specify each test i'm doing for the website.
Here the structure : http://imgur.com/g0VHfMw
My testing.xml files are like this :
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test suite website 1" parallel="tests" thread-count="2">
<test name="Test1">
<classes>
<class name="package.for.mytest1"></class>
</classes>
</test>
<test name="Test2">
<classes>
<class name="package.for.mytest2"></class>
</classes>
</test>
<test name="Test3">
<classes>
<class name="package.for.mytest3"></class>
</classes>
</test>
</suite>
How can I merge this 4 test suite in one testNG report ? And if it's possible, is there a way to only show the unstable or failed test on it ?
Any ideas ? Thanks for your help
Try Allure it could easily generate combined report for several suites and show only failed/broken tests by default.
Either you can Use Suite-XML tag in TestNG.xml file to report all your test in single report or implement a reporter of your own to collect all the results and then at the end flush everything into report of your own format with whatever information you need.
You can create a separate testng.xml file in which you can give the path of other testsuites files.
eg:-
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name= "Allsuites" verbose="1" preserve-order="true" >
<listeners>
<listener class-name="org.uncommons.reportng.HTMLReporter"/>
<listener class-name="org.uncommons.reportng.JUnitXMLReporter"/>
</listeners>
<suite-files preserve-order="true">
<suite-file path="testng1.xml" />
<suite-file path="testng2.xml" />
<suite-file path="testng3.xml" />
</suite-files>
</suite>