We defined one testng result listener which help us to send the testing result for each test case defined in testng.xml to one internal tool such like below:
public class TestResultsListener implements ITestListener, ISuiteListener {
#Override
public void onFinish(ISuite suite){
// some code to send the final suite result to internal tools
}
#Override
public void onTestSuccess(ITestResult iTestResult) {
this.sendResult(iTestResult,"PASS");Result
}
private void sendStatus(ITestResult iTestResult, String status){
// Set test case information
......
jsonArr.add(testResult);
}
}
And then we integrated this listener to other project's testng xml file such like:
<listeners>
<listener class-name="com.qa.test.listener.TestesultsListener" />
</listeners>
It worked as designed: once the test suite finishes, the test result will be uploaded to internal tools.
Now we have one requirement that in one project, one test case in testng.xml is related to 3 test cases in internal tool which means that for one test case in testng.xml we need to update 3 test cases in internal tools. How can we update our current testng listener to fulfill this?
Thanks a lot.
You can annotate each of your tests with the list of corresponding internal test tool ids:
Here I suppose that you have 2 testng tests: one is related to internal test IT-1, and the other one to internal tests IT-2, IT-3 and IT-4:
#Listeners(MyTestListener.class)
public class TestA {
#Test
#InternalTool(ids = "IT-1")
public void test1() {
System.out.println("test1");
fail();
}
#Test
#InternalTool(ids = {"IT-2", "IT-3", "IT-4"})
public void test2() {
System.out.println("test2");
}
}
The annotation is simply defined like this:
#Retention(RetentionPolicy.RUNTIME)
public #interface InternalTool {
String[] ids();
}
The your listener has just to figure out which annotation are present on successful/failed tests:
public class MyTestListener extends TestListenerAdapter implements ITestListener {
#Override
public void onTestSuccess(ITestResult tr) {
super.onTestSuccess(tr);
updateInternalTool(tr, true);
}
#Override
public void onTestFailure(ITestResult tr) {
super.onTestFailure(tr);
updateInternalTool(tr, false);
}
private void updateInternalTool(ITestResult tr, boolean success) {
InternalTool annotation = tr.getMethod().getConstructorOrMethod().getMethod().getAnnotation(InternalTool.class);
for (String id : annotation.ids()) {
System.out.println(String.format("Test with id [%s] is [%s]", id, success ? "successful" : "failed"));
}
}
}
The following output is produced:
test1
Test with id [IT-1] is [failed]
test2
Test with id [IT-2] is [successful]
Test with id [IT-3] is [successful]
Test with id [IT-4] is [successful]
You can also extend this mechanism to Suite listeners as well.
Disclaimer: The line
InternalTool annotation = tr.getMethod().getConstructorOrMethod().getMethod().getAnnotation(InternalTool.class); is not bullet-proof (high risk of null pointer exception). Should be more robust.
Related
I am trying to get some context of the result of the test run in the #AfterTest. I would like to have, at bare minimum, knowledge if it passed or not and ideally also the thrown exception if there is one.
However, every parameter I try seems to not be resolvable and I can't find any documentation on what should be available.
Code:
public class TestClass {
#AfterEach
public void afterEach(
TestInfo testInfo, //works, but no report on state of test
// none of these work
TestExecutionSummary summary,
TestExecutionResult result,
TestFailure fail,
Optional<Throwable> execOp,
Throwable exception
) {
// ...
}
}
What can I do to get this context?
Not sure if this is what you want, but you can use either a TestExecutionListener or a TestWatcher (there are also other ways that you can check in documentation).
An example of TestWatcher can be found here: TestWatcher in junit5 and a more detailed explanation here: https://www.baeldung.com/junit-testwatcher.
The following code example was partially taken from here.
public class TestResultLoggerExtension implements TestWatcher, AfterAllCallback {
...
#Override
public void testSuccessful(ExtensionContext context) {
System.out.println("Test Successful for test {}: ", context.getDisplayName();
}
#Override
public void testFailed(ExtensionContext context, Throwable cause) {
System.out.println("Test Failed for test {}, with cause {}", context.getDisplayName(), cause.getMessage());
}
}
You test class would be something like this:
#ExtendWith(TestResultLoggerExtension.class)
public class TestClass {
You can adapt the logic to your needs.
More References:
https://junit.org/junit5/docs/current/user-guide/#extensions-test-result-processing
https://junit.org/junit5/docs/current/user-guide/#launcher-api-listeners-custom
I have a class like this
public class LoginPresImpl implements LoginAPIInterface.LoginDataListener, LoginAPIInterface.LoginPresenter{
LoginAPIInterface.LoginView loginView;
LoginAPIInterface.LoginDataInteractor loginDataInteractor;
public LoginPresImpl(LoginAPIInterface.LoginView loginView) {
this.loginView = loginView;
loginDataInteractor=new LoginDataModel(this);
}
#Override
public void getLoginUpdateData(String username, String password,String registrationToken) {
loginDataInteractor.getLoginData(username,password,registrationToken);
}
}
I want to test if calling
getLoginUpdateData()
will call the getLoginDate() method of loginDataInteractor.
I have created a test class like this
public class LoginPresImplTest {
LoginAPIInterface.LoginDataInteractor loginDataInteractorMock;
LoginAPIInterface.LoginView loginViewMock;
LoginPresImpl loginPres;
#Before
public void setUp(){
loginDataInteractorMock = Mockito.mock(LoginAPIInterface.LoginDataInteractor.class);
loginViewMock = Mockito.mock(LoginAPIInterface.LoginView.class);
loginPres = Mockito.spy(LoginPresImpl.class);
}
#Test
public void getLoginUpdateData() {
loginPres.getLoginUpdateData("01","","");
verify(loginPres).getLoginUpdateData("01","","");
}
But I don't know how to check if calling
getLoginUpdateData()
will eventually call
loginDataInteractor.getLoginData()
method. How can I test this using JUnit or Mockito.
I want to test if calling
getLoginUpdateData()
will call the getLoginDate() method of loginDataInteractor.
loginDataInteractor is a dependency of the code under test (cut) you showed.
In a UnitTest you only verify the behavior of the cut. You do not verify the behavior of the dependencies. They get their own unit tests.
I am currently writing a test in TestNG. Currently I have the following method:
#Parameters({"clickRememberMe"})
#Test
public void runTest(String clickRememberMe) {
}
Now I have found that a TestNG class called org.testng.ITestContext can be used to share data between tests, which I want to use. The tutorial showing it told me to do this:
#Test
public void runTest(ITestContext itc){
}
And this also works. However, now I want to use parameters together with the ITestContext like follows:
#Parameters({"clickRememberMe"})
#Test
public void runTest(ITestContext itc, String clickRememberMe) {
}
Sadly, this throws the following exception:
java.lang.IllegalArgumentException: argument type mismatch
Is there any way to use #Parameters together with ITestContext in TestNG?
I already found the solution. Instead of:
#Parameters({"clickRememberMe"})
#Test
public void runTest(ITestContext itc, String clickRememberMe) {
}
I had to put the ITestContext after the parameters, like this:
#Parameters({"clickRememberMe"})
#Test
public void runTest(String clickRememberMe, ITestContext itc) {
}
I'm trying to search for an example of how to run a tests suite for test classes that use Robolectric, for example I have this class:
#Config(sdk = 16, manifest = "src/main/AndroidManifest.xml")
#RunWith(RobolectricTestRunner.class)
public class RestaurantModelTest {
//setUp code here...
#Test
public void testFindByLocation() throws Exception {
//test code here
}
}
All unit test and assertions of class RestaurantModelTest are passing, but I also have another class lets call it XModelTest (in which all assertions and unit test are passing.
My problem
I don't find any tutorial/example of how to use a test suite using robolectric.
Should this be done in the same package where my RestaurantModelTest and XModelTest are? If not where?
Also I tried doing this with TestSuite from JUnit but many questions arise should the class with my TestSuite extend extends TestSuite super class?
If someone can give me a short example using my RestaurantModelTestand XModelTestclasses it would be great.
I believe, I've also partially covered this question answering your second question - Can't run android test suite: Exception in thread “main”
Here's how to write a suite with Robolectric:
Let's say we have 2 model classes.
CartModel.java
public class CartModel {
public float totalAmount;
public int products;
public void addToCart(float productPrice) {
products++;
totalAmount += productPrice;
}
}
and RestaurantModel.java
public class RestaurantModel {
public int staff;
public void hire(int numberOfHires) {
staff += numberOfHires;
}
}
Let's write some dummy tests for them:
CartModelTest.java
#RunWith(RobolectricGradleTestRunner.class)
#Config(constants = BuildConfig.class, sdk=21)
public class CartModelTest {
#Test
public void addToCart() throws Exception {
CartModel cartModel = new CartModel();
assertEquals(0, cartModel.totalAmount, 0);
assertEquals(0, cartModel.products);
cartModel.addToCart(10.2f);
assertEquals(10.2f, cartModel.totalAmount, 0);
assertEquals(1, cartModel.products);
}
}
RestaurantModelTest.java
#RunWith(RobolectricGradleTestRunner.class)
#Config(constants = BuildConfig.class, sdk=21)
public class RestaurantModelTest {
#Test
public void hire() throws Exception {
RestaurantModel restaurantModel = new RestaurantModel();
assertEquals(0, restaurantModel.staff);
restaurantModel.hire(1);
assertEquals(1, restaurantModel.staff);
}
}
And now last step - to group them together into one ModelsTestSuite.java:
#RunWith(Suite.class)
#Suite.SuiteClasses({
RestaurantModelTest.class,
CartModelTest.class
})
public class ModelsTestSuite {}
To run - just right-click in ModelsTestSuite and click "Run ModelsTestSuite". That's it!
NB! In Android Studio 2.0 3b, you have to disable instant run (Preferences -> Build, Execution, Deployment -> Instant Run -> Enable Instant Run - uncheck) in order to run Robolectric tests (to avoid java.lang.RuntimeException: java.lang.ClassNotFoundException: Could not find a class for package: <package name> and class name: com.android.tools.fd.runtime.BootstrapApplication).
I hope, it helps
why does TestNG execute #AfterMethod as a #Test if my first test is passed?
For example the testoutput if my first test fails is:
1 test passed, 1 test failed.(3599,0 s)
TestSuite FAILED
run FAILED: check5=3 Expected: <3> got: <5>
run passed (1.0s)
But if I just switch the order of test cases, so the first will pass, i get this:
3 test passed, 2 test failed.(2867,0 s)
TestSuite FAILED
run passed (1.0s)
run FAILED: check5=3 Expected: <3> got: <5>
AfterMethod FAILED (4,0 s) // <--- wtf, this is not an #Test
AfterTest passed (5,0 s)
AfterSuite passed (15,0 s)
what is happening? my testngsuite.xml:
<suite name="TestSuite_03">
<test name="TestCase_17">
<groups>
<run><include name="functest"/></run>
</groups>
<classes>
<class name="TestStep_003" desc="will pass" />
<class name="TestStep_012" desc="will fail" />
</classes> ...
I am using Maven, TestNG and Java via NetBeans
my structure:
public abstract class TestCommon
{
#BeforeSuite(groups={"functest"})
public void BeforeSuite()
{
// clean report folder
}
#BeforeTest(groups={"functest"})
public void BeforeTest()
{
// start selenium browser
}
#AfterMethod(groups={"functest"}) // this is not a #test, still gets shown as failed
public void AfterMethod()
{
// check for failure and capture screenshot
}
#AfterTest(groups={"functest})
public void AfterTest()
{
// close browser
}
}
public class TestStep_003 extends TestCommon
{
#Test(groups = {"functest"})
public void run()
{
assertThat(5, Matchers.equalTo(5)); // will pass
}
}
public class TestStep_012 extends TestCommon
{
#Test(groups = {"functest"})
public void run()
{
assertThat(5, Matchers.equalTo(3)); // will fail
}
}
Annother issue is that the testoutput is not in order, the timestamps are
not older -> newer, there are newer ones between old ones:
thats how my output looks like:
1327840359762: TestStep_012.AfterMethod // this is not the oldest timestamp!
1327840359763: TestStep_003.run
1327840359765: TestStep_003.AfterMethod
1327840357189: TestStep_012.BeforeSuite
1327840357192: TestStep_012.BeforeTest
1327840359758: TestStep_012.run
1327840359762: TestStep_012.AfterMethod
1327840359763: TestStep_003.run
1327840359765: TestStep_003.AfterMethod
furthermore, there are only 2 methods TestStep_003.run and and TestStep_012.run,
and still it shows AfterMethod 4x