I do some cleanup in external systems using testng #AfterClass annotation. But when tests are failed I really need that data. Can I make testng perform some actions if only tests are passed?
There is an option to get information about all failed tests till current moment. You have to inject ITestContext into your "afterClass" method.
#AfterClass
public void after(ITestContext context) {
context.getFailedTests().getAllResults()
}
Iterate through all results and filter by TestClass
AFAIK there is nothing at afterclass/aftersuite level. What you can do is couple of things:
AfterMethod does take ITestResult as an argument which gives you the result of the currently executed test. Based on that you can cleanup.
Or
ISuiteListener gives you an onFinish method with the testresult object, which you can iterate and then do the cleanup.
Example, where you can delete test data just for current test class if only tests are passed:
#AfterClass
public void deleteCreatedData(ITestContext context) {
if (hasClassFailedTests(context)) return;
//do your cleanup for current test class
}
protected boolean hasClassFailedTests(ITestContext context) {
Class clazz = this.getClass();
return context.getFailedTests().getAllMethods().stream().anyMatch(it ->
it.getRealClass().equals(clazz));
}
Related
Background: I'm executing tests with TestNG and I have a class annotated with #Test that generates a number, or ID if you will, and that same number is the input value of my second test. Is it possible to pass values between TestNG tests?
Sure. For example if you have two tests that is related you can pass the values from one test to another via test context attributes:
#Test
public void test1(ITestContext context) { //Will be injected by testNG
/* Do the test here */
context.setAttribute("myOwnAttribute", "someTestResult");
}
#Test(dependsOnMethods = "test1")
public void test2(ITestContext context) { //Will be injected by testNG
String prevResult = (String) context.getAttribute("myOwnAttribute");
}
You should create one test that handles whole case. Tests can't depend on each other, it's considered as bad practise. If you are using maven order of tests execution can be different in different environments.
Bad practice or not, it can be accomplished by simply using class fields. Just make sure your cases are executed in predictable order (eg. using #Test(priority) or dependsOn TestNG feature).
I am using TestNG to run data driven tests
My data is read from an external file
I have a retry logic that is essentially a different test method in the same class but retries only the failed entities from the previous test. I am controlling that using priority
Test(dataProvider="customTestDataProvider" , priority = 1)
public void testSomething(final ITestContext testContext , final CustomTestDataItem testData) throws CustomTestException{
setTestData(testData, testContext);
performStep1();
performStep2();
validateResult();
}
#Test(dataProvider="customTestDataProvider" , priority = 2)
public void testSomethingRetry1(final ITestContext testContext ,final CustomTestDataItem testData) throws CustomTestException{
testSomething(testContext , testData);
}
#Test(dataProvider="customTestDataProvider" , priority = 3)
public void testSomethingRetry2(final ITestContext testContext ,final CustomTestDataItem testData) throws CustomTestException{
testSomething(testContext , testData);
}
customTestDataProvider knows which testData item the method has failed for so in testSomethingRetry1 only the failed test data will be supplied
If a test method fails in testSomething it is retried in testSomethingRetry1 but testNG considers it is failed since it failed in testSomething
So I need a custom logic to determine if the suite has passed or failed. How do i override the testNG result( pass/fail) with the result I have determined ?
Instead of duplicating test methods I would recommend to use org.testng.IRetryAnalyzer which basically runs failed test again. You can see some example here http://seleniumeasy.com/testng-tutorials/execute-only-failed-test-cases-using-iretryanalyzer.
But if you really want to override result you can use listeners and implement methods in which you get ITestResult. In this object you can check method class/name/result/etc. and change some of these attributes (including result).
http://testng.org/javadocs/org/testng/ITestListener.html
http://testng.org/javadocs/org/testng/IInvokedMethodListener.html
or for whole test suite
http://testng.org/javadocs/org/testng/ISuiteListener.html
So I have Selenium tests that use TestNG to test UI, but right now I'm struggling with a problem : how can I execute a specific method only once before a parameterized test ?
I have a parameterized test like this :
#Test(dataProvider="myProvider")
public void testMyThingToTest() {
// Things to test in here
}
My parameterized test works great and gets executed once for each parameter given. Now I have a initialization that is common to all my instances of my parameterized test, so I would like it to be ran only once, not before every instance of the parameterized test...
If I do something like using #BeforeTest or #BeforeMethod :
#BeforeMethod
public void initialization() {
// The initialization phase here...
}
The init method is run before every test... any idea how to achieve that ?
The purpose of this is to avoid doing some overwhelming job for each parameter, because my initialization takes something like 30 sec, and I have 7-8 parameters for my test, so I would rather have an initialization time of 30sec * 1 rather than 30sec * 8
If I understood correctly, you want to execute a method only once for all tests that are in you test class.
TestNG provide an annotation for that #BeforeClass: Annotates methods that will be run before the first method on the current test class is run.
You can find an example at http://java.dzone.com/articles/testng-beforeclass-annotation
Try
#BeforeSuite
public void initialization() {
// Init suite
}
If you want other scenarios. Pick annotation with following priority:
#BeforeSuite > #BeforeTest > #BeforeClass > #BeforeMethod
call function in dataprovider
#DataProvider
public Object[][] FeederOfResponseCodes()
{
initialization();
return new Object[][]
{
};
}
Is there any way I can get the test result in the teardown (#After) method? I'd like to do clean up after the tests depending on the result.
Could not find much details about #After in the junit docs.
The closest thing to what you're asking for would probably be the TestWatcher rule. That won't give you access to a returned result or anything, but you can use it (or create your own TestRule and combined with the Description object, you could annotate your methods differently to indicate what sort of clean-up is necessary.
Yes, if you use TestNG, it is a standard function, your #After method can look like this:
#AfterTest
public void cleanUp( ITestResult result ) {
boolean success = result.isSuccess();
....
If there is no standard possibility (I'm pretty sure there was no possibility in JUnit 3.x), you can just
write a Listener,
push the Listener-events to a static Collection,
and gather them from your #After- Method.
Why not set the result of a test in a class member and then act on it in the #After method?
public enum TestResult {
...
}
public class TestClass {
private TestResult result;
...
#Test
public void aTest() {
// set up test
// call class under test
// assert something and set result based upon outcome
this.result = ...;
}
...
#After
public void teardown() {
// clean up based upon this.result
}
}
I suspect you would not have too many different results and a finite set will suffice.
I am using something alike JamesB suggested. You might get to the point where you have to add timeouts to the tests, then =>>
"setting the result of a test in a class member and then act on it in the #After method" would not always work if you have more than 1 assert. That's is my problem today, when i have testCaces that timeout, but my afterClass is assuming everything went smooth because the most recent assert has passed..
The method onFinish(ITestContext context) is invoked after all the tests and configuration methods have run.
I'm searching an equivalent method which is invoked after a single test and all of it's configuration methods?
Guess you are looking for IInvokedMethodListener..
A method annotated with Junit's #after will be run after each method is run, you can also use #afterClass to be run after the who unit test class. For example
#Before
public final void setUp() { //Set up some data to use }
#After
public final void tearDown() { //remove the data }
According to Dependency injection section of TestNG documentation, you can specify a parameter of type java.lang.reflect.Method:
Any #BeforeMethod (and #AfterMethod) can declare a parameter of type
java.lang.reflect.Method. This parameter will receive the test method
that will be called once this #BeforeMethod finishes (or after the
method as run for #AfterMethod).