How to programmatically run all test cases forever ? Below is the setup I have … now I have to run this code infinitely …another requirement is I need to restart the app again after #aftersuite… I need to tear down the test cases because I need to produce report after each aftersuite …
For example :
public class SimpleTest extends TestBase
{
AppiumDriver driver;
#BeforeSuite
public void setUp() throws MalformedURLException {
// DesiredCapabilities and all setup
}
#Test(priority = 1)
public void testcase1()throws InterruptedException {
login();
}
#Test(priority = 2)
public void testcase2() throws InterruptedException {
//something
}
#Test(priority = 3)
public void testcase3() throws InterruptedException {
//something
}
#Test(priority = 4)
public void testcase4 throws InterruptedException {
//something
}
#Test(priority = 5)
public void testcase() throws InterruptedException{
//something
}
#Test(priority = 6)
public void testcase6() throws InterruptedException{
//something
}
#Test(priority = 7)
public void testcase_logout() throws InterruptedException {
logout();
}
#AfterSuite
public void testCaseTearDown()
{
driver.quit();
}
}
Create another class that would call these tests...
public SimpleTest {
public static void main(String[] args) {
while (true) {
org.junit.runner.JUnitCore.main("SimpleTest");
}
}
}
Arrange your tests in testNG.xml .
<?xml version="1.0" encoding="UTF-8"?>
<suite name="Regression suite" verbose="1" >
<test name="SampleTests" >
<classes>
<class name="com.first.example.SampleTest"/>
</classes>
</test>
</suite>
Then programmatically run them with TestNG object.
import org.testng.TestNG;
import java.util.List;
public class RunTestNGXML {
public static void main(String[] args) {
TestNG testng = new TestNG();
List<String> suites = Lists.newArrayList();
suites.add("testng.xml");
testng.setTestSuites(suites);
String outputDir ="src"+File.separator+"reports";
Calendar calendar = Calendar.getInstance();
SimpleDateFormat formatter = new SimpleDateFormat("dd_MM_yyyy_hh_mm_ss");
while(true){
testng.setOutputDirectory(outputDir+File.separator+formater.format(calendar.getTime()));
testng.run();
}
}
}
The caveat with infinite loop is that your emailable-report would be overwritten in each iteration. So configure the test-output directory path with timestamps. Otherwise log4j appender can easily do the job of continuous logging.
Related
Webdriver: close web driver only if test is successful.
Is there a way to check test results in #After method?
public class FooTest {
private WebDriver webDriver;
#Test
public void testFoo() {
}
#After
public void cleanUp() {
if (isTestSuccess()) { // How?
webDriver.close();
}
}
}
If you are using Junit, you can pass the Scenario parameter to the tear down method:
#After
public void cleanUp(Scenario scenario) {
if (!scenario.isFailed()) {
webDriver.close();
}
}
I ran the following test class and it reported that the first test took 2 seconds and the second took 4 seconds. I expected JUnit to output the time for each test case without putting into account the time of the #BeforeClass or #AfterClass method.
But obviously it includes the time of the #AfterClass method into only the last test case, which doesn't make sense at all if you ask me.
Is there a way make JUnit output the times only for the time taked for each test case ?
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
public class TimeTest {
#Test
public void test1() throws InterruptedException {
Thread.sleep(2000);
}
#Test
public void test2() throws InterruptedException {
Thread.sleep(2000);
}
#AfterClass
public static void afterClass() throws InterruptedException {
Thread.sleep(2000);
}
}
#BeforeClass & #AfterClass run ones for all tests
what you should do is to use #Before, and After instead, with the help of an instance variable
public class TimeTest {
long timeInMillis=0;
#Before
public void before(){
timeInMillis = System.currentTimeMillis();
}
#Test
public void test1() throws InterruptedException {
Thread.sleep(2000);
}
#Test
public void test2() throws InterruptedException {
Thread.sleep(2000);
}
#After
public static void after() throws InterruptedException {
long diff = System.currentTimeMillis() - timeInMillis;
// log the diff or cast it to seconds
}
}
You are not the first one with such a problem. There are a lot of examples of time logging on the internet.
I suggest you to use JUnit #Rules to solve your problem and avoid a code-duplication. (You don't want to write timeInMillis = System.currentTimeMillis() in each test class, do you?)
There is an example of rule implementation
I'm running into a strange JUnit and Selenium issue.
I want to write a screenshot taker that will take a screenshot on failed tests.
I define the screenshot taker class as:
public class SeleniumJUnitScreenshotTaker implements MethodRule
{
private WebDriver driverLocal;
public SeleniumJUnitScreenshotTaker(WebDriver driver)
{
driverLocal = driver;
}
public void takeScreenshot(String fileName)
{
try
{
new File("test-results/scrshots/").mkdirs(); // Make sure directory is there
FileOutputStream out = new FileOutputStream("test-results/scrshots/" + fileName + ".png");
out.write(((TakesScreenshot) driverLocal).getScreenshotAs(OutputType.BYTES));
out.close();
}
catch (Exception e)
{
// No need to crash the tests here
}
}
public Statement apply(final Statement statement, final FrameworkMethod frameworkMethod, final Object o)
{
return new Statement()
{
#Override
public void evaluate() throws Throwable
{
try
{
statement.evaluate();
}
catch (Throwable t)
{
takeScreenshot(frameworkMethod.getName());
throw t; // rethrow
}
}
};
}
I've a base test class where I setup the WebDriver as follows:
public class TestBase
{
protected static WebDriver driver;
#Rule
public SeleniumJUnitScreenshotTaker seleniumJUnitScreenshotTaker = new SeleniumJUnitScreenshotTaker(driver);
.
.
.
#BeforeClass
public static void intialize()
{
.
.
driver = SeleniumWebDriverFactory.getWebDriver(browser.toString());
if ((browser != Browser.Chrome) && (browser != Browser.ChromeCanary))
{
driver.manage().window().maximize();
}
hostUrl = Configuration.getHostUrl(envUnderTest);
// Open Browser and navigate to main page
driver.navigate().to(hostUrl);
}
#Before
protected void setup() throws Exception
{
this.users = getUsers();
}
#After
protected void teardown()
{
this.testCases.cleanupChecks();
}
#AfterClass
public static void terminate()
{
// Shut down WebDriver
if (driver != null)
driver.quit();
SeleniumWebDriverFactory.stopDriverService();
}
}
The problem is when I try to run a series of tests in the test class (derived from TestBase), the tests after the first failure throws TimeoutException. This is because we don't close the current browser between tests and the new test tries to find things that are absent in current browser.
Is there a way to close browser in between tests but not close the driver? Or reinstantiate the driver between tests? Else is there an alt. design that can make this work?
Thanks
Yana
#BeforeClass does not start my tests in Webdriver, Java, and I don't know where do I go wrong
#BeforeClass
public static void setup() {
driver = new FirefoxDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get(baseUrl + "login");
driver.findElement(By.id("username")).sendKeys("myUserName");
driver.findElement(By.id("password")).sendKeys("myPassword");
driver.findElement(By.id("loginBTN")).click();
}
After the code I start the regular testing:
#Test
public void firstTest() {
//myTestCode
}
After attempting to run, all tests fail, the webdriver does not start, etc...
It would be nice to have this since I have to test a page where I have to be logged in (with #Before the webdriver starts before each test, so obviously I would need the #BeforeClass for this one.)
#BeforeClass
public static void setup() {
//This needs to be here for this to run and having this here means its only local to this method
Webdriver driver;
driver = new FirefoxDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get(baseUrl + "login");
driver.findElement(By.id("username")).sendKeys("myUserName");
driver.findElement(By.id("password")).sendKeys("myPassword");
driver.findElement(By.id("loginBTN")).click();
}
Then your Test will work
#Test
public void firstTest() {
//myTestCode
}
Sample Code : Hopes this will work.
public class OpenBrowsers {
WebDriver driver = null;
#BeforeClass
public void beforeClass() {
System.out.println("beforeClass");
driver = new FirefoxDriver();
}
#Test
public void openGoogle() {
System.out.println("openGoogle");
driver.get("www.google.com");
}
#Test
public void openYahoo() {
System.out.println("openYahoo");
driver.get("www.yahoo.com");
}
#AfterClass
public void afterClass() {
driver.close();
System.out.println("afterClass");
}}
I'm trying to improve an existing system of automated Selenium test.
My goal is to repeat the tests that fails because of connections problem.
I've found and tried to follow this thread How to Re-run failed JUnit tests immediately? that revealed itself quite useful.
In my case the suite is composed by classes, so I've tried to substitute #Rule with #ClassRule, in order to repeat for each try also the #Before and #After parts.
I'm sorry for my ignorance, but where am I supposed to place this rule? In my Suite class? Or in the Classes representing the test?
I am the original answerer of How to Re-run failed JUnit tests immediately?
If I understand correctly, the problem that you are having is due to the #Before being executed before the code in the RetryRule, and the #After being executed afterwards.
So your current behaviour is something like:
#Before
#Retry
test code
#Retry
#After
But you can implement your #Before and #After as a rule - there is a rule ExternalResource which does exactly that. You would implement #Before and #After as a rule:
#Rule public ExternalResource beforeAfter = new ExternalResource() {
public void before() {
// code that was in #Before
}
public void after() {
// code that was in #After
}
}
Then you don't need the #Before and #After. You can then chain these rules using RuleChain. This forces an order of execution to your rules:
#Rule public RuleChain chain= RuleChain
.outerRule(new LoggingRule("outer rule")
.around(new LoggingRule("middle rule")
.around(new LoggingRule("inner rule");
so your final solution would be something like:
private ExternalResource beforeAfter = ...
private RetryRule retry = ...
#Rule public RuleChain chain = RuleChain
.outerRule(retry)
.around(beforeAfter);
Note that if you are using RuleChain, you no longer need the #Rule annotation on the ExternalResource and RetryRule, but you do on the RuleChain.
Here is my solution based on the one mentionned in the question.
It's a combinaison of a #Rule, FailedRule and a #ClassRule, RetryRule
public class RetryTest
{
public static class FailedRule implements TestRule
{
#Override
public Statement apply(final Statement base, final Description description)
{
return new Statement()
{
#Override
public void evaluate() throws Throwable
{
try
{
base.evaluate();
}
catch (Throwable t)
{
System.out.println(description.getDisplayName() + " failed");
retry.setNotGood();
if (retry.isLastTry())
{
System.out.println("No more retry !");
throw t;
}
else
{
System.out.println("Retrying.");
}
}
}
};
}
}
public static class RetryRule implements TestRule
{
private int retryCount, currentTry;
private boolean allGood = false;
public RetryRule(int retryCount)
{
this.retryCount = retryCount;
this.currentTry = 1;
}
public boolean isLastTry()
{
return currentTry == retryCount;
}
public void setNotGood()
{
allGood = false;
}
public Statement apply(final Statement base, final Description description)
{
return new Statement()
{
#Override
public void evaluate() throws Throwable
{
// implement retry logic here
for (; currentTry <= retryCount && !allGood; currentTry++)
{
allGood = true;
System.out.println("Try #" + currentTry);
base.evaluate();
}
}
};
}
}
#ClassRule
public static RetryRule retry = new RetryRule(3);
#Rule
public FailedRule onFailed = new FailedRule();
#BeforeClass
public static void before()
{
System.out.println("Before...");
}
#AfterClass
public static void after()
{
System.out.println("...After\n");
}
#Test
public void test1()
{
System.out.println("> test1 running");
}
#Test
public void test2()
{
System.out.println("> test2 running");
Object o = null;
o.equals("foo");
}
}
It gives :
Try #1
Before...
> test1 running
> test2 running
test2(RetryTest) failed
Retrying.
...After
Try #2
Before...
> test1 running
> test2 running
test2(RetryTest) failed
Retrying.
...After
Try #3
Before...
> test1 running
> test2 running
test2(RetryTest) failed
No more retry !
...After
If I am commenting the o.equals("foo"); in test2, everything runs fine in the firt try :
Try #1
Before...
> test1 running
> test2 running
...After
You decorate the test name itself with the #After or #Afterclass attributes:
#After
#Test
#Category(SmokeTests.class)
public void testProductPageOnly() throws TimeoutException {
//Some tests here.
}
#Afterclass
public static void SomeTest {
//Some test here.
}
Something to note, #Afterclass will always run; even if you are using a #Beforeclass that throws an exception.
May this can solve problem:
1) Test class should be inherited from junit.framework.TestCase
2) Run your tests with something like this
YourTestClass testClass = new YourTestClass();
TestResult result = testClass.run();
Enumeration<TestFailure> failures = result.failures();
if (result.failureCount() != 0)
{
TestFailure fail = failes.nextElement();
junit.framework.Test test = fail.failedTest();
test.run( result );
}
At the end result will contains last results of test running, so after analyzing what test was failed you can run it again.