I'm quite new to WebDriver and TestNG framework. I've started with a project that does a regression test of an e-commerce website. I'm done with the login and registration and so on. But there is something that I don't quite understand.
Example, I have this easy code that searches for a product.
driver.get(url + "/k/k.aspx");
driver.findElement(By.id("q")).clear();
driver.findElement(By.id("q")).sendKeys("xxxx"); //TODO: Make this dynamic
driver.findElement(By.cssSelector("input.submit")).click();
Now I want to check if xxxx is represented on the page. This can be done with
webdriver.findElement(By.cssSelector("BODY")).getText().matches("^[\\s\\S]*xxxxxx[\\s\\S]*$")
I store this in a Boolean and check if its true or false.
Now to the question, based on this Boolean value I want to say that the test result is success or fail. How can I do that? What triggers a testNG test to fail?
TestNG or any other testing tool decides success or failure of a test based on assertion.
Assert.assertEquals(actualVal, expectedVal);
So if actualVal and expectedVal are same then test will pass else it will fail.
Similarly you will find other assertion options if you using any IDE like Eclipse.
If you want to stop your test execution based on the verification of that text value, then you can use Asserts. However, if you want to log the outcome of the test as a failure and carry on, you should try using soft assertions, which log the verification as passed or failed and continue with the test. Latest Testng comes equipped to handle this - info at Cedric's blog
write this code where your if condition fails
throw new RuntimeException("XXXX not found: ");
u can use throw exception, and each method which will cal this meth should also throw Excetion after method name or you can use try catch. sample:
protected Boolean AssertIsCorrectURL(String exedctedURL) throws Exception {
String errMsg = String.format("Actual URL page: '%s'. Expected URL page: '%s'",
this.driver.getCurrentUrl(), exedctedURL);
throw new Exception(errMsg);
}
You can do this.
boolean result = webdriver.findElement(By.cssSelector("BODY")).getText().matches("^[\s\S]xxxxxx[\s\S]$")
Assert.assertTrue(result);
Related
I know it is possible to delete the entire Jmeter result tree, through the code:
import org.apache.jmeter.gui.GuiPackage;
import org.apache.jmeter.gui.JMeterGUIComponent;
import org.apache.jmeter.gui.tree.JMeterTreeNode;
import org.apache.jmeter.samplers.Clearable;
log.info("Clearing All ...");
guiPackage = GuiPackage.getInstance();
guiPackage.getMainFrame().clearData();
for (JMeterTreeNode node : guiPackage.getTreeModel().getNodesOfType(Clearable.class)) {
JMeterGUIComponent guiComp = guiPackage.getGui(node.getTestElement());
if (guiComp instanceof Clearable){
Clearable item = (Clearable) guiComp;
try {
item.clearData();
} catch (Exception ex) {
log.error("Can't clear: "+node+" "+guiComp, ex);
}
}
}
but I didn't want to delete the entire result tree, only the returns that returned with status == 500. Because my api returns 500 until the callback is available for consultation, when it finds the callback it returns "success", so while the api keeps retrying, these retries show up as an error in the report,but in fact the callback has not returned yet, when it returns the api returns the callback and is successful. I would like to remove these requests from the report, which are retry.
Add a JSR223 Post Processor with following code to ignore the test results when the response code is 500
if (prev.getResponseCode()=="500"){
prev.setIgnore()
}
prev - (SampleResult) - gives access to the previous SampleResult (if any)
API documentation for prev variable (SampleResult)
I don't think it's possible without modifying JMeter source code or heavily using reflection. In any case the answer will not fit here.
In general:
You should be using JMeter GUI only for tests development and debugging, when it comes to test execution you should run your test in command-line non-GUI mode
You should not be using listeners during test execution as they don't add any value, just consume valuable resources, all the necessary information is stored in .jtl results file
There is Filter Results plugin which allows removing the "unwanted" data from the .jtl results file
You can also (as well) generate the HTML Reporting Dashboard out of the .jtl results file, the Dashboard has its own responses filtering facilities
guys. First, thanks for the answers, I ended up giving up on the idea of removing the result from the tree and looking for a way to modify the way that result is presented.
I made a beanShell assertion, so that if Status==500, it would return "Success" in the result tree:
BeanShell
I also made it so that if it were a new attempt the name displayed in the results tree would indicate this, leaving the api name mutable depending on the return:
api name = variable
and I have this logic:
import org.apache.jmeter.samplers.SampleResult;
//process main sample
if (${status} == 500) {
SampleResult.setResponseCodeOK();
SampleResult.setSuccessful(true);
vars.put("Api_Fake_Client_name", "API_FAKE_CLIENT_RETRY");
I will configure the answers in the other conditions, but I believe that this way I will be able to solve my problem because the new attempts no longer appear as an error.
Is it not possible to have both SUCCCESS and FAILURE Outcome in the story?
Lifecycle:
After:
Outcome: FAILURE
Given I capture page screenshot
Given I close browser
Outcome: SUCCESS
Given I close browser
Scenario: Sample one
Given I open browser
When I do something
Scenario: Sample two
Given I open browser
When I do another thing
For example, for failures I want to take a screenshot before closing the browser. If successful I just want to close the browser.
I know I can just close the browser at the end of all my scenarios and only have the failure outcome remain. I would like to know if there is a way to do this in the Lifecycle After.
Thanks.
Quoting OP:
For example, for failures I want to take a screenshot before closing the browser. If successful I just want to close the browser.
Now, the interesting question will be which framework are you using for your assertions?
I'll assume you use Junit which comes bundled with JBehave as JBehave relies on knowing there is an error by JUnit's thrown exception.
The idea is to:
a) throw an exception when an error occurs (so need to check on every step)
b) take a screenshot
c) continue with testing (i.e. closing the browser)
So in order to throw an exception, you really do no need to do much as this is done automatically when using JUnit's Assert statement.
so for example
Assert(username.equals("expected_user").isTrue();
If the above fails an exception will be thrown.
You can capture it as such:
public class RunnerExtension implements AfterTestExecutionCallback {
#Override
public void afterTestExecution(ExtensionContext context) throws Exception {
Boolean testResult = context.getExecutionException().isPresent();
System.out.println(testResult); //false - SUCCESS, true - FAILED
}
}
#ExtendWith(RunnerExtension.class)
public abstract class Tests {
}
Taken from this answer:
JUnit5 - How to get test result in AfterTestExecutionCallback
So basically you override the standard behaviour -after- each assertion has been executed. In the case above you can add (when an exception is thrown --> take screenshot).
Here is the take a screenshot code for Selenium-Java:
WebDriver driver = new FirefoxDriver();
driver.get("http://www.google.com/");
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
// Now you can do whatever you need to do with it, for example copy somewhere
FileUtils.copyFile(scrFile, new File("c:\\tmp\\screenshot.png"));
Hope the above helps!
I have a PageObject startPage where I have a login and a logout method. The login method works fine and is executed in the #BeforeScenario:
#BeforeScenario
public void login() {
// {..} Declaration of baseUrl,user,password...
homeVM.setDefaultBaseUrl(baseUrl);
homeVM.open();
homeVM.login(user, password);
}
and login(user,password) in class homeVM is like:
typeInto(find(By.id(getUserFieldId())), user);
typeInto(find(By.id(getPasswordFieldId())), password);
findBy(getLoginButtonXPath()).then().click();
so nothing special, this works all fine.
Then I switch through several PageObjects in different test steps without a problem. When code reaches the #AfterScenario which looks like:
#AfterScenario
public void logout() {
homeVM.logoff();
}
and class homeVM with method logoff() looks like:
WebElement btnLogout = getDriver().findElement(By.xpath("//a [contains(#class,'lnkLogout')]"));
btnLogout.click();
But this isn't working (nothing happens, no exception, no click.. just nothing). Then I tried to log some information about getDriver() with:
System.out.println("WindowHandles:"+getDriver().getWindowHandles().size());
System.out.println("Title: "+getDriver().getTitle());
and both values are just empty (""). So it seems that getDriver() is just empty (not even null, so I don't get a NullPointerException). Why is it so? I tried to check getDriver() for the last PageObject I used in my test but there I get all the information I need, just getDriver() in the #AfterScenario is empty. Any idea or solution what to do next or why this is happening? I'm using chromeDriver.
EDIT:
Okay, I recognized something unexpected:
I have an assertThat(<something>) method in my last step and this step is actually producing an assignment failure (because the behaviour is not implemented yet)... and if I comment this assertThat() out, the #AfterScenario and its logout is executed correctly. So the webDriver gets "emptied" if the test fails? Is this on purpose?
EDIT2:
If I catch the AssertionErrorException Exception the test runs fine again but of course the test will be marked as "Test Passed". So it really has something to do that if the exception is thrown the current webDriver gets emptied. But this seems to be wrong...
Once Serenity (or Thucydides in this case) spots a test failure (e.g. from an assertion error), the test is switched to "dry-run" mode, as it considers that subsequent steps are compromised and may result in unnecessary (and slow) web driver calls.
As I found out from John Smart that once Serenity spots a test failure the test is switched to "dry-run" mode, so no web driver calls are possible anymore I had to find another way to perform a logout.
As my chromedriver runs by default all scenarios in the same session and browser I had to perform a manual logout after every scenario. But by setting
System.setProperty("restart.browser.each.scenario", "true");
it was possible to restart the browser and clean the session after every scenario. This worked for me so I don't need the #AfterScenario with logoff(); anymore.
overcoming the issue in cucumber watir framework
filename = DateTime.now.strftime("%Y-%m-%d--%Hh_%Mm_%Ss")
#browser.driver.save_screenshot ("#{filename}.png")
Note:
filename is the name of the screenshot file
you can pass the location of the screenshot file as well like this
#browser.driver.save_screenshot ("/Screenshots/#{filename}.png")
i have a form filling script
during the execution in any line (Step) if the test case is failed it is directly closing the browser .
ex: while filling the form if it couldn't find a element (Textbox or Checkbox) it throws an exception nosuchelements and directly closes the browser i am using assert (testNG) in my script
what i want is if the test case is failed anywhere middle i want the test case to continue its execution but in the report it should show that the test case is failed
in my code
String Jobvalue = driver.findElement(By.xpath("//label[#for='ctl00_ContentPlaceHolder1_RadPanelBar1_i0_chkColumns_5']")).getText();
Assert.assertEquals(Jobvalue, "Job Value ($)");
Reporter.log(Jobvalue+" Text Verification ---- Passed",true);
Assert will fail the test case if actual value is not equals to expected value
i want an alternate to assert or any code which continues executing the test case till the end and at reports it should show it is failed if the actual value is not equal to expected value
Santhosh, We need more info/code on what are you exactly trying.By assumption, here is my answer for your question.
Use validate() methods rather assert () in your test steps.
Any exception can be easily handled using try-catch.
try{
String Jobvalue = driver.findElement(By.xpath("//label[#for='ctl00_ContentPlaceHolder1_RadPanelBar1_i0_chkColumns_5']")).getText();
Assert.assertEquals(Jobvalue, "Job Value ($)");
}
catch()
{
//Write your code here about failing the test
}
If an Assert fails, it throws an AssertionError and that's why your program is stopped. You can use try/catch to handle the exception.
try{
Assert......
}catch(AssertionError e){
//handle the exception
}
Refer here for Assert documentation
How can I get the results of my JUnit assertions to be printed [to standard output]?
I have some tests like this:
#Test
public void test01()
{
Position p = getPositionAt('a', 1);
assertNotNull("a1 exists", p);
assertNotNull("figure exists a1", p.getFigure());
p = getPositionAt('a', 2);
assertNotNull("exists a2", p);
assertNull("figure exists a2", p.getFigure());
p = getPositionAt('b', 1);
assertNotNull("exists b1", p);
assertNull("figure exists b1", p.getFigure());
}
This is the printed output format I am hoping to get:
a1 exists -success
figure exists a1 -success
exists a2 -success
figure exists a2 -succcess
exists b1 -succcess
figure exists b1 -failed
Is there way to do this using runners and suites? Or does there exist any assertSuccess(), assertFailed() methods?
First, you have two issues not one. When an assertion fails, an AssertionError exception is thrown. This prevents any assertion past this point from being checked. To address this you need to use an ErrorCollector.
Second, I do not believe there is any way built in to JUnit to do this. However, you could implement your own methods that wrap the assertions:
public static void assertNotNull(String description, Object object){
try{
Assert.assertNotNull(description, object);
System.out.println(description + " - passed");
}catch(AssertionError e){
System.out.println(description + " - failed");
throw e;
}
}
All the assertXXX methods have a form that allows for displaying a String on error:
assertNotNull("exists a2", p); // prints "exists a2" if p is null
There is no particular value in printing a message on success.
EDIT
Junit typically provides 2 forms of an assert. To follow the example above, you can test for a null value in 1 of 2 ways:
assertNotNull(p)
or
assertNotNull("my message on failure", p)
The framework will print the error messages with no other effort required by you (it's provided by the framework).
To test for exceptions you would use the following pattern:
try{
someCall();
catch(Exception e){
fail(): // exception shouldn't happen, use assertTrue(true) if it should
}
Again, there are versions of these methods for adding a message
Check the API
One last resort option is to pair each assert with a corresponding System.out.println, though obviously that is less than ideal. Still, it will solve the problem if all else fails.
Existing Answers/Comments here contain enough info to understand how to print something based on JUnit assertions - but they also explain how doing so is probably not what you actually want to do, and is probably missing the point of running unit tests in the first place.
You should be viewing the results of the tests themselves, instead of trying to print something while you don't understand how/where to view test results themselves.
Now then how/where to view results themselves depends on how you are running your tests - you need to understand how you are running your tests, and then research how to view test results according to how you are running them. Here are a few (but not limited to) examples:
Running tests in IntelliJ
Running tests in Eclipse
Running tests on command line
Running tests in Jenkins