We Keep Coding

How to Deal with ElementNotFound Exception in Selenium WebDriver

Overview : I have already prepared the automation script in Selenium Web driver script in Java which will login into a website and make the selections automatically and once the selections are completed it will run the report.
What I want : I am facing issue while optimizing my automation script.
Brief Explanation : Actually I am familiar with different kinds of wait we are using in Selenium but all those wait i.e implicit,explicit or fluent wait didn't able to help me out in making the code more optimized.Currently I am using Thread.sleep() method everywhere in order to run the script completely without any fail but I know this should not be best practice to be get followed because sometime Elements loads fast and sometime slow because of that either my Script execution took long time or failed based on Element availability.I created one separate method for Webdriver wait and Which I can call for various webelements whenever I needed in my main script but this approach also sometime works or sometime not even though I am passing 800 Second as Time period to wait but if I use Thread.sleep(5000) it will work without any issue not sure why ??
What I want to be have a separate method for wait which can be called in main script whenever required and I want my script to be worked flawless the moment webeelment visible just like what we human do when we interacting with any web.
Note : I have tried ExpectedCondition methods like elementtobeclickable, visibilityOfElementLocated,presenceofElementLocated all of them sometime these work but sometime won't.
Separate Method of wait I have created
public static WebElement waiting(WebDriver driver,String path){
WebDriverWait wait = new WebDriverWait(driver,800);
WebElement element=wait.until(ExpectedConditions.presenceOfElementLocated(By.xpath(path)));
return element;
}
Piece of main code where I am calling this Method.
if(nam.equals("Some name"))
{
WebElement e=driver.findElement(By.xpath("1st Webelement path"));
e.click();
System.out.println("Value clicked under First Drop Down is:"+e);
Listing.waiting(driver,"2nd WebElement xpath").click();
//Thread.sleep(5000);
//driver.findElement(By.xpath("2nd WebElement xpath")).click();
System.out.println("Second Dropdown clicked");
}
When I commenting the Thread.sleep() then it will throw the ElementNotFound exception even though I have used 800 Seconds in Webdriver wait method but the moment I removed the comment from Thread.sleep() method it will work.
Kindly Help me in getting the reusable and useful wait method which I can call several times in my main code.
Thanks in Advance !!

This usually works for me (not FluentWait though):
WebDriverWait wait = new WebDriverWait(driver, TIMEOUT);
ExpectedCondition elementIsDisplayed = new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver arg0) {
try {
webElement.isDisplayed();
return true;
}
catch (NoSuchElementException e ) {
return false;
}
catch (StaleElementReferenceException f) {
return false;
}
}
};
wait.until(elementIsDisplayed);
Of course, setup TIMEOUT with the amount of time you want to wait for the element to be found (that is in seconds).

Going through Chrome://settings by Selenium

I am using Java and Selenium to write a test, I use the code below to get into Chrome:setting
driverChrome.manage().window().maximize();
driverChrome.get("chrome://settings");
But when the page is open I can not find any of its web Elements, for example when I try to find "show advanced setting...." by this code
driverChrome.findElement(By.xpath("//a[#id='advanced-settings-expander']")).click();
it throws an error saying that "no such element: Unable to locate element"
I tried to located other elements, but they all failed. I saw this post here but it did not help.
Find the code below:
driverChrome.manage().window().maximize();
driverChrome.get("chrome://settings");
Thread.sleep(5000);
WebElement w = driverChrome.findElement(By
.xpath("//iframe[#name='settings']"));
driverChrome = driverChrome.switchTo().frame(w);
Thread.sleep(1000);
while (true) {
try {
WebElement we = w.findElement(By
.xpath("//a[text()='Show advanced settings...']"));
if (we.isDisplayed()) {
we.click();
Thread.sleep(1000);
break;
}
} catch (Exception e) {
System.out.println(e.getMessage());
System.out.println("=========================");
}
}

I haven't tested this but I took your code snippet and cleaned it up a bit. Try this and see if it works. This should be pretty close.
Once you switch to the IFRAME context, you don't need to reference the IFRAME as you did with w.findElement().
In general, Thread.sleep() is not a good practice. You should prefer to use WebDriverWait with ExpectedConditions. Check the docs for all the different things you can wait for using ExpectedConditions. I used .elementToBeClickable() in my code below. This is perfect since you want to click an element. The .until() returns the element waited for so you can just append .click() on the end of the statement... or you can store the element in a WebElement variable and use it elsewhere.
driverChrome.manage().window().maximize();
driverChrome.get("chrome://settings");
WebElement w = driverChrome.findElement(By.xpath("//iframe[#name='settings']"));
driverChrome = driverChrome.switchTo().frame(w);
WebDriverWait wait = new WebDriverWait(driverChrome, 10);
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//a[text()='Show advanced settings...']"))).click();
// alternative example... store returned element and then click on a separate line... or use the variable elsewhere, etc.
// WebElement link = wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//a[text()='Show advanced settings...']")));
// link.click();

You will need to identify Shadow roots to interact with a lot of Chromes native pages including the Settings page.
See this Thread on how to work with them properly:
How to interact with the elements within #shadow-root (open) while Clearing Browsing Data of Chrome Browser using cssSelector

element is not attached to the page document

While clicking on javascript components Im getting:
org.openqa.selenium.StaleElementReferenceException: stale element
reference: element is not attached to the page document
One way to fix it is to stop script for some short period of time with :
Thread.sleep(200);
I have my implicitlyWait set for 10 seconds, and I think there wasnt such problems with older selenium
But is there any other way to do it globally ?
driver.findElement(By.cssSelector("button.btn.btn-default")).click();
driver.findElement(By.xpath("//div[#id='content']/div/form")).click();
driver.findElement(By.linkText("Order")).click();
But in the middle of this i have to put sleep's to make it work:
driver.findElement(By.cssSelector("button.btn.btn-default")).click();
Thread.sleep(200);
driver.findElement(By.xpath("//div[#id='content']/div/form")).click();
Thread.sleep(200);
driver.findElement(By.linkText("Order")).click();

Your problems are indicator that you need to construct the test cases better and/or lack of understanding of how the website you are automating works. Especially
stop script for some short period of time with : Thread.sleep(200);
is considered a really bad practice. Whatever you do don't mix implicit and explicit waits, things will start going wrong, explicit waits are the recommended solution.
Waiting for the page to be loaded won't work if (as it seems to be your case) the page is being modified by AJAX operations. Instead of waiting for the page to load, wait for the condition you are testing to become true. This way, you give the AJAX operation time to execute and if your there is a problem you will get an error when the time out occurs.
StaleElementReferenceException is caused by the DOM being refreshed
after you found an element. Remember that a WebElement is a reference
to a specific element on the page, if the DOM get's refreshed this
reference is broken and you need to find the element again to be able to
interact with it.

In my example replace getWebDriver() with your drive instance.
The best practice is to first assert & verify that, that particular
element is present or not. if you see function
assertAndVerifyElement() --- it continuously checks for element for 5
secs and then assert it accordingly.
package com.stackoverflow;
import org.openqa.selenium.By;
import org.testng.Assert;
import org.testng.annotations.Test;
import com.services.Init;
public class Issue3 extends Init {
#Test
public void solutionOfIssue() throws InterruptedException {
/*
* The best thing is to first assert & verify that, that particular
* element is present or not. if you see function
* assertAndVerifyElement() --- it continuously checks for element for 5
* secs and then assert it accordingly.
*/
assertAndVerifyElement(By.cssSelector("button.btn.btn-default"));
getWebDriver().findElement(By.cssSelector("button.btn.btn-default")).click();
assertAndVerifyElement(By.xpath("//div[#id='content']/div/form"));
getWebDriver().findElement(By.xpath("//div[#id='content']/div/form")).click();
assertAndVerifyElement(By.linkText("Order"));
getWebDriver().findElement(By.linkText("Order")).click();
}
public void assertAndVerifyElement(By element) throws InterruptedException {
boolean isPresent = false;
for (int i = 0; i < 5; i++) {
try {
if (getWebDriver().findElement(element) != null) {
isPresent = true;
break;
}
} catch (Exception e) {
// System.out.println(e.getLocalizedMessage());
Thread.sleep(1000);
}
}
Assert.assertTrue(isPresent, "\"" + element + "\" is not present.");
}
}
Hope this will work for you. :)

In order to avoid that, you should locate the element again. After you aquired the element, the reference of its Java object might become stale. Do something like:
WebElement element = WebDriver.findElement(By by) again.
EDIT: For your example try this
driver.findElement(By.cssSelector("button.btn.btn-default")).click();
//30s timeout, use timeout not sleep
WebDriverWait wait = new WebDriverWait(driver, 30);
By xpath = By.xpath("//div[#id='content']/div/form")
//wait for element to be clickable, then click
WebElement element = wait.until(ExpectedConditions.elementToBeClickable(xpath));
element.click();

Error "element is not attached to the page document" display when the element is not exist in current page anymore. It happen because you got element for page 1, and used it in page 1 (or the page was refresh after getting the element, HTML was change by AJAX)

Selenium : Handling Loading screens obscuring the web elements. (Java)

I'm writing an automated test case for a web page. Here's my scenario.
I have to click and type on various web elements in an html form. But, sometimes while typing on a text field, an ajax loading image appears , fogging all elements i want to interact with. So, I'm using web-driver wait before clicking on the actual elements like below,
WebdriverWait innerwait=new WebDriverWait(driver,30);
innerwait.until(ExpectedConditions.elementToBeClickable(By.xpath(fieldID)));
driver.findelement(By.xpath(fieldID)).click();
But the wait function returns the element even if it is fogged by another image and is not clickable. But the click() throws an exception as
Element is not clickable at point (586.5, 278).
Other element would receive the click: <div>Loading image</div>
Do I have to check every time if the loading image appeared before interacting with any elements?.(I can't predict when the loading image will appear and fog all elements.)
Is there any efficient way to handle this?
Currently I'm using the following function to wait till the loading image disappears,
public void wait_for_ajax_loading() throws Exception
{
try{
Thread.sleep(2000);
if(selenium.isElementPresent("id=loadingPanel"))
while(selenium.isElementPresent("id=loadingPanel")&&selenium.isVisible("id=loadingPanel"))//wait till the loading screen disappears
{
Thread.sleep(2000);
System.out.println("Loading....");
}}
catch(Exception e){
Logger.logPrint("Exception in wait_for_ajax_loading() "+e);
Logger.failedReport(report, e);
driver.quit();
System.exit(0);
}
}
But I don't know exactly when to call the above function, calling it at a wrong time will fail. Is there any efficient way to check if an element is actually clickable? or the loading image is present?
Thanks..

Given the circumstances that you describe, you are forced to verify one of two conditions:
Is the element that you want to click clickable?
Is the reason that blocks the clicks still present?
Normally, if the WebDriver is able to find the element and it is visible, then it is clickable too. Knowing the posible reasons that might block it, I would rather choose to verify those reasons. Besides, it would be more expressive in the test code: you clearly see what you are waiting for, what you are checking before clicking the element, instead of checking the "clickability" with no visible reason for it. I think it gives one (who reads the test) a better understanding of what is (or could be) actually going on.
Try using this method to check that the loading image is not present:
// suppose this is your WebDriver instance
WebDriver yourDriver = new RemoteWebDriver(your_hub_url, your_desired_capabilities);
......
// elementId would be 'loadingPanel'
boolean isElementNotDisplayed(final String elementId, final int timeoutInSeconds) {
try {
ExpectedCondition condition = new ExpectedCondition<Boolean>() {
#Override
public Boolean apply(final WebDriver webDriver) {
WebElement element = webDriver.findElement(By.id(elementId));
return !element.isDisplayed();
}
};
Wait w = new WebDriverWait(yourDriver, timeoutInSeconds);
w.until(condition);
} catch (Exception ex) {
// if it gets here it is because the element is still displayed after timeoutInSeconds
// insert code most suitable for you
}
return true;
}
Perhaps you will have to adjust it a bit to your code (e.g. finding the element once when the page loads and only checking if it is displayed).
If you are not sure when exactly the loading image comes up (though I suppose you do), you should call this method before every click on elements that would become "unclickable" due to the loading image. If the loading image is present, the method will return true as soon as it disappears; if it doesn't disappear within 'timeoutInSeconds' amount of time, the method will do what you choose (e.g. throw an exception with specific message).
You could wrap it together like:
void clickSkippingLoadingPanel(final WebElement elementToClick) {
if (isElementNotDisplayed('loadingPanel', 10)) {
elementToClick.click();
}
}
Hope it helps.

Handling self-refreshing pages from selenium

I have been running into intermittent errors with some java selenium-rc tests which I think are related to a page which has an ajax poll and automatically refreshes when some condition is reached on the server. In this scenario, I have no way of asking selenium to wait for the page to load, and so I run into a bunch of random "Couldn't access document.body" errors.
So, is there some way I can cause selenium to gracefully handle this situation? If not, is there some way I could detect whether the user is selenium from the page's javascript, and disable the automatic refresh?
If it helps at all, the javascript code in the page looks something like...
var ajax = new Ajax(url, {
update: state,
method: 'get',
onComplete: function(message) {
if (some_condition) {
window.location.replace(unescape(window.location));
}
}
});

One solution might be to always use a waitForCondition using isElementPresent before attempting to interact with the application under test. You could put the following method in a superclass to keep your tests more readable. Alternatively you could create helper methods for common Selenium commands that perform this wait.
/** Waits for an element to be present */
public static void waitForElementToBePresent(String locator) {
session().waitForCondition("var value = selenium.isElementPresent('" + locator.replace("'", "\\'") + "'); value == true", "60000");
}
You may also want to wait for the element to be visible, as waiting for it to just be present isn't always enough (imagine a textbox that is always present but hidden until a certain condition). You can combine this with the above method:
/** Waits for an element to be visible */
public static void waitForElementToBeVisible(String locator) {
waitForElementToBePresent(locator);
session().waitForCondition("var value = selenium.isVisible('" + locator.replace("'", "\\'") + "'); value == true", TIMEOUT);
}
Incidentally, the WebDriver (Selenium 2) team are working on having implicit waits, specifically to address AJAX issues where elements are not present immediately.

My solution was to disable the refresh in javascript by wrapping it in something like the following...
var isSeleniumActive = parent.seleniumAlert;
if (isSeleniumActive) {
alert("Selenium");
} else {
alert("Not selenium");
}
I'm not sure if the seleniumAlert function here is likely to sick around forever, so be aware if you're taking this that you may be relying on internal selenium implementation details of selenium.

There i was facing the same problem and i use a single line of code and it helps.
i was getting the error about the page is getting auto refresh
plus this warning:
-1490472087141 Marionette WARN Using deprecated data structure for setting timeouts
all i use is
Thread.sleep(2000)
and it worked for me.

I think that you can pause, or use a click and wait. There are a few good articles on the google. Good luck.
Edit for your comment:
How about the waitFor command?