I have a page with a dynamic table that periodically updates with new data. What I am trying to do is to click on a button and reload the page let's say every 3 seconds until the element from that table appears.
I know the xpath of the element I want to appear, but I just can't make it work using the FluentWait. I tried configuring as seen in the code below, but when I'm calling the method it keeps clicking on the button extremly fast disregarding the pollingEvery configuration and without giving the page enough time to fully reload itself. Code can be checked out below.
What I am not sure about is the return statement. I don't fully grasp what should it be if I only need to click on a button until that element appears.
What am I missing?
public void clickButtonUntilElementIsDisplayed(WebElement elementToBeClicked, String xPathOfElementToBeDisplayed){
FluentWait<WebDriver> wait = new FluentWait<>(getDriver());
wait.pollingEvery(Duration.ofSeconds(5));
wait.withTimeout(Duration.ofSeconds(200));
wait.ignoring(NoSuchElementException.class);
Function<WebDriver,WebElement> clickUntilAppearance = webDriver -> {
while(webDriver.findElements(By.xpath(xPathOfElementToBeDisplayed)).isEmpty())
click(elementToBeClicked);
return webDriver.findElement(By.xpath(xPathOfElementToBeDisplayed));
};
wait.until(clickUntilAppearance);
}
The class in which this method can be found extends Page
The return type of the Functional Interface need to be changed. There are 2 cases in which the functional interface will keep looping.
Value returned is null.
It threw an Exception mentioned as part of ignoring
Don't use while. Use if condition. For every pollingEvery this Functional Interface would get executed.
Function<WebDriver,WebElement> clickUntilAppearance = webDriver -> {
List<WebElement> tableElements = webDriver.findElements(By.xpath(xPathOfElementToBeDisplayed));
if(tableElements.isEmpty()) {
click(elementToBeClicked);
return null;
} else {
return tableElements.get(0);
}
};
WebDriverWait w = new WebDriverWait(driver, 5);// 5seconds
w.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("element_location")));
I have a method which allows me to get all URLs from the page (and optional - to check is it valid).
But it works only for 1 page, I want to check all the website. Need to make a recursion.
private static FirefoxDriver driver;
public static void main(String[] args) throws Exception {
driver = new FirefoxDriver();
driver.get("https://example.com/");
List<WebElement> allURLs = findAllLinks(driver);
report(allURLs);
// here are my trials for recursion
for (WebElement element : allURLs) {
driver.get(element.getAttribute("href"));
List<WebElement> allUrls = findAllLinks(driver);
report(allUrls);
}
}
public static List findAllLinks(WebDriver driver)
{
List<WebElement> elementList = driver.findElements(By.tagName("a"));
elementList.addAll(driver.findElements(By.tagName("img")));
List finalList = new ArrayList();
for (WebElement element : elementList)
{
if(element.getAttribute("href") != null)
{
finalList.add(element);
}
}
return finalList;
}
public static void report(List<WebElement> allURLs) {
for(WebElement element : allURLs){
System.out.println("URL: " + element.getAttribute("href")+ " returned " + isLinkBroken(new URL(element.getAttribute("href"))));
}
}
See comment "here are my trials for recursion". But it goes through the first page, then again through the first page and that's all.
You're trying to write a web crawler. I am a big fan of code reuse. Which is to say I always look around to see if my project has already been written before I spend the time writing it myself. And there are many versions of web crawlers out there. One written by Marilena Panagiotidou pops up early in a google search. Leaving out the imports, her basic version looks like this.
public class BasicWebCrawler {
private HashSet<String> links;
public BasicWebCrawler() {
links = new HashSet<String>();
}
public void getPageLinks(String URL) {
//4. Check if you have already crawled the URLs
//(we are intentionally not checking for duplicate content in this example)
if (!links.contains(URL)) {
try {
//4. (i) If not add it to the index
if (links.add(URL)) {
System.out.println(URL);
}
//2. Fetch the HTML code
Document document = Jsoup.connect(URL).get();
//3. Parse the HTML to extract links to other URLs
Elements linksOnPage = document.select("a[href]");
//5. For each extracted URL... go back to Step 4.
for (Element page : linksOnPage) {
getPageLinks(page.attr("abs:href"));
}
} catch (IOException e) {
System.err.println("For '" + URL + "': " + e.getMessage());
}
}
}
public static void main(String[] args) {
//1. Pick a URL from the frontier
new BasicWebCrawler().getPageLinks("http://www.mkyong.com/");
}
}
Probably the most important thing to note here is how the recursion works. A recursive method is one that calls itself. Your example above is not recursion. You have a method findAllLinks that you call once on a page, and then once for every link found in the page. Notice how Marilena's getPageLinks method calls itself once for every link it finds in a page at a given URL. And in calling itself it creates a new stack frame and a generates a new set of links from a page and calls itself again once for every link, etc. etc.
Another important thing to note about a recursive function is when does it stop calling itself. In this case Marilena's recursive function keeps calling itself until it can't find any new links. If the page you are crawling links to pages outside its domain, this program could run for a very long time. And, incidentally, what probably happens in that case is where this website got it's name: a StackOverflowError.
Make sure you are not visiting the same URL twice. Add some table where you store already visited URLs. Since every page might be starting with header that is linked to the home page you might end up visiting it over and over again, for example.
I've been researching this error for a while and have tried many things and nothing seems to work...
while(!driver.findElements(By.className("next")).isEmpty()) {
//elements = driver.findElements(By.xpath("//a[#class='name']"));
elements = findDynamicElements("//a[#class='name']");
for (WebElement e : elements) {
userName = e.getText(); //<--EXCEPTION HERE
check_visitor_profile(userName);//<--WE LEAVE THE PAGE HERE
Thread.sleep(3000); //<--NO TRY/CATCH BLOCK FOR READABILITY
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
elements = findDynamicElements("//a[#class='name']");
}
driver.findElement(By.xpath("VisitsNext")).click();
}
protected List<WebElement> findDynamicElements(String path) {
List<WebElement> result;
String xPath = path;
new WebDriverWait(driver, 25).until(ExpectedConditions.presenceOfAllElementsLocatedBy(By.xpath(xPath)));
//new WebDriverWait(driver, 25).until(elementIdentified(By.id(path)));
try {
result = driver.findElements(By.xpath(xPath));
return result;
}
catch(WebDriverException e) {
return null;
}
);
}
My code craps out on the first line of the for loop where userName is assigned. I've seen on this forum that you should use 'presenceOfElementLocated' and explicitly wait for the element to come back but that doesn't work either. I used 'presenceOfAllElementsLocatedBy' for a list but I have a method that uses 'presenceOfElementLocated' which doesn't work either.
I know stuff like the Thread.sleep and the implicitlyWait line is probably unnecessary at this point but I've literally tried everything and it doesn't work...
The error occurs because when I call 'check_visitor_profile' it leaves the page - when it comes back the elements are out of place so I have to find them again. Which I do but it still throws the exception.
Any Ideas?
Thanks.
The problem might occur because you are changing elements in the middle of the loop. It will cause you trouble even without the StaleElementReferenceException. Use a for loop instead of the for each loop
elements = findDynamicElements("//a[#class='name']");
int size = elements.size();
for (int i = 0 ; i < size ; ++i) {
elements = findDynamicElements("//a[#class='name']");
userName = elements.get(i).getText();
check_visitor_profile(userName);
}
Handle the exception explicitly as the element is no longer attached to the DOM or has changed at that moment you call "check_visitor_profile"
See the link below might help
catch(StateElementException e){
System.out.println("StaleElement dealt with since you successfully left page ");
}
http://docs.seleniumhq.org/exceptions/stale_element_reference.jsp
I'm using the PageObject/PageFactory design pattern for my UI automation. Using Selenium 2.0 WebDriver, JAVA, I randomly get the error: org.openqa.selenium.StaleElementReferenceException: Element is no longer attached to the DOM, when I attempt logic like this:
#FindBy(how = HOW.ID, using = "item")
private List<WebElement> items
private void getItemThroughName(String name) {
wait(items);
for(int i = 0; i < items.size(); i++) {
try {
Thread.sleep(0500);
} catch (InterruptedException e) { }
this.wait(items);
if(items.get(i).getText().contains(name)) {
System.out.println("Found");
break;
}
}
}
The error randomly happens at the if statement line, as you can see I've tried a couple things to avoid this, like sleeping a small amount of time, or waiting for the element again, neither works 100% of the time
First if you really have multiple elements on the by with the ID of "item" you should log a bug or talk to the developers on the site to fix that. An ID is meant to be unique.
As comments on the question already implied you should use an ExplicitWait in this case:
private void getItemThroughName(String name) {
new WebDriverWait(driver, 30)
.until(ExpectedConditions.presenceOfElementLocated(
By.xpath("id('item')[.='" + name + "']")
));
// A timeout exception will be thrown otherwise
System.out.println("Found");
}
Is there a way how to test if an element is present? Any findElement method would end in an exception, but that is not what I want, because it can be that an element is not present and that is okay. That is not a fail of the test, so an exception can not be the solution.
I've found this post: Selenium C# WebDriver: Wait until element is present.
But this is for C#, and I am not very good at it. What would the code be in Java? I tried it out in Eclipse, but I didn't get it right into Java code.
This is the code:
public static class WebDriverExtensions{
public static IWebElement FindElement(this IWebDriver driver, By by, int timeoutInSeconds){
if (timeoutInSeconds > 0){
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(timeoutInSeconds));
return wait.Until(drv => drv.FindElement(by));
}
return driver.FindElement(by);
}
}
Use findElements instead of findElement.
findElements will return an empty list if no matching elements are found instead of an exception.
To check that an element is present, you could try this
Boolean isPresent = driver.findElements(By.yourLocator).size() > 0
This will return true if at least one element is found and false if it does not exist.
The official documentation recommends this method:
findElement should not be used to look for non-present elements, use findElements(By) and assert zero length response instead.
Use a private method that simply looks for the element and determines if it is present like this:
private boolean existsElement(String id) {
try {
driver.findElement(By.id(id));
} catch (NoSuchElementException e) {
return false;
}
return true;
}
This would be quite easy and does the job.
You could even go further and take a By elementLocator as a parameter, eliminating problems if you want to find the element by something other than an id.
I found that this works for Java:
WebDriverWait waiter = new WebDriverWait(driver, 5000);
waiter.until( ExpectedConditions.presenceOfElementLocated(by) );
driver.FindElement(by);
public static WebElement FindElement(WebDriver driver, By by, int timeoutInSeconds)
{
WebDriverWait wait = new WebDriverWait(driver, timeoutInSeconds);
wait.until( ExpectedConditions.presenceOfElementLocated(by) ); //throws a timeout exception if element not present after waiting <timeoutInSeconds> seconds
return driver.findElement(by);
}
I had the same issue. For me, depending on a user's permission level, some links, buttons and other elements will not show on the page. Part of my suite was testing that the elements that should be missing, are missing. I spent hours trying to figure this out. I finally found the perfect solution.
It tells the browser to look for any and all elements based specified. If it results in 0, that means no elements based on the specification was found. Then I have the code execute an *if statement to let me know it was not found.
This is in C#, so translations would need to be done to Java. But it shouldn’t be too hard.
public void verifyPermission(string link)
{
IList<IWebElement> adminPermissions = driver.FindElements(By.CssSelector(link));
if (adminPermissions.Count == 0)
{
Console.WriteLine("User's permission properly hidden");
}
}
There's also another path you can take depending on what you need for your test.
The following snippet is checking to see if a very specific element exists on the page. Depending on the element's existence I have the test execute an if else.
If the element exists and is displayed on the page, I have console.write let me know and move on. If the element in question exists, I cannot execute the test I needed, which is the main reasoning behind needing to set this up.
If the element does not exist and is not displayed on the page, I have the else in the if else execute the test.
IList<IWebElement> deviceNotFound = driver.FindElements(By.CssSelector("CSS LINK GOES HERE"));
// If the element specified above results in more than 0 elements and is displayed on page execute the following, otherwise execute what’s in the else statement
if (deviceNotFound.Count > 0 && deviceNotFound[0].Displayed){
// Script to execute if element is found
} else {
// Test script goes here.
}
I know I'm a little late on the response to the OP. Hopefully this helps someone!
Try this:
Call this method and pass three arguments:
WebDriver variable. Assuming driver_variable as the driver.
The element which you are going to check. It should provide a from By method. Example: By.id("id")
Time limit in seconds.
Example: waitForElementPresent(driver, By.id("id"), 10);
public static WebElement waitForElementPresent(WebDriver driver, final By by, int timeOutInSeconds) {
WebElement element;
try{
driver.manage().timeouts().implicitlyWait(0, TimeUnit.SECONDS); // Nullify implicitlyWait()
WebDriverWait wait = new WebDriverWait(driver, timeOutInSeconds);
element = wait.until(ExpectedConditions.presenceOfElementLocated(by));
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); // Reset implicitlyWait
return element; // Return the element
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
This works for me:
if(!driver.findElements(By.xpath("//*[#id='submit']")).isEmpty()){
// Then click on the submit button
}
else{
// Do something else as submit button is not there
}
You can make the code run faster by shorting the Selenium timeout before your try-catch statement.
I use the following code to check if an element is present.
protected boolean isElementPresent(By selector) {
selenium.manage().timeouts().implicitlyWait(1, TimeUnit.SECONDS);
logger.debug("Is element present"+selector);
boolean returnVal = true;
try{
selenium.findElement(selector);
} catch (NoSuchElementException e) {
returnVal = false;
} finally {
selenium.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
}
return returnVal;
}
Write the following function/methods using Java:
protected boolean isElementPresent(By by){
try{
driver.findElement(by);
return true;
}
catch(NoSuchElementException e){
return false;
}
}
Call the method with the appropriate parameter during the assertion.
If you are using rspec-Webdriver in Ruby, you can use this script, assuming that an element should really not be present, and it is a passed test.
First, write this method first from your class RB file:
class Test
def element_present?
begin
browser.find_element(:name, "this_element_id".displayed?
rescue Selenium::WebDriver::Error::NoSuchElementError
puts "this element should not be present"
end
end
Then, in your spec file, call that method.
before(:all) do
#Test= Test.new(#browser)
end
#Test.element_present?.should == nil
If your element is not present, your spec will pass, but if the element is present, it will throw an error, and the test failed.
Personally, I always go for a mixture of the above answers and create a reusable static utility method that uses the size() > 0 suggestion:
public Class Utility {
...
public static boolean isElementExist(WebDriver driver, By by) {
return driver.findElements(by).size() > 0;
...
}
This is neat, reusable, maintainable, etc.—all that good stuff ;-)
public boolean isElementDisplayed() {
return !driver.findElements(By.xpath("...")).isEmpty();
}
This should do it:
try {
driver.findElement(By.id(id));
} catch (NoSuchElementException e) {
//do what you need here if you were expecting
//the element wouldn't exist
}
I would use something like (with Scala [the code in old "good" Java 8 may be similar to this]):
object SeleniumFacade {
def getElement(bySelector: By, maybeParent: Option[WebElement] = None, withIndex: Int = 0)(implicit driver: RemoteWebDriver): Option[WebElement] = {
val elements = maybeParent match {
case Some(parent) => parent.findElements(bySelector).asScala
case None => driver.findElements(bySelector).asScala
}
if (elements.nonEmpty) {
Try { Some(elements(withIndex)) } getOrElse None
} else None
}
...
}
so then,
val maybeHeaderLink = SeleniumFacade getElement(By.xpath(".//a"), Some(someParentElement))
The simplest way I found in Java was:
List<WebElement> linkSearch= driver.findElements(By.id("linkTag"));
int checkLink = linkSearch.size();
if(checkLink!=0) {
// Do something you want
}
To find if a particular Element is present or not, we have to use the findElements() method instead of findElement()...
int i = driver.findElements(By.xpath(".......")).size();
if(i=0)
System.out.println("Element is not present");
else
System.out.println("Element is present");
This is worked for me...
You can try implicit wait:
WebDriver driver = new FirefoxDriver();
driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));
driver.Url = "http://somedomain/url_that_delays_loading";
IWebElement myDynamicElement = driver.FindElement(By.Id("someDynamicElement"));
Or you can try explicit wait one:
IWebDriver driver = new FirefoxDriver();
driver.Url = "http://somedomain/url_that_delays_loading";
WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
IWebElement myDynamicElement = wait.Until<IWebElement>((d) =>
{
return d.FindElement(By.Id("someDynamicElement"));
});
Explicit will check if the element is present before some action. Implicit wait could be called in every place in the code. For example, after some Ajax actions.
More you can find at SeleniumHQ page.
I am giving my snippet of code. So, the below method checks if a random web element 'Create New Application' button exists on a page or not. Note that I have used the wait period as 0 seconds.
public boolean isCreateNewApplicationButtonVisible(){
WebDriverWait zeroWait = new WebDriverWait(driver, 0);
ExpectedCondition<WebElement> c = ExpectedConditions.presenceOfElementLocated(By.xpath("//input[#value='Create New Application']"));
try {
zeroWait.until(c);
logger.debug("Create New Application button is visible");
return true;
} catch (TimeoutException e) {
logger.debug("Create New Application button is not visible");
return false;
}
}
In 2022 this can now be done without an annoying delay, or affecting your current implicit wait value.
First bump your Selenium driver to latest (currently 4.1.2).
Then you can use getImplicitWaitTimeout then set timeout to 0 to avoid a wait then restore your previous implicit wait value whatever it was:
Duration implicitWait = driver.manage().timeouts().getImplicitWaitTimeout();
driver.manage().timeouts().implicitlyWait(Duration.ofMillis(0));
final List<WebElement> signOut = driver.findElements(By.linkText("Sign Out"));
driver.manage().timeouts().implicitlyWait(implicitWait); // Restore implicit wait to previous value
if (!signOut.isEmpty()) {
....
}
Try the below code using the isDispplayed() method to verify if the element is present or not:
WebElement element = driver.findElements(By.xpath(""));
element.isDispplayed();
There could be multiple reasons due to which you might observe exceptions while locating a WebElement using Selenium driver.
I would suggest you to apply the below suggestions for different scenarios:
Scenario 1: You just want to find out if a certain WebElement is present on the screen or not. For example, the Save button icon will only appear until the form is fully filled and you may want to check if Save button is present or not in your test.
Use the below code -
public Boolean isElementLoaded(By locator){
return !getWebElements(this.driver.findElements(locator), locator).isEmpty();
}
Scenario 2: You want to wait before a WebElement becomes visible in the UI
public List<WebElement> waitForElementsToAppear(By locator) {
return wait.until(visibilityOfAllElementsLocatedBy(by));
}
Scenario 3: Your test is flaky because the WebElement becomes stale sometimes and gets detached from the DOM.
protected final List<Class<? extends WebDriverException>> exceptionList =
List.of(NoSuchWindowException.class,
NoSuchFrameException.class,
NoAlertPresentException.class,
InvalidSelectorException.class,
ElementNotVisibleException.class,
ElementNotSelectableException.class,
TimeoutException.class,
NoSuchSessionException.class,
StaleElementReferenceException.class);
public WebElement reactivateWebElement(By by, WebElement element){
try {
wait.ignoreAll(exceptionList)
.until(refreshed(visibilityOf(element)));
logger.info(("Element is available.").concat(BLANK).concat(element.toString()));
} catch (WebDriverException exception) {
logger.warn(exception.getMessage());
} return this.driver.findElement(by);
}