We Keep Coding

Selenium wait for specific XHR request to be complete

Hi i am working on a selenium project and the top difficulty that i am having was waiting for XHR request to be completed. What i am currently doing is i wait for a request to be made using following expected condition,
public ExpectedCondition<Boolean> jQueryExpect (int expectedActive) {
ExpectedCondition<Boolean> jQLoad = new ExpectedCondition<Boolean>() {
#Override
public Boolean apply(WebDriver dr) {
try {
logger.log(Level.INFO,"Checking number of jQueries Active");
Long active = (Long) ((JavascriptExecutor) driver).executeScript("return jQuery.active");
logger.log(Level.INFO,"jQuery''s active: {0}",active);
return (active >= expectedActive);
}
catch (Exception e) {
logger.log(Level.WARNING,"Error executing script in jQueryLoad method");
// no jQuery present
return true;
}
}
};
return jQLoad;
}
And then i wait for the jQuery to load using this expected condition
public ExpectedCondition<Boolean> jQueryLoad (int expectedActive) {
ExpectedCondition<Boolean> jQLoad = new ExpectedCondition<Boolean>() {
#Override
public Boolean apply(WebDriver dr) {
try {
logger.log(Level.INFO,"Checking number of jQueries Active");
Long active = (Long) ((JavascriptExecutor) driver).executeScript("return jQuery.active");
logger.log(Level.INFO,"jQuery''s active: {0}",active);
return (active <= expectedActive);
}
catch (Exception e) {
logger.log(Level.WARNING,"Error executing script in jQueryLoad method");
// no jQuery present
return true;
}
}
};
return jQLoad;
}
This method is working pretty solid for now since i know how many requests to expect. But as you have already noticed it can easily break in future as number of requests made are changed for some reason.
I been looking at cypress documentation and found this. According to cypress documentation this waits for the specified requests to be made.
cy.wait(['#getUsers', '#getActivities', '#getComments']).then((xhrs) => {
// xhrs will now be an array of matching XHR's
// xhrs[0] <-- getUsers
// xhrs[1] <-- getActivities
// xhrs[2] <-- getComments
})
Is there any such method available in Selenium? or Is there any way this can be implemented? So far from what i have googled i got nothing. So any help will be appreciated.

You can locate Element and wait for element
There are Implicit and Explicit waits in selenium.
You can use either
WebDriverWait wait = new WebDriverWait(webDriver, timeoutInSeconds);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id<locator>));
or
wait.until(ExpectedConditions.elementToBeClickable(By.id<locator>));
More information: on this answer

About Automated testing, huge number of steps and perfomance issues [duplicate]

I am trying to check if web page is loaded completed or not (i.e. checking that all the control is loaded) in selenium.
I tried below code:
new WebDriverWait(firefoxDriver, pageLoadTimeout).until(
webDriver -> ((JavascriptExecutor) webDriver).executeScript("return document.readyState").equals("complete"));
but even if page is loading above code does not wait.
I know that I can check for particular element to check if its visible/clickable etc but I am looking for some generic solution

As you mentioned if there is any generic function to check if the page has completely loaded through Selenium the answer is No.
First let us have a look at your code trial which is as follows :
new WebDriverWait(firefoxDriver, pageLoadTimeout).until(webDriver -> ((JavascriptExecutor) webDriver).executeScript("return document.readyState").equals("complete"));
The parameter pageLoadTimeout in the above line of code doesn't really reseambles to actual pageLoadTimeout().
Here you can find a detailed discussion of pageLoadTimeout in Selenium not working
Now as your usecase relates to page being completely loaded you can use the pageLoadStrategy() set to normal [ the supported values being none, eager or normal ] using either through an instance of DesiredCapabilities Class or ChromeOptions Class as follows :
Using DesiredCapabilities Class :
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.DesiredCapabilities;
public class myDemo
{
public static void main(String[] args) throws Exception
{
System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
DesiredCapabilities dcap = new DesiredCapabilities();
dcap.setCapability("pageLoadStrategy", "normal");
FirefoxOptions opt = new FirefoxOptions();
opt.merge(dcap);
WebDriver driver = new FirefoxDriver(opt);
driver.get("https://www.google.com/");
System.out.println(driver.getTitle());
driver.quit();
}
}
Using ChromeOptions Class :
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.PageLoadStrategy;
public class myDemo
{
public static void main(String[] args) throws Exception
{
System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
FirefoxOptions opt = new FirefoxOptions();
opt.setPageLoadStrategy(PageLoadStrategy.NORMAL);
WebDriver driver = new FirefoxDriver(opt);
driver.get("https://www.google.com/");
System.out.println(driver.getTitle());
driver.quit();
}
}
You can find a detailed discussion in Page load strategy for Chrome driver (Updated till Selenium v3.12.0)
Now setting PageLoadStrategy to NORMAL and your code trial both ensures that the Browser Client have (i.e. the Web Browser) have attained 'document.readyState' equal to "complete". Once this condition is fulfilled Selenium performs the next line of code.
You can find a detailed discussion in Selenium IE WebDriver only works while debugging
But the Browser Client attaining 'document.readyState' equal to "complete" still doesn't guarantees that all the JavaScript and Ajax Calls have completed.
To wait for the all the JavaScript and Ajax Calls to complete you can write a function as follows :
public void WaitForAjax2Complete() throws InterruptedException
{
while (true)
{
if ((Boolean) ((JavascriptExecutor)driver).executeScript("return jQuery.active == 0")){
break;
}
Thread.sleep(100);
}
}
You can find a detailed discussion in Wait for ajax request to complete - selenium webdriver
Now, the above two approaches through PageLoadStrategy and "return jQuery.active == 0" looks to be waiting for indefinite events. So for a definite wait you can induce WebDriverWait inconjunction with ExpectedConditions set to titleContains() method which will ensure that the Page Title (i.e. the Web Page) is visible and assume the all the elements are also visible as follows :
driver.get("https://www.google.com/");
new WebDriverWait(driver, 10).until(ExpectedConditions.titleContains("partial_title_of_application_under_test"));
System.out.println(driver.getTitle());
driver.quit();
Now, at times it is possible though the Page Title will match your Application Title still the desired element you want to interact haven't completed loading. So a more granular approach would be to induce WebDriverWait inconjunction with ExpectedConditions set to visibilityOfElementLocated() method which will make your program wait for the desired element to be visible as follows :
driver.get("https://www.google.com/");
WebElement ele = new WebDriverWait(driver, 10).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("xpath_of_the_desired_element")));
System.out.println(ele.getText());
driver.quit();
References
You can find a couple of relevant detailed discussions in:
Selenium IE WebDriver only works while debugging
Selenium how to manage wait for page load?

I use selenium too and I had the same problem, to fix that I just wait also for the jQuery to load.
So if you have the same issue try this also
((Long) ((JavascriptExecutor) browser).executeScript("return jQuery.active") == 0);
You can wrap both function in a method and check until both page and jQuery is loaded

Implement this, Its working for many of us including me. It includes Web Page wait on JavaScript, Angular, JQuery if its there.
If your Application is containing Javascript & JQuery you can write code for only those,
By define it in single method and you can Call it anywhere:
// Wait for jQuery to load
{
ExpectedCondition<Boolean> jQueryLoad = driver -> ((Long) ((JavascriptExecutor) driver).executeScript("return jQuery.active") == 0);
boolean jqueryReady = (Boolean) js.executeScript("return jQuery.active==0");
if (!jqueryReady) {
// System.out.println("JQuery is NOT Ready!");
wait.until(jQueryLoad);
}
wait.until(jQueryLoad);
}
// Wait for ANGULAR to load
{
String angularReadyScript = "return angular.element(document).injector().get('$http').pendingRequests.length === 0";
ExpectedCondition<Boolean> angularLoad = driver -> Boolean.valueOf(((JavascriptExecutor) driver).executeScript(angularReadyScript).toString());
boolean angularReady = Boolean.valueOf(js.executeScript(angularReadyScript).toString());
if (!angularReady) {
// System.out.println("ANGULAR is NOT Ready!");
wait.until(angularLoad);
}
}
// Wait for Javascript to load
{
ExpectedCondition<Boolean> jsLoad = driver -> ((JavascriptExecutor) driver).executeScript("return document.readyState").toString()
.equals("complete");
boolean jsReady = (Boolean) js.executeScript("return document.readyState").toString().equals("complete");
// Wait Javascript until it is Ready!
if (!jsReady) {
// System.out.println("JS in NOT Ready!");
wait.until(jsLoad);
}
}
Click here for Reference Link
Let me know if you stuck anywhere by implementing.
It overcomes the use of Thread or Explicit Wait.

public static void waitForPageToLoad(long timeOutInSeconds) {
ExpectedCondition<Boolean> expectation = new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver driver) {
return ((JavascriptExecutor) driver).executeScript("return document.readyState").equals("complete");
}
};
try {
System.out.println("Waiting for page to load...");
WebDriverWait wait = new WebDriverWait(Driver.getDriver(), timeOutInSeconds);
wait.until(expectation);
} catch (Throwable error) {
System.out.println(
"Timeout waiting for Page Load Request to complete after " + timeOutInSeconds + " seconds");
}
}
Try this method

This works for me well with dynamically rendered websites:
Wait for complete page to load
WebDriverWait wait = new WebDriverWait(driver, 50);
wait.until((ExpectedCondition<Boolean>) wd -> ((JavascriptExecutor) wd).executeScript("return document.readyState").equals("complete"));
Make another implicit wait with a dummy condition which would always fail
try {
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//*[contains(text(),'" + "This text will always fail :)" + "')]"))); // condition you are certain won't be true
}
catch (TimeoutException te) {
}
Finally, instead of getting the html source - which would in most of one page applications would give you a different result , pull the outerhtml of the first html tag
String script = "return document.getElementsByTagName(\"html\")[0].outerHTML;";
content = ((JavascriptExecutor) driver).executeScript(script).toString();

There is a easy way to do it. When you first request the state via javascript, it tells you that the page is complete, but after that it enters the state loading. The first complete state was the initial page!
So my proposal is to check for a complete state after a loading state. Check this code in PHP, easily translatable to another language.
$prevStatus = '';
$checkStatus = function ($driver) use (&$prevStatus){
$status = $driver->executeScript("return document.readyState");
if ($prevStatus=='' && $status=='loading'){
//save the previous status and continue waiting
$prevStatus = $status;
return false;
}
if ($prevStatus=='loading' && $status=='complete'){
//loading -> complete, stop waiting, it is finish!
return true;
}
//continue waiting
return false;
};
$this->driver->wait(20, 150)->until($checkStatus);
Checking for a element to be present also works well, but you need to make sure that this element is only present in the destination page.

Something like this should work (please excuse the python in a java answer):
idle = driver.execute_async_script("""
window.requestIdleCallback(() => {
arguments[0](true)
})
""")
This should block until the event loop is idle which means all assets should be loaded.

PhantomJS - Element is not currently interactable

I'm currently using PhantomJS + Selenium to populate some form fields but having weird results. 50% of the time, the test runs fine. The other 50% it errors out and gives me the following
{"errorMessage":"Element is not currently interactable and may not be
manipulated"
I'm doing the following to make sure the page is loaded.
private static boolean waitForJQueryProcessing(WebDriver driver,
int timeOutInSeconds) {
boolean jQcondition = false;
try {
new WebDriverWait(driver, timeOutInSeconds) {
}.until(new ExpectedCondition<Boolean>() {
#Override
public Boolean apply(WebDriver driverObject) {
return (Boolean) ((JavascriptExecutor) driverObject)
.executeScript("return jQuery.active == 0");
}
});
jQcondition = (Boolean) ((JavascriptExecutor) driver)
.executeScript("return window.jQuery != undefined && jQuery.active === 0");
return jQcondition;
} catch (Exception e) {
logger.debug(e.getMessage());
}
return jQcondition;
}
And then to interact with the element(s):
pageWait.until(ExpectedConditions.presenceOfElementLocated(By
.cssSelector("#myForm-searchDate")));
driver.findElement(
By.cssSelector("#myForm-searchDate"))
.sendKeys(Keys.CONTROL + "a");
driver.findElement(
By.cssSelector("#myForm-searchDate"))
.sendKeys(Keys.DELETE);
driver.findElement(
By.cssSelector("#myForm-searchDate"))
.sendKeys(MY_TEST_DATE);
I could see if it failed all the time, but it doesn't fail all the time so it's hard to repeat the results when debugging.
Edit 1. I've tried swapping following the comment below; however, it doesn't work. I've since come to realize this seems to only happen when I fire up several (5+) instances of PhantomJS at once.

Selenium: Waiting for an element do disappear

I posed with a difficult task. I am fairly new to selenium and still working through the functionalities of waiting for elements and alike.
I have to manipulate some data on a website and then proceed to another. Problem: the manipulation invokes a script that makes a little "Saving..." label appear while the manipulated data is being processed in the background. I have to wait until I can proceed to the next website.
So here it is:
How do i wait for and element to DISAPPEAR? Thing is: It is always present in the DOM but only made visible by some script (I suppose, see image below).
This is what I tried but it just doesn't work - there is no waiting, selenium just proceeds to the next step (and gets stuck with an alert asking me if I want to leave or stay on the page because of the "saving...").
private By savingLableLocator = By.id("lblOrderHeaderSaving");
public boolean waitForSavingDone(By webelementLocator, Integer seconds){
WebDriverWait wait = new WebDriverWait(driver, seconds);
Boolean element = wait.until(ExpectedConditions.invisibilityOfElementLocated(webelementLocator));
return element;
}
UPDATE / SOLUTION:
I came up ith the following solution: I built my own method. Basically it checks in a loop for the CssValue to change.
the loops checks for a certain amount of time for the CSSVALUE "display" to go from "block" to another state.
public void waitForSavingOrderHeaderDone(Integer _seconds){
WebElement savingLbl = driver.findElement(By.id("lblOrderHeaderSaving"));
for (int second = 0;; second++) {
if (second >= _seconds)
System.out.println("Waiting for changes to be saved...");
try {
if (!("block".equals(savingLbl.getCssValue("display"))))
break;
} catch (Exception e) {
}
}

You can wait for a WebElement to throw a StaleElementReferenceException like this:
public void waitForInvisibility(WebElement webElement, int maxSeconds) {
Long startTime = System.currentTimeMillis();
try {
while (System.currentTimeMillis() - startTime < maxSeconds * 1000 && webElement.isDisplayed()) {}
} catch (StaleElementReferenceException e) {
return;
}
}
So you would pass in the WebElement you want to wait for, and the max amount of seconds you want to wait.

Webdriver has built in waiting functionality you just need to build in the condition to wait for.
Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
.withTimeout(30, SECONDS)
.pollingEvery(5, SECONDS)
.ignoring(NoSuchElementException.class);
WebElement foo = wait.until(new Function<WebDriver, WebElement>() {
public WebElement apply(WebDriver driver) {
return (driver.findElements(By.id("lblOrderHeaderSaving")).size() == 0);
}
});

I'm not sure, but you can try something like this :)
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); //time in second
WebElement we = driver.findElement(By.id("lblOrderHeaderSaving"));
assertEquals("none", we.getCssValue("display"));

This works with selenium 2.4.0. you have to use the invisibility mehtod to find it.
final public static boolean waitForElToBeRemove(WebDriver driver, final By by) {
try {
driver.manage().timeouts()
.implicitlyWait(0, TimeUnit.SECONDS);
WebDriverWait wait = new WebDriverWait(UITestBase.driver,
DEFAULT_TIMEOUT);
boolean present = wait
.ignoring(StaleElementReferenceException.class)
.ignoring(NoSuchElementException.class)
.until(ExpectedConditions.invisibilityOfElementLocated(by));
return present;
} catch (Exception e) {
return false;
} finally {
driver.manage().timeouts()
.implicitlyWait(DEFAULT_TIMEOUT, TimeUnit.SECONDS);
}
}

I used following C# code to handle this, you may convert it to Java
public bool WaitForElementDisapper(By element)
{
try
{
while (true)
{
try
{
if (driver.FindElement(element).Displayed)
Thread.Sleep(2000);
}
catch (NoSuchElementException)
{
break;
}
}
return true;
}
catch (Exception e)
{
logger.Error(e.Message);
return false;
}
}

You can also try waiting for the ajax calls to complete. I've used this to check when the page load is complete and all the elements are visible.
Here's the code - https://stackoverflow.com/a/46640938/4418897

You could use XPath and WebDriverWait to check whether display: none is present in the style attribute of an element. Here is an example:
// Specify the time in seconds the driver should wait while searching for an element which is not present yet.
int WAITING_TIME = 10;
// Use the driver for the browser you want to use.
ChromeDriver driver = new ChromeDriver();
WebDriverWait wait = new WebDriverWait(driver, WAITING_TIME);
// Replace ELEMENT_ID with the ID of the element which should disappear.
// Waits unit style="display: none;" is present in the element, which means the element is not visible anymore.
driver.wait.until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[#id='ELEMENT_ID'][contains(#style, 'display: block')]")));

Try using invisibilityOfElementLocated method.
You can reference example here How to wait until an element no longer exists in Selenium?

enter image description hereI created my own method for element disappearing from dom....
In Conditions class (In .m2\repository\org\seleniumhq\selenium\selenium-support\3.141.59\selenium-support-3.141.59.jar!\org\openqa\selenium\support\ui\ExpectedConditions.class)
we can see that 'isInvisible' method with 'isDisplayed' method,,, i wrote the same with 'isEnabled'
public static ExpectedCondition<Boolean> invisibilityOf(final WebElement element) {
return new ExpectedCondition<Boolean>() {
#Override
public Boolean apply(WebDriver webDriver) {
return isRemovedFromDom(element);
}
#Override
public String toString() {
return "invisibility of " + element;
}
};
}
private static boolean isRemovedFromDom(final WebElement element) {
try {
return !element.isEnabled();
} catch (StaleElementReferenceException ignored) {
return true;
}
}

Selenium.setTimeout does not wait for specified time

We have to continuously monitor a URL to check for its availability. I have used selenium for simulation. Pasted below is the piece of code.
driver = new InternetExplorerDriver();
selenium = new WebDriverBackedSelenium(driver, mainUrl);
selenium.setTimeout("90000");
selenium.open(mainUrl);
However, selenium.timeout does not work even if the URL does not get opened up in 90000 milliseconds. How can I fix this?

You can also use :
driver.manage().timeouts().implicitlyWait(90, TimeUnit.SECONDS);

According the selenium javadoc, the example looks like this:
driver = new InternetExplorerDriver();
driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);
selenium.open(mainUrl);

try use Thread.sleep():
final String sUrl = "http://www.google.ca/index.html";
// Instantiate the Internet Explorer browser.
WebDriver oWebDriver = new InternetExplorerDriver();
// Open the main google webpage.
oWebDriver.get(sUrl);
try
{
Thread.sleep(5000);
}
catch(InterruptedException ex)
{
System.out.println(ex.getMessage());
}
If you'd like to check for availability somtheing try also using fluentWait:
public WebElement fluentWait(final By locator){
Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
.withTimeout(30, TimeUnit.SECONDS)
.pollingEvery(5, TimeUnit.SECONDS)
.ignoring(NoSuchElementException.class);
WebElement foo = wait.until(
new Function<WebDriver, WebElement>() {
public WebElement apply(WebDriver driver) {
return driver.findElement(locator);
}
}
);
return foo; } ;
An implementation of the Wait interface that may have its timeout and polling interval configured on the fly.
Each FluentWait instance defines the maximum amount of time to wait for a condition, as well as the frequency with which to check the condition. Furthermore, the user may configure the wait to ignore specific types of exceptions whilst waiting, such as NoSuchElementExceptions when searching for an element on the page.
Hope this helps you)