I'm trying to extract a variety of information from a site. The site uses Javascript and allows you to search for a series of addresses on a map. What I have to do is: 1) enter a text in a text box 2) press the search button 3) wait for the list of addresses and the map to update (Gmaps) 4) extract data from the list When I use Selenium and chrome in the normal version, everything works. for point 3, to wait until the page is completely loaded, i use the following block of code:
public void waitForLoading() {
ExpectedCondition<Boolean> pageLoadCondition = new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver driver) {
boolean condition = ((JavascriptExecutor) driver).executeScript("return document.readyState").equals("complete");
System.out.println(condition);
return condition;
}
};
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(pageLoadCondition);
}
When I use Chrome headless, using exactly the same code, selenium doesn't wait. point 3 is always evaluated true, i.e. the above function return always true, also if the list is not loaded. The only solution that i found is to use Thread.sleep(), but is not the best and is very strange that the first solution doesn't work
could someone help me understand how to solve the problem?
Related
I am looking for function in Java +selenium where I can check or verify if page is fully loaded. I saw onLoad() of JS but nothing on java, is there is something for JAVA?
also I saw these:
WebDriver driver = new AnyDriverYouWant();
if (driver instanceof JavascriptExecutor) {
((JavascriptExecutor)driver).executeScript("yourScript();");
} else {
throw new IllegalStateException("This driver does not support JavaScript!");
}
but again JS and need to write script in JS
How to use JavaScript with Selenium WebDriver Java
update - also can try these solution too:
void waitForLoad(WebDriver driver) {
new WebDriverWait(driver, 30).until((ExpectedCondition<Boolean>) wd -> ((JavascriptExecutor) wd).executeScript("return document.readyState").equals("complete")); }
from here https://stackoverflow.com/a/15124562/12115696
In order to determine if a page is fully loaded, you will need to identify which WebElements on the page indicate a loading status. For example, if there is a load mask of some type, you will want to wait until the load mask is hidden to verify that the page is fully loaded.
Here's a simple "Wait until loaded" function that utilizes the ExpectedConditions class:
Given the following HTML for a load mask:
<div id='load-mask' style='display: block'/>
You can use the following code to wait until the load mask is hidden:
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(seconds));
wait.Until(ExpectedConditions.InvisibilityOfElementLocated(By.Id("load-mask")));
Edit -- added a JavaScript-only wait function, as requested by the asker:
wait.until(driver=> ((JavascriptExecutor)driver).executeScript("return document.readyState").equals("complete"));
This checks the document.readyState attribute in JavaScript, and completes the wait once readyState is set to complete.
I'm using web driver wait as explicit wait in selenium web driver. But It's not consistent it seems. before mouse over operations, links It's throwing unable to locate element error. Please see the below method and suggest where am I going wrong.
public WebElement waitForElement(String xPathExpression) {
WebDriverWait wait = new WebDriverWait(driver,30);
WebElement element = wait.until(ExpectedConditions.visibilityOf(driver.findElement(By.xpath(xPathExpression))));
return element;
}
Just a guess, but I might have seen similar behavior on very dynamic pages. I.e the element on the page changes while being evalued.
I'm able to workaround these kind of problems by using FluentWait#ignoring
WebDriverWait wait = new WebDriverWait(driver,30)
.ignoring(NoSuchElementException.class);
I was just wondering if there's an elegant way to utilize ExpectedConditions or something else to have my code wait for a page's source to contain a given string until a given timeout. I know I can use something like this if I wanted to use a specific element's locator ...
WebDriverWait wait = new WebDriverWait(driver,10);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.linkText("foobar")));
But I wanted to accomplish this without using a locator for a specific element, and just use the whole page source as my reference instead. Any suggestions?
You cant have the all elements as a condition for waiting. When switching page weddriver automaticly wait for the page to load. WHen it has finished loading the HTML elements it continues. But it doesnt wait for JavaScript to execute. A lot of webpages today uses JavaScript to populate the webpage after the HTML has loaded.
What you should do is wait for every element you want to use.
wait.until(ExpectedConditions.refreshed(ExpectedConditions.visibilityOfElementLocated(by)));
or
wait.until(ExpectedConditions.refreshed(ExpectedConditions.elementToBeClickable(element))h;
You can wait for document's readyState to become complete. Run the javascript return document.readyState").equals("complete") against the web page that is loading.
void waitForLoad(WebDriver driver) {
ExpectedCondition<Boolean> pageLoadCondition = new
ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver driver) {
return ((JavascriptExecutor)driver).executeScript("return document.readyState").equals("complete");
}
};
WebDriverWait wait = new WebDriverWait(driver, 30);
wait.until(pageLoadCondition);
}
And then you can get the page source:
driver.getPageSource();
And then verify that the pageSource contains what you are looking for:
driver.getPageSource().contains("your element/tag");
I hope this helps!
Without depending on visibility, you could check if an element is present in the DOM:
WebDriverWait wait = new WebDriverWait(driver,10);
wait.until(ExpectedConditions.presenceOfElementLocated(By.tagName("html")));
If you want to refer to individual text, you can implement ExpectedCondition<T> and create your own condition class. The mentioned interface has access to the WebDriver (due to super-interface com.google.common.base.Function<WebDriver,T>).
In the method apply you could use the WebDriver and call method getPageSource() to have String presenting the page source. Check the String for whatever you prefer.
I am using selenium webdriver along with TestNG in eclipse.The problem is the page relaods in the midway for some data and the time of this reload is flexible thats why I am not able apply explicit wait time.I want to make webdriver wait until this reload completes.
I am trying to do this through this code...but it is not working
public void waitForPageLoadingToComplete() throws Exception {
ExpectedCondition<Boolean> expectation = new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver driver) {
return ((JavascriptExecutor) driver).executeScript(
"return document.readyState").equals("complete");
}
};
Wait<WebDriver> wait = new WebDriverWait(driver, 30);
wait.until(expectation);
}
try the below code for handling page load/page refresh time outs
WebDriver driver = new FireFoxDriver();
driver.manage().timeouts().pageLoadTimeout(30, TimeUnit.SECONDS);
please use latest version of chrome driver, as the page wait is not handled in older version of chrome driver.
Waiting for an indefinite time is not a good idea. Timing of a website is also a part of testing. If possible find out the Service Level Agreement of the "page" you are testing. If not run a speed test for the website(here is a method to test : http://tools.pingdom.com/fpt/ ) and use an average of time you get. If this also doesn't work the last option is to work with industry wide standards.
document.readyState() does not reflect the correct page load time(example- it does not wait for images/scripts to load fully). It is suggested and tested option to wait for an element on the page(preferrably the one you will operate upon in your next step of test). As others have suggested use WebDriverWait with expected conditions methods like "visibilityOf", "presenceOfElement" or many more and it should be fine.
You should use WebDriverWait and set the timeout to the maximum time you can wait. As soon as you discover that the page loaded the required data (e.g. by checking for visibility of a certain element), you may proceed with the test case.
See an example in the selenium docs.
driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
For java 8 onwards:
JavascriptExecutor js = (JavascriptExecutor) driver;
new WebDriverWait(driver, 10).until(webDriver ->(js).executeScript("return document.readyState;").equals("complete"));
For java below 8 you can try the below solution from the below link. I am using it and it's working for me.
Wait for page load in Selenium
how to select a value from dropdown using selenium java webdriver using xpath?
depending on the option selected in the dropdown, fields appear .so that I need t enter values in it. My problem is I am not getting the fields after selecting the option in dropdown.After a long period of time it is appears, mean while error appears
You can click the drop down and wait for the options to get displayed and then you can click the option from it.
or
Select select = new Select(driver.findElement(By.id("drop_down_id")));
select.selectByIndex(`index_value_of_option`);
Jyotsna... Your script need to wait till the field appear. For this you need to use any of the wait condition.
implicit wait
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
or sleep condition
Thread.sleep(2000);
or you can use Fluent wait (the best one as per my suggestion)
public WebElement fluentWait(final By locator) {
Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
.withTimeout(30, TimeUnit.SECONDS)
.pollingEvery(5, TimeUnit.SECONDS)
.ignoring(NoSuchElementException.class);
WebElement foo = wait.until(new Function<WebDriver, WebElement>() {
public WebElement apply(WebDriver driver) {
return driver.findElement(locator);
}
});
return foo;
};
fluentWait function returns your found web element. From the documentation on fluentWait: An implementation of the Wait interface that may have its timeout and polling interval configured on the fly. Each FluentWait instance defines the maximum amount of time to wait for a condition, as well as the frequency with which to check the condition. Furthermore, the user may configure the wait to ignore specific types of exceptions whilst waiting, such as NoSuchElementExceptions when searching for an element on the page. Details you can get here
Usage of `fluentWait in your case be the following:
WebElement textbox = fluentWait(By.id("textbox"));
Sample statements to open browser, load URL and select value from dropdown
static WebDriver driver;
System.setProperty("webdriver.ie.driver","C:\\(Path)\\IEDriverServer.exe");
driver = new InternetExplorerDriver();
driver.manage().window().maximize();
driver.get("EnterURLHere");
driver.manage().timeouts().implicitlyWait(30,TimeUnit.SECONDS);
Select value1 = new Select(driver.findElement(By.id("LocateId")));
value1.selectByVisibleText("ValueToBeSelected"); //Select Character from dropdown list
you can add wait so that the problem of late loading would be resolve.
driver.manage().timeouts().implicitlyWait(20,TimeUnit.SECONDS);
or,
Thread.sleep(2000);
For selecting from drop down , there are multiple ways from which one can select :
Select dropdown = new Select(driver.findElement(By.id(""))); // By id
dropdown.selectByVisibleText(""); // By Visible text
dropdown.selectByIndex(1); // By index