We Keep Coding

Java Selenium - Waiting on slow websites

Im testing on a site that sometimes doenst really have a good connection.
Everything works great until it decides to not work properly.
I am seperating every element with a wait.until, everything works normally until the connection gets slow and it believes the element is present and decides to use it.
public static WebElement login_btn(WebDriver driver, WebDriverWait wait) {
wait.until(ExpectedConditions.refreshed(ExpectedConditions.presenceOfElementLocated(By.id("btnEntrar"))));
wait.until(ExpectedConditions.refreshed(ExpectedConditions.elementToBeClickable(By.id("btnEntrar"))));
element = driver.findElement(By.id("btnEntrar"));
return element;
}
I cannot find a way to simulate this until the website becomes terribly slow and I get to see if it works or not. After months of trying I am unable to find out a way to completely wait for the element to be present and not receive a staleElementReferenceException or the site actually going into an error page due to me using an element. Using the site manually without these tests cannot simulate what is happening with the webdriver. Are there any hints or suggests on what I could do?

There are a few good answers you can try in this question and this question. One is to use Chrome dev tools to throttle adjust the download speed, https://stackoverflow.com/a/34728002/2386774. Another is Charles Web Debugging Proxy.
To solve this issue more simply, my suggestion is to specifically wait for the page to load completely then scrape the page. This will allow you to wait once and then you won't have to add waits everywhere. You would need to add additional waits if your page is dynamic, e.g. you click a button and it updates part of the page. In that case, add a wait when the button is clicked and you're good again. No need to add waits everywhere.

Selenium Webdriver.load.strategy unstable

I'm having problems loading my page with webdriver. My current (problematic) solution involves using the unstable load with firefox, but I'm open to other solutions.
The Core Problem
The root of all my problems comes from the fact my page will never fully load when I call it normally with webdriver, and thus will never preform the 2nd step, it's just always loading. It loads fine when you just go to the site with a normal browser. I've tried out a few work arounds that work intermittently, including opening the driver to google, and then going to the page which sometimes makes it load, and with
IJavaScriptExecutor js = (IJavaScriptExecutor)Driver;
js.ExecuteScript("return window.stop");
as suggested by this question, which sometimes makes it continue without loading. But neither of those work consistently at all (probably <50% of the time)
The best I've got so far is using the unstable load strategy. However that has its own different problems, all of which only happen sometimes.
1) Sometimes it doesn't wait for the page to load at all, and I get an "Unable to locate element:" exception within milliseconds of loading the page, despite the fact that I have a wait set up:
new WebDriverWait(cdriver, 30).until(ExpectedConditions.visibilityOf(cdriver.findElement(By.id(myId))));
Thread.sleep(3000); solves that problem, but I've read that is a sloppy way to do things.
2) If I get passed that step, my test should click one link, then click another and continue on with the test. However, it often gets stuck after that first click. So if my code is like this:
Thread.sleep(3000);
element1.click();
System.out.println("clicked!");
element2.click();
The first click (which doesn't load a new page, by the way, just a pop up on the same page) will work, but then the system will never print out "clicked!", it's stuck in the same way would be loading the page initially (without the unstable load thing). If/when it makes it over that hurdle, I think the rest of the test is fine.
Any ideas 1) why it works sometimes but not others. 2) how to fix it 3) how to just get my page to load in the first place
Thank you!

Try the following approach:
Don't wait for visibility of element on the page - in your case By.id(myId).
The element can become visible much earlier than the page is fully loaded, before all java scripts are loaded and fired etc.
Wait for some element on the page to be clickeable instead. For example you can wait until link becomes clickeable in this way:
By locatorOfLink1 = By.xpath(....); // By.id, By.name, By.linkText etc.
wait.until(ExpectedConditions.elementToBeClickable(locatorOfLink1));
If the above will not help for problem #2 (the webdriver gets stuck after the click), then send ENTER key to the link instead of the click:
element1.sendKeys(Keys.ENTER);

Why can't the selenium WebDriver click on anything, even if the element exists and is on focus?

I have a problem with Selenium again.
It was (and I swear!) working a week ago without problems. Today I wanted to write some more tests, but I have an extremely weird problem.
Whenever I want to click on something, Selenium simply doesn't click. With the Firefox-Driver I can even see that the object I want to click is focused, but it happens... nothing.
It happens in IE as in FF, but in IE there are even more errors.
My test breaks in the next step, I do not even get any error-messages for the not-working click.
Also, as mentioned above, I have more errors/bugs with the IE-Driver. The FF-Driver does not click on CERTAIN links (sometimes it works, sometimes not, structure of the link that should be clicked and the code that clicks are equal between the links. Everything is reproducable, but I do not know on what it depends if he clicks it or not, the FF is just behaving the same way every time). The IE-Driver clicks on nothing.
I don't get what I am doing wrong and I invested more than 8 hours into this problem and tried to click those damn links in any way you could think of. Nothing. No errors, not a single step forward.
An example:
The specific HTML-Element:
<div id="search">
<form id="keytyp" onsubmit="return false;" target="ECS_GF_CENTER_BOTTOM" method="get" name="keytyp" action="keySearchResult.jsp;jsessionid=F90D1D4731343BF3EB03EACD56CF4314">
<input type="text" value="Suchbegriff" onkeyup="javascript:startSearchOnReturn(event);" onfocus="this.select()" name="searchString" size="25"></input>
<div style="float:right;width:20px;">
<a border="0" href="javascript:search()">
<img title="Suche nach Begriffen" alt="startSearch" src="images/arrow3.gif"></img>
</a>
</div>
</form>
</div>
This snippet represents the "search"-bar in our project. Please keep in mind that this is one of many examples. The error must not be searched here.
What I want to do is to simply click on the a-link. I am trying to do this with the following code:
#When("the user searches for the term (.*)")
public void searchForTerm(final String term) throws InterruptedException {
// Switch back to root-frame
webdriver.switchTo().defaultContent();
webdriver.switchTo().frame("start");
WebElement searchField = webdriver.findElement(By.name("searchString"));
searchField.clear();
searchField.sendKeys(term);
WebElement target = webdriver.findElement(By.id("search")).findElements(By.tagName("a")).get(0);
target.click();
Thread.sleep(5000);
}
As I said, I tried to click on something in so goddamn many ways.
What am I doing wrong here? I really do not know what to do.
Thanks in advance, Guys.
P.S. Yes, I do not have ANY information about what is going wrong as selenium does not tell me anything. It simply does not perform the click.
EDIT: I know this thread is dead, but maybe someone will find this in the future, thinking the issue still remains unfixed. That is not the case.
Actually everything was caused by a misbehaving Tomcat, which was only misbehaving on my machine, I did not had the idea to try accessing the webpage without selenium, because my coworkers were able to do, what they had to do, so I was not actually searching for the error in Tomcat! I do not know why, but when launched on my local system, the Tomcat does not perform any POST-Request or anything. This caused me to think that Selenium was not able to click things, but actually the Tomcat did not respond to these requests in a really weird way. He just ignored the click.
We now set up a server in our local network to run all our tests against this system, so everything is reproducable by every team-member.
Thanks for the tips and ideas by you guys!

Did Webdriver throw any exception to you? It is really hard to investigate an issue without any stack trace of error.
Your code:
WebElement target = webdriver.findElement(By.id("search")).findElements(By.tagName("a")).get(0); can cause a several problems:
The list of elements with tagName="a" can be empty.
The target element may be not fully loaded on the page.
Possible solutions:
You can try to use any other element attributes to check whether element is accesible.
Also you can try to use WebdriverWait WebDriverWait wait = new WebDriverWait(driver, 15);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("search"))); before initializing 'target' variable.
BDD framework also can cause some issues to your tests.

Selenium Webdriver with Java: Element not found in the cache - perhaps the page has changed since it was looked up

I am initializing a variable in the beginning of my class:
public WebElement logout;
Later on in the code, in some method, the first time I encounter the logout button, I assign a value to that variable (in the brackets of an if/else statement):
logout = driver.findElement(By.linkText("Logout"));
logout.click();
I then use "logout" once more, successfully, at another stage of my test:
logout.click();
And at the end of the test, at a place where the element is the same (By.linkText ("Logout")), I get this error:
Element not found in the cache - perhaps the page has changed since it was looked up
Why?
EDIT: Actually, I dont successfully use the logout.click(); commant at another stage of my test. Looks like I cant use it again. I have to create a logout1 webelement and use it...

If there has been any changes to the page after you have initially found the element the webdriver reference will now contain a stale reference. As the page has changed, the element will no longer be where webdriver expects it to be.
To solve your issue, try finding the element each time you need to use it - writing a small method that you can call as and when is a good idea.
import org.openqa.selenium.support.ui.WebDriverWait
public void clickAnElementByLinkText(String linkText) {
wait.until(ExpectedConditions.presenceOfElementLocated(By.linkText(linkText)));
driver.findElement(By.linkText(linkText)).click();
}
Then within your code you'd only need to:
clickAnElementByLinkText("Logout");
So each time it will find the element and click on it, as such even if the page changes as it is 'refreshing' the reference to that element it all successfully click it.

The browser rebuilds the DOM structure of the dynamic pages, so the elements do not need to keep you have to find them before to use.
Using XPath, for example. This approach is not correct(can cause exception org.openqa.selenium.StaleElementReferenceException in the future):
WebElement element = driver.findElement(By.xpath("//ul[#class=\"pagination\"]/li[3]/a"));
...// Some Ajax interaction here
element.click(); //<-- Element might not be exists
This approach is correct:
driver.findElement(By.xpath("//ul[#class=\"pagination\"]/li[3]/a")).click();

This is because you are not giving proper time to load the page.So you have to give Thread.sleep(); code for the Given page.
I am also getting the same issue for my project but after using the Thread.sleep(); its working fine for me give the web page as much as it is possible for 30 to 50 secs.

Selenium exception when attempting to click ID of a button

I'm attempting to use the Java Selenium client with JBehave to interact with a simple web page as a demonstration of BDD techniques.
When running the test, i'm trying to do a simple
selenium.click("joinButton");
but i'm getting a
(com.thoughtworks.selenium.SeleniumException: joinButton is not defined)
exception thrown.
I've tried qualifying the ID as a dom ID using id=joinButton, as a css selector and even as xpath but to no avail.
The element is definitely rendered in the page. I'm clearly doing something daft here.
Any pointers?
Thanks

Did you call waitForPageToLoad() after navigating to the page in Selenium? I'm betting your DOM hasn't loaded yet when your click is called.

Please confirm that while verifying with xpath and css you used selenium.click("//[#id=joinButton]") and selenium.click("css=#joinButton") respectively.
You can also keep selenium.isElementPresent("joinButton") in a loop until some timeout (30/60 sec) and the moment selenium finds this element, it would come out of this loop and execute click command.

I realised what the issue was here. The Selenium class was being extended, and the click method overridden to only accept CSS selectors. With this cruft now removed, the operation works as expected.
Thanks for your help folks.