We Keep Coding

Persistent background checking with Selenium

First time poster, long time lurker. I've gotten a lot of great advice to problems from this site, but I haven't found anything here for the topic of this question. Normally I would bug our SME at the office but he's indisposed.
So, we use Selenium Web Driver to do automated tests. I'm working on an application with some mapping and demographics features, so my tests are very function vs. form oriented.
My tests are written such that I have classes/methods that are a part of the puzzle (the site is essentially one workflow where you go from page 1 to page 5 and the same actions need to be performed in steps 2-3, for example, but test A might do something different on page 4 to see the result in page 5. Clear as mud?
Anyways, during manual tests, I can sometimes see an error message pop up on the site (a hidden div that will become visible if it detects an error, but it's usually a very generic/vague error). This error sometimes pops up even if you're able to go through the flow with no other ill-effects. However, I want to capture when these errors happen so I can look for patterns - if this means just logging it to console or failing the test...I can figure that out later.
The immediate problem is having a persistent check in place that will always look for this error during every test. I could create a method and call it in my "action" methods, though this would leave gaps and slow the tests down. Is there any clever way of implementing something like this without slowing the tests down or calling this check every time I do a step in the process? Also, forgive me, I'm still learning Java and the selenium web driver, so if I've said anything stupid, that's why.

Since this message is persistent if it is there, you might try adding a check for it in your test case teardown method. (I would recommend that you reduce the implicit wait time before you do that check, though, otherwise each test will take an extra amount of time waiting for an error message that isn't there.)
Another possible option is to define your own listener on your own test runner and update the testFinished() method to go check for your error message. See this for some ideas.

Since it sounds like the error messages are always in known locations on each page, I would create a method (or methods, depending on how many error message locations there are on a given page) that looks to see if an error exists and then log it before leaving the page. It sounds like you might be using the page object model. If so, you can add these methods to the each relevant page object for easy access.
NOTE: Checking for errors once before you leave the page may not be enough. You may need to check each time you do some action that might cause an error. This is probably not a bad practice anyway because it will help in debugging errors because you will notice an error closer to the time it was triggered, thus narrowing down what caused the error.
If you have the ability, do something like log it as a warning so that it doesn't fail your test but stands out (and is searchable) in your logs.
You seem concerned that checking for all these errors will significantly slow your script. If properly written, it shouldn't add a significant delay. One significant delay you might run into is if you have implicit waits turned on and are checking for elements that don't exist (e.g., unless there's an error). This will cause the implicit wait to be applied each time you search for the missing element and will likely add significant time to the run time. My suggestion is to turn off implicit waits and add explicit waits only where needed. Searching for any element will add some time but 25ms here and there should be negligible in an overall script run.

Have you tried using EventFiringWebDriver?
There is an answer here on what it does:
What is EventFiringWebDriver in selenium?
Newer selenium versions have more types of events present in the interface, which can broaden its use on these types of tests.

Selenium Webdriver.load.strategy unstable

I'm having problems loading my page with webdriver. My current (problematic) solution involves using the unstable load with firefox, but I'm open to other solutions.
The Core Problem
The root of all my problems comes from the fact my page will never fully load when I call it normally with webdriver, and thus will never preform the 2nd step, it's just always loading. It loads fine when you just go to the site with a normal browser. I've tried out a few work arounds that work intermittently, including opening the driver to google, and then going to the page which sometimes makes it load, and with
IJavaScriptExecutor js = (IJavaScriptExecutor)Driver;
js.ExecuteScript("return window.stop");
as suggested by this question, which sometimes makes it continue without loading. But neither of those work consistently at all (probably <50% of the time)
The best I've got so far is using the unstable load strategy. However that has its own different problems, all of which only happen sometimes.
1) Sometimes it doesn't wait for the page to load at all, and I get an "Unable to locate element:" exception within milliseconds of loading the page, despite the fact that I have a wait set up:
new WebDriverWait(cdriver, 30).until(ExpectedConditions.visibilityOf(cdriver.findElement(By.id(myId))));
Thread.sleep(3000); solves that problem, but I've read that is a sloppy way to do things.
2) If I get passed that step, my test should click one link, then click another and continue on with the test. However, it often gets stuck after that first click. So if my code is like this:
Thread.sleep(3000);
element1.click();
System.out.println("clicked!");
element2.click();
The first click (which doesn't load a new page, by the way, just a pop up on the same page) will work, but then the system will never print out "clicked!", it's stuck in the same way would be loading the page initially (without the unstable load thing). If/when it makes it over that hurdle, I think the rest of the test is fine.
Any ideas 1) why it works sometimes but not others. 2) how to fix it 3) how to just get my page to load in the first place
Thank you!

Try the following approach:
Don't wait for visibility of element on the page - in your case By.id(myId).
The element can become visible much earlier than the page is fully loaded, before all java scripts are loaded and fired etc.
Wait for some element on the page to be clickeable instead. For example you can wait until link becomes clickeable in this way:
By locatorOfLink1 = By.xpath(....); // By.id, By.name, By.linkText etc.
wait.until(ExpectedConditions.elementToBeClickable(locatorOfLink1));
If the above will not help for problem #2 (the webdriver gets stuck after the click), then send ENTER key to the link instead of the click:
element1.sendKeys(Keys.ENTER);

Selenium: sendKeys()

I tried using Java + Selenium to automatically login to my Gmail account however each time I am blocked by Google. When I log in manually (even on the very same server) I have no such issue. Further, if I use "sendKeys" type lib (https://pypi.python.org/pypi/SendKeys/0.3) I have no such problem.
My question is two fold:
What are the tech differences between how Selenium "sends keys" and how a normal user would? Or how the Python library does?
Is there a way to modify selenium so it mimics a real user? Or is there another library to use instead of Selenium that mimicks a real user?
Thanks

Actually its the difference of speed. Human being types slowly and script write it very fastly. So, you have to decrease your script speed. So on the whole you have to built better bot.
Better bots are not going to be detected by any server.

You should include some delay in your code to cheat google that your code is real human - that's it :) Thread.sleep could be the easiest solution here.

Open gmail & login & open latest received mail
DriverObj.get("http://www.gmail.com");
DriverObj.findElement(By.id("gmail-sign-in")).click();
DriverObj.findElement(By.id("Email")).sendKeys("youremailID");
DriverObj.findElement(By.id("next")).click();
DriverObj.findElement(By.id("Passwd")).sendKeys("yourpassword");
DriverObj.findElement(By.id("signIn")).click();
Thread.sleep(2000);
WebElement tableelement=DriverObj.findElement(By.xpath("//div[#class='Cp']/div/table//tbody"));
java.util.List<WebElement> tableelement1=tableelement.findElements(By.xpath("//div[#class='Cp']/div/table//tbody/tr"));
int tableelementsize=tableelement1.size();
int i;
if(tableelementsize>0){
//DriverObj.navigate().refresh();
for(i=0;i<tableelementsize;i++){
DriverObj.findElement(By.xpath("//div[#class='Cp']/div/table//tbody/tr/td[6]")).click();
}
}else {
if(tableelementsize==0){
DriverObj.navigate().refresh();
for(i=0;i<tableelementsize;i++){
DriverObj.findElement(By.xpath("//div[#class='Cp']/div/table//tbody/tr/td[6]")).click();
}
}
path of mail box might be change (you have need to change according to your work)

Answer to 1st question :
Sendkeys() method refers to user input to a particular text box on the web page where as we can also use javascriptExecuter for the same operation.
Answer to 2nd question :
As per my knowledge NO. but I am not so sure about it.
In Java use the following code to log in to Gmail.
However I can say after clicking on sign in button the Inbox page takes time to load (at around 10 seconds max)
So in time being if you are searching for another element without any wait (implicit or explicit) then it will throw error.
driver.get("http://www.gmail.com/");
driver.findElement(By.id("Email")).sendKeys("userName");
driver.findElement(By.id("next")).click();
driver.findElement(By.id("Passwd")).sendKeys("password");
driver.findElement(By.id("signIn")).click();
Then you can use Thread.sleep(10000) or if you have written any wait method to make the driver wait for the next object to be visible , You can use that one.

Absolutely, Selenium just provides library to write automated test scripts. If you wanted to act as a human. Then you should add some wait statement in between your test steps into entire script. Until you didn't add any wait in script, it will behave like a robotic test only. If you take any examples of test automation tool, tool doesn't add any extra information/action towards automation testing.Thats what automated testing is... It's not automatic testing. It's just computer aided testing done by machine. I hope now you'll be more clear with above example.

Another solution is send a wrong e-mail as first attempt, after the error message, you could apply the right e-mail, this is an human action.

Selenium Webdriver with Java: Element not found in the cache - perhaps the page has changed since it was looked up

I am initializing a variable in the beginning of my class:
public WebElement logout;
Later on in the code, in some method, the first time I encounter the logout button, I assign a value to that variable (in the brackets of an if/else statement):
logout = driver.findElement(By.linkText("Logout"));
logout.click();
I then use "logout" once more, successfully, at another stage of my test:
logout.click();
And at the end of the test, at a place where the element is the same (By.linkText ("Logout")), I get this error:
Element not found in the cache - perhaps the page has changed since it was looked up
Why?
EDIT: Actually, I dont successfully use the logout.click(); commant at another stage of my test. Looks like I cant use it again. I have to create a logout1 webelement and use it...

If there has been any changes to the page after you have initially found the element the webdriver reference will now contain a stale reference. As the page has changed, the element will no longer be where webdriver expects it to be.
To solve your issue, try finding the element each time you need to use it - writing a small method that you can call as and when is a good idea.
import org.openqa.selenium.support.ui.WebDriverWait
public void clickAnElementByLinkText(String linkText) {
wait.until(ExpectedConditions.presenceOfElementLocated(By.linkText(linkText)));
driver.findElement(By.linkText(linkText)).click();
}
Then within your code you'd only need to:
clickAnElementByLinkText("Logout");
So each time it will find the element and click on it, as such even if the page changes as it is 'refreshing' the reference to that element it all successfully click it.

The browser rebuilds the DOM structure of the dynamic pages, so the elements do not need to keep you have to find them before to use.
Using XPath, for example. This approach is not correct(can cause exception org.openqa.selenium.StaleElementReferenceException in the future):
WebElement element = driver.findElement(By.xpath("//ul[#class=\"pagination\"]/li[3]/a"));
...// Some Ajax interaction here
element.click(); //<-- Element might not be exists
This approach is correct:
driver.findElement(By.xpath("//ul[#class=\"pagination\"]/li[3]/a")).click();

This is because you are not giving proper time to load the page.So you have to give Thread.sleep(); code for the Given page.
I am also getting the same issue for my project but after using the Thread.sleep(); its working fine for me give the web page as much as it is possible for 30 to 50 secs.

Wait for ext-js dialog to load with Selenium

When testing an ext-js application with Selenium (WebDriver), I have a button that brings up a dialog. The test then fills out some form fields, clicks buttons, etc. The problem is that selenium starts performing these actions before the dialog is fully done building.
I know that I can tell selenium to wait for a specific element to appear, but if I want to generalize the case to "wait for the dialog to finish loading", is there a way to do that?

The short answer is, "No, theres no simple way to do that." to understand why this is so, consider this: How do you define "the whole DOM is loaded?" Is it when the last closing </html> tag is downloaded? When it's parsed and added to the DOM? Is it when the document readyState property is "interactive," "contentReady," or some other value? What about DOM elements added via JavaScript? What if that JavaScript is fired via setTimeout()? What about processing XmlHttpRequests? There is no one-size-fits-all answer to that question that is correct 100% of the time for all use cases.
Waiting for an element is the right approach. Identifying the right element can be challenging when using extJs, because of the generated IDs. I've often found it more effective to use class attributes with that particular JavaScript framework.