I want to send value to "search" input but sendKeys() method doesn't work. I checked the "xpath" path manually, added the clear() method before putting values but nothing changed. Also added WebdriverWait.
There is method:
public Seminar searchSeminarByName(String seminarName){
WebElement searchInp = driver.findElement(searchInput);
if (searchInp == null){
log.error("Can not find element by {}", searchInput);
}else{
log.info("Founded element ex {}", searchInp);
log.info("Founded element {}", searchInp.getTagName());
}
WebDriverWait wait = new WebDriverWait(driver,30);
wait.until(ExpectedConditions.elementToBeClickable(searchInp));
searchInp.click();
searchInp.sendKeys(seminarName);
wait.until(ExpectedConditions.textToBePresentInElement(searchInp, seminarName));
searchInp.sendKeys(Keys.ENTER);
WebElement seminarId = driver.findElement(By.xpath("//td[1]"));
WebElement seminarNametd = driver.findElement(By.xpath("//td[2]"));
WebElement seminarStatus =driver.findElement(By.xpath("//td[4]"));
return new Seminar(Long.parseLong(seminarId.getText()), seminarNametd.getText(),seminarStatus.getText());
}
This method sends value on "Search" input and returns found info from the page.
Check the info in another class by using the method:
#Test
public void searchFieldCheck(){
Seminar seminar = seminarPage.searchSeminarByName(groupName);
Assert.assertEquals(groupName, seminar.getName());
}
The logs to run test:
Starting ChromeDriver 87.0.4280.20 (c99e81631faa0b2a448e658c0dbd8311fb04ddbd-refs/branch-heads/4280#{#355}) on port 11594
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
[1612355832.736][WARNING]: This version of ChromeDriver has not been tested with Chrome version 88.
февр. 03, 2021 2:37:12 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
2021-02-03 14:37:22.387 [main] INFO SeminarPage - Founded element ex [[ChromeDriver: chrome on WINDOWS (3e6dd669a7b7ad62f24b6dd150da861b)] -> xpath: //input/parent::div/*]
2021-02-03 14:37:22.394 [main] INFO SeminarPage - Founded element input
org.openqa.selenium.TimeoutException: Expected condition failed: waiting for element ([[ChromeDriver: chrome on WINDOWS (3e6dd669a7b7ad62f24b6dd150da861b)] -> xpath: //input/parent::div/*]) to be selected (tried for 30 second(s) with 500 milliseconds interval)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'DESKTOP-PS8FAB8', ip: '172.18.76.97', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '14.0.2'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 88.0.4324.104, chrome: {chromedriverVersion: 87.0.4280.20 (c99e81631faa0..., userDataDir: C:\Users\Frederik\AppData\L...}, goog:chromeOptions: {debuggerAddress: localhost:41163}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: WINDOWS, platformName: WINDOWS, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:virtualAuthenticators: true}
Session ID: 3e6dd669a7b7ad62f24b6dd150da861b
at org.openqa.selenium.support.ui.WebDriverWait.timeoutException(WebDriverWait.java:95)
at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:272)
at SeminarPage.searchSeminarByName(SeminarPage.java:48)
at SeminarPageTest.searchFieldCheck(SeminarPageTest.java:34)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
Update
The issue was too fast work of Selenium added to code Thread.sleep(500);
Code(used lombok):
#SneakyThrows
public Seminar searchSeminarByName(String seminarName){
WebElement searchInp = driver.findElement(searchInput);
if (searchInp == null){
log.error("Can not find element by {}", searchInput);
}else{
log.info("Founded element ex {}", searchInp);
log.info("Founded element {}", searchInp.getTagName());
}
//WebDriverWait wait = new WebDriverWait(driver,30);
//wait.until(ExpectedConditions.elementToBeClickable(searchInp));
searchInp.click();
Thread.sleep(500); //Required waite, because "sendKeys" is ignored for some time after render.
//wait.until(ExpectedConditions.elementToBeSelected(searchInp));
searchInp.sendKeys(seminarName);
//wait.until(ExpectedConditions.textToBePresentInElement(searchInp, seminarName));
Thread.sleep(500);
searchInp.sendKeys(Keys.ENTER);
Thread.sleep(700);
WebElement seminarId = driver.findElement(By.xpath("//td[1]"));
WebElement seminarNametd = driver.findElement(By.xpath("//td[2]"));
WebElement seminarStatus =driver.findElement(By.xpath("//td[4]"));
return new Seminar(Long.parseLong(seminarId.getText()), seminarNametd.getText(),seminarStatus.getText());
}
The test is passed:
Starting ChromeDriver 88.0.4324.96 (68dba2d8a0b149a1d3afac56fa74648032bcf46b-refs/branch-heads/4324#{#1784}) on port 19322
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
февр. 03, 2021 9:27:01 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
2021-02-03 21:27:06.999 [main] INFO SeminarPage - Founded element ex [[ChromeDriver: chrome on WINDOWS (50d08a5eaebefd27100eb7990b7149f8)] -> xpath: //input/parent::div/*]
2021-02-03 21:27:07.006 [main] INFO SeminarPage - Founded element input
The problem is with this line:
wait.until(ExpectedConditions.elementToBeClickable(searchInp));
According to your log, searchInp is not a clickable element. Inspect your page and maybe change this expected condition to "element present". For example:
WebDriverWait wait = new WebDriverWait(driver, 30);
wait.until(ExpectedConditions.presenceOfElementLocated(locator));
Where locator is one of multiple localization strategies implemented in the org.openqa.selenium.By class:
className(String className)
cssSelector(String selectorName)
id(String id)
linkText(String linkText)
name(String name)
partialLinkText(String partialLinkText)
tagName(String tagName)
xpath(String xpath)
The most powerful in my opinion is by XPath. It is also the most complicated. For your case, you could do something like By locator = By.xpath("some xpath string here"); and then pass that locator to ExpectedConditions.
I also suggest you read on web driver click method so that you could have a better understanding under what conditions and what type of elements this function will work on.
UPDATE:
Without having intimate knowledge of the structure of the page, is very hard to figure out why sendKeys will not work. A potential problem is that the field you need to send keystrokes to is in an iframe. If this is the case, you must tell the web driver to switch to the iframe and then send they keystrokes to the field. Once you are done, you need to switch back from the iframe to the main window (frame). A quick search on the web for "selenium switch to iframe" should point you in the right direction IF THIS IS THE ISSUE.
Related
I am implementing a script that logs into a web page and perform certain actions, that are irrelevant in this case. This is the code that executes the login:
WebDriver driver;
JavascriptExecutor exec;
String dni = "...";
String passwd = "...";
driver = new ChromeDriver();
exec = (JavascriptExecutor) driver;
// Search web page
driver.get("http://example.com");
// Accept cookies
driver.findElement(By.id("aceptarCookies")).click();
// Login
driver.findElement(By.id("areaPersonal")).click();
driver.findElement(By.id("tab_login_user")).sendKeys(dni);
driver.findElement(By.id("tab_login_password")).sendKeys(passwd);
driver.findElement(By.id("loginFormHeader")).findElement(By.id("buttonHeader")).click();
The bot logs in successfully and, once logged in, if I want to execute any instruction I get very strange behaviors. If I try to search for any item within the home page it doesn't find it because it hasn't been loaded yet, for example adding this instruction just below:
// Search body item
driver.findElement(By.id("my_panel_page"));
However, if I add an implicit wait:
driver.manage().timeouts().implicitlyWait(20, TimeUnit.SECONDS);
// Search body item
driver.findElement(By.id("my_panel_page"));
Or an explicit wait:
WebDriverWait wait = new WebDriverWait(driver, 20);
wait.until(ExpectedConditions.presenceOfElementLocated(By.id("id_of_new_item")));
driver.findElement(By.id("my_panel_page"));
The script hangs and does not execute any more instructions. For example, if I add the following print, it never gets executed:
WebDriverWait wait = new WebDriverWait(driver, 20);
wait.until(ExpectedConditions.presenceOfElementLocated(By.id("id_of_new_item")));
System.out.println("Page loaded!!!");
driver.findElement(By.id("my_panel_page"));
The behavior is very strange, it is as if selenium does not work once the page has been loaded, it only works when the new page is still loading and the HTML objects I want to use are not yet available. I'm using the following versions:
ChromeDriver: 109.0.5414.74
Chrome: 109.0.5414.120
Selenium: 3.141.59
JDK: 11.0.16.1
I can't indicate specific data of the web page I'm using because it is detected as spam, but I have previously made sure that the items I'm searching on the script exists in the HTML code, in fact, the script worked correctly until a few months ago.
EDIT:
I add some examples of output to better understand the problem. If I execute the instruction driver.findElement(By.id("my_panel_page")); without using any type of wait the output is as follows:
... (Initialize ChromeDriver)
no such element: Unable to locate element: {"method":"css selector","selector":"#my_panel_page"}
(Session info: chrome=109.0.5414.120)
For documentation on this error, please visit: https://www.seleniumhq.org/exceptions/no_such_element.html
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'xxx', ip: 'xxx', os.name: 'Windows 11', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.16.1'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 109.0.5414.120, chrome: {chromedriverVersion: 109.0.5414.74 (e7c5703604da..., userDataDir: C:\Users\xxx\AppData\Loca...}, goog:chromeOptions: {debuggerAddress: localhost:52373}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: WINDOWS, platformName: WINDOWS, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: 533a8a11a33254b8ab788522dada77ab
*** Element info: {Using=id, value=my_panel_page}
If I use some kind of wait I didn't get an output, the ChromeDriver initializes and logs into the web page successfully, but its execution hangs when it reaches the wait statement. It only prints the ChromeDriver initialization info:
Starting ChromeDriver 109.0.5414.74 (e7c5703604daa9cc128ccf5a5d3e993513758913-refs/branch-heads/5414#{#1172}) on port 15136
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
feb. 01, 2023 5:28:28 P. M. org.openqa.selenium.remote.ProtocolHandshake createSession
INFORMACIËN: Detected dialect: W3C
presenceOfElementLocated()
presenceOfElementLocated() is the expectation for checking that an element is present on the DOM of a page. This does not necessarily mean that the element is visible.
Solution
So instead of presenceOfElementLocated(), you have to induce WebDriverWait for the visibilityOfElementLocated() and you can use the following solution:
So instead of presenceOfElementLocated(), you have to induce WebDriverWait for the visibilityOfElementLocated() and you can use the following solution:
try
{
new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfElementLocated(By.id("id_of_new_item")));
System.out.println("new_item is visible");
System.out.println("Page loaded!!!");
driver.findElement(By.id("my_panel_page"));
// lines of the program
}
catch(TimeoutException e) {
System.out.println("new_item is not visible");
}
// remaining lines of the program
My java selenium java test case with Chrome version 77 runs fine when options.setHeadless(false); but fails when I change this line options.setHeadless(true);
The java code was exported by the katalon selenium recorder chrome extension.
Output when it runs fine with options.setHeadless(false);:
Title of the page is 9 -> Scan History
Title of the page is 10 -> Scan History
Title of the page is 10.1 -> Scan History
Title of the page is 10.2 -> Scan History
Title of the page is 10.3 -> Scan History
Title of the page is 10.4 -> Scan History
Title of the page is 10.5 -> Scan History
When I run the same code with options.setHeadless(true); I get a below warning as well as an error message and the execution terminates before completion.
Title of the page is 10 -> Scan History
[0530/125542.102:INFO:CONSOLE(0)] "Error parsing header X-XSS-Protection: 1; mod
e=block, 1;mode=block: expected semicolon at character position 13. The default
protections will be applied.", source: https://qualysguard.myshop.com/fo/scan/sca
nList.php (0)
Title of the page is 10.1 -> Scan History
Title of the page is 10.2 -> Scan History
Title of the page is 10.3 -> Scan History
Exception in thread "main" org.openqa.selenium.NoSuchElementException: no such e
lement: Unable to locate element: {"method":"css selector","selector":"#ext\-gen
117"}
(Session info: headless chrome=77.0.3865.75)
For documentation on this error, please visit: https://www.seleniumhq.org/except
ions/no_such_element.html
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17
:03'
System info: host: 'myhostserver', ip: '10.9.111.32', os.name: 'Windows Serve
r 2012 R2', os.arch: 'amd64', os.version: '6.3', java.version: '1.8.0_45'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 7
7.0.3865.75, chrome: {chromedriverVersion: 77.0.3865.40 (f484704e052e0..., userD
ataDir: C:\Users\1886\AppData\Loc...}, goog:chromeOptions: {debuggerAddress: l
ocalhost:56128}, javascriptEnabled: true, networkConnectionEnabled: false, pageL
oadStrategy: normal, platform: XP, platformName: XP, proxy: Proxy(), setWindowRe
ct: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 30
0000, script: 30000}, unhandledPromptBehavior: dismiss and notify}
Session ID: 09a309a747f7bcb42735bb8b6c39ad1f
*** Element info: {Using=id, value=ext-gen117}
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou
rce)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(
W3CHttpResponseCodec.java:187)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpRe
sponseCodec.java:122)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpRe
sponseCodec.java:49)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExe
cutor.java:158)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(Driv
erCommandExecutor.java:83)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.ja
va:552)
at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDrive
r.java:323)
at org.openqa.selenium.remote.RemoteWebDriver.findElementById(RemoteWebD
river.java:372)
at org.openqa.selenium.By$ById.findElement(By.java:188)
at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDrive
r.java:315)
at pack.QualysScan.testQualysScan(QualysScan.java:182)
at pack.QualysScan.main(QualysScan.java:284)
Here is the part of the code concerning the issue:
System.out.println("Title of the page is 10 -> " + driver.getTitle());
driver.findElement(By.id("body-header")).click();
driver.get("https://qualysguard.myshop.com/fo/scan/scanList.php");
System.out.println("Title of the page is 10.1 -> " + driver.getTitle());
driver.findElement(By.xpath("(.//*[normalize-space(text()) and normalize-space(.)='Add to my Calendar'])[1]/following::em[2]")).click();
System.out.println("Title of the page is 10.2 -> " + driver.getTitle());
System.out.println("Title of the page is 10.3 -> " + driver.getTitle());
driver.findElement(By.id("ext-gen117")).click();
System.out.println("Title of the page is 10.4 -> " + driver.getTitle());
System.out.println("Title of the page is 10.5 -> " + driver.getTitle());
String your_title = "Launch Compliance Scan";
And here is how the driver is setup:
public static void setUp() throws Exception {
System.setProperty("webdriver.chrome.driver", "H:\\Downloads\\Qualys\\vdi\\chromedriver.exe");
ChromeOptions options = new ChromeOptions();
options.setHeadless(true);
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(CapabilityType.ForSeleniumServer.ENSURING_CLEAN_SESSION, true);
capabilities.setCapability("chrome.switches", Arrays.asList("--incognito"));
options.merge(capabilities);
driver = new ChromeDriver(options);
driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
}
I'm sharing snapshot of element ext\-gen117 as visible on the portal.
Video of the issue:
https://www.youtube.com/watch?v=abSwsPtOOG4
Can you please suggest how can I overcome the issue?
This is likely caused by a race condition between the browser and selenium, where selenium attempts to click on something before it exists in the DOM. Using an explicit wait should resolve the issue:
WebDriverWait wait = new WebDriverWait (driver, 30);
WebElement element = wait.until(ExpectedConditions.elementToBeClickable(By.id("ext-gen117");
element.click();
I have encountered a weird problem while working with AShot.
Ashot works fine for whole screenShots but while selective screenShot it is BAD.
It throws error in CoordsProvider Class while getting co-ordinates of element.
am I using a faulty build or product?
public abstract class CoordsProvider implements Serializable {
public abstract Coords ofElement(WebDriver driver, WebElement element);
public Set<Coords> ofElements(WebDriver driver, Iterable<WebElement> elements) {
Set<Coords> elementsCoords = new HashSet<>();
for (WebElement element : elements) {
***Coords elementCoords = ofElement(driver, element); //fails here***
if (!elementCoords.isEmpty()) {
elementsCoords.add(elementCoords);
}
}
return Collections.unmodifiableSet(elementsCoords);
}
#SuppressWarnings("UnusedDeclaration")
public Set<Coords> ofElements(WebDriver driver, WebElement... elements) {
return ofElements(driver, Arrays.asList(elements));
}
#SuppressWarnings("UnusedDeclaration")
public Set<Coords> locatedBy(WebDriver driver, By locator) {
return ofElements(driver, driver.findElements(locator));
}
}
Stack trace
org.openqa.selenium.JavascriptException: javascript error: $ is not defined
(Session info: chrome=79.0.3945.130)
Build info: version: '3.14.0', revision: 'aacccce0', time: '2018-08-02T20:19:58.91Z'
System info: host: 'OPTIMIZEQ-LTP03', ip: '192.168.99.1', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.2'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 79.0.3945.130, chrome: {chromedriverVersion: 78.0.3904.105 (60e2d8774a81..., userDataDir: C:\Users\SHAILE~1.SIN\AppDa...}, goog:chromeOptions: {debuggerAddress: localhost:61006}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: XP, platformName: XP, proxy: Proxy(manual, http=localhos..., setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webdriver.remote.sessionid: f5a47b27537f019dacb73462732...}
Session ID: f5a47b27537f019dacb734627324a790
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:na]
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[na:na]
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187) ~[selenium-remote-driver-3.14.0.jar:na]
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122) ~[selenium-remote-driver-3.14.0.jar:na]
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49) ~[selenium-remote-driver-3.14.0.jar:na]
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158) ~[selenium-remote-driver-3.14.0.jar:na]
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:548) ~[selenium-remote-driver-3.14.0.jar:na]
at org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:485) ~[selenium-remote-driver-3.14.0.jar:na]
at ru.yandex.qatools.ashot.util.JsCoords.findCoordsWithJquery(JsCoords.java:30) ~[ashot-1.5.2.jar:na]
at ru.yandex.qatools.ashot.coordinates.JqueryCoordsProvider.ofElement(JqueryCoordsProvider.java:13) ~[ashot-1.5.2.jar:na]
at ru.yandex.qatools.ashot.coordinates.CoordsProvider.ofElements(CoordsProvider.java:21) ~[ashot-1.5.2.jar:na]
at ru.yandex.qatools.ashot.AShot.takeScreenshot(AShot.java:115) ~[ashot-1.5.2.jar:na]
at ru.yandex.qatools.ashot.AShot.takeScreenshot(AShot.java:132) ~[ashot-1.5.2.jar:na]
at com.optq.main.util.SeleniumDriverUtility.captureAShotElement(SeleniumDriverUtility.java:563) ~[classes/:na]
Any help or Alternatives For Ashot are welcome,Please help me into this.
This error message...
org.openqa.selenium.JavascriptException: javascript error: $ is not defined
(Session info: chrome=79.0.3945.130)
Build info: version: '3.14.0', revision: 'aacccce0', time: '2018-08-02T20:19:58.91Z'
System info: host: 'OPTIMIZEQ-LTP03', ip: '192.168.99.1', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.2'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 79.0.3945.130, chrome: {chromedriverVersion: 78.0.3904.105 (60e2d8774a81..., userDataDir: C:\Users\SHAILE~1.SIN\AppDa...}, goog:chromeOptions: {debuggerAddress: localhost:61006}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: XP, platformName: XP, proxy: Proxy(manual, http=localhos..., setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webdriver.remote.sessionid: f5a47b27537f019dacb73462732...}
...implies that the ChromeDriver was unable to interact with the Browsing Context i.e. Chrome Browser session.
Deep dive
As per the documentation in ReferenceError: "x" is not defined this error means there is a non-existent variable referenced somewhere within the DOM Tree. This variable needs to be declared, or you need to make sure it is available in your current script or scope.
Hint: When loading a library (such as jQuery), make sure it is loaded before you access library variables, such as "$". Put the tag that loads the library before your code that uses it.
As per the discussion JavaScript/jQuery - “$ is not defined- $function()” error #Ketan mentions that this error occurs when you have not made jQuery available to your script, i.e. possibly the JavaScript / jQuery / AJAX haven't completed rendering the HTML DOM.
Solution
In these cases there are 3(three) different approaches available to solve the issue as follows:
The jQuery library is a single JavaScript file, and you reference it with the HTML <script> tag within the <head> section as follows:
<head>
<script src="jquery-3.4.1.min.js"></script>
</head>
This goes out and gets the jQuery code from the source.
Note: You do not have to include type="text/javascript" inside the <script> tag as this is not required in HTML5. JavaScript is the default scripting language in HTML5 and in all modern browsers.
You can also download the jQuery library and reference it locally on the server.
You can induce WebDriverWait inconjunction with ExpectedConditions for the element's desired state either to be present / visible / interactable.
You can find a detailed discussion in Selenium: How selenium identifies elements visible or not? Is is possible that it is loaded in DOM but not rendered on UI?
This usecase
As you mentioned, the following line fails:
Coords elementCoords = ofElement(driver, element);
This method takes one of the arguments (last) as element, but while defining you seem to be treating it as a list of elements, as in:
#SuppressWarnings("UnusedDeclaration")
public Set<Coords> ofElements(WebDriver driver, WebElement... elements) {
return ofElements(driver, Arrays.asList(elements));
}
Seems some mismatch in argument types here.
Additional considerations
You need to take care of a couple of things more:
You are using chromedriver=78.0.3904.105
Release Notes of chromedriver=78.0 clearly mentions the following :
Supports Chrome version 78
You are using chrome=79.0
Release Notes of ChromeDriver v79.0 clearly mentions the following :
Supports Chrome version 79
Your Selenium Client version is 3.14.0 of 2018-08-02T20:19:58.91Z which is almost 1.5 years older.
Your JDK version is 11.0.2.
So there is a clear mismatch between JDK v8u111 , Selenium Client v3.3.1 , ChromeDriver v2.41 and the Chrome Browser v79.0
Solution
Ensure that:
JDK is upgraded to current levels JDK 8u241.
Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v80.0 level.
Chrome is updated to current Chrome Version 80.0 level. (as per ChromeDriver v80.0 release notes)
I found a solution here, https://medium.com/virtualmind-io/jquery-injection-for-selenium-automation-tests-f6121ea57993. Basically it injects jquery into the page.
So I followed the steps and created this method that i used before calling AShot methods and could solve this issue, leaving the code here:
JavascriptExecutor js = (JavascriptExecutor) driver;
if(!(Boolean) js.executeScript("return (typeof jQuery != \"undefined\")")) {
js.executeScript(
"var headID = document.getElementsByTagName('head')[0];" +
"var newScript = document.createElement('script');" +
"newScript.type = 'text/javascript';" +
"newScript.src = 'https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js';" +
"headID.appendChild(newScript);");
WebDriverWait waitJQ = new WebDriverWait(driver, 30);
Function<WebDriver, Boolean> jQueryAvailable = WebDriver -> (
(Boolean) js.executeScript("return (typeof jQuery != \"undefined\")")
);
waitJQ.until(jQueryAvailable);
}
I am trying of different ways to select a specific button using seleninum webdriver with java but unfortunately, nothing is working.
When I tested using the Selenium IDE is working. I copied the same xpath, for example, but when I try to test in my java application nothing is working. I tried using different ways, By.cssSelector and By.path.
This is my html:
<section class="fd-section"><fd-action-bar><div class="fd-action-bar"><fd-action-bar-header class="fd-action-bar__header"><fd-action-bar-title><h1 class="fd-action-bar__title"> Applications </h1></fd-action-bar-title></fd-action-bar-header><fd-action-bar-actions class="fd-action-bar__actions"><y-list-search _nghost-c4="" hidden=""><!----><!----><div _ngcontent-c4="" clickoutsideevents="click,mousedown" excludebeforeclick="true" class="ng-star-inserted"><!----><button _ngcontent-c4="" fd-button="" class="fd-button xyz-icon--search fd-button--light ng-star-inserted"></button><!----></div></y-list-search><y-list-filter _nghost-c5="" hidden=""><!----></y-list-filter><!----><button class="open-create-namespace-modal fd-button xyz-icon--add ng-star-inserted" fd-button=""> Create Application </button></fd-action-bar-actions></div></fd-action-bar></section>
I need to select the button with the text " Create Application ".
When I created a test using Selenium IDE the xpath for this button is:
//button[contains(.,' Create Application')]
Basically, my java code is:
public WebElement wElement;
wElement = driver.findElement(By.xpath("//button[contains(.,' Create Application')]"));
wElement.click();
This is the exception message:
org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//button[contains(.,' Create Application')]"}
(Session info: chrome=76.0.3809.100)
(Driver info: chromedriver=72.0.3626.69 (3c16f8a135abc0d4da2dff33804db79b849a7c38),platform=Mac OS X 10.14.6 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '3.14.0', revision: 'aacccce0', time: '2018-08-02T20:19:58.91Z'
System info: host: 'C02WW0BZHTD8', ip: 'fe80:0:0:0:8f6:17e1:1a28:1e23%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.14.6', java.version: '1.8.0_171'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 72.0.3626.69 (3c16f8a135abc..., userDataDir: /var/folders/2r/99nyn7t16cz...}, cssSelectorsEnabled: true, databaseEnabled: false, goog:chromeOptions: {debuggerAddress: localhost:60374}, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, proxy: Proxy(), rotatable: false, setWindowRect: true, takesHeapSnapshot: true, takesScreenshot: true, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore, version: 76.0.3809.100, webStorageEnabled: true}
Session ID: b2341899cd9b62b0169b02371aaa3018
*** Element info: {Using=xpath, value=//button[contains(.,' Create Application')]}
When this piece of code is being executed, is the button loaded into the page already?
{implicit: 0, pageLoad: 300000, script: 30000}, suggests that the driver will not implicitly wait for finding any elements. i.e. if the element is not available, it will throw the exception immediately.
Try driver.manage().timeouts().implicitlyWait(10,TimeUnit.SECONDS); before you attempt to find the button.
Also try using below locators -
//button[contains(.,'Create Application')]
//button[contains(text(),'Create Application')]
If none of the above works, please can you provide a URL (if it is public)
Also check whether the button is inside a frame.
You can use JavascriptExecutor interface in this case and try to click the button.
WebElement element = driver.findElement(By.xpath("//button[contains(text(),'Create Application')]"));
JavascriptExecutor executor = (JavascriptExecutor)driver;
executor.executeScript("arguments[0].click();", element);
Also give a try with WebDriverWait
new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//button[contains(text(),'Create Application')]"))).click();
The desired element is an Angular element so to locate the element you have to induce WebDriverWait for the elementToBeClickable() and you can use either of the following Locator Strategies:
cssSelector:
new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.cssSelector("fd-action-bar-actions.fd-action-bar__actions button.open-create-namespace-modal.fd-button.xyz-icon--add.ng-star-inserted"))).click();
xpath:
new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//button[#class='open-create-namespace-modal fd-button xyz-icon--add ng-star-inserted' and contains(., 'Create Application')]"))).click();
Additional Considerations
Ensure that:
JDK is upgraded to current levels JDK 8u212.
Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v76.0 level.
Chrome is updated to current Chrome Version 76.0 level. (as per ChromeDriver v76.0 release notes)
Here you can find a detailed discussion on NoSuchElementException, Selenium unable to locate element
i try to make a web crawler with selenium.
My program fire a StaleElementReferenceException.
I thought that were because i crawl a page recursive and when a page have no more links the function navigate to next page and not previously to the parent page.
Therefore i have introduced a tree data structure to navigate back to the parent when the current url not equal the parent url. But this was not the solution for my problem.
Can anybody help me?
Code:
public class crawler {
private static FirefoxDriver driver;
private static String main_url = "https://robhammond.co/tools/seo-crawler";
private static List<String> uniqueLinks = new ArrayList<String>();
public static void main(String[] args) {
driver = new FirefoxDriver();
Node<String> root = new Node<>(main_url);
scrape(root, main_url);
}
public static void scrape(Node<String> node, String url) {
if(node.getParent() != null && (!driver.getCurrentUrl().equals(node.getParent().getData()))) {
driver.navigate().to(node.getParent().getData());
}
driver.navigate().to(url);
List<WebElement> allLinks = driver.findElements(By.tagName("a"));
for(WebElement link : allLinks) {
if(link.getAttribute("href").contains(main_url) && !uniqueLinks.contains(link.getAttribute("href")) && link.isDisplayed()) {
uniqueLinks.add(link.getAttribute("href"));
System.out.println(link.getAttribute("href"));
scrape(new Node<>(link.getAttribute("href")), link.getAttribute("href"));
}
}
}
}
And this is the output from the console:
D:\Programme\openjdk-12.0.1_windows-x64_bin\jdk-12.0.1\bin\java.exe "-javaagent:D:\Programme\JetBrains\IntelliJ IDEA 2019.1.2\lib\idea_rt.jar=60461:D:\Programme\JetBrains\IntelliJ IDEA 2019.1.2\bin" -Dfile.encoding=UTF-8 -classpath C:\Users\admin\Desktop\SeleniumWebScraper\out\production\SeleniumWebScraper;D:\Downloads\selenium-server-standalone-3.141.59.jar de.company.crawler.crawler
1557924446770 mozrunner::runner INFO Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "-marionette" "-foreground" "-no-remote" "-profile" "C:\\Users\\admin\\AppData\\Local\\Temp\\rust_mozprofile.YqmEqE8y1pjv"
1557924447037 addons.webextension.screenshots#mozilla.org WARN Loading extension 'screenshots#mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1557924447037 addons.webextension.screenshots#mozilla.org WARN Loading extension 'screenshots#mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1557924447037 addons.webextension.screenshots#mozilla.org WARN Loading extension 'screenshots#mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1557924448047 Marionette INFO Listening on port 60468
1557924448383 Marionette WARN TLS certificate errors will be ignored for this session
Mai 15, 2019 2:47:28 NACHM. org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
JavaScript warning: https://robhammond.co/js/jquery.min.js, line 4: Using //# to indicate sourceMappingURL pragmas is deprecated. Use //# instead
https://robhammond.co/tools/seo-crawler#content
https://twitter.com/intent/tweet?text=SEO%20Crawler&url=https://robhammond.co/tools/seo-crawler&via=robhammond
Exception in thread "main" org.openqa.selenium.StaleElementReferenceException: The element reference of <a href="/tools/"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
For documentation on this error, please visit: https://www.seleniumhq.org/exceptions/stale_element_reference.html
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'DESKTOP-admin', ip: '192.168.233.1', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '12.0.1'
Driver info: org.openqa.selenium.firefox.FirefoxDriver
Capabilities {acceptInsecureCerts: true, browserName: firefox, browserVersion: 66.0.5, javascriptEnabled: true, moz:accessibilityChecks: false, moz:geckodriverVersion: 0.24.0, moz:headless: false, moz:processID: 19124, moz:profile: C:\Users\admin\AppData\Loca..., moz:shutdownTimeout: 60000, moz:useNonSpecCompliantPointerOrigin: false, moz:webdriverClick: true, pageLoadStrategy: normal, platform: WINDOWS, platformName: WINDOWS, platformVersion: 10.0, rotatable: false, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify}
Session ID: b3b87675-57c8-4b48-9a20-8df5e4d37503
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:285)
at org.openqa.selenium.remote.RemoteWebElement.getAttribute(RemoteWebElement.java:134)
at de.company.crawler.crawler.scrape(crawler.java:33)
at de.company.crawler.crawler.scrape(crawler.java:38)
at de.company.crawler.crawler.main(crawler.java:20)
Process finished with exit code 1
When you navigate away from the first page all WebElements in the allLinks list get lost.
I would recommend converting it from the list of WebElement to the list of normal Strings like:
List<String> allLinksHrefs = allLinks.stream().map(link -> link.getAttribute("href")).collect(Collectors.toList());
and iterate through this new allLinksHrefs list instead.
You can use a hash-based collection for holding the uniqueLinks like HashSet - this way duplicates will be automatically eliminated
The current approach can take days to complete, consider using Selenium Grid and running your scraper in Parallel