We Keep Coding

Selenium chrome driver (Java) click() login button doesn't work programmatically but works manually

I'm just doing some test automation of web UI, specifically this page https://autorefi.capitalone.com/login/
I am locating the lastname, zipcode, ssn input boxes and typing in data (the data here doesn't matter). I am then simply using the locator to click the "Sign In" button. The problem is, everytime I run this within my code (Java) using selenium/chromedriver, I get an error
Sorry, we weren't able to log you in. If you continue to see this error, make sure you're using one of our supported browsers.
The problem is this is not the correct error message. You can try this yourself by simply opening another tab and entering a random lastname, zipcode, and last 4 digit of SSN. Conversely, if you actually had an offer with Capital one, it would bring up a different page completely. The point is, the first error message I posted only comes via selenium and is not correct. The correct error message is:
Sorry, it looks like you don't have an offer with Capital one.
I tried sleeping the thread before clicking the button ,because I thought it was maybe clicking it too fast, but it still didn't work. I a bit perplexed why doing the same set of operations manually seems to work, but launching this programmatically through selenium. Can anyone provide any insight here? My code is:
WebElement element;
WebDriver driver = null;
ChromeOptions options = new ChromeOptions();
options.setPageLoadStrategy(PageLoadStrategy.NONE);
driver = new ChromeDriver(options);
WebDriverManager.getInstance(CHROME).setup();
// TODO: PROD
driver.get("https://autorefi.capitalone.com/login/");
WebElement refiCommonLoginForm = new WebDriverWait(driver,10).until(ExpectedConditions.presenceOfElementLocated(By.tagName("refi-common-login-form")));
WebElement shadowRoot1 = expandRootElement(refiCommonLoginForm, driver);
WebElement refiCommonLastName = shadowRoot1.findElement(By.tagName("refi-common-last-name"));
WebElement refiCommonLastNameShadowRoot = expandRootElement(refiCommonLastName, driver);
WebElement refiCommonZip = shadowRoot1.findElement(By.tagName("refi-common-zip"));
WebElement refiCommonZipShadowRoot = expandRootElement(refiCommonZip, driver);
WebElement refiCommonLastFourSSN = shadowRoot1.findElement(By.tagName("refi-common-last-four-ssn"));
WebElement refiCommonLastFourSSNShadowRoot = expandRootElement(refiCommonLastFourSSN, driver);
refiCommonLastNameShadowRoot.findElement(By.id("loginLastName")).sendKeys("random last name");
refiCommonZipShadowRoot.findElement(By.id("loginZipCode")).sendKeys("43978");
refiCommonLastFourSSNShadowRoot.findElement(By.id("loginLastFourSSN")).sendKeys("3483");
Thread.sleep(2000);
shadowRoot1.findElement(By.tagName("button")).click();

Absolutely not sure the issue is that, but still possibly this will help.
Instead of clicking the "Sign in" button try submitting it i.e. shadowRoot1.findElement(By.tagName("button")).submit()
But I guess the issue here is that this site has some kind of anti bot defense that blocks automated access to it.

Extract Web Elements from an E-Commerce website?

I m trying write a code to access Aliexpress and search for an item, then extract the details, such as, Product name, price, etc.; on page by page to an Excel document. I seek through previous questions posted here to build it. Thanks to that.
Somehow I was able to search the item for first 5 or 6 test runs but then suddenly, Aliexpress asked me to either login or register.
1.) First question, Why any browser won't access the website without registering? Did they recognized my user-agent?
2.) Secondly, Then I was wrote a code to auto log in. Site contains lots of Javascripts, an it is an responsive site. Some html elements appear as we click them. When in the auto log in, my code won't detect the either E-mail or password elements of the page. Is there something preventing it from been detected? How can I solve this?
I here put my sample code:
public class Main {
public static void main(String[] args) throws IOException, InterruptedException {
//To input the user's search
Scanner nw1 = new Scanner(System.in);
System.out.println("What do you want to search?");
String a = nw1.nextLine();
//Open the driver
System.setProperty("webdriver.chrome.driver",
"E:\\JetBrains\\webdriver\\chrome\\chromedriver.exe");
WebDriver AE = new ChromeDriver();
//Open the web page and Login in.
AE.get("https://www.aliexpress.com/");
Thread.sleep(2000);
//xpath of account button
AE.findElement(By.xpath("//*[#id="nav-user-account"]/div/div/p[3]/a[2]")).click();
//xpath of Sign in button
AE.findElement(By.xpath("/html/body/div[9]/a")).click();
AE.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
//xpath of Email box
AE.findElement(By.xpath("//*[#id=\"fm-login-id\"]")).sendKeys("my-email");
//xpath of password section to type
AE.findElement(By.xpath("//*[#id=\'fm-login-password\']")).sendKeys("my-password");
AE.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
// xpath of submit button
AE.findElement(By.xpath("//*[#id="login-form"]/div[5]/button")).click()
Sry kind of my first time here.
Any helpful comments are welcome. Thanks.

It is because following x path in not clicking on account button instead it is clicking on sign in button
AE.findElement(By.xpath("//*[#id="nav-user-account"]/div/div/p[3]/a[2]")).click();
find element through id. Id is available to click on account
"nav-user-account"
while clicking verify that it is unfold through following class
"ng-item nav-pinfo-item nav-user-account user-account-unfold"
if unfold contain than box is open otherwise it is close. If close than click on it.
Try this one first.

Try applying delete cookies ,some sites access data from cache /cookies ,clearing it would solve.Also quit driver at the end of your script and open a new instance i.e driver.manage().deleteAllCookies(); and driver.quit();
xpath in your code needs to be corrected as AE.findElement(By.xpath("//*[#id='nav-user-account']/div/div/p[3]/a[2]")).click();
Use single quotes inside of your xpaths.

Sendkeys without element selenium java

I'm trying to write a simple java based selenium code where I would load a page, give the desired values to username & password.
Once the page loads username is already focused but I can enter values into username or password
Using MAC - Eclipse
I am new to coding sorry so any help would be greatfully received
public static void main(String[] args) throws InterruptedException {
// TODO Auto-generated method stub
WebDriver driver = new FirefoxDriver();
driver.get("http://dc1.racbusinessclub.co.uk/login/");
driver.manage().timeouts().implicitlyWait(20, TimeUnit.SECONDS);
Thread.sleep(2000);
driver.findElement(By.tagName("body")).sendKeys("rac-dc");
Thread.sleep(2000);
driver.findElement(By.tagName("body")).sendKeys(Keys.TAB);
Thread.sleep(2000);

Below is basic way of doing it.
driver.findElement(By.id("username")).sendKeys(username);
driver.findElement(By.id("password")).sendKeys(password);
pause(3);
WebElement button = driver.findElement(By.tagName("button"));
button.sendKeys(Keys.RETURN);
And once logged-in you can assert the url like below
Assert.assertTrue(driver.getCurrentUrl().endsWith("/yourURL"));
You can get username and password id from the browser by Right click on the page and select Inspect.

Even if the field is already focused, you must specify which element you wish to "send the keys" to. So grab the id of the username field, and then use sendKeys, like so:
driver.findElement(By.id("userNameInputBox")).sendKeys("rac-dc");
Also, worth noting that using Thread.sleep in your scripts is bad practice. If there's something you're waiting for, check for the presence of that element, rather than waiting an arbitrary amount of time.
Edit: Actually, seeing your response above I can see you're trying to interact with an alert popup, which you can do with:
driver.switchTo().alert().sendKeys(yourString);
Alternatively, you can skip that entirely and navigate to the page with your credentials in the URL - remember not to hard-code these:
driver.get("http://username:password#http://dc1.racbusinessclub.co.uk/");

How to close Popovers and in-line ads with selenium webdriver

I'm trying to learn Selenium Webdriver using tutorials online etc...
I'm struggling to overcome this obctacle which is to close this popover.
Using:
Laptop: Alienware
O.S: Windows 10 64bits
Browser: Firefox 51.0.1 (32-bit)
Eclipse: Version: Neon.2 Release (4.6.2) Build id: 20161208-0600
Selenium Webdriver: Java 3.0.1 2016-10-18
`package com.indeed.tests;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
public class IndeedJobSearch {
public static void main(String[] args) throws InterruptedException {
// TODO Auto-generated method stub
//Create firefox driver to drive the browser
System.setProperty("webdriver.gecko.driver", "C:\\Users......\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
//Open Indeed home page
driver.get("https://www.indeed.co.uk/");
//Find what field and enter Selenium
Thread.sleep(1000);
driver.findElement(By.id("what")).sendKeys("Selenium");
//Find location field and enter London
driver.findElement(By.id("where")).clear();
Thread.sleep(1000);
driver.findElement(By.id("where")).sendKeys("London");
//Find FindJobs button and click on it
Thread.sleep(1000);
driver.findElement(By.id("fj")).click();
//Close popup - popover, not popup
//prime-popover-div
//selenium webdriver cannot close bootstrap popovers
//Can't find a solution
//From job search results page, get page title and jobs count message
//searchCount
System.out.println(driver.getTitle());
System.out.println(driver.findElement(By.id("searchCount")).getText());
driver.close();
}
}
`
Expected Result: Selenium Webdriver would open firefox browser, load indeed.co.uk webpage, insert "Selenium" in the first field, insert "London" in the second field, hit the search button, get title and job count values on the console and driver window.
Actual Result: Selenium Webdriver would open firefox browser, load indeed.co.uk webpage, insert "Selenium" in the first field, insert "London" in the second field, hit the search button, STOPS the focus in on the url field and nothing else happens.
I've tried a few solutions but couldn't get it working
(https://sqa.stackexchange.com/questions/5310/how-to-close-pop-up-window-in-selenium-webdriver)
e.g.
driver.findElement(By.id("prime-popover-close-button")).click();
Driver.SwitchTo().frame("prime-popover-div");
Driver.findElement(By.id("prime-popover-close-button")).click();
Driver.SwitchTo().defaultContent();
driver.findElement(By.xpath("//*[#id='prime-popover-close-button']/a/img")).click();
Note: Not entirely sure my xpath was writen correctly, still learning.
None of these seem to work. I read something about Selenium WebDriver not handling bootstrap popovers, not sure if that's exactly my case, or if any of you has found a solution.
Would love solutions and or advice :)
Thank you very much in Advance.

Your code generally looks fine (other than the use of Thread.Sleep(), which I will address in a minute.
Basically what you want to do in these cases is to right-click on the close X of the dialog and treat it like any other element on the page. Find a locator for the X, in this case it also has an id, prime-popover-close-button, that we can use. All you need to do is grab that element using the id and click it to dismiss the popup. I've simplified the code below.
driver.get("https://www.indeed.co.uk/");
driver.findElement(By.id("what")).sendKeys("Selenium");
driver.findElement(By.id("where")).sendKeys("London");
driver.findElement(By.id("fj")).click();
driver.findElement(By.id("prime-popover-close-button")).click();
If you aren't trying to test the UI (entering text and clicking buttons) on the search page, you can just navigate directly to the url and even feed your own keywords in, if you like. See the code below for that.
String what = "selenium";
String where = "london";
driver.get("https://www.indeed.co.uk/jobs?q=" + what + "&l=" + where);
driver.findElement(By.id("prime-popover-close-button")).click();
Now back to Thread.Sleep(). This form of wait is generally a bad practice. You can do some research into the details but suffice it to say that it's not flexible. If you sleep for 10s and the element is present in 25ms, you've waited a long time that you didn't need to. Read up on WebDriverWait and ExpectedConditions. While you didn't need it here, you will eventually need to wait and these are best practices for waiting.

It looks like I'm doing the same tutorial you are :) I ran into the exact same issue you did, and tried almost everything you did to click that close button and kill that popover before finding this thread.
It appears that the problem lies in that the popover isn't immediately available for Selenium to close after we click Find Jobs. A 'wait.until..' has to be set in place to wait for the popover to appear so we can close it. Here's what I did:
package com.indeed.tests;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait; //**and this
public class IndeedJobSearch {
public static void main(String[] args) throws InterruptedException {
// TODO Auto-generated method stub
//Create firefox driver to drive the browser
WebDriver driver;
System.setProperty("webdriver.gecko.driver", "C:\\Users\\BURRITOBEAST\\Downloads\\jars\\geckodriver-v0.14.0-win64\\geckodriver.exe");
driver = new FirefoxDriver();
WebDriverWait wait = new WebDriverWait(driver,10); //**and this. 10 is the number of seconds it'll wait before an error is thrown.
//Open Indeed homepage
driver.get("http://www.indeed.com");
//Find the 'what' field and enter "selenium"
driver.findElement(By.id("what")).sendKeys("Selenium");
//Find the 'location' field and enter "San Diego, CA"
driver.findElement(By.id("where")).clear();
driver.findElement(By.id("where")).sendKeys("San Diego, CA");
//Find the 'findjobs' button and click on it
driver.findElement(By.id("fj")).click();
wait.until(ExpectedConditions.elementToBeClickable(By.id("prime-popover-close-button"))); //**this is where the magic happens
//Thread.sleep(1000); **tested my idea first using a sleep. then found the wait method after. plus, i want to avoid sleeps if possible to make things speedy.
driver.findElement(By.id("prime-popover-close-button")).click();
//From the job search results page, get page title and jobs count msg
}
}

How to fill in an autocomplete inputbox using Selenium? (Why an automated input does not load autocomplete options BUT a manual input does?)

The following code tests an autocomlete box of a webpage:
public class Test {
public static void main(String[] args) throws InterruptedException {
System.setProperty("webdriver.chrome.driver","chromedriver\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("http://www..............com");
driver.switchTo().frame("mainFrame");
WebDriverWait waitst = new WebDriverWait(driver, 120);
waitst.until(ExpectedConditions.visibilityOfElementLocated(By.name("sourceTitle")));
WebElement sourceTitle = driver.findElement(By.name("sourceTitle"));
WebElement small = driver.findElement(By.cssSelector("li#nameExampleSection label + small"));
sourceTitle.sendKeys("Times");
Thread.sleep(5000);
Actions actions = new Actions(driver);
actions.click(small).perform();
}
}
Why doesn't the autosuggest box load? IMPORTANT: try to type in "..........." manually ... the autocomplete box will load perfectly fine!!! So, why does not cssSelector work, why doesn't it load the autocomplete box?
How come an automated input does not allow for autocomplete options BUT a manual input does???
PS: I also tried fireEvent, sendKeys but nothing works.

I tried your code, it does exactly what the manual walkthough does. "Associated Press, The" returns only a "No Match, please try sources". In your code you then try to click on the next form list item, not the results popup. The autosuggest popup is dynamically populated at the top of your html page positioned under the input form. The following code does select the first option on your drop down.
#Test
public void test() throws InterruptedException {
WebDriver driver = new ChromeDriver();
driver.get("http://www.lexisnexis.com/hottopics/lnacademic/?verb=sf&sfi=AC00NBGenSrch");
driver.switchTo().frame("mainFrame");
WebDriverWait waitst = new WebDriverWait(driver, 0);
waitst.until(ExpectedConditions.visibilityOfElementLocated(By.name("sourceTitle")));
WebElement sourceTitle = driver.findElement(By.name("sourceTitle"));
sourceTitle.sendKeys("Times");
Thread.sleep(5000);
WebElement firstItem = driver.findElement(By.xpath("//*[#class='auto_suggest']/*[#class='title_item']"));
firstItem.click();
}

I found a workaround about this. My problem was:
Selenium inputted 'Mandaluyong' to an auto-suggest location field
The auto-suggest field appears together with the matched option
Then selenium left the auto-suggest drop-down open not selecting the matched option.
What I did was:
driver.findElement(By.name("fromLocation")).sendKeys("Mandaluyong");
driver.findElement(By.name("fromLocation")).sendKeys(Keys.TAB);
This is because on a manual test, when I try to press TAB key, two things were done by the system:
Picks the matched option from the auto-suggest drop-down
Closes the auto-suggest drop-down
Hope this helps.

// allow autopuplation value to fill the text box.
// wait for 6 sec make sure auto value is inserted
thread.sleep(6000L);
// clear auto filled value
driver.findElement(By.name("txtBox")).clear();
driver.findElement(By.name("txtBox")).sendKeys("value");

Try first clicking on the Input textbox.
This will trigger the auto populating dropdown box and then enter the required value using sendKeys

Either you can use tab or enter to Escape the scenario or,it is not possible by selenium if it is mandatory fields. (sad)