Extract Web Elements from an E-Commerce website? - java

I m trying write a code to access Aliexpress and search for an item, then extract the details, such as, Product name, price, etc.; on page by page to an Excel document. I seek through previous questions posted here to build it. Thanks to that.
Somehow I was able to search the item for first 5 or 6 test runs but then suddenly, Aliexpress asked me to either login or register.
1.) First question, Why any browser won't access the website without registering? Did they recognized my user-agent?
2.) Secondly, Then I was wrote a code to auto log in. Site contains lots of Javascripts, an it is an responsive site. Some html elements appear as we click them. When in the auto log in, my code won't detect the either E-mail or password elements of the page. Is there something preventing it from been detected? How can I solve this?
I here put my sample code:
public class Main {
public static void main(String[] args) throws IOException, InterruptedException {
//To input the user's search
Scanner nw1 = new Scanner(System.in);
System.out.println("What do you want to search?");
String a = nw1.nextLine();
//Open the driver
System.setProperty("webdriver.chrome.driver",
"E:\\JetBrains\\webdriver\\chrome\\chromedriver.exe");
WebDriver AE = new ChromeDriver();
//Open the web page and Login in.
AE.get("https://www.aliexpress.com/");
Thread.sleep(2000);
//xpath of account button
AE.findElement(By.xpath("//*[#id="nav-user-account"]/div/div/p[3]/a[2]")).click();
//xpath of Sign in button
AE.findElement(By.xpath("/html/body/div[9]/a")).click();
AE.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
//xpath of Email box
AE.findElement(By.xpath("//*[#id=\"fm-login-id\"]")).sendKeys("my-email");
//xpath of password section to type
AE.findElement(By.xpath("//*[#id=\'fm-login-password\']")).sendKeys("my-password");
AE.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
// xpath of submit button
AE.findElement(By.xpath("//*[#id="login-form"]/div[5]/button")).click()
Sry kind of my first time here.
Any helpful comments are welcome. Thanks.

It is because following x path in not clicking on account button instead it is clicking on sign in button
AE.findElement(By.xpath("//*[#id="nav-user-account"]/div/div/p[3]/a[2]")).click();
find element through id. Id is available to click on account
"nav-user-account"
while clicking verify that it is unfold through following class
"ng-item nav-pinfo-item nav-user-account user-account-unfold"
if unfold contain than box is open otherwise it is close. If close than click on it.
Try this one first.

Try applying delete cookies ,some sites access data from cache /cookies ,clearing it would solve.Also quit driver at the end of your script and open a new instance i.e driver.manage().deleteAllCookies(); and driver.quit();
xpath in your code needs to be corrected as AE.findElement(By.xpath("//*[#id='nav-user-account']/div/div/p[3]/a[2]")).click();
Use single quotes inside of your xpaths.

Related

Sendkeys without element selenium java

I'm trying to write a simple java based selenium code where I would load a page, give the desired values to username & password.
Once the page loads username is already focused but I can enter values into username or password
Using MAC - Eclipse
I am new to coding sorry so any help would be greatfully received
public static void main(String[] args) throws InterruptedException {
// TODO Auto-generated method stub
WebDriver driver = new FirefoxDriver();
driver.get("http://dc1.racbusinessclub.co.uk/login/");
driver.manage().timeouts().implicitlyWait(20, TimeUnit.SECONDS);
Thread.sleep(2000);
driver.findElement(By.tagName("body")).sendKeys("rac-dc");
Thread.sleep(2000);
driver.findElement(By.tagName("body")).sendKeys(Keys.TAB);
Thread.sleep(2000);
Below is basic way of doing it.
driver.findElement(By.id("username")).sendKeys(username);
driver.findElement(By.id("password")).sendKeys(password);
pause(3);
WebElement button = driver.findElement(By.tagName("button"));
button.sendKeys(Keys.RETURN);
And once logged-in you can assert the url like below
Assert.assertTrue(driver.getCurrentUrl().endsWith("/yourURL"));
You can get username and password id from the browser by Right click on the page and select Inspect.
Even if the field is already focused, you must specify which element you wish to "send the keys" to. So grab the id of the username field, and then use sendKeys, like so:
driver.findElement(By.id("userNameInputBox")).sendKeys("rac-dc");
Also, worth noting that using Thread.sleep in your scripts is bad practice. If there's something you're waiting for, check for the presence of that element, rather than waiting an arbitrary amount of time.
Edit: Actually, seeing your response above I can see you're trying to interact with an alert popup, which you can do with:
driver.switchTo().alert().sendKeys(yourString);
Alternatively, you can skip that entirely and navigate to the page with your credentials in the URL - remember not to hard-code these:
driver.get("http://username:password#http://dc1.racbusinessclub.co.uk/");

Selenium Cant find element inside Iframe event tho they are visible

I have been trying to create code to automatically log in into the website of a game I play called Life is Feudal to gather some information(Everything is legal, I have manually access to the informations I want but it would take years to gather everything).
So here's my code:
public static void main(String[] args) throws InterruptedException {
String Url = "https://lifeisfeudal.com/";
String email = "MYEMAIL";
String password = "MYPASSWORD";
WebDriver driver = setDriver();
// Open Life is feudal home page
driver.get(Url);
/*
* Logging into my account
*/
// Click on sign in button [WORK]
driver.findElement(By.linkText("Sign In")).click();
//Focus the Iframe
driver.switchTo().frame(driver.findElement(By.id("signin")));
/*
* Type in Email
*/
//Try #1 using name (DONT WORK)
driver.findElement(By.name("email")).sendKeys(email);
//Try #2 using xpath (DONT WORK)
driver.findElement(By.xpath("//*[#id=\"react-view\"]/div/div/div/div[2]/form/div[1]/div/input")).sendKeys(email);
/*
* Type in Password
*/
//Try #1 using name (DONT WORK)
driver.findElement(By.name("password")).sendKeys(password);
//Try #2 using xpath (DONT WORK)
driver.findElement(By.xpath("//*[#id=\"react-view\"]/div/div/div/div[2]/form/div[2]/div/input")).sendKeys(password);
/*
* Click the sign in button
*/
//Try (Havent tried it yet)
driver.findElement(By.linkText("Sign in")).click();
//Bring back to default
driver.switchTo().defaultContent();
//Rest of the code
}
And here's the html code inside the Iframe
HTML Iframe
Edit 1: Adding a thread.sleep(3000) after switching frame as suggested by Frank made it work.
I assume it wasnt fully loading the frame and was trying to access something not fully loading that wasnt existing at the time.
Add a Thread.Sleep(3000) after switching to the IFrame to figure out if it's a matter of timing. If so replace the Thread.Sleep(3000) with a more robust way of waiting like Explicit Wait.

Selenium code to open facebook messenger and send a message

I'm just starting to learn a little bit of Selenium scripting (in Java). Currently I'm trying to open a chat box in Facebook and send a message.
I have gotten up to being able to open the chat box through selenium, however I can't figure out how to actually type things into the text box.
Currently here is my code:
import org.openqa.selenium.*;
import org.openqa.selenium.chrome.ChromeDriver;
public class practice {
public static void main(String[] args) throws InterruptedException {
System.setProperty("webdriver.chrome.driver", "C:/max/chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("http://facebook.com/login/");
driver.manage().window().maximize();
driver.findElement(By.name("email")).sendKeys("myemail#yahoo.com");
driver.findElement(By.name("pass")).sendKeys("myPassword");
driver.findElement(By.id("loginbutton")).click();
driver.findElement(By.name("mercurymessages")).click();
driver.findElement(By.cssSelector("a[href*='https://www.facebook.com/messages/conversation-8148306']")).click();
// This has worked randomly. Sometimes the driver will work and open the chat box. Sometimes it will say element not found. I didn't include the full link of the conversation because apparently you don't have to. And it has worked like this in the past.
}
}
My current issue I'd like resolved is: why does this only work sometimes, and how do i find the text box area on the chat box? I did an Inspect Element and the chat box is very odd. It doesn't have anything like an id or a name, so i can't do By.id, or By.name. I can't figure out how to do it with By.cssSelector. This is what the inspect element for the text box looks like:
textarea class="uiTextareaAutogrow _552m" data-ft=".... more stuff here" onkeydown=" more stuff here" style= "more stuff here."
You have to learn Xpath and how to create Relative xpath.The xpath for the textarea
driver.findElement(By.xpath("//textarea[#class='uiTextareaAutogrow _552m']"));
Anyhow I've made few changes that Include instead of clicking on some other message.It will create a new message and send to your friend
driver.findElement(By.name("mercurymessages")).click();
//wait for 20 seconds
WebDriverWait wait = new WebDriverWait(driver, 20);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("u_0_5")));
driver.findElement(By.id("u_0_5")).click();//To click on Send a New Message Link
//To enter a name into the to field
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//input[#class='inputtext textInput']")));
WebElement friendName = driver.findElement(By.xpath("//input[#class='inputtext textInput']"));
friendName.sendKeys("Deep");//Change it with your friend name
//wait for the user list to appear
wait.until(ExpectedConditions.presenceOfElementLocated(By.xpath("//li[#class='user selected']")));
friendName.sendKeys(Keys.ENTER);
WebElement messageBox = driver.findElement(By.xpath("//textarea[#class='uiTextareaAutogrow _552m']"));
wait.until(ExpectedConditions.visibilityOf(messageBox));
messageBox.sendKeys("Hi there");
messageBox.sendKeys(Keys.ENTER);
Hi pal hope this gonna help you:
there is a method called By.className so in order to find elements without an id or a name you can use this method, the thing it's that sometimes the class name of the element it's used in other elements, so be aware of the uses in the current page, if you are lucky and the class name is unique then you can use it :)
System.setProperty("webdriver.chrome.driver", "C:/max/chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("http://facebook.com/login");
driver.manage().window().maximize();
Thread.sleep(5000);
driver.findElement(By.name("email")).sendKeys("myemail#yahoo.com");
driver.findElement(By.name("pass")).sendKeys("myPassword");
driver.findElement(By.id("loginbutton")).click();
Thread.sleep(5000);
driver.findElement(By.name("mercurymessages")).click();
Thread.sleep(7000);// here you have to use ExpectedConditions in order to verify that the elemnt it´s "clickable"
driver.findElement(By.cssSelector("a[href*='https://www.facebook.com/messages/conversation-8148306']")).click();
Thread.sleep(5000);
WebElement mssgbox = driver.findElement(By.cssSelector("textarea[class*='uiTextareaAutogrow _552m']"));
mssgbox.click();
mssgbox.sendKeys("hi");
hope this help you.
and some times elements in the pages changes so use ExpectedConditions to be sure the element its on the page
How to send message to your friend on Facebook:: Using Selenium WebDriver
// Find Username and Enter
driver.findElement(By.id("email")).sendKeys("Email");
System.out.println("Email");
// Find Password and Enter
driver.findElement(By.id("pass")).sendKeys("Password");
System.out.println("Password");
// Click Login
driver.manage().timeouts().implicitlyWait(2, TimeUnit.SECONDS);
driver.findElement(By.xpath(".//*[#id='u_0_q']")).click();
System.out.println("LogIn Successfull");
Thread.sleep(2000);
// Message
driver.findElement(By.xpath(".//*[#id='u_0_f']/a/div")).click();
driver.findElement(By
.xpath(".//*[#id='u_0_f']/div/div[3]/div/div[1]/div/div/ul/li[2]/a/div/div[2]/div/div[2]/div/div[1]/strong/span"))
.click();
Thread.sleep(2000);
driver.findElement(By.xpath("//*[#class='_1mf _1mj']//following :: div[11]")).click();
Thread.sleep(2000);
WebElement sendmsg = driver
.findElement(By.xpath("//div[#class='_1ia']/descendant::div[#class='_5rpu' and #role='combobox']"));
sendmsg.sendKeys("Just testing: using selenium webdriver" + Keys.ENTER);
IWebElement messageBox = driver.FindElement(By.ClassName("_1mj"));
messageBox.SendKeys(string.Format("{0} Do not try to scam people anymore!!!", iteratia++));
messageBox.SendKeys(Keys.Enter);
Using C#
I can access the message textbox to insert the message with the first line of code. The second line is inserting the message. The third line is sending it.
Result:

How to insert text into popup window using selenium WebDriver

I am trying to automate a web page using selenium WebDriver. I can move towards the webpages and can able to do all tasks on parent pages. But at one place I am getting a POPUP window which asks username and password. I am unable to identify the username and password text box id's, how can I find the element (TextBox) and send the username and password through selenium webDriver code.
You can try this by using your web URL,
#Test
public void testlinkedin() throws Exception {
driver.get("https://www.linkedin.com/"); //Use your web URL
driver.findElement(By.linkText("Sign Up")).click();
driver.findElement(By.cssSelector("button.fb-btn")).click();
Thread.sleep(3000);
Set <String>handles = driver.getWindowHandles();//To handle multiple windows
firstWinHandle = driver.getWindowHandle();
handles.remove(firstWinHandle);
String winHandle=handles.iterator().next();
if (winHandle!=firstWinHandle){
secondWinHandle=winHandle;
driver.switchTo().window(secondWinHandle); //Switch to popup window
Thread.sleep(2000);
driver.findElement(By.id("email")).clear();
driver.findElement(By.id("email")).sendKeys("Username");
driver.findElement(By.id("pass")).clear();
driver.findElement(By.id("pass")).sendKeys("Password");
driver.findElement(By.id("u_0_2")).click();
driver.findElement(By.name("__CONFIRM__")).click();}
Make sure to declare these string variables as,
public String firstWinHandle;
public String secondWinHandle;
You need to switch to the new pop-up window to select its elements. Here is an example to switch to the new window:
String handle = driver.getWindowHandles().toArray()[1]; // 1 stands for the new window and 0 for the parent window
driver.switchTo(handle);
Now you can select your elements present in this window.
*Note: * To get back to original window you again need to switch back to it with 0 as the index.
Maybe the popup window is HTTP basic authenticated site. If so, then you cannot use the window handles as stated in previous answer, but in that case you have to send the username and password directly to URL request:
driver.get("http://username:password#your-test-site.com");
Its working fine for me as I used like this...
String handle = driver.getWindowHandles().toArray()[1].toString();
driver.switchTo().window(handle);

How to fill a web form using Java (Don't want to submit it, but just want to fill up the fields from database)

There is a requirement to fill a webform of some particular site of which HTML source I have available. I want to fill the fields on this form from the database records. I don't want to POST the request to the server as the application user would first verify the filled data and will submit the form manually.
Is there any way in java through which I can achieve this? I have already achieved this in VB .Net but want to include as a part of bigger application which is already developed in java and implements a business logic.
You help would be appreciated.
Thanks,
SRA
Java would be running on the server.
In that case, you just modify the server so that it generates the HTML with the <input> elements prefilled with the values from the database; e.g. using the value attributes. It is a little bit more complicated for selectors, where you have to make sure that one of the choices is preselected.
So, you need to access DB and you also need to fill web form from the client side. If you use Selenium it will be this scenario:
User starts your program. Program opens browser window, logs in and fills form. User checks data and presses "send"-button. Browser window automatically closed by program (potentially possible to keep browser window open, but it's better to clean resources and finish program).
Code sample:
public static void main(String[] args) throws InterruptedException {
webDriver = new FirefoxDriver();
try {
processPage();
} finally {
webDriver.quit();
}
}
private static void processPage() throws InterruptedException {
webDriver.get("http://some_page_url");
// Page element search example
List<WebElement> elements = webDriver.findElements(By.xpath(XPATH_STR));
// Fill page here
}

Categories