I'm having trouble with clicking at an element, which I find using text which is a variable. This is the code of the page:
<div class="recommendedProfileList fl">
<h3>
<ul class="ctrlResearchProfiles">
<li>
<li>
<li>
<li>
<li>
<li>
<span class="profileBtn ctrlSelectDefProfile ctrlClickSubmit" data-value="143" data-form="formChooseProfile" data-profileid="143">Sales manager</span>
<span class="profileTooltip" style="display: none;">
<span class="arrow"/>
<span class="profileTooltipContent">
</span>
and the name of the variable is profile. This is how I've tried to do this, but did not work:
WebDriverWait wait = new WebDriverWait(driver, 5);
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[text()=' + profile + ']")));
second:
driver.findElement(By.xpath("//*[text()=' + profile + ']"));
also:
driver.findElement(By.linkText("" +profile)).click();
Do you know how to click such element?
You are almost there buddy...
wait = new WebDriverWait(driver, 5);
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[text()='" + profile + "']")));
second:
driver.findElement(By.xpath("//*[text()='" + profile + "']"));
The thing that you missed was double quotes to insert ur variable values in xpath.
Related
I'm new to selenium and following a small tutorial on scraping that scrapes jobs from indeed.com, but I am having issues as it seems some of the elements have been renamed since the tutorial was written. I'm stuck on this part:
List<WebElement> pagination = driver.findElements(By.xpath("//ul[#class='pagination-
list']/li"));
int pgSize = pagination.size();
for (int j = 1; j < pgSize; j++) {
Thread.sleep(1000);
WebElement pagei = driver.findElement(By.xpath("(//ul[#class='pagination-list']/li)[" + j + "]"));
pagei.click();
The xpath needs to be updated as the element can no longer be found. I have changed the xpath to this, but the list does not get populated:
//nav[#aria-label='pagination']
When I do this, it prints zero which means no elements have been added to the list:
List<WebElement> pagination = driver.findElements(By.xpath("//nav[#aria-label='pagination']"));
int size = pagination.size();
System.out.println(size);
Is this the correct xpath? I'm not certain as to what is supposed to get populated into the list of Webelements? Should they be page numbers?
You could try this path //nav[#role='navigation']/div This would give you 6 elements. 5 for the pages and 1 for the next button
If this is not the correct page, please share the URL and the element, I will try to provide a solution.
The website indeed.com have changed since the article Web Scraping using selenium and Java was written. The pagination elements have changed. Currently the pagination is implemented through:
<nav role="navigation" aria-label="pagination" class="css-jbuxu0 ecydgvn0">
<div class="css-tvvxwd ecydgvn1">
<button data-testid="pagination-page-current" class="css-1cpyzlr e8ju0x51">1</button>
</div>
<div class="css-tvvxwd ecydgvn1">
<a data-testid="pagination-page-2" rel="nofollow" aria-label="2" href="/jobs?q=Api+Testing&l=Pune&start=10" class="css-e9oyys e8ju0x50">2</a>
</div>
<div class="css-tvvxwd ecydgvn1">
<a data-testid="pagination-page-3" rel="nofollow" aria-label="3" href="/jobs?q=Api+Testing&l=Pune&start=20" class="css-e9oyys e8ju0x50">3</a>
</div>
<div class="css-tvvxwd ecydgvn1">
<a data-testid="pagination-page-4" rel="nofollow" aria-label="4" href="/jobs?q=Api+Testing&l=Pune&start=30" class="css-e9oyys e8ju0x50">4</a>
</div>
<div class="css-tvvxwd ecydgvn1">
<a data-testid="pagination-page-5" rel="nofollow" aria-label="5" href="/jobs?q=Api+Testing&l=Pune&start=40" class="css-e9oyys e8ju0x50">5</a>
</div>
<div class="css-tvvxwd ecydgvn1">
<a data-testid="pagination-page-next" rel="nofollow" aria-label="Next Page" href="/jobs?q=Api+Testing&l=Pune&start=10" class="css-cy0uue e8ju0x50">
<svg xmlns="http://www.w3.org/2000/svg" focusable="false" role="img" fill="currentColor" viewBox="0 0 24 24" aria-hidden="true" class="css-1xqhio eac13zx0">
<path d="M9.888 5.998a.506.506 0 00-.716-.008l-.707.707a.506.506 0 00.01.716L13.06 12l-4.587 4.587c-.2.2-.204.521-.009.716l.707.707a.507.507 0 00.717-.009l5.647-5.648c.1-.1.148-.233.144-.366a.492.492 0 00-.144-.34v-.001a.611.611 0 00-.009-.009L9.888 5.998z">
</path>
</svg>
</a>
</div>
</nav>
Solution
Instead of creating a list of the pagination elements, as the pagination keeps on increasing as you move forward you can use the following solution:
System.setProperty("webdriver.chrome.driver", "C:\\BrowserDrivers\\chromedriver.exe");
ChromeOptions options = new ChromeOptions();
options.addArguments("--start-maximized");
ChromeDriver driver = new ChromeDriver(options);
driver.get("https://in.indeed.com/");
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(5));
// Create a Scanner object to prompt for user input
Scanner myObj = new Scanner(System.in);
System.out.println("What jobs are you looking for ? ");
String job = myObj.nextLine();
System.out.println("Which city are you looking for ? ");
String city = myObj.nextLine();
wait.until(ExpectedConditions.elementToBeClickable(By.cssSelector("input#text-input-what"))).sendKeys(job);
driver.findElement(By.cssSelector("input#text-input-where")).sendKeys(city + Keys.RETURN);
for (int j=1; j<6; j++)
{
driver.findElement(By.xpath("//button[#data-testid='pagination-page-current']//following::div[1]/a[#data-testid]")).click();
Thread.sleep(15000);
}
Note:
Here 6 is the number of (pages-1) you want to scrape.
Thread.sleep(15000) is to provide the window to pulate Email Address after the first click on pagination item.
Here's my html structure and I am trying to skip a span within a div to get div's text only (which is dynamic) for testing.
<div class="items">
<div class="payment void" id="payment-000000899799">
<div class="payment__details">
<div class="method">CASH <span class="label"><span>Payment Voided</span></span></div>
<div class="date">2/12/2021, 3:02:15 PM</div>
</div>
<div class="payment__details">
<div class="amount">$20.00</div>
<div class="ref">Ref ID: REF-ID-01</div>
</div>
</div>
<div class="payment sale" id="payment-000000899806">
<div class="payment__details">
<div class="method">CASH </div>
<div class="date">2/12/2021, 3:02:21 PM</div>
</div>
<div class="payment__details">
<div class="amount">$100.00</div>
<div class="ref">Ref ID: REF-ID-02</div>
</div>
</div>
</div>
In my step definition I've
List<List<String>> data = capturedData.raw();
WebElement paymentDetails = driver.findElement(By.xpath("(//*[#class='payment__details']/div[#class='method'])[" + data.get(1).get(0) + "]"));
String paymentType = paymentDetails.getText();
System.out.println(paymentType); //This prints CASH Payment Voided
But actually I want only 'CASH' which is a text of div and skip the text 'Payment Voided' of span. And data is coming from a feature file.
How do I get text of div only and skip the text span which is inside the same div?
You cannot skip it directly since it is part of element.
1st Approach
String fulltext = driver.findElement(By.xpath("//*[#class='payment__details']/div[#class='method'][1]")).getText();
String spantext = driver.findElement(By.xpath("//*[#class='payment__details']/div[#class='method'][1]/span")).getText();
//Now replace span text with ""
String output = fulltext.replace(span,"");
2nd Approach
WebElement parent = driver.findElement(By.xpath("//*[#class='payment__details']/div[#class='method'][1]"));
JavascriptExecutor js = (JavascriptExecutor) driver;
String output = (String) js.executeScript("return arguments[0].firstChild.textContent",parent);
I have the task to click on element in the droplist, using exactly Select class.
So I have the HTML block:
<div id="_desktop_currency_selector">
<div class="currency-selector dropdown js-dropdown open">
<span>Валюта:</span>
<span class="expand-more _gray-darker hidden-sm-down" data-toggle="dropdown">UAH ₴</span>
<a data-target="#" data-toggle="dropdown" aria-haspopup="true" aria-expanded="true" class="hidden-sm-down">
<i class="material-icons expand-more"></i>
</a>
<ul class="dropdown-menu hidden-sm-down" aria-labelledby="dLabel" style="display: block;">
<li>
<a title="Евро" rel="nofollow" href="http://prestashop-automation.qatestlab.com.ua/ru/?SubmitCurrency=1&id_currency=2" class="dropdown-item">EUR €</a>
</li>
<li class="current">
<a title="Украинская гривна" rel="nofollow" href="http://prestashop-automation.qatestlab.com.ua/ru/?SubmitCurrency=1&id_currency=1" class="dropdown-item">UAH ₴</a>
</li>
<li>
<a title="Доллар США" rel="nofollow" href="http://prestashop-automation.qatestlab.com.ua/ru/?SubmitCurrency=1&id_currency=3" class="dropdown-item">USD $</a>
</li>
</ul>
<select class="link hidden-md-up">
<option value="http://prestashop-automation.qatestlab.com.ua/ru/?SubmitCurrency=1&id_currency=2">EUR €</option>
<option value="http://prestashop-automation.qatestlab.com.ua/ru/?SubmitCurrency=1&id_currency=1" selected="selected">UAH ₴</option>
<option value="http://prestashop-automation.qatestlab.com.ua/ru/?SubmitCurrency=1&id_currency=3">USD $</option>
</select>
</div>
</div>
In follow attempt, I have
org.openqa.selenium.TimeoutException: Expected condition failed:
waiting for element to be clickable: By.xpath:
//*[#id='_desktop_currency_selector']//select (tried for 30 second(s)
with 500 MILLISECONDS interval)
click(currencyDropListBtn);
WebElement dropListBtn = driver.findElement(By.xpath("//*[#id='_desktop_currency_selector']//i"));
waitToBeClickable(dropListBtn);
dropListBtn.click();
WebElement dropListElement = driver.findElement(By.xpath("//*[#id='_desktop_currency_selector']//select"));
waitToBeClickable(dropListElement);
Select select = new Select(dropListElement);
select.selectByIndex(1);
It will be work in the follow way:
WebElement dropListBtn = driver.findElement(By.xpath("//*[#id='_desktop_currency_selector']//i"));
waitToBeClickable(dropListBtn);
dropListBtn.click();
WebElement dropListElement = driver.findElement(By.xpath("//a[#title='Евро']"));
waitToBeClickable(dropListElement);
click(dropListElement);
But I need to use exactly Select class.
How to correctly select the droplist element via Select?
Normally for select element, you do not need to click the option inside it.
You can just set the value of the select element with value of the option you want to select.
I post a utility function that I use a lot.
set_value("//select[#class='link hidden-md-up']",'http://prestashop-automation.qatestlab.com.ua/ru/?SubmitCurrency=1&id_currency=3','value')
def set_value(xpath, val, field):
script = """(function()
{
node = document.evaluate("%s", document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
if (node==null)
return '';
node.%s='%s';
return 'ok';
})()"""%(xpath,field,val)
driver.execute_script(script)
I am sending defined data from excel file. I tried some code but they are not selecting all the data from excel file at some point of time code is giving me exception for WebElement not found.
Here is the HTML code:
<div class="ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix">
<div id="addDialog" class="hidden ui-dialog-content ui-widget-content" style="display: block; width: auto; min-height: 30px; height: auto; max-height: 351.05px; overflow-y: auto;">
<div class="field-container">
<fieldset class="field-container">
<legend>Contracts:</legend>
<a class="select-all" href="#">Select All</a>
<a class="deselect-all" href="#">Deselect All</a>
<select id="addContract" class="searchable" multiple="multiple" style="position: absolute; left: -9999px;">
<option value="93370956">93370956</option>
<option value="93796167">93796167</option>
<option value="94203239">94203239</option>
</select>
<div id="ms-addContract" class="ms-container">
<div class="ms-selectable">
<input class="search-input" type="text" placeholder="filter" autocomplete="off"/>
<ul class="ms-list" tabindex="-1">
<li id="86355560-selectable" class="ms-elem-selectable">
<span>93370956</span>
</li>
<li id="202890296-selectable" class="ms-elem-selectable">
<span>93796167</span>
</li>
<li id="938848030-selectable" class="ms-elem-selectable">
<span>94203239</span>
</li>
</ul>
</div>
</div>
Need to select values from list.
Efforts done:
This effort for code worked but it selected only one value and then gave exception
WebDriverWait Wait=new WebDriverWait(driver, 10);
Wait.until(ExpectedConditions.presenceOfElementLocated(By.xpath("//ul[#class='ms-list']/li/span")));
//now you can find element
List<WebElement>options=driver.findElements(By.xpath("//ul[#class='ms-list']/li/span[contains(text(),'"+testData+"')]"));
for (WebElement option: options) {
if(testData.equals(option.getText())) option.click();
}
Tried above code but it only selects one value !!
WebDriverWait Wait = new WebDriverWait(driver, 10);
Wait.until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[#id='ms-addContract']//descendant::div[#class='ms-selectable']/ul[#class='ms-list']]//span")));
List<WebElement> options = driver.findElements(By.xpath("//*[#id='ms-addContract']//descendant::div[#class='ms-selectable']/ul[#class='ms-list']]//span[contains(text(), '"+testData+"')]"));
for (WebElement option : options) {
if(testData.equals(option.getText()))
option.click();--tried this xpath-no success
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(".//*[#id='ms-addContract']/div[1]/input"))).click();
driver.findElement(By.xpath(".//*[#id='ms-addContract']/div[1]/input")).sendKeys(testData);
WebDriverWait wait1 = new WebDriverWait(driver, 10);
wait1.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(".//*[#id='ms-addContract']/div[1]/input"))).sendKeys(Keys.ARROW_DOWN,Keys.ARROW_DOWN,Keys.SPACE);
//Clear the input text value.
driver.findElement(By.xpath(".//*[#id='ms-addContract']/div[1]/input")).clear();---worked but not satisfactory
Please need help to locate the element. I am using keyword driven framework reading values from excel file.
It sounds like you are trying to get all span elements that are contained within a ul element (although correct me if I am wrong). This is the code you are using:
List<WebElement>options = driver.findElements(By.xpath("//ul[#class='ms-list']/li/span[contains(text(),'"+testData+"')]"));
In your case, this XPath will only return 1 value. This is because you are trying to match the contents of each span with an input value "testData". If you want to get all span elements inside of the ul then use:
List<WebElement>options = driver.findElements(By.xpath("//ul[#class='ms-list']/li/span"));
This XPath will select accomplish selecting all span elements in the list. If you need to select only span elements containing text within your testData, then you can iterate over that list selecting the appropriate elements:
ArrayList<WebElement> optionsInTestData = new ArrayList<WebElement>();
for(WebElement element: options){
for(String data: testData){
if(element.getText() == data){
optionsInTestData.add(element);
}
}
}
I am using the code below
WebElement inputele = driver.findElement(By.className("class_name"));
String inputeleval = inputele.getAttribute("value");
System.out.println(inputeleval);
but the value is empty. The HTML is below.
<div id="main">
<div id="hiddenresult">
<div class="tech-blog-list">
<label for="Question">1st Question</label>
<input id="txt60" class="form-control" type="text" value="sddf sd sdfsdf sdf sdfsdf sdfsdfsd fsd" />
</div>
</div>
<div class="pagination_main pull-left">
<div id="Pagination">
<div class="pagination">
<a class="previous" onclick="PreviousBtnClickEvent();" href="javascript:void(0)">Previous</a>
<a id="pg59" class="ep" onclick="PaginationBtnClickEvent(this);" href="javascript:void(0)" name="Textbox">1</a>
<a id="pg41" class="ep" onclick="PaginationBtnClickEvent(this);" href="javascript:void(0)" name="Textbox">2</a>
<a id="pg40" class="ep" onclick="PaginationBtnClickEvent(this);" href="javascript:void(0)" name="Textarea">3</a>
<a id="pg60" class="ep current" onclick="PaginationBtnClickEvent(this);" href="javascript:void(0)" name="Textbox">4</a>
</div>
</div>
</div>
</div>
Try using WebDriverWait to wait until element fully loaded on page and visible as below :-
WebDriverWait wait = new WebDriverWait(driver, 10);
WebElement inputele= wait.until(ExpectedConditions.visibilityOfElementLocated(By.className("class_name")));
String inputeleval = inputele.getAttribute("value");
System.out.println(inputeleval);
Note :-By.className("class_name") will give that element which class attribute equal to class_name. Make sure which element you want to locate is unique element with class attribute equal to class_name otherwise wise it will give first element with condition true.
Hope it will work..:)
Looks like your code is pretty close but you have the wrong class name? In your code above, you had "class_name" instead of "form-control". I'm assuming that was some sample code and not the actual code you are using? There is only one INPUT in the HTML and the code below should work. It also has an ID so that should be more specific in case there are more than one INPUTs on the page.
WebElement inputele= driver.findElement(By.className("form-control"));
String inputeleval = inputele.getAttribute("value");
System.out.println(inputeleval);