I just started learning how to use JSoup. I think I've successfully selected this section of the html, and I successfully took "DARK SOULS III Deluxe Edition" out by doing .select("span.title").text but I was trying to get the prices, in this case $84.98 and $55.23. I tried doing .select("div.col search_price responsive_secondrow").text but it comes up as blank. I was wondering if someone could help me figure out how to extract that part, thanks in advance! Here's the html of the section of the page.
The full html is view-source:http://store.steampowered.com/search/?filter=topsellers
<a href="http://store.steampowered.com/sub/94174/?snr=1_7_7_topsellers_150_1" data-ds-packageid="94174" data-ds-appid="374320,442010"onmouseover="GameHover( this, event, 'global_hover', {"type":"sub","id":94174,"public":1,"v6":1} );" onmouseout="HideGameHover( this, event, 'global_hover' )" class="search_result_row ds_collapse_flag" >
<div class="col search_capsule"><img src="http://cdn.edgecast.steamstatic.com/steam/subs/94174/capsule_sm_120.jpg?t=1476893662"></div>
<div class="responsive_search_name_combined">
<div class="col search_name ellipsis">
<span class="title">DARK SOULS III Deluxe Edition</span>
<p>
<span class="platform_img win"></span> </p>
</div>
<div class="col search_released responsive_secondrow">12 Apr, 2016</div>
<div class="col search_reviewscore responsive_secondrow">
<span class="search_review_summary positive" data-store-tooltip="Very Positive<br>86% of the 29,204 user reviews for games in this bundle are positive.">
</span>
</div>
<div class="col search_price_discount_combined responsive_secondrow">
<div class="col search_discount responsive_secondrow">
<span>-35%</span>
</div>
<div class="col search_price discounted responsive_secondrow">
<span style="color: #888888;"><strike>$84.98</strike></span><br>$55.23 </div>
</div>
</div>
<div style="clear: left;"></div>
</a>
Use doc.select("a.search_result_row") instead:
public class JsoupSteamTest {
public static void main(String[] args) throws IOException {
Document doc = Jsoup.connect("http://store.steampowered.com/search/?filter=topsellers").userAgent("Mozilla")
.get();
Elements table = doc.select("a.search_result_row");
Iterator<Element> ite = table.iterator();
while (ite.hasNext()) {
Element element = ite.next();
System.out.println(element.text());
}
}
}
You will get a list like this:
PLAYERUNKNOWN'S BATTLEGROUNDS 23 Mar, 2017 29,99€
Steel Division: Normandy 44 Coming Soon 39,99€
DARK SOULS™ III 11 Apr, 2016 -50% 59,99€ 29,99€
Your particular problem comes from the div that has multiple classes.
To select an element that has multiple classes, use a dot instead of a space in your select:
doc.select("div.col.search_price.discounted.responsive_secondrow");
Take a look at this question: JSOUP get element with multiple classes
Related
This question already has answers here:
'UnexpectedTagNameException' and Element should have been "select" but was "div" error using 'Select' function through Selenium java
(1 answer)
org.openqa.selenium.support.ui.UnexpectedTagNameException: Element should have been "select" but was "span" while selecting a dropdown value
(2 answers)
Closed 2 years ago.
I'm trying to select an option from a drop-down that has a div tag instead of select. With my below code, I am able to open the respective div, however unable to select the element.
This is the HTML tags:
<div id="selectator_LocationListDD" class="selectator_element single options-hidden" style="width:
100%; min-height: 35px; padding: 6px 12px; flex-grow: 0; position: relative;">
<span class="selectator_textlength" style="position: absolute; visibility: hidden;">
</span>
<div class="selectator_selected_items">
<div class="selectator_selected_item selectator_value_">
<div class="selectator_selected_item_title">--Select--</div>
<div class="selectator_selected_item_subtitle"></div>
</div>
</div>
<input class="selectator_input" placeholder="Search here..." autocomplete="false">
<ul class="selectator_options" style="top: 73px;"><li class="selectator_option selectator_value_">
<div class="selectator_option_title">--Select--</div><div class="selectator_option_subtitle">
</div>
<div class="selectator_option_subtitle2">
</div>
<div class="selectator_option_subtitle3">
</div>
</li>
<li class="selectator_option selectator_value_CST0003970">
<div class="selectator_option_title">21ST STREET</div>
<div class="selectator_option_subtitle">1031 21st</div>
<div class="selectator_option_subtitle2">Lewiston, ID</div>
</li>
<li class="selectator_option selectator_value_CST0003214">
<div class="selectator_option_title">3RD & STEVENS</div>
<div class="selectator_option_subtitle">508 W Third Ave</div>
<div class="selectator_option_subtitle2">Spokane, WA</div>
</li>
<li class="selectator_option selectator_value_CST0003956 active">
<div class="selectator_option_title">9TH AVE</div>
<div class="selectator_option_subtitle">600 S 9th Ave</div>
<div class="selectator_option_subtitle2">Walla Walla, WA</div>
</li>
<li class="selectator_option selectator_value_CST0003991">
<div class="selectator_option_title">10TH & BANNOCK</div>
<div class="selectator_option_subtitle">950 W Bannock St, Ste 100</div>
<div class="selectator_option_subtitle2">Boise, ID</div>
</li>
</ul>
</div>
The Code ni has so far is:
Page Object:
#FindBy(id="selectator_LocationListDD")
WebElement locationDD;
public void select_locationEI(int index) throws InterruptedException {
Thread.sleep(2000);
locationDD.click();
Select locationEI = new Select(locationDD);
locationEI.selectByIndex(index+1);
// wait.until(ExpectedConditions.visibilityOfElementLocated
(By.xpath("//div[#class=\"selectator_selected_item selectator_value_\"]//li["+
(index+1)+"]"))).click();
}
step definition:
#When("user added equipment for each location")
public void user_added_equipment_for_each_location() throws InterruptedException {
atmAgreement = new AgreementsATM(driver);
for(int ei: emptyLocation) {
atmAgreement.click_addNewEquipment_tab();
loaderVisibilityWait();
loaderInvisibilityWait();
atmAgreement.select_locationEI(ei);
atmAgreement.enter_modelText();
String dt = reader.getCellData("ATM", "Depositor Type", 2);
atmAgreement.select_depositorType(dt);
String manufacture = reader.getCellData("ATM", "Manufacturer", 2);
atmAgreement.select_manufacturer(manufacture);
atmAgreement.enter_terminalID();
atmAgreement.click_addButtonEI();
loaderVisibilityWait();
}
emptyLocation.clear();
}
I got an org.openqa.selenium.support.ui.UnexpectedTagNameException: Element should have been "select" but was "div".
I'm not sure how to handle this as I've only worked with selects before.
Let's say I wanted to select "9TH AVE" for the agent code. How would I go about this?
Any help is appreciated! Thanks.
Use this xpath and get all the option title (findelements).
//ul//li/div[#class='selectator_option_title']
store above element in yourListOFElements
Once you have the list of webelements you can iterate through each entry and compare the innerHTML
yourListOFElements.get(i).getAttribute("innerHTML")
and compare with your required text.
if matches you can click that element
hope you got the idea.
as I see your dropdown list contains search field
<input class="selectator_input" placeholder="Search here..." autocomplete="false">
the best way is to
Select the main div with id="selectator_LocationListDD"
select the search field inside the main div.
type in the search field a unique part of the option name.
then click on the only displayed <li> inside the main div.
that way you avoid using the index, which can change frequently and use the text in the selection which most likely depends on your inserted Test data so you have full control of it.
I am using the code below
WebElement inputele = driver.findElement(By.className("class_name"));
String inputeleval = inputele.getAttribute("value");
System.out.println(inputeleval);
but the value is empty. The HTML is below.
<div id="main">
<div id="hiddenresult">
<div class="tech-blog-list">
<label for="Question">1st Question</label>
<input id="txt60" class="form-control" type="text" value="sddf sd sdfsdf sdf sdfsdf sdfsdfsd fsd" />
</div>
</div>
<div class="pagination_main pull-left">
<div id="Pagination">
<div class="pagination">
<a class="previous" onclick="PreviousBtnClickEvent();" href="javascript:void(0)">Previous</a>
<a id="pg59" class="ep" onclick="PaginationBtnClickEvent(this);" href="javascript:void(0)" name="Textbox">1</a>
<a id="pg41" class="ep" onclick="PaginationBtnClickEvent(this);" href="javascript:void(0)" name="Textbox">2</a>
<a id="pg40" class="ep" onclick="PaginationBtnClickEvent(this);" href="javascript:void(0)" name="Textarea">3</a>
<a id="pg60" class="ep current" onclick="PaginationBtnClickEvent(this);" href="javascript:void(0)" name="Textbox">4</a>
</div>
</div>
</div>
</div>
Try using WebDriverWait to wait until element fully loaded on page and visible as below :-
WebDriverWait wait = new WebDriverWait(driver, 10);
WebElement inputele= wait.until(ExpectedConditions.visibilityOfElementLocated(By.className("class_name")));
String inputeleval = inputele.getAttribute("value");
System.out.println(inputeleval);
Note :-By.className("class_name") will give that element which class attribute equal to class_name. Make sure which element you want to locate is unique element with class attribute equal to class_name otherwise wise it will give first element with condition true.
Hope it will work..:)
Looks like your code is pretty close but you have the wrong class name? In your code above, you had "class_name" instead of "form-control". I'm assuming that was some sample code and not the actual code you are using? There is only one INPUT in the HTML and the code below should work. It also has an ID so that should be more specific in case there are more than one INPUTs on the page.
WebElement inputele= driver.findElement(By.className("form-control"));
String inputeleval = inputele.getAttribute("value");
System.out.println(inputeleval);
The input fields I am needing to grab are within this #id="contractorsWrapper".
In this example, there are 2 input fields within that wrapper (but this number is dynamic depending on the case) located at #class="contactEntry".
What I'm trying to do is say, how many className=contactEntry fields are there within the id=contractorsWrapper. Then, be able to input text into them independently.
<div id="contractorsWrapper" class="contactInputAndInfoDisplays_wrapper">
<div id="contractorsRow_5d1532ba-b37e-4aac-85c2-4a5e6c6c2796" class="contactInputAndInfoDisplay">
<div class="contactName">
<div class="contactFlag"/>
<a class="smallRemove removeAContact" href="#"/>
<span class="littleGreyTitles">
Name
<br/>
</span>
<input class="contactEntry " type="text" value=""/>
</div>
<div class="descriptionInput littleGreyTitles">
Description
<br/>
<input type="text"/>
</div>
<a class="contactLink" href="#" style="display: none;"/>
</div>
<div class="spacerDiv1"/>
<div id="contractorsRow_5fc58f1a-906f-4239-93ae-b0a2e4b8b70c" class="contactInputAndInfoDisplay">
<div class="contactName">
<div class="contactFlag"/>
<a class="smallRemove removeAContact" href="#"/>
<span class="littleGreyTitles">
Name
<br/>
</span>
<input class="contactEntry " type="text" value=""/>
</div>
<div class="descriptionInput littleGreyTitles">
Description
<br/>
<input type="text"/>
</div>
<a class="contactLink" href="#" style="display: none;"/>
</div>
<div class="spacerDiv1"/>
</div>
Find your wrapper:
WebElement wrapperElement = driver.findElement(By.id("contractorsWrapper"));
Number of input elements:
wrapperElement.findElements(By.className("contactEntry ")).size();
I don't know what you mean with "input text into them independently" but here's how you could enter the same thing in all of them:
for (WebElement element : wrapperElement.findElements(By.className(className))) {
element.sendKeys("keysToSend");
};
update
after more details from OP
If you want to insert some "unique" Strings to the element, you can use an ArrayList
// create as much array entries as you need
List<String> namesList = new ArrayList<String>();
namesList.add("John Doe");
namesList.add("Jane Doe");
...
// then
int count = 0;
for (WebElement element : wrapperElement.findElements(By.className(className))) {
element.sendKeys(namesList.get(count++));
};
of course you would then need to make sure, that your list is always longer than the number of input elements...
I am trying to find button add to cart is present or not using loop from all item box from following code
<div class="page-body">
<div class="product-selectors">
<div class="product-filters-wrapper">
<div class="product-grid">
<div class="item-box">
<div class="item-box">
<div class="item-box">
<div class="item-box">
</div>
in each item box folowing code
<div class="item-box">
<div class="product-item" data-productid="20">
<div class="picture">
<div class="details">
<h2 class="product-title">
<div class="product-rating-box" title="1 review(s)">
<div class="description"> 12x optical zoom; SuperRange Optical Image Stabilizer </div>
<div class="add-info">
<div class="prices">
<div class="buttons">
<input class="button-2 product-box-add-to-cart-button" type="button" onclick="AjaxCart.addproducttocart_catalog('/addproducttocart/catalog/20/1/1 ');return false;" value="Add to cart">
</div>
</div>
</div>
</div>
</div>
I need to find that all itembox have add to cart button present or not using loop. if anyone can help please
I suggest to avoid looping if not necessary. You do not need to do the loop to find out unless there is an explicit need of doing so. You can find the count of Add to cart button and compare with a known value
By byCss = By.cssSelector(".item-box>div input[value='Add to cart']");
int cartCount = driver.findElements(byCss).size();
if (cartCount != 4){
//fail the test
}
If you exactly one to looping and check if the input button exist or not.
By itemBoxes = By.className("item-box");
By button = By.cssSelector("[type='button'][value='Add to cart']");
List<WebElement> webElementList = driver.findElements(itemBoxes);
for (WebElement element: webElementList){
//simply taking size if exist it will return 1
if (element.findElements(button).size() != 1){
//fail
}
}
you can use searching by xpath inside of the loop.
Something like
".//input[#value='Add to cart'][1]"
".//input[#value='Add to cart'][2]"
".//input[#value='Add to cart'][3]"
etc
not sure that this xpath is correct, but generally it will work for you, bro.
Or something like this:
string xpath=".//input[#value='Add to cart']";
var AddToCartBtnsList = driver.findElements(By.Xpath(xpath));
foreach(IWebElement button in AddToCartBtnsList )
{
button.click();
}
I want to parse the data out of this HTML (CompanyName, Location, jobDescription,...) using JSoup (java). I get stuck when trying to iterate the joblistings
The extract from the HTML is one of many "JOBLISTING" divs which I want to iterate and extract the Data out of it. I just can't handle how to iterate the specific div objects. Sorry for this noob question, but maybe someone can help me who already knows which function to use. Select?
<div class="between_listings"><!-- local.spacer --></div>
<div id="joblisting-2944914" class="joblisting listing-even listing-even company-98028 " itemscope itemtype="http://schema.org/JobPosting">
<div class="company_logo" itemprop="hiringOrganization" itemscope itemtype="http://schema.org/Organization">
<a href="/stellenangebote-des-unternehmens--Delivery-Hero-Holding-GmbH--98028.html" title="Jobs Delivery Hero Holding GmbH" itemprop="url">
<img src="/upload_de/logo/D/logoDelivery-Hero-Holding-GmbH-98028DE.gif" alt="Logo Delivery Hero Holding GmbH" itemprop="image" width="160" height="80" />
</a>
</div>
<div class="job_info">
<div class="h3 job_title">
<a id="jobtitle-2944914" href="/stellenangebote--Junior-Business-Intelligence-Analyst-CRM-m-f-Berlin-Delivery-Hero-Holding-GmbH--2944914-inline.html?ssaPOP=204&ssaPOR=203" title="Arbeiten bei Delivery Hero Holding GmbH" itemprop="url">
<span itemprop="title">Junior Business Intelligence Analyst / CRM (m/f)</span>
</a>
</div>
<div class="h3 company_name" itemprop="hiringOrganization" itemscope itemtype="http://schema.org/Organization">
<span itemprop="name">Delivery Hero Holding GmbH</span>
</div>
</div>
<div class="job_location_date">
<div class="job_location target-location">
<div class="job_location_info" itemprop="jobLocation" itemscope itemtype="http://schema.org/Place">
<div class="h3 locality" itemprop="address" itemscope itemtype="http://schema.org/PostalAddress">
<span itemprop="addressLocality"> Berlin</span>
</div>
<span class="location_actions">
<a href="javaScript:PopUp('http://www.stepstone.de/5/standort.html?OfferId=2944914&ssaPOP=203&ssaPOR=203','resultList',800,520,1)" class="action_showlistingonmap showlabel" title="Google Maps" itemprop="maps">
<span class="location-icon"><!-- --></span>
<span class="location-label">Google Maps</span>
</a>
</span>
</div>
</div>
<div class="job_date_added" itemprop="datePosted"><time datetime="2014-07-04">04.07.14</time></div>
</div>
<div class="job_actions">
</div>
</div>
<div class="between_listings"><!-- local.spacer --></div>
File input = new File("C:/Talend/workspace/WEBCRAWLER/output/keywords_SOA.txt"); // Load file into extraction1 Document ParseResult = Jsoup.parse(input, "UTF-8", "http://example.com/"); Elements jobListingElements = ParseResult.select(".joblisting"); for (Element jobListingElement: jobListingElements) { jobListingElement.select(".companyName span[itemprop=\"name\"]"); // other element properties System.out.println(jobListingElements);
Java code:
File input = new File("C:/Talend/workspace/WEBCRAWLER/output/keywords_SOA.txt");
// Load file into extraction1
Document ParseResult = Jsoup.parse(input, "UTF-8", "http://example.com/");
Elements jobListingElements = ParseResult.select(".joblisting");
for (Element jobListingElement: jobListingElements) {
jobListingElement.select(".companyName span[itemprop=\"name\"]");
// other element properties
System.out.println(jobListingElements);
}
Thank you!
So you got your Jsoup document right? Than it seems pretty easy if the css class joblisting does not appear anywhere else.
Document document = Jsoup.parse(new File("d:/bla.html"), "utf-8");
Elements elements = document.select(".joblisting");
for (Element element : elements) {
Elements jobTitleElement = element.select(".job_title span");
Elements companyNameElement = element.select(".company_name spanspan[itemprop=name]");
String companyName = companyNameElement.text();
String jobTitle = jobTitleElement.text();
System.out.println(companyName);
System.out.println(jobTitle);
}
I don't know why the attribute [itemprop*=\"name\"] selector does not find the span (Further reading: http://jsoup.org/cookbook/extracting-data/selector-syntax )
Got it: span[itemprop=name] without any quotes or escapes. Other attributes or values also should work to get a more specific selection.