I am trying to capture the product description from a webpage using ID and tagname but when I print it, it is showing blank. However I think I have used correct locators to locate the element.
Page Source
<div id="item-description-block" class="layout-container layout-container-background clearfix">
<h2>About this item</h2>
<div id="social_share" class="details-row" onclick="javascript:clickPDP('Share','123070751499');">
<div class="details-row clear_both">
<div id="inspirational_copy">
<p>The big plus: Our new formula now helps strengthen skin's own moisture barrier. More moisture stays in. Skin feels soft, springy. Has a healthy-looking glow.</p>
</div>
Web Driver Code
driver.get("http://www.debenhams.com/webapp/wcs/stores/servlet/prod_10701_10001_123070751499_-1");
WebElement description =driver.findElement(By.id("inspirational_copy").tagName("p"));
String description1 = description.getText();
I just tested it in Python and if you replace By.id("inspirational_copy").tagName("p") with a valid css selector you can then use getText() to get the text you're looking for.
driver.get("http://www.debenhams.com/webapp/wcs/stores/servlet/prod_10701_10001_123070751499_-1");
WebElement description =driver.findElement(By.cssSelector("div[id='inspirational_copy']>p"));
String description1 = description.getText();
I did notice when I arrived on the page I got a welcome message. This message prevented me from getting the text. After closing it I could get the element and the text without problems.
WebElement close = driver.findElement(By.cssSelector("button[title='Close']");
close.click()
You can solve this problem easily by using xpath as locator
driver.findelement(By.xpath("//div[#id='inspirational_copy']/p"));
String description1 = description.getText();
Hope this will get the text you want.
String description1 = driver.findElement(By.xpath("//*[#id="inspirational_copy"]/p")).getText();
Related
Critical UPDATE:
It appears that, when we reach the page by using Selenium, the read only fields never loads. It's a document.jsp page which loads. But when we reach the page manually, we get that data. I am using ChromeDriver. I think that explains why I am unable to retrieve the read only fields while using Selenium. If anyone knows of a work around, please let me know.
UPDATE: Since writing this question I have tried innerText again with CSS but it returns " " instead of "Bronze". So it looks like I am able to retrieve something. But it's . How can I get "Bronze"
I am trying to retrieve the text from the field of a read only element using ChromDriver. Below is the HTML code. I want to retrieve the String "Bronze"
<div class="column label-left" style="width:25%">
<div class="form-item clearfix null" id="attr_wrapper_1_offerType_t">
<label class="form-label" for="offerType_t" style="width: 130px"><span style="padding-right: 5px">Offer Type:</span></label>
<div class="form-element field-wrapper" id="field_wrapper_1_offerType_t" style="padding-left:130px">
<div class="field" message=""><span class="readonly-wrapper" id="readonly_1_offerType_t">New Business</span></div>
<div id="msg_1_offerType_t" class="error-hover" data-action-message="" message=""></div>
</div>
</div>
<div class="form-item clearfix null" id="attr_wrapper_1_dealClass_t">
<label class="form-label" for="dealClass_t" style="width: 130px"><span style="padding-right: 5px">Deal Class:</span></label>
<div class="form-element field-wrapper" id="field_wrapper_1_dealClass_t" style="padding-left:130px">
<div class="field" message=""><span class="readonly-wrapper" id="readonly_1_dealClass_t">Bronze</span></div>
<div id="msg_1_dealClass_t" class="error-hover" data-action-message="" message=""></div>
</div>
</div>
<div class="form-item clearfix attr-spacer" style="height: 25px;"></div>
<div class="form-item clearfix attr-spacer" style="height: 25px;"></div>
</div>
I am using id="readonly_1_dealClass_t" but it returns null.
I have also tried xpath="//span[contains(#id,"dealClass")]". It returns null too.
First of all, getText() on id, xpath, CSS all of them returns null. Then I tried all the below options.
I have also tried using JavascriptExecutor and retrieving the text(), but it doesn't help either.
I have also tried innerText and textContent for the above id but without success.
I have waits for 60 seconds until element is visible. It returns true. Which means its visible. But it just refuses to retrieve the string "Bronze".
I also tried getAttribute("value") too. Without success obviously.
I also thought I could use the id="field_wrapper_1_dealClass_t" and use innerText on it. Still no success. That one just returns a lot of whitespace.
What else can I try to retrieve the string "Bronze"?
PS: I don't have issues with Firefox. Chrome just refuses to go ahead. And business need is to stick with only Chrome right now. So I have to get this working in Chrome. Please help.
I hope I have been clear and I hope I have furnished enough HTML code.
UPDATE:
This returns [] for value, ie blank.
final String script = "return arguments[0].getAttribute('innerHTML')";
WebElement randomRow = driver.findElement(By.xpath("(//div[#class='field']/span)[21]"));
String value = (String) ((JavascriptExecutor) driver).executeScript(script, randomRow);
UPDATE 2:
This returns [ ] too:
String myText = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//span[#class='readonly-wrapper' and starts-with(#id,'readonly_') and contains(#id,'_dealClass_t')]"))).getAttribute("innerHTML");
As per the HTML you have shared to retrieve the String Bronze you have to induce WebDriverWait for the visibility of the element as follows :
String myText = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//span[#class='readonly-wrapper' and starts-with(#id,'readonly_') and contains(#id,'_dealClass_t')]"))).getAttribute("innerHTML");
I think you'll be able to get that using below by fetching 'innerHTML' attribute:
final String script = "return arguments[0].getAttribute('innerHTML')";
WebElement randomRow = driver.findElement(By.xpath("(//div[#class='field']/span)[21]"));
String value = (String) ((JavascriptExecutor) driver).executeScript(script, randomRow);
So, after several trial and error runs it turns out that the page does not load completely when logging into it using Selenium. I copied the URL which opens using Selenium and compared it with the one which opens manually. They were different. The part after formCreate right after document.jsp was completely different for the Selenium loaded page. So, to avoid too much complications, I used driver.get("url") with the "url" being the actual URL and then let it open. This loaded the fields correctly and I was able to do the validations.
I want to locate the Text within the below HTML code but there are two duplicated classes.
<div id="header" class="cf">
<div class="cf">
<h1>
Text
</h1>
I located but not sure if that is the best way to do it because the text might `appear some where else.
WebElement LL = driver.findElement(By.linkText("Text"));
Anyone have a better way to locate this please? THANK you in advance!
Go by the following css to identify the element more precisely. And, the id header should be unique and that should be enough to uniquely identify this element
By css = By.cssSelector("#header div.cf>h1>a");
WebElement element = driver.findElement(css );
String text = element.getText();
You can use xpath as your selector and then use WebElements getText() method to extract the text.
WebElement element = driver.findElement(By.xpath(".//div[#class='cf']/h1/a"));
String text = element.getText();
I have the following HTML code:
div id="flashMessage" class="error">
<span>Saved Section.</span>
<div id="errors" class="clearfix">
</div>
I would like to get the text which is contained in span, I tried all the locators.
Could someone please help me with the respective command and css or xpath locator in selenium, for the above query?
This should work for you:
string spanText = driver.findElement(By.cssSelector("div#flashMessage>span")).getText();
I'm not sure why the same locator wouldn't work for success messages, but you can try this:
string spanText = driver.findElement(By.cssSelector("div#flashMessage.msg>span")).getText();
My goal is to parse a block of HTML code like below to obtain the text, comments and replies fields as separate parts of the block:
<div id='fooID' class='foo'>
<p>
This is the top caption of picture's description</p>
<p>
T=<img src="http://www.mysite.com/images/img23.jpg" alt="" width="64" height="108"/> </p>
<p>
And here is more text to describe the photo.</p>
<div class=comments>(3 comments)</div>
<div id='reply13' class='replies'>
<a href=javascript:getReply('13',1)>Show reply </a></div>
</div>
My problem is that Selenium's WebDriver does not seem to support non-string identifiers in the HTML (notice that the class field in the HTML is 'foo' and as opposed to "foo"). From all examples that I have seen in both the Selenium docs and in other SO posts, the latter format is what WebDriver commonly expects.
Here is the relevant part of my Java code with my various (unsuccessful) attempts:
java.util.List<WebElement> elementList = driver.findElements(By.xpath("//div[#class='foo']"));
java.util.List<WebElement> elementList = (List<WebElement>) ((JavascriptExecutor)driver).executeScript("return $('.foo')[0]");
java.util.List<WebElement> elementList = driver.findElements(By.xpath("//div[contains(#class, 'foo')]"));
java.util.List<WebElement> elementList = driver.findElements(By.cssSelector("div." + foo_tag)); // where foo_tag = "'foo'".replace("'", "\'");
java.util.List<WebElement> elementList = driver.findElements(By.cssSelector("'foo'"));
Is there a sure way of handling this? Or is there an alternative, better way of extracting the above fields?
Other info:
I'm an HTML noob, but have made efforts to understand the structure of the HTML code/tags
Using Firefox (and, accordingly, FirefoxDriver)
Your help/suggestions greatly appreciated!
It's invalid HTML, so Selenium won't have a chance. You should fix it.
You will have a better chance with HTMLAgilityPack:
http://htmlagilitypack.codeplex.com/
It is a little better when it comes to badly formed (which this is) HTML.
Below is a SO post which a few different options for a few different languages, with tools like HTMLAgilityPack. You should find a suitable one:
Options for HTML scraping?
The problem is that the html specification doesnt know single quotes as far as I know. Therefore you don't have a problem with the Selenum webdriver, the problem is the html.
Do you have the chance to edit the html code?
How do i get "this text" from the following html code using Jsoup?
<h2 class="link title"><a href="myhref.html">this text<img width=10
height=10 src="img.jpg" /><span class="blah">
<span>Other texts</span><span class="sometime">00:00</span></span>
</a></h2>
When I try
String s = document.select("h2.title").select("a[href]").first().text();
it returns
this textOther texts00:00
I tried to read the api for Selector in Jsoup but could not figure out much.
Also how do i get an element of class class="link title blah" (multiple classes?). Forgive me I only know both Jsoup and CSS a little.
Use Element#ownText() instead of Element#text().
String s = document.select("h2.link.title a[href]").first().ownText();
Note that you can select elements with multiple classes by just concatenating the classname selectors together like as h2.link.title which will select <h2> elements which have at least both the link and title class.