We Keep Coding

Webdriver getText not returning any meaningful value

I have the following html in a page
<div class="mst_updt" style=""> 27/06/2017 12:02:31 </div>
I am trying to extract the dynamic date value between the divs :
WebElement webElement =driver.findElement(By.cssSelector(".mst_updt"));
String text = webElement.getText();
System.out.println("i am text : " + text);
System.out.println("Most read 1 : " + webElement.getText());
String a = driver.findElement(By.cssSelector(".mst_updt")).getText();
System.out.println("Most read 2 : " + a);
System.out.println(webElement);
Boolean isTheTextPresent = driver.getPageSource().contains("mst_updt");
System.out.println("And did we find the string ? : " + isTheTextPresent);
you will see i am trying various methods , here are the results i am getting, why cant i extract the date and time ??
i am text :
Most read 1 :
Most read 2 :
[[FirefoxDriver: firefox on ANY (e4a9c548-6146-4685-9944-6b5d51308bff)] -> css selector: .mst_updt]
And did we find the string ? : true
Full Code which should help..
<div class="content">
<div role="main" class="content-inner content-full-width">
<div class="main-content">
<section class="component-list">
<div class="section group">
<div class="col-lrg span-lrg_1_of_3">
<div class="component-weekly-wrap">
<header class="header-weekly-wrap">
<h4 itemprop="name">
<a class="section-title-link" href="example.com/static/survey-panel">Computer Survey Panel</a>
</h4>
</header>
<article>
<img alt="" src="/w-images/fa8edbd1-2c78-4415-9abf-334f7087ff8b/2/CTGRS17OA344213-370x229.jpg" />
<div class="col-inner weekly-wrap-details">
<p><p>Join our <strong>Research Survey Panel</strong> and earn an Amazon voucher for each survey you complete!</p>
<p><strong>Find out more</strong></p></p>
<!--<a class="btn download" href="#">More information</a> -->
</div>
</article>
</div>
</div>
<!--most read homepage updated START-->
<div class= "mst_updt" style="display:none;">
28/06/2017 12:38:15
</div>
<!--most read homepage updated END-->
<div class="col-lrg component-list-most-read span-lrg_2_of_3">
<div class="col-inner component-most-read">
<header class="header-most-read">
<h4 itemprop="name">Most read</h4>
</header>
<div class="ol">
<ol>
<li>
</li>
<li>
</li>
<li>
</li>
<li>
</li>
<li>
</li>
</ol>
</div>
</div>
</div>
</div>
</section>
</div>
</div>
</div>

Here is the Answer to your Question:
To locate a particular div to get text using class attribute must be avoided. Class attribute is applied to multiple div tags as required. You can consider using some other locator preferably an xpath or css to identify this unique div tag and use getText() method to retrieve the text.
Here in this case as you have copied the single div tag it would be tough to help you. But you can consider to construct anxpath to traverse the HTML DOM by binding to an id or name attribute at the parent level and then access theclass property of this div.

Try this xpath, if that's working, it means the identifier you are using is giving you wrong element.
//*[contains(text(),'27/06/2017')]

the solution was supplied by : Rafał Laskowski
i could get the value by xpath of cssSelector, it needed returning by using : getAttribute‌​("innerHTML");
s String a = driver.findElement(By.cssSelector(".mst_updt")).getAttribute‌​("innerHTML");
String b = driver.findElement(By.xpath("//html/body/div[3]/div[2]/div[7‌​]/div/div/section/di‌​v/div[2]")).getAttri‌​bute("innerHTML");
System.out.println("Most read 2 : " + a);
System.out.println("Most read 3 : " + b);

Positioning of jQuery/Java toggle content

I have the following HTML-Code:
<div class="test-container">
<div class="slide-button" data-content="panel1">
<p><span class="panel-icon">+</span> Test1</p>
</div>
<div id="panel1" style="display: none">
<p> Test jquery menu1 </p>
</div>
<div class="slide-button" data-content="panel2">
<p><span class="panel-icon">+</span> Test2</p>
</div>
<div id="panel2" style="display: none">
<p> Test jquery menu2 </p>
</div>
</div>
And the following jQuery/Java-Code:
$(".slide-button").on('click', function() {
var panelId = $(this).attr('data-content');
$('#'+panelId).toggle(500);
$(this).find('.panel-icon').text(function(_, txt) {
return txt === "+" ? "-" : "+";
});
});
The toggle itself works perfectly. When I click on the slide-button the content will slide-down. However, after the slide-down animation is finished the content somehow "jumps up" to its final position.
How can I avoid this "jump" and get the content stays where it is after the slide-down animation is finished?
Thanks for any help :-)

I don't know exactly what your circumstances are, and whether you need <p>aragraph tags or not, but if you switch the <p> tags inside your "panels" to <span> tags it seems to fix your issue.
The HTML code I used which fixed the jump looks like this:
<div class="test-container">
<div class="slide-button" data-content="panel1">
<p><span class="panel-icon">+</span> Test1</p>
</div>
<div id="panel1" style="display: none">
<!-- Here is the first change from a paragraph to a span tag -->
<span>Test jquery menu1</span>
</div>
<div class="slide-button" data-content="panel2">
<p><span class="panel-icon">+</span> Test2</p>
</div>
<div id="panel2" style="display: none">
<!-- Here is the second change from a paragraph to a span tag -->
<span>Test jquery menu2</span>
</div>
Also, just a friendly tip, Java is not the same as JavaScript. (: Keep that in mind when tagging your questions.

JSoup scrape HTML document by attribute value

I want to make a dynamic website and need some pics off the internet. I decided to scrape them off flickr and include the owners on my website but am running into problems scraping. I'll post part of the HTML below but if you want to check the source code yourself, here's the website. https://www.flickr.com/explore
HTML:
<div class="thumb ">
<span class="photo_container pc_ju">
<a data-track="photo-click" href="/photos/sheilarogers13/15586482942/in/explore-2014-10-20" title="Lake District" class="rapidnofollow photo-click"><img id="photo_img_15586482942" src="https://c2.staticflickr.com/4/3945/15586482942_6a7154363f_z.jpg"width="508" height="339" alt="Lake District" class="pc_img " border="0"><div class="play"></div></a>
</span>
<div class="meta">
<div class="title"><a data-track="photo-click" href="/photos/sheilarogers13/15586482942/in/explore-2014-10-20" title="Lake District" class="title">Lake District</a></div>
<div class="attribution-block">
<span class="attribution">
<span>by </span>
******<a data-track="owner" href="/photos/sheilarogers13" title="sheilarogers22" class="owner">sheilarogers22</a>******
</span>
</div>
<span class="inline-icons">
<a data-track="favorite" href="#" class="rapidnofollow fave-star-inline canfave" title="Add this photo to your favorites?"><img width="12" height="12" alt="[★]" src="https://s.yimg.com/pw/images/spaceball.gif" class="img"><span class="fave-count count">99+</span></a>
<a title="Comments" href="#" class="rapidnofollow comments-icon comments-inline-btn">
<img width="12" height="12" alt="Comments" src="https://s.yimg.com/pw/images/spaceball.gif">
<span class="comment-count count">57</span>
</a>
<img width="12" height="12" alt="" src="https://s.yimg.com/pw/images/spaceball.gif">
</span>
</div>
</div>
I want the line where I put asterisks, in order to be able to give credit to the authors of the pictures.
My code:
Elements pgElem = doc.select("div.thumb").select("div.meta").select("[data-track]");
The above code however gives me all 4 data tracks in my div.meta though, and I only want the one that =owner.
I checked the JSoup documentation and it says that attributes with values are found using [attr=value], but I can't seem to get it to work. I've tried:
.select("[data-track=owner]")
.select("[data-track='owner']")
but neither work. Thoughts?

Elements pgElem = doc.select("div.thumb").select("div.meta").select("[data-track]");
Elements ownerElements = new Elements();
for(Element element:pgElem){
if(!element.getElementsByAttributeValueContaining("data-track","owner").isEmpty()){
ownerElements.add(element);
}
}
actually, I just gave it another spin and this works for me:
doc.select("div.thumb").select("div.meta").select("[data-track=owner]")

How to retrieve atomic values inside tags in HTMLUnit

I am new to HtmlUnit and I don't know how to get the text inside the [...]
A part of my html file:
<ul ......somethin....>
<li data-role="list-divider" role="heading" style="font-size:16px;" class="ui-bar-f">
INFORMATION_LINE_1
</li>
<li data-theme="d" class="ui-li ui-btn-icon-right ui-btn-up-d ui-odd-match-column ">
<div class="ui-btn-inner ui-li">
<div class="">
<div class="ui-btn-text">
<a href="/x/cxntay/13113/ndzvsssl/g1" class=" ui-link-inherit ui-link-hover">
<h3 class="ui-li-heading">
<span class="xheader">INFORMATION_LINE_2</span>
<span class="label live">INFORMATION_LINE_3</span>
</h3>
<div class="ui-live-scores">
<span class="team1-scores">
<span class="ui-team-name">INFORMATION_LINE_4</span>
<span style="font-weight:bold">INFORMATION_LINE_5</span>
</span>
<span>INFORMATION_LINE_6</span>
</div>
</a>
</div>
</div>
</div>
</li>
</ul>
Now, I want to retrieve "INFORMATION_LINE_X"(1,2...6) in between these tags..
This is what I tried:
List<HtmlUnorderedList> ls = (List<HtmlUnorderedList>) page.getByXPath("/ul");
List<DomNode> dls = ls.get(0).getChildNodes();
System.out.println(dls.get(0).getFirstByXPath("//li[#data-role='list-divider']/text()");
I just tried to get INFORMATION_LINE_1
But it printed null. I need to get all the INFORMATION_LINES.

It is better to use just XPath rather than mixing it with HTMLUnit methods. Something like this should work to get you the first information line:
HtmlElement e = page.getFirstByXPath("//li[#data-role='list-divider']");
System.out.println(e.asText());
In order to fetch the other information lines you should follow the same approach but changing the XPath string.
Bear in mind you should always debug the page by taking a look at the code by printing the output of page.asXml(). If you use a real browser you are not actually seeing exactly the same as HTMLUnit is seeing. You can stumble with differences particularly if the page executes JavaScript.

Extracting href from a class within other div/id classes with jsoup

Hello I am trying to extract the first href from within the "title" class from the following source (the source is only part of the whole page however I am using the entire page):
div id="atfResults" class="list results ">
<div id="result_0" class="result firstRow product" name="0006754023">
<div id="srNum_0" class="number">1.</div>
<div class="image">
<a href="http://www.amazon.co.uk/Essential-Modern-Classics-J-Tolkien/dp/0006754023/ref=sr_1_1?ie=UTF8&qid=1316504574&sr=8-1">
<img src="http://ecx.images-amazon.com/images/I/31ZcWU6HN4L._AA115_.jpg" class="productImage" alt="Product Details">
</a>
</div>
<div class="data">
<div class="title">
<a class="title titleHover" href="http://www.amazon.co.uk/Essential-Modern-Classics-J-Tolkien/dp/0006754023/ref=sr_1_1?ie=UTF8&qid=1316504574&sr=8-1">Essential Modern Classics - The Hobbit</a>
<span class="ptBrand">by J. R. R. Tolkien</span>
<span class="bindingAndRelease">(<span class="binding">Paperback</span> - 2 Apr 2009)</span>
</div>
I have tried several variations of both the select function and also getElementByClass but all have given me a "null" value such as:
Document firstSearchPage = Jsoup.connect(fullST).get();
Element link = firstSearchPage.select("div.title").first();
If someone could help me with a solution to this problem and recommend some areas of reading so I can avoid this problem in future it would be greatly appreciated.

The CSS selector div.title, returns a <div class="title">, not a link as you seem to think. If you want an <a class="title"> then you should use the a.title selector.
Element link = document.select("a.title").first();
String href = link.absUrl("href");
// ...
Or if an <a class="title"> can appear elsewhere in the document outside a <div class="title"> before that point, then you need the following more specific selector:
Element link = document.select("div.title a.title").first();
String href = link.absUrl("href");
// ...
This will return the first <a class="title"> which is a child of <div class="title">.