Locating elements in <li> using selenium webdriver with java - java

I need to locate each item and click on it.
I tired this:
List<WebElement> allTournaments = driver.findElements(By.xpath("//*[#id='main-section']/div/div/section/div[1]/ul/li/div/div[1]/div"));
for (WebElement tournament : allTournaments) {
tournament.click();
}
But got the error "element not visible"
The xpath of first element is //[#id='main-section']/div/div/section/div1/ul/li1/div/div1/div
xpath of second element is //[#id='main-section']/div/div/section/div1/ul/li[2]/div/div1/div
like wise..
can anybody help..?
HTML:
<div class="header-toggle-area">
<div class="header-notification-area text-right ng-scope" ng-controller="notificationController">
</header>
<!--HEADER ENDS-->
<div class="content-wrap" style="padding-top: 60px;">
<aside id="main-sidebar">
<!--MAIN SIDEBAR ENDS-->
<main id="main-section">
<div class="main-content">
<div class="ng-scope" ng-controller="NewTournamentsController as tournaments">
<header class="member-title dt-title-main">
<section class="dashboard-tournaments-listing">
<div class="" ng-show="tournaments.Page==1">
<ul class="row dashboard-listing">
<!-- ngRepeat: running in tournaments.runningTournaments -->
<li class="dashboard-listing-items ng-scope" ng-repeat="running in tournaments.runningTournaments" style="">
<div class="dtslider-block">
<div class="dtslider-content">
<img ng-src="../../category/categoryDefault.jpg" alt="slider" src="../../category/categoryDefault.jpg"/>
<div class="hover-contents">
<div class="hc-timer">
<p> Tournament ends in </p>
<div class="hct-ticker ng-binding"> 04:30 - 26/11/16 </div>
</div>
<div class="hc-details">
<div class="hcd-block ng-hide" ng-show="running.JoinedPlayers>0">
<ul class="hc-icons">
</div>
</div>
</div>


Check if each element is visible and clickable as follows:
for (WebElement tournament : allTournaments) {
if (tournament.isDisplayed()
&& ExpectedConditions.elementToBeClickable(tournament).equals(true)) {
tournament.click();
}
}

Related

Getting value with Selenium

I was searching quite a lot for finding a solution for my problem and tried a lot with related solutions but they all don´t work for me so i guess i´m missing smth. and would be glad if someone would help me out. So maybe my code first:
package Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.remote.server.handler.FindElement;
import org.openqa.selenium.support.ui.Wait;
import org.openqa.selenium.support.ui.WebDriverWait;
import java.io.FileNotFoundException;
import java.io.PrintWriter;
import java.util.List;
import java.util.concurrent.TimeUnit;
public class oeffnen {
WebDriver driver;
public static void main(String[] args) throws FileNotFoundException {
System.setProperty("webdriver.gecko.driver", "C:\\\\Program Files\\Java\\jdk-11.0.2\\bin\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
driver.get("https://www.bet365.com/#/IP/");
driver.findElement(By.className("lpdgl")).click();
driver.get("https://www.bet365.com/#/IP/");
driver.manage().timeouts().setScriptTimeout(30, TimeUnit.SECONDS);
String value = driver.findElement(By.cssSelector("div.ipo-Competition:nth-child(1) > div:nth-child(3) > div:nth-child(2) > div:nth-child(1) > div:nth-child(2) > div:nth-child(1) > div:nth-child(1) > span:nth-child(2)")).getText();
PrintWriter writer = new PrintWriter("D:\\Test.txt");
writer.println(value);
writer.close();
}
private static void WebDriverWait() {
// TODO Auto-generated method stub
}
}
So i´m trying to get the rate that a certain team wins game. Tried several ways of by.xpath and .getText() and ended up now with by.cssselector and still getting no value in my Test.txt. Or just isn´t it possible at this page (blocked)?
I would be really glad if someone could help me here.
Page: https://www.bet365.com/#/IP/
In regards
edit 13.03.2019
here is the html code of that page:
<div>
<div class="wc-WebConsoleModule ">
<div>
<div class="hm-HeaderModule ">
<div class="hm-HeaderModule_Primary ">
<div class="hm-BigButtons hm-BigButtons-noscrollernative "><nav class="hm-BigButtons_Inner "><a class="hm-BigButton ">Sport</a><a class="hm-BigButton hm-BigButton_Highlight ">Live</a><a class="hm-BigButton ">Casino</a><a class="hm-BigButton ">Live-Casino</a><a class="hm-BigButton ">Spiele</a><a class="hm-BigButton ">Poker</a><a class="hm-BigButton ">Vegas</a></nav></div>
<div class="hm-HeaderModule_UserAdmin ">
<div class="hm-Login ">
<div class="hm-Login_UserNameWrapper "><input class="hm-Login_InputField " type="text" />
<div class="hm-Login_InputText ">Jetzt Anmelden</div>
</div>
<div class="hm-Login_PasswordWrapper "><input class="hm-Login_InputField " type="text" /><input class="hm-Login_InputField Hidden " type="password" /><button class="hm-Login_LoginBtn " tabindex="0">OK</button>
<div class="hm-Login_InputText ">Daten vergessen?</div>
</div>
</div>
</div>
</div>
<div class="hm-HeaderModule_Secondary hm-HeaderModule_Secondary-search ">
<div class="sml-SearchHeader ">
<div class="sml-SearchHeader_SearchIcon "> </div>
<input class="sml-SearchTextInput " type="text" placeholder="Suchen" /></div>
<div class="hm-HeaderModule_Menus ">
<div> </div>
<a class="hm-HeaderModule_Link ">Spielsuchtprävention</a><a class="hm-HeaderModule_Link ">Live-Bilder</a><a class="hm-HeaderModule_Link ">Hilfe</a>
<div>
<div class="hm-LanguageDropDownSelections hm-DropDownSelections "><a class="hm-DropDownSelections_Button hm-DropDownSelections_DropLink ">Sprache<span class="hm-DropDownSelections_Highlight ">Deutsch</span></a></div>
</div>
<div>
<div class="hm-OddsDropDownSelections hm-DropDownSelections "><a class="hm-DropDownSelections_Button hm-DropDownSelections_DropLink ">Quoten<span class="hm-DropDownSelections_Highlight ">Dezimal</span></a></div>
</div>
<div class="hm-ServicesDropDownSelections hm-DropDownSelections "><a class="hm-DropDownSelections_Button hm-DropDownSelections_DropLink hm-DropDownSelections_Solo ">Service</a></div>
</div>
<div class="hm-HeaderModule_TimeLastLogin "><time class="hm-Clock ">20:19:32 CET</time></div>
</div>
</div>
</div>
<div class="wc-PageView ">
<div class="wc-PageView_Main wc-InPlayPage_MainContainer ">
<div>
<div class="ip-InPlayModule ">
<div class="ip-ControlBar ">
<div class="ip-ControlBar_BBarItem wl-ButtonBar_Selected ">Übersicht</div>
<div class="ip-ControlBar_BBarItem ">Einzelanzeige</div>
<div class="ip-ControlBar_BBarItem ">Kalender</div>
</div>
<div class="ipo-OverViewView ">
<div class="ipo-OverViewNativeScroller ipo-OverViewNativeScroller-scrollable ">
<div class="ipo-OverViewNativeScroller_ContentContainer " style="max-height: 424.3333282470703px;">
<div class="ipo-OverViewView_ContentContainer ipo-OverViewNativeScroller_Content ipo-ClassificationBarScrollable_RightArrowVisible ">
<div class="ipo-ClassificationBar ">
<div class="ipo-ClassificationBarScrollable ipo-ClassificationBarScrollable-notransition ">
<div class="ipo-ClassificationBarScrollable_LeftArrow ipo-ClassificationBarScrollable_Hidden "> </div>
<div class="ipo-ClassificationBarScrollable_HScroll">
<div class="ipo-ClassificationBar_ButtonContainer ipo-ClassificationBarScrollable_ScrollContent ipo-ClassificationBarScrollable_ScrollContentNoAnimation ">
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Favoriten</div>
</div>
<div class="ipo-ClassificationBarButtonBase ipo-ClassificationBarButtonBase_Selected ipo-ClassificationBarButtonBase_Selected-1 ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Fußball</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Tennis</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Badminton</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Baseball</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Basketball</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">E-Sports</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Eishockey</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Floorball</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Futsal</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Handball</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Pferderennen</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Snooker</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Squash</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Tischtennis</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Volleyball</div>
</div>
<div class="ipo-ClassificationBarButtonBase ">
<div class="ipo-ClassificationBarButtonBase_Icon "> </div>
<div class="ipo-ClassificationBarButtonBase_Label ">Windhundrennen</div>
</div>
</div>
</div>
<div class="ipo-ClassificationBarScrollable_RightArrow "> </div>
</div>
</div>
<div class="ipo-OverViewDetail ">
<div class="ipo-OverViewDetail_Container ipo-Classification ">
<div class="ipo-ClassificationHeader_Header ipo-ClassificationHeader_Header-1 ipo-ClassificationHeader_Header-lightgreenborder ipo-ClassificationHeader_Header-moremarkets ">
<div class="ipo-ClassificationHeader_HeaderLabel ">Fußball</div>
<div class="ipo-ClassificationHeader_EventButtonWrapper ">
<div class="ipo-ClassificationHeader_EventButtonInnerWrapper ">
<div class="ipo-ClassificationHeader_EventButton ipo-ClassificationHeader_EventButton-selected ">Alle Events</div>
<div class="ipo-ClassificationHeader_EventButton ">Live-Bilder</div>
</div>
</div>
<div class="ipo-ClassificationHeader_MarketsButtonOuterWrapper ">
<div class="ipo-ClassificationHeader_MarketsButtonWrapper ">
<div class="ipo-ClassificationHeader_MarketsButton ipo-ClassificationHeader_MarketsButton-transparent ">Alle Wettmöglichkeiten</div>
<div class="ipo-ClassificationHeader_MarketsButton ipo-InPlayClassificationMarketSelector ipo-ClassificationHeader_MarketsButton-selected ">
<div class="ipo-InPlayClassificationMarketSelector_DropDown ipo-InPlayClassificationMarketSelectorDropDown ip-DropDownContainer ">
<div class="ipo-InPlayClassificationMarketSelectorDropdownLabelContainer ">
<div class="ip-DropDownContainer_Button ipo-InPlayClassificationMarketSelectorDropDown_Button ipo-InPlayClassificationMarketSelectorDropDown_Button-1 ">Haupt-Wettmöglichkeiten</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="ipo-FavouritesContainer "> </div>
<div class="ipo-CompetitionRenderer ">
<div class="ipo-Competition ipo-Competition-open ">
<div class="ipo-CompetitionButton ">
<div class="ipo-CompetitionButton_NameLabel ipo-CompetitionButton_NameLabelHasMarketHeading ">Deutschland - 3. Liga</div>
<div class="ipo-CompetitionButton_MarketHeadingWrapper">
<div class="ipo-CompetitionButton_MarketHeadingLabel ">1X2</div>
<div class="ipo-CompetitionButton_MarketHeadingLabel ">Nächstes Tor</div>
<div class="ipo-CompetitionButton_MarketHeadingLabel ">Tore - Spiel</div>
</div>
</div>
<div class="ipo-Competition_FavouriteWrapper"> </div>
<div class="ipo-FixtureRenderer ipo-Competition_Container ">
<div class="ipo-Fixture ipo-Fixture_CL1 ipo-Fixture-hastimer ipo-Fixture_MainMarkets ">
<div class="ipo-Fixture_TableRow ">
<div class="ipo-Fixture_ScoreDisplay ipo-ScoreDisplayStandard ">
<div class="ipo-ScoreDisplayStandard_Wrapper ">
<div class="ipo-InPlayTimer ">58:27</div>
<div class="ipo-TeamStack ">
<div class="ipo-TeamStack_Team"><span class="ipo-TeamStack_TeamWrapper ">1. FC Kaiserslautern</span>
<div class="ipo-SoccerTeamStack_TeamGoalMarker "><span class="ipo-SoccerTeamStack_TeamGoalMarkerSpan ">TOR</span></div>
</div>
<div class="ipo-TeamStack_Team"><span class="ipo-TeamStack_TeamWrapper ">Eintracht Braunschweig</span>
<div class="ipo-SoccerTeamStack_TeamGoalMarker "><span class="ipo-SoccerTeamStack_TeamGoalMarkerSpan ">TOR</span></div>
</div>
<div class="ipo-TeamStack_Team"><span class="ipo-TeamStack_TeamWrapper ipo-TeamStack_DrawWrapper">Unentschieden</span></div>
</div>
<div class="ipo-TeamPoints ">
<div class="ipo-TeamPoints_TeamScoresWrapper ">
<div class="ipo-TeamPoints_TeamScore ipo-TeamPoints_TeamScore-teamone ">0</div>
<div class="ipo-TeamPoints_TeamScore ipo-TeamPoints_TeamScore-teamtwo ">0</div>
</div>
</div>
<div class="ipo-ScoreDisplayStandard_MediaButton "> </div>
</div>
</div>
<div class="ipo-MainMarkets ">
<div class="ipo-MainMarketRenderer ">
<div class="gl-ParticipantCentered gl-ParticipantCentered_NoHandicap ipo-AllMarketsParticipant gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Odds">1.83</span></div>
<div class="gl-ParticipantCentered gl-ParticipantCentered_NoHandicap ipo-AllMarketsParticipant gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Odds">11.00</span></div>
<div class="gl-ParticipantCentered gl-ParticipantCentered_NoHandicap ipo-AllMarketsParticipant gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Odds">2.30</span></div>
</div>
<div class="ipo-MainMarketRenderer ">
<div class="gl-ParticipantCentered gl-ParticipantCentered_NoHandicap ipo-AllMarketsParticipant ipo-ParticipantNoGoal gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Odds">1.72</span></div>
<div class="gl-ParticipantCentered gl-ParticipantCentered_NoHandicap ipo-AllMarketsParticipant ipo-ParticipantNoGoal gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Odds">6.50</span></div>
<div class="gl-ParticipantCentered gl-ParticipantCentered_NoHandicap ipo-AllMarketsParticipant ipo-ParticipantNoGoal gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Name">Kein 1.</span><span class="gl-ParticipantCentered_Odds">2.62</span></div>
</div>
<div class="ipo-MainMarketRenderer ">
<div class="gl-ParticipantCentered ipo-AllMarketsParticipant gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Handicap">Ü 0.5</span><span class="gl-ParticipantCentered_Odds">1.44</span></div>
<div class="gl-ParticipantCentered ipo-AllMarketsParticipant gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Handicap">U 0.5</span><span class="gl-ParticipantCentered_Odds">2.62</span></div>
<div class="ipo-MainMarketRenderer_BlankParticipant "> </div>`
I want to get the "1.83" in here <div class="gl-ParticipantCentered gl-ParticipantCentered_NoHandicap ipo-AllMarketsParticipant gl-ParticipantCentered_BlankName "><span class="gl-ParticipantCentered_Odds">1.83</span></div>
edit 13.03.2019
Now i am receiving this message:
Exception in thread "main" org.openqa.selenium.NoSuchElementException: Unable to locate element: div.ipo-Competition:nth-child(1) > div:nth-child(3) > div:nth-child(2) > div:nth-child(1) > div:nth-child(2) > div:nth-child(1) > div:nth-child(1) > span:nth-child(2)
edit 14.03.2019
Later, if its working to catch the rate, i would like to make a loop to get the rates of all games with x = 1->n "div:nth-child(x)".
edit 16.03.2019
Ok, since i didnt get forward this way i installed the Selenium IDE on Firefox, recorded just a click on a rate on that page.
Thats the result:
"comment": "",
"command": "click",
"target": "css=.ipo-Competition:nth-child(2) .ipo-MainMarketRenderer:nth-child(1) > .gl-ParticipantCentered:nth-child(3) > .gl-ParticipantCentered_Odds",
"targets": [
["css=.ipo-Competition:nth-child(2) .ipo-MainMarketRenderer:nth-child(1) > .gl-ParticipantCentered:nth-child(3) > .gl-ParticipantCentered_Odds", "css:finder"],
["xpath=//div[2]/div[3]/div/div/div[2]/div/div[3]/span[2]", "xpath:position"],
["xpath=//span[contains(.,'5.00')]", "xpath:innerText"]´
So i tried to get these two ways of locators into my script and just make a click to see if he gets it, but still doesnt work:
driver.findElement(By.cssSelector(".ipo-Competition:nth-child(2) .ipo-MainMarketRenderer:nth-child(1) > .gl-ParticipantCentered:nth-child(3) > .gl-ParticipantCentered_Odds")).click();
driver.findElement(By.xpath("//div[2]/div[3]/div/div/div[2]/div[2]/div[2]/span[2]")).click();

Just checking - Are you trying to get the value out of a button element ? I think you may need to try to capture it in a webElement object (org.openqa.selenium.WebElement).
This is a sample from one of my old Cucumber Test cases which pulls the value from a button element.
WebElement okButton = driver.findElement(By.id("confirmOk"));
WebElement cancelButton = driver.findElement(By.id("confirmCancle"));
assertEquals(okButton.getText(),"OK");
assertEquals(cancelButton.getText(),"Cancel");
You can use either By.Id or By.Xpath or By.cssselector but I think you just need that one extra step to capture the value. Not sure if that answers your question.

So the problem is that you are declaring a string value as null and then in the next line doing .getAttribute(value) which is .getAttribute(null) in this case.
String value = null;
driver.findElement(By.cssSelector(...)).getAttribute(value);
My guess is that you want to do something more like the below to get the text. You need to delete the first line, store the return in the value variable so that it will get printed.
// String value = null; <- DELETE THIS LINE
String value = driver.findElement(By.cssSelector(...)).getText();

How to select two (or more) HTML elements that exist at the same tree level with Jsoup?

I'm working on a project and I faced a problem. I need to scrape data from the website that contains following HTML code:
<div class="lin-curso" style="border: 0;">
<div class="lin-area-c3">
Vagas 2017
</div>
</div>
<div class="box10">
<div class="lin-area-c1">
L160
</div>
<div class="lin-area-c2">
Acupuntura
</div>
<div class="lin-area-c3">
[Lic-1º cic]
</div>
</div>
<div class="lin-curso">
<div class="lin-curso-c1">
</div>
<div class="lin-curso-c2">
3155
</div>
<div class="lin-curso-c3">
Instituto Politécnico de Setúbal - Escola Superior de Saúde
</div>
<div class="lin-curso-c4">
20
</div>
</div>
<br>
<div class="box10">
<div class="lin-area-c1">
9059
</div>
<div class="lin-area-c2">
Administração e Gestão de Empresas
</div>
<div class="lin-area-c3">
[Lic-1º cic]
</div>
</div>
<div class="lin-curso">
<div class="lin-curso-c1">
</div>
<div class="lin-curso-c2">
2270
</div>
<div class="lin-curso-c3">
Universidade Católica Portuguesa - Faculdade de Ciências Económicas e Empresariais
</div>
<div class="lin-curso-c4">
n.d.
</div>
</div>
<br>
<div class="box10">
<div class="lin-area-c1">
8056
</div>
<div class="lin-area-c2">
Administração e Gestão Pública
</div>
<div class="lin-area-c3">
[Lic-1º cic]
</div>
</div>
<div class="lin-curso">
<div class="lin-curso-c1">
</div>
<div class="lin-curso-c2">
4275
</div>
<div class="lin-curso-c3">
Instituto Superior de Ciências da Administração
</div>
<div class="lin-curso-c4">
20
</div>
</div>
<br>
<div class="box10">
<div class="lin-area-c1">
8194
</div>
<div class="lin-area-c2">
Administração da Guarda Nacional Republicana
</div>
<div class="lin-area-c3">
[Mest Integ]
</div>
</div>
<div class="lin-curso">
<div class="lin-curso-c1">
</div>
<div class="lin-curso-c2">
7510
</div>
<div class="lin-curso-c3">
Academia Militar
</div>
<div class="lin-curso-c4">
n.d.
</div>
</div>
<br>
<div class="box10">
<div class="lin-area-c1">
9672
</div>
<div class="lin-area-c2">
Administração e Marketing
</div>
<div class="lin-area-c3">
[Lic-1º cic]
</div>
</div>
BOX10 and line-curso should form an element and they don't.
Because in some lines there is only one BOX10 for one Lin-curso but there are lines that are like Lin-curso for one Box10 , if Box10 and Lin-curso were an element there wouldn't be a problem , is there a way I can associate those two ?
EDIT: The website link is this : http://www.dges.gov.pt/guias/indcurso.asp?letra=A
And the element is the ".inside"

Solution to this problem is fairly easy when you use sibling selector. In your case div with class box10 plays role of a header in the table and sibling divs with class lin-curso play role of table data rows. I would suggest firstly selecting all divs with class box10:
Elements boxes = doc.select("div.box10");
Then you can iterate over boxes and do two major things:
Extract data you are interested in from this div (it contains 3 child nodes, divs with classes lin-area-c1, lin-area-c2 and lin-area-c3)
Select sibling nodes with class lin-curso and extract data from them.
Jsoup provides a method called Element.nextElementSibling() that return sibling element to the element you called this method on. So when you call it on element div.box10 you will get sibling element div.lin-curso.
Sibling in this case means a node immediately following the specified node at the same tree level.
Exemplary solution
Below you can find exemplary code that parses given website and prints table to the console output:
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import java.io.IOException;
final class TestMain {
public static void main(String[] args) throws IOException {
Document doc = Jsoup.connect("http://www.dges.gov.pt/guias/indcurso.asp?letra=A").get();
Elements boxes = doc.select("div.box10");
for (Element box : boxes) {
String linAreaC1 = box.select(".lin-area-c1").text();
String linAreaC2 = box.select(".lin-area-c2").text();
String linAreaC3 = box.select(".lin-area-c3").text();
System.out.printf("%s: %s %s%n", linAreaC1, linAreaC2, linAreaC3);
Element linCurso = box.nextElementSibling();
while (linCurso.hasClass("lin-curso")) {
String linCursoC2 = linCurso.select(".lin-curso-c2").text();
String linCursoC3 = linCurso.select(".lin-curso-c3").text();
String linCursoC4 = linCurso.select(".lin-curso-c4").text();
System.out.printf("%s\t%s\t%s%n", linCursoC2, linCursoC3, linCursoC4);
linCurso = linCurso.nextElementSibling();
}
System.out.println("==============================");
}
}
}
I hope it helps.

Vaadin : CustomLayout don't generate <div> content

I'm using Vaadin 7. When I try to add html content to CustomLayout it does not generate a paragraph inside a div element. Here is the input :
<div style="margin-top: 10px; padding-left: 50px;" class="col-lg-10">
<div class="bs-component">
<ul class="nav nav-pills">
<li class="active"><a href="#Myprofile" class="active"
data-toggle="tab">Profile</a></li>
<li class="">setting</li>
<li class="">Tasks</li>
</ul>
<div class="tabbable">
<div class="tab-content">
<div id="Myprofile" location="Myprofile" class="tab-pane active"></div>
<div id="setting" location="setting" class="tab-pane">
<p>My paragraph</p>
</div>
</div>
</div>
</div>
In java I use this :
String html; //This var contain my input
InputStream is = new ByteArrayInputStream(html.getBytes());
CustomLayout custom = null;
try {
custom = new CustomLayout(is);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Finaly I add the customLayout to my mainLayout. When I run the application I get this output :
<div style="margin-top: 10px; padding-left: 50px;" class="col-lg-10">
<div class="bs-component">
<ul class="nav nav-pills">
<li class="active"><a href="#Myprofile" class="active"
data-toggle="tab">Profile</a></li>
<li class="">setting</li>
<li class="">Tasks</li>
</ul>
<div class="tabbable">
<div class="tab-content">
<div id="Myprofile" location="Myprofile" class="tab-pane active"></div>
<div id="setting" location="setting" class="tab-pane">
</div>
</div>
</div>
</div>
My paragraph inside div with id=setting was not generated by Vaadin. Any Ideas to fix this problem ?

I found the source of the problem. I removed the location="setting" part in the div element and it worked. I don't know why It does that. Probably because CustomLayout when generating the html looks for elements to put in the location, if it does not find one it clean all the element content.

Don't you need the location="setting" attribute? Don't you insert a component for that attribute?

Jsoup getElementsByClass gives size 0 Elements

I'm trying to write a program that fetches weather information from weather.com by using JSoup. This is what I have so far:
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.jsoup.Jsoup;
public class WeatherComHandler {
private Element ForeCastDiv;
public static void main(String[] args) {
WeatherComHandler wch = new WeatherComHandler("http://www.weather.com/weather/today/Brussels+BEXX0005:1:BE");
wch.getTodaysForeCast();
}
public WeatherComHandler(String url) {
getForeCastDiv(url);
}
private void getForeCastDiv(String url) {
try {
Document doc = Jsoup.connect(url).get();
ForeCastDiv = doc.getElementById("wx-forecast-container");
} catch(Exception e) {
e.getStackTrace();
}
}
public void getTodaysForeCast() {
System.out.println(ForeCastDiv.toString());
System.out.println(ForeCastDiv.hasClass("wx-data-part wx-first"));
Elements todaysElements = ForeCastDiv.getElementsByClass("wx-data-part wx-first");
System.out.println(todaysElements.size());
}
}
This is the output:
<div id="wx-forecast-container">
<div class="wx-today wx-module wx-grid3of6">
<div class="wx-title-container">
<h2 class="wx-title"><img src="http://i.imwx.com/common/My.png" /> Forecast for Today</h2>
<div class="wx-updated">
Updated: Oct 28, 5:45pm Local Time
</div>
<div class="wx-promo-links wx-cond wx-severe wx-severe-0">
<ul class="wx-links">
<li class="wx-icon wx-cond wx-desktop"> <a class="wx-icon wx-cond wx-desktop" href="/apps/desktop-weather" location="" from="today_promo_1" title="Weather on my Desktop">Desktop App</a> </li>
</ul>
</div>
</div>
<div class="wx-weather-details">
<div class="wx-timepart-title wx-first">
Right Now
<div class="wx-module wx-grid1of6 wx-weather wx-love-module wx-cond-bg">
<div class="wx-module-inner">
<div class="wx-icon wx-social wx-balloon-wt-lt wx-clickable wx-love-ugh" data-good="true" data-share-trx="" data-share-icon="http://s.imwx.com/img/common/social/lmw-love.jpg" title="Love My Weather" data-share-desc="It's 55°F, Clear">
<div class="wx-icon wx-social wx-heart"></div>
<h4 data-share-title="Love! Right now in Brussels, Belgium"></h4>
</div>
<div class="wx-icon wx-social wx-balloon-wt-rt wx-clickable wx-love-ugh" data-good="false" data-share-trx="" data-share-icon="http://s.imwx.com/img/common/social/lmw-ugg.jpg" title="Don't Love My Weather" data-share-desc="It's 55°F, Clear">
<h4 data-share-title="Ugh! Right now in Brussels, Belgium"></h4>
</div>
</div>
</div>
</div>
<div class="wx-timepart-title">
Earlier Today
</div>
<div class="wx-timepart-title">
Tonight
</div>
<div class="wx-data-part wx-first">
<img src="http://s.imwx.com/v.20131006.214956/img/wxicon/120/31.png" height="120" width="120" alt="Clear" class="wx-weather-icon" />
</div>
<div class="wx-data-part">
<img src="http://s.imwx.com/v.20131006.214956/img/wxicon/120/34.png" height="120" width="120" alt="Fair / Windy" class="wx-weather-icon" />
</div>
<div class="wx-data-part">
<img src="http://s.imwx.com/v.20131006.214956/img/wxicon/120/29.png" height="120" width="120" alt="Partly Cloudy" class="wx-weather-icon" />
</div>
<div class="wx-data-part wx-first">
<div class="wx-temperature">
<span itemprop="temperature-fahrenheit">55</span>
<span class="wx-degrees">°<span class="wx-unit">F</span></span>
</div>
<div class="wx-temperature-label">
FEELS LIKE
<span itemprop="feels-like-temperature-fahrenheit">55</span>°
</div>
</div>
<div class="wx-data-part">
<div class="wx-temperature">
62
<span class="wx-degrees">°</span>
</div>
<div class="wx-temperature-label">
HIGH AT 11:45 AM
</div>
</div>
<div class="wx-data-part">
<div class="wx-temperature">
47
<span class="wx-degrees">°</span>
</div>
<div class="wx-temperature-label">
LOW
</div>
</div>
<div class="wx-data-part wx-first">
<div class="wx-phrase ">
Clear
</div>
<div class="wx-obs-qualifier ">
<span class="wx-hide" itemprop="observation-qualifier">OQ1190</span>
<span class="wx-hide" itemprop="observation-qualifier-severity">2</span>
<span itemprop="observation-qualifier-phrase">Locations nearby are reporting rain with wind.</span>
</div>
</div>
<div class="wx-data-part">
<div class="wx-phrase ">
Fair / Windy
</div>
</div>
<div class="wx-data-part">
<div class="wx-phrase ">
Partly Cloudy
</div>
</div>
<div class="wx-data-part wx-first">
<h6 class="wx-label">Past 24-hr Precip:</h6>
<div class="wx-data">
0.34 in
<span class="wx-estimated">(est.)</span>
</div>
</div>
<div class="wx-data-part">
<h6 class="wx-label"></h6>
<div class="wx-data"></div>
</div>
<div class="wx-data-part">
<h6 class="wx-label">Chance of <span class="wx-firstletter">rain</span></h6>
<div class="wx-data">
<span class="wx-icon wx-raindrop"></span>20%
</div>
</div>
<div class="wx-collapsible">
<div class="wx-data-part wx-first wx-wind">
<h6 class="wx-label">Wind:</h6>
<div class="wx-data">
<div class="wx-icon wx-dir-arrow-sm wind-dir-SW"></div>
<div class="wx-wind-label">
SW at 17 mph
</div>
</div>
</div>
<div class="wx-data-part wx-wind">
<h6 class="wx-label"></h6>
<div class="wx-data">
<div class="wx-wind-label"></div>
</div>
</div>
<div class="wx-data-part wx-wind">
<h6 class="wx-label">Wind:</h6>
<div class="wx-data">
<div class="wx-icon wx-dir-arrow-sm wind-dir-SW"></div>
<div class="wx-wind-label">
SW at 18 mph
</div>
</div>
</div>
<div class="wx-data-part wx-first">
<h6 class="wx-label">Humidity:</h6>
<div class="wx-data">
76%
</div>
</div>
<div class="wx-data-part">
<h6 class="wx-label"></h6>
<div class="wx-data"></div>
</div>
<div class="wx-data-part">
<h6 class="wx-label">Humidity:</h6>
<div class="wx-data">
82%
</div>
</div>
<div class="wx-data-part wx-first">
<h6 class="wx-label">UV Index:</h6>
<div class="wx-data">
0 - Low
</div>
</div>
<div class="wx-data-part">
<h6 class="wx-label"></h6>
<div class="wx-data"></div>
</div>
<div class="wx-data-part"></div>
<div class="wx-data-part wx-first ">
<h6 class="wx-label"></h6>
<p class="wx-text"></p>
</div>
<div class="wx-data-part ">
<h6 class="wx-label"></h6>
<p class="wx-text"></p>
</div>
<div class="wx-data-part ">
<h6 class="wx-label">Tonight:</h6>
<p class="wx-text">Partly cloudy skies. Low 47F. Winds SW at 15 to 25 mph.</p>
</div>
<div class="wx-astro-neighbor">
<div class="wx-data-part wx-first">
<h6 class="wx-label">Pressure:</h6>
<div class="wx-data">
<span itemprop="barometric-pressure-incheshg"> 29.59 in </span>
<span class="wx-icon wx-dir-arrow-sm wx-pressure-steady"></span>
</div>
</div>
<div class="wx-data-part wx-first wx-dewpoint">
<h6 class="wx-label">Dew Point: <span class="wx-icon wx-long wx-info"> <i>Humidity becomes more noticeable as the dew point rises above 60°F. Readings above 75°F can feel stifling.</i> </span> </h6>
<div class="wx-data">
47°
</div>
</div>
<div class="wx-data-part wx-first">
<h6 class="wx-label">Visibility:</h6>
<div class="wx-data">
6.0 mi
</div>
</div>
</div>
<div class="wx-astro-details">
<h3 class="wx-astro-title">Sun & Moon</h3>
<dl class="wx-first">
<dt>
Sunrise:
</dt>
<dd>
7:28 am
</dd>
<dt>
Sunset:
</dt>
<dd>
5:25 pm
</dd>
</dl>
<dl>
<dt>
Moonrise:
</dt>
<dd>
12:27 am
</dd>
<dt>
Moonset:
</dt>
<dd>
2:32 pm
</dd>
</dl>
<dl class="wx-moonphase-container">
<dt>
Moonphase:
</dt>
<dd>
<div class="wx-icon wx-moonphase wx-moon-23"></div>
<div class="wx-moonphase-label">
Waning Crescent
</div>
</dd>
</dl>
</div>
</div>
</div>
<div class="wx-toggle-collapsible-container">
<div class="wx-toggle-collapsible wx-clickable">
<div class="wx-control wx-expand-control" tracktype="action" trackstr="today_rightnow_showmore">
<span class="wx-icon wx-arrow wx-mb-lg-dn"></span>Open Weather Details
</div>
<div class="wx-control wx-collapse-control wx-hide" tracktype="action" trackstr="today_rightnow_showless">
<span class="wx-icon wx-arrow wx-mb-lg-up"></span>Close Weather Details
</div>
</div>
</div>
<div class="wx-pagination-linkers">
<div class="wx-next-linker">
<ul class="wx-links">
<li class=""> <a class="" href="/weather/hourbyhour/graph/Brussels+BEXX0005:1:BE" location="" from="today_rightnow-Today_tempGraph_1" title="">Hourly Forecast</a> </li>
</ul>
</div>
</div>
</div>
<script>$("#wx-icon wx-long wx-info").attr('title','This is the hover-over text');</script>
<div id="wunderground-lightbox" class="wx-dialog">
<div class="wx-close" title="Close">
<div class="wx-icon"></div>
</div>
</div>
<div class="wx-combinedPromo wx-module wx-grid3of6">
<div class="wx-commuter-promo" data-classname="wx-grid-2" from="today-commuterpromo" data-layers="traffic,weather" data-locationinus="false" data-backgroundimageurl="" data-location-zoom="10" data-location-latitude="50.85045" data-location-longitude="4.34878">
<div class="wx-commute-promo-callout-title">
<div class="wx-titleIconHolder">
<img class="wx-titleIcon" src="http://s.imwx.com/img/module/TrafficCams_16x16.png" />
</div>
<div class="wx-titleText">
NEW! Traffic & Commuter Forecast
</div>
</div>
</div>
</div>
<div class="wx-clear"></div>
<div class="wx-scroll-tracking wx-scroll-pres" data-modulecode="ScrollTracking1_Today" data-location="BEXX0005:1:BE" data-groupname="weblocal"></div>
<div class="wx-free-title ">
<h3>Latest Stories</h3>
</div>
<div id="wx-rightnowsecond-container">
<div id="wx-rightnow-container">
<div class="wx-module wx-mod1 wx-corsican wx-1x155M wx-render-mode" id="mod_4" data-vr-zone="today_beta_Media_top2">
<div class="wx-inner" data-vr-contentbox=""></div>
</div>
<div class="wx-module wx-mod1 wx-corsican wx-1x155M wx-render-mode" id="mod_5" data-vr-zone="today_beta_Media_top1">
<div class="wx-inner" data-vr-contentbox=""></div>
</div>
</div>
<div class="wx-module wx-trending-stories wx-tabbed-content wx-grid2of6">
<div class="wx-inner">
<h2 class="wx-title">Most Popular</h2>
<ul class="wx-tabs" data-vr-zone="Most Popular">
<li class="wx-tab wx-active" data-tab="today" data-vr-contentbox="">Today</li>
<li class="wx-tab wx-clickable" data-tab="this_week" data-vr-contentbox="">This Week</li>
</ul>
<ul class="wx-tab-content" data-tab="today" data-vr-zone="Most Popular">
<li data-vr-contentbox="">Superstorm Sandy: Then and Now (EXCLUSIVE PHOTOS)</li>
<li data-vr-contentbox="">Eerie Nighttime Shots of Abandoned America</li>
<li data-vr-contentbox="">Astounding Fall Photos That Will Make You Say 'Wow!'</li>
<li data-vr-contentbox="">Nightmares Fear Factory: Terrified Reactions at Haunted House (PHOTOS)</li>
<li data-vr-contentbox="">The 15 Worst U.S. Cities for Asthma</li>
<li data-vr-contentbox="">Shocking End to Shark Attack <span class="wx-icon wx-video wx-trending-video"></span></li>
<li data-vr-contentbox="">Gigantic Hornets Are Breeding <span class="wx-icon wx-video wx-trending-video"></span></li>
<li data-vr-contentbox="">18-Foot WHAT? <span class="wx-icon wx-video wx-trending-video"></span></li>
</ul>
<ul class="wx-tab-content" data-tab="this_week" style="display:none;" data-vr-zone="Most Popular">
<li data-vr-contentbox="">Superstorm Sandy: Then and Now (EXCLUSIVE PHOTOS)</li>
<li data-vr-contentbox="">Nightmares Fear Factory: Terrified Reactions at Haunted House (PHOTOS)</li>
<li data-vr-contentbox="">Eerie Nighttime Shots of Abandoned America</li>
<li data-vr-contentbox="">8 Deadly Venomous Creatures and Those Who've Survived (PHOTOS)</li>
<li data-vr-contentbox="">Astounding Fall Photos That Will Make You Say 'Wow!'</li>
</ul>
</div>
</div>
</div>
<div class="wx-map wx-cc-map wx-grid3of6 wx-module pagelet " data-severelocation="false" data-location="BEXX0005" data-location-type="1" data-overlay="" data-zoom="" data-tropical="" data-mapsize="3" data-fragid="frag_newccMap_166739" data-modulecode="beta_CCMap" data-pageid="62288">
</div>
</div>
false
0
The problem is getTodaysForeCast() keeps saying that the size of todaysElements is 0, which it shouldn't be. Could anyone explain me why this is happening or what I'm doing wrong?
thanks in advance.

Jsoup will not accept two combined classes in the getElementsByClass method. Try something like:
Elements todaysElements = ForeCastDiv.getElementsByAttributeValueContaining("class", "wx-data-part wx-first");

You're passing two separate class names (separated by a space) to both hasClass() and getElementsByClass(), both of which can only handle one class at a time.
Try using select() instead. It allows you to use complex CSS selectors to find elements, like this:
ForeCastDiv.select(".wx-data-part.wx-first")

How to select a <a href> javascript button using selenium java

I'm trying to click the "Add to Shopping Bag" button by utilising selenium (Java) interface on my AUT, but I'm unable to do so.
My script as follows:
public void addToBag()
{
WebElement AddToBag = null;
List<WebElement> SizeList = driver.findElement(By.className("selection-wrapper")).findElements(By.className("step-wrapper"));
for(WebElement e : SizeList)
{
if (e.getText() == " ")
{
if(e.findElement(By.tagName("span")).findElement(By.tagName("a")).getAttribute("href").contains("javascript:AddToCart()"))
{
AddToBag = e.findElement(By.tagName("span")).findElement(By.tagName("a"));
AddToBag.click();
break;
}
}
}
Thread.sleep(2000);
}
Source code:
<div class="subitem-steps span-12 last" style="z-index: 890;">`
<div class="item-styles span-12 last" style="z-index: 885;">`
<div class="selection-wrapper" style="z-index: 870;">`
<div class="step-wrapper" style="z-index: 869;">
<div class="step-wrapper" style="z-index: 863;">
<div class="step-wrapper" style="z-index: 857;">
<div class="step-wrapper" style="z-index: 851;">
<div class="step step4" style="z-index: 850;"/>
<span class="ui-btn-grey btn-add-to-cart btn-add-to-cart-sel">
<a href="javascript:AddToCart()" onclick="s_objectID="javascript:AddToCart()_1";return this.s_oc?this.s_oc(e):true"/>
</span>
<div class="external-links" style="z-index: 849;">
</div>
</div>
</div>`

Categories