I am looking to get data from this webpage: http://www.sportinglife.com/greyhounds/racecards/29-10-2014/belle-vue
I have been using jSoup and Java, but can't seem to get the data I am looking for. I need the times of each race (Jump to: 14:18 14:37 14:57 15:17 15:38 15:58 16:18 16:37 16:57 17:17 17:33 17:47 18:04 18:18) and the link that each of them refers to.
I then need to go to each link and print out the 6 dogs in each race.
So the output would look like:
14:18
1 Golden Light
2 Always Late
3 Redley Rooster
4 Redstone Bo Dhu
5 Ballymac Oprah
6 Ballyhill Slide
For each race.
My current code is below, and uses jSoup to extract the runners from the race - but I can't seem to do the first step of getting the race "times" and links to each race page so I can loop through the links and output the runners for each race.
Document doc = Jsoup.connect(
"http://www.sportinglife.com/greyhounds/racecards/29-10-2014/belle-vue/card/834800").get();
Element tableHeader = doc.select("tbody").first();
Map<String, String> data = new HashMap<>();
for (Element element : tableHeader.children()) {
// Here you can do something with each element
String dog = element.select("td:eq(0)").text();
String race = element.select("td:eq(2)").text();
data.put(dog, race);
System.out.println(dog + " " + race);
}
Any help is very much appreciated.... thanks!
Rob
Looking at your page, the race information are not directly in the second TD but in a link (a) in the second TD, then you need to replace :
String race = element.select("td:eq(2)").text();
with :
String race = element.select("td:eq(2) a").text();
Related
There are 2 drop-down lists. Each has a similar meaning, for example, "Jorge". Lists in different modules. When I need to fill in, for example, a list that is lower in the tree, then the first match is taken along the XPath path, on an undisclosed list.
Not lists, but values in drop-down lists!
There are 2 drop-down lists. Each has a similar meaning, for example, "Jorge". Lists in different modules. When I need to fill in, for example, a list that is lower in the tree, then the first match is taken along the XPath path, on an undisclosed list.
Not lists, but values in drop-down lists!
I wanted to implement it in Java this way:
Example:
if (findElement(By.xpath("(//example//example)")).isDisplayed()) {
findElement(By.xpath("(//example//example)")).click();
}
But in this case, the element is not displayed.
How to implement a search of all values similar to the XPath path in order to get the one that is displayed?
I tried to do something like this: (//example//example)1 (//example//example)[2] (//example//example)[3]
In my case, we have that 1 - the element does not exist [2] - exists, but is not displayed (isDisplayed = false) [3] - exists, is displayed (isDisplayed = true)
iterating through the values in the loop for [n] cannot be implemented, because, for example, the value 1 is not.
Described as difficult as possible :D. Excuse me.
If someone understands my nonsense, please help me. How to implement my requirement?
enter image description here
UPD:
The problem was solved (for me) by substituting the first value into the expression ()"{1}" immediately.
Now I'm interested in why I get an exception after the first iteration:
Method threw 'org.openqa.selenium.ElementNotInteractableException' exception.
Code:
int number = 1;
String option = "(//ul[contains(#style, 'display: block')]//li//span[contains(text(),'" + valueField + "') or strong[contains(text(),'" + valueField.toUpperCase() + "')]])";
findElement(By.xpath(option+"["+number+"]"));
String[] words = valueField.split(" ");
StringBuilder builder = new StringBuilder();
for (int i = 0; i < words.length; i++) {
builder.append(words[i]);
setFieldByLabel(nameModule, nameLabel, builder.toString());
fastWaitLoading();
for (int y = 0; y < 10; y++) {
if (findElement(By.xpath(option+"["+number+"]")).isDisplayed()) {
new Actions(browser.getWebDriver())
.moveToElement(findElement(option))
.click()
.build()
.perform();
break;
}
number++;
}
}
So I am trying to fully understand your question, and I don't. What I would recommend for a situation like this is, iterate through all elements by creating a list with: findElements(By.xpath ... )
This way you will get a list of webelements and you can iterate through them. Then apply a foreach, assert if element is displayed (it exists as it has been found with findElements) and you should be able to interact with it.
Yeah, everything is in a prominent place)
Missed it
new actions(browser.getWebDriver()) .moveToElement(findElement(**option**)) .click() .build() .perform(); break;
Here
new actions(browser.getWebDriver())
.moveToElement(findElement(**option + "[" + number+"]"**))
.click()
.build()
.perform();
break;
I have a program which reads an XML file using Java DOM and processes certain element. For example, here is part of the document I am looking at:
<Flow>
<Id>306</Id>
<Type>Simple</Type>
<FlowContent Width="0.2000000000000000111">
<P Id="523"><T xml:space="preserve" Id="652">A spouse’s pension would be paid equal to <O Id="351"/>% of your Core pension at date of death.</T>
</P>
</FlowContent>
(Note: this is exported from a program called GMC Inspire Designer, so I have no control over its format.)
I can process most elements fine, but have issues with text content which also contains elements. In the example above, another layout object <O Id="351"/> (referencing another piece of text or a variable) occurs in the body of the text.
I can look up this element and retrieve it using the ID number. This is the element linked in the above snippet:
<Variable>
<Id>351</Id>
<Name>CAMT44</Name>
What I would then like to do is output information from the linked node (e.g., I could look up the node with ID 351 and retrieve the name etc. then display this information in place of where the element appears within the string).
I currently look up children and store the ID in a string array like so:
NodeList nl = e.getElementsByTagName("O");
sa = new String[nl.getLength()]; // Set up new array to hold child ids
for (int i = 0; i < nl.getLength(); i++) {
sa[i] = nodeToElement(nl.item(i)).getAttribute("Id");
}
I'm very much a Java beginner, so I've been wondering if DOM was the correct choice for this project. Perhaps I should have used SAX instead, but as I don't have much XML experience, I'm not sure which best suits my needs and, as I mentioned, I have managed to do most of the things I need, it's just this last tricky bit that I'm stuck on.
Currently my output looks like this:
IF CR.SCHEME == "EXCT" PRINT:
"A spouse’s pension would be paid equal to % of your Core pension at
date of death, ignoring the fact that you may have chosen to convert
part of your pension into a lump sum at retirement."
Child flow: 351
It would be great if there is some way to do this using DOM. Apologies if anything is unclear, I'm new to most of this.
You should be able to do something like this:
String output = "";
for (int i = 0; i < nl.getLength(); i++) {
Node n = nl.item(i);
if(n.getNodeType() == Node.TEXT_NODE) {
output += n.getTextContent();
} else if (n.getNodeType() == Node.ELEMENT_NODE && n.getNodeName().equals("O")) {
output += lookup(doc, ((Element)n).getAttribute("id"));
}
}
System.out.println(output);
The lookup method is something you would need to implement yourself but it would look something like this:
private static String lookup(Document doc, String id) {
return "<IMPLEMENT_LOOKUP_HERE>";
}
So I will have to explain what I'm looking for and I will use an example very general to this so this could help others with the same problem:
Lets assume I'm working at the police station and I'm creating a DB with a table called "Policestations" and the Police of my country has 3 policestations.
Those stations wan't to be as efficient as possible therefore they will pick those incidents if they fullfill some parameters for our example here those 3 policestations:
policeStation:1010
offender:GuyA
place:GB (iso-code)
delict:murder
policeStation:2020
offender:GuyA
place: * (* = "for not relevant")
delict:murder
policeStation:3030
offender:GuyB
place:*
delict:*
So now, I have a method that looks like that (could someone edit this the code thing doesnt work when I add that into it):
public String responsiblePoliceStation(String offender, String place, String delict) {
Query q = em.createQuery("SELECT a.Policestation FROM PoliceStations a WHERE a.offender = :offender AND a.place = :place AND a.delict = :delict );
q.setParameter("offender", offender);
q.setParameter("place", place);
q.setParameter("delict", delict);
String policestation = (String) q.getSingleResult();
return policestation;
}
so now I can call that method for example with that:
PSPRouteService psp = new PSPRouteService();
String pS = psp.responsiblePoliceStation("GuyA", "GB", "murder");
`
and this will work fine, I will get as response that the policeStation with String 1010 has to "solve" this incident.... but as I intended in the header I want a loop so for example thats assume I have that incident:
offender: GuyA
place: Brazil
delict: murder
then this wouldnt work because I dont know how to implement that, if some parameter doesn't fit in then just compare it with the next row and look if the next Policestation is more tolerant and doesn't care for example whether the incident took place.
So I want kinda a query-loop where I compare the given Parameter with my DB and if something doesnt fit I just skip to the next row compare that and if that doesnt fit I keep going to the next and as you see above the third policeStation is more tolerant they dont care, what happend and where they just take the incident if the offender was GuyB...... so any ideas and tipps how to implement that?
I am trying to manipulate this blocks of codes:
List<WebElement> more = driver.findElements(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[4]/button"));
if(more.size()!=0){
driver.findElement(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[4]/button")).click();
}else {
WebElement present = driver.findElement(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div/table/tbody"));
List<WebElement> list = present.findElements(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div/table/tbody/tr"));
System.out.println("Total Number of TR: " + list.size());
}
What Im trying to do hopefully is that before I execute what is in the IF statement I want to loop that everytime it sees the element /html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[4]/button it will be click and if it is not available then ill execute this:
WebElement present = driver.findElement(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div/table/tbody"));
List<WebElement> list = present.findElements(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div/table/tbody/tr"));
System.out.println("Total Number of TR: " + list.size());
For further info what I am trying to do here is this.
I am on a listview for a specific module, and then at the button there is a "Click here for more records" -> its XPath is /html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[4]/button
I want that if i am in the listview and that button is present as mentioned above -I want to click it. And if in the listview there is no button "Click here for more records" (for example the records is composed of 5records only so there is no pagination clearly) I want to execute a blocks of code.
What I understood from your query is that you want to loop your IF condition statement multiple times until your else condition satisfies. For this, you can try following code:
(1) try{
(2) String xpathVAL="/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[4]/button";
(3) int i=1;
(4) while(i!=0){
(5) if(driver.findElements(By.xpath(xpathVAL)).size() != 0)
(6) {
(7) driver.findElement(By.xpath(xpathVAL).click();
(8) Thread.sleep(2000);
(9) }
(10) else{
(11) Thread.sleep(2000);
(12) WebElement present = driver.findElement(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div/table/tbody"));
(13) List<WebElement> list = present.findElements(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div/table/tbody/tr"));
(14) LOGGER.debug("Total Number of TR: " + list.size());
(15) i=0;
(16) }
(17) }
(18) }catch(Exception e){
(19) LOGGER.debug("Exception Caught");
(20) WebElement present = driver.findElement(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div/table/tbody"));
List<WebElement> list = present.findElements(By.xpath("/html/body/div[1]/div/div[3]/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div/table/tbody/tr"));
}
I think your requirement would be to keep click a button until some window appears and the button disappears. What I suggest for this is, apply "Thread.sleep(//some 3-4 seconds//)". However, it is poor style of coding and using forceful thread is not suggested by many standards. You can put some other implicit wait there. Also, if your element is not found after certain period of time, you can apply break condition after certain time interval, else it will go to infinite loop. If you are having trouble writing the code, let me know and I will help you in that.
EDIT
Try following points to remove element not present in cache:-
Use Explicit wait(Thread.sleep) at line 8 and line 11
Change your xpath from position based to some id/name/class type. Changing it to NAME based gets more clearer picture of the element even if its position is changed.
Again re-initialize the element if exception is caught at line 20.
hello guys i am try to print the output of two element data simultaneously
Document document2 = Jsoup.parse(webPage2);
Document document22 = Jsoup.parse(webPage2);
Elements links2 = document2.select("a.yschttl");
Elements links22 = document22.select("div.abstr");
can we include both a.yschttl and div.abstr or...
for (Element link2 : links2) {
out.println(link2);
}
can we include two say links2 and links22 in same for loop...
or how to achive it...
You can do something like:
for (int i = 0; i < links2.size(); i++) {
out.println(links2.get(i));
out.println(links22.get(i));
}
But in this case you will get IndexOutOfBoundsException if size of links22 higher than size of links2.
What do you want to achieve?
If you are just trying to select both at the same time, you can do something like this:
for (Element link : document.select("a.yschttl, div.abstr") {
out.println(link);
}
If you are trying to make two selections and outputting those values in tandem, you will have to do something like #vacuum suggests, but being careful of the lengths of the lists.
A side note, you don't have to parse the document twice to make two selections. You can parse once and select twice.