We Keep Coding

Trying to get the text out of a List of webelements with Selenium WebDriver

In my code, I try to find all elements with a specific name, then try taking each elements' descendant and get its title, link and price. The price I'm having issues with because it sticks to the price tag of the first element from the WebElements list.
List<WebElement> autos = driver.findElements(By.xpath("//section[contains(#class ,'ui-search-results')]/ol/li//a[#class = 'ui-search-result__content ui-search-link']"));
for(WebElement auto : autos) {
String model = auto.getAttribute("title");
String page = auto.getAttribute("href");
String price = auto.findElement(By.xpath("//span[#class = 'price-tag-fraction']")).getText();
System.out.println(model + page + price);
}
Console is printing model and page just fine but the price is always the same one. I already tested the site and there is a price-tag-fraction per element.

When you use XPath and want to start searching from a specific element, you need to add a . to the start of the XPath. In your case
"//span[#class = 'price-tag-fraction']"
becomes
".//span[#class = 'price-tag-fraction']"
Your updated code
List<WebElement> autos = driver.findElements(By.xpath("//section[contains(#class ,'ui-search-results')]/ol/li//a[#class = 'ui-search-result__content ui-search-link']"));
for(WebElement auto : autos) {
String model = auto.getAttribute("title");
String page = auto.getAttribute("href");
String price = auto.findElement(By.xpath(".//span[#class = 'price-tag-fraction']")).getText();
System.out.println("Model: %s, Page: %s, Price: %s".formatted(model, page, price));
}
NOTE: I changed your print statement to make it easier to read. You could also write these to a CSV file and then open them later in Excel, etc. as a table.

Find dynamic style attribute in a webelement

I'm having a problem extracting the full style Attribute, as part of it still didn't reached the DOM- Its an image that sometimes takes 1-2 seconds to load in the screen, so what happens is that there IS a string with attribute, but it CHANGES when the image arrives from the server.
I'm doing a WebDriverWait for the element to be visible in the DOM, but when I try to take its "style" attribute (that has the "...;(url:"http://....")", it sometimes isn't there, and then my substring() fails.
Here is the code:
#Step("Print Image src url")
public String printImage(Integer imgNo){
WebElement imgStyle = (new WebDriverWait(driver, 15)).until(ExpectedConditions.visibilityOfElementLocated(By.xpath(Consts.ACTIVE_IMG_XPATH_1000 + "/div[" + (currentImg + 1) +"]/div[1]")));
String styleString = imgStyle.getAttribute("style");
Integer idxUrl = styleString.indexOf("url");
Integer idxJpg = styleString.indexOf("jpg");
String urlImage = styleString.substring(idxUrl+5,idxJpg+3); //styleString.indexOf("http"),20
Log.info("V - Image " + imgNo + " src is: " + urlImage);
return urlImage;
}
I can simply add System.wait(3000), but I don't wan't to use it. Any ideas how to solve it nicer? Can I wait for visibility of String?
Thanks for your time.

What is the HTML like? You could use contains or starts-with and the beginning of the url before the part that is dynamic and changes. Like WebDriverWait(driver, 15).until(EC.visibility_of_element_located(By.XPATH, "//div[contains(#style, 'text']"))

Apache POI PPT (Java) - Updating TextShape keeping text formatting/shape formatting

I am trying to produce several reports (i.e. N PPTX files) based on different inputs/for different users on the same PPTX template I created.
I have several preformatted XSLFTextShape on the PPTX template that contains a single XSLFTextParagraph already formatted (i.e. both the shape and the text). Each shape contains a particular placeholder that I need to substitute with a dynimic value. I have this value in a Map (placeholder,newValue). I am successful in updating the placeholder with the new value using:
textShape.clearText();
XSLFTextRun run = paragraph.addNewTextRun();
run.setText(newText);
So, when I produce the PPTX in output the text is updated but font color, font formatting, font size are changed compared to those I defined in the template. How can I keep the same formatting?
Any solutions to simply change the text while keeping original formatting?
Thanks in advance!

For everybody which may be interested in this topic in the future - I post the solution (working if one TextBox has a single Paragraph). This solution loops on all text boxes and in the case one contain one of the vales specified in the Placeholder->newValue map, it will update it maintaining the formatting.
public static void updateTextBoxesWithDesiredValues(XMLSlideShow ppt, Map<String, String> placeHolderDefinedValue) {
logger.info("ElapsedTime: " + tM.getTimeElapsedReadableFormat() + " ########## Updating single text box content...");
List<XSLFSlide> allSlides = ppt.getSlides();
int updatedElements = 0;
for (XSLFSlide currentSlide : allSlides) {
for (XSLFShape shape : currentSlide.getShapes()) {
if (shape instanceof XSLFTextShape) {
XSLFTextShape textBox = (XSLFTextShape) shape;
String elementTextContent = textBox.getText();
for (Object key : placeHolderDefinedValue.keySet()) {
if (elementTextContent.equals(key)) {
List<XSLFTextParagraph> textBoxParagraphs = textBox.getTextParagraphs();
List<XSLFTextRun> textBoxParagraphTextRuns = textBoxParagraphs.get(0).getTextRuns();
//System.out.println("########################## check paragraph number in textbox: " + textBoxParagraphs.size() + " - TextRuns: " + textBoxParagraphs.get(0).getTextRuns().size());
logger.info("ElapsedTime: " + tM.getTimeElapsedReadableFormat() + updatedElements + ") Updating: " + textBox.getText() + " --> " + placeHolderDefinedValue.get(key));
for (XSLFTextRun r : textBoxParagraphTextRuns) {
r.setText(placeHolderDefinedValue.get(key));
}
updatedElements++;
//break;
}
}
}
}
}
logger.info("ElapsedTime: " + tM.getTimeElapsedReadableFormat() + " Total Text Element Content Updated: " + updatedElements + " #########################");
}

It's kind of horrible - but yeah there's a reason they called it "POI".
Here's my approach to "only reset text" of an existing XSLFTextShape (that must have at least some text pre-set!):
textShape.getTextParagraphs().get(0).getTextRuns().get(0).setText(text);
for (int i = 1; i < textShape.getTextParagraphs().get(0).getTextRuns().size(); i++) {
textShape.getTextParagraphs().get(0).getTextRuns().get(i).setText("");
}
for (int i = 1; i < textShape.getTextParagraphs().size(); i++) {
textShape.getTextParagraphs().get(i).getTextRuns().stream().filter(tr -> !tr.getRawText().equals("\n")).forEach(tr -> tr.setText(""));
}
It will replace all existing text(paragraphs/runs) with "empty" text, but linebreaks can't be replaced for some reason. So this might leave you with some trailing lines - as they usually(!) are transparent this won't really hurt a lot.
.clearText / removing paragraphs either destoyed the formatting for me, or didn't work. Trying to reset the style (fontColor, fontFamily, fontSize, isBold, isItalit, ...) didn't result in satisfying results :(

Querying/Accessing/Retrieving Header row in Google Spreadsheet

I'm accessing a public Google Docs Spreadsheet using the Google Sheets API. The API says that when you query a list-feed for a worksheet, you get back a list of rows excluding the first row, which is the header row (by convention).
Is there a way to access the header row? I see that you can use the cells feed to request specific cells:
// Fetch column 4, and every row after row 1.
URL cellFeedUrl = new URI(worksheet.getCellFeedUrl().toString()
+ "?min-row=2&min-col=4&max-col=4").toURL();
CellFeed cellFeed = service.getFeed(cellFeedUrl, CellFeed.class);
Is there another way that is more explicit, to retrieve the header row?

I searched long and hard, but it appears there is no semantically explicit way to grab headers out of a spreadsheet. As I mentioned in the question, you can use the cell feed so this is what I did:
URL cellFeedUrl = new URL(worksheet.getCellFeedUrl().toString() + "?min-row=1&max-row=1");
CellFeed cellFeed = service.getFeed(cellFeedUrl, CellFeed.class);
for(CellEntry cellEntry : cellFeed.getEntries()) {
System.out.print(cellEntry.getCell().getValue() + ",");
}
System.out.println();
The important part is the ?min-row=1&max-row=1 part. This gives you all the cells in the first row of the sheet. By convention, the very first row in a worksheet is treated as the header.

getTags()
this might return an iterable with "name", "address", "manager", "employeeid"."
https://developers.google.com/gdata/javadoc/com/google/gdata/data/spreadsheet/CustomElementCollection#getTags()
Example - ListDemo.java
public void printAndCacheEntry(ListEntry entry) {
// We only care about the entry id, chop off the leftmost part.
// I.E., this turns http://spreadsheets.google.com/..../cpzh6 into cpzh6.
String id = entry.getId().substring(entry.getId().lastIndexOf('/') + 1);
// Cache all displayed entries so that they can be updated later.
entriesCached.put(id, entry);
out.println("-- id: " + id + " title: " + entry.getTitle().getPlainText());
for (String tag : entry.getCustomElements().getTags()) {
out.println(" <gsx:" + tag + ">"
+ entry.getCustomElements().getValue(tag) + "</gsx:" + tag + ">");
}
}
http://gdata-java-client.googlecode.com/svn-history/r497/trunk/java/sample/spreadsheet/list/ListDemo.java
I have never used this, so not 100% sure. But looks like what you want.

jsoup how to reach dropdownlist

Hello everybody I want to get the data from
http://sansoyunlari.hurriyet.com.tr/SayisalLoto/SayisalLotoSonuclari.aspx this adress by using jsoup ı can get them but only the latest results . There is a dropdownlist on the website which consists dates how can I reach other dates ? by the way I will move these codes to the android these are codes which is written in netbeans for now. ı will put a dropdownlist to my android program which get the data from this adress and also the results.
these are my java codes I wrote until now
public static void main(String[] args) {
String adres = "http://sansoyunlari.hurriyet.com.tr/SayisalLoto/SayisalLotoSonuclari.aspx";
ArrayList sayi = new ArrayList<>();
sayi.add("six");
sayi.add("five");
sayi.add("four");
sayi.add("three");
sayi.add("two");
sayi.add("one");
//Sayısal Loto
try {
Document doc = Jsoup.connect(adres).get();
Elements sonuclar = doc.select("div.hurriyet2010_so_sanstopu_no_bg");
//1. yi manuel almak gerek ilk yoldan çünkü resut diye kodlanmış
Elements sonuclar1 = doc.select("span#_ctl0_ContentPlaceHolder1_lblresut"+sayi.get(sayi.size()-1));
Element numaralar = sonuclar1.first();
System.out.println(numaralar.text());
//yol 1 numaraları almak için
for (int i = sonuclar.size();i>1;i--)
{
sonuclar1 = doc.select("span#_ctl0_ContentPlaceHolder1_lblresult"+sayi.get(i-2));
Element numaralar1 = sonuclar1.first();
System.out.println(numaralar1.text());
}
//yol 2 numaraları almak için
// for(Element el : sonuclar)
// {
// System.out.println(el.text());
// }
//kazanan kişi sayısı ve ikramiye tutarı için
for(int i = 0;i<4;i++)
{
int b = 6 -i;
System.out.println(b + " bilen kişi sayısı :");
sonuclar = doc.select("span#_ctl0_ContentPlaceHolder1_lblluckycount"+sayi.get(i));
Element el = sonuclar.first();
System.out.println(el.text());
System.out.println("Kişi başına düşen ikramiye :");
sonuclar = doc.select("span#_ctl0_ContentPlaceHolder1_lblluckyamount"+sayi.get(i));
el = sonuclar.first();
System.out.println(el.text());
}
}
catch(Exception e){
}
}

To get the select item you should do:
Element select = doc.select("#_ctl0_ContentPlaceHolder1_ddlSayisalLotoDates").first();
Now the children of this elements are the "option" items you want:
for (Element e : select) {
String date = e.text();
}
edit
I looked at the html source. In order to get the right page you need to do a post request at the URL "http://sansoyunlari.hurriyet.com.tr/SayisalLoto/SayisalLotoSonuclari.aspx" with following params:
__EVENTARGUMENT = empty
__EVENTTARGET = _ctl0$ContentPlaceHolder1$ddlSayisalLotoDates
__EVENTVALIDATION = a random value that you get from the html page
__LASTFOCUS = empty
__VIEWSTATE = another random value
_ctl0:ContentPlaceHolder1:ddlSayisalLotoDates = The ID of the date you want to search (i.e. 884 for 19 Ekim 2013)
txtSearch = can be empty
As you can see, it's quite annoying scraping an ASP.NET webpage..

Use an application like Fiddler (or another one) to find the params you need to post (hidden inputs, session cookies, your selected input). Probably you're missing some of them.
Hope it helps.