Hello everybody I want to get the data from
http://sansoyunlari.hurriyet.com.tr/SayisalLoto/SayisalLotoSonuclari.aspx this adress by using jsoup ı can get them but only the latest results . There is a dropdownlist on the website which consists dates how can I reach other dates ? by the way I will move these codes to the android these are codes which is written in netbeans for now. ı will put a dropdownlist to my android program which get the data from this adress and also the results.
these are my java codes I wrote until now
public static void main(String[] args) {
String adres = "http://sansoyunlari.hurriyet.com.tr/SayisalLoto/SayisalLotoSonuclari.aspx";
ArrayList sayi = new ArrayList<>();
sayi.add("six");
sayi.add("five");
sayi.add("four");
sayi.add("three");
sayi.add("two");
sayi.add("one");
//Sayısal Loto
try {
Document doc = Jsoup.connect(adres).get();
Elements sonuclar = doc.select("div.hurriyet2010_so_sanstopu_no_bg");
//1. yi manuel almak gerek ilk yoldan çünkü resut diye kodlanmış
Elements sonuclar1 = doc.select("span#_ctl0_ContentPlaceHolder1_lblresut"+sayi.get(sayi.size()-1));
Element numaralar = sonuclar1.first();
System.out.println(numaralar.text());
//yol 1 numaraları almak için
for (int i = sonuclar.size();i>1;i--)
{
sonuclar1 = doc.select("span#_ctl0_ContentPlaceHolder1_lblresult"+sayi.get(i-2));
Element numaralar1 = sonuclar1.first();
System.out.println(numaralar1.text());
}
//yol 2 numaraları almak için
// for(Element el : sonuclar)
// {
// System.out.println(el.text());
// }
//kazanan kişi sayısı ve ikramiye tutarı için
for(int i = 0;i<4;i++)
{
int b = 6 -i;
System.out.println(b + " bilen kişi sayısı :");
sonuclar = doc.select("span#_ctl0_ContentPlaceHolder1_lblluckycount"+sayi.get(i));
Element el = sonuclar.first();
System.out.println(el.text());
System.out.println("Kişi başına düşen ikramiye :");
sonuclar = doc.select("span#_ctl0_ContentPlaceHolder1_lblluckyamount"+sayi.get(i));
el = sonuclar.first();
System.out.println(el.text());
}
}
catch(Exception e){
}
}
To get the select item you should do:
Element select = doc.select("#_ctl0_ContentPlaceHolder1_ddlSayisalLotoDates").first();
Now the children of this elements are the "option" items you want:
for (Element e : select) {
String date = e.text();
}
edit
I looked at the html source. In order to get the right page you need to do a post request at the URL "http://sansoyunlari.hurriyet.com.tr/SayisalLoto/SayisalLotoSonuclari.aspx" with following params:
__EVENTARGUMENT = empty
__EVENTTARGET = _ctl0$ContentPlaceHolder1$ddlSayisalLotoDates
__EVENTVALIDATION = a random value that you get from the html page
__LASTFOCUS = empty
__VIEWSTATE = another random value
_ctl0:ContentPlaceHolder1:ddlSayisalLotoDates = The ID of the date you want to search (i.e. 884 for 19 Ekim 2013)
txtSearch = can be empty
As you can see, it's quite annoying scraping an ASP.NET webpage..
Use an application like Fiddler (or another one) to find the params you need to post (hidden inputs, session cookies, your selected input). Probably you're missing some of them.
Hope it helps.
Related
In my code, I try to find all elements with a specific name, then try taking each elements' descendant and get its title, link and price. The price I'm having issues with because it sticks to the price tag of the first element from the WebElements list.
List<WebElement> autos = driver.findElements(By.xpath("//section[contains(#class ,'ui-search-results')]/ol/li//a[#class = 'ui-search-result__content ui-search-link']"));
for(WebElement auto : autos) {
String model = auto.getAttribute("title");
String page = auto.getAttribute("href");
String price = auto.findElement(By.xpath("//span[#class = 'price-tag-fraction']")).getText();
System.out.println(model + page + price);
}
Console is printing model and page just fine but the price is always the same one. I already tested the site and there is a price-tag-fraction per element.
When you use XPath and want to start searching from a specific element, you need to add a . to the start of the XPath. In your case
"//span[#class = 'price-tag-fraction']"
becomes
".//span[#class = 'price-tag-fraction']"
Your updated code
List<WebElement> autos = driver.findElements(By.xpath("//section[contains(#class ,'ui-search-results')]/ol/li//a[#class = 'ui-search-result__content ui-search-link']"));
for(WebElement auto : autos) {
String model = auto.getAttribute("title");
String page = auto.getAttribute("href");
String price = auto.findElement(By.xpath(".//span[#class = 'price-tag-fraction']")).getText();
System.out.println("Model: %s, Page: %s, Price: %s".formatted(model, page, price));
}
NOTE: I changed your print statement to make it easier to read. You could also write these to a CSV file and then open them later in Excel, etc. as a table.
I need to fetch data from the website "https://www.arbatunity.com/index.php", I want the data from the top right of the website that says current market profit.
I need this as a string value that can be updated.
With the JSoup library, this is easy:
Document doc = Jsoup.connect("https://www.arbatunity.com/index.php").get();
Elements elements = doc.select("#id_profit b");
String percent = ""
for (Element e : elements) {
percent = e.html();
}
//percent holds the String you're looking for
I am doing a selenium test against a web page which returns a table with some rows and columns showing payment data. I'm trying to strip some characters/words from the result of the XPATH i'm using because i dont need the part while doing an assertion (check if the data in table is correct).
Normally the webpage also returns a "Dropdown Button" as text (there is an icon), just before the identification number (e.g 168.3285.6021 as seen below).
What i used is it.set(it.next().replaceAll("DropDown Arrow ","")); so the DropDown Arrow text is replaced with nothing, which only works for the first line, but the other 2 lines don't get replaced. Any tips?
public void check_receivals() {
// Check how many lines and assert the size (from xpath)
List<WebElement> Receivals = driver.findElements(By.xpath("//*[#id='received-overview']//div[#class='bdpo-overview-table-row']/div[#class='claims']"));
System.out.println(Receivals.size() + " receival lines found");
assertEquals(7, Receivals.size());
// Test data to compare against..aka expectedResultList
List<String> expectedResultList = new ArrayList<>();
expectedResultList.add ("168.3285.6021\n" + "Payment 2015\n" + "01-01-2015\n" + "€ 246");
expectedResultList.add ("143.8407.8413\n" + "Payment 2015\n" + "01-01-2015\n" + "€ 233");
expectedResultList.add ("154.2841.2407\n" + "Payment 2015\n" + "01-01-2015\n" + "€ 253");
// Assert
List<WebElement> ReceivalLines = driver.findElements(By.xpath("//*[#id='received-overview']//div[#class='bdpo-overview-table-row']/div[#class='claims']"));
List<String> ReceivalLines_List = ReceivalLines.stream().map(WebElement::getText).collect(Collectors.toList());
ListIterator<String> it = ReceivalLines_List.listIterator();
while(it.hasNext()) {
it.set(it.next().replaceAll("DropDown Arrow ",""));
assertEquals(ReceivalLines_List, expectedResultList);
THe issue is that you are modifying the iterator as you are working with it. I would suggest making the replace part of the stream operation using the map function.
List<String> ReceivalLines_List = ReceivalLines.stream().map(WebElement::getText).map(s -> s.replaceAll("DropDown Arrow ","")).collect(Collectors.toList());
I am trying to retrieve the records of users from the grid using the below code from OrangeHRM demo site with demo credentials. However, It always returns/prints the first record values i.e. Admin. Could you tell me where I did wrong?
URL: http://opensource.demo.orangehrmlive.com/
User: Admin
Password: admin
Users Grid data: Admin > User Management> User page
Objective: Read user names data
Code:
WebElement element;
List<WebElement> usersList = driver.findElements(By.xpath("//*[#id='resultTable']/tbody/tr"));
for (int i = 0; i < usersList.size(); i++) {
element = usersList.get(i);
System.out.println(i+":"+element.findElement(By.xpath("//td[2]/a")).getText());
}
I know, we can achieve it by usng xpath "//*[#id='resultTable']/tbody/tr/td[2]/a". But, I'd like to select checkbox based on the given name and what's my mistake.
Thanks for your help in advance.
Try below solutions
System.out.println(i+":"+element.findElement(By.xpath("td[2]/a")).getText());
System.out.println(i+":"+element.findElement(By.xpath("decendent:td[2]/a")).getText());
System.out.println(i+":"+element.findElement(By.xpath(".td[2]/a")).getText());
You can find it more convenient way like below.
WebElement table = driver.findElement(By.id("resultTable"));
List<String> list = new ArrayList<String>();
List<WebElement> oddNames = table.findElements(By.className("odd"));
for(WebElement elem : oddNames){
System.out.println(elem.findElement(By.tagName("a")).getText());
String name = elem.findElement(By.tagName("a")).getText();
list.add(name);
}
List<WebElement> evenNames = table.findElements(By.className("even"));
for(WebElement elem : evenNames){
System.out.println(elem.findElement(By.tagName("a")).getText());
String name = elem.findElement(By.tagName("a")).getText();
list.add(name);
}
use the below code you will get list of usernames
List<WebElement> _Name=driver.findElements(By.xpath("//table[#id='resultTable']/tbody/tr"));
for(int j=0;j<_Name.size();j++)
{
String str="//table[#id='resultTable']/tbody/tr["; System.out.println(driver.findElement(By.xpath(str+j+"]/td[2]")).getText());
}
First get all tr elements and keep them on a WebElement list. Then get the user link element using tagName a in the loop. And finally use getText() to retrieve each user name.
You can use like below:
List<WebElement> usersList = driver.findElements(By.xpath("//*[#id='resultTable']/tbody/tr"));
for (WebElement userElem : usersList) {
String username = userElem.findElement(By.tagName("a")).getText();
System.out.println(username);
}
I think, it is more convenient than other solutions as the solution not using any tr td index. You know, different browser behaves differently. so index may be changed sometimes for different browser.
I have a drop list as id="product-size" and the items S,M,L,XL.
<select id="product-size" onchange=" addToWishList();">
<option>Select</option>
<option id="2119362" value="4">S</option>
<option id="2119363" value="7">M</option>
<option id="2119364" value="8">L</option>
<option id="2119365" value="4">XL</option>
</select>
I have used an array to store these items and at Runtime I need to access the first element 'S'.The problem I am facing is ,I was not able to click on the first element S at runtime.
I have written the code as follows :
driver.get("https://m.staging.karmaloop.com/product/The-Infinity-Tee/407819");
WebElement j =driver.findElement(By.id("product-size"));
String text = j.getText();
String[] DDLcount =text.split("\n");
for (int i=1;i<=DDLcount.length-1;i++)
{
driver.findElement(By.xpath(Testconfiguration.size_dropdown_10deep)).click();
Thread.sleep(5000);
driver.findElement(By.name(DDLcount[i])).click();
}
Can anyone help me to sort out this problem ?
From the code you've supplied, you're using an invalid selector for the Options.
They don't appear to have a name attribute
Aside from modifying the loop, you could make the operation faster if the DOM isnt reconstructed.
WebElement selectBox = driver.findElement(By.xpath(Testconfiguration.size_dropdown_10deep));
List<WebElement> options = selectBox.findElements(By.tagName("option"));
for ( WebElement option : options )
{
selectBox.click();
option.click();
}
By locator = By.id("product-size");
Select select = new Select(webdriver.findElement(locator));
You can you use any of the three following options to select an item from dropdown
select.selectByIndex(index); // Give Index as parameter
select.selectByValue(value); // Give the value of the option tag
select.selectByVisibleText(value); // Give the visible text as parameter
WebElement element =driver.findElement(By.id("product-size"));
Select sel = new Select(element);
List<WebElement> items = sel.getOptions();
boolean stringExits = false;
for(int i =0; i<items.size(); i++)
{
String text = items.get(i).getText();
if(text.equals("S"))
{
stringExists = true;
break;
}
}
if(stringExits)
{
System.out.println("The string exists");
}else
{
System.out.println("The string does not exist");
}