I am working on HTML unit with Java to read HTML pages. I have a scenario where I have to read messages from the popup/ alert window. I have an index page page = form.getInputByName("index").click();
After I click on the index page I get the response page. But before that I get an alert some thing like
I want to read the above message and then proceed with ok.
I tried with alert handlers like `
ConfirmHandler okHandler = new ConfirmHandler(){
#Override
public boolean handleConfirm(Page page, String message) {
System.out.println(">>>>>>>>>>>>>>>>>>> message--"+message);
return true;
}
};
webClient.setConfirmHandler(okHandler);`
But this is not working for me.
You should be using the CollectingAlertHandler instead:
CollectingAlertHandler alertHandler = new CollectingAlertHandler();
webClient.setAlertHandler(alertHandler);
/*Your browsing codes here*/
List<String> alertmsgs = new ArrayList<String>();
alertmsgs = alertHandler.getCollectedAlerts();
Then you can use the message obtained as you want. Reminder: You do not need to click the OK button.
Related
I'm trying to parse several goods. The problem is I can parse only first element.
To be specific: I need goods which were sorted by "Newest" on the website below. And I click this button before parsing.
driver.get("https://www.banggood.com/new-arrivals.html");
driver.findElementByCssSelector("body > div.lastest-box.J-loading-box > div > ul.filtrate.J-filter-list > li:nth-child(2)").click();
So I want to get info about each good doing like:
// save goods in arrayList
List<Article> articleList = new ArrayList<>();
WebElement ulElement = driver.findElement(By.cssSelector("div.box > ul.prt-list.J-prt-list"));
List<WebElement> liElements = industries.findElements(By.tagName("li"));
// using loop for parsing
for (WebElement element : liElements) {
String price = element.findElement(By.cssSelector("div.price-box > span.price.notranslate")).getText();
String title = element.findElement(By.cssSelector("p.title")).getText();
String url = element.findElement(By.cssSelector("span.img > a")).getAttribute("href");
// switch link to get img
String newUrl = url.replace("https://www.banggood.com/new-arrivals.html", url);
// open new tab using another url
((JavascriptExecutor)driver).executeScript("window.open()");
ArrayList<String> tabs = new ArrayList<>(driver.getWindowHandles());
driver.switchTo().window(tabs.get(1));
driver.get(newUrl);
WebElement img = driver.findElement(By.id("landingImage"));
String srcImg = img.getAttribute("src");
articleList.add(new Article(srcImg, url, title, price));
// close new tab and back to the main window
((JavascriptExecutor)driver).executeScript("window.close()");
}
driver.close();
driver.quit();
}
Via debug I see it works:
but loop is going through only first element then the program is freezing. And I don't see any issues in the console.
Can someone tell me what I'm missing here? Thanks in advance.
Your driver is freezing because it is searching for a window that doesn't exist anymore. After closing your window you need to explicitly switch your window back to the first window:
// close new tab and back to the main window
((JavascriptExecutor)driver).executeScript("window.close()");
driver.switchTo().window(tabs.get(0));
Background : I have a webpage (.aspx) which have few dropdown lists.The list value is getting populated using Ajax call based on the selection of previous dropdown. After selecting the value of all drop down lists we can click on download button and the data will be downloaded based on the downloaded data we need to perform some other operations.
what i already did: I am able to set the drop down data via calling the ajax correctly but sending a post request is a problem. Here is the code snippet/pseudo Code.
Feel free to use any tool along with java
public static void main(String[] args) throws FailingHttpStatusCodeException, IOException {
final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17);
WebRequest request = new WebRequest(new URL(DataDownloader.MY_URL),HttpMethod.POST);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.setJavaScriptTimeout(10000);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getOptions().setTimeout(10000);
HtmlPage page = webClient.getPage(request);
HtmlSelect firstDd = (HtmlSelect) page.getElementById("dd1_id");
List<HtmlOption> firstOption = firstDd.getOptions();
firstDd.setSelectedAttribute(firstOption.get(2), true);
webClient.waitForBackgroundJavaScript(3000);
HtmlPage pgAfterFirstDd = (HtmlPage) webClient.getCurrentWindow().getEnclosedPage();
HtmlSelect secondDd = (HtmlSelect) pgAfterFirstDd.getElementById("dd2_id");
List<HtmlOption> secondOption = secondDd.getOptions();
secondDd.setSelectedAttribute(secondOption.get(2), true);
webClient.waitForBackgroundJavaScript(10000);
//set the value for all other dropdowns
HtmlPage finalpage = (HtmlPage) webClient.getCurrentWindow().getEnclosedPage();
HtmlForm form = finalpage.getHtmlElementById("aspnetForm");
webClient.waitForBackgroundJavaScript(10000);
request.setRequestBody("REQUESTBODY");
Page redirectPage = webClient.getPage(request);
// HtmlSubmitInput submitInput=form.getInputByName("btnSubmit");
// submitInput.click();
/*HtmlButton submitButton = (HtmlButton) pageAfterWard.createElement("btnSubmit");
submitButton.setAttribute("type", "submit");
form.appendChild(submitButton);
HtmlPage nextPage = (HtmlPage) submitButton.click();*/
}
Why you hide your error details? Is there any secret? If you like helpful answers you have to provide as many information as possible.
So i do a wild guess...
submitInput.click();
will return a PDF. In this case you have to do something like
Page pdfPage = submitInput.click();
WebResponse resp = pdfPage.getWebResponse();
if("application/pdf".equals(resp.getContentType())) {
.... process the bytes
.... resp.getContentAsStream()
}
HtmlUnit has four kind of pages HtmlPage/XmlPage/TextPage and UnexpectedPage. Binary content like PDF or office documents are handled as UnexpectedPage. Processing this content is up to you.
as you mentioned in the comment under RBRi's Answer that you were getting the typecast error.can you please mention
what the exact error you were getting
what type of file/response you were expecting.
Because the code looks good to me and it should work perfectly..
final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17);
Looks like you are using an old version, please use the latest one.
WebRequest request = new WebRequest(new URL(DataDownloader.MY_URL),HttpMethod.POST);
With HtmlUnit you usually do not work with requests. The idea is to work more 'browser like'. Use something like getPage(final URL url).
List<HtmlOption> firstOption = firstDd.getOptions();
firstDd.setSelectedAttribute(firstOption.get(2), true);
Do your work more 'browser like'
firstOption.get(2)setSelected(true);
This will do all the background work for you like deselection the other options and event processing for you.
Regarding submitting the form your idea of
HtmlSubmitInput submitInput=form.getInputByName("btnSubmit");
HtmlPage nextPage = submitInput.click();
looks correct. Maybe your have to wait after that also.
If you still have problems you have to provide the URL you are working with to enable us to reproduce/debug your case.
I'm using HTMLUnit in order to put text in a input box and then clicking on a link which is actually a JS call.
The problem comes up when I put text in an input , using inputBox.setValueAttribute("example input"); . In this case , after clicking the button the page does not change at all.
On the other hand , once I delete inputBox.setValueAttribute("example input"); and then click the button, the page content does change and includes an error for empty input.
Below is the code i've used in order to put text in the relevant input and then click the button.
public void addressToBlockPlot(){
WebClient client = new WebClient(BrowserVersion.FIREFOX_24);
client.getOptions().setThrowExceptionOnScriptError(false);
client.getOptions().setThrowExceptionOnScriptError(false);
client.setJavaScriptTimeout(10000);
client.getOptions().setJavaScriptEnabled(true);
client.setAjaxController(new NicelyResynchronizingAjaxController());
client.getOptions().setTimeout(10000);
try {
HtmlPage page = client.getPage("http://mapi.gov.il/Pages/LotAddressLocator.aspx");
HtmlInput inputBox = (HtmlInput)page.getHtmlElementById("AddressInput");
final HtmlAnchor a = (HtmlAnchor) page.getElementById("helkaSubmit");
inputBox.setValueAttribute("example input");
a.click();
client.waitForBackgroundJavaScript(2000);
HtmlPage page2= (HtmlPage) client.getCurrentWindow().getEnclosedPage();
System.out.println(page2.asXml());
} catch (Exception e) {
}
}
Any ideas for solving this issue ?
EDIT:
I've tried using inputBox.setValueAttribute(""); , which resulted in receiving the same error that I got when no input text was set at all.
I have almost similar issue, where JS take some time to load, I have solved this issue by setting wait and retries until page load successfully by adding a condition,
int input_length = page.getByXPath("//input").size();
int tries = 5;
while (tries > 0 && input_length < 12) {
tries--;
synchronized (page) {
page.wait(2000);
}
input_length = page.getByXPath("//input").size();
System.out.println("page loaded successfully");
}
here input_length determines that whether required page is loaded or not. you can use similar condition according to structure of your page.
I try to fill and submit an HTML form using HtmlUnit. One select element and its options are loaded using <body onLoad="...">.
My problem: I cannot retrieve this select element via getSelectByName, getChildElements etc. (ElementNotFoundException is thrown), although I can see that the data has been loaded when looking at the org.apache.http.wire log.
When printing page.asXml(), I see only the unaltered HTML document.
My code:
public static void main(final String[] args) throws Exception {
final URL url = new URL("http://www.rce-event.de/modules/meldung/annahme.php?oid=471&pid=1&ac=d98482bbf174f62eaaa4664c&tkey=468&portal=www.dachau.de&ortsbox=1&callpopup=1");
final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_3_6); // tried also FIREFOX_3
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
final HtmlPage page = webClient.getPage(url);
webClient.waitForBackgroundJavaScript(10000); // tried also Thread.sleep()
// tried also to use webClient.getCurrentWindow().getEnclosedPage() instead of 'page'
final HtmlForm form = page.getFormByName("formular");
// ElementNotFoundException thrown here:
final HtmlSelect select = form.getSelectByName("event.theme");
final HtmlOption option = select.getOptionByText("Sport/Freizeit");
final Page newPage = select.setSelectedAttribute(option, false);
// submit etc.
}
Stacktrace:
Exception in thread "main" com.gargoylesoftware.htmlunit.ElementNotFoundException: elementName=[select] attributeName=[name] attributeValue=[event.theme]
at com.gargoylesoftware.htmlunit.html.HtmlForm.getSelectByName(HtmlForm.java:449)
at Xyzzy.main(Xyzzy.java:58)
I tried everything written here, here, and here (and even more), without any success.
Update:
I simplified my code and started a bounty.
Your problem is that the select named "event.theme" only gets loaded once the select named "event.datapool" has a value of "1" as selected.
So you need to change the "event.datapool" select value to "1" :
[........]
final HtmlSelect selectBase = form.getSelectByName("event.datapool");
final HtmlOption optionBase = selectBase.getOptionByText("Freizeit / Tourismus");
final Page newPage = selectBase.setSelectedAttribute(optionBase, true);
[........]
But you may have problems because the "HTML" data for the select "event.theme" is loaded via ajax. So I do not think your java "HtmlSelect" class will loaded the select "event.theme" in the form as Javascript does with an actual user interation.
A solution to that would be to:
1. Load your page "http://www.rce-event.de/modules/meldung/annahme.php?oid=471&pid=1&ac=d98482bbf174f62eaaa4664c&tkey=468&portal=www.dachau.de&ortsbox=1&callpopup=1"
2. Load the page "http://www.rce-event.de/modules/meldung/js/xmlhttp_querys.php?get_kat=1&time=1338409551228&id=1&block=kat" > which will return the "event.theme" select data/values
3. Then use the data loaded in step 2 to update the page loaded in step 1 by inserting a "select list with id and name set to <event.theme>" in the HTML element "kat_content"
Then your form/loaded webpage should have the new select named "event.theme" and therfore the following code shouldn't produce errors anymore.
final HtmlSelect select = form.getSelectByName("event.theme");
final HtmlOption option = select.getOptionByText("Sport/Freizeit");
final Page newPage = select.setSelectedAttribute(option, false);
I'm using HtmlUnit in Java to navigate to a web page. From that webpage i need to log in and then go from there. I know how to type in the user name and password but then there is a dropdown box where i need to select one of the options. How do i select an option from a dropdown box in HtmlUnit?
Thanks
You can navigate and manipulate the page <select> elements using HtmlSelect:
WebClient client = ...
Page page = client.getPage(url);
HtmlSelect select = (HtmlSelect) page.getElementById(mySelectId);
HtmlOption option = select.getOptionByValue(desiredOptionValue);
select.setSelectedAttribute(option, true);
The JavaDoc shows that there are a lot of flexible API methods for doing things like this.
Add the follwoing lines:
protected void selectOption(WebElement el, String option)
{
Select select = new Select(el);
select.selectByVisibleText(option);
}
protected WebElement elById(String id)
{
return driver.findElement(By.id(id));
}
// "title" is your drop-down HTML id
public void populateForm(String elValue)
{
selectOption(elById("title"), elValue);
}
Following code:
HtmlSelect select = page.getElementById(mySelectId);
should be:
HtmlSelect select = (HtmlSelect)page.getElementById(mySelectId);