We Keep Coding

Convert a Jsoup "Element" into a WebDriver "WebElement"?

So I found that if you get the page source with WebDriver, you actually get the generated source of the entire DOM (not just the HTML source code of the page that loaded). You can then use this String to generate a Jsoup Document. This is cool, because Jsoup is much faster than WebDriver at searching for elements, it also has a much better API to do so.
So, is there anyway to turn a Jsoup Element into a WebDriver WebElement? I saw another post on stackoverflow about using a method to generate an xpath from the Jsoup document, but that's not what I'm looking for since WebDriver will still have to parse the page and use the Xpath to lookup the element, defeating the purpose (unless your porpuse is purely to use Jsoup for its superior Selector methods).
The reason I want to try and use Jsoup to find WebElements for WebDriver is because on some websites, WebDriver is very very slow (I work for a company that automation hundreds of 3rd party websites, we have no control over these sites).

There seems to be a confusion between interactive and non-interactive tools here.
WebDriver tests are very often slow (in my experience) due to unnecessary and defensive waits and delays, using improperly-understood frameworks, and often written by junior or outsourced developers - but fundamentally also because WebDriver is mimicking a real user's actions in 'real time' on a real browser, and communicating with the browser app using an API (based on a specification) and a protocol. It's interactive.
(Less so with HtmlUnit, PhantomJS etc.)
By contrast, Jsoup is just a glorified HTTP client with extra parsing capabilities. It's non-interactive, and ultimately works off a snapshot String of data. We'd expect it to be much faster for its particular use-cases.
Clearly both are HTTP clients of a sort, and can share static web content, which is why WebDriver could pass data off for processing by Jsoup (though I've never heard of this use-case before).
However, Jsoup can never turn one of its Elements (a Java snapshot object containing some properties) into a WebDriver WebElement, which is more a kind of 'live' proxy to a real and interactive object within a program like Firefox or Chrome. (Again, less so with HtmlUnit, PhantomJS etc.)
So you need to decide whether interactivity is important to you. If it's crucial to mimic a real user, WebDriver has to 'drive' the process using a real browser.
If it's not, then you can consider the headless browsers like HtmlUnit and (especially) PhantomJS, as they will be able to execute JavaScript and update the DOM in a way that the HTTP libraries and Jsoup can't. You can then pass the output to Jsoup etc.
Potentially, if you went down the PhantomJS route, you could do all your parsing there using the JavaScript API. See: Use PhantomJS to extract html and text etc.
For a lot of people, interactivity isn't important at all, and it's quicker to drop WebDriver completely and rely on the libraries.

I know this question is incredibly old, but just so anyone who comes to see this can find this answer. This will return an xpath from your Jsoup Element. This was translated to Java by me, but the original source I copied the code from was https://stackoverflow.com/a/48376038/13274510.
You can then use the xpath with WebDriver
Edit: Code works now
public static String jsoupToXpath(Element element) {
String xpath = "/";
List<String> components = new ArrayList<>();
Element child = element.tagName().isEmpty() ? element.parent() : element;
System.out.println(child.tag());
while (child.parent() != null){
Element parent = child.parent();
Elements siblings = parent.children();
String componentToAdd = null;
if (siblings.size() == 1) {
componentToAdd = child.tagName();
} else {
int x = 1;
for(Element sibling: siblings){
if (child.tagName().equals(sibling.tagName())){
if (child == sibling){
break;
} else {
x++;
}
}
}
componentToAdd = String.format("%s[%d]", child.tagName(), x);
}
components.add(componentToAdd);
child = parent;
}
List<String> reversedComponents = new ArrayList<>();
for (int i = components.size()-1; i > 0; i--){
reversedComponents.add(components.get(i));
}
xpath = xpath + String.join("/", reversedComponents);
return xpath;
}

WebDriver : SendKeys(Integer) NOT working in Firefox 29

I am using Firefox 29 and WebDriver java 2.41.0 bindings to automate test scenarios. Have one scenario to input an integer to a input-box which was working FINE with Firefox 28 and now failing with v29 i.e latest FF version. The code I wrote for the same is:
int inputString = 123456;
driver.FindElement(By.Id("tinymce")).SendKeys(inputString);
Please help me getting through of this.

This will be the result of this issue:
https://code.google.com/p/selenium/issues/detail?id=7291
Fixed by this revision in the Selenium code:
https://code.google.com/p/selenium/source/detail?r=afde40cbbf5c

Quick test below worked for me. I understand JS is not the right way to do browser simulation, one should always FIRST use webdriver methods since they use browsers native api, but thought it would unblock you while the bug is fixed in selenium
DesiredCapabilities desiredCapabilities = DesiredCapabilities.firefox();
desiredCapabilities.setCapability(CapabilityType.HAS_NATIVE_EVENTS,true);
WebDriver driver = new FirefoxDriver(desiredCapabilities);
driver.get("http://yizeng.me/2014/01/31/test-wysiwyg-editors-using-selenium-webdriver/");
WebElement frame = driver.findElement(By.id("tinymce-editor_ifr"));
driver.switchTo().frame(frame);
WebElement body = driver.findElement(By.id("tinymce"));
JavascriptExecutor js = (JavascriptExecutor)driver;
js.executeScript("arguments[0].innerHTML = '<h1>Heading</h1>Hello There'",body);

Can you check following points
Can you make sure there are no application changes, I mean the locator is returning single node element.
Can you also post the exception that you are getting. To troubleshoot this problem, need these details.

One thing I do before sending a value to a box is to clear it, also, the value I send is always a string, the parsing should be done by the page/code as you need to validate what the user has entered.
But I do agree with skv, we need to see the actual error being thrown.

How to copy Google Translate's Chinese transliteration using Selenium?

I'm trying to extract Google Translate's pinyin transliteration of a Chinese word using Selenium but am having some trouble finding its WebElement.
For example, the word I look up is "事". My code would be as follows:
String word = "事";
WebDriver driver = new HtmlUnitDriver();
driver.get("http://translate.google.com/#zh-CN/zh-CN/" + word);
When I go to the actual page using my browser, I can see that its pinyin is "Shì" and that its id, according to Inspect Element is src-translit. However, when I go to view source, though the id="src-translit" is present, you don't see anything resembling "Shì" nearby. It's simply empty.
Thinking that the page has had no time to load properly. I implemented a waiting period of 30 seconds (kind of a long wait, I know, but I just wanted to know if it would work).
int timeoutInSeconds = 30;
WebDriverWait wait = new WebDriverWait(driver, timeoutInSeconds);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("src-translit")));
Unfortunately, even with the wait time, transliteration and its text still returns as empty.
WebElement transliteration = driver.findElement(By.id("src-translit"));
String pinyin = transliteration.getText();
My question, then, is: what's happened to the src-translit? Why won't it display in the html code and how can I go about finding it and copying it from Google Translate?

Sounds like javascript isn't being executed. Looking at the docs, you can enable javascript like this
HtmlUnitDriver driver = new HtmlUnitDriver();
driver.setJavascriptEnabled(true);
or
HtmlUnitDriver driver = new HtmlUnitDriver(true);
See if that makes a difference.
EDIT:
I still think the problem is related to javascript. When I run it using FirefoxDriver, it works fine: the AJAX request is made, and src-translit element has been updated with Shi.
Workaround:
In any case, monitoring the network traffic, you can see that when you want to translate 事 , it makes an AJAX call to
http://translate.google.com/translate_a/t?client=t&sl=zh-CN&tl=zh-CN&hl=en&sc=2&ie=UTF-8&oe=UTF-8&pc=1&oc=1&otf=1&rom=1&srcrom=1&ssel=0&tsel=0&q=%E6%B2%92%E4%BA%8B
Which returns JSON:
[[["事","事","Shì","Shì"]],,"zh-CN",,[["事",,false,false,0,0,0,0]],,,,[],10]
Maybe you could parse that instead for now.

Type TAB in text area using Selenium Webdriver

Recenty I migrated from Selenium RC to Webdriver. Typing text containing delimiter TAB used to work fine on Selenium RC. But when using webdriver, typing tab moves focus to next input.
Sample text :
Name Age
Mark 35
I did the following :
if(text.contains("\t"))
{
data = text.split("\t");
for (String str : data)
{
element.sendKeys(str);
element.sendKeys(Keys.TAB);
}
}
else
{
element.sendKeys(text);
}
I tried using elements.sendKeys("\t") as well as elements.sendKeys("\\t").
Any suggestions on how to achieve this?
Thanks in advance.

You can use the Actions class for advanced operations. Refer this site http://www.guru99.com/keyboard-mouse-events-files-webdriver.html

As already said in the comments, the use of sendKeys will work as if you were a normal user of the browser. That means the browser will focus on the next input. If you really need to add a TAB in the input, I think the solution is to execute JavaScript.
JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript("document.getElementById('" + inputId + "').value = '\\t';");

How to deal with file uploading in test automation using selenium or webdriver

I think that everybody who uses Webdriver for test automation must be aware of its great advantages for web development.
But there is a huge issue if file uploading is part of your web flow. It stops being test automation. The security restriction of browsers (invoking file selection) practically makes it impossible to automate tests.
Afaik the only option is to have Webdriver click the file upload button, sleep the thread, have developer/tester manually select the file, and then do the rest of the web flow.
How to deal with this, is there a workaround for it? Because it really can't be done like this. It wouldn't make sense.
This is the only case I know of when browser security restrictions do not apply:
<script language=javascript>
function window.onload(){
document.all.attachment.focus();
var WshShell=new ActiveXObject("WScript.Shell")
WshShell.sendKeys("D:\MyFile.doc")
}
</script>

Webdriver can handle this quite easily in IE and Firefox. Its a simple case of finding the element and typing into it.
driver = webdriver.Firefox()
element = driver.find_element_by_id("fileUpload")
element.send_keys("myfile.txt")
The above example is in Python but you get the idea

Using AWT Robots is one option, if you're using Java, which you are. But it's not a good option, it is not very dependable, and not clean at all. Look here
I use HttpClient and run a few tests outside of Selenium. That's more dependable and cleaner.
See the code below. You'll need more exception handling and conditionals to get it to suit your job.
HttpClient c = new HttpClient();
String url = "http://" + cargoHost + ":" + cargoPort + contextPath + "/j_security_check";
PostMethod post = new PostMethod(url);
post.setParameter("j_username", username);
post.setParameter("j_password", password);
c.executeMethod(post);
url = "http://" + cargoHost + ":" + cargoPort + contextPath + "/myurl.html";
MultipartPostMethod mPost = new MultipartPostMethod(url);
String fileNameWithPath = this.getClass().getClassLoader().getResource(filename).getPath();
File f1 = new File(fileNameWithPath);
mPost.addParameter(elementName, f1);
mPost.addParameter("action", "upload");
mPost.addParameter("ajax", "true");
c.executeMethod(mPost);
mPost.getResponseBodyAsString();

The suggestion of typing into the text box works only if the textbox is enabled.
Quite a few applications force you to go through the file system file browser for obvious reasons.
What do you do then?
I don't think the WebDriver mavens thought of just presenting keys into the KeyBoard buffer (this used to be a "no brainer" in earlier automation days)
===
After several days of little sleep, head banging and hair pulling I was able to get some of the Robot-based solution suggested here (and elsewhere).
The problem i encountered was that the dialog text box that was populated with the correct file path and name could not respond to the KeyPress/Release Events of terminating the file name with VK_ENTER as in:
private final static int Enter = KeyEvent.VK_ENTER;
keyboard.keyPress(Enter);
keyboard.keyRelease(Enter);
What happens is that the file path and file name are typed in correctly but the dialog remains opened - against my constant hoping and praying that the key emulation will terminate it and get processed by the app under testing.
Does anyone know how to get this robot to behave a bit better?

Just thought I'd provide an FYI to author's original post of using ActiveX. Another workaround would be to integrate with desktop GUI automation tools to do the job. For example, google "Selenium AutoIt". For a more cross-platform solution, consider tools like Sikuli over AutoIt.
This of course, is not considering WebDriver's support for uploads on IE & Firefox via SendKeys, or considering for other browsers where that method doesn't work.

After banging my head on this problem for far too many hours, I wanted to share with the community that Firefox 7.0.1 seems to have an issue with the FirefoxDriver sendKeys() implementation noted above (at least I couldn't get it to work on my Windows 7 x64 box), I haven't found a workaround, but updating to Firefox 8.0.1 seems to have fixed the problem. For those of you wondering, it's also possible to use Selenium RC to solve this problem (though you need to account for all of your target operating systems and the native key presses required to interact with their file selection dialogs). Hopefully the issues I had to work around save other people some time, in summary:
https://gist.github.com/1511360

If you have your are using a grid, you could make the folder of the testfiles open for sharing.
This way you could select the upload input field and set its value to \\pc-name\myTestFiles
If you're not, you should go with local files on each system.