Selenium Tests take several minutes to start when loading a profile - java

I am just trying to figure out if anyone else has seen their Selenium tests run significantly slower (takes 2+ minutes to start) when they load a profile into the FirefoxDriver as shown in:
Selenium a default profile for the Firefox
The question originator of the above post mentioned this issue in a comment, but never updated whether he fixed this slowness issue.
At some point my tests stopped running all together and I started getting the error
org.openqa.selenium.WebDriverException: java.io.Exception: unexpected end of stream on Connection.
If I remove the profile option from the FirefoxDriver call then the test runs within 5 seconds of selecting "RUN" but the test fails because the default profile Selenium uses does not have the certificates I need to access my site.
Anyone else in the same boat or know how to fix this? How do you adjust how much information is saved within a profile?
Firefox Version: 60.3.0
Selenium Version: 3.14.0
GeckoDriver Version: 0.23.0
OS: Linux Redhat 6
Eclipse Version: Neon
Code:
WebDriver browser;
System.setProperty("webdriver.gecko.driver", "/path/to/geckodriver.exe");
ProfilesIni profile = new ProfilesIni();
FirefoxProfile ffprofile = profile.get("SeleniumUser");
FirefoxOptions options = new FirefoxOptions().setProfile(ffprofile);
browser = new FirefoxDriver(options); // takes a long time and eventually fails here
browser.get("site.url");
If you take out the {options} parameter from the new FirefoxDriver() call the test will start in about 5 seconds.
Keeping the options causes the error "org.openqa.selenium.WebDriverException: java.io.Exception: unexpected end of stream on Connection" as stated above.

When you initiate the process to load a new/existing FirefoxProfile through GeckoDriver the underlying framework consisting of:
The Driver (Selenium binding)
The Server (GeckoDriver)
The Client (Firefox Browser)
Needs to initialize and intercommunicate with different internal modules.
You can find a detailed discussion on how to access a FirefoxProfile through GeckoDriver with in Cannot resolve constructor FirefoxDriver(org.openqa.selenium.firefox.FirefoxProfile)
Additionally the saved:
Bookmarks
Password
User Preferences
are also loaded when an existing FirefoxProfile loads. Hence some added time is required.
You can find a detailed discussion in webdriver.FirefoxProfile(): Is it possible to use a profile without making a copy of it?

Related

Can I run Selenium using geckodriver without a webpage detecting marionette?

I want to use FirefoxDriver with Selenium but I keep getting detected by webpages. When I add the following code
System.setProperty("webdriver.gecko.driver", "../../../../../../../usr/bin/geckodriver");
FirefoxOptions opt = new FirefoxOptions();
opt.setCapability("marionette", false);
driver = new FirefoxDriver(opt);
The webpages cant't detect I'm using geckodriver but I can't use Selenium automation and that's my problem. I need the automation without detection.
QUESTIONS:
Can I change the setCapabilites on/off while the driver is running?
Is it easier to do this using ChromeDriver?
Using FirefoxDriver with Selenium but getting detected is quite common now as:
Selenium identifies itself
You can find a detailed discussion in How to make Selenium script undetectable using GeckoDriver and Firefox through Python?
Marionette
As per the documentation, Marionette, is the automation driver for Mozilla’s Gecko engine. It can remotely control the UI and the internal JavaScript of a Gecko platform, such as Mozilla Firefox. It can control both the chrome (i.e. menus and functions) or the content (the webpage loaded inside the browsing context), giving a high level of control and ability to replicate user actions. In addition to performing actions on the browser, Marionette can also read the properties and attributes of the DOM. Now, marionette shares much of the same API as Selenium/WebDriver, with additional commands to interact with Gecko’s chrome interface. Its goal is to replicate what Selenium does for web content, i.e. to enable the tester to have the ability to send commands to remotely control a user agent.
We have also discussed in details about Why Firefox requires GeckoDriver? within this thread
Finally, in the discussion Difference between webdriver.firefox.marionette & webdriver.gecko.driver we discussed about initializing Firefox sessions using legacy Firefox 47.x browsers and GeckoDriver enabled Firefox >47.x browsers. The conclusion was when using Firefox browsers > v77.x you have to mandatorily use GeckoDriver which extensively uses the marionette. So configuring marionette as false won't help us out. While using the latest version of geckodriver, selenium and firefox, you have to use the marionette by default.
If you still want to initialise a Firefox browsing session without using marionette you need to configure "marionette" to false as follows:
System.setProperty("webdriver.gecko.driver", "C://path//to//geckodriver.exe");
DesiredCapabilities dc = new DesiredCapabilities();
dc.setCapability("marionatte", false);
FirefoxOptions opt = new FirefoxOptions();
opt.merge(dc);
FirefoxDriver driver = new FirefoxDriver(opt);
driver.get("https://stackoverflow.com");
System.out.println("Application opened");
System.out.println("Page Title is : "+driver.getTitle());
driver.quit();
You can find a couple of relevant discussions in:
org.openqa.selenium.SessionNotCreatedException: Unable to find a matching set of capabilities while initiating Firefox v37 through Selenium v3.11.0
selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities with Firefox 46 through Selenium
How can Geckodriver/Firefox work without Marionette? (running python selenium 3 against FF 53)
The other questions:
Can I change the setCapabilites on/off while the driver is running?: The short answer is No, you can't change the capabilites while the session initiated by the webdriver is In Progress and you can find a couple of detailed discussions in:
Change ChromeOptions in an existing webdriver
Set capability on already running selenium webdriver
Is it easier to do this using ChromeDriver?: Again the precise answer is No, ChromeDriver also gets detected and you can find a couple of detailed discussions in:
Is there a version of selenium that is not detectable ? can selenium be truly undetectable?
Webpage Is Detecting Selenium Webdriver with Chromedriver as a bot
Outro
Here you can find a detailed discussion on Which Firefox browser versions supported for given Geckodriver version?

Selenium FirefoxDriver initialization

When I am trying to load the Selenium FireFoxDriver, process is taking more time to invoke the browser. I have tried for lot of options like disabling the auto updates and etc. But all efforts are in vain. So I have decided to load the FirefoxDriver on startup of the Server (i.e, with load-on-startup time as 0 in web.xml when container initializes). Are there any cons with this approach? Also please suggest if there is any better way to do this.
Thanks in advance!!!
In general, WebDriver supports Firfox and Safari browsers by default. There is no need to perform any configuration for this. Please find below for the sample code which initializes the browser
WebDriver driver = new FirefoxDriver();
driver.get("http://www.google.com/");
The above code will start the firefox browser installed in your machine and launches the google page. If there is a mismatch between the selenium version of firefox version the browser will be started, but would not launch the google application.
Hope this helps.

Can't upgrade Firefox with selenium

Temporarily I upgrade my sofware for automation web test.
I had Selenium 2.41 and Firefox27 working fine. When I try to upgrade the software, there is no problem about Selenium (I can upgrade to the latest release 2.45 and it's working fine with Firefox27) but I can't upgrade to any Firefox version higher than 29...and this is a problem for us.
I tried to run my test but I get this exception at some point:
"Failed to start new browser session, shutdown browser and clear all session data
org.openqa.selenium.server.RemoteCommandException: timed out waiting for window 'null' to appear"
Any idea?
Best regards and thanks
Disclaimer: I'm using Selenium's .NET bindings, not Java. I don't think that will make a difference in this case.
Selenium 2.44 had an issue with Firefox 36, but this was resolved in Selenium 2.45. It's possible that Firefox versions between 29 and 35 might be incompatible with Selenium 2.45, although this is just a guess on my part.
I had an issue with Seleinum 2.45 driving Firefox version 38: when instantiating the Firefox driver with no profile, an instance of Firefox would load and immediately crash to desktop; subsequently another instance would load as normal.
I found that the issue didn't occur when instantiating the Selenium Firefox driver from an existing profile, so my workaround was to create a blank profile, launch Selenium's Firefox driver using a temporary copy of that profile; then at the conclusion of the test, delete the temporary copy.
Try launching the Firefox driver with an existing profile.

How can a remote Selenium 2 Webdriver instance accept untrusted certificates?

I am trying to implement some Selenium 2 Webdriver tests with JUnit. The documentation on SeleniumHQ.org and the web is confusing to me because it seems to jump back and forth between Selenium RC and Webdriver. Plus, my Java is not very strong. I've took a few courses years ago, but haven't used it much. I want to have JUnit tests run from a headless CI server, and have Firefox run on a remote client system by using Webdriver.
From what I've gathered, I can use the following code to open a Webdriver-controlled instance of Firefox on my local system. The web site I'm testing has an untrusted SSL/TLS certificate, so I need to tell the Firefox driver to accept untrusted certificates. This works great locally:
FirefoxProfile profile = new FirefoxProfile();
profile.setAcceptUntrustedCertificates(true); // NOTE: this is the default behavior
RemoteWebDriver driver = new FirefoxDriver(profile);
Selenium selenium = new WebDriverBackedSelenium(driver, baseurl);
But I can't figure out how to do this on the remote system using Webdriver. The two approaches seem totally incompatible. The code above does not fit in any way into the following code that I have been using for using Webdriver remotely:
Selenium selenium = new DefaultSelenium(host, port, browser, baseurl);
selenium.start();
Now, I have spent many hours working with custom Firefox profiles on the remote test system. It worked in the summer of 2012, but after recent OS and browser updates, it stopped working. It seems much better to create the Firefox driver profile and call setAcceptUntrustedCertificates(true). Is it possible to use Webdriver to run tests in a browser on a remote system and also have the browser ignore untrusted SSL/TLS certificates?
As mentioned in your question, you don't need to set any property for accepting untrusted certificates explicitly. By default webdriver accepts untrusted certificates. Rather than using a webdriverbacked selenium, you should use the remotewebdriver directly like:
Webdriver wd = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), DesiredCapabilities.firefox());
Here http://localhost:4444/wd/hub is the URL of the Hub to which tests should be send for execution. When you start the tests, hub will look for remote nodes that have registered with firefox capability.
Personally I would suggest to read documentation at http://code.google.com/p/selenium/wiki/Grid2 rather than seleniumhq.org. As far as I know, selenium team is trying to get the seleniumhq documentation updated. You can also contribute to it :)
First of all i will recommend sticking to webdriver if you are using webdriver backed selenium just for profile. You can define profile to be used on your local machine as
File file = new File("firebug-1.8.1.xpi");
FirefoxProfile firefoxProfile = new FirefoxProfile();
firefoxProfile.addExtension(file);
firefoxProfile.setPreference("extensions.firebug.currentVersion", "1.8.1");
WebDriver driver = new FirefoxDriver(firefoxProfile);
To Answer Your Question: I will quote Simon Stewart's solution from here:
FirefoxProfile profile = new FirefoxProfile();
profile.setAcceptUntrustedCertificates(true);
DesiredCapabilities caps = DesiredCapabilities.firefox();
caps.setCapability(FirefoxDriver.PROFILE, profile);
Use this profile to create the remote driver.
Now if this doesn't work than may be we can write-up a bug (or at least a feature request).
post edit: I can not really test this solution as I dont have a cert issue site readily available. So in a way I would be looking towards you feedback to get the real picture... :)
When I asked this question, I didn't understand the distinction between Selenium objects and WebDriver objects. Even though I was specifically trying to learn about Selenium 2's "WebDriver" feature, I foolishly thought that I could write a "Selenium 2 Webdriver" project with Selenium 2 objects. That may sound obvious to someone who has experience with these tools, but that distinction was still not clear in my mind after reading "Selenium 2" books and project documentation.
As I result, I was writing Java code to instantiate a Selenium object to examine a web page, and trying to pass the Selenium object a WebDriver object, in the hope that the test would run on a remote server.
Now it seems clearer: The Selenium and WebDriver projects merged into a new umbrella project named (confusingly) Selenium 2.0, but they are distinct and separate tools within Selenium 2. If I want to use the WebDriver API, it seems that I must convert any existing Selenium objects to WebDriver objects. There appears to be no useful interaction between the two tools.
For example, in my project I had the following code. It ran great on my local desktop system's web browser:
Selenium selenium = new DefaultSelenium(host, port, browser, baseurl);
selenium.get(urlPath);
selenium.type(username_field, username);
selenium.type(password_field, password);
selenium.click(login_button);
But I want to be able to run that test on a headless continuous integration server, not my desktop system. I have converted the code to use a WebDriver object instead of a Selenium object. Now it runs on a remote system connected to a Selenium Grid 2 server:
WebDriver driver = new RemoteWebDriver(new URL("http://10.0.0.29:4444/wd/hub"), capability);
driver.get(urlPath);
driver.findElement(By.name(username_field)).sendKeys(username);
driver.findElement(By.name(password_field)).sendKeys(password);
driver.findElement(By.className(login_button)).submit();
I hope other people wanting to learn how to use WebDriver in Selenium 2 will not waste as much time as I did reading about Selenium objects, thinking that the Selenium object is part of WebDriver. My current [n00b] advice is to ignore anything that mentions Selenium objects, and focus purely on finding out as much as you can about WebDriver objects. A good place to start is the WebDriver documentation at SeleniumHQ.org:
http://seleniumhq.org/docs/03_webdriver.jsp#webdriver-and-the-selenium-server
As A.J. suggested in his answer, also take a look at the Selenium Grid documentation:
http://code.google.com/p/selenium/wiki/Grid2
And PS: a remote Selenium 2 Webdriver instance accepts untrusted SSL/TLS certificates automatically, by default. No code required.

How do I rerun Selenium 2.0 (webdriver) tests on the same browser?

I am trying to use Selenium 2.0 (Webdriver) to implement a series of tests. Before these tests can run, I have to login into the application. Since the application is not my 'own' (testing api-built functionality), each test should not be logging into my application to run.
I would prefer to do the following:
Connect my webdriver tests to my open Firefox browser (already loggedin)
Run my webdriver projects with the same browser.
I understand that Selenium usually assigns a session id to its browsers. However, the current Java implementation of Selenium 2.0 driver does not make use of session id (maybe it does but I don't know where to find it. )
Can someone provide some direction on how to resolve my issue (existing browser and run multiple tests with Selenium 2.0 (java))? Any code provided would also be helpful. Thanks!
Here is what I have learnt:
Selenium 1: As Ioan suggested earlier, use "-firefoxProfileTemplate" when starting up the Selenium RC server and point to the location of your Firefox profile.
Selenium 2: I suppose you can use the Selenium 1 RC server, however, since Selenium 2 uses WebDriver, you can point to the profile information within your code.
File profileDir = new File("/Users/_____/selenium/FFprofile");
FirefoxProfile profile =
new FirefoxProfile(profileDir);
WebDriver driver = new FirefoxDriver(
profile);
Notes:
Make sure you run "firefox -profilemanager" to create your initial profile and save your login information.
Allow the browser/website to always store your authentication credentials avoiding "popup"/"login" wwindows, etcs.
Hope this helps somebody who may run into a similar issue: Using the same browser profile in Selenium, etc.

Categories