For integration testing I want to use selenium, also for my TDD/BDD workflow but also for my remote build server to run through the tests etc.
What components do I need for this?
I know there is a browser add-in, but that is for helping to bootstrap the code (you can export the code, then tweak as necessary).
I find it slow to run the tests locally as it fires up the browser, is there a way to do this w/o the browser opening?
How does this work on the build server, is there a headless mode?
Can someone outline the various selenium tools and how they fit together and where they are used in the development cycle.
e.g. I see selenium drivers, selenium rc, browser add-on etc.
I want to use this for both rails and java.
I never tried Selenium with Java, only with Ruby, but your interaction should be similar. I don't think Selenium has a headless mode, you would have to use a different driver for that. For ruby, there is capybara-webkit, for example, which runs a headless webkit based browser.
I can tell you from my own experience, we had a lot of problems on CI (build server) with the headless browser so we ended up switching back to Selenium and Google Chrome running on top of Xvfb, which is kind of a virtual screen manager for unix that lets you run graphical apps in a headless environment. Most of the problems are related to inconsistent behavior between the webkit headless browser and an actual real driver. However, depending on how your app looks like and how you're going to drive it, it might work for you, you should give it a try.
So the only components you need are the Selenium server (that driver the browser) and the client, that sends the command to the server. They have java and ruby clients, so you can choose whichever language you prefer.
It is indeed a slow process and very flaky as well, as your tests might fail for no reason sometimes due to timeouts, 3rd party APIs that are down, etc. There are tons of things that can go wrong and cause your integration tests to fail.
Anyways it is still an invaluable tool and you should definitely use it. Just be aware that you should not rely entirely only on these kinds of tests to assure the quality of your app, however, these should only test a smaller surface of your system. Unit and component tests should make up the majority of your test suite.
Hope this helps clarify your questions.
I have evaluated Selenium somewhat myself and as I understand it you should use the Firefox add-on to record your tests. Then export to whatever code you want. Then, when the time comes and you want to test your system, boot the Selenium RC which acts as a server that you can then use to run your tests. The Selenium WebDrivers are part of this and allow the server to open and "drive" the browsers using your recorded tests.
Note that recording tests in browsers other to Firefox is more difficult as there is no handy add-on for use. Instead you have to build your own tests from scratch, which is time consuming.
app.test from Fabasoft (http://www.apptest.com/en/) is another free alternative that does the same job but in a different way, maybe take a look at that before you get too involved with Selenium.
Related
I've been asked a question by one of our developers here. They are asking whether it is possible for a selenium test suite to be tied in with a TFS build in visual studio to the test server? So that way, as soon as a build is done it can kick off a run of the regression test suite, directly after that.
Bear in mind that my scripts have been written in Eclipse, NOT Visual Studio, so I'm not sure if this will cause restrictions.
You can call UI scripts as you describe and I would additionaly recommend that you use Release Management to do this rather than build. It makes much more sense to use a deployment engine rather than a compilation engine to maintain this. One does not usually have an instance of ones application running on the build server.
http://nakedalm.com/execute-tests-release-management-visual-studio-2013/
You need to get a few things lined up, but it worked pretty good..
I have written a program in Ruby that checks a web page every so often, and performs some actions if data is found. I have written this using Watir-Webdriver.
I also want this program to run as a "daemon". The problem I had with Watir-Webdriver was that I could not run it without a browser popping up everytime.
I actually need to write this in Java? Which is best Java package thats similar in functionality to Watir-Webdriver?
Is there any way to implement this in Java - to check a webpage and do specific actions - all in the background?
Also, I would like this to be cross-platform.
Thank you in advance to help
You are looking for Selenium WebDriver. It can be written in Java and many other languages.
Yes. Webdriver can run different types of browsers. You are looking for what we call a headless driver. The most popular headless drivers are HtmlUnitDriver or GhostDriver. I personally prefer GhostDriver, as it runs a more modern version of Javascript.
All Java code can be run on different platforms, so that shouldn't be a problem.
I'm writing an application in groovy with grails that needs to do some automated usage of a couple of websites. So I need something that will perform a similar task to functional web application testing, without a browser being needed. I would also like it to be as lightweight as possible and of course completely headless.
There are two options that I am aware of so far. Canoo which has a nice grails plugin and conveniently also a firefox recorder plugin and EasyB/JWebUnit which can be recorded with Selenium.
Canoo seems very heavy and I don't know what I'd have to do to get it to run in a grails service neatly.
Perhaps I'm thinking about this wrongly though. Has anyone here done this kind of thing in Java or Groovy? Am I better off just dropping out to curl on the command line perhaps?
Have you though about just sending your web requests? Maybe you don't need a full blown headless browser and a line like
def html = new Url("http://example.com").text
is enough for your app?
A testing tool or framework is great when you have to click virtual buttons. But if you only have to remote control a website, sending a request could be enough.
Update: if you need to send POST requests and handle cookies, then the Url-Class isn't enough. But you can use the http builder instead. Here are two URLs which will get you started:
http://blog.swwomm.com/2011/01/groovy-httpbuilder-cookies.html
http://groovy.codehaus.org/modules/http-builder/doc/post.html
I'm trying to find a practical unit testing framework for JSF.
I know about JSFUnit, but this is very impractical to me. I need to include about 10 JARs to my project, and jump through many other hoops just to get it running.
I realize that -- due to the need to simulate a platform and a client -- unit testing web applications is difficult. But is there a better way?
Have you thought about doing integration testing with Selenium or another tool? Selenium allows you to record and run tests directly in the browser. You can also run tests in multiple browsers and on multiple platforms with Selenium Remote Control.
Writing unit tests is good, but it might provide more to create some functional integration tests rather than unit-testing the presentation layer code.
On the project I'm working on at the moment we dabbled with using selenium. We actually spent a lot of time writing these selenium tests, but found that they added little value because the UI changes so much and you just end up doubling your effort for very little return on investment.
Another problem with selenium is that it requires your code to be deployed, which means it doesn't play well with unit test frameworks eg maven.
What I would say is that writing really good unit tests for your managed beans is invaluable.
Have you taken a look at the jsfunitwar Ant task or alternatively the Maven plugin provided by JSFUnit? Both greatly reduce the complexity of generating the .war file to be tested. I'm using JSFUnit on my current project and find the combination of white box and black box testing capabilities to be very powerful. Because JSFUnit uses HtmlUnit under the covers, you can very easily and effectively examine the generated HTML, or conversely, verify the state of your internal JSF backing beans. I was able to incorporate the JSFUnit tests into my Continuous Integration process and have been quite pleased with the outcome.
HttpUnit can also be an alternative. It provides apis so you have a choice to automate the tests.
http://httpunit.sourceforge.net/index.html
Selenium is superficial, jsfunit is inward. I recommend that use jsfunit if project is not simple. Because team member can change jsf managedbean names or etc, you can catch that with jsfunit.
I'm with Paul on Selenium being very easy to setup and start working with. I use Selenium IDE in Firefox with some customization at that level, then you can export these to other platforms such as Java JUnit tests. It was quite easy to download and launch the selenium-server.jar, add the selenium-java-client-driver to my existing Eclipse Maven POM driver project; then launch the same exported JUnit test in Eclipse. I mainly wanted to use Java just for looping which the basic Selenium IDE didn't support.
I have configured JSF Unit for my project too which does require more time to configure... more importantly though with in-container tests like JSFUnit changes to the test require rebuilding the WAR, redeploying in the container and then executing from Eclipse or via a browser. So for quickly trying a small change this is time consuming. Of course with JSFUnit you have access to all the internals of the JSFSession etc so it depends what granularity of testing you need I guess.
I'd be interested if anybody knows a faster way to turnaround changes to a JSFUnit test and execute it. Definitely Selenium tests feel more like JUnit tests in that regard.
What's the best, crossplatform way to perform blackbox tests on AJAX web applications?
Ideally, the solution should have the following attributes:
Able to integrate into a continuous integration build loop
Cross platform so I you can run it on Windows laptops and Linux continuous integration servers
Easy way to script the interactions
Free-as-in-freedom so you can adapt it into your tool chain if necessary
I've looked into HttpUnit but I'm not conviced it can handle AJAX-heavy websites.
Selenium might be what you're looking for: http://selenium.openqa.org/
It allows you to script actions and evaluate the results. It's open-source (Apache 2.0), cross platform, and has nice tools.
I have used Selenium for exactly this task, but found it to be brittle.
Check out this talk by two Googlers: Does my button look big in this? Building testable AJAX applications
They isolate the testable javascript (non DOM-interaction) and test that using the Rhino javascript engine.