I just want to quickly ask, I found all over the internet and even here on SO, how Selenium IDE can create Java source files from what you are doing in browser. But all these sources result in some Unit Test. For Java I believe JUnit and some other are supported by Selenium IDE.
But I want to ask, why? I mean, if you still need to compile them before executing, why are Unit Tests used instead of just running the code and look if WebDriver throwns any exception? What is the advantage of using for example JUnit here? I know its mostly used this way, I just don´t know why. Thanks.
Here's a couple of reasons off the top of my head:
1) You can hook your selenium tests into your build process (and hence your CI process).
2) You can use JUnit assertions.
3) You can build up multiple suites of JUnit tests (which can then be run in parallel).
I'm sure there's more but I guess it depends on the number of tests you have and the size of the project you are working on. If you're project already has a set of JUnit tests then it's quite nice to be able to write selenium tests without too much effort.
If you use Junit you can quickly recover from failures before starting a new test with the #before and #after annotations. You can also TearDown the tests with it. This also makes the tests more organized.
Not always you get an exception. Your application can handle an exception/user input/ etc. and navigate to different page then expected without throwing an exception - that can be easily verified by JUnit - assert expepected title of a page / presence of an element with actual values.
Related
I need to build a datadriven framework using selenium webdriver with java binding. i am bit confused with regards to choosing the right tool for building framework
Could somebody please recommend which framework (i.e. Junit, TestNG, Cucumber) works better with large set of test data
Also, please suggest which data file format i.e. .xls or .tsv to use for executing automated tests from Jenkins(deployed on Linux box)
As fare as I know and what I have earlier used Selenium for is to implement automated test scenarios of a web-application. Would typically be accept-test og end-to-end test (e2e).
These kind of test are what you call Black-Box testing. You give a input and expect a output, but does not know what is going on inside the black-box (you application).
To implement and execute your tests scenarios you would use a test-framework as you also mentioned (JUnit, TestNG, Cucumber etc.).
All test are able to be executed on a build server running Jenkins whether it is unit/integration or e2e-test (which could be implemented with Selenium).
Selenium test are expensive to maintain and often you would run into timing issues which causes test to fail even though the code does not contain any errors.
Therefore it is important to implement the test correct, otherwise you would run into a maintenance hell.
I am not sure if you are new to testing?? Maybe you could benefit from reading a bit about software testing.
The following is a good link: https://www.softwaretestingmaterial.com/software-testing/
Let me know, if I have misunderstood you question totally :)
I am really new to Fit/Fitnesse and, in general, to test automation.
I am trying to use them from Eclipse.
I have several question about it:
is there a way to obtain the html tables that Fitnesse pass to Fit?
once I write several tests with Fitnesse, is there a way to call them several times from Java without clicking on the Test button of the wiki?
About passing objects from one table to another in a flow. I read about symbols but it seems that, in java, they works only with ColumnFixturewhile I would like to use DoFixture. how to do this?
Finally,is if there is any plugin for eclipse you suggest to use with Fit/Fitnesse?
Regarding you question 2: I would recommend using the JUnit integration (#RunWith(FitNesseRunner.class) to run the test page (or a suite) as a unit test from Eclipse. This also gives you the ability to debug inside your fixture code.
It takes a bit of configuration to get it running 'just right'. In my pre-packaged FitNesse I provide a unit test FixtureDebug where you only have to enter the test name (and you can also use that to run your tests on a build/continuous integration server).
I'm writing some system tests in Groovy and piggybacking on its unit testing infrastructure. It works pretty well except that I don't like the default JUnit test runner, which displays . for each test in the suite waiting until the end to report details about the errors and failures. The system tests can take a long time to run so it's useful to be able to interrupt them in the middle once you know that a failure or error exists but you can only do that if you know which testcase failed and what the failure was while the suite is running.
Are there any alternate JUnit test runners which I could use that provide this functionality out of the box?
I'm not exactly sure what you mean by "textui", but IntelliJ IDEA 11 Community Edition has an integrated JUnit test runner that works with Groovy exactly as you describe (but it has a graphical user interface only, as far as I know).
Question
When I run all our JUnit tests, using eclipse, can I set a default timeout?
Background
My manager insists on writing Unit tests that sometimes take up to 5 minutes to complete. When I try to run our entire test suite (only about 300 tests) it can take over 30 minutes. I want to put something in place that will stop any test that takes longer than 10 seconds.
I know an individual test can be annotated with:
#Test(timeout=10000)
But doing this would make his long tests always fail. I want them to work when he runs them on his box (if I have to make minor adjustments to the project before checking it in, that's acceptable. However, deleting the timeouts from 40 different test files is not practical).
I also know I can create an ant task to set a default timeout for all tests, along the lines of:
<junit timeout="10000">
...
</junit>
The problem with that we typically run our tests from inside eclipse with Right Click > Run As > JUnit Test.
Summary
So is there a relatively painless way to set a timeout for all tests, perhaps using a Run Configuration setting, or project setting, or JUnit preference, or environment variable, or something? I'd even settle for installing some other plugin that lets me right click on particular test folders and run all the tests in some other manner like through ant or something...
Possible solution:
Extend all your Test classes from another class: TestBase for example
Add to TestBase global timeout. This timeout will be applied to all extended classes:
public class TestBase {
#Rule
public Timeout globalTimeout = new Timeout(10000);
}
So maybe a combination of using Infinitest with the "Slow test warning" enabled together with the filtering feature would do the trick. You could identify tests that exceed your time-limit and add them to the filter list, this would only affect testing from inside Eclipse. Running the tests via a possible build script via CLI/CI etc would not be affected at all.
You can find more on setting this up here: http://improvingworks.com/products/infinitest/infinitest-user-guide/
If you want to configure the tests to run for a maximum of ten seconds you can try this:
#Test(timeout=10000)
My manager insists on writing Unit tests that sometimes take up to 5 minutes to complete
This almost certainly indicates that those tests are not in fact unit tests. Cut that Gordian knot: try refactoring your testsuite to provide equivalent test coverage without requiring a test-case that runs for that long.
Almost certainly your bosses tests are system tests pretending to be unit tests. If they are suppsoed to be unit tests and are just slow they should be refactored to use mocks so that they run quicker.
Anyway, a more pragmatic and diplomatic approach than confronting your boss over this might be to just try and run the faster ones yourself. I've seen a hack to do this in a project where slow tests had SytemTest in their names. Then there were two ant targets created in the build file. One that ran all tests and one that filtered out by class name the SytemTests . To implement this all you would have to do is rename some of the tests and write your ant target.
It sounds like test suites would help you out.
You can have two test suites; QuickTests and AllTests. Include QuickTests in the AllTests suite, as well as the tests that take a long time. Then all other tests would go into the quick tests suite.
From eclipse you can run an entire test suite at once. So you would run QuickTests and that way all the other slow tests will not run.
Or see this question on how to apply a timeout to a suite which will apply to nested suites and classes in the suite. Which can achieve similar to what you want when combined with my above suggestion.
I know this doesnt really answer your question but the simple answer is don't!
Setting timeouts conditionally is wrong beacsue then you would have Unit tests on your machine that you are always going to fail. The point of Unit tests is to be able to quickly see that you havnt broken anything. Having to check through the failed test list to make sure it's just the long running tests is ust going to make some bugs pass through the cracks.
As some of the commenters mentioned you should split out the tests into unit tests that run quickly and the slower runnning integration tests i.e. have a source folder called src/main/java for your code, src/test/java for unit tests and src/integration-test/java for the longer running tests.
What are the possible ways to report or capture the result?
Or should we keep on staring at the screen when the automation script runs?
Thanks in advance!
You can get the test report using testng.
First install testng by following this link
After installation of testng, write some test cases in a class using #Test
annotation. Then click on run and run as a testng test. Then refresh your project and in test-output folder, you will find the testng report.
If you google, you will find a lots of links about this.
You can go through this link
Selenium (like other browser automation frameworks) is most commonly used to write automated acceptance tests within a test framework such as Cucumber-JVM. I mention Cucumber-JVM since it's designed for acceptance testing and is the most common such framework used with Java applications, but you could use JUnit or TestNG or another test framework if you wanted.
In any such framework, within each test, Selenium or another browser driver simulates user actions in the UI and statements written in the test framework's API/DSL assert that the expected result occurred. The test framework runs all of the tests and reports any which failed.
It is still often useful to watch the screen while debugging a failing test, however.
you can take screenshots of the screens, then you can create PDF's/doc's if you need. to capture screenshot you the command
screenShotImage = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
use screenShotImage.getAbsolutePath() to get file path.
Test results are always linked to project needs, i have seen scenarios where test results should be in PDF.
Here are some suggestions on test results
Extent reports - This is the java library contains graphs as well as
detailed steps. This even allows screenshots
If you are using TestNG or Junit, it has default reporting. However if you are looking for more advanced graphical report, use
RealTimeReport
if you link the results in documentation, create a custom report code