Alternate JUnit textui TestRunner - java

I'm writing some system tests in Groovy and piggybacking on its unit testing infrastructure. It works pretty well except that I don't like the default JUnit test runner, which displays . for each test in the suite waiting until the end to report details about the errors and failures. The system tests can take a long time to run so it's useful to be able to interrupt them in the middle once you know that a failure or error exists but you can only do that if you know which testcase failed and what the failure was while the suite is running.
Are there any alternate JUnit test runners which I could use that provide this functionality out of the box?

I'm not exactly sure what you mean by "textui", but IntelliJ IDEA 11 Community Edition has an integrated JUnit test runner that works with Groovy exactly as you describe (but it has a graphical user interface only, as far as I know).

Related

application code coverage after running UI selenium functional tests

I am trying to get java source code coverage after testing with a GUI based functional test using selenium.
what all source (actual application) code( NOT selenium test code) was executed or covered after running the selenium functional tests in java.
Is this can be achieved using Emma or jacoco or any other?
Anyone who has done this could please give me an idea to start this ?
Our Java Test Coverage tool can do this.
The process is straightfoward:
Apply the Test Coverage tool instrumenter to your source code and compile.
Run the application, exercising it any way you desire (including running selenium, or hand interaction, or unit tests)
Cause the application to exit from main.
The instrumenter's code inserted in main will write coverage data
Display/explore the test coverage with the tools GUI interface
You may have a long-running (server) application that doesn't naturally exit; in that case you can't get the test coverage dumped by exiting main because you didn't exit. In this case, you add a call to the tool's DumpTestCoverage() method, at some place in your application that you can cause to be executed by some interaction with the application. Then when you want a snapshot, simply exercise that feature.
I always use Jacoco plugin with Gradle. It cool in a way that it generates report and makes everything simple. You can configure it in build.gradle file.

How to find test that affects execution of other tests?

I'm currently struggling with a pretty hard problem: I'm working on a project that has around 8 thousands of unit tests (that take 15 minutes to execute on pretty strong machine) and test that are currently failing don't fail when run on they or own (or when run with other tests that failed), so I guess there is some test that passes but leaves some mess behind.
I'm currently trying to run those tests with tests from specific packages, using gradle:
test {
filter {
includeTestsMatching 'some.package.*'
includeTestsMatching '*Test1'
includeTestsMatching '*Test2'
}
}
However there are some things I don't know how to control, like execution order of test classes (if someone has an idea how to change order that would also help me).
Perhaps someone already knows some nice process of finding tests affecting other tests?
Assuming junit tests, then:
define a test suite that specifies the ordering of test classes
ensure you are using junit 4.11 or later to get a deterministic method order
run only the test suite from gradle (or just directly from your ide)
adjust the ordering by editing the suite until you reproduce the problem

Code coverage for Fitnesse

I have just inherited a old java codebase (around 10 - 15 years old). It does not have any automated testing coverage, or at least the contemporary world knows about it. I am planning to write some Fitnesse scripts around it, to begin with.
I know about Concordion etc. And I have my reason to pick Fitnesse. I will keep away from that since that is not the topic of this question.
My question is, I don't know of a quick way to measure the code coverage done by the Fitnesse tests as I write them. I know that jacoco (or similar libraries) should be able to report them but I just can't figure out exactly how.
So, if anyone of you have worked on Fitnesse test scripts and have managed to have Jenkins report on the coverage achieved by the scripts, please help.
Thanks.
I have not done this myself, because I tend to use FitNesse to test deployed applications (and I cannot measure code coverage in the 'real installation'). But I believe this should be fairly straightforward if you run your FitNesse tests as part of a Jenkins (or any other build server's) jUnit run, which measures code coverage.
To have your FitNesse tests be executed as part of a jUnit run: create a Java class annotated with #RunWith(FitNesseRunner.class) and give it a #Suite("MyPageOrSuite.That.IWantToRun") annotation to indicate which tests to run. This will execute the specified page(s) in the same Java process that the jUnit process is using, so if that is instrumented in some way to determine code coverage the coverage of your FitNesse tests will be included in the report.
Sample jUnit test class, running FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestScriptTable:
#RunWith(FitNesseRunner.class)
#FitNesseRunner.Suite("FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestScriptTable")
#FitNesseRunner.FitnesseDir(".")
#FitNesseRunner.OutputDir("../target/fitnesse-results")
public class FitNesseRunnerTest {
}

Junit Tests returning as Unrooted Tests

I've similar questions asked on this site, but this is a bit of a different scenario than what I have seen.
We have a PC client execute JUnit 4 tests. However, we have a custom test runner that ships the JUnit 4 tests as JUnit 3 tests (using JUnit38ClassRunner) over ethernet to a target system that is running a service that executes JUnit tests using JUnit 3.8. The tests execute as intended, however, when they are returned to the PC client they are marked as Unrooted Tests. Is there a way to organize these tests as "non-unrooted tests"? It is somewhat difficult to sift through the failed results when they are all returned in one group when not using Eclipse. Using JUnit 4 is not an option on the remote system as the target embedded system uses Java 1.4.2, and this is not changing anytime in the near future. We really do not want to have to downgrade to JUnit 3.8 on the PC client side because of the #RunWith annotation, which will take us a little while to figure out how to re-implement.
Any assistance on this is appreciated, thanks in advance.
I wasn't able to find a solution for this in Eclipse, but using Ant to manage the junit execution generates xml reports, and I can view the summary in a web page by compiling those reports using the <junitreport> element.

Selenium Unit Test using JUnit (Java)

I just want to quickly ask, I found all over the internet and even here on SO, how Selenium IDE can create Java source files from what you are doing in browser. But all these sources result in some Unit Test. For Java I believe JUnit and some other are supported by Selenium IDE.
But I want to ask, why? I mean, if you still need to compile them before executing, why are Unit Tests used instead of just running the code and look if WebDriver throwns any exception? What is the advantage of using for example JUnit here? I know its mostly used this way, I just donĀ“t know why. Thanks.
Here's a couple of reasons off the top of my head:
1) You can hook your selenium tests into your build process (and hence your CI process).
2) You can use JUnit assertions.
3) You can build up multiple suites of JUnit tests (which can then be run in parallel).
I'm sure there's more but I guess it depends on the number of tests you have and the size of the project you are working on. If you're project already has a set of JUnit tests then it's quite nice to be able to write selenium tests without too much effort.
If you use Junit you can quickly recover from failures before starting a new test with the #before and #after annotations. You can also TearDown the tests with it. This also makes the tests more organized.
Not always you get an exception. Your application can handle an exception/user input/ etc. and navigate to different page then expected without throwing an exception - that can be easily verified by JUnit - assert expepected title of a page / presence of an element with actual values.

Categories