We use use junit for unit testing our java code. Today we use cobertura to get coverage numbers. It does not have an easy way of getting per test coverage number. Is there a tool to get per test code coverage - commercial/free?
(cobertura has a patch to get per test coverage numbers, out of date with latest cobertura).
we used clover to good effect. we wrote some ant tasks that allowed us to run it from a dev box, so we could view the coverage numbers locally, and we also integrated it into our continuos integration so we had a site for the official number.
http://www.atlassian.com/software/clover/
the only issue we had was it is a memory hog....
Emma provides detailed reports by overall/package/class for block and line coverage.
The obvious way to do this is, run one test and dump the test coverage data. (In fact, this is the only way to do this).
Our SD Java Test Coverage Tool has explicit DumpVectors and ResetVectors procedures that can be called anytime. By adjusting the unit test framework to just call these two procedures between tests, you can get one test coverage vector per unit test.
The display tool will display any individual test coverage vector. It can also give you the union of the entire set (as if you had run all the tests) or compute how one test overlaps with another.
Related
We want to teach student to make test with JUnit.
But we have a lot of student, so we want a program which automatically verify if they test a series of tests.
For example, if we have a program which add two Integer, we want to check if they test normal case, with String ...
Do you have any idea how I can do that ?
The closest thing that comes to mind is to use Jacoco to check the code coverage. You can either:
run it as part of an IDE like eclipse (e.g. Using ECLEmma plugin) or
programmatically
In both cases you can just navigate the report to check complete coverage or coverage per class.
Alternatively you can programmatically check the coverage percentages and pass just the ones above a certain %.
It goes without saying that high coverage doesn't necessarily mean that tests are well written.
.
I am currently dealing with a legacy Java Maven project that I need to alter. I have ran a test coverage report with cobertura and can see that the logic I have to change is currently covered by a test. Given that the code base is tens of thousands of classes in size, that the complexity of many of the classes and tests is very large, and that there they are not testing their code unit by unit opting for sort of semi-integration tests...I will have no luck manually tracking down which tests are providing this coverage.
What I'd ideally have is some way to find out that line x of source class y is covered by method t of test class r.
Is there a way with cobertura or some other tool (hopefully a Maven plugin) that I can view the tests which actually cover each LoC?
Put a breakpoint in your method, execute your test in debug mode, and then look at the call stack when the VM suspends.
I couldnt find this info anywhere. In order to get code coverage calculated using a plugin (like Jococo, Cobertura..etc) Do I need to run all the unit tests before? These look like relevant tasks, But still I think Code coverage should not be dependent on running unit tests before hand, unless coverage plugin really relies on the Junit
You do not need to run tests beforehand. The coverage tool instruments the code (if required), runs the tests (or your main) and then reports the stats back to you.
Having said that, if your code relies of fancy reflection/bytecode manipulation, it may be a good idea to run tests beforehand, just to make sure that failures reported during the coverage scan the the instrumentation's fault and not "real" test failures.
Right now I use Jacoco to know the coverage of the whole test suite, but I would need to break down the coverage information by test case.
I need to create a program that does this since we need to automate a large suite. I guess this should be done by using the API...
Is there a way of doing this? I'm new to Jacoco, please help.
The TestNG plug in for Eclipse allows you to select and run an individual test case. You can then inspect the result of running that particular test case.
However, there does not seem to be a way to, for instance, click on a single line of code and query what test cases visited that line.
What other use cases did you have in mind?
Is there a tool for Java which, given a set of JUnit tests, and a class to test, will tell you which lines of the class are tested by the tests? ie. required to be present for the tests to run successfully. I don't mean "code coverage", which only tells you whether a line is executed, but something stronger than that: Is the line required for the test to pass?
I often comment out a line of code and run a test to see if the test really is testing that line of code. I reckon this could be done automatically by a semi-smart tool (eg. something like an IDE that can work out what can be removed from a method whilst keeping it compilable).
There's an open source mutation-testing tool called Jester that changes the lines of your source code, then runs your tests, and reports whether your tests passed anyway. Sounds closer to what you're looking for than code coverage tools.
Jester is a test tester for testing your java JUnit tests (Pester is for Python PyUnit tests). It modifies your source code, runs the tests and reports if the tests pass despite the changes to the code. This can indicate missing tests or redundant code.
WRT the discussion about whether these tools are needed in a pure TDD project, there is a link on the Jester project webpage to a posting about the benefits of using Jester on code written during a TDD session (Uncle Bob's infamous bowling TDD example).
What you are looking for might be referred to as mutation testing. While mutation testing won't tell you which lines of code are required to pass, per se. What mutation testing does is modify your source code looking for changes it can make to your code but your test still passes. E.g. changing
if (a < b)
to
if (a >= b)
and seeing if the test still passes. This will highlight weaknesses in your test.
Another java library for mutation testing is jumble.
I use emma for most of my projects. i included it in my ant build file and it generates html files for the reports
two other coverage projects i read about but haven't tried yet are clover or cobertura
I love cobertura, because the generated reports are IMHO the most beautiful. And it has its own ant target!
In comparison to emma, it has also branch coverage, not only line coverage, which is misleading very often.