We want to teach student to make test with JUnit.
But we have a lot of student, so we want a program which automatically verify if they test a series of tests.
For example, if we have a program which add two Integer, we want to check if they test normal case, with String ...
Do you have any idea how I can do that ?
The closest thing that comes to mind is to use Jacoco to check the code coverage. You can either:
run it as part of an IDE like eclipse (e.g. Using ECLEmma plugin) or
programmatically
In both cases you can just navigate the report to check complete coverage or coverage per class.
Alternatively you can programmatically check the coverage percentages and pass just the ones above a certain %.
It goes without saying that high coverage doesn't necessarily mean that tests are well written.
.
Related
I'm using Junit with Eclipse. I have created test cases for few of the methods in my project. Is there any way/plugin to find the list of classes for which the test cases have been created(so that I need not manually check whether test cases created for all methods)?
You should look into using Code coverage tools, that are designed to do exactly this by telling you for instance:
which lines are covered by your tests
which branches of loops, ifs, switches, etc. are covered by your tests
More unit plug-in decorates all classes for which test classes are written.
Check here.
Using code coverage tools:
If used code coverage tools, then user has to run this(Ex:Eclemma) tool again and again to cover all use cases and merge the results of each run to check the method coverage.This takes more time.
Even this procedure is automated then also user has to open all source files to check whether the method is covered or not. See coverage view.
If the method is marked as 100% coverage that doesn’t mean that test case was written for that method separately. Because other method may called this method internally.
So using code coverage tool it is not possible to decide whether test cases were written for all the methods of all the classes.
Right now I use Jacoco to know the coverage of the whole test suite, but I would need to break down the coverage information by test case.
I need to create a program that does this since we need to automate a large suite. I guess this should be done by using the API...
Is there a way of doing this? I'm new to Jacoco, please help.
The TestNG plug in for Eclipse allows you to select and run an individual test case. You can then inspect the result of running that particular test case.
However, there does not seem to be a way to, for instance, click on a single line of code and query what test cases visited that line.
What other use cases did you have in mind?
Is there a program out there that can allow me to find all ignored junits?
By this I mean, I have seen unit tests that use the #Ignore and tests with method name like ignore_testFoo() or xtestBar() or xxtestBar1(), which all get ignored and they are very hard to find sometimes.
I could grep for those cases, but I was wondering if there was an application that would find any of those situations automatically.
I tried using cobertura to obtain coverage on junits, to see which methods were being executed and which were not being executed, and picking apart the bad unit tests that was.
I was just wondering if there was a program or another method to obtain this information without hacking something up.
A static analysis tool would serve you well here. Checkstyle is a decent choice amongst them, it has a long list of modules, and worst case you can easily write your own module to validate any coding convention you need.
You would locate or create a module for it then execute to find any non-conforming code.
Edit
PMD looks to be an excellent choice to handle this task. It actually comes with a set of JUnit rules already built in and its very easy to combine rules or create new ones.
It should be easy to detect ignored tests using junit3 by a grep on your java test files. Find all lines matching test and parenthesis but with a method name that doesn't start by test.
For junit4, you could
* implement your own test runner by extending the default one, print out ignored tests
* build a small app that loads test classes, get all declared methods through introspect, print out those markedas ignored.
There may be a tool to do that, maybe even some runners already do, but actually it could take a few hours to have those tools from scratch if you really need them.
Is there a tool for Java which, given a set of JUnit tests, and a class to test, will tell you which lines of the class are tested by the tests? ie. required to be present for the tests to run successfully. I don't mean "code coverage", which only tells you whether a line is executed, but something stronger than that: Is the line required for the test to pass?
I often comment out a line of code and run a test to see if the test really is testing that line of code. I reckon this could be done automatically by a semi-smart tool (eg. something like an IDE that can work out what can be removed from a method whilst keeping it compilable).
There's an open source mutation-testing tool called Jester that changes the lines of your source code, then runs your tests, and reports whether your tests passed anyway. Sounds closer to what you're looking for than code coverage tools.
Jester is a test tester for testing your java JUnit tests (Pester is for Python PyUnit tests). It modifies your source code, runs the tests and reports if the tests pass despite the changes to the code. This can indicate missing tests or redundant code.
WRT the discussion about whether these tools are needed in a pure TDD project, there is a link on the Jester project webpage to a posting about the benefits of using Jester on code written during a TDD session (Uncle Bob's infamous bowling TDD example).
What you are looking for might be referred to as mutation testing. While mutation testing won't tell you which lines of code are required to pass, per se. What mutation testing does is modify your source code looking for changes it can make to your code but your test still passes. E.g. changing
if (a < b)
to
if (a >= b)
and seeing if the test still passes. This will highlight weaknesses in your test.
Another java library for mutation testing is jumble.
I use emma for most of my projects. i included it in my ant build file and it generates html files for the reports
two other coverage projects i read about but haven't tried yet are clover or cobertura
I love cobertura, because the generated reports are IMHO the most beautiful. And it has its own ant target!
In comparison to emma, it has also branch coverage, not only line coverage, which is misleading very often.
We use use junit for unit testing our java code. Today we use cobertura to get coverage numbers. It does not have an easy way of getting per test coverage number. Is there a tool to get per test code coverage - commercial/free?
(cobertura has a patch to get per test coverage numbers, out of date with latest cobertura).
we used clover to good effect. we wrote some ant tasks that allowed us to run it from a dev box, so we could view the coverage numbers locally, and we also integrated it into our continuos integration so we had a site for the official number.
http://www.atlassian.com/software/clover/
the only issue we had was it is a memory hog....
Emma provides detailed reports by overall/package/class for block and line coverage.
The obvious way to do this is, run one test and dump the test coverage data. (In fact, this is the only way to do this).
Our SD Java Test Coverage Tool has explicit DumpVectors and ResetVectors procedures that can be called anytime. By adjusting the unit test framework to just call these two procedures between tests, you can get one test coverage vector per unit test.
The display tool will display any individual test coverage vector. It can also give you the union of the entire set (as if you had run all the tests) or compute how one test overlaps with another.