I want to have a unit test that checks if all methods defined in a class/interface are covered with unit tests.
There is an interface and a number of classes that implement that interface.
Since the classes are related, and some methods are critical for performance, there is some amount of copied/pasted/modified code.
I want to make sure that unit tests for each class (that implements that interface) cover:
all methods declared in the interface
all methods declared in the class
This is meaningful because such tests discover:
code that was not modified after copying/pasting
methods inherited from the base class that must be redefined
Currently I use the following tool (in the coding convention that I use all testing classes and methods must end with "Test"):
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.stream.Collectors;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
public class TestCoverage {
public static void check(Class appClass, Class testingClass) {
assertTrue("testingClass must end with \"Test\"", testingClass.getSimpleName().endsWith("Test"));
var appMethodNamesToTest =
Arrays.stream(appClass.getDeclaredMethods())
// .filter(m -> !Modifier.isStatic(m.getModifiers()))
.map(Method::getName)
.filter(s -> !s.contains("$")) // filter out $jacocoInit
.map(s -> s + "Test")
.collect(Collectors.toList());
var testingMethodNames =
Arrays.stream(testingClass.getDeclaredMethods())
.map(Method::getName)
.filter(s -> s.endsWith("Test"))
.collect(Collectors.toSet());
var missingMethods = appMethodNamesToTest.stream()
.filter(methodName -> !testingMethodNames.contains(methodName))
.collect(Collectors.joining(", "));
if (!missingMethods.isEmpty()) {
fail("missing test methods in " + testingClass.getSimpleName() + ": " + missingMethods);
}
}
}
which is used as:
public class FooTest {
// unit tests go here
#Test
public void coverageTest() {
TestCoverage.check(Baar.class, getClass()); // interface
TestCoverage.check(Foo.class, getClass()); // implementing class
}
}
What is really good is that I get an early warning about methods not covered by the tests.
Strictly speaking, coverageTest() is not a unit test.
In the above code, getClass() returns FooTest.class, but unlike FooTest.class, it need not be modified when you copy/paste coverageTest(). In theory, it should be possible to extract the calling class from an exception stack trace, but this is complex.
EDIT {
The problem that I want to solve is a bit different from what the coverage tools usually do. I am interested in per-interface test coverage, not per-method test coverage. With the existing coverage tools, in the following example there is no way to force the developer to define C.c():
interface I { void a(); void b(); default void c() {...} }
class A { // defines a() and b(); the default c() is correct
}
class B { // defines a() and b(); the default c() is correct
}
class C { // defines a() and b(); the default c() is NOT correct
}
If I.c() is covered by at least some test, it will be green, and C.c() does not exist and therefore cannot be marked red.
TestCoverage.check() does make the developer write tests for A.c(), B.c() and C.c() and hopefully mention that c() must be defined in C.
On the other hand, classes A and B share the common implementation I.c(), but the test for c() has to be copied.
In other words, what I want is make sure all interface methods are covered by tests in each implementing class.
}
Since it is very unlikely that I "invented a wheel",
Question:
Have such checks been already implemented in the existing testing frameworks?
Related
I am trying to improve my knowledge about testing I'm trying to achieve running the same JUnit test class with different objects derived from the same interface.
so we can assume the following:
interface Base {
void sort();
}
class A implements Base {
#Override
public void sort() {
//sort naively
}
}
class B implements Base {
#Override
public void sort() {
//sort using another better approach
}
}
class C implements Base {
#Override
public void sort() {
//sort using optimized approach
}
}
class Test {
#Test
void test1() {
Base obj = new A();
obj.sort();
obj.otherStuff();
}
}
class SecondTest {
//trying to avoid making multiple test classes that has only one line in difference
#Test
void test1() {
var obj = new B();
obj.sort();
obj.otherStuff();
}
So my question is how to run the test class with the objects from A,B,C without falling into the trap of duplicate code and redundancy?
Please note that I wrote this example just to illustrate my point, and the sort() doStuff() methods are just placeholders but when you have over ~70 line of code duplication in each test class it starts to look ugly and redundant.
*I have looked at #beforeEach, #Before, #After and I don't think I see a way where those might help me.
You can write a parameterized test with a MethodSource.
#ParameterizedTest
#MethodSource("bases")
void test1(Base obj) {
obj.sort();
obj.otherStuff();
}
static Stream<String> bases() {
return Stream.of(new A(), new B(), new C());
}
A way to fix it is the following, you create a method within your test class that takes as input the Base obj and contains all the duplicate lines. What you'll do then is to initialize the obj in different tests, then pass it to the method.
Here is a code that would do the job:
class Test {
#Test
void test1() {
Base obj = new A();
wrapperMethod(obj);
}
#Test
void test2() {
var obj = new B();
wrapperMethod(obj);
}
public static void wrapperMethod(Base obj){
obj.sort();
obj.otherStuff();
}
}
As a rule of thumb, testing can be much like normal programming where redundancy is avoided with methods to guarantee reusability.
Cheers,
D
First of all you have to fix your understanding of what UnitTesting is about.
UnitTesting is not about code (coverage).
UnitTesing is about verifying desired public behavior where "public behavior means return values and/or communication with dependencies.
Each test method should verify a single atomic assumption of the tested units desired behavior.
From this point of view it does not make sense to pass a bunch of objects sharing the same interface trough the same test method since these different interface implementations exist to implements the interfaces methods with their own unique behavior. In turn the assumption how the objects behave differ uniquely.
If all the objects where expected to behave identically (which is the only assumption a single test method could verify) there where no different objects (i.e. implementations) in the first place.
Assuming I have a couple of interfaces with exactly one abstract method. Having these interfaces, I can declare lambdas with it:
interface A {
int c();
}
interface B {
int c();
}
public class Main {
public static void main(String... args) {
A a = () -> 42;
B b = () -> 42;
}
}
Short question: is there some trick or hack to restrict using interface A for lambdas and fail the build on attempt to do so? Any hint, dirty or not, is welcome (by 'dirty' I mean hacks on compilation/bytecode level - something which won't affect sources and, preferably, public contracts).
Long story: for some interfaces implementors I consider defining equals/hashCode as a part of the contract. Also, I generate equals/hashCode automatically for them at build time.
In this context, lambdas are troublemakers. For ordinary and anonymous implementors of interface A I can find a .class file and instrument its bytecode at build time. For lambdas there is VM-anonymous class, produced at run time. Affecting such class seems impossible at build time, so I need to at least prohibit such occasions for a specific set of interfaces.
Please take a look at my solution on that:
package com.example.demo;
public class LambdaDemo {
public static void main(String[] args) {
//doesn't compile
//LambdaRestrictedInterface x = () -> {};
LambdaRestrictedInterface y = new Test();
y.print();
}
private static class Test implements LambdaRestrictedInterface {
#Override
public void print() {
System.out.println("print");
}
}
public interface MyInterface {
void print();
}
public interface LambdaRestrictedInterface extends MyInterface {
#Override
default void print() {
//hack prevents lambda instantiating
}
}
}
https://dumpz.org/2708733/
Idea is to override parent interface with default impl
Edit from originator: After some consideration, I decided to accept this answer, (since it suited my needs the best and is rather cheap to implement) with some formal additions. In fact, it was realized that the minimal instrumentation which is enough to prevent interface being used as lambda-type is to just add default implementation to its abstract method.
From playing around a bit, it looks like the desc field of the invokedynamic call contains the interface that's being implemented. For instance, when I created a simple () -> {} Runnable and then passed it through ASM's Bytecode Outline plugin, the "ASM-ified" call looked like:
mv.visitInvokeDynamicInsn("run", "()Ljava/lang/Runnable;", new Handle...
So if you're able to do the build-time hack on the call site (as opposed to somehow marking the annotation itself as non-lambda-able, which I don't think you can do) then you should be able to first compile a set of disallowed interfaces, and then check the invokedynamic's desc against that set.
This is my situation, I have 2 very simple classes:
public class B {
public void doSomething(){
System.out.println("doSomething B reached");
}
}
And:
public class A {
public void doSomething(){
B b = new B();
b.doSomething();
System.out.println("doSomething A reached");
}
}
I want to test method doSomething of class A with Mockito. Therefor, I want to mock an instance of class B and give this to A when it is instantiating class B. I don't want b.doSomething() to be reached at all, for isolation reasons.
I know I can reach this behaviour by creating the following unittest:
#RunWith(PowerMockRunner.class)
public class TestA {
#Test
#PrepareForTest(A.class)
public void testDoSomethingOfA() throws Exception{
A a = PowerMockito.spy(new A());
B b = PowerMockito.mock(B.class);
PowerMockito.whenNew(B.class).withNoArguments().thenReturn(b);
a.doSomething();
}
}
which results in output:
doSomething A reached
So this work! However, my problem now is that we use the Jococo plugin for test coverage. Jococo doesn't cover code tested with the #PrepareForTest(A.class) statement. And my company values accurate code testing coverage.
My question: Is there another way around to give A an instantiation of B without having to use the #PrepareForTest statement?
Many thanks in advance!
To answer my own question, yes, use an agent: https://github.com/jayway/powermock/wiki/PowerMockAgent
#Rens Groenveld: After integrating PowerMockAgent, did you make any changes in your test class or source code ? I tried integrating PowerMockAgent and removed my main source class from #PrepareForTest but the behavior is same (creating new instance instead of mock).
I have tried jacoco offline instruments and many other solution, It did not work.
I can not post to your comment (needs 50 points) hence added as answer.
I have a class A which instantiates an object from class B internally. That second class has lots of external side-effects, e.g. it uses a network connection.
public class A {
private B b;
public A() {
b = new B(); // 3rd party class that calls out externally (e.g. to a db)
}
}
I would like to unit test class A by providing a mock implementation of class B. I can easily create a mocked version of B by writing my own class or using something like Mockito and other frameworks, but how do I inject this mock implementation into class A's code?
I guess I could ask for an instance of class B in class A's constructor, but that seems ugly. No one really needs to know how class A does it's business.
Python has a "mock" library that allows you to "patch" functions at run time during a test. Does Java have something similar?
In this scenario I typically have a 2nd package (default) scope constuctor that allows you to pass a mock in for testing purposes.
public class A {
private B b;
/*
* Used by clients
*/
public A() {
this(new B());
}
/*
* Used by unit test
*
* #param b A mock implementation of B
*/
A(B b) {
this.b = b;
}
}
Check out Mockito. Ultimately, based on your design, you'll need to use reflection to get the mock instance of B into A.
This is trivial to solve with the JMockit mocking API:
public class ATest
{
#Test
public void myTest(#Mocked B mockB)
{
// Record expectations on "mockB", if needed.
new A().doSomethingUsingB();
// Verify expectations on "mockB", if applicable.
}
}
The more I read mock example, the more I get confused...
I have classA method eat() that calls FatDude class eatThemAll()
public class classA {
FatDude dude = new FatDude();
public String eat() {
String result = dude.eatThemAll();
}
}
public class FatDude {
public String eatThemAll() {
return "never full";
}
}
Now I want to test classA eat() method by mocking FatDude class.
public class MockFatDude extends FatDude {
//override
public String eatThemAll() {
return "very full";
}
}
------------- test --------------
public class DoTest {
public void runTest() {
classA cl = new ClassA();
String out = cl.eat();
assertEqual(out, "very full");
}
}
This DoTest runTest() won't use MockFatDude class of course. One way I can think is to change the code to pass FatDude to eat() method of ClassA like:
public class classA {
public String eat(FatDude dude) {
String result = dude.eatThemAll();
}
}
Then change my test method to:
public class DoTest {
public void runTest() {
classA cl = new ClassA();
String out = cl.eat(new MockFatDude());
assertEqual(out, "very full");
}
}
But as you can see, I had to change the source code to meet my need.
Is this the right way to do? What if I am not allowed to change my source code?
I know if I apply TDD concept, it is OK to change source code but I would like to hear
some opinion or advice if what I have shown above is right way to do.
Mocking and the Dependency Inversion Principle (DIP) go hand in hand, and in most languages, Mocks work best by decoupling classes by means of interfaces.
In your instance, this will work without you needing to change code: (Edit : I mean, in future, if you design your app this way, you won't need to change code to mock dependencies :))
Abstract an interface IDude
Concrete classes FatDude (and MockFatDude) should implement IDude interface
provide a mechanism for an IDude instance to be 'set' or injected into classA - Dependency Injection (constructor or get / sets); or Service Locator pattern work best (to replace a concrete classfactory)
Also note that many mocking frameworks actually allow you to build up the Mock concrete class 'on the fly' (see MoQ et al), so you can create the functionality of MockFatDude directly in your unit test.
Yes. You've stumbled on some good design directly because of your unit test. If you look more closely, you'll see that you removed the coupling between classA and FatDude. Now FatDude can be an interface for behavior that gets passed in when needed. ClassA doesn't need to know what kind of FatDude it's getting, or how to construct a FatDude (with cheeseburgers?).
Your solution is exactly what I would have done. There's nothing wrong with changing your code to accomodate TDD, as long as you understand the reasons and the benfits/drawbacks to making such changes.