We Keep Coding

what is the advantage of using #VisibleForTesting? [duplicate]

This question already has answers here:
How do I test a class that has private methods, fields or inner classes?
(58 answers)
Closed 5 years ago.
Who has a solution for that common need.
I have a class in my application.
some methods are public, as they are part of the api,
and some are private, as they for internal use of making the internal flow more readable
now, say I want to write a unit test, or more like an integration test, which will be located in a different package, which will be allowed to call this method, BUT, I want that normal calling to this method will not be allowed if you try to call it from classes of the application itself
so, I was thinking about something like that
public class MyClass {
public void somePublicMethod() {
....
}
#PublicForTests
private void somePrivateMethod() {
....
}
}
The annotation above will mark the private method as "public for tests"
which means, that compilation and runtime will be allowed for any class which is under the test... package , while compilation and\or runtime will fail for any class which is not under the test package.
any thoughts?
is there an annotation like this?
is there a better way to do this?
it seems that the more unit tests you write, to more your inforced to break your encapsulation...

The common way is to make the private method protected or package-private and to put the unit test for this method in the same package as the class under test.
Guava has a #VisibleForTesting annotation, but it's only for documentation purposes.

If your test coverage is good on all the public method inside the tested class, the privates methods called by the public one will be automatically tested since you will assert all the possible case.
The JUnit Doc says:
Testing private methods may be an indication that those methods should be moved into another class to promote reusability.
But if you must...
If you are using JDK 1.3 or higher, you can use reflection to subvert the access control mechanism with the aid of the PrivilegedAccessor. For details on how to use it, read this article.

Consider using interfaces to expose the API methods, using factories or DI to publish the objects so the consumers know them only by the interface. The interface describes the published API. That way you can make whatever you want public on the implementation objects and the consumers of them see only those methods exposed through the interface.

dp4j has what you need. Essentially all you have to do is add dp4j to your classpath and whenever a method annotated with #Test (JUnit's annotation) calls a method that's private it will work (dp4j will inject the required reflection at compile-time). You may also use dp4j's #TestPrivates annotation to be more explicit.
If you insist on also annotating your private methods you may use Google's #VisibleForTesting annotation.

An article on Testing Private Methods lays out some approaches to testing private code. using reflection puts extra burden on the programmer to remember if refactoring is done, the strings aren't automatically changed, but I think it's the cleanest approach.

Or you can extract this method to some strategy object. In this case you can easily test extracted class and don't make method public or some magic with reflection/bytecode.

Okay, so here we have two things that are being mixed. First thing, is when you need to mark something to be used only on test, which I agree with #JB Nizet, using the guava annotation would be good.
A different thing, is to test private methods. Why should you test private methods from the outside? I mean.. You should be able to test the object by their public methods, and at the end that its behavior. At least, that we are doing and trying to teach to junior developers, that always try to test private methods (as a good practice).

I am not aware of any such annotation, however the following may be of value: unit testing private methods
or the following: JMockit

You can't do this, since then how could you even compile your tests? The compiler won't take the annotation into account.
There are two general approaches to this
The first is to use reflection to access the methods anyway
The second is to use package-private instead of private, then have your tests in the same package (but in a different module). They will essentially be private to other code, but your tests will still be able to access them.
Of course, if you do black-box testing, you shouldn't be accessing the private members anyway.

We recently released a library that helps a lot to access private fields, methods and inner classes through reflection : BoundBox
For a class like
public class Outer {
private static class Inner {
private int foo() {return 2;}
}
}
It provides a syntax like :
Outer outer = new Outer();
Object inner = BoundBoxOfOuter.boundBox_new_Inner();
new BoundBoxOfOuter.BoundBoxOfInner(inner).foo();
The only thing you have to do to create the BoundBox class is to write #BoundBox(boundClass=Outer.class) and the BoundBoxOfOuter class will be instantly generated.

As much as I know there is no annotation like this. The best way is to use reflection as some of the others suggested. Look at this post:
How do I test a class that has private methods, fields or inner classes?
You should only watch out on testing the exception outcome of the method. For example: if u expect an IllegalArgumentException, but instead you'll get "null" (Class:java.lang.reflect.InvocationTargetException).
A colegue of mine proposed using the powermock framework for these situations, but I haven't tested it yet, so no idea what exactly it can do. Although I have used the Mockito framework that it is based upon and thats a good framework too (but I think doesn't solve the private method exception issue).
It's a great idea though having the #PublicForTests annotation.
Cheers!

I just put the test in the class itself by making it an inner class:
https://rogerkeays.com/how-to-unit-test-private-methods

Is it OK to make a private method to non-private just because to make it easier for testing?

We are maintaining some legacy code and we are adding tests for them.
In order to write tests easier, we will extract some code into methods(which will be private be default), then writing tests for them. Since they are private, it's hard to invoke them, so what we did is: remove the "private" keyword, make it "non-private", then write tests.
But it makes some methods non-private which are supposed to be "private". Is it OK to do this, or is there any better solutions?
Update:
I think I'd better to provide some more background:
The code is very complicated and not good, and has nearly no tests. Writing tests for public methods are not possible since they have lots of hard-coded external dependencies and which are not be mocked.
We need to fix some bugs, but without tests we can't do it. So we first extract some related code to some methods, add tests for them, then fix bugs.
Since the extracted methods are private by default, we have to make them non-private, otherwise we have to invoke them by reflection:
Method method = Somecls.class.getMethod("a-private-method");
method.setAccessible(true);
method.invoke(...)
But if we make it non-private, we can just:
Somecls cls = new Somecls();
cls.nonPrivateMethod(...)

If the method is private, then it will be invoked by a public method in that class. Ideally, the test cases should pass through the public method to reach all the possible flows in that private method.

I think this is something pretty personal. In most of the scenarios I don't see any harm on changing the scope of a method from private to protected/package private if this helps to write better unit tests. It not only allows to invoke the method from your test - which you might do either way using reflection -, but also to override/mock that method when testing another one invoking this.
However, if you don't want to do it you still can use tools like Spring's ReflectionTestUtils to make the invocation of private methods less painful.

Yes this is perfectly acceptable for creating unit tests for the private methods. By leaving out the private keyword you create what is a known as a package-private method which can only be accesed by other classes that are in the same package. This lets you create unit test classes in the same package that can call your methods but also provides encapsulation so clients that use your class can't call those methods.

Enforcing that a JUnit subclass test overrides a #BeforeClass method

I have what amounts to a lightweight test framework written as a JUnit Abstract test. What I would like to do is have the implementing subclasses each define their custom test class setup. My plan was to have the abstract superclass define an #BeforeClass method that calls into an abstract setup method that each subclass would be forced to define, but this fails as the #BeforeClass methods must be static and static methods cannot be made abstract nor can they call instance methods.
I could just assume that subclasses will do the setup by including what's required in the documentation or by throwing an IllegalStateException, but I'd really like to be able to enforce this at an interface level for a number of reasons. Can anyone think of a work around for this?
By the way, I had the same issue with making these tests parameterized (subclasses define the parameters, but #Parameters annotated methods must be static). I got around this by running with the 3rd party JUnitParams runner which allows method level parameters. Check it out here: https://github.com/Pragmatists/JUnitParams

One option is to have subclasses implement a, say, static doSetupOnce() method, and find and invoke that method reflectively from the base class #BeforeClass method. Since this needs to be a static method, its existence can only be enforced at runtime.
Another approach would be to have an abstract doSetupOnce instance method in the base class which gets invoked the first time the parent's #Before method gets invoked. This enforces the issue at compile time, but then implementors will have to be careful not to access instance fields from this method (since this is probably not what they want).
In general (and without knowing the details of your situation), I'm not very fond of either of these approaches, and would rather leave it up to implementors to declare a #BeforeClass method if needed. Locking them up in a rigid base class scheme can cause more problems than it solves. Also consider the use of JUnit rules, which often are a better choice than base classes (e.g. because they are composable). Of course you can also combine the two approaches, relying mainly on JUnit rules and additionally offering some base classes with predefined rules for convenience.

For your main question, why not make your parent class abstract and use the #Before annotation instead of #BeforeClass ? For example:
public abstract class TestParent {
#Before
public void setup() {
doSetup();
}
protected abstract void doSetup();
// other stuff...
}
public class RealTest extends TestParent {
protected void doSetup() {
// custom setup
}
// custom tests...
}
This will force the subclasses to redefine the doSetup() method without using static methods.

This may be out of scope or overkill, but I think it's worth mentioning since they're not that different. So I'm taking a leap of faith and suggest that you could try TestNG because methods annotated with #BeforeClass do not have to be static, nor do those annotated with #Parameters.
You can read about the differences between the 2 frameworks here and it looks like they also have support for migrating JUnit tests to TestNG

I don't think it is possible to do this in a clean OO way. Not only is #BeforeClass a static method, but JUnit will call the parent's #BeforeClass before the child's #BeforeClass.
Anyway you try to do this must necessarily expose the parent class's internal static state so a child class can set the parent's fields, breaking encapsulation.
I think the best way is to to use #Before, but also have a static flag that sets if the method has been called before, that way at least you can short circuit and only do the initialization for the first call...

#BeforeClass vs static{}

I am writing some test cases using JUnit. I need to initialize some static variables which will be used for all the test cases in that class.
For this I can use either
Static initializer block or
Static method with #BeforeClass
What are the advantages of using one over another?

There are very different semantics for #BeforeClass or a static initializer.
A static initializer is invoked by the JVM and not by JUnit. If an exception is thrown within a static initializer, the test framework might not be able to catch and report the exception. Furthermore, the invocation time of the static initializer is not well-defined compared to the #BeforeClass method. It will be run only once per class loader on its first actual use which is for example the access of a static property, a static method or one of its constructors. Sometimes, it might be hard to figure out when this will be. (If you do not use inheritence: You might one day or some coworker will refactor your test case. If not today, the choice for a static initializer might introduce nasty bugs in the future.)
On the other hand, #BeforeClass is run before each class's tests are run. If a class would be subject to different tests, for example due to tests built on inheritance, the static initializer will only run for the first test using this class. This means that you made your test order dependent what is something you never want.
Note that the semantic difference between the two options is bigger than between using #Before or a constructor for a test. As a final argument, think about the documentary value of the annotations. It makes your intentions more readable.
The only exception for this rule would be immutable constants. Those should be initialized within their declaration in order to keep your code concise and in order to respect compile time constants. If your values are however mutable, you should not use static values at all. Again, mutable values that are altered in a test introduce an order dependency to your test which is to be avoided.
TL;DR: Use #BeforeClass!

Here's a couple of considerations that could be taken into account when deciding whether to use the static initialization block or #BeforeClass:
#BeforeClass is the antagonist of #AfterClass. So if you do initializations that require cleaning up later (like opening external resources) it would be better (from a semantic point of view) to use the annotated methods.
If you do complex initializations that might throw checked exceptions it's much more comfortable to use #BeforeClass, because you don't have to catch and wrap it into an unchecked exception.
If you want to use constant semantics (private static final String VARIABLE) with complex initialization, you will have no choice but to use the static initialization block or a static method.
There is a related post on SO: unit testing - What's the difference between using #BeforeClass and using instance or static variable in JUnit 4 Java?

If it's both static and final then you only have one choice: static initializer.
The compiler will prevent you from writing to a final field from inside a method, static or not.
EDIT: OK, you've removed the word 'final' from your question. In that case, it makes little difference. The static initializer will run once; so will the #BeforeClass method. Just pick whichever one you think is more readable.

It's not exactly the same behavior. #BeforeClass runs everytime the tests are run, a static initializer only once when the class is loaded. If your testrunner uses the same classloader, then in the case of #BeforeClass, you would re-run the initialization of the static variable. It depends on what you want to achieve here.

PowerMock + Mockito VS Mockito alone

Can anyone please summarize, what exactly features gives you adding PowerMock on top of the Mockito?
So far I've found these:
mock static, final and private methods
remove static initializers
allow mocking without dependency injection - this one isn't clear to me. Can you elaborate?
Does it add anything else? Can you please sum up in several lines?
And do I need to sacrifice something when using PowerMock?

I don't know of other benefits offhand, but I want to address 2 of your sub-questions (and this is way too long for a comment):
allow mocking without dependency injection - this one isn't clear to me. Can you elaborate?
I think this came from the Motivation wiki page where they describe a way of refactoring code to not invoke static methods to make it testable. For a concrete example of what I think they're getting at, let's say you have this code and you want to test the method mocking the behaviour of the static method, without using powermock:
public class MyClass {
public void doGetString() {
...
OtherClass.getString(); //It's complex and scary and needs mocking!
...
}
}
One solution, would be to pull the static invocation into its own object, then inject an object that can be mocked come test time. For example, without using other frameworks, this could look like:
public class MyClass {
public static class StringGetter {
public getString() {
return OtherClass.getString();
}
}
private final StringGetter getter;
//Existing Constructor
public MyClass() {
this(new StringGetter());
}
//DI Constructor
MyClass(StringGetter getter) {
this.getter = getter;
}
public void doGetString() {
...
getter.getString();
...
}
}
I've seperated the behaviour of my method from the behaviour of the static invocation, and can use the DI constructor to inject mocks easily at test time. Of course with powermock I could just mock the static method in place, and run with it.
And do I need to sacrifice something when using PowerMock?
Physically no, but I'd say philosophically yes :). The below are my opinions, and I try to give good reasons behind them, but of course they are opinions so take them with a grain of salt:
The potentially scary thing that is happening with PowerMock is that in order to accomplish the feats of mocking private and static methods, they are using a custom class loader (which shouldn't be present at runtime in production) and changing the bytecode of your classes. Arguably, this should not matter with the vast majority of classes most of the time, but if you think about it, if the bytecode has changed, and certain side effects are no longer present, you're effectively testing different Classes albiet based upon your existing Classes. Yes this is a very academic argument.
You can somewhat mitigate this first argument by having good comprehensive integration and higher level tests that don't use PowerMock. In this way you can be more confident in the behaviours of your objects even if your unit tests are using PowerMock.
The other argument I have against PowerMock, is that it could almost too easily become a crutch. I agree that PowerMock can help with testing code that uses legacy code and other code that you do not have control over. However I would argue that when you have control over the classes that you need to mock, you should avoid its use. If you write a class with a private method or static method that you need to explicitly mock in order to test other methods, my gut instinct would say that this method may be doing too much and should be refactored and broken up. Having PowerMock already available in a project, you may be tempted to just mock it and move on, which would mitigate the pain that should encourage you to refactor the same. Yes there are sometimes due to various technical and non-technical constraints this is not possible, but it's good to solve pain points instead of avoid them :)

PowerMock is an extension to Mockito that allows mocking of static methods, constructors, final classes and methods, private methods, removal of static initializers and more.

Another feature of the Powermock mockito extension is that it supports mocking and stubbing of equals and hashcode.
As with all powermock features to be used with care, but adding (value-based) equality for specific results can be helpful.

One more feature of PowerMock is that we can mock construction of new objects in a method. It is helpful when we cannot change the code of the method to be tested.

For mocking final class we can use org.mockito.plugins.MockMaker. What you would need to do is
Create a folder in your test/resource folder with namemockito-extensions.
Create a file under it with the name org.mockito.plugins.MockMaker.
In that file have just one line mock-maker-inline
This will not require you to do add any new library and hence save some runtime.