We Keep Coding

Calling real method in Mockito, but intercepting the result

Simplifying a bit, our system has two parts. "Our" part, which in turn uses an lower level part implemented by another team (in the same codebase). We have a fairly complicated functional test setup, where we wrap the entry points to the lower level in spy objects. In positive tests we use the real implementation of that level, but we mock calls that should fail with some predefined error.
Now I am trying to add support for more complicated scenarios, where I would like to add an artificial delay for the calls made to the underlying level (on a fake clock obviously). To do this I would like to define a mock that would (1) Call the real implementation (2) Get the resulting Future object that is returned and combine it with a custom function that would inject the delay accordingly. So Ideally I would like to have something like:
doAnswer(invocationOnMock ->
{
result = call real method on mySpy;
return Futures.combile(result, myFunction);
}).when(mySpy).myMethod();
How can I achieve it?

As for me the easiest way it's just to save the link to the read object when you initialize your Spy object:
Foo realFoo = new Foo();
Foo spyFoo = Mockito.spy(realFoo);
Now you can stub it like this:
doAnswer(invocation -> realFoo.getSome() + "spyMethod").when(spyFoo).getSome();
One more way is to call invocation.callRealMethod():
doAnswer(invocation -> invocation.callRealMethod() + "spyMethod").when(spyFoo).getSome();
But in this case you may need to cast the return value as long as invocation.callRealMethod() returns Object.

Using both junit assertions and mockito verification

I'm using Junit in conjunction with Mockito. I used mockito's verify method + junit assertion to do complete verification. Is this not desirable? Should we use one or the other but not both?

There's nothing wrong with using both.
Mockito's verify is used to assert a method was called (with the expected arguments) on the given mock.
JUnit's assertXYZ is used to assert that some result has the expected value.
Both are valid verifications, and if both are relevant, both should be used.
For example, consider the following (admittedly artificial) situation:
You have an interface that performs some mathematical calculation:
public interface ValueProducer {
public int getValue(int val);
}
And a class that doubles any result produced by it:
public class Doubler {
public static int doubleThatResult (ValueProducer producer, int val) {
return 2 * producer.getValue(val);
}
}
Testing it would require to assert two things:
That getValue was properly called
That the result is doubled
So, e.g.:
public class DoublerTest {
#Test
public void testDoubleThatResult() throws Exception {
int value = 7; // Or any other value
int returnMock = 13; // Or any other value
ValueProducer producerMock = mock(ValueProducer.class);
when(producerMock.getValue(value)).thenReturn(returnMock);
int actual = Doubler.doubleThatResult(producerMock, value);
verify(producerMock);
assertEquals(26, actual);
}
}

Mureinik's answer is absolutely correct—assertions and verifications are complementary, and work quite well together—but doing both for a single behavior may be redundant.
The goal is usually to express the test in the most flexible and implementation-agnostic way possible. State testing (with assertions) is often the most appropriate choice, in that sense: it doesn't matter which methods were called as long as the resulting state is correct. A Mockito verification, while possible, may introduce brittleness (where the code behaves correctly but the test is broken). Mockito itself warns of this in verify(T)'s Javadoc and the linked article by Mockito's creator Szczepan Faber.
This redundancy is certainly not the worst thing: It does test a property of the system, and may be a worthwhile endeavor in testing interactions with sensitive, legacy, third-party, or frequently-changing dependencies. If the interaction is unimportant, though, these extra verifications may come at the expense of brittleness.
Other times, confirming correct behavior means testing side-effects—for example, ensuring that someSystem.reset() was called, or testing a cache by verifying a provider was NOT called. These are perfect examples for verification, because the interaction with the external service is critical testable behavior. In some of these cases, Mockito verifications are the only way to make the correct assertions.
Many test cases are a combination of the above, so feel free to use assertions and verifications freely together; just remember that state testing is often sufficient and preferable.

Should I check the state of objects returned by methods I am testing

I am writing unit tests for my service layer. I have a method named placeOrder that returns an Order object. I have a unit-test that checks that the returned Order object is not null.
Should I also check the state of the returned object to ensure that the placeOrder method has set it correctly? For example, checking the order date, number of order lines, the user that placed it.
Or is my check that the returned object is not null enough?

I use a programming-by-contract approach to produce my code. If I am testing a method, my test cases check all the post-conditions (including the invariants) of the method that can be expressed reasonably simply in terms of the arguments to the method, the state of the object before the method call and the state of the object after the method call.
Why? Because I can not predict which post-condition or invariant might be broken by an incorrect refactoring or enhancement, so I must test them all, or nearly all of them.
Conversely, if the field of a returned object is not part of a post-condition, I tend not to check its value in tests, unless I know I must check it to ensure correct operation of the method. And how do I know which fields those are? I also use a tdd approach; if the minimum required change to code under development to make it pass the existing set of tests is to not both setting some field values, I deliberately refrain from setting those fields, and add to my TODO list a test-case (or cases) that check that those fields are correctly set.

IMHO it depends on what the method's purpose is - to populate the object's members or just to return an already populated (from somewhere else) object.
Let's say your method does this:
public Order placeOrder() {
Order order = new Order();
order.setUser("John Smith");
//more members are set...
return order;
}
In this case, obviously, the methods purpose is to populate the order's members and then to return the object. In this case your unit test has to ensure that the returned object has (all) it's members populated, e.g.
#Test
public void testPlaceOrder() {
Order order = myServiceInstance.placeOrder();
assertNotNull(order);
assertNotNull(order.getUser());
//more assertions
}
The other case is the one in which your placeOrder method just returns an already populated object. For example:
public Order placeOrder(Order order) {
someExternalMethodThatPopulatesTheOrder(order);
return order;
}
In this case, your unit-test should just test the functionality of the placeOrder method and should not case if the someExternalMethodThatPopulatesTheOrder() works correctly. Given this, you can mock the someExternalMethodThatPopulatesTheOrder()'s functionality (with libraries like PowerMock or Mockito), so that you can test the functional purpose of your placeOrder method.

Can I place unit test result validation in the #After method?

I'm writing a Unit test for the behavior of a method, and want to try to reduce my code duplication - I'm about to double the test cases again, and the file is getting a bit hard to look at.
If anyone's really interested, the code is here (warning to the future - link may not be permanent).
Right now, I do the following, repeated several times in slightly different ways:
#Test
public void testAdd64OfMax64() {
// Set up container
TileEntityChest nmsinv = new TileEntityChest();
Inventory container = new CraftInventory(nmsinv);
// Call method
HashMap<Integer,ItemStack> leftover = container.addItem(new ItemStack(Foo, 64));
// Set up expected result
ItemStack[] expected = empty.clone();
expected[0] = new ItemStack(Foo, 64);
// Verify result
assertThat(container.getContents(), is(expected));
assertThat(leftover, is(Collections.EMPTY_MAP));
}
(Note: assertThat comes from org.hamcrest.Matchers.)
I know that I can put the "Set up container" and cloning the empty array into a #Before method.
My question is, can I put the assertThat method into an #After method and have the test still fail (when it should fail)?

No. The After method should be used to release external resources after a test run. Also, note that it is guaranteed to run even if your Test method throws an exception which may not be desirable in your case.
Also, think about the people who will have to maintain your code in the future. You are violating the PLA, by validating the results of your tests in the After method. Nobody expects that!
It would be better if you created a helper method e.g. checkResults which was called at the end of each of your Test methods to validate your results.

Value of Behavior Verification

I've been reading up on (and experimenting with) several Java mocking APIs such as Mockito, EasyMock, JMock and PowerMock. I like each of them for different reasons, but have ultimately decided on Mockito. Please note though, that this is not a question about which framework to use - the question really applies to any mocking framework, although the solution will look different as the APIs are (obviously) different.
Like many things, you read the tutorials, you follow the examples, and you tinker around with a few code samples in a sandbox project. But then, when it comes time to actually use the thing, you start to choke - and that's where I am.
I really, really like the idea of mocking. And yes, I am aware of the complaints about mocking leading to "brittle" tests that are too heavily coupled with the classes under test. But until I come to such a realization myself, I really want to give mocking a chance to see if it can add some good value to my unit tests.
I'm now trying to actively use mocks in my unit tests. Mockito allows both stubbing and mocking. Let's say we have a Car object that has a getMaxSpeed() method. In Mockito, we could stub it like so:
Car mockCar = mock(Car.class);
when(mockCar.getMaxSpeed()).thenReturn(100.0);
This "stubs" the Car object to always return 100.0 as the max speed of our car.
My problem is that, after writing a handful of unit tests already...all I'm doing is stubbing my collaborators! I'm not using a single mock method (verify, etc.) available to me!
I realize that I'm stuck in a "stubbing state of mind" and I'm finding it impossible to break. All this reading, and all this excitement building up to using mocks in my unit testing and... I can't think of a single use case for behavior verification.
So I backed up and re-read Fowler's article and other BDD-style literatures, and still I'm just "not getting" the value of behavior verification for test double collaborators.
I know that I'm missing something, I'm just not sure of what. Could someone give me a concrete example (or even a set of examples!) using, say, this Car class, and demonstrate when a behavior-verifying unit test is favorable to a state-verifying test?
Thanks in advance for any nudges in the right direction!

Well, if the object under test calls a collaborator with a computed value, and the test is supposed to test that the computation is correct, then verifying the mock colaborator is the right thing to do. Example:
private ResultDisplayer resultDisplayer;
public void add(int a, int b) {
int sum = a + b; // trivial example, but the computation might be more complex
displayer.display(sum);
}
Clearly, in this case, you'll have to mock the displayer, and verify that its display method has been called, with the value 5 if 2 and 3 are the arguments of the add method.
If all you do with your collaborators is call getters without arguments, or with arguments which are direct inputs of the tested method, then stubbing is probably sufficient, unless the code might get a value from two different collaborators and you want to verify that the appropriate collaborator has been called.
Example:
private Computer noTaxComputer;
private Computer taxComputer;
public BigDecimal computePrice(Client c, ShoppingCart cart) {
if (client.isSubjectToTaxes()) {
return taxComputer.compute(cart);
}
else {
return noTaxComputer.compute(cart);
}
}

I like #JB Nizet's answer, but here's another example. Suppose you want to persist a Car to a database using Hibernate after making some changes. So you have a class like this:
public class CarController {
private HibernateTemplate hibernateTemplate;
public void setHibernateTemplate(HibernateTemplate hibernateTemplate) {
this.hibernateTemplate = hibernateTemplate;
}
public void accelerate(Car car, double mph) {
car.setCurrentSpeed(car.getCurrentSpeed() + mph);
hibernateTemplate.update(car);
}
}
To test the accelerate method, you could just use a stub, but you wouldn't have a compete test.
public class CarControllerTest {
#Mock
private HibernateTemplate mockHibernateTemplate;
#InjectMocks
private CarController controllerUT;
#Test
public void testAccelerate() {
Car car = new Car();
car.setCurrentSpeed(10.0);
controllerUT.accelerate(car, 2.5);
assertThat(car.getCurrentSpeed(), is(12.5));
}
}
This test passes and does check the computation, but we don't know if the car's new speed was saved or not. To do that, we need to add:
verify(hibernateTemplate).update(car);
Now, suppose that if you try to accelerate past max speed, you expect the acceleration and the update not to happen. In that case, you would want:
#Test
public void testAcceleratePastMaxSpeed() {
Car car = new Car();
car.setMaxSpeed(20.0);
car.setCurrentSpeed(10.0);
controllerUT.accelerate(car, 12.5);
assertThat(car.getCurrentSpeed(), is(10.0));
verify(mockHibernateTemplate, never()).update(car);
}
This test will not pass with our current implementation of CarController, but it shouldn't. It shows you need to do more work to support this case and that one of the requirements is that you don't try to write to the database in this case.
Basically, verify should be used for exactly what it sounds like - to verify that something happened (or didn't happen). If the fact that it happened or didn't isn't really what you are trying to test, then skip it. Take the second example I made. One could argue that since the value wasn't changed, it doesn't really matter whether update was called or not. In that case, you can skip the verify step in the second example since the implementation of accelerate would be correct either way.
I hope it doesn't sound like I'm making a case for using verify a lot. It can make your tests very brittle. But it can also 'verify' that important things that were supposed to happen did happen.

My take on this is that every test case should contain EITHER
stubbing, plus one or more asserts OR
one or more verifys .
but not both.
It seems to me that in most test classes, you end up with a mixture of "stub and assert" test cases and "verify" test cases. Whether a test case does a "stub and assert" or does a "verify" depends on whether the value returned by a collaborator is important to the test. I need two examples to illustrate this.
Suppose I have an Investment class, which has a value in dollars. Its constructor sets the initial value. It has an addGold method, which increases the value of an Investment by the amount of gold times the price of gold in dollars per ounce. I have a collaborator called PriceCalculator that calculates the price of gold. I might write a test like this.
public void addGoldIncreasesInvestmentValueByPriceTimesAmount(){
PriceCalculator mockCalculator = mock( PriceCalculator.class );
when( mockCalculator.getGoldPrice()).thenReturn( new BigDecimal( 400 ));
Investment toTest = new Investment( new BigDecimal( 10000 ));
toTest.addGold( 5 );
assertEquals( new BigDecimal( 12000 ), toTest.getValue());
}
In this case, the result from the collaborator method is important to the test. We stub it, because we're not testing the PriceCalculator at this point. There's no need to verify, because if the method hadn't been called, the final value of the investment value would not be correct. So all we need is the assert.
Now, suppose there's a requirement that the Investment class notifies the IRS whenever anyone withdraws more than $100000 from an Investment. It uses a collaborator called IrsNotifier to do this. So a test for this might look like this.
public void largeWithdrawalNotifiesIRS(){
IrsNotifier mockNotifier = mock( IrsNotifier.class );
Investment toTest = new Investment( new BigDecimal( 200000 ));
toTest.setIrsNotifier( mockNotifier );
toTest.withdraw( 150000 );
verify( mockNotifier ).notifyIRS();
}
In this case, the test doesn't care about the return value from the collaborator method notifyIRS(). Or maybe it's void. What matters is just that the method got called. For a test like this, you'll use a verify. There may be stubbing in a test like this (to set up other collaborators, or return values from different methods), but it's unlikely that you'll ever want to stub the same method that you verify.
If you find yourself using both stubbing and verification on the same collaborator method, you should probably ask yourself why. What is the test really trying to prove? Does the return value matter to the test? Because this is usually a testing code smell.
Hope these examples are helpful to you.