We Keep Coding

How to test that a method is called by Guava Cache only once

I'm trying to test that a cache I created is working properly, and duplicate calls to a method aren't actually called multiple times, but loaded from my cache.
I'm using LoadingCache from Google Guava to accomplish this.
So my naive approach was to create a spy, and verify that the method was called once. I then figured out that because spy() is a decorator, I can only see if the method was called on that object. Since the method is being called by LoadingCache, my spy cannot verify it.
How do I best test that my cache is being used properly?
(note: I could make a LoadingCache dependency, and check that the correct method is called, but then I don't know if my cache is working. Maybe the hash is calculated in a way I didn't anticipate, and so it actually is calling the method every time. I want to actually see the effect)
private final LoadingCache<TaskDetails, byte[]> cache;
...
cache = CacheBuilder.newBuilder()
.maximumSize(cacheSize)
.expireAfterAccess(Duration.ofMinutes(cacheDurationInMinutes))
.build(CacheLoader.from(this::doTask));
public byte[] taskCaller(...) {
...
return cache.getUnchecked();
}
public byte[] doTask(...) {
if(something.doSomething() ... ) {
...
}
First test attempt doesn't work, Mockito says: Wanted but not invoked (the method was "never called", since the spy doesn't know the cache called it)
#Test
public void test() {
// given
TaskService spy = spy(
new TaskService(...)
);
// when
for (int i = 0; i < 3; i++) {
spy.taskCaller(...);
}
// then
//if caching is working, we should only do the real work once
verify(spy, times(1)).doTask(any(TaskDetails.class));
}
Second attempt: NullPointerException on first statement in the method ( something.doSomething() ), I'm not completely sure why, but I'm convinced for the same reason as before
#Test
public void test() {
// given
TaskService spy = spy(
new TaskService(...)
);
final int[] counter = {0};
when(spy.doTask(any())).thenAnswer(invocation -> {
counter[0]++;
return new byte[]{0,0,0,0};
});
// when
int run = 3;
for (int i = 0; i < run; i++) {
spy.taskCaller(...);
}
// then
//if caching is working, we should only do the real work once
assertThat(run).isEqualTo(counter[0]);
verify(spy, times(1)).doTask(any(TaskDetails.class));
}

Two ideas:
Enable cache stats recording, interact with the cache, get the cache stats, then verify that the load count (and possibly other stats) are what you expect.
Make your CacheLoader a separate named class rather than a method reference, an instance of which is passed to CacheBuilder.build(), and arrange for that instance to be a mock in your test. You can then verify method call counts and arguments on your mock loader.

I'm using LoadingCache from Google Guava to accomplish this.
That's the point. If you trust Guava enough, why don't you trust that its tests work?
Did you write the cache? No, you didn't. Then don't test it as part of unit tests: only as part of integration tests, with the rest of the whole application.
Just test that the cache-loading method is called when you invoke your cache-using method.
If you want to test that you have a cache, perform two calls on the general method, and verify that the cache-loading method is called only once.

confused by unit testing and Mockito

I'm new to unit test and Mockito. I got confused by what I should test and verify. I have a class A as follows:
public class A{
#Autowired
private B b;
public double doSomething(Request r){
r = b.process1(r);
r = b.process2(r);
return calculateFinal(r);
}
public void reportSomething(Request r){
r = b.process1(r);
r = b.process2(r);
b.report(r);
}
private int calculateFinal(Request r){
return r.getFinalScore() * 2;
}
}
Suppose I want to test these two methods with Junit test. Since I have a dependency B in A, I mock it with Mockito. For both tests, I was told that I should assume that the dependency b is fully tested and properly working since we want to test the business logic in A.
At first it looks like that I don't have to test anything for reportSomething() since it only involves calls to b and they are all "working"? The only thing I can think of to test is whether they are actually called and the order of the calls, correct? So should I just call a.reportSomething() and then do the verification? One thing that bugs me is that whether I should stub the b.process1() and b.process2() to return anything. I tried without stubbing anything and it worked, but why?
For testDoSomething(), I think what I'm really testing is the calculateFinal() method. But since it uses the data from the Request object, I need to set that data in Request r first. Since r directly comes from b.process2(), I should stub the method call to return a Request object with that data. But I could skip the stubbing of b.process1(), right?
Is this a right thinking process? Did I miss something or misunderstand something? If it's right, is there a better and cleaner way to write it? Thank you!
public class ATest{
private static final int SCORE = 100;
#Mock
private B mockB;
#InjectMocks
private A aClient;
#Before
public void setUpTest{
MockitoAnnotations.initMocks(this);
}
#Test
public void testReportSomething(){
// what should I test here?
Request r = new Request();
// is it necessary to include the following two lines?
when(mockB.process1(any(Request.class))).return(r);
when(mockB.process2(any(Request.class))).return(r);
aClient.reportSomething(r);
InOrder inOrder = inOrder(mockProcesser);
inOrder.verify(mockProcesser).process1(any(Request.class));
inOrder.verify(mockProcesser).process2(any(Request.class));
inOrder.verify(mockProcesser).report(any(Request.class));
}
#Test
public void testDoSomething(){
// Is this correct?
Request r = new Request();
r.setFinal(SCORE);
// I skipped this line and it still works
when(mockB.process1(any(Request.class))).return(r);
when(mockB.process2(any(Request.class))).return(r);
assert(SCORE * 2, aClient.doSomething(r));
// is it still necessary to verify the call to mockB?
}
}

You are doing your test incorrectly. Let's look at the method you want to test:
public void reportSomething(Request r){
r = b.process1(r);
r = b.process2(r);
b.report(r);
}
First of all, you need to mock that when b processes a request, it returns the expected result; DO NOT therefore use the same return value for two invocations.
Here is how I would write the test:
final Request r = mock(Request.class);
final Request r1 = mock(Request.class);
final Request r2 = mock(Request.class);
when(mockB.process1(r)).thenReturn(r1);
when(mockB.process2(r1)).thenReturn(r2);
doNothing().when(mockB).report(any(Request.class));
final InOrder inOrder = inOrder(mockB);
// Launch... And then verify:
inOrder.verify(mockB).process1(r);
inOrder.verify(mockB).process2(r1);
inOrder.verify(mockB).report(r2);
inOrder.verifyNoMoreInteractions();
As to:
// is it necessary to include the following two lines?
Yes. By default, when unspecified, a mocked instance will return Java's defaults: 0 for numeric primitives, false for boolean, null for objects. You MUST specify what you want to be returned by stubbing.

EasyMock : java.lang.IllegalStateException: 1 matchers expected, 2 recorded

I am having a problem with EasyMock 2.5.2 and JUnit 4.8.2 (running through Eclipse). I have read all the similar posts here but have not found an answer. I have a class containing two tests which test the same method. I am using matchers.
Each test passes when run alone.
The first test always passes - this is true if I switch the order of the tests in the file.
Here is a simplified version of the test code:
private Xthing mockXthing;
private MainThing mainThing;
#Before
public void setUp() {
mockXthing = EasyMock.createMock(Xthing.class);
mainThing = new MainThing();
mainThing.setxThing(mockXthing);
}
#After
public void cleanUp() {
EasyMock.reset(mockXthing);
}
#Test
public void testTwo() {
String abc = "abc";
EasyMock.expect(mockXthing.doXthing((String) EasyMock.anyObject())).andReturn(abc);
EasyMock.replay(mockXthing);
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
assertEquals("abc", testResult);
EasyMock.verify(mockXthing);
}
#Test
public void testOne() {
String xyz = "xyz";
EasyMock.expect(mockXthing.doXthing((String) EasyMock.anyObject())).andReturn(xyz);
EasyMock.replay(mockXthing);
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
assertEquals("xyz", testResult);
EasyMock.verify(mockXthing);
}
The second (or last) test always fails with the following error:
java.lang.IllegalStateException: 1 matchers expected, 2 recorded
Any insight to this would be greatly appreciated.
Thanks,
Anne

I haven't looked meticulously closely yet, but this looks suspect:
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
anyObject() is a matcher and you're calling it after the replay. It's not used to produce any object. It's used to instruct EasyMock to allow any object. EasyMock is detecting that extra matcher but it is not harmful until the second test. At that point, the number of matchers that EasyMock has recorded but hasn't yet used (2) doesn't line up with the number of parameters expected for the second doXthing call (1).
You should be passing in real parameters to testCallingXthing (or a mock that is in replay mode). Try passing in null directly, or a real value like 2.

for me this failure (in my case 2 matchers expected, 4 recorded.) meant "you are mixing easymock and mockito in the same unit test, so accidentally calling easymock's notNull() method for a mockito argument. Which causes the failure but only if the tests are run in a certain order.

Try:
String testResult = mainThing.testCallingXthing(eq(EasyMock.anyLong()));
There are more refined matchers than anyObject(). These allow you to make type-based assertions about collaborators.
From the EasyMock documentation:
eq(X value)
Matches if the actual value is equals the expected value. Available for all primitive types and for objects.
anyBoolean(), anyByte(), anyChar(), anyDouble(), anyFloat(), anyInt(), anyLong(), anyObject(), anyShort()

You should reset mock after each test method to get rid of this problem. Adding below code will solve this problem.
#After
public void after(){
EasyMock.reset(mockXthing)
}

Verifying partially ordered method invocations in JMockit

I'm trying to write a unit test (using JMockit) that verifies that methods are called according to a partial order. The specific use case is ensuring that certain operations are called inside a transaction, but more generally I want to verify something like this:
Method beginTransaction is called.
Methods operation1 through to operationN are called in any order.
Method endTransaction is called.
Method someOtherOperation is called some time before, during or after the transaction.
The Expectations and Verifications APIs don't seem to be able to handle this requirement.
If I have a #Mocked BusinessObject bo I can verify that the right methods are called (in any order) with this:
new Verifications() {{
bo.beginTransaction();
bo.endTransaction();
bo.operation1();
bo.operation2();
bo.someOtherOperation();
}};
optionally making it a FullVerifications to check that there are no other side-effects.
To check the ordering constraints I can do something like this:
new VerificationsInOrder() {{
bo.beginTransaction();
unverifiedInvocations();
bo.endTransaction();
}};
but this does not handle the someOtherOperation case. I can't replace the unverifiedInvocations with bo.operation1(); bo.operation2() because that puts a total ordering on the invocations. A correct implementation of the business method could call bo.operation2(); bo.operation1().
If I make it:
new VerificationsInOrder() {{
unverifiedInvocations();
bo.beginTransaction();
unverifiedInvocations();
bo.endTransaction();
unverifiedInvocations();
}};
then I get a "No unverified invocations left" failure when someOtherOperation is called before the transaction. Trying bo.someOtherOperation(); minTimes = 0 also doesn't work.
So: Is there a clean way to specify partial ordering requirements on method calls using the Expectations/Verifications API in JMockIt? Or do I have to use a MockClass and manually keep track of invocations, a la:
#MockClass(realClass = BusinessObject.class)
public class MockBO {
private boolean op1Called = false;
private boolean op2Called = false;
private boolean beginCalled = false;
#Mock(invocations = 1)
public void operation1() {
op1Called = true;
}
#Mock(invocations = 1)
public void operation2() {
op2Called = true;
}
#Mock(invocations = 1)
public void someOtherOperation() {}
#Mock(invocations = 1)
public void beginTransaction() {
assertFalse(op1Called);
assertFalse(op2Called);
beginCalled = true;
}
#Mock(invocations = 1)
public void endTransaction() {
assertTrue(beginCalled);
assertTrue(op1Called);
assertTrue(op2Called);
}
}

if you really need such test then: don't use mocking library but create your own mock with state inside that can simply check the correct order of methods.
but testing order of invocations is usually a bad sign. my advice would be: don't test it, refactor. you should test your logic and results rather than a sequence of invocations. check if side effects are correct (database content, services interaction etc). if you test the sequence then your test is basically exact copy of your production code. so what's the added value of such test? and such test is also very fragile (as any duplication).
maybe you should make your code looks like that:
beginTransaction()
doTransactionalStuff()
endTransaction()
doNonTransactionalStuff()

From my usage of jmockit, I believe the answer is no even in the latest version 1.49.
You can implement this type of advanced verification using a MockUp extension with some internal fields to keep track of which functions get called, when, and in what order.
For example, I implemented a simple MockUp to track method call counts. The purpose of this example is real, for where the Verifications and Expectations times fields did not work when mocking a ThreadGroup (useful for other sensitive types as well):
public class CalledCheckMockUp<T> extends MockUp<T>
{
private Map<String, Boolean> calledMap = Maps.newHashMap();
private Map<String, AtomicInteger> calledCountMap = Maps.newHashMap();
public void markAsCalled(String methodCalled)
{
if (methodCalled == null)
{
Log.logWarning("Caller attempted to mark a method string" +
" that is null as called, this is surely" +
" either a logic error or an unhandled edge" +
" case.");
}
else
{
calledMap.put(methodCalled, Boolean.TRUE);
calledCountMap.putIfAbsent(methodCalled, new AtomicInteger()).
incrementAndGet();
}
}
public int methodCallCount(String method)
{
return calledCountMap.putIfAbsent(method, new AtomicInteger()).get();
}
public boolean wasMethodCalled(String method)
{
if (method == null)
{
Log.logWarning("Caller attempted to mark a method string" +
" that is null as called, this is surely" +
" either a logic error or an unhandled edge" +
" case.");
return false;
}
return calledMap.containsKey(method) ? calledMap.get(method) :
Boolean.FALSE;
}
}
With usage like the following, where cut1 is a dynamic proxy type that wraps an actual ThreadGroup:
String methodId = "activeCount";
CalledCheckMockUp<ThreadGroup> calledChecker = new CalledCheckMockUp<ThreadGroup>()
{
#Mock
public int activeCount()
{
markAsCalled(methodId);
return active;
}
};
. . .
int callCount = 0;
int activeCount = cut1.activeCount();
callCount += 1;
Assertions.assertTrue(calledChecker.wasMethodCalled(methodId));
Assertions.assertEquals(callCount, calledChecker.methodCallCount(methodId));
I know question is old and this example doesn't fit OP's use case exactly, but hoping it may help guide others to a potential solution that come looking (or the OP, god-forbid this is still unsolved for an important use case, which is unlikely).
Given the complexity of what OP is trying to do, it may help to override the $advice method in your custom MockUp to ease differentiating and recording method calls. Docs here: Applying AOP-style advice.

PowerMock, mockito, verify static method

I'm trying to get PowerMock to work with mockito, and I'm following the documentation here: http://code.google.com/p/powermock/wiki/MockitoUsage13.
To simplify a bit, lets say that I have a static method:
StaticObj.put(String key, String val) { ... }
And the class to be tested does something like this:
public class ClassToTest {
public void doSomething(Params p) {
if (StringUtils.isNotBlank(p.getK()) StaticObj.put("k1", p.getK());
if (StringUtils.isNotBlank(p.getX()) StaticObj.put("x1", p.getX());
}
}
In my unit test I'd like to verify that StaticObj.put is called for K and X when they are not blank or null, so I do something like this:
public void testNormalCase() {
// assume that mocking setup for statics already happened in some #Before function..
Params params = new Params("k", "x");
ClassToTest classToTest = new ClassToTest();
classToTest.doSomething(params);
// now I want to verify:
PowerMockito.verifyStatic(times(1));
StaticObj.put("k1", "k1");
PowerMockito.verifyStatic(times(1));
StaticObj.put("x1", "x");
}
This works, and it's what I'd expect. What doesn't work, is if I comment out the verification for K, then the verification for X fails! The error message indicates that ("x1", "x") is expected but got ("k1", "k"). Why is this? Am I not coding this correctly?
Also it leads me to believe that the following type of test, which passes, might pass for the wrong reason entirely:
public void testOtherCase() {
// assume that mocking setup for statics already happened in some #Before function..
Params params = new Params("k", null);
ClassToTest classToTest = new ClassToTest();
classToTest.doSomething();
// now I want to verify:
PowerMockito.verifyStatic(never());
StaticObj.put(eq("x1"), anyString());
}
E.g. I wonder if powermock sees "k1", decides that "x1" was never called, and passes. (?)
To state it generally, I have a static method that is called N times (where N changes depending on the input params). And I want to verify that it was called in the correct cases (which can be determined by input params). It seems like powermock doesn't handle this well, unless I misunderstand.
Thanks for any ideas!

I read this question and the issue carefully but not sure if I understood them clearly - From my understanding, it's correct that powermock raise the exception when you pass k and x but only verify k.
Because you are mocking the static method StaticObj.put, when you pass parameter k and x and verify it with
PowerMockito.verifyStatic(times(1));
StaticObj.put("k1", "k1");
PowerMockito.verifyStatic(times(1));
StaticObj.put("x1", "x");
This should work. And when you verify parameter k and x with verification for k is commented out.
// PowerMockito.verifyStatic(times(1));
// StaticObj.put("k1", "k1");
PowerMockito.verifyStatic(times(1));
StaticObj.put("x1", "x");
Powermock will get the invocation with put("k1"...) first apparently, so the verification of x will raise an error. Your verification process is sequenced.

I don't know as of which version, but PowerMockito.verifyStatic(VerificationMode) is deprecated. Just wanted to point that out to anyone else finding this years after the last post.