I am experiencing a certain bug in JUnit/JMock. I am trying to mock a couple of objects and then assert that all expectations is satisfied. I am running a simple test such as :
#Test
public void sellingPutOptionProductDoesNotCauseDisclosure() throws PositionVerificationException, DataLoadException, MissingPriceException {
final OptionProduct optionProduct = setupOptionProduct();
context.assertIsSatisfied();
}
private OptionProduct setupOptionProduct() {
final Option optionProduct = context.mock(Option.class);
context.checking(new Expectations() {
{
oneOf(optionProduct).getUnderlyingProduct();
will(returnValue(new Object()));
}
});
return optionProduct;
}
The Option is an object and I am using Mockery like this:
context = new Mockery() {
{
setImposteriser(ClassImposteriser.INSTANCE);
}
};
If I run the above test I am gettiing Test passed, where JVM does not terminate and the last print out in console is:
Exception in thread "main"
ANy ideas what might be causing this?
I'm a little late to the party, but I just had a similar problem and was able to track down the cause.
Normally, when a method is called that is not part of the expectations, JMock builds the log telling you what was expected that was not found. In my case, it tried to create that log message after encountering an unexpected call. The act of creating the log message threw an exception, which got JMock all confused and it reported that the test passed, when it had actually failed.
In my case, the reason that the exception was thrown was that one of the parameters being passed to the "unexpected" function call was an instance of a class. That class was initialized, in part, with mocked objects. When JMock was trying to build the error message to tell me about the unexpected invocation, it needed to describe the parameter. Usually it will say something like unexpected invocation: myobject.myMethod(param1,param2).
Because param1, in my case, contained member variables that were mock objects, and the param1 class did not define toString(), the default, Object.toString() was used.
Object.toString() is defined as: return getClass().getName() + "#" + Integer.toHexString(hashCode());
My implementation of hashcode for param1 was using some of those mocked objects. Those calls on the mocked objects to calculate the hashcode were 'unexpected' invocations themselves, leading JMock to throw an exception when trying to describe the test failure to me.
Unfortunately, instead of recognizing this condition and still reporting as much of the failure as it could, JMock seemed to give up altogether and report the test as passing and offhandedly mention Exception in thread "main."
To see if this is happening to you, I recommend you check the parameters involved in the offending function calls. If any of them are classes, not interfaces, you should see if their use of equals / hashcode / toString use any suspicious calls to member variables that may not be playing nice with how your scenario is being mocked.
Related
I am trying to write a Unit Test for the following service method:
public CommandDTO update(UUID uuid, QuantityRequest request) {
Quantity quantity = quantityRepository.findByUuid(uuid)
.orElseThrow(() -> new EntityNotFoundException(QUANTITY));
Quantity updated = saveQuantity(quantity, request);
return CommandDTO.builder().uuid(updated.getUuid()).build();
}
private Quantity saveQuantity(Quantity quantity, QuantityRequest request) {
//map fields (code omitted for brevity)
return quantityRepository.save(quantity);
}
I use ArgumentCaptor so that I catch the quantity parameter in the private method that my service method calls: quantityRepository.save(quantity).
#Test
public void test() {
Quantity quantity = new Quantity();
QuantityRequest request = new QuantityRequest();
request.setQuantity(100);
when(quantityRepository.findByUuid(uuid)).thenReturn(Optional.of(quantity));
// It seems to be meaningless this stubbing. because I already stb it in verify method below
when(quantityRepository.save(any())).thenReturn(quantity);
quantityService.update(uuid, request);
verify(quantityRepository).save(quantityCaptor.capture());
Quantity captured = quantityCaptor.getValue();
// assertions...
}
The test is working, but if I remove when(quantityRepository.save(any())).thenReturn(quantity); line, it throws "null pointer exception error" because in this case updated parameter in the update method is null. So, do I have to use the mentioned stubbing in the when() method? I think I do not need and the verify already perform that task via verify(quantityRepository).save(quantityCaptor.capture()). Is that true?
The problem lies in the following lines:
Quantity updated = saveQuantity(quantity, request);
return CommandDTO.builder().uuid(updated.getUuid()).build();
Which is essentially the same as:
Quantity updated = quantityRepository.save(quantity)
return CommandDTO.builder().uuid(updated.getUuid()).build();
The stubbing is necessary, because you're expecting the save method to return something, when you call updated.getUuid().
Without the stub, updated is null and your call results in a NullPointerException.
No, you need the stubbing here. You can't delete the when(save) call because your test depends on what return value you have for save. However, you're on the right track to question whether any given thing needs to be both stubbed and verified.
You're right that it's often redundant to verify things that you've stubbed, and the docs to verify tell you so:
Although it is possible to verify a stubbed invocation, usually it's just redundant. Let's say you've stubbed foo.bar(). If your code cares what foo.bar() returns then something else breaks(often before even verify() gets executed). If your code doesn't care what foo.bar() returns then it should not be stubbed.
And as attributed to Aaron Jensen, quoted in "Asking and Telling" by Mockito's original author Szczepan Faber:
If you’re verifying you don’t need to stub unless of course that method returns something that is critical to the flow of your test (or code), in which case you don’t really need to verify, because the flow would have verified.
In general, if you've stubbed something, then you make your test assertions at the end and you don't need to test that the method was called—the assertions would fail. And if you verify that a method was called but nobody cares about the result—or the result is passed directly back to the caller—then you might not need to stub, because Mockito's default return values like 0 and null should work fine.
Here, neither of those are true: A default null value causes an NPE, and you need to verify because that's the only supported way to get a value out of your ArgumentCaptor. This means that you call verify, but not for the sake of actual verification, but rather to get the value out of ArgumentCaptor. Part of the verify call is redundant, but there isn't another practical way to get to the necessary ArgumentCaptor part, so your code is fine with both when and verify.
I have a class which contains the following three methods:
void add(Service... objs)
void add(Collection<Service> objs)
void add(Stream<Service> objs)
As you might expect, these all support the adding of zero or more objects, which can be specified individually or as part of an array, a collection or a stream. The first two variants create a stream from their arguments, and pass them to the third variant which actually does the adding.
In testing an object that uses this class, I have created a Mockito mock object to represent an instance of this class, using Spring's #MockBean annotation. I can see in the debugger that the object under test contains the mock object, and that the call I am expecting (with a single argument of type Service) is being addressed to the mock. Because the method that should be called is the first variant (the varargs one), and I know that varargs parameters are a little tricky, I coded the test to check that the mock is called with the correct parameter as follows:
ArgumentCaptor<Service> captor = ArgumentCaptor.forClass(Service.class);
verify(theMock).add(captor.capture());
assertThat(captor.getAllValues()).containsExactly(expectedService);
However, when I run this code the assertion fails because the List returned by captor.getAllValues() contains not a Service, but a Stream: the failure message says:
java.lang.AssertionError:
Expecting:
<[java.util.stream.ReferencePipeline$Head#2cfe272f]>
to contain exactly (and in same order):
<[com.xxx.data.Service#37c5]>
but some elements were not found:
<[com.xxx.data.Service#37c5]>
and others were not expected:
<[java.util.stream.ReferencePipeline$Head#2cfe272f]>
When I run the code in the debugger, I can see that the call from the object under test to add(Service...) invokes the real implementation; this invokes add(Stream<Service>) and it is that call which is intercepted by the mock. That explains why I am seeing the failure, but I don't understand why the mock is failing to intercept the original call, or what I can do to make it do so.
Update your ArgumentCaptor to accept Service[]
ArgumentCaptor<Service[]> serviceCaptor = ArgumentCaptor.forClass(Service[].class);
And assert
Service[] actualServices = serviceCaptor.getAllValues();
assertEquals(actualServices.length, 1);
assertEquals(actualServices[0], service);
And it's best practice to use ErrorCollector in Junit to assert more than one
assert and SoftAssect in Testng and call after your assertion softAssert.assertAll()
I have come up with a workaround to the problem, but the actual problem still exists and I think it is probably a Mockito bug (raised as https://github.com/mockito/mockito/issues/1929).
The workaround is to add this method to my test class. I've added a generic method because it is not just a call to the add() method that has the problem, but also a call to a similar overloaded remove() method that takes String arguments.
private <T, V> void verifyCall(T mock, BiConsumer<T, V> call,
V expectedArg, Class<V> type)
{
ArgumentCaptor<V> captor = ArgumentCaptor.forClass(type);
call.accept(verify(mock), captor.capture());
List<?> values = captor.getAllValues();
try {
assertThat(values.get(0)).isEqualTo(expectedArg);
} catch (AssertionFailedError ex) {
assertThat((Stream<V>) values.get(0)).containsExactly(expectedArg);
}
}
This should work whether the call intercepted by the mock was to the varargs variant of the method (as it should be) - in which case the assertion in the body of the try block will not throw an exception - or to the Stream variant (as it currently is) - in which case the assertion in the body will throw an exception and the assertion in the catch block will be executed.
Then, when I want to verify that my mock's add() method was called with the expected Service object, I do so with:
verifyCall(theMock, Datastore::add, expectedService, Service.class);
And similarly, for the remove() method:
verifyCall(theMock, Datastore::remove, expectedDeletedKey, String.class);
Very pleasingly, when I finally got this working the test failed because I had made a mistake in the method under test. Which made it all worthwhile.
D'oh. I failed to spot that the two varargs methods were declared as final. Removed this and it all works as expected.
I'm creating a database wrapper. This wrapper has a series of states:
it is created
you are signing in
you are signed id
you are connected to a specific table
Methods on my database wrapper can trigger the state transitions.
But the database interactions are asynchronous, so you don't get immediate feedback. Instead, you can register as a listener for onStateChanged events.
I'm using mockito to test the transitions, and so far I'm really amazed at how clean the use of InOrder and verify are.
But now I'm running into a problem. I'm trying to test a forbidden transition. My Database wrapper will catch that you're not in the correct state and throw an AssertionException. This Exception let's the testing crash. I'm trying to register this as an expected exception with Mockito, but I can't figure out how to do this in the correct way.
The following code is all in one function, the exception is expected by JUnit:
#Test(expected = AssertionError.class)
public void cantCreateTwice() throws Exception{
...
First I'm creating a connection, and I'm setting up a StateChangeListener which attempts the forbidden transition:
final FirebaseConnection connection = new FirebaseConnection();
StateChangeListener stateChangeListener = new StateChangeListener() {
#Override
public void onStateChanged(DBState newState) {
if(newState==DBState.SignedIn){
connection.createParty();
connection.createParty(); // forbidden
}
}
};
Next, I'm registering this listener and a mock:
StateChangeListener mockedListener = mock(StateChangeListener.class);
connection.addStateChangeListener(stateChangeListener);
connection.addStateChangeListener(mockedListener);
Then I tell Mockito about the expected exception and start the test:
doThrow(new AssertionError()).when(mockedListener).onStateChanged(DBState.SignedIn);
connection.signIn();
verify(mockedListener, times(1)).onStateChanged(DBState.SigninIn);
verify(mockedListener, times(1)).onStateChanged(DBState.SignedIn);
This will result in:
Test running failed: Instrumentation run failed due to 'Process
crashed.'
The exception that crashed the process is the expected AssertionError:
java.lang.AssertionError: the connection is not in the necessary state
I think the problem is that my exception is not triggered when a function is called on the mocked listener, but on the anonymous implementation.
But I need the anonymous implementation to trigger the forbidden functionality.
I also need the verify since my callbacks are asynchronous. If I remove the verify then the test will simply return immediately without any exception being thrown.
It seems to me that I somehow need to convert my anonymous implementation into a Mock object that can be supervised by Mockito. But mock() doesn't work on anonymous classes. And I think it's not correct to have an individual class for every snippet of logic in my test cases.
How can I tell Mockito to expect this exception ?
EDIT:
I think I've found a way, I'm using verify with a timeout:
verify(mockedListener, timeout(1000).times(1)).onStateChanged(DBState.SigninIn);
verify(mockedListener, timeout(1000).times(1)).onStateChanged(DBState.SignedIn);
connection.createParty();
connection.createParty();
This blocks until the sign-in was completed. Then it tries to create a party twice, which fails and throws the expected exception.
But I think I'm now basically misusing a feature of Mockito for synchronization. Is this the correct way to use Mockito for my problem ?
I'm pondering on exception handling and unit tests best practices because we're trying to get some code best practices in place.
A previous article regarding best practices, found on our company wiki, stated "Do not use try/catch, but use Junit4 #Test(expect=MyException.class)", without further information. I'm not convinced.
Many of our custom exception have an Enum in order to identify the failure cause.
As a result, I would rather see a test like :
#Test
public void testDoSomethingFailsBecauseZzz() {
try{
doSomething();
} catch(OurCustomException e){
assertEquals("Omg it failed, but not like we planned", FailureEnum.ZZZ, e.getFailure());
}
}
than :
#Test(expected = OurCustomException.class)
public void testDoSomethingFailsBecauseZzz() {
doSomething();
}
when doSomethig() looks like :
public void doSomething throws OurCustomException {
if(Aaa) {
throw OurCustomException(FailureEnum.AAA);
}
if(Zzz) {
throw OurCustomException(FailureEnum.ZZZ);
}
// ...
}
On a side note, I am more than convinced that on some cases #Test(expected=blabla.class) IS the best choice (for example when the exception is precise and there can be no doubt about what's causing it).
Am I missing something here or should I push the use of try/catch when necessary ?
It sounds like your enum is being used as an alternative to an exception hierarchy? Perhaps if you had an exception hierarchy the #Test(expected=XYZ.class) would become more useful?
If you simply want to check that an exception of a certain type was thrown, use the annotation's expected property.
If you want to check properties of the thrown exception (e.g. the message, or a custom member value), catch it in the test and make assertions.
In your case, it seems like you want the latter (to assert that the exception has a certain FailureEnum value); there's nothing wrong with using the try/catch.
The generalization that you should "not use try/catch" (interpreted as "never") is bunk.
Jeff is right though; the organization of your exception hierarchy is suspect. However, you seem to recognize this. :)
If you want to check the raw exception type, then the expected method is appropriate. Otherwise, if you need to test something about the exception (and regardless of the enum weirdness testing the message content is common) you can do the try catch, but that is a bit old-school. The new JUnit way to do it is with a MethodRule. The one that comes in the API (ExpectedException) is about testing the message specifically, but you can easily look at the code and adapt that implementation to check for failure enums.
In your special case, you want to test (1) if the expected exception type is thrown and (2) if the error number is correct, because the method can thrown the same exception with different types.
This requires an inspection of the exception object. But, you can stick to the recommendation and verify that the right exception has been thrown:
#Test(expected = OurCustomException.class)
public void testDoSomethingFailsBecauseZzz() {
try {
doSomething();
} catch (OurCustomException e) {
if (e.getFailureEnum.equals(FailureEnum.ZZZ)) // use *your* method here
throw e;
fail("Catched OurCostomException with unexpected failure number: "
+ e.getFailureEnum().getValue()); // again: your enum method here
}
}
This pattern will eat the unexpected exception and make the test fail.
Edit
Changed it because I missed the obvious: we can make a test case fail and capture a message. So now: the test passes, if the expected exception with the expected error code is thrown. If the test fails because we got an unexpected error, then we can read the error code.
I came across this when searching how to handle exceptions.
As #Yishai mentioned, the preferred way to expect exceptions is using JUnit rules and ExpectedException.
When using #Test(expected=SomeException.class) a test method will pass if the exception is thrown anywhere in the method.
When you use ExpectedException:
#Test
public void testException()
{
// If SomeException is thrown here, the test will fail.
expectedException.expect(SomeException.class);
// If SomeException is thrown here, the test will pass.
}
You can also test:
an expected message: ExpectedException.expectMessage();
an expected cause: expectedException.expectCause().
As a side note: I don't think using enums for exception messages/causes is good practice. (Please correct me if I'm wrong.)
I made catch-exception because I was facing the same problem as you did, Stph.
With catch-exception your code could look like this:
#Test
public void testDoSomethingFailsBecauseZzz() {
verifyException(myObj, OurCustomException.class).doSomething();
assertEquals("Omg it failed, but not like we planned", FailureEnum.ZZZ,
((OurCustomException)caughtException()).getFailure() ;
}
Is this possible?
I tried with EasyMock.expectLastCall().times(0); but EasyMock complains that times must be >=1
You could use .andThrow(new AssertionFailedError()).anyTimes(); - this is the same exception that Assert.fail() throws, but is less verbose than making an Answer.
with easymock 3.0, you need to add a .anyTimes() on the expectLastCall or the test will fail:
Expectation failure on verify: myMethod(): expected: 1, actual: 0`
based on nkr1pt example:
expectLastCall().andAnswer(new IAnswer() {
public Object answer() {
Assert.assertFail();
return null;
}
}).anyTimes();
The fact that some method is not called is controlled by Mock or StrictMock. They will throw an exception, when that not recorded method is called. This problem occurs only when using NiceMocks, where default values are returned when calling for not recorded methods.
So a solution can be not to use NiceMocks.
Looks like a bug to me. The internal class Range does not allow to set a maximum less than 1.
Couldn't you mock that method, and just call Assert.fail() ?
If you expect your method not to be called then just don't record it. But I agree it won't work with a nice mock.
I managed to come up with a solution:
expectLastCall().andAnswer(new IAnswer() {
public Object answer() {
Assert.assertFail();
return null;
}
});