I have a Unit Test in a Java app and I get error while executing the following line:
when(Arrays.asList(Locale.getISOCountries()).contains(countryCode)).thenReturn(true);
The error message does not mention mocking, but I think the problem is related to not mocking
Arrays.asList(Locale.getISOCountries() part in the when method. So, how should I treat this line? Normally I mock services and repositoories, but ı am not sure if I should mock this part like #Mock Locale locale;. I tried it but not worked.
Here is the error message:
org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
Boolean cannot be returned by toString()
toString() should return String
Note: I also tried doReturn(true).when(Arrays.asList(Locale.getISOCountries()).contains(countryCode)); but not worked.
TL;DR: don't try to mock this. For one thing, you can't; for another, you shouldn't.
It is best to use the default behavior. Remove the line where you attempt to configure this behavior with Mockito.
If you insist that you need to be able to use an arbitrary list of countries, see the section on "dependency injection" below, noting the caveat mentioned.
Attempting to mock the return value of Arrays.asList(Locale.getISOCountries()).contains(countryCode) would specifically mean that you are trying to alter the behavior of the List<String> returned by Arrays.asList when given a particular String[] as an argument.
You can't with Mockito, because it isn't magic: it doesn't allow replacement of behavior for arbitrary expressions.
The way something like when(aMock.method(123)).thenReturn("hello") works is that the mock object - aMock - records that method was invoked with an argument 123. This information is pushed onto a stack, from which the when method is able to retrieve it and deal with it.
The pushing-onto-the-stack is only done if aMock is an object that Mockito has created: Mockito implements the methods of an interface/overrides the methods of a class to do this recording.
None of the objects involved in Arrays.asList(Locale.getISOCountries()).contains("GB") were created by Mockito. As such, none of these objects have methods which capture the invocation and push it onto the stack; Mockito basically can't see that anything is going on, so when the when call comes, it's just using whatever state is hanging around in Mockito. The error message shown:
org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
Boolean cannot be returned by toString()
toString() should return String
indicates that it thinks you are configuring the behavior of a toString method, nothing obviously to do with Arrays.asList, Locale.getISOCountries() or List.contains.
For the same reason, you can't mock the return value of Arrays.asList(Locale.getISOCountries()) or Locale.getISOCountries(): they're just not objects Mockito knows anything about.
So, that's why this mocking doesn't work. On to why you don't want to do it in the first place:
List.contains has specific semantics, namely that it returns true if and only if the argument is in the list. There are implications of this, such as aList.contains(o) == true implying that a call to aList.indexOf(o) would have returned a non-negative value.
This means that the consequences of mocking contains would be either:
You have to also configure the mocking of other methods on the list such that the behavior of the list is consistent with that result of List.contains (so, indexOf, subList, iterator etc) - and, how the mock should behave if you were to set the element equal to o to something else (because Arrays.asList allows that);
or
You don't configure the other mocking, and your List doesn't have behavior consistent with an actual List, which will have unpredictable effects on the behavior of your code.
But you don't actually have to worry about doing 1. (which is good, because it would be essentially impossible to do correctly, e.g. such that copies of the list have the same contains behavior) or 2. (which is good, because introducing the unpredictability of a broken List is simply a bad idea): Arrays.asList has a perfectly-working implementation of the List interface; all you need to make sure is that the argument you pass in (in this case Locale.getISOCountries()) contains the element that you want.
Mocking of the return value of Arrays.asList(...).contains is neither necessary nor desirable.
So, the problem now is shifted from one of ensuring that Arrays.asList(Locale.getISOCountries()).contains(countryCode) to one of ensuring that Locale.getISOCountries() has at least one element that is equal to countryCode.
As stated in the Javadoc of Locale.getISOCountries():
Returns a list of all 2-letter country codes defined in ISO 3166
Given that GB (as was originally asked) is a 2-letter country code defined in ISO 3166, it will always (or, at least, until ISO3166 is changed) be an element of the array returned by Locale.getISOCountries().
Hence, the array returned by Locale.getISOCountries() is going to contain "GB", so Arrays.asList(Locale.getISOCountries()).contains("GB"), without actually doing anything.
But then there is the question of making Arrays.asList(Locale.getISOCountries()).contains(countryCode) true, for an arbitrary countryCode. By the argument above, you want to effect this only by ensuring that Locale.getISOCountries() has an element equal to countryCode.
If countryCode is another two-letter ISO-3166-2 country code, this already works, as in the GB case.
If countryCode is not an ISO country code, you absolutely should not want to make a method return it if that method is documented to return only ISO-3166-2 country codes, because this wouldn't happen in production code.
Mocking should not be used as a way to do arbitrary things in tests.
You only ever want a test to test things which can actually happen in production code. Ideally, you use "the real thing"; test doubles (of which a mock is one type) come into play only if using "the real thing" is hard because the real thing is slow, expensive, difficult to reproduce (e.g. an error condition like a network error or full disk) etc. But, the testing double should only be doing things you'd see for real.
So, even if you could mock Locale.getISOCountries() to ensure that it returned an array containing a non-ISO-3166-2 countryCode, you shouldn't, because this will never actually happen in production; and a test that tests something that cannot happen in production has very limited value in telling you something useful.
Actually, you can mock static methods like Locale.getISOCountries() using PowerMock; but changing the behavior of a static method is highly inadvisable, because it doesn't just change the behavior for you - it changes it for anybody who calls it. So, there could be unintended consequences in the behavior, both nearby and in the rest of the code.
For example:
when(Arrays.asList(Locale.getISOCountries())).thenReturn(Collections.singletonList(countryCode.toUpperCase(Locale.ENGLISH)));
Aside from changing the mutability semantics of the returned list (Arrays.asList allows set; Collections.singletonList doesn't), it is now inconsistent with other country-code-returning methods in the Locale class (e.g. getISOCountryCodes(Type). Chasing down and fixing all such inconsistencies is nigh-on impossible.
What if we could use PowerMock to mock the return value of Arrays.asList(Locale.getISOCountries()), i.e. a less generic use-case of the Locale.getISOCountries() method? This still suffers from the problems of unintended consequences - there could be other parts of the program which invoke Arrays.asList(Locale.getISOCountries()) where the mocked behavior isn't desirable.
What if we could use PowerMock to mock the return value of just one specific call to Arrays.asList(Locale.getISOCountries())? That's brittle, for example, if another call is added, you'd have to make sure the test is correctly updated, otherwise the behavior would be inconsistent between the calls.
There isn't a good way to win the PowerMock battle.
There are an awful lot of words here, but the key point is that there are really rather difficult-to-deal-with consequences of trying to use mocking inappropriately: using the actual behavior (without mocks) is best; but, if a mock has to be used, it should not behave in a way that the real code never will.
Fortunately, Mockito is stopping you from doing this; but hopefully this answer has given a thorough explanation as to why it was the wrong approach in the first place.
Dependency injection
With all of this said, there is a way to make your code work in the face of arbitrary country codes: dependency injection.
Whilst there are lots of DI frameworks (e.g. Guice, Spring) which introduce a lot of power (and complexity and horror), dependency injection simply means: passing things as arguments.
If, for example, the code in which you want Arrays.asList(Locale.getISOCountries()).contains(countryCode) to be true occurs in a method, inject the country list as a parameter to that method:
class MyClass {
void myMethod(List<String> countryCodes, String countryCode) {
if (countryCodes.contains(countryCode)) {
// ...
}
}
}
or make it a constructor parameter:
class MyClass {
private final List<String> countryCodes;
MyClass(List<String> countryCodes) {
// Defensive copy.
this.countryCodes = Collections.unmodifiableList(new ArrayList<>(countryCodes));
}
void myMethod(String countryCode) {
if (countryCodes.contains(countryCode)) {
// ...
}
}
}
In your production code, pass in Arrays.asList(Locale.getISOCountries()); in test code, pass in whatever list you like.
But still: beware of the interaction between this code and code which uses Locale.getISOCountries() and allied methods directly in tests. If there is a risk of such an interaction, it remains safer to write your tests using the static Local.getISOCountries().
Related
I never had to stub a call that accepts a functional interface/method reference as argument, so I just find out about the obvious incapacity for compare with "equals" these, aside from the reference comparison.
So how does one deal with this scenario during testing? Do we have to use any() either we want it or not?
Comparator<String> comparator = Comparator.naturalOrder();
when(myColaborator.isCalledWith(comparator)).thenReturn("foo"); //This is not gonna work as long as the real call does not use theat very same reference
It is necessary to verify its collaborator whether has recieved an exactly Comparator in a simple test case since such a communication protocol test in a book of GOOS describes whether they will work together correctly. and then skip checking the rest of tests by using any(Comparator.class). you can read it further in Martin's blog:
in the second test case is that I've relaxed the constraints on the expectation by using withAnyArguments. The reason for this is that the first test checks that the number is passed to the warehouse, so the second test need not repeat that element of the test. If the logic of the order needs to be changed later, then only one test will fail, easing the effort of migrating the tests. As it turns out I could have left withAnyArguments out entirely, as that is the default.
I want to check preconditions on a base class so that I know subtypes will always use constructor arguments that are valid.
Let's take as an example a constructor that:
takes 2 or more parameters
takes parameters of different types
for one parameter, it performs multiple checks (e.g. String is not null and not empty)
How would one best use the Guava preconditions approach in that case?
In a mock example like this: (this is contrived!)
protected AbstractException(String errorMessage, Throwable errorCause) {
super(errorMessage, errorCause);
checkNotNull(errorMessage,
ErrorMessage.MethodArgument.CANNOT_BE_NULL_CHECK, "errorMessage");
checkArgument(!errorMessage.isEmpty(),
ErrorMessage.MethodArgument.CANNOT_BE_EMPTY_STRING_CHECK,
"errorMessage");
checkNotNull(errorCause, ErrorMessage.MethodArgument.CANNOT_BE_NULL_CHECK,
"errorCause");
}
I end up calling super before checking the arguments because a call to super needs to be the first line of the method and, although I could do super(checkNoNull(errorMessage)), I cannot do the same wrapping using checkArgument because that returns void. So the dilemma is:
Where do I put the checks on all arguments? I don't want to create a Builder just for that
How do I "group" checks as in a fictitious checkStringNotNullAndNotEmpty()
Should I rather think about integration with matchers frameworks? (hamcrest, fest assertions...)
I use the odd-looking ErrorMessage.MethodArgument.CANNOT_BE_NULL_CHECK because the default throw does not include an error message so from the testing side I cannot recognise this as an argument validation failure rather than a "any" NPE?
Am I doing it all wrong?
This should have been a comment, but it's too long.
Calling super before the test is harmless provided that the super ctor doesn't do things which it shouldn't do anyway.
It could be prevented via a static builder method, you need no Builder. But it's not worth it.
I doubt that grouping tests is generally useful; if it was, then there would be a such method already. But if you need one such a concrete thing more than twice, then write your own; if it comes often, report it to the Guava team as an RFE.
I'm pretty sure, matchers are an overkill here as you're just creating an exception, i.e., something used only rarely (I hope). As your test is runtime only, it can't really help to catch errors. It would be nice, if you could statically ensure a "properly" constructed exception, but it's impossible in pure java.
More important: The exceptions you're throwing are probably inferior to the ones you'd get without all the checks. Imagine the user provides a cause and no message. That's bad, you think, but you replace it with an NPE lacking any cause. That's worse.
Look at Guava's Preconditions.format (package private). They could check the correct number of arguments first, but they do not. You can provide too few or too many, which is an error, but ignoring it is the best way to handle it.
Lets say I have a method:
someMethod(X anObject)
Where X is a type of object that is extremely complex. By this I mean it is not something one can easily instantiate on the fly. I need to somehow unit test someMethod, but I cannot so simply create an X object to put in as parameters.
So I first think to try and mock the object, but the problem I run in to is the someMethod function calls many methods of anObject, meaning this X object that is being mocked has a latge amount of functions that need to be called, and thus need to be mock-expected. To make things worse, these X object methods being called return more X objects, meaning I have to mock objects, to expect mock method calls, to return more mock objects.
Regarding this scenario I have a few questions, as I'm new to the concept of unit testing:
The lengthy unit test method aside, I find my unit test to not only be testing as to whether a method works or not, but also specifying the implementation (because I'm basically specifying most of the code that is being called in the method itself with the mock-expects). Is this a problem (mostly to the concept of unit testing itself)?
Is there any way to get around this, even if only to make my unit test methods be a lot less verbose and more maintainable?
I thought about taking a serialized X object from somewhere else, saving that, and then whenever I call my unit test method, I would unserialize my X object and run that as parameters. This is just some random idea I thought of the top of my head; does anyone actually do this?
In case anyone is wondering what exactly I'm doing, I'm using the IDebugContextListener interface to grab debugging information regarding data on a stackframe at a given step on the java debugger. The "X" that I am referring to are objects that are defined by the interface here, including objects such as IValue, IVariable, and IStackframe. All these variables are provided to me by the Java debugger during runtime.
The fact that you have this difficulty is a symptom of a design problem. When something is hard to test, refactor until it isn't hard to test.
If one object needs to call too many methods of another, then encapsulation is poor, and responsibilities are poorly placed. Presumably, the Single Responsibility Principle is not being followed. If code calls methods that return objects, and must call methods on those in turn, then the Law of Demeter is not being followed.
Your pain comes from the fact, that your method does not comply with the Single Responsibility Principle. Your method does a lot of things with X -- and X also sounds a too comlex. This makes testing very hard -- even with mocking.
Break your method down into manageble chunks, that only do one thing each.
Mockito seems like a pretty sweet stubbing/mocking framework for Java. The only problem is I can't find any concrete documentation on the best ways of using their API. Common methods used in tests include:
doXXX(???) : Stubber
when(T) : OngoingStubbing
then(T) : OngoingStubbing
verify(???) : T
given(T) : BDDOngoingStubbing
willXXX(???) : BDDStubber
When you see examples of Mockito in practice, you see code like:
when(yourMethod()).thenReturn(5);
From all the docs I've read, I've identified several "patterns" of Mockito "grammars" obtained from daisy-chaining these method calls together like the example above. Some common patterns I've found are:
When/Then: when(yourMethod()).thenReturn(5);
Given/Will: given(yourMethod()).willThrow(OutOfMemoryException.class);
Do/When: doReturn(7).when(yourMock.fizzBuzz());
Will/Given/Do: willReturn(any()).given(yourMethod()).doNothing();
Verify/Do: verify(yourMethod()).doThrow(SomeException.class);
What I'm choking on is how to select the right pattern/combination of method calls to model my test cases. It seems like you can daisy-chain these together in seemingly endless combos and I'm not sure what pattern is right for which problem.
Can some Mockito Guru help shed some light as to which patterns/combinations of Mockito methods are used for which types of test cases (and why)? Thanks in advance!
There are several disadvantages to the when/thenReturn, when/thenThrow and when/then syntaxes. For example,
In the case of when/thenReturn, if the return type is a generic with a
wildcard, and you wish to return a mock of the same type, you will be unable
to avoid a compile warning.
You can't use when/thenThrow and when/then for a void method.
You can't use these syntaxes on Mockito spies.
You can only call when once for each combination of mock object,
method and arguments, unless you call reset on the mock.
Calling when multiple times for one combination of mock
object and method, when you are using argument matchers, can lead to problems.
I find these cases difficult to remember. So instead of trying to keep track of when the
when/thenReturn, when/thenThrow and when/then syntaxes will and won't work, I prefer to avoid them completely, in favour of the doReturn/when, doThrow/when and doAnswer/when alternatives. That is to say, since you'll occasionally need doReturn/when, doThrow/when and doAnswer/when, and you can ALWAYS use these methods, there is no point in learning how to use when/thenReturn, when/thenThrow and when/then.
Note that doReturn, doThrow and doAnswer can be chained together in the same way as thenReturn, thenThrow and then. What they don't have is an option for returning several values (or throwing several exceptions, or running several answers) within a single call to doReturn, doThrow and doAnswer. But I find that I need to do this so seldom, that it doesn't really matter.
There's one more disadvantage to doReturn, which I consider insignificant. You don't get compile time checking of the type of its argument, like you do with when/thenReturn. So if you get the argument type wrong, you won't find out until you run your test. Frankly, I don't care.
In summary then, I have been using Mockito for more than two years, and I consider the consistent use of doReturn, doThrow and doAnswer to be a Mockito best practice. Other Mockito users disagree.
Mockito often have several ways of doing things.
I find myself using mostly:
// Setup expectations
when(object.method()).thenReturn(value);
when(object.method()).thenThrow(exception);
doThrow(exception).when(object.voidMethod());
// verify things
verify(object, times(2)).method();
verify(object, times(1)).voidMethod();
I've found that i can do 95% of what i need to with these three kinds of calls.
Also, what version of Mockito are you using? "given" and "will" constructs are not present in the latest version (1.9.0+)
However, there are cases where I want the return value or exception to respond to the input. In this case, you can use the Answer interface to inspect the method arguments and return an appropriate value.
public class ReturnFirstArg<T> implements Answer<T> {
public T answer(InvocationOnMock invocation) {
return invocation.getArguments()[0];
}
}
when(object.method(7)).thenAnswer(new ReturnFirstArg<Integer>());
The things in fact looks much simpler than you thought
REF: http://static.javadoc.io/org.mockito/mockito-core/2.7.12/org/mockito/Mockito.html
Verify:
In order to use Mockito, you need to understand one basic philosophy of Mockito: Stubbing and Verification is separated. Therefore, the "Verify/Do" you mentioned is in fact doing the "Verification" job, while the other 4 "grammars" is for stubbing. Stubbing define how the mock object will react in different situation. Verification is to make sure what the mocks is invoked as expected, in previous invocation to system under test (SUT).
When/Then , Given/Will:
Then it come to the "When" and "Given" families. You can simply treat them as aliases of each other. "Given" family is added in Mockito 1.8.x as to make it looks more aligned to BDD practices.
DoXxx:
In normal case we mostly use when(xxx).then(...) (and given(...).will(...) ) . However there are some case that syntax is not working. The most obvious case is when return type of the stubbed method is void. In such case when(mockObj.voidMethod()).thenThrow(anException) is not going to compile. As a workaround, an alternative syntax of Do/When is created, so you can write the previous line as doThrow(anException).when(mockObj.voidMethod())
If I have a basic accessor method that returns an ArrayList
What exactly would i test for it?
I am very inexperienced when it comes to testing.
That depends on how you expect the method to behave. For example: If someone has called the method and changed the list that was retrieved, do you want those changes to show up the next time the getter is called? Either way, test that behaviour. What does the getter return when the list would be empty? Null or an empty list? This should also be tested.
Typically writing explicit Junit tests for accessors is usually a little overkill (what are you testing? return foo;). Using a code coverage tool such as clover can help you target your testing efforts at your most complicated code first.
Interessting question for you:
How much should a unit test "test"
How to use Junit and Hibernate usefully
What should not be unit tested
Edit:
Added some of my favorite Questions here on Stackoverflow regarding JUnit and Unit Testing.
Always keeps in mind that test code is also code and for every 1,000 lines of code, you produce at least 4 bugs. So test what doesn't work and don't write tests for something that can't possibly break (like code generated by your IDE). If it does break, write a test :)
In general unit tests should test that your method does what it states it should do.
If your method returns an arraylist your basic test is to assert that an arraylist is indeed returned when it is called.
The next level of detail in the test is to check is the arraylist constructed correctly? Have the values you expect in it been filled correctly? If it's supposed to be an empty list, is that the case?
Now you have your "sunny day" case (i.e the method works under normal conditions) you should add some negative (or "rainy day") conditions if appropriate. If the method accepts a length for the array, what if you pass in a negative number or a int.Max etc.
As stated in another answer, this is probably overkill for a simple accessor, but the principles apply to any unit tests you need to write.
Depends on your requirements. You might test:
If the return value is not null
If the returned collection is not empty
If the returned collection is modifiable/unmodifiable
If the returned collection is sorted
If the returned collection contains all expected values
If the accessor method does not throw a runtime exception
But, as I said, it depends on the requirements, it depends on the 'kind' of collection you expect when you call the accessor. Maybe you allow setting the list to null but create an empty list. The the test could make sure that you really get an empty list when you set the list to null.
Hope it helps to give you an idea!