We Keep Coding

Mockito verify the last call on a mocked object

I have a bit of logic that needs to be tested such as:
{
...
A.add("1");
...
A.add("what ever");
...
A.add("2");
A.delete("5");
...
}
I have already mocked A in my test and I can test the add method is called once on argument ("2") such as:
Mockito.verify(mockedA).add("2");
My question is how can I test if I can verify the last call on method add is add("2") instead of other arguments.
Since the test above can't catch if somebody by accident adds another call such as add("3") in the last. Please notice that we don't care about other method invocations on A again afterwards. We also don't care about the times of the method called, the sequence of the methods called. The key point here is if we can verify the last true argument on a certain method of a certain mockedObject.
If you ask why do you need such functionality, I'd say in real world we might need to handle some logic that set something and the last set wins, and in order to avoid someone by accident set some other thing unexpected and I'd like to use our UT to catch this. And in order not to make the test too complex and neat, so I only expect to verify the last call on a certain method of a object instead verify something like order/noMoreInteractions/AtMostTimes and so on.

About the order of the invocations
By default, Mockito.verify() doesn't matter of the invocation order.
To take it into consideration, wrap the mock in an InOrder instance and perform the invocation verification on this instance.
About the no more interations
If the mock is no more invoked after the methods that you want to verify, you could use Mockito.verifyNoMoreInteractions(Object... mocks) that checks if any of given mocks has any unverified interaction such as :
InOrder inOrder = Mockito.inOrder(mockedA);
inOrder.verify(mockedA).add("1");
inOrder.verify(mockedA).add("2");
Mockito.verifyNoMoreInteractions(mockedA);
If the mock may still be invoked after the methods that you want to verify, you could add after your verifies an invocation to verify(T mock, VerificationMode mode) by passing a VerificationMode that checks that at most 2 invocations were performed.
InOrder inOrder = Mockito.inOrder(mockedA);
inOrder.verify(mockedA).add("1");
inOrder.verify(mockedA).add("2");
Mockito.verify(mockedA, Mockito.atMost(2)).add(Mockito.anyString());
A warning about your think and this way of mocking
Since the test above can't catch if somebody by accident adds another
call such as add("3") in the last.
Mockito provides a powerful and broad toolkit to work with mocks. Some features such as verify and more particularly verify that no more interaction was detected about a mock or a specific method of the mock make your test more complex to read and to maintain.
As well as, currently you want to check that the invocations on a mock were performed in a specific order. But you generally want to use these checks only as required that is according to the business/logic scenarios, not technical invocations.
For example supposing that in the tested method you have a case where for business reasons the mocked method is invoked 3 times and another case where the mocked method is invoked 2 times. It could make sense to check that it is only invoked 2 times and not more in the case with two expected invocations.
But in a general way, you should be cautious that your unit test doesn't overuse mocking verify that could look like as a assertion on the description of the flow and not a assertion on the behavior/logic.

Thanks #staszko032, inspired by the ArgumentCaptor, instead of getAllValues and verify the sequence, we can use getValue of captor since captor's getValue always get the last true argument. We can do it like this:
ArgumentCaptor<String> captor = ArgumentCaptor.forClass(String.class);
Mockito.verify(mockedA, Mockito.atLeastOnce()).add(captor.capture());
Assert.assertEquals("2", captor.getValue());

What this looks like to me is that you are mocking a data class. In my experience it's better to leave (stateful) data classes and mock (stateless) services. This way, you can verify that the method under test produces the correct data, rather than just verifying a series of invocations. Along with testdata builders (making it easy to instantiate your data classes with some default state, using builder pattern for instance), it becomes real easy to write tests.
If you do need to mock, the only way to test what you want is to use InOrder, and verify each of the invocations on the mock, and end with verifyNoMoreInteractions.

I've ended up implementing a new VerificationMode to do this. Here's the code in case it's useful to anyone.
class MostRecently implements VerificationMode {
#Override
public void verify(VerificationData data) {
MatchableInvocation target = data.getTarget();
List<Invocation> invocations = data.getAllInvocations();
if (invocations.isEmpty()) {
throw wantedButNotInvoked(target);
}
List<Invocation> targets = findInvocations(invocations, target);
if (targets.isEmpty()) {
throw wantedButNotInvoked(target);
}
Invocation lastInvocation = invocations.get(invocations.size() - 1);
if (target.matches(lastInvocation)) {
return;
}
ListIterator<Invocation> iterator = invocations.listIterator(invocations.size());
Invocation previous = iterator.previous();
Invocation undesired = previous;
while (!target.matches(previous)) {
undesired = previous;
previous = iterator.previous();
}
Invocation lastGoodInvocation = previous;
throw new MockitoAssertionError(join(
"Wanted most recent on '" + lastGoodInvocation.getMock() + "' to be " + lastGoodInvocation.getMethod(),
"No more interactions wanted after " + lastGoodInvocation.getLocation(),
"but found this interaction on mock '"
+ MockUtil.getMockName(undesired.getMock()) + "':",
undesired.getLocation(),
allLocations(invocations)));
}
static String allLocations(List<Invocation> invocations) {
StringBuilder sb = new StringBuilder("***\nFor your reference, here is the list of all invocations.\n");
int counter = 0;
for (Invocation i : invocations) {
sb.append(++counter).append(". ");
sb.append(i.getLocation()).append("\n");
}
return sb.toString();
}
}

Is my unit test violating "test should do one thing principle"?

I wrote some unit tests around my class and at the end I am confused whether to combine these tests as their reason of failure will be same.
PatternBuilder class
public class PatternBuilder {
private static final String PATTERN_INITIALS = "#0.%s";
String buildPatternFromScale(int scale) {
return String.format(Locale.ENGLISH, PATTERN_INITIALS, StringUtils.repeat("0", scale));
}
}
Unit tests for the above implementation
#RunWith(Parameterized.class)
public class PatternBuilderTest {
private PatternBuilder patternBuilder;
#Parameterized.Parameter
public int scale;
#Parameterized.Parameter(1)
public String expectedPattern;
#Before
public void setUp() {
patternBuilder = new PatternBuilder();
}
#Parameterized.Parameters(name = "buildPatternFromScale({0}) = {1}")
public static Collection<Object[]> data() {
return Arrays.asList(new Object[][]{
{1, "#0.0"},
{2, "#0.00"},
{5, "#0.00000"},
});
}
#Test
public void testShouldVerifyThatNonNullPatternIsBuilt() {
//given
//when
String pattern = patternBuilder.buildPatternFromScale(scale);
//then
assertThat(pattern, is(notNullValue()));
}
#Test
public void testShouldVerifyThatCorrectPatternOfSpecifiedScaleShouldBeCreated() {
//when
String pattern = patternBuilder.buildPatternFromScale(scale);
//then
assertThat(pattern, is(equalTo(expectedPattern)));
}
}
Will I violate the 'test should do only one thing' if I combine first test with second such that it asserts for no-null-value and correct pattern string ?
Effectively the one single test will then have following two assertions-
assertThat(pattern, is(notNullValue()));
assertThat(pattern, is(equalTo(expectedPattern)));
Reason - I am thinking to combine them both because if null pattern string is created than it will be fail both tests for one single reason

Will I violate the 'test should do only one thing' if I combine first
test with second such that it asserts for no-null-value and correct
pattern string ?
In this case I don't think.
Why ?
Because the first test method :
assertThat(pattern, is(notNullValue()));
doesn't valid any expected behavior of the buildPatternFromScale() method.
This method is designed to return a Pattern according what it received as parameter.
The single thing to check is that.
By writing a test method to do a not null assertion, you don't cover the specification of the method. You write just a first step to validate it.
If your method could return null in some cases and not null in others, it could make sense.
But it is not the case.
So I would replace :
assertThat(pattern, is(notNullValue()));
assertThat(pattern, is(equalTo(expectedPattern)));
by just :
assertThat(pattern, is(equalTo(expectedPattern)));

Given your code: you don't need the first test. The second test makes sure that your builder returns an object that is equal to an expected pattern.
This test will obviously fail if the builder returns a null object!
In that sense: having two different tests doesn't add much value anyway!
In other words: you might start with both test, or more precisely: when you follow TDD; you would probably start with that first test case. Later on, when you have implemented your real builder, and the "real" test case is in place - then you don't need that first check-for-null test case any more.
Keep in mind: any line of source code also resembles cost for you. As long as it is around, it might be read and is required to be understand (to fix bugs or add functionality). Therefore even unit tests should provide "return on investment" to you. Meaning: tests that don't add significant value to your test bucket are subject to deletion.
Edit, based on the changes to the question. The additional test case
#Test
public void testShouldVerifyThatNonNullPatternIsBuilt() {
//given
int scale = 1;
is still redundant. And "worse" - keep in mind that #Parameterized tests work somehow different than "normal JUnit" tests. You see, with the #Parameterized runner in place, each test is invoked for each data value. Meaning: the above test is called three times. The idea is that all tests use the data elements. In that sense: it doesn't make sense to have a test in there that does not use the "data values". Especially given the fact that this test here tests something that the other test checks already.
In other words: assertThat(pattern, is(equalTo(expectedPattern))) will fail when pattern is null (unless you happen to put null into your expectedPattern).

One purpose of "One assertion per test method" rule is that only name of the test method will provide you enough information to understand reason why test failed.
Where if you have multiple assertions in test - you need to read failed message to understand why test failed.
Also with multiple assertions, if first assertion fail other will not be executed.
So for getting "full picture" of possible reason you need re-run tests after fixing first assertion
In your particular case assertion for non-null is same as assertion for equality to expected result. Null result will fail always if expected not null.

I think I'd stick with two separate tests.here. If this was done using TDD, you would most likely end up with two tests anyway.
Although its worth mentioning that a test can have a single concept (rather than a thing), which can mean multiple asserts in a single test case, if all related logically.
As mentioned before they key thing you are looking forward is clarity in the test to quickly identify what fails and why, you way you have it gives you this which is important in terms of the value added from having the test .

Test code for getting list of fileNames from s3 bucket from a specific folder

I have this function which gets the List of fileNames from a specific s3 folder from a specific folder bucket which looks like this :
public List<String> listFilesFromS3Bucket(String bucketName, String folderName, String fileNamePrefix) {
List<String> fileNames = new ArrayList<>();
String folderPathPrefix = folderName + fileNamePrefix;
ObjectListing listing = s3Client.listObjects(bucketName, folderPathPrefix);
List<S3ObjectSummary> s3ObjectSummaries = listing.getObjectSummaries();
for (ListIterator<S3ObjectSummary> iterator = s3ObjectSummaries.listIterator(); iterator.hasNext();) {
String keyName = iterator.next().getKey();
if (!keyName.endsWith("/")) {
fileNames.add(StringUtils.substringAfterLast(keyName, "/"));
}
}
log.info("Total {} files to process", fileNames.size());
return fileNames;
}
Now i need to test this piece of code, I have created a mock of s3client and verified call to listObjects. I wan't the test to verify the call to listObjects and which i did by -
Mockito.Verify(s3Client).listObjects(bucketname, prefix);
but how should i test the remaining code, i am pretty new to testing ?

What I normaly test are 3 things.
1) The output for a given input (usually more then 1 test)
2) Are the correct calls executed on injected objects. e.g. are for a given input the right parameters passed to the method call on that injected object.
3) When it is possible for a method to throw an exception . test if for a given input the right exception is thrown.
Also make sure "all" your paths are covered. Your IDE can help you with this
for your method I would test the following:
1) test if Mockito.Verify(s3Client).listObjects(bucketname, prefix); is executed exactly once.
2) test for multiple inputs the list of output filenames. Use inputs which makes the loop executing 0,1 and multiple times
hope this answers your question

One flaw that I see in your code that you assume , listing to be always non - null. Verify that its always the case since your code assumes that. Similar is the case with - s3ObjectSummaries & keyName references.
If listing is empty , would your code work OK?
Execute a test when !keyName.endsWith("/") is true and another when its false.
How your code behaves for any RuntimeException? How do you want to handle that?
These are just sample things that you would test for your method and that is achieved not by one test method but many test methods i.e. for your one single target method, you would write many many test methods covering all scenarios & code paths and eventually correcting your code ( if you find any flaw ).
Put any code coverage plug in to your IDE ( if your IDE doesn't provide that ) and try to achieve 100% code coverage through all these various test methods.
You shouldn't restrict yourself to a few number of test methods for a target method - you should go on unless you achieve 100% code coverage ( if that is possible ) . You should also be open to changing your code if your unit test results say so.
Mockito when are used to set various data for your various scenarios and verify would be used to check invocations etc on that mock. Few very good samples are listed here
Also, unit tests are not integration tests so in my opinion, you should always write unit tests mocking your s3Client or any such dependencies.

Your question is a somewhat broad question, but if I were you, I would test the entire listFilesFromS3Bucket method. This way, you can be certain that your codes properly functions as a unit instead of the functioning of the individual pieces. Once you have done that, then test various expected inputs of your function to validate expected output, error handling, etc. This also will address and separation of concerns in your function that you may need to implement.

Java Mockito one test causes another to fail

#Test
public void onConnectionCompletedTest() {
connectionProvider.initialize();
connectionProvider.addEventObserver(SocketEvent.Type.SOCKET_CONNECT, mockedObserver);
connectionProvider.onConnectionCompleted(mockedChannel);
verify(mockedObserver).socketEventObserved(socketEventCaptor.capture());
Assert.assertEquals(SocketEvent.Type.SOCKET_CONNECT, socketEventCaptor.getValue().getType());
}
#Test
public void onConnectionClosedTest() {
connectionProvider.initialize();
connectionProvider.addEventObserver(SocketEvent.Type.SOCKET_DISCONNECT, mockedObserver);
connectionProvider.onConnectionClosed(mockedChannel);
verify(mockedObserver).socketEventObserved(socketEventCaptor.capture());
Assert.assertEquals(SocketEvent.Type.SOCKET_DISCONNECT, socketEventCaptor.getValue().getType());
}
The problem is when I run both of these tests, the 2nd one fails. But if I comment out
verify(mockedObserver).socketEventObserved(socketEventCaptor.capture());
Assert.assertEquals(SocketEvent.Type.SOCKET_CONNECT, socketEventCaptor.getValue().getType());
then the 2nd test will pass. There's a lot of different classes/methods involved in this so hopefully this is enough information to be able to come up with an explanation.
The error I get:
wanted but not invoked:
mockedObserver.socketEventObserved(
<Capturing argument>
);
-> at com.company.cnx.cip.io.ConnectionProviderTest.onConnectionClosedTest(ConnectionProviderTest.java:188)
Actually, there were zero interactions with this mock.
My question exactly: What could be happening that when I #Ignore the first test, the 2nd will pass?
EDIT: I have an #Before class that's important.
#Before
public void init() {
MockitoAnnotations.initMocks(this);
JsonParser parser = new JsonParser();
JsonElement jsonElement = parser.parse(json);
configurationService.loadConfiguration(jsonElement, "id");
AppContext.getContext().applyConfiguration(configurationService);
connectionProvider = ConnectionProvider.newInstance();
}
You can ignore everything that isn't the first and last line of it. I make a new ConnectionProvider object, so I would think that one test shoudldn't affect another, because they're operating on two separate objects.

This is more straightforward than you think: The test works when you comment out those lines because socketEventObserved is not getting called when it's the second test of the run. This is probably a problem with code that you haven't posted above.
Since it seems that the culprit may be buried in an impractical volume of code, here are a few debugging tips and general sources of test pollution:
First and foremost, set a breakpoint everywhere that socketEventObserved is called, and compare between single-test runs and multi-test runs. If you see the same behavior Mockito sees, then it's not Mockito.
As it turned out to be in this case, keep an eye out for actions that may occur on other threads (particularly to listeners). Using a Mockito timeout can help there:
verify(mockedObserver, timeout(2000))
.socketEventObserved(socketEventCaptor.capture());
You seem to be working with I/O channels, which sometimes involve buffering or flushing policies that can be triggered only when a certain number of tests are run, or if tests are run in certain order. Make sure that your #Before method fully resets state, including any modes or buffers that your code may touch.
You interact with AppContext.getInstance() and ConnectionProvider.newInstance(), both of which are static method calls. I'm less worried about the latter, unless it conserves instances in spite of its name, but the former may not take kindly to multiple initializations. In general, in your system-under-test, keep an eye out for writes to global state.
Mockito itself keeps static state. Mockito keeps its internal state static and thread-scoped (through ThreadLocal), and at times a test can leave Mockito in an invalid internal state that it cannot detect (because an operation is half-completed, for instance). Add an #After method to detect this case:
#After public void checkMockito() { Mockito.validateMockitoUsage(); }
...or switch to using MockitoRule or MockitoJUnitRunner, which do this and initMocks(this) automatically.
Finally, Mockito isn't perfect: It uses proxy objects to override methods, which also means it will silently fail to mock final methods and some non-public methods that work with nested classes (as those require the compiler to generate synthetic methods you can't see). If your mockedObserver has any final or limited-visibility methods, it may cause real code and mocked code to interact in a way that makes the system's behavior hard-to-predict.

Specifying order of execution in JUnit test case [duplicate]

This question already has answers here:
How to run test methods in specific order in JUnit4?
(23 answers)
Closed 9 years ago.
I have a test case where I add an entity, update it and delete the same. Hence, the order of execution is important here. I want it to be :
Create
Update
Delete
Strangely, for just one test case ( out of 15) , JUnit executes it in the following order :
Delete
Update
Create .
How do I tell JUnit to execute them in a specific order ? In other cases, JUnit works totally fine ( executing serially ) . And why does JUnit behave weirdly in this one case ?
Relevant code snippet below :
private static Date date;
private static int entity;
static Parking p;
public ParkingTests(String name) {
super(name);
}
public void testAdd() throws Exception {
//Add code here
}
public void testUpdate() throws Exception {
//update code here
}
public void testDelete() throws Exception {
//delete code here
}
}
It gets weirder. I run a lot of test cases as part of a suite. If I run just the Parking case, the order is maintained. If I run it along with others, it is sometimes maintained, sometimes not !

Your kind of situation is awkward, as it feels bad to keep duplicating work in order to isolate the tests (see below) - but note that most of the duplication can be pulled out into setUp and tearDown (#Before, #After) methods, so you don't need much extra code. Provided that the tests are not running so slowly that you stop running them often, it's better to waste a bit of CPU in the name of clean testing.
public void testAdd() throws Exception {
// wipe database
// add something
// assert that it was added
}
public void testUpdate() throws Exception {
// wipe database
// add something
// update it
// assert that it was updated
}
public void testDelete() throws Exception {
// wipe database
// add something
// delete it
// assert that it was deleted
}
The alternative is to stick everything into one test with multiple asserts, but this is harder to understand and maintain, and gives a bit less information when a test fails:
public void testCRUD() throws Exception {
// wipe database
// add something
// assert that it was added
// update it
// assert that it was updated
// delete it
// assert that it was deleted
}
Testing with databases or collections or storage of any kind is tricky because one test can always affect other tests by leaving junk behind in the database/collection. Even if your tests don't explicitly rely on one another, they may still interfere with one another, especially if one of them fails.
Where possible, use a fresh instance for each test, or wipe the data, ideally in as simple a way as possible - e.g. for a database, wiping an entire table is more likely to succeed than a very specific deletion that you might accidentally get wrong.
Update: It's usually better to wipe data at the start of the test, so one failed test run doesn't affect the next run.

Generally junit tests(test methods) should not depend on each other.
Following is taken from junit FAQ
Each test runs in its own test fixture to isolate tests from the
changes made by other tests. That is, tests don't share the state of
objects in the test fixture. Because the tests are isolated, they can
be run in any order...... The ordering of test-method invocations is not
guaranteed.
So if you want to do some common initialization stuff then you could do that in the method annotated with #Before and cleanup in method annotated with #After. Or else if that initialization is not required for all tests methods in your test class then you could put that in private methods and call them appropriately from your tests.
On a side note, if you still want to do ordering of tests then you may have a look at TestNG.

If you're determined you would want to have order of execution for your tests, JUnit 4.11 now supports this through an annotation. See this thread for more discussion - basically, you would use
#FixMethodOrder
to guarantee some test order that way. It is discouraged though.

If you are using Java 7 then you should know that Junit gets the list of all tests using "Method[] getDeclaredMethods()" from java.lang.Class. You can read from the javadoc of this method or from junit docs that: "The elements in the array returned are not sorted and are not in any particular order.", but in previous jvm implementation methods list was ordered as they were in source code.
This was taken from this blog and he provides a work around.

In general, JUnit does not guarantee the ordering of test cases. It's not guaranteed to be alphabetical, nor the order in the file. If the ordering of tests were important, then one depends on the output of the previous. What if the first one failed? Should we even bother with the later (and dependent) tests? Probably not.
So if we had this:
#Test
public void first(){...}
#Test
public void second() {...}
#Test
public void third() {...}
We don't know what order they will run in. Since we are hoping they go in order, and we should probably not bother running second or third if the previous one(s) failed, we can do this instead:
#Test
public void firstThree(){
first();
second();
third();
}
public void first(){...}
public void second() {...}
public void third() {...}
Notice that we only have one #Test this time, and it guarantees ordering.

If you want to run junit tests in order "just as they present in your source code",
see my note about this here:
How to run junit tests in order as they present in your source code
But it is really not a good idea, tests must be independent.

What you can do :
Cleanup the database before every test
Start by testing the first logical operation first. When you have enough confidence, assume it is correct and move to the next, etc...
Write white box tests first, but start with black box tests. For example if you have triggers or similar in your database, start with that.