We Keep Coding

How to write JUnit5 or Mockito test for a retry mechanism built using Supplier<CompletableFuture> as a method param?

Have a common framework that isn't server specific which we use to import and re-use objects inside our microservices but can be used for any particular Java program which supports Java 8.
Was tasked to create a retry() mechanism that would take the following two parameters:
Supplier<CompletableFuture<R>> supplier
int numRetries
What I needed it to do is to make a generic component that conducts a retry operation for a CompletableFuture every time there's an exception based on the numRetries specified.
Using Java 8, I wrote the following generic helper class which uses a Supplier to retry a method based on a number of retries specified.
Am new to CompletableFuture, in general, so am wondering how to write a JUnit 5 (and if Mockito is better) test for this method? Am testing for all edge cases and have made this as generic as possible for others to re-use.
Note this is inside an in-house common framework which is imported by other microservices as a Maven dependency but the aim is to be able to re-use this at the Java SE level.
public class RetryUtility {
private static final Logger log =
LoggerFactory.getLogger(RetryUtility.class);
public static <R> CompletableFuture<R> retry(
Supplier<CompletableFuture<R>> supplier,
int numRetries) {
CompletableFuture<R> completableFuture = supplier.get();
if (numRetries > 0) {
for (int i = 0; i < numRetries; i++) {
completableFuture =
completableFuture
.thenApply(CompletableFuture::completedFuture)
.exceptionally(
t -> {
log.info("Retrying for {}", t.getMessage());
return supplier.get();
})
.thenCompose(Function.identity());
}
}
return completableFuture;
}
}
Usage: Presume this code compiles and works, I was able to put in a wrong URL in the config file for client and log.info(Error getting orderResponse = {}); was printed twice via grep against app.log file.
This is the calling class that imports the above class as Maven dependency:
public class OrderServiceFuture {
private CompletableFuture<OrderReponse> getOrderResponse(
OrderInput orderInput,BearerToken token) {
int numRetries = 1;
CompletableFuture<OrderReponse> orderResponse =
RetryUtility.retry(() -> makeOrder(orderInput, token), numRetries);
orderResponse.join();
return orderResponse;
}
public CompletableFuture<OrderReponse> makeOrder() {
return client.post(orderInput, token),
orderReponse -> {
log.info("Got orderReponse = {}", orderReponse);
},
throwable -> {
log.error("Error getting orderResponse = {}", throwable.getMessage());
}
}
}
Although this example's calling class uses OrderSerice with an HttpClient making a call, this is a generic utility class that was specifically written to be reusable for any type of method call which returns CompletableFuture<OrderReponse>.
Question(s):
How to write a JUnit 5 test case (or Mockito) for this:
public static <R> CompletableFuture<R> RetryUtility.retry(Supplier<CompletableFuture<R>> supplier, int numRetries) method?
Is there any edge cases or nuances that someone can see with its design and/or implementation?

Before you ask "How do I test method X?" it helps to clarify "What does X actually do?". To answer
this latter question I personally like to rewrite methods in a way such that individual steps become
clearer.
Doing that for your retry method, I get:
public static <R> CompletableFuture<R> retry(
Supplier<CompletableFuture<R>> supplier,
int numRetries
) {
CompletableFuture<R> completableFuture = supplier.get();
for (int i = 0; i < numRetries; i++) {
Function<R, CompletableFuture<R>> function1 = CompletableFuture::completedFuture;
CompletableFuture<CompletableFuture<R>> tmp1 = completableFuture.thenApply(function1);
Function<Throwable, CompletableFuture<R>> function2 = t -> supplier.get();
CompletableFuture<CompletableFuture<R>> tmp2 = tmp1.exceptionally(function2);
Function<CompletableFuture<R>, CompletableFuture<R>> function3 = Function.identity();
completableFuture = tmp2.thenCompose(function3);
}
return completableFuture;
}
Note that I removed the unnecessary if statement and also the log.info call. The latter isn't
testable unless you pass in a logger instance as a method argument (or make retry non-static and
have the logger as an instance variable that is passed via the constructor, or maybe use some
dirty hacks like redirecting the logger output stream).
Now, what does retry actually do?
It calls supplier.get() once and assigns the value to the completableFuture variable.
If numRetries <= 0, then it returns completableFuture which is the same CF instance that the
Supplier's get method returned.
If numRetries > 0, then it does the following steps numRetries times:
It creates a function function1 that returns a completed CF.
It passes function1 to the thenApply method of completableFuture, creating a new
CF tmp1.
It creates another function function2 which is a function that ignores its input argument
but calls supplier.get().
It passes function2 to the exceptionally method of tmp1, creating a new
CF tmp2.
It creates a third function function3 which is the identity function.
It passes function3 to the thenCompose of tmp2, creating a new CF and assigning it to
the completableFuture variable.
Having a clear breakdown of what a function does lets you see what you can test and what you cannot
test and, therefore, might want to refactor. Steps 1. and 2. are very easy to test:
For step 1. you mock a Supplier and test that it's get method is called:
#Test
void testStep1() { // please use more descriptive names...
Supplier<CompletableFuture<Object>> mock = Mockito.mock(Supplier.class);
RetryUtility.retry(mock, 0);
Mockito.verify(mock).get();
Mockito.verifyNoMoreInteractions(mock);
}
For step 2. you let the supplier return some predefined instance and check that retry returns
the same instance:
#Test
void testStep2() {
CompletableFuture<Object> instance = CompletableFuture.completedFuture("");
Supplier<CompletableFuture<Object>> supplier = () -> instance;
CompletableFuture<Object> result = RetryUtility.retry(supplier, 0);
Assertions.assertSame(instance, result);
}
The tricky part is, of course, step 3. The first thing you should look out for is which variables
are influenced by our input arguments. These are: completableFuture, tmp1, function2,
and tmp2. In contrast, function1 and function3 are practically constants and are not
influenced by our input arguments.
So, how can we influence the listed arguments then? Well, completableFuture is the return value
of the get method of our Supplier. If we let get return a mock, we can influence the return
value of thenApply. Similarly, if we let thenApply return a mock, we can influence the return
value of expectionally. We can apply the same logic to thenCompose and then test that
our methods have been called the correct amount of times in the correct order:
#ParameterizedTest
#ValueSource(ints = {0, 1, 2, 3, 5, 10, 100})
void testStep3(int numRetries) {
CompletableFuture<Object> completableFuture = mock(CompletableFuture.class);
CompletableFuture<CompletableFuture<Object>> tmp1 = mock(CompletableFuture.class);
CompletableFuture<CompletableFuture<Object>> tmp2 = mock(CompletableFuture.class);
doReturn(tmp1).when(completableFuture).thenApply(any(Function.class));
doReturn(tmp2).when(tmp1).exceptionally(any(Function.class));
doReturn(completableFuture).when(tmp2).thenCompose(any(Function.class));
CompletableFuture<Object> retry = RetryUtility.retry(
() -> completableFuture, // here 'get' returns our mock
numRetries
);
// While we're at it we also test that we get back our initial CF
Assertions.assertSame(completableFuture, retry);
InOrder inOrder = Mockito.inOrder(completableFuture, tmp1, tmp2);
for (int i = 0; i < numRetries; i++) {
inOrder.verify(completableFuture, times(1)).thenApply(any(Function.class));
inOrder.verify(tmp1, times(1)).exceptionally(any(Function.class));
inOrder.verify(tmp2, times(1)).thenCompose(any(Function.class));
}
inOrder.verifyNoMoreInteractions();
}
What about the method arguments of thenApply and thenCompose? There is no way to test
that these methods have been called with function1 and function3, respectively, (unless you
refactor your code and move the functions out of that method call) as these functions are local to
our method and not influenced by our arguments.
But what about function2? While we cannot test that exceptionally is called with function2 as
an argument, we can test that function2 calls supplier.get() exactly numRetries times. When
does it do that? Well, only if the CF fails:
#ParameterizedTest
#ValueSource(ints = {0, 1, 2, 3, 5, 10, 100})
void testStep3_4(int numRetries) {
CompletableFuture<Object> future = CompletableFuture.failedFuture(new RuntimeException());
Supplier<CompletableFuture<Object>> supplier = mock(Supplier.class);
doReturn(future).when(supplier).get();
Assertions.assertThrows(
CompletionException.class,
() -> RetryUtility.retry(supplier, numRetries).join()
);
// remember: supplier.get() is also called once at the beginning
Mockito.verify(supplier, times(numRetries + 1)).get();
Mockito.verifyNoMoreInteractions(supplier);
}
Similarly, you can test that retry calls the get method n+1 times by providing a supplier,
that returns a completed future after it has been called n times.
What we still need to do (and what we should probably have had done as a first step) is to test
whether the return value of our method behaves correctly:
If the CF fails numRetries tries, the CF should complete exceptionally.
If the CF fails fewer than numRetries tries, the CF should complete normally.
#ParameterizedTest
#ValueSource(ints = {0, 1, 2, 3, 5, 10})
void testFailingCf(int numRetries) {
RuntimeException exception = new RuntimeException("");
CompletableFuture<Object> future = CompletableFuture.failedFuture(exception);
CompletionException completionException = Assertions.assertThrows(
CompletionException.class,
() -> RetryUtility.retry(() -> future, numRetries).join()
);
Assertions.assertSame(exception, completionException.getCause());
}
#ParameterizedTest
#ValueSource(ints = {0, 1, 2, 3, 5, 10})
void testSucceedingCf(int numRetries) {
final AtomicInteger counter = new AtomicInteger();
final String expected = "expected";
Supplier<CompletableFuture<Object>> supplier =
() -> (counter.getAndIncrement() < numRetries / 2)
? CompletableFuture.failedFuture(new RuntimeException())
: CompletableFuture.completedFuture(expected);
Object result = RetryUtility.retry(supplier, numRetries).join();
Assertions.assertEquals(expected, result);
Assertions.assertEquals(numRetries / 2 + 1, counter.get());
}
Some other cases you might want to consider testing and catching are what happens if numRetries is
negative or very large? Should such method calls throw an exception?
Another question we haven't touched so far is: Is this the right way to test your code?
Some people might say yes, others
would argue that you shouldn't test the internal structure of your method but only its output given
some input (i.e. basically only the last two tests). That's clearly debatable and, like most things,
depends
on your requirements. (For example, would you test the internal structure of some sorting
algorithm, like array assignments and what not?).
As you can see, by testing the internal structure, the tests become quite complicated and involve a
lot of mocking. Testing the internal structure can also make refactoring more troublesome, as your
tests might start to fail, even though you haven't changed the logic of your method. Personally, in
most situations I wouldn't write such tests. However, if a lot of people depended on the correctness
of my code, for example on the order in which steps are executed, I might consider it.
In any case, if you choose to go that route, I hope these examples are useful to you.

Access Lambda Arguments with Groovy and Spock Argument Capture

I am trying to unit test a Java class with a method containing a lambda function. I am using Groovy and Spock for the test. For proprietary reasons I can't show the original code.
The Java method looks like this:
class ExampleClass {
AsyncHandler asynHandler;
Component componet;
Component getComponent() {
return component;
}
void exampleMethod(String input) {
byte[] data = input.getBytes();
getComponent().doCall(builder ->
builder
.setName(name)
.data(data)
.build()).whenCompleteAsync(asyncHandler);
}
}
Where component#doCall has the following signature:
CompletableFuture<Response> doCall(Consumer<Request> request) {
// do some stuff
}
The groovy test looks like this:
class Spec extends Specification {
def mockComponent = Mock(Component)
#Subject
def sut = new TestableExampleClass(mockComponent)
def 'a test'() {
when:
sut.exampleMethod('teststring')
then:
1 * componentMock.doCall(_ as Consumer<Request>) >> { args ->
assert args[0].args$2.asUtf8String() == 'teststring'
return new CompletableFuture()
}
}
class TestableExampleClass extends ExampleClass {
def component
TestableExampleClass(Component component) {
this.component = component;
}
#Override
getComponent() {
return component
}
}
}
The captured argument, args, shows up as follows in the debug window if I place a breakpoint on the assert line:
args = {Arrays$ArrayList#1234} size = 1
> 0 = {Component$lambda}
> args$1 = {TestableExampleClass}
> args$2 = {bytes[]}
There are two points confusing me:
When I try to cast the captured argument args[0] as either ExampleClass or TestableExampleClass it throws a GroovyCastException. I believe this is because it is expecting Component$Lambda, but I am not sure how to cast this.
Accessing the data property using args[0].args$2, doesn't seem like a clean way to do it. This is likely linked to the casting issue mentioned above. But is there a better way to do this, such as with args[0].data?
Even if direct answers can't be given, a pointer to some documentation or article would be helpful. My search results discussed Groovy closures and Java lambdas comparisons separately, but not about using lambdas in closures.

Why you should not do what you are trying
This invasive kind of testing is a nightmare! Sorry for my strong wording, but I want to make it clear that you should not over-specify tests like this, asserting on private final fields of lambda expressions. Why would it even be important what goes into the lambda? Simply verify the result. In order to do a verification like this, you
need to know internals of how lambdas are implemented in Java,
those implementation details have to stay unchanged across Java versions and
the implementations even have to be the same across JVM types like Oracle Hotspot, OpenJ9 etc.
Otherwise, your tests break quickly. And why would you care how a method internally computes its result? A method should be tested like a black box, only in rare cases should you use interaction testing,where it is absolutely crucial in order to make sure that certain interactions between objects occur in a certain way (e.g. in order to verify a publish-subscribe design pattern).
How you can do it anyway (dont!!!)
Having said all that, just assuming for a minute that it does actually make sense to test like that (which it really does not!), a hint: Instead of accessing the field args$2, you can also access the declared field with index 1. Accessing by name is also possible, of course. anyway, you have to reflect on the lambda's class, get the declared field(s) you are interested in, make them accessible (remember, they are private final) and then assert on their respective contents. You could also filter by field type in order to be less sensitive to their order (not shown here).
Besides, I do not understand why you create a TestableExampleClass instead of using the original.
In this example, I am using explicit types instead of just def in order to make it easier to understand what the code does:
then:
1 * mockComponent.doCall(_ as Consumer<Request>) >> { args ->
Consumer<Request> requestConsumer = args[0]
Field nameField = requestConsumer.class.declaredFields[1]
// Field nameField = requestConsumer.class.getDeclaredField('arg$2')
nameField.accessible = true
byte[] nameBytes = nameField.get(requestConsumer)
assert new String(nameBytes, Charset.forName("UTF-8")) == 'teststring'
return new CompletableFuture()
}
Or, in order to avoid the explicit assert in favour of a Spock-style condition:
def 'a test'() {
given:
String name
when:
sut.exampleMethod('teststring')
then:
1 * mockComponent.doCall(_ as Consumer<Request>) >> { args ->
Consumer<Request> requestConsumer = args[0]
Field nameField = requestConsumer.class.declaredFields[1]
// Field nameField = requestConsumer.class.getDeclaredField('arg$2')
nameField.accessible = true
byte[] nameBytes = nameField.get(requestConsumer)
name = new String(nameBytes, Charset.forName("UTF-8"))
return new CompletableFuture()
}
name == 'teststring'
}

How to create JUnit tests for different input files as separate cases?

Right now, I have around 107 test input cases for my interpreter, and I have my JUnit tester set up to manually handle each case independently so as to not lump them all together. That is, if I use a loop to iterate over the test files as such
for (int i = 0; i < NUM_TESTS; i++) {
String fileName = "file_" + (i + 1) + ".in";
testFile(fileName);
}
JUnit will create one giant test result for all 107 tests, meaning if one fails, the entire test fails, which I don't want. As I said, right now I have something like
#Test
public static void test001() {
testFile("file1.in");
}
#Test
public static void test002() {
testFile("file2.in");
}
While this works, I imagine there's a much better solution to get what I'm after.

You can use #ParameterizedTest with #MethodSource annotations.
For exemple :
#ParameterizedTest
#MethodSource("fileNameSource")
void test(final String fileName) {
testFile(fileName);
}
private static Stream<String> fileNameSource() {
return IntStream.range(0,NUM_TESTS).mapToObj(i -> "file_" + (i + 1) + ".in");
}
Check the documentation at https://junit.org/junit5/docs/current/user-guide/#writing-tests-parameterized-tests
For each params returned by fileNameSource(), the corresponding test will be considered as a different case.

You have to define own structure based on your need,
one way to define your input to json file in like list of values as below.
{
[
"value1",
"value2"
]
}
Read this value when you test case execute with the help of object mapper.
objectMapper.readValue(fixture("filePathName.json"),CustomInput.class);
Where CustomInput would be something like below.
public class CustomInput {
List<String> values;
}
You can keep increase & decrease your inputs in json.

How to generate multiple test method from JSON?

I have a huge JSON-File which contains testcases. (No I don't wanna show it here because it doesn't matter in this case to know the file)
I parse the json file to my junit test - that works fine.
But I've got 50 testcases and if I want to show each in Junit like: "test 0 from 50 passed"
and have a list like: test 1 passed, test 2 passed, test 3 failed..
I have to put each testcase into a method. How can I dynamically do this? Is this possible in Junit? Because when I'm parsing the json, I don't know how many cases I have.

JUnit has direct support for CSV files, which means you can import and use them easily using #CSVFileSource.
However, since your case does not involve CSV files, I tried to create parametrized tests in JUnit 5 using JSON files.
Our class under test.
public class MathClass {
public static int add(int a, int b) {
return a + b;
}
}
Here's the JSON file I am using.
[
{
"name": "add positive numbers",
"cases": [[1, 1, 2],[2, 2, 4]]
},
{
"name": "add negative numbers",
"cases": [[-1, -1, -2 ], [-10, -10, -20 ]]
}
]
So, in JUnit 5 there is an annotation called #MethodSource which gives you the opportunity to provide arguments to your parametrized test. You only need to provide the method name. Here's my argument provider method.
#SneakyThrows
private static Stream<TestCase> getAddCases() {
final ObjectMapper mapper = new ObjectMapper();
TypeReference<List<Case>> typeRef = new TypeReference<>() {};
final File file = new File("src/test/resources/add-cases.json");
final List<Case> cases = mapper.readValue(file, typeRef);
return cases.stream()
.flatMap(caze -> caze.getCases()
.stream()
.map(el -> new TestCase(caze.getName(), el)));
}
In the code above, the class Case is used to map from the json object to Java Object and since the "cases" field is a multidimensional array, to represent each test case there is a class called TestCase. (Overall, this is not important for you, since you already are able to parse it, but I wanted to put it here anyway).
Finally, the test method itself.
#ParameterizedTest(name = "{index} : {arguments}")
#MethodSource("getAddCases")
void add_test(TestCase testCase) {
final List<Integer> values = testCase.getValues();
int i1 = values.get(0);
int i2 = values.get(1);
int e = values.get(2);
assertEquals(e, MathClass.add(i1, i2));
}
#ParametrizedTest annotation takes a name argument where you can provide a template for the test names. I just played around with the toString method of TestCase class to achieve a better description for each test case.
#Override
public String toString() {
return String.format("%s : (%s, %s) ==> %s", name, values.get(0), values.get(1), values.get(2));
}
And voila!

Dataproviders and Asserts

When using DataProviders, on TestNG, my test method has asserts that will fail since the data passed in navigates to a different url. Is there a way to work around this, i.e. a way for the data to only be injected to certain/specific asserts?
Instead of testing one scenario with different data, I am instead testing multiple scenarios with different data which is where my conflict arises.
#DataProvider(name = "VINNumbers")
public String[][] VINNumbers() {
return new String[][] {
{"2T1BU4ECC834670"},
{"1GKS2JKJR543989"},
{"2FTDF0820A04457"}
};
}
#Test(dataProvider = "VINNumbers")
public void shouldNavigateToCorrespondingVinEnteredIn(String VIN) {
driver.get(findYourCarPage.getURL() + VIN);
Assert.assertTrue(reactSRP.dealerListingMSRPIsDisplayed());
}
The assert test whether or not the page has an MSRP displayed, but not all dataproviders will have an MSRP displayed so it will fail. The only dataprovider that has it is the first array. Is there a way for dataproviders to be called to specific asserts?

If depending on the VIN, MSRP is displayed or not (boolean), you could for example create a provider the way it provides VIN and expected result:
#Test(dataProvider = "VINNumbers")
public void shouldNavigateToCorrespondingVinEnteredIn(String VIN, boolean isMSRPDisplayed) {
// act
// assert
assertThat(reactSRP.dealerListingMSRPIsDisplayed()).is(isMSRPDisplayed);
}
This way you end up with an provider like below:
{
{"2T1BU4ECC834670", true},
{"1GKS2JKJR543989", false},
{"2FTDF0820A04457", true},
}
In my opinion this is acceptable for simple cases. To make assertion more readable, I would add a custom message to it that is also parameterized.
I hope this helps.