We Keep Coding

How to write unit test upload directory to S3?

I have a method to upload directory to s3.
I would like to write a unit test for it and I've been searching, but most test are integration tests.
Maybe I'm not supposed to do a unit test for this kind of process?
I don't have a privileges to use testcontainers for integration test, so I can’t use testcontainers.
Here is my code
public void uploadDir(final Path path, final String key) throws InternalServerError {
final TransferManager transferManager = TransferManagerBuilder.standard()
.withS3Client(s3Client)
.build();
final ProgressListener progressListener = progressEvent -> {
if (progressEvent.getBytesTransferred() > 0) {
double percentTransferred = progressEvent.getBytesTransferred() * 100.0 / progressEvent.getBytes();
log.info("Transferred " + percentTransferred + "%");
}
};
final MultipleFileUpload xfer = transferManager.uploadDirectory("mybucket", key, path.toFile(), true);
xfer.addProgressListener(progressListener);
xfer.waitForCompletion();
log.info("Upload has been completed.");
} catch (final AmazonServiceException | InterruptedException exception) {
... do something here
} finally {
transferManager.shutdownNow();
}
}

It seems there's not much to unit test in the method but arguments and interactions.
Mocking TransferManager isn't really a solution because you'd need to mock MultipleFileUpload, and also the code that calls ProgressListener needs to be added. You'd end up mocking everything which makes it useless. Testing a widely used library also has not much value for you.
I'd test the exception case which is an unlikely case to happen (aka not repeatable) during integration testing.
Roughly something like this (assuming mockito):
#Test
void uploadDirFailsTest() {
TransferManager mockTM = mock(TransferManager.class);
doThrow(new AmazonServiceException(...)) // hopefully it has a public constructor
.when(mockTM)
.uploadDirectory(eq("mybucket"), any(), any(), eq(true)); //more matching if needed
YourClassName instance = new YourClassName(mockTM);
instance.uploadDir(path, key);
/*
verify correct exception handling here or
if you rethrow the exception wrap the call with
assertThrows()
*/
}
public class YourClassName {
final TransferManager transferManager;
public YourClassName(TransferManager transferManager) {
this.transferManager = transferManager;
}
public void uploadDir(final Path path, final String key) throws InternalServerError {
final ProgressListener progressListener = progressEvent -> {
if (progressEvent.getBytesTransferred() > 0) {
double percentTransferred = progressEvent.getBytesTransferred() * 100.0 / progressEvent.getBytes();
log.info("Transferred " + percentTransferred + "%");
}
};
final MultipleFileUpload xfer = transferManager.uploadDirectory("mybucket", key, path.toFile(), true);
xfer.addProgressListener(progressListener);
xfer.waitForCompletion();
log.info("Upload has been completed.");
} catch (final AmazonServiceException | InterruptedException exception) {
... do something here
} finally {
transferManager.shutdownNow();
}
}
//real use
new YourClassName(
TransferManagerBuilder.standard()
.withS3Client(s3Client)
.build()
);

How to mock KafkaAdminClient

I am trying to do a unit test for the code below. I am able to test the exception block but unable to test the below block as I am getting an exception.
How can I mock or set values to the ListTopicsResult topics = client.listTopics(), so that the flow goes into the if block?
if(!topics.names.get().isEmpty()) { response = true; }
public boolean isBrokerRunning() {
boolean response = false;
Properties property = new Properties();
try(AdminClient client = KafkaAdminClient.create(property)) {
ListTopicsResult topics = client.listTopics();
if(!topics.names.get().isEmpty()) {
response = true;
}
} catch(Exception ex) {
response = false;
}
}

KafkaAdminClient.create
This is a static function call, so you need to mock static function, You can use powermockit on top of mockito to mock static functions.
see this example

use mockito-inline can do this. need some trick
#Test
public void mockCreateAdmin() {
AdminClient mock = mock(KafkaAdminClient.class);
try (MockedStatic<Admin> staticMock = mockStatic(Admin.class)) {
staticMock.when(() -> Admin.create(any(Properties.class))).thenReturn(mock);
KafkaAdminClient adminClient = (KafkaAdminClient) KafkaAdminClient.create(new Properties());
// when
// then
assertEquals(mock, adminClient);
}
}

Unable to mock accountStatus with Mockito

From below piece of code I am not able to mock checkAccountStatus and its coming as null. What changes do I need to do to resolve this issue?
Class
public AccessIDSearchResponse searchAccessID(AccessIDSearchRequest accessIDRequest) {
String[] productTypes = accessIDRequest.getProductTypes();
AccountResponse actResponse = checkAccountStatus(accessIDRequest);
System.out.println("Response is---->"+JsonService.getJsonFromObject(actResponse));
if (accessIDRequest.getSearchtype().equalsIgnoreCase("accountId") && !Utility.isEmpty(actResponse)
&& !"FREEVIEW".equalsIgnoreCase(actResponse.getAccountStatus())) {
errorHandler.error(ErrorMessages.EPO_EXISTINGTV_ERR_07, ErrorMessages.ACCESS_ID_NOT_FOUND);
}
}
public AccountResponse checkAccountStatus(AccessIDSearchRequest request) {
AccessIDSearchResponse response = new AccessIDSearchResponse();
SearchAccessIdContent content = new SearchAccessIdContent();
DTVNAccountDetails accountDetails = new DTVNAccountDetails();
accountDetails.setAccountNumber(request.getSearchvalue());
List<DTVNAccountDetails> list = new ArrayList<>();
list.add(accountDetails);
content.setDtvAccountList(list);
response.setContent(content);
return helper.getAccountStatus(response);
}
Helper
public AccountResponse getAccountStatus(AccessIDSearchResponse accessIDResponse) {
AccountResponse accountResponse = null;
AccountRequest request = new AccountRequest();
Account account = new Account();
account.setCustomerID(accessIDResponse.getContent().getDtvAccountList().get(0).getAccountNumber());
request.setAccount(account);
String response = dtvnClients.callandGetDtvnStatus(request);
System.out.println("Response is--->"+response);
if (!Utility.isEmpty(response)) {
accountResponse = JqUtil.runJqQueryAndGetString(".content.accountResponse", response,
AccountResponse.class);
if (!Utility.isEmpty(accountResponse) && accountResponse.isSubscribable()
&& !Utility.isEmpty(accountResponse.getAccountStatus())
&& accountResponse.getAccountStatus().equalsIgnoreCase("FREEVIEW")) {
return accountResponse;
}
}
return accountResponse;
}
Test Class
#Test(expected = ServiceException.class)
public void test_searchAccessID_3_sample() throws Exception {
AccessIDSearchRequest request = new AccessIDSearchRequest();
CommonData commonData = new CommonData();
commonData.setAppName("IDSE");
commonData.setLoginId("qay_slid_sr1281");
request.setCommonData(commonData);
request.setSearchtype("accountId");
request.setSearchvalue("qay_slid_sr1281");
request.setMode("abc");
SearchAccessIdContent content = new SearchAccessIdContent();
AccountResponse accountResponse = new AccountResponse();
accountResponse.setAccountStatus("Sucess");
accountResponse.setSubscribable(true);
Mockito.when(helper.getAccountStatus(accessIDResponse)).thenReturn(accountResponse);
Mockito.when(service.checkAccountStatus(request)).thenReturn(accountResponse);
service.searchAccessID(header, request);
}

Your mocks are not properly configured.
When you call
service.searchAccessID(header, request);
it was make the underlying call
checkAccountStatus(request);
(which is correctly mocked and returns accountResponse), but this one does instanciate its result object, so your first mock will never be triggered.
Updating your first mock to something more permissive will probably fix your problem
Mockito.when(helper.getAccountStatus(any(AccessIDSearchResponse.class))).thenReturn(accountResponse);
To be honest, your code is hardly testable because you instanciate too many objects everywhere. Going for mocks here will be a pain in the future when you refactor something. If I were you I would rewrite this piece of code using a TDD approach and favorizing more testable patterns.

How get results of test in a Dynamic Test in Junit5?

my function is similar to:
#TestFactory
public Stream<DynamicTest> dynamicTest() throws Exception {
String geocodingAnasJsonTest = properties.getProperty("smart-road.simulator.json.geocoding-it.anas.testSuite.test");
String endpoint = properties.getProperty("smart-road.simulator.endpoint.anasGeocoding");
RequestSpecification request = RestAssured.given().header("Authorization", auth);
request.accept(ContentType.JSON);
request.contentType(ContentType.JSON);
JsonNode jsonObjectArray = JsonMappingUtil.getJsonFileFromPath(geocodingAnasJsonTest);
Stream<JsonNode> elementStream = StreamSupport.stream(Spliterators
.spliteratorUnknownSize(jsonObjectArray.elements(),
Spliterator.ORDERED), false);
return elementStream.map(jsonNode -> DynamicTest.dynamicTest(String.format("Test ID: %s", jsonNode.get("test_name")),
() -> {request.body(jsonNode.get("request").toString());
Response response = request.post(endpoint);
int statusCode = response.getStatusCode();
boolean res = false;
if (statusCode >= 200 && statusCode < 300) {
res = true;
}
try {
assertEquals(true, res, properties.getProperty("smart-road.response.smart-road.message.status.ok"));
logger.info(properties.getProperty("smart-road.response.smart-road.message.status.ok"));
String responseOK=jsonNode.get("response").toString();
assertEquals(responseOK, response.asString(), properties.getProperty("smart-road.response.smart-road.message.status.right-end"));
logger.info(properties.getProperty("smart-road.response.smart-road.message.status.right-end"));
} catch (AssertionFailedError er) {
logger.error(properties.getProperty("smart-road.response.smart-road.message.status.assertion-failed"));
fail("Test Fallito");
Assertions.assertTrue(true);
}
}
)//fine dynamicTest
);//fine map
}//fine metodo
I have 20 children test.
I run test in main:
SummaryGeneratingListener listener = new SummaryGeneratingListener();
LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
.selectors(selectMethod(Test.class,"dynamicTest"))
.build();
Launcher launcher = LauncherFactory.create();
launcher.registerTestExecutionListeners(listener);
launcher.execute(request);
Now with summary= listener.getSummary() i dont read all tests result but only count Failed or Successfull test.
How i read all result fail/success for all tests?
I will want a map like this:
TEST_ID - RESULTS
test0001 Success
test0002 Fail
test0003 Success
test0004 Success
test0005 Fail
How i get this? Is possible?
Thanks
Regards

One approach is to create your own implementation of org.junit.platform.launcher.TestExecutionListener and register it with the launcher. You may look at the source code of SummaryGeneratingListener as a first start. You could change executionFinished(..) to build up the map of test results. Here's a sketch:
class MySummaryListener implements TestExecutionListener {
private Map<String, TestExecutionResult.Status> summary = new HashMap<>();
#Override
public void executionFinished(TestIdentifier testIdentifier, TestExecutionResult testExecutionResult) {
summary.put(testIdentifier.getDisplayName(), testExecutionResult.getStatus());
}
}
There's probably more you want to do in the listener but it should give you an idea where to start.

Wanted but not invoke: Mockito PrintWriter

Hi I am working on a project and using PrintWriter class for opening and writing in the file. But when I am writing the test case for same it gives following error at Line 153
Wanted but not invoked:
mockPrintWriter.println("ID url1
");
-> at x.y.z.verify(ProcessImageDataTest.java:153)
Actually, there were zero interactions with this mock.
Code: (Uses Lombok Library)
ProcessImageData.java
#Setter
#RequiredArgsConstructor
public class ProcessImageData implements T {
private final File newImageDataTextFile;
#Override
public void execute() {
LineIterator inputFileIterator = null;
try {
File filteredImageDataTextFile = new File(filteredImageDataTextFilepath);
PrintWriter writer = new PrintWriter(newImageDataTextFile);
inputFileIterator = FileUtils.lineIterator(filteredImageDataTextFile, StandardCharsets.UTF_8.displayName());
while (inputFileIterator.hasNext()) {
if(someCondition)
**Line51** writer.println(imageDataFileLine);
//FileUtils.writeStringToFile(newImageDataTextFile, imageDataFileLine + NEWLINE, true);
}
}
} catch (Exception e) {
} finally {
LineIterator.closeQuietly(inputFileIterator);
**LINE63** writer.close();
}
}
ProcessImageDataTest.java
#RunWith(PowerMockRunner.class)
#PrepareForTest({ ProcessImageData.class, FileUtils.class, Printwriter.class })
public class ProcessImageDataTest {
private ProcessImageData processImageData;
private static final String FILTERED_IMAGE_DATA_TEXT_FILE_PATH = "filteredFilepath";
private File FILTEREDFILE = new File(FILTERED_PATH);
private static final File IMAGE__FILE = new File("imageFilePath");
private LineIterator lineIterator;
#Mock
private PrintWriter mockPrintWriter;
#Before
public void init() throws Exception {
MockitoAnnotations.initMocks(this);
processImageData = new ProcessImageData(Palettes_file, FILTERED_PATH, IMAGE_FILE);
PowerMockito.mockStatic(FileUtils.class);
PowerMockito.whenNew(PrintWriter.class).withArguments(IMAGE_FILE).thenReturn(mockPrintWriter);
PowerMockito.when(FileUtils.lineIterator(FILTERED_FILE, StandardCharsets.UTF_8.displayName())).thenReturn(lineIterator);
PowerMockito.when(lineIterator.hasNext()).thenReturn(true, true, false);
}
#Test
public void testTaskWhenIDInDBAndStale() throws IOException {
PowerMockito.when(lineIterator.nextLine()).thenReturn(ID2 + SPACE + URL1, ID1 + SPACE + URL2);
processImageData.execute();
List<String> exepctedFileContentOutput = Arrays.asList(ID2 + SPACE + URL1 + NEWLINE);
verify(exepctedFileContentOutput, 1, 1);
}
#Test
public void testTaskWhenIDNotInDB() throws IOException {
PowerMockito.when(lineIterator.nextLine()).thenReturn(ID2 + SPACE + URL1, ID3 + SPACE + URL2);
processImageData.execute();
List<String> exepctedFileContentOutput = Arrays.asList(ID3 + SPACE + URL2 + NEWLINE);
verify(exepctedFileContentOutput, 1, 1);
}
private void verify(List<String> exepctedFileContentOutput, int fileWriteTimes, int fileReadTimes) throws IOException {
for (String line : exepctedFileContentOutput){
**Line153** Mockito.verify(mockPrintWriter, Mockito.times(fileWriteTimes)).print(line);
}
PowerMockito.verifyStatic(Mockito.times(fileReadTimes));
FileUtils.lineIterator(FILTERED_IMAGE_DATA_TEXT_FILE, StandardCharsets.UTF_8.displayName());
}
}
I am mocking a new operator for PrintWriter also, injecting using beans. What is the mistake I am doing?? I am stuck on it from long time and not getting the error?
Any help is appreciated.
Updated :
I did changes suggested below and updated the code, but now I get the error:
Wanted but not invoked: mockPrintWriter.print("ASIN2 url1 "); ->
at softlines.ctl.ruleExecutor.tasks.ProcessImageDataTest.verify‌​(ProcessImageDataTes‌​t.java:153)
However, there were other interactions with this mock: -> at softlines.ctl.ruleExecutor.tasks.ProcessImageData.execute(Pr‌​ocessImageData.java:‌​51) ->
at softlines.ctl.ruleExecutor.tasks.ProcessImageData.execute(Pr‌​ocessImageData.java:‌​51) ->
at softlines.ctl.ruleExecutor.tasks.ProcessImageData.execute(Pr‌​ocessImageData.java:‌​58) –

I see 3 issues in your test:
You don't try to mock the correct constructor, indeed in the method execute, you create your PrintWriter with only one argument of type File while you try to mock the constructor with 2 arguments one of type File and the other one of type String.
So the code should rather be:
PowerMockito.whenNew(PrintWriter.class)
.withArguments(IMAGE_FILE)
.thenReturn(mockPrintWriter);
To be able to mock a constructor you need to prepare the class creating the instance which is ProcessImageData in this case, so you need to add ProcessImageData.class in the annotation #PrepareForTest. (I'm not sure ProcessImageDataTest.class is needed there)
The field lineIterator should be annotated with #Mock.
Instead of verifying print with a new line, you should verify directly println without new line it is much less error prone.
I simplified your code to show the idea.
Assuming that ProcessImageData is:
public class ProcessImageData {
private final File newImageDataTextFile;
public ProcessImageData(final File newImageDataTextFile) {
this.newImageDataTextFile = newImageDataTextFile;
}
public void execute() throws Exception{
try (PrintWriter writer = new PrintWriter(newImageDataTextFile)) {
LineIterator inputFileIterator = FileUtils.lineIterator(
newImageDataTextFile, StandardCharsets.UTF_8.displayName()
);
while (inputFileIterator.hasNext()) {
writer.println(inputFileIterator.nextLine());
}
}
}
}
My unit test would then be:
#RunWith(PowerMockRunner.class)
#PrepareForTest({ProcessImageData.class, FileUtils.class})
public class ProcessImageDataTest {
private File file = new File("imageFilePath");
private ProcessImageData processImageData;
#Mock
private PrintWriter mockPrintWriter;
#Mock
private LineIterator lineIterator;
#Before
public void init() throws Exception {
MockitoAnnotations.initMocks(this);
processImageData = new ProcessImageData(file);
PowerMockito.whenNew(PrintWriter.class)
.withArguments(file)
.thenReturn(mockPrintWriter);
PowerMockito.mockStatic(FileUtils.class);
PowerMockito.when(
FileUtils.lineIterator(file, StandardCharsets.UTF_8.displayName())
).thenReturn(lineIterator);
PowerMockito.when(lineIterator.hasNext()).thenReturn(true, true, false);
}
#Test
public void testExecute() throws Exception {
PowerMockito.when(lineIterator.nextLine()).thenReturn("Foo", "Bar");
processImageData.execute();
Mockito.verify(mockPrintWriter, Mockito.times(1)).println("Foo");
Mockito.verify(mockPrintWriter, Mockito.times(1)).println("Bar");
}
}
For more details please refer to How to mock construction of new objects.
how can I add verification in unit test for writer.close?
One way could be to simply check that close() at be called once by adding the next line to your unit test:
Mockito.verify(mockPrintWriter, Mockito.times(1)).close();

Your construction of the PrintWriter doesn't match the mock. You told PowerMockito to return your mock like this:
PowerMockito.whenNew(PrintWriter.class).withArguments(IMAGE_FILE , StandardCharsets.UTF_8.name()).thenReturn(mockPrintWriter);
So you would have to say:
new PrintWriter(IMAGE_FILE, "UTF-8"); // 2 arguments
But instead in your execute method in the code that is being tested, you do:
PrintWriter writer = new PrintWriter(newImageDataTextFile); // only 1 argument
So you either need to change the PowerMockito withArguments clause, or you need to add "UTF-8" to the constructor invocation in the execute method.