I'm new to writing tests in general. The class that I need to test has one method and it needs to be tested:
public String run(final Map<String, Dataset> datasets)
throws ApiException {
final String sourcePath = ElementsUtil.getElementFromDatasets(inputElementNames.get(0), datasets).getValue();
final String destinationPath = ElementsUtil.getElementFromDatasets(inputElementNames.get(1), datasets).getValue();
final File source = new File(sourcePath);
final File destination = new File(destinationPath);
if (source.exists()) {
if (source.isDirectory()) {
final IOFileFilter filter = new WildcardFileFilter(pattern);
final Iterator<File> it = FileUtils.iterateFiles(source, filter, null);
while (it.hasNext()) {
final File file = it.next();
moveFileToDirectory(file, destination);
}
} else {
moveFileToDirectory(source, destination);
}
} else {
LOGGER.error("Source file/folder at path {} doesn't exist.", sourcePath);
}
return "0";
}
At first, with my limited knowledge of writing unit tests, my unit test looked like this:
#Test(description = "Test in case the source is a file.")
public void moveFileTest1() {
// setup
final String fileName = UUID.randomUUID().toString() + ".txt";
final String folderName = UUID.randomUUID().toString();
final Element source = new Element("source", "./" + fileName);
final Element destination = new Element("destination", "./" + folderName);
...
final Path sourcePath = Paths.get(source.getValue());
final Path destinationPath = Paths.get(destination.getValue());
final Path fileDestination = Paths.get(destination.getValue() + "/" + fileName);
try {
Files.createFile(sourcePath);
Files.createDirectory(destinationPath);
// exercise
moveFile.run("", datasets, null);
// verify
Assert.assertEquals(Files.exists(fileDestination), true);
Assert.assertEquals(Files.exists(sourcePath), false);
} catch (ApiException | IOException e) {
LOGGER.error("Exception : ", e);
} finally {
// teardown
try {
Files.deleteIfExists(sourcePath);
} catch (final IOException e) {
LOGGER.error("Exception in teardown: ", e);
}
try {
Files.deleteIfExists(fileDestination);
} catch (IOException e) {
LOGGER.error("Exception in teardown: ", e);
}
try {
Files.deleteIfExists(destinationPath);
} catch (IOException e) {
LOGGER.error("Exception in teardown: ", e);
}
}
}
After reading some articles about unit testing I found out that my test isn't exactly testing a unit since my method depends on different util methods. Also I found out about mocking objects in tests and how everything should be mocked. My question is: Should I use mocking in each of these util methods/new object calls etc. or is there a different approach? How would you test this piece of code?
What you are doing is called integration testing. Integration testing is testing an object/method without mocking nothing, on real data or data that you produce during the test itself. What integration testing does is to test both your unit, the units that your unit uses and the flow of the use. If you would like to test only your unit you should mock all other units that your unit uses and create flows of those units doing their job successfully or not. Meaning you should replicate a test in which a used unit throws an exception/returns a value that you don't expect as a good value(only if it really can do it) and return a value that you do expect and now how to work with.
Usually when writing tests you do both kinds of testing, unit and integration tests
There is such good test principle as "Don't Mock What You Don't Own". What doesn't it mean? It means your shouldn't mock/stub any interface which you cannot control, even if this interface has been written by your company, but not by your team.
But what about writing unit tests, not integration tests, where you may want to mock all classes except a class which is being tested? Indeed, this is a really tricky question. And an answer more about system design than about testing. You may read about how the issue could be resolved here and here.
What does it means for your?
As your mentioned ElementsUtil has been written by you, so this class exactly should be mocked. How? It depends is it the legacy code or new code which you're writing now. If you have legacy code - then you need PowerMock, otherwise you may change design and use an instance of ElementsUtil.
For example, divide the ElementsUtil class into three classes: Elements- interface, ElementsImpl- implementation, ElementsUtil - class with static access to keep compatibility. The ElementsUtil may have method
public static Elements getInstance()
And the method could be used by class which contains run method in constructor. But you may provide either constructor with parameter or setter. By the way, as I remember Mockito can inject mocks into private fields. After this refactoring you don't need PowerMock and can use only Mockito.
Now about FileUtils. The class doesn't belong to you, so if follow good practice, then this class should be mock. But FileUtils works with file and it will already be integration tests. So answer - the FileUtils should be wrapped by a new your class.
Related
This is the function that I want to test:
#Component
public class DataSourceAttributes {
...
...
public AWSSecretDB getAttribsBySecret() throws Exception {
AbstractConnector abstractConnector = new AWSSecretManagerConnector("secretsmanager." + region + ".amazonaws.com", region);
GenericManager genericManager = new GenericManager(abstractConnector);
System.out.println("Generic Manager: " + genericManager);
AWSSecretDB awsSecretDB;
try {
awsSecretDB = genericManager.getSecretModel(secretName, AWSSecretDB.class);
System.out.println("awsSecretDB: " + awsSecretDB.getEngine()); // It must be mocked
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
throw e;
}
return awsSecretDB;
}
}
This is my current unit test:
public class DataSourceAttributesTest {
#InjectMocks
private DataSourceAttributes dataSourceAttributes;
#Mock
private GenericManager genericManagerMock;
#Test
public void AWSSecretDBGetAttribsBySecret() throws Exception {
AWSSecretDB awsSecretDB = new AWSSecretDB();
awsSecretDB.setEngine("Engine Test");
awsSecretDB.setDbname("DB Test");
awsSecretDB.setHost("Host Test");
when(genericManagerMock.getSecretModel(ArgumentMatchers.any(), ArgumentMatchers.any())).thenReturn(awsSecretDB);
dataSourceAttributes.getAttribsBySecret();
// The assert is missing, but it's not important for this question
}
}
I need to mock genericManager to control getSecretModel() function, but It doesn't work.
When I run my test, the System.out.println (located in getAttribsBySecret) prints the following message evidencing that mock is not working:
Generic Manager: co.com.bancolombia.commons.secretsmanager.manager.GenericManager#1349883
I know if I use the following code, the mock works great, but I don't want to recode something that it's already working in the src directory:
#Component
public class DataSourceAttributes {
private GenericManager genericManager; // First change
public DataSourceAttributes () { // Second cahnge
AbstractConnector abstractConnector = new AWSSecretManagerConnector("secretsmanager." + region + ".amazonaws.com", region);
this.genericManager = new GenericManager(abstractConnector);
}
public AWSSecretDB getAttribsBySecret() throws Exception {
System.out.println("Generic Manager: " + genericManager);
AWSSecretDB awsSecretDB;
try {
awsSecretDB = genericManager.getSecretModel(secretName, AWSSecretDB.class);
System.out.println("awsSecretDB: " + awsSecretDB.getEngine());
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
throw e;
}
return awsSecretDB;
}
}
When I run my test, the System.out.println (located in getAttribsBySecret) prints:
GENERIC MANAGER: genericManagerMock
awsSecretDB: Engine Test
How it shows, the mock works great. So, this is my question: how can I use a mock inside a class and avoid to declare a new attribute and a constructor in main code. I ask this because the first code works and I don't want to edit it, I think that is not the filosophy of unit tests.
Thanks!
To use unit tests you need to make your code testable first. And this might require some code changes. There is no way to mock local variables inside methods so you need to either pass those variables as parameters to methods or create properties inside objects and pass mocked objects into constructor.
#Component
public class DataSourceAttributes {
private AbstractConnector abstractConnector;
private GenericManager genericManager;
#Autowired // to ask Spring to inject dependencies
public DataSourceAttributes(AbstractConnector abstractConnector, GenericManager genericManager) {
this.abstractConnector = abstractConnector;
this.genericManager = genericManager;
}
public AWSSecretDB getAttribsBySecret() throws Exception {
System.out.println("Generic Manager: " + genericManager);
AWSSecretDB awsSecretDB;
try {
awsSecretDB = genericManager.getSecretModel(secretName, AWSSecretDB.class);
System.out.println("awsSecretDB: " + awsSecretDB.getEngine()); // It must be mocked
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
throw e;
}
return awsSecretDB;
}
}
And then in test you create instance of you object passing mocked dependencies into constructor
If you know that the code works, why unit-test it then in the first place? Or, why perform an activity that can bring up a bug if you are not intending to change the source to fix it?
But, more seriously, you seem to be stuck in the "legacy code dilemma": To modify the code, you'd rather have tests in place. To get tests in place you have to modify the code.
The escape from this is to start with minimally invasive changes that are just sufficient to create those tests you need. Since this is a migrational approach, sometimes even dirty tricks are acceptable to overcome the dilemma. This is discussed in much detail with lots of practical advice by Michael Feathers in "Working Effectively with Legacy Code".
In languages with introspection like Java, sometimes introspection can be a solution along the way towards a better testable solution. That, however, falls in the category of dirty tricks and except for serving as a way to migrate to a better solution, however, I don't consider it advisable.
I am trying to test the functionality of persisting the data into elasticsearch using JUnits. This is the first time I am using JUnits and this is my first test case.
I have an interface which is like the below
public interface ElasticSearchIndexer {
void writeTo(String inputJson) throws IOException;
}
The interface is implemented by multiple classes. The sample implementation looks like below
public class HardwareEOXIndexer implements ElasticSearchIndexer {
private static final Logger logger = LoggerFactory.getLogger(HardwareEOXIndexer.class);
private final String es_index = "/hardwareeox/inv/";
private String hostname;
public HardwareEOXIndexer(String hostname) {
this.hostname = hostname;
}
public void writeTo(String inputJson) throws IOException {
ReadContext ctx = JsonPath.parse(inputJson);
String hardwareEOXId = Integer.toString(ctx.read("$.EoXBulletin.items[0].hardwareEOXId"));
StringBuilder documentID = new StringBuilder().append(hardwareEOXId);
logger.info("Indexing the document with ID :: {} ", documentID.toString());
try {
new ElasticSearchContext().getContext(hostname, inputJson, es_index, documentID);
} catch (Exception e) {
e.printStackTrace();
logger.error("HardwareEOXIndexer : es_index: " + es_index + " ------> " + e.getMessage());
}
}
}
How do I mock the behavior of the elasticsearch and how to write unit tests.
The interface part is bogus within the question, the core point is:
How do I mock the behavior of the elasticsearch and how to write unit tests.
And there are basically two answers:
you create an abstraction layer that hides the details of ElasticSearch. Meaning: instead of creating a new ElasticSearch object, you create an object of your own class (which you don't create via new, but via a factory object for example).
you read about PowerMock, and how to use that to mock calls to new.
I definitely suggest you to go for the first option: simply because that will improve your design. You see, why do you want to tightly couple all of your code to elastic search? But assuming that this implementation is already meant as abstraction layer around elastic search - then you should still use dependency injection to acquire that ElasticSearch object you need to actually invoke methods on. As said, use a factory or a real DI framework. That will allow you to say with "simple" mocking frameworks such as Mockito or EasyMock.
I'm writing a Junit to test the following method in Client.java:
public FSDataInputStream getObj(String hName, Path p) throws IOException {
String myKey = pathToKey(hName, p);
FileStatus status = memoryCache.getStatus(p.toString());
if (status == null) {
status = getStatus(hName, p, "getObject");
}
if (status.isDirectory()) {
throw new FileNotFoundException("Can't open " + path
+ " because it is a directory");
}
InputStream inputStream = new InputStream(bucket, myKey,
status.getLen(), client, readAhead, inputPolicy);
return new FSDataInputStream(inputStream);
}
Initially I want to test if status == null then getStatus() is invoked and if status.isDirectory(), the FileNotFoundException is thrown
I'm new to Junit so not completely sure what I'm at but to the best of my knowledge I think I need to mock the following:
List item
Client
status
inputStream
possibly memoryCache
So far this is what I've got:
#Before
public final void before() {
private COSAPIClient myClient;
private String myBucket;
FileStatus myStatus;
InputStream myInputStream;
myClient = PowerMockito.mock(AmazonS3.class);
myInputStream = PowerMockito.mock(InputStream.class);
myFileStatus = PowerMockito.mock(FileStatus.class);
}
#Test
public void getObjTest() throws Exception {
URI uri = new URI("xyz://aa-bb-cc/data7-1-23-a.txt");
String hName = "xyz://aa-bb-cc/";
Path p = new Path("cos://aa-bb-cc/data7-1-23-a.txt");
Configuration conf = new Configuration();
myClient = spy(new Client(uri, conf));
myStatus = spy(new FileStatus());
myMemoryCache.getStatus(p.toString());
InputStream = spy(new InputStream(myBucket, objectKey, 300, myClient, 12345678910L, myInputPolicy));
}
It returns a NullPointerError at this line in my program:
FileStatus status = memoryCache.getStatus(p.toString());
I wonder is anybody could advice if/what I'm doing wronfg and how I should go about resolving this?
First, the real answer: step back for a moment. Don't start with JUnit and Mockito and your production code as input. Rather have a look into a tutorial (like here) that step-by-step explains all the relevant elements and how to "bring" them together.
In your case, the are various problems with your code:
Why are you using PowerMock? Try to go with "plain vanilla" Mockito. If your production code is so that it requires PowerMock, rather consider to rework your production instead of turning to PowerMock.
You seem to really not know where/how to apply mocking. In other words: you only mock the elements that you need to control when running your code under test. And you only use mocking, if you can't control them otherwise. Meaning: you almost never mock a list - you simply create a "normal" list to then add the things that this list should contain.
Creating a mock allows to invoke methods on that mock object. But by default, any method that returns something will return null (or maybe an empty collection, or 0 for primitive return types, see here for details). Thus you rather need a statement such as when(mockedCache.getStatus("some string")).thenReturn(someResult).
I have written Junit test case for following function. When checked JACOCO test coverage. It is showing only try block is covered by test case. I am newbie to writing test cases. How the exceptions and catch block can be covered in test cases
Here is a method
public static List<Student> readCsvFile(String fileName)
{
BufferedReader fileReader = null;
//logic to read file
}
catch (Exception e)
{
System.out.println("Error in CsvFileReader !!!");
e.printStackTrace();
} finally
{
try
{
fileReader.close();
} catch (IOException e)
{
System.out.println("Error while closing fileReader !!!");
e.printStackTrace();
}
}
return students;
}
And TestMethod
#Test
public void ReadCsvFileTest()
{
String fileName = "test.csv";
List<Student> result = new ArrayList<Student>();
result = CsvFileReader.readCsvFile(fileName);
Student student1 = null;
Iterator<Student> it = result.iterator();
while (it.hasNext())
{
Student s = it.next();
if ("471908US".equals(s.getId()))
{
student1 = s;
break;
}
}
assertTrue(student1 != null);
}
In such situations you may often consider the introduction of additional dependencies to your class. Here is what I mean with a rough example. Create a factory for readers:
interface BufferedReaderFactory
{
public BufferedReader createBufferedReader(String fileName) throws IOException;
}
Then you will have a trivial implementation that hardly needs any testing, e.g. something similar:
class BufferedReaderFactoryImpl implements BufferedReaderFactory
{
#Override
public BufferedReader createBufferedReader(String fileName) throws IOException
{
return new BufferedReader(new FileReader(fileName));
}
}
Then you have to find a way to inject this dependency into your class. I usually use Guice in my daily work but you may try something as simple as using constructor injection and making your method non static. Here is an example:
class CsvFileReader
{
private final BufferedReaderFactory factory;
public CsvFileReader(BufferedReaderFactory factory)
{
this.factory = factory;
}
public List<Student> readCsvFile(String fileName)
{
BufferedReader fileReader = null;
try
{
fileReader = factory.createBufferedReader(fileName);
...
}
catch(IOException e)
{
...
}
finally
{
...
}
return new LinkedList<>();
}
}
With a mocking framework like Mockito the behavior of this class in case of IOException-s is easier to test now (note that you may also return mocks that throw exceptions from the factory). Here is a sample:
#RunWith(MockitoJUnitRunner.class)
public class MyTest
{
#Mock
private BufferedReaderFactory mockFactroy;
#Test
public void testIOException() throws IOException
{
String ivalidFileName = "invalid.txt";
//throw exception in case that invalid file name is passed to the factory
Mockito.when(mockFactroy.createBufferedReader(ivalidFileName)).thenThrow(new IOException("Hello!"));
CsvFileReader csvFileReader = new CsvFileReader(mockFactroy);
//invoke with a factory that throws exceptions
csvFileReader.readCsvFile(ivalidFileName);
//...
//and make a sensible test here, e.g. check that empty list is returned, or proper message is logged, etc.
}
}
You may do that without Mockito, of course - by implementing a test factory. But this is more cumbersome especially in more complicated use cases. Once the IOException is thrown you will get appropriate coverage report by JaCoCo.
Also mind a limitation of JaCoCo mentioned here, in section Source code lines with exceptions show no coverage. Why?
Given the current signature of your method under test, getting to full coverage isn't easy: your catch block is only executed when an exception is thrown within your try block.
One way to solve this: do not pass in the file name, but the reader object itself. Like:
public static List<Student> readCsvFile(String fileName) {
return readCsvFile(new BufferedReader(fileName));
}
static List<Student> readCsvFile(BufferedReader reader) {
try {
...
} catch( ...
Now you can write several specific unit tests for that second method. You keep your tests that simply do "correct" reading; but you add one where you pass in a mocked reader object ... that simply throws an Exception at some point. Please note that I made that new method just package protected - you probably don't want to use that "public"; and making it private would prevent it from being unit tested.
That should help you achieving full coverage. Of course you will also need at least one test to "cover" the string-taking method, too.
Some notes:
Be careful about re-inventing the wheel. There are many existing CSV parsers already. And be assured: writing a correct CSV parser that is able to deal with all "correct" input CSV is much harder than it sounds. If this is not for "learning purposes" I strongly advise to not write your own CSV parser.
Be careful about making such things static. As said, a real CSV parser is a complicated thing, and worth its complete own class. So no static helper methods - a normal class which you instantiate to then call methods on it (that would also for using dependency injection which would help with the problem you are asking about ... getting exceptions thrown within try blocks)
You are catching Exception in your code example. Don't do that - try to catch exactly those exceptions that your code can actually produce (probably IOException in your case).
Java Path API is a better replacement of Java File API but massive usage of static methods makes it difficult to mock with Mockito.
From my own class, I inject a FileSystem instance which I replace with a mock during unit tests.
However, I need to mock a lot of methods (and also creates a lot of mocks) to achieve this. And this happens repeatedly so many times across my test classes. So I start thinking about setup a simple API to register Path-s and declare associated behaviour.
For example, I need to check error handling on stream opening.
The main class:
class MyClass {
private FileSystem fileSystem;
public MyClass(FileSystem fileSystem) {
this.fileSystem = fileSystem;
}
public void operation() {
String filename = /* such way to retrieve filename, ie database access */
try (InputStream in = Files.newInputStream(fileSystem.getPath(filename))) {
/* file content handling */
} catch (IOException e) {
/* business error management */
}
}
}
The test class:
class MyClassTest {
#Test
public void operation_encounterIOException() {
//Arrange
MyClass instance = new MyClass(fileSystem);
FileSystem fileSystem = mock(FileSystem.class);
FileSystemProvider fileSystemProvider = mock(FileSystemProvider.class);
Path path = mock(Path.class);
doReturn(path).when(fileSystem).getPath("/dir/file.txt");
doReturn(fileSystemProvider).when(path).provider();
doThrow(new IOException("fileOperation_checkError")).when(fileSystemProvider).newInputStream(path, (OpenOption)anyVararg());
//Act
instance.operation();
//Assert
/* ... */
}
#Test
public void operation_normalBehaviour() {
//Arrange
MyClass instance = new MyClass(fileSystem);
FileSystem fileSystem = mock(FileSystem.class);
FileSystemProvider fileSystemProvider = mock(FileSystemProvider.class);
Path path = mock(Path.class);
doReturn(path).when(fileSystem).getPath("/dir/file.txt");
doReturn(fileSystemProvider).when(path).provider();
ByteArrayInputStream in = new ByteArrayInputStream(/* arranged content */);
doReturn(in).when(fileSystemProvider).newInputStream(path, (OpenOption)anyVararg());
//Act
instance.operation();
//Assert
/* ... */
}
}
I have many classes/tests of this kind and mock setup can be more tricky as static methods may call 3-6 non-static methods over the Path API. I have refactored test to avoid most redundant code but my simple API tends to be very limited as my Path API usage grown. So again it's time to refactor.
However, the logic I'm thinking about seems ugly and requires much code for a basic usage. The way I would like to ease API mocking (whatever is Java Path API or not) is based on the following principles:
Creates abstract classes that implements interface or extends class to mock.
Implements methods that I don't want to mock.
When invoking a "partial mock" I want to execute (in preference order) : explicitly mocked methods, implemented methods, default answer.
In order to achieve the third step, I think about creating an Answer which lookup for implemented method and fallback to a default answer. Then an instance of this Answer is passed at mock creation.
Are there existing ways to achieve this directly from Mockito or other ways to handle the problem ?
Your problem is that you are violating the Single Responsibility Principle.
You have two concerns:
Find and locate a file, get an InputStream
Process the file.
Actually, this should most likely be broken into sub concerns also, but that's outside the scope of this question.
You are attempting to do both of those jobs in one method, which is forcing you to do a ton of extra work. Instead, break the work into two different classes. For example, if your code were instead constructed like this:
class MyClass {
private FileSystem fileSystem;
private final StreamProcessor processor;
public MyClass(FileSystem fileSystem, StreamProcessor processor) {
this.fileSystem = fileSystem;
this.processor = processor;
}
public void operation() {
String filename = /* such way to retrieve filename, ie database access */
try (InputStream in = Files.newInputStream(fileSystem.getPath(filename))) {
processor.process(in);
} catch (IOException e) {
/* business error management */
}
}
}
class StreamProcessor {
public StreamProcessor() {
// maybe set dependencies, depending on the need of your app
}
public void process(InputStream in) throws IOException {
/* file content handling */
}
}
Now we've broken the responsibilities into two places. The class that does all the business logic work that you want to test, from an InputStream, just needs an input stream. In fact, I wouldn't even mock that, because it's just data. You can load the InputStream any way you want, for example using a ByteArrayInputStream as you mention in your question. There doesn't need to be any code for Java Path API in your StreamProcessor test.
Additionally, if you are accessing files in a common way, you only need to have one test to make sure that behavior works. You can also make StreamProcessor be an interface, and then, in the different parts of your code base, do the different jobs for different types of files, while passing in different StreamProcessors into the file API.
In the comments you said:
Sounds good but I have to live with tons of legacy code. I'm starting to introduce unit test and don't want to refactor too much "application" code.
The best way to do it is what I said above. However, if you want to do the smallest amount of changes to add tests, here is what you should do:
Old code:
public void operation() {
String filename = /* such way to retrieve filename, ie database access */
try (InputStream in = Files.newInputStream(fileSystem.getPath(filename))) {
/* file content handling */
} catch (IOException e) {
/* business error management */
}
}
New code:
public void operation() {
String filename = /* such way to retrieve filename, ie database access */
try (InputStream in = Files.newInputStream(fileSystem.getPath(filename))) {
new StreamProcessor().process(in);
} catch (IOException e) {
/* business error management */
}
}
public class StreamProcessor {
public void process(InputStream in) throws IOException {
/* file content handling */
/* just cut-paste the other code */
}
}
This is the least invasive way to do what I describe above. The original way I describe is better, but obviously it's a more involved refactor. This way should involve almost no other code changes, but will allow you to write your tests.