I wrote some unit tests around my class and at the end I am confused whether to combine these tests as their reason of failure will be same.
PatternBuilder class
public class PatternBuilder {
private static final String PATTERN_INITIALS = "#0.%s";
String buildPatternFromScale(int scale) {
return String.format(Locale.ENGLISH, PATTERN_INITIALS, StringUtils.repeat("0", scale));
}
}
Unit tests for the above implementation
#RunWith(Parameterized.class)
public class PatternBuilderTest {
private PatternBuilder patternBuilder;
#Parameterized.Parameter
public int scale;
#Parameterized.Parameter(1)
public String expectedPattern;
#Before
public void setUp() {
patternBuilder = new PatternBuilder();
}
#Parameterized.Parameters(name = "buildPatternFromScale({0}) = {1}")
public static Collection<Object[]> data() {
return Arrays.asList(new Object[][]{
{1, "#0.0"},
{2, "#0.00"},
{5, "#0.00000"},
});
}
#Test
public void testShouldVerifyThatNonNullPatternIsBuilt() {
//given
//when
String pattern = patternBuilder.buildPatternFromScale(scale);
//then
assertThat(pattern, is(notNullValue()));
}
#Test
public void testShouldVerifyThatCorrectPatternOfSpecifiedScaleShouldBeCreated() {
//when
String pattern = patternBuilder.buildPatternFromScale(scale);
//then
assertThat(pattern, is(equalTo(expectedPattern)));
}
}
Will I violate the 'test should do only one thing' if I combine first test with second such that it asserts for no-null-value and correct pattern string ?
Effectively the one single test will then have following two assertions-
assertThat(pattern, is(notNullValue()));
assertThat(pattern, is(equalTo(expectedPattern)));
Reason - I am thinking to combine them both because if null pattern string is created than it will be fail both tests for one single reason
Will I violate the 'test should do only one thing' if I combine first
test with second such that it asserts for no-null-value and correct
pattern string ?
In this case I don't think.
Why ?
Because the first test method :
assertThat(pattern, is(notNullValue()));
doesn't valid any expected behavior of the buildPatternFromScale() method.
This method is designed to return a Pattern according what it received as parameter.
The single thing to check is that.
By writing a test method to do a not null assertion, you don't cover the specification of the method. You write just a first step to validate it.
If your method could return null in some cases and not null in others, it could make sense.
But it is not the case.
So I would replace :
assertThat(pattern, is(notNullValue()));
assertThat(pattern, is(equalTo(expectedPattern)));
by just :
assertThat(pattern, is(equalTo(expectedPattern)));
Given your code: you don't need the first test. The second test makes sure that your builder returns an object that is equal to an expected pattern.
This test will obviously fail if the builder returns a null object!
In that sense: having two different tests doesn't add much value anyway!
In other words: you might start with both test, or more precisely: when you follow TDD; you would probably start with that first test case. Later on, when you have implemented your real builder, and the "real" test case is in place - then you don't need that first check-for-null test case any more.
Keep in mind: any line of source code also resembles cost for you. As long as it is around, it might be read and is required to be understand (to fix bugs or add functionality). Therefore even unit tests should provide "return on investment" to you. Meaning: tests that don't add significant value to your test bucket are subject to deletion.
Edit, based on the changes to the question. The additional test case
#Test
public void testShouldVerifyThatNonNullPatternIsBuilt() {
//given
int scale = 1;
is still redundant. And "worse" - keep in mind that #Parameterized tests work somehow different than "normal JUnit" tests. You see, with the #Parameterized runner in place, each test is invoked for each data value. Meaning: the above test is called three times. The idea is that all tests use the data elements. In that sense: it doesn't make sense to have a test in there that does not use the "data values". Especially given the fact that this test here tests something that the other test checks already.
In other words: assertThat(pattern, is(equalTo(expectedPattern))) will fail when pattern is null (unless you happen to put null into your expectedPattern).
One purpose of "One assertion per test method" rule is that only name of the test method will provide you enough information to understand reason why test failed.
Where if you have multiple assertions in test - you need to read failed message to understand why test failed.
Also with multiple assertions, if first assertion fail other will not be executed.
So for getting "full picture" of possible reason you need re-run tests after fixing first assertion
In your particular case assertion for non-null is same as assertion for equality to expected result. Null result will fail always if expected not null.
I think I'd stick with two separate tests.here. If this was done using TDD, you would most likely end up with two tests anyway.
Although its worth mentioning that a test can have a single concept (rather than a thing), which can mean multiple asserts in a single test case, if all related logically.
As mentioned before they key thing you are looking forward is clarity in the test to quickly identify what fails and why, you way you have it gives you this which is important in terms of the value added from having the test .
Related
What does removed call to "com.some.Filename::someMethodName" --> SURVIVED mean in pitest.
Does it mean that if that method call is removed, the code will still work properly?
When pitest says the mutantion has survived it means it changed the codebase, and not a single test detected the code has changed. So you are not being very demanding on your test suite.
Ideally each mutation created should be killed by at least 1 unit test.
some more information regarding mutation test that may help you: https://pedrorijo.com/blog/intro-mutation/. (disclaimer, I'm the tutorial author)
Pi-Mutation wants:-
Every line of code got executed by unit test cases
And data being modified by the source code must asserted/validated.
Just master's these above two point's to be master of pi-mutation API.
Suppose you have following source code for which pi-mutation must be passed.
public Optional<CodeRemovedModel> method1(List<CodeRemovedModel> list) {
if(list.isEmpty()) {
return Optional.empty();
}
return doSomething(list);
}
private Optional<CodeRemovedModel> doSomething(List<CodeRemovedModel> list) {
// iterating over list item and modifying two fields.
// as per mutation this forEach loop must be executed
// And, the modified fields must be asserted - if not you will get "removed call .... -> SURVIVED" error
list.forEach(s -> {
s.setFirstName("RAHUL");
s.setLastName("VSK");
});
return Optional.of(list.get(0));
}
In following test case I'm ignoring assertion of one field, hence the error will shows up.
#Test
public void testMethod1_NON_EMPTY_LIST() {
List<CodeRemovedModel> l = new ArrayList<>();
l.add(new CodeRemovedModel());
Optional<CodeRemovedModel> actual = this.codeRemovedMutation.method1(l);
assertEquals("RAHUL", actual.get().getFirstName());
//assertEquals("VSK", actual.get().getLastName());
}
In Simple mutation testing terms:
Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived/survived.
In most cases, developers tend to write Unit Test cases targeting the Code coverage only which may not test each statement.So using a mutation testing tool like PIT is very helpful because it actually detects any faults in the executed code. But in most cases, to kill the mutants introduced by PIT, you have to write assert statements. Mutation testing improves the testing standard because it enforces to write test cases which tests each statement of the code.
Does it mean that if that method call is removed the code will still work properly?
It does not mean that when the method call is removed, the actual code will work but it means even the method call is removed, at least one of the test cases has not detected the change and the test case passed and mutant survived.
Let's say you have a public method like this
public Sprite spriteAt(int x, int y)
{
assert withinBorders(x, y) : "PRE: x and y should be inside the borders";
return tileAt(x, y).topSprite();
}
Versus
public Sprite spriteAt(int x, int y)
{
if (!withinBorders(x, y)) throw new InvalidArgumentException();
return tileAt(x, y).topSprite();
}
In the bottom case, I would usually have a unit test cases that checks if an exception is thrown when invalid value for x and/or y are given like:
#Test(expected = InvalidArgumentException.class)
public void SpriteAt_InvalidX_ThrowsInvalidArgumentException()
{
sut.spriteAt(-100, 0);
}
This test case is to ensure that the argument validating logic is implemented in the method.
However, for the assertion method at the top, I am not sure what I am supposed to do. The assertions are not production code and I think this means I don't have to test the assertions.
On the other hand, I think unit tests should notify the developer by failing when there is a change of logic in a method. If I do not write a test case for checking if there is an assertion that checks invalid arguments (just like how I do for exception method), then I may not realize I have made a mistake when I accidentally remove the assertion line of code.
Therefore, what I want to do is to check whether assertion is in place if junit is ran with assertion enabled and don't do anything when assertion is not enabled. Below code will contain pseudocode.
#Test
public void SpriteAt_InvalidX_AssertionThrowsException()
{
if (assertion is enabled)
{
try
{
sut.spriteAt(-100, 0);
fail();
}
catch (AssertionFailureException e)
{
}
}
}
So back to my point. I want to know whether unit tests are supposed to test assertions. If so, am I going in the right direction? If not, how do you prevent from accidentally removing assertion code without unit tests?
Interesting question.
In my opinion if you want to test the case where xor yare not within range explicitly than you should throw the IllegalArgumentException and not use assert. On the other hand you have no method comment that tells the caller of this method that xor ymust be within a range. So if you see it from the callers perspective you have no right to assert this at all. :-p
I would use assert for checking pre conditions I have no unit tests for. But on the other hand you simply should have a unit test for that example.
At the end of the day assert throws a RuntimeException as well and - as you told - only during development time when a compiler flag is that. So why not simply use the IllegalArgumentException which seems perfect for that case. And please give the caller of the method the hint in which case it is thrown with the #throws comment.
My conclusion: assert is rather useless as long as you have good unit test coverage. It's only good for manual testing during development to check up pre conditions where you don't want or can write explicit unit tests for.
You can always test the assert functionality if you turn it on by using -ea java command argument.
assert throws java.lang.AssertionError.
Unit tests should test the behavior, not the implementation. So if the asert behavior is not explicitly defined in the contract (JavaDoc) you should not test it.
assert is pretty useless and is almost never used in real systems.
Ok, I know this is considered an anti-pattern, and I am certainly open to a better way of doing this.
I have a map of enum values. I want to ensure that each of those enum values is assigned to something. My test looks like this.
#Test
public void eachRowRequiresCellCalc()
{
Model model = new Model();
EnumValues[] values = EnumValues.values();
for (EnumValues value : values)
{
Assert.assertTrue(String.format("%s must be assigned", value.name()), model.hasEnumValue(value));
}
}
This works and accomplishes 90% of what I'm looking for. What it doesn't do is show me if multiple enum values are unassigned (it fails on the first). Is there a way with JUnit to have multiple fails per test?
Ideally you would not want to check for all values once you get a failure since it is anyways going to fail.
But a workaround I would suggest, but not sure if it works for you:
#Test
public void eachRowRequiresCellCalc()
{
Model model = new Model();
EnumValues[] values = EnumValues.values();
List<EnumValues> isFalse = new ArrayList<EnumValues>;
for (EnumValues value : values)
{
if(!model.hasEnumValue(value)) {
isFalse.add(value);
}
}
//Now you have the array of incorrect values in 'isFalse'
}
You cannot have multiple fails per test. But you can do something similar by tracking the failures in the for loop. Then outside the for loop print out your string in a single assert.
#Test
public void eachRowRequiresCellCalc()
{
Model model = new Model();
EnumValues[] values = EnumValues.values();
String errors = "";
for (EnumValues value : values)
{
if(!model.hasEnumValue(value))
errors += String.format("%s must be assigned", value.name()+". ");
}
if(!errors.isEmpty()){
fail(errors);
}
}
A way to express this using junit-quickcheck would be:
#RunWith(Theories.class)
public class Models {
#Theory public void mustHaveValue(#ForAll #ValuesOf EnumValues e) {
assertTrue(e.name(), new Model().hasEnumValue(e));
}
}
This would run the theory method for every value of your enum.
Another way to express this would be via a parameterized test.
(Full disclosure: I am the creator of junit-quickcheck.)
I know you asked about JUnit, but if you are in a position to consider TestNG, you can define a method as #DataProvider, which will supply parameters to a #Test method. What you seem to be looking for fits this perfectly.
Another option that comes to mind is that you might want to look into MatcherAssert.assertThat with collections matchers. Also it has better logging for your asserts - you might not need to use string format.
A suggestion regarding unit tests, if you don't mind: If you brake down a test method into blocks of given - when - then parts, it greatly improves their readability. given sets up the test case (variables, mocks, etc), when executes the method that is being tested, then is the part where you check the result (assert, verify, ...). I have found that following this structure helps others as well as myself weeks after the test has been written to understand what is going on.
I am aware this is an older question, but for anyone who stumbles upon it now, like I did - as of today, JUnit provides an assertAll() function, which can be used just for that, so there's no more need to fiddle around trying to get the results of multiple chained assertions :)
Here's the reference, hope this saves you some time! Would for sure have saved some of mine if I had known about it earlier.
Say in the main code, you've got something like this:
MyClass.java
public class MyClass {
public List<Obj1> create(List<ObjA> list) {
return (new MyClassCreator()).create(list);
}
// Similar methods for other CRUD operations
}
MyClassCreator.java
public class MyClassCreator {
Obj1Maker obj1Maker = new Obj1Maker();
public List<Obj1> create(List<ObjA> list) {
List<Obj1> converted = new List<Obj1>();
for(ObjA objA : list)
converted.add(obj1Maker.convert(objA));
return converted;
}
}
Obj1Maker.java
public class Obj1Maker {
public Obj1 convert(ObjA objA) {
Obj1 obj1 = new Obj1();
obj1.setProp(formatObjAProperty(objA));
return obj1;
}
private String formatObjAProperty(ObjA objA) {
// get objA prop and do some manipulation on it
}
}
Assume that the unit test for Obj1Maker is already done, and involves a method makeObjAMock() which mocks complex object A.
My questions:
For unit testing MyClassCreator, how would I test create(List<ObjA> list)? All the method really does is delegate the conversion from ObjA to Obj1 and runs it in a loop. The conversion itself is already tested. If I were to create a list of ObjA and tested each object in the list of Obj1 I get back, I would have to copy makeObjAMock() into MyClassCreator's unit test. Obviously, this would be duplicate code, so is using verify() enough to ensure that create(List list) works?
For unit testing MyClass, again, its create(List<ObjA>) method just delegates the operation to MyClassCreator. Do I actually need to test this with full test cases, or should I just verify that MyClassCreator's create method was called?
In the unit test for Obj1Maker, I checked that the properties Obj1 and ObjA corresponded to each other by doing assertEquals(obj1.getProp(), formatObjAProperty(objA)). However, that means I had to duplicate the code for the private formatObjAProperty method from the Obj1Maker class into its unit test. How can I prevent code repetition in this case? I don't want to make this method public/protected just so I can use it in a unit test. Is repetition acceptable in this case?
Thanks, and sorry for the lengthy questions.
My opinion is here. Picking which methods to test is a hard thing to do.
You have to think about a) whether you are meeting your requirements and b) what could go wrong when someone stupid makes changes to the code in the future. (Actually, the stupid person could be you. We all have bad days.)
I would say, writing new code to verify the two objects have the same data in two formats would be a good idea. Probably there is no reason to duplicate the code from the private method and copying the code over is a bad idea. Remember that you are verifying requirements. So if the original string said "6/30/13" and the reformatted one said "June 30th 2013", I would just hard code the check:
assertEquals("Wrong answer", "June 30th 2013", obj.getProp());
Add some more asserts for edge cases and errors. (In my example, use "2/30/13" and "2/29/12" and "12/1/14" to check illegal date, leap year day and that it gets "1st" not "1th" perhaps.)
In the test on the create method, I would probably just go for the easy error and verify that the returned array had the same number as the one passed in. The one I passed in would have two identical elements and some different ones. I'd just check that the identical ones came back identical and the different ones non-identical. Why? Because we already know the formatter works.
I wouldn't test the constructor but would make sure some test ran the code in it. It's good to make sure most of the code actually runs in a test to catch dumb errors like null pointers you missed.
The balance point is what you are looking for.
Enough tests, testing enough different things, to feel good about the code working.
Enough tests, testing obvious things, that stupid changes in the future will get found.
Not so many tests that the tests take forever to run and all the developers (including you) will put off running them because they don't want to wait or lose their train of thought while they run.
Balance!
I'm writing a Unit test for the behavior of a method, and want to try to reduce my code duplication - I'm about to double the test cases again, and the file is getting a bit hard to look at.
If anyone's really interested, the code is here (warning to the future - link may not be permanent).
Right now, I do the following, repeated several times in slightly different ways:
#Test
public void testAdd64OfMax64() {
// Set up container
TileEntityChest nmsinv = new TileEntityChest();
Inventory container = new CraftInventory(nmsinv);
// Call method
HashMap<Integer,ItemStack> leftover = container.addItem(new ItemStack(Foo, 64));
// Set up expected result
ItemStack[] expected = empty.clone();
expected[0] = new ItemStack(Foo, 64);
// Verify result
assertThat(container.getContents(), is(expected));
assertThat(leftover, is(Collections.EMPTY_MAP));
}
(Note: assertThat comes from org.hamcrest.Matchers.)
I know that I can put the "Set up container" and cloning the empty array into a #Before method.
My question is, can I put the assertThat method into an #After method and have the test still fail (when it should fail)?
No. The After method should be used to release external resources after a test run. Also, note that it is guaranteed to run even if your Test method throws an exception which may not be desirable in your case.
Also, think about the people who will have to maintain your code in the future. You are violating the PLA, by validating the results of your tests in the After method. Nobody expects that!
It would be better if you created a helper method e.g. checkResults which was called at the end of each of your Test methods to validate your results.