I recently found out that JUnit > 4.10 allows the usage of #Rule and ExpectedException. Since I'm not big on duplicating code I tried the following. For a better understanding I scaled it down from several tests to just these two. The MockitoJUnitRunner is intentional although it's not used in the small scaled example.
pom.xml
<dependencies>
<!-- Test dependencies -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.10.19</version>
<scope>test</scope>
</dependency>
</dependencies>
TestBase
#RunWith(MockitoJUnitRunner.class)
public class TestBase {
/** JUnit > 4.10 allows expected exception handling like this */
#Rule
public ExpectedException exception = ExpectedException.none();
#Before
public void setup() {
this.expectBadParam();
}
protected void expectBadParam() {
this.exception.expect(NullPointerException.class);
}
}
The problem is that the following test is not working as I would expect it to. What I'm trying is by default expect an exception type and in some cases run a normal JUnit test. I can't reset the expected exception once it's set.
public class ExpectedExceptionTest extends TestBase {
#Test
public void error() {
throw new NullPointerException();
}
#Test
public void success() {
this.exception = ExpectedException.none();
// this should be a success
}
}
I already found a different solution by duplicating the expectBadParam method in each method I expect an exception as well as overriding the #Before annotation in the test class. However I'm hoping someone can help me understand why this is not working?
The reason it's not working as you expect (no pun intended) is related to how TestRules work with JUnit.
Effectively what happens is that the test framework inspects the test case for TestRule instances, and then calls the TestRule.apply() method on each on in turn. This method takes a Statement object and returns a Statement(). Your test case object is initially wrapped in a Statement, given to the first TestRule, which returns a brand new Statement wrapping the original Statement. So, basically, the TestRule is being given the opportunity to adapt the original TestCase, generally adding new functionality. Once the framework has gone through all the TestRule instances it calls the Statement.evaluate() method, like it would do for any 'standard' test case it builds up which doesn't have any TestRules attached to it.
The key thing here is that all the interaction between the framework and the TestRule instances happens at test case construction time. Once the test case has been built up, the fields containing the rules are no longer queried or directly interacted with by the test framework. Their main purpose afterwards is for tests to interact with the mutable state contained within the rules. So, if you change the instance field as you do in your test case success() you'll have absolutely no effect on the outcome of the rules, because the rule expecting an IllegalArgumentException has already been applied to the test case.
There's a 'typical' shape to a TestRule implementation. They look like this...
public void apply(Statement base, Description description) {
return new Statement() {
public void evaluate( ) {
// some initialisation
try {
base.evaluate();
} finally {
// some tidy up here
}
}
}
}
Here the TestRule gets an opportunity to run some code after the test case has completed. This is how ExpectedException works (although it has a 'catch Exception(e)' block also). During the course of the test you can call methods on the rule instance which builds up state within the TestRule object which is then used when the finally block is called. So, when you call 'exception.expect(IllegalArgumentException.class)`, the test rule stores a matcher in a list and basically matches caught exceptions using that matcher and any others you may have set up. When you reset the instance field in your test case all that state in the original instance is still there and so the test still fails.
To do what you want to do you need a means of resetting the internal state of the ExpectedException instance. Unfortunately, there are no methods on the ExpectedException class which allow you to remove expectations which have been added. It's only really possible to add expectations. And, to be honest, this is for a good reason - your tests should be logically grouped, with finer grained details being added the closer you get to the test case. The act of 'resetting' expectations is an act of removing rather than adding a detail and so suggests that your tests are not logically grouped well enough. It creates maintainability difficulties if some part of a test suite adds some expectations and another part removes some / all of them.
You have 2 options if you want to use ExpectedException here. The first is to split your test class, or test base class, in two. One suite should be for tests which expect the IllegalArgumentException and another for ones which don't or which have some kind of alternative exception they expect. The second is to accept the duplication inherent in having 44 tests which have to explicitly declare they expect an exception and only 4 tests which don't.
It's possible you may be able to achieve what you want with JUnit Theories in some way, although that would depend very much on how your test cases work.
The solution is to Override the setup method,
You could also do it manually:
#Test
public void success() {
try{
... all your code
} catch (Exception e){
// check your nested clauses
if(e.getCause() instanceof ExpectedException){
// pass
} else {
Assert.fail("unexpected exception");
}
}
Please find bellow interesting links to learn more about :
testing-custom-exceptions-w-junits
Code showing how to test a custom exception by using a Hamcrest
Matcher
Good luck :)
Related
I want to mock a dependency and return a default value in most test cases since most of them should not care about the values returned but there are some certain cases like I would like to test like the dependency returns some weird values or just throw. So I am modeling it in this way. Most cases, it should return a nice and valid value.
Test Setup which return the 20L by default for all test classes.
Dependency dependency = Mockito.mock(Dependency.class);
when(dependency.returnSomeVal()).thenReturn(20L);
In a specific test cases class, I would like to override the behavior like below:
when(dependency.returnSomeVal()).thenThrow(); //failure cases
when(dependency.returnSomeVal()).thenReturn(Weird_Val); //failure cases
But I don't find a good solution to override the existing behavior? Any idea?
You can reset the mock and add behavior. In the test, do
Mockito.reset(dependency);
when(dependency.returnSomeVal()).thenThrow(); //failure cases
when(dependency.returnSomeVal()).thenReturn(Weird_Val); //failure cases
Resetting will remove all mocked behavior on this class though. If you want to remock only some methods, then you have to create the mock from scratch.
I ended using myself this pattern to mock a bunch of methods of a class providing configurations.
In a #Before method I setup a bunch of stubs for a mocked object that provide a correct configuration for each test. Afterwards, in each test it was extremely convenient to only override one of those stubs to provide a different configuration and test a different error case.
I think the response from Hari Menon is correct but it somehow defeats the purpose explained in the question. If the mock is reset, all the stubs would need to be added again, making this pattern very confusing (it would be better to not use any overriding than using reset in this case, the code would be way more straightforward).
The comments added to the question provide indeed an indirect answer on how to achieve this, and why it works, but it took me a bit to get it working.
In spite of one of the comments, I made everything work by using in my #Before fixture when().thenReturn() and overriding the concrete stub with doReturn().when()
Example:
public class WorkerTest {
private ConfigProvider mockedConfigProvider = mock(ConfigProvider.class);
#Before
public void setup() {
// Setup stubs with a correct config
when(mockedConfigProvider.getValue("property1")).thenReturn("value1");
when(mockedConfigProvider.getValue("property2")).thenReturn("value2");
when(mockedConfigProvider.getValue("property3")).thenReturn("value3");
when(mockedConfigProvider.getValue("property4")).thenReturn("value4");
}
#Test
public void test_GoodConfig(){
// The config object gets injected in the test worker
Worker testWorker = new Worker(mockedConfigProvider);
// testWorker.execute() returns true if everything went well
assertTrue(testWorker.execute());
}
#Test
public void test_BadConfigProp1(){
// Test now with a broken 'property1', overriding that stub.
doReturn(null).when(mockedConfigProvider).getValue("property1");
Worker testWorker = new Worker(mockedConfigProvider);
// testWorker.execute() returns false if there is a problem.
assertFalse(testWorker.execute());
}
#Test
public void test_BadConfigProp2(){
// This test needs to only override the result of property2
doReturn("crazy result").when(mockedConfigProvider).getValue("property2");
...
}
I'm having issues with SonarQube raising issues with several of my unit tests, prompting the following issue:
Add at least one assertion to this test case.
Each test case resembles this format (where a number of assertions are delegated to a method with common assertions, to avoid duplication):
#Test
public void companyNameOneTooLong() throws Exception {
AddressFormBean formBean = getValidBean();
formBean.setCompanyNameOne("123456789012345678901234567890123456");
assertViolation(validator.validate(formBean), "companyNameOne", "length must be between 0 and 35");
}
private void assertViolation(Set<ConstraintViolation<AddressFormBean>> violations, String fieldname, String message) {
assertThat(violations, hasSize(1));
assertEquals(fieldname, violations.iterator().next().getPropertyPath().iterator().next().getName());
assertEquals(message, violations.iterator().next().getMessage());
}
Now, obviously I could just pull the three assertions out of the private method and put them in the test method - but I'm performing the same checks (on different fields) multiple times.
So, I thought I'd try to emulate the behaviour of the assertion methods, by (re) throwing an AssertionError:
private void assertViolation(Set<ConstraintViolation<AddressFormBean>> violations, String fieldname, String message) throws AssertionError {
try {
assertThat(violations, hasSize(1));
assertEquals(fieldname, violations.iterator().next().getPropertyPath().iterator().next().getName());
assertEquals(message, violations.iterator().next().getMessage());
} catch (AssertionError e) {
throw e;
}
}
Unfortunately, this approach does not work either.
What's special about the JUnit assert methods / what is SonarQube looking for specifically to check that an assertion has been made for each test?
Alternatively - are there other approaches to achieve the same end result (avoiding duplicating the shared assertion code over and over)?
The rule S2699 (Tests should include assertions) from the SonarQube Java Analyzer does not perform cross-procedural analysis and only explore body of methods being identified as test method (usually annotated with #Test).
Consequently, if the only assertions which will be called when executing the test method are done by a dedicated method (to avoid duplication), then the rule will raise an issue. This is a known limitation of the rule and we will deal with it only when we will be able to efficiently perform cross-procedural analysis.
Regarding the issues raised by SonarQube on such cases, you can safely mark them as Won't Fix.
Regarding the detected assertions, the rule consider as assertions the usual assert/fail/verify/expect methods from the following (unit test) frameworks :
JUnit
Fest (1.x & 2.x)
AssertJ
Hamcrest
Mockito
Spring
EasyMock
If you don't expect any exception to be throw from your test, this can be a workaround:
#Test(expected = Test.None.class /* no exception expected */)
Alternatively, you can suppress the warning for the test method/test class:
#SuppressWarnings("squid:S2699")
One thing I have done in the past is to have the helper method return true, and assert on that:
#Test
public void testSomeThings() {
Thing expected = // . . .
Thing actual = service.methodReturningThing(42);
assertTrue(assertViolation(expected, actual));
}
private boolean assertViolation(Thing expected, Thing actual) {
assertEquals(expected.getName(), actual.getName());
assertEquals(expected.getQuest(), actual.getQuest());
assertEquals(expected.getFavoriteColor(), actual.getFavoriteColor());
return true;
}
I hate this, but I hate duplicated code even more.
The other thing we've done at times, is to simply mark any such objections from SonarQube as Won't Fix, but I hate that, too.
Sometimes you don't need to have any code or assertation, for example, the test of load the context of spring boot successfully. In this case, to prevent Sonar issue when you don't expect any exception to be throw from your test, you can use this part of the code:
#Test
void contextLoads() {
Assertions.assertDoesNotThrow(this::doNotThrowException);
}
private void doNotThrowException(){
//This method will never throw exception
}
I was trying to write unit test using jmocks and junit. (My Project uses core java- no frameworks-) I could not write unit test for some of my classes, by mocking external dependencies, when dependencies were initialized in a a no arg-constructor.
As i cannot provide the actual code, trying to explain the scenario by an example
public interface Apple {
String variety();
}
Implementation.
public class MalgovaApple implements Apple {
#Override
public String variety() {
return "Malgova";
}
}
Class to be tested
public class VarietyChecker {
private Apple apple;
VarietyChecker(){
this.apple = new MalgovaApple();
// instead of new, a factory method is used in actual application
}
public String printAppleVariety(){
String variety = apple.variety();
if(variety.length() < 3){
System.out.println("Donot use Code names- Use complete names");
return "bad";
}
return "good";
}
}
Junit test using jmock
public class VarietyCheckerUnitTest{
Mockery context = new JUnit4Mockery();
#Before
public void setUp() throws Exception {
}
#After
public void tearDown() throws Exception {
}
#Test
public void test_VarietyChecker() throws Exception{
final Apple mockapple = context.mock(Apple.class);
VarietyChecker printer = new VarietyChecker();
context.checking(new Expectations(){{
oneOf(mockapple).variety();will(returnValue("as"));
}});
String varietyNameValid = printer.printAppleVariety();
assertEquals("bad",varietyNameValid);
} }
This test fails - Mocking does not work the values "as" is not injected, the test class executes with MalgovaApple ...
Now if we add below constructor to VarietyChecker and use it test case - it gives expected output...
public VarietyChecker(Apple apple) {
super();
this.apple = apple;
}
and in unit test create test class object like
VarietyChecker printer = new VarietyChecker(mockapple);
Exposing a new constructor just for the purpose of testing is not a good idea. After all it is said that you should not alter the code for testing alone, more than that, i am afraid we have already written "some"(amount) code...
Am i missing something in junit or jmock that can make mocking work even incase of no-arg constructors. Or is this a limitation of simple junit and jmocks and should i migrate to something powerful like Jmockit /PowerMock
You should consider two choices.
Use a constructor parameter as you describe.
In this case, you're not "exposing a new constructor just for the purpose of testing". You're making your class more flexible by allowing callers to use a different factory implementation.
Don't mock it.
In this case, you are declaring that it never makes sense to use a different factory. Sometimes this is okay. At that point, the question changes, though. Instead of, "How do I mock this?" your question is now, "What am I gaining from writing this test?" You might not be gaining much of anything, and it might not make much sense to write the test at all.
If you don't mock it and decide a unit test is still worth it, then you should be asserting on other aspects of the code. Either an end state or some output. In this case, the factory call becomes an implementation detail that's not appropriate for mocking.
It's important not to fall for a "unit test everything" mentality. That is a recipe for Test-induced Design Damage. Evaluate your tests on a case by case basis, deciding whether they're providing you any real value or not. Not writing a unit test is a valid option and is even appropriate at times, even if it's option you try very hard to avoid.
Only you can make a determination which one makes the most sense in this case. From the the fact that this is a factory object we're talking about, I'd probably lean toward the former.
I have a series of data each one containing info_A and info_B. I would like to:
if(info_A) {
run Test A
} else if(!info_A) {
run Test B
}
It is very important that only the Test actually run is shown in the JUnit GUI tree. How can I do this?
The following solutions do not work:
If I use the Assume.assumeTrue(conditon), I can Ignore a test but then it is still displayed as passed in the test.
Doing this:
Result result = JUnitCore.runClasses(TestStep1.class);
leads to the correct result but the JUnit tree is not built.
Using #Catagories also shows the failed tests, which is also what I don't want.
You can use Assume to turn tests on/off conditionally:
A set of methods useful for stating assumptions about the conditions in which a test is meaningful. A failed assumption does not mean the code is broken, but that the test provides no useful information.
You can do this with JUnit Rules.
There's a whole blog post dedicated to exactly your question here, but in short, you do the following:
Create a class that implements the org.junit.rules.TestRule interface. This interface contains the following method:
Statement apply(Statement base, Description description);
The base argument is actually your test, which you run by calling base.execute() -- well, the apply() method actually returns a Statement anonymous instance that will do that. In your anonymous Statement instance, you'll add the logic to determine whether or not the test should be run. See below for an example of a TestRule implementation that doesn't do anything except execute your test -- and yes, it's not particularly straighforward.
Once you've created your TestRule implementation, then you need to add the following lines to to your JUnit Test Class:
#Rule
public MyTestRuleImpl conditionalTests;
Remember, the field must be public.
And that's it. Good luck -- as I said, you may have to hack a little, but I believe there's a fair amount explained in the blog post. Otherwise, there should be other information on the internets (or in the JUnit sourc code).
Here's a quick example of a TestRule implementation that doesn't do anything.
public abstract class SimpleRule implements TestRule {
public Statement apply(final Statement base, final Description description) {
return new Statement() {
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable t) {
t.printStackTrace();
}
}
};
}
}
I don't know why, but I have always written my JMock tests like this:
#Test
public void testMyThing() throws Exception {
mockery.checking(new Expectations() {{
oneOf(mockObj).foo();
}});
testObj.bar(); // calls mockObj.foo()
mockery.assertIsSatisfied();
}
But when there are many tests, is it better to move assertIsSatisfied to the tear-down?
#After
public void tearDown() throws Exception {
mockery.assertIsSatisfied();
}
The recommended way to do this is to use the JMock runner. Annotate the class with
#RunWith(JMock.class)
public class TestClass {
This will call the assertion at the right place in the test lifecycle. Teardown isn't the right place as a failure might not be reported in correctly and might mess up other cleanup.
We also have a mockery rule in the repository that works with the new #Rule infrastructure.
Yes, I tend to do it in teardown. It keeps the focus of the individual test methods on what they're actually testing, by removing the boilerplate out into the #After- it's critical to me that tests are as expressive and readable as possible.
In fact, I sometimes take it further than that, and use a JMockSupport base class that handles the Mockery for me (as well as providing convenience implementations of mock(...)). This is just a convenience, of course, and by no means a requirement like it was in JUnit 3.