I've made an MethodRule and #Rule-annotation to make my test-life a bit easier.
It checks if a specific exception had been thrown and checks if the exception-message equals or contains the given message.
Now when i run a testmethod with more lines to test, it only takes the first line and than is ready. How do I make so all my lines in the testmethod are tested?
This is my code:
Annotation:
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.METHOD})
public #interface ExpectedDomeinValidatieMessage {
String value() default "";
String contains() default "";
}
MethodRule:
#Override
public Statement apply(final Statement base, final FrameworkMethod method, final Object target) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
ExpectedDomeinValidatieMessage message = method.getAnnotation(ExpectedDomeinValidatieMessage.class);
if (message == null) {
base.evaluate();
} else {
try {
base.evaluate();
Assert.fail("DomeinValidatieException not thrown");
} catch (DomeinValidatieException e) {
if (StringUtils.isNotBlank(message.value())) {
if (!e.getMessage().equals(message.value())) {
throwException(e, "", message.value(), e.getMessage());
}
}
if (StringUtils.isNotBlank(message.contains())) {
if (!e.getMessage().contains(message.contains())) {
throwException(e, "Segment niet gevonden:", message.contains(), e.getMessage());
}
}
}
}
}
private void throwException(Throwable exception, String message, String expected, String actual) {
ComparisonFailure cf = new ComparisonFailure(message, expected, actual);
cf.setStackTrace(exception.getStackTrace());
throw cf;
}
};
Usage:
#Test
#ExpectedDomeinValidatieMessage("[Werkzaamheden] WerkzaamMetGevaarlijkeStoffen niet gevuld")
public void valideerWerkzaamMetGevaarlijkeStoffen() throws DomeinValidatieException {
aanvraag.getVerzekerde().getWerkzaamheden().setWerkzaamMetGevaarlijkeStoffen(null);
validator.valideer();
}
If I use it like this, it only tests the first test in the method:
#Test
#ExpectedDomeinValidatieMessage("[Werkzaamheden] WerkzaamMetGevaarlijkeStoffen niet gevuld")
public void valideerWerkzaamMetGevaarlijkeStoffen() throws DomeinValidatieException {
aanvraag.getVerzekerde().getWerkzaamheden().setWerkzaamMetGevaarlijkeStoffen(null);
validator.valideer(); //Only this one is tested
aanvraag.getVerzekerde().getWerkzaamheden().setWerkzaamMetGevaarlijkeStoffen("bla");
validator.valideer(); //This is NOT tested
}
Run the code through a debugger. My guess is that the first call to valideer() does indeed throw an exception even though you don't expect it.
The JUnit assertXXX methods work by throwing exceptions (specifically AssertionError). So when an exception is thrown (either by your code, or by an assert) control exits from the test method. There isn't any way to restart from the place where the exception is thrown.
You probably want Parameterized, which allows you to run the same tests multiple times with different parameters.
EDIT: I suspect that valideer() is throwing an Exception. To explain a bit further, let's paraphrase your code. When you define a rule, what you're effectively doing is the following:
try {
base.evaluate(); // this calls valideerWerkzaamMetGevaarlijkeStoffen()
Assert.fail("DomeinValidatieException not thrown");
} catch (DomeinValidatieException e) {
// evaluate whether or not the test has failed or not
}
This means that if your first call to valideer() throws an Exception, then control is transferred to the catch block above. There isn't a chance to continue executing the test, because the control has passed elsewhere. You can pass/fail the test as much as you like, but control has passed to the catch block above.
By the way, MethodRule has been deprecated in the later versions, you should be using TestRule instead.
Related
I am writing a void validate method like this one:
void validate(SomeObject someObject) {
if (check someObject with constraints) {
throw new SomeException();
}
}
To unit test this method, I would like to test both legal and illegal input:
for illegal input, I used (expected = SomeException.class)
I am wondering how we could test the case when someObject is legal?
Legal in this case means that the validation completes without exceptions. To test it you just need to call validate.
#Test
public void testLegal() throws SomeException {
validator.validate(/* some legal object*/);
}
This test will fail only if an exception is thrown during validation.
1st Test Case :
#Test
public void testOK() throws Exception {
validate(someObject);
// At Least 1 assertion
}
2nd Test Case :
#Test(expected=SomeException.class)
public void testException() throws Exception {
try {
validate(someObject);
} catch(SomeException e) {
// At Least 1 assertion
throw e;
}
}
In this 2nd test case, if the exception is not thrown, the test will fail because it expects SommeException.class.
At least 1 assertion is mandatory because you must verify that the exception that has been thrown is well the one you were expected.
To test the case where someObject is legal, just call validate in a try block. If an exception is thrown, fail the test. If no exception is thrown, let the test end normally and pass, since that is the expected behavior.
Example would look something like:
#Test
public void myTest() {
try {
validate(new SomeObject());
} catch (Exception e) {
fail("expected no exception, but one was thrown.");
}
}
I always tested my exceptions with annotations.
#Test (expected = Exception.class)
public void test1() {
methodToTest() // throws an exception
}
I finally switched to Java 8, and I came across lambda Expressions. Now there is another option to get the desired result.
#Test
public void test2() {
assertThrows(Exception.class, () -> methodToTest());
}
public static <X extends Throwable> Throwable assertThrows(
final Class<X> exceptionClass, final Runnable block) {
try {
block.run();
} catch(Throwable ex) {
if (exceptionClass.isInstance(ex))
return ex;
}
fail("Failed to throw expected exception");
return null;
}
I understand that with the second version you can check for single methods more precisely, and you don't have to worry about other methods within a single test that could throw the expected exception as well. Furthermore, with an "assertThrows" method, all tests can have the same structure, because it all comes down to a call for an assertion.
Besides those two points, are there any pro arguments for the new way? For me, it feels like its still superior to go with the annotations, as long as I am only testing a single method within a single test.
You missed a third way, the ExpectedException jUnit rule:
public class SimpleExpectedExceptionTest {
#Rule
public ExpectedException thrown= ExpectedException.none();
#Test
public void myMethod_throws_no_exception_when_passed_greeting() {
fixture.myMethod("hello");
}
#Test
public void myMethod_throws_MyException_when_passed_farewell() {
thrown.expect(MyException.class);
fixture.myMethod("goodbye");
}
}
I find this clearer than the #Test (expected = ...) version, since the expectation goes closer to the method call.
There is also the plain old Java version, which we used to make do with:
try {
fixture.myMethod("should throw");
fail("Expected an exception");
} catch (MyException e) {
// expected
}
Which of the many is "better" depends entirely on context. Don't adopt one universally. Pick the one that gives you the clearest test in a given situation.
When you begin coding non-test code in a lambda-centric style, it is likely that you'll find yourself wanting to use the lambda-centric assertThrows().
If all you want to test is that an exception is thrown, the first syntax is better. It is standard, it is concise, and it prevents you from writing the same ugly try-catch over and over.
You could have a test slightly more complicated where you want to assert that an exception is thrown and some method is not called. In this case, manually catching the exception is reasonable.
#Test
public void test1() {
DBClient dbClient = spy(new DBClient());
try {
new Server().doRequest(new InvalidRequest(), dbClient);
fail("should have thrown");
} catch (InvalidRequestException e) {
verify(dbClient, times(0)).query(any(Query.class));
}
}
Regarding the use on lambdas more specifically, it's up to you. Just note that Runnable cannot throw checked exceptions so you would need something like
#FunctionalInterface
public interface ThrowingRunnable<E extends Exception> {
void run() throws E;
}
I don't see a problem with either approach--there is no right or wrong in matters of style; just use the one that best suits each situation. I suggest that assertThrows should also check for thrown exceptions that aren't of the expected type, and, as #Dici suggests, that a functional interface that allows checked exceptions be used:
public static <X extends Throwable> Throwable assertThrows(
final Class<X> exceptionClass, final CheckedRunnable block) {
try {
block.run();
} catch(Throwable ex) {
if (exceptionClass.isInstance(ex)) {
return ex;
} else {
throw new AssertionError("Unexpected exception was thrown", ex);
}
}
fail("Failed to throw expected exception");
}
#FunctionalInterface
public interface CheckedRunnable<R extends RuntimeException> {
void run () throws Exception;
}
I have a code segment like shown below. Each line of code throw same exception. However, in practice, when first line throws an exception, testFoo finishes its job and does not continue, as expected. But, I want a bit more different thing; since they are throwing same exception, I want to continue and check these three lines w.r.t the exception which they all throw. If they throw, test should be continue.
How can I test these three line w.r.t same exception?
#test
void testFoo(){
assertNull( /*errorMessage*/, ClassFoo.foo(null)); // foo will throw `AssertionError` due to null parameter
assertNull( /*errorMessage*/, ClassBar.bar(null)); // foo will throw `AssertionError` due to null parameter
assertNull( /*errorMessage*/, ClassGbr.gbr(null)); // foo will throw `AssertionError` due to null parameter
}
Just catch the exception yourself:
#Test
void testFoo() {
boolean fooHasThrownException = false;
boolean barHasThrownException = false;
boolean gbrHasThrownException = false;
try {
ClassFoo.foo(null);
fail();
} catch (AssertionError e) {
fooHasThrownException = true;
}
try {
ClassBar.bar(null);
fail();
} catch (AssertionError e) {
barHasThrownException = true;
}
try {
ClassGbr.gbr(null);
fail();
} catch (AssertionError e) {
gbrHasThrownException = true;
}
assertThat(true, equalTo(fooHasThrownException),
equalTo(barHasThrownException),
equalTo(gbrHasThrownException));
}
Note that your assertNull() is redundant. If a method throws an exception, it will not return anything.
On the side, this is a very weird scenario to be testing. If a method throws an exception, it just seems more logical to stop any further processing, if those processes down the line are going to also throw exceptions anyway.
I have tried to implement precondition for each parameters to a method with Java built-in Assert.assertTrue...
This is not built into java, this is from junit: void junit.framework.Assert.assertTrue(...). You are confusing Java Assertions with Junit assertions.
Junit assertions should be used in your unit tests. They look like Assert.assertEquals(result, "expected result"); They are intended to test the validity of the methods under test in your unit tests.
Java assertions should be used when verifying assumptions. They look like assert param!=null:"param should not be null!"; They are part of the java language and can be turned on and off at compile time. They are intended to double check assumptions in your code and to produce zero overhead when turned off.
Programming with assertions is a great thing. Using Junit assertions outside of unit tests is dubious.
My interpretation of this question is that you are expecting an AssertFailedError in your unit test and this is meant to be part of this test. If that is the case, you can use the following junit method structure:
#Test(expected = AssertFailedError.class)
public void testFoo() throws AssertFailedError
{
assertNotNull(null);
}
You can use this when you are testing a block of code that you know will throw an exception.
I wrote some test cases to test some method. But some methods throw an exception. Am I doing it correctly?
private void testNumber(String word, int number) {
try {
assertEquals(word, service.convert(number));
} catch (OutOfRangeNumberException e) {
Assert.fail("Test failed : " + e.getMessage());
}
}
#Test
public final void testZero() {
testNumber("zero", 0);
}
If I pass -45, it will fail with OutOfRangeException but I am not able to test specific exception like #Test(Expected...)
An unexpected exception is a test failure, so you neither need nor want to catch one.
#Test
public void canConvertStringsToDecimals() {
String str = "1.234";
Assert.assertEquals(1.234, service.convert(str), 1.0e-4);
}
Until service does not throw an IllegalArgumentException because str has a decimal point in it, that will be a simple test failure.
An expected exception should be handled by the optional expected argument of #Test.
#Test(expected=NullPointerException.class)
public void cannotConvertNulls() {
service.convert(null);
}
If the programmer was lazy and threw Exception, or if he had service return 0.0, the test will fail. Only an NPE will succeed. Note that subclasses of the expected exception also work. That's rare for NPEs, but common with IOExceptions and SQLExceptions.
In the rare case that you want to test for a specific exception message, you use the newish ExpectedException JUnit #Rule.
#Rule
public ExpectedException thrown= ExpectedException.none();
#Test
public void messageIncludesErrantTemperature() {
thrown.expect(IllegalArgumentException.class);
thrown.expectMessage("-400"); // Tests that the message contains -400.
temperatureGauge.setTemperature(-400);
}
Now, unless the setTemperature throws an IAE and the message contains the temperature the user was trying to set, the test fails. This rule can be used in more sophisticated ways.
Your example can best be handled by:
private void testNumber(String word, int number)
throws OutOfRangeNumberException {
assertEquals(word, service.convert(number));
}
#Test
public final void testZero()
throws OutOfRangeNumberException {
testNumber("zero", 0);
}
You can inline testNumber; now, it does not help much. You can turn this into a parametrized test class.
Remove the try-catch block and add throws Exception to your test method, like:
#Test
public final void testZero() throws Exception {
assertEquals("zero", service.convert(0));
}
JUnit expects failing tests will throw Exceptions, your catching them is just stopping JUnit from being able to report them properly. Also this way the expected property on the #Test annotation will work.
You don't need to catch the exception to fail the test. Just let it go (by declaring throws) and it will fail anyway.
Another case is when you actually expect the exception, then you put fail at the end of try block.
For example:
#Test
public void testInvalidNumber() {
try {
String dummy = service.convert(-1));
Assert.fail("Fail! Method was expected to throw an exception because negative numbers are not supported.")
} catch (OutOfRangeException e) {
// expected
}
}
You can use this kind of test to verify if your code is properly validating input and handles invalid input with a proper exception.
There are several strategies that are open to you to deal with expected exceptions in your tests. I think the JUnit annotations and try/catch idiom have already been mentioned above. I'd like to draw attention to the Java 8 option of Lambda expressions.
For instance given:
class DummyService {
public void someMethod() {
throw new RuntimeException("Runtime exception occurred");
}
public void someOtherMethod(boolean b) {
throw new RuntimeException("Runtime exception occurred",
new IllegalStateException("Illegal state"));
}
}
You can do this:
#Test
public void verifiesCauseType() {
// lambda expression
assertThrown(() -> new DummyService().someOtherMethod(true))
// assertions
.isInstanceOf(RuntimeException.class)
.hasMessage("Runtime exception occurred")
.hasCauseInstanceOf(IllegalStateException.class);
}
Take a look at this blog which covers most of the options with examples.
http://blog.codeleak.pl/2013/07/3-ways-of-handling-exceptions-in-junit.html
And this one explains the Java 8 Lambda option more fully:
http://blog.codeleak.pl/2014/07/junit-testing-exception-with-java-8-and-lambda-expressions.html
I have some unit tests which exercise code which makes calls out to a test server, in order to make sure that the requests are well-formed (i.e. we get valid data back in response). However, this means that the unit tests, and hence the build, can get blocked if this test server is down. This does not conform to good unit test practices, but as a thought experiment let's say I'm not allowed to delete these tests or change them so they don't actually call out to the server. I want to change them so that they will still pass if the server is down (i.e. trying to connect results in ConnectException), but fail if any other exception occurs. Making it more difficult, the code under test doesn't throw the ConnecException directly, but throws a wrapper exception that contains it. So initially, that means each test will go from looking like this:
#Test
public void testNumberOne() {
// body of test...
}
To this:
#Test
public void testNumberOne() {
try {
// body of test...
} catch (ThirdPartyWrapperException e) {
if (!(e.getRootCause() instanceof ConnectException) {
throw e;
}
}
}
Is there any way I can avoid having to paste that try/catch into each unit test?
I know I can refactor out at least some of it, ala:
#Test
public void testNumberOne() {
try {
// body of test...
} catch (ThirdPartyWrapperException e) {
handleException(e);
}
}
private void handleException(ThirdPartyWrapperException e)
throws ThirdPartyWrapperException {
if (!(e.getRootCause() instanceof ConnectException) {
throw e;
}
}
But is there anything further I can do?
I would add a line to the start to determine if the required resources are available
#Test
public void testNumberOne() {
if (!requiredServerAvailable()) return;