I have set up some JUnit (4.12) test with the ExpectedException feature, and I would like the test to continue after the expected exception. But I never see the log '3', as the execution seems to stop after the exception, event if catch?
Is this actually possible, and how?
#Rule
public ExpectedException exception = ExpectedException.none();
#Test
public void testUserAlreadyExists() throws Exception {
log.info("1");
// Create some users
userService.createUser("toto1");
userService.createUser("toto2");
userService.createUser("toto3");
Assert.assertTrue( userService.userExists("toto1") );
Assert.assertTrue( userService.userExists("toto2") );
Assert.assertTrue( userService.userExists("toto3") );
log.info("2");
// Try to create an existing user
exception.expect(AlreadyExistsException.class);
userService.createUser("toto1");
log.info("3");
}
You cannot do that, when the exception is thrown it's thrown for real, ExpectedException rule or not.
If you really want this kind of behaviour, you can go back to the "old school" pattern:
try {
userService.createUser("toto1");
Assert.fail("expecting some AlreadyExistsException here")
} catch (AlreadyExistsException e) {
// ignore
}
log.info("3");
But I wouldn't bother for some log.
This SO solution seems to do what you want to do: JUnit continue to assert things after expected exception
I myself was thinking something similar. To continue with the test, you would have to catch the exception yourself in the test. This solution shows an elegant way of doing that.
Note: If you make a rule to expect an exception (as you did), the test will return successful as soon as that exception is thrown.
Reference: http://junit.org/javadoc/latest/org/junit/rules/ExpectedException.html
If you don't want to add a lot of similar test methods for something that has many options to throw the expected exception and want to verify that it actually throws on all of the desired cases within a single unit-test instead, I'd suggest this (not pretty maybe) helpful schema:
#Test
public void testThatSomethingExpectedlyFails() {
for (int i = 1; i <= 3; i++) {
try {
switch (i) {
case 1: // smth here throws the exception when configuration #1;
case 2: // smth here throws the exception when configuration #2;
case 3: // smth here throws the exception when configuration #3;
}
} catch (ExceptionThatIsExpected expected) {
continue;
} catch (Exception unexpected) {
/* the test must fail when an unexpected exception is thrown */
fail("The test has failed due to an unexpected exception: " + unexpected.getMessage()); // or just re-throw this exception
}
/* the test must fail when a case completes without the expected exception */
fail("No expected exception occurred at case " + i);
}
}
The one could also iterate items (and even execute functions) of some preliminarily prepared list instead of switch-case with hard-coded integers.
First of all your test doesn't test one thing. It tests "userExists" and "createUser" under different conditions a.k.a. different scenarios. This is called an AssertionRoulette. You wouldn't need a hack to continue to log "3", if you would write tests, that fail fo the right reason.
If the tests fail for the right reason, you can see the scenario why it fails without doing all the logging stuff. The Junit-Runner does the logging for you already.
#Test
public void testUserExists_UserCreatedUserNotExistent_expectTrue()
{
// Create some users
userService.createUser("toto1");
// Assert That user exists
Assert.assertTrue( userService.userExists("toto1") );
}
#Test
public void testCreateUser_UserAlreadyCreated_expectAlreadyExistsExceptionIsThrown()
{
// Create some users
userService.createUser("toto1");
// Try to create an existing user
exception.expect(AlreadyExistsException.class);
userService.createUser("toto1");
}
Related
I am trying to test that my controller will indeed throw an error with the response code of 409 conflict. My controller looks like this:
if (Object != null) {
return Object;
} else {
throw new WebApplicationException(Response.Status.CONFLICT);
}
In the unit test I have something like this:
assertThrows(WebApplicationException.class, () -> controller.createObject();
Although I feel like this test isn't comprehensive enough as I haven't verified the response code of the call. How does one do that, if it is possible?
Instead of asserting that an exception is thrown, you need to enclose the code that should throw the exception within a try / catch, catch the exception, then get the response from the exception object and check its status code. Something like this:
try {
controller.createObject();
assertFail("should have thrown an exception");
} catch (WebApplicationClass ex) {
assertEquals(404, ex.getResponse().getStatusCode());
}
assetThrow method description
Assert that execution of the supplied executable throws an exception of the expectedType and return the exception.
If no exception is thrown, or if an exception of a different type is thrown, this method will fail.
If you do not want to perform additional checks on the exception instance, simply ignore the return value.
the return value of the assertThrows method is your exception instance, so you can do this
WebApplicationClass e = assertThrows(HttpException.class, () -> controller.createObject());
// perform additional checks
assertEquals(xxx, e.getResponse().getxxx);
Using below test (JUnit4) the Asserts are not invoked. Is this the expected behaviour ? I expect ResponseStatusException to be thrown, the test verifies this but then I wish to assert on the response. Or is the reasoning as an exception is thrown checking the content of the response is not valid ?
#Test(expected = ResponseStatusException.class)
public void testResponse(){
final Long ticketId = null
when(service.getTicket(null))
.thenThrow(new NullPointerException("ticketId cannot be null"));
//Execute
ResponseEntity<List<TicketResponse>> response = service.getTicket(null);
assertEquals(HttpStatus.OK, response.getStatusCode());
}
Yes, this is normal, though note you won't be able to verify the response anyway, as an exception is thrown so you don't get a response! You could verify the exception state however.
For that, you might want to read up on the "Exception Testing" page of the official Junit 4 documentation (code taken from there), where you basically use the assertThrows method and not #Test(expected=), which allows you to do more verifications.
Another alternative would be using the ExpectedException Rule. Again, see the link for an example.
https://github.com/junit-team/junit4/wiki/Exception-testing
#Test
public void testExceptionAndState() {
List<Object> list = new ArrayList<>();
IndexOutOfBoundsException thrown = assertThrows(
IndexOutOfBoundsException.class,
() -> list.add(1, new Object()));
// assertions on the thrown exception
assertEquals("Index: 1, Size: 0", thrown.getMessage());
// assertions on the state of a domain object after the exception has been thrown
assertTrue(list.isEmpty());
}
I'm testing a method that throws two different exceptions. This is my header:
#Test (expected = A8InvalidInputException.class)
public void testGuessCharacter() throws A8InvalidInputException, A8AlreadyGuessedException
{
...
}
The body has two try/catch blocks (a search on SO resulted in a post that said that's how you test that exceptions are thrown), one for each exception. It seems to me I should break this up into two test methods, especially because I can only have one expected attribute. However, when I do that, the method that is supposed to be testing A8InvalidInputException is requiring a try/catch for A8AlreadyGuessedException, and the method that is supposed to be testing A8AlreadyGuessedException is requiring a try/catch for A8InvalidInputException. I'm not really sure how to write this test. This is the method I'm trying to test:
/**
* This method returns whether a specified character exists in the keyPhrase field
* #param guess a character being checked for in the keyPhrase field
* #return returns whether a specified character exists in the keyPhrase field
* #throws A8InvalidInputException if a non-valid character is passed as input to this method
* #throws A8AlreadyGuessedException if a valid character which has already been guessed is passed as input to this method
*/
public boolean guessCharacter(char guess) throws A8InvalidInputException, A8AlreadyGuessedException
{
if(isValidCharacter(guess))
{
guess = Character.toLowerCase(guess);
if(guessedCharacters.contains(guess) )
{
throw new A8AlreadyGuessedException("" + guess);
}
else
{
guessedCharacters.add(guess);
if(keyPhrase.contains("" + guess))
return true;
else
{
numberOfGuessesLeft--;
return false;
}
}
}
else
{
throw new A8InvalidInputException("" + guess);
}
}
Just add both exceptions in the throws clause:
#Test (expected = A8InvalidCharacterException.class)
public void testInvalidCharacterScenario() throws A8InvalidInputException, A8AlreadyGuessedException { ... }
#Test (expected = A8InvalidInputException.class)
public void testInvalidInputScenario() throws A8InvalidInputException, A8AlreadyGuessedException { ... }
Then, if one test throws the other exception (the unexpected one) then your test will automatically fail.
A method can throw only one exception when it is ran. That’s why there can be only one expected attributed.
You may want to have three test cases: one when the method throws one exception, one when the method throws the other, and one when the method doesn’t throw any exception at all.
Do not put any try/catch statement in your #Test methods, just declare that they throw exceptions.
Yes, you should break this up into two unit tests. One with an invalid input to trigger the A8InvalidInputException and another with an "already guessed" input to trigger the A8AlreadyGuessedException.
Consider, to make writing your tests a little simpler, breaking them out into two distinct portions - testing isValidCharacter and testing guessCharacter separately.
Presuming that isValidCharacter(guess) will fail if you receive an invalid guess, I think that throwing A8InvalidInputException in that method would be ideal.
public boolean isValidCharacter(char guess) throws A8InvalidInputException {
// have your logic to check the guess, and if it's invalid, throw
}
Then all you would need to do is test that particular method to see if it throws the exception on bogus input.
#Test (expected = A8InvalidInputException.class)
public void testIsValidCharacterWithInvalidCharacter() {
// write your test here.
}
Next, you can change your method to only care about the happy-path of the isValidCharacter method, since if you don't return the boolean, you've thrown the exception.
Lastly, you would only concern the tests for guessCharacter around whether or not it throws A8AlreadyGuessedException.
#Test (expected = A8AlreadyGuessedException.class)
public void testGuessCharacterWithAlreadyGuessedValue() {
// write your test here.
}
I am trying my hand at writing test cases. From what I have read, my tests should fail from the start and I should strive to make tests pass. However, I find myself writing tests checking boundaries and the exceptions they should cause:
#Test(expected=NegativeArraySizeException.class)
public void testWorldMapIntInt() {
WorldMap w = new WorldMap(-1, -1);
}
#Test(expected=IndexOutOfBoundsException.class)
public void testGetnIntnInt() {
WorldMap w = new WorldMap(10,10);
Object o = w.get(-1, -1);
}
However, this test passes by default because Java will throw the exception anyway. Is there a better way to handle these kinds of expected exceptions, possibly a way that fails by default-- forcing me to strive to handle these cases?
I agree that the style you present is not so good. The problem is that it doesn't check where in the method the exception is thrown, so it's possible to get false negatives.
We usually write tests for exceptions like this:
public void testWorldMapIntInt() {
try {
WorldMap w = new WorldMap(-1, -1);
Assert.fail("should have thrown IndexOutOfBoundsException");
}
catch (IndexOutOfBoundsException e) {}
}
Expected behaviour for WorldMap is to throw an exception if (-1, -1) passed into it
Initially it doesn't do that, so your test will fail as it does not see expected exception.
You implement the code for WorldMap correctly, including throwing exception when (-1, -1) passed in.
You rerun your test, it passes.
Sound like good TDD to me!
That seems like a fair test to write. WorldMap is no standard Java class. Presumably it's your own class. Therefore the test wouldn't be passing if you hadn't already written some code. This test will force you to throw (or propagate) an appropriate exception from your class. That sounds like a good test to me, which you should write before implementing the behavior.
I personally look for mistakes like that in the WorldMap constructor and throw an IllegalArgumentException, that way you can provide a better error message, such as what the value passed in was and what the expected range is.
As for having that test fail by default, I cannot think of a reasonable way of doing that if you are going to have it actually do something (if you are writing the tests first then it should fail because the constructor won't have any code).
Agree with accepted answer, try-fail-catch idiom, although ugly and cluttering the test, is much better than #Test(expcted=...) as it might report false positives.
A while back I implemented very simple JUnit rule to deal with exception testing in both safe and readable manner:
public class DefaultFooServiceTest {
#UnderTest
private FooService fooService = new DefaultFooService();
#Rule
public ExceptionAssert exception = new ExceptionAssert();
#Test
public void shouldThrowNpeWhenNullName() throws Exception {
//given
String name = null;
//when
fooService.echo(name);
//then
exception.expect(NullPointerException.class);
}
#Test
public void shouldThrowIllegalArgumentWhenNameJohn() throws Exception {
//given
String name = "John";
//when
fooService.echo(name);
//then
exception.expect(IllegalArgumentException.class)
.expectMessage("Name: 'John' is not allowed");
}
}
See blog post and source.
I'm pretty new to JUnit, and I don't really know what best practices are for exceptions and exception handling.
For example, let's say I'm writing tests for an IPAddress class. It has a constructor IPAddress(String addr) that will throw an InvalidIPAddressException if addr is null. As far as I can tell from googling around, the test for the null parameter will look like this.
#Test
public void testNullParameter()
{
try
{
IPAddress addr = new IPAddress(null);
assertTrue(addr.getOctets() == null);
}
catch(InvalidIPAddressException e)
{
return;
}
fail("InvalidIPAddressException not thrown.");
}
In this case, try/catch makes sense because I know the exception is coming.
But now if I want to write testValidIPAddress(), there's a couple of ways to do it:
Way #1:
#Test
public void testValidIPAddress() throws InvalidIPAddressException
{
IPAddress addr = new IPAddress("127.0.0.1");
byte[] octets = addr.getOctets();
assertTrue(octets[0] == 127);
assertTrue(octets[1] == 0);
assertTrue(octets[2] == 0);
assertTrue(octets[3] == 1);
}
Way #2:
#Test
public void testValidIPAddress()
{
try
{
IPAddress addr = new IPAddress("127.0.0.1");
byte[] octets = addr.getOctets();
assertTrue(octets[0] == 127);
assertTrue(octets[1] == 0);
assertTrue(octets[2] == 0);
assertTrue(octets[3] == 1);
}
catch (InvalidIPAddressException e)
{
fail("InvalidIPAddressException: " + e.getMessage());
}
}
Is is standard practice to throw unexpected exceptions to JUnit or just deal with them yourself?
Thanks for the help.
Actually, the old style of exception testing is to wrap a try block around the code that throws the exception and then add a fail() statement at the end of the try block. Something like this:
public void testNullParameter() {
try {
IPAddress addr = new IPAddress(null);
fail("InvalidIPAddressException not thrown.");
} catch(InvalidIPAddressException e) {
assertNotNull(e.getMessage());
}
}
This isn't much different from what you wrote but:
Your assertTrue(addr.getOctets() == null); is useless.
The intend and the syntax are clearer IMO and thus easier to read.
Still, this is a bit ugly. But this is where JUnit 4 comes to the rescue as exception testing is one of the biggest improvements in JUnit 4. With JUnit 4, you can now write your test like this:
#Test (expected=InvalidIPAddressException.class)
public void testNullParameter() throws InvalidIPAddressException {
IPAddress addr = new IPAddress(null);
}
Nice, isn't it?
Now, regarding the real question, if I don't expect an exception to be thrown, I'd definitely go for way #1 (because it's less verbose) and let JUnit handle the exception and fail the test as expected.
For tests where I don't expect an exception, I don't bother to catch it. I let JUnit catch the exception (it does this reliably) and don't cater for it at all beyond declaring the throws cause (if required).
I note re. your first example that you're not making use of the #expected annotation viz.
#Test (expected=IndexOutOfBoundsException.class) public void elementAt() {
int[] intArray = new int[10];
int i = intArray[20]; // Should throw IndexOutOfBoundsException
}
I use this for all tests that I'm testing for throwing exceptions. It's briefer than the equivalent catch/fail pattern that I had to use with Junit3.
Since JUnit 4.7 you have the possibility to use an ExpectedException rule and you should use it. The rule gives you the possibility to define exactly the called method where the exception should be thrown in your test code. Moreover, you can easily match a string against the error message of the exception. In your case the code looks like this:
#Rule
public ExpectedException expectedException = ExpectedException.none();
#Test
public void test() {
//working code here...
expectedException.expect(InvalidIPAddressException.class);
IPAddress addr = new IPAddress(null);
}
UPDATE: In his book Practical Unit Testing with JUnit and Mockito Tomek Kaczanowski argues against the use of ExpectedException, because the rule "breaks the arrange/act/assert [...] flow" of a Unit test (he suggests to use Catch Exception Library instead). Although I can understand his argument, I think using the rule is fine if you do not want to introduce another 3rd-party library (using the rule is better than catching the exception "manually" anyway).
For the null test you can simply do this:
public void testNullParameter() {
try {
IPAddress addr = new IPAddress(null);
fail("InvalidIPAddressException not thrown.");
}
catch(InvalidIPAddressException e) { }
}
If the exception has a message, you could also check that message in the catch if you wish. E.g.
String actual = e.getMessage();
assertEquals("Null is not a valid IP Address", actual);
For the valid test you don't need to catch the exception. A test will automatically fail if an exception is thrown and not caught. So way #1 would be all you need as it will fail and the stack trace will be available to you anyway for your viewing pleasure.
if i understand your question, the answer is either - personal preference.
personally i throw my exceptions in tests. in my opinion a test failing by assertion is equivalent to a test failing by an uncaught exception. both show something that needs to be fixed.
the important thing to remember in testing is code coverage.
In general way #1 is the way to go, there is no reason to call out a failure over an error - either way the test essentially failed.
The only time way #2 makes sense if you need a good message of what went wrong, and just an exception won't give that to you. Then catching and failing can make sense to better announce the reason of the failure.
Reg: Testing for Exceptions
I agree with "Pascal Thivent", ie use #Test (expected=InvalidIPAddressException.class)
Reg: Testing for testValidIPAddress
IPAddress addr = new IPAddress("127.0.0.1");
byte[] octets = addr.getOctets();
I would write a test like
class IPAddressTests
{
[Test]
public void getOctets_ForAValidIPAddress_ShouldReturnCorrectOctect()
{
// Test code here
}
}
The point is when testinput is VALID ipAddress
The test must be on the public methods/capabilities on the class asserting that they are working as excepted
IMO it is better to handle the exception and show appropriate messaging from the test than throwing it from a test.