Can you please help me better understand, what is an appropriate use of “assert” vs “throwing an exception? When is each scenario appropriate?
Scenario 1
CODE
public Context(Algorythm algo) {
if (algo == null) {
throw new IllegalArgumentException("Failed to initialize Context");
}
this.algo = algo;
}
TEST
public void testContext_null() {
try {
context = new Context(null);
fail();
} catch (IllegalArgumentException e) {
assertNotNull(e);
}
}
Scenario 2
CODE
public Context(Algorythm algo) {
assert (algo != null);
this.algo = algo;
}
TEST
public void testContext_null() {
try {
context = new Context(null);
fail();
} catch (AssertionFailedError e) {
assertNotNull(e);
}
}
The main difference with assert is;
the ability to turn on/off selected tests by class/package.
the error thrown.
assert is more approriate for tests which will be turned off in production.
If you want a test which is checked every time, esp if validating data from an input, you should use the check which runs every time.
Assert is a macro (in C/C++, or a function in other languages) that validates a given expression as true or false, and throw an exception in case of false values.
Assert is something to use when ddebugging an application, like when you must check if a math expression really gives you an appropriate value, or if an object/structure member is not null or missing something important, and things like that.
An Exception throwing is more of a real error treatment. Exceptions are errors too and can stop your application, but they are used as the (let's say) "retail version" error treatment of the application. That's because Exceptions can be caught and taken differently to the user, with a little non-technical message instead of symbols and memory addresses, while you can just serialize that into an app log, for example.
On the other hand, asserts will just stop the running process and give you a message like "Assertion failed on source_file.ext, line X. The process will be terminated." And that's not user-friendly :)
The assert keyword should be used when failure to meet a condition violates the integrity of the program. These are intended to be non-recoverable error situations.
Exceptions, on the other hand, alert calling methods to the presence and location of an error but can be handled or ignored at the programmer's discretion.
When testing, you should use the Assert functions when a condition must be met for a test to pass. If you're expecting an exception in that particular test, JUnit 4 has an annotation to signify that an test should throw a particular Exception:
#Test(expected=MyException.class)
Outside of test code, asserts are generally a bad idea. the reason is that unless there are very strict company guidelines in place, you invariably end up with mixed usage, which is bad. there are basically 2 usage scenarios for assert:
extra, possibly slow tests which will be turned off in production
normal, quick code sanity tests which should never be disabled (like requiring a given method parameter to be non-null)
As long as you always follow one of the scenarios, things are fine. however, if your code base ends up with both scenarios, then you are stuck. you have asserts which follow scenario 2 which you don't want to disable, and you have asserts which follow scenario 1 (and are slowing down your production code) which you want to disable. what to do?
most codebases which i have worked with which used asserts in normal code, never ended up disabling them in the production build for exactly this reason. therefore, my recommendation is always to avoid them outside of test code. use normal exceptions for the normal code, and stick the extra, possibly slow code (with asserts) in separate test code.
Related
This question is similar to this one, but the focus is somewhat different
I have a function that loads a file on disk in a try catch block. Because the file is crucial to the program, it will terminate if it fails to load the file:
String loadSuperImportantFile() {
try {
// ... file loading code ...
assert fileContent != null;
return fileContent;
} catch(IOException ex) {
System.err.println("Failed to load super important file. Please check path.");
System.exit(Codes.FAIL); // terminate the program
}
return null; // statement is unreachable
}
IntelliJ IDEA does not recognize that it is impossible for this function to return null. Because of this, I get warnings whenever I use the return string:
foo(loadSuperImportantFile()); // WARNING: Argument "loadSuperImportantFile()" might be null.
From reading the question I linked up above, I believe it may be possible to use Method Contracts to tell IntelliJ that the method cannot return null. I tried using the #Contract("null -> fail") decorator, but the warnings did not disappear.
Does anyone have an ideas on how to make the warnings disappear in the method itself, using a Method Contract or something similar, rather than null checking externally?
System.exit(Codes.FAIL); is not a terminating statement in Java (like return or throw).
In some wired cases, you could imagine, that it is possible to override exit method (or mock it), in such a way it doesn't terminate the application. Then this null would be returned.
For a concise and robust application, if you want to terminate the application, throw an Exception that would propagate up. Especially, if you want to terminate your application because an error occurred (invalid path). It would be even more pragmatic, to let the application die on it's own. Why do you have to call System.exit();?
PS: You can also see the answers of #Stephen C or #user31601, and that would surely fix your problem with method returning null (because it uses flow control statement - throw), but I wouldn't suggest that option. In my opinion, it would be better to design a better structue, one that is concise, and doesn't let such situation occur - in opposite to allow it to happen, and then throw AssertionException when it does.
PS2: You could also add #NotNull like #yole suggested, but again - instead of reacting to thing that shouldn't happen - just don't let it happen. And throw (for example) throw new InvalidPathException();.
My suggestion:
String loadSuperImportantFile() {
try {
return fileContent;
} catch(IOException ex) {
throw new ImportantFileMissingException("Failed to load super important file. Please check path.");
}
}
class ImportantFileMissingException extends RuntimeException {}
IntelliJ IDEA does not recognize that it is impossible for this function to return null.
IntelliJ is simply following the standard Java reachability rules. These say that the return null; statement is reachable.
Does anyone have an ideas on how to make the warnings disappear in the method itself.
You could replace the last statement with this:
throw new AssertionError("unreachable statement executed");
Or better still, put it after the System.exit(...) call.
Any unchecked exception would do, but to my mind an AssertionError is the strongest indication that something totally wrong has happened. Note that you need to explicitly throw the exception. Using an assert is not sufficient to avoid the need for a return ... since assertion checking can be turned off.
The runtime cost of a throw statement that is never executed is minimal to zero.
Another idea is to return a dummy non-null value. In this example, an empty string would do.
The problem with annotating the method as #NotNull is that a static code analyzer (using the same incomplete logic as IntelliJ) may flag that the method does return null. If you can suppress that warning, there is also the possibility that some framework might insert a runtime check for null ... which 1) serves no purpose and 2) maybe can't be optimized away.
Unfortunately, there is no practical way to tag System.exit with an annotation that says "this statement never returns". Why? Because it is a standard method, and you can't change it ... without hacking the class libraries and (potentially) breaking portability.
The best you can do (I think) to fix the problem would be to develop and submit a patch to the Intellij maintainers. One that "understands" the special behavior of System.exit.
Finally, it is generally speaking a bad idea for a library method to call System.exit in a method like in your example. A better approach is to throw a custom exception which you catch at / near the base of the call stack ... on the application's controlling thread. That way, you can have all of the code and logic that governs (controlled) application exit in the same place.
A subsidiary method should not be deciding about the "fate" of the overall application.
Why don't you just add a terminating statement to the end of catch block? It will never be reached, so all it should do is help the compiler infer that the final return statement isn't necessary.
For example:
String loadSuperImportantFile() {
try {
// ... file loading code ...
assert fileContent != null;
return fileContent;
} catch(IOException ex) {
System.err.println("Failed to load super important file. Please check path.");
System.exit(Codes.FAIL); // terminate the program
throw new AssertionError("Unreachable");
}
}
Simply annotate the method as #NotNull. The #Contract annotation you've tried to apply says that the method will fail if you pass null to it as a parameter, which does not make sense because it does not have any parameters.
You could also return an Optional<String> instead of a String.
This way, you could avoid returning null and return an Optional.empty() instead to avoid null checking and possible NullPointerException.
See: https://docs.oracle.com/javase/8/docs/api/java/util/Optional.html
I need to define a set of functions as part of an interface for a public facing SDK and trying to figure out the best way of signalling to the caller code if something was successful or not, and if not, why not.
My first consideration was return value vs exception. I think I need to use a combination of both for this one. Exceptions for erroneous states/errors caused by bugs. Return value to be used for times when function ran as expected and just want to return the result of the call to the caller.
Now, for return type. I could use ints (flags) or booleans. It has been argued internally that booleans should be used as it is simpler, and no need for flags. Booleans limit you to specifying success or failure, with no explanation.
I would make two arguments for using flags instead. First, you can return more than two possible values. Allowing for { success, fail, fail_reason_1, fail_reason_2, fail_reason_3 }. Note as this is an interface for implementations on various hardware devices it is probably desirable to be able to notify why an operation failed. e.g. the connected device doesn't support beep, has no LCD, doesn't support embedded crypto etc.
Who knows what the requirements will be in the future. Returning a bool locks you in now, whereas using flags allows you greater flexibility in the future. So what if you never need more than two values in the future. At least you have the option.
Considering this will be a public facing SDK I want as much flexibility so as to prevent breaking changes in the future.
Thanks
In my opinion the difference between returning a value to indicate the result of a method call and to throw an exception is that a value is just a notification about what happened. The method call should be considered as being performed successfully regarding the contract it defines. For example have a look how boolean : Set.add() is defined.
If a method throws an exception, this should indicate either an incorrect use of the method or a call while the object / whole system was in an illegal state. For example, trying to buy something for an user while his account does not have enough credits.
An exception is perfectly suited for capturing the different failure types: either by an exception hierarchy or by adding properties to the exception like getFailureCode() or combining them.
I would not use flags to indicate a failure condition in case the failure must be handled. Because ignoring return values is much to easy and can be missed by programmers, while exception have to be ignored actively.
The short answer is that it varies a lot based on your domain, clients, and the specifics of what you're providing. If you haven't already, you need to sit down with your clients and figure out how they're going to be using the library. Just off the top of my head, here are some things I'd be thinking about:
First, all the failure types you identified are basically the same thing - UnsupportedOperationException, with a reason why. Are there other failure types? If not, and you use exceptions, you possibly want one exception class with a type/cause/whatever as an enum property.
Second, it looks like many/most/all of your methods are going to have one or more failure types like this. Is that true? If so, does your API provide any way of determining what features a device supports? Consider this:
// Best if I probably don't care about the result if it's not SUCCESS
final Result result = myApiObject.someMethod();
if (result == Result.BEEP_UNSUPPORTED) {
....
} else if (result == Result.NO_DISPLAY) {
....
} else if ...
and this:
// Best if I have to handle every possible failure condition, and I care what
// the failure type is
try {
myApiObject.someMethod();
} catch (final BeepUnsupportedException e) {
....
} catch (final NoDisplayException e) {
....
}
and this:
// Best if I have to consider every possible failure condition, but it probably
// doesn't matter what the failure type is
try {
myApiObject.someMethod();
} catch (final MyApiException e) {
if (Cause.BEEP_UNSUPPORTED == e.getCause()) {
....
} else ....
and this:
// Best if I know in advance what features I may need
// someMethod() may still return response codes or throw an exception. In this case
// it's possibly OK to make it a RuntimeException, since clients are expected to
// poll for features.
if (myApiObject.supports(Feature.BEEP)
&& myApiObject.supports(Feature.DISPLAY)) {
myApiObject.someMethod();
} else ...
Which of these is closest to the way your clients will want to use your system?
Third, how fatal are these errors? I totally agree with #Harmlezz that failures which must be handled need to be Exceptions. What neither of us can tell you without more information is whether your failures need to be handled or not. If clients will mostly be ignoring failures (they usually want the code to fail silently), or only mostly only care about SUCCESS or !SUCCESS, then return codes are probably okay. If failures need to be handled, then you should be throwing a checked exception. You probably want to stay away from unchecked exceptions, since these failures aren't programming errors and may be handled correctly by clients.
I have an object as the result of an API call and I want to assert the values of a member variable.
This value can be one of the two expected values depending on what the API call "sees" first and sets first. So if the assertion on one value fails, I want to assert for another value before declaring the test as a failure.
What is the best way of doing this? What I have right now is:
try {
assertEquals("message", someObject.getValue1(), expectedValue1);
} catch(AssertionError ae) {
assertEquals("message", someObject.getValue1(), expectedValue2);
}
I am just not sure if this is an acceptable practice. Please comment.
Using exceptions as a sort of glorified goto statement is generally not a good practice. Or at least you will run into people in your career who take a dim view of using exceptions for program flow control.
How about:
Assert.assertTrue((someObject.getValue1().equals(expectedValue1) || (someObject.getValue2().equals(expectedValue2));
Depends on the purpose, automated functional testing or unit testing. I sometimes do this for the former:
try {
assertTrue(boolean condition from earlier in test method to check);
}
catch(AssertionError uhOh) {
Logger.err("condition X failed: detailed info msg"); // broken item #1
}
try {
assertTrue(another condition in same method to check);
}
catch(AssertionError yuck) {
Logger.err("condition X failed: detailed info msg"); // another broken item
fail(); // now blow up as we've checked everything
}
Of course that's using logback Logger and JUnit's Assert.fail() which fails the test method. This way I know of all failures for this method rather than blowing up after the first. In my case, I'm testing a rich-content web app (dialogs and pages that take a lot of user input).
Downside of "fail-fast" (using no catches) is finding one problem, fixing it, running again and finding a new one ("rinse and repeat"), but if used for unit testing this is an asset due to the modularity of the tests (ideally you are only testing one aspect of an item per test).
I'll agree with Aquilon regarding this not being good practice.
However, can you use mocking or some other mechanism to "force" the API "see" one item before the other? That way your tests can reflect the conditions that lead to one assertion being true in one test, and the other assertion being true in another test.
In this specific scenarios, are asserts more appropriate then exceptions?
It is my understanding that assert should be used when program is FUBAR to a degree where it can not recover and will exit.
I also was told to always throw exceptions for clarity and error message handling.
Is there a fine line between when to use each? Is there an example where assert must be used in place of exception unconditionally?
public void subscribe(DataConsumer c) throws IllegalArgumentException {
if (c == null) {
// Almost certainly FUBAR
throw new IllegalArgumentException("Can't subscribe null as a DataConsumer. Object not initialized");
}
if (dataConsumerList == null) {
// Definetely FUBAR
throw new IllegalArgumentException("Nothing to subscribe to. DataConsumerList is null");
}
dataConsumerList.add(c);
}
Personally I'm not keen on using assertions for this sort of thing, simply because they can be turned off. Some places use assertions when running tests, but then disable them for production for the sake of speed. To me this is like taking your driving test as normal, but then removing your seatbelt when you get on the motorway.
An assertion is only going to throw an exception anyway, of course. If you absolutely want to take the JVM down right now, you'd need to use something like Runtime.halt.
So I'm a fan of exceptions here, although I'd typically use a NullPointerException when given a null argument, and if dataConsumerList is part of your state then I would personally use IllegalStateException to differentiate that situation. (It's a shame that Java doesn't have the same ArgmentNullException that .NET has, given what a common check it is.)
Guava has the useful Preconditions class which lets you write this more concisely:
public void subscribe(DataConsumer c) throws IllegalArgumentException {
Preconditions.checkNotNull(c,
"Can't subscribe null as a DataConsumer. Object not initialized");
Preconditions.checkState(dataConsumerList != null,
"Nothing to subscribe to. DataConsumerList is null");
dataConsumerList.add(c);
}
General rule (copied from here)
assertions should protect from (not always obvious) mistakes of the
developer, e.g. using a pointer despite its being NULL.
exceptions are a way to handle errors that may legitimately occur at
runtime, e.g. the failure of trying to connect to some server (which may
not respond for various reasons).
And there is a better way of writing the above code using google-guava Preconditions.checkNotNull() class.
public void subscribe(DataConsumer c) throws IllegalArgumentException
{
checkNotNull(c, "Can't subscribe null as a DataConsumer. Object not initialized");
checkNotNull(dataConsumerList , "Nothing to subscribe to. DataConsumerList is null");
dataConsumerList.add(c);
}
If you could put this in English terms, use assert for "gotta" (Got to, Must) and exceptions for "otta" (Ought to, should).
Use the assert for show-stopping, critical conditions that must be true for the execution to continue. Examples might be that a division happens correctly (think of the Intel chip floating point bug) or that your database connection is not null after you have correctly opened it. If these have occurred, then program execution should not continue.
Use the throw for foreseeable errors that your method may handle. The throw is a part of a contract that declares to you and other programmers that certain types of errors may be encountered (and that it's not your responsibility).
In your example, my guess is that a null consumer or an empty list should never happen under normal circumstances. If my guess is correct, then you would want to use an assert here, declaring that subscribe() will handle it.
If my guess is wrong and a null consumer happens, say 1 out of 50 times, then the throw would be better and you would be declaring that subscribe() forms a contract with a calling method, whereby the calling method handles the error.
The Java technote Programming With Assertions contain this explicit line in with regards to usage:
Do not use assertions for argument checking in public methods.
That should be a pretty definitive answer to your question.
I'd like to write a TestNG test to make sure an exception is thrown under a specific condition, and fail the test if the exception is not thrown. Is there an easy way to do this without having to create an extra boolean variable?
A related blog post on this subject: http://konigsberg.blogspot.com/2007/11/testng-and-expectedexceptions-ive.html
#Test(expectedExceptions) is useful for the most common cases:
You expect a specific exception to be thrown
You need the message of that exception to contain specific words
Per the documentation a test will fail if no expectedException is thrown:
The list of exceptions that a test method is expected to throw. If no exception or a different than one on this list is thrown, this test will be marked a failure.
Here are a few scenarios where #Test(expectedExceptions) is not sufficient:
Your test method has several statements and only one of them is expected to throw
You are throwing your own type of exception and you need to make sure it matches a certain criterion
In such cases, you should just revert to the traditional (pre-TestNG) pattern:
try {
// your statement expected to throw
fail();
}
catch(<the expected exception>) {
// pass
}
Use #Test annotation to check expected exceptions.
#Test(
expectedExceptions = AnyClassThatExtendsException.class,
expectedExceptionsMessageRegExp = "Exception message regexp"
)
Or if you don't want to check for exception message, only below is enough
#Test(expectedExceptions = AnyClassThatExtendsException.class)
In that way, you don't need to use ugly try catch block, just invoke you exception-thrower method inside the test.
If you are using Java 7 and TestNG, the below example can be used. For Java 8 you can also use Lambda Expressions.
class A implements ThrowingRunnable {
#Override
public void run() throws AuthenticationFailedException{
spy.processAuthenticationResponse(mockRequest, mockResponse, authenticationContext);
}
}
assertThrows(AuthenticationFailedException.class, new A());
I have to disagree with the the article on the nature of the testing techniques employed. The solution employs a gate, to verify if the test should succeed or fail in an intermediate stage.
In my opinion, it is better to employ Guard Assertions, especially for such tests (assuming that the test does not turn out to be long-winded and complex, which is an anti-pattern in itself). Using guard-assertions forces you to design the SUT in either of the following ways:
design the method itself to provide enough information in the result on whether the invocation passed or succeeded. Sometimes, this cannot be done as the intention of the designer is to not return a result, and instead throw an exception (this can be handled in the second case).
design the SUT so that it's state can be verified after each significant method invocation.
But before we consider the above possibilities, have a look at the following snippet again:
plane.bookAllSeats();
plane.bookPlane(createValidItinerary(), null);
If the intention is to test bookPlane() and verify for execution of that method, it is better to have bookAllSeats() in a fixture. In my understanding, invoking bookAllSeats() is equivalent to setting up the SUT to ensure that the invocation of bookPlane() fails, and hence having a fixture to do the same would make for a more readable test. If the intention are different, I would recommend testing the state after every transition (as I normally would do in functional tests), to help pinpoint the original cause of failure.
Why don't you use the try/fail/catch pattern mentioned in the blog post you linked to?
catch-exception provides probably everything you need to test for expected exceptions.
I created a custom Stack data structure which is backed by an array. The push() method throws a custom exception when the stack is full and you still try to push() data into the stack. You could handle it like this :
public class TestStackDataStructure {
//All test methods use this variable.
public Stack<String> stack;//This Stack class is NOT from Java.
#BeforeMethod
public void beforeMethod(){
//Don't want to repeat this code inside each test, especially if we have several lines for setup.
stack = new Stack<>(5);
}
#Test
public void pushItemIntoAFullStack(){
//I know this code won't throw exceptions, but what if we have some code that does ?
IntStream.rangeClosed(1,5).mapToObj(i -> i + "").forEach(stack::push);
try{
stack.push("6");
Assert.fail("Exception expected.");
}catch (StackIsFullException ex) {
// do nothing;
}
}
//Other tests here.
}
Alternately, you could change your api as suggested here :
#Test
public void pushItemIntoAFullStack(){
IntStream.rangeClosed(1,5).mapToObj(i -> i + "").forEach(stack::push);
Assert.assertFalse( stack.push("6"), "Expected push to fail." );
}
I updated the push method to return true or false if the operation passed or failed, instead of returning void. The Java Stack.push(item) returns the element you tried to insert instead of void. I don't know why. But, it also inherits a similar method addElement(item) from Vector which returns void.
One minor downside I see to making push(item) return a boolean or void is that you are stuck with those return types. If you return Stack instead then you can write code conveniently like this stack.push(1).push(2).push(3).pop(). But, I don't know how often one would have to write code like that.
Similarly, my pop() method used to return a generic type "T" and used to throw an exception if the stack was empty. I updated it to return Optional<T> instead.
#Test
public void popEmptyStack(){
Assert.assertTrue(stack.pop().isEmpty());
}
I guess I am now free of the clunky try-catch blocks and TestNg expectedExceptions. Hopefully, my design is good now.