I did a small debug session and stumbled over the following code snippet in Sun/Oracle's code:
try {
XmlSchema s = null;
s.location();
} catch (NullPointerException e) {
// as epxected
} catch (NoSuchMethodError e) {
// this is not a 2.1 API. Where is it being loaded from?
...
}
I think this has several flaws by first of all expecting a NullPointerException, using ((XmlSchema)null).location(); and expecting a NoSuchMethodError. If I would do the code review I would flag this as unappropriated code by triggering an exception as a regular event.
In my opinion using XmlSchema.getClass().getMethod("location", new Object[0]) would be a some what better idea.
Since this code is used in the official API implementation I wonder if there are edge cases, I am not aware of which prevent class inspection being a good solution and one has to use this code snippet as a fall back.
UPDATE:
This question is related to "Strange NullPointerException CATCH in OpenJDK JAXB but it is not answering my question about the why do they do it. It is just an explaination what it does not about why they do it.
UPDATE2:
Using the test code:
#Test
public void test() throws NoSuchMethodException, SecurityException {
Method method = ArrayList.class.getMethod("size");
if(method == null)
throw new UnsupportedOperationException();
}
and
#Test
#SuppressWarnings("null")
public void test2() {
ArrayList<?> list = null;
try {
list.size();
}
catch(NullPointerException e) {
}
catch(NoSuchMethodError e) {
throw new UnsupportedOperationException();
}
}
I found that TestNG reports 0s for both tests (after you warm up). Since the code is executed within a static method of a class this code is executed at most once.
So the timing is not the reason. Another problem is that a modern compiler detects the attempt to execute the location method on a null reference and therefore one need to suppress the execution.
So the timing is not the reason (anymore?) for preferring this hack instead of a proper class inspection.
Does anyone know more answers to this?
If you use a catch for handling the NoSuchMethodError, the slow code is used exceptionally. If instead you use reflection for checking the existence of the method, then the slow code is always used.
It appears that the code fragment is now referred to as old legacy code that was never maintained in the meantime. Especially if one notice that any modern IDE flags this code snippet as a accessing a NULL reference.
The reason for using this might have been the problems with reflections (slow) 15 years ago but at least today do not make sense anymore.
Also generating a NullPointerException everytime within the normal application flow and therefore generate such an important exception and unnecessarily interfere with a debug session this can be considered a defect.
So final conlusion: Do not use this code snippet to verify a compatible class version, use reflection instead.
Related
I know there are a number of questions on this topic, but all of them seem to assume one of two things:
You just want to test if an exception was thrown and not caught,
You should test the function that is inside of the try block
directly
I'm not sure how I can apply those options to this case. I have a small try/catch block, like so:
try {
o.getDataContainer().build(...);
o2.setDataContainer(o.getDataContainer());
} catch (final Exception e) {
LOGGER.error("Data set failed", e);
}
As you can see, if o.getDataContainer() returns null, an exception would be triggered. However, that exception is then caught, and the test tool considers it a successful test. Is it possible to test that the exception occurred without changing the code?
I ask because our logging system triggers a trouble ticket if it picks up an exception that is caught and logged. Since it is common human error to forget to guard something like this, I would like to write UTs that can test if an exception was triggered and caught. I can't remove the whole-program protection provided by the catch block, but the error would also cause a degradation of the user experience, since the data isn't being passed along. (I work in a place where minutes of site downtime equal millions of dollars lost.)
In other words: The exception is an error, and we want to log it and investigate it, but in case this manages to trigger on prod, we don't want to risk the whole site going down.
Note: This try/catch sits inside a much larger function with many such try/catch blocks. One could easily argue bad overall design, but fixing it is not an option (without a huge amount of free dev time, at least).
Update: As the task at hand does not allow me to spend a great deal of time on this, I went with a very simple generic test that would fail if the guard and catch were both removed so that I could move on. But, I'm leaving the question unanswered for now in hopes of continuing conversation. I would love to be able to write a simple UT for each new feature that fails if any exceptions are triggered and caught.
Ignoring the issues with this code (Sometimes you've gotta put lipstick on a pig, I guess), this is how I might handle the situation.
I'd use Mockito and mock o2, then use an Answer to ensure the method is invoked.
A test might look like this:
#RunWith(MockitoJUnitRunner.class)
public class TestClass{
#Mock
O2 o2;
#Mock
O1 o1;
boolean exceptionThrown = false;
#Test
public void test(){
Mockito.doAnswer(new Answer<Void>(){
public Void answer(InvocationOnMock invocation) throws Throwable {
exceptionThrown = true;
throw new RuntimeException("some message");
}
}).when(o2).setDataContainer(DataContainer.class);
}
}
Essentially, you can Mock out o2 in your example, and force the exception.
If this doesn't quite do what you want, you may need to mock LOGGER and verify that it's invoked with LOGGER.error("some message");. Unfortunately, mocking statics is not at all elegant, but it can be done with PowerMock.
You could add a custom handler to LOGGER that just throws when an error is logged. For java.util.logging you could do something like:
LOGGER.addHandler(new Handler() {
public void publish(LogRecord record) {
if ("Data set failed".equals(record.getMessage())) {
throw new RuntimeException(record.getThrown());
}
}
public void flush() {}
public void close() throws SecurityException {}
});
I think log4j calls it "Appender," but the same principle should work. See How to create a own Appender in log4j? or How to Create a Custom Appender in log4j2?
So I am working with AspectJ to refactor my program to remove all logging calls from the main classes. Instead, all logging calls will occur via aspects. So far, here's what I've done successfully:
Catching all exceptions. Since I already have my exception handling pretty unified to begin with (Exceptions propagate up call stack until they reach my controller, then get caught), this wasn't too difficult. Using handle(Exception) I've got every catch block tied to an aspect that will log the exception before the catch block executes.
Debug information logging. Basic things like entering a particular method, etc. Again, relatively simple using before() and, sometimes, args() or thisJoinPoint.getArgs() to capture parameters.
The final thing I need to do, though, is properly log successful completion of key methods. Here is a minor example, of a block of code as it currently exists in my program, so I can better explain my meaning here:
private static void loadDefaultProperties(){
defaultProperties = new Properties();
try {
defaultProperties.load(
Main.class.getClassLoader().getResourceAsStream(
"default.properties"));
if(defaultProperties.size() == 0){
throw new IOException();
}
//THE CURRENT LOGGING LINE THAT NEEDS TO BE MOVED TO AN ASPECT
LOGGER.logp(Level.INFO, Main.class.getName(),
"loadDefaultProperties()",
"Default Properties file loaded successfully");
}
catch (IOException ex){
//THIS EXCEPTION IS LOGGED USING HANDLE(EXCEPTION) IN AN ASPECT
displayExceptionDialog(ex);
}
}
So, this method loads the default properties from the classpath. If it fails, an IOException is thrown. However, if an exception is not thrown, and it performs its task successfully, I want to log that as well.
This particular example is a relatively minor item that wouldn't be a huge deal to miss, but this is the style currently used everywhere in the application.
How would I configure a pointcut/advice in AspectJ to only run after this method completes, and recognize that it does so successfully?
PS. I've considered propagating the exception further up the call stack. That would allow me to use a simple after() returning(Object o) advice to accomplish this. If an exception is thrown, this advice will not run because the method won't return properly.
The problem with that is there aren't many levels higher than this in the call stack. This particular method in my example is called directly by my main method immediately following the program's initialization. In my controller class, there's only one level higher in the call stack for all of my methods there as well.
I guess I could do this, but I want to see if there's another option first, rather than pool all of those exceptions together in one place.
You can certainly use a pointcut to wrap anything in the call stack, including main():
public aspect WrapMain
{
pointcut mainMethod() : execution(public static void main(String[]));
before() : mainMethod() {
System.out.println("Starting application...");
}
after() : mainMethod() {
System.out.println("Terminating application...");
}
}
If you are generating aspects using Java, you can use the #AfterReturning and #AfterThrowing annotations to manage successful or exceptional cases.
#Pointcut("execution(your-method-signature-here)
public void yourPointcut(){}
#AfterReturning(pointcut = "yourPointcut()", returning ="returnValue")
public void afterSuccessAdvice(JoinPoint jp, Object returnValue) {
// log here
}
#AfterThrowing(pointcut = "yourPointcut()", returning ="returnValue")
public void afterFailureAdvice(JoinPoint jp, Object returnValue) {
// log here
}
Here's a Javaworld article with examples that explains logging method entry and exit.
I have implemented a custom TestRunner based on BlockJUnit4ClassRunner.
My assumption was that any failed assertions (indicating product/requirement issues) would be reported to the notifier via addFailedAssumption() while other exceptions would be reported via addFailure() indicating bugs in the unit test itself.
Looking at the results, addFailedAssumption() was never called. In the source code of ParentRunner.runLeaf(), I see
try {
statement.evaluate();
} catch (AssumptionViolatedException e) {
eachNotifier.addFailedAssumption(e);
} catch (Throwable e) {
eachNotifier.addFailure(e);
} finally {
eachNotifier.fireTestFinished();
}
The exceptions I get are all of type java.lang.AssertionError.
Should ParentRunner catch AssertionError or is there a misunderstanding on my side?
Reading more about this topic, this seems to be a language / translation issue on my side because I'm not a native speaker.
Finding the class Assume helped me getting it right (I hope) and I'll explain it with an example:
Usage of the AssumtionViolatedException
Tests can e.g. run in different environments, let's say different operating systems. Maybe the product behaves or needs to behave slightly different on different operating systems, e.g. because it can use an API call on a newer OS which does not exist in an older version of the OS. This may result in code like
if(isApiPresent())
SimpleAPICall();
else
// Do some crazy stuff here, potentially slower than the API call
The isApiPresent() call will return different results depending on the OS, so you write 2 unit tests and add an assumption about the environment:
#Test
public void isApiPresent_returns_true_on_Win8()
{
assumeTrue(System.getProperty("os.version").equals("6.2"));
assertTrue(isApiPresent());
}
#Test
public void isApiPresent_returns_false_on_Win7()
{
assumeTrue(System.getProperty("os.version").equals("6.1"));
assertFalse(isApiPresent());
}
If the assumption about the operating system is not given, the test still gets executed due to the #Test annotation, but it should actually be ignored. The assume...() statements take care of that: they throw an AssumptionViolatedException which can be used to ignore the test.
Eclipse marks a test with a violated assumption (try assumeFalse(true);) with an ignore icon:
Usage of AssertionError
What I wanted to achieve with my custom implementation of a TestRunner is a bit different. I wanted to find out which unit tests fail due to a requirement issue and which tests fail due to other exceptions which could indicate a bug in the unit test itself, reconstructing the icons in Eclipse. Eclipse already distinguishes these two kinds of issues: AssertionErrors are marked with blue icons while Exceptions are marked with red icons.
For me, this means that I have to implement the decision in fireTestFailure():
public void fireTestFailure(Failure failure) {
originalNotifier.fireTestFailure(failure);
if (failure.getException().getClass() == AssertionError.class) {
// Requirement issue
} else {
// Unit test issue
}
}
As you know, exception is thrown at the condition of abnormal scenarios. So how to analog these exceptions? I feel it is challenge. For such code snippets:
public String getServerName() {
try {
InetAddress addr = InetAddress.getLocalHost();
String hostname = addr.getHostName();
return hostname;
}
catch (Exception e) {
e.printStackTrace();
return "";
}
}
Does anybody have good ideas?
You can tell junit that the correct behavior is to get an exception.
In JUnit 4, it goes something like:
#Test(expected = MyExceptionClass.class)
public void functionUnderTest() {
…
}
Other answers have addressed the general problem of how to write a unit test that checks that an exception is thrown. But I think your question is really asking about how to get the code to throw the exception in the first place.
Take your code as an example. It would be very hard to cause your getServerName() to internally throw an exception in the context of a simple unit test. The problem is that in order for the exception to happen, the code (typically) needs to be run on a machine whose networking is broken. Arranging for that to happen in a unit test is probably impossible ... you'd need to deliberately misconfigure the machine before running the test.
So what is the answer?
In some cases, the simple answer is just to take the pragmatic decision and not go for total test coverage. Your method is a good example. It should be clear from code inspection what the method actually does. Testing it is not going to prove anything (except see below **). All you are doing is improve your test counts and test coverage numbers, neither of which should be project goals.
In other cases, it may be sensible to separate out the low-level code where the exception is being generated and make it a separate class. Then, to test the higher level code's handling of the exception, you can replace the class with a mock class that will throw the desired exceptions.
Here is your example given this "treatment". (This is a bit contrived ... )
public interface ILocalDetails {
InetAddress getLocalHost() throws UnknownHostException;
...
}
public class LocalDetails implements ILocalDetails {
public InetAddress getLocalHost() throws UnknownHostException {
return InetAddress.getLocalHost();
}
}
public class SomeClass {
private ILocalDetails local = new LocalDetails(); // or something ...
...
public String getServerName() {
try {
InetAddress addr = local.getLocalHost();
return addr.getHostName();
}
catch (Exception e) {
e.printStackTrace();
return "";
}
}
}
Now to unit test this, you create a "mock" implementation of the ILocalDetails interface whose getLocalHost() method throws the exception you want under the appropriate conditions. Then you create a unit text for SomeClass.getServerName(), arranging that the instance of SomeClass uses an instance of your "mock" class instead of the normal one. (The last bit could be done using a mocking framework, by exposing a setter for the local attribute or by using the reflection APIs.)
Obviously, you would need to modify your code to make it testable like this. And there are limits to what you can do ... for example, you now cannot create a unit test to make the real LocalDetails.getLocalHost() method to throw an exception. You need to make a case-by-case judgement as to whether it is worth the effort of doing this; i.e. does the benefit of the unit test outweigh the work (and extra code complexity) of making the class testable in this way. (The fact that there is a static method at the bottom of this is a large part of the problem.)
** There is a hypothetical point to this kind of testing. In your example, the fact that the original code catches an exception and returns an empty string could be a bug ... depending on how the method's API is specified ... and a hypothetical unit test would pick it up. However, in this case, the bug is so blatant that you would spot it while writing the unit test! And assuming that you fix bugs as you find them, the unit test becomes somewhat redundant. (You wouldn't expect someone to re-instate this particular bug ...)
Okay there are a few possible answers here.
Testing for an exception itself is easy
import static org.hamcrest.core.Is.is;
import static org.junit.Assert.assertThat;
#Test
public void TestForException() {
try {
doSomething();
fail();
} catch (Exception e) {
assertThat(e.getMessage(), is("Something bad happened"));
}
}
Alternately, you can use the Exception Annotation to note that you expect an exception to come out.
Now, as to you specific example, Testing that something you are creating inside your method, either via new or statically as you did, when you have no way to interact with the object is tricky. You normally need to encapsulate that particular generator and then use some mocking to be able to override the behavior to generate the exception you expect.
Since this question is in community wiki I'll add a new one for completeness:
You can use ExpectedException in JUnit 4
#Rule
public ExpectedException thrown= ExpectedException.none();
#Test
public void TestForException(){
thrown.expect(SomeException.class);
DoSomething();
}
The ExpectedException makes the thrown exception available to all test methods.
Is is also possible to test for a specific error message:
thrown.expectMessage("Error string");
or use matchers
thrown.expectMessage(startsWith("Specific start"));
This is shorter and more convenient than
public void TestForException(){
try{
DoSomething();
Fail();
}catch(Exception e) {
Assert.That(e.msg, Is("Bad thing happened"))
}
}
because if you forget the fail, the test can result in a false negative.
Many unit testing frameworks allow your tests to expect exceptions as part of the test. JUnit, for example, allows for this.
#Test (expected=IndexOutOfBoundsException.class) public void elementAt() {
int[] intArray = new int[10];
int i = intArray[20]; // Should throw IndexOutOfBoundsException
}
Today I spent my afternoon with analysing a NoClassDefFoundError. After verifying the classpath again and again, it turned out that there was a static member of a class that threw an Exception that was ignored the first time. After that every use of the class throw a NoClassDefFoundError without a meaningful stacktrace:
Exception in thread "main" java.lang.NoClassDefFoundError:
Could not initialize class InitializationProblem$A
at InitializationProblem.main(InitializationProblem.java:19)
That's all. No more lines.
Reduced to the point, this was the problem:
public class InitializationProblem {
public static class A {
static int foo = 1 / 0;
static String getId() {
return "42";
}
}
public static void main( String[] args ) {
try {
new A();
}
catch( Error e ) {
// ignore the initialization error
}
// here an Error is being thrown again,
// without any hint what is going wrong.
A.getId();
}
}
To make it not so easy, all but the last call of A.getId() was hidden somewhere in the initialization code of a very big project.
Question:
Now that I've found this error after hours of trial and error, I'm wondering if there is a straight forward way to find this bug starting from the thrown exception. Any ideas on how to do this?
I hope this question will be a hint for anyone else analysing an inexplicable NoClassDefFoundError.
Really, you shouldn't ever ever catch Error, but here's how you can find initializer problems wherever they might occur.
Here's an agent that will make all ExceptionInInitializerErrors print the stack trace when they are created:
import java.lang.instrument.*;
import javassist.*;
import java.io.*;
import java.security.*;
public class InitializerLoggingAgent implements ClassFileTransformer {
public static void premain(String agentArgs, Instrumentation inst) {
inst.addTransformer(new InitializerLoggingAgent(), true);
}
private final ClassPool pool = new ClassPool(true);
public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) {
try {
if (className.equals("java/lang/ExceptionInInitializerError")) {
CtClass klass = pool.makeClass(new ByteArrayInputStream(classfileBuffer));
CtConstructor[] ctors = klass.getConstructors();
for (int i = 0; i < ctors.length; i++) {
ctors[i].insertAfter("this.printStackTrace();");
}
return klass.toBytecode();
} else {
return null;
}
} catch (Throwable t) {
return null;
}
}
}
It uses javassist to modify the classes. Compile and put it in a jar file with javassist classes and the following MANIFEST.MF
Manifest-Version: 1.0
Premain-Class: InitializerLoggingAgent
Run your app with java -javaagent:agentjar.jar MainClass and every ExceptionInInitializerError will be printed even if it is caught.
My advice would be to avoid this problem by avoiding static initializers as much as you can. Because these initializers get executed during the classloading process, many frameworks don't handle them very well, and in fact older VMs don't handle them very well either.
Most (if not all) static initializers can be refactored into other forms, and in general it makes the problems easier to handle and diagnose. As you've discovered, static initializers are forbidden from throwing checked exceptions, so you've got to either log-and-ignore, or log-and-rethrow-as-unchecked, none of which make the job of diagnosis any easier.
Also, most classloaders make one-and-only-one attempt to load a given class, and if it fails the first time, and isn't handled properly, the problem gets effectively squashed, and you end up with generic Errors being thrown, with little or no context.
If you ever see code with this pattern:
} catch(...) {
// no code
}
Find out who wrote it and BEAT THE CRAP OUT OF THEM. I'm serious. Try to get them fired--they do not understand the debugging portion of programming in any way, shape or form.
I guess if they are an apprentice programmer you might just beat the crap out of them and then let them have ONE second chance.
Even for temporary code--it's never worth the possibility that it will somehow be brought forward into production code.
This kind of code is caused by checked exceptions, an otherwise reasonable idea made into a huge language pitfall by the fact that at some point we'll all see code like that above.
It can take DAYS if not WEEKS to solve this problem. So you've got to understand that by coding that, you are potentially costing the company tens of thousands of dollars. (There's another good solution, fine them for all the salary spent because of that stupidity--I bet they never do THAT again).
If you do expect (catch) a given error and handle it, make sure that:
You know that the error you handled is the ONLY POSSIBLE source of that exception.
Any other exceptions/causes incidentally caught are either rethrown or logged.
You aren't catching to broad an exception (Exception or Throwable)
If I sound aggressive and angry, it's because I've gotten screwed spending weeks finding hidden bugs like this and, as a consultant, haven't found anyone to take it out on. Sorry.
The only hints the error gives are the name of the class and that something went terribly wrong during initialization of that class. So either in one of those static initializers, field initialization or maybe in a called constructor.
The second error has been thrown because the class has not been initialized at the time A.getId() was called. The first initialization was aborted. Catching that error was a nice test for the engineering team ;-)
A promising approach to locate such an error is to initialize the class in a test environment and debug the initialization (single steps) code. Then one should be able to find the cause of the problem.
Today I spent my afternoon with analysing a NoClassDefFoundError. After verifying the classpath again and again, it turned out that there was a static member of a class that threw an Exception that was ignored the first time.
There is your problem! Don't ever catch and ignore Error (or Throwable). NOT EVER.
And if you've inherited some dodgy code that might do this, use your favourite code search tool / IDE to seek and destroy the offending catch clauses.
Now that I've found this error after hours of trial and error, I'm wondering if there is a straight forward way to find this bug starting from the thrown exception.
No there isn't. There are complicated/heroic ways ... like using doing clever things with a java agent to hack the runtime system on the fly ... but not the sort of thing that a typical Java developer is likely to have in their "toolbox".
Which is why the advice above is so important.
I really don't understand your reasoning. You ask about "find this bug starting from the thrown exception" and yet you catch that error and ignore it ...
If you can reproduce the problem (even occasionally), and it's possible to run the app under debug, then you may be able to set a break point in your debugger for (all 3 constructors of) ExceptionInInitializerError, and see when they git hit.