As the title suggested, I have some code wrapped in a while(true) infinite loop, and all of them are fully caught by try and catch block. This thread is started in the main method, however, after long run, this worker thread is vanished mysteriously when I check using the jstack and causing work accumulated.
Below is my code:
public void run() {
while (true) {
try {
// Consumer consumes from Kafka server
Global.KAFKA_METRIC_DATA_CONSUMER.consume(topic, handler);
} catch (Exception e) {
logger.error("Kafka consumer process was interrupted by exception!");
} finally {
try {
// Prevent restart too often
Thread.sleep(30 * BaseConst.SECOND);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
For my understanding, this structure will keep the thread running so is the consumer. Even if the consume() methods failed, it will restart infinitely. However, as I mentioned above, the whole thread disappear silently without any error log. Could anyone provide some clue please?
Some more information that might be helpful:
I have checked the consume method will never shutdown the consumer
nor close the socket to the server. It will continuously try to
connect server after fail.
I analysed the java heap dump, and I found there is a memory leak
somewhere else in the project, causing memory occupation extremely
high and the gc very often. However, the main method is still
running.
OutOfMemoryError is not an Exception. It's an Error derived from Throwable.
If that was thrown somewhere in your consume(topic, handler), finally would still be called, delaying the inevitable some 30s... but after that the error would be passed upward and your loop would be terminated.
You are catching Exception so there's a chance that a java.lang.Error or a java.lang.Throwable is being thrown (eg OutOfMemoryError)
If you really want to catch everything, you'll need to catch Throwable and not just Exception subclasses.
Your thread is probably killed by an error.
An error is not an exception! But they both extend Throwable
Add another catch block that catches errors.
Throwable should never be caught, because errors require a different handling than exceptions
Related
I have a service, and I would like it to have the following behavior:
If service receives InterruptedException, or JVM shuts down, it should try to stop gracefully
If there's some "catastrophic" event, the service should just quit
Any other exception should be logged, state should reset, and loop should keep running
Loop should not quit under any other circumstance.
So here's a overly simplified version of what I came up with.
On class level:
private static volatile boolean keepRunning = true;
static {
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
keepRunning = false;
}
});
}
And then the loop:
while(keepRunning) {
try {
// Do something useful
Thread.sleep(10000); // just an example of something that can cause InterruptedException
}
catch(InterruptedException e) {
keepRunning = false;
}
catch(Exception e) { // catches all exceptions, but not any errors
// log it
// reset state
}
}
if(!keepRunning) {
// shut down gracefully
}
It seems satisfies all 4 conditions, but there are some problems and unclear parts:
(problem) Catching Exception and not doing anything is against all good practices. But is it acceptable in this case, or there's a better solution?
(question) Is not catching Error all I need to do to satisfy condition #2? Or are there other situations I should be aware of?
(unclear) Usually it's recommended that the "shut down gracefully" code goes into finally section after exception, and I don't really like that I have to check it after loop (if(!keepRunning)). But it seems doesn't fit in finally in this case. Is there a better way to do that?
(question) This code is written for Java 7. Would anything be changed/improved with Java 8?
I will appreciate either direct answers to my questions, or pointers to different patterns/solutions. Thanks in advance.
It is ok to catch the Exception in your case.
Not catching Error is a good practice if you run tests.
The finally block is what you should use to shut down gracefully and yes - the if statement in the finally block is needed and generally ok.
If an error occurs, your finally block will still execute so it is all good.
This code is ok for both Java 7 and Java 8
I have an application that runs headless and needs to be able to notify administrators of problems. I built an email notification framework for it to use, and basically if an exception is thrown and caught, depending on the area in code and the criticality of the exception, it may be passed into the alert notification framework which triggers an email to go out to the listed admins with the stack trace and other debug info.
This works, pretty well.
Before deploying, I was putting it through it's paces. One of my tests is to throw an unhandled exception from a random spot in code - simulating a potentially serious runtime problem where exceptions are being thrown that we did not expect.
For example, here is a method I inserted the test unhandled exception:
/**
* Closes connection.
*/
public void closeConnection() {
if (true)
throw new NullPointerException("Test unexpected exception NPE");
LOG.info("Closing SFTP connection");
getSftpChannel().exit();
getSession().disconnect();
LOG.debug("SFTP Connection closed");
}
The code runs, and when it gets to this unhandled exception, the program hardlocks (because exception is thrown and the sftp connection thread is never closed, so it holds the jvm open until the connection timeout).
I had thought it would crash the JVM, or pass it up to it's caller which eventually would bubble up to the alert system.
In this scenario, I had assumed this NPE would throw out of this exception, isn't handled by it's caller or the caller's caller, etc, so it should bubble to the main() and then crash the JVM since even main does not catch Exception or NPE's.
Question: What is going on here and how can I ensure a scenario like this won't hang in production? Do I just had a huge catch-all catch clause to my main() and have it catch all Exception so that every exception gets handled?
EDIT FOR CLARITY: The question is more-or-less -- Why does an unhandled exception that is not explicitly thrown in a method's signature, nor handled by a caller, not crash the JVM?
To answer your question: Why does an unhandled exception that is not explicitly thrown in a method's signature, nor handled by a caller, not crash the JVM?
Assuming this code is running in a Thread, the reason is, that unless you've set an UnhandledExceptionHandler on either the Thread class or the Thread instance, then the 'main' thread group is the default UnhandledExceptionHandler. By default, the thread group handles unhandled exceptions by logging the stack trace to system.out, the thread 'dies' and the JVM does not crash.
You may want to consider implementing an UnhandledExceptionHandler that uses your email framework to notify you of these failures.
As other posters have suggested, the code should clean up resources like Channels and Session in a finally block.
use a finally block to ensure the connection is closed.
public void closeConnection() {
try {
if (true)
throw new NullPointerException("Test unexpected exception NPE");
} finally {
LOG.info("Closing SFTP connection");
getSftpChannel().exit();
getSession().disconnect();
LOG.debug("SFTP Connection closed");
}
}
We're calling "lock()" on a ReentrantLock and threads are getting stuck there when they apparently shouldn't.
When debugging with a breakpoint just before the call to "lock()", the first thread would stop there with the program pointer going to "Thread.exit()".
The lock object's toString() says "unlocked" and it's "state" attribute is "0".
The behavior is not always the same. Sometimes the first thread goes past the lock as expected.
userLock.lock(); //first thread sometimes gets stuck here (and the following ones as well)
//"userLock" has "state=0" and toString() says "UNLOCKED"
try {
Transaction tr = HibernateConfig.getSessionFactory().getCurrentSession().beginTransaction();
try {
execute();
tr.commit();
} catch (ConstraintViolationException e) {
//probably traces with repeated time
System.err.println(e.getMessage());
if (tr.isActive()) {
tr.rollback();
}
} catch (RuntimeException e) {
e.printStackTrace();
if (tr.isActive()) {
tr.rollback();
}
}
} catch (Throwable e) {
e.printStackTrace();
} finally {
userLock.unlock();
}
try to put a breakpoint after userLock.lock(); then you should get the thread, that gets the lock.
alternatively you could use userLock.getOwner(); right behind .lock() to see wich thread got the lock.
The problem was my breakpoint was not before "lock()" like I said, but on it.
What happened is a bunch of threads would be blocked in that line by the breakpoint, one of them would still acquire the lock, and then the debugger would give me control over a random one of them which hadn't acquired the lock. And I was failing to check every thread blocked by the breakpoint to find the free one.
In the end I put the breakpoint actually before the lock and it behaved as expected.
This was confusing and I hope the question will still help someone.
Note: I'm still confused by the fact the lock's state said "unlocked" in the debugger while I was controlling a locked thread.
Sometimes, you just have to catch Throwable, e.g. when writing a dispatcher queue that dispatches generic items and needs to recover from any errors (said dispatcher logs all caught exceptions, but silently, and then execution is continued on other items).
One best practice I can think of is to always rethrow the exception if it's InterruptedException, because this means someone interrupted my thread and wants to kill it.
Another suggestion (that came from a comment, not an answer) is to always rethrow ThreadDeath
Any other best practices?
Probably the most important one is, never swallow a checked exception. By this I mean don't do this:
try {
...
} catch (IOException e) {
}
unless that's what you intend. Sometimes people swallow checked exceptions because they don't know what to do with them or don't want to (or can't) pollute their interface with "throws Exception" clauses.
If you don't know what to do with it, do this:
try {
...
} catch (IOException e) {
throw new RuntimeException(e);
}
The other one that springs to mind is to make sure you deal with exceptions. Reading a file should look something like this:
FileInputStream in = null;
try {
in = new FileInputStream(new File("..."));;
// do stuff
} catch (IOException e) {
// deal with it appropriately
} finally {
if (in != null) try { in.close(); } catch (IOException e) { /* swallow this one */ }
}
Depends on what you are working on.
if you are developing an API to be used by some one else, its better to re-throw the Exception or wrap it into a custom exception of yours and throw.
Whereas if you are developing an enduser application you need to handle this exception and do the needful.
What about OutOfMemoryError (or perhaps its super class VirtualMachineError)? I can't imagine there is much you can do after something that serious.
If you're writing a dispatcher queue, then by the time the exception comes back to you there's no point in doing anything with it other than logging it. The Swing event queue has basically that type of behavior.
Alternatively, you could provide a hook for an "uncaught exception handler," similar to ThreadGroup. Be aware that the handler could take a long time, and end up delaying your dispatcher.
As far as InterruptedException goes: the only thing that cares about that is your dispatch loop, which should be checking some external state to see if it should stop processing.
I have the following code in a Runnable that gets passed to a thread t:
public void run() {
logger.debug("Starting thread " + Thread.currentThread());
try {
doStuff();
} catch (Exception e) {
logger.debug("Exception in Thread " + Thread.currentThread());
}
logger.debug("End of thread " + Thread.currentThread());
}
I've hit a bug where I see deadlock with the following conditions
only the start of thread message has been printed by my logs
A thread dump shows that the thread t (supposed to be executing this) is no longer running
Is there some magical way this thread could have terminated early without either logging an end of thread message OR throwing an exception?
Are you sure that doStuff() did not throw an Error? Change catch (Exception e) to catch (Throwable t). It's possible to kill threads in Java with Thread.stop(), but that is highly unlikely.
Are you sure that where you start() the Thread, you also join() it afterwards?
Runnable myRunnable=new Runnable(){
#Override
public void run(){
// your original code goes here
}
};
Thread myThread=new Thread(myRunnable);
myThread.start();
myThread.join(); // magic happens here! it waits for the thread to finish ;)
By the way, join() may throw an InterruptedException,
so if something interrupts your thread while it's running,
join will inform you about this by throwing this exception.
Hope this helps.
When you catch Exception, you will catch any RunnableException and any declared thrown Exception, but you will not catch anything that extends Error. If you truly want to catch anything, then you need to catch Throwable.
If you want to do this for logging purposes only and you don't care why the Thread exits, you can do this:
public void run() {
logger.debug("Starting thread " + Thread.currentThread());
try {
// The work of your Thread
} finally {
logger.debug("End of thread " + Thread.currentThread());
}
}
and the finally statement is guaranteed to execute unless the Thread is stopped or deadlocks or in some other way stops executing without an Exception.
In most of my programs, I install an UncaughtExceptionHandler so that I'll know about every Thread that dies unexpectedly. It's been a tremendous help in tracking failures. This was added to the language as of Java 5.
Sure, the thread could get interrupted after the end of the try/catch block but before the last logger.debug statement. In that case it would throw an InterruptedException that, in principle, might not get recorded anywhere (if the default exception handler is set to ignore such things).
That seems like a rather unlikely scenario, though... it's kind of hard to tell what's going on without knowing more about the rest of your program, though.
When ever you trap a general exception I suggest you log it so you know what the exception was and it cause. Failing to do so will not help you at all.