HealthChecker for Java Process - java

I want to create a health checker, which will check the health of a java process. My process does a lot of things and is multi threaded. Various exceptions could be thrown, like Service / SQL / IO, etc. My plan is to call the HealthChecker to check for the process, from the catch block, in the individual threads. This will check for all the different healths, and in the case where there is any issue it will pause the threads, and log appropriately. There will be other processes which will read the logs by the process, and alert support to take appropriate actions.
Below is the general structure of the java process.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class Schedular {
private static int numOfTasks = 10 ;
public static void main(String[] args) {
ExecutorService service = Executors.newFixedThreadPool(5);
while(true){
for(int i=0;i<numOfTasks;i++){
service.execute(new Workers());
}
}
}
}
class Workers implements Runnable{
#Override
public void run() {
/*
* This can throw different exceptions , eg:
*/
try{
}catch(Exception e){
e.printStackTrace();
HealthChecker.checkHealth();
}
}
}
class HealthChecker{
public static void checkHealth() {
//Check health and then , log and pause all the threads
}
}
I am not able to figure out a way to pause all the threads. If there is a db exception I want all the threads to pause. I am requesting some suggestions.

You need a way to block the threads until some event occurs that allows the threads to continue. I see some major issues with the code:
1) The while(true) in your main thread might lead to a StackOverflowError. With each iteration of the while loop, you will add 10 more threads to the executor, and this will just continue unbounded.
2) There is no loop in your run() so that even if an exception is caught and we wait for the HealthCheck, the run() method would still exit. While a loop is not needed in your run() if you can constantly execute new Threads from your main thread to take the place of the terminated one, but that logic is not presently there in the main loop.
But setting those concerns aside here is one way to block worker threads until some event (presumably a HealthCheck all clear) occurs.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class Schedular {
private static int numOfTasks = 10 ;
public static void main(String[] args) {
ExecutorService service = Executors.newFixedThreadPool(5);
HealtchChecker hChecker = new HealthChecker();
for(int i=0;i<numOfTasks;i++){
service.execute(new Workers(hChecker));
}
}
}
class Workers implements Runnable{
private HealtchChecker hChecker;
public Workers(HealtchChecker hChecker){
this.hChecker = hChecker;
}
#Override
public void run() {
/*
* This can throw different exceptions , eg:
*/
while(true) {
try{
}catch (InterruptedException ie) {
throw ie;
}catch(Exception e){
e.printStackTrace();
HealthChecker.checkHealth();
}
}
}
}
class HealthChecker implements Runnable {
private final Semaphore semaphore = new Semaphore(1, true);
public void checkHealth() {
try {
semaphore.acquire();
} finally {
semaphore.release();
}
}
#Override
public void run(){
//code to check for errors that cause threads to pause.
if (inErrorState) {
semaphore.acquire();
} else {
semaphore.release();
}
}
}
A few things worth mentioning.
1) The main thread only creates 10 threads, versus an unbounded amount. You can adjust this as needed.
2) The Worker thread is long lived, meaning it will continue running even if it encounters Exceptions, except for an InterruptException.
3) HealthCheck is no longer a static object. it is instead a shared object.
4) HealthCheck is a runnable that can be executed in its own thread for monitoring for errors. I did not add the code to execute this thread.
5) HealCheck uses a Semaphore to cause the threads to block until the error state is cleared. I looked for other objects that can do this, like CountDownLatch or CyclicBarrier or Phaser, but this one came closest to giving us what we need to block all the threads from one point (the run() method).
Its not perfect but I think it gets you a little bit closer to what you want.

You're venturing pretty far afield from best practices, but you didn't ask about best practices for monitoring the health of threads - so I won't answer that question. Instead, I'll just answer the question you asked: how can I pause a set of threads managed by an ExecutorService?
Assuming that your Workers.run() will eventually end without intervention (in other words, it's not in an infinite loop - intentional or otherwise), the right thing to do is to call service.shutdown() (where service is your instance of ExecutorService). To do this, you can pass service in to HealthCheck.healthCheck() as a new parameter. Calling shutdown() will allow the currently-running threads to complete, then stop the executor.
If Workers.run() will not naturally complete, best practice says that you need to change your code such that it will. There is a Thread.stop() method you can call to halt the thread and a Thread.suspend() method you can call to suspend the thread. Both of these are double-bad ideas for you to use for two reasons:
They are Deprecated and will leave the Threads in a super-unhealthy state. You will have very difficult problems in the future if you use them.
You are using ExecutorService. That means you are delegating thread management to that class. If you go messing with the state of the Threads underneath ExecutorService, it can't manage the thread pool for you and, again, you will have very difficult problems in the future.

Related

How to change my Thread implementation to use ExecutorService

I had been making a game, and was using Threads in my program to carry out tasks. So let me explain the scenario a bit. I have a BattleManager class which implements Runnable and keep looping in the battle queue for battles, if there are any.
#Override
public void run() {
while(serverRunning){
synchronized (battleQueue) {
for(Battle battle : battleQueue){
if(battle != null){
if (battle instanceof WildBattle) {
if(!((WildBattle) battle).isBattleOver()){
((WildBattle) battle).tryExecuteBattleTurn();
}else{
battleQueue.remove(battle);
battle = null;
}
}
}
}
}
try {
Thread.sleep(3);
} catch (InterruptedException e)
e.printStackTrace();
}
}
currentThread = null;
}
Then I check if battle is not over, and if not I try to execute the battle turn. Since there can be more than 100 battles running at the same time and there are complex calculations inside every battle, I inside WildBattle class spawn a child thread to execute the task, so that the battles run in parallel.
Here is the method which is invoked inside wild battle class, which spawns a new thread.
public void tryExecuteBattleTurn() {
if (!isBattleTurnRunning && battleThread == null) {
battleThread = new Thread(new Runnable() {
#Override
public void run() {
//long startTime = System.currentTimeMillis();
executeBattle();
battleLog.setBattleLog("");
battleThread = null;
//System.err.println("Total execution time : " +(System.currentTimeMillis() - startTime));
}
}, "Battle thread");
battleThread.start();
}
}
Now the main question is, I want to learn about executor service and I read at few places that it is always better to use executor service rather than spawning new child threads. How can I change this to use executor service.
I am not sure though. I am not a java expert and still learning the language so spare me if you see something is wrong, and please let me know if I can change anything to make it more efficient.
Let me know if you are not clear about anything.
I'll show you a basic example and you'll manage how to integrate it with your code
First you create ExecutorService somewhere in your application.
ExecutorService executorService = Executors.newFixedThreadPool(NUMBER_OF_THREADS);
You should choose NUMBER_OF_THREADS based on your application needs. Threads are not created immediately - only when you submit a task to service and there are no available threads for it. If all NUMBER_OF_THREADS are busy, task will wait in queue until one of the threads will be able to handle it. ExecutorService will reuse threads, this will save time on thread instantiation and is a generally good concept to work with threads.
Then you manage how to access executor service from your battles. Then, when you need to perform an asynchronous work you submit task to service:
executorService.submit(new Runnable() {
#Override public void run() {
// your code here
}
}
If your application has a lifecycle and can be somehow shutdown, you'd like to shutdown ExecutorService as well. There are two options - shutdown() and shutdownNow(), first one waits for all current tasks to be executed, second one performs shutdown immediately and returns list of tasks that were not completed.
As was mentioned in comments, you should figure out how to preserve model state and organize thread synchronization based on your real situation.

Java threads scheduling

I have a MainClass, a Worker class and a Supervisor class. In MainClass i create 10 Worker classes and a Supervisor class that run in separate threads.
class MainClass {
public static void main(String args[]) {
for (int i=0; i<10 ;i++) {
Thread t = new Thread( new Worker());
t.start();
}
(new Thread(new Supervisor()).start();
}
.
class Worker extends Thread {
public void run() {
while(true) {
if(some_condition) {
//do stuff
} else {
// pause thread execution for undefined time.
}
}
}
}
.
class Supervisor extends Thread {
public void run() {
while(true) {
if(some_condition) {
// restart Workers thread that are paused.
}
// do other stuff
}
}
}
I don't know how to implement this, cause the conditions in every thread are independent from each other so i don't need to synchronize, so i can't use wait-notify.
I don't know how to implement this, cause the conditions in every thread are independent from each other so i don't need to synchronize, so i can't use wait-notify.
Sure you can.
The subtlety here is that presumably Supervisor doesn't actually know whether worker threads are really paused. (If it does, then the conditions are not independent.)
Since Supervisor doesn't know whether the threads are actually paused (by assumption), you have to design what you want to happen if it tries to unpause an already-unpaused thread.
a) Should an unpause do nothing?
b) Or should it immediately unpause the next time a worker tries to pause itself?
If the answer is (b), then you have to worry about thread safety. If the answer is (a), then you don't (unless you have some other data to pass between threads!)
Either way, you can still use wait and notify.
As per my understanding u want to create separate thread pools which consist of 10 workers or number as per your requirement.
As far as pools are concerned you can check for ThreadPoolExecutor in java.util.concurrent api. Internally ThreadPoolexecutor also creates worker Threads for running tasks.
Try reading ThreadPoolExecutor it might help you or please elaborate your question whats your ultimate objective you wish to achieve by this problem.

Using sleep() and interrupt() to reuse thread

In a swing application, I would like to re-utilize a spawned thread instead of creating a new one to serve requests. This is because the requests would be coming in short intervals of time and the cost of creating a new thread for every request could be high.
I am thinking of using the interrupt() and sleep() methods to do this as below and would like to know any potential performance problems with the code:
public class MyUtils {
private static TabSwitcherThread tabSwitcherThread = null;
public static void handleStateChange(){
if(tabSwitcherThread == null || !tabSwitcherThread.isAlive()){
tabSwitcherThread = new TabSwitcherThread();
tabSwitcherThread.start();
}
else
tabSwitcherThread.interrupt();
}
private static class TabSwitcherThread extends Thread{
#Override
public void run() {
try {
//Serve request code
//Processing complete, sleep till next request is received (will be interrupted)
Thread.sleep(60000);
} catch (InterruptedException e) {
//Interrupted execute request
run();
}
//No request received till sleep completed so let the thread die
}
}
}
Thanks
I wouldn't use sleep() and interrupt() - I'd use wait() and notify() if I absolutely had to.
However, is there any real need to do this instead of using a ThreadPoolExecutor which can handle the thread reuse for you? Or perhaps use a BlockingQueue in a producer/consumer fashion?
Java already provides enough higher-level building blocks for this that you shouldn't need to go down to this level yourself.
I think what you're looking for is a ThreadPool. Java 5 and above comes with ThreadPoolExecutor. I would suggest you use what is provided with Java instead of writing your own, so you can save yourself a lot of time and hairs.
Of course, if you absolutely has to do it the way you described (hey, sometimes business requirement make our life hard), then use wait() and notify() as Jon suggested. I would not use sleep() in this case because you have to specified timeout, and you never know when the next request will come in. Having a thread that keep waking up then go back to sleep seems a bit wasteful of CPU cycle for me.
Here is a nice tutorial about the ThreadPoolExecutor.
EDIT:
Here is some code example:
public class MyUtils {
private static UIUpdater worker = null;
private static ExecutorService exeSrv = Executors.newFixedThreadPool(1);
public static void handleStateChange(){
if(tabSwitcherThread == null || !tabSwitcherThread.isAlive()){
worker = new UIUpdater();
}
//this call does not block
exeSrv.submit(worker, new Object());
}
private static class UIUpdater implements Runnable{
#Override
public void run() {
//do server request and update ui.
}
}
}

How to indefinitely pause a thread in Java and later resume it?

Maybe this question has been asked many times before, but I never found a satisfying answer.
The problem:
I have to simulate a process scheduler, using the round robin strategy. I'm using threads to simulate processes and multiprogramming; everything works fine with the JVM managing the threads. But the thing is that now I want to have control of all the threads so that I can run each thread alone by a certain quantum (or time), just like real OS processes schedulers.
What I'm thinking to do:
I want have a list of all threads, as I iterate the list I want to execute each thread for their corresponding quantum, but as soon the time's up I want to pause that thread indefinitely until all threads in the list are executed and then when I reach the same thread again resume it and so on.
The question:
So is their a way, without using deprecated methods stop(), suspend(), or resume(), to have this control over threads?
Yes, there is:
Object.wait( ), Object.notify() and a bunch of other much nicer synchronization primitives in java.util.concurrent.
Who said Java is not low level enough?
Here is my 3 minute solution. I hope it fits your needs.
import java.util.ArrayList;
import java.util.List;
public class ThreadScheduler {
private List<RoundRobinProcess> threadList
= new ArrayList<RoundRobinProcess>();
public ThreadScheduler(){
for (int i = 0 ; i < 100 ; i++){
threadList.add(new RoundRobinProcess());
new Thread(threadList.get(i)).start();
}
}
private class RoundRobinProcess implements Runnable{
private final Object lock = new Object();
private volatile boolean suspend = false , stopped = false;
#Override
public void run() {
while(!stopped){
while (!suspend){
// do work
}
synchronized (lock){
try {
lock.wait();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return;
}
}
}
}
public void suspend(){
suspend = true;
}
public void stop(){
suspend = true;stopped = true;
synchronized (lock){
lock.notifyAll();
}
}
public void resume(){
suspend = false;
synchronized (lock){
lock.notifyAll();
}
}
}
}
Please note that "do work" should not be blocking.
Short answer: no. You don't get to implement a thread scheduler in Java, as it doesn't operate at a low enough level.
If you really do intend to implement a process scheduler, I would expect you to need to hook into the underlying operating system calls, and as such I doubt this will ever be a good idea (if remotely possible) in Java. At the very least, you wouldn't be able to use java.lang.Thread to represent the running threads so it may as well all be done in a lower-level language like C.

How do you kill a Thread in Java?

How do you kill a java.lang.Thread in Java?
See this thread by Sun on why they deprecated Thread.stop(). It goes into detail about why this was a bad method and what should be done to safely stop threads in general.
The way they recommend is to use a shared variable as a flag which asks the background thread to stop. This variable can then be set by a different object requesting the thread terminate.
Generally you don't..
You ask it to interrupt whatever it is doing using Thread.interrupt() (javadoc link)
A good explanation of why is in the javadoc here (java technote link)
In Java threads are not killed, but the stopping of a thread is done in a cooperative way. The thread is asked to terminate and the thread can then shutdown gracefully.
Often a volatile boolean field is used which the thread periodically checks and terminates when it is set to the corresponding value.
I would not use a boolean to check whether the thread should terminate. If you use volatile as a field modifier, this will work reliable, but if your code becomes more complex, for instead uses other blocking methods inside the while loop, it might happen, that your code will not terminate at all or at least takes longer as you might want.
Certain blocking library methods support interruption.
Every thread has already a boolean flag interrupted status and you should make use of it. It can be implemented like this:
public void run() {
try {
while (!interrupted()) {
// ...
}
} catch (InterruptedException consumed)
/* Allow thread to exit */
}
}
public void cancel() { interrupt(); }
Source code adapted from Java Concurrency in Practice. Since the cancel() method is public you can let another thread invoke this method as you wanted.
One way is by setting a class variable and using it as a sentinel.
Class Outer {
public static volatile flag = true;
Outer() {
new Test().start();
}
class Test extends Thread {
public void run() {
while (Outer.flag) {
//do stuff here
}
}
}
}
Set an external class variable, i.e. flag = true in the above example. Set it to false to 'kill' the thread.
I want to add several observations, based on the comments that have accumulated.
Thread.stop() will stop a thread if the security manager allows it.
Thread.stop() is dangerous. Having said that, if you are working in a JEE environment and you have no control over the code being called, it may be necessary; see Why is Thread.stop deprecated?
You should never stop stop a container worker thread. If you want to run code that tends to hang, (carefully) start a new daemon thread and monitor it, killing if necessary.
stop() creates a new ThreadDeathError error on the calling thread and then throws that error on the target thread. Therefore, the stack trace is generally worthless.
In JRE 6, stop() checks with the security manager and then calls stop1() that calls stop0(). stop0() is native code.
As of Java 13 Thread.stop() has not been removed (yet), but Thread.stop(Throwable) was removed in Java 11. (mailing list, JDK-8204243)
There is a way how you can do it. But if you had to use it, either you are a bad programmer or you are using a code written by bad programmers. So, you should think about stopping being a bad programmer or stopping using this bad code.
This solution is only for situations when THERE IS NO OTHER WAY.
Thread f = <A thread to be stopped>
Method m = Thread.class.getDeclaredMethod( "stop0" , new Class[]{Object.class} );
m.setAccessible( true );
m.invoke( f , new ThreadDeath() );
I'd vote for Thread.stop().
As for instance you have a long lasting operation (like a network request).
Supposedly you are waiting for a response, but it can take time and the user navigated to other UI.
This waiting thread is now a) useless b) potential problem because when he will get result, it's completely useless and he will trigger callbacks that can lead to number of errors.
All of that and he can do response processing that could be CPU intense. And you, as a developer, cannot even stop it, because you can't throw if (Thread.currentThread().isInterrupted()) lines in all code.
So the inability to forcefully stop a thread it weird.
The question is rather vague. If you meant “how do I write a program so that a thread stops running when I want it to”, then various other responses should be helpful. But if you meant “I have an emergency with a server I cannot restart right now and I just need a particular thread to die, come what may”, then you need an intervention tool to match monitoring tools like jstack.
For this purpose I created jkillthread. See its instructions for usage.
There is of course the case where you are running some kind of not-completely-trusted code. (I personally have this by allowing uploaded scripts to execute in my Java environment. Yes, there are security alarm bell ringing everywhere, but it's part of the application.) In this unfortunate instance you first of all are merely being hopeful by asking script writers to respect some kind of boolean run/don't-run signal. Your only decent fail safe is to call the stop method on the thread if, say, it runs longer than some timeout.
But, this is just "decent", and not absolute, because the code could catch the ThreadDeath error (or whatever exception you explicitly throw), and not rethrow it like a gentlemanly thread is supposed to do. So, the bottom line is AFAIA there is no absolute fail safe.
'Killing a thread' is not the right phrase to use. Here is one way we can implement graceful completion/exit of the thread on will:
Runnable which I used:
class TaskThread implements Runnable {
boolean shouldStop;
public TaskThread(boolean shouldStop) {
this.shouldStop = shouldStop;
}
#Override
public void run() {
System.out.println("Thread has started");
while (!shouldStop) {
// do something
}
System.out.println("Thread has ended");
}
public void stop() {
shouldStop = true;
}
}
The triggering class:
public class ThreadStop {
public static void main(String[] args) {
System.out.println("Start");
// Start the thread
TaskThread task = new TaskThread(false);
Thread t = new Thread(task);
t.start();
// Stop the thread
task.stop();
System.out.println("End");
}
}
There is no way to gracefully kill a thread.
You can try to interrupt the thread, one commons strategy is to use a poison pill to message the thread to stop itself
public class CancelSupport {
public static class CommandExecutor implements Runnable {
private BlockingQueue<String> queue;
public static final String POISON_PILL = “stopnow”;
public CommandExecutor(BlockingQueue<String> queue) {
this.queue=queue;
}
#Override
public void run() {
boolean stop=false;
while(!stop) {
try {
String command=queue.take();
if(POISON_PILL.equals(command)) {
stop=true;
} else {
// do command
System.out.println(command);
}
} catch (InterruptedException e) {
stop=true;
}
}
System.out.println(“Stopping execution”);
}
}
}
BlockingQueue<String> queue=new LinkedBlockingQueue<String>();
Thread t=new Thread(new CommandExecutor(queue));
queue.put(“hello”);
queue.put(“world”);
t.start();
Thread.sleep(1000);
queue.put(“stopnow”);
http://anandsekar.github.io/cancel-support-for-threads/
Generally you don't kill, stop, or interrupt a thread (or check wheter it is interrupted()), but let it terminate naturally.
It is simple. You can use any loop together with (volatile) boolean variable inside run() method to control thread's activity. You can also return from active thread to the main thread to stop it.
This way you gracefully kill a thread :) .
Attempts of abrupt thread termination are well-known bad programming practice and evidence of poor application design. All threads in the multithreaded application explicitly and implicitly share the same process state and forced to cooperate with each other to keep it consistent, otherwise your application will be prone to the bugs which will be really hard to diagnose. So, it is a responsibility of developer to provide an assurance of such consistency via careful and clear application design.
There are two main right solutions for the controlled threads terminations:
Use of the shared volatile flag
Use of the pair of Thread.interrupt() and Thread.interrupted() methods.
Good and detailed explanation of the issues related to the abrupt threads termination as well as examples of wrong and right solutions for the controlled threads termination can be found here:
https://www.securecoding.cert.org/confluence/display/java/THI05-J.+Do+not+use+Thread.stop%28%29+to+terminate+threads
Here are a couple of good reads on the subject:
What Do You Do With InterruptedException?
Shutting down threads cleanly
I didn't get the interrupt to work in Android, so I used this method, works perfectly:
boolean shouldCheckUpdates = true;
private void startupCheckForUpdatesEveryFewSeconds() {
Thread t = new Thread(new CheckUpdates());
t.start();
}
private class CheckUpdates implements Runnable{
public void run() {
while (shouldCheckUpdates){
//Thread sleep 3 seconds
System.out.println("Do your thing here");
}
}
}
public void stop(){
shouldCheckUpdates = false;
}
Thread.stop is deprecated so how do we stop a thread in java ?
Always use interrupt method and future to request cancellation
When the task responds to interrupt signal, for example, blocking queue take method.
Callable < String > callable = new Callable < String > () {
#Override
public String call() throws Exception {
String result = "";
try {
//assume below take method is blocked as no work is produced.
result = queue.take();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
return result;
}
};
Future future = executor.submit(callable);
try {
String result = future.get(5, TimeUnit.SECONDS);
} catch (TimeoutException e) {
logger.error("Thread timedout!");
return "";
} finally {
//this will call interrupt on queue which will abort the operation.
//if it completes before time out, it has no side effects
future.cancel(true);
}
When the task does not respond to interrupt signal.Suppose the task performs socket I/O which does not respond to interrupt signal and thus using above approach will not abort the task, future would time out but the cancel in finally block will have no effect, thread will keep on listening to socket. We can close the socket or call close method on connection if implemented by pool.
public interface CustomCallable < T > extends Callable < T > {
void cancel();
RunnableFuture < T > newTask();
}
public class CustomExecutorPool extends ThreadPoolExecutor {
protected < T > RunnableFuture < T > newTaskFor(Callable < T > callable) {
if (callable instanceof CancellableTask)
return ((CancellableTask < T > ) callable).newTask();
else
return super.newTaskFor(callable);
}
}
public abstract class UnblockingIOTask < T > implements CustomCallable < T > {
public synchronized void cancel() {
try {
obj.close();
} catch (IOException e) {
logger.error("io exception", e);
}
}
public RunnableFuture < T > newTask() {
return new FutureTask < T > (this) {
public boolean cancel(boolean mayInterruptIfRunning) {
try {
this.cancel();
} finally {
return super.cancel(mayInterruptIfRunning);
}
}
};
}
}
After 15+ years of developing in Java there is one thing I want to say to the world.
Deprecating Thread.stop() and all the holy battle against its use is just another bad habit or design flaw unfortunately became a reality... (eg. want to talk about the Serializable interface?)
The battle is focusing on the fact that killing a thread can leave an object into an inconsistent state. And so? Welcome to multithread programming. You are a programmer, and you need to know what you are doing, and yes.. killing a thread can leave an object in inconsistent state. If you are worried about it use a flag and let the thread quit gracefully; but there are TONS of times where there is no reason to be worried.
But no.. if you type thread.stop() you're likely to be killed by all the people who looks/comments/uses your code. So you have to use a flag, call interrupt(), place if(!flag) all around your code because you're not looping at all, and finally pray that the 3rd-party library you're using to do your external call is written correctly and doesn't handle the InterruptException improperly.

Categories