I'm currently working on a daemon that will be doing A LOT of different tasks. It's multi threaded and is being built to handle almost any kind of internal-error without crashing. Well I'm getting to the point of handling a shutdown request and I'm not sure how I should go about doing it.
I have a shutdown hook setup, and when it's called it sets a variable telling the main daemon loop to stop running. The problem is, this daemon spawns multiple threads and they can take a long time. For instance, one of these threads could be converting a document. Most of them will be quick (I'm guessing under 10 seconds), but there will be threads that can last as long as 10+ minutes.
What I'm thinking of doing right now is when a shutdown hook has been sent, do a loop for like 5 seconds on ThreadGroup.activeCount() with a 500ms (or so) Sleep (all these threads are in a ThreadGroup) and before this loop, I will send a notification to all threads telling them a shutdown request has been called. Then they will have to instantly no matter what they're doing cleanup and shutdown.
Anyone else have any suggestions? I'm interested in what a daemon like MySQL for instance does when it gets told to stop, it stops instantly. What happens if like 10 query's are running that are very slow are being called? Does it wait or does it just end them. I mean servers are really quick, so there really isn't any kind of operation that I shouldn't be able to do in less than a second. You can do A LOT in 1000ms now days.
Thanks
The java.util.concurrent package provides a number of utilities, such as ThreadPoolExecutor (along with various specialized types of other Executor implementations from the Executors class) and ThreadPoolExecutor.awaitTermination(), which you might want to look into - as they provide the same exact functionality you are looking to implement. This way you can concentrate on implementing the actual functionality of your application/tasks instead of worrying about things like thread and task scheduling.
Are your thread jobs amenable to interruption via Thread#interrupt()? Do they mostly call on functions that themselves advertise throwing InterruptedException? If so, then the aforementioned java.util.concurrent.ExecutorService#shutdownNow() is the way to go. It will interrupt any running threads and return the list of jobs that were never started.
Similarly, if you hang on to the Futures produced by ExecutorService#submit(), you can use Future#cancel(boolean) and pass true to request that a running job be interrupted.
Unless you're calling on code out of your control that swallows interrupt signals (say, by catching InterruptedException without calling Thread.currentThread().interrupt()), using the built-in cooperative interruption facility is a better choice than introducing your own flags to approximate what's already there.
Related
I get that with threads being nonblocking, we don't need to have Thread sprawl depending on N concurrent requests, but rather we put our tasks in a single event loop in our reactive web programming pattern.
Yes, that can help, but since the event loop is a queue, what if the first task to be processed blocks forever? Then the event loop will never progress and thus end of responses and processing other than queueing more tasks. Yes, timeouts are probably possible, but I can't wrap my head around how the event loop can be a good solution.
Say you have 3 tasks that take 3 seconds to wait for IO and run each executions and they got submitted to the event queue. Then they will still take 9 seconds to be able to be processed and also to execute once IO resolved. In the case of making threads that block, this would have resolved in 3 seconds since they run concurrently.
Where I can see a benefit is if the event loop is not really a queue and upon signal that a task is ready to be processed, it dispatches that task to be processed. In that case though, this would mean that order of task execution is not maintained and also each task has to still be running a thread in order to be able to tell when IO is resolved.
Maybe I am not understanding the event loop and thread handling correctly. Can someone correct me please because it seems like this Reactor pattern seems to make things possibly worse.
Lastly, upon X requests in Spring Reactor, does only 1 thread get created to run handlers instead of the traditional X threads? In that case, if someone accidently wrote blocking code, doesnt that mean each subsequent requests get queued?
It is not a good idea to use the event loop for long running tasks. This is considered an anti-pattern. Usually it is merely used for quickly picking up imminent events, but not actually doing the work associated with these events if the work would block the event loop noticeably. You would want to use a separate thread pool for executing long running tasks. So the event loop would usually only initiate work using asynchronous and hence non-blocking structures (or actually doing the work only if it can be done very quickly) and pass the heavier and possibly blocking tasks to a separate thread pool (for CPU intensive computations) or to the operating system (such as data buffers to be sent over the network).
Also, don't be fooled by the fact that only one thread is dealing with the events, it is very fast and is usually enough for even demanding applications. Platforms like NodeJS or frameworks like Netty (used in Akka, Play framework, Apache Cassandra, etc.) are using an event loop at their heart with great success. One should just be aware of the fact, that performing blocking operations inside the event loop is generally a bad idea.
Please have a look at some of these posts for more information:
The reactor pattern and non blocking IO
Unix Network Programming
Kotlin Webflux
Slightly off topic but still a very prominent example: Don't Block the Event Loop (NodeJS)
I am currently running the JAR that I cannot change, and sometimes it simply gets stuck for no good reason. I have tried finding the ways to interrupt the thread, stop the thread, etceteras, but no luck.
Each solution offered was about doing the complete exit or waiting for a thread to complete.
What I want to do is to simply close the thread, exactly when the timeout completes, and carry on with the program.
What I do not want to do is use the while loop with a timeout, java.util.concurrent.Future, System.exit, and make a Thread.interrupt call.
None of these will help!
You can't forcibly stop a thread in mid-execution. The Thread.destroy() method would have done that, but it was never implemented, and its documentation explains why it would be unsafe to use even if it worked.
There are some other deprecated methods like Thread.stop() and Thread.suspend() which may actually work, but they're also unsafe to use; again, their documentation explains why.
Telling the thread that it should terminate itself, and then waiting for it to do so, is the only safe way to stop a thread.
As an workaround, you could run your task in an entirely separate process, so that you can destroy it when you want it to stop. That is safe, since processes are isolated from each other and destroying the child process can't leave the parent process in an unstable state.
Interacting with a separate process is more difficult, though, since you can't share variables between processes like you can with threads. You'd need to send messages through the process's input and output streams.
Actually, you can't really solve this!
What I mean is: even if you would manage to kill "your" thread that you used to trigger the 3rd party code - you have no way of killing threads or processes created by the code you are invoking.
If you want to be absolutely sure to kill all and anything, you might have to look into rather complex solutions like:
instead of just using a thread, you create a new process with a new JVM B
in that JVM B, you can call that library
but of course, that requires that you put additional code around; so that "your" code in JVM A can talk to "your" code in JVM B
And now you might be able to tear down that process, and all artifacts belonging to it. Maybe.
And seriously: to be really really sure that the 3rd party library didn't kick of anything that you can't stop; you might even have to run that JVM inside some kind of container (for example a docker instance). That you could tear down and be sure that everything is gone.
Long story short: I think there is no way to absolutely control the threads created in a thread. If you need that level of control, you need to look into "outsourcing" those calls.
You can use Executor for this. It allows you to submit tasks (e.g. runnable) and executes those tasks parallely. Also, once you call shutdown(), it lets you configure the timeout and kills all the workers if they are not finished by that time. An example would look like this:
ExecutorService executor = Executors.newFixedThreadPool(1);
executor.execute(() -> {
//logic to call the method of third party jar
});
//Other business logic
executor.awaitTermination(1, TimeUnit.MINUTES);
executor.shutdownNow();
TimeUnit is an enum, with values like SECONDS, HOURS, MINUTES etc (here's javadoc) so you can configure different time units. A couple of points:
Once shutdownNow is called, no new tasks will be accepted (i.e. you can't call execute or submit) and existing tasks will be stopped. So, we are basically waiting for a minute for tasks to be complete and if it is not complete, we are killing that task.
awaitTermination throws InterruptedException (as it interrupts the threads internally if they are not finished) so you will have to wrap it inside try-catch block.
Here's javadoc for Executor.
I am working on a multithreaded game in java. I have several worker threads that fetch modules from a central thread manager, which then executes it on its own. Now I would like to be able to pause such a thread if it temporarily has nothing to execute. I have tried calling the wait() method on it from the thread manager, but that only resulted in it ignoring the notify() call that followed it.
I googled a bit on it too, only finding that most sites refer to functions like suspend(), pause(), etc, which are now marked a deprecated on the java documentation pages.
So in general, what is the way to pause or continue a thread on demand?
You can use an if block in the thread with a sentinal variable that is set to false if you want to halt the thread's action. This works best if the thread is performing loops.
Maybe I'm missing the point, but if they have nothing to do, why not just let them die? Then spawn a new thread when you have work for one to do again.
It sounds to me like you're trying to have the conversation both ways. In my (humble) opinion, you should either have the worker threads responsible for asking the central thread manager for work (or 'modules'), or you should have the central thread manager responsible for doling out work and kicking off the worker threads.
What it sounds like is that most of the time the worker threads are responsible for asking for work. Then, sometimes, the responsibility flips round to the thread manager to tell the workers not to ask for a while. I think the system will stay simpler if this responsibility stays on only one side.
So, given this, and with my limited knowledge of what you're developing, I would suggest either:
Have the thread manager kick of worker threads when there's stuff to do and keep track of their progress, letting them die when they're done and only creating new ones when there's new stuff to do. Or
Have a set number of always existing worker threads that poll the thread manager for work and (if there isn't any) sleep for a period of time using Thread.sleep() before trying again. This seems pretty wasteful to me so I would lean towards option 1 unless you've a good reason not to?
In the grand tradition of not answering your question, and suggest that You Are Doing It Wrong, I Offer this :-)
Maybe you should refactor your code to use a ExecutorService, its a rather good design.
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html
There are many ways to do this, but in the commonest (IMO), the worker thread calls wait() on the work queue, while the work generator should call notify(). This causes the worker thread to stop, without the thread manager doing anything. See e.g. this article on thread pools and work queues.
use a blocking queue to fetch those modules using take()
or poll(time,unit) for a timed out wait so you can cleanly shutdown
these will block the current thread until a module is available
I have a java program that spawns out 4 threads. In each thread, I have multiple socket timeouts. However, it seems that these timeouts are not respected i.e. the readLine() function might block for a longer period of time.
I want the following behavior: If I set a socket timeout to 300 ms, then I want the readLine() function to return within 300 ms from when the readLine() (i.e. the underlying select call) was invoked, no matter what. I understand that the OS scheduler would be putting the threads to sleep while doing processor sharing, but is there any way in Java to force the threads to always be woken up to ensure this behavior? Or is this just not the right way to think when doing multi-threaded programming?
Ideally, since I am spawning out 4 threads and running on a 6-core machine, each thread should be able to get its own CPU and run in parallel, and respect the select timeout ... but that's perhaps too much to expect ...
PS: I actually do use Thread.interrupt() to ensure that each of my threads exits within a certain time (I check the time elapsed in the master thread, and interrupt the child threads if its been too long). In each of my threads, I connect to a (different) server, make a request, and wait for a response. I do not know how long the response will be. So I keep on calling the readLine() method, till it times out with a SocketTimeoutException. I enforce a timeout of 300 ms, since I expect the server to start responding within this time. The reason I want to enforce this timeout is that the server behaves in a broadcast fashion, and sends responses to a request by a single client to all clients. So if I don't have a timeout, I will keep on getting data in response to requests by some other clients.
If I really understood your problem, you can always try to invoke Thread.interrupt() in the thread that is performing the readLine() operation. As you didn't provide any code, I leave this link for you to read. It's general, but it provides enough information about interrupting threads.
These two links may also be of use to you: How do you kill a thread in Java? and How to abort a thread in a fast and clean way in java?.
Regarding your question about the OS scheduler, you should be aware that in general purpose OSes you do not have total control on the way the OS schedules tasks. For instance, in Linux the interrupts are the highest priority tasks, then there are scheduling policies that enable you to put "some" determinism on how the tasks are scheduled. In Java, you can use the setPriority() method to change the priority of the thread, but in fact it is the same as using the nice command, and still you don't get any guarantees that this thread will be scheduled ahead of other OS threads.
I hope this helps.
You are making some wrong assumptions here:
that the timeout will be exactly 300ms. In fact the timeout is that will be at least 300ms.
the OS Scheduler does not do anything (rather than schedule the java os processes) within the java threads.
having 6 core, does not mean that each one of your threads will run at separate core, it is not possible to bind thread->core at java
for last, you consider that the jvm has only yours 4 threads running, but in fact there has more threads, by example, the garbage collector thread(s).
Asking your question: "is there any way in Java to force the threads to always be woken up to ensure this behavior? "
Yes, depending how it is your code, if the thread is thread.sleep(), you can use thread.interrupt() (for readline() uses it) and handle the InterruptionException or if they are object.wait(), you can use object.notify() or object.notifyAll().
When writing a multithread internet server in java, the main-thread starts new
ones to serve incoming requests in parallel.
Is any problem if the main-thread does not wait ( with .join()) for them?
(It is obviously absurd create a new thread and then, wait for it).
I know that, in a practical situation, you should (or "you must"?) implement a pool
of threads to "re-use" them for new requests when they become idle.
But for small applications, should we use a pool of threads?
You don't need to wait for threads.
They can either complete running on their own (if they've been spawned to perform one particular task), or run indefinitely (e.g. in a server-type environment).
They should handle interrupts and respond to shutdown requests, however. See this article on how to do this correctly.
If you need a set of threads I would use a pool and executor methods since they'll look after thread resource management for you. If you're writing a multi-threaded network server then I would investigating using (say) a servlet container or a framework such as Mina.
The only problem in your approach is that it does not scale well beyond a certain request rate. If the requests are coming in faster than your server is able to handle them, the number of threads will rise continuously. As each thread adds some overhead and uses CPU time, the time for handling each request will get longer, so the problem will get worse (because the number of threads rises even faster). Eventually no request will be able to get handled anymore because all of the CPU time is wasted with overhead. Probably your application will crash.
The alternative is to use a ThreadPool with a fixed upper bound of threads (which depends on the power of the hardware). If there are more requests than the threads are able to handle, some requests will have to wait too long in the request queue, and will fail due to a timeout. But the application will still be able to handle the rest of the incoming requests.
Fortunately the Java API already provides a nice and flexible ThreadPool implementation, see ThreadPoolExecutor. Using this is probably even easier than implementing everything with your original approach, so no reason not to use it.
Thread.join() lets you wait for the Thread to end, which is mostly contrary to what you want when starting a new Thread. At all, you start the new thread to do stuff in parallel to the original Thread.
Only if you really need to wait for the spawned thread to finish, you should join() it.
You should wait for your threads if you need their results or need to do some cleanup which is only possible after all of them are dead, otherwise not.
For the Thread-Pool: I would use it whenever you have some non-fixed number of tasks to run, i.e. if the number depends on the input.
I would like to collect the main ideas of this interesting (for me) question.
I can't totally agree with "you
don't need to wait for threads".
Only in the sense that if you don't
join a thread (and don't have a
pointer to it) once the thread is
done, its resources are freed
(right? I'm not sure).
The use of a thread pool is only
necessary to avoid the overhead of
thread creation, because ...
You can limit the number of parallel
running threads by accounting, with shared variables (and without a thread pool), how many of then
were started but not yet finished.