I am in a wierd situation. In my web-server (tomcat), on web request, I basically need to cancel a previous request. I have a reference to the thread that was executing the previous request. So I can directly interrupt that thread and the node will do the rest.
I know you are not suppose to interrupt the thread which you do not own. But is it safe to interrupt tomcat thread in this case? What can be the other way? Maintaining own thread pool is waste of resources and ovehead
Maintaining your own thread pool is a waste of resource but it's also a gain in every other respect, like stability of your application server. So you need to decide what is more important: A few thousand bytes of memory and CPU cycles or a stable, reliable application.
The problem with interrupting another thread is that you usually can't know for sure where in the code that other thread is. You might want to use locking for this:
Thread A locks something while it's safe to interrupt, thread B checks the lock and if it can't get it, it interrupts A.
But what happens when A is just about to give up the lock, B checks the lock, A unlocks and starts with cleanup, B sends interrupt?
So you should really use your own thread pool.
I would not do it. Not 100% sure why, but I think those threads come from Tomcat's own worker thread pool and killing them one-by-one would/could eventually result in a non-responding Tomcat instance. (This is just a hypothesis).
I would argue that "maintaining own threadpool is waste of resources and overhead". I think it is a minor thing, threadpools are great guys, do no be afraid of them. I do not know the details of your application but I think if you measured the overhead by JConsole you could decide the point to do some optimization and it is not probable that the threadpool would be the bottleneck.
The best think I could suggest to you is a complete redesign: use short-returning HTTP requests to start long-running asynchronous operations in the background by submitting tasks to an ExecutorService or stuff. This way there is no need to harm Tomcat's own threads and the overall usability of your application could also be improved from a user/client perspective.
To sum up: I think it is not safe to do what you mentioned and one possible other way to do what you want is described in the above paragraph.
Related
For every request, there is a lot of computation happens. On an average the reply takes about 10 minutes to process. Now in the mean time, if a user sends a new request; There is absolutely no point for the previous request to continue.
So I have written a code where I basically interrupt the previous thread executing it. Is it a good practice in tomcat environment? Can there be a better solution to deal with it. Is it alright to interrupt tomcat threads.
Or should I manage my own threadpool and let pool do the computation for me?
More Information:
Basically the whole task is wrapped with a FutureTask. For every request, this task is executed and the reference to the task is stored by a ConcurrentHashMap. For every request, all the future's in the map is "cancelled" and then proceeds to execute the latest request. Thus cancelling the previous requests.
Q> I basically interrupt the previous thread executing it. Is it a good practice in tomcat environment?
A> I think it's fine as long as you're happy having HTTP thread(s) blocked for 10 minutes. This means no other user would be able to process HTTP requests. Otherwise create your own thread pool and manage it.
Q> Is it a good practice in tomcat environment?
A> Interrupting Runnables or Callables can be tricky. For example, if your thread is in the middle of I/O operation, interrupting can leave the data in a corrupt state. Other than that, this is quite normal practice. I also recommend using your own thread pool in order for your server capacity to be predictable.
Can you break your large task into a lot of smaller tasks? Sticking a conditional and exiting early could be a good alternative to interruption.
Alternatively, does waiting for/ensuring the first task finishes the operation and others just return the same value make sense in your environment? If so, I'd rather prefer that instead of your approach. There's LoadingCache from guava library which does exactly that.
I am not sure I understand your question completely...
But, if you are talking of running a threadpool within a tomcat application and cancelling its future tasks, I see no problem with that.
I would not interrupt a thread allocated by tomcat, unless I would write code to deal with the interrupt personally (such as within the servlet class)
Can anybody explain with examples about why should we use Thread-pools.
I have know about use of threadpools with Executors theoretically.
I have gone through number of tutorials, but I didn't get any practically examples about why should we use Threadpools, it can be newFixedThreadPool or newCachedThreadPool or newSingleThreadExecutor
in terms of scalability and performance .
If anybody explain me with respect to performance and scalability with examples about it?
First off, check this description of thread pools that I wrote yesterday: Android Thread Pool to manage multiple bluetooth handeling threads? (ok, it was about android but it's the same for classic java).
The main use I always seem to find for using a threadpool is that is very nicely manages a very common problem: producer-consumer. In this pattern, someone needs to constantly send work items (the producer) to be processed by someone else (the consumers). The work items are obtained from some stream-like source, like a socket, a database, or a collection of disk files, and needs multiple workers in order to be processed efficiently. The main components identifiable here are:
the producer: a thread that keeps posting jobs
a queue where the jobs are posted
the consumers: worker threads that take jobs from the queue and execute them
In addition to this, synchronization needs to be employed to make all this work correctly, since reading and writing to the queue without synchronization can lead to corrupted and inconsistent data. Also, we need to make the system efficient, since the consumers should not waste CPU cycles when there is nothing to do.
Now this pattern is very common, but to implement it from scratch it takes a considerable effort, which is error prone and needs to be carefully reviewed.
The solution is the thread pool. It very conveniently manages the work queue, the consumer threads and all the synchronization needed. All you need to do is play the role of the producer and feed the pool with tasks!
I would start with a problem and only then try to find a solution for it.
If you start the way you have, you can have a solution looking for a problem to solve and you are likely to use it inappropriately.
If you can't think of a use for thread pools, don't use them. ;)
A common mistake people make is to assume that because they have lots of cpus now, they have to use them all as if this were a reason in itself. Its like saying I have lots of disk space, I must find a way to use all of it.
A good reason to use thread pools is to improve the performance of CPU bounds processes and the simplicity of IO bound processes (rather than using non-blocking IO with one thread)
If you have a busy CPU bound process which performs tasks which can be executed independently you have a good use case for a thread pool.
Note: Thread pool often has just one thread. There are specific static factories for these. If you want a simple background worker, this may be an option.
Note 2: A common mistake is to assume that a CPU bound tasks will run best on hundreds or thousands of threads. The optimial number of threads can be the number of core or cpus you have. Once all these are busy, you may find additional threads just add overhead.
Initializing a new thread (and its own stack) is a costly operation.
Thread pools are use to avoid this cost by reusing threads already created. Thus using thread pools you get better performance then creating new threads every time.
Also note that created threads might need to be "deleted" after they have been used, which increases the cost of garbage collection and the frequency it will happen (as the memory fills up faster).
This analysis is just from the performance point of view. I cannot think of an advantage of using thread pools in terms of scalability at the moment.
I googled "why use java thread pools" and found:
A thread pool offers a solution to both the problem of thread
life-cycle overhead and the problem of resource thrashing.
http://www.ibm.com/developerworks/library/j-jtp0730/index.html
and
The newCachedThreadPool method creates an executor with an expandable
thread pool. This executor is suitable for applications that launch
many short-lived tasks.
The newSingleThreadExecutor method creates an
executor that executes a single task at a time.
http://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html
I am working on a multithreaded game in java. I have several worker threads that fetch modules from a central thread manager, which then executes it on its own. Now I would like to be able to pause such a thread if it temporarily has nothing to execute. I have tried calling the wait() method on it from the thread manager, but that only resulted in it ignoring the notify() call that followed it.
I googled a bit on it too, only finding that most sites refer to functions like suspend(), pause(), etc, which are now marked a deprecated on the java documentation pages.
So in general, what is the way to pause or continue a thread on demand?
You can use an if block in the thread with a sentinal variable that is set to false if you want to halt the thread's action. This works best if the thread is performing loops.
Maybe I'm missing the point, but if they have nothing to do, why not just let them die? Then spawn a new thread when you have work for one to do again.
It sounds to me like you're trying to have the conversation both ways. In my (humble) opinion, you should either have the worker threads responsible for asking the central thread manager for work (or 'modules'), or you should have the central thread manager responsible for doling out work and kicking off the worker threads.
What it sounds like is that most of the time the worker threads are responsible for asking for work. Then, sometimes, the responsibility flips round to the thread manager to tell the workers not to ask for a while. I think the system will stay simpler if this responsibility stays on only one side.
So, given this, and with my limited knowledge of what you're developing, I would suggest either:
Have the thread manager kick of worker threads when there's stuff to do and keep track of their progress, letting them die when they're done and only creating new ones when there's new stuff to do. Or
Have a set number of always existing worker threads that poll the thread manager for work and (if there isn't any) sleep for a period of time using Thread.sleep() before trying again. This seems pretty wasteful to me so I would lean towards option 1 unless you've a good reason not to?
In the grand tradition of not answering your question, and suggest that You Are Doing It Wrong, I Offer this :-)
Maybe you should refactor your code to use a ExecutorService, its a rather good design.
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html
There are many ways to do this, but in the commonest (IMO), the worker thread calls wait() on the work queue, while the work generator should call notify(). This causes the worker thread to stop, without the thread manager doing anything. See e.g. this article on thread pools and work queues.
use a blocking queue to fetch those modules using take()
or poll(time,unit) for a timed out wait so you can cleanly shutdown
these will block the current thread until a module is available
When writing a multithread internet server in java, the main-thread starts new
ones to serve incoming requests in parallel.
Is any problem if the main-thread does not wait ( with .join()) for them?
(It is obviously absurd create a new thread and then, wait for it).
I know that, in a practical situation, you should (or "you must"?) implement a pool
of threads to "re-use" them for new requests when they become idle.
But for small applications, should we use a pool of threads?
You don't need to wait for threads.
They can either complete running on their own (if they've been spawned to perform one particular task), or run indefinitely (e.g. in a server-type environment).
They should handle interrupts and respond to shutdown requests, however. See this article on how to do this correctly.
If you need a set of threads I would use a pool and executor methods since they'll look after thread resource management for you. If you're writing a multi-threaded network server then I would investigating using (say) a servlet container or a framework such as Mina.
The only problem in your approach is that it does not scale well beyond a certain request rate. If the requests are coming in faster than your server is able to handle them, the number of threads will rise continuously. As each thread adds some overhead and uses CPU time, the time for handling each request will get longer, so the problem will get worse (because the number of threads rises even faster). Eventually no request will be able to get handled anymore because all of the CPU time is wasted with overhead. Probably your application will crash.
The alternative is to use a ThreadPool with a fixed upper bound of threads (which depends on the power of the hardware). If there are more requests than the threads are able to handle, some requests will have to wait too long in the request queue, and will fail due to a timeout. But the application will still be able to handle the rest of the incoming requests.
Fortunately the Java API already provides a nice and flexible ThreadPool implementation, see ThreadPoolExecutor. Using this is probably even easier than implementing everything with your original approach, so no reason not to use it.
Thread.join() lets you wait for the Thread to end, which is mostly contrary to what you want when starting a new Thread. At all, you start the new thread to do stuff in parallel to the original Thread.
Only if you really need to wait for the spawned thread to finish, you should join() it.
You should wait for your threads if you need their results or need to do some cleanup which is only possible after all of them are dead, otherwise not.
For the Thread-Pool: I would use it whenever you have some non-fixed number of tasks to run, i.e. if the number depends on the input.
I would like to collect the main ideas of this interesting (for me) question.
I can't totally agree with "you
don't need to wait for threads".
Only in the sense that if you don't
join a thread (and don't have a
pointer to it) once the thread is
done, its resources are freed
(right? I'm not sure).
The use of a thread pool is only
necessary to avoid the overhead of
thread creation, because ...
You can limit the number of parallel
running threads by accounting, with shared variables (and without a thread pool), how many of then
were started but not yet finished.
I'm using a java.util.concurrent.ExecutorService that I obtained by calling Executors.newSingleThreadExecutor(). This ExecutorService can sometimes stop processing tasks, even though it has not been shutdown and continues to accept new tasks without throwing exceptions. Eventually, it builds up enough of a queue that my app shuts down with OutOfMemoryError exceptions.
The documentation seem to indicate that this single thread executor should survive task processing errors by firing up a new worker thread if necessary to replace one that has died. Am I missing something?
It sounds like you have two different issues:
1) You're over-feeding the work queue. You can't just keep stuffing new tasks into the queue, with no regard for the consumption rate of the task executors. You need to figure out some logic for knowing when you to block new additions to the work queue.
2) Any uncaught exception in a task's thread can completely kill the thread. When that happens, the ExecutorService spins up a new thread to replace it. But that doesn't mean you can ignore whatever problem is causing the thread to die in the first place! Find those uncaught exceptions and catch them!
This is just a hunch (cuz there's not enough info in your post to know otherwise), but I don't think your problem is that the task executor stops processing tasks. My guess is that it just doesn't process tasks as fast as you're creating them. (And the fact that your tasks sometimes die prematurely is probably orthogonal to the problem.)
At least, that's been my experience working with thread pools and task executors.
Okay, here's another possibility that sounds feasible based on your comment (that everything will run smoothly for hours until suddenly coming to a crashing halt)...
You might have a rare deadlock between your task threads. Most of the time, you get lucky, and the deadlock doesn't manifest itself. But occasionally, two or more of your task threads get into a state where they're waiting for the release of a lock held by the other thread. At that point, no more task processing can take place, and your work queue will pile up until you get the OutOfMemoryError.
Here's how I'd diagnose that problem:
Eliminate ALL shared state between your task threads. At first, this might require each task thread making a defensive copy of all shared data structures it requires. Once you've done that, it should be completely impossible to experience a deadlock.
At this point, gradually reintroduced the shared data structures, one at a time (with appropriate synchronization). Re-run your application after each tiny modification to test for the deadlock. When you get that crashing situation again, take a close look at the access patterns for the shared resource and determine whether you really need to share it.
As for me, whenever I write code that processes parallel tasks with thread pools and executors, I always try to eliminate ALL shared state between those tasks. As far as the application is concerned, they may as well be completely autonomous applications. Hunting down deadlocks is a drag, and in my experience, the best way to eliminate deadlocks is for each thread to have its own local state rather than sharing any state with other task threads.
Good luck!
My guess would be that your tasks are blocking indefinitely, rather than dying. Do you have evidence, such as a log statement at the end of your task, suggest that your tasks are successfully completing?
This could be a deadlock, or an interaction with some external process that is blocking.
Although you don't leave enough detail to be sure, the first thing I'd try is to have your tasks catch "Exception" at the top level and log the message.
I know it doesn't seem right, but occasionally (depending on a lot of variables) I've worked on code where stuff happening in a thread throws an exception and it is never logged, or it just doesn't show up on the console--yet the "executing" code exits out of it's top level loop or whatever code is causing your task to run.
I guess I'm just saying, make sure your tasks are not throwing an exception out.