I am using ThreadPoolExecutor to throttle call to my servlet, I referred to my below link:
http://www.softwareengineeringsolutions.com/blogs/2010/08/13/asynchronous-servlets-in-servlet-spec-3-0/
This gives great example. But need add delays. How do it here.
I am maintaining a pool of 7 threads, which run simultaneously. But new threads should creted from queue after all threads get complete and i need put before that.
so every 7 calls, i need a delay before more threads are pooled from queue and runned as task.
Will ScheduledThreadPoolExecutor help, if so how. Or how can modify the existing ThreadPoolExecutor will help.
if you just need to throttle servlet requests, asynchronous requests with a thread pool is not the way to solve the problem. the easiest way is to add a Filter which limits the incoming requests using something like a semaphore.
if you need to use asynchronous requests and want the throttle them, then just use a limited size thread pool.
Related
We need to do some asynchronous task processing where in around 30-40 requests will be coming at the same moment and each request will intiate a asynch task which will approximately take around 7-8 seconds to complete.
If java executorservice has been identified to do such task, what would be the idle type of executor for such purpose?
I thought of using CachedThreadPool but my worry is if too many threads are created would it have any performance impact on the application?
Another option would be to use FixedThreadPool but I am struggling to think of a idle no threads which it should be instantiated with...
What is the recommended Executor for such a scenario or how we go about finding the right one?
I think you are limiting your research to just the Executors.* factory methods. You should review what the range of constructors of ThreadPoolExecutor, you'll find a maximum thread pool size limit, among other things.
I thought of using CachedThreadPool but my worry is if too many
threads are created would it have any performance impact on the
application?
You need to test for the application for performance impact.
If none of them fits into the application or having some issues then you can use customized thread pool executor java.util.concurrent.ThreadPoolExecutor
You can customize according your needs with configuiring core pool size, configuring the blocking queue. Blocking queue will be used and task will be queued when pool size is reached.
When we talk about the processing of asynchronous events using an Executors service, why does creating a new fixed thread pool, involve the use of LinkedBlockingQueue ? The events which are arriving are not dependent at all, so why use a queue because the consumer thread would still involve the contention for take lock? Why doens't the Executors class have some hybrid data structure(such as a concurrent Map implementation) where there is no need for a take lock in most of the cases ?
There is very good reason what thread pool executor works with BlockingQueue (btw, you are not obliged to use LinkedBlockingQueue implementation, you can use different implementations of the BlockingQueue). The queue should be blocking in order to suspend worker threads when there are no tasks to execute. This blocking is done using wait on condition variables, so waiting worker threads do not consume any CPU resources when queue is empty.
If you use non-blocking queue in the thread pool, then how would worker threads poll for tasks to execute? They would have to implement some kind of polling, which is unnecessary wasting of CPU resources (it will be "busy waiting").
UPDATE:
Ok, now I fully understood the use case. Still you need blocking collection anyway. The reason is basically the same - since you implement Producer-Consumer you should have means for worker threads to wait for messages to arrive - and this you simply can't do without mutex + condition variable (or simply BlockingQueue).
Regarding map - yes, I understand how you want to use it, but unfortunately there is no such implementation provided. Recently I solved the similar problem: I needed to group incoming tasks by some criteria and execute tasks from each group serially. As a result I implemented my own GroupThreadPoolExecutor that does this grouping. The idea is simple: group incoming tasks into map and then add them to the executor queue when previous task from the group completes.
There is big discussion here - I think it's relevant to your question.
I'm working on a project where execution time is critical. In one of the algorithms I have, I need to save some data into a database.
What I did is call a method that does that. It fires a new thread every time it's called. I faced a runoutofmemory problem since the loaded threads are more than 20,000 ...
My question now is, I want to start only one thread, when the method is called, it adds the job into a queue and notifies the thread, it sleeps when no jobs are available and so on. Any design patterns available or examples available online ?
Run, do not walk to your friendly Javadocs and look up ExecutorService, especially Executors.newSingleThreadExecutor().
ExecutorService myXS = Executors.newSingleThreadExecutor();
// then, as needed...
myXS.submit(myRunnable);
And it will handle the rest.
Yes, you want a worker thread or thread pool pattern.
http://en.wikipedia.org/wiki/Thread_pool_pattern
See http://www.ibm.com/developerworks/library/j-jtp0730/index.html for Java examples
I believe the pattern you're looking for is called producer-consumer. In Java, you can use the blocking methods on a BlockingQueue to pass tasks from the producers (that create the jobs) to the consumer (the single worker thread). This will make the worker thread automatically sleep when no jobs are available in the queue, and wake up when one is added. The concurrent collections should also handle using multiple worker threads.
Are you looking for java.util.concurrent.Executor?
That said, if you have 20000 concurrent inserts into the database, using a thread pool will probably not save you: If the database can't keep up, the queue will get longer and longer, until you run out of memory again. Also, note that an executors queue is volatile, i.e. if the server crashes, the data in it will be gone.
When writing a multithread internet server in java, the main-thread starts new
ones to serve incoming requests in parallel.
Is any problem if the main-thread does not wait ( with .join()) for them?
(It is obviously absurd create a new thread and then, wait for it).
I know that, in a practical situation, you should (or "you must"?) implement a pool
of threads to "re-use" them for new requests when they become idle.
But for small applications, should we use a pool of threads?
You don't need to wait for threads.
They can either complete running on their own (if they've been spawned to perform one particular task), or run indefinitely (e.g. in a server-type environment).
They should handle interrupts and respond to shutdown requests, however. See this article on how to do this correctly.
If you need a set of threads I would use a pool and executor methods since they'll look after thread resource management for you. If you're writing a multi-threaded network server then I would investigating using (say) a servlet container or a framework such as Mina.
The only problem in your approach is that it does not scale well beyond a certain request rate. If the requests are coming in faster than your server is able to handle them, the number of threads will rise continuously. As each thread adds some overhead and uses CPU time, the time for handling each request will get longer, so the problem will get worse (because the number of threads rises even faster). Eventually no request will be able to get handled anymore because all of the CPU time is wasted with overhead. Probably your application will crash.
The alternative is to use a ThreadPool with a fixed upper bound of threads (which depends on the power of the hardware). If there are more requests than the threads are able to handle, some requests will have to wait too long in the request queue, and will fail due to a timeout. But the application will still be able to handle the rest of the incoming requests.
Fortunately the Java API already provides a nice and flexible ThreadPool implementation, see ThreadPoolExecutor. Using this is probably even easier than implementing everything with your original approach, so no reason not to use it.
Thread.join() lets you wait for the Thread to end, which is mostly contrary to what you want when starting a new Thread. At all, you start the new thread to do stuff in parallel to the original Thread.
Only if you really need to wait for the spawned thread to finish, you should join() it.
You should wait for your threads if you need their results or need to do some cleanup which is only possible after all of them are dead, otherwise not.
For the Thread-Pool: I would use it whenever you have some non-fixed number of tasks to run, i.e. if the number depends on the input.
I would like to collect the main ideas of this interesting (for me) question.
I can't totally agree with "you
don't need to wait for threads".
Only in the sense that if you don't
join a thread (and don't have a
pointer to it) once the thread is
done, its resources are freed
(right? I'm not sure).
The use of a thread pool is only
necessary to avoid the overhead of
thread creation, because ...
You can limit the number of parallel
running threads by accounting, with shared variables (and without a thread pool), how many of then
were started but not yet finished.
I created multiple ExecutorService instances in my code, usually each UI page has one ExecutorService instance. Each ExecutorService instance will execute some http get request threads.
private ExecutorService m_threadPool = Executors.newCachedThreadPool();
Is it OK to do that?
The problem I met is that sometimes the http get requests got response code -1 from HttpURLConnection getResponseCode() call. I don't know whether it is caused by multiple threadpool instances.
Thanks.
ExecutorService per se is just another object so there's no big overhead. But each thread pool comes with a number of idle threads by default and those are a cause of a major resource waste. I would suggest setting the default number of pre-generated threads in each pool small (1 or 0 if you are not sure whether any requests are sent) in order to reduce the cost of creating extra objects. Threads would be created on demand and you'll be able to keep your code clean.
Another solution is to use a single thread pool but to maintain a separate list of tasks for each UI window. In this case when window gets closed you'll have to iterate over all tasks and cancell the running ones manually (this can also be done in a separate thread). A task may be represented by a Future<?> (it has handy isDone() and cancel() methods).
It shouldn't be caused by your thread pool instances. However, I'd say that having more than one thread pool is questionable. Why would you need it? It could lead to a lot of unnecessary threads, and thereby unnecessary memory use.