We have an application which processes items and on each iteration, starts a thread to do an update on an other database - it is not hugely important what happens on that other thread, it is a very straightforward update.
Our original intention was (by using a thread) to make sure the main processing is not held up by initalizing a connection to this other db and running the update.
Yesterday, we had an issue where (for a yet unknown reason) the database slowed down and the number of parallel threads went to the sky, resulting in 1000+ connections in this DB. So we realized we need more control over the threads.
I need a lib or tool for our software which can:
1) Put threads / jobs / tasks (anything - we can rewrite the code if required, we have Thread objects at the mintue) into a queue like system
2) we can define how many threads are running at most at the same time
3) After the thread finished, the thread is removed from the queue so GC can remove all the entities involved.
I was doing a fair bit of reading and i found ExecutorService (Executors.newFixedThreadPool(5);) but may problem is that it fails with 3) because according to the javadocs:
The threads in the pool will exist until it is explicitly shutdown.
Which, i believe, means that if the app keeps adding threads, the threads will exists until the application is restarted (or if i shutdown and reinstantiate the ExecutorService, but that seems to be a hack to me).
Am i right in thinking Executors.newFixedThreadPool(5) is failing at my 3) requirement?
Am i actually getting the problem from the good end? Do i need Threads or something different?
The sentence you are afraid of:
The threads in the pool will exist until it is explicitly shutdown
describes only calls to Executors.newFixedThreadPool(). To keep more control on the thread pool behavior, use ThreadPoolExecutor constructor expicitly, e.g
new ThreadPoolExecutor(1, //minimal Pool Size,
10, // maximal Pool Size,
30, TimeUnit.SECONDS // idle thread dies in 30 seconds
new ArrayBlockingQueue<Runnable>())
Here you need to understand the difference between Thread and the Runnable/Callable Task. so the meaning of The threads in the pool will exist until it is explicitly shutdown. is that at any point of time there will be 5 threads in the thread pool if you use Executors.newFixedThreadPool(5); . And the work that you want these threads to do would be submitted as Tasks (Runnable/Callable). So essentially at any point of time at max there will be 5 threads executing via this thread pool which in your case would be 5 connections.
The threads will stay there (waiting for other tasks to run), but it won't hold on to all the data that you put there. When the thread in the threadpool has executed the task, it will take a next task and won't reference the existing task anymore.
So your fears are baseless, unless you explicitly keep references to the tasks.
Use the ScheduledExecutorService with a fixed pool of threads for however many connections you need.
Have a BlockingQueue that you put requests in, the worker threads wait on the queue and process the requests as they appear.
The ExecutorService is the way to go. It provides you an interface (Future) to the state of the underlying thread, the ability to detect exceptions and return a value from the completed thread.
Here is a simple example of how to use ExecutorSerivce and the Future interface.
public class Updater implements Future< boolean > {
public Updater() { }
public boolean call() throws Exception {
System.out.println( "Hello, World!" );
return true;
}
}
public class Main {
public Main() { }
public static void main( String[] args ) {
ExecutorService pool = Executors.newFixedThreadPool( 1 );
boolean again = true;
do {
if ( again ) {
Future< ? > update = pool.submit( new Updater() );
}
/* Do other work while waiting for update to finish */
if( update.isDone() ) { //may be because of completion or an exception
try {
again = update.get(); // This would block if the Updater was still running
} catch( ExecutionException ee ) { // This is thrown by get() if an exception occurred in Updater.call()
again = false;
ee.printStackTrace();
}
}
} while ( true );
}
}
The example above will start an update to your database if the last update succeeded without an exception. This way you are controlling how many threads are trying to connect, and catching any errors that are causing the update to fail.
Related
Am building a spring boot rest api application deployed on weblogic 12c.
One of my requirement is to run some long running tasks on every incoming request.
An incoming rest request could result into multiple asynchronous task executions.
Since I dont care for the response and nor any exceptions that will result from these tasks I chose to use the ExecutorService and not Callable or CompletableFuture.
ExecutorService executorService =
Executors.newFixedThreadPool(2, new CustomizableThreadFactory("-abc-"));
Then for the incoming request that I receive in controller run two for loops and assign those tasks to the ExecutorService:
for (final String orderId : orderIds) {
for (final String itemId : itemIds) {
exec.execute(new Runnable() {
public void run() {
try {
//call database operation
}catch(Throwable t) {
logger.error("EXCEPTION with {} , {}" ,orderId,itemId
)
}
});
}//for
}//for
My question is regarding shutting down of the ExecutorService.
I am aware about graceful shutdown ( shutdown ) a hybrid shutdown ( awaitTermination ) or an abrupt shutdown ( shutdownNow )
what would be the preferred approach between the three for a rest api application ?
also is there any limit on how many thread pools can get created viz a viz as the number of ExecutorService thread pools getting created will be driven by the number of incoming requests
We currently have similar requirements, this is a difficult problem to solve as you want to use the right hammer if you will. There are very heavy weight solutions to orchestrating long running processes, for example SpringBatch.
Firstly though don't bother stop and starting the ExecutorService. The whole point of that class is to take the burden of Thread management off your hands, so you don't need to create and stop Threads yourself. So you don't need to manage the manager.
But be careful with your approach. Without using queues or another load balancing technique to smartly balance the long running processes across instances in your app. Or managing what happens when a Thread dies, you may get into a world of trouble. In general I would say nowadays it doesn't make much sense to interact directly with Threads or ThreadPools, and to use higher level solutions for this type of problem.
awaitTermination is usually a bit safer, while shutdownNow is more forceful. It's usually a good idea to use awaitTermination in a functional method, or even a runnable, if you would like the executor to shut down as soon as possible, but only after it has completed doing everything that it was created to do. In other words, when there are no active tasks that the executor is executing.
Ex.)
ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime.availableProcessors);
Observable.of(items).schedule(Schedulers.from(executor)).flatMap(item -> {
... // this block represents a task that the executor will execute in a worker thread
}).onSubscribe(onNext ->
logItem(onNext), throwable ->
throwable.printStackTrace(), /* onComplete */ () ->
executor.awaitTermination(60, TimeUnit.Seconds)
);
... // you need to shutdown asap because these other methods below are also doing some computation/io-intensive stuff
Now, when this method is finished, it will call awaitTermination, which will either close the pool immediately if it is not executing any tasks, or wait up to 60 seconds if tasks are still being executed.
Threads, or workers, will cease to be active for 60 seconds of inactivity in most cases, since that is usually the default.
On the other hand, if you want tasks to stop executing as soon as (to give some examples) an exception is thrown, there was a breach in security, or another module/service has failed, you might want to use shutdownNow() to stop all tasks immediately without the option of waiting.
My advice for choosing between the two would be to use shutdownNow in you catch block if you do not want tasks to continue to be executed if there is an exception - i.e., there is no longer a reason to return the list of items to the client given that one of the items did not get added to the list.
Otherwise, I'd recommend using awaitTermination after your try-catch, set to one minute, to safely shut down the thread pool as soon as it has executed all the tasks you have given it. But only do that if you know that the executor will not responsible for executing any more tasks down the line.
The simple shutdown, if that is an option for you, is also a good method. shutdown will reject all incoming tasks but wait until current tasks are finished executing, according to the Oracle docs.
If your not sure when you need to close the executor, it might be a good idea to use an #PreDestroy method so that the executor will just before the destroy method has been called on your bean:
#PreDestroy
private void cleanup(){
executor.shutdown();
}
I have an ExecutorService to execute my tasks concurrently. Most of these tasks are simple actions that require ~300ms to complete each. But a few of these tasks are background processing queues that take in new sub-tasks all the time and execute them in order. These background tasks will remain active as long as there are normal tasks running.
The ThreadPool is generated through one of the Executors' methods (don't know which yet) with a user-specified Thread count. My fear is that the following situation might happen: There are less threads than there are background queues. At a given moment, all background queues are working, blocking all the threads of the ExecutorService. No normal tasks will thus be started and the program hang forever.
Is there a possibility this might happen and how can I avoid it? I'm thinking of a possibility to interrupt the background tasks to leave the place to the normal ones.
The goal is to limit the number of threads in my application because Google said having a lot of threads is bad and having them idle for most of the time is bad too.
There are ~10000 tasks that are going to be submitted in a very short amount of time at the begin of the program execution. About ~50 background task queues are needed and most of the time will be spent waiting for a background job to do.
Don't mix up long running tasks with short running tasks in same ExecutorService.
Use two different ExecutorService instances with right pool size. Even if you set the size as 50 for background threads with long running tasks, performance of the pool is not optimal since number of available cores (2 core, 4 core, 8 core etc.) is not in that number.
I would like to create two separate ExecutorService initialized with Runtime.getRuntime().availableProcessors()/2;
Have a look at below posts for more details to effectively utilize available cores:
How to implement simple threading with a fixed number of worker threads
Dynamic Thread Pool
You can have an unlimited number of threads, check out cache thread pool
Creates a thread pool that creates new threads as needed, but will
reuse previously constructed threads when they are available. These
pools will typically improve the performance of programs that execute
many short-lived asynchronous tasks. Calls to execute will reuse
previously constructed threads if available. If no existing thread is
available, a new thread will be created and added to the pool. Threads
that have not been used for sixty seconds are terminated and removed
from the cache. Thus, a pool that remains idle for long enough will
not consume any resources. Note that pools with similar properties but
different details (for example, timeout parameters) may be created
using ThreadPoolExecutor constructors.
Another option is create two different pools and reserve one for priority tasks.
The solution is that the background tasks stop instead of being idle when there is no work and get restarted if there are enough tasks again.
public class BackgroundQueue implements Runnable {
private final ExecutorService service;
private final Queue<Runnable> tasks = new ConcurrentLinkedQueue<>();
private final AtomicBoolean running = new AtomicBoolean(false);
private Future<?> future;
public BackgroundQueue(ExecutorService service) {
this.service = Objects.requireNonNull(service);
// Create a Future that immediately returns null
FutureTask f = new FutureTask<>(() -> null);
f.run();
future = f;
}
public void awaitQueueTermination() throws InterruptedException, ExecutionException {
do {
future.get();
} while (!tasks.isEmpty() || running.get());
}
public synchronized void submit(Runnable task) {
tasks.add(task);
if (running.compareAndSet(false, true))
future = service.submit(this);
}
#Override
public void run() {
while (!running.compareAndSet(tasks.isEmpty(), false)) {
tasks.remove().run();
}
}
}
currently I am experimenting with Concurrency in Java/JavaFX. Printing must run in a different thread otherwise it will make the JavaFX main thread freeze for a couple seconds. Right now my printing is done with this simplified example.
public void print(PrintContent pt) {
setPrintContent(pt);
Thread thread = new Thread(this);
thread.start();
}
#Override
public void run() {
// send content to printer
}
With this code I am sending many print jobs parallel to my printer. Therefore I get the error telling me that my printer can only handle 1 print job at a time. Since I know that Threads cannot be reused, I would like to know if there is a possibility to queue up Threads, so that my printer only handles one print job at a time.
Thank you very much for your effort and your time.
Use a single threaded executor to execute the print jobs. It will create one (and only one) background thread and queue the jobs:
// it might be better not to make this static; but you need to ensure there is
// only one instance of this executor:
private static final Executor PRINT_QUEUE = Executors.newSingleThreadExecutor();
// ...
public void print(PrintContent pt) {
PRINT_QUEUE.execute(() -> {
// send content to printer
});
}
~~> WAY 1
You can implement your own BlockingQueue read this is very useful or use a default implementation from Java libraries tutorial
So after reading the above links,you add a method in your class like
public void addJob(Object job){
queue.put(job);
}
Secondly you implement a Thread that is running into an infinite while loop.Inside it you call the method
queue.take();
When the queue is empty this Thread is blocked waiting until a new Object is added,so you dont have to worry about spending cpu time.
Finally you can set some upper bounds so for example queue can contain .. 27 items.
Mention that in case of Thread failure you have to recreate it manually.
~~>WAY 2 Better Approach
You can use an Executors Interface:
ExecutorService executorService1 = Executors.newSingleThreadExecutor();
From documentation:
Creates an Executor that uses a single worker thread operating off an
unbounded queue. (Note however that if this single thread terminates
due to a failure during execution prior to shutdown, a new one will
take its place if needed to execute subsequent tasks.) Tasks are
guaranteed to execute sequentially, and no more than one task will be
active at any given time.
With the method below you retrieve a result if the job has successfully done.
Future future = executorService.submit(new Callable(){ public Object call() throws Exception { System.out.println("Asynchronous Callable"); return "Callable Result"; } });
System.out.println("future.get() = " + future.get());
If future.get() returns null, the job has been done successfully.
Remember to call
executorService.shutdown(); because the active threads inside this ExecutorService may prevent the JVM from shutting down.
Full tutorial here
I havea threadpool of 5, which I keep in array:
private static final Collection<Thread> workerThreads = new ArrayList<Thread>();
But when I reupload my osgi plugin, the threads keep running, but the array will be empty, so it will be populated with new 5 threads. So eventually I have tons of threads running.
My thread pool is designed to run forever, they just stay idle and wait for jobs to come into queue.
while (!queue.isEmpty()) {
try {
Job takenJob = queue.poll(5000, TimeUnit.MILLISECONDS);
if (takenJob != null) {
takenJob.execute();
}
} catch (InterruptedException e) {
log.error("ERROR", e);
}
}
So basically the problem is, that after I reupload my osgi project, I will lose reference to old threads.
Possible solution: I need to define a lifetime for threads, so I don't have a pool, but each thread will live ~15 minutes and then end. Meanwhile new threads a being created every 15 minutes so I will always have some thread looking at the queue.
Just using standard java.util.Date getTime() seems like not best way. Any suggestions how to implement this?
You must finish your threads and release every resources when the bundle is stopping. You can do it for example in the BundleActivator stop method.
In case you have new threads you should also be sure that the threads finish their job before the stop function returns. This means that if your jobs need to run for a long time before finish (e.g. due to an iteration) they should be designed in a way that they can be interrupted.
The best solution is to use an ExecutorService and close it when the bundle ends. The https://github.com/bndtools/bndtools-rt project contains an bundle that registers such an executor as a service, ensuring all life cycles issues are properly addressed.
I was trying to run ExecutorService object with FixedThreadPool and I ran into problems.
I expected the program to run in nanoseconds but it was hung. I found that I need to use Semaphore along with it so that the items in the queue do not get added up.
Is there any way I can come to know that all the threads of the pool are used.
Basic code ...
static ExecutorService pool = Executors.newFixedThreadPool(4);
static Semaphore permits = new Semaphore(4);
try {
permits.acquire();
pool.execute(p); // Assuming p is runnable on large number of objects
permits.release();
} catch ( InterruptedException ex ) {
}
This code gets hanged and I really don't know why. How to know if pool is currently waiting for all the threads to finish?
By default, if you submit more than 4 tasks to your pool then the extra tasks will be queued until a thread becomes available.
The blog you referenced in your comment uses the semaphore to limit the amount of work that can be queued at once, which won't be a problem for you until you have many thousands of tasks queued up and they start eating into the available memory. There's an easier way to do this, anyway - construct a ThreadPoolExecutor with a bounded queue.* But this isn't your problem.
If you want to know when a task completes, notice that ExecutorService.submit() returns a Future object which can be used to wait for the task's completion:
Future<?> f = pool.execute(p);
f.get();
System.out.println("task complete");
If you have several tasks and want to wait for all of them to complete, either store each Future in a list and then call get() on each in turn, or investigate ExecutorService.invokeAll() (which essentially does the same but in a single method call).
You can also tell whether a task has completed or not:
Future<?> f = pool.execute(p);
while(!f.isDone()) {
// do something else, task not complete
}
f.get();
Finally, note that even if your tasks are complete, your program may not exit (and thus appears to "hang") if you haven't called shutdown() on the thread pool; the reason is that the threads are still running, waiting to be given more work to do.
*Edit: sorry, I just re-read my answer and realised this part is incorrect - ThreadPoolExecutor offers tasks to the queue and rejects them if they aren't accepted, so a bounded queue has different semantics to the semaphore approach.
You do not need the Semaphore.
If you are hanging it is probably because the threads are locking themselves elsewhere.
Run the code in a Debuger and when it hangs pause it and see what the threads are doing.
You could change to using a ThreadPoolExecutor. It contains a getActiveCount() method which returns an approximate count of the active threads. Why it is approximate I'm not sure.