I have many I/O intensive jobs that are triggerred via Jersey webservice calls like this:
https://localhost/rest/execute/job/job1
I want to globally control the number of threads these jobs are using. How do I do this? I have thought of following two options, please suggest if I am in right direction or if there is a better solution.
Approach 1:
Create a wrapper class over ThreadPoolExecutor that provides threads to various services to submit runnables.
class GlobalThreadPool {
int corePoolSize = 1;
int maximumPoolSize = 4;
BlockingQueue<Runnable> q = BlockingQueue<Runnable>(10);
private ExecutorService pool = ThreadPoolExecutor(corePoolSize, maximumPoolSize, ..., q);
public Future<?> run (Runnable task) { pool.submit(task); }
}
Then use this class as ServletContextListener to start and shutdown with webservice.
(+) REST resource gets an instance of this class from ServletContext and submits the requested job.
(-)If a submitted task wants to use additional thread, it won't be able to do that as the run() method won't have access to service context.
Approach 2:
Create a Singleton of GlobalThreadPool. Then we Will not start it via web services listener. However, whenever a job needs it, it will instantiate the class and submit the runnable.
(+) any job has access to the pool irrespective of whether it has access to ServletContext
(-) shutting down the ExecutorService is not tied with webservice
Do you see any particular problem in any of these approaches that I might be missing? Is there any better (read standard) approach to do these things?
Related
I have a case to be implemented in my project.Below is a sample rest service which has to be implemented
#GET
#Path("/test/{id}")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public String getData(#PathParam("id") String id) {
//Some processing to get value of String
String result = doSomeProcessing();
//I want to return this result to GUI and call one more rest api
// and end this process without waiting for response from second
//call
new Thread(){
//call second rest api
}.start();
return result;
}
Is this good approach using new Thread to call second rest API and return result without waiting for response from second rest API ?
I have also looked into Asynchronous Rest call, but it doesn't exactly suit my requirement. Please advice. Thanks in Advance
Avoid starting Threads directly. Consider an ExecutorService instead as shown below:
#Singleton
#Path("foo")
public class FooResource {
private ExecutorService executor;
#PostConstruct
public void onCreate() {
// Creates a thread pool that reuses a fixed number
// of threads operating off a shared unbounded queue
this.executor = Executors.newFixedThreadPool​(10);
}
#GET
public Response getFoo() {
String result = doSomeProcessing();
// Submits a Runnable task for execution
executor.submit(new LongRunningTask());
return Response.ok(result).build();
}
#PreDestroy
public void onDestroy() {
// Initiates an orderly shutdown in which previously submitted
// tasks are executed, but no new tasks will be accepted.
this.executor.shutdownNow();
}
}
public class LongRunningTask implements Runnable {
#Override
public void run() {
try {
// Simulate a long running task
// Don't do it in a real application
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Explore the Executors API for details on how to create an ExecutorService instance.
In Java SE and in a Servlet container, you can use an ExecutorService for your long running task. In a Java EE container, you should use a ManagedExecutorService instead:
#Resource
ManagedExecutorService executor;
Once it's a resource managed by the container, you don't need to instantiate and dispose it manually.
I not sure what do you exactly mean by "calling the second REST API" so I assume that you're sending an HTTP request to another external API, i.e "second REST API".
You can use Apache HC to send the request and skip waiting for the response. See its fluent API which is easy to use. Async.execute is the one that ought to be used in your case. The Async class uses a thread-pool underneath to handle background requests.
I should mention that I haven't used any other HTTP client libraries. There might be other choices out there with almost the same functionality.
Side Note I strongly suggest ExecutorService, esp. ThreadPoolExecutor instead of creating new threads. It has more control over life-cycle of threads and manages system resource efficiently. ExecutorService has methods for fire and forget scenarios (submit). However, this only makes sense when your "second REST API" call is actually implemented in another method in your application and lives within the same JRE.
I'd say yes and no; there is nothing "wrong" about calling another service asynchronously (ex: a metrics subsystem to register that a call to that service was made). However, I wouldn't create threads straight in the getData method, I would use another (injected) object to hide the implementation detail (ex: just calling some sort of myOtherService.submit(something). That way you can easily switch from something synchronous, to asnychronous and even change the async method to your liking.
As to the actual implementation, instead of creating threads, I would submit callables to an executor service instead so you save the overhead of creating actual system threads, or you could even have one or more threads waiting on a blocking queue and just put jobs in that queue for threads to pick them up.
I have used Spring Framework's Scheduled to schedule my job to run at every 5 mins using cron. But sometime my job waits infinitely for an external resource and I can't put timeout there. I can't use fixedDelay as previous process sometime goes in wait infinitely mode and I have to refresh data at every 5 mins.
So I was looking any option in Spring Framework's Scheduled to stop that process/thread after a fixed-time either it run successfully or not.
I have found below setting which initialized ThreadPoolExecutor with 120 seconds for keepAliveTime which I put in #Configuration class. Can anybody tell me will this work as I expected.
#Bean(destroyMethod="shutdown")
public Executor taskExecutor() {
int coreThreads = 8;
int maxThreads = 20;
final ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(
coreThreads, maxThreads, 120L,
TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>()
);
threadPoolExecutor.allowCoreThreadTimeOut(true);
return threadPoolExecutor;
}
I'm not sure this will work as expected. Indeed the keepAlive is for IDLE thread and I don't know if your thread waiting for resources is in IDLE. Furthermore it's only when the number of threads is greater than the core so you can't really know when it happen unless you monitor the threadpool.
keepAliveTime - when the number of threads is greater than the core, this is the maximum time that excess idle threads will wait for new tasks before terminating.
What you can do is the following:
public class MyTask {
private final long timeout;
public MyTask(long timeout) {
this.timeout = timeout;
}
#Scheduled(cron = "")
public void cronTask() {
Future<Object> result = doSomething();
result.get(timeout, TimeUnit.MILLISECONDS);
}
#Async
Future<Object> doSomething() {
//what i should do
//get ressources etc...
}
}
Don't forget to add #EnableAsync
It's also possible to do the same without #Async by implementing a Callable.
Edit: Keep in mind that it will wait until timeout but the thread running the task won't be interrupted. You will need to call Future.cancel when TimeoutException occurs. And in the task check for isInterrupted() to stop the processing. If you are calling an api be sure that isInterrupted() is checked.
allowCoreThreadTimeOut and timeout setting doesn't help cause it just allow work thread to be ended after some time without work (See javadocs)
You say your job waits infinitely for an external resource. I'am sure it's because you (or some third-party library you using) use sockets with time out infinite-by-default.
Also keep in mind what jvm ignores Thread.interrupt() when it blocked on socket.connect/read.
So find out witch socket library used in your task (and how exactly it used) and change it's default timeout settings.
As example: there is RestTemplate widely used inside Spring (in rest client, in spring social, in spring security OAuth and so on). And there is ClientHttpRequestFactory implementation to create RestTemplate instances. By default, spring use SimpleClientHttpRequestFactory which use JDK sockets. And by default all it's timeouts are infinite.
So find out where exactly you freeze, read it's docs and configure it properly.
P.S. If you don't have enough time and "feeling lucky" try to run your app with setting jvm properties sun.net.client.defaultConnectTimeout and
sun.net.client.defaultReadTimeout to some reasonable values (See docs for more details)
The keepAliveTime is just for cleaning out worker threads that hasn't been needed for a while - it doesn't have any impact on the execution time of the tasks submitted to the executor.
If whatever is taking time respects interrupts you can start a new thread and join it with a timeout, interrupting it if it doesn't complete in time.
public class SomeService {
#Scheduled(fixedRate = 5 * 60 * 1000)
public void doSomething() throws InterruptedException {
Thread taskThread = new TaskThread();
taskThread.start();
taskThread.join(120 * 000);
if(taskThread.isAlive()) {
// We timed out
taskThread.interrupt();
}
}
private class TaskThread extends Thread {
public void run() {
// Do the actual work here
}
}
}
Is it necessary to shutdown an ExecutorService at some point if it runs in a Tomcat container for a Servlet. If yes, then where should I call the shutdown? I tried adding it after the submit() call but when I fire another request to the Servlet from a client browser, I get a RejectedExecutionException which is probably because I did a shutdown? I am trying to understand how it works in a Servlet within Tomcat and how I should use it.
I am doing the following in my webApplication (which seems to work fine without any shutdown):
// In configuration class
#Bean (name = "executorService")
public ExecutorService executorService() {
return Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() + 1);
}
// In some other class
#Qualifier("executorService")
#Autowired
private ExecutorService executorService;
....
private void load() {
executorService.submit(new Runnable() {
#Override
public void run() {
doSomethingInTheBackground();
}
});
// If I enable this I will get a RejectedExecutionException
// for a next request.
// executorService.shutdown();
}
The idea behind the ExecutorService is to reuse threads. Creating threads is expensive and usually it is more efficient to create a thread once and then use that same thread multiple times. This is exactly what an ExecutorService does: it manages a pool of (possibly idle) threads and assigns work to them when you call its submit methods.
In a typical application you therefore do not want to shutdown the ExecutorService. You should however shut the ExecutorService down properly if your application is terminated. Since you are using Spring you don't have to worry about that:
By default, beans defined using Java config that have a public close or shutdown method are automatically enlisted with a destruction callback. [See the documentation.]
That means, if you close the ApplicationContext, Spring will automatically shutdown the ExecutorService for you.
I have a spring #Controller that has a request mapping.
When requests are sent to it it creates a deferred result and runs a method on a delegator class.
In the controller:
#ResponseBody
#RequestMapping(MenuModelCodes.MENU_NAME)
DeferredResult<HttpEntity<?>> getModelCodesWithClass(String target) {
final DeferredResult<HttpEntity<?>> deferredResult = getNewDeferredResult();
delegator.doStuff(target);
return deferredResult;
}
The doStuff method in the delegator is #Async annotated - so it's running in another thread.
In this thread a new Phaser is created to keep track of child threads it will create.
The delegator thread itself registers with the phaser, and triggers a call to a method in "AnotherClass" that is also #Async annotated. I.e. the delegator is now spawning children.
The delegator:
public class Delegator {
#Async
public Object doStuff(String requestURI) {
Phaser phaser = new Phaser();
phaser.register();
Object result = anotherClass.createThreadDoWork(phaser);
phaser.arriveAndDeregister();
//Wait until phaser is completed
while (!phaser.isTerminated()) {}
return result;
}
}
Each child spawned thread created by a call in the delegator to createThreadDoWork here registers itself with the phaser, does some work and then arrives and deregisters.
AnotherClass:
public class AnotherClass {
#Async
public void createThreadDoWork(Phaser phaser) throws Throwable {
phaser.register();
//some kind of logic / rest call etc.
phaser.arriveAndDeregister();
}
}
When all child threads complete, the delegator which is sitting at this piece of code:
while (!phaser.isTerminated()) {}
Will continue, set its deferred result and the delegator thread will end.
So here's the question(s):
I want a Phaser for every request - i.e. I create a new Phaser() everytime a request creates a new delegator thread.
If I receive 5 requests to my controller, they each spawn a delegator thread, each delegator creates a new Phaser and passes it to their children.
However, as soon as one of the threads completes its processing and its specific instance of the Phaser is terminated, the other delegator threads continue and return without having done their tasks.
Can Phasers be used like this? Am I doing something wrong? I thought this would be a useful way to track the completion of child threads and wait for them to complete?
Thanks.
Can Phasers be used like this?
Yes, it can be used like this, and usually would be as a synchronization barrier.
Am I doing something wrong?
Yes. Do not do
while (!phaser.isTerminated()) {}
The busy spinning will kill you. Instead do:
phaser.awaitAdvance();
This would force you to register a little differently. You would have to register a thread before you actually start running.
Also if you are in advancing the phaser where you have a multiple phases then you would have to re-evaluate using a phaser.
I thought this would be a useful way to track the completion of child
threads and wait for them to complete?
If you do have a number of unknown threads being created it would be useful to use a Phaser.
We have a service implementation in our Spring-based web application that increments some statistics counters in the db. Since we don't want to mess up response time for the user we defined them asynchronous using Spring's #Async:
public interface ReportingService {
#Async
Future<Void> incrementLoginCounter(Long userid);
#Async
Future<Void> incrementReadCounter(Long userid, Long productId);
}
And the spring task configuration like this:
<task:annotation-driven executor="taskExecutor" />
<task:executor id="taskExecutor" pool-size="10" />
Now, having the pool-size="10", we have concurrency issues when two threads try two create the same initial record that will contain the counter.
Is it a good idea here to set the pool-size="1" to avoid those conflicts? Does this have any side affects? We have quite a few places that fire async operations to update statistics.
The side-effects would depend on the speed at which tasks are added to the executor in comparison to how quickly a single thread can process them. If the number of tasks being added per second is greater than the number that a single thread can process in a second you run the risk of the queue increasing in size over time until you finally get an out of memory error.
Check out the executor section at this page Task Execution. They state that having an unbounded queue is not a good idea.
If you know that you can process tasks faster than they will be added then you are probably safe. If not, you should add a queue capacity and handle the input thread blocking if the queue reaches this size.
Looking at the two examples you posted, instead of a constant stream of #Async calls, consider updating a JVM local variable upon client requests, and then have a background thread write that to the database every now and then. Along the lines of (mind the semi-pseudo-code):
class DefaultReportingService implements ReportingService {
ConcurrentMap<Long, AtomicLong> numLogins;
public void incrementLoginCounterForUser(Long userId) {
numLogins.get(userId).incrementAndGet();
}
#Scheduled(..)
void saveLoginCountersToDb() {
for (Map.Entry<Long, AtomicLong> entry : numLogins.entrySet()) {
AtomicLong counter = entry.getValue();
Long toBeSummedWithTheValueInDb = counter.getAndSet(0L);
// ...
}
}
}