I have a spring #Controller that has a request mapping.
When requests are sent to it it creates a deferred result and runs a method on a delegator class.
In the controller:
#ResponseBody
#RequestMapping(MenuModelCodes.MENU_NAME)
DeferredResult<HttpEntity<?>> getModelCodesWithClass(String target) {
final DeferredResult<HttpEntity<?>> deferredResult = getNewDeferredResult();
delegator.doStuff(target);
return deferredResult;
}
The doStuff method in the delegator is #Async annotated - so it's running in another thread.
In this thread a new Phaser is created to keep track of child threads it will create.
The delegator thread itself registers with the phaser, and triggers a call to a method in "AnotherClass" that is also #Async annotated. I.e. the delegator is now spawning children.
The delegator:
public class Delegator {
#Async
public Object doStuff(String requestURI) {
Phaser phaser = new Phaser();
phaser.register();
Object result = anotherClass.createThreadDoWork(phaser);
phaser.arriveAndDeregister();
//Wait until phaser is completed
while (!phaser.isTerminated()) {}
return result;
}
}
Each child spawned thread created by a call in the delegator to createThreadDoWork here registers itself with the phaser, does some work and then arrives and deregisters.
AnotherClass:
public class AnotherClass {
#Async
public void createThreadDoWork(Phaser phaser) throws Throwable {
phaser.register();
//some kind of logic / rest call etc.
phaser.arriveAndDeregister();
}
}
When all child threads complete, the delegator which is sitting at this piece of code:
while (!phaser.isTerminated()) {}
Will continue, set its deferred result and the delegator thread will end.
So here's the question(s):
I want a Phaser for every request - i.e. I create a new Phaser() everytime a request creates a new delegator thread.
If I receive 5 requests to my controller, they each spawn a delegator thread, each delegator creates a new Phaser and passes it to their children.
However, as soon as one of the threads completes its processing and its specific instance of the Phaser is terminated, the other delegator threads continue and return without having done their tasks.
Can Phasers be used like this? Am I doing something wrong? I thought this would be a useful way to track the completion of child threads and wait for them to complete?
Thanks.
Can Phasers be used like this?
Yes, it can be used like this, and usually would be as a synchronization barrier.
Am I doing something wrong?
Yes. Do not do
while (!phaser.isTerminated()) {}
The busy spinning will kill you. Instead do:
phaser.awaitAdvance();
This would force you to register a little differently. You would have to register a thread before you actually start running.
Also if you are in advancing the phaser where you have a multiple phases then you would have to re-evaluate using a phaser.
I thought this would be a useful way to track the completion of child
threads and wait for them to complete?
If you do have a number of unknown threads being created it would be useful to use a Phaser.
Related
I have a case to be implemented in my project.Below is a sample rest service which has to be implemented
#GET
#Path("/test/{id}")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public String getData(#PathParam("id") String id) {
//Some processing to get value of String
String result = doSomeProcessing();
//I want to return this result to GUI and call one more rest api
// and end this process without waiting for response from second
//call
new Thread(){
//call second rest api
}.start();
return result;
}
Is this good approach using new Thread to call second rest API and return result without waiting for response from second rest API ?
I have also looked into Asynchronous Rest call, but it doesn't exactly suit my requirement. Please advice. Thanks in Advance
Avoid starting Threads directly. Consider an ExecutorService instead as shown below:
#Singleton
#Path("foo")
public class FooResource {
private ExecutorService executor;
#PostConstruct
public void onCreate() {
// Creates a thread pool that reuses a fixed number
// of threads operating off a shared unbounded queue
this.executor = Executors.newFixedThreadPool​(10);
}
#GET
public Response getFoo() {
String result = doSomeProcessing();
// Submits a Runnable task for execution
executor.submit(new LongRunningTask());
return Response.ok(result).build();
}
#PreDestroy
public void onDestroy() {
// Initiates an orderly shutdown in which previously submitted
// tasks are executed, but no new tasks will be accepted.
this.executor.shutdownNow();
}
}
public class LongRunningTask implements Runnable {
#Override
public void run() {
try {
// Simulate a long running task
// Don't do it in a real application
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Explore the Executors API for details on how to create an ExecutorService instance.
In Java SE and in a Servlet container, you can use an ExecutorService for your long running task. In a Java EE container, you should use a ManagedExecutorService instead:
#Resource
ManagedExecutorService executor;
Once it's a resource managed by the container, you don't need to instantiate and dispose it manually.
I not sure what do you exactly mean by "calling the second REST API" so I assume that you're sending an HTTP request to another external API, i.e "second REST API".
You can use Apache HC to send the request and skip waiting for the response. See its fluent API which is easy to use. Async.execute is the one that ought to be used in your case. The Async class uses a thread-pool underneath to handle background requests.
I should mention that I haven't used any other HTTP client libraries. There might be other choices out there with almost the same functionality.
Side Note I strongly suggest ExecutorService, esp. ThreadPoolExecutor instead of creating new threads. It has more control over life-cycle of threads and manages system resource efficiently. ExecutorService has methods for fire and forget scenarios (submit). However, this only makes sense when your "second REST API" call is actually implemented in another method in your application and lives within the same JRE.
I'd say yes and no; there is nothing "wrong" about calling another service asynchronously (ex: a metrics subsystem to register that a call to that service was made). However, I wouldn't create threads straight in the getData method, I would use another (injected) object to hide the implementation detail (ex: just calling some sort of myOtherService.submit(something). That way you can easily switch from something synchronous, to asnychronous and even change the async method to your liking.
As to the actual implementation, instead of creating threads, I would submit callables to an executor service instead so you save the overhead of creating actual system threads, or you could even have one or more threads waiting on a blocking queue and just put jobs in that queue for threads to pick them up.
I'm creating a REST service using Spring with Jersey and I have a use case where for every request I get, I need to make several calls (N) to an upstream API.
I get one request, it has n items, for each item, I create a thread to call my dependency (REST) and process the response. At the end I collect all the responses together, maintaining the order, and return them as a single response to the client.
I am using Java 8's CompletableFuture and wanted to know if I was using the ExecutorService framework correctly.
#Component // automatic singleton by Spring
class A {
private ExecutorService executorService = Executors.newCachedThreadPool();
private RawResponse getRawResponse(Item item) {
// make REST call
}
private Response processResponse(RawResponse rawResponse) {
// process response
}
public List<Response> handleRequest(Request request) {
List<CompletableFuture> futureResponses = new ArrayList<>();
for(Item item : request.getItems()) {
CompletableFuture<Response> futureResponse = CompletableFuture.supplyAsync(() -> getRawResponse(item), executorService)
.thenApply(rawResponse -> processResponse(rawResponse))
.handle((response, throwable) {
if(throwable != null) { // log and return default response
} else { return response;}});
futureResponses.add(futureResponse);
}
List<Response> result = new ArrayList<>();
for (CompletableFuture<Resource> futureResponse : futureResponses) {
try {
result.add(futureResponse.get());
} catch (Exception e) {
// log error
}
}
return result;
}
}
The question I have now is, should I move the creation of the executorService right above:
List<CompletableFuture> futureResponses = new ArrayList<>();
and call shutdown on it right above:
return result;
because at this time, I am not really calling shutdown anywhere since the app will always run in it's docker container.
Is it costly to keep creating and discarding the pool, or is the current way going to leak memory? And I think calling the pool static as a private field var is redundant since the class is a spring bean anyways (singleton).
Any advice will be appreciated, also should I be using a cachedThreadPool? I am not sure how to approximate the number of threads I need.
should I move the creation of the executorService right above?
No you don't, you have your ExecutorService the right place in your example code. Think it as a thread pool, you will not want to init a new thread pool and close it for each method call of handleRequest. Of course ExecutorService does more job than a thread pool, actually it'll manage a thread pool underneath, and provides life-circle management for async tasks.
I am not really calling shutdown anywhere since the app will always run in it's docker container.
In most of cases you init your ExecutorService when applications starts and shut it down when applications shuts down. So you can just leave it there, because it'll be closed when application exits, or you can add some kind of shutdown hooks if you need to do graceful shutdown.
Is it costly to keep creating and discarding the pool.
Kind of, we don't want to create and discard Thread very often, so we have thread pool, if you create/discard pool for each method call, what's the point to have a thread pool.
or is the current way going to leak memory?
No, as long as the task you submitted does not leak memory. The implementation of ExecutorService itself is good to use.
And I think calling the pool static as a private field var is redundant since the class is a spring bean anyways (singleton)
Yes, you're correct. You can also define ExecutorService as Spring Bean and inject it to service bean, if you want to do some customized init process.
should I be using a cachedThreadPool, I am not sure how to approximate the number of threads I need.
That's hard to say, you need to do some test to get the right number of threads for your application. But most of NIO or EventDriven framework has the twice number of available cores to be the number of threads by default.
As you are using Spring, you might want to let it handle the asynchronous execution instead.
Just put #EnableAsync in one of your #Configuration classes to enable the #Async annotation on methods.
You would then change your getRawResponse() to
#Async
private CompletableFuture<RawResponse> getRawResponse(Item item) {
// make REST call
return CompletableFuture.completedFuture(rawResponse);
}
(you might need to put this method in a separate service to allow proper proxying, depending on how AOP is configured in your project)
and change your loop to simply
for(Item item : request.getItems()) {
CompletableFuture<Response> futureResponse = getRawResponse(item)
.thenApply(rawResponse -> processResponse(rawResponse))
.handle((response, throwable) {
if(throwable != null) { // log and return default response
} else { return response;}
});
futureResponses.add(futureResponse);
}
As you can see, you do not need to care about the executor in your service anymore.
You can also customize your executor by declaring its Spring bean, for example:
#SpringBootApplication
#EnableAsync
public class Application extends AsyncConfigurerSupport {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(2);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("GithubLookup-");
executor.initialize();
return executor;
}
}
You can even configure several executors and select one by providing its name as parameter to the #Async annotation.
See also Getting Started: Creating Async Methods and The #Async annotation.
we experience the following problem: We have got some service bean that does some handling and will iterate over a set and start an async thread for each entry, which will look like following:
#Service
public class ServiceBean {
#Autowired
private AsyncHandler asyncHandler;
public void doService(Set<?> theSet) {
for (Object obj : theSet) {
Future<?> future = asyncHandler.doHandle(obj);
// ...
}
// wait for all futures to complete
}
}
#Service
public class AsyncHandler {
#Async
public Future<?> doHandle(Object object) {
// dosth
return future;
}
}
Now we see that each thread will hold its own transaction and thereby database connection, which will cause the connection pool to run empty very fast.
What is the desired way to share the transaction accross all threads?
Thanks in advance!
What is the desired way to share the transaction accross all threads?
Don't do that. That makes it incredibly hard to understand what is going on. There's a reason why a transaction is attached to the thread that has created it. I guess your plan is to start all those asynchronous threads and then wait for them to complete to send the commit signal.
If you don't, who's going to complete the transaction? If a thread rollbacks the transaction, what happens in other threads that are still using it? You could pass the Connection around obviously but I'd expect things to be serialized anyway as most connection implementation are using synchronized in various places.
I have a pipeline of tasks (each task in the pipeline has different parallelism requirements), each task works in a different ExecutorService. Tasks work on packets of data, so if we have 10 datapackets then 10 tasks will be submitted to service1, one task per data packet. Once a task submitted to service1 has actually invoked it may submit a new task to work further on the datapacket to service2, service3 or not.
The following code works fine, i.e.:
shutdown() is invoked on service1 after everything has been submitted to service1
Then awaitTermination() does not return until all the tasks that were submitted before the shutdown() have actually completed running.
-- shutdown() is then invoked on service2 but because all tasks submitted to service1 have completed, and all tasks are submitted to service2 from tasks on service1 all tasks have been submitted to service2 before shutdown() is called on service2.
-- and so on for service3
ExecutorService[] services = {
service1,
service2,
service3};
int count = 0;
for(ExecutorService service: services)
{
service.shutdown();
service.awaitTermination(1, TimeUnit.HOURS);
}
However I have now added a case whereby service2 can break a datapacket into a smaller packet and submit additional tasks on service2 and the code is now failing. The problem is that shutdown() is called on service2 once all the tasks on service1 have completed, but now we want to submit additional service2 tasks from a task running in service2
My questions:
Does shutdown() rerun after all submitted tasks have finished running, or does it return immediately but just doesn't stop already submitted tasks from running ? Update:answered below
How do I solve my new problem ?
"shutdown" simply tells the pool not to accept any more work. It does nothing more. All existing submitted work will be executed as normal. When the queue is drained, the pool will actually destroy all it's threads and terminate.
The problem here is that you're saying that tasks in service2 will submit additional tasks to service2 for processing. There seems to be no way to know when you should actually call a shutdown. But alas, there is an alternative, assuming these smaller packets don't break down further into service.
List<Future<Void>> service2Futures = new ArrayList<Future<Void>>();
service2Futures.add(service2.submit(new Callable<Void>() {
public Void call() throws Exception {
// do your work, submit more stuff to service2
// if you submit Callables, you could use Future.get() to wait on those
// results.
return null;
}
}));
for (Future<Void> future : service2Futures) {
future.get();
}
service2.shutdown();
...
What's going on here is that you're storing Future objects for the top level submitted tasks (you'll have to use Callable and not Runnable). Instead of immediately shutting the pool down after submission, you simply collect up the Future objects. You then wait until they are all done running by cycling through them, and calling get() on each one. The "get()" method blocks until the thread running that task has completed.
At that point, all of the top level tasks are complete, and they will have submitted second level tasks. You can now issue a shutdown. This assumes the second level tasks don't submit more stuff to service2.
This all being said, if you're using java 7, you should consider taking a look at ForkJoinPool and RecursiveTask instead. It probably makes more sense for what you're doing.
ForkJoinPool forkJoinPool = new ForkJoinPool();
RecursiveAction action = new RecursiveAction() {
protected void compute() {
// break down here and build actions
RecursiveAction smallerActions[] = ...;
invokeAll(smallerActions);
}
};
Future<Void> future = forkJoinPool.submit(action);
ExecutorService#shutdown lets already submitted tasks finish whatever they are doing - javadoc extract:
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down.
This method does not wait for previously submitted tasks to complete execution. Use awaitTermination to do that.
In practice, you can consider that a call to shutdown does several things:
the ExecutorService can't accept new jobs any more
existing threads are terminated once they have finished running
So to answer your questions:
if you have submitted all your tasks to service1 before you call service1.shutdown (if you submit anything after that call you will get an exception anyway), you are fine (i.e. if those tasks submit something to service2 and service2 is not shutdown, they will be executed).
shutdown returns immediately and does not guarantee that already submitted tasks will stop (they could run forever).
The problem you are having is probably linked to how you submit your tasks from one service to another and it seems difficult to solve it with only the information you have given.
The best way would be to include a SSCCE in your question that replicates the behaviour you are seeing.
Instead of shutting down the ExecutorService, you should track the tasks themselves. they can pass around a "job state" object in which they use to keep track of outstanding work, e.g.:
public class JobState {
private int _runningJobs;
public synchronized void start() {
++_runningJobs;
}
public synchronized void finish() {
--_runningJobs;
if(_runningJobs == 0) { notifyAll(); }
}
public synchronized void awaitTermination() {
while(_runningJobs > 0) { wait() }
}
}
public class SomeJob implements Runnable {
private final JobState _jobState;
public void run() {
try {
// ... do work here, possibly submitting new jobs, and pass along _jobState
} finally {
_jobState.finish();
}
}
}
// utility method to start a new job
public static void submitJob(ExecutorService executor, Runnable runnable, JobState jobState) {
// call start _before_ submitting
jobState.start();
executor.submit(runnable);
}
// start main work
JobState state = new JobState();
Runnable firstJob = new SomeJob(state);
submitJob(executor, firstJob, state);
state.awaitTermination();
When you call shutdown it does not wait all task will be finished. Do it as you do with awaitTermination.
But once shutdown was called - new task are blocked. Your executor service reject all new task. For ThreadPoolExecutor rejected task handle in RejectedExecutionHandler. If you specify you custom handler you can process rejected after shutdown task. This is one of workarounds.
Matts question looks like it may well work but Im concerned it may cause new issues.
Ive come up with a solution which works without many code changes for my scenario, although it seems a bit clunky
Ive introduced a new service (service2a) that runs the same task as service2. When a task in service2 wants to submit a small data packet it submits it to service2a rather than service2 so all sub packets are submitted to service2a before service 2 shutdowns. This works for me as the smaller data packets dont need to be broken down into further subpackets and the subpackets idea only applies to service2(a) not any of the other services.
I am using the Executor framework to kick off several threads using a threadpool i.e newFixedThreadPool. I use threadpool.submit(aThread) to submit jobs to be executed by the threadpool and this works fine however I need to determine when all the threads are complete so that I can continue with other processing. I looked at using Future.get() which blocks until the thread is complete the problem here being that it blocks until a result is available. I also looked at using continuously calling the isTerminated() method followed by a sleep after issuing the shutdown to check if all threads are complete but this doesn't seem neat to me. Is there another cleaner way to this? Also if there is an exception raised in any one of the threads I want to be able to terminate all other running threads and also stop any queued up thread in the pool from starting. What is the best mechanism to do this?
Look forward to hearing your replies
TIA
Use ExecutorService#shutdown() and then ExecutorService#awaitTermination()
For example:
ExecutorService service = Executors.newCachedThreadPool();
service.submit(...);
service.submit(...);
service.shutdown();
service.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
// All tasks have now finished
As far as notifying you when a task fails with an exception. You'll have to provide a ThreadFactory to the ExecutorService that sets an "uncaught Exception handler" for each thread it creates. This Exception handler can then terminate the running tasks.
One of the cleaner ways to approach this is by modifying the tasks that are submitted. By registering a callback with each task, it can notify of normal completion or an exception without any polling by the main thread.
You can write a simple wrapper that will do this for any Runnable.
Or, following that example, you can extend the idea to wrap any Callable.
class CallbackTask<T>
implements Callable<T>
{
private final Callable<? extends T> task;
private final Callback<T> callback;
CallbackTask(Callable<? extends T> task, Callback<T> callback)
{
this.task = task;
this.callback = callback;
}
public T call()
throws Exception
{
try {
T result = task.call();
callback.complete(result);
return result;
}
catch (Exception ex) {
callback.failed(ex);
throw ex;
}
}
}