I'm creating a REST service using Spring with Jersey and I have a use case where for every request I get, I need to make several calls (N) to an upstream API.
I get one request, it has n items, for each item, I create a thread to call my dependency (REST) and process the response. At the end I collect all the responses together, maintaining the order, and return them as a single response to the client.
I am using Java 8's CompletableFuture and wanted to know if I was using the ExecutorService framework correctly.
#Component // automatic singleton by Spring
class A {
private ExecutorService executorService = Executors.newCachedThreadPool();
private RawResponse getRawResponse(Item item) {
// make REST call
}
private Response processResponse(RawResponse rawResponse) {
// process response
}
public List<Response> handleRequest(Request request) {
List<CompletableFuture> futureResponses = new ArrayList<>();
for(Item item : request.getItems()) {
CompletableFuture<Response> futureResponse = CompletableFuture.supplyAsync(() -> getRawResponse(item), executorService)
.thenApply(rawResponse -> processResponse(rawResponse))
.handle((response, throwable) {
if(throwable != null) { // log and return default response
} else { return response;}});
futureResponses.add(futureResponse);
}
List<Response> result = new ArrayList<>();
for (CompletableFuture<Resource> futureResponse : futureResponses) {
try {
result.add(futureResponse.get());
} catch (Exception e) {
// log error
}
}
return result;
}
}
The question I have now is, should I move the creation of the executorService right above:
List<CompletableFuture> futureResponses = new ArrayList<>();
and call shutdown on it right above:
return result;
because at this time, I am not really calling shutdown anywhere since the app will always run in it's docker container.
Is it costly to keep creating and discarding the pool, or is the current way going to leak memory? And I think calling the pool static as a private field var is redundant since the class is a spring bean anyways (singleton).
Any advice will be appreciated, also should I be using a cachedThreadPool? I am not sure how to approximate the number of threads I need.
should I move the creation of the executorService right above?
No you don't, you have your ExecutorService the right place in your example code. Think it as a thread pool, you will not want to init a new thread pool and close it for each method call of handleRequest. Of course ExecutorService does more job than a thread pool, actually it'll manage a thread pool underneath, and provides life-circle management for async tasks.
I am not really calling shutdown anywhere since the app will always run in it's docker container.
In most of cases you init your ExecutorService when applications starts and shut it down when applications shuts down. So you can just leave it there, because it'll be closed when application exits, or you can add some kind of shutdown hooks if you need to do graceful shutdown.
Is it costly to keep creating and discarding the pool.
Kind of, we don't want to create and discard Thread very often, so we have thread pool, if you create/discard pool for each method call, what's the point to have a thread pool.
or is the current way going to leak memory?
No, as long as the task you submitted does not leak memory. The implementation of ExecutorService itself is good to use.
And I think calling the pool static as a private field var is redundant since the class is a spring bean anyways (singleton)
Yes, you're correct. You can also define ExecutorService as Spring Bean and inject it to service bean, if you want to do some customized init process.
should I be using a cachedThreadPool, I am not sure how to approximate the number of threads I need.
That's hard to say, you need to do some test to get the right number of threads for your application. But most of NIO or EventDriven framework has the twice number of available cores to be the number of threads by default.
As you are using Spring, you might want to let it handle the asynchronous execution instead.
Just put #EnableAsync in one of your #Configuration classes to enable the #Async annotation on methods.
You would then change your getRawResponse() to
#Async
private CompletableFuture<RawResponse> getRawResponse(Item item) {
// make REST call
return CompletableFuture.completedFuture(rawResponse);
}
(you might need to put this method in a separate service to allow proper proxying, depending on how AOP is configured in your project)
and change your loop to simply
for(Item item : request.getItems()) {
CompletableFuture<Response> futureResponse = getRawResponse(item)
.thenApply(rawResponse -> processResponse(rawResponse))
.handle((response, throwable) {
if(throwable != null) { // log and return default response
} else { return response;}
});
futureResponses.add(futureResponse);
}
As you can see, you do not need to care about the executor in your service anymore.
You can also customize your executor by declaring its Spring bean, for example:
#SpringBootApplication
#EnableAsync
public class Application extends AsyncConfigurerSupport {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(2);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("GithubLookup-");
executor.initialize();
return executor;
}
}
You can even configure several executors and select one by providing its name as parameter to the #Async annotation.
See also Getting Started: Creating Async Methods and The #Async annotation.
Related
I am writing a service where I want to expose an endpoint which will call another service and if the service call is successful then I want to send back the result to UI/ calling app.
In parallel before sending back the response, I want to execute/submit a task which should run in background and my call should not be dependent on success or failure of this task.
Before returning the response i want to do the-
executorService.execute(object);
This should not be a blocking call..
Any suggestion
Spring Async methods is the way to go here as was suggested in comments.
Some caveats:
Async methods can have different return types, its true that they can return CompletableFuture but this is in case if you called them from some background process and would like to wait/check for their execution status or perhaps, execute something else when the future is ready. In your case it seems that you want "fire-and-forget" behavior. So you should use void return type for your #Async annotated method.
Make sure that you place #EnableAsync. Under that hood it works in a way that it wraps the bean that has #Async methods with some sort of proxy, so the proxy is actually injected into your service. So #EnableAsync turns on this proxy generation mechanism. You can verify that this is the case usually in the debugger and checking the actual type of the injected reference object.
Consider customizing the the task executor to make sure that you're running the async methods with executor that matches your needs. For example, you won't probably want that every invocation of async method would spawn a new thread (and there is an executor that behaves like this). You can read about various executors here for example
Update
Code-wise you should do something like this:
public class MyAsyncHandler {
#Async
public void doAsyncJob(...) {
...
}
}
#Service
public class MyService {
#Autowired // or autowired constructor
private MyAsyncHandler asyncHandler;
public Result doMyMainJob(params) {
dao.saveInDB();
// do other synchronous stuff
Result res = prepareResult();
asyncHandler.doAsyncJob(); // this returns immediately
return res;
}
}
I have a case to be implemented in my project.Below is a sample rest service which has to be implemented
#GET
#Path("/test/{id}")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public String getData(#PathParam("id") String id) {
//Some processing to get value of String
String result = doSomeProcessing();
//I want to return this result to GUI and call one more rest api
// and end this process without waiting for response from second
//call
new Thread(){
//call second rest api
}.start();
return result;
}
Is this good approach using new Thread to call second rest API and return result without waiting for response from second rest API ?
I have also looked into Asynchronous Rest call, but it doesn't exactly suit my requirement. Please advice. Thanks in Advance
Avoid starting Threads directly. Consider an ExecutorService instead as shown below:
#Singleton
#Path("foo")
public class FooResource {
private ExecutorService executor;
#PostConstruct
public void onCreate() {
// Creates a thread pool that reuses a fixed number
// of threads operating off a shared unbounded queue
this.executor = Executors.newFixedThreadPool​(10);
}
#GET
public Response getFoo() {
String result = doSomeProcessing();
// Submits a Runnable task for execution
executor.submit(new LongRunningTask());
return Response.ok(result).build();
}
#PreDestroy
public void onDestroy() {
// Initiates an orderly shutdown in which previously submitted
// tasks are executed, but no new tasks will be accepted.
this.executor.shutdownNow();
}
}
public class LongRunningTask implements Runnable {
#Override
public void run() {
try {
// Simulate a long running task
// Don't do it in a real application
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Explore the Executors API for details on how to create an ExecutorService instance.
In Java SE and in a Servlet container, you can use an ExecutorService for your long running task. In a Java EE container, you should use a ManagedExecutorService instead:
#Resource
ManagedExecutorService executor;
Once it's a resource managed by the container, you don't need to instantiate and dispose it manually.
I not sure what do you exactly mean by "calling the second REST API" so I assume that you're sending an HTTP request to another external API, i.e "second REST API".
You can use Apache HC to send the request and skip waiting for the response. See its fluent API which is easy to use. Async.execute is the one that ought to be used in your case. The Async class uses a thread-pool underneath to handle background requests.
I should mention that I haven't used any other HTTP client libraries. There might be other choices out there with almost the same functionality.
Side Note I strongly suggest ExecutorService, esp. ThreadPoolExecutor instead of creating new threads. It has more control over life-cycle of threads and manages system resource efficiently. ExecutorService has methods for fire and forget scenarios (submit). However, this only makes sense when your "second REST API" call is actually implemented in another method in your application and lives within the same JRE.
I'd say yes and no; there is nothing "wrong" about calling another service asynchronously (ex: a metrics subsystem to register that a call to that service was made). However, I wouldn't create threads straight in the getData method, I would use another (injected) object to hide the implementation detail (ex: just calling some sort of myOtherService.submit(something). That way you can easily switch from something synchronous, to asnychronous and even change the async method to your liking.
As to the actual implementation, instead of creating threads, I would submit callables to an executor service instead so you save the overhead of creating actual system threads, or you could even have one or more threads waiting on a blocking queue and just put jobs in that queue for threads to pick them up.
I am using CXF to call web service. It is used in a simple way like it is described in the documentation:
HelloService service = new HelloService();
Hello client = service.getHelloHttpPort();
String result = client.sayHi("Joe");
How can I terminate this service call when it takes time?
I found only one related question but this doesn't provide any solution.
How to terminate CXF webservice call within Callable upon Future cancellation
I think this is more of a function of the web server. For example, if you use Jetty to serve your CXF content, then you can set the thread pool to something that'll watch your threads.
ThreadPoolExecutor pool = new ThreadPoolExecutor(...);
ExecutorService svc = new ControlledExecutorService(pool);
server.setThreadPool(new org.eclipse.jetty.util.thread.ExecutorThreadPool(svc));
Then for the custom executor service (sorry, all code typed in the browser directly. I'm on an iPad with no Java. So you'll likely need to make slight adjustments, but the useful parts should be here):
public class ControlledExecutorService implements ExecutorService {
private ExecutorService es;
public ControlledExecutorService(ExecutorService wrapped) {
es = wrapped;
}
#Override
public void execute(final Runnable command) {
Future<Boolean> future = submit(new Callable< Boolean >() {
public Boolean call() throws Exception {
command.run();
return true;
}
});
// Do the proper monitoring of your Future and interrupt it
// using Future.cancel(true) if you need to.
}
}
Be sure to pass true to cancel() so that it sends the interrupt.
Also remember than just like with any thread, just because you send it an interrupt, it doesn't mean it'll comply. You have to do some work in your threads to make sure they're well behaved. Notably, periodically check Thread.currentThread().isInterrupted() and properly handling InterruptedException to pick it up and stop the task gracefully instead of just letting the exception blow everything up.
Is it necessary to shutdown an ExecutorService at some point if it runs in a Tomcat container for a Servlet. If yes, then where should I call the shutdown? I tried adding it after the submit() call but when I fire another request to the Servlet from a client browser, I get a RejectedExecutionException which is probably because I did a shutdown? I am trying to understand how it works in a Servlet within Tomcat and how I should use it.
I am doing the following in my webApplication (which seems to work fine without any shutdown):
// In configuration class
#Bean (name = "executorService")
public ExecutorService executorService() {
return Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() + 1);
}
// In some other class
#Qualifier("executorService")
#Autowired
private ExecutorService executorService;
....
private void load() {
executorService.submit(new Runnable() {
#Override
public void run() {
doSomethingInTheBackground();
}
});
// If I enable this I will get a RejectedExecutionException
// for a next request.
// executorService.shutdown();
}
The idea behind the ExecutorService is to reuse threads. Creating threads is expensive and usually it is more efficient to create a thread once and then use that same thread multiple times. This is exactly what an ExecutorService does: it manages a pool of (possibly idle) threads and assigns work to them when you call its submit methods.
In a typical application you therefore do not want to shutdown the ExecutorService. You should however shut the ExecutorService down properly if your application is terminated. Since you are using Spring you don't have to worry about that:
By default, beans defined using Java config that have a public close or shutdown method are automatically enlisted with a destruction callback. [See the documentation.]
That means, if you close the ApplicationContext, Spring will automatically shutdown the ExecutorService for you.
I am trying to implement a spring #Async task which has a return type of Future,
but I can't really figure out how to do it properly.
What will I gain by doing so ? will I now have a controll over my task so I can stop it and run it ?
Is there any reference implementation on how I can do it ? springsource does not provide any.
EDIT
From spring source and spring refrence manual :
Even methods that return a value can be invoked asynchronously. However, such methods are required
to have a Future typed return value. This still provides the benefit of asynchronous execution so that
the caller can perform other tasks prior to calling get() on that Future.
and the it gives an example like so :
#Async
Future<String> returnSomething(int i) {
// this will be executed asynchronously
}
How to implement this correctly ?
Check out this blog post.
Using #Async allows you to run a computation in a method asynchronously. This means that if it's called (on a Spring managed bean), the control is immediately returned to the caller and the code in the method is run in another thread. The caller receives a Future object that is bound to the running computation and can use it to check if the computation is running and/or wait for the result.
Creating such a method is simple. Annotate it with #Async and wrap the result in AsyncResult, as shown in the blog post.
Check out this blog post.
Important configuration is:
#Async on Spring managed bean method.
Enable async in Spring config XML by defining:
<!--
Enables the detection of #Async and #Scheduled annotations
on any Spring-managed object.
-->
<task:annotation-driven />
SimpleAsyncTaskExecutor will be used by default.
Wrap the response in a Future<> object.
Example
#Async
public Future<PublishAndReturnDocumentResult> generateDocument(FooBarBean bean) {
// do some logic
return new AsyncResult<PublishAndReturnDocumentResult>(result);
}
You can then check if the result is done using result.isDone() or wait to get the response result.get().
The ExecutorService can schedule Callable and return a Future object.
The Future is a placeholder that contains result once it's available. It allows you to check if result is there, cancel the task, or block and wait for the results. The Future is useful only when you are expecting some object/value from the task.
Correct way to do the Future call is:
Future<Integer> futureEvenNumber = executorService.submit(new NextEvenNumberFinder(10000));
// Do something.
try {
Integer nextEvenNumber = futureEvenNumber.get();
} catch (ExecutionException e) {
System.err.println("NextEvenNumberFinder threw exception: " + e.getCause());
}
The NextEvenNumberFinder class:
public class NextEvenNumberFinder implements Callable<Integer> {
private int number;
public NextEvenNumberFinder(int number) { this.number = number; }
#Override
public Integer call() throws Exception {
for (;;)
if (isEvenNumber(++number)) return number;
}
}
Spring Integration Reference Manual: http://static.springsource.org/spring-integration/reference/htmlsingle/