This is a design question and I am asking for some ideas.
I have a rest method and it will trigger long-time tasks (10~15 minutes)
As the function takes long time, I run it as a thread,
this can avoid method timeout, but how can I know if the thread went wrong?
Runnable loader = new Runnable() {
public void run() {
//tasks
}
};
(new Thread(loader)).start();
Update: the rest service like this
#path()
beginload(){
//let thread run and return info first
//how can i know if this thread went wrong?
(new Thread(loader)).start();
return "need 15 minutes";
}
Conceptually there has to be a way for the service to communicate a failure to the client. There are multiple ways you can do this. Here are three examples:
After the client calls the service, the service immediately returns a job ID. The client can use the job ID later to query the service for the status (including error). For example, when you launch instances at AWS EC2, it takes a while for EC2 to service the request, so the launch request returns a so-called "reservation ID" that you can use in subsequent operations (like querying for status, terminating the launch, etc.).
Pro: Usable in a wide variety of cases, and easy enough to implement.
Con: Requires polling. (I.e. more chatty.)
The client offers a callback URI that the service invokes upon job completion. The callback URI can either be configured into the service, or else passed along as a request parameter. (Don't hardcode the callback URI in the service since services shouldn't depend on their clients.)
Pro: Still pretty simple, and avoids polling.
Con: Client has to have URI for the service to call, which may not be convenient. (E.g. the client may be a desktop app rather than a service, firewall may prevent it, etc.)
The client pushes a notification into a message queue, and the client listens to that queue.
Pro: Avoids polling, and client doesn't need endpoints to call.
Con: More work to set up (requires messaging infrastructure).
There are other possibilities but those are typical approaches.
Do you need to differentiate between different requests? If several tasks to perform, you need an ID.
You can do something like the following:
private static final ExecutorService es = Executors.newFixedThreadPool(10);
private static final Map<Long, Future<Void>> map = new HashMap<>();
#GET
#Path("/submit")
public Response submitTask() {
long id = System.currentTimeMillis();
Future<Void> future = es.submit(new Callable<Void>() {
public Void call() throws Exception {
// long task
// you must throw exception for bad task
return null;
}
});
map.put(id, future);
return Response.ok(id, MediaType.TEXT_PLAIN).build();
}
#GET
#Path("/status/{id}")
public Response submitTask(#PathParam("id") long id) {
Future<Void> future = map.get(id);
if (future.isDone()) {
try {
future.get();
return Response.ok("Successful!", MediaType.TEXT_PLAIN).build();
} catch (InterruptedException | ExecutionException e) {
// log
return Response.ok("Bad task!", MediaType.TEXT_PLAIN).build();
}
}
return Response.ok("Wait a few seconds.", MediaType.TEXT_PLAIN).build();
}
This can give you an idea. Remember purge the map of old tasks.
If you want to get the return value of your thread and throw/catch possible exception, consider use Callable rather than Runnable, and it can be used along with ExecutorService which provide more functionality.
Callable : A task that returns a result and may throw an exception.
Implementors define a single method with no arguments called call.
public interface Callable<V> {
V call() throws Exception;
}
Related
I have a Spring Boot application that will call several microservice URLs using the GET method. These microservice URL endpoints are all implemented as #RestControllers. They don't return Flux or Mono.
I need my application to capture which URLs are not returning 2xx HTTP status.
I'm currently using the following code to do this:
List<String> failedServiceUrls = new ArrayList<>();
for (String serviceUrl : serviceUrls.getServiceUrls()) {
try {
ResponseEntity<String> response = rest.getForEntity(serviceUrl, String.class);
if (!response.getStatusCode().is2xxSuccessful()) {
failedServiceUrls.add(serviceUrl);
}
} catch (Exception e){
failedServiceUrls.add(serviceUrl);
}
}
// all checks are complete so send email with the failedServiceUrls.
mail.sendEmail("Service Check Complete", failedServiceUrls);
}
The problem is that each URL call is slow to respond and I have to wait for one URL call to complete prior to making the next one.
How can I change this to make the URLs calls be made concurrently? After all call have completed, I need to send an email with any URLs that have an error that should be collected in failedServiceUrls.
Update
I revised the above post to state that I just want the calls to be made concurrently. I don't care that rest.getForEntity call blocks.
Using the executor service in your code, you can call all microservices in parallel this way:
// synchronised it as per Maciej's comment:
failedServiceUrls = Collections.synchronizedList(failedServiceUrls);
ExecutorService executorService = Executors.newFixedThreadPool(serviceUrls.getServiceUrls().size());
List<Callable<String>> runnables = new ArrayList<>().stream().map(o -> new Callable<String>() {
#Override
public String call() throws Exception {
ResponseEntity<String> response = rest.getForEntity(serviceUrl, String.class);
// do something with the response
if (!response.getStatusCode().is2xxSuccessful()) {
failedServiceUrls.add(serviceUrl);
}
return response.getBody();
}
}).collect(toList());
List<Future<String>> result = executorService.invokeAll(runnables);
for(Future f : result) {
String resultFromService = f.get(); // blocker, it will wait until the execution is over
}
If you just want to make calls concurrently and you don't care about blocking threads you can:
wrap the blocking service call using Mono#fromCallable
transform serviceUrls.getServiceUrls() into a reactive stream using Flux#fromIterable
Concurrently call and filter failed services with Flux#filterWhen using Flux from 2. and asynchronous service call from 1.
Wait for all calls to complete using Flux#collectList and send email with invalid urls in subscribe
void sendFailedUrls() {
Flux.fromIterable(erviceUrls.getServiceUrls())
.filterWhen(url -> responseFailed(url))
.collectList()
.subscribe(failedURls -> mail.sendEmail("Service Check Complete", failedURls));
}
Mono<Boolean> responseFailed(String url) {
return Mono.fromCallable(() -> rest.getForEntity(url, String.class))
.map(response -> !response.getStatusCode().is2xxSuccessful())
.subscribeOn(Schedulers.boundedElastic());
}
Blocking calls with Reactor
Since the underlying service call is blocking it should be executed on a dedicated thread pool. Size of this thread pool should be equal to the number of concurrent calls if you want to achieve full concurrency. That's why we need .subscribeOn(Schedulers.boundedElastic())
See: https://projectreactor.io/docs/core/release/reference/#faq.wrap-blocking
Better solution using WebClient
Note however, that blocking calls should be avoided when using reactor and spring webflux. The correct way to do this would be to replace RestTemplate with WebClient from Spring 5 which is fully non-blocking.
See: https://docs.spring.io/spring-boot/docs/2.0.3.RELEASE/reference/html/boot-features-webclient.html
I'm working on a backend Spring Boot project which is called by multiple clients. One of the functionalities is to merge data from two different databases and return the result, which may take up to 2 minutes.
I would like to be able to make concurrent calls to this endpoint wait for an already running request and return the same result without running the query again.
As shown below I've tried to setup a CompletableFuture field in the service singleton bean (which I know is a code smell since singleton service beans should be stateless).
//RestController
#Async
#GetMapping
public CompletableFuture<List<Foo>> getSyncedFoo() {
return service.syncFoo();
}
//ServiceImpl
private CompletableFuture<List<Foo>> syncTask;
#Override
#Async
#Transactional
public CompletableFuture<List<Foo>> syncFoo() {
if (this.syncTask == null || this.syncTask.isDone()) {
this.syncTask = CompletableFuture.supplyAsync(() -> {
// long running task
return new ArrayList<>();
});
}
return this.dbaseSyncTask;
}
I expected multiple frontend clients calling the api endpoint to receive the same response at roughly the same time, resulting in the backend performing the long running operation just once.
The operation was in fact executed just once but one of the clients received a 503 (Service Unavailable) while the other client received the expected response.
I suspect it's due to the shared use of the CompletableFuture, but I'm at a loss on what approach I should take. Could RxJava be of any use with the Observable strategy?
I've found a functional answer, for now.
#Service
public class FooServiceImpl implements FooService {
private CompletableFuture<List<Foo>> syncFuture;
private Observable<List<Foo>> syncObservable;
#Override
public Single<List<Foo>> syncFoo() {
if (syncFuture == null || syncFuture .isDone()) {
syncFuture = syncFooAsync();
syncObservable = Observable.fromFuture(syncFuture).share();
}
return Single.fromObservable(syncObservable);
}
private CompletableFuture<List<Foo>> syncFooAsync() {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(10_000);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
return new ArrayList<>();
}
});
}
}
By using the RxJava library it s possible to multicast the created observable to multiple listeners using Observable::share method and the #RestController will happily work with the returned Single(s).
Sadly it still uses state in a singleton which is accessed concurrently by multiple threads so I fear situations where concurrency issues like the Observable completing while a new request is still in the process of creating a new subscription.
Hence I do not recommend this as a best practice so I'm not marking this as a final answer.
I want to read a message at a specific position in an class other than InboundHandler. I can't find a way to read it expect in the channelRead0 method, which is called from the netty framework.
For example:
context.writeMessage("message");
String msg = context.readMessage;
If this is not possible, how can I map a result, which I get in the channelRead0 method to a specific call I made in another class?
The Netty framework is designed to be asynchronously driven. Using this analogy, it can handle large amount of connections with minimal threading usage. I you are creating an api that uses the netty framework to dispatch calls to a remote location, you should use the same analogy for your calls.
Instead of making your api return the value direct, make it return a Future<?> or a Promise<?>. There are different ways of implementing this system in your application, the simplest way is creating a custom handler that maps the incoming requests to the Promises in a FIFO queue.
An example of this could be the following:
This is heavily based on this answer that I submitted in the past.
We start with out handler that maps the requests to requests in our pipeline:
public class MyLastHandler extends SimpleInboundHandler<String> {
private final SynchronousQueue<Promise<String>> queue;
public MyLastHandler (SynchronousQueue<Promise<String>> queue) {
super();
this.queue = queue;
}
// The following is called messageReceived(ChannelHandlerContext, String) in 5.0.
#Override
public void channelRead0(ChannelHandlerContext ctx, String msg) {
this.queue.remove().setSuccss(msg);
// Or setFailure(Throwable)
}
}
We then need to have a method of sending the commands to a remote server:
Channel channel = ....;
SynchronousQueue<Promise<String>> queue = ....;
public Future<String> sendCommandAsync(String command) {
return sendCommandAsync(command, new DefaultPromise<>());
}
public Future<String> sendCommandAsync(String command, Promise<String> promise) {
synchronized(channel) {
queue.offer(promise);
channel.write(command);
}
channel.flush();
}
After we have done our methods, we need a way to call it:
sendCommandAsync("USER anonymous",
new DefaultPromise<>().addListener(
(Future<String> f) -> {
String response = f.get();
if (response.startWidth("331")) {
// do something
}
// etc
}
)
);
If the called would like to use our a api as a blocking call, he can also do that:
String response = sendCommandAsync("USER anonymous").get();
if (response.startWidth("331")) {
// do something
}
// etc
Notice that Future.get() can throw an InterruptedException if the Thread state is interrupted, unlike a socket read operation, who can only be cancelled by some interaction on the socket. This exception should not be a problem in the FutureListener.
My goal is to support long-polling for multiple web service callers, and to keep track of which callers are currently "parked" on a long poll (i.e., connected). By "long polling," I mean that a caller calls a web service and the server (the web service) does not return immediately, but keeps the caller waiting for some preset period of time (an hour in my application), or returns sooner if the server has a message to send to the caller (in which case the server returns the message by calling asyncResponse.resume("MESSAGE")).
I'll break this into two questions.
First question: is this a reasonable way to "park" the callers who are long-polling?
#GET
#Produces(MediaType.TEXT_PLAIN)
#ManagedAsync
#Path("/poll/{id}")
public Response poller(#Suspended final AsyncResponse asyncResponse, #PathParam("id") String callerId) {
// add this asyncResponse to a HashMap that is persisted across web service calls by Jersey.
// other application components that may have a message to send to a caller will look up the
// caller by callerId in this HashMap and call resume() on its asyncResponse.
callerIdAsyncResponseHashMap.put(callerId, asyncResponse);
asyncResponse.setTimeout(3600, TimeUnit.SECONDS);
asyncResponse.setTimeoutHandler(new TimeoutHandler() {
#Override
public void handleTimeout(AsyncResponse asyncResponse) {
asyncResponse.resume(Response.ok("TIMEOUT").build());
}
});
return Response.ok("COMPLETE").build();
}
This works fine. I'm just not sure if it's following best practices. It seems odd to have the "return Response..." line at the end of the method. This line is executed when the caller first connects, but, as I understand it, the "COMPLETE" result is never actually returned to the caller. The caller either gets "TIMEOUT" response or some other response message sent by the server via asyncResponse.resume(), when the server needs to notify the caller of an event.
Second question: my current challenge is to accurately reflect the population of currently-polling callers in the HashMap. When a caller stops polling, I need to remove its entry from the HashMap. A caller can leave for three reasons: 1) the 3600 seconds elapse and so it times out, 2) another application component looks up the caller in the HashMap and calls asyncResponse.resume("MESSAGE"), and 3) the HTTP connection is broken for some reason, such as somebody turning off the computer running the client application.
So, JAX-RS has two callbacks I can register to be notified of connections ending: CompletionCallback (for my end-poll reasons #1 and #2 above), and ConnectionCallback (for my end-poll reason #3 above).
I can add these to my web service method like this:
#GET
#Produces(MediaType.TEXT_PLAIN)
#ManagedAsync
#Path("/poll/{id}")
public Response poller(#Suspended final AsyncResponse asyncResponse, #PathParam("id") String callerId) {
asyncResponse.register(new CompletionCallback() {
#Override
public void onComplete(Throwable throwable) {
//?
}
});
asyncResponse.register(new ConnectionCallback() {
#Override
public void onDisconnect(AsyncResponse disconnected) {
//?
}
});
// add this asyncResponse to a HashMap that is persisted across web service calls by Jersey.
// other application components that may have a message to send to a caller will look up the
// caller by callerId in this HashMap and call resume() on its asyncResponse.
callerIdAsyncResponseHashMap.put(callerId, asyncResponse);
asyncResponse.setTimeout(3600, TimeUnit.SECONDS);
asyncResponse.setTimeoutHandler(new TimeoutHandler() {
#Override
public void handleTimeout(AsyncResponse asyncResponse) {
asyncResponse.resume(Response.ok("TIMEOUT").build());
}
});
return Response.ok("COMPLETE").build();
}
The challenge, as I said, is to use these two callbacks to remove no-longer-polling callers from the HashMap. The ConnectionCallback is actually the easier of the two. Since it receives an asyncResponse instance as a parameter, I can use that to remove the corresponding entry from the HashMap, like this:
asyncResponse.register(new ConnectionCallback() {
#Override
public void onDisconnect(AsyncResponse disconnected) {
Iterator<Map.Entry<String, AsyncResponse>> iterator = callerIdAsyncResponseHashMap.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, AsyncResponse> entry = iterator.next();
if (entry.getValue().equals(disconnected)) {
iterator.remove();
break;
}
}
}
});
For the CompletionCallback, though, since the asyncResponse is already done or cancelled at the time the callback is triggered, no asyncResponse parameter is passed in. As a result, it seems the only solution is to run through the HashMap entries checking for done/cancelled ones and removing them, like the following. (Note that I don't need to know whether a caller left because resume() was called or because it timed out, so I don't look at the "throwable" parameter).
asyncResponse.register(new CompletionCallback() {
#Override
public void onComplete(Throwable throwable) {
Iterator<Map.Entry<String, AsyncResponse>> iterator = callerIdAsyncResponseHashMap.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, AsyncResponse> entry = iterator.next();
if (entry.getValue().isDone() || entry.getValue().isCancelled()) {
iterator.remove();
}
}
}
});
Any feedback would be appreciated. Does this approach seem reasonable? Is there a better or more Jersey/JAX-RS way to do it?
Your poller() method does not need to return a Response in order to participate in asynchronous processing. It can return void. If you are doing anything complex in the poller however you should consider wrapping the whole method in a try/catch block that resumes your AsyncResponse object with the exception to ensure that any RuntimeExceptions or other unchecked Throwables are not lost. Logging these exceptions in the catch block here also seems like a good idea.
I'm currently researching the question of how to reliably catch an asynchronous request being cancelled by the client and have read at one question that suggests the mechanism isn't working for the questioner[1]. I'll leave it to others to fill out this information for the moment.
[1] AsyncResponse ConnectionCallback does not fire in Jersey
I read answers from similar Q&A
How do you create an asynchronous HTTP request in JAVA? |
Asynchronous programming design pattern |
AsyncTask Android - Design Pattern and Return Values
I see a lot of solutions , but none really satifies me.
Listener way
Once the results are caught, the processing is implemented in onResult method.
public interface GeolocationListener {
public void onResult(Address[] addresses);
public void onError(Exception e);
}
This solution doesn't quite satify me , because I want to handle the results in the main method. I hate this interface because when the response is returned, it is processed in onResult resulting in chains of processing and no way to go back to the "main" method.
The servlet way
public class SignGuestbookServlet extends HttpServlet {
public void doPost(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
// ...
resp.sendRedirect("/guestbook.jsp");
}
}
There is no exposed Java code calling the servlet. All the configuration is done in the web.xml
The way I want
Wait for the response like this
Response a = getResponse();
// wait until the response is received, do not go further
// process
Response b = getResponse();
// wait until the response is received, do not go further
process(a,b);
Is there a design pattern to handle the async request and wait for the response like above ? Other way than the listener.
Please no library or framework.
EDIT
Thanks so far the responses. I didn't give you the full picture so I exposed the Geolocation class
I started the implementation . I don't know how to implement the method . Can someone shows "how to" ? He (or she) must also implement the listener to retrieve the results
private Address getFullAddress (String text, AddressListener listener, ... ){
// new Geolocation(text, listener, options).start()
// implements Geolocation.GeolocationListener
// how to return the Address from the onResult ?
}
First, you should not reject the first two methods you discuss. There are very good reasons people are using those techniques and you should try to learn them instead of creating new ones.
Otherwise, you should look at java.util.concurrent:
ExecutorService es = Executors.newFixedThreadPool(2);
...
Future<Response> responseA = es.submit(responseGetter);
Future<Response> responseB = es.submit(responseGetter);
process(responseA.get(), responseB.get());
where responseGetter is of type Callable<Response> (you must implement the method public Response call()).
Asynchronous code can always be made synchronous. The simplest/crudest way is to make the async call, then enter a while loop that just sleeps the current thread until the value comes back.
Edit: Code that turns an asynchronous callback into synchronous code--again, a crude implementation:
import java.util.concurrent.*;
public class MakeAsynchronousCodeSynchronous {
public static void main(String[] args) throws Exception {
final Listener listener = new Listener();
Runnable delayedTask = new Runnable() {
#Override
public void run() {
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
throw new IllegalStateException("Shouldn't be interrupted", e);
}
listener.onResult(123);
}
};
System.out.println(System.currentTimeMillis() + ": Starting task");
Executors.newSingleThreadExecutor().submit(delayedTask);
System.out.println(System.currentTimeMillis() + ": Waiting for task to finish");
while (!listener.isDone()) {
Thread.sleep(100);
}
System.out.println(System.currentTimeMillis() + ": Task finished; result=" + listener.getResult());
}
private static class Listener {
private Integer result;
private boolean done;
public void onResult(Integer result) {
this.result = result;
this.done = true;
}
public boolean isDone() {
return done;
}
public Integer getResult() {
return result;
}
}
}
You could also use a CountDownLatch as recommended by hakon's answer. It will do basically the same thing. I would also suggest you get familiar with the java.util.concurrent package for a better way to manage threads. Finally, just because you can do this doesn't make it a good idea. If you're working with a framework that's based on asynchronous callbacks, you're probably much better off learning how to use the framework effectively than trying to subvert it.
Could CountDownLatch help you? In the main method, you call getResponse and then countDownLatch.await(). Pass a count down latch to the getResponse method and then count down once getResponse the result from getResponse is finished:
CountDownLatch latch = new CountDownLatch(1);
Response a = getResponse(latch);
latch.await();
latch = new CountDownLatch(1);
Response b = getResponse(latch);
latch.await();
process(a, b);
Your getResponse needs to call latch.countDown() once it's asynch parts return a result.
e.g.:
public Response getResponse(CountDownLatch latch) {
someAsychBloc(final CountDownLatch latch) {
do work
latch.countDown();
}
}
Essentially you need a "listener" of sorts no matter what. This is because you do not know WHEN your return message will come back, if at all (that is one of the downsides of asynchronous processing...what to do if you do not get a return message).
So you either need to implement a listener that waits for events (ie, it is nudged by the returning message to be processed).
Or you could do a hybrid on that by having a separate thread that "polls" (or pulls) a response area on your service to see if the return message exists.
So it really comes down to whether you want more of a "pull" or "push" method of retrieving messages.
The SCA (Service Component Architecture) framework might be something to consider, but depending on what you are doing, it could be overkill too. But something to consider.
EDIT:
I just found this in the Java SE 6 Javadocs that may be helpful. The
interface CompletionService which abstracts the very thing you care
about --> asynchronous work. I suggest you take a look.
If you want a page flow in a web application, you have to handle in the web way : storing some data either in the session, or cookies or hidden fields, etc.
The problem you're trying to tackle, as far as I understand it, doesn't come from asynchronousness but from the stateless http protocole.
Regards,
Stéphane