I have an architechture related question. This is a language independent question, but as I come from Java background, it will be easier for me if someone guides me in the Java way.
Basically, the middleware I'm writing communicates with a SOAP based third party service. The calls are async - in a way that, when a service is called, it returns with a response 01 - processing; meaning that the third party has successfully received the request. In the original SOAP request, one callback URL has to be submitted each time, where third party actually sends the result. So, calling a particular service doesn't actually return the result immediately; the result is received in a separate HTTP endpoint in the middleware.
Now in our frontend, we don't want to complicate the user experience. We want our users to call a middleware function (via menu items/buttons), and get the result immediately; and leave the dirty work to the middleware.
Please note that the middleware function (lets say X()) which was invoked from the front end and the middleware endpoint URL(lets call it Y) where third party pushes the result are completely separate from each other. X() somehow has to wait and then fetch the result grabbed in Y and then return the result to the frontend.
How can I build a robust solution to achieve the above mentioned behavior?
The picture depicts my case perfectly. Any suggestions will be highly appreciated.
This question could be more about integration patterns than it is about multi-threading. But requests in the same application/JVM can be orchestrated using a combination of asynchronous invocation and the observer pattern:
This is better done using an example (exploiting your Java knowledge). Check the following simplistic components that try to replicate your scenario:
The third-party service: it exposes an operation that returns a correlation ID and starts the long-running execution
class ExternalService {
public String send() {
return UUID.randomUUID().toString();
}
}
Your client-facing service: It receives a request, calls the third-party service and then waits for the response after registering with the result receiver:
class RequestProcessor {
public Object submitRequest() {
String correlationId = new ExternalService().send();
return new ResultReceiver().register(correlationId).join();
}
}
The result receiver: It exposes an operation to the third-party service, and maintains an internal correlation registry:
class ResultReceiver {
Map<String, CompletableFuture<Object>> subscribers;
CompletableFuture<Object> register(String responseId) {
CompletableFuture<Object> future = new CompletableFuture<Object>();
this.subscribers.put(responseId, future);
return future;
}
public void externalResponse(String responseId, Object result) {
this.subscribers.get(responseId).complete(result);
}
}
Futures, promises, call-backs are handy in this case. Synchronization is done by the initial request processor in order to force the execution to block for the client.
Now this can raise a number of issues that are not addressed in this simplistic class set. Some of these problems may be:
race condition between new ExternalService().send() and new ResultReceiver().register(correlationId). This is something that can be solved in ResultReceiver if it undestands that some responses can be very fast (2-way wait, so to say)
Never-coming results: results can take too long or simply run into errors. These future APIs typically offer timeouts to force cancellation of the request. For example:
new ResultReceiver().register(correlationId)
.get(10000, TimeUnit.SECONDS);
Well what exactly is the problem with doing that? You just create an API (middleware) which doesn't return response until the third party returns the processed result. Front end sends request to X(), X() processes that request by sending a request to Y() and then keep polling Y() to see when the result is ready, then X() takes the results from Y() and sends it back to the front end. Like a facade.
There are some problems regarding using third party services that you don't control which you should consider. First of all you need to implement some kind of circuit breaker or timeout. Because the third party service might hang and never process the results (or process them so long that it makes no sense to wait). Also you should consider some meaningful way to keep the site running even if the third party service is not available or has updated their API or something else prevents you from using it.
And just one last thought in the end. Why would you want to make something that is already implemented asynchronous synchronous? It is made like that probably because it might take time. Blocking the front end for a long periods of time to wait for results makes the user experience unpleasant and the UI unresponsive. It is usually better to stick to asynchronous requests and show the users they are processing but let them do something else meanwhile.
Related
EJB Spec says you shouldn't manage threads. I have seen Bean code that sends remote requests and loops with a Thread.sleep waiting for a response to reduce CPU usage. From what I understand this breaks spec. Does simply calling the logic from a separate POJO or library that is instantiated then referenced in the EJB's method fix this? Does simply removing Thread.sleep fix the issue at the cost of additional CPU consumption? How should external synchronous requests be coded in EJBs?
That depends on the business case. EJB spec provides plenty of resources for async/sync processing without boilerplate code using Thread, Runnable or any other mechanism.
To execute a piece or code asynchronously (that is, the caller won't wait for the response, but carry on), use #Asynchronous, and Future<T> if you want to listen for responses afterwords.
A synchronous call, as you called, is a call that waits for the response, so "How should external synchronous requests be coded in EJBs" is something that doesn't need any kind of asynchronous/background execution. You just make the call and the code itself wait for the response (otherwise it would be asynchronous), being the tipical case a Web Service (either REST or SOAP).
Web Services calls can actually be synchronous or asynchronous, that depends on the business case, but they are usualy synchronous, you make the call and receive a response with the data. In cases of business logic that takes a while to execute, the Web Service receives the resquest and may launch the business logic asynchronously (with an #Asynchronous for instance) and respond immediately with a plain HTTP 202 - Accepted, which basically means "Hey! The request you just sent me is gonna take a while, so I'll do it in the backround".
In that case, may be you have another web service that you need to check to see how that long lasting process is going. That is the only case I can think of in which someone will want that Thread.sleep(...) in a loop, checking the Web Service until it tells you that the process have finished.
Luckily, EJB also provides a solution for that business case:
You can use #Schedule methods in case you need to check/do something indefenately, in specific intervals: something to do every day at 02:00, or every first day of month, or even every 2 seconds.
Or TimerService and #Timeout, in case you want to programatically schedule a single task. This last fits better in the business case we are talking.
So you call the TimerService with the timespan you want to wait for the next check. When time comes the #Timeout method is fired, in which you can check whatever you need, and shcedule another execution in case you need it, even with a new timespan.
I am creating an endpoint which retrieves me some data and in this call it calls 3 different REST calls and due to this it hampers the performance of my application.
My Endpoint Code:
1. REST call to get the userApps()
2. iterate over userAPPs
2.1 make REST call to get the appDetails
2.2 make use of above response to call the 3rd REST call which returns list.
2.3 iterate over the above list and filter out the required fields and put it in main response object.
3.return response
So, this much complexity hampers the performance.
I have tried to add the multithreading concept but the time taken by normal code and multi threading is almost same.
Condition is like, We can not modify the 3 external REST calls to support the pagination.
We can not add the pagination because we don't have any database.
Is there any solution?
You shouldn't add threading, you should remove threads altogether. I.e. you should make all your code non-blocking. This just means that all the work will basically be done in the http-client's threadpool, and all the waiting can be done in the operating system's selector (which we want).
Here is some code how this core logic would work, assuming your http calls return CompletableFuture.
public CompletableFuture<List<Something>> retrieveSomethings() {
return retrieveUserApps().thenCompose(this::retriveAllAppsSomethings);
}
public CompletableFuture<List<Something>> retrieveAllAppsSomethings(List<UserApp> apps) {
return CompletableFuture.allOf(
apps.stream().map(this::retriveAppSomethings).toArray(CompletableFuture[]::new))
.apply(listOfLists -> listOfLists.stream().flatMap(List::stream).collect(Collectors.toList()))
.apply(this::filterSomethings);
}
public CompletableFuture<List<Something>> retreiveAppSomethings(UserApp app) {
return retrieveAppDetails(app).thenCompose(this::retreiveAppDetailSomethings);
}
All this does, is to make everything non-blocking, so everything that can be run in parallel will run in parallel. There is no need to limit anything, since everything will be run in the http-client's threadpool, which is most likely limited. It doesn't matter anyway, because waiting will not take up a thread.
All you have to do for the above code is to implement retrieveUserApps(), retrieveAppDetails(app) and retrieveAppDetailSometings(appDetail). All of which should return a CompletableFuture<> and be implemented with the async-enabled version of your http client.
This will make retrieving data for 1 app or 100 apps the same, since all of those will run in parallel (assuming they all take the same time and the downstream systems can handle this many parallel requests).
Essentially I've written a service in Java that will do initial synchronous processing (a couple simple calls to other web services). Then, after that processing is done, I return an acknowledgement message to the caller, saying I've verified their request and there is now downstream processing happening in the background asynchronously.
In a nutshell, what I'm concerned about is the complexity of the async processing. The sum of those async calls can take up to 2-3 minutes depending on certain parameters sent. My thought here is: what if there's a lot of traffic at once hitting my service, and there are a bunch of hanging threads in the background, doing a large chunk of processing. Will there be bad data as a result? (like one request getting mixed in with a previous request etc)
The code follows this structure:
Validation of headers and params in body
Synchronous processing
Return acknowledgement message to the caller
Asynchronous processing
For #4, I've simply made a new thread and call a method that does all the async processing within it. Like:
new Thread()
{
#Override
public void run()
{
try {
makeDownstreamCalls(arg1, arg2 , arg3, arg4);
} catch (Exception e) {
e.printStackTrace();
}
}
}.start();
I'm basically wondering about unintended consequences of lots of traffic hitting my service. An example I'm thinking about: a thread executing downstream calls for request A, and then another request comes in, and a new thread has to be made to execute downstream calls for request B. How is request B handled in this situation, and what happens to request A, which is still in-progress? Will the async calls in request A just terminate in this case? Or can each distinct request, and thread, execute in parallel just fine and complete, without any strange consequences?
Well, the answer depends on your code, of which you posted a small part, so my answer contains some guesswork. I'll assume that we're talking about some sort of multi-threaded server which accepts client requests, and that those request come to some handleRequest() method which performs the 4 steps you've mentioned. I'll also assume that the requests aren't related in any way and don't affect each other (so for instance, the code doesn't do something like "if a thread already exists from a previous request then don't create a new thread" or anything like that).
If that's the case, then your handleRequest() method can be simultaneously invoked by different server threads concurrently. And each will execute the four steps you've outlined. If two requests happen simultaneously, then a server thread will execute your handler for request A, and a different one will execute it for B at the same time. If during the processing of a request, a new thread is created, then one will be created for A, another for B. That way, you'll end up with two threads performing makeDownstreamCalls(), one with A's parameters one with B's.
In practice, that's probably a pretty bad idea. The more threads your program will create, the more context-switching the OS has to do. You really don't want the number of requests to increase the number of threads in your application endlessly. Modern OSes are capable of handling hundreds or even thousands of threads (as long as they're bound by IO, not CPU), but it comes at a cost. You might want to consider using a Java executor with a limited number of threads to avoid crushing your process or even OS.
If there's too much load on a server, you can't expect your application to handle it. Process what you can within the limit of the application, and reject further request. Accepting more requests when you're fully loaded means that your application crashes, and none of the requests are processed - this is known as "Load Shedding".
Lets say I create an async REST API in Spring MVC with Java 8's Completeable.
How is this called in the client? If its non blocking, does the endpoint return something before processing? Ie
#RequestMapping("/") //GET method
public CompletableFuture<String> meth(){
thread.sleep(10000);
String result = "lol";
return CompletableFuture.completedFuture(result);
}
How does this exactly work? (This code above is just a randomly made code I just thought of).
When I send a GET request from say google chrome # localhost:3000/ then what happens? I'm a newbie to async APIs, would like some help.
No, the client doesn't know it's asynchronous. It'll have to wait for the result normally. It's just the server side that benefits from freeing up a worker thread to handle other requests.
In this version it's pointless, because CompletableFuture.completedFuture() creates a completed Future immediately.
However in a more complex piece of code, you might return a Future that is not yet complete. Spring will not send the response body until some other thread calls complete() on this Future.
Why not just use a new thread? Well, you could - but in some situations it might be more efficient not to. For example you might put a task into an Executor to be handled by a small pool of threads.
Or you might fire off a JMS message asking for the request to be handled by a completely separate machine. A different part of your program will respond to incoming JMS messages, find the corresponding Future and complete it. There is no need for a thread dedicated to this HTTP request to be alive while the work is being done on another system.
Very simple example:
#RequestMapping("/employeenames/{id}")
public CompletableFuture<String> getName(#PathVariable String id){
CompletableFuture<String> future = new CompletableFuture<>();
database.asyncSelect(
name -> future.complete(name),
"select name from employees where id = ?",
id
);
return future;
}
I've invented a plausible-ish API for an asynchronous database client here: asyncSelect(Consumer<String> callback, String preparedstatement, String... parameters). The point is that it fires off the query, then does not block the tread waiting for the DB to respond. Instead it leaves a callback (name -> future.complete(name)) for the DB client to invoke when it can.
This is not about improving API response times -- we do not send an HTTP response until we have a payload to provide. This is about using the resources on the server more efficiently, so that while we're waiting for the database to respond it can do other things.
There is a related, but different concept, of asynch REST, in which the server responds with 202 Accepted and a header like Location: /queue/12345, allowing the client to poll for the result. But this isn't what the code you asked about does.
CompletableFuture was introduced by Java to make handling complex asynchronous programming. It lets the programmer combine and cascade async calls, and offers the static utility methods runAsync and supplyAsync to abstract away the manual creation of threads.
These methods dispatch tasks to Java’s common thread pool by default or a custom thread pool if provided as an optional argument.
If a CompletableFuture is returned by an endpoint method and #complete is never called, the request will hang until it times out.
My function is to send an HTTP request to a server and get a response. It looks like :
public Acknowledgement function() {
Acknowledgement ack = new Acknowledgement("received");
//send http request and get response
return ack;
}
The purpose of the Acknowledgement is to inform the caller that function() has been successfully called and processing has been started.
I want to return acknowledgement before sending HTTP request. One way is to use separate thread(implementing Runnable) for sending request and getting response. But Threads have been pretty old. What are other latest alternative to threads to achieve this.
you could try using a callback listener, like:
interface LifecycleListener {
void onStarting();
}
public Acknowledgement function(LifecycleListener listener) {
listener.onStarting();
...
...
}
Many things exist in Java for almost 20 years. Doesn't mean that we should stop using String, or Integer do we?
But you are correct insofar that there are (slightly) newer abstractions on top of bare metal threads.
So, concepts to study would be ExecutorServices and things like Future.
Using threads directly; especially in conjunction with low level primitives such as wait/notify is probably not the first choice in 2017; you would rather look into the aforementioned ExecutorService; or even going one step further and learn about frameworks like RXJava.
threads are not old,if you want to do it better,you can use thread pool to manage them