I am trying to implement a simple RPC-like (or request-response) system over WebSockets in Java (there will be JS on the front-end but I am working on the back-end for now).
I am trying to apply the Java CompletableFuture pattern to handle sending messages asynchronously. But I am currnently stuck on error handling.
I have a class (let's call it the rpc class) that is responsible to send the message over a WebSocket session (using Spring WebSocket support classes here), then to wait for "reply" type messages, and matches them with the pending request and returning the content to the caller.
The flow is :
Client code calls the method on the rpc class, specifying the name of the procedure to call on the remote process, the session to which to send the message, and a map of arguments to send along.
The rpc class uses another lower level class to sends the message asynchronously using an Executor (thread pool), and receives a CompletableFuture<Void> for the "send the message" operation
It stores the pending request in a map, builds a CompletableFuture<Map<String, Object>> and associates it with the pending request, and stores them in a map. It returns the completable future.
When a "reply" type message is received, a method is called on the same class, this method tries to match the response with one of the pending requests (they have an ID for this) and then completes the CompletableFuture with the content received in the response.
So there are 3 threads involved : the caller thread, the thread that sends the message, and the thread that receives the message and completes the future.
Now, how should I handle an error in the sending of the message (e.g. IO error) in order to make the returned completableFuture also fail (or maybe implement a retry strategy, and a time out...) ?
Here is the code of the rpc class method that sends the message :
/**
* Invoke a remote procedure over WS on the specified session, with the given arguments.
* #param session The target session on which to send the RPC message
* #param target The name of the procedure to call
* #param arguments The arguments to be sent in the message
* #return
*/
public CompletableFuture<Map<String,Object>> invoke(WebSocketSession session, String target, Map<String, Object> arguments){
Invocation invocationMessage = new Invocation(target, arguments);
invocationMessage.setId(getNextId());
// completeable future for the result. It does nothing, will be completed when reply is received which happen in a different thread, see completeInvocation
CompletableFuture<Map<String, Object>> invocationFuture = new CompletableFuture<>();
CompletableFuture<Void> senderFuture = sender.sendMessage(session, invocationMessage);
// handle problem in the sending of the message
senderFuture.exceptionally(e -> {
// is this correct ??
invocationFuture.completeExceptionally(e);
return null;
});
// store the pending invocation in the registry
registry.addPendingInvocation(new PendingInvocation(invocationMessage, session, invocationFuture));
// return the future so the caller can have access to the result once it is ready
return invocationFuture;
}
The simplest way to do it would be to simply chain the futures using thencompose():
// completeable future for the result. It does nothing, will be completed when reply is received which happen in a different thread, see completeInvocation
CompletableFuture<Map<String, Object>> invocationFuture = new CompletableFuture<>();
CompletableFuture<Void> senderFuture = sender.sendMessage(session, invocationMessage);
// store the pending invocation in the registry
registry.addPendingInvocation(new PendingInvocation(invocationMessage, session, invocationFuture));
// return the future so the caller can have access to the result once it is ready
return senderFuture.thenCompose(__ -> invocationFuture);
In case of exceptional completion of senderFuture, the returned future will be completed exceptionally as well, with a CompletionException holding the exception as its cause (c.f. CompletionStage api).
Note that there are other concerns you might want to tackle as well:
Shouldn't you also cancel the pending invocation in case of exception?
What happens if the response arrives before addPendingInvocation is called? Shouldn't you call it before calling sendMessage to avoid issues?
Since you don't do anything with invocationFuture, wouldn't it be better to create it inside addPenidngInvocation?
Related
So my use-case is to consume messages from Kafka in a Spring Webflux application while programming in the reactive style using Project Reactor, and to perform a non-blocking operation for each message in the same order as the messages were received from Kafka. The system should also be able to recover on its own.
Here is the code snippet that is setup to consume from :
Flux<ReceiverRecord<Integer, DataDocument>> messages = Flux.defer(() -> {
KafkaReceiver<Integer, DataDocument> receiver = KafkaReceiver.create(options);
return receiver.receive();
});
messages.map(this::transformToOutputFormat)
.map(this::performAction)
.flatMapSequential(receiverRecordMono -> receiverRecordMono)
.doOnNext(record -> record.receiverOffset().acknowledge())
.doOnError(error -> logger.error("Error receiving record", error))
.retryBackoff(100, Duration.ofSeconds(5), Duration.ofMinutes(5))
.subscribe();
As you can see, what I do is: take the message from Kafka, transform it into an object intended for a new destination, then send it to the destination, and then acknowledge the offset to mark the message as consumed and processed. It is critical to acknowledge the offset in the same order as the messages being consumed from Kafka so that we don't move the offset beyond messages that were not fully processed (including sending some data to the destination). Hence I'm using a flatMapSequential to ensure this.
For simplicity let's assume the transformToOutputFormat() method is an identity transform.
public ReceiverRecord<Integer, DataDocument> transformToOutputFormat(ReceiverRecord<Integer, DataDocument> record) {
return record;
}
The performAction() method needs to do something over the network, say call an HTTP REST API. So the appropriate APIs return a Mono, which means the chain needs to be subscribed to. Also, I need the ReceiverRecord to be returned by this method so that the offset can be acknowledged in the flatMapSequential() operator above. Because I need the Mono subscribed to, I'm using flatMapSequential above. If not, I could have used a map instead.
public Mono<ReceiverRecord<Integer, DataDocument>> performAction(ReceiverRecord<Integer, DataDocument> record) {
return Mono.just(record)
.flatMap(receiverRecord ->
HttpClient.create()
.port(3000)
.get()
.uri("/makeCall?data=" + receiverRecord.value().getData())
.responseContent()
.aggregate()
.asString()
)
.retryBackoff(100, Duration.ofSeconds(5), Duration.ofMinutes(5))
.then(Mono.just(record));
I have two conflicting needs in this method:
1. Subscribe to the chain that makes the HTTP call
2. Return the ReceiverRecord
Using a flatMap() means my return type changes to a Mono. Using doOnNext() in the same place would retain the ReceiverRecord in the chain, but would not allow the HttpClient response to be subscribed to automatically.
I can't add .subscribe() after asString(), because I want to wait till the HTTP response is completely received before the offset is acknowledged.
I can't use .block() either since it runs on a parallel thread.
As a result, I need to cheat and return the record object from the method scope.
The other thing is that on a retry inside performAction it switches threads. Since flatMapSequential() eagerly subscribes to each Mono in the outer flux, this means that while acknowledgement of offsets can be guaranteed in order, we can't guarantee that the HTTP call in performAction will be performed in the same order.
So I have two questions.
Is it possible to return record in a natural way rather than returning the method scope object?
Is it possible to ensure that both the HTTP call as well as the offset acknowledgement are performed in the same order as the messages for which these operations are occurring?
Here is the solution I have come up with.
Flux<ReceiverRecord<Integer, DataDocument>> messages = Flux.defer(() -> {
KafkaReceiver<Integer, DataDocument> receiver = KafkaReceiver.create(options);
return receiver.receive();
});
messages.map(this::transformToOutputFormat)
.delayUntil(this::performAction)
.doOnNext(record -> record.receiverOffset().acknowledge())
.doOnError(error -> logger.error("Error receiving record", error))
.retryBackoff(100, Duration.ofSeconds(5), Duration.ofMinutes(5))
.subscribe();
Instead of using flatMapSequential to subscribe to the performAction Mono and preserve sequence, what I've done instead is delayed the request for more messages from the Kafka receiver until the action is performed. This enables the one-at-a-time processing that I need.
As a result, performAction doesn't need to return a Mono of ReceiverRecord. I also simplified it to the following:
public Mono<String> performAction(ReceiverRecord<Integer, DataDocument> record) {
HttpClient.create()
.port(3000)
.get()
.uri("/makeCall?data=" + receiverRecord.value().getData())
.responseContent()
.aggregate()
.asString()
.retryBackoff(100, Duration.ofSeconds(5), Duration.ofMinutes(5));
}
At first I just want to say that I'm new to akka and Futures. So be gentle :).
I have init method in some class which returns ListenableFuture<Boolean>. This method should execute some time consuming code in separate thread and create akka actor which is listening to some messages in akka cluster. Future returned by init method should be completed after this actor receives certain message AND that time consuming code is finished.
How can I achieve this using ListenableFuture from Guava?
Refer to Akka documentation about futures: http://doc.akka.io/docs/akka/current/java/futures.html
You can create 2 different futures, one executing the time consuming code:
Future<String> f1 = future(new Callable<String>() {
public String call() {
return "Hello" + "World";
}
}, system.dispatcher());
And another one sending a message to an actor with an ask:
Timeout timeout = new Timeout(Duration.create(5, "seconds"));
Future<Object> f2 = Patterns.ask(actor, msg, timeout);
Finally you can use Future.sequence to create a single future that gets completed when both your futures are completed.
Iterable<Future<Integer>> listOfFutureInts = source;
Future<Iterable<Integer>> futureListOfInts = sequence(listOfFutureInts, ec);
I solved it like this. I created ListenableFuture containing time consuming code. At the end of this furute I created CountDownLatch and passed it (using creator) to akka actor. This listenable future was then blocked by countDownLatch.await(). Created actor with injected latch then listened to certain message in cluster. After consuming this certain message I called countDownLatch.countDown(). This unblocks future, where was await() called and future finished and returned value.
I am using the StreamObserver class found in the grpc-java project to set up some bidirectional streaming.
When I run my program, I make an undetermined number of requests to the server, and I only want to call onCompleted() on the requestObserver once I have finished making all of the requests.
Currently, to solve this, I am using a variable "inFlight" to keep track of the requests that have been issued, and when a response comes back, I decrement "inFlight". So, something like this.
// issuing requests
while (haveRequests) {
MessageRequest request = mkRequest();
this.requestObserver.onNext(request);
this.inFlight++;
}
// response observer
StreamObserver<Message> responseObserver = new StreamObserver<Message> {
#Override
public void onNext(Message response) {
if (--this.onFlight == 0) {
this.requestObserver.onCompleted();
}
// work on message
}
// other methods
}
A bit pseudo-codey, but this logic works. However, I would like to get rid of the "inFlight" variable if possible. Is there anything within the StreamObserver class that allows this sort of functionality, without the need of an additional variable to track state? Something that would tell the number of requests issued and when they completed.
I've tried inspecting the object within the intellij IDE debugger, but nothing is popping out to me.
To answer your direct question, you can simply call onComplete from the while loop. All the messages passed to onNext. Under the hood, gRPC will send what is called a "half close", indicating that it won't send any more messages, but it is willing to receive them. Specifically:
// issuing requests
while (haveRequests) {
MessageRequest request = mkRequest();
this.requestObserver.onNext(request);
this.inFlight++;
}
requestObserver.onCompleted();
This ensures that all responses are sent, and in the order that you sent them. On the server side, when it sees the corresponding onCompleted callback, it can half-close its side of the connection by calling onComplete on its observer. (There are two observers on the server side one for receiving info from the client, one for sending info).
Back on the client side, you just need to wait for the server to half close to know that all messages were received and processed. Note that if there were any errors, you would get an onError callback instead.
If you don't know how many requests you are going to make on the client side, you might consider using an AtomicInteger, and call decrementAndGet when you get back a response. If the return value is 0, you'll know all the requests have completed.
I currently have code which dispatches a request using the Ask Pattern. The dispatched request will generate an Akka Actor which sends a HTTP request and then returns the response. I'm using Akka's circuit breaker API to manage issues with the upstream web services i call.
If the circuitbreaker is in an open state then all subsequent requests are failing fast which is the desired effect. However when the actor fails fast it just throws a CircuitBreakerOpenException, stops the actor however control does not return to the code which made the initial request until an AskTimeoutException is generated.
This is the code which dispatches the request
Timeout timeout = new Timeout(Duration.create(10, SECONDS));
Future<Object> future = Patterns.ask(myActor, argMessage, timeout);
Response res = (Response ) Await.result(future, timeout.duration());
This is the circuitbreaker
getSender().tell(breaker.callWithSyncCircuitBreaker(new Callable<Obj>()
{
#Override
public Obj call() throws Exception {
return fetch(message);
}
}), getSelf()
);
getContext().stop(getSelf());
When this block of code is executed and if the circuit is open it fails fast throwing an exception however i want to return control back to the code which handles the future without having to wait for a timeout.
Is this possible?
When an actor fails out and is restarted, if it was processing a message, no response will be automatically sent to that sender. If you want to send that sender a message on that particular failure then catch that exception explicitly and respond back to that sender with a failed result, making sure to capture the sender first before you go into any future callbacks to avoid closing over this mutable state. You could also try to do this in the preRestart, but that's not very safe as by that time the sender might have changed if you are using futures inside the actor.
My client wants my servlet to be able to return results within X seconds if there is something to return, else return zt X seconds with a message like "sorry could not return within specified time"
This is really a synchronos call to a servlet with a timeout. Is there an established pattern for this sort of behavior ?
What would be the ideal way to do this ?
The request handler thread creates a BlockingQueue called myQueue and gives it to a worker thread which will place its result in the queue when it is finished. Then the handler thread calls "myQueue.poll(X, TimeUnit.SECONDS)" and returns the serialized result if it gets one or your "timeout" error if it gets null instead (meaning the "poll" call timed out). Here is an example of how it might look:
// Servlet handler method.
BlockingQueue<MyResponse> queue = new ArrayBlockingQueue<MyResponse>();
Thread worker = new Thread(new MyResponseGenerator(queue));
worker.start();
MyResponse response = queue.poll(10, TimeUnit.SECONDS);
if (response == null) {
worker.interrupt();
// Send "timeout" message.
} else {
// Send serialized response.
}
Note that thread management in general (but especially in a servlet container) is full of pitfalls, so you should become very familiar with the servlet specification and behavior of your particular servlet container before using this pattern in a production system.
Using a ThreadPool is another option to consider but will add another layer of complexity.