I know that my question has been asked already multiple times, but I feel like there is still no satisfying answer to it.
Basically I have two downstream services which I want to call (in parallel) and then I want to combine the results and return it (as Json). Both calls can fail but both results are not mandatory, so also an empty combined response is possible:
class FirstResponse {...}
class SecondResponse {...}
class CombinedResponse {
private FirstResponse first;
private SecondResponse second;
}
class FirstService {
Mono<FirstResponse> get(){
return webclient.get(...)
.bodyToMono(FirstResponse.class)
.onErrorResume(throwable -> Mono.empty);
}
}
class SecondService {
Mono<SecondResponse> get(){
return webclient.get(...)
.bodyToMono(SecondResponse.class)
.onErrorResume(throwable -> Mono.empty);
}
}
#RestController(...)
class CombinationController {
#GetMapping(...)
Mono<CombinedResponse> getCombined() {
Mono.zip(firstService.get(), secondService.get(), (first, second) -> {
return new CombinedResponse(first, second);
})
}
}
Now in case the calls to firstService fails, also the response from secondService gets ignored. But what I actually would like to have, is that CombinedResponse still gets (partially populated).
As a disclaimer I have to say, that I am currently migrating my code from rxjava1 and there in case of downstream errors I just return Single.just(null). This allows me to zip both results and just sets the values to null.
About Mono.zip() :
An error or empty completion of any source will cause other sources to
be cancelled and the resulting Mono to immediately error or complete,
respectively.
Also, reactor does not allow null values, so you should do some workaround in your case. In some simple cases it is easy to define some default value in case of error (for example, empty String), but for custom types it would be weird to create an empty object.
As an alternative for such cases I would suggest to use Optional.
This solution adds some boilerplate code, though.
First service:
class FirstService {
Mono<Optional<FirstResponse>> get(){
return webclient.get(...)
.bodyToMono(FirstResponse.class)
.map(Optional::of)
.onErrorReturn(Optional.empty());
}
}
Second service:
class SecondService {
Mono<Optional<SecondResponse>> get(){
return webclient.get(...)
.bodyToMono(SecondResponse.class)
.map(Optional::of)
.onErrorReturn(Optional.empty());
}
}
And "combiner" :
#GetMapping(...)
Mono<CombinedResponse> getCombined() {
Mono.zip(firstService.get(), secondService.get())
.map(tuple -> {
// check optionals here from tuple.getT1() and tuple.getT2()
// and do whatever you want
})
...
}
Related
I have an interface that I am implementing that is generated from a WSDL, so I can't change the signatures. One of the methods returns a List<Attributes> with Attributes being another WSDL generated class. The call flow has been updated to be reactive, and the Mono is bubbled all the way up the chain to this implementing class.
public List<Attributes> getUserProfile(final String filterValue);
The return statement calls a method that returns a Mono<UserProfile> which is then passed to another method that returns the List, but I cannot figure out how to subscribe to this to get the List so that my code compiles and works in a non-blocking way?
#Override
public List<Attributes> getUserProfile(final String filterValue) {
return getUserProfile(serviceName, filterValue)
.map(userProfile -> userProfileToAttributeList(userProfile));
// .subscribe(userProfile -> userProfileToAttributeList(userProfile));
// .map(listAttributes -> listAttributes);
// Mono<UserProfile> userProfileMono = getUserProfile(serviceName, filterValue);
// return userProfileMono.subscribe(userProfile -> userProfileToAttributeList(userProfile));
}
private Mono<UserProfile> getUserProfile(final String serviceName,final String filterValue) {
return myService.getUserProfile(serviceName, filterValue);
}
private List<Attributes> userProfileToAttributeList(final UserProfile userProfile) {
return userProfile.getAttributes().stream().map(MyServiceEndpoint::newAttribute).collect(Collectors.toList());
}
Since you can't change the method signature to return Mono or Flux there is no other way around it, you need to block to get the "real" List<Attributes>.
Reactive only works if the complete operation is reactive on each step. Since this method "says" it will return List<Attributes> and not "promises" to return a List<Attributes> at some point in the future when needed and requested you have no option here.
I have a DTO class like this :
public class User {
#Field("id")
private String id;
private String userName;
private String emailId;
}
I have to provide an update and delete feature through API.
I have written the following code to delete the record:
public Mono<String> userData(User body) {
repo.removeUserDetails(userObj).subscribe();
return Mono.just("Remove Successful");
}
RemoveUserDetails method is something like this :
public Mono<User> removeUserDetails(User userObj) {
return findByUsername(userObj.getUsername())
.flatMap(existingUser -> {
// logic to delete the data from database which working as expected
}).switchIfEmpty(
Mono.defer(() -> {
return Mono.error(new Exception("User Name " + userObj.getUsername() + " doesn't exist."));
})
);
}
The problem with this code is even if the user is not existing, it is not showing the Mono error I'm returning. In every case, this always returns "Remove Successful".
How can I change my service layer method so that it can return whatever is received by the repo method? I'm new to Reactor code, so unable to figure out how to write it.
Whenever you call subscribe, consider it an immediate red flag. Subscription is something that should be handled by the framework you're using (Webflux in this case.)
If you subscribe yourself, such as in this example:
public Mono<String> userData(User body) {
repo.removeUserDetails(userObj).subscribe();
return Mono.just("Remove Successful");
}
...then you've essentially created a "fire and forget" type subscription, where you have no way of knowing if that publisher completed successfully, if it caused an error, how long it took to complete, whether it completed at all, or whether it emitted an element. So in this case, you're saying "send a request to remove user details, forget you sent it, and then before waiting for any kind of result, always return 'Remove successful'." This is almost never what you want.
You could use something like:
public Mono<String> userData(User body) {
return repo.removeUserDetails(userObj)
.then(Mono.just("Remove Successful"));
}
...which is much better as it includes everything as part of the reactive chain. In this case, you'll either get an error signal, or you'll get "Remove Successful".
However, chances are you don't need that String to be returned at all - you just need to know if it's successful or not. The standard way of doing that (I just need to know that it's completed successfully or not, I don't need it to return a value) is to use Mono<Void> as the return type and then(), something like:
public Mono<Void> userData(User body) {
return repo.removeUserDetails(userObj).then();
}
...which will give you a standard completion if the deletion was successful, and an error signal otherwise.
A common pattern you find when using reactive java code is handling nulls when collecting a list.
The following code is a simple example showing how to handle nulls returned by a Location by wrapping getLocation in a Mono.defer then handling a null using onErrorReturn.
The test code
List<String> items = inventory.testList().block();
items.forEach(System.out::println);
USA
Not Found
SPAIN
private List<Integer> clusters;
private List<Mono<Location>> locations;
private List<String> countryCodes;
public Mono<List<String>> testList() {
clusters = Arrays.asList(0, 1, 2);
locations = Arrays.asList(Mono.just(new Location(0)), null, Mono.just(new Location(2)));
countryCodes = Arrays.asList("USA", "FRANCE", "SPAIN");
return Flux.fromIterable(clusters)
.flatMap(cluster -> getLocation(cluster))
.collectList();
}
public Mono<String> getLocation(int clusterID) {
return Mono.defer(() -> locations.get(clusterID))
.flatMap(location -> Mono.just(location.id))
.flatMap(id -> Mono.just(countryCodes.get(id)))
.onErrorReturn(Exception.class, "Not Found");
}
I use Spring WebFlux (Project Reactor) and I'm facing the following problem:
I have to get some data from db to use them to call another service - everything in one stream. How to do that?
public Mono<MyObj> saveObj(Mono<MyObj> obj) {
return obj
.flatMap(
ob->
Mono.zip(
repo1.save(
...),
repo2
.saveAll(...)
.collectList(),
repo3
.saveAll(...)
.collectList())
.map(this::createSpecificObject))
.doOnNext(item-> createObjAndCallAnotherService(item));
}
private void createObjAndCallAnotherService(Prot prot){
myRepository
.findById(
prot.getDomCred().stream()
.filter(Objects::nonNull)
.findFirst()
.map(ConfDomCred::getCredId)
.orElse(UUID.fromString("00000000-0000-0000-0000-000000000000")))
.doOnNext( //one value is returned from myRepository -> Flux<MyObjectWithNeededData>
confCred-> {//from this point the code is unreachable!!! - why????
Optional<ConfDomCred> confDomCred=
prot.getDomCreds().stream().filter(Objects::nonNull).findFirst();
confDomCred.ifPresent(
domCred -> {
ProtComDto com=
ProtComDto.builder()
.userName(confCred.getUsername())
.password(confCred.getPassword())
.build();
clientApiToAnotherService.callEndpintInAnotherService(com); //this is a client like Feign that invokes method in another service
});
});
}
UPDATE
When I invoke
Flux<MyObj> myFlux = myRepository
.findById(
prot.getDomCred().stream()
.filter(Objects::nonNull)
.findFirst()
.map(ConfDomCred::getCredId)
.orElse(UUID.fromString("00000000-0000-0000-0000-000000000000")));
myFlux.subscribe(e -> e.getPassword())
then the value is printed
UPDATE2
So as a recap - I think the code below is asynchronous/non-blocking - am I right?
In my
ProtectionCommandService
I had to use subscribe() twice - only then I can call my other service and store them my object: commandControllerApi.createNewCommand
public Mono<Protection> saveProtection(Mono<Protection> newProtection) {
return newProtection.flatMap(
protection ->
Mono.zip(
protectorRepository.save(//some code),
domainCredentialRepository
.saveAll(//some code)
.collectList(),
protectionSetRepository
.saveAll(//some code)
.collectList())
.map(this::createNewObjectWrapper)
.doOnNext(protectionCommandService::createProtectionCommand));
}
ProtectionCommandService class:
public class ProtectionCommandService {
private final ProtectionCommandStrategyFactory protectionCommandFactory;
private final CommandControllerApi commandControllerApi;
public Mono<ProtectionObjectsWrapper> createProtectionCommand(
ProtectionObjectsWrapper protection) {
ProductType productType = protection.getProtector().getProductType();
Optional<ProtectionCommandFactory> commandFactory = protectionCommandFactory.get(productType);
commandFactory
.get()
.createCommandFromProtection(protection)
.subscribe(command -> commandControllerApi.createNewCommand(command).subscribe());
return Mono.just(protection);
}
}
And one of 2 factories:
#Component
#AllArgsConstructor
#Slf4j
public class VmWareProtectionCommandFactory implements ProtectionCommandFactory {
private static final Map<ProductType, CommandTypeEnum> productTypeToCommandType =
ImmutableMap.of(...//some values);
private final ConfigurationCredentialRepository configurationCredentialRepository;
#Override
public Mono<CommandDetails> createCommandFromProtection(ProtectionObjectsWrapper protection) {
Optional<DomainCredential> domainCredential =
protection.getDomainCredentials().stream().findFirst();
return configurationCredentialRepository
.findByOwnerAndId(protection.getOwner(), domainCredential.get().getCredentialId())
.map(credential -> createCommand(protection, credential, domainCredential.get()));
}
and createCommand method returns Mono object as a result of this factory.
private Mono<CommandDetails> createCommand(Protection protection
//other parameters) {
CommandDto commandDto =
buildCommandDto(protection, confCredential, domainCredentials);
String commands = JsonUtils.toJson(commandDto);
CommandDetails details = new CommandDetails();
details.setAgentId(protection.getProtector().getAgentId().toString());
details.setCommandType(///some value);
details.setArguments(//some value);
return Mono.just(details);
UPDATE3
My main method that calls everything has been changed a little bit:
public Mono<MyObj> saveObj(Mono<MyObj> obj) {
return obj
.flatMap(
ob->
Mono.zip(
repo1.save(
...),
repo2
.saveAll(...)
.collectList(),
repo3
.saveAll(...)
.collectList())
.map(this::wrapIntoAnotherObject)
.flatMap(protectionCommandService::createProtectionCommand)
.map(this::createMyObj));
Stop breaking the chain
This is a pure function it returns something, and always returns the same something whatever we give it. It has no side effect.
public Mono<Integer> fooBar(int number) {
return Mono.just(number);
}
we can call it and chain on, because it returns something.
foobar(5).flatMap(number -> { ... }).subscribe();
This is a non pure function, we can't chain on, we are breaking the chain. We can't subscribe, and nothing happens until we subscribe.
public void fooBar(int number) {
Mono.just(number)
}
fooBar(5).subscribe(); // compiler error
but i want a void function, i want, i want i want.... wuuaaa wuaaaa
We always need something to be returned so that we can trigger the next part in the chain. How else would the program know when to run the next section? But lets say we want to ignore the return value and just trigger the next part. Well we can then return a Mono<Void>.
public Mono<Void> fooBar(int number) {
System.out.println("Number: " + number);
return Mono.empty();
}
foobar(5).subscribe(); // Will work we have not broken the chain
your example:
private void createObjAndCallAnotherService(Prot prot){
myRepository.findById( ... ) // breaking the chain, no return
}
And some other tips:
Name your objects correctly not MyObj and saveObj, myRepository
Avoid long names createObjAndCallAnotherService
Follow single responsibility createObjAndCallAnotherService this is doing 2 things, hence the name.
Create private functions, or helper functions to make your code more readable don't inline everything.
UPDATE
You are still making the same misstake.
commandFactory // Here you are breaking the chain because you are ignoring the return type
.get()
.createCommandFromProtection(protection)
.subscribe(command -> commandControllerApi.createNewCommand(command)
.subscribe()); // DONT SUBSCRIBE you are not the consumer, the client that initiated the call is the subscriber
return Mono.just(protection);
What you want to do is:
return commandFactory.get()
.createCommandFrom(protection)
.flatMap(command -> commandControllerApi.createNewCommand(command))
.thenReturn(protection);
Stop breaking the chain, and don't subscribe unless your service is the final consumer, or the one initiating a call.
I have 2 Single. In first I receive token and in second I need use it, and then I must save some info from second single and return completable.
I thought do this with completable and andThen, like this:
String token;
IStoreProvider storeProvider;
IWebProvider webProvider;
public Completable getUserInfo(){
return Completable.fromSingle(Completable
.fromSingle(storeProvider
.getToken()
.doOnSuccess(x->token=x))
.andThen(webProvider.getUserInfo(token)
.doOnSuccess(x->storeProvider.saveUserInfo(x)));
interface IStoreProvider{
Single<String> getToken();
Completable saveUserInfo(UserInfo userInfo);
}
interface IWebProvider{
Single<UserInfo> getUserInfo(token);
}
But its dosen't work. How I can do it?
Your stream is created before its executed, so during creation token=null and therefore you will get null in getUserInfo.
flatMap will help you.
public Completable getUserInfo() {
return storeProvider.getToken()
.flatMap(token -> webProvider.getUserInfo(token))
.flatMapCompletable(userInfo -> storeProvider.saveUserInfo(userInfo));
}
Here, I am trying to make asynchronous and non-blocking calls using reactor and for each request, I may have to call two services in sequence (in my case below, getAccountInfoFromAAA and getAccountInfoFromBBB).
Here is my ItemRequest object:
public class ItemRequest {
private Account account;
private Result firstServiceResult;
private Result secondServiceResult;
private PostingParameterCode postingParameterCode; //enum
//...
//...
//getters and setters
}
So, my request input will contain multiple itemRequests and for each itemRequest, I am doing asynchronous calls as:
public void getAccountData(List<ItemRequest> itemRequests) {
ImmutableList<ItemRequest> list = ImmutableList.copyOf(itemRequests);
Flux.fromIterable(list).flatMap(this::callBothSors).blockLast();
}
public Mono<ItemRequest> callBothSors(ItemRequest itemRequest) {
return getAccountDataService.getAccountDataFromAAAandBBB(itemRequest);
//here, it will enter into a sequential call for each itemRequest
}
This is my first service call interface:
public Mono<ItemRequest> getAccountDataFromAAA(ItemRequest itemRequest);
This is my second service call interface:
public Mono<ItemRequest> getAccountDataFromBBB(ItemRequest itemRequest);
This method will have upto two calls in sequence based on the condition:
public Mono<ItemRequest> getAccountDataFromAAAandBBB(ItemRequest itemRequest){
Mono<ItemRequest> firstCallResult = Mono.empty();
Mono<ItemRequest> secondCallResult = Mono.empty();
if(isFirstServiceCallRequired(itemRequest)){
firstCallResult = this.firstServiceCallImpl.getAccountDataFromAAA(itemRequest);
//basically, firstService call will update the accountKey information and
//will also set the result status to OK which is required to decide
//whether to make secondService call.
} else {
//Account key is already present, so just update the result status which I need later.
Result result = new Result();
result.setStatus(Result.Status.OK);
result.setMessageText("First call not required as account info is set for item request");
itemRequest.setFirstServiceResult(result);
}
//Now, before calling the second service, I need to check the following:
if(null!= itemRequest.getFirstServiceResult() &&
itemRequest.getFirstServiceResult().getStatus().equals(Result.Status.OK) &&
itemRequest.getPostingParameterCode().equals(PostingParameterCode.MOBILECREDIT)){
secondCallResult = this.secondServiceCallImpl.getAccountDataFromBBB(itemRequest);
}
return firstCallResult.then(secondCallResult); //attaching the
//firstCallResult and secondCallResult to produce a single Mono
}
This is working fine when firstCallResult is not required. But when the first call is required, this condition check will not pass since I won't have first call result object updated:
if(null != itemRequest.getFirstServiceResult() &&
itemRequest.getFirstServiceResult().getStatus().equals(Result.Status.OK) &&
itemRequest.getPostingParameterCode().equals(PostingParameterCode.MOBILECREDIT))) { ... }
//this condition check will not pass because first service call is not actually executing
Both cases works fine if I put the following statement:
if(isFirstServiceCallRequired(itemRequest)){
firstCallResult = this.firstServiceCallImpl.getAccountDataFromAAA(itemRequest);
firstCallResult.block(); //adding this case will work on both cases
}
But, I don't think I will get the reactors benefit this way.
I was thinking to have the logic like this:
Mono<ItemRequest> result = firstService.call(...)
.doOnNext(/*do something */)
.then( ... secondService.call())
But couldn't figure out the way to chain the secondService with firstService to get the mono result and have those condition checks too.
Condition check is important since I don't always want to execute the second service. Is there any way to chain the secondService with firstService to get the result and have those condition checks too?
Apologies for the long question. Any suggestions/help would be greatly appreciated.
After offering the bounty points to this question, I was really excited and expecting some answers.
But anyways, I am able to improve my initial solution and have those condition checks too.
I did the following:
I changed the return type from Mono<ItemRequest> to Mono<Void> in both service calls since I am basically updating the data to ItemRequest list:
Handling the parallel call here (each parallel call has a sequential call):
public void getAccountData(List<ItemRequest> itemRequests) {
ImmutableList<ItemRequest> list = ImmutableList.copyOf(itemRequests);
Flux.fromIterable(list).flatMap(this::callBothSors).blockLast();
}
public Mono<Void> callBothSors(ItemRequest itemRequest) {
return getAccountDataService.getAccountDataFromAAAandBBB(itemRequest);
//here, it will enter into a sequential call for each itemRequest
}
and these are my firstServiceCall and secondServiceCall interface changes:
public Mono<Void> getAccountDataFromAAA(ItemRequest itemRequest);
public Mono<Void> getAccountDataFromBBB(ItemRequest itemRequest);
and I chained the secondServiceCall with firstServiceCall to get the mono result and have those condition checks too as:
public Mono<Void> getAccountDataFromAAAandBBB(ItemRequest itemRequest){
Mono<Void> callSequence = Mono.empty();
if(isFirstServiceCallRequired(itemRequest)){
callSequence = this.firstServiceCallImpl.getAccountDataFromAAA(itemRequest);
} else {
//Account key is already present, so just update the result status which I need later.
Result result = new Result();
result.setStatus(Result.Status.OK);
result.setMessageText("First call not required as account info is set for item request");
itemRequest.setFirstServiceResult(result);
}
return callSequence.thenEmpty(Mono.defer(() -> {
//note: Mono.defer ==>> Create a Mono provider that will supply a target Mono to subscribe to
//for each subscriber downstream.
//only if the firstServiceCall result is successful & other condition check successful,
// I am calling secondServiceCall:
if(shouldCallSecondService(itemRequest)){
return this.secondServiceCallImpl.getAccountDataFromAAAandBBB(itemRequest);
} else {
return Mono.empty();
}
}))
Here are some news: A Reactor is not a silver bullet! :)
Whenever you need the response of a call to determine if you need to do something else, this will never be able to be fully parallelized. E.g. you could always do you last suggestion. However, this doesn't mean that using the Reactor doesn't give you any benefits!
Some of the benefits you get:
You are using Netty under the hood instead of Servlet, which helps to avoid locking on I/O operations. This can lead to better allocation of resources, making your system more resilient.
You can do other operations while waiting for a response. If you have things to do where the order doesn't matter, you can always put them there (e.g. auditing, logging etc).
I hope this answers your question :)
public Mono<ItemRequest> getAccountDataFromAAAandBBB(ItemRequest itemRequest) {
Mono<ItemRequest> firstCallResult = Mono.empty();
Mono<ItemRequest> secondCallResult = Mono.empty();
if (isFirstServiceCallRequired(itemRequest)) {
firstCallResult = this.firstServiceCallImpl.getAccountDataFromAAA(itemRequest);
//basically, firstService call will update the accountKey information and
//will also set the result status to OK which is required to decide
//whether to make secondService call.
} else {
/*Account key is already present, so just update the result status which I need
later.*/
firstCallResult = Mono.defer(() -> {
Result result = new Result();
result.setStatus(Result.Status.OK);
result.setMessageText("First call not required as account info is set for item request");
itemRequest.setFirstServiceResult(result);
return Mono.just(itemRequest);
});
}
return firstCallResult.flatMap(itReq -> {
//Now, before calling the second service, I need to check the following:
if (null != itemRequest.getFirstServiceResult() &&
itemRequest.getFirstServiceResult().getStatus().equals(Result.Status.OK) &&
itemRequest.getPostingParameterCode().equals(PostingParameterCode.MOBILECREDIT)) {
return secondCallResult = this.secondServiceCallImpl.getAccountDataFromBBB(itemRequest);
} else {
return itReq;
}
});
}
The next simple example can help you with flatMap understanding:
public static void main(String[] args) {
callExternalServiceA.flatMap(response -> {
if(response.equals("200")){
return Mono.just(response);
} else {
return callExtertnalServiceB();
}
}).block();
}
public static Mono<String> callExtertnalServiceA() {
return Mono.defer(() -> {
System.out.println("Call external service A");
return Mono.just("400");
});
}
public static Mono<String> callExtertnalServiceB() {
return Mono.defer(() -> {
System.out.println("Call external service B");
return Mono.just("200");
});
}