I have three WebClients that look something like this:
WebClient
public Mono<MyObject> getResponseOne() {
return webClient.get()
.uri(URI)
.header("header", header)
.bodyValue(body)
.retrieve()
.bodyToMono(MyObject.class);
}
Then I have a controller which calls multiple WebClients:
Controller
#ResponseBody
#PostMapping("/get")
public Mono<MyObject> processResponse() {
MyObject obj = getResponseOne().toFuture().get();
system.out.println("Got first response");
String str = getResponseTwo().toFuture().get();
system.out.println("Got second response");
//process and send with third WebClient
MyObject newObj = getResponseThree(obj, str).toFuture().get();
//process response from third WebClient and send to fourth WebClient
//return statement
}
When I call /get, the console only prints "Got first response" then it just stops there and anything below doesn't seem to be executing. I'm using Postman to send the request, so it keeps on waiting without getting any response.
This may or may not be relevant, but I'm using blocking calls because I need the both responses to be processed before sending it to a third WebClient, the response from the third WebClient will also go through additional processing before being returned as the response of processResponse().
Solution
I used Mono.zip() like Alex suggested:
#ResponseBody
#PostMapping("/get")
public Mono<MyObject> processResponse() {
//TupleN depends on the amount of Monos you want to process
Mono<Tuple2<MyObject,String>> output = Mono.zip(getResponseOne(),getResponseTwo());
return output.map(result ->{
// getT1() & getT2() is automatically generated by the tuple
MyObject obj = result.getT1();
String str = result.getT2();
getResponseThree(obj, str);
//process and return
});
}
More about Mono.zip()
In reactive API nothing happens until you subscribe. Your method returns Mono and you need to construct a flow combining publishers. There are multiple ways to combine depending on the required logic.
For example, if you need the result of the predecessor you could use flatMap to resolve Mono sequentially
return getResponseOne()
.flatMap(res -> getResponseTwo(res))
.flatMap(res -> getResponseThree(res));
In case calls are independent you could use then
return getResponseOne()
.then(getResponseTwo())
.then(getResponseThree());
You could also execute in parallel using Mono.when(getResponseOne(), getResponseTwo(), getResponseThree()) or Mono.zip(getResponseOne(), getResponseTwo(), getResponseThree()).
There are many other operators but the key here is to construct the flow and return Mono or Flux.
Related
I'm trying to reactively fetch data from external API using two methods from some Service class.
I'm new to reactive and Spring in general, so it could be a very obvious mistake but I just can't find it
These are the two methods:
public Mono<SomeClass> get(int value) {
return webClient.get()
.uri("/" + value)
.retrieve()
.onRawStatus(HttpStatus.CONFLICT::equals, clientResponse -> {
return Mono.error(new SomeException1("Some message", clientResponse.rawStatusCode()));
})
.onRawStatus(HttpStatus.NOT_FOUND::equals, clientResponse -> {
return requestGeneration(value)
.flatMap(res -> Mono.error(new SomeException1("Some message", clientResponse.rawStatusCode())));
})
.bodyToMono(SomeClass.class)
.retryWhen(Retry.backoff(5, Duration.ofSeconds(8))
.filter(throwable -> throwable instanceof SomeException1));
}
private Mono<Void> requestGeneration(int value) {
return webClient.post()
.uri("/" + value)
.retrieve()
.onRawStatus(HttpStatus.BAD_REQUEST::equals, clientResponse -> {
return Mono.error(new SomeException2("Wrong value", clientResponse.rawStatusCode()));
})
.bodyToMono(Void.class);
}
Baasically what I'm trying to achieve is:
first GET from http://api.examplepage.com/{value}
if that API returns HTTP404 it means I need to first call POST to the same URL, because the data is not yet generated
the second function does the POST call and returns Mono<Void> because it is just HTTP200 or HTTP400 on bad generation seed (i don't need to process the response)
first function (GET call) could also return HTTP429 which means the data is generating right now, so I need to call again after some time period (5-300 seconds) and check if data has been generated already
then after some time it results in HTTP200 with generated data which I want to map to SomeClass and then return mapped data in controller below
#PostMapping("/follow/{value}")
public Mono<ResponseEntity<?>> someFunction(#PathVariable int value) {
return Mono.just(ResponseEntity.ok(service.get(value)));
}
all the code I posted is very simplified to the issues I'm struggling with and doesn't contain some things I think are not important in this question
and now the actual question:
it doesn't actually make the call? i really don't know what is happening
program doesn't enter onRawStatus, even if i change it to onStatus 2xx or whatever other httpstatus and log inside i see nothing as if it doesn't even enter the chains
when i manually call with postman it seems like the program calls have never been made because the GET call returns 404 (the program didn't request to generate data)
the controller only returns "scanAvailable": true, when i expect it to return mapped SomeClass json
// edit
i changed the code to be a full chain as suggested and it didn't solve the problem. all the status are still unreachable (code inside any onStatus nor onRawStatus never executes)
*** I am sure there are better ways of doing this, but I am starting new on webflux, and still iearning***
I have a requirement where I need to make asynchronous calls to different web services fro one service( It is same end point with differet queryparameters each time, so calling it a different service). There are dynamic number of downstream calls
Service[A] ---> ServiceB[1],ServiceB[2],ServiceB[3],ServiceB[4], etc
Now, when I get the response from each instance of ServiceB[n] , I need to know 'this' response is for which request ( 1 or 2 or 3 or 4 , etc). Reason is tha I need to append thais at the end of request to send as response in specific order. They need to be appended in the same order as they were in the original request. ( so I need to identify which response to its request ). I am geiing a List that has the order in which they were requested.
I have the below code which works functionally. I.e. when I make one request to Service-A ( which makes 4 calls to 4 Servcie B calls)
BUT, When there is parallel requests to Service-A, I see the Flux.merge(asyncRequestList).collectList().block() call does not wait for ALL resposnes and still returns from the method. So I am missing some responses.
public List<ResultSet> invoke(List<ResultSet> requestPayload) {
String endpoint = "https://ourserver.hostname.com/our-service/api/v1/context?reference={servicereference}"
List<ResultSet> responsePayload = requestPayload;
ArrayList<Mono<String>> asyncRequestList = new ArrayList<>();
/*
"Request" is a Pojo with 4 fields
JsonNode request;
String reference;
JsonNode response;
int order;
*/
for (Requests req : requestPayload) {
asyncRequestList.add(
req.getOrder(),
webClient.post()
.uri(new UriTemplate(endpoint).expand(reference))
.body(BodyInserters.fromValue(req.getRequest()))
.retrieve()
.bodyToMono(String.class)
.timeout(Duration.ofSeconds(20)));
// register the callback
asyncRequestList.get(req.getOrder()).subscribe(s2 -> {
logger.info("Got response :: Request Id : {}, for Reference{}",
req.getRequestId(), req.getReference());
responsePayload.get(req.getOrder()).setResponse(mapper.mapFromString(s2));
});
}
//wait for all responses
Flux.merge(asyncRequestList).collectList().block();
logger.info("Got all responses. ");
return responsePayload;
}
Can anyone point me in right direction here ?
and what tools can I use to debug this ( any network monitors, etc)
The following method will return a list of Strings with status OK.
#Async
#RequestMapping(path = "/device-data/search/asyncCompletable", method = RequestMethod.GET)
public CompletableFuture<ResponseEntity<?>> getValueAsyncUsingCompletableFuture() {
logger.info("** start getValueAsyncUsingCompletableFuture **");
// Get a future of type String
CompletableFuture<List<String>> futureList = CompletableFuture.supplyAsync(() -> processRequest());
return futureList.thenApply(dataList -> new ResponseEntity<List<String>>(dataList, HttpStatus.OK));
}
I would like to make this method more robust by adding the following features:
1) Since fetching the data will take some time ( processRequest())) than in the meanwhile I would like to return status ACCEPTED so there will be some notification for the user.
2) In case the list is empty then I want to return status NO_CONTENT.
How can I add these enhancements in the same method?
Thank you
Even though this is an #Async function, you're waiting for processRequest() to complete and thenApply() function takes the results and set HttpStatus.NO_CONTENT to ResponseEntity despite the results status.
You return that already processed result list, just like a synchronized invoke. I would rather let the client to decide if the List is empty or not, after an immediate response.
#Async
#RequestMapping(path = "/device-data/search/asyncCompletable", method = RequestMethod.GET)
public CompletableFuture<ResponseEntity<List<String>>> getValueAsyncUsingCompletableFuture() {
return CompletableFuture.supplyAsync(() -> ResponseEntity.accepted().body(processRequest()));
}
In the client code, once the get response via RestTemplate of WebClient wait for the response or continue the rest of the tasks.
CompletableFuture<ResponseEntity<List<String>>> response = ... // The response object, should be CompletableFuture<ResponseEntity<List<String>>> type hence returns from the upstream endpoint.
System.out.println(response.get().getBody()); // 'get()' returns a instance of 'ReponseEntity<List<String>>', that's why 'getBody()' invokes.
Well this CompletableFuture.get() is a blocking function and it would wait the current thread until the response arrives. You can continue without blocking for the response until particular response is required in the code.
I am trying to build an application A (like an adaptor) that will:
1) Receive POST requests with some key (JSON format)
2) It should modify that key somehow and create POST request to another system B.
3) Application A should parse the response from application B and modify that response.
4) After that my application A should answer to the initial POST request.
#RestController
#RequestMapping("/A")
public class Controller {
#ResponseStatus(HttpStatus.OK)
#PostMapping(value = "B", consumes = APPLICATION_JSON_VALUE)
// to return nested Flux is a bad idea here
private Flux<Flux<Map<String, ResultClass>>> testUpdAcc(#RequestBody Flux<Map<String, SomeClass>> keys) {
return someMethod(keys);
}
// the problem comes here when I will get Flux<Flux<T>> in the return
public Flux<Flux<Map<String, ResultClass>>> someMethod(Flux<Map<String, SomeClass>> keysFlux) {
return keysFlux.map(keysMap -> {
// do something with keys and create URL
// also will batch keys here
<...>
// for each batch of keys:
WebClient.create(hostAndPort)
.method(HttpMethod.POST)
.uri(url)
.body(BodyInserters.fromObject(body))
.header(HttpHeaders.CONTENT_TYPE, "application/x-www-form-urlencoded")
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(schema) // response will be parsed into some schema here
.retryWhen (// will make a retry mechanism here)
// ===== will join all Mono batches into single Flux
Flux.concat(...);
}
);
}
}
Of course this can be fixed by not reading keysFlux as Flux and read that as Map. But that should make everything less reactive, no? :)
#ResponseStatus(HttpStatus.OK)
#PostMapping(value = "B", consumes = APPLICATION_JSON_VALUE)
// to return nested Flux is a bad idea here
private Flux<Map<String, ResultClass>> testUpdAcc(#RequestBody Map<String, SomeClass> keys) {
return someMethod(keys);
}
Also I have tried to use block()/blockFirst() in the last moment before returning the request, but I have got an error:
block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor...
Thank you for your ideas!
Forget about my question - we can easily use "flatMap" instead of "map".
That will solve a problem with Flux inside Flux.
Try to zip all the flux like this
Flux.zip(flux1,flux2)
It will create Tuple2 so that you can do flatMap
Thanks,
Vimalesh
The application i'm writing performs an initial API call with Retrofit which returns a URL. That response is then .flatMap'd into another API call depending on the text contained in the URL. However, the two secondary API calls are defined to return different response models.
To make things clearer, here is some code:
APIService service = retrofit.create(APIService.class);
service.getURL() // returns response model containing a URL.
.flatMap(new Function<GetURLResponse, ObservableSource<?>>() {
#Override
public ObservableSource<?> apply(GetURLResponse getURLResponse) throws Exception {
// Determine whether the returned url is for "first.com".
if (getURLResponse.url.contains("first.com")) {
return service.first(getURLResponse.url);
}
// Otherwise, the URL is not for "first.com", so use our other service method.
return service.second(getURLResponse.url);
}
})
Here are the interface definitions for service.first() and service.second():
#GET
Observable<retrofit2.Response<ResponseBody>> first(#Url String url);
#GET
Observable<SecondModel> second(#Url String url);
How can I better handle these two different possible types (retrofit2.Response<ResponseBody> and SecondModel) for the rest of the stream? Eg. If the initial URL contains first.com then the service.first() API call should fire, and operators down the stream should received a retrofit2.Response<ResponseBody>. Conversely, if the initial URL does not contain first.com, the service.second() API call should fire and operators down the stream should receive a SecondModel.
The easiest way would be to have both your model classes implement an interface and return that interface, alternatively the models could both extend an abstract class to achieve the same effect. You would then do an instanceOf check to see which model it is and continue with your preferred transformations.
That having said you mentioning downstream operators, makes me think that this would cause an annoying amount of checks. So what I would do is split the stream using the publish operator, and then apply your further transformations to each sub-stream. Finally you should merge the two streams back together and return a single model encompassing both models.
Below a code example to get you started.
Observable<Integer> fooObservableSecondaryRequest(String foo) {
return Observable.just(1);
}
Observable<Integer> booObservableSecondaryRequest(String boo) {
return Observable.just(2);
}
Observable<String> stringObservable = Observable.just("foo", "boo");
stringObservable.publish(shared -> Observable.merge(
shared.filter(a -> a.equals("foo"))
.flatMap(fooString -> fooObservableSecondaryRequest(fooString))
.map(num -> num * 2),
shared.filter(a -> a.equals("boo"))
.flatMap(booString -> booObservableSecondaryRequest(booString))
.map(num -> num * 10)
)).subscribe(result -> System.out.println(result)); // will print 2 and 20