BTW I'm still learning weblux;
I don't know if this is possible or I have the wrong approach but given this parallel flux.
Flux<String> enablers = Flux.fromIterable(enablersList)
.parallel()
.runOn(Schedulers.elastic())
.flatMap(element -> service.getAMono(string, entity, element))
.sequential();
who calls a method that has a webclient request (service.getAMono)
webClient.post()
.uri(url)
.headers(headers -> headers.addAll(httpHeaders))
.body(BodyInserters.fromObject(request))
.retrieve()
.bodyToMono(entity2.class);
I need to wait for enablers flux's flow ends and process all the responses inside it, the reason is if one of them gives me error or a negative response i won't run this other Parallel Flux for blockers
Flux<String> blockers = Flux.fromIterable(blockersList)
.parallel()
.runOn(Schedulers.elastic())
.flatMap(element -> service.callAMono(string, entity, element))
.sequential();
I though about "zip" method, but this one merge both response and is not what I want
If anybody could help me with this.
UPDATE
enablers. //handle enablers response and if error return a custom Mono<response> with .reduce
And if no error in the handle of enablers proceed to the .thenMany with the other Flux
I found the way to do it by conditional any in the first flux, like this
Flux.fromIterable(enablersList)
.parallel()
.runOn(Schedulers.elastic())
.flatMap(element -> service.getAMono(string, entity, element))
.sequential()
.any(element -> *stuff here)//condition
.flatMap(condition->{
if(condition.equals(Boolean.FALSE)){
return Flux.fromIterable(blockersList)
.parallel()
.runOn(Schedulers.elastic())
.flatMap(element -> service.callAMono(string, entity, element))
.sequential()
.reduce(**stuff here)// handle noError response and return;
}
return Mono.just(**stuff here);//handle error response and return
});
If there is another way to do this please I'll be glad you post it here Thanks, :D
Related
I have the following method:
public void countLetters() {
Flux.just("alpha", "bravo", "charlie")
.map(String::toUpperCase)
.flatMap(s -> Flux.fromArray(s.split("")))
.groupBy(String::toString)
.sort(Comparator.comparing(GroupedFlux::key))
.flatMap(group -> Mono.just(group.key()).and(group.count()))
.map(keyAndCount ->
keyAndCount.getT1() + " => " + keyAndCount.getT2())
.subscribe(System.out::println);
}
It gives me these errors:
Cannot resolve method 'getT1' in 'Void'
Cannot resolve method 'getT2' in 'Void'
I'm not sure what this means. Is it because my method is void or is it some other reason?
Instead of
.flatMap(group -> Mono.just(group.key()).and(group.count()))
You need
.map(group -> new Tuple2<>(group.key(), group.count()))
The issues in your flatMap:
It doesn't need to be a flat map at all. You are mapping your response in a "1 to 1" fashion without any further reactive streams, so the Mono.just isn't needed.
Mono.and joins 2 Monos and will produce a new Mono<Void> which is a mono that only completes (when all joined monos complete), meaning it has no result (doesn't emit any values). This means that .map and .flatMap on this mono will accept a Void as param (main reason for your compile time error). Not just that, but they will never be called, as Mono<Void> don't emit any Void values, they just complete (similar to Completable from RxJava).
For example, if you have
Mono<File> downloadFile = ...;
Mono<Long> calculateNumber = ...;
and you perform
Mono<Void> test = downloadFile.and(calculateNumber);
You would create a new Mono that completes when both monos complete, but throws away both values!
So:
test.map(v -> {
// v is type of "Void", and this map will never be called!
System.out.println("this will never be printed!");
return 0;
}).doFinally((signalType) -> {
System.out.println("this will be called!");
}).subscribe(
v -> System.out.println("wont be called, nothing is emitted!"),
err -> System.out.println("this might be called, if the stream emits an error"),
() -> System.out.println("this will be called, as its a completion handler!")
);
The mono test will terminate/complete after both monos joined with and complete.
If you really want a flatMap with Mono.just-ing the values of the tuple, you could use Mono.zip.
Tuple2 Cannot be used directly.
So,
Instead of
.flatMap(group -> Mono.just(group.key()).and(group.count()))
You need
.flatMap(group -> Mono.just(group.key()).zipWith(group.count()))
zipWith function will combine the result from this mono and another into a Tuple2.
I have a fully reactive web app that aggregates the information from two other backend-services.
Incoming request -> sends request to service A and B -> aggregates responses -> response is emitted.
pseudocode:
public Mono<ResponseEntity<List<String>>> getValues() {
return Mono.zip(getValuesA(), getValuesB(),
(a, b) -> Stream.concat(a.stream(), b.stream()).collect(Collectors.toList()))
.map(result -> ResponseEntity.ok(result));
}
public Mono<String> getValuesA() {
return webClient.get()
.uri(uriA)
.retrieve()
.bodyToMono(new ParameterizedTypeReference<>() {});
}
// getValuesB same as A, but with uriB.
Because of the high request frequency, I want to bundle requests to the backend-services. I thought using Sinks would be the right way to go. A sink is returned as mono to every requesting party. After a threshold of 10 requests has been exceeded, the request will be handled and the response will be emitted to every sink.
public Mono<ResponseEntity<List<String>>> getValues() {
return Mono.zip(getValuesA(), getValuesB(),
(a, b) -> Stream.concat(a.stream(), b.stream()).collect(Collectors.toList()))
.map(result -> ResponseEntity.ok(result));
}
public Mono<String> getValuesA() {
Sink.One<List<String>> sink = Sinks.one();
queue.add(sink);
if(queue.size() > 10) {
webClient.get()
.uri(uriA)
.retrieve()
.bodyToMono(new ParameterizedTypeReference<>() {})
.subscribe(response -> {
for(Sink.One<List<String>> sinkItem : queue) {
sink.tryEmitValue(response);
}
});
}
return sink.asMono();
}
// getValuesB same as A, but with uriB.
The problem in this code is the 'subscribe' part. As soon as we're subscribing to the webclient's response, it will block the thread. This will only happen in 10% of the requests, but this is already too much for an endpoint that's being called very frequently. What can I do to 'unblock' this part. If using sinks wasn't the best choice, what could have been a better one?
PS. All pseudocode used is NOT production code. It may have many flaws and it is only meant to visualize the problem I'm facing at this moment.
Because of the high request frequency, I want to bundle requests to the backend-services. I thought using Sinks would be the right way to go.
You shouldn't need a sink to do this at all - assuming a Flux as input, you should be able to do this in 3 steps with a standard reactive chain:
Buffer the input with a length of 10, which transforms your Flux<Foo> into a Flux<List<Foo>> where each element is a list of size 10 (or lower than 10 if the flux completes with fewer than 10 remaining elements);
Flatmap to a zipped mono which contains the original list, the "A" web service response given the list, and the "B" web service response given the list;
Implement a method (let's call it expand()) which takes the original list of 10 items, the A service response, and the B service response, and then splits it out into a flux of multiple items. Flatmap to this method.
The end result would be a reactive chain that looked something like:
input.buffer(10)
.flatMap(list -> Mono.zip(Mono.just(list), getResponseFromA(list), getResponseFromB(list)))
.flatMap(response -> expand(response.getT1(), response.getT2(), response.getT3()))
I want to:
subscribe to multiple endpoints returning Flux and output the messages I receive.
wait until all messages have been output from all endpoints before continuing.
avoid processing messages from multiple endpoints "together" (e.g. Flux.zip) because the endpoints will return an uneven number of messages and are not logically connected to each other
block forever if one or more endpoints generate an infinite number of messages
The following code satisfies 1 and 3 but not 2 and 4:
Stream.of("http://service1.com", "http://service2.com", "http://service3.com")
.forEach(service -> {
webClient.get()
.retrieve()
.bodyToFlux(String.class)
.map(message -> service + ": " + message)
.subscribe(System.out::println);
});
System.out.println("Received all messages");
The line "Received all messages" should not be printed until all endpoints have finished. However, because subscribe is asynchronous, that line is instead printed almost immediately and my application continues instead of waiting.
What should I do differently?
I believe the following code snippet achieves 3 out of 4 points in your question though I do not feel like I fully understand the 3rd requirement. Let me if this example meets what is needed and if not, what is missing.
Flux.just("http://service1.com", "http://service2.com", "http://service3.com")
.flatMap(url -> webClient.get()
.uri(url)
.retrieve()
.bodyToFlux(String.class)
.map(body -> url + ":" + body)
)
.collectList()
.doOnNext(list -> LOG.info("Received all messages"))
.subscribe(list -> LOG.info("" + list));
flatMap is one way to merge fluxes together but you can also use Flux.merge
List<Flux<String>> individualResults =
Stream.of("http://service1.com", "http://service2.com", "http://service3.com")
.map(
service ->
webClient //
.get()
.retrieve()
.bodyToFlux(String.class))
.collect(toList());
Flux<String> mergedResults = Flux.merge(individualResults); // Will not complete until all individual Fluxes have completed.
mergedResults //
.doOnNext(System.out::println)
.then()
.block(); // block this thread until mergedResults completes
System.out.println("Received all messages");
I have a java webclient code , the response of which I convert to Mono. I want to iterate on the api call until the Mono response matches certain condition. Of course I do not want to iterate till infinity. I want to iterate after every 5 seconds until 30 seconds. So far I have tried this
client.get()
.uri("https://someUri")
.retrieve()
.bodyToMono(Response.class)
.delayElement(Duration.ofSeconds(5))
.retryBackoff(5, Duration.ofSeconds(5))
.delayUntil(r -> {
System.out.print("Looping");
if(condition) {
System.out.print(r.getStatus());
return Mono.just(r);
}
return Mono.empty();
})
But no use.
You can use a filter, repeatWhenEmpty and Repeat like so
client.get()
.uri("https://someUri")
.retrieve()
.bodyToMono(Response.class)
.filter(response -> condition)
.repeatWhenEmpty(Repeat.onlyIf(r -> true)
.fixedBackoff(Duration.ofSeconds(5))
.timeout(Duration.ofSeconds(30)))
The Repeat class is part of the reactor-extra library
<dependency>
<groupId>io.projectreactor.addons</groupId>
<artifactId>reactor-extra</artifactId>
</dependency>
I have a series of thenCompose calls, similar to
myObject.updateDB(payload)
.thenCompose(__ -> getUserID(payload.ID()))
.thenCompose(id -> getProfile(id))
.thenCompose(userProfile -> updateSomething(userProfile))
.thenCompose(__ -> notifyUser(id))
.thenAccept(__ -> doSomething())
.exceptionally(t -> doSomethingElse());
The getUserID call returns a CompletionStage<String> which I use in the next call for getProfile. I need the same id again for the notifyUser call. How to make it available there? The IDE is showing
Cannot resolve symbol id.
The issue with your current code is that by the time you reach .thenCompose(__ -> notifyUser(id)), the variable id is not in scope anymore.
A simple solution in this case would be to invoke multiple thenCompose directly on the CompletionStage returned by getProfile:
myObject.updateDB(payload)
.thenCompose(__ -> getUserID(payload.ID()))
.thenCompose(id ->
getProfile(id)
.thenCompose(userProfile -> updateSomething(userProfile))
.thenCompose(__ -> notifyUser(id))
)
// rest of chain calls
I think, your code becomes simpler, if you don’t insist on using thenCompose for every step:
myObject.updateDB(payload)
.thenCompose(__ -> getUserID(payload.ID()))
.thenAccept(id -> {
updateSomething(getProfile(id).join());
notifyUser(id);
})
.thenRun(() -> doSomething())
.exceptionally(t -> doSomethingElse());
If having each step effectively sequential is your requirement, you can simply use join:
myObject.updateDB(payload)
.thenCompose(__ -> getUserID(payload.ID()))
.thenAccept(id -> {
updateSomething(getProfile(id).join()).join();
notifyUser(id).join();
})
.thenRun(() -> doSomething())
.exceptionally(t -> doSomethingElse());
Considering that the whole chain is effectively sequential, you may just write it straight-forward:
myObject.updateDB(payload)
.thenRun(() -> {
YourUserIDType id = getUserID(payload.ID()).join();
updateSomething(getProfile(id).join()).join();
notifyUser(id).join();
doSomething();
})
.exceptionally(t -> doSomethingElse());