The below is my code snippet, and sometimes the Observable does not time out,
Observable<A> AObservable = Observable.fromCallable(() ->
//External Service Call
).timeout(800, TimeUnit.MILLISECONDS)
.subscribeOn(Schedulers.io())
.onErrorReturn(throwable -> {
LOGGER.warn(format("Server did not respond within %s ms for id=%s", 800, id));
return null;
});
Observable<B> BObservable = Observable.fromCallable(() ->
//External Service Call
).timeout(800, TimeUnit.MILLISECONDS)
.subscribeOn(Schedulers.io())
.onErrorReturn( throwable -> {
LOGGER.warn(format("Service did not respond within %s ms for id=%s", 800, Id));
return null;
});
// Build Default response
Observable<C> CObservable = Observable.fromCallable(() ->
// Build Default one
).subscribeOn(Schedulers.io());
return Observable.zip(AObservable, BObservable,CObservable,
(AResponse, BResponse, CResponse) -> {
// Handle response and combine them
}).toBlocking().first();
It appears to me that at times the if the service takes more than 800ms the timeout does not happen. Do I miss any attribute here please advise.
Related
I have the following code where I call external APIs via webclient and return Mono.
I need to execute some logic when I receive data. And after all, requests are processed, execute one logic for all gathered data. I can collect all Monos and put them to flux and then execute some logic at the end. But I have serviceName filed which is accessible only in the loop, so I need to execute logic for mono in loop and here I'm stuck and don't know how to wait for all data to complete and do it in a reactive way.
#Scheduled(fixedDelay = 50000)
public void refreshSwaggerConfigurations() {
log.debug("Starting Service Definition Context refresh");
List<SwaggerServiceData> allServicesApi = new ArrayList<>();
swaggerProperties.getUrls().forEach((serviceName, serviceSwaggerUrl) -> {
log.debug("Attempting service definition refresh for Service : {} ", serviceName);
Mono<SwaggerServiceData> swaggerData = getSwaggerDefinitionForAPI(serviceName,
serviceSwaggerUrl);
swaggerData.subscribe(swaggerServiceData -> {
if (swaggerServiceData != null) {
allServicesApi.add(swaggerServiceData);
String content = getJSON(swaggerServiceData);
definitionContext.addServiceDefinition(serviceName, content);
} else {
log.error("Skipping service id : {} Error : Could not get Swagger definition from API ",
serviceName);
}
});
});
//I need to wait here for all monos to complete and after that proceed for All gathered data...
//Now it's empty And I know why, just don't know how to make it.
Optional<SwaggerServiceData> swaggerAllServicesData = getAllServicesApiSwagger(allServicesApi);
if (swaggerAllServicesData.isPresent()) {
String allApiContent = getJSON(swaggerAllServicesData.get());
definitionContext.addServiceDefinition("All", allApiContent);
}
}
private Mono<SwaggerServiceData> getSwaggerDefinitionForAPI(String serviceName, String url) {
log.debug("Accessing the SwaggerDefinition JSON for Service : {} : URL : {} ", serviceName,
url);
Mono<SwaggerServiceData> swaggerServiceDataMono = webClient.get()
.uri(url)
.exchangeToMono(clientResponse -> clientResponse.bodyToMono(SwaggerServiceData.class));
return swaggerServiceDataMono;
}
I would add a temporary class to group data and serivce name :
record SwaggerService(SwaggerServiceData swaggerServiceData, String serviceName) {
boolean hasData() {
return swaggerServiceData != null;
}
}
And then change your pipeline :
Flux.fromStream(swaggerProperties.getUrls().entrySet().stream())
.flatMap((e) -> {
Mono<SwaggerServiceData> swaggerDefinitionForAPI = getSwaggerDefinitionForAPI(e.getKey(),
e.getValue());
return swaggerDefinitionForAPI.map(swaggerServiceData -> new SwaggerService(swaggerServiceData, e.getKey()));
})
.filter(SwaggerService::hasData)
.map(swaggerService -> {
String content = getJSON(swaggerService.swaggerServiceData());
definitionContext.addServiceDefinition(swaggerService.serviceName(), content);
return swaggerService.swaggerServiceData();
})
// here we will collect all datas and they will be emmited as single Mono with list of SwaggerServiceData
.collectList()
.map(this::getAllServicesApiSwagger)
.filter(Optional::isPresent)
.map(Optional::get)
.subscribe(e -> {
String allApiContent = getJSON(e);
definitionContext.addServiceDefinition("All", allApiContent);
});
This does not deal with logging error when SwaggerServiceData is null but you can further change it if you want. Also I assume that DefinitionContext is thread safe.
Solution with error logging (using flatMap and Mono.empty()) :
Flux.fromStream(swaggerProperties.getUrls().entrySet().stream())
.flatMap((e) -> {
Mono<SwaggerServiceData> swaggerDefinitionForAPI = getSwaggerDefinitionForAPI(e.getKey(),
e.getValue());
return swaggerDefinitionForAPI
.flatMap(swaggerServiceData -> {
if(swaggerServiceData != null) {
return Mono.just(new SwaggerService(swaggerServiceData, e.getKey()));
} else {
log.error("Skipping service id : {} Error : Could not get Swagger definition from API ",
e.getKey());
return Mono.empty();
}
});
})
.map(swaggerService -> {
String content = getJSON(swaggerService.swaggerServiceData());
definitionContext.addServiceDefinition(swaggerService.serviceName(), content);
return swaggerService.swaggerServiceData();
}).collectList()
.map(this::getAllServicesApiSwagger)
.filter(Optional::isPresent)
.map(Optional::get)
.subscribe(e -> {
String allApiContent = getJSON(e);
definitionContext.addServiceDefinition("All", allApiContent);
});
You can also wrap those lambads into some meaningful methods to improve readibility.
I'd like to do the following using the WebClient from spring webflux:
Call endpoint1
If it fails with an expected error then
call endpoint2 and
retry endpoint1 only once
I've got this far:
webclient.get()
.uri("/endpoint1")
.retrieve()
.bodyToFlux(MyBody.class)
.retry(error -> {
if (error == expectedError) {
webclient.get()
.uri("/endpoint2")
.retrieve().block();
return true;
} else {
false;
});
I cannot block when requesting endpoint2 since I would get the following error: block()/blockFirst()/blockLast() are blocking, which is not supported in thread (I wouldn't like to block either).
Maybe I should use retryWhen but I'm not really sure how to use it.
The only way I made this work was with retryWhen I could not use reactor.retry.Retry#doOnRetry because it only accepts a Consumer not a Mono or Flux or Publisher.
The snippet is as follows:
webclient.get()
.uri("/endpoint1")
.retrieve()
.bodyToFlux(MyBody.class)
.retryWhen(errorCurrentAttempt -> errorCurrentAttempt
.flatMap(currentError -> Mono.subscriberContext().map(ctx -> Tuples.of(currentError, ctx)))
.flatMap(tp -> {
Context ctx = tp.getT2();
Throwable error = tp.getT1();
int maxAttempts = 3;
Integer rl = ctx.getOrDefault("retriesLeft", maxAttempts);
if (rl != null && rl > 0 && error == myExpectedError) {
// Call endpoint and retry
return webclient.get()
.uri("/endpoint2")
.retrieve()
.thenReturn(ctx.put("retriesLeft", rl - 1));
} else {
// Finish retries
return Mono.<Object>error(error);
}
}));
I have defined completableFuture.supplyAsync() inside foreach loop, so each entry(each asynchronous task) adding a list and I need to get final list(after all asynchronous task adding list)from completableFuture.supplyAsync().How to achieve this?
Code snippet:
unporcessedList.forEach(entry -> {
CompletableFuture<List<ChangeLog>> cf =
CompletableFuture.supplyAsync((Supplier<List<ChangeLog>>) () -> {
mongoDBHelper.processInMongo(entry, getObject(entry, map),entryList);
return entryList;
}, executor);
});
Non blocking version
General example:
List<String> entries = new ArrayList<>(2);
entries.add("first");
entries.add("second");
List<CompletableFuture<String>> completableFutures = entries.stream()
.map((entry) -> {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(new Random().nextInt(5000) + 500);
} catch (InterruptedException e) {
e.printStackTrace();
}
return entry.concat(String.valueOf(entry.length()));
}).thenApply((e) -> new StringBuilder(e).reverse().toString());
}
).collect(Collectors.toList());
CompletableFuture
.allOf(completableFutures.toArray(new CompletableFuture[completableFutures.size()]))
.thenApply((v) -> completableFutures.stream().map((cf) -> cf.join()))
.get()
.forEach(System.out::println);
Your case:
List<CompletableFuture<List<ChangeLog>>> completableFutures = unporcessedList.stream()
.map((entry) -> {
return CompletableFuture.supplyAsync((Supplier<List<ChangeLog>>) () -> {
mongoDBHelper.processInMongo(entry, getObject(entry, map), entryList);
return entryList;
}, executor);
}
).collect(Collectors.toList());
CompletableFuture
.allOf(completableFutures.toArray(new CompletableFuture[completableFutures.size()]))
.thenApply((v) -> completableFutures.stream().map((cf) -> cf.join()))
.get()
.forEach(System.out::println);
You can use the get() Method that will block your application until the future is completed. So use something like this:
// Block and get the result of the Future
Supplier<List<ChangeLog>> result = cf.get();
More examples are described here: https://www.callicoder.com/java-8-completablefuture-tutorial/
Hope this helps.
I found below code(modified for brevity), which is buggy, and has flaws as far as I can tell.
Its using toBlocking() which is not recommended in general, and has been used within reactive context
It returns single(); so as soon as it gets the single item from the stream, it terminates it, ignoring the rest of the items which is not desirable.
While I believe I can solve the issue, by removing single to last? Can someone explain, how can I get about removing the use of toBlocking() here?
observableList.map(incentiveDetailsList -> {
List<SomeObject> list = mapThisList(incentiveDetailsList);
return Observable.just(list)
.flatMap(Observable::from)
.flatMap(item -> {
Request request = createRequest(item);
String accountNumber = item.getAccountNumber();
return serviceThatReturnsObservable.load(request)
.doOnError(onError -> {
Observable.error(new Exception("some context"));
})
.map(response -> {
handleError(response);
return responseMap.put(accountNumber, buildResponse(response.getResponse()));
});
})
.map(resp -> mapResponse(store, incentiveDetailsList, responseMap))
.toBlocking()
.single();
})
You can replace map + toBlocking with flatMap (or concatMap):
observableList.flatMap(incentiveDetailsList -> {
List<SomeObject> list = mapThisList(incentiveDetailsList);
return Observable.from(list)
.flatMap(item -> {
Request request = createRequest(item);
String accountNumber = item.getAccountNumber();
return serviceThatReturnsObservable.load(request)
/* this has no effect:
.doOnError(onError -> {
Observable.error(new Exception("some context"));
})
*/
.map(response -> {
handleError(response);
return responseMap.put(accountNumber,
buildResponse(response.getResponse()));
});
})
.map(resp -> mapResponse(store, incentiveDetailsList, responseMap));
})
I have following code written in RXJava.
validateProduct(payload)
.map.(r -> {
if(r.getBoolean("valid")){
return createProduct(productPayload);
}else{
return null; // request.end() | end the chain here with some message as invalid product.
}
})
.map(r -> {
return linkCategories(catPayload);
})
.map(r -> {
return linkTags(tagPayload);
})
.doOnError(e -> log.error(e))
.subscribe(r -> {
JsonObject response = new JsonObject().put("status", true);
request.end(response);
}, e -> {
JsonObject response = new JsonObject().put("status", false);
request.end(response);
});
The first block has a condition check, this code is not working right now. Whats the best way to handle conditional chains in RX?
It looks like you are probably running into a null pointer exception. nulls are not acceptable in RxJava v2. Your first map is likely causing problems.
Generally when you need conditional logic in rxjava and may not be returning an object you have two options:
Return an object that signifies null (and possibly filter it out)
Use an operator like flatMap and add on an empty Observable
It looks like you are probably running into a null pointer exception. nulls are not acceptable in RxJavav2. Your first map is likely causing problems.
Option 1.
validateProduct(payload)
.map.(r -> {
if(r.getBoolean("valid")){
return createProduct(productPayload);
}else{
return createEmptyProduct(); // generate non null placeholder object
}
})
.filter(r->{
// check here via method call or instanceOf to filter out empty products
r instanceof ValidProduct
}).map(r -> {
return linkCategories(catPayload);
})
.map(r -> {
return linkTags(tagPayload);
})
.doOnError(e -> log.error(e))
.subscribe(r -> {
JsonObject response = new JsonObject().put("status", true);
request.end(response);
}, e -> {
JsonObject response = new JsonObject().put("status", false);
request.end(response);
});
Option 2
validateProduct(payload)
.flatMap(r -> {
if(r.getBoolean("valid")){
return createProduct(productPayload); // Assuming this returns an observable if not use Observable.just(createProduct(productPayload))
}else{
return Observable.empty(); // request.end() | end the chain here with some message as invalid product.
}
})
.map(r -> {
return linkCategories(catPayload);
})
.map(r -> {
return linkTags(tagPayload);
})
.doOnError(e -> log.error(e))
.subscribe(r -> {
JsonObject response = new JsonObject().put("status", true);
request.end(response);
}, e -> {
JsonObject response = new JsonObject().put("status", false);
request.end(response);
});