I was wondering how to create "logical stream" using Reactor.
Lets assume that I want to implement following scenario:
As input I have object to save in database. As output I would like to get Mono representing execution message.
Option 1: if object to save has all fields filled then I perform additionall operations, save it to database and finally return "Success" message
Option 2: if object to save has at least one field not filled I return "Error"
I have created such code:
Mono<String> message = Mono.just(new User("Bob the Reactor master")) // user with name = can be saved
.flatMap(user -> {
if(user.getName() != null && user.getName().length() > 1){
// Perform additional operations e.g. user.setCreatedDate(new Date())
// Save to repository e.g. repository.save(user)
return Mono.just("Success!");
}
else{
return Mono.just("Error!");
}
})
.doOnNext(System.out::println); // print stream result
message.subscribe();
Is this code 100% reactive (has all its benefits)? If no then what it will look like?
The answer depends on your commented repository.
Repository is non-blocking and returns Mono or Flux
You should subscribe it then return Success Mono. In your if statement:
return repository.save(user).then(Mono.just("Success!"));
Repository is blocking
You should make your repository call non-blocking moving its execution to separate thread. Reactor way is to wrap it with Mono and subscribe on elastic scheduler or your custom scheduler.
Mono<String> message = Mono.just(new User("Bob the Reactor master"))
.filter(user -> user.getName() != null && user.getName().length() > 1)
.map(user -> user) // Perform additional operations e.g. user.setCreatedDate(new Date())
.flatMap(user -> Mono.fromRunnable(() -> repository.save(user))
.subscribeOn(Schedulers.elastic())
.then(Mono.just("Success!")))
.switchIfEmpty(Mono.just("Error!"));
Related
Wrote a class as below:
class KeyFetcher {
String key = "";
Mono getKeys() {
try {
key = keyService.getKeys();
if(key == null) {
throw new RuntimeException("key value is null");
} else {
this.key = key;
}
} catch(Exception e) {
return Mono.error(e);
}
return Mono.just(key)
}
#Scheduled // spring scheduler
void fetchDataFromExternalService() {
client.fetchData(key) // returns Mono
.retry(3)
.map(this::processResponse)
.doOnError(this::handleError)
.subscribe();
}
}
I want to modify the fetchDataFromExternalService method to include the getKeys() also before calling external service, and retry three times on both methods. Retry will be like:
get keys - try 3 times on failure - then go to handleError and do not call external service.
if key are fetched within 3 retry boundary, then now fetch data from external service with max 3 retries.
if error encountered in client.fetchData(), then go to handleError but not retry getKeys9) again.
I tried below:
Mono.defer(()-> getKeys())
.flatMap(obj -> client.fetchData(key))
.retry(3)
.map(this::processResponse)
.doOnError(this::handleError)
.subscribe();
But the problem is that when getKey() returns response in say 2nd retry, it enters fetchData(), but if this methods throws exception, it enters handleError() before again re-entering getKey() which is not desirable. All I want is once getKey() retry is exhausted, control should enter handleError() and close the Mono. Same for fetchData(). I think I am doing something wrong above, but need suggestion as I am new to this. Thanks.
You are well describing your issue, and if you apply strictly your description to your code, you will get what you need.
In your code sample, you create a pipeline that fetch a key, then fetch data. Therefore, the retry is applied to the entire chain, because it receive a publisher that already chain both actions.
You want:
to fetch a key with 3 max retries:
Mono<Key> getKeyWithRetry = Mono.defer(() -> getKeys()).retry(3);
Fetch data from an input key, with retry:
Function<Key, Mono<Data>> fetchDataWithRetry = key -> fetchData(key).retry(3)
And finally, assemble all that with error handling:
getKeyWithRetry()
.flatMap(fetchDataWithRetry)
.map(this::processResponse)
.doOnError(this::handleError)
.subscribe();
I am using ReactiveCassandraRepository and I can create new record as below.
public Mono<String> saveAbc(Abc toBeSaved) {
return abcRepository.save(toBeSaved).map(saved -> saved.getId());
}
But I could not imagine how to update a specific field in a DB record since 2 reactive operations (findById & save) are involved.
I wrote a code as below to create or update status if exists, but seems to be not working.
public Mono<String> saveAbc(Abc toBeSaved) {
return abcRepository.findById(toBeSaved.getId())
.map(current -> abcRepository.save(transform(toBeSaved, current)).map(saved -> saved.getId()))
.flatMap(id -> id);
}
private Abc transform(Abc toBeSaved, Abc current) {
if(current == null) {
return toBeSaved;
} else {
current.setStatus(toBeSaved.getStatus());
return current;
}
}
Can someone please assist on that?
I expect your abcRepository's method to look something like this:
interface AbcRepository {
Mono<Abc> findById(String id);
Mono<Abc> save(Abc abc);
}
I guess from your code, for a given Abc you want to
read an Abc from repository with the same id,
map the data from the given Abc to the found one,
or just use the given Abc if the repository did not find any,
asynchronously save this Abc
and return the id of the saved element as Mono
I would do it like this:
public Mono<String> saveAbc(Abc toBeSaved) {
return abcRepository.findById(toBeSaved.getId()) // (1)
.map(abc -> transform(toBeSaved, abc)) // (2)
.defaultIfEmpty(toBeSaved) // (3)
.flatMap(abcRepository::save) // (4)
.map(Abc::getId); // (5)
}
private Abc transform(Abc toBeSaved, Abc current) {
current.setStatus(toBeSaved.getStatus());
return current;
}
A Mono can only receive one or no element, so when using Mono:map (2) you don't need to handle null values. The Mono returned by the abcRepository will receive the found Abc in which case the transformation call (2) is done or it will just emit a complete signal in which case the map does nothing and defaultIfEmpty (3) emits toBeSaved as fallback.
If you have a transformation that is asynchronous itself and thus results in another Mono use flatMap (4), else your intermediate result would be an Mono<Mono<Abc>>.
And always remember: nothing happens until subscribe is called.
saveAbc(myNewAbc).subscribe(id -> System.out.println("Saved Abc with id: " + id));
In the above example I expected your repository to just emit a complete signal when findById doesn't find any matching Abc that will complete the Mono as empty (which is the case when using ReactiveCassandraRepository!). If instead the repository emits an exception in this case you can for example use
.onErrorResume(t -> Mono.just(toBeSaved))
instead of defaultIfEmpty (3).
I have a remote call(retrofit) - which I converted into an Observable. Let's call it Observable Y.
Now, I also have a certain code that looks for geo location with GPS and NETWORK providers. I have a Timer there, that basically limits the time that the geo search can be performed for. Let's call it Task X. I want to convert it into an Observable X.
Then, I want to have a subcription, that will perform Observable X(that is, find location), once it will return a Location, I will "analyze" in a certain way, and then I will either pass that Location into the Observable Y(the retrofit call), or simply quit(if that "raw" location will be enough in my case)
At ALL time, I want to be able to interrupt all that "process". From what I gather, I can achieve that by simply unsubscribing the subscription, right?
and then next time just subscribe this subscription once again in the future.
Questions:
1. Can all of that be implemented via RxJava/RxAndroid ?
2. Does it even make sense implementing it with Rx ? or is there a more efficient way?
3. How is it done with Rx?
(More specifically : (a) How do I convert task Y into an Observable Y?
(b) How do I perform them in sequence with only one subscription?)
1- It can be implemented via RxJava
2- This is your best option so far
3-
3-a Observable.fromCallable() does the trick
3-b flatmap operator is used to chain observable calls
you can proceed like this:
private Location searchForLocation() {
// of course you will return not null location
return null;
}
// your task X
//mock your location fetching
private Observable<Location> getLocationObservableX() {
return Observable.fromCallable(() -> searchForLocation());
}
//your task Y
//replace CustomData with simple String
//just to mock your asynchronous retrofit call
private Observable<List<String>> getRetrofitCallObservableY(String param){
return Observable.just(new ArrayList<String>());
}
//subscribe
private void initialize() {
getLocationObservableX()
.filter(location -> {
//place your if else here
//condition
//don't continue tu retrofit
boolean condition = false;
if (condition) {
//process
//quit and pass that Location in Broadcas
//you shall return false if you don't want to continue
return false;
}
return true;
})
//filter operation does not continue here if you return false
.flatMap(location -> getRetrofitCallObservableY("param"))
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(response -> {
//do what you want with response
});
}
The application i'm writing performs an initial API call with Retrofit which returns a URL. That response is then .flatMap'd into another API call depending on the text contained in the URL. However, the two secondary API calls are defined to return different response models.
To make things clearer, here is some code:
APIService service = retrofit.create(APIService.class);
service.getURL() // returns response model containing a URL.
.flatMap(new Function<GetURLResponse, ObservableSource<?>>() {
#Override
public ObservableSource<?> apply(GetURLResponse getURLResponse) throws Exception {
// Determine whether the returned url is for "first.com".
if (getURLResponse.url.contains("first.com")) {
return service.first(getURLResponse.url);
}
// Otherwise, the URL is not for "first.com", so use our other service method.
return service.second(getURLResponse.url);
}
})
Here are the interface definitions for service.first() and service.second():
#GET
Observable<retrofit2.Response<ResponseBody>> first(#Url String url);
#GET
Observable<SecondModel> second(#Url String url);
How can I better handle these two different possible types (retrofit2.Response<ResponseBody> and SecondModel) for the rest of the stream? Eg. If the initial URL contains first.com then the service.first() API call should fire, and operators down the stream should received a retrofit2.Response<ResponseBody>. Conversely, if the initial URL does not contain first.com, the service.second() API call should fire and operators down the stream should receive a SecondModel.
The easiest way would be to have both your model classes implement an interface and return that interface, alternatively the models could both extend an abstract class to achieve the same effect. You would then do an instanceOf check to see which model it is and continue with your preferred transformations.
That having said you mentioning downstream operators, makes me think that this would cause an annoying amount of checks. So what I would do is split the stream using the publish operator, and then apply your further transformations to each sub-stream. Finally you should merge the two streams back together and return a single model encompassing both models.
Below a code example to get you started.
Observable<Integer> fooObservableSecondaryRequest(String foo) {
return Observable.just(1);
}
Observable<Integer> booObservableSecondaryRequest(String boo) {
return Observable.just(2);
}
Observable<String> stringObservable = Observable.just("foo", "boo");
stringObservable.publish(shared -> Observable.merge(
shared.filter(a -> a.equals("foo"))
.flatMap(fooString -> fooObservableSecondaryRequest(fooString))
.map(num -> num * 2),
shared.filter(a -> a.equals("boo"))
.flatMap(booString -> booObservableSecondaryRequest(booString))
.map(num -> num * 10)
)).subscribe(result -> System.out.println(result)); // will print 2 and 20
The Problem
I have two Apis. Api 1 gives me a List of Items and Api 2 gives me more detailed Information for each of the items I got from Api 1. The way I solved it so far results in bad Performance.
The Question
Efficent and fast solution to this Problem with the help of Retrofit and RxJava.
My Approach
At the Moment my Solution Looks like this:
Step 1: Retrofit executes Single<ArrayList<Information>> from Api 1.
Step 2: I iterate through this Items and make a request for each to Api 2.
Step 3: Retrofit Returns Sequentially executes Single<ExtendedInformation> for
each item
Step 4: After all calls form Api 2 completely executed I create a new Object for all Items combining the Information and Extended Information.
My Code
public void addExtendedInformations(final Information[] informations) {
final ArrayList<InformationDetail> informationDetailArrayList = new ArrayList<>();
final JSONRequestRatingHelper.RatingRequestListener ratingRequestListener = new JSONRequestRatingHelper.RatingRequestListener() {
#Override
public void onDownloadFinished(Information baseInformation, ExtendedInformation extendedInformation) {
informationDetailArrayList.add(new InformationDetail(baseInformation, extendedInformation));
if (informationDetailArrayList.size() >= informations.length){
listener.onAllExtendedInformationLoadedAndCombined(informationDetailArrayList);
}
}
};
for (Information information : informations) {
getExtendedInformation(ratingRequestListener, information);
}
}
public void getRatingsByTitle(final JSONRequestRatingHelper.RatingRequestListener ratingRequestListener, final Information information) {
Single<ExtendedInformation> repos = service.findForTitle(information.title);
disposable.add(repos.subscribeOn(Schedulers.io()).observeOn(AndroidSchedulers.mainThread()).subscribeWith(new DisposableSingleObserver<ExtendedInformation>() {
#Override
public void onSuccess(ExtendedInformation extendedInformation) {
ratingRequestListener.onDownloadFinished(information, extendedInformation);
}
#Override
public void onError(Throwable e) {
ExtendedInformation extendedInformation = new ExtendedInformation();
ratingRequestListener.onDownloadFinished(extendedInformation, information);
}
}));
}
public interface RatingRequestListener {
void onDownloadFinished(Information information, ExtendedInformation extendedInformation);
}
tl;dr use concatMapEager or flatMap and execute sub-calls asynchronously or on a schedulers.
long story
I'm not an android developer, so my question will be limited to pure RxJava (version 1 and version 2).
If I get the picture right the needed flow is :
some query param
\--> Execute query on API_1 -> list of items
|-> Execute query for item 1 on API_2 -> extended info of item1
|-> Execute query for item 2 on API_2 -> extended info of item1
|-> Execute query for item 3 on API_2 -> extended info of item1
...
\-> Execute query for item n on API_2 -> extended info of item1
\----------------------------------------------------------------------/
|
\--> stream (or list) of extended item info for the query param
Assuming Retrofit generated the clients for
interface Api1 {
#GET("/api1") Observable<List<Item>> items(#Query("param") String param);
}
interface Api2 {
#GET("/api2/{item_id}") Observable<ItemExtended> extendedInfo(#Path("item_id") String item_id);
}
If the order of the item is not important, then it is possible to use flatMap only:
api1.items(queryParam)
.flatMap(itemList -> Observable.fromIterable(itemList)))
.flatMap(item -> api2.extendedInfo(item.id()))
.subscribe(...)
But only if the retrofit builder is configured with
Either with the async adapter (calls will be queued in the okhttp internal executor). I personally think this is not a good idea, because you don't have control over this executor.
.addCallAdapterFactory(RxJava2CallAdapterFactory.createAsync()
Or with the scheduler based adapter (calls will be scheduled on the RxJava scheduler). It would my preferred option, because you explicitly choose which scheduler is used, it will be most likely the IO scheduler, but you are free to try a different one.
.addCallAdapterFactory(RxJava2CallAdapterFactory.createWithScheduler(Schedulers.io()))
The reason is that flatMap will subscribe to each observable created by api2.extendedInfo(...) and merge them in the resulting observable. So results will appear in the order they are received.
If the retrofit client is not set to be async or set to run on a scheduler, it is possible to set one :
api1.items(queryParam)
.flatMap(itemList -> Observable.fromIterable(itemList)))
.flatMap(item -> api2.extendedInfo(item.id()).subscribeOn(Schedulers.io()))
.subscribe(...)
This structure is almost identical to the previous one execpts it indicates locally on which scheduler each api2.extendedInfo is supposed to run.
It is possible to tune the maxConcurrency parameter of flatMap to control how many request you want to perform at the same time. Although I'd be cautious on this one, you don't want run all queries at the same time. Usually the default maxConcurrency is good enough (128).
Now if order of the original query matter. concatMap is usually the operator that does the same thing as flatMap in order but sequentially, which turns out to be slow if the code need to wait for all sub-queries to be performed. The solution though is one step further with concatMapEager, this one will subscribe to observable in order, and buffer the results as needed.
Assuming retrofit clients are async or ran on a specific scheduler :
api1.items(queryParam)
.flatMap(itemList -> Observable.fromIterable(itemList)))
.concatMapEager(item -> api2.extendedInfo(item.id()))
.subscribe(...)
Or if the scheduler has to be set locally :
api1.items(queryParam)
.flatMap(itemList -> Observable.fromIterable(itemList)))
.concatMapEager(item -> api2.extendedInfo(item.id()).subscribeOn(Schedulers.io()))
.subscribe(...)
It is also possible to tune the concurrency in this operator.
Additionally if the Api is returning Flowable, it is possible to use .parallel that is still in beta at this time in RxJava 2.1.7. But then results are not in order and I don't know a way (yet?) to order them without sorting after.
api.items(queryParam) // Flowable<Item>
.parallel(10)
.runOn(Schedulers.io())
.map(item -> api2.extendedInfo(item.id()))
.sequential(); // Flowable<ItemExtended>
the flatMap operator is designed to cater to these types of workflows.
i'll outline the broad strokes with a simple five step example. hopefully you can easily reconstruct the same principles in your code:
#Test fun flatMapExample() {
// (1) constructing a fake stream that emits a list of values
Observable.just(listOf(1, 2, 3, 4, 5))
// (2) convert our List emission into a stream of its constituent values
.flatMap { numbers -> Observable.fromIterable(numbers) }
// (3) subsequently convert each individual value emission into an Observable of some
// newly calculated type
.flatMap { number ->
when(number) {
1 -> Observable.just("A1")
2 -> Observable.just("B2")
3 -> Observable.just("C3")
4 -> Observable.just("D4")
5 -> Observable.just("E5")
else -> throw RuntimeException("Unexpected value for number [$number]")
}
}
// (4) collect all the final emissions into a list
.toList()
.subscribeBy(
onSuccess = {
// (5) handle all the combined results (in list form) here
println("## onNext($it)")
},
onError = { error ->
println("## onError(${error.message})")
}
)
}
(incidentally, if the order of the emissions matter, look at using concatMap instead).
i hope that helps.
Check below it's working.
Say you have multiple network calls you need to make–cals to get Github user information and Github user events for example.
And you want to wait for each to return before updating the UI. RxJava can help you here.
Let’s first define our Retrofit object to access Github’s API, then setup two observables for the two network requests call.
Retrofit repo = new Retrofit.Builder()
.baseUrl("https://api.github.com")
.addConverterFactory(GsonConverterFactory.create())
.addCallAdapterFactory(RxJavaCallAdapterFactory.create())
.build();
Observable<JsonObject> userObservable = repo
.create(GitHubUser.class)
.getUser(loginName)
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Observable<JsonArray> eventsObservable = repo
.create(GitHubEvents.class)
.listEvents(loginName)
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Used Interface for it like below:
public interface GitHubUser {
#GET("users/{user}")
Observable<JsonObject> getUser(#Path("user") String user);
}
public interface GitHubEvents {
#GET("users/{user}/events")
Observable<JsonArray> listEvents(#Path("user") String user);
}
After we use RxJava’s zip method to combine our two Observables and wait for them to complete before creating a new Observable.
Observable<UserAndEvents> combined = Observable.zip(userObservable, eventsObservable, new Func2<JsonObject, JsonArray, UserAndEvents>() {
#Override
public UserAndEvents call(JsonObject jsonObject, JsonArray jsonElements) {
return new UserAndEvents(jsonObject, jsonElements);
}
});
Finally let’s call the subscribe method on our new combined Observable:
combined.subscribe(new Subscriber<UserAndEvents>() {
...
#Override
public void onNext(UserAndEvents o) {
// You can access the results of the
// two observabes via the POJO now
}
});
No more waiting in threads etc for network calls to finish. RxJava has done all that for you in zip().
hope my answer helps you.
I solved a similar problem with RxJava2. Execution of requests for Api 2 in parallel slightly speeds up the work.
private InformationRepository informationRepository;
//init....
public Single<List<FullInformation>> getFullInformation() {
return informationRepository.getInformationList()
.subscribeOn(Schedulers.io())//I usually write subscribeOn() in the repository, here - for clarity
.flatMapObservable(Observable::fromIterable)
.flatMapSingle(this::getFullInformation)
.collect(ArrayList::new, List::add);
}
private Single<FullInformation> getFullInformation(Information information) {
return informationRepository.getExtendedInformation(information)
.map(extendedInformation -> new FullInformation(information, extendedInformation))
.subscribeOn(Schedulers.io());//execute requests in parallel
}
InformationRepository - just interface. Its implementation is not interesting for us.
public interface InformationRepository {
Single<List<Information>> getInformationList();
Single<ExtendedInformation> getExtendedInformation(Information information);
}
FullInformation - container for result.
public class FullInformation {
private Information information;
private ExtendedInformation extendedInformation;
public FullInformation(Information information, ExtendedInformation extendedInformation) {
this.information = information;
this.extendedInformation = extendedInformation;
}
}
Try using Observable.zip() operator. It will wait until both Api calls are finished before continuing the stream. Then you can insert some logic by calling flatMap() afterwards.
http://reactivex.io/documentation/operators/zip.html