I started to play around with RxJava and ReactFX, and I became pretty fascinated with it. But as I'm experimenting I have dozens of questions and I'm constantly researching for answers.
One thing I'm observing (no pun intended) is of course lazy execution. With my exploratory code below, I noticed nothing gets executed until the merge.subscribe(pet -> System.out.println(pet)) is called. But what fascinated me is when I subscribed a second subscriber merge.subscribe(pet -> System.out.println("Feed " + pet)), it fired the "iteration" again.
What I'm trying to understand is the behavior of the iteration. It does not seem to behave like a Java 8 stream that can only be used once. Is it literally going through each String one at a time and posting it as the value for that moment? And do any new subscribers following any previously fired subscribers receive those items as if they were new?
public class RxTest {
public static void main(String[] args) {
Observable<String> dogs = Observable.from(ImmutableList.of("Dasher", "Rex"))
.filter(dog -> dog.matches("D.*"));
Observable<String> cats = Observable.from(ImmutableList.of("Tabby", "Grumpy Cat", "Meowmers", "Peanut"));
Observable<String> ferrets = Observable.from(CompletableFuture.supplyAsync(() -> "Harvey"));
Observable<String> merge = dogs.mergeWith(cats).mergeWith(ferrets);
merge.subscribe(pet -> System.out.println(pet));
merge.subscribe(pet -> System.out.println("Feed " + pet));
}
}
Observable<T> represents a monad, a chained operation, not the execution of the operation itself. It is descriptive language, rather than the imperative you're used to. To execute an operation, you .subscribe() to it. Every time you subscribe a new execution stream is created from scratch. Do not confuse streams with threads, as subscription are executed synchronously unless you specify a thread change with .subscribeOn() or .observeOn(). You chain new elements to any existing operation/monad/Observable to add new behaviour, like changing threads, filtering, accumulation, transformation, etc. In case your observable is an expensive operation you don't want to repeat on every subscription, you can prevent recreation by using .cache().
To make any asynchronous/synchronous Observable<T> operation into a synchronous inlined one, use .toBlocking() to change its type to BlockingObservable<T>. Instead of .subscribe() it contains new methods to execute operations on each result with .forEach(), or coerce with .first()
Observables are a good tool because they're mostly* deterministic (same inputs always yield same outputs unless you're doing something wrong), reusable (you can send them around as part of a command/policy pattern) and for the most part ignore concurrence because they should not rely on shared state (a.k.a. doing something wrong). BlockingObservables are good if you're trying to bring an observable-based library into imperative language, or just executing an operation on an Observable that you have 100% confidence it's well managed.
Architecting your application around these principles is a change of paradigm that I can't really cover on this answer.
*There are breaches like Subject and Observable.create() that are needed to integrate with imperative frameworks.
Related
I still do not understand when to apply this method. In fact, it is similar to Mono.just, but I heard that callback is used for heavy operations if it needs to be performed separately from other flows. Now I use it like this, but is it correct.
Here is an example of use, I wrap sending a firebase notification in a callback since the operation is long
#Override
public Mono<NotificationDto> sendMessageAllDevice(NotificationDto notification) {
return Mono.fromCallable(() -> fcmProvider.sendPublicMessage(notification))
.thenReturn(notification);
}
maybe I still had to wrap up here in Mono.just ?
It depends which thread you want fcmProvider.sendPublicMessage(...) to be run on.
Either the one currently executing sendMessageAllDevice(...):
T result = fcmProvider.sendPublicMessage(notification);
return Mono.just(result);
Or the one(s) the underlying mono relies on:
Callable<T> callable = () -> fcmProvider.sendPublicMessage(notification);
return Mono.fromCallable(callable);
I would guess you need the latter approach.
If you use Mono.just(computeX()), computeX() is called immediately. No want you want(I guess).
If you use Mono.fromCallable(() -> computeX()), the computation is still not performed. I mean computeX() is only called when you subscribe to it. Maybe using .map, .flatMap, etc.
Important: if computeX() return Mono you doe not need to use Mono.fromCallable. It's only for blocking code
As you explained in the description, Mono.fromCallable is used when you want to compute a result with an async execution (mostly some heavy operation).
Since, you have already generated the Mono with Mono.fromCallable you do not have to wrap it again with Mono.just.
Suppose that we need to transform a hot Observable in a way that we need to know all of its previously emitted items to be able to determine what to emit next. The solution which I find the most convenient is to pass an instance of a Func1 subclass, which has a global state (e.g. a map or list of previously emitted items) to flatMap. In each call, the Func1 instance would update its state and based on that, decide what to return.
However, I am worried about the "niceness" of this solution. As far as I know, RxJava does not go well with global and mutable state, which this solution seems to be in contrast with. On the other hand, I am sure that my Observable fulfills the Observable contract, so it seems to be at least a working solution, and event if it could be called concurrently, a synchronization would solve the problem.
Other possible solutions could be:
Creating an Operator. Mutable state in Operators is allowed, I guess. Anyways, I try to avoid custom operators, as they are more tricky.
Propagating the history of the Observable through scan (in a List or Map). I would either use the same object (List or Map) for every emitted item, which introduces a mutable object into the stream, or copy the entire object every time, which would waste a lot of performance.
Subscribe to the original Observable, modify some global state from the subscriber, and emit items on a Subject (the transformed Observable) using this global state. I thought about this because it seems to exit the scope of RxJava when it deals with the global state (and synchronization).
So the question is: Should I use the Func1 implementation with mutable state in flatMap for transforming items based on the history of previously emitted items (which works, btw), and if not, what alternatives should I use? In general, I am confused about the recommended way to handle a complex mutable state needed for the transformation of Observables.
I hope I have expressed my problem clearly. Otherwise, let me know and I will try to describe it with the help of some specific problems and code.
Flows with functions containing mutable state are generally not recommended as the mutable state could be potentially shared across multiple Subscribers to a particular Observable chain. Often though, most developers assemble Observables when needed and rarely ever reuse the same Observable. For example, a button click handler will create an Observable that, through composition, forks off two other Observables to get data from two different places asynchronously, and then subscribe to this thread-local Observable instance. A new button click will repeat the process with a fresh and independent Observable.
Here lies the solution to your stateful-function problem: make the existence of the stateful bits depend on the individual Subscribers subscribing: defer()
Observable<Integer> o = Observable.defer(() -> {
return Observable.range(1, 10)
.map(new Func1<Integer, Integer>() {
int sum;
#Override
public Integer call(Integer v) {
sum += v;
return sum;
}
});
});
o.subscribe(System.out::println);
o.subscribe(System.out::println);
Since the Func1 inner class will be created for each of the subscribe call, its sum field will be local to each individual consumer. Note also that sum is returned and auto-boxed into an immutable Integer which then can be freely read after in some other thread (think observeOn) as it is then completely detached of the sum field then on.
Mutable state and shared, mutable state often are required for useful work. The issue is how well we isolate the mutability from outside parties.
Creating an operator hides the mutability within the operator instance. The downside is that the state is private to the observable chain.
scan(), reduce() and fold() (if it existed) would be good candidates, but they have very limited implementations, export their state in non-obvious ways and are also limited to the observable chain they are attached to.
Subject or Relay objects provide useful cut-out points.
Going back to basics, using a privately accessible data structure in thread-safe ways is not a bad thing. If you are only concerned about the one observer chain, then either of options 1 or 3 will do the job readily.
In Reactive Java, we're told that the .subscribe() call returns "a Subscription reference". But Subscription is an interface, not a class. So what kind of object are we handed that implements this interface? Do we have any control over this?
There is the class Subscriptions that can create and return several different kinds of Subscription, but what does one do with them? If I write
Subscription mSub = Subscriptions.create(<some Action0>);
mSub = someObservable.subscribe();
won't my just-created Subscription simply be overwritten by whatever the .subscribe() call returns? How do you use a Subscription you create?
(On a somewhat related note, what is the point of Subscriptions.unsubscribed(), which "returns a Subscription to which unsubscribe does nothing, as it is already unsubscribed. Huh?)
Short answer: You shouldn't care.
Longer answer: a subscription gives you two methods:
unsubscribe(), which causes the subscription to terminate.
isUnsubscribed(), which checks whether that has already happened.
You can use these methods to a) check whether an Observable chain terminated and b) to cause it to terminate prematurely, for example if the user switched to a different Activity.
That's it. You aren't exposed to the internals on purpose. Also, do you notice that there's no resubscribe method? That's because if you want to restart the operation, you need to resubscribe to the Observable, giving you a new Subscription.
As you know Subscriptions are used to keep references to ongoing Observables, mainly for resources' management. For example in Android applications, when you change an Activity (screen) you flush old Activity Observables. In this scenario, Subscription instances are given by .subscribe() (as you mentioned) and stored. So, for which reason would one create a Subscription directly, especially Subscriptions.unsubscribed()? I encountered two cases:
Default implementation; avoid declaration like Subscription mSub; that would be filled latter and could create an NPE. It's especially true if you use Kotlin that require property initialization.
Testing
On a somewhat related note, what is the point of Subscriptions.unsubscribed(), which "returns a Subscription to which unsubscribe does nothing, as it is already unsubscribed. Huh?
In 1.x, Subscriptions.unsubscribed() is used to return a Subscription instance the operation was completed (or never run in the first place) when the control is returned to your code from RxJava. Since being unsubscribed is stateless and a constant state, the returned Subscription is a singleton because just by looking at the interface Subscription there is no (reasonable) way to distinguish one completed/unsubscribed Subscription from another.
In 2.x, there is a public and internal version of its equivalent interface, Disposable. The internal version is employed mostly to swap out a live Disposable with a terminated one, avoiding NullPointerException and null checks in general and to help the GC somewhat.
what does one do with them?
Usually you don't need to worry about Subscriptions.create(); it is provided for the case you have a resource you'd like to attach to the lifecycle of your end-subscriber:
FileReader file = new FileReader ("file.txt");
readLines(file)
.map(line -> line.length())
.reduce(0, (a, b) -> a + b)
.subscribe(new Subscriber<Integer>() {
{
add(Subscriptions.create(() -> {
Closeables.closeSilently(file); // utility from Guava
});
}
#Override public void onNext(Integer) {
// process
}
// onError(), onCompleted()
});
This example, demonstrating one way of usage, can be expressed via using instead nonetheless:
Observable.using(
() -> new FileReader("file.txt"), // + try { } catch { }
file -> readLines(file).map(...).reduce(...),
file -> Closeables.closeSilently(file)
)
.subscribe(...)
TL;DR
How to convert Task.whenAll(List<Task>) into RxJava?
My existing code uses Bolts to build up a list of asynchronous tasks and waits until all of those tasks finish before performing other steps. Essentially, it builds up a List<Task> and returns a single Task which is marked as completed when all tasks in the list complete, as per the example on the Bolts site.
I'm looking to replace Bolts with RxJava and I'm assuming this method of building up a list of async tasks (size not known in advance) and wrapping them all into a single Observable is possible, but I don't know how.
I've tried looking at merge, zip, concat etc... but can't get to work on the List<Observable> that I'd be building up as they all seem geared to working on just two Observables at a time if I understand the docs correctly.
I'm trying to learn RxJava and am still very new to it so forgive me if this is an obvious question or explained in the docs somewhere; I have tried searching. Any help would be much appreciated.
You can use flatMap in case you have dynamic tasks composition. Something like this:
public Observable<Boolean> whenAll(List<Observable<Boolean>> tasks) {
return Observable.from(tasks)
//execute in parallel
.flatMap(task -> task.observeOn(Schedulers.computation()))
//wait, until all task are executed
//be aware, all your observable should emit onComplete event
//otherwise you will wait forever
.toList()
//could implement more intelligent logic. eg. check that everything is successful
.map(results -> true);
}
Another good example of parallel execution
Note: I do not really know your requirements for error handling. For example, what to do if only one task fails. I think you should verify this scenario.
It sounds like you're looking for the Zip operator.
There are a few different ways of using it, so let's look at an example. Say we have a few simple observables of different types:
Observable<Integer> obs1 = Observable.just(1);
Observable<String> obs2 = Observable.just("Blah");
Observable<Boolean> obs3 = Observable.just(true);
The simplest way to wait for them all is something like this:
Observable.zip(obs1, obs2, obs3, (Integer i, String s, Boolean b) -> i + " " + s + " " + b)
.subscribe(str -> System.out.println(str));
Note that in the zip function, the parameters have concrete types that correspond to the types of the observables being zipped.
Zipping a list of observables is also possible, either directly:
List<Observable<?>> obsList = Arrays.asList(obs1, obs2, obs3);
Observable.zip(obsList, (i) -> i[0] + " " + i[1] + " " + i[2])
.subscribe(str -> System.out.println(str));
...or by wrapping the list into an Observable<Observable<?>>:
Observable<Observable<?>> obsObs = Observable.from(obsList);
Observable.zip(obsObs, (i) -> i[0] + " " + i[1] + " " + i[2])
.subscribe(str -> System.out.println(str));
However, in both of these cases, the zip function can only accept a single Object[] parameter since the types of the observables in the list are not known in advance as well as their number. This means that that the zip function would have to check the number of parameters and cast them accordingly.
Regardless, all of the above examples will eventually print 1 Blah true
EDIT: When using Zip, make sure that the Observables being zipped all emit the same number of items. In the above examples all three observables emitted a single item. If we were to change them to something like this:
Observable<Integer> obs1 = Observable.from(new Integer[]{1,2,3}); //Emits three items
Observable<String> obs2 = Observable.from(new String[]{"Blah","Hello"}); //Emits two items
Observable<Boolean> obs3 = Observable.from(new Boolean[]{true,true}); //Emits two items
Then 1, Blah, True and 2, Hello, True would be the only items passed into the zip function(s). The item 3would never be zipped since the other observables have completed.
Of the suggestions proposed, zip() actually combines observable results with each other, which may or may not be what is wanted, but was not asked in the question. In the question, all that was wanted was execution of each of the operations, either one-by-one or in parallel (which was not specified, but linked Bolts example was about parallel execution). Also, zip() will complete immediately when any of the observables complete, so it's in violation of the requirements.
For parallel execution of Observables, flatMap() presented in the other answer is fine, but merge() would be more straight-forward. Note that merge will exit on error of any of the Observables, if you rather postpone the exit until all observables have finished, you should be looking at mergeDelayError().
For one-by-one, I think Observable.concat() static method should be used. Its javadoc states like this:
concat(java.lang.Iterable> sequences)
Flattens an Iterable of Observables into one Observable, one after the other, without interleaving them
which sounds like what you're after if you don't want parallel execution.
Also, if you're only interested in the completion of your task, not return values, you should probably look into Completable instead of Observable.
TLDR: for one-by-one execution of tasks and oncompletion event when they are completed, I think Completable.concat() is best suited. For parallel execution, Completable.merge() or Completable.mergeDelayError() sounds like the solution. The former one will stop immediately on any error on any completable, the latter one will execute them all even if one of them has an error, and only then reports the error.
With Kotlin
Observable.zip(obs1, obs2, BiFunction { t1 : Boolean, t2:Boolean ->
})
It's important to set the type for the function's arguments or you will have compilation errors
The last argument type change with the number of argument :
BiFunction for 2
Function3 for 3
Function4 for 4
...
You probably looked at the zip operator that works with 2 Observables.
There is also the static method Observable.zip. It has one form which should be useful for you:
zip(java.lang.Iterable<? extends Observable<?>> ws, FuncN<? extends R> zipFunction)
You can check out the javadoc for more.
I'm writing some computation heave code in Kotlin with JavaRx Observables and RxKotlin. I want to observe a list of observables to be completed and in the meantime giving me an update with the progress and latest result. At the end it returns the best calculation result. An extra requirement was to run Observables in parallel for using all my cpu cores. I ended up with this solution:
#Volatile var results: MutableList<CalculationResult> = mutableListOf()
fun doALotOfCalculations(listOfCalculations: List<Calculation>): Observable<Pair<String, CalculationResult>> {
return Observable.create { subscriber ->
Observable.concatEager(listOfCalculations.map { calculation: Calculation ->
doCalculation(calculation).subscribeOn(Schedulers.computation()) // function doCalculation returns an Observable with only one result
}).subscribeBy(
onNext = {
results.add(it)
subscriber.onNext(Pair("A calculation is ready", it))
},
onComplete = {
subscriber.onNext(Pair("Finished: ${results.size}", findBestCalculation(results))
subscriber.onComplete()
},
onError = {
subscriber.onError(it)
}
)
}
}
I had similar problem, I needed to fetch search items from rest call while also integrate saved suggestions from a RecentSearchProvider.AUTHORITY and combine them together to one unified list. I was trying to use #MyDogTom solution, unfortunately there is no Observable.from in RxJava. After some research I got a solution that worked for me.
fun getSearchedResultsSuggestions(context : Context, query : String) : Single<ArrayList<ArrayList<SearchItem>>>
{
val fetchedItems = ArrayList<Observable<ArrayList<SearchItem>>>(0)
fetchedItems.add(fetchSearchSuggestions(context,query).toObservable())
fetchedItems.add(getSearchResults(query).toObservable())
return Observable.fromArray(fetchedItems)
.flatMapIterable { data->data }
.flatMap {task -> task.observeOn(Schedulers.io())}
.toList()
.map { ArrayList(it) }
}
I created an observable from the array of observables that contains lists of suggestions and results from the internet depending on the query. After that you just go over those tasks with flatMapIterable and run them using flatmap, place the results in array, which can be later fetched into a recycle view.
If you use Project Reactor, you can use Mono.when.
Mono.when(publisher1, publisher2)
.map(i-> {
System.out.println("everything is done!");
return i;
}).block()
Background
I have a number of RxJava Observables (either generated from Jersey clients, or stubs using Observable.just(someObject)). All of them should emit exactly one value. I have a component test that mocks out all the Jersey clients and uses Observable.just(someObject), and I see the same behaviour there as when running the production code.
I have several classes that act upon these observables, perform some calculations (& some side-effects - I might make them direct return values later) and return empty void observables.
At one point, in one such class, I'm trying to zip several of my source observables up and then map them - something like the below:
public Observable<Void> doCalculation() {
return Observable.zip(
getObservable1(),
getObservable2(),
getObservable3(),
UnifyingObject::new
).concatMap(unifyingObject -> unifyingObject.processToNewObservable())
}
// in Unifying Object
public Observable<Void> processToNewObservable() {
// ... do some calculation ...
return Observable.empty();
}
The calculating classes are then all combined and waited on:
// Wait for rule computations to complete
List<Observable<Void>> calculations = ...;
Observable.zip(calculations, results -> results)
.toBlocking().lastOrDefault(null);
The problem
The trouble is, processToNewObservable() is never being executed. By process of elimination, I can see it's getObservable1() that's the trouble - if I replace it with Observable.just(null), everything's executed as I'd imagine (but with a null value where I want a real one).
To reiterate, getObservable1() returns an Observable from a Jersey client in production code, but that client is a Mockito mock returning Observable.just(someValue) in my test.
Investigation
If I convert getObservable1() to blocking, then wrap the first value in just(), again, everything executes as I'd imagine (but I don't want to introduce the blocking step):
Observable.zip(
Observable.just(getObservable1().toBlocking().first()),
getObservable2(),
getObservable3(),
UnifyingObject::new
).concatMap(unifyingObject -> unifyingObject.processToNewObservable())
My first thought was that perhaps something else was consuming the value emitted from my observable, and zip was seeing that it was already complete, thus determining that the result of zipping them should be an empty observable. I've tried adding .cache() onto every observable source I can think is related, however, but that hasn't altered the behaviour.
I've also tried adding next / error / complete / finally handlers on getObservable1 (without converting it to blocking) just before the zip, but none of them executed either:
getObservable1()
.doOnNext(...)
.doOnCompleted(...)
.doOnError(...)
.finallyDo(...);
Observable.zip(
getObservable1(),
getObservable2(),
getObservable3(),
UnifyingObject::new
).concatMap(unifyingObject -> unifyingObject.processToNewObservable())
The question
I'm very new to RxJava, so I'm pretty sure I'm missing something fundamental. The question is: what stupid thing could I be doing? If that's not obvious from what I've said so far, what can I do to help diagnose the issue?
The Observable must emit to start the chain. You have to think of your pipeline as a declaration of what will happen when the Observable emits.
You didn't share what was actually being observed, but Observable.just() causes the Observable to emit the wrapped object immediately.
Based on the response in the comment, either one of the getObservable doesn't return any value but just completes or the Mockito mocking does something wrong. The following standalone example works for me. Could you check it and start slowly mutating it to see where things break?
Observable.zip(
Observable.just(1),
Observable.just(2),
Observable.just(3),
(a, b, c) -> new Integer[] { a, b, c })
.concatMap(a -> Observable.from(a))
.subscribe(System.out::println)
;
Note: I didn't find my answer here very satisfying, so I dug in a bit further and found a much smaller reproduction case, so I've asked a new question here: Why does my RxJava Observable emit only to the first consumer?
I've figured out at least part of my troubles (and, apologies to all who tried to answer, I don't think you stood much of a chance given my explanation).
The various classes which perform these calculations were all returning Observable.empty() (as per processToNewObservable() in my original example). As far as I can tell, Observable.zip() doesn't subscribe to the Nth observable it's zipping until the N-1th observable has emitted a value.
My original example claimed it was getObservable1() that was misbehaving - that was actually slight inaccurate, it was a later observable in the parameter list. As I understand it, the reason making it blocking, then turning that value into an Observable again worked is because making it blocking and calling first forced its execution, and I got the side-effects I wanted.
If I change all my calculating classing to return Observable.just(null) instead, everything works: the final zip() of all the calculation classes' observables works through them all, so all the expected side-effects happen.
Returning a null Void seems like I'm definitely Doing Something Wrong from a design point of view, but at least this particular question is answered.