Observable and Flowable interfaces seem to be identical. Why Flowable was introduced in RxJava 2.0? When should I prefer to use Flowable over Observable?
As stated in the documentation:
A small regret about introducing backpressure in RxJava 0.x is that
instead of having a separate base reactive class, the Observable
itself was retrofitted. The main issue with backpressure is that many
hot sources, such as UI events, can't be reasonably backpressured and
cause unexpected MissingBackpressureException (i.e., beginners don't
expect them).
We try to remedy this situation in 2.x by having
io.reactivex.Observable non-backpressured and the new
io.reactivex.Flowable be the backpressure-enabled base reactive class.
Use Observable when you have relatively few items over time (<1000) and/or there's no risk of producer overflooding consumers and thus causing OOM.
Use Flowable when you have relatively large amount of items and you need to carefully control how Producer behaves in order to to avoid resource exhaustion and/or congestion.
Backpressure
When you have an observable which emits items so fast that consumer can’t keep up with the flow leading to the existence of emitted but unconsumed items.
How unconsumed items, which are emitted by observables but not consumed by subscribers, are managed and controlled is what backpressure strategy deals with.
Ref link
Related
It seems like mental overhead to use both Observable and Flowable in the same project for RXJava 2.
Say to distinguish in the interfaces that this method return Observable so there is not back-pressure this one does so it is with back-pressure support.
But in the end of the day we can just use Flowable to make things simpler?
Then:
Can I make Flowable to be faster to turn off pressure to behave same way as Observable does?
My answer to myself: use Flowable everywhere. For all API that exposed themselves through the Net.
My use case: I use RX Java to defining RX API to the Web and itneracting with other microservices by Net.
From my understanding, the projectreactor https://projectreactor.io/ (another rx iml) project there is even no question whether to go with backpressure or not. It is there for all Flux
Going back to RxJava2. It seems to me that in the RX Java it was decided to keep Observable for the backward compatibility purposes primary. (there is no good explanation why not)
And when it comes to HTTP I found that I need it even for smaller sets. Usually. Event if I return less than 1000 items to the client it does not mean that it can handle it at once (it can be busy doing other stuff), so I would go with backpressure.
So I do not won't to overcomplicate things / api / code, will go with Flowable everywhere until I see any of Observable.
The main benefit of reactive programming is that it is fault-tolerant and can process a lot more events than a blocking implementation, despite the fact that the processing will usually happen slower.
What I don't fully understand is how and where the events are stored. I know that there is an event buffer and it can be tweaked but that buffer can easily overload the memory if the queue is unbound, can't it? Can this buffer flush onto disk? Isn't it a rist to have it in-memory? Can it be configured similarly to Lagom event-sourcing or persistent Akka actors where events can be stored in DB?
The short answer is no, this buffer cannot be persisted. At least in reference implementation.
The internal in-memory buffer can hold up to 128 emited values by default, but there are some points. First of all, there is a backpressure — situatuion when the source emits items faster than observer or operator consumes them. Thus, when this internal buffer is overloaded you get a MissingBackpressureException and there are no any disk or some other way to persist it. However you can tweak the behavior, for instance keep only latest emit or just drop new emits. There are special operators for that — onBackpressureBuffer, onBackpressureDrop, onBackpressureLatest.
RxJava2 introduces a new type — Flowable which supports backpressure by default and gives more ways to tweak internal buffer.
Rx is a way to process data streams and you should care if you can consume all the items and how to store them if you can't.
One of the main advantages of rxjava is contract and there are ways to create your own operators or use some extensions like rxjava-extras
I have been looking at new rx java 2 and I'm not quite sure I understand the idea of backpressure anymore...
I'm aware that we have Observable that does not have backpressure support and Flowable that has it.
So based on example, lets say I have flowable with interval:
Flowable.interval(1, TimeUnit.MILLISECONDS, Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Consumer<Long>() {
#Override
public void accept(Long aLong) throws Exception {
// do smth
}
});
This is going to crash after around 128 values, and thats pretty obvious I am consuming slower than getting items.
But then we have the same with Observable
Observable.interval(1, TimeUnit.MILLISECONDS, Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Consumer<Long>() {
#Override
public void accept(Long aLong) throws Exception {
// do smth
}
});
This will not crash at all, even when I put some delay on consuming it still works. To make Flowable work lets say I put onBackpressureDrop operator, crash is gone but not all values are emitted either.
So the base question I can not find answer currently in my head is why should I care about backpressure when I can use plain Observable still receive all values without managing the buffer? Or maybe from the other side, what advantages do backpressure give me in favour of managing and handling the consuming?
What backpressure manifests in practice is bounded buffers, Flowable.observeOn has a buffer of 128 elements that gets drained as fast as the dowstream can take it. You can increase this buffer size individually to handle bursty source and all the backpressure-management practices still apply from 1.x. Observable.observeOn has an unbounded buffer that keeps collecting the elements and your app may run out of memory.
You may use Observable for example:
handling GUI events
working with short sequences (less than 1000 elements total)
You may use Flowable for example:
cold and non-timed sources
generator like sources
network and database accessors
Backpressure is when your observable (publisher) is creating more events than your subscriber can handle. So you can get subscribers missing events, or you can get a huge queue of events which just leads to out of memory eventually. Flowable takes backpressure into consideration. Observable does not. Thats it.
it reminds me of a funnel which when it has too much liquid overflows. Flowable can help with not making that happen:
with tremendous backpressure:
but with using flowable, there is much less backpressure :
Rxjava2 has a few backpressure strategies you can use depending on your usecase. by strategy i mean Rxjava2 supplies a way to handle the objects that cannot be processed because of the overflow (backpressure).
here are the strategies.
I wont go through them all, but for example if you want to not worry about the items that are overflowed you can use a drop strategy like this:
observable.toFlowable(BackpressureStrategy.DROP)
As far as i know there should be a 128 item limit on the queue, after that there can be a overflow (backpressure). Even if its not 128 its close to that number. Hope this helps someone.
if you need to change the buffer size from 128 it looks like it can be done
like this (but watch any memory constraints:
myObservable.toFlowable(BackpressureStrategy.MISSING).buffer(256); //but using MISSING might be slower.
in software developement usually back pressure strategy means your telling the emitter to slow down a bit as the consumer cannot handle the velocity your emitting events.
The fact that your Flowable crashed after emitting 128 values without backpressure handling doesn't mean it will always crash after exactly 128 values: sometimes it will crash after 10, and sometimes it will not crash at all. I believe this is what happened when you tried the example with Observable - there happened to be no backpressure, so your code worked normally, next time it may not. The difference in RxJava 2 is that there is no concept of backpressure in Observables anymore, and no way to handle it. If you're designing a reactive sequence that will probably require explicit backpressure handling - then Flowable is your best choice.
I'm unclear on how to implement task cancellation in RXJava.
I'm interested in porting an existing API built using Guava's ListenableFuture. My use case is as follows:
I have an single operation that's composed of a sequence of futures joined by Futures.transform()
Multiple subscribers observe the operation's final future.
Each observer can cancel the final future, and all observers witness the cancellation event.
Cancellation of the final future results in the cancellation of its dependencies, e.g. in sequence 1->2->3, cancellation of 3 is propagated to 2, and so on.
There's very little info in the RxJava wiki about this; the only references I can find to cancellation mention Subscription as an equivalent to .NET's Disposable, but as far as I can see, Subscription only offers the ability to unsubscribe from subsequent values in the sequence.
I'm unclear on how to implement "any subscriber can cancel" semantics through this API. Am I thinking about this in the wrong way?
Any input would be appreciated.
It's important to learn about Cold vs Hot Observables. If your Observables are cold, then their operations will not execute if you have no subscribers. Hence to "cancel", just make sure all Observers unsubscribe from the source Observable.
However, if only one Observer of the source unsubscribes, and there are other Observers still subscribed to the source, this will not incur a "cancelling". In that case you can use (but it's not the only solution) ConnectableObservables. Also see this link about Rx.NET.
A practical way of using ConnectableObservables is to simply call .publish().refCount() on any cold Observable. What that does is create one single "proxy" Observer which relays the events from the source to the actual Observers. The proxy Observer unsubscribes when the last actual Observer unsubscribes.
To manually control a ConnectableObservable, call just coldSource.publish() and you will get an instance of ConnectableObservable. Then you can call .connect() which will return you the Subscription of the "proxy" Observer. To manually "cancel" the source, you just unsubscribe the Subscription of the proxy Observer.
For your specific problem, you can also use the .takeUntil() operator.
Suppose your "final future" is ported as finalStream in RxJava, and suppose that "cancel events" are Observables cancelStream1, cancelStream2, etc, then it becomes fairly simple to "cancel" operations resulting from finalStream:
Observable<FooBar> finalAndCancelableStream = finalStream
.takeUntil( Observable.merge(cancelStream1, cancelStream2) );
In diagrams, this is how takeUntil works, and this is how merge works.
In plain english, you can read it as "finalAndCancelableStream is the finalStream until either cancelStream1 or cancelStream2 emit an event".
I am trying to figure out if I can switch from a blocking scenario to a more reactive pattern.
I have incoming update commands arriving in a queue, and I need to handle them in order, but only those regarding the same entity. In essence, I can create as many parallel streams of update events as I wish, as long as no two streams contain events regarding the same entity.
I was thinking that the consumer of the primary queue would possibly be able to leverage amqp's routing mechanisms, and temporary queues, by creating temporary queues for each entity id, and hooking a consumer to them. Once the subscriber is finished and no other events regarding the entity in question are currently in the queue, the queue could be disposed of.
Is this scenario something that is used regularly? Is there a better way to achieve this? In our current system we use a named lock based on the id to prevent concurrent updates.
There are at least 2 Options:
A single queue for each entity
And n Consumers on one Entity-Queue.
One queue with messages of all entities. Where the message contains data what it is for an entity. You could than split this up into several queues (One AMQP-Queue for one type of entity) or by using a BlockingQueue implementation.
Benefits of splitting up the Entities in qmqp-queues
You could create an ha-setup with rabbitmq
You could route messages
You could maybe have more than one consumer of an entity queue if it
is necessary someday (scalability)
Messages could be persistent and therefore recoverable on an
application-crash
Benefits of using an internal BlockingQueue implementation
It is faster (no net-io obviously)
Everything has to happen in one JVM
Anyway it does depend on what you want since both ways could have their benefits.
UPDATE:
I am not sure if I got you now, but let me give you some resources to try some things out.
There are special rabbitmq extensions maybe some of them can give you an idea. Take a look at alternate exchanges and exchange to exchange bindings.
Also for basic testing, I am not sure if it covers all rabbitmq features or at all all amqp features but this can sometimes be usefull. Keep in mind the routing key in this visualization is the producer name, you can also find there some examples. Import and Export your configuration.