I'm trying to implement a class that emits changes using an Observable.
When a subscription is done to this observable I want to send an starting/initialization event. Then I want to send the usual events.
For example. Lets say I have two different subscribers A and B. A and B starts subscribing at different times. If MyClass.getChanges() emits event no. 1,2,3,4 and 5.
If A starts it subscription between event 1,2 then it should receive the following events:
InitialEvent, 2, 3, 4, 5.
If B starts it subscription between event 4 and 5, then B should receive the following events:
InitialEvent, 5.
How to do this using RxJava?
Thanks!
Edit 1
I think I need to explain that the "InitialEvent" is different each time it's emitted. It's calculated by MyClass each time a new subscriber starts to subscribe from getChanged().
My scenario is that MyClass contains a list. The "initialEvent" contains the list at the moment when the subscription is done. Then each change to this list is emitted from getChanges().
Sorry to post this 2 years later, but I had the same need and found this question unanswered.
What I did is the following:
public Observable<Event> observe() {
return Observable.defer(() ->
subject.startWith(createInitialEvent())
);
}
The idea is the following:
defer() executes the passed-in lambda expression when an observer subscribes to the Observable returned by the method observe(). So basically, it executes subject.startWith(...), which returns an Observable that is the actual source of event for the subscriber.
subject.startWith(...) emits an initial event (specified by startWith(...)) followed by those emitted by the subject.
So, if I come back to the original post:
if an observer starts it subscription between event 1,2 then it should receive the following events: InitialEvent, 2, 3, 4, 5.
What you're looking for is PublishSubject. Subjects are hot Observables, in that they do not wait for Observers to subscribe to them before beginning to emit their items. Here's a bit of info on Subjects.
Here's a quick demo of your use-case
PublishSubject<String> subject = PublishSubject.create();
Observable<String> InitEvent = Observable.just("init");
Observable<String> A = subject.asObservable();
Observable<String> B = subject.asObservable();
subject.onNext("1");
A.startWith(InitEvent)
.subscribe(s -> System.out.println("A: " + s));
subject.onNext("2");
subject.onNext("3");
subject.onNext("4");
B.startWith(InitEvent)
.subscribe(s -> System.out.println("B: " + s));
subject.onNext("5");
Possibly not really elegant way how about just using a flag? It looks like you just want to replace the first emitted event.
e.g. for one subscription the following logic:
boolean firstTimeA = true;
myCustomObservable.subscribe(s -> {
System.out.println(firstTimeA ? "initEvent" : s.toString());
if(firstTimeA) firstTimeA = false;
});
And since you want to have a second subscription just create a firstTimeB and update it your B subscription.
If I understand what you are asking something like this should work for you
int last = 0;
Observable obs;
List<Integer> list = new ArrayList<>();
public SimpleListObservable() {
obs = Observable.create(new Observable.OnSubscribe<Integer>() {
#Override
public void call(Subscriber<? super Integer> subscriber) {
while(last < 30) {
last++;
list.add(last);
subscriber.onNext(last);
}
subscriber.onCompleted();
}
});
}
public Observable<Integer> process() {
return Observable.from(list).concatWith(obs);
}
As the source observable collects values they are added to the List (you can transform the items as you see fit, filter them out, etc) and then when ObserverB subscribes it will get a replay of the items already collected in the List before continuing with the source observable output.
This simple test should demonstrate the outcome
public void testSequenceNext() {
final SimpleListObservable obs = new SimpleListObservable();
final Observer<Integer> ob2 = Mockito.mock(Observer.class);
obs.process().subscribe(new Observer<Integer>() {
#Override
public void onCompleted() {
ob1Complete = true;
}
#Override
public void onError(Throwable e) {
e.printStackTrace();
}
#Override
public void onNext(Integer integer) {
System.out.println("ob1: " + integer);
if (integer == 20) {
obs.process().subscribe(ob2);
}
}
});
ArgumentCaptor<Integer> captor = ArgumentCaptor.forClass(Integer.class);
Mockito.verify(ob2, Mockito.times(30)).onNext(captor.capture());
for (Integer value : captor.getAllValues()) {
System.out.println(value);
}
}
What do you think of this, I've made part of my API of course as I'm on a phone :
public class StreamOfSomething {
new StreamOfSomething() {
// source of events like
events = Observable.range(0, 1_000_000_000)
.doOnNext(set::add) // some operation there
.map(Event::change)
.publish()
.refCount();
}
public Observable<Event> observeChanges() {
return events.startWith(
Observable.just(Event.snapshot(set))); // start stream with generated event
}
}
And the client can do something like :
Observable.timer(2, 4, TimeUnit.SECONDS)
.limit(2)
.flatMap(t -> theSourceToWatch.observeChanges().limit(10))
.subscribe(System.out::println);
Note however if you are in a multithreaded environment you may have to synchronize when you are subscribing to block any modification, otherwise the list may change before it get's emitted. Or rework this class completely around observables, I don't know yet how to achieve this though.
Related
I need to chain sequentially in order Vertx CompositeFutures in a RxJava style for dependent CompositeFuture, avoiding callback hell.
The use case:
Each CompositeFuture.any/all do some async operations that return futures, lets say myList1, myList2, myList3, but I must wait for CompositeFuture.any(myList1) to complete and return success before doing CompositeFuture.any(myList2), and the same from myList2 to myList3. Naturally, the CompositeFuture itself does the jobs async, but just for its set of operations, since the next set have to be done just after the first set goes well.
Doing it in a "callback-hell style" would be:
public static void myFunc(Vertx vertx, Handler<AsyncResult<CompositeFuture>> asyncResultHandler) {
CompositeFuture.any(myList1 < Future >)
.onComplete(ar1 -> {
if (!ar1.succeeded()) {
asyncResultHandler.handle(ar1);
} else {
CompositeFuture.any(myList2 < Future >)
.onComplete(ar2 -> {
if (!ar2.succeeded()) {
asyncResultHandler.handle(ar2);
} else {
CompositeFuture.all(myList3 < Future >)
.onComplete(ar3 -> {
asyncResultHandler.handle(ar3);
.... <ARROW OF CLOSING BRACKETS> ...
}
Now I tried somenthing like this:
public static void myFunc(Vertx vertx, Handler<AsyncResult<CompositeFuture>> asyncResultHandler) {
Single
.just(CompositeFuture.any(myList1 < Future >))
.flatMap(previousFuture -> rxComposeAny(previousFuture, myList2 < Future >))
.flatMap(previousFuture -> rxComposeAll(previousFuture, myList3 < Future >))
.subscribe(SingleHelper.toObserver(asyncResultHandler));
}
public static Single<CompositeFuture> rxComposeAny(CompositeFuture previousResult, List<Future> myList) {
if (previousResult.failed()) return Single.just(previousResult); // See explanation bellow
CompositeFuture compositeFuture = CompositeFuture.any(myList);
return Single.just(compositeFuture);
}
public static Single<CompositeFuture> rxComposeAll(CompositeFuture previousResult, List<Future> myList) {
if (previousResult.failed()) return Single.just(previousResult);
CompositeFuture compositeFuture = CompositeFuture.any(myList);
return Single.just(compositeFuture);
}
}
Much more compact and clear. But, I am not succeeding in passing the previous fails to the asyncResultHandler.
My idea was as follows: The flatMap passes the previous CompositeFuture result and I want to check if it failed. The next rxComposeAny/All first checks to see if previous failed, if so, just returns the failed CompositeFuture and so on until it hits the handler in the subscriber. If the previous passed the test, I`m ok to continue passing the current result till the last successful CompositeFuture hits the handler.
The problem is that the check
if (previousResult.failed()) return Single.just(previousResult); // See explanation bellow
doesn't work, and all the CompositeFutures are processed, but not tested for successful completion, just the last one ends up being passed to the asyncResultHandler which will test for overall failure (but in the case of my code, it ends up cheking just the last one)
I`m using Vertx 3.9.0 and RxJava 2 Vertx API.
Disclosure: I have experience in Vertx, but I'm totally new in RxJava. So I appreciate any answer, from technical solutions to conceptual explanations.
Thank you.
EDIT (after excellent response of #homerman):
I need to have the exact same behavior of the "callback hell style" of sequentially dependent CompositeFutures, ie, the next must be called after onComplete and test for completed with failure or success. The complexity comes from the fact that:
I have to use vertx CompositeAll/Any methods, not zip. Zip provides behaviour similar to CompositeAll, but not CompositeAny.
CompositeAll/Any return the completed future just inside onComplete method. If I check it before as showed above, since it is async, I will get unresolved futures.
CompositeAll/Any if failed will not throw error, but failed future inside onComplete, so I cannot use onError from rxJava.
For example, I tried the following change in the rxComposite function:
public static Single<CompositeFuture> rxLoadVerticlesAny(CompositeFuture previousResult, Vertx vertx, String deploymentName,
List<Class<? extends Verticle>> verticles, JsonObject config) {
previousResult.onComplete(event -> {
if (event.failed()) {
return Single.just(previousResult);
} else {
CompositeFuture compositeFuture = CompositeFuture.any(VertxDeployHelper.deploy(vertx, verticles, config));
return Single.just(compositeFuture);
}
}
);
}
But naturally it does not compile, since lambda is void. How can I reproduce this exact same behavior it rxJava in Vertx?
Just to clarify something...
Each CompositeFuture.any/all do some async operations that return
futures, lets say myList1, myList2, myList3, but I must wait for
CompositeFuture.any(myList1) to complete and return success before
doing CompositeFuture.any(myList2), and the same from myList2 to
myList3.
You've offered CompositeFuture.any() and CompositeFuture.all() as points of reference, but the behavior you describe is consistent with all(), which is to say the resulting composite will yield success only if all its constituents do.
For the purpose of my answer, I'm assuming all() is the behavior you expect.
In RxJava, an unexpected error triggered by an exception will result in termination of the stream with the underlying exception being delivered to the observer via the onError() callback.
As a small demo, assume the following setup:
final Single<String> a1 = Single.just("Batch-A-Operation-1");
final Single<String> a2 = Single.just("Batch-A-Operation-2");
final Single<String> a3 = Single.just("Batch-A-Operation-3");
final Single<String> b1 = Single.just("Batch-B-Operation-1");
final Single<String> b2 = Single.just("Batch-B-Operation-2");
final Single<String> b3 = Single.just("Batch-B-Operation-3");
final Single<String> c1 = Single.just("Batch-C-Operation-1");
final Single<String> c2 = Single.just("Batch-C-Operation-2");
final Single<String> c3 = Single.just("Batch-C-Operation-3");
Each Single represents a discrete operation to be performed, and they are logically named according to some logical grouping (ie they are meant to be executed together). For example, "Batch-A" corresponds to your "myList1", "Batch-B" to your "myList2", ...
Assume the following stream:
Single
.zip(a1, a2, a3, (s, s2, s3) -> {
return "A's completed successfully";
})
.flatMap((Function<String, SingleSource<String>>) s -> {
throw new RuntimeException("B's failed");
})
.flatMap((Function<String, SingleSource<String>>) s -> {
return Single.zip(c1, c2, c3, (one, two, three) -> "C's completed successfully");
})
.subscribe(
s -> System.out.println("## onSuccess(" + s + ")"),
t -> System.out.println("## onError(" + t.getMessage() + ")")
);
(If you're not familiar, the zip() operator can be used to combine the results of all the sources supplied as input to emit another/new source).
In this stream, because the processing of the B's ends up throwing an exception:
the stream is terminated during the execution of the B's
the exception is reported to the observer (ie the onError() handler is triggered)
the C's are never processed
If what you want, however, is to decide for yourself whether or not to execute each branch, one approach you could take is to pass the results from previous operations down the stream using some sort of state holder, like so:
class State {
final String value;
final Throwable error;
State(String value, Throwable error) {
this.value = value;
this.error = error;
}
}
The stream could then be modified to conditionally execute different batches, for example:
Single
.zip(a1, a2, a3, (s, s2, s3) -> {
try {
// Execute the A's here...
return new State("A's completed successfully", null);
} catch(Throwable t) {
return new State(null, t);
}
})
.flatMap((Function<State, SingleSource<State>>) s -> {
if(s.error != null) {
// If an error occurred upstream, skip this batch...
return Single.just(s);
} else {
try {
// ...otherwise, execute the B's
return Single.just(new State("B's completed successfully", null));
} catch(Throwable t) {
return Single.just(new State(null, t));
}
}
})
.flatMap((Function<State, SingleSource<State>>) s -> {
if(s.error != null) {
// If an error occurred upstream, skip this batch...
return Single.just(s);
} else {
try {
// ...otherwise, execute the C's
return Single.just(new State("C's completed successfully", null));
} catch(Throwable t) {
return Single.just(new State(null, t));
}
}
})
.subscribe(
s -> {
if(s.error != null) {
System.out.println("## onSuccess with error: " + s.error.getMessage());
} else {
System.out.println("## onSuccess without error: " + s.value);
}
},
t -> System.out.println("## onError(" + t.getMessage() + ")")
);
After some research in Vertx source code, I found a public method that the rx version of CompositeFuture uses to convert 'traditional' CompositeFuture to its rx version. The method is io.vertx.reactivex.core.CompositeFuture.newInstance. With this workaround, I could use my traditional method and then convert it to use in the rx chain. This was what I wanted, because it was problematic to change the existing traditional method.
Here is the code with comments:
rxGetConfig(vertx)
.flatMap(config -> {
return rxComposeAny(vertx, config)
.flatMap(r -> rxComposeAny(vertx, config))
.flatMap(r -> rxComposeAll(vertx, config));
})
.subscribe(
compositeFuture -> {
compositeFuture.onSuccess(event -> startPromise.complete());
},
error -> startPromise.fail(error));
public static Single<JsonObject> rxGetConfig(Vertx vertx) {
ConfigRetrieverOptions enrichConfigRetrieverOptions = getEnrichConfigRetrieverOptions();
// the reason we create new vertx is just to get an instance that is rx
// so this ConfigRetriever is from io.vertx.reactivex.config, instead of normal io.vertx.config
ConfigRetriever configRetriever = ConfigRetriever.create(io.vertx.reactivex.core.Vertx.newInstance(vertx), enrichConfigRetrieverOptions);
return configRetriever.rxGetConfig();
}
public static Single<io.vertx.reactivex.core.CompositeFuture> rxComposeAny(Vertx vertx, JsonObject config) {
// instead of adapted all the parameters of myMethodsThatReturnsFutures to be rx compliant,
// we create it 'normally' and the converts bellow to rx CompositeFuture
CompositeFuture compositeFuture = CompositeFuture.any(myMethodsThatReturnsFutures(config));
return io.vertx.reactivex.core.CompositeFuture
.newInstance(compositeFuture)
.rxOnComplete();
}
I am trying to set up an exponential back off via an Observable.timer if the network is down or if a given service is down. I have a retryWhen when there are errors.
I have two issue, I cannot get the timer to work, no matter the time set, it always runs immediately. From what I know in the docs it should run the delay then send a complete, but when I look at the logs, I see no delay.
Second is because of I wanted to get the value of the retry when it is returned I used subscribe to get it, however when Observable error is returned it throws an exception when I do the calculations. For the second issue, I plan to do a check on the type of Observable and action it depending on the type.
If I could get ideas on what I may be doing wrong that would be great
return Observable.zip(
locationObservable,
oAdapterService.getIssuerInformation(sponsorCode),
oAdapterService.getOfferInformation(sponsorCode, activity.getOfferCode()),
(LocationInfo a, IssuerInfo b, OfferInfo c) -> {
OAdapterUtil.setLocationInfo(activity, a);
OAdapterUtil.setIssuerInfo(activity, b);
OAdapterUtil.setOfferInfo(activity, c);
return activity;
})
.retryWhen(errors -> errors.zipWith(Observable.range(1, maxRetries), (error, retries) -> {
if (retries++ < maxRetries) {
log.debug("Issues with Service call for transaction ID {} with initiator ID {}, retry count {}"
,activity.getTransactionId(),activity.getInitiatorId() ,retries);
return Observable.just(retries);
}
log.error("Tried to call Service {} time(s) for for transaction ID {} with initiator ID {}, error is {} "
,maxRetries,activity.getTransactionId(),activity.getInitiatorId(),error);
return Observable.error(error);
}
).flatMap(x -> {
log.debug("X value in flat map is {}",x.toString());
x.subscribe(currentValue -> {
log.debug("X value in subscribe is with subscribe {}",currentValue.toString());
double retryCount = Double.parseDouble(currentValue.toString()) + 2.0 ;
log.debug("retry count {}",retryCount);
long exponentialBackOff =(long)Math.pow(2.0, retryCount);
log.debug("exp back off {}",exponentialBackOff);
// Observable.timer(exponentialBackOff, TimeUnit.SECONDS);
});
Observable.timer(10, TimeUnit.SECONDS);
return x;
// Observable.timer(backoffPeriod, TimeUnit.MILLISECONDS);
}
));
You have an orphan line of code:
Observable.timer(10, TimeUnit.SECONDS);
The only thing this line of code does is to create an observable. The result is discarded because nothing is done with it.
If you need to back off, then do:
return x.delay(10, TimeUnit.SECONDS);
inside of the flatMap() operator. Remove the x.subscriber(); any logging should be done before returning.
I have a Flowable that we are returning in a function that will continually read from a database and add it to a Flowable.
public void scan() {
Flowable<String> flow = Flowable.create((FlowableOnSubscribe<String>) emitter -> {
Result result = new Result();
while (!result.hasData()) {
result = request.query(skip, limit);
partialResult.getResult()
.getFeatures().forEach(feature -> emmitter.emit(feature));
}
}, BackpressureStrategy.BUFFER)
.subscribeOn(Schedulers.io());
return flow;
}
Then I have another object that can call this method.
myObj.scan()
.parallel()
.runOn(Schedulers.computation())
.map(feature -> {
//Heavy Computation
})
.sequential()
.blockingSubscribe(msg -> {
logger.debug("Successfully processed " + msg);
}, (e) -> {
logger.error("Failed to process features because of error with scan", e);
});
My heavy computation section could potentially take a very long time. So long in fact that there is a good chance that the database requests will load the whole database into memory before the consumer finishes the first couple entries.
I have read up on backpressure with rxjava but the only 4 options essentially make me drop data or replace it with the last.
Is there a way to make it so that when I call emmitter.emit(feature) the call blocks until there is more room in the Flowable?
I.E I want to treat the Flowable as a blocking queue where push will sleep if the queue is past the capacity.
I'm having a very specific problem or misunderstanding with rxjava that someone hopefully can help with.
I'm running rxjava 2.1.5 and have the following code snippet:
public static void main(String[] args) {
final Observable<Object> observable = Observable.create(emitter -> {
// Code ...
});
observable.subscribeOn(Schedulers.io())
.retryWhen(error -> {
System.out.println("retryWhen");
return error.retry();
}).subscribe(next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"));
}
After executing this, the program prints:
retryWhen
Process finished with exit code 0
My question, and what I don't understand is: Why is retryWhen called instantly upon subscribing to an Observable? The observable does nothing.
What I want is retryWhen to be called when onError is called on the emitter. Am I misunderstanding how rx works?
Thanks!
Adding new snippet:
public static void main(String[] args) throws InterruptedException {
final Observable<Object> observable = Observable.create(emitter -> {
emitter.onNext("next");
emitter.onComplete();
});
final CountDownLatch latch = new CountDownLatch(1);
observable.subscribeOn(Schedulers.io())
.doOnError(error -> System.out.println("doOnError: " + error.getMessage()))
.retryWhen(error -> {
System.out.println("retryWhen: " + error.toString());
return error.retry();
}).subscribe(next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"),
() -> latch.countDown());
latch.await();
}
Emitter onNext and complete is called. DoOnError is never called. Output is:
retryWhen: io.reactivex.subjects.SerializedSubject#35fb3008
subscribeNext
Process finished with exit code 0
retryWhen calls the provided function when an Observer subscribes to it so you have a main sequence accompanied by a sequence that emits the Throwable the main sequence failed with. You should compose a logic onto the Observable you get in this Function so at the end, one Throwable will result in a value on the other end.
Observable.error(new IOException())
.retryWhen(e -> {
System.out.println("Setting up retryWhen");
int[] count = { 0 };
return e
.takeWhile(v -> ++count[0] < 3)
.doOnNext(v -> { System.out.println("Retrying"); });
})
.subscribe(System.out::println, Throwable::printStackTrace);
Since the e -> { } function body is executed for each individual subscriber, you can have a per subscriber state such as retry counter safely.
Using e -> e.retry() has no effect because the input error flow never gets its onError called.
One issue is, that you don't receive any more results because you'r creating a Thread using retryWhen() but your app seems to finish. To see that behaviour you may want to have a while loop to keep your app running.
That actually means that you need to add something like that to the end of your code:
while (true) {}
Another issue is that you dont emit any error in your sample. You need to emit at least one value to call onNext() else it wont repeat because it's waiting for it.
Here's a working example which a value, then it emits an error and repeat. you can use
.retryWhen(errors -> errors)
which is the same as
.retryWhen(errors -> errors.retry())
Working sample:
public static void main(String[] args) {
Observable
.create(e -> {
e.onNext("test");
e.onError(new Throwable("test"));
})
.retryWhen(errors -> errors.retry())
.subscribeOn(Schedulers.io())
.subscribe(
next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"),
() -> System.out.println("onCompleted")
);
while (true) {
}
}
The reason why you need to emit a result is, that Observable needs to emit a value, else it wait until it receives a new one.
This is because onError can only be called onec (in subscribe), but onNext emits 1..* values.
You can check this behaviour by using doOnError() which provides you the error everytime it retrys the Observable.
Observable
.create(e -> e.onError(new Exception("empty")))
.doOnError(e -> System.out.println("error received " + e))
.retryWhen(errors -> errors.retry())
.subscribeOn(Schedulers.io())
.subscribe(
nextOrSuccess -> System.out.println("nextOrSuccess " + nextOrSuccess),
error -> System.out.println("subscribeError")
);
This question already has answers here:
RxJava delay for each item of list emitted
(17 answers)
Closed 6 years ago.
I have an observable I've created from a list of objects. For each object in the list I make a network request but I'd like to put a delay between each item in the list as to space out the requests a bit. Here's a snippet of my code.
return Observable.from(documentGroupModels).flatMap(new Func1<DocumentGroupModel, Observable<Boolean>>() {
#Override
public Observable<Boolean> call(DocumentGroupModel documentGroupModel) {
return refreshDocumentWithUri(documentGroupModel.getUri(), documentGroupModel.sectionGroupId,
includeExceptions, false);
}
});
Using delay or buffer doesn't quite work for this scenario as far as I can tell.
You can use a combination of Zip and interval operator if your delay is static, so you can emit an item of your zip every time configure on your interval.
Check the example
#Test
public void delaySteps() {
long start = System.currentTimeMillis();
Subscription subscription =
Observable.zip(Observable.from(Arrays.asList(1, 2, 3)), Observable.interval(200, TimeUnit.MILLISECONDS),
(i, t) -> i)
.subscribe(n -> System.out.println("time:" + (System.currentTimeMillis() - start)));
new TestSubscriber((Observer) subscription).awaitTerminalEvent(3000, TimeUnit.MILLISECONDS);
}
Also you can create an Observable with your list and use concatMap, then you can use delay for every item emitted. Maybe this solution is more elegant and no so Hacky
#Test
public void delayObservableList() {
Observable.from(Arrays.asList(1, 2, 3, 4, 5))
.concatMap(s -> Observable.just(s).delay(100, TimeUnit.MILLISECONDS))
.subscribe(n -> System.out.println(n + " emitted"),
e -> {
},
() -> System.out.println("All emitted"));
new TestSubscriber().awaitTerminalEvent(1000, TimeUnit.MILLISECONDS);
}
You can see another examples of delay here https://github.com/politrons/reactive/blob/master/src/test/java/rx/observables/utils/ObservableDelay.java
Use the `delay' operator e.g.
return Observable.from(documentGroupModels).flatMap(new Func1<DocumentGroupModel, Observable<Boolean>>() {
#Override
public Observable<Boolean> call(DocumentGroupModel documentGroupModel) {
return refreshDocumentWithUri(documentGroupModel.getUri(), documentGroupModel.sectionGroupId,
includeExceptions, false).delay(2000, TimeUnit.MILLISECONDS);
}
});