Good time of day everyone.
I wonder if it's possible to somehow emit element of a Flowable on a different thread than next ones.
For example I have a hot in-memory cache of database objects and I don't want to go to io thread to get elements from there.
Whant I want to do is basically:
if (cache.contains(e)) {
emiter.emit(cache.get(e));
} else {
Io.post(() -> emiter.emit(db.get(e)));
}
I need the same Flowable to use different threads.
I haven't found a way to do this so far.Is it possible?
Consider following method:
private Flowable<String> getDbOnlyIfNotCached(String key) {
if (cache.contains(key)) {
return Flowable.just(cache.get(key));
} else {
return Flowable.fromCallable(() -> db.get(key))
.subscribeOn(Schedulers.io());
}
}
If cache.contains(key) is true, everything will run in the calling thread. If the value is not cached, db.get(key) will be called using the io scheduler.
Update: Examples in Android
You can use above method like this:
getDbOnlyIfNotCached("hit")
.subscribe(s -> {
// If "hit" is cached, this will be executed in the current thread.
Log.d(TAG, Thread.currentThread().getName());
});
getDbOnlyIfNotCached("miss")
.subscribe(s -> {
// If "miss" is cached, this will be executed in another thread.
Log.d(TAG, Thread.currentThread().getName());
});
Or you can use it in a Flowable chain using flatMap.
Flowable.just("hello")
./* some other operators */
.flatMap(s -> getDbOnlyIfNotCached(s))
// If "hit" is cached, chain still runs in the current thread.
.subscribe(s -> {
Log.d(TAG, s + " " + Thread.currentThread().getName());
});
Flowable.just("miss")
./* some other operators */
.flatMap(s -> getDbOnlyIfNotCached(s))
// If "miss" is cached, chain switches to another thread.
.subscribe(s -> {
Log.d(TAG, Thread.currentThread().getName());
});
If you want to observe on the main thread, then specify observeOn at the end of the chain.
Flowable.just("miss")
./* some other operators */
.flatMap(s -> getDbOnlyIfNotCached(s))
// if "miss" is cached, chain switches to another thread.
.observeOn(AndroidSchedulers.mainThread())
// Now switched to the main thread.
.subscribe(s -> {
Log.d(TAG, Thread.currentThread().getName());
});
Related
I am making multiple async calls to my database. I store all those async calls on a List<CompletableFuture<X>> list. I want to collect all the results together, so I need to wait for all of those calls to complete.
One way is to create a CompletableFuture.allOf(list.toArray(...))...
Another way is to use: list.stream.map(cf -> cf.join())...
I was just wondering if there are any advantages of creating the global CompletableFuture and waiting for it to complete (when all the individual CompletableFuture complete) over directly waiting for the individual CompletableFutures to complete.
The main thread gets blocked either way.
static CompletableFuture<Void> getFailingCF() {
return CompletableFuture.runAsync(() -> {
System.out.println("getFailingCF :: Started getFailingCF.. ");
throw new RuntimeException("getFailingCF:: Failed");
});
}
static CompletableFuture<Void> getOkCF() {
return CompletableFuture.runAsync(() -> {
System.out.println("getOkCF :: Started getOkCF.. ");
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(3));
System.out.println("getOkCF :: Completed getOkCF.. ");
});
}
public static void main(String[] args) {
List<CompletableFuture<Void>> futures = new ArrayList<>();
futures.add(getFailingCF());
futures.add(getOkCF());
// using CompletableFuture.allOf
var allOfCF = CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]));
allOfCF.join();
// invoking join on individual CF
futures.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList());
}
In the code snippet above, the difference lies in handling exception: The CompletableFuture.allOf(..) wraps any exception thrown by any of the CompletableFutures while allowing rest of the threads (executing the CompletableFuture) continue their execution.
The list.stream.map(cf -> cf.join())... way immediately throws the exception and terminates the app (and all threads executing the CFs in the list).
Note that invoking join() on allOf throws the wrapped exception, too. It will also terminate the app. But, by this time, unlike list.stream.map(cf -> cf.join())..., the rest of the threads have completed their processing.
allOfCF.whenComplete(..) is one of the graceful ways to handle the execution result (normal or exceptional) of all the CFs:
allOfCF.whenComplete((v, ex) -> {
System.out.println("In whenComplete...");
System.out.println("----------- Exception Status ------------");
System.out.println(" 1: " + futures.get(0).isCompletedExceptionally());
System.out.println(" 2: " + futures.get(1).isCompletedExceptionally());
});
In the list.stream.map(cf -> cf.join())... way, one needs to wrap the join() call in try/catch.
I have a Flowable that we are returning in a function that will continually read from a database and add it to a Flowable.
public void scan() {
Flowable<String> flow = Flowable.create((FlowableOnSubscribe<String>) emitter -> {
Result result = new Result();
while (!result.hasData()) {
result = request.query(skip, limit);
partialResult.getResult()
.getFeatures().forEach(feature -> emmitter.emit(feature));
}
}, BackpressureStrategy.BUFFER)
.subscribeOn(Schedulers.io());
return flow;
}
Then I have another object that can call this method.
myObj.scan()
.parallel()
.runOn(Schedulers.computation())
.map(feature -> {
//Heavy Computation
})
.sequential()
.blockingSubscribe(msg -> {
logger.debug("Successfully processed " + msg);
}, (e) -> {
logger.error("Failed to process features because of error with scan", e);
});
My heavy computation section could potentially take a very long time. So long in fact that there is a good chance that the database requests will load the whole database into memory before the consumer finishes the first couple entries.
I have read up on backpressure with rxjava but the only 4 options essentially make me drop data or replace it with the last.
Is there a way to make it so that when I call emmitter.emit(feature) the call blocks until there is more room in the Flowable?
I.E I want to treat the Flowable as a blocking queue where push will sleep if the queue is past the capacity.
I'm having a very specific problem or misunderstanding with rxjava that someone hopefully can help with.
I'm running rxjava 2.1.5 and have the following code snippet:
public static void main(String[] args) {
final Observable<Object> observable = Observable.create(emitter -> {
// Code ...
});
observable.subscribeOn(Schedulers.io())
.retryWhen(error -> {
System.out.println("retryWhen");
return error.retry();
}).subscribe(next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"));
}
After executing this, the program prints:
retryWhen
Process finished with exit code 0
My question, and what I don't understand is: Why is retryWhen called instantly upon subscribing to an Observable? The observable does nothing.
What I want is retryWhen to be called when onError is called on the emitter. Am I misunderstanding how rx works?
Thanks!
Adding new snippet:
public static void main(String[] args) throws InterruptedException {
final Observable<Object> observable = Observable.create(emitter -> {
emitter.onNext("next");
emitter.onComplete();
});
final CountDownLatch latch = new CountDownLatch(1);
observable.subscribeOn(Schedulers.io())
.doOnError(error -> System.out.println("doOnError: " + error.getMessage()))
.retryWhen(error -> {
System.out.println("retryWhen: " + error.toString());
return error.retry();
}).subscribe(next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"),
() -> latch.countDown());
latch.await();
}
Emitter onNext and complete is called. DoOnError is never called. Output is:
retryWhen: io.reactivex.subjects.SerializedSubject#35fb3008
subscribeNext
Process finished with exit code 0
retryWhen calls the provided function when an Observer subscribes to it so you have a main sequence accompanied by a sequence that emits the Throwable the main sequence failed with. You should compose a logic onto the Observable you get in this Function so at the end, one Throwable will result in a value on the other end.
Observable.error(new IOException())
.retryWhen(e -> {
System.out.println("Setting up retryWhen");
int[] count = { 0 };
return e
.takeWhile(v -> ++count[0] < 3)
.doOnNext(v -> { System.out.println("Retrying"); });
})
.subscribe(System.out::println, Throwable::printStackTrace);
Since the e -> { } function body is executed for each individual subscriber, you can have a per subscriber state such as retry counter safely.
Using e -> e.retry() has no effect because the input error flow never gets its onError called.
One issue is, that you don't receive any more results because you'r creating a Thread using retryWhen() but your app seems to finish. To see that behaviour you may want to have a while loop to keep your app running.
That actually means that you need to add something like that to the end of your code:
while (true) {}
Another issue is that you dont emit any error in your sample. You need to emit at least one value to call onNext() else it wont repeat because it's waiting for it.
Here's a working example which a value, then it emits an error and repeat. you can use
.retryWhen(errors -> errors)
which is the same as
.retryWhen(errors -> errors.retry())
Working sample:
public static void main(String[] args) {
Observable
.create(e -> {
e.onNext("test");
e.onError(new Throwable("test"));
})
.retryWhen(errors -> errors.retry())
.subscribeOn(Schedulers.io())
.subscribe(
next -> System.out.println("subscribeNext"),
error -> System.out.println("subscribeError"),
() -> System.out.println("onCompleted")
);
while (true) {
}
}
The reason why you need to emit a result is, that Observable needs to emit a value, else it wait until it receives a new one.
This is because onError can only be called onec (in subscribe), but onNext emits 1..* values.
You can check this behaviour by using doOnError() which provides you the error everytime it retrys the Observable.
Observable
.create(e -> e.onError(new Exception("empty")))
.doOnError(e -> System.out.println("error received " + e))
.retryWhen(errors -> errors.retry())
.subscribeOn(Schedulers.io())
.subscribe(
nextOrSuccess -> System.out.println("nextOrSuccess " + nextOrSuccess),
error -> System.out.println("subscribeError")
);
I am new to threading in Java and was wondering how I can keep the following Observable running,
private void init() {
System.out.println("Running...");
Observable o = Observable.interval(5, TimeUnit.SECONDS)
.flatMap(o -> serverService.listServers("all"))
.retryWhen(o -> o.flatMap(s -> Observable.timer(30, TimeUnit.SECONDS)))
.subscribe(serverModels -> System.out.println("onNext"),
e -> System.out.println("ERROR " + e),
() -> System.out.println("Completed"));
}
If I call this method from main(), the process exits immediately. How do I refrain from a process exit?
Sleeping can cause bugs. Instead checkout the blockingSubscribe operator.
In code bellow I need to release some resources on unsubscription (where it logs "release").
Observable first = Observable.create(new Observable.OnSubscribe<Object>() {
#Override
public void call(Subscriber<? super Object> subscriber) {
subscriber.add(Subscriptions.create(() -> {
log(“release”);
}));
}
}).doOnUnsubscribe(() -> log(“first”));
Observable second = Observable.create(…).doOnUnsubscribe(() -> log(“second”));
Observable result = first.mergeWith(second).doOnUnsubscribe(() -> log(“result”));
Subscription subscription = result.subscribe(…);
//…
subscription.unsubscribe();
But it logs only “result”. Looks like unsubscription is not propagated to merge’s child observables. So how to handle unsubscription inside of first observable’s Observable.OnSubscribe?
Most of the time, calling unsubscribe has only effect on a live sequence and may not propagate if certain sequences have completed: the operators may not keep their sources around so they can avoid memory leaks. The main idea would be that operators release any resources they manage on termination just before or just after they call their downstream's onError or onCompleted methods, but this is somewhat inconsistent with 1.x.
If you want to make sure resources are releases, look at the using operator which will release your resource upon termination or unsubscription:
Observable.using(
() -> "resource",
r -> Observable.just(r),
r -> System.out.println("Releasing " + r))
.subscribe(System.out::println);