I've got the following piece of code and it works perfectly when my inner data structure of the writerObj is CopyOnWriteArrayList (concurrent one) and it crashes when I use ArrayList.
Here're my questions:
But there's only one thread in RxJava by default, isn't it?
Will the lines (between player { ... }) execute in sync way?
My code looks as follows:
.flatMapCompletable { player -> {
writerObj.write(player); // void write(Player player) adds player to inner data structure using ds.add()
return Completable.complete();
}
}
Depends on how the rest of your chain is coded.
Have a look at the following:
List<String> writerObj = new ArrayList<>();
Observable.range(0, 1000)
.map(i -> Observable.just("hello world"))
.flatMap(obs -> obs
.flatMapCompletable(elem -> {
writerObj.add(elem);
System.out.println(Thread.currentThread().getName() + " executing");
return Completable.complete();
})
.toObservable()
.subscribeOn(Schedulers.io())
)
.blockingSubscribe();
//Size of the list is not always 1000
System.out.println("The size of the list is : " + writerObj.size());
If you execute the code above, you might notice that the size of the List at the end is not always 1000. If you change the Implementation of List to CopyOnWriteArrayList, we get the desired result.
If you want the code in the flatMap to execute sequentially and by one thread at a time, change the flatMap to concatMap.
List<String> writerObj = new ArrayList<>();
Observable.range(0, 1000)
.map(i -> Observable.just("hello world"))
.concatMap(obs -> obs
.flatMapCompletable(elem -> {
writerObj.add(elem);
System.out.println(Thread.currentThread().getName() + " executing");
return Completable.complete();
})
.toObservable()
.subscribeOn(Schedulers.io())
)
.blockingSubscribe();
// Size is always 1000
System.out.println("The size of the list is : " + writerObj.size());
Hope it helps!
Related
I'm kinda stuck with a trivial task: whenever I query an external API with reactive spring WebClient or query reactive MongoDBRepository, I'd like to log how many entities got through my flux, eg. to log message like "Found n records in the database.". Eg:
return repository.findAll()
.doOnComplete { log.info("Found total n records!") } // how to get the n?
.filter { it.age > 10 }
.distinct { it.name }
TLDR: How to get a flux size (and perhaps it's contents) when it completes?
You can use ConnectableFlux. In your example:
var all = repository.findAll()
.filter { it.age > 10 }
.distinct { it.name }
.publish()
.autoConnect(2)
all.count()
.subscribe {c -> log.info("Found total {} records!", c)}
return all;
By calling the count(). It should emit a Mono when onComplete is observed.
Here was what I did,
AtomicInteger i = new AtomicInteger();
Flux<UserDetails> stringFlux =
Flux.using(() -> stringStream, Flux::fromStream,
Stream::close)
.doOnNext(s -> i.getAndIncrement())
.log()
.map(UserDetails::createUserDetails);
stringFlux
.subscribe(updateUserDetailsService::updateUserDetails);
log.info("number of records: {}", i);
I've been trying out rxJava in our code base, mostly looking to add concurrency boosting performance. However, there seems to be overhead/startup cost issues when I use rxJava. In the example below, in "doRx()" it takes ~130ms before getAllElements() is triggered, while in "doOld" it takes 0ms before getAllElements() is triggered. Any explanation to why I'm loosing 130ms initially in doRx()?
This is the logging I do, by using System.currentTimeMillis(). The () is elapsed time from init().
Existing implementation
(0) 2016-10-11T13:34:07.060: OldImpl: init()
(0) 2016-10-11T13:34:07.060: OldImpl: Call getAllElements()
(327) 2016-10-11T13:34:07.387: OldImpl: Received getAllElements()
RX implementation
(0) 2016-10-11T13:34:07.703: RxImpl: init()
(160) 2016-10-11T13:34:07.863: RxImpl: Call
getAllElements()
(392) 2016-10-11T13:34:08.095: RxImpl:
Received getAllElements()
The reasoning behind the code is that I first want to collect all elements, and then run them in parallel (under h2) since that is where we can save time as there are many backend invocations. I've used this blog as guidance for this setup.
public List<Element> doRx() {
List<Element> elements = new ArrayList<>();
Observable
.from(getAllElements())
.flatMap(
s -> Observable
.just(Element::new)
.subscribeOn(Schedulers.io())
.flatMap(
e -> {
List<Element> elements = new ArrayList<>();
for (SubElement se : e.getSubElements()) {
elements.add(se);
}
return Observable.from(elements);
}
)
)
.flatMap(
h1 -> Observable
.just(h1)
.subscribeOn(Schedulers.computation())
.flatMap(
h2 -> {
// Do additional things in parallell on all elements
return Observable
.just(h2);
}
)
)
.toBlocking()
.getIterator()
.forEachRemaining(myList::add);
return elements;
}
public List<Element> doOld() {
List<Element> elements = getAllElements();
for (Element e : elements) {
// Do stuff, same as under h2
}
return elements;
}
If I understand you code correctly, it's equivalent to the following:
public List<Element> doRx() {
return Observable
.from(getAllElements())
.flatMap(element -> Observable
.just(new Element(element))
.subscribeOn(Schedulers.io())
.flatMaplIterable(e -> e.getSubElements())
)
.observeOn(Schedulers.computation())
.doOnNext(element -> {
// Do additional things in parallell on all elements
})
.toList()
.toBlocking()
.single();
}
This has at minimum 2 context switches per element more than the sequential version. How are you doing your timings? X runs, ignore biggest and smallest numbers?
So here is what I'm trying to do:
for each event i want to call some Service.getList() 10 times, then merge the 10 lists into one.
Now I tried this two ways, they both work in UTs but fail in real app (I'm guessing I am not doing correctly the reduce operation given async http calls in real app).
For both cases I cannot see the logs put after reduce(), neither onNext() nor onError(), so I am guessing the reduce() operation does not complete.
Try #1:
public Observable<List<Event>> getEventsForLocation(Location location) {
List<Observable<List<Event>>> obs = new ArrayList<>();
for (Venue v : location.getVenues()) {
obs.add(getEventsForVenue(v)); //does one http call, returns Observable<List<Event>>
}
return Observable.concat(Observable.from(obs))
.reduce((List<Event>) new ArrayList<Event>(), (events, events2) -> {
events.addAll(events2);
return events;
})
.doOnNext(events -> Log.d("reduce ", events.toString()))
.doOnError(throwable -> Log.e("reduce error", throwable.toString()));}
Try #2:
public Observable<List<Event>> getEventsForLocation(Location location) {
return Observable
.from(location.getVenues())
.flatMap(venue -> getEventsForVenue(venue)) //does one http call, returns Observable<List<Event>>
.reduce((List<Event>) new ArrayList<Event>(), (events, events2) -> {
events.addAll(events2);
return events;
})
.doOnNext(events -> Log.d("service", "total events " + events.toString()))
.doOnError(t -> Log.e("service", "total events error2 " + t.toString()));}
And the UT which passes for both approaches:
#Test
public void getEventsForLocation() {
Location loc = new Location("test", newArrayList(new Venue("v1", "url1"),new Venue("v2", "url2")));
when(httpGateway.downloadWebPage(Mockito.anyString())).thenReturn(
Observable.just(readResource("eventsForVenue1.html")),
Observable.just(readResource("eventsForVenue2.html"))
);
TestSubscriber<List<Event>> probe = new TestSubscriber<>();
service.getEventsForLocation(loc).subscribe(probe);
probe.assertNoErrors();
//assert the next event containts contents of all lists
List<Event> events = probe.getOnNextEvents().get(0);
//first list
Assert.assertEquals("Unexpected title", "event1", events.get(0).getName());
Assert.assertEquals("Unexpected artist", "artist1", events.get(0).getArtist());
//second list
Assert.assertEquals("Unexpected title", "event2", events.get(1).getName());
Assert.assertEquals("Unexpected artist", "artist2", events.get(1).getArtist());
}
UPDATE
Here is the more complete code, with the schedulers.
Observable
.just(loc)
.subscribeOn(AndroidSchedulers.mainThread())
.observeOn(Schedulers.io())
.flatMap(location -> service.getEventsForLocation(location))
.observeOn(AndroidSchedulers.mainThread())
.subscribe(getObserver();
If you don't care about the final order in the list, you can just use from + flatMap + flatMapIterable + toList:
Observable.from(location.getVenues())
.flatMap(venue -> getEventsForVenue(venue))
.flatMapIterable(list -> list)
.toList();
If order matters and you want to execute getEventsForVenue in "parallel", you can replace flatMap with concatMapEager:
Observable.from(location.getVenues())
.concatMapEager(venue -> getEventsForVenue(venue))
.concatMapIterable(list -> list)
.toList();
You can use Collect and it will do the trick. But in this scenarios where I´m merging severals items, normally I much rather use Scan. Which for every new item it will give you the last processed item. So you can append every item with the previous items.
Take a look the marble diagram in case is suitable for your case
http://reactivex.io/documentation/operators/scan.html
Try to start with this simple example and then try to apply it into your code
/**
* apply this function for every item against the previous emitted item from the source.
* Emitted:
0
1
3
6
10
15
*/
#Test
public void scanObservable() {
Integer[] numbers = {0, 1, 2, 3, 4, 5};
Observable.from(numbers)
.scan((lastItemEmitted, newItem) -> (lastItemEmitted + newItem))
.subscribe(System.out::println);
}
I finally found one solution, using zip() but i don't like it.
The problem should be able to be resolved with a combination of flatMap/reduce
public Observable<List<Event>> getEventsForLocation(Location location) {
List<Observable<List<Event>>> venues = new ArrayList<>();
for (Venue v : location.getVenues()) {
venues.add(Observable.just(v).flatMap(venue -> getEventsForVenue(venue)));
}
return Observable.zip(venues, new FuncN<List<Event>>() {
#Override
public List<Event> call(Object... args) {
List<Event> allEvents = new ArrayList<Event>();
for (Object o : args) {
List<Event> le = (List<Event>) o;
allEvents.addAll(le);
}
return allEvents;
}
});
I have a list of number that every number represents an index.
I would like to go through the list one by one and for every number emit an observable.
How can I do that?
I have this snippet, but I having troubles understanding how to continue...
List<Short> seats = Arrays.asList(new Short[]{0,2,3,1});
rx.Observable.from(seats).takeWhileWithIndex((seatIdx, idx) -> {
// Here I would like to emit an Observable
updateGame(seatIdx, idx == 0 ? true : false)
.subscribe(results -> {
System.out.println(results);
});
return !seats.isEmpty();
}).subscribe();
There must be a better way to do this...
If you have any idea...
Thanks!
EDIT
I would like to go one step further and after omitting the loop of 4 iterations (which I don't care about its results), I would like to continue with the chain and concatenate more Observables.
List<Short> seats = Arrays.asList(new Short[]{0,2,3,1});
rx.Observable.range(0, seats.size())
.zipWith(seats, (idx, seatIdx) -> {
return updateGame(seatIdx, idx == 0 ? true : false);
})
.takeLast(1)
.flatMap(x -> x)
.concatWith(calculatePayout(...))
.concatWith(persistGame(...))
.subscribe(results -> {
System.out.println(results);
}, e -> {
}, () -> {
System.out.println("COMPLETED");
});
Notice, I use 'takeLast(1)' in order to avoid calling 'calculatePay' 4 times! - is there more elegant way to do this?
zip with Iterable gives you what you need:
List<Short> seats = Arrays.asList(new Short[]{0,2,3,1});
Observable.range(0, seats.size())
.zipWith(seats,
(idx, seatIdx) ->
updateGame(seatIdx, idx == 0 ? true : false))
.flatMap(x -> x)
.doOnNext(System.out::println)
.subscribe();
I have a list of users of unknown size. What i want is to query first 30 and update UI. Then i want to query all others by offset with step of 100 until i get last pack of users - should i use takeUntil here?) and when i get - i update UI by adding remaining users (combined with reduce() i belive).
This is my code:
final int INITIAL_OFFSET = 0;
final int INITIAL_LIMIT = 30;
// Loading first 30 users to immediately update UI (better UX)
getServerApi().getAllFriends(userId, "photo_50", INITIAL_OFFSET, INITIAL_LIMIT)
// Loading remaining users 100 by 100 and updating UI after all users been loaded
.flatMap(users -> {
AtomicInteger newOffset = new AtomicInteger(INITIAL_LIMIT);
return Observable.just(users)
.flatMap(users1 -> getServerApi().getAllFriends(userId, "photo_50", newOffset.get(), Config.DEFAULT_FRIEND_REQUEST_COUNT))
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.collect(() -> new ArrayList<User>(), (b, s) -> {
b.addAll(s);
newOffset.set(newOffset.get() + Config.DEFAULT_FRIEND_REQUEST_COUNT);
})
.repeat()
.takeUntil(friends -> friends.size() == 0);
})
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(users -> getView().appendAllFriends(users),
throwable -> getView().setError(processFail(throwable, ServerApi.Action.GET_ALL_FRIENDS), false));
But seems i do something wrong because onNext is called each time the retrofit call is made.
Answering my own question. Adels answer is good, but i needed to have a single subscription (i'm using Nucleus MVP library) and i wanted to use collect() and takeUntil() instead of while loop (which requires blocking retrofit interface method).
Spent some hours and finally got it:
final int INITIAL_LIMIT = 30;
// Loading first 30 users to immediately update UI (better UX)
getServerApi().getAllFriends(userId, "photo_50", null, INITIAL_LIMIT)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
// Updating UI 1st time or show error
.doOnNext(users -> getView().appendAllFriends(users))
.doOnError(throwable -> getView().setError(processFail(throwable, ServerApi.Action.GET_ALL_FRIENDS), false))
// Loading remaining users 100 by 100 and updating UI after all users been loaded
.flatMap(users -> {
AtomicInteger newOffset = new AtomicInteger(INITIAL_LIMIT);
ArrayList<User> remainingUsers = new ArrayList<>();
AtomicBoolean hasMore = new AtomicBoolean(true);
return Observable.just(users)
.flatMap(users1 -> getServerApi().getAllFriends(userId, "photo_50", newOffset.get(), Config.DEFAULT_FRIEND_REQUEST_COUNT))
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.collect(() -> remainingUsers, (b, s) -> {
// Needed for takeUntil
hasMore.set(b.addAll(s));
newOffset.set(newOffset.get() + Config.DEFAULT_FRIEND_REQUEST_COUNT);
})
.repeat()
.takeUntil(friends -> !hasMore.get())
// Grab all items emitted by collect()
.last()
// Updating UI last time
.doOnNext(users2 -> getView().appendAllFriends(users2));
})
.subscribe();
Maybe it will be useful for other people which are also using Nucleus.
// cache() will ensure that we load the first pack only once
Observable<Users> firstPack = firstPack().cache();
// this subscription is for updating the UI on the first batch
firstPack
.observeOn(AndroidSchedulers.mainThread())
.subscribe(x -> draw(x), e -> whoops(e));
// this subscription is for collecting all the stuff
// do whatever tricks you need to do with your backend API to get the full list of stuff
firstPack
.flatMap(fp -> rest(fp))
.observeOn(AndroidSchedulers.mainThread())
.subscribe(x -> allUsers(x), e -> whoops(e));
// I would do this in a simple while loop
Observable<List<User>> rest(List<User> firstPack) {
return Observable.create(sub -> {
final List<User> total = firstPack;
try {
while (!sub.isUnsubscribed()) {
final List<User> friends = api.getFriendsBlocking(total.size());
if (friends.isEmpty()) {
sub.onNext(total);
sub.onCompleted();
} else {
total.addAll(friends);
}
}
} catch(IOException e) {
sub.onError(e);
}
})
}