An application I am working on makes heavy use of asynchronous processing, and I am looking for a better way to organize the code.
The external input to the system is received on a servlet. The raw data collected by this servlet is deposited in to a queue. A thread pool runs against this queue, and parses the raw data into a structured record which is then deposited in to one of a set of N queues. The queue is chosen such that all records of the same kind go to the same queue. These N queues are serviced by a single thread each, which collects records of the same kind into a set. Every minute a scheduled tasks wakes up and writes into a file all records collected in the previous minute for each kind.
Currently, this code is organized using a bunch of queues, thread pools, and ever-running runnables, which makes the logic hard to follow. I’d like to refactor this code into something where the data-flow described above is more explicit. I am looking for tools and approaches to achieve that.
Do tools like RxJava help with this? If so, how?
Are there other approaches I should consider?
Here is an example of RxJava according to your description. Hope it would help you.
public class TestServlet extends HttpServlet {
private static final PublishSubject<String> o = PublishSubject.<String>create();
public static Observable<String> getServletObservable() {
return o.observeOn(Schedulers.computation());
}
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
synchronized (TestServlet.class) {
o.onNext("value");
}
}
}
class Foo {
public static void main(String[] args) {
TestServlet.getServletObservable().map(new Func1<String, String>() {
#Override
public String call(String t1) {
// do something
return null;
}
}).groupBy(new Func1<String, String>() {
#Override
public String call(String t1) {
// do something
return null;
}
}).subscribe(new Observer<GroupedObservable<String, String>>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(GroupedObservable<String, String> group) {
group.observeOn(Schedulers.io()).subscribe(new Observer<String>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(String t) {
// store t
}
});
}
});
}
}
Related
I have the next RestController:
#PostMapping("/play")
public Mono<PlayResponse> play(#RequestBody String body) {
//business logic here;
}
And I wanna add some additional logic after the controller call. For example, I want to add to my application some modes logic:
timeout mode - after the successful call, the response will wait until timeout happens
long answer mode - after the successful call, the response will wait a particular amount of ms
failure mode - after the successful call, the response will answer with FORBIDDEN code
etc.
I'm trying to achieve that through WebFilter:
#Component
public class OutgoingFilter implements WebFilter {
Mode mode = new TimeoutMode();
#NonNull
#Override
public Mono<Void> filter(#NonNull final ServerWebExchange exchange, final WebFilterChain chain) {
return chain.filter(exchange)
.doOnNext(this::onNext)
.map(this::onMap)
.doFinally(this::onFinally);
}
private Void onMap(final Void unused) {
mode.run();
return unused;
}
private void onNext(final Void unused) {
mode.run();
}
private void onFinally(final SignalType signalType) {
mode.run();
}
}
As you can see, I've tried onMap, doOnNext andonFinally methods and none of them seem not working.
Is it the right way to use WebFilter there? Maybe I'm doing something wrong?
How can I implement such logic in the Spring WebFlux application?
Update
There is my Mode interface:
public interface Mode {
void run() throws Forbidden;
}
Implementations:
public class Fail implements Mode {
#Override
public void run() throws Forbidden {
throw new Forbidden("Fail mode enabled");
}
}
public class Wait implements Mode {
private final int ms;
public Wait() {
this(0);
}
public Wait(final int ms) {
this.ms = ms;
}
#Override
public void run() throws Forbidden {
sleep();
}
private void sleep() {
try {
Thread.sleep(ms);
} catch (InterruptedException e) {
throw new IllegalStateException(e);
}
}
}
But the implementations aren't final. I will change them to a reactive style if it's needed.
I'm fairly new to RxJava and I have a basic understanding as to how to wrap a callback into an Observable but what I'm having difficulty with is doing so when the callback/listener is pre-instanced. Every example that I have found only shows instancing the callback directly into the Observable being created.
Some example code of what I'm talking about. I'm working with an Api that's works like this:
public class Api {
private ApiCallback callback;
void initialize(ApiCallback callback){
this.callback = callback;
}
void doAction1(){
this.callback.onAction1Complete();
}
}
interface ApiCallback {
void onInitialized();
void onAction1Complete();
}
With the real api I am working with I have no control over how it works so I must work with it in this state. In terms of trying to work with this Api using observables here is the struggle I am having. I have a member variable that holds the Api object:
private Api mApi = new Api();
Now in order to initialize this I have one of two options it seems.
Option 1:
Completable startApi() {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
mApi.initialize(new ApiCallback() {
#Override
public void onInitialized() {
emitter.onComplete();
}
#Override
public void onAction1Complete() {
}
});
}
});
}
Option 2:
private ApiCallback premadeCallback = new ApiCallback() {
#Override
public void onInitialized() {
}
#Override
public void onAction1Complete() {
}
};
Completable startApi() {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
mApi.initialize(premadeCallback);
}
});
}
Now the issue I have is that Option 2 makes more sense to me when I need to know when the other methods in the callback are called from Api calls. With my understanding of RxJava however I don't understand how I can reach these method calls with an Api that works like this.
For example:
Completable doAction1() {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
// Api is already initialized with callback
// How do I reach the callback from here?
}
});
}
The only what that I can currently think of as to how to achieve this would be to create a member variable as an emitter (or a dictionary of emitters) and then call its appropriate method in the api callback when needed. My concerns with this are A. I'm unsure if RxJava can work this way B. This sounds like a terrible idea.
I've a continuously generated log stream i.e a method which get called whenever a new log is available in the system. I don't want to process the log every time it is generated(because logs are generated every milliseconds or so).
I want to collect logs which are emitted over a period of time let say 5 seconds and then process them in batch.
How can I achieve this using rxjava.
I've tried something like
private static void logResults(LogData logData) {
Observable.create((ObservableOnSubscribe<LogData>) e -> {
e.onNext(logData);
}).buffer(5, TimeUnit.SECONDS).subscribeWith(new DisposableObserver<List<LogData>>() {
#Override
public void onNext(List<LogData> logData) {
System.out.print(logData.toString()));
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
}
});
}
/**
This method get called every time when new log is there
*/
public static void logGenerated(LogData log) {
logResults(log);
}
You need to create a flow that stays active across multiple calls to logResults. The simplest way is to use a static PublishSubject:
private static final Subject<LogData> subject =
PublishSubject.<LogData>create(); // .toSerialized();
private static final Disposable logProcessing =
subject.buffer(5, TimeUnit.SECONDS)
.subscribeWith(new DisposableObserver<List<LogData>>() {
#Override
public void onNext(List<LogData> logData) {
System.out.print(logData.toString()));
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
}
});
private static void logResults(LogData logData) {
subject.onNext(logData);
}
/**
* This method get called every time when new log is there
*/
public static void logGenerated(LogData log) {
logResults(log);
}
So I have a list of Track Ids that for each track Id I need to execute a network request to get the track details, I am using a for loop to launch all the requests and a latch to wait for all the requests to be completed. After they are completed then the callback is sent with the List of Tracks that have already populated.
I would like to know if there is any better way to do this, maybe with RxJava ?
I am using Retrofit 2.0 in Android.
public IBaseRequest batchTracksById(final TrackIdList trackIdListPayload, final IRequestListener<TracksList> listener) {
final TracksList tracks = new TracksList();
final Track[] trackArray = newrack[trackIdListPayload.getTrackIds().length];
tracks.setTrack(trackArray);
final CountDownLatch latch = new CountDownLatch(trackArray.length);
Thread t = new Thread(new Runnable() {
#Override
public void run() {
try {
latch.await();
handler.post(new Runnable() {
#Override
public void run() {
listener.onRequestUpdate(null, tracks, null, true);
}
});
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
t.start();
for (String id : trackIdListPayload.getTrackIds()) {
getTrackById(id, new IRequestListener<Track>() {
#Override
public void onRequestFailure(IBaseRequest request, Exception exception) {
latch.countDown();
}
#Override
public void onRequestUpdate(IBaseRequest request, Track track, RequestState state, boolean requestComplete) {
//iterate through the tracks and update the ones in the thing
int i = 0;
for (String s : trackIdListPayload.getTrackIds()) {
if (s.equals(track.getTrackId())) {
trackArray[i] = track;
// don't break here, as we may have a case where we have multiple instances of the same trackId (although
// at the moment a request will be made for each anyway...
}
i++;
}
latch.countDown();
}
});
}
return null;
}
If you want to make all the requests asynchronously and wait for them to return you can do this (lambdas for brevity and readability):
tracks.flatMap(track -> getTrackDetails(track.id)
.subscribeOn(Schedulers.io()))
.toList()
.doOnNext(list -> processTrackList())
...
If you require that the results are returned in the order of tracks but are still requested asynchronously then in soon to be released rxjava 1.0.15 you will be able to do this
tracks.concatMapEager(track -> getTrackDetails(track.id)
.subscribeOn(Schedulers.io())
.toList()
.doOnNext(list -> processTrackList())
...
If I understand correctly, you have a list of tracks as input and you want a list of webservice results. Here's a simple way to do that with RxJava if you can make your network call synchronous (rxjava will handle the background processing for you).
Observable.from(trackList)
.map(new Func1<Track, Response>() {
#Override
public Response call(Track track) {
return makeRequestSynchronously(track.id());
}
})
.toList()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Subscriber<List<Response>>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(List<Response> responses) {
}
});
Edit: You can change Retrofit to return observables from webservice, if you do that you will need to change map to the following
.flatMap(new Func1<Track, Observable<Response>>() {
#Override
public Observable<Response> call(Track track) {
return makeRequestThatReturnsObservable(track.id());
}
})
In order to not repeat myself, I want to re-use a Subscriber variable between two observables. How do you do accomplish this? My current code below does not work, because after the subscriber is used once, it is unsubscribed and no longer works again. If I new a Subscriber instead of reusing a variable, my subscription works. I don't want to write the same code twice, if possible.
public class HomePresenter extends BasePresenter<HomeView> {
ArticleRepo articleRepo;
#Inject
public HomePresenter(ArticleRepo articleRepo) {
this.articleRepo = articleRepo;
}
#Override
public void onCreate(#Nullable PresenterBundle bundle) {
super.onCreate(bundle);
}
public void onEvent(ArticleCategoryClickedEvent event) {
Timber.v("Adapter position clicked at position: '%d'", event.getAdapterPosition());
view.launchArticleActivity(event.getArticleCategory());
}
public void onEvent(SeabeeOnlineExternalLinkClickedEvent event) {
view.launchExternalLink(event.getSeabeeOnlineExternalLink());
}
public void loadArticleImages() {
articleRepo.getArticleBuckets()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(subscriber);
}
public void refreshData() {
articleRepo.refreshAndSaveArticles()
.flatMap(new Func1<List<ArticleEntity>, Observable<List<ImageArticleCategoryEntity>>>() {
#Override
public Observable<List<ImageArticleCategoryEntity>> call(List<ArticleEntity> articleEntityList) {
return articleRepo.getArticleBuckets();
}
}).subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(subscriber);
}
final Subscriber<List<ImageArticleCategoryEntity>> subscriber = new Subscriber<List<ImageArticleCategoryEntity>>() {
#Override
public void onCompleted() {
Timber.v("Loading article images complete!");
view.hideLoadingAnimation();
}
#Override
public void onError(Throwable e) {
Timber.e("Error loading article images", e);
Log.e("tag", "Error loading article images", e);
}
#Override
public void onNext(List<ImageArticleCategoryEntity> integerImageArticleCategoryEntityHashMap) {
view.loadArticleImages(integerImageArticleCategoryEntityHashMap);
}
};
}
A Subscriber should not be reused. It will not work because it is a Subscription and once unsubscribed it is done.
Use an Observer instead if you want to reuse it.
source
You can reuse your subscriber, you just need to create an actual class out of it.
private static class MySubscriber extends Subscriber<List<ImageArticleCategoryEntity>> {...}
Subscriber<> subscriber1 = new MySubscriber();
Subscriber<> subscriber2 = new MySubscriber();
And there you go.