I've a continuously generated log stream i.e a method which get called whenever a new log is available in the system. I don't want to process the log every time it is generated(because logs are generated every milliseconds or so).
I want to collect logs which are emitted over a period of time let say 5 seconds and then process them in batch.
How can I achieve this using rxjava.
I've tried something like
private static void logResults(LogData logData) {
Observable.create((ObservableOnSubscribe<LogData>) e -> {
e.onNext(logData);
}).buffer(5, TimeUnit.SECONDS).subscribeWith(new DisposableObserver<List<LogData>>() {
#Override
public void onNext(List<LogData> logData) {
System.out.print(logData.toString()));
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
}
});
}
/**
This method get called every time when new log is there
*/
public static void logGenerated(LogData log) {
logResults(log);
}
You need to create a flow that stays active across multiple calls to logResults. The simplest way is to use a static PublishSubject:
private static final Subject<LogData> subject =
PublishSubject.<LogData>create(); // .toSerialized();
private static final Disposable logProcessing =
subject.buffer(5, TimeUnit.SECONDS)
.subscribeWith(new DisposableObserver<List<LogData>>() {
#Override
public void onNext(List<LogData> logData) {
System.out.print(logData.toString()));
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
}
});
private static void logResults(LogData logData) {
subject.onNext(logData);
}
/**
* This method get called every time when new log is there
*/
public static void logGenerated(LogData log) {
logResults(log);
}
Related
Once the new Object is been added in a pipeline i can see the value(obj.url) of the new Object being printed without getting executing the Flux part.
Note: If i restart my application, it able to read all the objects and executes completely.
Issue: Seems like flux.interval is not getting subscribed for the
new events.
The new real time value will be added into a Map<String, BaseSubscriber>
Once added it will invoke startPipeline() method where it get printed with a new value but it wont execute after that . So here the real time added objects wont be executed with getMessages() and doSomeTask() method.
class pipelineProcess extends BaseSubscriber<Data>{
private final XYZ obj;
#PostConstruct
public void startPipeline() {
log.info("New Objected read {} ",obj.getUrl())
Flux.interval(Duration.ofMillis(500))
.flatMapIterable(aLong -> getMessages())
.flatMap(message -> doSomeTask(message))
.subscribe(this);
}
}
#Override
protected void hookOnSubscribe(Subscription subscription) {
log.info("Hook on Subscribe called for {} : ",obj.getUrl());
request(1);
}
#Override
protected void hookOnNext(ScanRequestData next) {
log.info("Hook on Next called for {}: ", obj.getUrl());
request(1);
}
#Override
protected void hookOnError(Throwable throwable) {
log.error("hookOnError exception", throwable);
request(1);
}
#Override
protected void hookOnComplete() {
log.warn("The reactor pipeline of {} has been completed.", obj.getUrl());
}
#Override
protected void hookOnCancel() {
log.warn("The reactor pipeline of {} has been cancelled.", obj.getUrl());
}
}
Can anyone tell me how to overcome this issue
I'm trying make a reactive application that listens to a network socket on a separate thread for prices and got a bit stumped with how exactly to construct the Observable. Much of the interfaces I have are constrained by the API I am using and therefore cannot change. I distilled what I am trying to do as a test below, but I can't see how to fill in the body of the getPriceReactive() method such that the prices are printed on the console by the subscriber (see the comment in the code).
public class PriceObservableTest {
// This interface is defined externally and used by the API
private interface ITickHandler {
void priceReceived(double price);
}
// Stores the price (currently just one double for illustration)
private class Tick {
double price = Double.NaN;
}
// Implementation of handler called by API when it receives a price
private class TickHandler implements ITickHandler {
private final Tick tick;
TickHandler() { this.tick = new Tick(); }
#Override public void priceReceived(double x) { tick.price = x; }
}
// This class emulates the API delivering prices from the socket
private class PriceSource {
private final Thread thread;
PriceSource(final ITickHandler handler) {
thread = new Thread(new Runnable() {
final Random r = new Random();
#Override public void run() {
while (!Thread.currentThread().isInterrupted()) {
try {
Thread.sleep(100);
handler.priceReceived(r.nextDouble() * 100);
} catch (InterruptedException e) {
break;
}
}
System.out.println("Price thread closed");
}
});
}
void subscribe() { thread.start(); }
void unsubscribe() { thread.interrupt(); }
}
#Test
public void simpleTest() throws Exception {
final ITickHandler handler = new TickHandler();
// Simulate some prices received periodically from a socket
PriceSource prices = new PriceSource(handler);
Observable<Tick> reactive = getPriceReactive(handler);
reactive.subscribe(new Subscriber<Tick>() {
#Override public void onCompleted() { }
#Override public void onError(Throwable e) { }
#Override public void onNext(Tick tick) {
System.out.println("Received price: " + tick.price);
}});
// Observe prices for 1 second. The subscriber should print them to console
prices.subscribe();
Thread.sleep(1000);
prices.unsubscribe();
}
// Returns an observable that reacts to price changes
private Observable<Tick> getPriceReactive(ITickHandler handler) {
return Observable.create(new Observable.OnSubscribe<Tick>() {
#Override public void call(Subscriber<? super Tick> subscriber) {
// How to call subscriber.onNext() whenever
// priceReceived() is called with a new price?
}
});
}
}
Somehow subscriber.onNext() needs to be called whenever the API calls priceReceived(), but I can't quite see how to achieve this. Of course I could store a reference to the subscriber in the TickHandler but this kind of defeats the purpose of having an Observable, doesn't it?
Transition to Observable in ITickHandler implementation. You are not controlling the subscriber(s) but the publisher
private class TickHandler implements ITickHandler {
private final Tick tick;
private final PublishSubject<Tick> priceSubject;
TickHandler() {
this.tick = new Tick();
this.priceSubject = PublishSubject.create();
}
#Override public void priceReceived(double x)
{
tick.price = x;
priceSubject.onNext(tick);
}
public Observable<Tick> priceReceivedObservable()
{
return priceSubject.asObservable();
}
}
And you can use it in your tests like:
final ITickHandler handler = new TickHandler();
PriceSource prices = new PriceSource(handler);
handler.priceReceivedObservable()
.subscribe(new Subscriber<Tick>() {
#Override public void onCompleted() { }
#Override public void onError(Throwable e) { }
#Override public void onNext(Tick tick) {
System.out.println("Received price: " + tick.price);
}});
I warn you, it's not tested since I don't do a lot of Java :)
In order to not repeat myself, I want to re-use a Subscriber variable between two observables. How do you do accomplish this? My current code below does not work, because after the subscriber is used once, it is unsubscribed and no longer works again. If I new a Subscriber instead of reusing a variable, my subscription works. I don't want to write the same code twice, if possible.
public class HomePresenter extends BasePresenter<HomeView> {
ArticleRepo articleRepo;
#Inject
public HomePresenter(ArticleRepo articleRepo) {
this.articleRepo = articleRepo;
}
#Override
public void onCreate(#Nullable PresenterBundle bundle) {
super.onCreate(bundle);
}
public void onEvent(ArticleCategoryClickedEvent event) {
Timber.v("Adapter position clicked at position: '%d'", event.getAdapterPosition());
view.launchArticleActivity(event.getArticleCategory());
}
public void onEvent(SeabeeOnlineExternalLinkClickedEvent event) {
view.launchExternalLink(event.getSeabeeOnlineExternalLink());
}
public void loadArticleImages() {
articleRepo.getArticleBuckets()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(subscriber);
}
public void refreshData() {
articleRepo.refreshAndSaveArticles()
.flatMap(new Func1<List<ArticleEntity>, Observable<List<ImageArticleCategoryEntity>>>() {
#Override
public Observable<List<ImageArticleCategoryEntity>> call(List<ArticleEntity> articleEntityList) {
return articleRepo.getArticleBuckets();
}
}).subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(subscriber);
}
final Subscriber<List<ImageArticleCategoryEntity>> subscriber = new Subscriber<List<ImageArticleCategoryEntity>>() {
#Override
public void onCompleted() {
Timber.v("Loading article images complete!");
view.hideLoadingAnimation();
}
#Override
public void onError(Throwable e) {
Timber.e("Error loading article images", e);
Log.e("tag", "Error loading article images", e);
}
#Override
public void onNext(List<ImageArticleCategoryEntity> integerImageArticleCategoryEntityHashMap) {
view.loadArticleImages(integerImageArticleCategoryEntityHashMap);
}
};
}
A Subscriber should not be reused. It will not work because it is a Subscription and once unsubscribed it is done.
Use an Observer instead if you want to reuse it.
source
You can reuse your subscriber, you just need to create an actual class out of it.
private static class MySubscriber extends Subscriber<List<ImageArticleCategoryEntity>> {...}
Subscriber<> subscriber1 = new MySubscriber();
Subscriber<> subscriber2 = new MySubscriber();
And there you go.
An application I am working on makes heavy use of asynchronous processing, and I am looking for a better way to organize the code.
The external input to the system is received on a servlet. The raw data collected by this servlet is deposited in to a queue. A thread pool runs against this queue, and parses the raw data into a structured record which is then deposited in to one of a set of N queues. The queue is chosen such that all records of the same kind go to the same queue. These N queues are serviced by a single thread each, which collects records of the same kind into a set. Every minute a scheduled tasks wakes up and writes into a file all records collected in the previous minute for each kind.
Currently, this code is organized using a bunch of queues, thread pools, and ever-running runnables, which makes the logic hard to follow. I’d like to refactor this code into something where the data-flow described above is more explicit. I am looking for tools and approaches to achieve that.
Do tools like RxJava help with this? If so, how?
Are there other approaches I should consider?
Here is an example of RxJava according to your description. Hope it would help you.
public class TestServlet extends HttpServlet {
private static final PublishSubject<String> o = PublishSubject.<String>create();
public static Observable<String> getServletObservable() {
return o.observeOn(Schedulers.computation());
}
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
synchronized (TestServlet.class) {
o.onNext("value");
}
}
}
class Foo {
public static void main(String[] args) {
TestServlet.getServletObservable().map(new Func1<String, String>() {
#Override
public String call(String t1) {
// do something
return null;
}
}).groupBy(new Func1<String, String>() {
#Override
public String call(String t1) {
// do something
return null;
}
}).subscribe(new Observer<GroupedObservable<String, String>>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(GroupedObservable<String, String> group) {
group.observeOn(Schedulers.io()).subscribe(new Observer<String>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(String t) {
// store t
}
});
}
});
}
}
The below method onReceivedTitlegets called 2-3 times with in a second when webview url changes. I want to call a method in it, when onReceivedTitle is being called last time. I am doing this because I just want to monitor url changes with in webview. shouldOverrideUrlLoading is not getting called when url changes through ajax.
class MyWebChromeClient extends WebChromeClient {
#Override
public void onReceivedTitle(WebView view, String title) {
Log.v("onReceivedTitle", "=>" + title);
// callAMehod();
super.onReceivedTitle(view, title);
}
}
If you want to throttle how often a method call causes another method call you can do so for example via a Handler. The simplest version enqueues a delayed message on the first call and any subsequent call while there is an enqueued message will not enqueue a new one. That results in 1 call every X time to go though - but it take at least that amount of time until the first action happens.
Example implementation (you can put that class unmodified somewhere in your code)
public abstract class ThrottleExecutor {
private final long mMinDelay;
public ThrottleExecutor(long minDelay) {
mMinDelay = minDelay;
}
/** Implement to do something */
public abstract void doThrottled();
public final void scheduleExecution() {
if (mHandler.hasMessages(0)) {
// message already enqueued, do nothing
} else {
// otherwise enqueue a message for later
mHandler.sendEmptyMessageDelayed(0, mMinDelay);
}
}
public final void cancelExecution() {
mHandler.removeMessages(0);
}
private final Handler mHandler = new Handler(Looper.getMainLooper()) {
#Override
public void handleMessage(Message msg) {
doThrottled();
}
};
}
And then use it for example like so
class Usage {
private ThrottleExecutor mThrottle = new ThrottleExecutor(2000) {
#Override
public void doThrottled() {
// happens at most every 2000ms
methodToBeThrottled();
}
};
void methodThatHappensTooOften() {
mThrottle.scheduleExecution();
}
void methodToBeThrottled() {
Log.d("TAG", "triggered at 2000ms before");
}
}
You might want to use Handler and do something like this:
class MyWebChromeClient extends WebChromeClient {
private boolean mOnReceivedTitleInvoked;
#Override
public synchronized void onReceivedTitle(final WebView view, final String title) {
if (!mOnReceivedTitleInvoked) {
mOnReceivedTitleInvoked = true;
Log.v("onReceivedTitle", "=>" + title);
handler.postDelayed(new Runnable() {
#Override
public void run() {
super.onReceivedTitle(view, title);
mOnReceivedTitleInvoked = false;
}
}, 1000);
}
}
}
Although you might want to reconsider the onReceivedTitle behaviour.