I'm trying to integrate twitter4j with vertx event loop and I don't know if I'm doing it the right way.
I'm new to vertx, although I'm a Node.js developer, so I'm familiar with the event loop/single thread concept.
In my test, I want to subscribe to a twitter stream and publish the stream on the vertx event bus.
I've created an agnostic TwitterAPI class which will transform the twitter stream into an Observable (this will be hooked to vertx later):
public class TwitterAPI {
public Observable<Status> getTwitterObservable() {
return Observable.create(emitter -> {
final TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
twitterStream.addListener(new StatusListener(){
public void onStatus(Status status) {
emitter.onNext(status);
}
public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {}
public void onTrackLimitationNotice(int numberOfLimitedStatuses) {}
public void onException(Exception ex) {
emitter.onError(ex);
}
#Override
public void onScrubGeo(long userId, long upToStatusId) {}
#Override
public void onStallWarning(StallWarning warning) {}
});
//twitterStream.filter("some keyword");
twitterStream.sample();
});
}
}
Then I created a TwitterVerticle, which will listen to the above Observable and publish the stream on the event bus, so some other verticles can subscribe to it and process it:
public class TwitterVerticle extends AbstractVerticle {
public void start() {
EventBus eb = this.vertx.eventBus();
TwitterAPI twitterAPI = new TwitterAPI();
twitterAPI.getTwitterObservable()
.map(Status::getText)
.filter(text -> text.startsWith("my keyword"))
.subscribe(text -> {
eb.publish("tweet-feed", text);
});
}
}
For example, I created another verticle which will listen to "twitter-feed" on event bus and publish it on WebSocket, so you can see the feed in the browser.
Everything is working well at the first look...but...my main question is: I'm not sure twitter4j will play well with the event loop, maybe my integration technique is wrong. Maybe I should make TwitterVerticle a Worker Verticle?
Could somebody take a look and let me know if this is the best way to accomplish such task?
Thanks a lot!
EDIT
Publishing on event bus directly is a better pattern in this case?
Here is the modified code:
public class TwitterAPI {
public void publishStreamOnEventBus(Vertx vertx) {
EventBus eb = vertx.eventBus();
TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
twitterStream.addListener(new StatusListener(){
public void onStatus(Status status) {
eb.publish("tweet-feed", status.getText());
}
public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {}
public void onTrackLimitationNotice(int numberOfLimitedStatuses) {}
public void onException(Exception ex) {
//emitter.onError(ex);
}
#Override
public void onScrubGeo(long userId, long upToStatusId) {}
#Override
public void onStallWarning(StallWarning warning) {}
});
twitterStream.sample();
}
}
public class Main {
public static void main(String[] args) {
Vertx vertx = Vertx.vertx();
new TwitterAPI().publishStreamOnEventBus(vertx);
//vertx.deployVerticle(new TwitterVerticle()/*, new DeploymentOptions().setWorker(true)*/);
vertx.deployVerticle(new WebServerVerticle());
}
}
First a couple of general recommendations.
1/ If subscribing to an Observable involves invoking blocking APIs, use a blockingScheduler:
Scheduler blockingScheduler = io.vertx.rx.java.RxHelper.blockingScheduler(vertx);
Observable<String> obs = twitterObservable.subscribeOn(blockingScheduler);
2/ Then I assume twitter4j uses its own threads to invoke the StatusListener, so the EventBus#publish call will be made on one of those. To come back to the verticle context use the #observeOn operator:
Scheduler contextScheduler = io.vertx.rx.java.RxHelper.scheduler(context);
Observable<String> obs = twitterObservable.observeOn(contextScheduler);
Combining the two changes:
public class TwitterVerticle extends AbstractVerticle {
public void start() {
EventBus eb = this.vertx.eventBus();
Scheduler contextScheduler = io.vertx.rx.java.RxHelper.scheduler(context);
Scheduler blockingScheduler = io.vertx.rx.java.RxHelper.blockingScheduler(vertx);
TwitterAPI twitterAPI = new TwitterAPI();
twitterAPI.getTwitterObservable()
.map(Status::getText)
.filter(text -> text.startsWith("my keyword"))
.observeOn(contextScheduler)
.subscribeOn(blockingScheduler)
.subscribe(text -> {
eb.publish("tweet-feed", text);
});
}
}
All that said, if this the only job of the verticle, I would recommend to get rid of it and simply publish to the event bus. The event bus instance is thread safe and it's perfectly fine to invoke publish from the outside (non Vert.x) world. Actually, this is a good pattern to combine Vert.x code with legacy code.
Related
I am developing an application that returns Multi<String>, I would like to make some modifications to it, so I have added some methods, but for some reason it does not enter the next method at all.
My other methods are working absolutely fine. Because I am able to collect it and add it to a List, but I want to do some execution asynchronously, so using this approach.
private final ManagedExecutor managedExecutor;
public void writeTo(StreamingInfo streamingInfo) {
streamingInfo
.getEvents()
.runSubscriptionOn(managedExecutor)
.subscribe()
.withSubscriber(
new Subscriber < String > () {
#Override
public void onSubscribe(Subscription s) {
System.out.println("OnSubscription Method");
System.out.println("ON SUBS END");
}
#Override
public void onNext(String event) {
System.out.println("On Next Method");
}
#Override
public void onError(Throwable t) {
System.out.println("OnError Method");
}
#Override
public void onComplete() {
System.out.println("On Complete Method");
}
});
}
I get the following output:
OnSubscription Method
ON SUBS END
Which means that your subscription is not working for some reason. If I do not add subscription and directly collect to List then everything works as expected. Can anyone suggest what am I doing wrong here?
This is because the underlying Reactive Streams specification that SmallRye Mutiny implements has a built-in backpressure mechanism. The client (in your case your subscriber) needs to request the next item manually from the producer (events) otherwise, no item is sent down the reactive pipeline.
You need to save the Subscription object you receive in the onSubscribe method and call its request(long) method when you can process next item(s):
.withSubscriber(
new Subscriber<String>() {
private Subscription subscription;
#Override
public void onSubscribe(Subscription s) {
System.out.println("OnSubscription Method");
System.out.println("ON SUBS END");
subscription = s;
subscription.request(1);
}
#Override
public void onNext(String event) {
System.out.println("On Next Method");
subscription.request(1);
}
#Override
public void onError(Throwable t) {
System.out.println("OnError Method");
}
#Override
public void onComplete() {
System.out.println("On Complete Method");
}
});
In SmallRye there is also an easier way to do this:
events
.onSubscription()
.invoke(() -> {
System.out.println("OnSubscription Method");
System.out.println("ON SUBS END");
})
.onItem()
.invoke(event -> System.out.println("On Next Method"))
.onFailure()
.invoke(t -> System.out.println("OnError Method"))
.onCompletion()
.invoke(() -> System.out.println("On Complete Method"))
.subscribe()
.with(value -> {});
I'm fairly new to RxJava and I have a basic understanding as to how to wrap a callback into an Observable but what I'm having difficulty with is doing so when the callback/listener is pre-instanced. Every example that I have found only shows instancing the callback directly into the Observable being created.
Some example code of what I'm talking about. I'm working with an Api that's works like this:
public class Api {
private ApiCallback callback;
void initialize(ApiCallback callback){
this.callback = callback;
}
void doAction1(){
this.callback.onAction1Complete();
}
}
interface ApiCallback {
void onInitialized();
void onAction1Complete();
}
With the real api I am working with I have no control over how it works so I must work with it in this state. In terms of trying to work with this Api using observables here is the struggle I am having. I have a member variable that holds the Api object:
private Api mApi = new Api();
Now in order to initialize this I have one of two options it seems.
Option 1:
Completable startApi() {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
mApi.initialize(new ApiCallback() {
#Override
public void onInitialized() {
emitter.onComplete();
}
#Override
public void onAction1Complete() {
}
});
}
});
}
Option 2:
private ApiCallback premadeCallback = new ApiCallback() {
#Override
public void onInitialized() {
}
#Override
public void onAction1Complete() {
}
};
Completable startApi() {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
mApi.initialize(premadeCallback);
}
});
}
Now the issue I have is that Option 2 makes more sense to me when I need to know when the other methods in the callback are called from Api calls. With my understanding of RxJava however I don't understand how I can reach these method calls with an Api that works like this.
For example:
Completable doAction1() {
return Completable.create(new CompletableOnSubscribe() {
#Override
public void subscribe(final CompletableEmitter emitter) throws Exception {
// Api is already initialized with callback
// How do I reach the callback from here?
}
});
}
The only what that I can currently think of as to how to achieve this would be to create a member variable as an emitter (or a dictionary of emitters) and then call its appropriate method in the api callback when needed. My concerns with this are A. I'm unsure if RxJava can work this way B. This sounds like a terrible idea.
In order to not repeat myself, I want to re-use a Subscriber variable between two observables. How do you do accomplish this? My current code below does not work, because after the subscriber is used once, it is unsubscribed and no longer works again. If I new a Subscriber instead of reusing a variable, my subscription works. I don't want to write the same code twice, if possible.
public class HomePresenter extends BasePresenter<HomeView> {
ArticleRepo articleRepo;
#Inject
public HomePresenter(ArticleRepo articleRepo) {
this.articleRepo = articleRepo;
}
#Override
public void onCreate(#Nullable PresenterBundle bundle) {
super.onCreate(bundle);
}
public void onEvent(ArticleCategoryClickedEvent event) {
Timber.v("Adapter position clicked at position: '%d'", event.getAdapterPosition());
view.launchArticleActivity(event.getArticleCategory());
}
public void onEvent(SeabeeOnlineExternalLinkClickedEvent event) {
view.launchExternalLink(event.getSeabeeOnlineExternalLink());
}
public void loadArticleImages() {
articleRepo.getArticleBuckets()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(subscriber);
}
public void refreshData() {
articleRepo.refreshAndSaveArticles()
.flatMap(new Func1<List<ArticleEntity>, Observable<List<ImageArticleCategoryEntity>>>() {
#Override
public Observable<List<ImageArticleCategoryEntity>> call(List<ArticleEntity> articleEntityList) {
return articleRepo.getArticleBuckets();
}
}).subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(subscriber);
}
final Subscriber<List<ImageArticleCategoryEntity>> subscriber = new Subscriber<List<ImageArticleCategoryEntity>>() {
#Override
public void onCompleted() {
Timber.v("Loading article images complete!");
view.hideLoadingAnimation();
}
#Override
public void onError(Throwable e) {
Timber.e("Error loading article images", e);
Log.e("tag", "Error loading article images", e);
}
#Override
public void onNext(List<ImageArticleCategoryEntity> integerImageArticleCategoryEntityHashMap) {
view.loadArticleImages(integerImageArticleCategoryEntityHashMap);
}
};
}
A Subscriber should not be reused. It will not work because it is a Subscription and once unsubscribed it is done.
Use an Observer instead if you want to reuse it.
source
You can reuse your subscriber, you just need to create an actual class out of it.
private static class MySubscriber extends Subscriber<List<ImageArticleCategoryEntity>> {...}
Subscriber<> subscriber1 = new MySubscriber();
Subscriber<> subscriber2 = new MySubscriber();
And there you go.
I am trying to understand mechanism of callback handler. How is the handle() method invoked? Can anybody give an example of usage of custom callback handler (other than those used in Login Modules of JASS or so) in non Swing application?
Define an interface to handle the callback.
public interface ServiceListener<T> {
void callback(T result);
}
Define a method that takes ServiceListener as parameter and returns void.
Public void runInBackground(ServiceListener listener) {
...code that runs in the background...
listener.callback(...data to return to caller...);
}
And you can now do this from your main code:
runInBackground(new ServiceListener() {
#Override
public void callback(..returned data...) {
...Do stuff with returned data...
}
});
This is a basic example for requesting data from a webserver using the AsyncTask from an Android application.
First define the async class. Note that the constructor takes a listener which we use to publish the result once ready.
public class Webservice extends AsyncTask<String, Void, String> {
private DialogListener dialogListener;
public Webservice(final DialogListener dialogListener) {
this.dialogListener = dialogListener;
}
#Override
protected String doInBackground(final String... strings) {
// We cant trigger onComplete here as we are not on the GUI thread!
return "";
}
protected void onPostExecute(final String result) {
dialogListener.onComplete(result);
}
}
Basic server class for handling various network communications:
public class Server {
public void queryServer(final String url, final DialogListener service) {
// Simulate slow network...
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
new Webservice(service).execute(url);
}
}
We can now use this code inside our activity without having to worry how long the call takes as it is not going to halt the GUI as it is executed async.
Server s = new Server();
// Async server call.
s.queryServer("http://onto.dk/actions/searchEvents.jsp?minLatE6=55640596&minLngE6=12078516&maxLatE6=55642654&maxLngE6=12081948", new DialogListener() {
#Override
public void onComplete(final String result) {
toast("complete");
}
#Override
public void onError() {
toast("error");
}
});
I'm in the midst of porting a C# program over to Java that makes heavy use of delegates and the delegate's BeginInvoke method to notify of an event asynchronously. Take a data communication thread for example. It might have to notify another worker thread of its state as well as the GUI.
It seems to me that the best way to notify of the various events for different classes is to have an IClassNameHereWatcher interface that defines all of the types of events that the class "publishing" the event would need to notify about and then each class that needs to listen would implement this interface and register itself as a listener. The thing I'm not quite sure about is how to make this asynchronous. Here's approximately what I'm referring to:
public interface IFrobWatcher {
void frobDidSomething();
void frobReceivedData(object data);
}
public class Frob implements Runnable {
List<IFrobWatcher> watchers = new ArrayList<IFrobWatcher>();
private void run() {
while (true) {
// Long running task
if (blah) notifyWeDidSomething();
notifyOfData(someDataObject);
}
}
public void addWatcher(IFrobWatcher watcher) {
watchers.Add(watcher);
}
private void notifyWeDidSomething() {
for (IFrobWatcher watcher : watchers) {
watcher.frobDidSomething(); // How do I make this asynchronous?
}
}
private void notifyOfData(object someDataObject) {
for (IFrobWatcher watcher : watchers) {
watcher.frobReceivedData(someDataObject); // How do I make this asynchronous?
}
}
}
public class FrobWatcher implements IFrobWatcher {
private Frob frobToWatch;
public FrobWatcher(Frob frob) {
frobToWatch = frob;
frobToWatch.addListener(this);
}
public void FrobDidSomething() {
System.out.println("Frob did something!");
}
public void FrobReceivedData(object received) {
System.out.println("Frob received: " + received.ToString());
}
public static void main(string[] args) {
Frob f = new Frob();
FrobWatcher fw = new FrobWatcher(f);
(new Thread(f)).start();
}
}
And this is a fairly simplified example, but hopefully it conveys what I'm looking for. If something isn't clear or doesn't make sense, please comment and I'll try to clarify or fix the example.
I would recommend an Executor from the java.util.concurrent package.
This is how asynchronous callbacks are handled in the Raptor framework:
scheduler = Executors.newSingleThreadScheduledExecutor();
scheduler.scheduleAtFixedRate(
filesPoller, 0 /*initial delay*/,
checkInterval,
TimeUnit.MILLISECONDS
);
NB. this scheduling is recurring.