We Keep Coding

RxJava2 batch items

I've a continuously generated log stream i.e a method which get called whenever a new log is available in the system. I don't want to process the log every time it is generated(because logs are generated every milliseconds or so).
I want to collect logs which are emitted over a period of time let say 5 seconds and then process them in batch.
How can I achieve this using rxjava.
I've tried something like
private static void logResults(LogData logData) {
Observable.create((ObservableOnSubscribe<LogData>) e -> {
e.onNext(logData);
}).buffer(5, TimeUnit.SECONDS).subscribeWith(new DisposableObserver<List<LogData>>() {
#Override
public void onNext(List<LogData> logData) {
System.out.print(logData.toString()));
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
}
});
}
/**
This method get called every time when new log is there
*/
public static void logGenerated(LogData log) {
logResults(log);
}

You need to create a flow that stays active across multiple calls to logResults. The simplest way is to use a static PublishSubject:
private static final Subject<LogData> subject =
PublishSubject.<LogData>create(); // .toSerialized();
private static final Disposable logProcessing =
subject.buffer(5, TimeUnit.SECONDS)
.subscribeWith(new DisposableObserver<List<LogData>>() {
#Override
public void onNext(List<LogData> logData) {
System.out.print(logData.toString()));
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
}
});
private static void logResults(LogData logData) {
subject.onNext(logData);
}
/**
* This method get called every time when new log is there
*/
public static void logGenerated(LogData log) {
logResults(log);
}

Simple Apache Camel SNMP trap

I am new to Apache Camel and trying to receive a simple SNMP trap.
I have the Maven project set up with camel-core and org.apache.servicemix.bundles.snmp4j.
I have not been able to find any SNMP examples, but based on other examples I have come up with this Main class:
public class Main {
public static Processor myProcessor = new Processor() {
public void process(Exchange arg0) throws Exception {
// save to database
}
};
public static void main(String[] args) {
CamelContext context = new DefaultCamelContext();
context.addComponent("snmp", new SnmpComponent());
RouteBuilder builder = new RouteBuilder() {
public void configure() {
from("snmp:127.0.0.1:162?protocol=udp&type=TRAP").process(myProcessor);
}
};
try {
context.addRoutes(builder);
context.start();
} catch (Exception e) {
e.printStackTrace();
}
}
}
However when I run it in Eclipse as Java application it just exits after running for half a second. I was expecting it to keep running and listening to 127.0.0.1:162 ...
Any help is greatly appreciated

One way to at least pick up a trap and print to System.out is like so:
public class SNMPTrap {
private Main main;
public static void main(String[] args) throws Exception {
SNMPTrap snmpTrap = new SNMPTrap();
snmpTrap.boot();
}
#SuppressWarnings("deprecation")
public void boot() throws Exception {
main = new Main();
main.bind("snmp", new SnmpComponent());
main.addRouteBuilder(new MyRouteBuilder());
main.addMainListener(new Events());
System.out.println("Starting SNMPTrap. Use ctrl + c to terminate the JVM.\n");
main.run();
}
private static class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
from("snmp:127.0.0.1:162?protocol=udp&type=TRAP").process(myProcessor)
.bean("snmp");
}
}
public static Processor myProcessor = new Processor() {
public void process(Exchange trap) throws Exception {
System.out.println(trap.getIn().getBody(String.class));
// Save to DB or do other good stuff
}
};
public static class Events extends MainListenerSupport {
#Override
public void afterStart(MainSupport main) {
System.out.println("SNMPTrap is now started!");
}
#Override
public void beforeStop(MainSupport main) {
System.out.println("SNMPTrap is now being stopped!");
}
}
}
However, I get warning that Main which is part of Camel core is deprecated now.

Binding an API callback to an RxJava Observable

I'm trying make a reactive application that listens to a network socket on a separate thread for prices and got a bit stumped with how exactly to construct the Observable. Much of the interfaces I have are constrained by the API I am using and therefore cannot change. I distilled what I am trying to do as a test below, but I can't see how to fill in the body of the getPriceReactive() method such that the prices are printed on the console by the subscriber (see the comment in the code).
public class PriceObservableTest {
// This interface is defined externally and used by the API
private interface ITickHandler {
void priceReceived(double price);
}
// Stores the price (currently just one double for illustration)
private class Tick {
double price = Double.NaN;
}
// Implementation of handler called by API when it receives a price
private class TickHandler implements ITickHandler {
private final Tick tick;
TickHandler() { this.tick = new Tick(); }
#Override public void priceReceived(double x) { tick.price = x; }
}
// This class emulates the API delivering prices from the socket
private class PriceSource {
private final Thread thread;
PriceSource(final ITickHandler handler) {
thread = new Thread(new Runnable() {
final Random r = new Random();
#Override public void run() {
while (!Thread.currentThread().isInterrupted()) {
try {
Thread.sleep(100);
handler.priceReceived(r.nextDouble() * 100);
} catch (InterruptedException e) {
break;
}
}
System.out.println("Price thread closed");
}
});
}
void subscribe() { thread.start(); }
void unsubscribe() { thread.interrupt(); }
}
#Test
public void simpleTest() throws Exception {
final ITickHandler handler = new TickHandler();
// Simulate some prices received periodically from a socket
PriceSource prices = new PriceSource(handler);
Observable<Tick> reactive = getPriceReactive(handler);
reactive.subscribe(new Subscriber<Tick>() {
#Override public void onCompleted() { }
#Override public void onError(Throwable e) { }
#Override public void onNext(Tick tick) {
System.out.println("Received price: " + tick.price);
}});
// Observe prices for 1 second. The subscriber should print them to console
prices.subscribe();
Thread.sleep(1000);
prices.unsubscribe();
}
// Returns an observable that reacts to price changes
private Observable<Tick> getPriceReactive(ITickHandler handler) {
return Observable.create(new Observable.OnSubscribe<Tick>() {
#Override public void call(Subscriber<? super Tick> subscriber) {
// How to call subscriber.onNext() whenever
// priceReceived() is called with a new price?
}
});
}
}
Somehow subscriber.onNext() needs to be called whenever the API calls priceReceived(), but I can't quite see how to achieve this. Of course I could store a reference to the subscriber in the TickHandler but this kind of defeats the purpose of having an Observable, doesn't it?

Transition to Observable in ITickHandler implementation. You are not controlling the subscriber(s) but the publisher
private class TickHandler implements ITickHandler {
private final Tick tick;
private final PublishSubject<Tick> priceSubject;
TickHandler() {
this.tick = new Tick();
this.priceSubject = PublishSubject.create();
}
#Override public void priceReceived(double x)
{
tick.price = x;
priceSubject.onNext(tick);
}
public Observable<Tick> priceReceivedObservable()
{
return priceSubject.asObservable();
}
}
And you can use it in your tests like:
final ITickHandler handler = new TickHandler();
PriceSource prices = new PriceSource(handler);
handler.priceReceivedObservable()
.subscribe(new Subscriber<Tick>() {
#Override public void onCompleted() { }
#Override public void onError(Throwable e) { }
#Override public void onNext(Tick tick) {
System.out.println("Received price: " + tick.price);
}});
I warn you, it's not tested since I don't do a lot of Java :)

Refactoring code into a reactive style

An application I am working on makes heavy use of asynchronous processing, and I am looking for a better way to organize the code.
The external input to the system is received on a servlet. The raw data collected by this servlet is deposited in to a queue. A thread pool runs against this queue, and parses the raw data into a structured record which is then deposited in to one of a set of N queues. The queue is chosen such that all records of the same kind go to the same queue. These N queues are serviced by a single thread each, which collects records of the same kind into a set. Every minute a scheduled tasks wakes up and writes into a file all records collected in the previous minute for each kind.
Currently, this code is organized using a bunch of queues, thread pools, and ever-running runnables, which makes the logic hard to follow. I’d like to refactor this code into something where the data-flow described above is more explicit. I am looking for tools and approaches to achieve that.
Do tools like RxJava help with this? If so, how?
Are there other approaches I should consider?

Here is an example of RxJava according to your description. Hope it would help you.
public class TestServlet extends HttpServlet {
private static final PublishSubject<String> o = PublishSubject.<String>create();
public static Observable<String> getServletObservable() {
return o.observeOn(Schedulers.computation());
}
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
synchronized (TestServlet.class) {
o.onNext("value");
}
}
}
class Foo {
public static void main(String[] args) {
TestServlet.getServletObservable().map(new Func1<String, String>() {
#Override
public String call(String t1) {
// do something
return null;
}
}).groupBy(new Func1<String, String>() {
#Override
public String call(String t1) {
// do something
return null;
}
}).subscribe(new Observer<GroupedObservable<String, String>>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(GroupedObservable<String, String> group) {
group.observeOn(Schedulers.io()).subscribe(new Observer<String>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(String t) {
// store t
}
});
}
});
}
}

GWT - synchronous return of a method

Ok, there isn't synchronous call in GWT.
But if i have something like this:
class XXX {
Config c=new Config();
c.doSomething();
}
and the class Config
public class Config {
private static HashMap<String,String> map;
public Config(){
final ServerProxyAsync serverProxy= GWT.create(ServerProxy.class);
serverProxy.getParameters(new AsyncCallback<HashMap<String,String>>() {
#Override
public void onFailure(Throwable caught) {
System.out.println("getParameters: FAILED!");
map=null;
}
#Override
public void onSuccess(HashMap<String, String> result) {
System.out.println("getParameters: OK!");
map=result;
// RETURN NOW!!!!
}
});
}
if the asynchronous call doesn't terminate I can't do doSomething(), because sometime the method is invoked before the map is initialized.
Is there a way?

How about doSomething() in onSuccess() ?
You can't use synchronous call in GWT.Even though you made it somehowits not a good practise
When you need this you have to do the stuff in the onSuccess(); of your RPC call .
So, the best idea is to simply use the asynchronous method and continue execution via the callbacks provided.You will end up with a much better user experience, and a more professional appearing
application.

Just extends your Config class to take a Runnable or a Callback. Something like this:
public class Config {
private static HashMap<String,String> map;
private final Runnable callback;
public Config(Runnable callback){
this.run = run;
final ServerProxyAsync serverProxy= GWT.create(ServerProxy.class);
serverProxy.getParameters(new AsyncCallback<HashMap<String,String>>() {
#Override
public void onFailure(Throwable caught) {
System.out.println("getParameters: FAILED!");
map=null;
}
#Override
public void onSuccess(HashMap<String, String> result) {
System.out.println("getParameters: OK!");
map=result;
callback.run();
}
});
}
}
Then you can use it like this:
class XXX {
final Runnable callback = new Runnable() {
#Override
public void run() {
dosomething1();
}
};
Config c=new Config(callback);
}