We Keep Coding

Observing closeable resources to be closed when the subscriber is removed/disposed

I'm working on a small subsystem that integrates two simple components using RxJava 2.
These two components work in a simple client-server manner where the first component produces observable data opening a resource under the hood.
The resource is not exposed to the second component.
Moreover, it must be open as long as the observable is in use, however the observable object cannot determine when it should be closed.
Speaking in code, an example implementation is like this:
private Disposable disposable;
public void onCreate() {
final Maybe<Object> maybeResource = Maybe.defer(() -> {
System.out.println("open");
// here is the resource under the hood, it is encapsulated in the observable and never gets exposed
final Closeable resource = () -> { };
return Maybe.just(resource)
.doOnDispose(() -> {
// this "destructor" is never called, resulting in a resource leak
System.out.println("close");
resource.close();
})
// arbitrary data, does not represent the data I'm working with, but it hides the resource away
.map(closeable -> new Object());
});
disposable = maybeResource.subscribe(data -> System.out.println("process: " + data));
}
public void onUserWorflow() {
// ...
System.out.println("... ... ...");
// ...
}
public void onDestroy() {
disposable.dispose();
}
The output I'd anticipate to get is:
open
process: <...>
... ... ...
close <-- this is never produced
but the last line, close, is never produced as the doOnDispose method is not invoked and does not work as I might think it's supposed to.
Therefore the resource gets never released.
There is also Maybe.using that does a similar thing, but it does not allow to "span" across the "user workflow".
Is there an RxJava/RxJava 2 way that allows managing "closeable" resources closed on disposing a subscriber?

i guess you need to use Observable.create() instead of Maybe.
Something like that:
final Observable<Object> resourceObservable = Observable.create<Object> {(emitter ->
// do you staff
emitter.onNext(new Object()); //to make observable emit something
emitter.setCancellable (
System.out.println("close");
resource.close();
)
);
disposable = resourceObservable.subscribe(data -> System.out.println("process: " + data));

ListenableFuture callback execution order

Guava's ListenableFuture library provides a mechanism for adding callbacks to future tasks. This is done as follows:
ListenableFuture<MyClass> future = myExecutor.submit(myCallable);
Futures.addCallback(future, new FutureCallback<MyClass>() {
#Override
public void onSuccess(#Nullable MyClass myClass) {
doSomething(myClass);
}
#Override
public void onFailure(Throwable t) {
printWarning(t);
}}, myCallbackExecutor);
}
You can wait for a ListenableFuture to complete by calling its get function. For instance:
MyClass myClass = future.get();
My question is, are all callbacks for a certain future guaranteed to run before the get terminates. I.e. if there is a future with many callbacks on many callback executors registered, will all the callbacks complete before get returns?
Edit
My use case is, I pass a builder around to many classes. Each class populates one field of the builder. I want all fields to be populated asynchronously because each field requires an external query to generate the data for the field. I want the user who calls my asyncPopulateBuilder to receive a Future on which she can call get and be assured that all the fields have been populated. The way I thought to do it is as follows:
final Builder b;
ListenableFuture<MyClass> future = myExecutor.submit(myCallable);
Futures.addCallback(future, new FutureCallback<MyClass>() {
#Override
public void onSuccess(#Nullable MyClass myClass) {
b.setMyClass(myClass);
}
#Override
public void onFailure(Throwable t) {
printWarning(t);
}}, myCallbackExecutor);
}
// Do the same thing for all other fields.
What is the recommended way to block until all fields are populated in such a case?

Callbacks are not guaranteed to run before get returns. More on that below.
As for how to address this use case, I would suggest turning the query for each field's data into a separate Future, combining them with allAsList+transform, and taking action on that. (We may someday provide a shortcut for the "combine" step.)
ListenableFuture<MyClass> future = myExecutor.submit(myCallable);
final ListenableFuture<Foo> foo =
Futures.transform(
future,
new Function<MyClass, Foo>() { ... },
myCallbackExecutor);
final ListenableFuture<Bar> bar = ...;
final ListenableFuture<Baz> baz = ...;
ListenableFuture<?> allAvailable = Futures.allAsList(foo, bar, baz);
ListenableFuture<?> allSet = Futures.transform(
allAvailable,
new Function<Object, Object>() {
#Override
public Object apply(Object ignored) {
// Use getUnchecked, since we know they already succeeded:
builder.setFoo(Futures.getUnchecked(foo));
builder.setFoo(Futures.getUnchecked(bar));
builder.setFoo(Futures.getUnchecked(baz));
return null;
}
}
};
Now the user can call allSet.get() to await population.
(Or maybe you want for allSet to be a Future<Builder> so that the user is handed a reference to the builder. Or maybe you don't need a full-on Future at all, only a CountDownLatch, in which you could use addCallback instead of transform and count down the latch at the end of the callback.)
This approach may also simplify error handling.
RE: "Do callbacks run before get?"
First, I am pretty sure that we don't guarantee this anywhere in the spec, so thanks for asking rather than just going for it :) If you do end up wanting to rely on some behavior of the current implementation, please file an issue so that we can add documentation and tests.
Second, if I take your question very literally, what you're asking for isn't possible: If get() waits for all listeners to complete, then any listener that calls get() will hang!
A slightly more lenient version of your question is "Will all the listeners at least start before get() returns?" This turns out to be impossible, too: Suppose that I attach two listeners to the same Future to be run with directExecutor(). Both listeners simply call get() and return. One of the listeners has to run first. When it calls get(), it will hang, since the second listener hasn't started yet -- nor can it until the first listener is done. (More generally, it can be dangerous to rely on any given Executor to execute a task promptly.)
A still more lenient version is "Will the Future at least call submit() for each of the listeners before get() returns?" But this ends up with a problem in the same scenario as I just described: Calling submit(firstListener) on a directExecutor() runs the task and calls get(), which can't complete until the second listener is started, which can't happen until the first listener completes.
If anything, it's starting to sound much more likely that get() will return before any listeners execute. But thanks to the unpredictability of thread scheduling, we can't rely on that, either. (And again: It's not documented, so please don't rely on it unless you ask for it to be documented!)

final Builder b;
CountDownLatch latch = new CountDownLatch(1);
ListenableFuture<MyClass> future = myExecutor.submit(myCallable);
Futures.addCallback(future, new FutureCallback<MyClass>() {
#Override
public void onSuccess(#Nullable MyClass myClass) {
b.setMyClass(myClass);
latch.countDown();
}
#Override
public void onFailure(Throwable t) {
printWarning(t);
latch.countDown();
}, myCallbackExecutor);
try {
latch.await();
} catch (InterruptedException e) {
LOG.error("something InterruptedException", e);
} finally {
myCallbackExecutor.shutdown();
}
Edit
code is inspired by #Chris Povirk
(Or maybe you want for allSet to be a Future so that the user is handed a reference to the builder. Or maybe you don't need a full-on Future at all, only a CountDownLatch, in which you could use addCallback instead of transform and count down the latch at the end of the callback.)
This approach may also simplify error handling.

Java 8 equivalent of (RxJava) Observable#onComplete()

I'm getting to know Java 8 Stream API and I am unsure how to signal to a consumer of a stream that the stream is completed.
In my case the results of the stream-pipeline will be written in batches to a database or published on a messaging service. In such cases the stream-pipeline should invoke a method to "flush" and "close" the endpoints once the stream is closed.
I had a bit of exposure to the Observable pattern as implemented in RxJava and remember the Observer#onComplete method is used there for this purpose.
On the other hand Java 8 Consumer only exposes an accept method but no way to "close" it. Digging in the library I found a sub-interface of Consumer called Sink which offers an end method, but it's not public. Finally I thought of implementing a Collector which seems to be the most flexible consumer of a stream, but isn't there any simpler option?

The simplest way of doing a final operation is by placing the appropriate statement right after the terminal operation of the stream, for example:
IntStream.range(0, 100).parallel().forEach(System.out::println);
System.out.println("done");
This operation will be performed in the successful case only, where a commit is appropriate. While the Consumers run concurrently, in unspecified order, it is still guaranteed that all of them have done their work upon normal return.
Defining an operation that is also performed in the exceptional case is not that easy. Have a look at the following example:
try(IntStream is=IntStream.range(0, 100).onClose(()->System.out.println("done"))) {
is.parallel().forEach(System.out::println);
}
This works like the first one but if you test it with an exceptional case, e.g.
try(IntStream is=IntStream.range(0, 100).onClose(()->System.out.println("done"))) {
is.parallel().forEach(x -> {
System.out.println(x);
if(Math.random()>0.7) throw new RuntimeException();
});
}
you might encounter printouts of numbers after done. This applies to all kind of cleanup in the exceptional case. When you catch the exception or process a finally block, there might be still running asynchronous operations. While it is no problem rolling back a transaction in the exceptional case at this point as the data is incomplete anyway, you have to be prepared for still running attempts to write items to the now-rolled-back resource.
Note that Collector-based solutions, which you thought about, can only define a completion action for the successful completion. So these are equivalent to the first example; just placing the completing statement after the terminal operation is the simpler alternative to the Collector.
If you want to define operations which implement both, the item processing and the clean up steps, you may create your own interface for it and encapsulate the necessary Stream setup into a helper method. Here is how it might look like:
Operation interface:
interface IoOperation<T> {
void accept(T item) throws IOException;
/** Called after successfull completion of <em>all</em> items */
default void commit() throws IOException {}
/**
* Called on failure, for parallel streams it must set the consume()
* method into a silent state or handle concurrent invocations in
* some other way
*/
default void rollback() throws IOException {}
}
Helper method implementation:
public static <T> void processAllAtems(Stream<T> s, IoOperation<? super T> c)
throws IOException {
Consumer<IoOperation> rollback=io(IoOperation::rollback);
AtomicBoolean success=new AtomicBoolean();
try(Stream<T> s0=s.onClose(() -> { if(!success.get()) rollback.accept(c); })) {
s0.forEach(io(c));
c.commit();
success.set(true);
}
catch(UncheckedIOException ex) { throw ex.getCause(); }
}
private static <T> Consumer<T> io(IoOperation<T> c) {
return item -> {
try { c.accept(item); }
catch (IOException ex) { throw new UncheckedIOException(ex); }
};
}
Using it without error handling might be as easy as
class PrintNumbers implements IoOperation<Integer> {
public void accept(Integer i) {
System.out.println(i);
}
#Override
public void commit() {
System.out.println("done.");
}
}
processAllAtems(IntStream.range(0, 100).parallel().boxed(), new PrintNumbers());
Dealing with errors is possible, but as said, you have to handle the concurrency here. The following example does also just print number but use a new output stream that should be closed at the end, therefore the concurrent accept calls have to deal with concurrently closed streams in the exceptional case.
class WriteNumbers implements IoOperation<Integer> {
private Writer target;
WriteNumbers(Writer writer) {
target=writer;
}
public void accept(Integer i) throws IOException {
try {
final Writer writer = target;
if(writer!=null) writer.append(i+"\n");
//if(Math.random()>0.9) throw new IOException("test trigger");
} catch (IOException ex) {
if(target!=null) throw ex;
}
}
#Override
public void commit() throws IOException {
target.append("done.\n").close();
}
#Override
public void rollback() throws IOException {
System.err.print("rollback");
Writer writer = target;
target=null;
writer.close();
}
}
FileOutputStream fos = new FileOutputStream(FileDescriptor.out);
FileChannel fch = fos.getChannel();
Writer closableStdIO=new OutputStreamWriter(fos);
try {
processAllAtems(IntStream.range(0, 100).parallel().boxed(),
new WriteNumbers(closableStdIO));
} finally {
if(fch.isOpen()) throw new AssertionError();
}

Terminal operations on Java 8 streams (like collect(), forEach() etc) will always complete the stream.
If you have something that is processing objects from the Stream you know when the stream ends when the Collector returns.
If you just have to close your processor, you can wrap it in a try-with-resource and perform the terminal operatoni inside the try block
try(BatchWriter writer = new ....){
MyStream.forEach( o-> writer.write(o));
}//autoclose writer

You can use Stream#onClose(Runnable) to specify a callback to invoke when the stream is closed. Streams are pull-based (contrary to push-based rx.Observable), so the hook is associated with stream, not it's consumers

This is a bit of a hack but works well.
Create a stream concatenation of the original + a unique object.
Using peek(), see if the new object is encountered, and call the onFinish action.
Return the stream with filter, so that the unique object won't be returned.
This preserves the onClose event of the original stream.
public static <T> Stream<T> onFinish(Stream<T> stream, Runnable action) {
final Object end = new Object(); // unique object
Stream<Object> withEnd = Stream.concat(stream.sequential(), Stream.of(end));
Stream<Object> withEndAction = withEnd.peek(item -> {
if (item == end) {
action.run();
}
});
Stream<Object> withoutEnd = withEndAction.filter(item -> item != end);
return (Stream<T>) withoutEnd;
}
Another option is to wrap the original spliterator, and when it returns false, call the action.
public static <T> Stream<T> onFinishWithSpliterator(Stream<T> source, Runnable onFinishAction) {
Spliterator<T> spliterator = source.spliterator();
Spliterator<T> result = new Spliterators.AbstractSpliterator<T>(source.estimateSize(), source.characteristics()) {
#Override
public boolean tryAdvance(Consumer<? super T> action) {
boolean didAdvance = source.tryAdvance(action);
if (!didAdvance) {
onFinishAction.run();
}
return didAdvance;
}
};
// wrap the the new spliterator with a stream and keep the onClose event
return StreamSupport.stream(result, false).onClose(source::close);
}

Observable which does not pass anything in onNext()

I would need an Observable, for example to provide a system clock, which does not need to pass anything in onNext(). I couldn't find a signature that would allow me to do that.
Sure, I could use any object and then pass null, but that doesn't make much sense. So my question is if there is a better way to do that.
Observable.create(new Observable.OnSubscribe<Anyobject>() { // use any object in the signature
#Override public void call(Subscriber<? super Anyobject> subscriber) {
subscriber.onNext(null); // then pass null
subscriber.onCompleted();
}
})

You don't need to call onNext if your Observable doesn't emit anything.
You could use Void in your signature and do something like
Observable<Void> o = Observable.create(new Observable.OnSubscribe<Void>() {
#Override
public void call(Subscriber<? super Void> subscriber) {
// Do the work and call onCompleted when you done,
// no need to call onNext if you have nothing to emit
subscriber.onCompleted();
}
});
o.subscribe(new OnCompletedObserver<Void>() {
#Override
public void onCompleted() {
System.out.println("onCompleted");
}
#Override
public void onError(Throwable e) {
System.out.println("onError " + e.getMessage());
}
});
You can define an OnCompletedObserver to simplify your Observer callback so that you don't have to override the onNext since you don't need it.
public abstract class OnCompletedObserver<T> implements Observer<T> {
#Override
public void onNext(T o) {
}
}
If I've understood what you're asking then this should do the trick.

If you need something to be passed to onNext() before onCompleted() is called:
Observable.<Void>just(null)
If you only need onCompleted() to be called:
Observable.empty()

RxJava 2 Wiki:
RxJava 2.x no longer accepts null values and the following will yield
NullPointerException immediately or as a signal to downstream.
...
This means that Observable<Void> can no longer emit any values but
only terminate normally or with an exception. API designers may
instead choose to define Observable<Object> with no guarantee on what
Object will be (which should be irrelevant anyway)
It means that you can't use Void and do Observable.just(null).
Use Object or some other simple type instead:
Observable.just(new Object());

Starting with RxJava 2, the propper way to do this is to use a Completable
From the docs:
Represents a deferred computation without any value but only
indication for completion or exception. The class follows a similar
event pattern as Reactive-Streams: onSubscribe (onError|onComplete)?

One of the light solutions is to use Observable<Boolean>
And then onNext(Boolean.TRUE) which you then just ignore.
But probably you shouldn't use Observable in that case.
Consider using Completable instead

I don't know this will helps you or not.
The code written in RxSwift.
// Create an observable and emit somethings
let myObservable = Observable<Void>.create{ observer in
observer.on(.next(Void()))
return Disposables.create()
}
// Observer subscribe changes
myObservable.subscribe(onNext: {
_ in
print("Hello")
}).disposed(by: disposeBag)
Or use the Variable object
// Create a Variable object that contanins nothing
var myValueWatcher:Variable<Void> = Variable<Void>(Void())
// Observer subscribe changes
myValueWatcher.asObservable().skip(1).subscribe(onNext: {
_ in
print("Changes!")
}).disposed(by: disposeBag)
// The emit code
myValueWatcher.value = Void()

Variable arguments with different types?

I have a List of event listeners in my code:
private List<EventListener> listeners = new ArrayList<EventListener>();
listeners.add(new EventListener() {
#Override
public void someEvent(String arg1, MyClass arg2) {
// ...
}
#Override
public void someOtherEvent(AnotherClass arg1, int arg2) {
// ...
}
}
Currently, I am calling the listeners using a for loop:
for (EventListener listener : listeners) {
listener.someEvent("Hello world", (MyClass) myObject);
}
I would like to call it using a single method, like this:
fireEvent("someEvent", "Hello world", (MyClass) myObject);
Or possibly an array or something for the event arguments.
One way to do this would be to create some sort of event object, but I don't particularly want to do this as it seems messy (someone tell me if I'm wrong here; I'm inexperienced with Java). Is there a way to create a fireEvent similar to above? EventListener is an interface, if that helps.

Theoretically, you can do this using java reflection:
public void fireEvent(String name, Object... args) {
Method method = null;
// 1. find method
for (Method m : EventListener.class.getMethods()) {
if (m.getName().equals(name)) {
method = m;
break;
}
}
if (method == null) {
throw new IllegalArgumentException("Unknown event method: " + name);
}
// 2. call method on all listeners
for (EventListener l : listeners) {
try {
method.invoke(l, args);
} catch (IllegalAccessException | InvocationTargetException e) {
throw new IllegalArgumentException(e);
}
}
}
(Note that this is a trivial version that does not cover all eventualities like overloaded event methods etc.)
But I strongly advice do NOT do that! It's not only ugly and hard to read, it also makes your program being NOT typesafe any more! When you use the wrong name String or the wrong number or types of parameters, the compiler won't notice, but the program will break at runtime!
So I suggest to simply have one protected fireXXXEvent method for every event handler method of your listener (using a simple for loop). In your case this would be:
protected void fireSomeEvent(String arg1, MyClass arg2);
protected void fireSomeOtherEvent(AnotherClass arg1, int arg2);
Of course you can also introduce event objects (possibly by subclassing java.util.EventObject). But this might not necessarily reduce the number of methods in your listener (but it can - depends on the types of events you've got).
There are some fancier strategies e.g. using a generic event handler with event type objects and generic event objects as in JavaFX, but I think this is not recommended if you're not very familiar with java.
Sidenote: Use either a CopyOnWriteArrayList for storing your listeners or iterate over a copy of listeners when firing the events otherwise you might get a ConcurrentModificationException if a listeners tries to remove itself from the listener list.