I was doing some experiments with reactiveX Zip, and I notice that the observables that I define inside my zip are executed sequentially one after the other. I thought that the good thing of the zip was that every single observable defined inside the zip was executed by one thread, so all of them were executed in parallel. There´s any way to achieve what I want?.
Here is my zip example
#Test
public void testZip() {
Observable.zip(obString(), obString1(), obString2(), (s, s2, s3) -> s.concat(s2)
.concat(s3))
.subscribe(System.out::println);
}
public Observable<String> obString() {
System.out.println(Thread.currentThread().getId());
return Observable.just("hello");
}
public Observable<String> obString1() {
System.out.println(Thread.currentThread().getId());
return Observable.just(" world");
}
public Observable<String> obString2() {
System.out.println(Thread.currentThread().getId());
return Observable.just("!");
}
You are looking at the wrong thing.
obString* are all executed on the same thread because they are executed when you call them in testZip.
What you want to be looking at is what happens in the observable, which is not possible using simply just, you'd need a custom observable and look at the current thread in the body of onSubscribe.
Also, you probably want to use scheduleOn to give either a specifically new thread or a thread pool to your Observable.
Related
I have an application with a couple runAsync(). These runAsync() call a variety of other methods, and I'd like to run some code before each of them but within the same thread.
So for example I have main thread, then I call runAsync(MyClass::myMethod), Thread1 is created and before myMethod() gets called, within the same thread (Thread1), another method is called.
I assume this would involve some kind of wrapper of some sorts but since this uses lambda expressions and async threads I'm a bit lost on how that'd be done.
Thanks in advance
Edit: I'd like to clarify that methodToRunBeforeAsync() should be hidden from other devs. Something like using a wrapper so you don't have to worry to make the calls to methodToRunBeforeAsync()
In order to run code around those lambdas there are a couple of options. One would be AOP but that can be complex to set up so if you're able to change the calls, you could do the following:
Option 1: Create a WrapperRunnable
Just create a wrapper/decorator that executes whatever additional code you need. You can use that approach wherever a Runnable is required.
class WrapperRunnable implements Runnable {
Runnable delegate;
WrapperRunnable(Runnable r) {
delegate= r;
}
public void run() {
//any code before
delegate.run();
//any code after
}
}
Usage: CompletableFuture.runAsync(new WrapperRunnable(MyClass::myMethod))
Option 2: wrap runAsync()
Provide your own runAsync(Runnable) method that internally creates a decorator lambda or uses the decorator defined in option 1. That calls CompletableFuture.runAsync() internally and can only be used as a replacement for this method.
class MyCompletables {
public static CompletableFuture<Void> runAsync(Runnable runnable) {
return CompletableFuture.runAsync(() -> {
//any code before
runnable.run();
//any code after
});
}
}
Using the decorator of option 1:
class MyCompletables {
public static CompletableFuture<Void> runAsync(Runnable runnable) {
return CompletableFuture.runAsync(new WrapperRunnable(runnable));
}
}
Usage: MyCompletables.runAsync(MyClass::myMethod)
Note that there are other options as well, some being more flexible, some more elegant, but this should get you started while still being easy to understand.
Something like this? Just wrap the task and make sure people use myRunAsync instead of the standard one. Give it a better name, obviously.
public static void main(String[] args) {
myRunAsync(() -> System.out.println("Task")).join();
}
private static CompletableFuture<Void> myRunAsync(Runnable runnable) {
return CompletableFuture.runAsync(() -> {
preTask();
runnable.run();
});
}
private static void preTask() {
System.out.println("Pre");
}
One simple example would be:
runAsync(() -> {
myOtherObject.myMethodToRunBefore();
myObject.myMethod();
}
)
You can either add the call to myMethodToRunBefore() in the first line of the body myMethod() or create wrapper object.The choice depends if the myMethod should be separated from the call to myMethodToRunBefore (then use wrapper) or they always need to be called together in same order (then add the call to the beforeMethod in the first line of myMethod).
I want to release current thread when invoking a blocking call and continue when the call returns in asynchronous coding in java. The example is as follows:
public class Thread1 implements Runnable {
public void run() {
someBlockingCall(); // when do this calling, I want the current thread can be relased to do some other stuff, like execute some other Runnable object
getResult(); // when return from the blocking call, something can inform the thread to continue executing, and we can get the result
}
}
How can I realize this? Please help me.
You need to explicitly call someBlockingCall() asynchronously, then block to wait for the result when it's due
public void run() {
CompletableFuture<ResultType> result =
CompletableFuture.supplyAsync(() -> someBlockingCall());
//do some other work here while someBlockingCall() is running async
//this other work will be done by the first (main?) thread
ResultType finalResult = result.join(); //get (or wait for) async result
//Now use the result in the next call
getResult();
}
If someBlockingCall() has a void return type, you can use CompletableFuture.runAsync(() -> someBlockingCall());, with the future being of type CompletableFuture<Void>
In conclusion, currently there is no way to realize my thought as I wrote in this question, because you can not just make two parallele statements execute in two different threads.
I want to do something finally after stream terminates for any reason including cancellation, and I
found the doFinally method, but it dose not work when cancellation, because https://github.com/reactor/reactor-core/issues/1090#issuecomment-367633241 show :
Cancellation travels only upstream
So, how to capture the cancel signal?
There is my code:
public Mono<Void> myFunction() {
return Mono.just("hello")
.flatMap(s -> foo(s))
.doFinally(signalType -> {
// do something finally, but the doFinally won't be called
System.out.println(signalType);
});
}
// some other library's function that I cant not modify any way
public Mono<Void> foo(String s) {
// return a reactive stream, and will cancel it after it be subscribed, like:
return Mono.just(s)
.doOnSubscribe(subscription -> subscription.cancel())
.then();
}
You can't in that particular arrangement, because the foo() method/library seems to manage the subscription (the cancellation) itself, instead of leaving that responsibility to the consumer. Managing the subscription like that is thus not necessarily a good thing.
I'm playing around with implementing my own observables or porting them from other languages for fun and profit.
The problem I've run into is that there's very little info on how to properly test observables or async code in general.
Consider the following test code:
// Create a stream of values emitted every 100 milliseconds
// `interval` uses Timer internally
final Stream<Number> stream =
Streams.interval(100).map(number -> number.intValue() * 10);
ArrayList<Number> expected = new ArrayList<>();
expected.add(0);
expected.add(10);
expected.add(20);
IObserver<Number> observer = new IObserver<Number>() {
public void next(Number x) {
assertEquals(x, expected.get(0));
expected.remove(0);
if(expected.size() == 0) {
stream.unsubscribe(this);
}
}
public void error(Exception e) {}
public void complete() {}
};
stream.subscribe(observer);
As soon as the stream is subscribed to, it emits the first value. onNext is called... And then the test exits successfully.
In JavaScript most test frameworks nowadays provide an optional Promise to the test case that you can call asynchronously on success/failure. Is anything similar available for Java?
Since the execution is asyncronious, you have to wait until is finish. You can just wait for some time in an old fashion way
your_code
wait(1000)
check results.
Or if you use Observables you can use TestSubscriber
In this example you can see how having an async operation we wait until the observer consume all items.
#Test
public void testObservableAsync() throws InterruptedException {
Subscription subscription = Observable.from(numbers)
.doOnNext(increaseTotalItemsEmitted())
.subscribeOn(Schedulers.newThread())
.subscribe(number -> System.out.println("Items emitted:" + total));
System.out.println("I finish before the observable finish. Items emitted:" + total);
new TestSubscriber((Observer) subscription)
.awaitTerminalEvent(100, TimeUnit.MILLISECONDS);
}
You can see more Asynchronous examples here https://github.com/politrons/reactive/blob/master/src/test/java/rx/observables/scheduler/ObservableAsynchronous.java
What is the proper way to implement concurrency in Java applications? I know about Threads and stuff, of course, I have been programming for Java for 10 years now, but haven't had too much experience with concurrency.
For example, I have to asynchronously load a few resources, and only after all have been loaded, can I proceed and do more work. Needless to say, there is no order how they will finish. How do I do this?
In JavaScript, I like using the jQuery.deferred infrastructure, to say
$.when(deferred1,deferred2,deferred3...)
.done(
function(){//here everything is done
...
});
But what do I do in Java?
You can achieve it in multiple ways.
1.ExecutorService invokeAll() API
Executes the given tasks, returning a list of Futures holding their status and results when all complete.
2.CountDownLatch
A synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes.
A CountDownLatch is initialized with a given count. The await methods block until the current count reaches zero due to invocations of the countDown() method, after which all waiting threads are released and any subsequent invocations of await return immediately. This is a one-shot phenomenon -- the count cannot be reset. If you need a version that resets the count, consider using a CyclicBarrier.
3.ForkJoinPool or newWorkStealingPool() in Executors is other way
Have a look at related SE questions:
How to wait for a thread that spawns it's own thread?
Executors: How to synchronously wait until all tasks have finished if tasks are created recursively?
I would use parallel stream.
Stream.of(runnable1, runnable2, runnable3).parallel().forEach(r -> r.run());
// do something after all these are done.
If you need this to be asynchronous, then you might use a pool or Thread.
I have to asynchronously load a few resources,
You could collect these resources like this.
List<String> urls = ....
Map<String, String> map = urls.parallelStream()
.collect(Collectors.toMap(u -> u, u -> download(u)));
This will give you a mapping of all the resources once they have been downloaded concurrently. The concurrency will be the number of CPUs you have by default.
If I'm not using parallel Streams or Spring MVC's TaskExecutor, I usually use CountDownLatch. Instantiate with # of tasks, reduce once for each thread that completes its task. CountDownLatch.await() waits until the latch is at 0. Really useful.
Read more here: JavaDocs
Personally, I would do something like this if I am using Java 8 or later.
// Retrieving instagram followers
CompletableFuture<Integer> instagramFollowers = CompletableFuture.supplyAsync(() -> {
// getInstaFollowers(userId);
return 0; // default value
});
// Retrieving twitter followers
CompletableFuture<Integer> twitterFollowers = CompletableFuture.supplyAsync(() -> {
// getTwFollowers(userId);
return 0; // default value
});
System.out.println("Calculating Total Followers...");
CompletableFuture<Integer> totalFollowers = instagramFollowers
.thenCombine(twitterFollowers, (instaFollowers, twFollowers) -> {
return instaFollowers + twFollowers; // can be replaced with method reference
});
System.out.println("Total followers: " + totalFollowers.get()); // blocks until both the above tasks are complete
I used supplyAsync() as I am returning some value (no. of followers in this case) from the tasks otherwise I could have used runAsync(). Both of these run the task in a separate thread.
Finally, I used thenCombine() to join both the CompletableFuture. You could also use thenCompose() to join two CompletableFuture if one depends on the other. But in this case, as both the tasks can be executed in parallel, I used thenCombine().
The methods getInstaFollowers(userId) and getTwFollowers(userId) are simple HTTP calls or something.
You can use a ThreadPool and Executors to do this.
https://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html
This is an example I use Threads. Its a static executerService with a fixed size of 50 threads.
public class ThreadPoolExecutor {
private static final ExecutorService executorService = Executors.newFixedThreadPool(50,
new ThreadFactoryBuilder().setNameFormat("thread-%d").build());
private static ThreadPoolExecutor instance = new ThreadPoolExecutor();
public static ThreadPoolExecutor getInstance() {
return instance;
}
public <T> Future<? extends T> queueJob(Callable<? extends T> task) {
return executorService.submit(task);
}
public void shutdown() {
executorService.shutdown();
}
}
The business logic for the executer is used like this: (You can use Callable or Runnable. Callable can return something, Runnable not)
public class MultipleExecutor implements Callable<ReturnType> {//your code}
And the call of the executer:
ThreadPoolExecutor threadPoolExecutor = ThreadPoolExecutor.getInstance();
List<Future<? extends ReturnType>> results = new LinkedList<>();
for (Type Type : typeList) {
Future<? extends ReturnType> future = threadPoolExecutor.queueJob(
new MultipleExecutor(needed parameters));
results.add(future);
}
for (Future<? extends ReturnType> result : results) {
try {
if (result.get() != null) {
result.get(); // here you get the return of one thread
}
} catch (InterruptedException | ExecutionException e) {
logger.error(e, e);
}
}
The same behaviour as with $.Deferred in jQuery you can archive in Java 8 with a class called CompletableFuture. This class provides the API for working with Promises. In order to create async code you can use one of it's static creational methods like #runAsync, #supplyAsync. Then applying some computation of results with #thenApply.
I usually opt for an async notify-start, notify-progress, notify-end approach:
class Task extends Thread {
private ThreadLauncher parent;
public Task(ThreadLauncher parent) {
super();
this.parent = parent;
}
public void run() {
doStuff();
parent.notifyEnd(this);
}
public /*abstract*/ void doStuff() {
// ...
}
}
class ThreadLauncher {
public void stuff() {
for (int i=0; i<10; i++)
new Task(this).start();
}
public void notifyEnd(Task who) {
// ...
}
}