How does the CompletableFuture introduced in JDK 8 compare with the io.netty.util.concurrent.Future provided by Netty ?
Netty documentation mentions that
JDK 8 adds CompletableFuture which somewhat overlaps
io.netty.util.concurrent.Future
http://netty.io/wiki/using-as-a-generic-library.html
The questions I'm trying to get answers to are:
What would their similarities and differences be?
How would the performance characteristics of the two differ? Which one would be able to scale better?
With respect to the similarities/ differences, I have been able to come up with the following:
Similarities:
The fundamental similarity being that both are non-blocking as compared to the Java Future. Both the classes have methods available to add a listener to the future, introspect failure and success of the task and get results from the task.
Differences:
CompletableFuture seems to have a much richer interface for things like composing multiple async activities etc. Netty's io.netty.util.concurrent.Future on the other hand allows for multiple listeners to be added to the same Future, and moreover allows for listeners to be removed.
If we look at that whole paragraph (especially the first sentence)
Java sometimes advances by adopting ideas that subsume constructs
provided by Netty. For example, JDK 8 adds CompletableFuture which
somewhat overlaps io.netty.util.concurrent.Future. In such a case,
Netty's constructs provide a good migration path to you; We will
diligently update the API with future migration in mind.
What it's basically saying is that the netty Future and CompletableFuture are the same concept, but implemented at different times by different people.
Netty made their future because there wasn't one available in java, and they didn't want to pull one in as a dependency from something like Guice. But now, java has created one, and it's available for use.
In the end of the paragraph they're basically saying that the netty API may replace Future with CompletableFuture in the future.
As far as similarities/differences, they're both just one of many implementations of the future/promise pattern. Use the netty one when you're using the netty api and netty specific stuff, otherwise use CompletableFuture.
Related
In a critical part of my project which basically allows objects to be received by a controller asynchronously, put into a Queue, processed sequentially from the queue one at a time by a thread, then service responds, older processed objects are kept in the queue until newer item insertion.
Back in time (months ago), my Queue implementation for solving this particular business specific issue behind this was to use Guava's EvictingQueue, which now is marked as #Beta, and so this part of the application can break in future Guava releases.
private final Queue<SomeRandomBusinessObject> items = Queues.synchronizedQueue(EvictingQueue.create(queueSize));
Are there any thread-safe and fixed-size alternatives to EvictingQueue to achieve this goal?
There are couple of inaccuracies / mistakes in your post, so let's just try to find common ground.
First, any new feature in Guava is annotated as #Beta from the beginning, same is true for EvictingQueue in 15.0 (this links to 15.0 docs). So you probably missed that fact couple months ago, but that's OK, because...
...#Beta doesn't really mean it'll be changed without any notice -- on the contrary, some time ago, after some feedback from the community, Guava devs established pretty strict policy about what and when can be changed. See PhilosophyExplained wiki page, which says (emphasis mine):
Beta APIs
Beta APIs represent Guava features that we aren't ready to freeze for whatever reason: because the methods might not find enough users, because they might be moved, because their uses might be too narrow to include them in Guava.
That said, #Beta APIs are fully tested and supported, and treated with all the care and affection that the rest of Guava receives.
This means EvictingQueue quality is not worse than if it wasn't a "beta feature".
The biggest connotation of the #Beta annotation is that annotated classes or methods are subject to change. They can be modified in any way, or even removed, at any time. If your code is a library itself (i.e. it is used on the CLASSPATH of users outside your own control), you should not use beta APIs, unless you repackage them (e.g. using ProGuard).
This could be the concern you brought up when talking about "braking up in the future", but...
All this said, #Beta features tend to remain relatively stable. If we decide to delete a #Beta feature, we will typically deprecate it for one release before deleting it.
So it won't happen silently (as far as I observed, usually there's more than one release with deprecating though).
Which brings me the the last point:
On the other hand, if you want something taken out of #Beta, file an issue. We generally promote features out of #Beta only when it's specifically requested, so if you don't ask, it won't happen.
To sum up: I'd suggest you to file a ticket to promote EvictingQueue and make it non-#Beta, which would remove any doubts about it. On the other hand, the EvictingQueue's implementation is quite simple and standalone, so if it's removed (unlikely) you can repakckage it (i.e. use ProGuard) or even copy the code to your project (with all the licenses).
We are working on rewrite of an existing application, and need support for high number of read/write to database. For this, we are proceeding with sharding on MySQL. Since we are allowing bulk APIs for read/write, this would mean parallel execution of queries on different shards.
Can you suggest frameworks which would support the same in Java, mainly focussing on split-and-aggregate jobs. Basically I will define two interfaces ReadTask and WriteTask, and implementation of these tasks will be jobs and they would be submitted as a list for parallel execution.
I might not have termed this question in the right way, but I hope you got the context from the description. Let me know if there is any info needed for answer.
BLUF: This sounds like a common processing pattern in Akka.
This sounds like a Scatter-Gather patterned API.
If you have 1 job, you should first answer if that job will touch only one shard or more? If it will touch many shards you may choose to reject it (allowing only single-shard actions) or you may choose to break it up (scatter) it across other workers.
Akka gives you APIs, especially the Streaming API, that talk about this style of work. Akka is best expressed in Scala, but it has a Java API that gives you all the functionality of the Scala one. That you are talking about "mapping" and "reducing" (or "folding") data, these are functional operations and Scala gives you the functional idioms.
If you scatter it across other workers, you'll need to communicate the manifest of jobs to the gather side of the system.
Hope that's helpful.
You can use the ThreadPoolExecutor & Executors(factory) in Java to create Thread pools to which you can submit your read & write tasks. It allows for Runnable & Callable based on your situation.
It seems on every iteration of Java for the last few major releases, there are consistently new ways to manage concurrent tasks.
In Java 9, we have the Flow API which resembles the Flowable API of RxJava but with Java 9 has a much simpler set of classes and interfaces.
Java 9
Has a Flow.Publisher, Flow.Subscriber, Flow.Processor, Flow.Subscription, and SubmissionPublisher, and that's about it.
RxJava
Has whole packages of Flow API-like classes, i.e. io.reactivex.flowables, io.reactivex.subscribers, io.reactivex.processors, io.reactivex.observers, and io.reactivex.observables which seem to do something similar.
What are the main differences between these two libraries? Why would someone use the Java 9 Flow library over the much more diverse RxJava library or vice versa?
What are the main differences between these two libraries?
The Java 9 Flow API is not a standalone library but a component of the Java Standard Edition library and consists of 4 interfaces adopted from the Reactive Streams specification established in early 2015. In theory, it's inclusion can enable in-JDK specific usages, such as the incubating HttpClient, maybe the planned Async Database Connection in parts, and of course SubmissionPublisher.
RxJava is Java library that uses the ReactiveX style API design to provide a rich set of operators over reactive (push) dataflows. Version 2, through Flowable and various XxxProcessors, implements the Reactive Streams API which allows instances of Flowable to be consumed by other compatible libraries and in turn one can wrap any Publisher into a Flowable to consume those and compose the rich set of operators with them.
So the Reactive Streams API is the minimal interface specification and RxJava 2 is one implementation of it, plus RxJava declares a large set of additional methods to form a rich and fluent API of its own.
RxJava 1 inspired, among other sources, the Reactive Streams specification but couldn't capitalize on it (had to remain compatible). RxJava 2, being a full rewrite and a separate main version, could embrace and use the Reactive Streams specification (and even expand upon it internally, thanks to the Rsc project) and has been released almost a year before Java 9. In addition, it was decided both v1 and v2 keeps supporting Java 6 and thus a lot of Android runtimes. Therefore it couldn't capitalize directly on the Flow API provided now by Java 9 directly but only through a bridge. Such bridge is required by and/or provided in other Reactive Streams-based libraries too.
RxJava 3 may target the Java 9 Flow API but this hasn't been decided yet and depending on what features the subsequent Java versions bring (i.e., value types), we may not have v3 within a year or so.
Till then, there is a prototype library called Reactive4JavaFlow which does implement the Flow API and offers a ReactiveX style rich fluent API over it.
Why would someone use the Java 9 Flow library over the much more diverse RxJava library or vice versa?
The Flow API is an interoperation specification and not an end-user API. Normally, you wouldn't use it directly but to pass flows around to various implementations of it. When JEP 266 was discussed, the authors didn't find any existing library's API good enough to have something default with the Flow API (unlike the rich java.util.Stream). Therefore, it was decided that users will have to rely on 3rd party implementations for now.
You have to wait for existing reactive libraries to support the Flow API natively, through their own bridge implementation or new libraries to be implemented.
Providing a rich set of operators over the Flow API is only reason a library would implement it. Datasource vendors (i.e., reactive database drivers, network libraries) can start implementing their own data accessors via the Flow API and rely on the rich libraries to wrap those and provide the transformation and coordination for them without forcing everybody to implement all sorts of these operators.
Consequently, a better question is, should you start using the Flow API-based interoperation now or stick to Reactive Streams?
If you need working and reliable solutions relatively soon, I suggest you stick with the Reactive Streams ecosystem for now. If you have plenty of time or you want to explore things, you could start using the Flow API.
At the beginning, there was Rx, version one. It was a language agnostic specification of reactive APIs that has implementations for Java, JavaScript, .NET. Then they improved it and we saw Rx 2. It has implementations for different languages as well. At the time of Rx 2 Spring team was working on Reactor — their own set of reactive APIs.
And then they all thought: why not make a joint effort and create one API to rule them all. That was how Reactive Commons was set up. A joint research effort for building highly optimized reactive streams compliant operators. Current implementors include RxJava2 and Reactor.
At the same time JDK developers realized that reactive stuff is great and worth including in Java. As it is usual in Java world the de facto standard become de jure. Remeber Hibernate and JPA, Joda Time and Java 8 Date/Time API? So what JDK develpers did is extracting the very core of reactive APIs, the most basic part, and making it a standard. That is how j.u.c.Flow was born.
Technically, j.u.c.Flow is much more simpler, it consists only of four simple interfaces, while other libraries provide dozens of classes and hundreds of operators.
I hope, this answers the question "what is the difference between them".
Why would someone choose j.u.c.Flow over Rx? Well, because now it is a standard!
Currently JDK ships with only one implementation of j.u.c.Flow: HTTP/2 API. It is actually an incubating API. But in future we might expect support of it from Reactor, RxJava 2 as well as from other libraries, like reactive DB drivers or even FS IO.
"What are the main differences between these two libraries?"
As you noted yourself, the Java 9 library is much more basic and basically serves as a general API for reactive streams instead of a full-fledged solution.
"Why would someone use the Java 9 Flow library over the much more diverse RxJava library or vice versa?"
Well, for the same reason people use basic library constructs over libraries - one less dependency to manage. Also, due to the fact that the Flow API in Java 9 is more general, it is less constrained by the specific implementation.
What are the main differences between these two libraries?
This mostly holds true as an informative comment(but too long to fit in), the JEP 266: More Concurrency Updates responsible for the introduction of the Flow API in Java9 states this in its description(emphasis mine) -
Interfaces supporting the Reactive Streams publish-subscribe
framework, nested within the new class Flow.
Publishers produce items
consumed by one or more Subscribers, each managed by a Subscription.
Communication relies on a simple form of flow control (method
Subscription.request, for communicating back pressure) that can be
used to avoid resource management problems that may otherwise occur in
"push" based systems. A utility class SubmissionPublisher is provided
that developers can use to create custom components.
These (very
small) interfaces correspond to those defined with broad participation
(from the Reactive Streams initiative) and support interoperability
across a number of async systems running on JVMs.
Nesting the interfaces within a class is a conservative policy allowing
their use across various short-term and long-term possibilities. There
are no plans to provide network- or I/O-based java.util.concurrent
components for distributed messaging, but it is possible that future JDK
releases will include such APIs in other packages.
Why would someone use the Java 9 Flow library over the much more diverse RxJava library or vice versa?
Looking at a wider prospect this is completely opinion based on factors like the type of application a client is developing and its usages of the framework.
The introduction of CompletableFutures in Java 8 brought to the language features available in the scala.concurrent.Future such as monadic transformations.
What are the differences, and why a Scala developer should prefer Scala Futures over java 8 CompletableFuture ?
Are there still good reasons to use the scala.concurrent.Futurein Java through akka.dispatch bridge?
What are the differences, and why a Scala developer should prefer Scala Futures over java 8 CompletableFuture ?
Rephrasing what #dk14 pointed out in comments I'd say that CompletableFuture doesn't have idiomatic Scala api.
For scala developer the implications are:
lack of for comprehensions due to fact that it does not follow Scala method conventions common for monadic types
java-scala interop overhead needed when using big part of it's methods
It is also worth noting that java CompletableFuture is not exactly equivalent of scala Future. It is rather a fuse of scala Future and Promise.
Considering the cons listed above there isn't much sense in using CompletableFuture in scala unless you are designing public api that should be seamlessly interoperable with java.
Are there still good reasons to use the scala.concurrent.Future in Java through akka.dispatch bridge?
I am particularly looking for reasons to use akka.dispatch in Java, if there are still any
Akka is build on top of scala and it sometimes uses scala Futures. This means that in cases when you have some portion of code written in java it is worth to wrap it in scala api (with akka.dispatch java api) to be able to easily use it with akka.
For example, you are implementing akka actor in java. When processing message you want to do some non-blocking reading that, when done, should produce result as a message to another actor.
What you could do is to put your I/O into java Callable, then use akka.dispatch.Futures#future to get scala Future out of it, and then you could leverage akka pipe to make result of the future be delivered as a message to some actor.
As part of a study I am doing, I am exploring the supposed simplicity of using languages like Scala & Clojure to achieve concurrency on the JVM.
By simplicity, I am hoping to prove that these languages provide easier concurrency constructs than what Java 7 provides.
Therefore, I am hoping to find some good references that explain the complexities of Java's concurrency model.
Outside of pointing me in the direction of Google (which I have already searched with limited success), I would appreciate if those in-the-know could provide me with some good references to get me started off in this area.
Thanks
Java does not support lambda expressions. Creating an inline callback (eg, for the completion of an asynchronous call) requires 5 lines of boilerplate for an anonymous type.
This strongly discourages people from using callbacks. This is probably why Java 7 still does not have an interface for a callback that takes a value (as opposed to Runnable and Callbable), whereas C# has had one since 2005.
Therefore, the JDK does not have any real support for asynchronous operations.
The key to an asynchronous operation is the ability to kick off a long-running request, and have it run a callback when it finishes, without consuming a thread for the duration of the request. In Java, you can only do this by making a separate thread call get() on a Future<V>. This limits the concurrency of an application using the standard API to the number of threads you can sanely support.
To solve this problem, Google's Guava framework for better Java code introduces a ListenableFuture<V> interface which does have completion callbacks.
Languages like Scala fix this problem by supporting lambda expressions (which compile to anonymous classes) and adding their own Promise / Future types.
While higher level languages are easier to use multiple cores, what is often forgotten is why you want to use multiple cores which is to make the program faster e.g. increase its throughput.
When you consider options which increase concurrency, you need to test whether these options actually improve performance in some way. (Because very often they don't)
e.g. STM (Software Transactional Memory) makes it easier to write multi-threaded applications without having to worry about concurrency issues. The problem is that for trivial examples, it would be faster to not use STM and only use one thread.
Using multiple threads adds complexity and makes your application more fragile, so there has to be a good reason to do it otherwise you should stick to the simplest solution possible.
For more discussion
http://vanillajava.blogspot.co.uk/2011/11/why-concurency-examples-are-confusing.html