Sorry, I'm new to Play Framework.
I use it with Java API.
Let's say I want to have a controller action that runs some kind of import and displays result after import is finished.
Import requires expensive HTTP communication with the 3rd party service (fetching data from 3 URLs, processing data, updating database after all 3 resources were processed).
So I'd like to implement an import itself as a Promise in controller (Controller shouldn't be aware of import implementation).
Then I'd like to run fetching the data from URLs and processing in 3 parallel threads. I think it would be nice to implement it as a 3 separate Promises.
Database should be updated only when (and if) all three promises were completed successfully.
And finally controller should be notified after database was updated.
I'm able to implement the whole import as a Promise, but I don't know how to implement nested promises.
Could you suggest how to implement it or correct me if I'm trying to use wrong approach?
You can achieve this with flatmap. The syntax in Java is sadly a bit clunky because of the anonymous interfaces (will get better with Java 8 and lambdas). Promise<T>.flatMap accepts a Function<T, Promise<U>> and will return a Promise<U>. This means you can nest flatMaps from all your three operations and collect them with a flatmap, like this:
final Promise<String> promise1 = Promise.pure("one");
final Promise<String> promise2 = Promise.pure("two");
final Promise<String> promise3 = Promise.pure("three");
Promise<String> allThreeCombined = promise1.flatMap(new Function<String, Promise<String>>() {
#Override
public Promise<String> apply(final String result1) throws Throwable {
return promise2.flatMap(new Function<String, Promise<String>>() {
#Override
public Promise<String> apply(final String result2) throws Throwable {
return promise3.map(new Function<String, String>() {
#Override
public String apply(String result3) throws Throwable {
return result1 + result2 + result3;
}
});
}
});
}
});
If there is no special meaning to each of the different things you are fetching - for example if they are to be treated as a list of values you can also use Promise.sequence() which accepts a list of Promise<T> and return a Promise<List<T>> so you can react on all values arriving.
Related
I am working on a project which provides a list of operations to be done on an entity, and each operation is an API call to the backend. Let's say the entity is a file, and operations are convert, edit, copy. There are definitely easier ways of doing this, but I am interested in an approach which allows me to chain these operations, similar to intermediate operations in java Streams, and then when I hit a terminal operation, it decides which API call to execute, and performs any optimisation that might be needed. My API calls are dependent on the result of other operations. I was thinking of creating an interface
interface operation{
operation copy(Params ..); //intermediate
operation convert(Params ..); // intermediate
operation edit(Params ..); // intermediate
finalresult execute(); // terminal op
}
Now each of these functions might impact the other based on the sequence in which the pipeline is created. My high level approach would be to just save the operation name and params inside the individual implementation of operation methods and use that to decide and optimise anything I'd like in the execute method. I feel that is a bad practice since I am technically doing nothing inside the operation methods, and this feels more like a builder pattern, while not exactly being that. I'd like to know the thoughts on my approach. Is there a better design for building operation pipelines in java?
Apologies if the question appears vague, but I am basically looking for a way to build an operation pipeline in java, while getting my approach reviewed.
You should look at a pattern such as
EntityHandler.of(remoteApi, entity)
.copy()
.convert(...)
.get();
public class EntityHandler {
private final CurrentResult result = new CurrentResult();
private final RemoteApi remoteApi;
private EntityHandler(
final RemoteApi remoteApi,
final Entity entity) {
this.remoteApi = remoteApi;
this.result.setEntity(entity);
}
public EntityHandler copy() {
this.result.setEntity(new Entity(entity)); // Copy constructor
return this;
}
public EntityHandler convert(final EntityType type) {
if (this.result.isErrored()) {
throw new InvalidEntityException("...");
}
if (type == EntityType.PRIMARY) {
this.result.setEntity(remoteApi.convertToSecondary(entity));
} else {
...
}
return this:
}
public Entity get() {
return result.getEntity();
}
public static EntityHandler of(
final RemoteApi remoteApi,
final Entity entity) {
return new EntityHandler(remoteApi, entity);
}
}
The key is to maintain the state immutable, and handle thread-safety on localized places, such as in CurrentResult, in this case.
I need to create an observable where 2 blocks of retrofit based calls are run sequentially. I know I can just run the second retrofit call inside one Observer call but it will be pretty messy. I have my Observable code in a separate class from the caller and it returns an Observable. I'd like to pass the result of the first call to the second then when the second call is done pass back an Observable to the calling class. (I'm using Java 7 and not 8)
public class GetFollowing {
public Observable< ArrayList<Media> > init() {
return Observable.create(
new Observable.OnSubscribe< ArrayList<Media> >() {
#Override
public void call(Subscriber<? super ArrayList<Media> > subscriber) {
...
I also need to pass back to the calling class a different type than I pass to teh second retrofit call. I been reading about map flatMap and concat but I can't seem to figure out how to structure them for my use here.
UPDATE
I came up with this, not sure if its the most elegant or if it will work at all...but if it does work is there any way to pass the result of first observable to second? Also how would I handle an issue if first observable fails?
Observable< ArrayList<Media> > test;
Observable.concat(
Observable.create(
new Observable.OnSubscribe< ArrayList<User> >() {
#Override
public void call(Subscriber<? super ArrayList<User> > subscriber) {
}
}
),
test = Observable.create(
new Observable.OnSubscribe< ArrayList<Media> >() {
#Override
public void call(Subscriber<? super ArrayList<Media> > subscriber) {
}
}
)
);
return test;
If the the requirement can be rephrased as below:
You have two methods to be executed and both return Observables.
The items emitted on first method's Observable needs to be fed into the second method as and when they occur.
The output of second method is an Observable which is based on some computation on items of first Observable.
The readily available flatMap feature in RxJava is the solution for you. Below is a simple implementation to assist you.
public static void main(String[] args) {
Observable<Integer> o1 = Observable.just(1, 2);
Observable<String> result = o1.flatMap(result1 -> Observable.just("Value is: "+result1));
result.subscribe(finalResult -> System.out.println("Final result: "+finalResult));
}
Output:
Final result: Value is: 1
Final result: Value is: 2
On the other side, if second method does not return an Observable, but performs some operation on the emitted item, you can implement the same using map.
I am trying to learn the Play Framework 2.4. I am trying to get the time it takes to access different webpages asynchronously using Promise. Below is the code for that:
final long start = System.currentTimeMillis();
F.Function<WSResponse,Long> timing = new F.Function<WSResponse, Long>() {
#Override
public Long apply(WSResponse wsResponse) throws Throwable {
return System.currentTimeMillis()-start;
}
};
F.Promise<Long> google = ws.url("http://www.google.com").get().map(timing);
F.Promise<Long> yahoo = ws.url("http://www.yahoo.com").get().map(timing);
F.Promise<Long> bing = ws.url("http://www.bing.com").get().map(timing);
As you can see I am using the get function to get the requested pages and putting the result in a Future Promise. Then I convert/map it to long. What I am not able to do is how do I compose these three promises into one and once all of the three promises are redeemed convert/map it to json and return the result. In earlier versions of Play it could have been done by F.Promise.waitAll(google,yahoo,bing).map(...) however I am unable to do it in Play 2.4. Please advice
EDIT1: Based on the answer below i used sequence like below:
return F.Promise.sequence(google, yahoo, bing).map(new F.Function<List<Long>, Result>() {
#Override
public Result apply(List<Long> longs) throws Throwable {
Map<String, Long> data = new HashMap<String, Long>();
data.put("google", google.get());
data.put("yahoo", yahoo.get());
data.put("bing", bing.get());
return ok(Json.toJson(data));
}
});
However, i am getting error that google.get() method cannot be resolved and that Json cannot be applied. What am i missing here?
EDIT 2. The Json error was fixed by using return ok((JsonNode) Json.toJson((Writes<Object>) data)); However, i am still not able to resolve the earlier error that google.get() method cannot be resolved in the line data.put("google", google.get());
EDIT 3. It seems Play2.4 has no get() method which returns the value of a Promise once it has been redeemed. What should i use then?
waitAll has been replaced with F.Promise.sequence.
From the docs
public static <A> F.Promise<java.util.List<A>> sequence(java.lang.Iterable<F.Promise<A>> promises)
Combine the given promises into a single promise for the list of results. The sequencing operations are performed in the default ExecutionContext.
Parameters:
promises - The promises to combine
Returns:
A single promise whose methods act on the list of redeemed promises
Update
Regarding the second half of the question, you don't need to call .get() because the promises have already completed.
In fact, you can get rid of the individual promise variables and just pass them directly into the sequence. The resulting list will contain results in the same order (Google first, then Yahoo, then Bing, in this case).
The whole thing should look something like this:
package controllers;
import java.util.HashMap;
import java.util.Map;
import play.libs.F;
import play.libs.Json;
import play.libs.ws.WS;
import play.libs.ws.WSResponse;
import play.mvc.Controller;
import play.mvc.Result;
import play.mvc.Results;
public class Application extends Controller {
public F.Promise<Result> index() {
final long start = System.currentTimeMillis();
final F.Function<WSResponse,Long> timing = response -> System.currentTimeMillis() - start;
return F.Promise.sequence(WS.url("http://www.google.com").get().map(timing),
WS.url("http://www.yahoo.com").get().map(timing),
WS.url("http://www.bing.com").get().map(timing))
.map(list -> {
Map<String, Long> data = new HashMap<>();
data.put("google", list.get(0));
data.put("yahoo", list.get(1));
data.put("bing", list.get(2));
return data;
})
.map(Json::toJson)
.map(Results::ok);
}
}
Finally, since Play 2.4 requires Java 8, this is a good opportunity to play around with lambdas!
I am starting to embrace reactive programming a bit more, and I'm trying to apply it to my typical business problems. One pattern I often design with is database-driven classes. I have some defined unit class like ActionProfile whose instances are managed by an ActionProfileManager, which creates the instances off a database table and stores them in a Map<Integer,ActionProfile> where Integer is the actionProfileId key. The ActionProfileManager may clear and re-import the data periodically, and notify all dependencies to re-pull from its map.
public final class ActionProfileManager {
private volatile ImmutableMap<Integer,ActionProfile> actionProfiles;
private ActionProfileManager() {
this.actionProfiles = importFromDb();
}
public void refresh() {
this.actionProfiles = importFromDb();
notifyEventBus();
}
//called by clients on their construction or when notifyEventBus is called
public ActionProfile forKey(int actionProfileId) {
return actionProfiles.get(actionProfiles);
}
private ImmutableMap<Integer,ActionProfile> importFromDb() {
return ImmutableMap.of(); //import data here
}
private void notifyEventBus() {
//notify event through EventBus here
}
}
However, if I want this to be more reactive creating the map would kind of break the monad. One approach I could do is make the Map itself an Observable, and return a monad that looks up a specific key for the client. However the intermediate imperative operations may not be ideal, especially if I start using the rxjava-jdbc down the road. But the hashmap may help lookup performance significantly in intensive cases.
public final class ActionProfileManager {
private final BehaviorSubject<ImmutableMap<Integer,ActionProfile>> actionProfiles;
private ActionProfileManager() {
this.actionProfiles = BehaviorSubject.create(importFromDb());
}
public void refresh() {
actionProfiles.onNext(importFromDb());
}
public Observable<ActionProfile> forKey(int actionProfileId) {
return actionProfiles.map(m -> m.get(actionProfileId));
}
private ImmutableMap<Integer,ActionProfile> importFromDb() {
return ImmutableMap.of(); //import data here
}
}
Therefore, the most reactive approach to me seems to be just pushing everything from the database on each refresh through an Observable<ActionProfile> and filtering for the last matching ID for the client.
public final class ActionProfileManager {
private final ReplaySubject<ActionProfile> actionProfiles;
private ActionProfileManager() {
this.actionProfiles = ReplaySubject.create();
importFromDb();
}
public void refresh() {
importFromDb();
}
public Observable<ActionProfile> forKey(int actionProfileId) {
return actionProfiles.filter(m -> m.getActionProfileID() == actionProfileId).last();
}
private void importFromDb() {
// call onNext() on actionProfiles and pass each new ActionProfile coming from database
}
}
Is this the optimal approach? What about old data causing memory leaks and not being GC'd? Is it more practical to maintain the map and make it observable?
What is the most optimal reactive approach above to data driven classes? Or is there a better way I have not discovered?
Using BehaviorSubject is the right thing to do here if you don't care about earlier values.
Note most post discouraging Subjects were written in the early days of Rx.NET and is mostly quoted over and over again without much thought. I attribute this to the possibility that such authors didn't really understand how Subjects work or run into some problems with them and just declared they shouldn't be used.
I think Subjects are a great way to multicast events (coming from a single thread usually) where you control or you are the source of the events and the event dispatching is somewhat 'global' (such as listening to mouse move events).
I have provided a callback to a third party library that calls the provided method at various times providing me with an object that has changed. I am then carrying out an async web request to get further details and set them on that object, below is a made up similar example;
public void update(Person person) {
if (person.getId() == -1) {
mService.getPersonDetails()
.flatMap(..)
.skip(..)
.subscribe(personResult -> person.setId(personResult.getId()))
}
}
The update is called quite a few times and should only executes the query if the object has no ID. The problem is that at least two requests get sent off as the first query has not yet completed.
How can I synchronise this method call so that only one request is sent for each Object that get passed via the callback? I only want to block requests for that exact Object, so if the update() is supplying different objects it would be ok for new requests to be sent out.
The solution provided by Adam S looks good but soon or later will cause OOM problems. It is due to distinct operator which has to store all unique values.
Other option that comes to my mind is usage of ConcurrentMap to store processed persons and doOnTerminate to clean it.
private Map<Person, Boolean> map = new ConcurrentHashMap<>();
public void update(final Person person) {
if (person.getId() == -1) {
if(map.putIfAbsent(person, true)==null){
mService.getPersonDetails()
.flatMap(..)
.skip(..)
.doOnTerminate(()->map.remove(person))
.subscribe(personResult -> person.setId(personResult.getId()))
}
}
}
You can filter the inputs to your observable using the distinct operator. Here's a general idea of how you could do that using a PublishSubject (JavaDoc) (note this is written from memory, I haven't tested this):
private PublishSubject<Person> personSubject;
public void update(Person person) {
if (personSubject == null) {
personSubject = new PublishSubject();
personSubject
.filter(person -> person.getId() == -1)
.distinct()
.flatMap(person -> mService.getPersonDetails())
.skip(..)
.subscribe(personResult -> person.setId(personResult.getId()));
}
personSubject.onNext(person);
}
You will, of course, have to either implement the equals method on your Person class (which, as Marek points out, will result in all objects passed in being cached in memory) or implement the distinct(Func) variant.
That method takes a 'key selector' function used to differentiate between objects. If your objects are fairly heavy and you're concerned about memory (if you're on Android, for example) this might be a better path. Something like this:
.distinct(new Func1<Person, Integer>() {
#Override
public Integer call(Person person) {
return person.hashCode();
}
})
Well you can just simply synchronize it.
public synchronized void update(Person person)