We Keep Coding

Spring Reactive. How wait for all monos to finish?

I have the following code where I call external APIs via webclient and return Mono.
I need to execute some logic when I receive data. And after all, requests are processed, execute one logic for all gathered data. I can collect all Monos and put them to flux and then execute some logic at the end. But I have serviceName filed which is accessible only in the loop, so I need to execute logic for mono in loop and here I'm stuck and don't know how to wait for all data to complete and do it in a reactive way.
#Scheduled(fixedDelay = 50000)
public void refreshSwaggerConfigurations() {
log.debug("Starting Service Definition Context refresh");
List<SwaggerServiceData> allServicesApi = new ArrayList<>();
swaggerProperties.getUrls().forEach((serviceName, serviceSwaggerUrl) -> {
log.debug("Attempting service definition refresh for Service : {} ", serviceName);
Mono<SwaggerServiceData> swaggerData = getSwaggerDefinitionForAPI(serviceName,
serviceSwaggerUrl);
swaggerData.subscribe(swaggerServiceData -> {
if (swaggerServiceData != null) {
allServicesApi.add(swaggerServiceData);
String content = getJSON(swaggerServiceData);
definitionContext.addServiceDefinition(serviceName, content);
} else {
log.error("Skipping service id : {} Error : Could not get Swagger definition from API ",
serviceName);
}
});
});
//I need to wait here for all monos to complete and after that proceed for All gathered data...
//Now it's empty And I know why, just don't know how to make it.
Optional<SwaggerServiceData> swaggerAllServicesData = getAllServicesApiSwagger(allServicesApi);
if (swaggerAllServicesData.isPresent()) {
String allApiContent = getJSON(swaggerAllServicesData.get());
definitionContext.addServiceDefinition("All", allApiContent);
}
}
private Mono<SwaggerServiceData> getSwaggerDefinitionForAPI(String serviceName, String url) {
log.debug("Accessing the SwaggerDefinition JSON for Service : {} : URL : {} ", serviceName,
url);
Mono<SwaggerServiceData> swaggerServiceDataMono = webClient.get()
.uri(url)
.exchangeToMono(clientResponse -> clientResponse.bodyToMono(SwaggerServiceData.class));
return swaggerServiceDataMono;
}

I would add a temporary class to group data and serivce name :
record SwaggerService(SwaggerServiceData swaggerServiceData, String serviceName) {
boolean hasData() {
return swaggerServiceData != null;
}
}
And then change your pipeline :
Flux.fromStream(swaggerProperties.getUrls().entrySet().stream())
.flatMap((e) -> {
Mono<SwaggerServiceData> swaggerDefinitionForAPI = getSwaggerDefinitionForAPI(e.getKey(),
e.getValue());
return swaggerDefinitionForAPI.map(swaggerServiceData -> new SwaggerService(swaggerServiceData, e.getKey()));
})
.filter(SwaggerService::hasData)
.map(swaggerService -> {
String content = getJSON(swaggerService.swaggerServiceData());
definitionContext.addServiceDefinition(swaggerService.serviceName(), content);
return swaggerService.swaggerServiceData();
})
// here we will collect all datas and they will be emmited as single Mono with list of SwaggerServiceData
.collectList()
.map(this::getAllServicesApiSwagger)
.filter(Optional::isPresent)
.map(Optional::get)
.subscribe(e -> {
String allApiContent = getJSON(e);
definitionContext.addServiceDefinition("All", allApiContent);
});
This does not deal with logging error when SwaggerServiceData is null but you can further change it if you want. Also I assume that DefinitionContext is thread safe.
Solution with error logging (using flatMap and Mono.empty()) :
Flux.fromStream(swaggerProperties.getUrls().entrySet().stream())
.flatMap((e) -> {
Mono<SwaggerServiceData> swaggerDefinitionForAPI = getSwaggerDefinitionForAPI(e.getKey(),
e.getValue());
return swaggerDefinitionForAPI
.flatMap(swaggerServiceData -> {
if(swaggerServiceData != null) {
return Mono.just(new SwaggerService(swaggerServiceData, e.getKey()));
} else {
log.error("Skipping service id : {} Error : Could not get Swagger definition from API ",
e.getKey());
return Mono.empty();
}
});
})
.map(swaggerService -> {
String content = getJSON(swaggerService.swaggerServiceData());
definitionContext.addServiceDefinition(swaggerService.serviceName(), content);
return swaggerService.swaggerServiceData();
}).collectList()
.map(this::getAllServicesApiSwagger)
.filter(Optional::isPresent)
.map(Optional::get)
.subscribe(e -> {
String allApiContent = getJSON(e);
definitionContext.addServiceDefinition("All", allApiContent);
});
You can also wrap those lambads into some meaningful methods to improve readibility.

Keycloak: how to delete and edit a Group

I already tried a bunch of different ways and none of them work.
(First of all im using this, and works with other methods, like create/delete user, create group etc etc)
public void startKeycoak(String username, String password) {
Keycloak kc = KeycloakBuilder.builder()
.serverUrl(uri)
.realm(realmName)
.username(username)
.password(password)
.clientId(client)
.resteasyClient(
new ResteasyClientBuilder()
.connectionPoolSize(10).build())
.build();
this.kc = kc;
}
Problem starts here:
public void deleteGroup(String groupName) {
GroupRepresentation groupRepresentation = kc.realm(realmName)
.groups()
.groups()
.stream()
.filter(group -> group.getName().equals(groupName)).collect(Collectors.toList()).get(0);
// kc.realm(realmName).groups().group(existingGroups.getName()).remove(); -> Not Working
// boolean a = kc.realm(realmName).groups().groups().remove(groupRepresentation); -> Not Workings - returns a false
}
public void updateGroup(String newName, String oldName) {
GroupRepresentation groupRepresentation = kc.realm(realmName)
.groups()
.groups()
.stream()
.filter(group -> group.getName().equals(oldName)).collect(Collectors.toList()).get(0);
//groupRepresentation.setName(newName); -> 1 - Not working
//kc.realm(realmName).groups().groups().stream().filter(g -> { -> 2 - Not Working
//g.setName(oldName);
//return false;
//});
}
Like I said before its working with a lot of methods except those two.

kc.realm(realmName).groups().group(groupRepresentation.getId()).remove();
try to delete it with the group representation id it works.

Is there a cleaner way adapt a standard observer/listener service using RxJava?

I've been tinkering with wrapping an old style listener interface using RxJava. What i've come up with seems to work, but the usage of Observable.using feels a bit awkward.
The requirements are:
Only one subscription per id to the underlying service.
The latest value for a given id should be cached and served to new subscribers.
We must unsubscribe from the underlying service if nothing is listening to an id.
Is there a better way? The following is what I've got.
static class MockServiceRXAdapterImpl1 implements MockServiceRXAdapter {
PublishSubject<MockResponse> mockResponseObservable = PublishSubject.create();
MockService mockService = new MockService(mockResponse -> mockResponseObservable.onNext(mockResponse));
final ConcurrentMap<String, Observable<String>> subscriptionMap = new ConcurrentHashMap<>();
public Observable<String> getObservable(String id) {
return Observable.using(() -> subscriptionMap.computeIfAbsent(
id,
key -> mockResponseObservable.filter(mockResponse -> mockResponse.id.equals(id))
.doOnSubscribe(disposable -> mockService.subscribe(id))
.doOnDispose(() -> {
mockService.unsubscribe(id);
subscriptionMap.remove(id);
})
.map(mockResponse -> mockResponse.value)
.replay(1)
.refCount()),
observable -> observable,
observable -> {
}
);
}
}

You may use Observable.create
So code may look like this
final Map<String, Observable<String>> subscriptionMap = new HashMap<>();
MockService mockService = new MockService();
public Observable<String> getObservable(String id) {
log.info("looking for root observable");
if (subscriptionMap.containsKey(id)) {
log.info("found root observable");
return subscriptionMap.get(id);
} else {
synchronized (subscriptionMap) {
if (!subscriptionMap.containsKey(id)) {
log.info("creating new root observable");
final Observable<String> responseObservable = Observable.create(emitter -> {
MockServiceListener listener = emitter::onNext;
mockService.addListener(listener);
emitter.setCancellable(() -> {
mockServices.removeListener(listener);
mockService.unsubscribe(id);
synchronized (subscriptionMap) {
subscriptionMap.remove(id);
}
});
mockService.subscribe(id);
})
.filter(mockResponse -> mockResponse.id.equals(id))
.map(mockResponse -> mockResponse.value)
.replay(1)
.refCount();
subscriptionMap.put(id, responseObservable);
} else {
log.info("Another thread created the observable for us");
}
return subscriptionMap.get(id);
}
}
}

I think I've gotten it to work using .groupBy(...).
In my case Response.getValue() returns an int, but you get the idea:
class Adapter
{
Subject<Response> msgSubject;
ThirdPartyService service;
Map<String, Observable<Integer>> observables;
Observable<GroupedObservable<String, Response>> groupedObservables;
public Adapter()
{
msgSubject = PublishSubject.<Response>create().toSerialized();
service = new MockThirdPartyService( msgSubject::onNext );
groupedObservables = msgSubject.groupBy( Response::getId );
observables = Collections.synchronizedMap( new HashMap<>() );
}
public Observable<Integer> getObservable( String id )
{
return observables.computeIfAbsent( id, this::doCreateObservable );
}
private Observable<Integer> doCreateObservable( String id )
{
service.subscribe( id );
return groupedObservables
.filter( group -> group.getKey().equals( id ))
.doOnDispose( () -> {
synchronized ( observables )
{
service.unsubscribe( id );
observables.remove( id );
}
} )
.concatMap( Functions.identity() )
.map( Response::getValue )
.replay( 1 )
.refCount();
}
}

Chaining async methods in a loop with RxJava

I have this code and I want to know if this is possible with RxJava:
Function queries for a User from a server (async)
The server returns a User JSON object with a list of ID's of associated UserProfile(s)
Then for each of this ID, it needs to fetch the UserProfile given the ID (async also)
For each asynchronously fetched UserProfile append it to the User object, below is my pseudo-code.
I cannot use any blocking code, all request should be async.
Here's the code:
#Override
public Single<User> retrieve(String entityId) {
BaasUser baasUser = new BaasUser();
baasUser.setEntityId(entityId);
baasUser.setIncludes(Arrays.asList("userProfile"));
return baasUser.retrieve().map(user -> {
String username = user.getUsername();
String dateCreated = user.getDateCreated();
String dateUpdated = user.getDateUpdated();
List<UserProfile> userProfiles = new LinkedList<>();
BaasLink userProfileLink = user.getFirstLink();
userProfileLink.getEntities().forEach(stubEntity -> {
Single<UserProfile> observable = stubEntity.retrieve().map(entity -> {
UserProfile userProfile = new UserProfile();
userProfile.setEntityId(entity.getStringProperty("entityId"));
userProfile.setName(entity.getStringProperty("name"));
return userProfile;
});
observable.subscribe(userProfile -> {
// until all UserProfile is fetched this 'retrieve' "callback" should not return
userProfiles.add(userProfile);
}, error -> {
// err
});
});
User user1 = new User();
user1.setDateCreated(dateCreated);
user1.setDateUpdated(dateUpdated);
user1.setUsername(username);
user1.setUserProfiles(userProfiles);
return user1;
});
}

Here you have en example how to do your jobe maybe there is any typeerror becouse i dont have your objects
Single.just("user")
.observeOn(Schedulers.io())
.flatMap(user -> Observable.zip(getCurrentUserData(user),getUserProfiles(user),(t1, t2) -> {
//first func will return user eith some data next function wll return List<UserProfile> userProfiles
return newuser;
}))
.subscribeOn(Schedulers.io())
}
private Single<List<UserProfile>> getUserProfiles(User user) {
Observable.fromIterable( user.getFirstLink().getEntities())
.flatMap(stubEntity ->stubEntity.retrieve())
.map(o -> {
//set user profile data
return userprofile
})
.toList();
}
private Single<User> getCurrentUserData(User user) {
Observable.just(user)
.map(s -> {
//set data
return user;
})
}

rxjava - getting response and inserting diff parallelly

I am using rxjava for parallel processing of two requests using Observable.zip. What I am trying to do is , in one observable say response I am getting one response and in other observable say diff I am trying to get the response and save this difference in DB. The problem is I am not sure how to achieve my requirement as the diff observable is not getting completed if response observable gets the response
Here is what I am doing ...
public ServiceResponse getDummyResponse(ServiceRequest serviceRequest, String prodId){
Observable<ServiceResponse> subInfoDummyObservable = getDummyResonseGenericObservable();
Observable<ServicesDiff> reObservable = getServicesDiffGenericObservable(serviceRequest, prodId);
Observable<ServiceResponse> responseObservable = Observable.zip(
subInfoDummyObservable,
reObservable,
new Func2<ServiceResponse, ServicesDiff, ServiceResponse>() {
#Override
public ServiceResponse call(ServiceResponse serviceResponse, ServicesDiff diffResponse) {
return serviceResponse;
}
}
);
ServiceResponse serviceResponse = responseObservable.toBlocking().single();
return serviceResponse;
}
Observable<ServiceResponse> getDummyResonseGenericObservable() {
return GenericHystrixCommand.toObservable("getDummyResonseGenericObservable", "getDummyResonseGenericObservable", () -> new ServiceResponse(),(t) -> {return null;} );
}
Observable<ServicesDiff> getServicesDiffGenericObservable(ServiceRequest serviceRequest, String prodId) {
return GenericHystrixCommand.toObservable("getServicesDiffGenericObservable", "getServicesDiffGenericObservable", () -> getBothServiceResponses(serviceRequest, prodId),(t) -> {return null;} );
}
public ServicesDiff getBothServiceResponses(ServiceRequest serviceRequest, String prodId) {
Observable<String> service1ResponseObservable = getService1GenericObservable(prodId);
Observable<ServiceResponse> service2ResponseObservable = getService2GenericObservable(serviceRequest, prodId);
Observable<ServicesDiff> observable = Observable.zip(
service1ResponseObservable, service2ResponseObservable,
new Func2<String, ServiceResponse, ServicesDiff>() {
#Override
public ServicesDiff call(String service1Response, ServiceResponse service2Response) {
return aggregate(service1Response, service2Response); // never reaches this point**********
}
}
);
ServicesDiff response = observable.toBlocking().single();
return response;
}
I am inserting the diff to DB in aggregate method but it never reaches to aggregate at all. Please let me know what I am doing wrong here? Thanks.

Observable are a description of how to consume data. In your code sample, you don't subscribe, you don't actually consume the data. You just described how to request, but the subscribe part, the part that trigger the requests, is missing.
So if I rewrite a little your code:
class Aggregate {
Aggregate(String reponse, ServicesDiff diff) {
...
}
}
Observable<String> getService1GenericObservable(String prodId) {
...
}
Observable<ServicesDiff> getServicesDiffGenericObservable(ServiceRequest serviceRequest, String prodId) {
...
}
public Observable<Aggregate> getBothServiceResponses(ServiceRequest serviceRequest, String prodId) {
Observable<String> service1ResponseObservable = getService1GenericObservable(prodId);
Observable<ServiceResponse> service2ResponseObservable = getService2GenericObservable(serviceRequest, prodId);
return Observable<Aggregate> observable = Observable.zip(
service1ResponseObservable, service2ResponseObservable,
new Func2<String, ServiceResponse, ServicesDiff>() {
#Override
public ServicesDiff call(String service1Response, ServiceResponse service2Response) {
return aggregate(service1Response, service2Response);
}
}
);
}
You will just have to do this to access the result aggregate:
getBothServiceResponses(serviceRequest, prodId).subscribe(...)