We Keep Coding

Spring Reactive. How wait for all monos to finish?

I have the following code where I call external APIs via webclient and return Mono.
I need to execute some logic when I receive data. And after all, requests are processed, execute one logic for all gathered data. I can collect all Monos and put them to flux and then execute some logic at the end. But I have serviceName filed which is accessible only in the loop, so I need to execute logic for mono in loop and here I'm stuck and don't know how to wait for all data to complete and do it in a reactive way.
#Scheduled(fixedDelay = 50000)
public void refreshSwaggerConfigurations() {
log.debug("Starting Service Definition Context refresh");
List<SwaggerServiceData> allServicesApi = new ArrayList<>();
swaggerProperties.getUrls().forEach((serviceName, serviceSwaggerUrl) -> {
log.debug("Attempting service definition refresh for Service : {} ", serviceName);
Mono<SwaggerServiceData> swaggerData = getSwaggerDefinitionForAPI(serviceName,
serviceSwaggerUrl);
swaggerData.subscribe(swaggerServiceData -> {
if (swaggerServiceData != null) {
allServicesApi.add(swaggerServiceData);
String content = getJSON(swaggerServiceData);
definitionContext.addServiceDefinition(serviceName, content);
} else {
log.error("Skipping service id : {} Error : Could not get Swagger definition from API ",
serviceName);
}
});
});
//I need to wait here for all monos to complete and after that proceed for All gathered data...
//Now it's empty And I know why, just don't know how to make it.
Optional<SwaggerServiceData> swaggerAllServicesData = getAllServicesApiSwagger(allServicesApi);
if (swaggerAllServicesData.isPresent()) {
String allApiContent = getJSON(swaggerAllServicesData.get());
definitionContext.addServiceDefinition("All", allApiContent);
}
}
private Mono<SwaggerServiceData> getSwaggerDefinitionForAPI(String serviceName, String url) {
log.debug("Accessing the SwaggerDefinition JSON for Service : {} : URL : {} ", serviceName,
url);
Mono<SwaggerServiceData> swaggerServiceDataMono = webClient.get()
.uri(url)
.exchangeToMono(clientResponse -> clientResponse.bodyToMono(SwaggerServiceData.class));
return swaggerServiceDataMono;
}

I would add a temporary class to group data and serivce name :
record SwaggerService(SwaggerServiceData swaggerServiceData, String serviceName) {
boolean hasData() {
return swaggerServiceData != null;
}
}
And then change your pipeline :
Flux.fromStream(swaggerProperties.getUrls().entrySet().stream())
.flatMap((e) -> {
Mono<SwaggerServiceData> swaggerDefinitionForAPI = getSwaggerDefinitionForAPI(e.getKey(),
e.getValue());
return swaggerDefinitionForAPI.map(swaggerServiceData -> new SwaggerService(swaggerServiceData, e.getKey()));
})
.filter(SwaggerService::hasData)
.map(swaggerService -> {
String content = getJSON(swaggerService.swaggerServiceData());
definitionContext.addServiceDefinition(swaggerService.serviceName(), content);
return swaggerService.swaggerServiceData();
})
// here we will collect all datas and they will be emmited as single Mono with list of SwaggerServiceData
.collectList()
.map(this::getAllServicesApiSwagger)
.filter(Optional::isPresent)
.map(Optional::get)
.subscribe(e -> {
String allApiContent = getJSON(e);
definitionContext.addServiceDefinition("All", allApiContent);
});
This does not deal with logging error when SwaggerServiceData is null but you can further change it if you want. Also I assume that DefinitionContext is thread safe.
Solution with error logging (using flatMap and Mono.empty()) :
Flux.fromStream(swaggerProperties.getUrls().entrySet().stream())
.flatMap((e) -> {
Mono<SwaggerServiceData> swaggerDefinitionForAPI = getSwaggerDefinitionForAPI(e.getKey(),
e.getValue());
return swaggerDefinitionForAPI
.flatMap(swaggerServiceData -> {
if(swaggerServiceData != null) {
return Mono.just(new SwaggerService(swaggerServiceData, e.getKey()));
} else {
log.error("Skipping service id : {} Error : Could not get Swagger definition from API ",
e.getKey());
return Mono.empty();
}
});
})
.map(swaggerService -> {
String content = getJSON(swaggerService.swaggerServiceData());
definitionContext.addServiceDefinition(swaggerService.serviceName(), content);
return swaggerService.swaggerServiceData();
}).collectList()
.map(this::getAllServicesApiSwagger)
.filter(Optional::isPresent)
.map(Optional::get)
.subscribe(e -> {
String allApiContent = getJSON(e);
definitionContext.addServiceDefinition("All", allApiContent);
});
You can also wrap those lambads into some meaningful methods to improve readibility.

FlatMap is not being called after a webclient call

I have this function inside a #Service in Spring Webflux and it is called with a list of friends to know if everyone has been joined to a group. If some friends have no group this method calls an API to get its user's information and then calls another API to tag these guys with joined tag false.
#Service
...
public Flux<Boolean> checkUserHaveGroup(final List<String> friends) {
MatchOperation match1 = Aggregation.match(Criteria.where("friends").in(friends).and("status").is("ACTIVE"));
UnwindOperation unwind1 = Aggregation.unwind("friends");
MatchOperation match2 = Aggregation.match(Criteria.where("friends").in(friends));
GroupOperation group1 = Aggregation.group("friends");
TypedAggregation<Group> a = Aggregation.newAggregation(
Group.class,
match1, unwind1, match2, group1);
return this.reactiveMongoTemplate.aggregate(a, FriendInGroup.class)
.map(friendInGroup -> friendInGroup.id)
.collectList()
.map(users -> haveNoGroupsList(users, friends))
.flatMapMany(noGroupUsers -> {
return Flux.fromIterable(noGroupUsers)
.flatMap(pn -> crmService.deleteAttribute(pn, "joinedAGroup"));
});
}
(this method get the user information)
...
public Mono<UserInfo> userInfoById(final String userId) {
return webClient.get()
.uri(uriBuilder -> uriBuilder.path(constants.getByIdPath() + "/{id}")
.build(userId))
.header("auth", tokenService.token())
.exchange()
.flatMap(response -> {
Mono<UserInfo> responseMono;
if (response.statusCode().equals(HttpStatus.UNAUTHORIZED)) {
responseMono = Mono.error(new UnauthorizedException());
} else if (response.statusCode().equals(HttpStatus.OK)) {
responseMono = response.bodyToMono(UserInfoResponse.class)
.flatMap(uir -> Mono.just(uir.getData()));
} else {
responseMono = Mono.error(new UnhandledException());
}
return responseMono;
});
}
...
private Mono<UserInfo> getUserInfo(String userId) {
return userInfoAdapter.userInfoById(userId);
}
...
public Mono<Boolean> deleteAttribute(final String userId, final String attribute) {
return getUserInfo(userId) <<<<< here we get the users info
.flatMap(ui -> crmDeleteAttribute(ui, attribute)); <<<< this call is never done.
}
...
public Mono<Boolean> crmDeleteAttribute(final UserInfo user, final String attribute) {
return webClient.delete()
.uri(uriBuilder -> uriBuilder
.path(contants.path())
.build(user.getId(), attribute))
.header("auth", tokenService.token())
.exchange().flatMap(response -> {
if (response.statusCode().equals(HttpStatus.OK)) {
return Mono.just(Boolean.TRUE);
}
if (response.statusCode().equals(HttpStatus.BAD_REQUEST)) {
return Mono.error(CrmServiceBadRequestException::new);
}
if (response.statusCode().equals(HttpStatus.UNAUTHORIZED)) {
return Mono.error(CrmServiceUnauthorizedException::new);
}
return Mono.error(CrmServiceUnhandledException::new);
});
}
After getting the users' info on the API, the API for assign a tag is never been called no matter what I do. I can see in the debugger terminal that the call to UserInfo API was done but after that, the application returns to the controller. Someone could point me to what I'm doing wrong?
Any help is welcome.
Thanks!

Update row if references is exist, otherwise delete using spring boot

Need guideline -
How to do hard delete when no reference is available and do soft delete when reference is available, this operation should be performed in a single method itself.
E.g.
I have 1 master table and 3 transactional tables and the master reference is available in all 3 transactional tables.
Now while deleting master row - I have to do the following: If master reference is available then update the master table row and if no master ref. is available delete the row.
I tried following so far.
Service Implementation -
public response doHardOrSoftDelete(Employee emp) {
boolean flag = iMasterDao.isDataExist(emp);
if(flag) {
boolean result = iMasterDao.doSoftDelete(emp);
} else {
boolean result = iMasterDao.doHardDelete(emp);
}
}
Second Approach:
As we know that while deleting a record if the reference is available then it throws ConstraintViolationException so simply we can catch it and check that caught exception is of type ConstraintViolationException or not, if yes then call doSoftDelete() method and return the response. So here you don't need to write method or anything to check the references. But I'm not sure whether it is the right approach or not. Just help me with it.
Here is what I tried again -
public Response deleteEmployee(Employee emp) {
Response response = null;
try{
String status= iMasterDao.deleteEmployeeDetails(emp);
if(status.equals("SUCCESS")) {
response = new Response();
response.setStatus("Success");
response.setStatusCode("200");
response.setResult("True");
response.setReason("Record deleted successfully");
return response;
}else {
response = new Response();
response.setStatus("Fail");
response.setStatusCode("200");
response.setResult("False");
}
}catch(Exception e){
response = new Response();
Throwable t =e.getCause();
while ((t != null) && !(t instanceof ConstraintViolationException)) {
t = t.getCause();
}
if(t instanceof ConstraintViolationException){
boolean flag = iMasterDao.setEmployeeIsDeactive(emp);
if(flag) {
response.setStatus("Success");
response.setStatusCode("200");
response.setResult("True");
response.setReason("Record deleted successfully");
}else{
response.setStatus("Fail");
response.setStatusCode("200");
response.setResult("False");
}
}else {
response.setStatus("Fail");
response.setStatusCode("500");
response.setResult("False");
response.setReason("# EXCEPTION : " + e.getMessage());
}
}
return response;
}
Dao Implementation -
public boolean isDataExist(Employee emp) {
boolean flag = false;
List<Object[]> tbl1 = session.createQuery("FROM Table1 where emp_id=:id")
.setParameter("id",emp.getId())
.getResultList();
if(!tbl1.isEmpty() && tbl1.size() > 0) {
flag = true;
}
List<Object[]> tbl2 = session.createQuery("FROM Table2 where emp_id=:id")
.setParameter("id",emp.getId())
.getResultList();
if(!tbl2.isEmpty() && tbl2.size() > 0) {
flag = true;
}
List<Object[]> tbl3 = session.createQuery("FROM Table3 where emp_id=:id")
.setParameter("id",emp.getId())
.getResultList();
if(!tbl3.isEmpty() && tbl3.size() > 0) {
flag = true;
}
return flag;
}
public boolean doSoftDelete(Employee emp) {
empDet = session.get(Employee.class, emp.getId());
empDet .setIsActive("N");
session.update(empDet);
}
public boolean doHardDelete(Employee emp) {
empDet = session.get(Employee.class, emp.getId());
session.delete(empDet);
}
No matter how many transactional tables will be added with master tbl reference, my code should do the operations(soft/hard delete) accordingly.
In my case, every time new transactional tables get added with a master reference I've do the checks, so Simply I want to skip the isDataExist() method and do the deletions accordingly, how can I do it in a better way?
Please help me with the right approach to do the same.

There's a lot of repeated code in the body of isDataExist() method which is both hard to maintain and hard to extend (if you have to add 3 more tables the code will double in size).
On top of that the logic is not optimal as it will go over all tables even if the result from the first one is enough to return true.
Here is a simplified version (please note that I haven't tested the code and there could be errors, but it should be enough to explain the concept):
public boolean isDataExist(Employee emp) {
List<String> tableNames = List.of("Table1", "Table2", "Table3");
for (String tableName : tableNames) {
if (existsInTable(tableName, emp.getId())) {
return true;
}
}
return false;
}
private boolean existsInTable(String tableName, Long employeeId) {
String query = String.format("SELECT count(*) FROM %s WHERE emp_id=:id", tableName);
long count = (long)session
.createQuery(query)
.setParameter("id", employeeId)
.getSingleResult();
return count > 0;
}
isDataExist() contains a list of all table names and iterates over these until the first successful encounter of the required Employee id in which case it returns true. If not found in any table the method returns false.
private boolean existsInTable(String tableName, Long employeeId) is a helper method that does the actual search for employeeId in the specified tableName.
I changed the query to just return the count (0 or more) instead of a the actual entity objects as these are not required and there's no point to fetch them.
EDIT in response to the "Second approach"
Is the Second Approach meeting the requirements?
If so, then it is a "right approach" to the problem. :)
I would refactor the deleteEmployeeDetails method to either return a boolean (if just two possible outcomes are expected) or to return a custom Enum as using a String here doesn't seem appropriate.
There is repeated code in deleteEmployeeDetails and this is never a good thing. You should separate the logic which decides the type of the response from the code that builds it, thus making your code easier to follow, debug and extend when required.
Let me know if you need a code example of the ideas above.
EDIT #2
Here is the sample code as requested.
First we define a Status enum which should be used as return type from MasterDao's methods:
public enum Status {
DELETE_SUCCESS("Success", "200", "True", "Record deleted successfully"),
DELETE_FAIL("Fail", "200", "False", ""),
DEACTIVATE_SUCCESS("Success", "200", "True", "Record deactivated successfully"),
DEACTIVATE_FAIL("Fail", "200", "False", ""),
ERROR("Fail", "500", "False", "");
private String status;
private String statusCode;
private String result;
private String reason;
Status(String status, String statusCode, String result, String reason) {
this.status = status;
this.statusCode = statusCode;
this.result = result;
this.reason = reason;
}
// Getters
}
MasterDao methods changed to return Status instead of String or boolean:
public Status deleteEmployeeDetails(Employee employee) {
return Status.DELETE_SUCCESS; // or Status.DELETE_FAIL
}
public Status deactivateEmployee(Employee employee) {
return Status.DEACTIVATE_SUCCESS; // or Status.DEACTIVATE_FAIL
}
Here is the new deleteEmployee() method:
public Response deleteEmployee(Employee employee) {
Status status;
String reason = null;
try {
status = masterDao.deleteEmployeeDetails(employee);
} catch (Exception e) {
if (isConstraintViolationException(e)) {
status = masterDao.deactivateEmployee(employee);
} else {
status = Status.ERROR;
reason = "# EXCEPTION : " + e.getMessage();
}
}
return buildResponse(status, reason);
}
It uses two simple utility methods (you can make these static or export to utility class as they do not depend on the internal state).
First checks if the root cause of the thrown exception is ConstraintViolationException:
private boolean isConstraintViolationException(Throwable throwable) {
Throwable root = throwable;
while (root != null && !(root instanceof ConstraintViolationException)) {
root = root.getCause();
}
return root != null;
}
And the second one builds the Response out of the Status and a reason:
private Response buildResponse(Status status, String reason) {
Response response = new Response();
response.setStatus(status.getStatus());
response.setStatusCode(status.getStatusCode());
response.setResult(status.getResult());
if (reason != null) {
response.setReason(reason);
} else {
response.setReason(status.getReason());
}
return response;
}
If you do not like to have the Status enum loaded with default Response messages, you could strip it from the extra info:
public enum Status {
DELETE_SUCCESS, DELETE_FAIL, DEACTIVATE_SUCCESS, DEACTIVATE_FAIL, ERROR;
}
And use switch or if-else statements in buildResponse(Status status, String reason) method to build the response based on the Status type.

rxjava - getting response and inserting diff parallelly

I am using rxjava for parallel processing of two requests using Observable.zip. What I am trying to do is , in one observable say response I am getting one response and in other observable say diff I am trying to get the response and save this difference in DB. The problem is I am not sure how to achieve my requirement as the diff observable is not getting completed if response observable gets the response
Here is what I am doing ...
public ServiceResponse getDummyResponse(ServiceRequest serviceRequest, String prodId){
Observable<ServiceResponse> subInfoDummyObservable = getDummyResonseGenericObservable();
Observable<ServicesDiff> reObservable = getServicesDiffGenericObservable(serviceRequest, prodId);
Observable<ServiceResponse> responseObservable = Observable.zip(
subInfoDummyObservable,
reObservable,
new Func2<ServiceResponse, ServicesDiff, ServiceResponse>() {
#Override
public ServiceResponse call(ServiceResponse serviceResponse, ServicesDiff diffResponse) {
return serviceResponse;
}
}
);
ServiceResponse serviceResponse = responseObservable.toBlocking().single();
return serviceResponse;
}
Observable<ServiceResponse> getDummyResonseGenericObservable() {
return GenericHystrixCommand.toObservable("getDummyResonseGenericObservable", "getDummyResonseGenericObservable", () -> new ServiceResponse(),(t) -> {return null;} );
}
Observable<ServicesDiff> getServicesDiffGenericObservable(ServiceRequest serviceRequest, String prodId) {
return GenericHystrixCommand.toObservable("getServicesDiffGenericObservable", "getServicesDiffGenericObservable", () -> getBothServiceResponses(serviceRequest, prodId),(t) -> {return null;} );
}
public ServicesDiff getBothServiceResponses(ServiceRequest serviceRequest, String prodId) {
Observable<String> service1ResponseObservable = getService1GenericObservable(prodId);
Observable<ServiceResponse> service2ResponseObservable = getService2GenericObservable(serviceRequest, prodId);
Observable<ServicesDiff> observable = Observable.zip(
service1ResponseObservable, service2ResponseObservable,
new Func2<String, ServiceResponse, ServicesDiff>() {
#Override
public ServicesDiff call(String service1Response, ServiceResponse service2Response) {
return aggregate(service1Response, service2Response); // never reaches this point**********
}
}
);
ServicesDiff response = observable.toBlocking().single();
return response;
}
I am inserting the diff to DB in aggregate method but it never reaches to aggregate at all. Please let me know what I am doing wrong here? Thanks.

Observable are a description of how to consume data. In your code sample, you don't subscribe, you don't actually consume the data. You just described how to request, but the subscribe part, the part that trigger the requests, is missing.
So if I rewrite a little your code:
class Aggregate {
Aggregate(String reponse, ServicesDiff diff) {
...
}
}
Observable<String> getService1GenericObservable(String prodId) {
...
}
Observable<ServicesDiff> getServicesDiffGenericObservable(ServiceRequest serviceRequest, String prodId) {
...
}
public Observable<Aggregate> getBothServiceResponses(ServiceRequest serviceRequest, String prodId) {
Observable<String> service1ResponseObservable = getService1GenericObservable(prodId);
Observable<ServiceResponse> service2ResponseObservable = getService2GenericObservable(serviceRequest, prodId);
return Observable<Aggregate> observable = Observable.zip(
service1ResponseObservable, service2ResponseObservable,
new Func2<String, ServiceResponse, ServicesDiff>() {
#Override
public ServicesDiff call(String service1Response, ServiceResponse service2Response) {
return aggregate(service1Response, service2Response);
}
}
);
}
You will just have to do this to access the result aggregate:
getBothServiceResponses(serviceRequest, prodId).subscribe(...)

Rxjava Android how to use the Zip operator

I am having a lot of trouble understanding the zip operator in RxJava for my android project.
Problem
I need to be able to send a network request to upload a video
Then i need to send a network request to upload a picture to go with it
finally i need to add a description and use the responses from the previous two requests to upload the location urls of the video and picture along with the description to my server.
I assumed that the zip operator would be perfect for this task as I understood we could take the response of two observables (video and picture requests) and use them for my final task.
But I cant seem to get this to occur how I envision it.
I am looking for someone to answer how this can be done conceptually with a bit of psuedo code.
Thank you

Zip operator strictly pairs emitted items from observables. It waits for both (or more) items to arrive then merges them. So yes this would be suitable for your needs.
I would use Func2 to chain the result from the first two observables.
Notice this approach would be simpler if you use Retrofit since its api interface may return an observable. Otherwise you would need to create your own observable.
// assuming each observable returns response in the form of String
Observable<String> movOb = Observable.create(...);
// if you use Retrofit
Observable<String> picOb = RetrofitApiManager.getService().uploadPic(...),
Observable.zip(movOb, picOb, new Func2<String, String, MyResult>() {
#Override
public MyResult call(String movieUploadResponse, String picUploadResponse) {
// analyze both responses, upload them to another server
// and return this method with a MyResult type
return myResult;
}
}
)
// continue chaining this observable with subscriber
// or use it for something else

A small example:
val observableOne = Observable.just("Hello", "World")
val observableTwo = Observable.just("Bye", "Friends")
val zipper = BiFunction<String, String, String> { first, second -> "$first - $second" }
Observable.zip(observableOne, observableTwo, zipper)
.subscribe { println(it) }
This will print:
Hello - Bye
World - Friends
In BiFunction<String, String, String> the first String the type of the first observable, the second String is the type of the second observable, the third String represents the type of the return of your zipper function.
I made a small example that calls two real endpoints using zip in this blog post

Here I have an example that I did using Zip in asynchronous way, just in case you´re curious
/**
* Since every observable into the zip is created to subscribeOn a diferent thread, it´s means all of them will run in parallel.
* By default Rx is not async, only if you explicitly use subscribeOn.
*/
#Test
public void testAsyncZip() {
scheduler = Schedulers.newThread();
scheduler1 = Schedulers.newThread();
scheduler2 = Schedulers.newThread();
long start = System.currentTimeMillis();
Observable.zip(obAsyncString(), obAsyncString1(), obAsyncString2(), (s, s2, s3) -> s.concat(s2)
.concat(s3))
.subscribe(result -> showResult("Async in:", start, result));
}
/**
* In this example the the three observables will be emitted sequentially and the three items will be passed to the pipeline
*/
#Test
public void testZip() {
long start = System.currentTimeMillis();
Observable.zip(obString(), obString1(), obString2(), (s, s2, s3) -> s.concat(s2)
.concat(s3))
.subscribe(result -> showResult("Sync in:", start, result));
}
public void showResult(String transactionType, long start, String result) {
System.out.println(result + " " +
transactionType + String.valueOf(System.currentTimeMillis() - start));
}
public Observable<String> obString() {
return Observable.just("")
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> "Hello");
}
public Observable<String> obString1() {
return Observable.just("")
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> " World");
}
public Observable<String> obString2() {
return Observable.just("")
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> "!");
}
public Observable<String> obAsyncString() {
return Observable.just("")
.observeOn(scheduler)
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> "Hello");
}
public Observable<String> obAsyncString1() {
return Observable.just("")
.observeOn(scheduler1)
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> " World");
}
public Observable<String> obAsyncString2() {
return Observable.just("")
.observeOn(scheduler2)
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> "!");
}
You can see more examples here https://github.com/politrons/reactive

zip operator allow you to compose a result from results of two different observable.
You 'll have to give am lambda that will create a result from datas emitted by each observable.
Observable<MovieResponse> movies = ...
Observable<PictureResponse> picture = ...
Observable<Response> response = movies.zipWith(picture, (movie, pic) -> {
return new Response("description", movie.getName(), pic.getUrl());
});

i have been searching for a simple answer on how to use the Zip operator, and what to do with the Observables i create to pass them to it, i was wondering if i should call subscribe() for every observable or not, non of these answers were simple to find, i had to figure it out by my self, so here is a simple example for using Zip operator on 2 Observables :
#Test
public void zipOperator() throws Exception {
List<Integer> indexes = Arrays.asList(0, 1, 2, 3, 4);
List<String> letters = Arrays.asList("a", "b", "c", "d", "e");
Observable<Integer> indexesObservable = Observable.fromIterable(indexes);
Observable<String> lettersObservable = Observable.fromIterable(letters);
Observable.zip(indexesObservable, lettersObservable, mergeEmittedItems())
.subscribe(printMergedItems());
}
#NonNull
private BiFunction<Integer, String, String> mergeEmittedItems() {
return new BiFunction<Integer, String, String>() {
#Override
public String apply(Integer index, String letter) throws Exception {
return "[" + index + "] " + letter;
}
};
}
#NonNull
private Consumer<String> printMergedItems() {
return new Consumer<String>() {
#Override
public void accept(String s) throws Exception {
System.out.println(s);
}
};
}
the printed result is :
[0] a
[1] b
[2] c
[3] d
[4] e
the final answers to the questions that where in my head were as follows
the Observables passed to the zip() method just need to be created only, they do not need to have any subscribers to them, only creating them is enough ... if you want any observable to run on a scheduler, you can specify this for that Observable ... i also tried the zip() operator on Observables where they should wait for there result, and the Consumable of the zip() was triggered only when both results where ready (which is the expected behavior)

This is my implementation using Single.zip and rxJava2
I tried to make it as easy to understand as possible
//
// API Client Interface
//
#GET(ServicesConstants.API_PREFIX + "questions/{id}/")
Single<Response<ResponseGeneric<List<ResponseQuestion>>>> getBaseQuestions(#Path("id") int personId);
#GET(ServicesConstants.API_PREFIX + "physician/{id}/")
Single<Response<ResponseGeneric<List<ResponsePhysician>>>> getPhysicianInfo(#Path("id") int personId);
//
// API middle layer - NOTE: I had feedback that the Single.create is not needed (but I haven't yet spent the time to improve it)
//
public Single<List<ResponsePhysician>> getPhysicianInfo(int personId) {
return Single.create(subscriber -> {
apiClient.getPhysicianInfo(appId)
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(response -> {
ResponseGeneric<List<ResponsePhysician>> responseBody = response.body();
if(responseBody != null && responseBody.statusCode == 1) {
if (!subscriber.isDisposed()) subscriber.onSuccess(responseBody.data);
} else if(response.body() != null && response.body().status != null ){
if (!subscriber.isDisposed()) subscriber.onError(new Throwable(response.body().status));
} else {
if (!subscriber.isDisposed()) subscriber.onError(new Throwable(response.message()));
}
}, throwable -> {
throwable.printStackTrace();
if(!subscriber.isDisposed()) subscriber.onError(throwable);
});
});
}
public Single<List<ResponseQuestion>> getHealthQuestions(int personId){
return Single.create(subscriber -> {
apiClient.getBaseQuestions(personId)
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(response -> {
ResponseGeneric<List<ResponseQuestion>> responseBody = response.body();
if(responseBody != null && responseBody.data != null) {
if (!subscriber.isDisposed()) subscriber.onSuccess(response.body().data);
} else if(response.body() != null && response.body().status != null ){
if (!subscriber.isDisposed()) subscriber.onError(new Throwable(response.body().status));
} else {
if (!subscriber.isDisposed()) subscriber.onError(new Throwable(response.message()));
}
}, throwable -> {
throwable.printStackTrace();
if(!subscriber.isDisposed()) subscriber.onError(throwable);
});
});
}
//please note that ResponseGeneric is just an outer wrapper of the returned data - common to all API's in this project
public class ResponseGeneric<T> {
#SerializedName("Status")
public String status;
#SerializedName("StatusCode")
public float statusCode;
#SerializedName("Data")
public T data;
}
//
// API end-use layer - this gets close to the UI so notice the oberver is set for main thread
//
private static class MergedResponse{// this is just a POJO to store all the responses in one object
public List<ResponseQuestion> listQuestions;
public List<ResponsePhysician> listPhysicians;
public MergedResponse(List<ResponseQuestion> listQuestions, List<ResponsePhysician> listPhysicians){
this.listQuestions = listQuestions;
this.listPhysicians = listPhysicians;
}
}
// example of Single.zip() - calls getHealthQuestions() and getPhysicianInfo() from API Middle Layer
private void downloadHealthQuestions(int personId) {
addRxSubscription(Single
.zip(getHealthQuestions(personId), getPhysicianInfo(personId), MergedResponse::new)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(response -> {
if(response != null) {
Timber.i(" - total health questions downloaded %d", response.listQuestions.size());
Timber.i(" - physicians downloaded %d", response.listPhysicians.size());
if (response.listPhysicians != null && response.listPhysicians.size()>0) {
// do your stuff to process response data
}
if (response.listQuestions != null && response.listQuestions.size()>0) {
// do your stuff to process response data
}
} else {
// process error - show message
}
}, error -> {
// process error - show network error message
}));
}

You use the zip from rxjava with Java 8:
Observable<MovieResponse> movies = ...
Observable<PictureResponse> picture = ...
Observable<ZipResponse> response = Observable.zip(movies, picture, ZipResponse::new);
class ZipResponse {
private MovieResponse movieResponse;
private PictureResponse pictureResponse;
ZipResponse(MovieResponse movieResponse, PictureResponse pictureResponse) {
this.movieResponse = movieResponse;
this.pictureResponse = pictureResponse;
}
public MovieResponse getMovieResponse() {
return movieResponse;
}
public void setMovieResponse(MovieResponse movieResponse) {
this.movieResponse= movieResponse;
}
public PictureResponse getPictureResponse() {
return pictureResponse;
}
public void setPictureResponse(PictureResponse pictureResponse) {
this.pictureResponse= pictureResponse;
}
}

You can use .zipWith operator for Observable chains.
If uploadMovies() and uploadPictures() return Observable,
uploadMovies()
.zipWith(uploadPictures()) { m, p ->
"$m with $p were uploaded"
}
.subscribe { print(it) }