I manage to write a REST API using Stripe Framework. Inside my API, I have several tasks which need to execute and combine their results. I come up with an approach, borrowed from JavaScript, which will spawn tasks into several threads and join rather than chronological implementation. Thus, I used ExecutorService but I found a bottleneck on the implementation when the number of requests is quite big, tasks are finished on a longer time than I expect.
My question is related to an alternate way to achieve the same purpose.
How can I create an Executors per request
How can I expand Executors' size
To demonstrate, let consider this way on Javascript
import Promise from 'bluebird';
let tasks = [];
tasks.push(task01);
tasks.push(task02);
Promise.all(tasks).then(results => { do_sth_here!} )
Bring this idea to Java, I have implemented like below
ExecutorService exec = Executors.newCachedThreadPool();
List<Callable<Promise>> tasks = new ArrayList<>();
List<Future<Promise>> PromiseAll;
try {
tasks.add(() -> TaskPromises(Input));
tasks.add(() -> TaskPromise(Input));
PromiseAll = exec.invokeAll(tasks);
for (Future<Promise> fr : PromiseAll) {
// do_some_thing_next
}
}
Related
I have a microservice on which I am using Kotlin coroutines to perform a bunch of db queries asynchronously, and I want to monitor the execution time for each one of those queries for potential performance optimization.
The implementation I have is like this:
val requestSemaphore = Semaphore(5)
val baseProductsNos = productRepository.getAllBaseProductsNos()
runBlocking {
baseProductsNos
.chunked(500)
.map { batchOfProductNos ->
launch {
requestSemaphore.withPermit {
val rawBaseProducts = async {
productRepository.getBaseProducts(batchOfProductNos)
}
val mediaCall = async {
productRepository.getProductMedia(batchOfProductNos)
}
val productDimensions = async {
productRepository.getProductDimensions(batchOfProductNos)
}
val allowedCountries = async {
productRepository.getProductNosInCountries(batchOfProductNos, countriesList)
}
val variants = async {
productRepository.getProductVariants(batchOfProductNos)
}
// here I wait for all the results and then some processing on thm
}
}
}.joinAll()
}
As you can see I use Semaphore to limit the number of parallel jobs, and all the repository methods are suspendable and those are the ones I want to measure the execution time for. Here is an example of an implementation inside ProductRepository:
suspend fun getBaseProducts(baseProductNos: List<String>): List<RawBaseProduct> =
withContext(Dispatchers.IO) {
namedParameterJdbcTemplateMercator.query(
getSqlFromResource(baseProductSql),
getNamedParametersForBaseProductNos(baseProductNos),
RawBaseProductRowMapper()
)
}
And to do that I tried this :
val rawBaseProductsCall = async {
val startTime = System.currentTimeMillis()
val result = productRepository.getBaseProducts(productNos)
val endTime = System.currentTimeMillis()
logger.info("${TemporaryLog("call-duration", "rawBaseProductsCall", endTime - startTime)}")
result
}
But this measurement always returns inconsistent results for the average in contrast to the sequential implementation(without coroutines), and the only explanation I can come up with is that this measurement includes the suspension time, and obviously I am only interested in the time that the queries take to execute without a suspension time if there was any.
I don't know if what I am trying to do is possible in Kotlin, but it looks like python supports this. So I will appreciate any help to do something similar in Kotlin.
UPDATE:
In my case I am using a regular java library to query the db, so my DB queries are just regular blocking calls which means that the way I am measuring time right now is correct.
The assumption I made in the question would have been valid if I was using some implementation of R2DBC for querying my DB.
you do not want to measure the coroutine startup or suspension time so you need to measure over a block of code that will not suspend, ie.. your database calls from a java library
stdlib for example provides a few nice functions like measureTimedValue
val (result, duration) = measureTimedValue {
doWork()
// eg: productRepository.getBaseProducts(batchOfProductNos)
}
logger.info("operation took $duration")
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.time/measure-timed-value.html
I don't know if this is intentional or by a mistake, but you use only a single thread here. You start tens or even hundreds of coroutines and they all fight each other for this single thread. If you perform any CPU-intensive processing in "here I wait for all the results and then some processing on thm" then while it is working, all other coroutines have to wait to be resumed from withContext(Dispatchers.IO). If you want to utilize multiple threads, replace runBlocking {} with runBlocking(Dispatchers.Default) {}.
Still, it doesn't fix the problem, but rather lessens its impact. Regarding the proper fix: if you need to measure the time spent in the IO only then... measure the time in the IO only. Just move your measurements inside withContext(Dispatchers.IO) and I think results will be closer to what you expect. Otherwise, it is like measuring the size of a room by standing outside the building.
I'm picking up Java/Reactor after moving over from C#. I'm well versed in the C# async-await approach to non-blocking calls and am struggling to adapt to Flux/Mono.
I'm implementing a solution where I need to make a call to ElasticSearch via the Java SDK, get results, apply additional filters to strip out ES results, and keep paging through ES until my final collection of results is complete.
The ES SDK doesn't support Reactor but there are examples of Java adapter code that takes the ES callback and converts to a mono (I see a direct correlation to the C# async-await here as this is a non-blocking call to ES). What I then struggle with is the next bit - I need to take the results from the ES mono, filter them.
I do this by calling out to other external services to get additional data based on the results from the ES call, so I need to know the ids of each page of content the ES mono result before I can apply the filtering (effectively a kind of block), then apply the in-memory filters and if I don't have enough content, then go back to ES to get the next page... repeat until I have enough data or there are no more results from ES.
This appears to be very difficult to achieve compared to C# but I probably just don't understand the Java paradigm correctly.
My problem is that I can't use "block()" as this throws an error in Reactor 3.2 so I don't really know how to "wait" until the mono calls to ES and external services are complete until continuing. In C#, this would be as simple as call to an Async method with an await to handle the implicit callbacks
My blocking version (works in IntellJ, fails when published via maven and then run in a webserver) is effectively:
do {
var sr = GetSearchRequest(xxxx);
this.elasticsearch.results(sr)
.map(r -> chunk.addAdd(r))
.block();
if (chunk.size() == 0 {
isComplete = true;
}
else {
var filtered = postFilterResults(chunk);
finalResults.add(filtered);
if (finalResults.size() = MAXIMUM_RESULTS) {
isComplete = true;
}
esPage = esPage + 1;
while (isComplete == false);
If I try to subscribe() or other non-blocking reaktor calls, then (obviously) the code skips over the "get ES" bit and hits the do-while, looping repeatedly until the callback from ES finally happens and the subscribed map is invoked.
I think I need to perform an "async block" for each ES call but I don't know how.
To answer my own question... The underlying issue IMO is that Flux/Mono simply is not like any existing programming style in that it absolutely forces you to work within the fluent style that reactor mandates. This is very similar to C# Linq but it's almost a "false friend" as even things like loops need to be in Reactor.
In this case, the key issue to solve is one of paging and to keep doing this within a loop. it is very unclear how to achieve this as a subscription to a flux "locks in" the original parameters so repeating the subscription call simply gets the same page again. The solution is to use the Flux.defer method which forces lazy building of the subscription on each repeated invoke. You then need Atomic integers to keep track of the page counter across different calls. Again, this is something that C# handles for you, so it can catch a .net developer out.
Something like:
//The response from the elasticsearch adapter is a Flux<T> but we do not want to filter
//results on a row by row basis as this incurs one call for each row to the DB/Network
//(as appropriate). We choose to batch these up
var result = new SearchResult();
var page = new AtomicInteger();
var chunkSize = new AtomicInteger();
//Use a defer so we recalculate the subscription to the search with the new page count
var results = Flux.defer(() -> elasticsearch.results(GetSearchRequest(request, lc, pf, page.get()))
.doOnComplete(() -> {
chunkSize.set(0);
page.getAndAdd(1);
})
.collectList()
.map(chunk -> {
chunkSize.set(chunk.size());
return chunk;
})
.map(chunk -> postFilterResults(request, chunk, pf))
.map(filtered -> result.getDocuments().addAll(filtered)));
//Repeat the deferred flux (recalculating each time) until we have enough content or we don't get anything from the search engine
return results
.repeat()
.takeUntil(r -> chunkSize.get() == 0 || result.getDocuments().size() >= this.elasticsearch.getMaximumSearchResults())
.take(this.elasticsearch.getMaximumSearchResults())
.collectList()
.flatMap(r -> {
result.setTotalHits(result.getDocuments().size());
return Mono.just(result);
});
I have a List<HystrixCommand<?>>> commands, what is the best way to execute these commands and collect the results such that the commands run in parallel?
I have tried something like this:
List<Future<?>> futures = commands.stream()
.map(HystrixCommand::queue)
.collect(Collectors.toList());
List<?> results = futures.stream()
.map(Future::get)
.collect(Collectors.toList());
Does this run the commands in parallel?
I.e. when calling HystrixCommand.queue() followed by Future.get() on the same thread, the .get() call does not block on some command and delay the other commands?
I ask because I couldn't find any documentation for this.
I have also looked at HystrixCollapser, but this still requires creating and running the individual commands (like above) in the createCommand method.
Ok I have investigated this and figured it out... by creating some simple examples rather than debugging production code...
My initial code was correct:
List<Request> requests = ...; // some expensive requests
List<HystrixCommand<?>>> commands = getCommands(requests);
List<Future<?>> futures = commands.stream()
.map(HystrixCommand::queue)
.collect(Collectors.toList());
List<?> results = futures.stream()
.map(Future::get)
.collect(Collectors.toList());
The commands do indeed run in parallel.
The .get() method does block, but since all the commands have been queued (prior to any .get() call) they are all running (or queued to run).
Say the second command is faster to completion than the first. The first .get() will block, but when it eventually returns, the second .get() call will return immediately, as the second command was able to complete while the first command was blocking. (Assuming core size >=2.)
In terms of HystrixCollapser, I misunderstood the API. HystrixCollapser is used to combine many HystrixCollapser instances into one HystrixCommand not the other way around. So I had to modify my code to wrap my requests with HystrixCollapserrather than HystrixCommand:
List<Request> requests = ...; // some expensive requests
List<HystrixCollapser<?>>> commands = getCommands(requests);
HystrixRequestContext context = HystrixRequestContext.initializeContext();
try {
List<Future<?>> futures = commands.stream()
.map(HystrixCollapser::queue)
.collect(Collectors.toList());
List<?> results = futures.stream()
.map(Future::get)
.collect(Collectors.toList());
} finally {
context.shutdown();
}
JMH benchmarks and full example source here
Given some code using streams to process a large number of items, what's the best way to instrument the various steps for logging and performance/profiling?
Actual example:
ReactiveSeq.fromStream(pairs)
.filter(this::satisfiesThreshold)
.filter(this::satisfiesPersistConditions)
.map((pair) -> convertToResult(pair, jobId))
.flatMap(Option::toJavaStream)
.grouped(CHUNK_SIZE)
.forEach((chunk) ->
{
repository.save(chunk);
incrementAndReport();
});
reportProcessingTime();
Logging progress is important so I can trigger progress events in another thread that update a user interface.
Tracking the performance characteristics of the filtering and mapping steps in this stream is desireable to see where optimizations can be made to speed it up.
I see three options:
put logging/profiling code in each function
use peek around each step without actually using the value
some sort of annotation based or AOP solution (no idea what)
Which is the best? Any ideas on what #3 would look like? Is there another solution?
You have a couple of options here (if I have understood correctly) :-
We can make use of the elapsed operator to track the elapsed time between element emissions e.g.
ReactiveSeq.fromStream(Stream.of(1,2))
.filter(this::include)
.elapsed()
.map(this::logAndUnwrap)
Long[] filterTimeTakenMillis = new Long[maxSize];
int filterIndex = 0;
private <T> T logAndUnwrap(Tuple2<T, Long> t) {
//capture the elapsed time (t.v2) and then unwrap the tuple
filterTimeTakenMillis[filterIndex++]=t.v2;
return t.v1;
}
This will only work on cyclops-react Streams.
We can make use of the AOP-like functionality in FluentFunctions
e.g.
ReactiveSeq.fromStream(Stream.of(1,2))
.filter(this::include)
.elapsed()
.map(this::logAndUnwrap)
.map(FluentFunctions.of(this::convertToResult)
.around(a->{
SimpleTimer timer = new SimpleTimer();
String r = a.proceed();
mapTimeTakenNanos[mapIndex++]=timer.getElapsedNanos();
return r;
}));
This will also work on vanilla Java 8 Streams.
I've a web application which needs to be extremely fast. But for processing it requires access for multiple data sources. Therefore I decided that it might be useful to make a parallel calls for optimization.
Basically I want to make many different db calls in parallel. Could you please recommend me simple and reliable way and technologies for achieving my goal, it would be useful if you could provide few frameworks and design patterns.
Right now I am using Spring.
You can use the new Java 8 CompletableFuture. It allows to use asynchronously existing synchronous method.
Say you have a list of requests in the form List<DBRequest> listRequest that you want to run in parallel. You can make a stream and launching all requests asynchronously in the following way.
List<CompletableFuture<DBResult>> listFutureResult =
listRequest.stream()
.map(req -> CompletableFuture.supplyAsync(
() -> dbLaunchRequest(req), executor))
.collect(Collectors.toList());
List<DBResult> listResult =
listFutureResult.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList());
To be effective you have to write your own custom Executor
private final Executor executor =
Executors.newFixedThreadPool(Math.min(listRequest.size(), 100),
new ThreadFactory(){
public Thread newThread(Runnable r){
Thread t = new Thread(r);
t.setDaemon(true);
return t;
}
});
Like this you can have enough threads but not too much. Marking threads to deamons allows you to finish the program even if one thread is blocked.
You can find clear explanations about these techniques in the chapter 11 of the book Java 8 in action
== UPDATE for Java 7 ==
If you are stick with Java 7, you can use the following solution:
class DBResult{}
class DBRequest implements Callable<DBResult>{
#Override
public DBResult call(){return new DBResult();}
}
class AsyncTest{
public void test(){
try {
for(Future<DBResult> futureResult : ((ExecutorService)executor).invokeAll(listRequest)){
futureResult.get();
}
} catch (InterruptedException | ExecutionException ex) {
Logger.getLogger(SoTest.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
All requests are run asynchronously and you then wait for their completion, in the order of the list.
Finally, to answer the subsidiary question in the comment, you don't have to create a thread pool for each request.