I have a web application in Java where as part of client HTTP request handling, I need to make 2 API calls. The way I am planning to implement is to offload 1 API call to thread pool and do the other call in the same thread and then combine the result.
I want to process API1 call in parallel but don't want to block it in queue. Hence if no threads available, I am doing it sequentially.
This is what I have come up with.
//this is already created in setup, just listing here for reference.
ThreadPoolExecutor tpe = new ThreadPoolExecutor(1, 2, 300, TimeUnit.SECONDS, new SynchronousQueue<>());
.....
private Future<Integer> getDataFromAPI1(ThreadPoolExecutor tpe){
try {
return tpe.submit(new Callable<Integer>() {
#Override
public Integer call() throws Exception {
//....make API call here
return 1; //return result
}
});
}catch (RejectedExecutionException r){
//do sequentially and throw any exception encountered
///.....
return CompletableFuture.completedFuture(1); //return the result
}
}
public Integer handle(String reqStub){
Future<Integer> f1 = getDataFromAPI1(tpe);
//make API call2 here in same thread
//... this populates r2
Integer r1 = f1.get();
//now return final result based on 2 results
return r1+r2;
}
Assume that exception handling is done by caller of handle() method.
Does the code snippet look good in terms of correctness and performance.
Are there better ways of achieving the same?
Related
I have an execute method which is running multiple test cases one by one, the test cases are passed in a list of Strings arrays.
I am trying to run this test cases in multi-threaded way, also writing data in CSV file in parallel.
Here is what I have done but it seems that the code is not working in a multithreaded way. I have passed nThread 2,5,7 in newFixedThreadPool() but it is taking the same time to execute the code.
private void executeTest(List<String[]> inputArray) throws ExecutionException, InterruptedException {
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(nThreads);//2, 5, 7
long start = System.currentTimeMillis();
for (String[] listOfArray : inputArray) {
Callable c2 = new Callable() {
public ApiResponse call() {
response = runTestCase(listOfArray);
try {
csvWriter.writeCsv(listOfArray[0], response);
} catch (IOException e) {
e.printStackTrace();
}
return response;
}
};
System.out.println("nThread :"+nThreads);
Future<ApiResponse> result = executor.submit(c2);
result.get();
}
long stop = System.currentTimeMillis();
long timeTaken = stop - start;
System.out.println("Total time taken :"+timeTaken+"No of Theads :"+nThreads);
}
The call to future result.get(0) blocks until the action is completed, so you are just executing the tasks one by one inside your loop - even if they are actioned on different threads by the executor service.
// result.get();
Instead remove the line above and await termination at the end so that the full number of threads in your pool may receive tasks at same time, such as:
// All task submitted, mark for shutdown (only call after ALL submits done)
executor.shutdown();
// Wait for the executor service to finish
// You should consider how long this should be:
if (!executor.awaitTermination(whateverTimeIsReasonable, TimeUnit.SECONDS))
throw new RuntimeException("Test failed");
Tests that hiding exceptions are no help for testing, changing this:
e.printStackTrace();
to throw new UncheckedIOException(e); will ensure that all errors are reported.
I am using Java 8, and I want to know the recommended way to enforce timeout on 3 async jobs that I would to execute async and retrieve the result from the future. Note that the timeout is the same for all 3 jobs. I also want to cancel the job if it goes beyond time limit.
I am thinking something like this:
// Submit jobs async
List<CompletableFuture<String>> futures = submitJobs(); // Uses CompletableFuture.supplyAsync
List<CompletableFuture<Void>> all = CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]));
try {
allFutures.get(100L, TimeUnit.MILLISECONDS);
} catch (TimeoutException e){
for(CompletableFuture f : future) {
if(!f.isDone()) {
/*
From Java Doc:
#param mayInterruptIfRunning this value has no effect in this
* implementation because interrupts are not used to control
* processing.
*/
f.cancel(true);
}
}
}
List<String> output = new ArrayList<>();
for(CompeletableFuture fu : futures) {
if(!fu.isCancelled()) { // Is this needed?
output.add(fu.join());
}
}
return output;
Will something like this work? Is there a better way?
How to cancel the future properly? Java doc says, thread cannot be interrupted? So, if I were to cancel a future, and call join(), will I get the result immediately since the thread will not be interrupted?
Is it recommended to use join() or get() to get the result after waiting is over?
It is worth noting that calling cancel on CompletableFuture is effectively the same as calling completeExceptionally on the current stage. The cancellation will not impact prior stages. With that said:
In principle, something like this will work assuming upstream cancellation is not necessary (from a pseudocode perspective, the above has syntax errors).
CompletableFuture cancellation will not interrupt the current thread. Cancellation will cause all downstream stages to be triggered immediately with a CancellationException (will short circuit the execution flow).
'join' and 'get' are effectively the same in the case where the caller is willing to wait indefinitely. Join handles wrapping the checked Exceptions for you. If the caller wants to timeout, get will be needed.
Including a segment to illustrate the behavior on cancellation. Note how downstream processes will not be started, but upstream processes continue even after cancellation.
public static void main(String[] args) throws Exception
{
int maxSleepTime = 1000;
Random random = new Random();
AtomicInteger value = new AtomicInteger();
List<String> calculatedValues = new ArrayList<>();
Supplier<String> process = () -> { try { Thread.sleep(random.nextInt(maxSleepTime)); System.out.println("Stage 1 Running!"); } catch (InterruptedException e) { e.printStackTrace(); } return Integer.toString(value.getAndIncrement()); };
List<CompletableFuture<String>> stage1 = IntStream.range(0, 10).mapToObj(val -> CompletableFuture.supplyAsync(process)).collect(Collectors.toList());
List<CompletableFuture<String>> stage2 = stage1.stream().map(Test::appendNumber).collect(Collectors.toList());
List<CompletableFuture<String>> stage3 = stage2.stream().map(Test::printIfCancelled).collect(Collectors.toList());
CompletableFuture<Void> awaitAll = CompletableFuture.allOf(stage2.toArray(new CompletableFuture[0]));
try
{
/*Wait 1/2 the time, some should be complete. Some not complete -> TimeoutException*/
awaitAll.get(maxSleepTime / 2, TimeUnit.MILLISECONDS);
}
catch(TimeoutException ex)
{
for(CompletableFuture<String> toCancel : stage2)
{
boolean irrelevantValue = false;
if(!toCancel.isDone())
toCancel.cancel(irrelevantValue);
else
calculatedValues.add(toCancel.join());
}
}
System.out.println("All futures Cancelled! But some Stage 1's may still continue printing anyways.");
System.out.println("Values returned as of cancellation: " + calculatedValues);
Thread.sleep(maxSleepTime);
}
private static CompletableFuture<String> appendNumber(CompletableFuture<String> baseFuture)
{
return baseFuture.thenApply(val -> { System.out.println("Stage 2 Running"); return "#" + val; });
}
private static CompletableFuture<String> printIfCancelled(CompletableFuture<String> baseFuture)
{
return baseFuture.thenApply(val -> { System.out.println("Stage 3 Running!"); return val; }).exceptionally(ex -> { System.out.println("Stage 3 Cancelled!"); return ex.getMessage(); });
}
If it is necessary to cancel the upstream process (ex: cancel some network call), custom handling will be needed.
After calling cancel you cannot join the furure, since you get an exception.
One way to terminate the computation is to let it have a reference to the future and check it periodically: if it was cancelled abort the computation from inside.
This can be done if the computaion is a loop where at each iteration you can do the check.
Do you need it to be a CompletableFuture? Cause another way is to avoid to use a CompleatableFuture, and use a simple Future or a FutureTask instead: if you execute it with an Executor calling future.cancel(true) will terminate the computation if possbile.
Answerring to the question: "call join(), will I get the result immediately".
No you will not get it immediately, it will hang and wait to complete the computation: there is no way to force a computation that takes a long time to complete in a shorter time.
You can call future.complete(value) providing a value to be used as default result by other threads that have a reference to that future.
I don't understand how to use AsyncRestTemplate effectively for making external service calls. For the code below:
class Foo {
public void doStuff() {
Future<ResponseEntity<String>> future1 = asyncRestTemplate.getForEntity(
url1, String.class);
String response1 = future1.get();
Future<ResponseEntity<String>> future2 = asyncRestTemplate.getForEntity(
url2, String.class);
String response2 = future2.get();
Future<ResponseEntity<String>> future3 = asyncRestTemplate.getForEntity(
url3, String.class);
String response3 = future3.get();
}
}
Ideally I want to execute all 3 calls simultaneously and process the results once they're all done. However each external service call is not fetched until get() is called but get() is blocked. So doesn't that defeat the purpose of AsyncRestTemplate? I might as well use RestTemplate.
So I don't understaand how I can get them to execute simultaneously?
Simply don't call blocking get() before dispatching all of your asynchronous calls:
class Foo {
public void doStuff() {
ListenableFuture<ResponseEntity<String>> future1 = asyncRestTemplate
.getForEntity(url1, String.class);
ListenableFuture<ResponseEntity<String>> future2 = asyncRestTemplate
.getForEntity(url2, String.class);
ListenableFuture<ResponseEntity<String>> future3 = asyncRestTemplate
.getForEntity(url3, String.class);
String response1 = future1.get();
String response2 = future2.get();
String response3 = future3.get();
}
}
You can do both dispatch and get in loops, but note that current results gathering is inefficient as it would get stuck on the next unfinished future.
You could add all the futures to a collection, and iterate through it testing each future for non blocking isDone(). When that call returns true, you can then call get().
This way your en masse results gathering will be optimised rather than waiting on the next slow future result in the order of calling get()s.
Better still you can register callbacks (runtimes) within each ListenableFuture returned by AccyncRestTemplate and you don't have to worry about cyclically inspecting the potential results.
If you don't have to use 'AsyncRestTemplate' I would suggest to use RxJava instead. RxJava zip operator is what you are looking for. Check code below:
private rx.Observable<String> externalCall(String url, int delayMilliseconds) {
return rx.Observable.create(
subscriber -> {
try {
Thread.sleep(delayMilliseconds); //simulate long operation
subscriber.onNext("response(" + url + ") ");
subscriber.onCompleted();
} catch (InterruptedException e) {
subscriber.onError(e);
}
}
);
}
public void callServices() {
rx.Observable<String> call1 = externalCall("url1", 1000).subscribeOn(Schedulers.newThread());
rx.Observable<String> call2 = externalCall("url2", 4000).subscribeOn(Schedulers.newThread());
rx.Observable<String> call3 = externalCall("url3", 5000).subscribeOn(Schedulers.newThread());
rx.Observable.zip(call1, call2, call3, (resp1, resp2, resp3) -> resp1 + resp2 + resp3)
.subscribeOn(Schedulers.newThread())
.subscribe(response -> System.out.println("done with: " + response));
}
All requests to external services will be executed in separate threads, when last call will be finished transformation function( in example simple string concatenation) will be applied and result (concatenated string) will be emmited from 'zip' observable.
What I Understand by Your question is You have a predefined asynchronous method and you try to do is call this method asynchoronously using RestTemplate Class.
I have wrote a method that will help you out to call Your method asynchoronously.
public void testMyAsynchronousMethod(String... args) throws Exception {
// Start the clock
long start = System.currentTimeMillis();
// Kick of multiple, asynchronous lookups
Future<String> future1 = asyncRestTemplate
.getForEntity(url1, String.class);;
Future<String> future2 = asyncRestTemplate
.getForEntity(url2, String.class);
Future<String> future3 = asyncRestTemplate
.getForEntity(url3, String.class);
// Wait until they are all done
while (!(future1 .isDone() && future2.isDone() && future3.isDone())) {
Thread.sleep(10); //10-millisecond pause between each check
}
// Print results, including elapsed time
System.out.println("Elapsed time: " + (System.currentTimeMillis() - start));
System.out.println(future1.get());
System.out.println(future2.get());
System.out.println(future3.get());
}
You might want to use CompletableFuture class (javadoc).
Transform your calls into CompletableFuture. For instance.
final CompletableFuture<ResponseEntity<String>> cf = CompletableFuture.supplyAsync(() -> {
try {
return future.get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
});
Next call CompletableFuture::allOf method with your 3 newly created completable futures.
Call join() method on the result. After the resulting completable future is resolved you can get the results from each separate completable future you've created on step 3.
I think you are misunderstanding a few things here. When you call the getForEntity method, the requests are already fired. When the get() method of the future object is called, you are just waiting for the request to complete. So in order fire all those three requests on the same subsecond, you just have to do:
// Each of the lines below will fire an http request when it's executed
Future<ResponseEntity<String>> future1 = asyncRestTemplate.getForEntity(url1, String.class);
Future<ResponseEntity<String>> future2 = asyncRestTemplate.getForEntity(url2, String.class);
Future<ResponseEntity<String>> future3 = asyncRestTemplate.getForEntity(url3, String.class);
After all these codes are run, all the requests are already fired (most probably in the same subsecond). Then you can do whatever you want in the meanwhile. As soon as you call any of the get() method, you are waiting for each request to complete. If they are already completed, then it will just return immediately.
// do whatever you want in the meantime
// get the response of the http call and wait if it's not completed
String response1 = future1.get();
String response2 = future2.get();
String response3 = future3.get();
I don't think any of the previous answers actually achieve parallelism. The problem with #diginoise response is that it doesn't actually achieve parallelism. As soon as we call get, we're blocked. Consider that the calls are really slow such that future1 takes 3 seconds to complete, future2 2 seconds and future3 3 seconds again. With 3 get calls one after another, we end up waiting 3 + 2 + 3 = 8 seconds.
#Vikrant Kashyap answer blocks as well on while (!(future1 .isDone() && future2.isDone() && future3.isDone())). Besides the while loop is a pretty ugly looking piece of code for 3 futures, what if you have more? #lkz answer uses a different technology than you asked for, and even then, I'm not sure if zip is going to do the job. From Observable Javadoc:
zip applies this function in strict sequence, so the first item
emitted by the new Observable will be the result of the function
applied to the first item emitted by each of the source Observables;
the second item emitted by the new Observable will be the result of
the function applied to the second item emitted by each of those
Observables; and so forth.
Due to Spring's widespread popularity, they try very hard to maintain backward compatibility and in doing so, sometimes make compromises with the API. AsyncRestTemplate methods returning ListenableFuture is one such case. If they committed to Java 8+, CompletableFuture could be used instead. Why? Since we won't be dealing with thread pools directly, we don't have a good way to know when all the ListenableFutures have completed. CompletableFuture has an allOf method that creates a new CompletableFuture that is completed when all of the given CompletableFutures complete. Since we don't have that in ListenableFuture, we will have to improvise.
I've not compiled the following code but it should be clear what I'm trying to do. I'm using Java 8 because it's end of 2016.
// Lombok FTW
#RequiredArgsConstructor
public final class CounterCallback implements ListenableFutureCallback<ResponseEntity<String>> {
private final LongAdder adder;
public void onFailure(Throwable ex) {
adder.increment();
}
public void onSuccess(ResponseEntity<String> result) {
adder.increment();
}
}
ListenableFuture<ResponseEntity<String>> f1 = asyncRestTemplate
.getForEntity(url1, String.class);
f1.addCallback(//);
// more futures
LongAdder adder = new LongAdder();
ListenableFutureCallback<ResponseEntity<String>> callback = new CounterCallback(adder);
Stream.of(f1, f2, f3)
.forEach {f -> f.addCallback(callback)}
for (int counter = 1; adder.sum() < 3 && counter < 10; counter++) {
Thread.sleep(1000);
}
// either all futures are done or we're done waiting
Map<Boolean, ResponseEntity<String>> futures = Stream.of(f1, f2, f3)
.collect(Collectors.partitioningBy(Future::isDone));
Now we've a Map for which futures.get(Boolean.TRUE) will give us all the futures that have completed and futures.get(Boolean.FALSE) will give us the ones that didn't. We will want to cancel the ones that didn't complete.
This code does a few things that are important with parallel programming:
It doesn't block.
It limits the operation to some maximum allowed time.
It clearly separates successful and failure cases.
I'm using Jetty HTTP Client to make about 50 HTTP calls asynchronously. The code looks something like this:
List<Address> addresses = getAddresses();
final List<String> done = Collections.synchronizedList(new LinkedList<String>());
List<ContentExchange> requests;
for (Address address : addresses) {
ContentExchange ce = new ContentExchange() {
#Override
protected void onResponseComplete() throws IOException {
//handle response
done.add("done");
}
}
ce.setURL(createURL(address));
requests.add(ce);
}
for (ContentExchange ce : requests) {
httpClient.send(ce);
}
while (done.size() != addresses.size()) {
Thread.yield();
}
System.out.println("All addresses processed");
It's calling a rest service that returns back some data about the address. What I expect it to do is this:
Make 50 asynchronous (non-blocking) http calls.
The thread will wait until all 50 are finished.
However, it's not working. It works fine if I don't have the while loop, but I need to wait until all 50 are done. Is there some way to wait until all 50 are done?
Also I know about ExecutorService and multiple thread solution, but I need a single thread solution with non-blocking IO.
Use the java.util.concurrent.CountDownLatch to manage this.
Example from Eclipse Jetty 8.1.10.v20130312's Siege.java test class:
final CountDownLatch latch = new CountDownLatch(concurrent);
for (int i=0;i<concurrent;i++)
{
ConcurrentExchange ex = new ConcurrentExchange(client,latch,uris,repeats);
if (!ex.next()) // this executes the client.send()
{
latch.countDown(); // count down if client.send() was in error
}
}
latch.await(); // wait for all ConcurrentExchange's to complete (or error out)
Note: ConcurrentExchange is a private class within Siege.java.
Then in your HttpExchange object, use the CountDownLatch.countDown() call in the following methods
onConnectionFailed(Throwable x) - example
onException(Throwable x) - example
onExpire() - example
onResponseComplete() - example
Note that all of the examples use a AtomicBoolean counted to make sure that they are only counted once.
if (!counted.getAndSet(true)) // get the value, then set it to true
{
// only get here if counted returned false. (and that will only happen once)
latch.countDown(); // count down this exchange as being done.
}
I am trying to learn the java concurrency API, and for my exercise i want to schedule a job to run periodically every X seconds. The job will compute a random number.
I want to get the result of the scheduled task as soon as it is finished.
I could not get this done only with the API so i hacked it.
Is there a way to do this better, without using low level mechanisms?
I would like to be able to remove the synchronization in MyRandomGiverTask.getResult() and instead use
something like the ScheduledFuture.get(). But in my code ScheduledFuture is never done/completed.
This is my current solution:
class A {
public static void main() {
MyRandomGiverTask task = new MyRandomGiverTask(200);
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
ScheduledFuture<Double> scheduledDouble =
(ScheduledFuture<Double>) scheduler
.scheduleAtFixedRate(task, 1, 4, TimeUnit.SECONDS);
while (true) {
System.out.println(" >> " + task.getResult());
}
}
public class MyRandomGiverTask implements Runnable {
MyRandomGiver giver = new MyRandomGiver();
int param;
double result;
public MyRandomGiverTask(int param) { this.param = param; }
#Override public void run() { result = giver.getRandom(param); }
public double getResult() {
try {
while (result == 0d) {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
return result;
} finally {
result = 0d;
}
}
}
}
Your task is scheduled at fixed rate. This means that until you cancel the task, it will be executed again and again by the executor with a fixed rate. The only thing that such a task can do is to have a side-effect. It can't return anything, since the future that is returned by the executor represents all the pending executions of the task. BTW, you'll notice that the schedule method takes a Callable as argument (which can result something), whereas the sceduleAtFixedRate method only takes a Runnable as argument (which returns void, thus can't return anything).
So, if you want to print the result of each execution, then simply make the task itself (the Runnable) print its result, or have the runnable put its result in a blocking queue, and have the main thread take from the queue. The main thread will thus be blocked untile some result is put in the queue.
If you want to get every random number computed, use a BlockingQueue. the scheduled tasks put()s new random numbers in the queue and whatever wants them can take() them.
also, if you were going to use something like your solution, you would want to use wait()/notify(), not sleep().