how to execute multiple java methods asynchronously and get the results on job done for each method?
Lets say I have this:
for(int i=1; i<=10000; i++) {
kur(i);
}
//on kur(i) finish -> System.out.println(kur(i)) or System.out.println(Exception e)
Info kur(int i) throws Exception {
//do some stuff
return new Info(...);
}
I can use spring boot also. I checked this: https://spring.io/guides/gs/async-method/ but it is different than my case.
Could you help me?
(ORIGINAL ANSWER) Maybe an ExecutorService could help you?
ExecutorService executorService = Executors.newFixedThreadPool(10);
IntStream.rangeClosed(1, 10000)
.forEach(i -> {
executorService.submit(() -> {
try {
System.out.println(kur(i));
} catch (Exception e) {
// do something useful here - remember you're in a separate thread
//
// this is not useful.
e.printStackTrace();
}
});
});
executorService.shutdown();
executorService.awaitTermination(1, TimeUnit.MINUTES);
(EDIT WITH POOR MAN'S SOLUTION TO WAIT FOR EVERYTHING TO COMPLETE):
ExecutorService executorService = Executors.newFixedThreadPool(10);
List<Future<?>> futures = IntStream.rangeClosed(1, 10000)
.mapToObj(i -> {
return executorService.submit(() -> {
try {
System.out.println(kur(i));
} catch (Exception e) {
// do something useful here - remember you're in a separate thread
//
// this is not useful.
e.printStackTrace();
}
});
})
.collect(Collectors.toList());
for (Future<?> f : futures) {
f.get();
}
executorService.shutdown();
executorService.awaitTermination(1, TimeUnit.MINUTES);
Related
I have 3 methods that I need to run in parallel, since they are independent to each other and combine the results of each one at the end and send it as the response. I need to handle exception as well.
In different post I found the below code and modified accordingly.
public Response getResponse() {
Response resultClass = new Response();
try {
CompletableFuture<Optional<ClassA>> classAFuture
= CompletableFuture.supplyAsync(() -> service.getClassA() );
CompletableFuture<ClassB> classBFuture
= CompletableFuture.supplyAsync(() -> {
try {
return service.getClassB();
}
catch (Exception e) {
throw new CompletionException(e);
}
});
CompletableFuture<ClassC> classCFuture
= CompletableFuture.supplyAsync(() -> {
try {
return service.getClassC();
} catch (Exception e) {
throw new CompletionException(e);
}
});
CompletableFuture<Response> responseFuture =
CompletableFuture.allOf(classAFuture, classBFuture, classCFuture)
.thenApplyAsync(dummy -> {
if (classAFuture.join().isPresent() {
ClassA classA = classAFuture.join();
classA.setClassB(classBFuture.join());
classA.setClassC(classCFuture.join());
response.setClassA(classA)
}
return response;
});
responseFuture.join();
} catch (CompletionExecution e) {
throw e;
}
return response;
}
Should the above run correctly in parallel? I see it takes some more time, and I wanted to make sure I am doing it right.
If you want to run methods in parallel you should use ExecutorService. Try something like that:
ExecutorService myExecutor = Executors.newFixedThreadPool(3);
List<Future<Object>> futures = myExecutor.invokeAll(
Arrays.asList(
() -> service.getClassA(),
() -> service.getClassB(),
() -> service.getClassC(),
)
);
myExecutor.shutdown();
The idea is correct, but this all could be done with a lot less code:
public Response getResponse() {
CompletableFuture<Optional<ClassA>> classAFuture = CompletableFuture.supplyAsync(() -> service.getClassA());
CompletableFuture<ClassB> classBFuture = CompletableFuture.supplyAsync(() -> service.getClassB());
CompletableFuture<ClassC> classCFuture = CompletableFuture.supplyAsync(() -> service.getClassC());
try {
return CompletableFuture.allOf(classAFuture, classBFuture, classCFuture)
.thenApply(() -> {
Response response = new Response();
Optional<ClassA> maybeA = classAFuture.get();
if (maybeA.isPresent()) {
ClassA classA = maybeA.get();
classA.setClassB(classBFuture.get());
classA.setClassC(classCFuture.get());
response.setClassA(classA);
}
return response;
}).get();
} catch (ExecutionException e) { // Ususally the exception is wrapped to ExecutionException by java concurrency framework itself
Throwable cause = e.getCause();
if (cause != null) {
throw cause;
} else {
throw e;
}
}
}
Main things:
You don't need to wrap your exceptions to CompletionException.
You don't need to use thenApplyAsync. Just thenApply is the same thing unless you want to be very specific on the type of thread you want to use. Check this for more information https://stackoverflow.com/a/47489654/3020903
You don't need to join() anything. By the time CompletableFuture.all has finished, you can be very sure that all the supplied jobs have finished and calling get() on then will just return the value.
As for can you be sure jobs A, B and C will be run in parallel. Yes and no. It will be run in parallel if there are enough system resources to run them in parallel. You have done your best to ask them to run in parallel. Maybe at some point you also want to supply your custom thread pool to have more control, but that's a topic for another day.
List<CompletableFuture<GreetHolder>> completableFutures = langList.stream()
.map(lang -> getGreeting(lang))
.collect(Collectors.toList());
CompletableFuture<Void> allFutures = CompletableFuture.allOf(completableFutures.toArray(new CompletableFuture[completableFutures.size()]));
CompletableFuture<List<GreetHolder>> allCompletableFuture = allFutures.thenApply(future -> {
return completableFutures.stream().map(completableFuture -> completableFuture.join()).collect(Collectors.toList());
});
CompletableFuture completableFuture = allCompletableFuture.thenApply(greets -> {
return greets.stream().map(GreetHolder::getGreet).collect(Collectors.toList());
});
completableFuture.get();
private CompletableFuture<GreetHolder> getGreeting (String lang){
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("Task execution started.");
Thread.sleep(2000);
logger.info("Task execution stopped.");
} catch (InterruptedException e) {
e.printStackTrace();
}
return new GreetHolder(getGreet(lang));
}, pool);
}
}
I want to read the List of greeting values returned from either from allCompletableFuture or completableFutureCompleted. I am not sure how to read the values returned by getGreeting method and output the list to the screen. Please feel free to modify the code if needed in the last step.
Your greetings are returned by completableFuture.get(); but you aren't capturing them in your example.
Assuming "getGreet" returns a string, does this work?
List<String> greetings = completableFuture.get();
I am having a Springboot java application that talks to cassandra database and also using google guava libaries.
currently i am facing a an issue. I have a semaphore object in the code.
in my my method i have to perform two write queries simulatenously using two objects( mapper and parameterisedListMsisdnMapper).
Firing each queries using the mappers returns ListenableFuture future & ListenableFuture future1 objects . How can I rewrited the below code, so that i will release the semaphore upon completion of both future and future1 object.
public class ParameterisedListItemRepository {
public ParameterisedListItemRepository() {
this.executor = MoreExecutors.directExecutor();
this.semaphore = new Semaphore(getNumberOfRequests(session));
}
public void saveAsync(ParameterisedListItem parameterisedListItem) {
try {
semaphore.acquire();
ListenableFuture<Void> future = mapper.saveAsync(parameterisedListItem);
ListenableFuture<Void> future1 = parameterisedListMsisdnMapper.saveAsync( mapParameterisedList(parameterisedListItem));
future.addListener(() -> semaphore.release(), executor);
} catch (InterruptedException e) {
throw new RuntimeException("Semaphore was interrupted.");
}
}
}
appreciate any help
I have used Futures.whenAllSucceed and it worked
public void saveAsync(ParameterisedListItem parameterisedListItem) {
if (parameterisedListItem.getId() == null) {
parameterisedListItem.setId(UUID.randomUUID());
}
Set<ConstraintViolation<ParameterisedListItem>> violations = validator.validate(parameterisedListItem);
if (violations != null && !violations.isEmpty()) {
throw new ConstraintViolationException(violations);
}
Callable releasePermit = () -> { semaphore.release();
return null;
};
try {
semaphore.acquire();
ListenableFuture<Void> future1 = mapper.saveAsync(parameterisedListItem);
ListenableFuture<Void> future2 = parameterisedListMsisdnMapper.saveAsync( mapParameterisedList(parameterisedListItem));
Futures.whenAllSucceed(future1, future2).call(releasePermit, executor);
} catch (InterruptedException e) {
//FIXME handle exception in better way
throw new RuntimeException("Semaphore was interrupted.");
}
}
very basic code to run asynchronous method.
when I run the following code the runAsync doesn't run.
what I'm missing?
the result run only the sync code .
public class Main {
public static void main(String[] args) {
runAsync("run async command ans wait 10000");
System.out.println("sync commands ");
}
public static void runAsync(String inputStr) {
CompletableFuture.runAsync(() -> {
List<String> strings = Arrays.asList(inputStr.split(" "));
int sleep = Integer.parseInt(strings.get(strings.size() - 1));
try {
sleep(sleep);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("async command ");
});
}
}
I expect to get first the "sync commands" then the "async command "
but get only the sync message
Your task will run in some other Thread(by default in a Thread from ForkJoinPool) and you are not waiting for it to finish - the main Thread ends before your async task is executed/submitted. You can call CompletableFuture::join to wait for it to finish and it will block the main Thread until it finishes :
CompletableFuture.runAsync(() -> {
List<String> strings = Arrays.asList(inputStr.split(" "));
int sleep = Integer.parseInt(strings.get(strings.size() - 1));
try {
sleep(sleep);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("async command ");
}).join(); //here call the join
or like :
CompletableFuture<Void> cf = CompletableFuture.runAsync(() -> {
//...
});
cf.join();
You need to wait for the async task to be completed by using join, for example:
public static void main(String[] args) {
CompletableFuture<Void> future = runAsync("run async command ans wait 10000");
future.join();
System.out.println("sync commands ");
}
public static CompletableFuture<Void> runAsync(String inputStr) {
return CompletableFuture.runAsync(() -> {
List<String> strings = Arrays.asList(inputStr.split(" "));
int sleep = Integer.parseInt(strings.get(strings.size() - 1));
try {
sleep(sleep);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("async command ");
});
}
It does run, but it runs on another thread and you're not waiting or doing anything with the result. As Javadoc of CompletableFuture.runAsync() says:
Returns a new CompletableFuture that is asynchronously completed by a
task running in the ForkJoinPool.commonPool() after it runs the given
action.
runAsync() is useful for tasks that don't return anything. If you want a result from it you should use supplyAsync() which returns a CompletableFuture<T>
Then you can get the result from it:
// Run a task specified by a Supplier object asynchronously
CompletableFuture<String> future = CompletableFuture.supplyAsync(new Supplier<String>() {
#Override
public String get() {
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
throw new IllegalStateException(e);
}
return "Result of the asynchronous computation";
}
});
// Block and get the result of the Future
String result = future.get();
System.out.println(result);
I have this simple code below. All futures should start at the same time. future13 is supposed to run right after futures 1 and 3 finish, but in the logs I see that it waits until after futures 1, 2, 3, and 4 all finish. Why does it wait for futures 2 and 4?
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import org.junit.Test;
public class Test1 {
private void loop(Long id, int max) {
try {
for (int i = 0; i < max; i++) {
System.out.println(id);
Thread.sleep(100);
}
} catch (Throwable t) {
System.out.println(t);
}
}
private CompletableFuture<Void> createConfigFuture(Long id) {
return CompletableFuture.supplyAsync(() -> {
loop(id, 100);
return null;
});
}
#Test
public void testMe() {
CompletableFuture<Void> future1 = createConfigFuture(1L);
CompletableFuture<Void> future2 = createConfigFuture(2L);
CompletableFuture<Void> future3 = createConfigFuture(3L);
CompletableFuture<Void> future4 = createConfigFuture(4L);
try {
CompletableFuture<Void> future13 = CompletableFuture.allOf(future1, future3)
.thenApply(v -> {
loop(999L, 5);
return null;
});
CompletableFuture<Void> mainFuture = CompletableFuture.allOf(future13, future2, future4);
mainFuture.get();
} catch (InterruptedException | ExecutionException e) {
System.out.println(e);
}
}
}
There is a queue to get an execution slot in the JRE's default fork-join Executor which all async tasks will serialize on.
The task #2 is ahead of the task #3 in that Executor queue, so before you observe the execution of the task #3 (and, respectively, of the completion task #13) the #2 should get its execution slot first.
This may be seen as #3 linked to #2, but other than that there should not be any additional coupling between the tasks.