Trying to use Futures and IntelliJ is giving me various warnings, not sure how to code it 'correctly'. The code works but obviously want to learn best practice.
try {
public void futuresTest() {
try {
List<String> valuesToProcess = List.of("A","B","C","D","E");
ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
List<Future<MyObject>> futures = new ArrayList<>();
for(String s : valuesToProcess) {
futures.add((Future<MyObject>) executor.submit(new MyObject(s))); //<THIS
}
LOG.info("Waiting for threads to finish...");
boolean termStatus = executor.awaitTermination(10, TimeUnit.MINUTES);
if (termStatus) {
LOG.info("Success!");
} else {
LOG.warn("Timed Out!");
for(Future<MyObject> f : futures) {
if(!f.isDone()) {
LOG.warn("Failed to process {}:",f);
}
}
}
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
Gives Unchecked cast: 'java.util.concurrent.Future<capture<?>>' to 'java.util.concurrent.Future<model.MyObject>'
List<Future> futures = new ArrayList<>();
for(String s : valuesToProcess) {
futures.add( executor.submit(new MyObject(s)));
}
Gives Raw use of parameterized class 'Future'
is it just supposed to be List<Future<?>> futures = new ArrayList<>(); that has no warnings but I would think I should be specifying my Object.
Based on the comments it does sound like the correct approach is
List<Future<?>> futures = new ArrayList<>();
for(String s : valuesToProcess) {
futures.add(executor.submit(new MyObject(s)));
}
Related
I am having a Springboot java application that talks to cassandra database and also using google guava libaries.
currently i am facing a an issue. I have a semaphore object in the code.
in my my method i have to perform two write queries simulatenously using two objects( mapper and parameterisedListMsisdnMapper).
Firing each queries using the mappers returns ListenableFuture future & ListenableFuture future1 objects . How can I rewrited the below code, so that i will release the semaphore upon completion of both future and future1 object.
public class ParameterisedListItemRepository {
public ParameterisedListItemRepository() {
this.executor = MoreExecutors.directExecutor();
this.semaphore = new Semaphore(getNumberOfRequests(session));
}
public void saveAsync(ParameterisedListItem parameterisedListItem) {
try {
semaphore.acquire();
ListenableFuture<Void> future = mapper.saveAsync(parameterisedListItem);
ListenableFuture<Void> future1 = parameterisedListMsisdnMapper.saveAsync( mapParameterisedList(parameterisedListItem));
future.addListener(() -> semaphore.release(), executor);
} catch (InterruptedException e) {
throw new RuntimeException("Semaphore was interrupted.");
}
}
}
appreciate any help
I have used Futures.whenAllSucceed and it worked
public void saveAsync(ParameterisedListItem parameterisedListItem) {
if (parameterisedListItem.getId() == null) {
parameterisedListItem.setId(UUID.randomUUID());
}
Set<ConstraintViolation<ParameterisedListItem>> violations = validator.validate(parameterisedListItem);
if (violations != null && !violations.isEmpty()) {
throw new ConstraintViolationException(violations);
}
Callable releasePermit = () -> { semaphore.release();
return null;
};
try {
semaphore.acquire();
ListenableFuture<Void> future1 = mapper.saveAsync(parameterisedListItem);
ListenableFuture<Void> future2 = parameterisedListMsisdnMapper.saveAsync( mapParameterisedList(parameterisedListItem));
Futures.whenAllSucceed(future1, future2).call(releasePermit, executor);
} catch (InterruptedException e) {
//FIXME handle exception in better way
throw new RuntimeException("Semaphore was interrupted.");
}
}
how to execute multiple java methods asynchronously and get the results on job done for each method?
Lets say I have this:
for(int i=1; i<=10000; i++) {
kur(i);
}
//on kur(i) finish -> System.out.println(kur(i)) or System.out.println(Exception e)
Info kur(int i) throws Exception {
//do some stuff
return new Info(...);
}
I can use spring boot also. I checked this: https://spring.io/guides/gs/async-method/ but it is different than my case.
Could you help me?
(ORIGINAL ANSWER) Maybe an ExecutorService could help you?
ExecutorService executorService = Executors.newFixedThreadPool(10);
IntStream.rangeClosed(1, 10000)
.forEach(i -> {
executorService.submit(() -> {
try {
System.out.println(kur(i));
} catch (Exception e) {
// do something useful here - remember you're in a separate thread
//
// this is not useful.
e.printStackTrace();
}
});
});
executorService.shutdown();
executorService.awaitTermination(1, TimeUnit.MINUTES);
(EDIT WITH POOR MAN'S SOLUTION TO WAIT FOR EVERYTHING TO COMPLETE):
ExecutorService executorService = Executors.newFixedThreadPool(10);
List<Future<?>> futures = IntStream.rangeClosed(1, 10000)
.mapToObj(i -> {
return executorService.submit(() -> {
try {
System.out.println(kur(i));
} catch (Exception e) {
// do something useful here - remember you're in a separate thread
//
// this is not useful.
e.printStackTrace();
}
});
})
.collect(Collectors.toList());
for (Future<?> f : futures) {
f.get();
}
executorService.shutdown();
executorService.awaitTermination(1, TimeUnit.MINUTES);
I am trying to call a method multiple times every 60 seconds until a success response from the method which actually calls a rest end point on a different service. As of now I am using do while loop and using
Thread.sleep(60000);
to make the main thread wait 60 seconds which I feel is not the ideal way due to concurrency issues.
I came across the CountDownLatch method using
CountDownLatch latch = new CountDownLatch(1);
boolean processingCompleteWithin60Second = latch.await(60, TimeUnit.SECONDS);
#Override
public void run(){
String processStat = null;
try {
status = getStat(processStatId);
if("SUCCEEDED".equals(processStat))
{
latch.countDown();
}
} catch (Exception e) {
e.printStackTrace();
}
}
I have the run method in a different class which implements runnable. Not able to get this working. Any idea what is wrong?
You could use a CompletableFuture instead of CountDownLatch to return the result:
CompletableFuture<String> future = new CompletableFuture<>();
invokeYourLogicInAnotherThread(future);
String result = future.get(); // this blocks
And in another thread (possibly in a loop):
#Override
public void run() {
String processStat = null;
try {
status = getStat(processStatId);
if("SUCCEEDED".equals(processStat))
{
future.complete(processStat);
}
} catch (Exception e) {
future.completeExceptionally(e);
}
}
future.get() will block until something is submitted via complete() method and return the submitted value, or it will throw the exception supplied via completeExceptionally() wrapped in an ExecutionException.
There is also get() version with timeout limit:
String result = future.get(60, TimeUnit.SECONDS);
Finally got it to work using Executor Framework.
final int[] value = new int[1];
pollExecutor.scheduleWithFixedDelay(new Runnable() {
Map<String, String> statMap = null;
#Override
public void run() {
try {
statMap = coldService.doPoll(id);
} catch (Exception e) {
}
if (statMap != null) {
for (Map.Entry<String, String> entry : statMap
.entrySet()) {
if ("failed".equals(entry.getValue())) {
value[0] = 2;
pollExecutor.shutdown();
}
}
}
}
}, 0, 5, TimeUnit.MINUTES);
try {
pollExecutor.awaitTermination(40, TimeUnit.MINUTES);
} catch (InterruptedException e) {
}
Threadpool rejecting tasks while submitting. Threadpool size is fixed and it is 8. Even though i am not sumbitting the tasks more than 8 it is rejecting. I tried using the blocking queue but it is not helping me.
Here is my code snippet
try {
List<Future> tasks = new ArrayList<Future>();
ThreadPoolExecutor tpe = (ThreadPoolExecutor) Executors.newFixedThreadPool(10);
Process process = new Process();
ProcessingJobMeta meta = process.getPJM();
List<CuratedInput> cil = meta.getCuratedInputList();
for (final CuratedInput ci : cil) {
for (final Preperation prep : Preperation.values()) {
for (final Export export : Export.values()) {
Runnable runnable = new Runnable() {
public void run() {
LOGGER.info("Executing.................." + prep.toString() );
LOGGER.info("Executing.................." + export.toString());
PreperationFactory.getPreperation(prep.toString(), ci);
ExportFactory.getExport(export.toString(), ci);
}
};
// tpe.submit(runnable);
tasks.add((Future) tpe.submit(runnable));
for (Future p : tasks) {
LOGGER.info("---------------inside the futures for loop------------");
LOGGER.info("Result of the future executed ------> " + p.get());
}
tpe.shutdown();
while (!tpe.isShutdown()) {
}
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
Your problem is that your shutting down the pool within the loop and attemting to add more threads to an already shutdown pool. place these lines out of the loop
tpe.shutdown();
while (!tpe.isShutdown()) {
}
something like this
List<Future> tasks = new ArrayList<Future>();
ThreadPoolExecutor tpe = (ThreadPoolExecutor) Executors.newFixedThreadPool(10);
Process process = new Process();
ProcessingJobMeta meta = process.getPJM();
List<CuratedInput> cil = meta.getCuratedInputList();
for (final CuratedInput ci : cil) {
.....
}
tpe.shutdown();
while (!tpe.isShutdown()) {
}
Please try that
I call the below testMethod, after putting it into a Callable(with other few Callable tasks), from an ExecutorService. I suspect that, the map.put() suffers OutOfMemoryError, as I'm trying to put some 20 million entries.
But, I'm not able to see the error trace in console. Just the thread stops still. I tried to catch the Error ( I know.. we shouldnt, but for debug I caught). But, the error is not caught. Directly enters finally and stops executing.. and the thread stands still.
private HashMap<String, Integer> testMethod(
String file ) {
try {
in = new FileInputStream(new File(file));
br = new BufferedReader(new InputStreamReader(in), 102400);
for (String line; (line= br.readLine()) != null;) {
map.put(line.substring(1,17),
Integer.parseInt(line.substring(18,20)));
}
System.out.println("Loop End"); // Not executed
} catch(Error e){
e.printStackTrace(); //Not executed
}finally {
System.out.println(map.size()); //Executed
br.close();
in.close();
}
return map;
}
Wt could be the mistake, I'm doing?
EDIT: This is how I execute the Thread.
Callable<Void> callable1 = new Callable<Void>() {
#Override
public Void call() throws Exception {
testMethod(inputFile);
return null;
}
};
Callable<Void> callable2 = new Callable<Void>() {
#Override
public Void call() throws Exception {
testMethod1();
return null;
}
};
List<Callable<Void>> taskList = new ArrayList<Callable<Void>>();
taskList.add(callable1);
taskList.add(callable2);
// create a pool executor with 3 threads
ExecutorService executor = Executors.newFixedThreadPool(3);
List<Future<Void>> future = executor.invokeAll(taskList);
//executor.invokeAll(taskList);
latch.await();
future.get(0);future.get(1); //Added this as per SubOptimal'sComment
But, this future.get() didn't show OOME in console.
You should not throw away the future after submitting the Callable.
Future future = pool.submit(callable);
future.get(); // this would show you the OOME
example based on the informations of the requestor to demonstrate
public static void main(String[] args) throws InterruptedException, ExecutionException {
Callable<Void> callableOOME = new Callable<Void>() {
#Override
public Void call() throws Exception {
System.out.println("callableOOME");
HashMap<String, Integer> map = new HashMap<>();
// some code to force an OOME
try {
for (int i = 0; i < 10_000_000; i++) {
map.put(Integer.toString(i), i);
}
} catch (Error e) {
e.printStackTrace();
} finally {
System.out.println("callableOOME: map size " + map.size());
}
return null;
}
};
Callable<Void> callableNormal = new Callable<Void>() {
#Override
public Void call() throws Exception {
System.out.println("callableNormal");
// some code to have a short "processing time"
try {
TimeUnit.SECONDS.sleep(5);
} catch (InterruptedException ex) {
System.err.println(ex.getMessage());
}
return null;
}
};
List<Callable<Void>> taskList = new ArrayList<>();
taskList.add(callableOOME);
taskList.add(callableNormal);
ExecutorService executor = Executors.newFixedThreadPool(3);
List<Future<Void>> future = executor.invokeAll(taskList);
System.out.println("get future 0: ");
future.get(0).get();
System.out.println("get future 1: ");
future.get(1).get();
}
Try catching Throwable as it could be an Exception like IOException or NullPointerException, Throwable captures everything except System.exit();
Another possibility is that the thread doesn't die, instead it becomes increasingly slower and slower due to almost running out of memory but never giving up. You should be able to see this with a stack dump or using jvisualvm while it is running.
BTW Unless all you strings are exactly 16 characters long, you might like to call trim() on the to remove any padding in the String. This could make them shorter and use less memory.
I assume you are using a recent version of Java 7 or 8. If you are using Java 6 or older, it will use more memory as .substring() doesn't create a new underlying char[] to save CPU, but in this case wastes memory.