I have an executor service that accepts new tasks :
ExecutorService executor = Executors.newFixedThreadPool(1000);
//stupid example with several parralel tasks
for (int i = 0; i < 1000; i++) {
try{
Runnable task = new Runnable() {
public void run() {
throw new RuntimeException("foo");
}
};
executor.submit(task);
}
catch (ExecutionException e) {
System.out.println(e.getMessage());
}
}
My problem is that I'm not able to catch any exception thrown by the Runnable, unless I'm doing this :
Future<?> future = executor.submit(task);
try {
future.get();
} catch (Exception e) {
System.out.println("############### exception :" + e.getMessage());
}
The problem is that future.get() is blocking, so if I 'm not able to run my tasks asynchronously and my tasks will not run in parallel, but sequentially.
I would like to be able to use Java 8 and CompletableFuture but I can't ...
Do you have any other idea?
Thanks
The code inside the Runnable is executing on a separate thread, so you must handle its exceptions inside the run() method.
If you need to gather all the exceptions for later handling, I would do something like this:
ExecutorService executor = Executors.newFixedThreadPool(1000);
final List<Exception> exceptions = // a place to put exceptions
Collections.synchronizedList(new ArrayList<Exception>());
for (int i = 0; i < 1000; i++) {
Runnable task = new Runnable() {
public void run() {
try {
throw new RuntimeException("foo");
} catch (Exception e) {
exceptions.add(e); // save the exception for later
}
}
};
executor.submit(task);
}
// wait for all the tasks to finish, then...
for (Exception e: exceptions) {
// whatever you want to do
}
Otherwise, if you just want to get information about each exception as it occurs:
Runnable task = new Runnable() {
public void run() {
try {
throw new RuntimeException("foo");
} catch (Exception e) {
e.printStackTrace();
}
}
};
Anything you need to do after the task asynchronously can be added to the task itself.
for (int i = 0; i < 1000; i++) {
final Runnable task = new Runnable() {
public void run() {
throw new RuntimeException("foo");
}
};
executor.submit(new Runnable() {
public void run() {
try {
task.run();
} catch (Throwable e) {
e.printStackTrace();
}
}
});
}
or you combine them into one Runnable.
This may not be the best solution but we could make a parent Runnable which will do the work of the actual Runnable. The parent will catch all the exceptions you need to know about. Here is slight convoluted approach:
public static void main(String[] args){
ExecutorService executor = Executors.newFixedThreadPool(1000);
//stupid example with several parralel tasks
for (int i = 0; i < 5; i++) {
Runnable task = new Runnable() {
public void run() {
throw new RuntimeException("foo");
}
};
ParentRunnable t = new ParentRunnable();
t.setRunnable(task, i);
executor.submit(t);
}
}
static class ParentRunnable implements Runnable {
Runnable r;
int index;
public void setRunnable(Runnable r, int index){
this.r = r;
this.index = index;
}
public void run() {
try{
System.out.println("\n" + index + "\n");
r.run();
}catch(Exception e){
e.printStackTrace();
}
}
}
Related
I want to use snmp method to query some data by mutithread in sometime,if over time,then the task will be cancel,I write the code like this, if there is something wrong with my code(It means that the thread may only do step 1and step 2,but do not do the step 4:close snmp connetction),how to fix ?Can I have the method that is the task is cancel,I can still close the snmp clent at step 4?
public static void main(String[] args) throws InterruptedException {
var pool = new ThreadPoolExecutor(
100, 100, 0L, TimeUnit.NANOSECONDS, new LinkedBlockingQueue<>(100));
System.out.println("Executing first batch of tasks...");
submitTasks(pool);
System.out.println("Finish first batch of tasks...");
//call submitTasks(pool) many times
...
}
private static void submitTasks(ExecutorService executor) throws InterruptedException {
var tasks = new ArrayList<Callable<String>>(100);
for (int i = 0; i < 100; i++) {
tasks.add(() -> {
try {
//1.create snmp client
//2.query data with udp link
//3.return result
return result;
}catch (Exception ex){
log.error(String.valueOf(ex));
}
} finally {
//4.close snmp
if (snmp != null) {
snmp.close();
}
}
});
}
List<Future<String>> futureList=executor.invokeAll(tasks,1,TimeUnit.SECONDS);
List<String> list=new ArrayList<>();
for (int i = 0; i < futureList.size(); i++) {
Future<String> future = futureList.get(i);
try {
list.add(future.get());
} catch (CancellationException e) {
log.info("timeOut Task：{}", i);
} catch (Exception e) {
log.error(e.getMessage(), e);
}
}
}
I have change the code from Alexander Pavlov's example,it seems that if the task is cancel,the finally code will never run.
public static void main(String[] args) throws InterruptedException {
var pool = new ThreadPoolExecutor(
3, 3, 0L, TimeUnit.NANOSECONDS, new LinkedBlockingQueue<>(1));
try {
System.out.println("Executing first batch of tasks...");
submitTasks(pool);
System.out.println("Executing second batch of tasks...");
} finally {
pool.shutdown();
}
}
private static void submitTasks(ExecutorService executor) throws InterruptedException {
var tasks = new ArrayList<Callable<String>>(3);
final var latch = new CountDownLatch(3);
log.info(String.valueOf(latch.getCount()));
for (int i = 0; i < 3; i++) {
tasks.add(() -> {
try {
String s="1";
log.info("this.name:"+Thread.currentThread().getName());
for(int a=0;a<1000000L;a++) {
s=a+"";
}
return s;
}catch (Exception ex){
log.error(String.valueOf(ex));
}finally {
latch.countDown();
}
return null;
});
}
List<Future<String>> futureList=executor.invokeAll(tasks,1,TimeUnit.NANOSECONDS);
List<String> list=new ArrayList<>();
//latch.await();//
for (int i = 0; i < futureList.size(); i++) {
Future<String> future = futureList.get(i);
try {
list.add(future.get());
} catch (CancellationException e) {
log.info("timeOut Task：{}", i);
} catch (Exception e) {
log.error(e.getMessage(), e);
}
}
log.info(String.valueOf(latch.getCount()));
log.info("start to await:");
try {
latch.await(); // WAIT UNTIL ALL TASKS ARE REALLY DONE
} catch (InterruptedException ex) {
log.error(String.valueOf(ex));
}
//never log end to await
log.info("end to await:");
log.info(String.valueOf(latch.getCount()));
}
invokeAll with timeout just calls Thread.interrupt() for running tasks. It does not wait until task stops, it notifies task that interruption is requested. Moreover, some tasks may ignore interruption and continue working. So when you are returning from submitTasks then it does not mean all underlying tasks are really stopped.
If you want to be 100% sure that all tasks are stopped when you exit from submitTasks then use CountDownLatch to control how many tasks have been really finished and exit when running tasks count is zero.
private static void submitTasks(ExecutorService executor) throws InterruptedException {
var c = 100;
final var latch = new CountDownLatch(c); // EXPECT 100 TASKS TO BE COMPLETED
var tasks = new ArrayList<Callable<String>>(c);
for (int i = 0; i < c; i++) {
tasks.add(() -> {
try {
//1.create snmp client
//2.query data with udp link
//3.return result
return result;
} catch (Exception ex) {
log.error(String.valueOf(ex));
} finally {
//4.close snmp
if (snmp != null) {
snmp.close();
}
latch.countDown(); // 1 MORE TASK IS COMPLETED
}
});
}
List<Future<String>> futureList = executor.invokeAll(tasks, 1, TimeUnit.SECONDS);
List<String> list = new ArrayList<>();
for (int i = 0; i < futureList.size(); i++) {
Future<String> future = futureList.get(i);
try {
list.add(future.get());
} catch (CancellationException e) {
log.info("timeOut Task：{}", i);
} catch (Exception e) {
log.error(e.getMessage(), e);
}
}
try {
latch.await(); // WAIT UNTIL ALL TASKS ARE REALLY DONE
} catch (InterruptedException ex) {
log.error(ex);
}
}
The Following Code, A ScheduledExecutor has a task. In Production it will have several tasks, but for now I'm testing with one. Anyway, I need to be able to process Exceptions. My code below does this, but causes my GUI to become unresponsive. The line marked appears the be the issue. How can I gather all the Exceptions as they happen?
SwingUtilities.invokeLater(new Runnable() {
#Override
public void run() {
CommandInterface ci = tc.getTask().getCommandInterface();
ScheduledExecutorService scheduler =
taskManager.getComponentInterface().getThreadPool();
ScheduledFuture<?> st = scheduler.schedule(
new TimedRunnable(ci), new Date(
ci.getScheduledDate().getTime()
- System.currentTimeMillis()).getTime(), TimeUnit.MILLISECONDS);
//This line causes blocking ->
SwingUtilities.invokeLater(new AlertRunnable(
taskManager.getComponentInterface().getAlertList(), st));
}
});
SwingUtilities.invokeLater(new Runnable() {
#Override
public void run() {
CommandInterface ci = tc.getTask().getCommandInterface();
taskManagerInterface.getComponentInterface().getThreadPool().schedule(new TimedRunnable(ci, taskManagerInterface.getComponentInterface()), new Date(ci.getScheduledDate().getTime() - System.currentTimeMillis()).getTime(), TimeUnit.MILLISECONDS);
}
});
with timed runnable being:
/**
* This class runs at a specific time.
*/
public static class TimedRunnable implements Runnable, Serializable {
ExecutorService workerExecutor = Executors.newSingleThreadExecutor();
CommandInterface commandInterface;
ComponentInterface componentInterface;
public TimedRunnable(CommandInterface commandInterface, ComponentInterface componentInterface) {
this.commandInterface = commandInterface;
this.componentInterface = componentInterface;
}
#Override
public void run() {
//submit the callable to the progressHelper
Future future = workerExecutor.submit(new Callable() {
#Override
public Object call() throws Exception {
try {
ReturnInterface returnInterface = (ReturnInterface) commandInterface.call();
returnInterface.submitResult();
} catch (Exception ex) {
throw new RuntimeException(ex);
}
return null;
}
});
try {
Object get = future.get();
} catch (InterruptedException | ExecutionException ex) {
Throwable cause = ex.getCause();
Throwable cause1 = cause.getCause();
if (cause1 instanceof CommandInterfaceException) {
System.out.println("[MyItemTree].scheduleTask Cause 1= COMMANDINTERFACE EXCEPTION");
this.componentInterface.getAlertList().addAlert(((CommandInterfaceException) cause1).getResolverFormInterface());
}
}
}
}
I am using single thread executor for long-running threads like this:
executor = Executors.newSingleThreadExecutor(THREAD_FACTORY);
executor.submit(new LongRunnable());
which checks a flag to be stopped:
private class LongRunnable implements Runnable {
#Override
public void run() {
while (isRunning.get()) {
try {
doSomething();
} catch (InterruptedException e) {
...
}
}
}
}
and whole execution is interrupted that way:
#Override
public void close() throws Exception {
isRunning.set(false);
executor.shutdownNow();
}
Still I can see some threads not gc-ed in profiler (while by logs, runnable they were executing has quit outermost while loop).
Question: does provided working with threads strategy memory-leak-free and thread-leak-free?
I am not able to see any issue with executor or shutDownNow. Probably you are looking at different threads in your profiler.
Try this program which is similar to the one in your question and you can see the thread is no longer there after successful shutdown.
public class ExecutorShutdownTest {
private static ExecutorService executor;
private static AtomicLong executorThreadId = new AtomicLong(0);
public static void main(String[] args) {
// get thread MX bean
ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();
// create an executor and start the task
executor = Executors.newSingleThreadExecutor(new TestThreadFactory());
LongRunnable runnable = new LongRunnable();
executor.submit(runnable);
// main thread: keep running for sometime
int count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace(
"\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// main thread: stop the task
try {
runnable.close();
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (Exception e) {
e.printStackTrace();
}
// main thread: run some more time to verify the executor thread no longer exists
count = 5;
while (count-- > 0) {
try {
Thread.sleep(1000);
System.out.println(String.valueOf(threadMXBean.getThreadInfo(executorThreadId.longValue())).replace("\r", "").replace("\n", ""));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private static class LongRunnable implements Runnable {
private volatile boolean isRunning = true;
#Override
public void run() {
while (isRunning) {
System.out.println("Running");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
//ignore
}
}
System.out.println("Stopped");
}
public void close() throws Exception {
System.out.println("Stopping");
isRunning = false;
executor.shutdownNow();
}
}
private static class TestThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String namePrefix;
TestThreadFactory() {
SecurityManager s = System.getSecurityManager();
group = (s != null) ? s.getThreadGroup() : Thread.currentThread().getThreadGroup();
namePrefix = "pool-" + poolNumber.getAndIncrement() + "-thread-";
}
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0) {
#Override protected void finalize() throws Throwable {
super.finalize();
// probably bad idea but lets see if it gets here
System.out.println("Executor thread removed from JVM");
}
};
if (t.isDaemon())
t.setDaemon(false);
if (t.getPriority() != Thread.NORM_PRIORITY)
t.setPriority(Thread.NORM_PRIORITY);
executorThreadId.set(t.getId());
System.out.println("Executor thread created");
return t;
}
}
}
Here's a sample program using the single-thread Executor that manages to strand a thread so that the JVM can't shut down, but it only manages to do it by not calling shutdownNow:
import java.util.concurrent.*;
public class Exec {
public static void main(String[] args) throws Exception {
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(new MyTask());
Thread.sleep(20000L);
// executor.shutdownNow();
int retryCount = 4;
while (!executor.isTerminated() && retryCount > 0) {
System.out.println("waiting for tasks to terminate");
Thread.sleep(500L);
retryCount -= 1;
}
}
}
class MyTask implements Runnable {
public void run() {
int count = 0;
try {
while (!Thread.currentThread().isInterrupted() && count < 10) {
Thread.sleep(1000L);
count += 1;
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("all done");
}
}
The thread used by the executor has a separate life cycle from the task, this example shows how the task finishes but the thread goes on. Uncommenting the shutdownNow results in the executor's thread terminating. Otherwise the main thread sleeps for a while and exits, leaving the executor's thread hanging out, preventing the JVM from exiting.
My guess is that your close method isn't getting called and your executor never gets shut down. To get more useful answers please add a MVCE so that we can reproduce the problem.
Consider that with interruption there's no need to keep a reference to the Runnable to set the flag. As I read the question the task not finishing is not an issue here, but it would still be better to make the Runnable respond to interruption and lose the flag, just because having less things to keep track of is always an improvement.
i'm trying create a thread, which return a value, the process is running correctly but my screen is still locked. I want a thread that return a value but my main thread continues running.
I've done that:
public void showPartidas(int maximumDistance){
ExecutorService es = Executors.newFixedThreadPool(1);
Future<ArrayList<Partida>> partidas= es.submit(new FilterPartidas(maximumDistance));
try {
loadInListView(partidas.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
es.shutdown();
}
class FilterPartidas implements Callable<ArrayList<Partida> > {
private final int distance;
private ArrayList<Partida> partidas;
FilterPartidas(int distance) {
this.distance = distance;
}
#Override
public ArrayList<Partida> call() throws Exception {
partidas=null;
Download load = new Download();
Date fecha = new Date();
DateFormat fechaFormat = new SimpleDateFormat("yyyy-MM-dd");
String query = "select * from partidas where fecha >='"+fechaFormat.format(fecha)+"'";
partidas=load.obtainPartidas(query, distance, myPosition);
return partidas;
}
}
partidas.get() action is the cause that main thread is waiting for the completion of Callable method in executor. If you want main thread are still running during Callable action execution you must place partidas.get() action into dedicated separate thread e.g.:
replace
Future<ArrayList<Partida>> partidas= es.submit(new FilterPartidas(maximumDistance));
try {
loadInListView(partidas.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
into
final Future<ArrayList<Partida>> partidas= es.submit(new FilterPartidas(maximumDistance));
new Thread(new Runnable() {
#Override
public void run() {
try {
loadInListView(partidas.get());
} catch (InterruptedEArrayList<Partida>xception e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
}).start();
or similar action with threads (maybe using executor, Runnable, etc).
Or you can change you logic (if possible) and hide call to method from Callable into Runnable class. E,g.:
ExecutorService es = Executors.newFixedThreadPool(1);
es.submit(new Runnable() {
#Override
public void run() {
ArrayList<Partida> partidas = logic from you Callable call;
loadInListView(partidas);
}
});
I am trying to figure out how I can track all the threads that my application is spawning. Initially, I thought I had it figured out using a CyclicBarrier, however I am seeing threads executing after my await call.
Below is the working pseudo code:
public class ThreadTesterRunner {
public static void main(String[] args) throws InterruptedException {
final CyclicBarrier cb = new CyclicBarrier(1);
ThreadRunner tr = new ThreadRunner(cb);
Thread t = new Thread(tr, "Thread Runner");
t.start();
boolean process = true;
// wait until all threads process, then print reports
while (process){
if(tr.getIsFinished()){
System.out.println("Print metrics");
process = false;
}
Thread.sleep(1000);
}
}
}
class ThreadRunner implements Runnable {
static int timeOutTime = 2;
private ExecutorService executorService = Executors.newFixedThreadPool(10);
private final CyclicBarrier barrier;
private boolean isFinished=false;
public ThreadRunner(CyclicBarrier cb) {
this.barrier = cb;
}
public void run(){
try {
boolean stillLoop = true; int i = 0;
while (stillLoop){
int size;
Future<Integer> future = null;
try {
future = executorService.submit(new Reader()); // sleeps
size = future.get();
} catch (InterruptedException | ExecutionException ex) {
// handle Errs
}
if(i == 3){
stillLoop = false;
this.barrier.await();
this.isFinished=true;
}
//System.out.println("i = "+i+" Size is: "+size+"\r");
i++;
}
} catch (InterruptedException | BrokenBarrierException e1) {
e1.printStackTrace();
}
}
public boolean getIsFinished(){
return this.isFinished;
}
}
class Reader implements Callable {
private ExecutorService executorService = Executors.newFixedThreadPool(1);
#Override
public Object call() throws Exception {
System.out.println("Reading...");
Thread.sleep(2000);
executorService.submit(new Writer());
return 1000;
}
}
class Writer implements Callable {
#Override
public Void call() throws Exception {
Thread.sleep(4000);
System.out.println("Wrote");
return null;
}
}
Can anyone suggest a way to ONLY print "print metrics" after all threads have run?
It doesn't seem like you're doing anything to coordinate with your Reader and Writer threads, which are the ones you want to wait for. If you pass your synchronization barrier through to those threads so that they can register and signal when they are done, it works just fine.
Here's a version rewritten to do so, using a Phaser instead of a CyclicBarrier. Note that each Reader and Writer registers itself upon construction, and notifies the synchronization barrier when it is done executing:
public class ThreadTesterRunner {
public static void main(String[] args) throws InterruptedException {
final Phaser cb = new Phaser();
ThreadRunner tr = new ThreadRunner(cb);
Thread t = new Thread(tr, "Thread Runner");
t.start();
boolean process = true;
// wait until all threads process, then print reports
while (process){
if(tr.getIsFinished()){
System.out.println("Print metrics");
process = false;
}
//else {
// System.out.println("Waiting: registered=" + cb.getRegisteredParties() + ", arrived=" + cb.getArrivedParties() + ", unarrived=" + cb.getUnarrivedParties());
//}
Thread.sleep(1000);
}
}
}
class ThreadRunner implements Runnable {
static int timeOutTime = 2;
private ExecutorService executorService = Executors.newFixedThreadPool(10);
private final Phaser barrier;
private boolean isFinished=false;
public ThreadRunner(Phaser phaser) {
this.barrier = phaser;
}
public void run(){
try {
boolean stillLoop = true; int i = 0;
while (stillLoop){
int size;
Future<Integer> future = null;
try {
future = executorService.submit(new Reader(this.barrier)); // sleeps
size = future.get();
} catch (InterruptedException | ExecutionException ex) {
// handle Errs
}
if(i == 3){
stillLoop = false;
this.barrier.awaitAdvance(0);
this.isFinished=true;
}
//System.out.println("i = "+i+" Size is: "+size+"\r");
i++;
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
public boolean getIsFinished(){
return this.isFinished;
}
}
class Reader implements Callable {
private Phaser barrier;
private ExecutorService executorService = Executors.newFixedThreadPool(1);
public Reader(Phaser phase) {
phase.register();
this.barrier = phase;
}
#Override
public Object call() throws Exception {
System.out.println("Reading...");
Thread.sleep(2000);
executorService.submit(new Writer(this.barrier));
this.barrier.arrive();
return 1000;
}
}
class Writer implements Callable {
private Phaser barrier;
public Writer(Phaser phase) {
phase.register();
this.barrier = phase;
}
#Override
public Void call() throws Exception {
Thread.sleep(4000);
System.out.println("Wrote");
this.barrier.arrive();
return null;
}
}
From what I can see you aren't waiting for the Writer to finish in the Reader. Is that the problem you are seeing?
You are also accessing isFinished from more than one thread without synchronization (which however, merely may delay the termination of the loop in this situation).
I don't see CyclicBarrier doing anything.
Not sure what you are trying to do, but I'd think about how simpler I can make it. For example, can Reader and Writer be combined into one task? Then, waiting for them to finish would merely be:
executorService.invokeAll(tasks);
System.out.println("Print metrics");
where tasks is a collection of tasks (see also this javadoc)