Threadpool rejecting tasks while submitting. Threadpool size is fixed and it is 8. Even though i am not sumbitting the tasks more than 8 it is rejecting. I tried using the blocking queue but it is not helping me.
Here is my code snippet
try {
List<Future> tasks = new ArrayList<Future>();
ThreadPoolExecutor tpe = (ThreadPoolExecutor) Executors.newFixedThreadPool(10);
Process process = new Process();
ProcessingJobMeta meta = process.getPJM();
List<CuratedInput> cil = meta.getCuratedInputList();
for (final CuratedInput ci : cil) {
for (final Preperation prep : Preperation.values()) {
for (final Export export : Export.values()) {
Runnable runnable = new Runnable() {
public void run() {
LOGGER.info("Executing.................." + prep.toString() );
LOGGER.info("Executing.................." + export.toString());
PreperationFactory.getPreperation(prep.toString(), ci);
ExportFactory.getExport(export.toString(), ci);
}
};
// tpe.submit(runnable);
tasks.add((Future) tpe.submit(runnable));
for (Future p : tasks) {
LOGGER.info("---------------inside the futures for loop------------");
LOGGER.info("Result of the future executed ------> " + p.get());
}
tpe.shutdown();
while (!tpe.isShutdown()) {
}
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
Your problem is that your shutting down the pool within the loop and attemting to add more threads to an already shutdown pool. place these lines out of the loop
tpe.shutdown();
while (!tpe.isShutdown()) {
}
something like this
List<Future> tasks = new ArrayList<Future>();
ThreadPoolExecutor tpe = (ThreadPoolExecutor) Executors.newFixedThreadPool(10);
Process process = new Process();
ProcessingJobMeta meta = process.getPJM();
List<CuratedInput> cil = meta.getCuratedInputList();
for (final CuratedInput ci : cil) {
.....
}
tpe.shutdown();
while (!tpe.isShutdown()) {
}
Please try that
Related
Trying to use Futures and IntelliJ is giving me various warnings, not sure how to code it 'correctly'. The code works but obviously want to learn best practice.
try {
public void futuresTest() {
try {
List<String> valuesToProcess = List.of("A","B","C","D","E");
ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
List<Future<MyObject>> futures = new ArrayList<>();
for(String s : valuesToProcess) {
futures.add((Future<MyObject>) executor.submit(new MyObject(s))); //<THIS
}
LOG.info("Waiting for threads to finish...");
boolean termStatus = executor.awaitTermination(10, TimeUnit.MINUTES);
if (termStatus) {
LOG.info("Success!");
} else {
LOG.warn("Timed Out!");
for(Future<MyObject> f : futures) {
if(!f.isDone()) {
LOG.warn("Failed to process {}:",f);
}
}
}
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
Gives Unchecked cast: 'java.util.concurrent.Future<capture<?>>' to 'java.util.concurrent.Future<model.MyObject>'
List<Future> futures = new ArrayList<>();
for(String s : valuesToProcess) {
futures.add( executor.submit(new MyObject(s)));
}
Gives Raw use of parameterized class 'Future'
is it just supposed to be List<Future<?>> futures = new ArrayList<>(); that has no warnings but I would think I should be specifying my Object.
Based on the comments it does sound like the correct approach is
List<Future<?>> futures = new ArrayList<>();
for(String s : valuesToProcess) {
futures.add(executor.submit(new MyObject(s)));
}
I have this code and what it does is wait for all tasks are complished and only then return values to the widget. WorkerThread is a runnable that needs to finish before next loop.
final ScheduledExecutorService ecs = Executors.newScheduledThreadPool(size2/2);
while(size2>1) {
for (int i = 0; i < size2/2; i++) {
Runnable worker = null;
try {
worker = new WorkerThread(players.take(), players.take() ,area,players);
} catch (InterruptedException interruptedException) {
interruptedException.printStackTrace();
}
ecs.submit(worker);
}
area.append("\n\n next Round");
size2=size2/2;
}
So what I need to do is to have in this case 8 tasks complete, get values that they append to the widget, then wait for example 2 seconds and proceed with the loop now with 4 tasks.
You can use Phaser.
Phaser phaser = new Phaser(1); // Create Phaser instance
final ScheduledExecutorService ecs = Executors.newScheduledThreadPool(size2/2);
while(size2>1) {
for (int i = 0; i < size2/2; i++) {
Runnable worker = null;
try {
worker = new WorkerThread(players.take(), players.take() ,area,players);
} catch (InterruptedException interruptedException) {
interruptedException.printStackTrace();
}
phaser.register(); // Register a party
ecs.submit(worker);
}
phaser.arriveAndAwaitAdvance(); // Wait for completion of all tasks
area.append("\n\n next Round");
size2=size2/2;
}
Here is your consumer.
public class WorkerThread {
public WorkerThread(Phaser phaser) {
this.phaser = phaser;
}
public void run() {
processThings();
phaser.arriveAndDeregister(); // Deregister a party
}
}
Below is my code:
public class Controller {
public Button button_submitWork;
#FXML
public void handleSubmitWork(ActionEvent event) {
final ExecutorService executorService = Executors.newFixedThreadPool(1, r -> {
Thread t = Executors.defaultThreadFactory().newThread(r);
t.setDaemon(true);
return t;
});//set thread daemon, let all threads terminate when the program is closed.
Callable<String> callable = new Callable<String>() {
#Override
public String call() throws Exception {
System.out.println("Executor Service thread");
StringBuilder stringBuilder_output = new StringBuilder();
for (int k = 0; k < 5; k++) {
stringBuilder_output.append(k);
}
//Thread.sleep(1000);
return stringBuilder_output.toString() + "\n";
}
};
Future<String> future = executorService.submit(callable);//Weird line.
//This line must be placed inside the "watchThread" to get the result, but why???
Thread watchThread = new Thread(new Runnable() {
#Override
public void run() {
//<----------Moving to here solve the problem!
System.out.println("Watch thread");
while (!Thread.currentThread().isInterrupted() && !future.isDone()) {
try {
String result = future.get();
System.out.println(result);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
} finally {
executorService.shutdownNow();
}
}
}
});
watchThread.setDaemon(true);
watchThread.start();
System.out.println("FX thread");
}
}
The question is that the System.out.println(result); inside "watchThread" is never been called. The console output looks like this:
Executor Service thread
FX thread
Watch thread
But when I move the Future<String> future = executorService.submit(callable); to the inside of run method of "watchThread", the output change to:
FX thread
Watch thread
Executor Service thread
01234
which is I expected.
I also discovered that if the call() method has a longer task, say a Thread.sleep(1000), the output change to the result I expected.
So why is that?
The thread you submit to executorService finishes before this line:
while (!Thread.currentThread().isInterrupted() && !future.isDone()) { is called so future.isDone returns true and the while loop is not executed.
If you add Thread.sleep(1000) then it still runs and future.isDone returns false and the while loop executes. The same thing happens when you move Future<String> future = executorService.submit(callable); inside watchThread.
I call the below testMethod, after putting it into a Callable(with other few Callable tasks), from an ExecutorService. I suspect that, the map.put() suffers OutOfMemoryError, as I'm trying to put some 20 million entries.
But, I'm not able to see the error trace in console. Just the thread stops still. I tried to catch the Error ( I know.. we shouldnt, but for debug I caught). But, the error is not caught. Directly enters finally and stops executing.. and the thread stands still.
private HashMap<String, Integer> testMethod(
String file ) {
try {
in = new FileInputStream(new File(file));
br = new BufferedReader(new InputStreamReader(in), 102400);
for (String line; (line= br.readLine()) != null;) {
map.put(line.substring(1,17),
Integer.parseInt(line.substring(18,20)));
}
System.out.println("Loop End"); // Not executed
} catch(Error e){
e.printStackTrace(); //Not executed
}finally {
System.out.println(map.size()); //Executed
br.close();
in.close();
}
return map;
}
Wt could be the mistake, I'm doing?
EDIT: This is how I execute the Thread.
Callable<Void> callable1 = new Callable<Void>() {
#Override
public Void call() throws Exception {
testMethod(inputFile);
return null;
}
};
Callable<Void> callable2 = new Callable<Void>() {
#Override
public Void call() throws Exception {
testMethod1();
return null;
}
};
List<Callable<Void>> taskList = new ArrayList<Callable<Void>>();
taskList.add(callable1);
taskList.add(callable2);
// create a pool executor with 3 threads
ExecutorService executor = Executors.newFixedThreadPool(3);
List<Future<Void>> future = executor.invokeAll(taskList);
//executor.invokeAll(taskList);
latch.await();
future.get(0);future.get(1); //Added this as per SubOptimal'sComment
But, this future.get() didn't show OOME in console.
You should not throw away the future after submitting the Callable.
Future future = pool.submit(callable);
future.get(); // this would show you the OOME
example based on the informations of the requestor to demonstrate
public static void main(String[] args) throws InterruptedException, ExecutionException {
Callable<Void> callableOOME = new Callable<Void>() {
#Override
public Void call() throws Exception {
System.out.println("callableOOME");
HashMap<String, Integer> map = new HashMap<>();
// some code to force an OOME
try {
for (int i = 0; i < 10_000_000; i++) {
map.put(Integer.toString(i), i);
}
} catch (Error e) {
e.printStackTrace();
} finally {
System.out.println("callableOOME: map size " + map.size());
}
return null;
}
};
Callable<Void> callableNormal = new Callable<Void>() {
#Override
public Void call() throws Exception {
System.out.println("callableNormal");
// some code to have a short "processing time"
try {
TimeUnit.SECONDS.sleep(5);
} catch (InterruptedException ex) {
System.err.println(ex.getMessage());
}
return null;
}
};
List<Callable<Void>> taskList = new ArrayList<>();
taskList.add(callableOOME);
taskList.add(callableNormal);
ExecutorService executor = Executors.newFixedThreadPool(3);
List<Future<Void>> future = executor.invokeAll(taskList);
System.out.println("get future 0: ");
future.get(0).get();
System.out.println("get future 1: ");
future.get(1).get();
}
Try catching Throwable as it could be an Exception like IOException or NullPointerException, Throwable captures everything except System.exit();
Another possibility is that the thread doesn't die, instead it becomes increasingly slower and slower due to almost running out of memory but never giving up. You should be able to see this with a stack dump or using jvisualvm while it is running.
BTW Unless all you strings are exactly 16 characters long, you might like to call trim() on the to remove any padding in the String. This could make them shorter and use less memory.
I assume you are using a recent version of Java 7 or 8. If you are using Java 6 or older, it will use more memory as .substring() doesn't create a new underlying char[] to save CPU, but in this case wastes memory.
I have problem converting my code with the runnable interface to the callable interface in the following code. I need to change, because I need to return a Sting[][] isRs by the threads.
When I just change the interface to callable and chande .run() to .call(), then new Thread(new Worker(startSignal, doneSignal, i)).start(); wont work.
CountDownLatch startSignal = new CountDownLatch(1);
CountDownLatch doneSignal = new CountDownLatch(3); // 3 tasks
class Worker implements Runnable {
private final CountDownLatch startSignal;
private final CountDownLatch doneSignal;
private final int threadNumber;
// you can pass additional arguments as well
Worker(CountDownLatch startSignal, CountDownLatch doneSignal, int threadNumber) {
this.startSignal = startSignal;
this.doneSignal = doneSignal;
this.threadNumber = threadNumber;
}
public void run() {
try {
startSignal.await();
if (threadNumber == 1) {
String[][] isRs = getIS(erg1, erg2, request);
}
if (threadNumber == 2) {
getIW(erg1, erg2, request);
}
if (threadNumber == 3) {
getIN(search_plz, request);
}
doneSignal.countDown();
} catch (InterruptedException ex) {
System.out.println(ex);
}
}
}
// 3 new threads are started
for (int i = 1; i <= 3; i++) {
new Thread(new Worker(startSignal, doneSignal, i)).start();
}
startSignal.countDown(); // let all threads proceed
try {
doneSignal.await(); // wait for all to finish
// all 3 tasks are finished and do whatever you want to do next
} catch (Exception e) {
}
You cannot pass a Callable into a Thread to execute.
Use the ExecutorService to execute the Callable object.
You can give it Callable objects to run using its submit() method:
<T> Future<T> submit(Callable<T> task)
Your class should look like:
class Worker {
private final CountDownLatch startSignal;
private final CountDownLatch doneSignal;
private final int threadNumber;
Worker(
CountDownLatch startSignal,
CountDownLatch doneSignal,
int threadNumber
){
this.startSignal = startSignal;
this.doneSignal = doneSignal;
this.threadNumber = threadNumber;
}
public String[][] getSomeStrArrArr() {
try {
startSignal.await();
if (threadNumber == 1) {
System.out.println("Running thread number 1");
}
if (threadNumber == 2) {
System.out.println("Running thread number 2");
}
if (threadNumber == 3) {
System.out.println("Running thread number 3");
}
doneSignal.countDown();
} catch (InterruptedException ex) {
System.out.println(
"Thread number "+threadNumber+" has been interrupted."
);
}
// replace these 2 lines with the actual code to get the String[][]
String[][] someStrArrArr = new String[1][1];
someStrArrArr[0][0] = "Done with thread number "+threadNumber;
return someStrArrArr;
}
public Callable<String[][]> getSomeCallableStrArrArr(){
return new Callable<String[][]>() {
public String[][] call() throws Exception {
return getSomeStrArrArr();
}
};
}
}
And you'd start it like:
ExecutorService pool = Executors.newFixedThreadPool(3);
Set<Future<String[][]>> set = new HashSet<Future<String[][]>>();
CountDownLatch startSignal = new CountDownLatch(1);
CountDownLatch doneSignal = new CountDownLatch(3);
for (int i=1;i<=3;i++) {
Worker worker = new Worker(startSignal,doneSignal,i);
Callable<String[][]> callable =
worker.getSomeCallableStrArrArr();
Future<String[][]> future = pool.submit(callable);
set.add(future);
}
And, to get and print the result strings:
for(Future<String[][]> future : set){
String[][] result = future.get();
for (String[] strArr: result){
for (String str: strArr){
System.out.println(str);
}
}
}
But this design can be improved. Have a look at the following documentation on Callable to see how it differenciates from Runnable and how you can get advantage from those differences and implent it properly:
Interface Callable
https://blogs.oracle.com/CoreJavaTechTips/entry/get_netbeans_6?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+corejavatechtips+(Core+Java+Technologies+Tech+Tips)
Also check out this link where I've written an example based on your code you can run and fiddle with: http://ideone.com/blUQm0
Once you your class implements callable interface you will have method call and its having return type.
You can use the below code for ExecutorService :-
ExecutorService service = Executors.newSingleThreadExecutor();
Worker worker= new Worker (tartSignal, doneSignal,threadNumber);
Future<Integer> future = service.submit(worker);
Object result = future.get();
Hope this will help to resolve your issue.