We Keep Coding

Java executor service: Waiting for all tasks to finish

I am trying to introduce concurrency in my program. Structure of program is something like this:
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);
List<String> initialData = dao.fetchFromDB("input");
Queue queue = new MyQueue();
queue.add(initialData);
while(queue.length() > 0) {
int startingLength = queue.length();
for (int i = 0; i < startingLength; i++) {
String input = queue.remove();
if(input.equals("some value")) {
missionAccomplished = true;
break;
} else {
MyRunnable task = new MyRunnable(input, queue, dao);
executor.execute(task);
}
}
if(missionAccomplished) {
break;
}
executor.shutdown();
}
So queue contains the data needed to be processed one by one. Inside while loop I run a for loop which picks data from queue one by one sequentially and performs some check on it, and if check fails I create a runnable task with this data and hands it over to executor(as DB operation is time consuming, I want to use parallelism for it). for loop picks data only upto certain length in given iteration of while.
What I want to achieve is that 'while' loop goes to next iteration only when all tasks submitted to executor in current iteration are finished.
How can this be achieved?

try-with-resources in Project Loom
You asked:
What I want to achieve is that 'while' loop goes to next iteration only when all tasks submitted to executor in current iteration are finished.
Project Loom promises to make this simpler.
One of the changes brought by Project Loom is that the ExecutorService interface is a sub-interface of AutoCloseable. This means we can use try-with-resources syntax. The try-with-resources automatically blocks until all submitted tasks are done/failed/canceled — just what you asked for.
Also, the executor service is automatically shut down when exiting the try. These changes mean your code becomes simpler and clearer.
Also, for code that blocks often, such as database access, you will see dramatically faster performance using virtual threads (a.k.a. fibers). Virtual threads is another new feature of Project Loom. To get this feature, call Executors.newVirtualThreadExecutor.
Experimental builds of Project Loom are available now, based on early-access Java 17. The Loom team is asking for feedback. For more info, see recent presentations and interviews by Ron Pressler of Oracle.
System.out.println( "INFO - executor service about to start. " + Instant.now() );
try (
ExecutorService executorService = Executors.newVirtualThreadExecutor() ;
)
{
for ( int i = 0 ; i < 7 ; i++ )
{
executorService.submit( ( ) -> System.out.println( Instant.now() ) );
}
}
// Notice that when reaching this point we block until all submitted tasks still running are fin
// because that is the new behavior of `ExecutorService` being `AutoCloseable`.
System.out.println( "INFO - executor service shut down at this point. " + Instant.now() );
When run.
INFO - executor service about to start. 2021-02-08T06:27:03.500093Z
2021-02-08T06:27:03.554440Z
2021-02-08T06:27:03.554517Z
2021-02-08T06:27:03.554682Z
2021-02-08T06:27:03.554837Z
2021-02-08T06:27:03.555015Z
2021-02-08T06:27:03.555073Z
2021-02-08T06:27:03.556675Z
INFO - executor service shut down at this point. 2021-02-08T06:27:03.560723Z

Is there a when process ends method in java?

I am trying to search when GTA5.exe ends. It launches with "steam://rungameid" protocol, so I can't use Process#waitFor(); method.
The method below checks if (currently) the process is running
public static boolean isProcessRunning(String processName) throws IOException {
ProcessBuilder processBuilder = new ProcessBuilder("tasklist.exe");
Process process = processBuilder.start();
String tasksList = toString(process.getInputStream());
return tasksList.contains(processName);
}
But what I want is something like
waitUntilProcessEnds(String processname) {
}
It could be
while (isProcessRunning) {
Thread.sleep(1000);
}
And, as expected, my JavaFX app freezes.
If I try to run the while method in another Thread, my JavaFX App gives this error:
java.lang.IllegalStateException: Not on FX application thread; currentThread = Thread-6
at com.sun.javafx.tk.Toolkit.checkFxUserThread(Toolkit.java:279)
at com.sun.javafx.tk.quantum.QuantumToolkit.checkFxUserThread(QuantumToolkit.java:444)
at javafx.scene.control.Dialog.show(Dialog.java:294)
at com.thunderstorm.gta5.mod_installer.app.AppController$1$1.run(AppController.java:101)
at java.lang.Thread.run(Thread.java:748)
I also tried this
Platform.runLater();
and this
Platform.setImplicitExit(false);
But nothing different...
How to fix this
This question is edited

There is an API, but it requires Java 9 or newer. Then, you can do it smoothly:
String name = "GTA5.exe";
ProcessHandle.allProcesses()
.filter(ph -> ph.info().command()
.filter(s -> s.regionMatches(true,s.length()-name.length(),name,0,name.length()))
.isPresent())
.findAny()
.ifPresentOrElse(ph -> {
System.out.println("Waiting for "+ph.pid()+" "+ph.info().command().get());
ph.onExit().thenRunAsync(() -> {
// your actual action
}, Platform::runLater);
}, () -> System.out.println("No matching process found"));
ProcessHandle.onExit() returns a CompletableFuture which allows to chain dependent actions, to be performed on the process’ completion. So you don’t need to poll the process yourself.
Further note that Platform.runLater(Runnable) matches the functional signature of the Executor interface which allows us to use thenRunAsync(…, Platform::runLater) to mandate that the dependent action (Runnable) should be executed in the JavaFX Application Thread, once the process has exited.

I think your design is ok, but you're a bit confused by the threads. First when you start your wait method, go ahead and start a new Thread.
new Thread( ()->{
waitUntilProcessEnds(processname);
Platform.runLater( ()-> callBackOnPlatformThread() );
}).start();
Then you need.
public void callBackOnPlatformThread(){
System.out.println("go ahead and modify fx components.");
}
This will start waiting for the process to end on a new thread, so your javafx UI will continue to be responsive. When it is finished, it calls back on the javafx Platform thread, so you can modify components.
This doesn't check if the thread has already been started, so if you have a button that starts the thread, you could start a bunch of them.

What are you trying to achieve? If you want to execute some code when your application ends then you can add a shutdown hook:
Runtime.getRuntime().addShutdownHook(() -> { ... });
If you want to wait until the external process ends, then you can use:
process.waitFor()
This will block your current thread until the external process exits.

Not if your program didn't create the process in the first place.
Java does not provide operating-system level facilities that are not provided in the underlying OS in the first place. In Unix-like systems, for example, "wait for process exit" can only be executed by an ancestor of the process in question. In Windows, for another example, you have to possess a handle on the process, which you would have obtained from creating the process.
You'll have to roll your own. Create a thread which periodically checks for the existence of the process you are monitoring, by whatever means is suitable to that process and your operating environment. Worst-case you'll have to fork a 'ps' command (or similar) and parse the output.
I can't advise on the JavaFX issue; not my area.

Android Java Running List of Action In separate Thread similar to C# style

I'm .NET Developer but i have to make some code for Android in Java. I need to run 4 calculation operation and i want to make it in separate threads. I can simply create 4 thread and just run it but when device will have less than 4 cores ( e.g 1) it will be same or even less effective like just run in one thread. In C# i can create list of Action and then just run it in Parrarel like this :
C# CODE :
var actions = new List<Action>
{
() => { // some action1},
() => { // some action2},
() => { // some action3},
() => { // some action4},
}
Parallel.ForEach(actions, new ParallelOptions { MaxDegreeOfParallelism = GetCoreCount() }, x => x());
And this will run and this will guarantee that each Action will be process on separate core ( More-Less)
Can I do some thing like this in Java on Android ?
I have no idea how to start this.
Only one thing that come to my head is some like this but this is ugly and ineffective.
int cores = Runtime.getRuntime().availableProcessors();
if(cores = 1)
{
// Create 1 Thread with action1,action2,action3,action4
}
if(cores = 2)
{
// Create 1 Thread with action1,action2
// And Create 2nd Thread with action3,action4
}
if(cores = 3)
{
// Create 1 Thread with action1,
// And Create 2nd Thread with action2
// And Create 3rd Thread with action3
// Thread that first end his action will pick up Action4
}
if(cores >= 4)
{
// Create 1 Thread with action1,
// And Create 2nd Thread with action2
// And Create 3nd Thread with action3
// And Create 4th Thread with action4
}
See even not codded this look very unclear and dirty ( and what when i add 5th action ? or will have 8 action and 8 cores ? )
Can i do something similar to C# ? With List of Action and Parallel execution limited by numbers of core ?
Thanks for advises :)

Try using a threadPoolExecutor https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
Java handles the efficiency issues your worried about, and fyi a computer with one core/cpu will have multiple threads running, there will only be one running at any given time. But any modern computer has a lot of threads running simultaneously, it doesn't depend on how many cores/cpus are present/available.

Java parallel db calls

I've a web application which needs to be extremely fast. But for processing it requires access for multiple data sources. Therefore I decided that it might be useful to make a parallel calls for optimization.
Basically I want to make many different db calls in parallel. Could you please recommend me simple and reliable way and technologies for achieving my goal, it would be useful if you could provide few frameworks and design patterns.
Right now I am using Spring.

You can use the new Java 8 CompletableFuture. It allows to use asynchronously existing synchronous method.
Say you have a list of requests in the form List<DBRequest> listRequest that you want to run in parallel. You can make a stream and launching all requests asynchronously in the following way.
List<CompletableFuture<DBResult>> listFutureResult =
listRequest.stream()
.map(req -> CompletableFuture.supplyAsync(
() -> dbLaunchRequest(req), executor))
.collect(Collectors.toList());
List<DBResult> listResult =
listFutureResult.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList());
To be effective you have to write your own custom Executor
private final Executor executor =
Executors.newFixedThreadPool(Math.min(listRequest.size(), 100),
new ThreadFactory(){
public Thread newThread(Runnable r){
Thread t = new Thread(r);
t.setDaemon(true);
return t;
}
});
Like this you can have enough threads but not too much. Marking threads to deamons allows you to finish the program even if one thread is blocked.
You can find clear explanations about these techniques in the chapter 11 of the book Java 8 in action
== UPDATE for Java 7 ==
If you are stick with Java 7, you can use the following solution:
class DBResult{}
class DBRequest implements Callable<DBResult>{
#Override
public DBResult call(){return new DBResult();}
}
class AsyncTest{
public void test(){
try {
for(Future<DBResult> futureResult : ((ExecutorService)executor).invokeAll(listRequest)){
futureResult.get();
}
} catch (InterruptedException | ExecutionException ex) {
Logger.getLogger(SoTest.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
All requests are run asynchronously and you then wait for their completion, in the order of the list.
Finally, to answer the subsidiary question in the comment, you don't have to create a thread pool for each request.

Interrupt java thread running nashorn script

In the code below i have javascript running in a separate thread from the main one. That script is an infinite loop, so it needs to be terminated somehow. How?
Calling .cancel() is not working AFTER the script begins running. But if i call .cancel() just after the thread initialization, it will terminate it (the commented out line).
package testscriptterminate;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.Timer;
import java.util.TimerTask;
public class TestScriptTerminate extends TimerTask{
private ExecutorService threads;
private Future runScript;
private Timer t;
public TestScriptTerminate(){
t = new Timer();
t.schedule(this, 6000); //let the script run for a while before attempt to cancel
threads = Executors.newFixedThreadPool(1);
runScript = threads.submit(new ScriptExec());
//runScript.cancel(true); //will cancel here, before the script had a change to run, but useless, i want to cancel at any time on demand
}
#Override
public void run(){
//after script has fully initialized and ran for a while - attempt to cancel.
//DOESN'T WORK, thread still active
System.out.println("Canceling now...");
runScript.cancel(true);
}
public static void main(String[] args) {
new TestScriptTerminate();
}
}
class ScriptExec implements Runnable{
private ScriptEngine js;
private ScriptEngineManager scriptManager;
public ScriptExec(){
init();
}
#Override
public void run() {
try {
js.eval("while(true){}");
} catch (ScriptException ex) {
System.out.println(ex.toString());
}
}
private void init(){
scriptManager = new ScriptEngineManager();
js = scriptManager.getEngineByName("nashorn");
}
}

So this is old, but i just wrote this up and thought it would probably be valuable to share. By default there is ~nothing you can do to stop a Nashorn script executing, .cancel() Thread.stop() Thread.interrupt() do nothing, but if you are willing to put in a bit of effort and are ok with rewriting some bytecode, it is achieveable. Details:
http://blog.getsandbox.com/2018/01/15/nashorn-interupt/

JavaScript (under Nashorn), like Java, will not respond to an interrupt in the middle of a tight loop. The script needs to poll for interruption and terminate the loop voluntarily, or it can call something that checks for interruption and let InterruptedException propagate.
You might think that Nashorn is "just running a script" and that it should be interrupted immediately. This doesn't apply, for the same reason that it doesn't apply in Java: asynchronous interruption risks corruption of the application's data structures, and there is essentially no way to avoid it or recover from it.
Asynchronous interruption brings in the same problems as the long-deprecated Thread.stop method. This is explained in this document, which is an updated version of the document linked in the comments.
Java Thread Primitive Deprecation
See also Goetz, Java Concurrency In Practice, Chapter 7, Cancellation and Shutdown.
The easiest way to check for interruption is to call Thread.interrupted(). You can call this quite easily from JavaScript. Here's a rewrite of the example program that cancels the running script after five seconds:
public class TestScriptTerminate {
ScheduledExecutorService pool = Executors.newScheduledThreadPool(2);
void script() {
ScriptEngineManager scriptManager = new ScriptEngineManager();
ScriptEngine js = scriptManager.getEngineByName("nashorn");
try {
System.out.println("Script starting.");
js.eval("while (true) { if (java.lang.Thread.interrupted()) break; }");
System.out.println("Script finished.");
} catch (ScriptException ex) {
ex.printStackTrace();
}
}
void init() throws Exception {
Future<?> scriptTask = pool.submit(this::script);
pool.schedule(() -> {
System.out.println("Canceling now...");
scriptTask.cancel(true);
}, 5, TimeUnit.SECONDS);
pool.shutdown();
}
public static void main(String[] args) throws Exception {
new TestScriptTerminate().init();
}
}
Since we're starting up a thread pool, might as well make it a scheduled thread pool so that we can use it for both the script task and the timeout. That way we can avoid Timer and TimerTask, which are mostly replaced by ScheduledExecutorService anyway.
The usual convention when handling and interrupt is either to restore the interrupt bit or to let an InterruptedException propagate. (One should never ignore an interrupt.) Since breaking out of the loop can be considered to have completed the handling of the interrupt, neither is necessary, and it seems sufficient simply to let the script exit normally.
This rewrite also moves a lot of work out of the constructor into an init() method. This prevents the instance from being leaked to other threads from within the constructor. There is no obvious danger from this in the original example code -- in fact, there almost never is -- but it's always good practice to avoid leaking the instance from the constructor.

Unfortunately it does not work for simple infinite loops: while (true) { }. I tried Thread.cancel(); does not cause the thread to exit. I wanted something foolproof for running scripts in an IntelliJ plugin where a user can make a mistake an cause an infinite loop, hanging the plugin.
The only thing I found to work in most cases is Thread.stop(). Even that does not work for a script like this:
while(true) {
try {
java.lang.Thread.sleep(100);
} catch (e) {
}
}
javascript catches the java.lang.ThreadDeath exception and keeps going. I found that the above sample is impossible to interrupt even with several Thread.stop() issued one after the other. Why would I use several? Hoping that one of them will catch the thread in its exception processing code and abort it. Which does work if there is something in the catch block to process as simple as var i = "" + e; that is enough to cause the second Thread.stop() to end it.
So the moral of the story is there is no fail safe way of ending a runaway script in Nashorn, but there is something that will work on most cases.
My implementation issues a Thread.interrupt(), then politely waits 2 seconds for the thread to terminate and if that fails then it issues Thread.stop() twice. If that does not work, then nothing else will either.
Hope it helps someone eliminate hours of experimentation to find a more reliable method to stop nashorn runaway scripts than hoping on the cooperation of the running script to respect Thread.cancel().

I have a similar problem where I let users write their own scripts.
But before I allow the script to be executed, I parse the script.
and if I find any of the following
(System.sleep. Exit, Thread.sleep, goto) etc
I don't even start the script, and I give user an error.
and then I do a search for all
(for,loops, while, doWhile), and I inject a method.
checkForLoop() just after the loop identifier.
I inject checkForLoop(); into allow user submitted script.
while(users code)
{
}
becomes
while ( checkForLoop() && users code )
{
}
This way before every iteration of their loop, my method is called.
and I can count how many times I was called or check internal timers.
Than I can stop the loops or timers from inside checkForLoop();
Honestly I think its a big security issue anyway, just to blindly let users write script and just execute it.
You need to build in a system that injects your code into their code loops.
Which is not that hard.
There are 100s of safety mechanisms you can apply to users submitted code, there is no RULE that says you need to run their code as is.
I have edited this answer to include a very simple example.
//Step 1
put the users submitted JS code into a Java String called userJSCode;
Step 2
//inject code at the start of their code.
String safeWhile ="var sCount=0; var sMax=10;
function safeWhileCheck(){ sCount++;
if ( return ( sCount > sMax )}";
userJSCode = safeWhile + userJSCode;
//Step 3: inject the custom while code
String injectSsafeWHile = "while( safeWhileCheck() && ";
userJSCode = userJSCode.replace("while(", injectSsafeWHile);
//Step 4: execute custom JS code
nashhorn.execute(injectSsafeWHile);
//Here is users bad submitted code, note no i increment in the loop, it would go on for ever.
var i=0;
while ( i <1000 )
console.log("I am number " + i);
using the steps above we end up with
var sCount=0;var sMax=10;
function safeWhileCheck(){
sCount++;
return ( sCount > sMax )};
var i=0;
while ( safeWhileCheck() && i <1000 )
console.log("I am number " + i)"
Here the while loop only executes a max of 10 times, so whatever you set the limit to.