I run analysis on a file directly from an analyzer program and it finishes under one minute. But if I make jar of analyzer and run it via ProcessBuilder, it does not get finished even in 8 minutes (500 sec). Here is the code I am using for ProcessBuilder. Can someone please explain the reason?
I cannot run the analyzer program directly because depending upon the input file, it may actually take 15-20 minutes which I don't want. I want to finish it under 8 minutes.
Here is code for ProcessBuilder.
public void myFun(){
String fileName = file.getName();
System.out.println(file.getAbsolutePath());
Process p;
ProcessBuilder pb;
String filePath = file.getAbsolutePath();
pb = new ProcessBuilder("C:\\Program Files\\Java\\jdk1.7.0_13\\bin\\java.exe", "-jar", "ta.jar", filePath);
pb.directory(new File("D:\\Softwares\\analyzerRun\\bin"));
long startTime = System.currentTimeMillis();
try {
p = pb.start();
long currTime = System.currentTimeMillis();
long diff = currTime - startTime;
boolean isBreak = false;
while(diff < 500000)
{
currTime = System.currentTimeMillis();
diff = currTime - startTime;
if(processIsTerminated(p))
{
isBreak = true;
break;
}
}
if(!isBreak)
{
System.out.println("Interrupting current thread!!");
p.destroy();
Thread.currentThread().interrupt();
}
else
{
System.out.println("process terminated peacefully");
}
System.out.println("Done with "+ fileName);
} catch (IOException e) {
e.printStackTrace();
}
}
}
private static boolean processIsTerminated (Process process) {
try {
process.exitValue();
} catch (IllegalThreadStateException itse) {
return false;
}
return true;
}
Edit1:
So after given suggestions, I removed while loop and my try code looks like below now:
try {
p = pb.start();
p.waitFor();
}
Interestingly, I run this program and it does not get finished under 5 minutes. But if I run my analyzer directly, it gets finished under 30 seconds. Does ProcessBuilder not get enough priority from CPU?
Your polling loop is insane.
Suggestion:
1) start() your process
2) Save the start time.
3) Call waitFor() to block until the process is done.
4) Then take the delta.
You'll get MUCH better results - honest!
If you absolutely must run asynchronously (if your app cannot block while your subprocess runs), then create a new Java thread for steps 1) ... 4).
IMHO...
ALSO:
Run Windows Performance Monitor (or equivalent for your OS) and monitor I/O wait, CPU utilization, etc while the process is running to try to figure out where the latency might be coming from.
But your polling loop is itself introducing excessive CPU utilization. Don't do it!
Because you're wasting most of the time smoking the CPU with sleep-free polling, instead of:
(a) consuming the process's output and error streams, and then
(b) calling waitFor().
To answer this, "Interestingly, I run this program and it does not get finished under 5 minutes. But if I run my analyzer directly, it gets finished under 30 seconds. Does ProcessBuilder not get enough priority from CPU? "
After a long-time, I finally figured out that I was writing two lines on standard output in my original program. So, when I ran the jar file using ProcessBuilder, I didn't read from standard output. The process was getting hanged ultimately getting killed by the main-class thread. I just removed those two lines from the original program and now it takes equal time in both ways.
Related
I'm using maven surefire plugin (LTS Version) to execute tests on two testing frameworks (e.g, jUnit, jBehave).
Have tried to implement parallelisation by spawning couple of Threads which in turn create processes to execute surefire jar, taking it from -
ManagementFactory.getRuntimeMXBean().getSystemProperties().get("sun.java.command")
Code snippet to show process creation -
CustomRunner.java
void run() {
ProcessBuilder processBuilder = new ProcessBuilder(commandArray);
Map<String, String> environment = processBuilder.environment();
environment.put("platformIndex", String.valueOf(platformIndex));
try {
processBuilder.inheritIO();
Process p = processBuilder.start();
LOGGER.info("Is Alive {} {}", p.isAlive(), LocalTime.now());
int statusCode = p.waitFor();
} catch (Exception e) {
e.printStackTrace();
}
}
EntryPoint.java
for (int i = 0; i < 3; i++) {
Thread thread = new Thread(new CustomRunner(commandArray, String.valueOf(i)));
thread.start();
threadList.add(thread);
}
threadList.forEach(thread -> {
try {
thread.join();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
});
System.exit(exitcode);
After running two or sometimes three processes in corresponding Threads, the process execution got stuck on p.waitFor();
Then the process exits after 30 secs and with error message "Surefire is going to kill self fork JVM. The exit has elapsed 30 seconds after System.exit(0)." resulting in Build Failure (sometimes it doesn't) though the tests have passed in their respective processes.
Seems like surefire execution is stuck in some processes. Could you please let me know what can be the possible reasons for it and how to mitigate this? Tried extending the ForkedProcessTimeoutInSeconds to few minutes but no luck.
Any help is much appreciated.
We are executing script and Linux command using process builder. Below is a rough example of the way we are using process builder.
class Example {
class Stopper extends TimerTask {
Thread t ;
Stopper( Thread t ){
this.t = t;
}
public void run (){
t.interrupt();
}
}
String command= <some command or script>;
String workingDirectory = <working directory>;
ProcessBuilder processbuilder = new ProcessBuilder(command);
processbuilder.setWorkingDirectory(workingDirectory);
processBuilder.redirectErrorStream(true);
Timer timer = new Timer(true);
Stopper stop = new Stopper(Thread.currentThread());
timer.schedule(stop , 5); // 5 millisec
Process process = processBuilder.start();
// code to call the thread to read the output stream.
process.waitFor();
}
In spite of reading from the output stream, the process.waitFor() get interrupted by timer thread. When thread get interrupted below is snippet of error we see ( pasting only the relevant part )
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.lang.UNIXProcess.waitFor(UNIXProcess.java:395)
The thread is interrupted intermittently and not always which means out by 1000 times same command run, it will timeout 10 times only. We tried increasing the timeout period but still it's not able to complete the command execution.
After searching for the issue, I see a bug in java which I feel is the issue but not sure. Any help on this is appreciated.
https://bugs.openjdk.java.net/browse/JDK-8169565
I'm trying to execute a visual basic script code in my java application using process builder. As script provided by the user might not finish its execution in time, I want to provide means to limit this execution time. In the following code, you can see my logic but it doesn't really do what it supposed to do. How can I make this waitfor work in order to limit the execution time?
private void run(String scriptFilePath) throws ScriptPluginException {
BufferedReader input = null;
BufferedReader error = null;
try {
ProcessBuilder p = new ProcessBuilder("cscript.exe", "//U", "\"" + scriptFilePath + "\"");
String path = "";
if (scriptFilePath.indexOf("/") != -1) {
path = scriptFilePath.substring(0, scriptFilePath.lastIndexOf("/"));
}
path += "/" + "tempvbsoutput.txt";
p.redirectOutput(new File(path));
Process pp = p.start();
try {
pp.waitFor(executionTimeout, TimeUnit.MINUTES);
} catch (InterruptedException e) {
SystemLog.writeError(jobId, ScriptConsts.COMPONENT_ID, "VBScriptExecutor", "run", 80401104,
"VB Script executes fail.");
}
if (!pp.isAlive()) {
pp.getOutputStream().close();
}
// rest of the code flow
}
Process.waitFor(long, TimeUnit) waits until the process has terminated or the specified time elapsed (Javadoc). The return value indicates whether the process exited or not.
if (process.waitFor(1, TimeUnit.MINUTES)) {
System.out.println("process exited");
} else {
System.out.println("process is still running");
}
waitFor() does not kill the process after the time elapsed.
If you want to kill the subprocess, use either destroy() or destroyForcibly().
My application uses some daemon subprocesses for subtasks. The subprocesses are launched using ProcessBuilder and working fine on their own, but then starting them as subprocesses every associated Process.isAlive() method return FALSE. As following, no access to process is possible.
Further investigation shows the subprocesses are not started at all (don't exist in Task Manager) with no error generated at all.
Daemons typically start a separate process and exit almost immediately, which makes checks like isAlive() useless.
Often the program will have a command line switch that make the program stay in the foreground, not becoming a daemon - use that if possible. Otherwise you'll need some other way of monitoring the daemon execution, for example using the daemon's PID file.
Is the command really running? Often there are weird little issues when trying to run a program from inside Java.
For example, the PATH environment variable may not be set correctly so it fails to load a dependency.
Use this method to see if there is any console output and what the exit code is. This uses the old Runtime class instead of ProcessBuilder. It can probably be adapted to use ProcessBuilder.
public static void runExe(String[] command) throws IOException {
Runtime runtime = Runtime.getRuntime();
long start = System.currentTimeMillis();
Process proc = runtime.exec(command);
BufferedReader br = new BufferedReader(new InputStreamReader(proc.getInputStream()));
try {
while (true) {
// enter a loop where we read what the program has to say and wait for it to finish
// read all the program has to say
while (br.ready()) {
String line = br.readLine();
System.out.println("CMD: " + line);
}
try {
int exitCode = proc.exitValue();
System.out.println("exit code: " + exitCode);
// if we get here then the process finished executing
break;
} catch (IllegalThreadStateException ex) {
// ignore
}
// wait 200ms and try again
Thread.sleep(200);
}
} catch (InterruptedException ex) {
ex.printStackTrace();
}
long end = System.currentTimeMillis();
System.out.println("Command took: " + (end - start) + "ms");
}
I wrote a java class in order to perform multithreaded tasks, each task running an external process.
The process is in charge of converting ".chp" files into ".txt" files. It is written in C.
This process breaks at one point because it disappears when looking at a "top" in my terminal (probably due to a corrupted chp file). The problem is that the process in my java thread does not return. The "process.waitFor()" seems to go on forever (at least 'til the 12 hours I specified for the ExecutorService.
Am I doing something wrong (not catching an exception?)?
I tried setting a class variable of type String in MyThread and putting an error message in place of throwing a new RuntimeException, then print the String at the end of the main, but the thread code doesn't reach to this point. It still gets stuck at the waitFor().
Shouldn't the process terminate once the C program has failed?
The program prints on the terminal (cf: MyThread):
A
B
C
main:
String pathToBin = "/path/to/bin";
List<MyThread> threadList = new ArrayList<MyThread>();
for (File f : folderList) {
File[] chpFilesInFolder = f.listFiles(new FilenameFilter() {
#Override
public boolean accept(File dir, String name) {
if (name.endsWith(".chp")){
return true;
}else{
return false;
}
}
});
File chpFile = writeChpFiles(chpFilesInFolder);
String[] cmd = {pathToBin, "--arg1", chpFile, "--out-dir", outputFolder};
MyThread t = new MyThread(cmd, f, chpFilesInFolder);
threadList.add(t);
}
ExecutorService threadExecutor = Executors.newFixedThreadPool(4);
for(MyThread th : threadList){
threadExecutor.execute(th);
}
threadExecutor.shutdown();
try {
threadExecutor.awaitTermination(12, TimeUnit.HOURS);
} catch (InterruptedException e) {
e.printStackTrace();
}
MyThread:
class MyThread extends Thread{
private String[] cmd;
private File chpFolder;
private File[] chpFilesInFolder;
public MyThread(String[] cmd, File chpFolder, File[] chpFilesInFolder){
this.cmd = cmd;
this.chpFolder = chpFolder;
this.chpFilesInFolder = chpFilesInFolder;
}
#Override
public void run() {
Process process = null;
try{
System.err.println("A ");
ProcessBuilder procBuilder = new ProcessBuilder(cmd);
procBuilder.redirectErrorStream(true);
System.err.println("B");
process = procBuilder.start();
System.err.println("C");
process.waitFor();
System.err.println("D");
if(process.exitValue()!=0) System.err.println("ERROR !"+process.exitValue());
System.err.println("E");
}catch(IOException e){
e.printStackTrace();
}catch(InterruptedException e){
e.printStackTrace();
}catch(Throwable e){
e.printStackTrace();
}finally{
System.err.println("F");
if(process!=null) {try { process.destroy();} catch(Exception err) {err.printStackTrace();}}
}
File[] txtFilesInFolder = chpFolder.listFiles(new FilenameFilter() {
#Override
public boolean accept(File dir, String name) {
if (name.endsWith(".chp.txt")){
return true;
}else{
return false;
}
}
});
if (txtFilesInFolder.length==chpFilesInFolder.length){
for (File chp : chpFilesInFolder) {
chp.delete();
}
File logFile = new File(chpFolder, "apt-chp-to-txt.log");
if (logFile.exists()){
logFile.delete();
}
}else{
throw new RuntimeException("CHPs have not all been transformed to TXT in "+chpFolder.getAbsolutePath());
}
Is it possible that your C program is producing output on stdout? If so, you need to read Process.getOutputStream() before Process.waitFor() returns - see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4254231
Alternatively, call your C program that a shell script that redirects stdout.
You can use the jstack command to confirm that the thread is indeed blocked at Process.waitFor().
You could have the main thread wait for a reasonable amount of time and then call some method on the MyThread class to kill the started process, thus causing the thread to finish.
as often I would suggest to use a more robust and professional point of view while using a messsaging solution to make your C program interact with your Java application, it will be easy and clean to avoid those non daemon threads waiting for ever because of the crash of your C application... now all brokers have a STOMP interface which is pretty cool for any kind of application to invoke (just use any Http library), broker configuration will enable to restart non finished jobs, to put some timeouts and so one..Even if JMS does not support request and response it's quite easy to implement such paradigm....
HTH
Jerome
If I understad correctly, your Java threads remain waiting after the C program crashes.
Make the spawned C process send heart beats. You can do this even by printing sth to console (or inserting in a table) and have the Java thread every so often wake up and check the heartbeat. If it's not there, assume the C process died and terminate the thread.
Launching external processes in Java can get a little bit tricky. I usually try to avoid them as you'll have to deal with different error codes and some terminal madness. I recommend you use specialized libraries such as commons-exec (http://commons.apache.org/exec/)