We Keep Coding

Groovy cannot run batch command "Caused: java.io.IOException: Cannot run program "cmd.exe": error=2, No such file or directory"

I'm trying to run some command on a Windows agent with cmd using the groovy String.execute() in a Jenkins pipeline.
I know that there exists the bat script which I can use and I intentionally don't want to use it in this specific case since it bloats the logs and the Blue Ocean plugin has a limit of number of steps it can shows for any stage.
Basically what I have is basically some cleanup function that I call very often which does a lot of checks and runs multiple commands (at the moment with sh and bat in addition to isUnix() and similar). The result is usually a very bloated logs of steps like sh and bat which makes the logs larger and more difficult to analyze.
I decided to use some native way of running shell commands "silently" and only print its stdout and stderr in case the command fails.
I have written such a function that executes the command
# NonCPS
def exec(cmd) {
def isUnix = Jenkins.instance.getNodes().find { it.getNodeName() == env.NODE_NAME }.toComputer().isUnix()
cmd = (isUnix ? ["/bin/bash", "-c"] : ["cmd.exe", "/c"]) + [cmd]
def proc = cmd.execute()
proc.waitFor()
def result = [out: proc.in.text.trim(), err: proc.err.text.trim(), exitCode: proc.exitValue()]
return result
}
The above function works perfectly for linux agents but not on Windows. I always get the following error:
java.io.IOException: error=2, No such file or directory
at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at java.base/java.lang.Runtime.exec(Runtime.java:591)
at java.base/java.lang.Runtime.exec(Runtime.java:415)
at java.base/java.lang.Runtime.exec(Runtime.java:312)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
Caused: java.io.IOException: Cannot run program "cmd.exe": error=2, No such file or directory
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at java.base/java.lang.Runtime.exec(Runtime.java:591)
at java.base/java.lang.Runtime.exec(Runtime.java:415)
at java.base/java.lang.Runtime.exec(Runtime.java:312)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Or mainly the following line Cannot run program "cmd.exe": error=2, No such file or directory.
I tried replacing "cmd.exe" with the full path "C:\Windows\System32\cmd.exe" or "C:\Windows\SysWOW64\cmd.exe", replace cmd.exe with ["start", "cmd.exe", "/C"], using Runtime.getRuntime().exec() and many others and I cannot make it work.
If I do "git version".execute(), that works fine but I cannot run some more complex commands like "echo hi && git version".execute() which will print "hi && git version" instead of "hi" and the real git version. I thought maybe the PATH environment variable is not set properly so I ran the following command "set".execute() which resulted in the same error above ("set" no such file or directory).
I have been trying various combinations of this for the past 2 days and that is not going anywhere.
Any help is much appreciated. Thanks in advance.
Update:
Found the reason why it is failing on Windows but not on Linux nodes with Jenkins and hit another "harder" wall here.
The problem is that String.exucute() method behaves internally like Runtime.getRuntime().exec() and this uses the runtime environment from the master node even if encapsulated within the node() directive. This basically means that when it was running fine on linux, it was actually running all the time on the master node which is also linux. The reason why it cannot run cmd or start or any similar is because those commands (obviously) do not exist on linux.

Shell script not recording Java exit status on failure

I am trying to write a shell script that records the exit status of a Java program. The script should simple launch a Java app, and if the Java app doesn't run for some reason, the shell script should detect this and take mitigating measures.
The following is my script:
#!/bin/bash
APPNAME="app"
APPFOLDER=$APPNAME
BACKUP=$APPFOLDER"-backup"
LOGFOLDER=$APPNAME"-log"
echo "Starting new app"
java -jar $APPFOLDER/$APPNAME*.jar > $LOGFOLDER/$APPNAME"_$(date+%Y.%m.%d.%s).log"
wait
STATUS=$?
if [ $STATUS -eq 0 ]
then
echo "Deployment successful" $?
else
echo "Deployment failed: ... derp" $?
fi
I have written a simple Swing GUI that runs fine. However, I packaged it as a jar without specifying an entry point. Hence, I should get the error:
Exception in thread "main" java.lang.NoClassDefFoundError: Demo$1
and the script should detect that the application failed to start.
All of this works FINE until I try to launch the Java app in the background using &. Whenever I do this:
java -jar $APPFOLDER/$APPNAME*.jar > $LOGFOLDER/$APPNAME"_$(date+%Y.%m.%d.%s).log" &
the script always returns a 0 for $?, indicating it passed.
What am I doing wrong? Is there a better way to go about detecting if the app failed to launch?
Thanks!

Wait! you are recording the exit status of wait!
This is why you see unexpected result with your script. Look at the man page for bash (wait is a bash built-in so you need to read the bash manual):
wait [-n] [n ...]
Wait for each specified child process and return its termination status. Each n may be a process ID... If n is not given, all currently active child processes are waited for, and the return status is zero(!). If n specifies a non-existent process or job, the return status is 127. Otherwise, the return status is the exit status of the last process ... waited for.
Since you have not specified the n (child pid to wait for) the return status is zero as per spec.
Another question is: do you really need a wait.
If you don't need to run your app in the background then just do this:
echo "Starting new app"
java -jar $APPFOLDER/$APPNAME*.jar > $LOGFOLDER/$APPNAME"_$(date+%Y.%m.%d.%s).log"
STATUS=$?
the only difference is that i removed unnecessary wait.
If for some reason you need to run your app in the background and read exit status later, then you need wait for that pid. To find out the pid of the last background process use special variable $!:
echo "Starting new app"
java -jar $APPFOLDER/$APPNAME*.jar > $LOGFOLDER/$APPNAME"_$(date+%Y.%m.%d.%s).log" &
CHILDPID=$!
wait "${CHILDPID}"
STATUS=$?
Here's short example of how it works:
user#s:~$ (sleep 10 && exit 42)&
[1] 27792
user#s:~$ wait "$!"
[1]+ Exit 42 ( sleep 10 && exit 42 )
user#s:~$ echo $?
42
What I want to know is if the app fails on startup or not. In the case of the former, my script would bag up the app and role out the previous version.
This purpose is too vague. Are you only interested in missing dependencies?
I don't think there is an easy way to distinguish between JRE non-zero exit code and you java application non-zero exit-code.
I can imagine lots of other reasons to unroll deployment many of which do not lead to non-zero exit code.

JVM hangs on exit

I am working on a small app that should sign documents using digital signature and quit.
The signature can be in a PKCS#12 archive (.pfx file) or on a smartcard device.
Working with the pfx file is easy and working fine.
However, sometimes using the smartcard device, the process hangs on Windows 8 PCs.
The document is signed correctly, but the process doesn't terminate. It just hangs.
I'm using the Sun's PKCS#11 provider - sun.security.pkcs11.SunPKCS11
Basically I'm doing this:
SunPKCS11 provider = new SunPKCS11(configuration);
Security.addProvider(provider);
..... some work .....
provider.logout()
Security.removeProvider(provider);
Now... even if I call System.exit(0) or throw an exception at the end of the main method, I can see the stacktrace in the output but the process doesn't terminate.
I've added a shutdown hook to see if it is executed and it is, i.e. the JVM is trying to stop.
The hang occures rarely, only on Windows 8 PCs. Tried with different smartcards and it happens only with cards that use cmp11.dll (dlls are provided from the vendors of the smartcards).
Using the same dll for communication with the smartcard, however, works fine on Windows 7, XP or some Windows 8 PCs
Running it with Java 8, Update 45, on either x86 or x64 Windows 8
Tried to get a thread dump to see what is hanging:
public static void main(String[] args) {
// do my job, register provider, sign documents, remove provider ...
for(int i = 0; i < 20; ++i) {
System.err.println("Sleep... " + i);
Thread.sleep(2 * 1000);
}
System.err.println("Exiting...");
}
If I execute jstack -l 3232 > dump.log 2>&1 when Sleep... x is printing, everything looks OK.
However, if I execute jstack -F -l 3232 > dump2.log 2>&1 when Exiting... is printed and the app hangs (using -F because the process hangs), i got the following:
Attaching to process ID 3232, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.45-b02
Deadlock Detection:
No deadlocks found.
Thread Exception in thread "main"
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at sun.tools.jstack.JStack.runJStackTool(JStack.java:140)
at sun.tools.jstack.JStack.main(JStack.java:106)
Caused by: sun.jvm.hotspot.debugger.DebuggerException: Windbg Error: GetThreadIdBySystemId failed!
at sun.jvm.hotspot.debugger.windbg.WindbgDebuggerLocal.getThreadIdFromSysId0(Native Method)
at sun.jvm.hotspot.debugger.windbg.WindbgDebuggerLocal.getThreadIdFromSysId(WindbgDebuggerLocal.java:284)
at sun.jvm.hotspot.debugger.windbg.amd64.WindbgAMD64Thread.getThreadID(WindbgAMD64Thread.java:88)
at sun.jvm.hotspot.debugger.windbg.amd64.WindbgAMD64Thread.toString(WindbgAMD64Thread.java:81)
at java.lang.String.valueOf(String.java:2982)
at java.io.PrintStream.print(PrintStream.java:683)
at sun.jvm.hotspot.runtime.win32_amd64.Win32AMD64JavaThreadPDAccess.printThreadIDOn(Win32AMD64JavaThreadPDAccess.java:114)
at sun.jvm.hotspot.runtime.JavaThread.printThreadIDOn(JavaThread.java:265)
at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:79)
at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
at sun.jvm.hotspot.tools.JStack.run(JStack.java:66)
at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
at sun.jvm.hotspot.tools.JStack.main(JStack.java:92)
... 6 more
I can see the process with PID 3232 in the task manager!
Any idea why it is not terminating or why jstack fails?
EDIT
Ok, tried to extract the signing in a separate process, execute it with Runtime.exec and then kill it with Process.destroy but... doesn't seem to help. The child process still stays in the task manager.
Aaaaand... now I have no other choice but to make it kill itself ;(
try {
String name = java.lang.management.ManagementFactory.getRuntimeMXBean().getName();
Runtime.getRuntime().exec("taskkill.exe /F /PID " + name.split("#")[0]);
}
catch(Throwable t) {
Runtime.getRuntime().exec("taskkill.exe /F /IM java.exe");
}
EDIT 2
Tried with Runtime.halt as well. Still doesn't terminate the process...
I would appreciate any ideas!

This won't address your root cause, but this method can be used to force the JVM to terminate:
http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html#halt(int)
As the Javadoc says, use with extreme caution ;-)

I have some problem with sun.security.pkcs11.SunPKCS11 on Windows 8 PCs.This is working for me:
Runtime.getRuntime().exec("taskkill.exe /F /PID " + name.split("#")[0]);
Thread.sleep(500);

Using an environment variable for java classpath not working within shell script

I have noticed an issue that crops up repeatedly: using an env var for the classpath on the java command line within a shell script does not work.
First off, let us see what does work: both using hard-coded classpath in a script as follows: (note: the "classpath is" statement is printed within the java program itself)
steve#mithril:/shared$ java -classpath .:/shared/mysql-connector-java-5.1.25-bin.jar DbPing com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/mysql user password
classpath is .:/shared/mysql-connector-java-5.1.25-bin.jar
Attempting connection to com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/mysql password
Connecting to user using URL=jdbc:mysql://localhost:3306/mysql
Successfully connected.
and also using env var directly within the shell:
steve#mithril:/shared$ export CP=.:/shared/mysql-connector-java-5.1.25-bin.jar
steve#mithril:/shared$ java -classpath $CP DbPing com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/mysql user password
classpath is .:/shared/mysql-connector-java-5.1.25-bin.jar
Attempting connection to com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/mysql password
Connecting to user using URL=jdbc:mysql://localhost:3306/mysql
Successfully connected.
What does *not * work: running the same commands as shown above within a shell script:
steve#mithril:/shared$ cat dbping.mysql
CP=.:/shared/mysql-connector-java-5.1.25-bin.jar
echo $CP
java -classpath $CP DbPing com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/mysql user password
#java -classpath .:/shared/mysql-connector-java-5.1.25-bin.jar DbPing com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/mysql user password
steve#mithril:/shared$ ./dbping.mysql
.:/shared/mysql-connector-java-5.1.25-bin.jar
classpath is .:/shared/mysql-connector-java-5.1.25-bin.jar
Attempting connection to com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/mysql password
Could not load db driver com.mysql.jdbc.Driver
java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at DbPing.getConnection(DbPing.java:34)
at DbPing.main(DbPing.java:22)
Exception in thread "main" java.sql.SQLException: com.mysql.jdbc.Driver
at DbPing.getConnection(DbPing.java:41)
at DbPing.main(DbPing.java:22)
Follow-up: The script had windows style newlines in it. Apparently the \r clobbered the internal environment variable. found this using
od -cx
. I am going to give credit to stephen c anyways, since his prodding got me on the right track to finding the solution

The symptoms you describe are rather puzzling. I can't see the problem (the script looks correct), but I've got an idea on how to start tracking it down.
Get rid of the commented out lines from the script.
Add a #!/bin/sh line to the start of the script to make sure that you are actually using the right shell to execute it. (It is always a good idea to do this ... even if you think that you will get the right shell by default. That might change, depending on the platform.)
To figure out what the shell is doing, add set -vx as after the #!/bin/sh line.
The "-v" says echo each script line read, and the "-x" says echo the actual command lines that are executed. This will tell you exactly what commands are being run ... so that you can figure out what the command arguments really are.

Ruby Call to Maui or Bash

I am trying to run a java command line operation from Ruby through a series of system commands. The first command works fine, I never receive any success from the second (maui java command). Is there something that I am doing wrong?
Dir.chdir('/var/lib/maui') do
system 'cat %s/*.txt > %s/topics.txt' % [text_path, identifier_path]
system 'java maui.main.MauiTopicExtractor -l %s -m go_model' % [identifier_path]
end
I have also tried calling a bash script that processes the java commands:
output = %x[/var/lib/maui/process.sh #{identifier_path}]
However, the same result occurs-- the files are concatenated, but never reach the Maui.
When I capture the stdoutput from the maui command, I get the following (even though I have all the environment vars setup as needed and works when I run the command outside of ruby):
Exception in thread "main" java.lang.NoClassDefFoundError: maui/main/MauiTopicExtractor
Caused by: java.lang.ClassNotFoundException: maui.main.MauiTopicExtractor
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: maui.main.MauiTopicExtractor. Program will exit
Any help appreciated.

Based on your comment about class not found I think you need to invoke like this:
system 'java -cp /path/to/maui.jar maui.main.MauiTopicExtractor -l %s -m go_model' % [identifier_path]
if it works from the command line doing simply:
java maui.main.MauiTopicExtractor -l /path/to/file -m go_model
then can you post the output of (from the shell):
echo $CLASSPATH
and from ruby:
system "echo $CLASSPATH"