We Keep Coding

Apache Commons Exec Problems Handling Regex Argument?

I'm trying to use Apache Commons Exec to run a git command which uses a regex.
When I form my CommandLine and print it out it looks like this:
[git, --no-pager, grep, --line-number, --untracked, --extended-regexp, "^\s*public void\s+(testFindByAdAccount).*", --, *Test.java]
However when I execute this, git returns no results, resulting in an exit code 1.
When I run this command manually though, it returns plenty of results and succeeds. Changing the --extended-regexp argument to just a string like testFindByAdAccount does yield results when run via Exec, so I think Apache Commons is doing something to the regexp argument making it invalid. Any ideas what is going on?
EDIT: Adding a reproducible example
Clone https://github.com/ragurney/min-example
Run gradlew shadowJar to produce jar file for project
Run the app with java -jar app/build/libs/app-all.jar
Note the output which shows the command printed fails with an exit code 1 (because there are no results returned by the git command)
$ java -jar app/build/libs/app-all.jar
HELLOOOOOO
WD::: null
[git, --no-pager, grep, --line-number, --untracked, --extended-regexp, "^\s*public void\s+(testAppHasAGreeting)\(\).*", --, *Test.java]
WD::: /Users/rgurney/Src/personal/min-example
Exception in thread "main" java.lang.RuntimeException: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at min.example.App.lambda$runCommand$1(App.java:74)
at io.vavr.control.Try.getOrElseThrow(Try.java:748)
Running the command manually does produce expected results:
$ git --no-pager grep --line-number --untracked --extended-regexp "^\s*public void\s+(testAppHasAGreeting)\(\).*" -- "*Test.java"
app/src/test/java/min/example/AppTest.java:11: public void testAppHasAGreeting() {

I got a clue as to what's going on here when the sample you provided worked just fine on my Windows laptop but failed on my Linux desktop.
Once I made sure the git version wasn't the culprit (tested several versions between 2.17 and 2.39 on both machines), I figured the difference must be in the way different shells handle quoting. Specifically, the only argument here that has any potential quoting issues is the regex ("^\s*public void\s+(testFindByAdAccount).*"), which is added to the command line by commandLine.addArgument(regex);.
addArgument may look innocuous, but under the hood, it allows the CommandLine to handle the quoting itself (i.e., addArgument(String argument) calls addArgument(String argument, true). Since you've handled the quoting yourself, you should not allow the CommandLine to handle the quoting, and should explicitly call it with the second argument false. i.e.:
public static List<String> grep(String regex, String filePattern, String wd) {
CommandLine commandLine = CommandLine.parse("git");
commandLine.addArgument("--no-pager");
commandLine.addArgument("grep");
commandLine.addArgument("--line-number");
commandLine.addArgument("--untracked");
commandLine.addArgument("--extended-regexp");
commandLine.addArgument(regex, false);
// Here -----------------------^
commandLine.addArgument("--");
commandLine.addArgument(filePattern);
System.out.println(commandLine);
return List.of(runCommand(commandLine, wd).split("\n"));
}
This takes the quote-handling logic away and ensures the same code runs smoothly both on Windows and Linux (at least those I've tested).

Java/JavaFX ProcessHandle possibly not finding all processes (Linux/Debian)

I've a JavaFX application where I've a list of a bunch of script files. Once the application loads, it reads it and and checks which ones are running.
To do that I use a ProcessHandle, as mentioned in various examples here on StackOverflow and other guides/tutorials on the internet.
The problem is, it never finds any of them. There for I programmatically started one, which I know for a fact that it will be running, via Process process = new ProcessBuilder("/path/to/file/my_script.sh").start(); - and it won't find this one either.
Contents of my_script.sh:
#!/bin/bash
echo "Wait for 5 seconds"
sleep 5
echo "Completed"
Java code:
// List of PIDs which correspond to the processes shown after "INFO COMMAND:"
System.out.println("ALL PROCESSES: " + ProcessHandle.allProcesses().toList());
Optional<ProcessHandle> scriptProcessHandle = ProcessHandle.allProcesses().filter(processHandle -> {
System.out.println("INFO COMMAND: " + processHandle.info().command());
Optional<String> processOptional = processHandle.info().command();
return processOptional.isPresent() && processOptional.get().equals("my_script.sh");
}).findFirst();
System.out.println("Script process handle is present: " + scriptProcessHandle.isPresent());
if (scriptProcessHandle.isPresent()) { // Always false
// Do stuff
}
Thanks to the good old fashioned System.out.println(), I noticed that I get this in my output console every time:
ALL PROCESSES: [1, 2, 28, 85, 128, 6944, 21174, 29029, 29071]
INFO COMMAND: Optional[/usr/bin/bwrap]
INFO COMMAND: Optional[/usr/bin/bash]
INFO COMMAND: Optional[/app/idea-IC/jbr/bin/java]
INFO COMMAND: Optional[/app/idea-IC/bin/fsnotifier]
INFO COMMAND: Optional[/home/username/.jdks/openjdk-17.0.2/bin/java]
INFO COMMAND: Optional[/usr/bin/bash]
INFO COMMAND: Optional[/home/username/.jdks/openjdk-17.0.2/bin/java]
INFO COMMAND: Optional[/home/username/.jdks/openjdk-17.0.2/bin/java]
INFO COMMAND: Optional[/usr/bin/bash]
Script process handle is present: false
The first line in the Javadoc of ProcessHandle.allProcess() reads:
Returns a snapshot of all processes visible to the current process.
So how come I can't see the rest of the operating system's processes?
I'm looking for a non-os-dependent solution, if possible. Why? For better portability and hopefully less maintenance in the future.
Notes:
A popular solution for GNU/Linux seems to be to check the proc entries, but I don't know if that would work for at least the majority of the most popular distributions - if it doesn't, adding support for them in a different way, would create more testing and maintenance workload.
I'm aware of ps, windir, tasklist.exe possible solutions (worst comes to worst).
I found the JavaSysMon library but it seems dead and unfortunately:
CPU speed on Linux only reports correct values for Intel CPUs
Edit 1:
I'm on Pop_OS! and installed IntelliJ via the PopShop as flatpak.
In order to start it as root as suggested by mr mcwolf, I went to /home/username/.local/share/flatpak/app/com.jetbrains.IntelliJ-IDEA-Community/x86_64/stable/active/export/bin and found com.jetbrains.IntelliJ-IDEA-Community file.
When I run sudo ./com.jetbrains.IntelliJ-IDEA-Community or sudo /usr/bin/flatpak run --branch=stable --arch=x86_64 com.jetbrains.IntelliJ-IDEA-Community in my terminal, I get error: app/com.jetbrains.IntelliJ-IDEA-Community/x86_64/stable not installed
So I opened the file and ran its contents:
exec /usr/bin/flatpak run --branch=stable --arch=x86_64 com.jetbrains.IntelliJ-IDEA-Community "$#"
This opens IntelliJ, but not as root, so instead I ran:
exec sudo /usr/bin/flatpak run --branch=stable --arch=x86_64 com.jetbrains.IntelliJ-IDEA-Community "$#"
Which prompts for a password and when I write it in, the terminal crashes.
Edit 1.1:
(╯°□°）╯︵ ┻━┻ "flatpak run" is not intended to be ran with sudo
Edit 2:
As mr mcwolf said, I downloaded the IntelliJ from the official website, extracted it and ran the idea.sh as root.
Now a lot more processes are shown. 1/3 of them show up as INFO COMMAND: Optional.empty.
scriptProcessHandle.isPresent() is still unfortunately returning false. I searched through them and my_script.sh is nowhere to be found. I also tried processOptional.isPresent() && processOptional.get().equals("/absolute/path/to/my_script.sh") but I still get false on isPresent() and it's not in the list of shown processes.
Though the last sentence might be a different problem. I'll do more digging.
Edit 3:
Combining .commandLine() and .contains() (instead of .equals()) solves the problem mentioned in "Edit 2".
Optional<ProcessHandle> scriptProcessHandle = ProcessHandle.allProcesses().filter(processHandle -> {
System.out.println("INFO COMMAND LINE: " + processHandle.info().commandLine());
Optional<String> processOptional = processHandle.info().commandLine();
return processOptional.isPresent() && processOptional.get().contains("/absolute/path/to/my_script.sh");
}).findFirst();
System.out.println("Script process handle is present: " + scriptProcessHandle.isPresent());
if (scriptProcessHandle.isPresent()) { // Returns true
// Do stuff
}
.commandLine() also shows script arguments, so that must be kept in mind.

Jenkinsfile: pass env from another script

Consider a folder with two jenkins files, JenkinsfileA and JenkinsfileB.
# JenkinsfileA
node {
environment {
APP_NAME = 'APPLICATION_A'
# Load the other jenkins file
load `JenkinsfileB'
}
#JenkinsfileB
node{
echo "Got parameter $APP_NAME"
stage ('do something'){
echo 'doing something'
}
}
In B, the $APP_NAME param is coming up null. What is the canonical way to pass APP_NAME parameter from A into B? Should I be using load here or something else? Are env vars. preferred or can we make use of parameters somehow?
Update
I think the reason env isn't working for me is because jenkins is triggering second job in a completely separate workspace. I'd like to do something like this:
load 'JenkinsfileB', parameters: [[$class: 'StringParameterValue', name: 'MYPARAM', value: "some-value" ]]
But keep getting error
java.lang.IllegalArgumentException: Expected named arguments but got [{parameters=[{$class=StringParameterValue, name=MYPARAM, value=some-value}]}, JenkinsfileB]
And when I try to use build job syntax instead of load the build can't even find JenkinsfileB, so I am stumped.
Unreleated: is node still the correct directive to use to ensure builds run on any agent, or should I be using pipeline{ agent: any}? Got node from this article

Is there any way to automatically setting windows path in a string in groovy?

My project root directory is:
D:/Project/Node_Project
I am using a gradle plugin to install nodejs temporarily in my project root directory so that some nodejs command can run in the project while the thoject builds. The plugin is as below:
plugins {
id "com.github.node-gradle.node" version "2.2.4"
}
node {
download = true
version = "10.10.0"
distBaseUrl = 'https://nodejs.org/dist'
workDir = file("${project.buildDir}/nodejs")
}
So, nodejs is getting installed inside the project in the location:
D:/Project/Node_Project/build/nodejs/node-v10.10.0-win-x64
Now, I am using a .execute(String[] "path to set at environment variable", String path of file to be executed which is in the project root directory) method to run a windows command with node dependency. Code below:
cmd = "node connect.js"
def process = cmd.execute(["PATH=${project.projectDir}/build/nodejs/node-v10.10.0-win-x64"],null)
In the above .execute method, is there a way to auto-populate the "build/nodejs/node-v10.10.0-win-x64" part of the string instead of hardcoding it into the method?
Something like:
def process = cmd.execute(["PATH=${project.projectDir}/.*"],null)
Syntax of .execute method:
https://docs.groovy-lang.org/latest/html/groovy-jdk/java/lang/String.html#execute(java.lang.String[],%20java.io.File)
All the codes are inside "build.gradle" file. Please help!

I asked why you don't just write a task of type NodeTask, but I understand that you like to run a it in the background, which you can't do with that.
You could list the content of a directory and use that as part of the command. But you could also just grab it from the extension provided by the plugin.
This is not documented and it might break in future releases of the plugin, but you can do something like this (Groovy DSL):
task connectJS {
dependsOn nodeSetup
doFirst {
def connectProcess = "$node.variant.nodeExec $projectDir/src/js/connect.js".execute()
// Blocking readers (if async, pipe to a log file instead)
connectProcess.in.eachLine { logger.info(it) }
connectProcess.err.eachLine { logger.err(it) }
}
}

Running a Java program from Adobe AIR's native process

I've use Adobe native process to run java program from my air app. Here the code and it works fine. But i should write absolute path to java runtime for that:
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java.
If user installed java runtime in diff folder, or have diff version then this code would not work. How i can detect where java were installed or maybe there is another right way to run java applications from air applications? If i run java library from terminal command line then i could just write "java -jar pdfbox-app-1.6.0.jar" etc. and it runs fine.
private function convertPdf2Txt():void{
var arg:Vector.<String> = new Vector.<String>;
arg.push("-jar");
arg.push(File.applicationDirectory.resolvePath("pdfbox-app-1.6.0.jar").nativePath);
arg.push("ExtractText");
arg.push("-force");
arg.push(File.applicationStorageDirectory.resolvePath("Data/1.pdf").nativePath);
arg.push(File.applicationStorageDirectory.resolvePath("Data/1.txt").nativePath);
var fjava:File = new File("/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java");
if (fjava.exists==false){
Alert.show("Can't find Java Runtime in default folder.","Idea Rover",mx.controls.Alert.OK, null,null,imgInfo);
return;
}
var npInfo:NativeProcessStartupInfo;
npInfo = new NativeProcessStartupInfo();
npInfo.executable = fjava;
npInfo.arguments = arg;
var nativeProcess:NativeProcess;
nativeProcess = new NativeProcess();
nativeProcess.addEventListener(NativeProcessExitEvent.EXIT,onNativeProcessExit);
nativeProcess.start(npInfo);
}

Absolute path is:
Mac OS: /usr/bin/java
Win OS: (default)
64bit : C:\Program Files\Java
32bit : C:\Program Files (x86)\Java

rather than popping up an Alert, you could open a file selection dialog, using File.browseForOpen(). then, the File you want is contained in the event passed by the Event.SELECT handler. this flow seems standard for applications i've used that need to access other applications, but aren't sure where to find their executables.
var npInfo:NativeProcessStartupInfo = new NativeProcessStartupInfo();
// setup npInfo, nativeProcess...
var fjava:File = new File("/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java");
if (!fjava.exists) {
fjava.addEventListener(Event.SELECT, onFileSelected);
fjava.browseForOpen("Where is Java located?");
}
private function onFileSelected (evt:Event) :void {
npInfo.executable = evt.target;
nativeProcess.start(npInfo);
fjava.removeEventListener(Event.SELECT, onFileSelected);
}
of course, you can use the same logic to find the file java needs to launch as well.

You may be able to determine where the Java binaries are by looking at the JAVA_HOME environment variable. I'd like to do the same thing as you're doing, so I'll post more after I do more research.