We Keep Coding

Apache Commons Exec Problems Handling Regex Argument?

I'm trying to use Apache Commons Exec to run a git command which uses a regex.
When I form my CommandLine and print it out it looks like this:
[git, --no-pager, grep, --line-number, --untracked, --extended-regexp, "^\s*public void\s+(testFindByAdAccount).*", --, *Test.java]
However when I execute this, git returns no results, resulting in an exit code 1.
When I run this command manually though, it returns plenty of results and succeeds. Changing the --extended-regexp argument to just a string like testFindByAdAccount does yield results when run via Exec, so I think Apache Commons is doing something to the regexp argument making it invalid. Any ideas what is going on?
EDIT: Adding a reproducible example
Clone https://github.com/ragurney/min-example
Run gradlew shadowJar to produce jar file for project
Run the app with java -jar app/build/libs/app-all.jar
Note the output which shows the command printed fails with an exit code 1 (because there are no results returned by the git command)
$ java -jar app/build/libs/app-all.jar
HELLOOOOOO
WD::: null
[git, --no-pager, grep, --line-number, --untracked, --extended-regexp, "^\s*public void\s+(testAppHasAGreeting)\(\).*", --, *Test.java]
WD::: /Users/rgurney/Src/personal/min-example
Exception in thread "main" java.lang.RuntimeException: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at min.example.App.lambda$runCommand$1(App.java:74)
at io.vavr.control.Try.getOrElseThrow(Try.java:748)
Running the command manually does produce expected results:
$ git --no-pager grep --line-number --untracked --extended-regexp "^\s*public void\s+(testAppHasAGreeting)\(\).*" -- "*Test.java"
app/src/test/java/min/example/AppTest.java:11: public void testAppHasAGreeting() {

I got a clue as to what's going on here when the sample you provided worked just fine on my Windows laptop but failed on my Linux desktop.
Once I made sure the git version wasn't the culprit (tested several versions between 2.17 and 2.39 on both machines), I figured the difference must be in the way different shells handle quoting. Specifically, the only argument here that has any potential quoting issues is the regex ("^\s*public void\s+(testFindByAdAccount).*"), which is added to the command line by commandLine.addArgument(regex);.
addArgument may look innocuous, but under the hood, it allows the CommandLine to handle the quoting itself (i.e., addArgument(String argument) calls addArgument(String argument, true). Since you've handled the quoting yourself, you should not allow the CommandLine to handle the quoting, and should explicitly call it with the second argument false. i.e.:
public static List<String> grep(String regex, String filePattern, String wd) {
CommandLine commandLine = CommandLine.parse("git");
commandLine.addArgument("--no-pager");
commandLine.addArgument("grep");
commandLine.addArgument("--line-number");
commandLine.addArgument("--untracked");
commandLine.addArgument("--extended-regexp");
commandLine.addArgument(regex, false);
// Here -----------------------^
commandLine.addArgument("--");
commandLine.addArgument(filePattern);
System.out.println(commandLine);
return List.of(runCommand(commandLine, wd).split("\n"));
}
This takes the quote-handling logic away and ensures the same code runs smoothly both on Windows and Linux (at least those I've tested).

Python 2.7 subprocess call method not working to run java command

I am trying to use a python script to manipulate the input files for my java program. The way I am doing it is I am generating the file name and passing it to subprocess.call() method to execute. Here is my program:
def execJava(self):
self.thisCmd="pause"
call(self.javaCmd,shell=True)
call(self.pauseCmd,shell=True)
Where,
self.javaCmd = 'java -ea -esa -Xfuture -Xss64m -classpath "C:\FVSDK_9_1_1\lib\x86_64\msc_12.0-sse2_crtdll\*" -Djava.library.path="C:\FVSDK_9_1_1\lib\x86_64\msc_12.0-sse2_crtdll;C:\FVSDK_9_1_1\lib\x86_64\share" com.cognitec.jfrsdk.examples.MatchFIRAgainstGallery C:\FVSDK_9_1_1\etc\frsdk.cfg 0 .\tmp\frsdk-scratch\probe_1.fir .\tmp\test\*'
Yes, it's a long complex java instruction but when I run it in the command prompt it works fine. Only when I pass it as a string it doesn't run and returns:
Exception in thread "main" java.lang.Error
After some exploration into the problem, I have discovered that it is due to \x, \t in the instruction, so it is executing
.\tmp\test\*
as
mp est\*
as it replaces \t with tab space while executing. I have looked up quite a bit and didn't find any solution. Any help is much appreciated.

Use forward slashes "/" instead of back slashes "\" for your paths.

Passing pre-escaped command-line arguments to ProcessBuilder

I bumped into this problem today when setting up a local set of communicating programs. Basically one of my applications is sending some data to another, and part of this data is a string containing a command to execute (like you would from the command-line). Let's say, for example:
g++ foo.cc bar.cc -o foobar
is the command sent by my first application. The second application, which receives the command (amongst other things), needs to execute this command after doing some other processing.
Now, at first I thought this would be trivial using a ProcessBuilder:
String exampleCommand = "g++ foo.cc bar.cc -o foobar";
ProcessBuilder builder = new ProcessBuilder(exampleCommand);
builder.start().waitFor();
However this is where the problem occurs.
CreateProcess error=2, The system cannot find the file specified
Okay, no worries I guess I can't just dump the whole thing into the builder. The first part of the command is usually a trivial string so I thought I could probably get away with a split around the first ' ' to separate the program name and arguments.
String exampleCommand = "g++ foo.cc bar.cc -o foobar";
String[] parts = exampleCommand.split(" ", 2);
ProcessBuilder builder = new ProcessBuilder(parts[0], parts[1]);
builder.start().waitFor();
And this brought me a little closer, the g++ file could now be found correctly, however after examining the stderr of g++ I found that the following error had occurred:
g++.exe: error: foo.cc bar.cc -o foobar: No such file or directory
At this point I realised that the ProcessBuilder class must be escaping all arguments passed to it in preparation for the command-line (hence the reason it usually takes arguments as an array of individual arguments rather than just a predefined argument string).
My question is, "Is there any way to pass a raw string of arguments to a ProcessBuilder and say THERE, execute EXACTLY this?"
Because the command comes from another application and is in no way static I can't just break the arguments down into an array beforehand and pass them to the ProcessBuilder constructor properly. The arguments are not so trivial that simply splitting the string around a ' ' will work properly either; arguments might contain spaces escaped with double quotes. For example:
g++ "..\my documents\foo.cpp" bar.cpp -o foobar
Could be a command coming from the application and splitting that string around ' ' and passing it to the ProcessBuilder will result in corrupt arguments.
If there is no proper way to do this can someone please point me to a standalone command line argument parser (in Java) that can turn a command-line string into a valid String[]?

Okay I feel rather foolish now but I achieved my desired result by simply reverting back to the good old Runtime.getRuntime().exec(...). I'll leave the question up in case anyone is as silly as me and find it useful.
String exampleCommand = "g++ foo.cc bar.cc -o foobar";
Runtime sys = Runtime.getRuntime();
sys.exec(exampleCommand);
Easy.

A comment to the Runtime.getRuntime().exec(...) solution:
The Runtime.getRuntime().exec(...) is not good anymore. In java executed on OSX El Capitan, 'Runtime.getRuntime().exec(...)' contains an error that sometimes closes the opened process when the java program exits. It works fine on previous OSX versions. However, ProcessBuilder works on all OSX versions.
(Haven't posted enough to have a enough rep points to make this as a normal comment.)

Getting IOException while getting free Space on the working drive

I am working on a application which first require to check the available free disk space before running any operation. We have set some Default required Space limit like 512MB, So if any working drive does not have more then 512mb space my program will prompt for less memory space available, please make sufficient space to run the program.
I am using following code for it.
long freeSpace = FileSystemUtils.freeSpaceKb() * 1024;
here I am coverting size into byte first to compare with our standard required size.
Due to the above statement i am gettign following exception:
Error-Command line returned OS error code '3' for command [cmd.exe, /C, dir /-c "F:\MyApp\"]Stacktrace java.io.IOException: Command line returned OS error code '3' for command [cmd.exe, /C, dir /-c "F:\MyApp"]
at org.apache.commons.io.FileSystemUtils.performCommand(FileSystemUtils.java:506)
at org.apache.commons.io.FileSystemUtils.freeSpaceWindows(FileSystemUtils.java:303)
at org.apache.commons.io.FileSystemUtils.freeSpaceOS(FileSystemUtils.java:270)
at org.apache.commons.io.FileSystemUtils.freeSpaceKb(FileSystemUtils.java:206)
at org.apache.commons.io.FileSystemUtils.freeSpaceKb(FileSystemUtils.java:240)
at org.apache.commons.io.FileSystemUtils.freeSpaceKb(FileSystemUtils.java:222)...
The OS returned Error Code is '3' thats mean it is not normal termination.
So now how can I resolve this issue ?
I also found alternative method available in java 1.6 - How to find how much disk space is left using Java?
new File("c:\\").getFreeSpace();
---------------------------------
**More Details :**
---------------------------------
OS Architecture : amd64
Temp Dir : c:\temp\
OS Name : Windows 7
OS Version : 6.1 amd64
Jre Version : 1.6.0_45-b06
User Home : C:\Users\Tej.Kiran
User Language : en
User Country: US
File Separator : \
Current Working Directory : F:\MyApp\

You can try executing that command from a prompt. Run cmd.exe and enter the following:
cmd.exe /C dir /-c "F:\MyApp\"
echo %errorlevel%
Error code 3 means the path doesn't exist, but in this case I wonder if it is related to permissions. Any non-zero errorlevel is a problem. If your Java app needs to know the free space on the drive it is installed on, you can do something like this:
// returns something like "file:/C:/MyApp/my/pkg/MyClass.class"
// -OR- "jar:file:/C:/MyApp/myjar.jar!/my/pkg/MyClass.class"
String myPath = my.pkg.MyClass.class.getResource(MyClass.class).toString();
int start = myPath.indexOf("file:/") + 6;
FileSystemUtils.freeSpaceKb(myPath.substring(start, myPath.indexOf("/", start));
Obviously this code wouldn't work in an applet, but that shouldn't be surprising. The substring logic should also be more robust, but this is just a simple example.

How would I run an .sh file using NSTask and get its output?

I need to run an .sh file and get its output. I need to see the setup of the file as well.
The .sh file simply runs a java app through terminal.
Any ideas? I'm truly stuck on this.....
Elijah
The server.sh file:
echo Starting Jarvis Program D.
ALICE_HOME=.
SERVLET_LIB=lib/servlet.jar
ALICE_LIB=lib/aliceserver.jar
JS_LIB=lib/js.jar
# Set SQL_LIB to the location of your database driver.
SQL_LIB=lib/mysql_comp.jar
# These are for Jetty; you will want to change these if you are using a different http server.
HTTP_SERVER_LIBS=lib/org.mortbay.jetty.jar
PROGRAMD_CLASSPATH=$SERVLET_LIB:$ALICE_LIB:$JS_LIB:$SQL_LIB:$HTTP_SERVER_LIBS
java -classpath $PROGRAMD_CLASSPATH -Xms64m -Xmx128m org.alicebot.server.net.AliceServer $1
My current code:
NSTask *server = [NSTask new];
[server setLaunchPath:#"/bin/sh"];
[server setArguments:[NSArray arrayWithObject:#"/applications/jarvis/brain/server.sh"]];
NSPipe *outputPipe = [NSPipe pipe];
[server setStandardInput:[NSPipe pipe]];
[server setStandardOutput:outputPipe];
[server launch];
NSMutableString *outputString = [NSMutableString string];
while ([outputString rangeOfString:#"Jarvis>"].location == NSNotFound) {
[outputString appendString:[[[NSString alloc] initWithData:[[outputPipe fileHandleForReading] readDataToEndOfFile] encoding:NSUTF8StringEncoding] autorelease]];
NSRunAlertPanel(#"", outputString, #"", #"", #"");
}
The NSRunAlertPanel is just for checking the output. Now my code is freezing and not even getting to the alertpanel.

See answer to this question.
There are a couple of things that should be fixed in your script:
The script should begin with a
shebang. Also make sure that the
script has its executable bit set.
Because the environment variables are set up relative to the shell script directory, you need to make sure that the script directory is the current directory.
You need to export the environment variables that should be visible to the Java process.
In the last line you can use exec to replace the shell process with the Java executable that runs Jetty.
Here is a revised version of your script:
#!/bin/sh
echo Starting Jarvis Program D.
cd "`dirname \"$0\"`"
export ALICE_HOME=.
export SERVLET_LIB=lib/servlet.jar
export ALICE_LIB=lib/aliceserver.jar
export JS_LIB=lib/js.jar
# Set SQL_LIB to the location of your database driver.
export SQL_LIB=lib/mysql_comp.jar
# These are for Jetty; you will want to change these if you are using a different http server.
export HTTP_SERVER_LIBS=lib/org.mortbay.jetty.jar
export PROGRAMD_CLASSPATH=$SERVLET_LIB:$ALICE_LIB:$JS_LIB:$SQL_LIB:$HTTP_SERVER_LIBS
exec java -classpath $PROGRAMD_CLASSPATH -Xms64m -Xmx128m org.alicebot.server.net.AliceServer $1
Invoking the shell script in Objective-C with multiple arguments:
NSTask *server = [NSTask new];
[server setLaunchPath:#"/bin/sh"];
[server setArguments:[NSArray arrayWithObjects:#"/applications/jarvis/brain/server.sh", #"argument", nil]];
...

Using AMShellWrapperTest.app you can filter (save, ...) the stdout stream of server.sh by modifying "- (void)appendOutput:(NSString *)output" in BannerController.m. (... but maybe there is a better way to do this ...)
/*
// output from stdout
- modified AMShellWrapper/AMShellWrapperTest/BannerController.m (http://www.harmless.de/cocoa-code.php)
to print server.sh setup information to "Error Messages:" text output field (or Console.app as an
alternative) and the Q & A dialog to the "Output:" text field
- use of default charliebot, http://sourceforge.net/projects/charliebot/, modified only to run server.sh
with complete path (here: ~/Desktop/charliebot/server.sh) in AMShellWrapperTest.app
*/
- (void)appendOutput:(NSString *)output
{
NSMutableString *outputString = [NSMutableString string];
if (
([output rangeOfString:#"Charlie>"].location != NSNotFound ) || \
([output rangeOfString:#"[Charlie] user>"].location != NSNotFound )
) {
[self write: output];
[self write: #"\n"];
} else {
[outputString appendString: output];
//[outputString writeToFile:#"/dev/console" atomically: NO]; // alternative
[errorOutlet setString:[[errorOutlet string] stringByAppendingString: outputString]];
}
}

yes, but why isn't my code (posted above) not working?
I guess your "Jarvis>" line is the first line of the server.sh ouput stream that expects some user input, which means that this line is incomplete without a terminating newline character "\n". If server.sh had been run in Terminal.app, the user would have to press the return key to let the dialog continue. The conditional code of the while loop (NSNotFound) cannot finish its job on this incomplete line (which would be to abort the while loop) and gets stuck.
You have to drop the while loop and use the 'readInBackgroundAndNotify' mode on NSFileHandle to get non-blocking I/O stdout stream behaviour!
See: NSTask/NSPipe STDIN hangs on large data, sometimes...
So, if you like, just transform the source code of AMShellWrapperTest.app into a pure command-line tool by removing the GUI code.