Steps using runtime api
echo %PATH% (output will be something like "c:\windows\system32;d:\test")
execute ping or any system command, the output will be success
delete one value for the path like d:\test
echo %PATH% (output will be "%system32%\system32;")
Now if I execute the same command executed in step 2 like ping, then I get "command not found."
Plesae note:- all steps executed in same java process
Anybody suggest whats going wrong in this process
Looks like variables are not being expanded the second time you show the PATH (step 4). Probably you are corrupting the path when you delete one value in step 3.
Show us a code snippet demonstrating the problem.
Are you using ProcessBuilder for this? That allows simple access to the environment variables passed to sub-processes. It sounds like you are trying to modify the global environment, which is likely not what you want to do.
Here's an example:
ProcessBuilder pb = new ProcessBuilder();
Map<String, String> env = pb.environment();
System.out.println("Current environment: " + env.toString());
String path = env.get("PATH");
path = path.substring(0, path.indexOf("d:\\test")) + path.substring(path.indexOf("d:\\test") + "d:\\test".length());
env.put("PATH", path);
pb.command("ping");
Process p = pb.start();
// ...
Because it sounds like you are on Windows, you'll need to be a bit more careful about finding the path variable in the environment, since the Windows environment is case-insensitive, but Map.get() is case-sensitive. Probably best to loop through the keys looking for a equalsIgnoreCase("PATH").
Also, you may want to clean up the path before putting it back into the map (make sure it doesn't contain extra System.getProperty("path.separator")s.
Your question is not clear to me but i do see some problem :
%System32% is usually c:\windows\system32.
In your example,assuming the environment is set of %system32% correctly, the step(4) "%system32%\system32;" will resolve to c:\windows\system32\system32
Which may not what you want.
Related
Below is basically an MCVE of my full problem, which is much messier. What you need to know is that the following line runs when directly put in terminal:
java -classpath /path/to/weka.jar weka.filters.MultiFilter \
-F "weka.filters.unsupervised.attribute.ClusterMembership -I first" \
-i /path/to/in.arff
This is relatively straightforward. Basically, all I am doing is trying to cluster the data from in.arff using all of the default settings for the ClusterMembership filter, but I want to ignore the first attribute. I have the MultiFilter there because in my actual project, there are other filters, so I need this to stay. Like previously mentioned, this works fine. However, when I try to run the same line with ProcessBuilder, I get a "quote parse error", and it seems like the whole structure of nesting quotes breaks down. One way of demonstrating this is trying to get the following to work:
List<String> args = new ArrayList<String>();
args.add("java");
args.add("-cp");
args.add("/path/to/weka.jar");
args.add("weka.filters.MultiFilter");
args.add("-F");
args.add("\"weka.filters.unsupervised.attribute.ClusterMembership");
args.add("-I");
args.add("first\"");
args.add("-i");
args.add("/path/to/in.arff");
ProcessBuilder pb = new ProcessBuiler(args);
// ... Run the process below
At first glance, you might think this is identical to the above line (that's certainly what my naive self thought). In fact, if I just print args out with spaces in between each one, the resulting strings are identical and run perfectly if directly copy and pasted to the terminal. However, for whatever reason, the program won't work as I got the message (from Weka) Quote parse error. I tried googling and found this question about how ProcessBuilder adds extra quotes to the command line (this led me to try numerous combinations of escape sequences, all of which did not work), and read this article about how ProcessBuilder/Runtime.exec() work (I tried both ProcessBuilder and Runtime.exec(), and ultimately the same problem persisted), but couldn't find anything relevant to what I needed. Weka already had bad documentation, and then their Wikispace page went down a couple weeks ago due to Wikispaces shutting down, so I have found very little info on the Weka side.
My question then is this: Is there a way to get something like the second example I put above to run such that I can group arguments together for much larger commands? I understand it may require some funky escape sequences (or maybe not?), or perhaps something else I have not considered. Any help here is much appreciated.
Edit: I updated the question to hopefully give more insight into what my problem is.
You don't need to group arguments together. It doesn't even work, as you've already noted. Take a look what happens when I call my Java programm like this:
java -jar Test.jar -i -s "-t 500"
This is my "program":
public class Test {
public static void main(String[] args) {
for( String arg : args ) {
System.out.println(arg);
}
}
}
And this is the output:
-i
-s
-t 500
The quotes are not included in the arguments, they are used to group the arguments. So when you pass the arguments to the ProcessBuilder like you did, it is essentially like you'd written them with quotes on the command line and they are treated as a single argument, which confuses the parser.
The quotes are only necessary when you have nested components, e.g. FilteredClassifier. Maybe my answer on another Weka question can help you with those nested components. (I recently changed the links to their wiki to point to the Google cache until they established a new wiki.)
Since you didn't specify what case exactly caused you to think about grouping, you could try to get a working command line for Weka and then use that one as input for a program like mine. You can then see how you would need to pass them to a ProcessBuilder.
For your example I'd guess the following will work:
List<String> args = new ArrayList<String>();
args.add("java");
args.add("-cp");
args.add("/path/to/weka.jar");
args.add("weka.filters.MultiFilter");
args.add("-F");
args.add("weka.filters.unsupervised.attribute.ClusterMembership -I first");
args.add("-i");
args.add("/path/to/in.arff");
ProcessBuilder pb = new ProcessBuiler(args);
Additional details
What happens inside Weka is basically the following: The options from the arguments are first processed by weka.filters.Filter, then all non-general filter options are processed by weka.filters.MultiFilter, which contains the following code in setOptions(...):
filters = new Vector<Filter>();
while ((tmpStr = Utils.getOption("F", options)).length() != 0) {
options2 = Utils.splitOptions(tmpStr);
filter = options2[0];
options2[0] = "";
filters.add((Filter) Utils.forName(Filter.class, filter, options2));
}
Here, tmpStr is the value for the -F option and will be processed by Utils.splitOption(tmpStr) (source code). There, all the quoting and unquoting magic happens, so that the next component will receive an options array that looks just like it would look if it was a first-level component.
I am trying to create a GUI using java swing. From there I have to run linux system commands. I tried using exec(). But the exec() function is unable to parse the string if it contains single quotes. The code which I have used is as follows-
Process p = Runtime.getRuntime().exec("cpabe-enc pub_key message.txt '( it_department or ( marketing and manager ) )'")
BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
But I am getting error when I run the program as--syntax error at "'(".
The same command runs when I write
Process p = Runtime.getRuntime().exec("cpabe-enc pub_key message.txt default")
Please help. Thanks in advance for your help.
Split up the parameters into an array instead, one string for each argument, and use the exec-method that takes as String[] instead, that generally works better for arguments.
Somethign along the lines of:
Runtime.getRuntime().exec(new String[] {"cpabe-enc", "pub_key", "message.txt", "( it_department or ( marketing and manager ) )"});
or whatever what your exact parameters are.
Its because the runtime does not interpret the '(...)' as a single parameter like you intend.
Try using ProcessBuilder instead:
http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/ProcessBuilder.html
I recently got this kind of problem solved. I was using javaFX to call shell scripts on button click .. which is very much similar to your swing application scenario...
Here are the links hope it might help you...
How to code in java to run unix shell script which use rSync internally in windows environment using cygwin?
Getting error in calling shell script in windows environment using java code and cygwin...!
Happy coding... :)
In particular, I'd like to use the (unfortunately not visible) sun.nio.fs.Globs.toUnixRegexPattern(String glob).
Ok, stepping back and giving a bit of context
I have an iterator of pathes into a remote, unix-like file system (think ssh unixhost find path -type f). I also have a user-supplied glob pattern which I now want to match each path against.
On a unix machine, the following works just fine:
matcher = FileSystems.getDefault().getPathMatcher("glob:" + glob);
// ...
for (String s : remoteFind(...)) {
if (matcher.matches(Paths.get(s))) {
// matches, do something
}
}
But when this is run on Windows, the same program totally fails because the FileSystems.getDefault() returns a Windows filesystem (the horror, the horror) and '\' is used as separator, etc. You get the picture. Nothing matches.
Of course I can stop all this nonsense and just rewrite (or rather, copy) sun.nio.fs.Globs.toUnixRegexPattern(String glob), but is there another, more elegant way?
Ok, so just to close this question, as stated in the comments I ended up writing a method in my FileUtil that is almost verbatim a copy of sun.nio.fs.Globs.toUnixRegexPattern(String glob). Works great.
If somebody finds a better way please add a different answer here.
If you do not make any file system operations locally, you could try to set
-Dfile.separator=/
system variable to mimic the unix path separator. This variable should be passed to JVM on startup
As sun.nio.fs.UnixFileSystem is not even part of my Windows JDK, I went one step back and looked for FileSystemProviders that are available on all platforms. So I found JrtFileSystemProvider, which can be (mis-)used to get a Unix-like path matcher on Windows (the following is copy & paste from some Kotlin code, but you get the idea):
val jrtFileSystem = FileSystems.getFileSystem(URI("jrt:/"))
// ...
val pattern = "..."
val matcher = jrtFileSystem.getPathMatcher("glob:$pattern")
// ...
matcher.matches(jrtFileSystem.getPath("path/to/match"))
I've got a HDFS structure something like
a/b/file1.gz
a/b/file2.gz
a/c/file3.gz
a/c/file4.gz
I'm using the classic pattern of
FileInputFormat.addInputPaths(conf, args[0]);
to set my input path for a java map reduce job.
This works fine if I specify args[0] as a/b but it fails if I specify just a (my intention being to process all 4 files)
the error being
Exception in thread "main" java.io.IOException: Not a file: hdfs://host:9000/user/hadoop/a
How do I recursively add everything under a ?
I must be missing something simple...
As Eitan Illuz mentioned here, in Hadoop 2.4.0 a mapreduce.input.fileinputformat.input.dir.recursive configuration property was introduced that when set to true instructs the input format to include files recursively.
In Java code it looks like this:
Configuration conf = new Configuration();
conf.setBoolean("mapreduce.input.fileinputformat.input.dir.recursive", true);
Job job = Job.getInstance(conf);
// etc.
I've been using this new property and find that it works well.
EDIT: Better yet, use this new method on FileInputFormat that achieves the same result:
Job job = Job.getInstance();
FileInputFormat.setInputDirRecursive(job, true);
This is a bug in the current version of Hadoop. Here is the JIRA for the same. It's still in open state. Either make the changes in the code and build the binaries or wait for it to be fixed in the coming releases. Processing of the files recursively can be turned on/off, check the patch attached to the JIRA for more details.
I must be missing something here, but how do I call something like "cd /root/some/dir/" with Ganymed SSH API?
I created a Connection object
In the first session created, I called "cd /root/some/dir"
In the second session created, I called "ls ." or "./myApp"
That didnt work, because ganymed probably starts each session with its own directory
So do I need to perform both commands on the same session? something like:
session.getStdin().write("cd /root/somedir \n".getBytes());
session.getStdin().write("ls . ".getBytes());
Is that the correct way?? if so, why do we need Session.execCommand?
After doing some research, the only good solution I managed to find is calling the "cd" command within the same code as the "ls" command, like this
session.execCommand("cd /root/somedir ; ls .");
The semicolon will separate the two commands as in any bash code.
In this way, you can query the session's result [session.getExitStatus()] of both the cd and ls commands, which is much better then writing the two commands to session.getStdIn() (after writing to stdin, you kinda loose all the ability to check for exit status...)
Hope this will help the rest
Eyal
According to the Ganymed FAQ (http://www.ganymed.ethz.ch/ssh2/FAQ.html), you are not allowed to send more than one command per Session object you generate. This is how SSH-2 apparently wants you to handle it. Your two options are to either combine the two commands like
session.execCommand("cd /root/somedir ; ls .");
However this wont always work and it get very ugly if you have more than a couple commands. The other way to do this is to open an interactive shell session and write the commands to standard in. This could look something like this:
Session sess = conn.openSession();
sess.requestDumbPTY();
sess.startShell();
OutputStream os = sess.getStdin();
os.write("cd /root/somedir\n".getBytes());
os.write("ls -1\n".getBytes());
os.write("exit\n".getBytes());
InputStream stdout = new StreamGobbler(sess.getStdout());
BufferedReader br = new BufferedReader(new InputStreamReader(stdout));
//TODO
Note the use of the final exit command. Since this is being treated like a terminal window, if you do not exit from the program any loop you have reading the output of the server will never terminate because the server will be expecting more input
OK, I took a quick look on the Ganymed javadoc and although I did not try it myself I assume that you should use method execCommand() of session instead of writing into the STDIN. I am pretty sure that session is connected to remote shell and therefore handles the shell state including current directory, environment variables etc.
So, just do the following:
session.execCommand("cd /root/somedir \n".getBytes());
session.execCommand("ls . ".getBytes());
I hope this will work for you. Good luck.