I need to use Java to rsync several files using one command
the following command works fine in shell
rsync -avrz --timeout=100 rsync://10.149.21.211:8730/'logflow/click/file1 logflow/click/file2' /home/kerrycai/puller"
but when i use the following Java code , it does not work
String cmd = "rsync -avrz --timeout=100 rsync://10.149.21.211:8730/'logflow/click/file1 logflow/click/file2' /home/kerrycai/puller";
Process p = Runtime.getRuntime().exec(cmd);
int ret = p.waitFor();
the ret value is not equal to 0 (5 in my test), and the command is not executed succeed, after some debugging , it seem the problem is caused by the single quote
So, my questions is
Can I using java to execute a shell command which has single quote in it (Pls note, the single quote is in the middle of a parameter, not start/end) ?
Can I have a shell command to rsync several files in one command , and the command does not have single(double) quotes in it ?
Note to #Chris: this combination of multiple filenames (really modified-partly-like-filenames) in one argument is indeed very unusual and even 'suspicious' for Unix in general, but is (or at least was) correct for rsync in particular.
Preface: Java Runtime.exec does NOT 'execute a shell command' (unless you explicitly run a shell and give it a command); it runs a program, with arguments. These different things are often confused because most of the shell commands used by normal users are commands to run programs, but this is one case where the difference matters. In particular quoting a space to shell causes the shell to pass a single argument to the program containing the space instead of splitting into two (or more) arguments, but the quote itself is NOT included in the argument.
First you should look at the man page (on your system or online at https://download.samba.org/pub/rsync/rsync.html) under ADVANCED USAGE. Current (and IME even moderately old) versions of rsync have a more convenient syntax with separate arguments to get multiple files, which the simple parsing used by Runtime.exec(String) can handle like this:
rsync -avrz --timeout=100 --port=8730 10.149.21.211::logflow/click/file1 ::logflow/click/file2 /home/kerrycai/puller
But if you need (or really want) to use the quoted-space form then you need to do the tokenization yourself and use the String[] overload as suggested by #EJP -- although you can still use Runtime, you don't need ProcessBuilder for this. Specifically do something along the lines of:
String[] cmdarray = {"rsync",
"-avrz",
"--timeout=100",
"rsync://10.149.21.211:8730/logflow/click/file1 logfile/click/file2",
// separated argument contains space but not single (or other) quote
"/home/kerrycai/puller" };
... Runtime.getRuntime.exec(cmdarray); ...
You're calling a somewhat large command from Java. Why not just use a shell script? Put your gnarly command in myScript.sh and then have Java invoke /bin/bash myScript.sh. Makes all the weirdness to do with string handling in Java go away.
Related
I am running the following command to generate the hash for a given password on the Linux terminal.
openssl passwd -apr1 'Test123$Pwd'
$apr1$lWLP.Qc0$f/YdAVqsYFglu1EnLvzUS.
I have written a simple shell script that generates a hash for the given password. Here is my shell script: (Should I modify anything here in shell script to include single quotes surrounding $1)
echo -n "$1" | openssl passwd -apr1 $1
Now, the problem is, when I call the same shell script from java program, for some reason if the password contains a dollar($) character, it is not working as expected.
Here is my java code which calls the above shell script:
void test(String password){
String script = "sh test.sh %s";
String command = String.format(script, password);
- - -
execute(command); // execute using ProcessBuilder
- - -
}
The above code works fine if the password doesn't contain a dollar character. But, if the password contains $ char, it ignores the text after the $ symbol and generates a hash for the remaining password only up to $.
For example: If I pass the password as Test$Pwd
The hash is getting generated for only Test instead of Test$Pwd
I tried to set a single quote in java code as below but this also didn't work. When executed using java for some reason it still generates the hash for Test123 instead of Test123$Pwd
Modified java code to include a single quote as below:
"sh test.sh '%s'";
Observed the followed command in logs which has single quotes, and this also didn't work when I ran shell script from java.
openssl passwd -apr1 'Test123$Pwd'
From the Linux terminal when I run the script it works fine.
sh test.sh 'Test123$Pwd' -- this works
sh test.sh Test123$Pwd -- this doesn't work (generates hash only for Test123 even when run directly on the linux terminal)
I am confused about whether I need to modify the shell script or something in java?
Any pointers would be helpful. Thanks!
In shell scripts, double quotes is a way to get around spaces being seen as delimiters. It doesn't exclude variable substitution. So that's what you want here; single quotes would pass the literal $1 as in, the dollar, then the 1, and not 'substitute the first argument of the script here'. However, you pass $1 straight up without quotes for the second.
But, this is putting rather a lot of dependency on the exact shell in use and opens up doors to passwords with certain symbols messing up your program.
Why not forego the shell script entirely and call openssl directly? You can replace the standard in (and thus, send what echo is sending in your script), and the args passed via ProcessBuilder's List<String> / String... options (never use the single string option, really, it's hard to read and is asking java to attempt to turn spaces into separators, bad idea here) - as now you're passing the raw options without bash or whatever shell you have attempting to 'interpret' that password which you don't want it to do.
Or, better yet, why are you invoking a shell script?
apr1 is, if memory serves, the crypt key indicating 'MD5, done 1000 times'. This is a HORRIBLE, in that it is trivially rainbow tabled and broken, way of hashing passwords. It's also something you don't need openssl for at all, you can easily do that from java.
So, here's my advice:
Reconsider apr1. It's insecure as heck, you don't want it. Use bcrypt, scrypt, or pbkdf2. Probably bcrypt.
Stop farming this job out via an exec, do it in java. For example, use jBCrypt.
I'm having difficulties to startup a java program from a shell script (bash) where nested variables are used
export MAIN_CLASS="xxxxx"
MAIN_CLASS_ARGS=("$FirstArg" "$SEC_ARG" )
CMD="java some args here ${MAIN_CLASS} ${MAIN_CLASS_ARGS[#]}"
exec $CMD
And I am passing parameter as
export FirstArg = hello
export SEC_ARG ="hi Jam"
But In my main java class I have getting 3 parameter hello, hi ,Jam. But I am expecting it to be only two. What I am missing here can anyone help me.
I have checked some of the link as
link
But not able to fix it.
When you run exec $CMD, then word splitting is performed on the contents of $CMD. It doesn't matter how the variable was built up; at this point, it's just a string which is split by the shell.
Since you appear to be using a shell with support for arrays, then one option would be to do this instead:
CMD=( java some args here "${MAIN_CLASS}" "${MAIN_CLASS_ARGS[#]}" )
exec "${CMD[#]}"
That is, build up an array of all the arguments, then use a quoted array expansion, which prevents word splitting from taking place.
I run egrep using Java Runtime.exec()
String command = "egrep \'(Success|Loading\\.\\.\\.|Loaded : READY|Found a running instance)\' "+ instance.getPath() + "/log";
Runtime.getRuntime().exec(command);
The stdout is always null and stderr shows "egrep: Unmatched ( or (". but when I copy the command to shell and run, it returns the correct value.
The solution is pretty simple: (Success|Loading\\.\\.\\.|Loaded is not a valid regex.
You can't protect white space with quotes when using Process.exec(String). Always use the versions of exec() that take an array or, even better, use ProcessBuilder.
That way, you can pass each argument as a single Java String and spaces and other special characters won't create any problems.
The single quotes should not be escaped. You don't escape them on the command line, either, do you?
Does anyone know the best place where I can go to see the differences between the variations of JEXL? I've noted the following so far.
Expression
This only allows for a single command to be executed and the result from that is returned. If you try to use multiple commands it ignores everything after the first semi-colon and just returns the result from the first command.
Script
This allows you to put multiple commands in the expression and you can use variable assignments, loops, calculations, etc. The result from the last command is returned from the script.
Unified
This is ideal for text. To get a calculation you use the EL-like syntax as in ${someVariable}. The expression that goes between the brackets behaves like a script, not an expression. You can use semi-colons to execute multiple commands and the result from the last command is returned from the script.
I want to attach meta data to a file in Unix file system.
attr command lets me do that but
the command syntax requires the path of the attached variable to be in double qoutes.
attr -s outpipe0 "/mnt/FUse/FileB" FileA
how can i Use System.Runtime.exec in java to run the above command. When ever i try to run using a string array argument I have to give the above "/mnt/FUse/FileB" which causes problem in java program as it considers the double quotes as end of string in java. I basically want to send a string argument which in itself has double quotes.
Can someone suggest a work around .
Thanks
You can escape the quotes within your literal string in Java, like this:
"\"/mnt/FUse/FileB\""
That will address your question of how to include double quotes in a string, but I doubt it will solve your program. That's because I doubt the attr program actually wants (or accepts) double quotes. Instead, the shell eats them. For example, if the command you type in the shell is the one you mentioned, the double quotes will be consumed by the shell before the arguments are passed to attr. So I have doubts that you need the double quotes at all (but if you do, see above).