We Keep Coding

Java Runtime.getRuntime().exec(cmd) command contain single quote

I need to use Java to rsync several files using one command
the following command works fine in shell
rsync -avrz --timeout=100 rsync://10.149.21.211:8730/'logflow/click/file1 logflow/click/file2' /home/kerrycai/puller"
but when i use the following Java code , it does not work
String cmd = "rsync -avrz --timeout=100 rsync://10.149.21.211:8730/'logflow/click/file1 logflow/click/file2' /home/kerrycai/puller";
Process p = Runtime.getRuntime().exec(cmd);
int ret = p.waitFor();
the ret value is not equal to 0 (5 in my test), and the command is not executed succeed, after some debugging , it seem the problem is caused by the single quote
So, my questions is
Can I using java to execute a shell command which has single quote in it (Pls note, the single quote is in the middle of a parameter, not start/end) ?
Can I have a shell command to rsync several files in one command , and the command does not have single(double) quotes in it ?

Note to #Chris: this combination of multiple filenames (really modified-partly-like-filenames) in one argument is indeed very unusual and even 'suspicious' for Unix in general, but is (or at least was) correct for rsync in particular.
Preface: Java Runtime.exec does NOT 'execute a shell command' (unless you explicitly run a shell and give it a command); it runs a program, with arguments. These different things are often confused because most of the shell commands used by normal users are commands to run programs, but this is one case where the difference matters. In particular quoting a space to shell causes the shell to pass a single argument to the program containing the space instead of splitting into two (or more) arguments, but the quote itself is NOT included in the argument.
First you should look at the man page (on your system or online at https://download.samba.org/pub/rsync/rsync.html) under ADVANCED USAGE. Current (and IME even moderately old) versions of rsync have a more convenient syntax with separate arguments to get multiple files, which the simple parsing used by Runtime.exec(String) can handle like this:
rsync -avrz --timeout=100 --port=8730 10.149.21.211::logflow/click/file1 ::logflow/click/file2 /home/kerrycai/puller
But if you need (or really want) to use the quoted-space form then you need to do the tokenization yourself and use the String[] overload as suggested by #EJP -- although you can still use Runtime, you don't need ProcessBuilder for this. Specifically do something along the lines of:
String[] cmdarray = {"rsync",
"-avrz",
"--timeout=100",
"rsync://10.149.21.211:8730/logflow/click/file1 logfile/click/file2",
// separated argument contains space but not single (or other) quote
"/home/kerrycai/puller" };
... Runtime.getRuntime.exec(cmdarray); ...

You're calling a somewhat large command from Java. Why not just use a shell script? Put your gnarly command in myScript.sh and then have Java invoke /bin/bash myScript.sh. Makes all the weirdness to do with string handling in Java go away.

'Communicate' in Python does not work

I'm trying to write a python program to test a java program that takes input from stdin using Scanner.
All other posts point to using communicate with popen, but for me it absolutely does not work. When i run my python program, it just calls popen and then stops while the java program waits for input. I wrote a print statement after popen to check. It never prints.
Its very simple. I just want to give this program that waits for input some input.
here is the code:
import os.path, subprocess
from subprocess import PIPE
p = subprocess.Popen(['java', 'Main'], stdin=PIPE, stdout=PIPE)
print 'after subprocess' #this never get's printed
output = p.communicate(input='5 5 4 3 2 1'.encode())[0]
print output

Without more information (like some sample Java code) it's hard to be sure, but I'll bet the problem is that the Java code is waiting for a complete line, and you haven't sent one.
If so, the fix is simple:
output = p.communicate(input='5 5 4 3 2 1\n'.encode())[0]
As a side note, why exactly are you calling encode on that string? It's already encoded in whatever character set your source code uses. So, when you call encode, it has to first decode that to Unicode. And then, because you didn't pass an argument to encode, it's going to encode it to your default character set (sys.getdefaultencoding()), which doesn't seem any more likely to match what the Java code is expecting than what you already have. It's rarely worth calling encode with an argument, and you should almost* never call it on a str, only a unicode.
* In case you're wondering, the exception is when you're using a handful of special codecs like hex or gzip. In Python 3, they decided that the occasional usefulness of those special cases was nowhere near as much as the frequent bug-magnet of calling encode on already-encoded strings, so they took it out of the language.

parsing strings with or without regex

I am trying create a basic file system to imitate the terminal. I am currently stuck on getting the names after the command. My first thought was to use regex to parse the commands.
Examples of commands would be:
mkdir hello
ls
cd hello
However to account for many whitespaces an input could be mkdir hello. I was wondering if there is another way without using regex? Also I was curious to which method is faster? With or without regex?

You could try splitting the lines like
String[] tokens = line.split(" ");
And for basic commands, most likely your command will be at tokens[0] followed by arguments.

for(String current: line.split("\\s+"){
//do something.
}

Usually, regex is faster because you can compile it.
see java.util.regex - importance of Pattern.compile()?
(Internally, I think the JVM always compile the regex at some point, but if you do it explicitly, you can reuse it. I am not sure if the JVM is smart enough to reuse a compiled regex locally, maybe it is)

Bash input redirection with newlines

So for an assignment at school I am required to make a program in java that manages some students at a college, the user may invoke a number of function, but I started by testing 'add'.
My prof recommends that I use input redirection, because I can easily test the program over and over, without much typing. Now here is the issue.
echo `< input.txt`
add Student Name 123 Street,City,Province,Postal Code q
#Input rediraction ate my newlines
cat input.txt
add
Student Name
Street,City,Province,Postal Code
q
#What I actually want to run
java -cp . TestCollege < input.txt #fail horribly
As you can see bash removes the newline characters I use to delimit my input, and thus my program crashes, because when you run it in interactive mode, Scanner.getLine() just blocks until input occurs.
Any idea how I can give my program the input with the newlines, it seems to work fine in windows "cmd" but not in "bash".

Could you try wrapping BufferedInputStream around System.in, because it can handle both cases: \r\n and \n; and I think that's where your problem lies.
Or rather, call useDelimiter(System.getProperty("line.separator")) on Scanner.

Well, you can see how bash does not eat the newlines running
$ wc -l < input.txt
Which I guess will give you the value you'd expect.

Weird Error JAVA process execution [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
run shell command from java
I'm trying to run a process with some arguments from inside Java and before running the particular command I print it using println.
Now the problem is from inside Java, the process is not doing good.. its sending some error on the Error Stream instead of output. But If I run the same command printed by Java on screen in console, it works perfectly.
String command="abc -def -hhij";
System.out.println(command);
Process p = Runtime.getRuntime.exec(command);
Anyone know whats going on wrong?

The Runtime.exec(String) using the StringTokenizer, which doesn't know how to process quoted text. The simplest fix is to use the exec that accepts your parameters in an array.
String[] command={ "abc", "-def", "-hhij"};
Process p = Runtime.getRuntime.exec(command);

I would try just to escape those '(' with \: '\\(' (double \ due to Java escaping rules)

Most likely the command needs the full path to the command. C:\dir\abc.exe or whatever.
If it's a script make sure it' executed by the correct shell.