Everything works fine on the command line, but when I translate what I want into Java, the receiving process never gets anything on stdin.
Here's what I have:
private void deployWarFile(File warFile, String instanceId) throws IOException, InterruptedException {
Runtime runtime = Runtime.getRuntime();
// FIXME(nyap): Use Jsch.
Process deployWarFile = runtime.exec(new String[]{
"ssh",
"gateway",
"/path/to/count-the-bytes"});
OutputStream deployWarFileStdin = deployWarFile.getOutputStream();
InputStream deployWarFileStdout = new BufferedInputStream(deployWarFile.getInputStream());
InputStream warFileInputStream = new FileInputStream(warFile);
IOUtils.copy(warFileInputStream, deployWarFileStdin);
IOUtils.copy(deployWarFileStdout, System.out);
warFileInputStream.close();
deployWarFileStdout.close();
deployWarFileStdin.close();
int status = deployWarFile.waitFor();
System.out.println("************ Deployed with status " + status + " file handles. ************");
}
The script 'count-the-bytes' is simply:
#!/bin/bash
echo "************ counting stdin bytes ************"
wc -c
echo "************ counted stdin bytes ************"
The output indicates that the function hangs at the 'wc -c' line -- it never gets to the 'counted stdin bytes' line.
What's going on? Would using Jsch help?
You might try closing the output stream before you expect wc -c to return.
IOUtils.copy(warFileInputStream, deployWarFileStdin);
deployWarFileStdin.close();
IOUtils.copy(deployWarFileStdout, System.out);
warFileInputStream.close();
deployWarFileStdout.close();
Would using Jsch help?
Using JSch would only help if you would be using the setInputStream() and setOutputStream() methods of the channel instead of the IOUtils.copy method, since they manage the copying on a separate thread.
ChannelExec deployWarFile = (ChannelExec)session.openChannel("exec");
deployWarFile.setCommand("/path/to/count-the-bytes");
deployWarFile.setOutputStream(System.out);
deployWarFile.setInputStream(new BufferedInputStream(new FileInputStream(warFile)));
deployWarFile.connect();
(Here you somehow have to wait until the other side closes the channel.)
If you simply replaced the Runtime.exec with opening an ChannelExec (and starting it after getting the streams), the problem would be completely the same, and could be solved by the same solution mentioned by antlersoft, i.e. closing the input before reading the output:
ChannelExec deployWarFile = (ChannelExec)session.openChannel("exec");
deployWarFile.setCommand("/path/to/count-the-bytes");
OutputStream deployWarFileStdin = deployWarFile.getOutputStream();
InputStream deployWarFileStdout = new BufferedInputStream(deployWarFile.getInputStream());
InputStream warFileInputStream = new FileInputStream(warFile);
deployWarFile.connect();
IOUtils.copy(warFileInputStream, deployWarFileStdin);
deployWarFileStdin.close();
warFileInputStream.close();
IOUtils.copy(deployWarFileStdout, System.out);
deployWarFileStdout.close();
(Of course, if you have longer output, you will want to do input and output in parallel, or simply use the first method.)
You probably get an error, but the process hangs because you are not reading the error stream.
Taken from the Process JavaDoc
All its standard io (i.e. stdin, stdout, stderr) operations will be redirected to the parent process through three streams (Process.getOutputStream(), Process.getInputStream(), Process.getErrorStream()). The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
So you need to read all of them. Using the ProcessBuilder is probably easier
Related
I have a Spring Boot REST application with 2 endpoints: first starts the Process (db shell) with a command like 'mysql -e root'. Second one accepts command (query) and writes it to OutputStream (which is BufferedOutputStream from Process implementation).
Starting the Process (MySQL shell):
ProcessBuilder builder = new ProcessBuilder(commands);
builder.redirectErrorStream(true);
process = builder.start();
out = process.getInputStream();
in = process.getOutputStream();
Executing a command (e.g. 'select * from db.some_table;\n'):
byte[] commandBytes = command.getBytes(Charset.defaultCharset());
in.write(commandBytes, 0, commandBytes.length);
in.flush();
After running a command (query) I want to return its result (or at least, output it to the console) with:
int no = out.available();
if (no > 0) {
int n = out.read(buffer, 0, Math.min(no, buffer.length));
System.out.println(new String(buffer, 0, n));
}
The problem is that out.available() is always 0.
If I call close() on an output stream, out.available() returns all the input stream length and I can read from it. But that is not what I want.
Can I somehow force BufferedInputStream to make result available to be read without closing the stream?
I see that internally BufferedInputStream uses FileInputStream and FileChannel, but I haven't found a way to capture the result when output stream is not closed.
I think what's happening is that the mysql client detects that standard input is not a terminal, and runs in batch mode rather than in interactive mode. This isn't caused by the behaviour of BufferedReader: it's blocking indefinitely on read, and reporting 0 bytes available because there genuinely isn't anything to read from the output of the subprocess.
In batch mode, the client expects to read a list of commands from standard input, and only executes them once the end of file is reached. In other words, the subprocess will not produce any output on the InputStream you see in your parent process until your parent process closes the OutputStream of the subprocess.
It appears that there's no way to force mysql to run in interactive mode (according to this question: "How to force mysql.exe to run in "interactive" mode?", and the documentation of command line options).
The mysqlsh client can be forced into interactive mode, but it is worth considering whether this is really the best solution for your use case. Other alternatives include:
Embracing batch mode and executing all of the commands together, if the form of each command does not depend on the results of previous ones
Sequentially invoking the subprocess multiple times in batch mode, if subsequent commands do depend on the results of previous ones
Performing the queries using JDBC (as g00se recommended in the comments)
I failed to do both reading from process's OutputStream and redirecting it to standard io at the same time. I can do any one of the above, but not both.
I tried to use both inheritIO() and redirectOutput(ProcessBuilder.Redirect.PIPE) and it didn't work, I can read the output but it didn't appear in the standard output.
#Test
void testRedirectOutput() throws IOException, InterruptedException {
// when
Process proc = new ProcessBuilder()
.redirectErrorStream(true)
.inheritIO()
.redirectOutput(ProcessBuilder.Redirect.PIPE)
.command("where", "where")
.start();
proc.waitFor();
// then
String output = readAllOutput(proc);
assertNotNull(output);
}
private static String readAllOutput(Process process) throws IOException {
StringBuilder builder = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
do {
line = reader.readLine();
builder.append(line);
}
while (line != null);
reader.close();
return builder.toString();
}
If I remove .redirectOutput(ProcessBuilder.Redirect.PIPE), it appears on the standard io but I can't read it (output=null).
Is there an elegant way to achieve it rather than calling System.out.println(output)?
Thanks.
A Process runs in parallel to your Java program. While it runs, it reads something from standard input and writes something to standard output and standard error. Typically, a process does not read all input at once and then write all output at once. Instead, it reads some input, does something with it, and then produces some output. This repeats itself until the process detects end of input, at which point it writes the remaining output to the standard output and closes it. In such cases, if you tried to write everything at once and then read everything at once, the OS buffers would fill up and your streams would become blocked in read/write operation.
This means you need a separate Thread to handle each stream. In addition to the thread that is running this code (probably main), you will need two other threads. If the input and output can fit into memory, you can simply pass the data around as ByteArrayInputStream and ByteArrayOutputStream to corresponding threads, and let them do the "pumping" of the data to/from the process.
I have a process created as follows:
Process p = Runtime.getRuntime().exec(new String[]{"su"});
In my program, I only want to create this process once. I am developing a root file explorer application for Android, and whenever this process is created, the Android device will prompt the user to grant root permissions. This is a very slow operation, and as this is a file browser, it will need root permissions often. So, I have decided to create this process once and write commands to its OutputStream in the following manner (stdin is this OutputStream):
stdin.writeBytes(command + "\n");
Before I can read the output of the command, I need my program to wait until the command written by writeBytes has terminated. I have tried p.waitFor(), but this causes the program to hang.
Here is how I read bytes from the InputStream:
int read;
String out = "";
stdout = p.getInputStream();
byte[] buffer = new byte[262144];
while (true) {
read = stdout.read(buffer);
out += new String(buffer, 0, read);
if (read < BUFF_LEN) {
//we have read everything
break;
}
}
Note that although the read(buffer) method blocks until input data is available, it does not block in this case because it thinks it has reached the end of the InputStream.
I have tried to include only relevant portions of my code in this post, but if you would like to take a look at the entire source code of the class where this is contained, see here: http://pastebin.com/t6JdWmQr.
How can I make sure the command has finished running before reading the process' InputStream?
I also encounter similar problem, and I found the answer here:
Wait until a command in su finishes
If you don't need any read stream in this shell process, simply add shell read stream may completed the shell process.
Or in XDA also have better way:
[HowTo]Execute Root Commands and read output
I have a minor Problem with a small Project I'm trying to do.
I'm trying to use a Java-Program to call a Python-Script.
Java:
ProcessBuilder pb = new ProcessBuilder("python3", "tmp.py");
process = pb.start();
OutputStream stdin = process.getOutputStream();
InputStream stderr = process.getErrorStream();
InputStream stdout = process.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stdout));
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(stdin));
writer.write("example" + "\n");
String output = reader.readLine();
Python-Script tmp.py (example):
import sys
sys.stdin.readline()
print("Hello World")
It wont terminate, seemingly because sys.sdin.readline() isnt catching any input and if I remove this line it terminates just fine.
(stderror was empty too)
I tried different things to fix this but nothing seems to work.
I would really appreciate any advice. Thanks in advance.
(Update: I tried to modify the Java-Program to access a .jar File instead of a Python-Script but the same error occurs here to. Using the Python Method subprocess.Popen() to access the Script however works just fine.)
Make sure you start python unbuffered with -u flag:
Force the binary layer of the stdout and stderr streams (which is
available as their buffer attribute) to be unbuffered. The text I/O
layer will still be line-buffered if writing to the console, or
block-buffered if redirected to a non-interactive file.
Edit
I recommend reviewing the buffering all the same. The python unbuffered is a common one, but you possibly still have some buffering. Make sure you flush java writer, writer.flush().
I'm trying to use the WMIC command 'DESKTOP' in an application through a method which is implemented from an interface. Before using WMIC commands in cmd, it's necessary to first input 'WMIC' and press enter, as I'm sure you all know. I can do this just fine, however after the command line enters WMIC mode, I need to subsequently enter the DESKTOP command, or any other WMIC command (see list : http://ss64.com/nt/wmic.html). Here's the overridden method code I'm currently using to enter WMIC mode :
#Override
public void Desktop(){
try {
Runtime rt = Runtime.getRuntime();
String cmd = "cmd /c WMIC";
Process pr = rt.exec(cmd);
}
Perhaps I'm going about this the wrong way? Any help would be much appreciated.
You need to get the output stream of the process you started and write to it.
OutputStream os = pr.getOutputStream();
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(os));
bw.write("DESKTOP");
bw.newLine();
bw.flush();
If you need to see the output/errors from the process you started you need to use getInputStream() and getErrorStream() methods to get the output/error streams and read from it.
Check the javadoc of Process for more details.
http://docs.oracle.com/javase/7/docs/api/java/lang/Process.html
Pay special attention to this part -
By default, the created subprocess does not have its own terminal or
console. All its standard I/O (i.e. stdin, stdout, stderr) operations
will be redirected to the parent process, where they can be accessed
via the streams obtained using the methods getOutputStream(),
getInputStream(), and getErrorStream(). The parent process uses these
streams to feed input to and get output from the subprocess. Because
some native platforms only provide limited buffer size for standard
input and output streams, failure to promptly write the input stream
or read the output stream of the subprocess may cause the subprocess
to block, or even deadlock.
It's a fairly silly answer.
WMIC is normally used in non interactive mode.
wmic desktop get
However you are a programmer. Therefore you are supposed to program.
Here's vbs script that does it. The help is filled with sample scripts in C, VB, VBS, and JScript.
Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\cimv2")
Set colItems = objWMIService.ExecQuery("Select * From Win32_Desktop")
For Each objItem in colItems
msgbox objItem.Name & " " & objItem.Wallpaper
Next