We Keep Coding

Force BufferedInputStream to return captured content

I have a Spring Boot REST application with 2 endpoints: first starts the Process (db shell) with a command like 'mysql -e root'. Second one accepts command (query) and writes it to OutputStream (which is BufferedOutputStream from Process implementation).
Starting the Process (MySQL shell):
ProcessBuilder builder = new ProcessBuilder(commands);
builder.redirectErrorStream(true);
process = builder.start();
out = process.getInputStream();
in = process.getOutputStream();
Executing a command (e.g. 'select * from db.some_table;\n'):
byte[] commandBytes = command.getBytes(Charset.defaultCharset());
in.write(commandBytes, 0, commandBytes.length);
in.flush();
After running a command (query) I want to return its result (or at least, output it to the console) with:
int no = out.available();
if (no > 0) {
int n = out.read(buffer, 0, Math.min(no, buffer.length));
System.out.println(new String(buffer, 0, n));
}
The problem is that out.available() is always 0.
If I call close() on an output stream, out.available() returns all the input stream length and I can read from it. But that is not what I want.
Can I somehow force BufferedInputStream to make result available to be read without closing the stream?
I see that internally BufferedInputStream uses FileInputStream and FileChannel, but I haven't found a way to capture the result when output stream is not closed.

I think what's happening is that the mysql client detects that standard input is not a terminal, and runs in batch mode rather than in interactive mode. This isn't caused by the behaviour of BufferedReader: it's blocking indefinitely on read, and reporting 0 bytes available because there genuinely isn't anything to read from the output of the subprocess.
In batch mode, the client expects to read a list of commands from standard input, and only executes them once the end of file is reached. In other words, the subprocess will not produce any output on the InputStream you see in your parent process until your parent process closes the OutputStream of the subprocess.
It appears that there's no way to force mysql to run in interactive mode (according to this question: "How to force mysql.exe to run in "interactive" mode?", and the documentation of command line options).
The mysqlsh client can be forced into interactive mode, but it is worth considering whether this is really the best solution for your use case. Other alternatives include:
Embracing batch mode and executing all of the commands together, if the form of each command does not depend on the results of previous ones
Sequentially invoking the subprocess multiple times in batch mode, if subsequent commands do depend on the results of previous ones
Performing the queries using JDBC (as g00se recommended in the comments)

Java: Wait for subprocess of process to finish before reading process's InputStream

I have a process created as follows:
Process p = Runtime.getRuntime().exec(new String[]{"su"});
In my program, I only want to create this process once. I am developing a root file explorer application for Android, and whenever this process is created, the Android device will prompt the user to grant root permissions. This is a very slow operation, and as this is a file browser, it will need root permissions often. So, I have decided to create this process once and write commands to its OutputStream in the following manner (stdin is this OutputStream):
stdin.writeBytes(command + "\n");
Before I can read the output of the command, I need my program to wait until the command written by writeBytes has terminated. I have tried p.waitFor(), but this causes the program to hang.
Here is how I read bytes from the InputStream:
int read;
String out = "";
stdout = p.getInputStream();
byte[] buffer = new byte[262144];
while (true) {
read = stdout.read(buffer);
out += new String(buffer, 0, read);
if (read < BUFF_LEN) {
//we have read everything
break;
}
}
Note that although the read(buffer) method blocks until input data is available, it does not block in this case because it thinks it has reached the end of the InputStream.
I have tried to include only relevant portions of my code in this post, but if you would like to take a look at the entire source code of the class where this is contained, see here: http://pastebin.com/t6JdWmQr.
How can I make sure the command has finished running before reading the process' InputStream?

I also encounter similar problem, and I found the answer here:
Wait until a command in su finishes
If you don't need any read stream in this shell process, simply add shell read stream may completed the shell process.
Or in XDA also have better way:
[HowTo]Execute Root Commands and read output

Getting too many arguments error while executing tr command from java

file = D:\Unix\tr.exe "Æ" "~" < "C:\SourceFiles\source.csv" > "D:\tgt"
When i execute this command using the below code in java
Process process = Runtime.getRuntime().exec(file);
am getting the following error
D:\Unix\tr.exe: too many arguments
PS : File contains Æ characters am trying to replace all those characters with ~
Any suggestions please ?

You're passing unicode in the CMD, I guess that causes the problem.
Set chcp xxx on the CMD and try.
refer below link for chcp codes
https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/chcp.mspx?mfr=true

You have to program the io redirection yourself. This is usually the bash that takes care of it.
The result of the exec() call is a process. This process can then be used to get the STDIN of the process. Send the data to that process using that stream.
The way you call it, you send another command line option to tr, which is the < and > redirectors and the filenames.
Process process = System.getRuntime().exec(strBatchFileName);
OutputStream stdin = process.getOutputStream();
sendFileToStream(out);
InputStream stdout = process.getInputStream();
loadResultFromStream(stdout);
This is a pseudocode example, where sendFileToStream(...) feeds the input file into the TR process, and the loadResultFromStream(...) will load the result.
You may need to utilize threads to feed and read if the data is larger than the stream's buffer.
The question is kind of a duplicate and you will find a ProcessBuilder example here: Runtime's exec() method is not redirecting the output

How does a java program obtain inputstream after being created on a command line

I am trying to write a Java program where Program1 will create Program2 by calling "java -jar Program2.jar" on command line and write bytes of data using outputstream of process.
The program2 after starting to run after command line call should be able to read data written by program1.
How can the program2 find the input stream of the pipe on which program1 has written data.
Any examples to obtain process instance could be helpful
Thanks

Read from System.in of the second program.
See the Java Tutorial on I/O from the Command Line.

You should use the Process class.
Process process = Runtime.getRuntime ().exec ("java -jar Program2.jar");
Using process you can obtain Input or Output stream.
OutputStream stdin = process.getOutputStream ();
InputStream stderr = process.getErrorStream ();
InputStream stdout = process.getInputStream ();

That is not possible as far as I know. But the original invoking program can write the command line parameters to be sent to the second program.
"java -jar Program2.jar param1 param2"
And that's about it....

How to write in Java to stdin of ssh?

Everything works fine on the command line, but when I translate what I want into Java, the receiving process never gets anything on stdin.
Here's what I have:
private void deployWarFile(File warFile, String instanceId) throws IOException, InterruptedException {
Runtime runtime = Runtime.getRuntime();
// FIXME(nyap): Use Jsch.
Process deployWarFile = runtime.exec(new String[]{
"ssh",
"gateway",
"/path/to/count-the-bytes"});
OutputStream deployWarFileStdin = deployWarFile.getOutputStream();
InputStream deployWarFileStdout = new BufferedInputStream(deployWarFile.getInputStream());
InputStream warFileInputStream = new FileInputStream(warFile);
IOUtils.copy(warFileInputStream, deployWarFileStdin);
IOUtils.copy(deployWarFileStdout, System.out);
warFileInputStream.close();
deployWarFileStdout.close();
deployWarFileStdin.close();
int status = deployWarFile.waitFor();
System.out.println("************ Deployed with status " + status + " file handles. ************");
}
The script 'count-the-bytes' is simply:
#!/bin/bash
echo "************ counting stdin bytes ************"
wc -c
echo "************ counted stdin bytes ************"
The output indicates that the function hangs at the 'wc -c' line -- it never gets to the 'counted stdin bytes' line.
What's going on? Would using Jsch help?

You might try closing the output stream before you expect wc -c to return.
IOUtils.copy(warFileInputStream, deployWarFileStdin);
deployWarFileStdin.close();
IOUtils.copy(deployWarFileStdout, System.out);
warFileInputStream.close();
deployWarFileStdout.close();

Would using Jsch help?
Using JSch would only help if you would be using the setInputStream() and setOutputStream() methods of the channel instead of the IOUtils.copy method, since they manage the copying on a separate thread.
ChannelExec deployWarFile = (ChannelExec)session.openChannel("exec");
deployWarFile.setCommand("/path/to/count-the-bytes");
deployWarFile.setOutputStream(System.out);
deployWarFile.setInputStream(new BufferedInputStream(new FileInputStream(warFile)));
deployWarFile.connect();
(Here you somehow have to wait until the other side closes the channel.)
If you simply replaced the Runtime.exec with opening an ChannelExec (and starting it after getting the streams), the problem would be completely the same, and could be solved by the same solution mentioned by antlersoft, i.e. closing the input before reading the output:
ChannelExec deployWarFile = (ChannelExec)session.openChannel("exec");
deployWarFile.setCommand("/path/to/count-the-bytes");
OutputStream deployWarFileStdin = deployWarFile.getOutputStream();
InputStream deployWarFileStdout = new BufferedInputStream(deployWarFile.getInputStream());
InputStream warFileInputStream = new FileInputStream(warFile);
deployWarFile.connect();
IOUtils.copy(warFileInputStream, deployWarFileStdin);
deployWarFileStdin.close();
warFileInputStream.close();
IOUtils.copy(deployWarFileStdout, System.out);
deployWarFileStdout.close();
(Of course, if you have longer output, you will want to do input and output in parallel, or simply use the first method.)

You probably get an error, but the process hangs because you are not reading the error stream.
Taken from the Process JavaDoc
All its standard io (i.e. stdin, stdout, stderr) operations will be redirected to the parent process through three streams (Process.getOutputStream(), Process.getInputStream(), Process.getErrorStream()). The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
So you need to read all of them. Using the ProcessBuilder is probably easier