We Keep Coding

Issue in calling Python code from Java (without using jython)

I found this as one of the ways to run (using exec() method) python script from java. I have one simple print statement in python file. However, my program is doing nothing when I run it. It neither prints the statement written in python file nor throws an exception. The program just terminates doing nothing:
Process p = Runtime.getRuntime().exec("C:\\Python\\Python36-32\\python.exe C:\\test2.py");
Even this is not creating the output file:
Process p = Runtime.getRuntime().exec("C:\\Python\\Python36-32\\python.exe C:\\test2.py output.txt 2>&1");
What is the issue?

I think you could try your luck with the ProcessBuilder class.
If I read the Oracle documentation correctly, the std inputs and outputs are directed to pipes by default but the ProcessBuilder has an easy method for you to explicitly set output (or input) to a file on your system or something else.
If you want your Python program to use the same output as your Java program (likely stdout and stderr), you can use stg like this:
ProcessBuilder pb = new ProcessBuilder("C:\\Python\\Python36-32\\python.exe", "C:\\test2.py");
pb.redirectOutput(Redirect.INHERIT);
Process p = pb.start();

You can use the ProcessBuilder API, redirecting the output to a file and then wait for the result.
public class Main {
public static final String PYTHON_PATH = "D:\\Anaconda3\\python.exe";
public static final String PATH_TO_SCRIPT = "D:\\projects\\StartScript\\test.py";
public static void main(String[] args) throws IOException, InterruptedException {
ProcessBuilder builder = new ProcessBuilder();
builder.command(PYTHON_PATH, PATH_TO_SCRIPT);
// Redirect output to a file
builder.redirectOutput(new File("output.txt"));
builder.start().waitFor();
// Print output to console
ProcessBuilder.Redirect output = builder.redirectOutput();
File outputFile = output.file();
BufferedReader br = new BufferedReader(new FileReader(outputFile));
String st;
while ((st = br.readLine()) != null) {
System.out.println(st);
}
}
}
The python file test.py contains a simple print statement:
print("Hello from python")
I guess it would be even simpler, if you do not need to wait for the result.
Using the Process API should work, too.
Like in your example (I am using the same constants declared above):
Process p = Runtime.getRuntime().exec(PYTHON_PATH + " " + PATH_TO_SCRIPT);
p.waitFor();
byte[] buffer = new byte[1024];
byte[] errBuffer = new byte[1024];
p.getInputStream().read(buffer);
p.getErrorStream().read(errBuffer);
System.out.println(new String(buffer));
System.out.println(new String(errBuffer));
To see the output of the print statement, you need to wait and redirect the streams. Same for the error stream.
Now if you break the python script like this:
print("Hello from python')
you should be able to see the error printed as well.

One way to start a python process is using an entrypoint - test.cmd
echo Hello
python hello.py
here is hello.py
#!/usr/bin/env python3
import os
if not os.path.exists('dir'):
os.makedirs('dir')
Here is my Java code:
public static void main(String[] args) throws IOException {
try {
Process p = Runtime.getRuntime().exec("test.cmd");
p.waitFor();
Scanner sc = new Scanner(p.getInputStream());
while(sc.hasNextLine()){
System.out.println(sc.nextLine());
}
sc.close();
} catch (Exception err) {
err.printStackTrace();
}
}

Creating named pipes in Java

I am experimenting with creating named pipes using Java. I am using Linux. However, I am running into a problem where writing to the pipe hangs.
File fifo = fifoCreator.createFifoPipe("fifo");
String[] command = new String[] {"cat", fifo.getAbsolutePath()};
process = Runtime.getRuntime().exec(command);
FileWriter fw = new FileWriter(fifo.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);
bw.write(boxString); //hangs here
bw.close();
process.waitFor();
fifoCreator.removeFifoPipe(fifo.toString());
fifoCreator:
#Override
public File createFifoPipe(String fifoName) throws IOException, InterruptedException {
Path fifoPath = propertiesManager.getTmpFilePath(fifoName);
Process process = null;
String[] command = new String[] {"mkfifo", fifoPath.toString()};
process = Runtime.getRuntime().exec(command);
process.waitFor();
return new File(fifoPath.toString());
}
#Override
public File getFifoPipe(String fifoName) {
Path fifoPath = propertiesManager.getTmpFilePath(fifoName);
return new File(fifoPath.toString());
}
#Override
public void removeFifoPipe(String fifoName) throws IOException {
Files.delete(propertiesManager.getTmpFilePath(fifoName));
}
I am writing a string that consists of 1000 lines. Writing 100 lines work but 1000 lines doesn't.
However, if I run "cat fifo" on an external shell, then the program proceeds and writes everything out without hanging. Its strange how the cat subprocess launched by this program doesn't work.
EDIT: I did a ps on the subprocess and it has the status "S".

External processes have input and output that you need to handle. Otherwise, they may hang, though the exact point at which they hang varies.
The easiest way to solve your issue is to change every occurrence of this:
process = Runtime.getRuntime().exec(command);
to this:
process = new ProcessBuilder(command).inheritIO().start();
Runtime.exec is obsolete. Use ProcessBuilder instead.
UPDATE:
inheritIO() is shorthand for redirecting all of the Process's input and output to those of the parent Java process. You can instead redirect only the input, and read the output yourself:
process = new ProcessBuilder(command).redirectInput(
ProcessBuilder.Redirect.INHERIT).start();
Then you will need to read the process's output from process.getInputStream().

How to execute "less" with paging from a Java console application?

I need to execute the less command, with paging, from my Java console application. However, the only method I found to execute external commands is Runtime.getRuntime().exec(), which requires me to write/read input/output via streams. So commands like cat work (and less does in fact act like cat), but I need the paging functionality.
In C, I'd use system(). In Ruby, Kernel.exec does the job.
Is there any way to get this done in Java?

When you execute an external process with Runtime.exec() its standard input and output streams are not connected to the terminal from which you are running the Java program. You can use shell redirection to connect it, but first you need to know what terminal to use. There is no way to find the terminal using the standard API but probably you can find an open source library that does it.
To see that it can be done, this program opens itself in less:
public class Test {
public static void main(String[] args) throws Exception {
Process p = Runtime.getRuntime().exec(
new String[] {"sh", "-c",
"less Test.java < "+args[0] + " > "+args[0]});
System.out.println("=> "+p.waitFor());
}
}
To run it you should use java Test $(tty). The tty program prints the name of the terminal connected to its stdin.
I'm not too sure about the portability of this solution; at least it works on Linux.

List item
The following program will work, initially it prints 10 lines , then press enter it will print next line till end of the file.
run program like
java Less $fileName
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.Scanner;
public class Less
{
public static void main(String args[]) throws IOException
{
FileReader reader = new FileReader(args[0]);
BufferedReader buff = new BufferedReader(reader);
String readLine;
int lineCount = 0;
while ((readLine = buff.readLine()) != null)
{
System.out.println(readLine);
lineCount++;
if (lineCount > 10)
{
Scanner scanner = new Scanner(System.in);
scanner.nextLine();
}
}
}
}

How to run command line within java using thread async

I am new to java. I am tasked to write java program to run the command lines. I tested the command line under the DOS prompt since i do not have have access to Linux box yet. it worked fine. See the PROGRAM below for full command line syntax. the job will take 6 input files and generate some output files. Next i tried to create a class to and using getruntime and process to process this job. Even it compiled without error but when i run it just show the cursor blinking... i thought i need to use Thread async technique. please provide some advices since i do not have enough time for the projects. I also would like to implement a call back or return values when the job is done. an example would be greatly appreciated. Thanks
import java.io.*;
public class RunJob {
// public static final String PROGRAM = "c:\\wrk\\java.exe Hello";
//one command line below
public static final String PROGRAM = "c:/java.exe -cp \"wrk/jmp.jar;wrk/colt.jar\" gov.lanl.yadas.reliability.UltimateMissileReliabilityModel 10000 \"wrk/\" x1.dat x2c.dat x3.dat x4.dat x5.dat x6.dat true";
// Set to true to end the loop
static boolean done = false;
public static void main(String argv[]) throws IOException {
BufferedReader is;
String line;
String returnMsg = "Start ";
final Process p = Runtime.getRuntime().exec(PROGRAM);
System.out.println("start");
Thread waiter = new Thread() {
public void run() {
try {
p.waitFor();
} catch (InterruptedException ex) {
System.out.println("InterruptedException");
return;
}
System.out.println("Program terminated!");
done = true;
}
};
waiter.start();
is = new BufferedReader(new InputStreamReader(p.getInputStream()));
while (!done && ((line = is.readLine()) != null))
{
System.out.println(line);
returnMsg = returnMsg + line;
}
System.out.println(returnMsg);
System.out.println("End");
return;
}// main
}

I assume that there is a good reason why you want to run a java program from another java program and not just from a shell script, or by invoking an API - but if not - please reconsider.
As to your problem - if your application produces a lot of output (the one you are running as a process) - your application will hang. The p.waitFor() will halt until the process ends. But if you don't read the information from the InputStream - it will overflow and hang!
Advice #1: put the p.waitFor() at the end.
Advice #2: read this article. If I remember correctly it is the one I read when I had a similar problem. You can also google for "StreamGobbler" - it is a common name for a separate thread that "gobbles" your streams.
Advice #3: Don't forget the ErrorStream - if your application will produce too many errors - that stream will cause the process to hang as well.

Problem ProcessBuilder running script sh

trying to execute an script, using this piece of code:
String command = "./myScript.sh";
pb = new ProcessBuilder(command, param1, param2);
pb.directory(directory);
pb.start();
I am not getting any kind of error, but neither the supposed results. Anyway, I tryed to run the same command, direclty in the terminal, and everything working correctly.
Am I missing something??
Thanks in advance

When you start a process (pb.start()) you get back a Process instance. If your script reads input or writes output to stdout or stderr you need to handle this on separate threads using Process.getInputStream(), ...getOutputStream() and getErrorStream(). If you don't do this the process can hang. You also should call Process.waitFor() and then Process.exitValue() to get the return status of the process. If it's a negative number then the system was unable to launch your script.
EDIT: Here is a short simplified example. This is a toy only and will work reliably ONLY under the following conditions:
The script does not require any input
The script does not produce a large amount of output on both stdout and stderr. If it does, then since the program reads all of stdout before stderr, the stderr buffer may fill up and block the process from completing. In a 'real' implementation you would read stdout and stderr in separate threads (hint, wrap the loadStream() method in a class that implements Runnable).
public class PBTest
{
public static void main(String[] args) throws Exception
{
ProcessBuilder pb = new ProcessBuilder("sc","query","wuauserv");
Process p = pb.start();
String output = loadStream(p.getInputStream());
String error = loadStream(p.getErrorStream());
int rc = p.waitFor();
System.out.println("Process ended with rc=" + rc);
System.out.println("\nStandard Output:\n");
System.out.println(output);
System.out.println("\nStandard Error:\n");
System.out.println(error);
}
private static String loadStream(InputStream s) throws Exception
{
BufferedReader br = new BufferedReader(new InputStreamReader(s));
StringBuilder sb = new StringBuilder();
String line;
while((line=br.readLine()) != null)
sb.append(line).append("\n");
return sb.toString();
}
}

The problem was not on the way I called the script, which was right.
But it was inside the script. At first it was:
#!/bin/bash
inputFolder=$1
outputFolder=$2
cd $inputFolder
for file in `ls ` ; do
ffmpeg -i $inputFolder/$file -ar 22050 $outputFolder/$file.mp4
done
But I got ffmpeg command not found, so I changed it to:
#!/bin/bash
inputFolder=$1
outputFolder=$2
cd $inputFolder
for file in `ls ` ; do
/usr/local/bin/ffmpeg -i $inputFolder/$file -ar 22050 $outputFolder/$file.mp4
done
with the hole path. But I have still doubts, why this is necessary, if I have ffmpeg in my path and I cand execute in console direclty form any directory??
If someone can give me an answer, it will be welcome :)