We Keep Coding

Java Fails to Run Large Python Files

I have a Python script which I am attempting to run via code in Java.
The Python script runs fine when run through a Linux terminal command on my Ubuntu virtual machine using an identical command to the one being passed through the Java script.
The Java code runs fine when running a different Python script that runs faster than the Python script I'm attempting to run..
However, despite both the Python script running fine and the Java script running fine, somehow, when I put the two together, nothing happens: The .txt file isn't updated, so the Java script prints out whatever old value it contains.
System.out.println("starting...");
try {
Process process = Runtime.getRuntime().exec("python3 /home/.../PycharmProjects/.../fraudanalysis.py abc def");
Thread.sleep(900000);
# Or try System.out.println(process.waitFor());
File file = new File("/home/.../PycharmProjects/.../output.txt");
Scanner newLineReader = new Scanner(file);
System.out.println(newLineReader.nextLine());
} catch(Exception e) {
System.out.println(e);
}
The code above should run the Python3 script at the absolute directory provided, using two arguments. The Python3 script completes after around 13 minutes and updates the output.txt file, which is then read by the Java program after waiting 15 minutes (or you can tell the thread to wait for completion-- process.WaitFor() returns 1).
def testScript():
time.sleep(780)
return_string1 = sys.argv[1]
return_string2 = sys.argv[2]
outputFile = open(os.path.dirname(os.path.abspath(__file__)) + "/output/output.txt", "w+")
outputFile.write(return_string1 + " " + return_string2)
print("Python run complete")
if __name__ == "__main__":
testScript()
The script above is a good stand-in for the Python script. If you lower the sleep time to 10 minutes for the Python script, it runs when Java sends the command. But, at the sleep times shown above, Java apparently fails to run the script, or the script run attempt ends in failure.
Additional info: the Java command is activated using a JavaFX button. The Java script has been developed in IntelliJ IDEA and the Python script was created using PyCharm.
My question is, what are possible causes for this problem, when both scripts work fine on their own?

As a simple suggestion, you should not rely on Thread.sleep method with a fixed parameter such as 15 minutes. Your data may grow or shrink and that way of proceeding is not efficient.
You could try to call the Process.waitFor() method so that when the python process is over, your thread continues.
Moreover, you could try to use ProcessBuilder that sometimes helps when facing buggy System exec cases.
Here is some code. in sub(), you can not change the python program, but for sub2() to work, you have to modify the python program so that its output is on the standard out and Java would do the redirect to the output.txt file.
public void sub() {
System.out.println("startig...");
Scanner newLineReader = null;
try {
Process process = Runtime.getRuntime().exec("python3 /home/.../PycharmProjects/.../fraudanalysis.py /home/.../PycharmProjects/.../fraudAnalysis.db 500");
process.waitFor();
File file = new File("/home/.../PycharmProjects/.../output.txt");
newLineReader = new Scanner(file);
String line;
while((line=newLineReader.nextLine())!=null) {
System.out.println(line);
}
} catch(IOException ioe) {
ioe.printStackTrace();
}catch(InterruptedException ie) {
ie.printStackTrace();
}finally {
newLineReader.close();
}
}
public void sub2() {
ProcessBuilder pb =
new ProcessBuilder("python3",
"/home/.../PycharmProjects/.../fraudanalysis.py",
"/home/.../PycharmProjects/.../fraudAnalysis.db", "500");
File log = new File("/home/.../PycharmProjects/.../output.txt");
pb.redirectErrorStream(true);
pb.redirectOutput(Redirect.appendTo(log));
Process p = null;
try {
p = pb.start();
} catch (IOException e) {
e.printStackTrace();
}
try {
p.waitFor();
} catch (InterruptedException e) {
e.printStackTrace();
}
Scanner newLineReader = null;
try{
newLineReader = new Scanner(log);
String line;
while((line=newLineReader.nextLine())!=null) {
System.out.println(line);
}
}catch(IOException ioe) {
ioe.printStackTrace();
}
}

I was able to get it to work with a small modification. I used relative file locations and TimeUnit.MINUTES.sleep(15);
package org.openjfx;
import java.io.File;
import java.util.Scanner;
import java.util.concurrent.TimeUnit;
public class TestWait {
public static void main(String[] args) {
System.out.println("starting...");
String dir="src/main/resources/org/openjfx/";//location of the python script
try {
System.out.println("Working Directory = " + System.getProperty("user.dir"));
//System.out.println("python3 " + dir+"fraudanalysis.py abc def");
Process process = Runtime.getRuntime().exec("python3 " + dir+"fraudanalysis.py abc def");
System.out.println(process.waitFor());
TimeUnit.MINUTES.sleep(15);
File file = new File("src/main/resources/org/openjfx/output.txt");
Scanner newLineReader = new Scanner(file);
System.out.println(newLineReader.nextLine());
} catch (Exception e) {
System.out.println(e);
}
}
}
Here is the python I used.
import sys
import time
def testScript():
return_string1 = sys.argv[1]
return_string2 = sys.argv[2]
time.sleep(780)
outputFile = open("src/main/resources/org/openjfx/output.txt", "w+")
outputFile.write(return_string1 + " " + return_string2)
print("Python run complete")
if __name__ == "__main__":
testScript()

it's a timeout error. can't be fixed. just pick between Java and Python and write everything in it. no reason to use both.

How do I run GDB and accept GDB commands through Java output console?

I am trying to build a GUI for GCC which has some basic functionalities like compile, link, execute, debug, etc for C++ programs using Java. I am creating strings of command which I pass to the ProcessBuilder and run it via command prompt and GCC.
command = "cd src & cd Resources & g++ " + compileFile.cpp +" -Wall "+ " -o "+ "tempOut";
This is a sample code for compiling the file.
Part of this is the debug functionality for which I am using GDB. Now the problem is GDB needs additional input to add breakpoints, remove breakpoints and so on. I am having trouble on how to pass these necessary inputs to GDB via Java terminal. If I pass the commands in the command prompt, it is working fine and I get the desired output.
enter image description here
But whenever I fire the GDB command from the Java program, I cannot pass any inputs from the IDE's terminal. I am aware that each GDB command uses a different process and I tried attaching Java's process ID to GDB but I just get a blank output console. It seems that the GDB session has started but there is no way to interact with that process through the IDE's output console.
int pid = Integer.parseInt(ManagementFactory.getRuntimeMXBean().getName().split("#")[0]);
command = "gdb attach "+ pid;
fireCommand();
EDIT
This is the method that interacts with command prompt to take input and display output in the IDE's output console:
public void fireCommand() {
String line;
String os = System.getProperty("os.name");
ProcessBuilder builder;
if(os.startsWith("Win")) {
builder = new ProcessBuilder("cmd.exe", "/c", command);
}
else{
builder = new ProcessBuilder("bash", "-c", command);
}
try {
process = builder.start();
BufferedReader reader = new BufferedReader(
new InputStreamReader(process.getInputStream()));
while ((line = reader.readLine()) != null) {
if(line.contains("Input the value")) {
//any other user input in non debug execution
String value = JOptionPane.showInputDialog(line);
BufferedWriter writer = new BufferedWriter(
new OutputStreamWriter(process.getOutputStream()));
writer.write(value, 0, value.length());
writer.newLine();
writer.close();
}
else {
output.append(line).append("\n");
}
}
int exitVal = process.waitFor();
if (exitVal == 0) {
//display("Success!");
display(output.toString());
} else {
String lineErr;
BufferedReader readerErr = new BufferedReader(
new InputStreamReader(process.getErrorStream()));
while ((lineErr = readerErr.readLine()) != null) {
outputErr.append(lineErr).append("\n");
}
//display(exitVal);
display(outputErr.toString()); //Display the uncatched errors
}
} catch (IOException e) {
display("There was a problem with the I/O");
e.printStackTrace();
} catch (InterruptedException e) {
display("There was a interruption with the execution");
e.printStackTrace();
}
if(!outputErr.toString().isEmpty())
errorFormatDisplay(); //display Error output function
}
Any leads on this would be very helpful. Thank you.

I am aware that each GDB command uses a different process
No, gdb runs as one process. Did you mean to say that you are creating a new gdb process every time you try to pass it a gdb command ?
It seems that the GDB session has started but there is no way to interact with that process through the IDE's output console.
Maybe you should use The GDB/MI Interface, which e.g. the Emacs debugger mode gud uses.

running multiple cmd commands through java separated by some another code

I want to start a cmd command, then after the first command is done, I want to run a code to adjust some text in a file, then execute another command on the same cmd window. I don't know how to do that and everywhere I looked the answer is for the commands after each other which is not this case. the code for editing the text works fine without starting the cmd but if I execute the cmd command it does not change. code below.
public static void main(String[] args)throws IOException
{
try
{
Main m1 = new Main();
Process p= Runtime.getRuntime().exec("cmd /c start C:/TERRIERS/terrier/bin/trec_setup.bat");
p.waitFor();
/*code to change the text*/
m1.answerFile(1);
m1.questionFile(1);
/**********************/
//code to add another command here (SAME WINDOW!)
/************************/
}
catch(IOException ex){
}
catch(InterruptedException ex){
}

Execute cmd and send your command lines (.bat) to the standard input.
Process p = Runtime.getRuntime().exec("cmd");
new Thread(() -> {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()))) {
String line;
while ((line = reader.readLine()) != null)
System.out.println(line);
} catch (Exception e) {
e.printStackTrace();
}
}).start();
try (PrintStream out = new PrintStream(p.getOutputStream())) {
out.println("C:/TERRIERS/terrier/bin/trec_setup.bat");
out.println("another.bat");
// .....
}
p.waitFor();

For starters, the \C option terminates CMD after executing the initial command. Use \K instead.
You won't be able to use waitFor() to detect when the initial command is done, because if you wait until the CMD terminates, you won't be able to re-use the same process.
Instead, you'll need to read the output of CMD process to detect when the batch file is complete and you are prompted for another command. Then write the next command line that you want to execute though the input stream of the Process.
Sounds like a pain. Why would do you need to use the same window?

Executing Fabric Script through JAVA

I have this simple fab script and I want to execute this through JAVA:
from __future__ import with_statement
from fabric.api import *
from fabric.contrib.console import confirm
env.hosts = ['localhost']
def updatefile():
with shell_env(TERM='vt100'):
with cd('/Users/'):
run("pwd")
run("ls -l")
def execute():
updatefile()
When I execute this script from command line it works : fab -f test.py executes but I want to execute via java. Tried with ..
public class ExecuteScript {
#Test
public void testExecuteScript() throws NumberFormatException, IOException{
StringBuffer output = new StringBuffer();
Process p;
try {
p = Runtime.getRuntime().exec("fab -f src/test/resources/scripts/test.py execute");
p.waitFor();
BufferedReader reader =
new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine())!= null) {
output.append(line + "\n");
}
} catch (Exception e) {
e.printStackTrace();
}
System.out.println(output.toString());
}
}
It doesn't work.. looked at the documentation for java examples in http://fabric8.io/gitbook/quickstarts.html the links are broken.

Environment, make sure you're using your environment. I would suggest you use virtualenv, looks something like this:
virtualenv .env (dont do this in your src folder, if anything it should be one directory down, and also please don't check this into your source control)
source .env/bin/activate
then install your dependencies:
pip install fabric
?? what ever else you need.
then this is the important part:
./.env/bin/fab -f src/test/resources/scripts/test.py execute

Not getting any output after calling C executable file from Java code

I am trying to execute the C code from Java code which is already compiled and executed, but, I am not getting any output from the executable file. Can anyone help me to complete this task?
Code is as follows.
public class Test {
public static void main(String args[]) {
try {
Process processCompile = Runtime.getRuntime().exec("e:/Sample.exe");
} catch(Exception ex) {
ex.printStackTrace();
}
}
}

Try this:
BufferedReader stdInput = new BufferedReader(new
InputStreamReader(processCompile .getInputStream()));
// read the output from the command
System.out.println("EXE OUTPUT");
while ((s = stdInput.readLine()) != null) {
System.out.println(s);
}

This method would work only if you run the java program with admin privileges.
If you have privileges, then can you try running your process under "cmd" shell (Which is forked by your java process). An implementation do so this is done here "LinuxInteractor" ( But is in linux). Just minor change needed to port to Windows version.
Finding hard and soft open file limits from within jvm in linux (ulimit -n and ulimit -Hn)