I am using Java as a front end for a chess AI i am writing. The Java handles all the graphics, and then executes some C using a few command line arguments. Sometimes the C will never finish, and not get back to the Java. I have found cases in which this happens, and tested them with just the .exe and no java. When i take out the java, these cases work everytime. I am not sure where to go from here. Here is some code that i think is relavant, and the whole project as at https://github.com/AndyGrant/JChess
try{
Process engine = Runtime.getRuntime().exec(buildCommandLineExecuteString(lastMove));
engine.waitFor();
int AImoveIndex = engine.exitValue();
String line;
BufferedReader input = new BufferedReader(new InputStreamReader(engine.getInputStream()));
while ((line = input.readLine()) != null)
System.out.println(line);
input.close();
if (AImoveIndex == -1){
activeGame = false;
System.out.println("Fatal Error");
while (true){
}
}
else{
JMove AIMove = JChessEngine.getAllValid(types,colors,moved,lastMove,!gameTurn).get(AImoveIndex);
AIMove.makeMove(types,colors,moved);
lastMove = AIMove;
validMoves = JChessEngine.getAllValid(types,colors,moved,lastMove,gameTurn);
}
waitingOnComputer = false;
parent.repaint();
}
catch(Exception e){
e.printStackTrace();
}
Sometimes, the external process will get stuck on IO, trying to write to the console. If the console buffer is full, the next printf will block.
How much text is it writing to the console?
Try moving your engine.waitFor() after the part where you read all the input from it.
An alternative would be to have the external process write to a temp file, and then you read the temp file.
Maybe remove
while (true){
}
If your AImoveIndex == -1, your program will enter in a never ending loop.
Related
I am reading a huge text file of words (one word per line) but I have to stop it from time to time to resume the read the next day. Right now I'm using Apache's lineiterator but it's totally the wrong solution. My file is 7Gb and I had to interrupt reading it around at 1Gb. To resume the read I saved the number of line already read. This means that I have an if statement on the while loop. Apache's FileUtils doesn't allow to seek so that was my solution.
What is the best/fastest solution? I thought to use RandomAccessfile to get to the right line and continue reading, but I'm not sure if I can go to the right place AND how do I save the correct place I read last. I can reead again a couple of lines, so the precision is not so important, but I haven't found a way to get the pointer. I have a BufferedReader to read the File and a RandomAccessFile to seek to the right place, but I don't know how to periodically save a position with the BufferedReader.
Any hints?
Code: (note the "SOMETHING" where I should print the value I can use on the seekToByte )
try {
RandomAccessFile rand = new RandomAccessFile(file,"r");
rand.seek(seekToByte);
startAtByte = rand.getFilePointer();
rand.close();
} catch(IOException e) {
// do something
}
// Do it using the BufferedReader
BufferedReader reader = null;
FileReader freader = null;
try {
freader = new FileReader(file);
reader = new BufferedReader(freader);
reader.skip(startAtByte);
long i=0;
for(String line; (line = reader.readLine()) != null; ) {
lines.add(line);
System.out.print(i+" ");
if (lines.size()>1000) {
commit(lines);
System.out.println("");
lines.clear();
System.out.println(SOMETHING?);
}
}
} catch(Exception e) {
// handle this
} finally {
if (reader != null) {
try {reader.close();} catch(Exception ignore) {}
}
}
RandomAccessfile is indeed one way to go. Use
long position = file.getFilePointer();
When you stop reading to save where you are in the file, and then restore with:
file.seek(position);
To resume reading at the same place.
However, be careful when using RandomAccessfile, as its readLine method does not completely support Unicode.
Can you somehow use predetermined offsets, for instance chop the file into four pieces (offset0, offset1) (offset1, offset2)..etc, and use RecursiveAction (ForkJoin API) to take advantage of parallelism.
I want to use an external tool while extracting some data (loop through lines).
For that I first used Runtime.getRuntime().exec() to execute it.
But then my extraction got really slow. So I am searching for a possibility to exec the external tool in each instance of the loop, using the same instance of shell.
I found out, that I should use ProcessBuilder. But it's not working yet.
Here is my code to test the execution (with input from the answers here in the forum already):
public class ExecuteShell {
ProcessBuilder builder;
Process process = null;
BufferedWriter process_stdin;
BufferedReader reader, errReader;
public ExecuteShell() {
String command;
command = getShellCommandForOperatingSystem();
if(command.equals("")) {
return; //Fehler! No error handling yet
}
//init shell
builder = new ProcessBuilder( command);
builder.redirectErrorStream(true);
try {
process = builder.start();
} catch (IOException e) {
System.out.println(e);
}
//get stdout of shell
reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
//get stdin of shell
process_stdin = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
System.out.println("ExecuteShell: Constructor successfully finished");
}
public String executeCommand(String commands) {
StringBuffer output;
String line;
try {
//single execution
process_stdin.write(commands);
process_stdin.newLine();
process_stdin.flush();
} catch (IOException e) {
System.out.println(e);
}
output = new StringBuffer();
line = "";
try {
if (!reader.ready()) {
output.append("Reader empty \n");
return output.toString();
}
while ((line = reader.readLine())!= null) {
output.append(line + "\n");
return output.toString();
}
if (!reader.ready()) {
output.append("errReader empty \n");
return output.toString();
}
while ((line = errReader.readLine())!= null) {
output.append(line + "\n");
}
} catch (Exception e) {
System.out.println("ExecuteShell: error in executeShell2File");
e.printStackTrace();
return "";
}
return output.toString();
}
public int close() {
// finally close the shell by execution exit command
try {
process_stdin.write("exit");
process_stdin.newLine();
process_stdin.flush();
}
catch (IOException e) {
System.out.println(e);
return 1;
}
return 0;
}
private static String getShellCommandForOperatingSystem() {
Properties prop = System.getProperties( );
String os = prop.getProperty( "os.name" );
if ( os.startsWith("Windows") ) {
//System.out.println("WINDOWS!");
return "C:/cygwin64/bin/bash";
} else if (os.startsWith("Linux") ) {
//System.out.println("Linux!");
return"/bin/sh";
}
return "";
}
}
I want to call it in another Class like this Testclass:
public class TestExec{
public static void main(String[] args) {
String result = "";
ExecuteShell es = new ExecuteShell();
for (int i=0; i<5; i++) {
// do something
result = es.executeCommand("date"); //execute some command
System.out.println("result:\n" + result); //do something with result
// do something
}
es.close();
}
}
My Problem is, that the output stream is always empty:
ExecuteShell: Constructor successfully finished
result:
Reader empty
result:
Reader empty
result:
Reader empty
result:
Reader empty
result:
Reader empty
I read the thread here: Java Process with Input/Output Stream
But the code snippets were not enough to get me going, I am missing something. I have not really worked with different threads much. And I am not sure if/how a Scanner is of any help to me. I would really appreciate some help.
Ultimatively, my goal is to call an external command repeatetly and make it fast.
EDIT:
I changed the loop, so that the es.close() is outside. And I wanted to add, that I do not want only this inside the loop.
EDIT:
The problem with the time was, that the command I called caused an error. When the command does not cause an error, the time is acceptable.
Thank you for your answers
You are probably experiencing a race condition: after writing the command to the shell, your Java program continues to run, and almost immediately calls reader.ready(). The command you wanted to execute has probably not yet output anything, so the reader has no data available. An alternative explanation would be that the command does not write anything to stdout, but only to stderr (or the shell, maybe it has failed to start the command?). You are however not reading from stderr in practice.
To properly handle output and error streams, you cannot check reader.ready() but need to call readLine() (which waits until data is available) in a loop. With your code, even if the program would come to that point, you would read only exactly one line from the output. If the program would output more than one line, this data would get interpreted as the output of the next command. The typical solution is to read in a loop until readLine() returns null, but this does not work here because this would mean your program would wait in this loop until the shell terminates (which would never happen, so it would just hang infinitely).
Fixing this would be pretty much impossible, if you do not know exactly how many lines each command will write to stdout and stderr.
However, your complicated approach of using a shell and sending commands to it is probably completely unnecessary. Starting a command from within your Java program and from within the shell is equally fast, and much easier to write. Similarly, there is no performance difference between Runtime.exec() and ProcessBuilder (the former just calls the latter), you only need ProcessBuilder if you need its advanced features.
If you are experiencing performance problems when calling external programs, you should find out where they are exactly and try to solve them, but not with this approach. For example, normally one starts a thread for reading from both the output and the error stream (if you do not start separate threads and the command produces large output, everything might hang). This could be slow, so you could use a thread pool to avoid repeated spawning of processes.
I am currently interfacing a GUI to an UCI chess engine. For this purpose i am creating the engine process using:
try {
process = Runtime.getRuntime().exec(enginePath);
} catch (IOException e) {
System.err.println("ENGINE NOT FOUND");
e.printStackTrace();
}
and i am sure that i am able to open the engine.
When engine is opened, there is no significant stream outputted from engine. In order to initiate communication, I have to send specific commands to the engine. The engine will respond then... Therefore it is working in a command/response approach(not immediately streaming data when opened or talking without spoken to). In order to communicate i have a send message block. In this block write a message to the engine using its outputstream and get input using its standart input stream as in the following send method:
private String sendCommand(String command) {
stdin = new PrintWriter(new OutputStreamWriter(process.getOutputStream()));
BufferedReader inputReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedReader errReader = new BufferedReader(newInputStreamReader(process.getErrorStream()));
String answer = "";
stdin.println(command);
stdin.flush();
stdin.close();
try {
String line = "";
while ((line = inputReader.readLine()) != null) {
System.out.println(line);
answer = line;
}
inputReader.close();
} catch (IOException ioe) {
System.err.println("READ ERROR");
ioe.printStackTrace();
}
try {
String line = "";
while ((line = errReader.readLine()) != null) {
System.err.println(line);
}
errReader.close();
} catch (IOException ioe) {
System.err.println("READ ERROR");
ioe.printStackTrace();
}
return answer;
}
By the way the whole engine is working in a single seperate thread other than the main GUI thread. Therefore no overloading for the GUI. However i didnot seperate the stdin stderr and output methods in individual threads since i suppose there will be only stdin and the one that i will write to the engine. For the time being i assume they will not collide.
With these codes i am able to communicate with the engine and i can see the outputs in the console. However due to stdin.close(); i can only use this method once (In the seconds time i have a STREAM CLOSED error). The chess engine is needed to be communicated in command/response approach many times when opened, without restarting the exe in each time. The problem is that if i remove the line : stdin.close(); yes the communication continues, but my console is blocked by this communication. I.e. i cannot println to the console for debugging purposes anymore. Which is very critical because my main debugging weapon is system.out.println. If i do not remove stdin.close(); i have to restart executable each time i want to send message and i do not want that. **The strange part is that i can see the messages coming from the engine in my console due to "System.out.println(line);" line in the send method; however i cannot print anything on the console once the code exits the method. **
EDIT: Actually after this block:
while ((line = inputReader.readLine()) != null) {
System.out.println(line);
answer = line;
}
System.out.println starts not to work.
EDIT: The problem seems not to be with println but any statement after while loop.
Why statements after while loop is not getting executed?
according to this, it seems that since the stream is never closed, "while loop" is stuck(?). Actually when printing inside while loop, at some point(after a stream is finished), the prints stop, which must indicate while loop is finished. Therefore if it is finished, it should continue on the next statements, shouldn't it? Anyway, the messages from the engine had a set of strings at end of each stream; therefore when i encounter one of them i am breaking the while loop.
Note: it is also interesting that : accumulating data as: "receivedString += line"; doesnot work,i.e. data is not accumulated. In order to fix it i luckily made it "receivedString = receivedString + line + "\n""; and it worked.. I dont know why.
You're reading the input until end of stream, so it won't stop reading until end of stream occurs. End of stream means that the peer has closed the connection in this case. That won't happen for a process's output or error stream until the process exits. You'll have to find some other loop termination condition, or else consume the stdout and stderr in separate threads.
I have a project where program has to open notepad file and after entering text and saving that notepad file program should display number of words in that file and it should delete the entered content in the file.
iam getting this error Error not derjava.lang.NullPointerException after running the program.
though after entering some text in Mytext.txt and saving it?
my question is why BufferedReader is reading empty file even though iam saving the file with some content.
Appreciate the help..
public class Notepad_Example {
public static void main(String[] jfb) {
try {
ProcessBuilder proc = new ProcessBuilder("notepad.exe", "C:\\Java Projects\\Reverse String\\src\\Mytext.txt");
proc.start();
BufferedReader br;
String s;
br = new BufferedReader(new FileReader("C:\\Java Projects\\Reverse String\\src\\Mytext.txt"));
s = br.readLine();
char c[] = new char[s.length()];
int j = 0;
for (int i = 0; i < s.length(); i++) {
if (s.charAt(i) != ' ') {
c[i] = s.charAt(i);
} else {
j++;
}
}
System.out.println("number of words are " + (j + 1));
br.close();
} catch (Exception hj) {
System.out.println("Error not der" + hj);
}
try {
FileWriter fw = new FileWriter("C:\\Java Projects\\Reverse String\\src\\Mytext.txt");
fw.close();
} catch (Exception hj) {
System.out.println("Error not der" + hj);
}
}
}
The issue you are having is here:
ProcessBuilder proc=new ProcessBuilder("notepad.exe","C:\\Java Projects\\Reverse String\\src\\Mytext.txt");
proc.start();
proc.start() is returning the freshly started process. You'll have to give the user the chance to edit and save the file and close the editor before you can read from that file. That is you have to wait for that process to finish before you can start using the results (the saved file) of that process.
So do instead something like this:
Process process = proc.start();
int result = process.waitFor();
if (result == 0) {
// Do your rest here
} else {
// give error message as the process did not finish without error.
}
Some further remarks:
The rest of your code also appears to have some issues.
You are only reading one line of that file. What if the user is using new lines?
The exception handling is not very good, at the very least print the stack trace of the exception which will give you further hints of where an exception was occuring
If you are using Java 7, read on try with resources; if you are using Java 6, add finally blocks to make sure your resources (the streams) are getting closed.
When you run proc.start(); it is not going to block and waitfor the process to end, it will continue running.
You will need to call the proc.waitFor() method, to block until it has finished.
NOTE
we have had some weird behaviour when using the process builder...
we used to start the process with a
new ProcessBuilder("notepad.exe", "C:\\Java Projects\\Reverse String\\src\\Mytext.txt");
but that started to fail wen we upgraded to Win7 and Java7 - we we not sure where this problem really originated, but we changed out Code like this:
String[] cmd = new String[]{"notepad.exe", "C:\\Java Projects\\Reverse String\\src\\Mytext.txt"};
new ProcessBuilder(cmd);
and since then it worked correct!
I have one application that prints messages from Test.exe in console .My java program creates one process by executing this Test.exe.
This application prints messages by reading from input-stream of that process.
The problem, that I am facing is,
I have two scenarios:
1) When I double click test.exe, messages("Printing : %d") are printing for every second.
2)But when I run my java application,whole messages are printing at last(not for every second) before terminating Test.exe.If .exe has a very huge messages to print,then it will print those messages(I think whenever buffer becomes full)and flushing will be done.
But how can I print messages same as 1st case.
Help from anyone would be appreciated. :)
Here is the code for this Test.exe.
#include <stdio.h>
#include <windows.h>
void main(void)
{
int i=0;
while (1)
{
Sleep(500);
printf("\nPrinting : %d",i);
i++;
if (i==10)
//if(i==100)
{
return 0;
}
}
}
And my Java application is below:
public class MainClass {
public static void main(String[] args) {
String str = "G:\\Charan\\Test\\Debug\\Test.exe";
try {
Process testProcess = Runtime.getRuntime().exec(str);
InputStream inputStream = new BufferedInputStream(
testProcess.getInputStream());
int read = 0;
byte[] bytes = new byte[1000];
String text;
while (read >= 0) {
if (inputStream.available() > 0 ) {
read = inputStream.read(bytes);
if (read > 0) {
text = new String(bytes, 0, read);
System.out.println(text);
}
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Is it possible in reverse order.If I input some text from console,Java should read and pass that String to .exe(or testProcess).How .exe scan something from Java program.
Could anyone help me..
Given that you're trying to print stdout from that process line by line, I would created a BufferedReader object using the process' input stream and use the readLine() method on that. You can get a BufferedReader object using the following chain of constructors:
BufferedReader testProcessReader = new BufferedReader(new InputStreamReader(testProcess.getInputStream()));
And to read line by line:
String line;
while ((line = testProcessReader.readLine()) != null) {
System.out.println(line);
}
The assumption here is that Test.exe is flushing its output, which is required by any read from the Java side. You can flush the output from C by calling fflush(stdout) after every call to printf().
If you don't flush, the data only lives in a buffer. When considering performance, it's a trade-off, how often you want the data to be written vs. how many writes / flush operations you want to save. If performance is critical, you can consider looking into a more efficient inter-process communication mechanism to pass data between the processes instead of stdout. Since you are on Windows, the first step might be to take a look at the Microsoft IPC help page.
Seems to have something to do with not flushing. I guess it's on both sides - The C library you use seems to only automatically flush output when writing to a terminal. Flush manually after calling printf.
On the Java side, try reading from a non-buffered stream.