I need to monitor console output in Java, I have tried several different ways of retrieving the output as it is streamed, but I fell into a few pitfalls (looping the same output, only one part of output being captured, losing output) and have decided to go about it a different way.
I am developing a Java plugin for a server of the popular game, Minecraft, and I need to be able to monitor console output from other plugins.
I think a good way to do this is by redirecting console output to file, and then set up a recurring async task that checks the file, carries out anything it needs to do, and then clears the file for more input. I think I can do this with a simple use of System.setOut(PrintStream stream); and one of the many guides I can find on Google.
But there is a problem, and that is why I am asking here today. I need console output to stay on the console like normal, hence "mirroring". I cannot edit any of the plugins to output somewhere else, it needs to be from what my Java plugin can do and that only. Sure, maybe on every scheduled check on the file I could reprint it all back to the console but that would result in blocks being printed at a time, which would not be ideal. I do hope this is possible. Thank you!
EDIT: I don't think I explained fully. "plugins"'s classes are run by the program that I am developing for. I cannot change how classes are run and I cannot change how other plugins print to console.
I also do not want to direct console output to files for logging, that idea is only to keep it in a temporary place while it is parsed. Ideally, I need to pass each line of console output to a function (parseString(String line)) which carries out operations depending on the contents of the line. Also, I don't understand how I can read stream contents line-by-line properly, so if someone has any idea on how to, please let me know. :)
Capture System.out before changing it, and use TeeOutputStream:
OutputStream myCaptureStream = ...;
PrintStream original = System.out;
OutptStream outputtee = new TeeOutputStream(originalOut, myCaptureStream);
PrintStream printTee = new PrintStream(outputTee);
System.setOut(printTee);
Convert output stream (myCaptureStream) to an input stream
Use Java standard library PipedOutputStream: new PipedOutputStream; read carefully - you'll need your reader running on another thread.
Convert the input stream to a Reader,
and that to a BufferedReader.
BufferedReader gives you a readLine method.
Pseudo code:
// Feed myCaptureStream to TeeOutputStream
OutputStream myCaptureStream = new PipedOutputStream();
// Prepare to capture data being written to that output stream
InputStream myCaptureAsInputStream = new PipedInputStream(myCaptureStream);
Reader myCaptureReader = new InputStreamReader(myCaptureAsInputStream);
BufferedReader myCaptureBuffered = new BufferedReader(myCaptureReader, 1024);
// This must run on separate reader thread; in spin loop:
myCaptureBuffer.readLine
You can change the System.out and System.err to be references to your own custom PrintStreams. This solution is cross platform, unlike using a platform specific helper executable such as tee.
Documentation for System.setOut(PrintStream)
You can write your own PrintStream Subclass that looks something like:
class MirroringPrintStream extends PrintStream
{
PrintStream first, second;
public MirroringPrintStream(PrintStream first, PrintStream second)
{
// fail now rather than later
if (first == null || second == null) throw new NullPointerException();
this.first = first;
this.second = second;
}
#override
// methods go here (I do believe there are rather a lot of them, but they will all look the same)
}
You can then use System.setOut(new MirroringPrintStream(System.out, myLogStream)) where myLogStream is the PrintStream you opened to your log file. I'm not sure how this will handle appending to a file that something else periodically truncates, you might want to experiment with that a bit.
All else fails, go to irc.esper.net and ask about it in #risucraft
Assuming this is a friendly takeover of console output, why not log to your own stream
and mirror that to the console?
Related
I have a C executable which I can run it from CYGWIN. I also want to run same file from JAVA. The C program gets input and output via stdin and stdout. It mainly gets string and outputs string.
I think I can start the program with process builder successfully. However I can not interact with the C program. To start .exe I use ProcessBuilder, see following.
Process cmd = new ProcessBuilder("path to exe").start();
The main method of my C program is here:
int main(argc, argv)
{
/* set command line or config file parms */
config(argc, argv);
/* read grammar, initialize parser, malloc space, etc */
init_parse(dir, dict_file, grammar_file, frames_file, priority_file);
/* for each utterance */
while( fgets(line, LINE_LEN-1, fp) ) {
/* assign word strings to slots in frames */
parse(line, gram);
/* print parses to buffer */
for(i= 0; i < num_parses; i++ )
print_parse(i, out_ptr, extract, gram);
/* clear parser temps */
reset(num_nets);
}
}
My goal is to send input and get output from Java.
If you only need stdinput/output then you can get the appropriate streams using a ProcessBuilder or some form of System.exec quite easily.
After that just generate output and parse input but beware. The input and output streams generally should be processed in different threads. Otherwise it is very easy to get a deadlock, since most programs won't expect stdin and stdout to be tied to a single process (e.g. the stdout fills your input buffer while you are still trying to write to the stdin stream. Your write is blocked waiting for the program to read more and it won't since its write is blocked waiting for you to read more. Classic.)
Be careful with threads but have fun!
You need to start reading about JNI before going any further. Google is your friend here.
Frankly, your main C method is short. Why don't you want to write this in Java again?
Other good library allow easy acces to native file is JNA. Maybe Runtime class might help you.
I would like to be able to spawn an external process from Java, and periodically write to its input and read the response as if it were the console. Much of the time, however, when I read the process' output, nothing is available. Is there a good practice to do this sort of thing (even avoiding it)?
Here's a stripped down example of what doesn't work:
import org.apache.commons.exec.*;
import java.io.*;
//...
CommandLine cl = CommandLine.parse("/usr/bin/awk {print($1-1)}");
System.out.println(cl.toStrings());
Process proc = new ProcessBuilder(cl.toStrings()).start();
OutputStream os = proc.getOutputStream(); // avoiding *Buffered* classes
InputStream is = proc.getInputStream(); // to lessen buffering complications
os.write(("4" + System.getProperty("line.separator")).getBytes());
os.flush(); // Doesn't seem to flush.
// os.close(); // uncommenting works, but I'd like to keep the process running
System.out.println("reading");
System.out.println(is.read()); // read even one byte? usually is.available() -> 0
Strangely, if I wrap up the OutputStream in a BufferedWriter, the I can read from some processes (cat), but not others (awk, grep).
Generally, the approach taken is proper. Few things though:
InputStream.read() is a blocking method. It waits for input and is not CPU-intensive. You should revolve around it...
More than one byte can be read, just use read(byte[] buffer, int offset, int len)
Wrapping input streams to BufferedInputStream eases the access (readLine() method). This is an alternative to BufferedReader.
Don't forget to use Process.waitFor().
Also, make sure that the external process writes to standard output (and not to standard error). Two possibilities here:
use Process.getErrorStream() (and treat it as another InputStream)
change command line to /usr/bin/awk {print($1-1)} 2>&1. This will redirect standard error to standard output.
Using either JLine (or JLine2), is it possible to issue a call to readline on a ConsoleReader and have, in addition to the standard prompt, the buffer be pre-filled with a string of my choosing?
I have tried to do, e.g.:
reader.getCursorBuffer().write("Default");
reader.readLine("Prompt> ");
This seems to indeed write into the buffer, but the line only displays the prompt. If I press enter, readLine returns "Default" as I would expect. If I clear the screen, the buffer is redrawn and shown correctly.
My understanding is that I should somehow call reader.redrawLine() right after the call to readLine. This last one however is blocking, which makes it hard (not impossible, but it certainly feels wrong to use a second thread for that).
I ran into exactly this use case today.
It's a bit of a hack, but I was able to preload text into the JLine buffer and then let the user edit it by doing this:
String preloadReadLine(ConsoleReader reader, String prompt, String preload)
throws IOException
{
reader.resetPromptLine(prompt, preload, 0);
reader.print("\r");
return reader.readLine(prompt);
}
Yeah, the printing of \r is a hack, but it seems to make the thing work.
I'm using JLine-2.13.
I managed to do that using a thread (yes, it does feel wrong, but I found no other way).
I took inspiration from code found in JLine itself that also uses a thread for similar purposes.
In Scala:
val thr = new Thread() {
override def run() = {
reader.putString("Default")
reader.flush()
// Another way is:
// reader.getCursorBuffer.write("Default") // writes into the buffer without displaying
// out.print("D*f*ult") // here you can choose to display something different
// reader.flush()
}
}
thr.setPriority(Thread.MAX_PRIORITY)
thr.setDaemon(true)
thr.start()
I think you want either resetPromptLine or putStream if you already have the prompt set.
Not to hijack your question but I can't figure out how to simply print a line replacing the prompt (ostensibly or visually pushing the prompt down with a message above it).
Update for JLine3:
This can be accomplished with one of the existing overloads of readLine:
readLine(String prompt, Character mask, String buffer)
For example, reader.readLine("> ", null, "abc") will yield > abc where abc is part of the buffer being edited.
This has been asked before, but was not clarified to the point where I get it. Similar to the one or two other threads I've seen on this subject, I'm working on a chat client with command line inputs for logging in/off, disconnecting, etc. and I am unsure how to simulate this in a JUnit test case. Other responses indicated that I should try changing the System.in to a separate InputStream but...then what?
tl;dr: I have a method in my actual code for parsing command line input, and need a JUnit way of testing that these were entered and appropriately processed.
EDIT: It seems I misunderstood the question. I usually use the term "command line input" to refer to command line arguments given to the process to start with, rather than interactive console input. However...
Handing your real code either a different InputStream or possibly even a Reader or Scanner would indeed help - anything to separate the "getting input" part from the console. You can then fake the input all in one go pretty easily, using a String as input in your test code, and then either converting it to bytes and wrapping those bytes in a ByteArrayInputStream or wrapping the string directly in StringReader.
The downside of this is that there's no easy way of making this "pause" after one command in order to check the results.
You may want to alter the design somewhat so that the part which reads the input is separated from the part which handles the input. The reading part could be a very simple loop, on the order of:
String line;
while ((line = reader.readLine()) != null) {
handleInput(line);
}
You could then potentially leave that part untested by unit tests, or write some relatively primitive tests - but you can then test handleInput extensively, as it's now separated from the input source.
Original answer
If you've extracted the parsing code from the code which really starts the application, it's easy: run that code, and check the results. This will be easiest if you have some sort of class encapsulating the options, of course. For example, your main method might look like this:
public static void main(String[] args) {
Options options = Options.parse(args);
// Use options here
}
Then you can just test Options.parse very easily.
I am trying to develop a class that reads the standard output of an external program(using an instance of Process, Runtime.getRuntime().exec(cmdLine, env, dir)).
The program takes user inputs during the process, and would not proceed until a valid input is given; this seems to be causing a problem in the way I am trying to read its output:
egm.execute(); // run external the program with specified arguments
BufferedInputStream stdout = new BufferedInputStream(egm.getInputStream());
BufferedInputStream stderr = new BufferedInputStream(egm.getErrorStream());
BufferedOutputStream stdin = new BufferedOutputStream(egm.getOutputStream());
int c; //standard output input stream
int e; //standadr error input stream
while((c=stdout.read()) != -1) //<-- the Java class stops here, waiting for input?
{
egm.processStdOutStream((char)c);
}
while((e=stderr.read()) != -1)
{
egm.processStdErrStream((char)e);
}
//...
How can I fix this so that the program takes in a valid input and proceed? Any help resolving this problem will be great!
You have to consume both the program's stdout and stderr concurrently to avoid blocking scenarios.
See this article for more info, and in particular note the StreamGobbler mechanism that captures stdout/err in separate threads. This is essential to prevent blocking and is the source of numerous errors if you don't do it properly!
In this situation you should have separate Threads reading InputStream and ErrStream.
Also you may want to do something like:
public void run() {
while( iShouldStillBeRunning ) {
int c;
while( stdout.available() > 0 && ((c=stdout.read()) != -1)) {
egm.processStdOutStream((char)c);
}
Thread.sleep(100);
}
}
Because you will get blocked on stdout.read() until there is input.
For one thing, this may block if it's writing to the error stream and has exhausted the buffer - you're not reading from the error stream until the output stream has completely finished.
Next, you say it takes user input during the process - are you giving it any user input by writing to stdin? If it's waiting for input, you should write to stdin appropriately, and flush it.
You're not saying in your question what is actually happening when you try to run this. Please update with a detailed description of what happens, and what you would expect to happen instead. Bonus points for telling us what the command is.
Also, is this UNIX/Linux or Windows? If this is UNIX/Linux (or some other POSIX platform), the program may be looking for input on /dev/console instead of /dev/stdin for security reasons.
For the benefit of others looking for solutions to this type of problem I just want to add that I had a very similar problem. But in my case the program was also waiting for a single line of input. So there need to be three threads involved to asynchronously handle all three channels.
Without writing to the output (i.e. the stdin of the executing program) caused the input (i.e. the output from the executing program) not to be captured completely. Writing to it hanged the process.
The solution was the three words by Jon Skeet: "and flush it". After adding the flush, no hang. Thanks!