I am trying to read continuously from a named pipe using java. This question answers it for python/bash.
public class PipeProducer {
private BufferedReader pipeReader;
public PipeProducer(String namedPipe) throws IOException {
this.pipeReader = new BufferedReader(new FileReader(new File(namedPipe)));
}
public void process() {
while ((msg = this.pipeReader.readLine()) != null) {
//Process
}
}
public static void main(String args[]) throws JSONException,IOException {
PipeProducer p = new PipeProducer("/tmp/testpipe");
while(true) {
p.process();
System.out.println("Encountered EOF");
now = new Date();
System.out.println("End : " + now);
}
}
Questions
What happens if there is no data from pipe for some time ?
Can Reader object be reused when EOF is encountered ?
Is EOF is sent by pipe only when it terminate and not otherwise ?
Does pipe guarantees to be alive and working unless something really goes wrong ?
Environment is CentOS 6.7 with Java 7
This is tested and works fine but corner cases needs to be handled so that continuous operation is ensured.
What happens if there is no data from pipe for some time ?
The program blocks until there is data to read or until EOF is detected, just like a Reader connected to any other kind of file.
Can Reader object be reused when EOF is encountered ?
I wouldn't count on it. It would be safer to close the Reader and create a new one in that case.
Is EOF is sent by pipe only when it terminate and not otherwise ?
On Unix, EOF will be received from the pipe after it goes from having one writer to having zero, when no more data are available from it. I am uncertain whether Windows named pipe semantics differ in this regard, but since you're on Linux, that doesn't matter to you.
Does pipe guarantees to be alive and working unless something really goes wrong ?
If the named pipe in fact exists on the file system and you have sufficient permission, then you should reliably be able to open it for reading, but that may block until there is at least one writer. Other than that, I'm not sure what you mean.
What happens if there is no data from pipe for some time?
Nothing. It blocks.
Can Reader object be reused when EOF is encountered?
Reused for what? It's got to the end of the data. The question does not arise.
Is EOF is sent by pipe only when it terminate and not otherwise?
It is sent when the peer closes its end of the pipe.
Does pipe guarantees to be alive and working unless something really goes wrong?
Nothing is guaranteed, in pipes or in life, but in the absence of an error you should continue to read any data that is sent.
Related
I went through the documentation of ProcessBuilder but I am having trouble understanding the difference between
processBuiler.inheritIO()
processBuilder.redirectInput(ProcessBuilder.Redirect.PIPE)
processBuilder.redirectInput(ProcessBuilder.Redirect.INHERIT)
FROM DOC: ProcessBuilder.Redirect PIPE:
Indicates that subprocess I/O will be connected to the current Java process over a pipe. This is the default handling of subprocess standard I/O.
what is a pipe here?
when and where would you use each one of them?
based on the docs what is the diff between 1 and 3 then?
If this question is not fit for stackoverflow, please point me in the right direction as to where to ask this.
PIPE means that the I/O streams of the process will be obtainable via Process#getInputStream, Process#getErrorStream and Process#getOutputStream. This is different from INHERIT, which merges a stream with the stream of the current Java process, that is, System.out, System.in, or System.err.
When you call ProcessBuilder#inheritIO, you are redirecting all I/O streams to the System.
When you call redirectInput(ProcessBuilder.Redirect.PIPE), you are redirecting only the input to the PIPE.
When you call redirectInput(ProcessBuilder.Redirect.INHERIT), you are redirecting only the input to the System.
So, the difference between 1st and 3d is that 3d call redirects only the input of the process, while the 1st one redirects all I/O streams.
You should use INHERIT when you want to use a stream of the Process via the stream of the System.
And you should use PIPE when you want to handle a stream of the Process separately. Consider following example:
public class ProcessTest {
#Test
public void testProcess() throws IOException {
//all streams go to System streams
new ProcessBuilder("java","--version").inheritIO().start();
//all streams are kept separate, and are obtained via .get...Stream() calls
Process process2=new ProcessBuilder("java","--version").start();
BufferedReader bufferedReader=new BufferedReader(new InputStreamReader(process2.getInputStream()));
String s;
while ((s=bufferedReader.readLine())!=null)
{
System.out.println(s);
}
bufferedReader.close();
//only the output of the Process is going to the System.out
new ProcessBuilder("java","--version").redirectOutput(ProcessBuilder.Redirect.INHERIT).start();
}
}
I am writing a java program to read the error stream from a process . Below is the structure of my code --
ProcessBuilder probuilder = new ProcessBuilder( command );
Process process = probuilder.start();
InputStream error = process.getErrorStream();
InputStreamReader isrerror = new InputStreamReader(error);
BufferedReader bre = new BufferedReader(isrerror);
while ((linee = bre.readLine()) != null) {
System.out.println(linee);
}
The above code works fine if anything is actually written to the error stream of the invoked process. However, if anything is not written to the error stream, then the call to readLine actually hangs indefinitely. However, I want to make my code generic so that it works for all scenarios. How can I modify my code to achieve the same.
Regards,
Dev
readline() is a blocking call. It will block until there's a line to be read (terminated by an end of line character) or the underlying stream is closed (returning EOF).
You need to have logic that is checking BufferedReader.ready() or just using BufferedReader.read() and bailing out if you decide you're waiting long enough (or want to do something else then check again).
Edit to add: That being said, it shouldn't hang "indefinitely" as-is; it should return once the invoked process terminates. By any chance is your invoked process also outputting something to stdout? If that's the case ... you need to be reading from that as well or the buffer will fill and will block the external process which will prevent it from exiting which ... leads to your problem.
This is a late reply, but the issue hasn't really solved and it's on the first page for some searches. I had the same issue, and BufferedReader.ready() would still set up a situation where it would lock.
The following workaround will not work if you need to get a persistent stream. However, if you're just running a program and waiting for it to close, this should be fine.
The workaround I'm using is to call ProcessBuilder.redirectError(File). Then I'd read the file and use that to present the error stream to the user. It worked fine, didn't lock. I did call Process.destroyForcibly() after Process.waitFor() but this is likely unnecessary.
Some pseudocode below:
File thisFile = new File("somefile.ext");
ProcessBuilder pb = new ProcessBuilder(yourStringList);
pb.redirectError(thisFile);
Process p = pb.start();
p.waitFor();
p.destroyForcibly();
ArrayList fileContents = getFileContents(thisFile);
I hope this helps with at least some of your use cases.
Something like this might also work and avoid the blocking behaviour (without requiring to create a File)
InputStream error = process.getErrorStream();
// Read from InputStream
for (int k = 0; k < error.available(); ++k)
System.out.println("Error stream = " + error.read());
From the Javadoc of InputStream.available
Returns an estimate of the number of bytes that can be read (orskipped over) from this input stream without blocking by the nextinvocation of a method for this input stream. The next invocationmight be the same thread or another thread. A single read or skip of thismany bytes will not block, but may read or skip fewer bytes.
The simplest answer would be to simply redirect the error stream to stdout:
process.getErrorStream().transferTo(System.out);
I am trying to run a .csh script and read it's output into a StringBuffer.
the output sometime returns empty although running the script from console returns some output. the same running flow can sometimes returns output and sometimes not, although nothing is changed in the way the process starts (same script, path , args) and the script isn't changed as well.
I'm not getting any exceptions thrown.
what might cause output now to be read correctly/successfully ?
the code segment is
public static String getOutpoutScript(Process p) {
InputStream outpout = p.getInputStream();
logger.info("Retrived script output stream");
BufferedReader buf = new BufferedReader(new InputStreamReader(outpout));
String line = "";
StringBuffer write = new StringBuffer();
try {
while ((line = buf.readLine()) != null) {
write.append(line);
}
} catch (IOException e) {
// do something
}
return write.toString().trim();
}
beside the fact not closing the streams is not good, could this or something else in the code might prevent output from being read correctly under some circumstances ?
thanks,
If you launch it with ProcessBuilder, you can combine the error stream into the output stream. This way if the program prints to stderr you'll capture this too. Alternatively you could just read both. Additionally, you may not want to use readLine, you could be stuck for awhile if the program does not print end of line character at the end.
Maybe you must replace p.getInputStream() with p.getOutputStream()
Besides this sometimes processes can block waiting on input, so you must read and write asynchronously - one possible solution is to use different threads - e.g. one thread is reading, other is writing and one that is monitoring the process.
If you have an error, this will write to getErrorStream() by default. If you have a problem, I would ensure you are reading this somewhere.
If the buffer for this stream fills, your program will stop, waiting for you to read it.
A simple way around these issues is to use ProcessBuilder.redirectErrorStream(true)
I'm having trouble transitioning to Java from C/C++ for my "Telnet" interface to some modules we work with here. I want to be able to establish a connection with a card that, after starting it's command line interface, waits for a connection and serves up a prompt ("OK>") to the clients. This works fine for both C and C# clients I've written, but the Java has given me some issues. I've attached some code that I grabbed from some examples online, but so far, all I can ascertain for sure is that the socket is being created.
Code:
private boolean CreateTelnetSession()
{
try
{
_socket = new Socket();
_socket.connect(new InetSocketAddress(_ipAddr, _ipPort));
_socket.setSoTimeout(10000);
_socket.setKeepAlive(true);
_out = new PrintWriter(_socket.getOutputStream(), true);
_in = new BufferedReader(new InputStreamReader(_socket.getInputStream()));
_out.println("\r\n");
System.out.println(_in.readLine());
return true;
}
catch(Exception e)
{
System.out.println("Exception!");
}
return false;
}
The socket SEEMS to be created correctly, and when the program shuts down, I can see the session close on the card(s) I'm trying to talk to, but I don't see the carriage return/line feed echoed on the card as I would expect, or a prompt returned via the InputStream. Is it possible that it's a character encoding issue? Am I doing something incorrectly with the streams (crossing them!?!)? Any insight at all? When I get over this initial learning curve, I would like to acknowledge how easy Java makes these socket reads and writes, but until then...
I read this post:
java simple telnet client using sockets
It seems similar to what I'm running up against, but it's not the same. I'm willing to take the rep hit if someone has seen something on here that resolves my issue, so feel free to let me know, bluntly, what I missed.
Edit:
private boolean CreateTelnetSession()
{
try
{
_socket = new Socket();
_socket.connect(new InetSocketAddress(_ipAddr, _ipPort));
_socket.setSoTimeout(10000);
_socket.setKeepAlive(true);
_out = new DataOutputStream(_socket.getOutputStream());
_in = new DataInputStream(_socket.getInputStream());
_outBuffer = ByteBuffer.allocate(2048);
_outBuffer.order(ByteOrder.LITTLE_ENDIAN);
_inBuffer = ByteBuffer.allocate(2048);
_inBuffer.order(ByteOrder.LITTLE_ENDIAN);
System.out.println("Connection Response: " + _in.read(_inBuffer.array()));
System.out.println("Response: " + WriteCommand("DRS\r\n"));
return true;
}
catch(Exception e)
{
System.out.println("Exception!");
}
return false;
}
private String WriteCommand(String command)
{
try
{
_outBuffer = encoder.encode(CharBuffer.wrap(command));
_out.write(_outBuffer.array());
_out.flush();
_in.read(_inBuffer.array());
String retString = decoder.decode(_inBuffer).toString();
return retString.substring(0, retString.indexOf('>') + 1);
}
catch(Exception e)
{
System.out.println("Exception!");
}
return "E1>";
}
There are many things to clean up and I'm going to experiment with whether I need to do it in quite this way, but this is the gist of the "solution". The big killer was the endian-ness. It should be mentioned, once again, that this is ugly and non-production code, but any other input would still be appreciated.
I have a couple things you can try. You are using a PrintWriter for your output, this is a fairly high-level Writer (i.e. it encapsulates a lot of things from you). My concern is that the println() method in PrintWriter adds a line terminating character(s) at the end automatically (as appropriate for your OS). So what you are really sending is "/r/n(line terminator)" so on a unix box you would be sending "/r/n/n".
I would recommend switching to a DataOutputStream which gives you much more control over the raw bytes that are sent: http://docs.oracle.com/javase/6/docs/api/java/io/DataOutputStream.html
Remember if you switch to DataOutputStream you need to call flush on the output stream.
My other thought is it might be an endianess problem. Java is strictly Big Endian (network byte order). Is it possible your "card" is reading things in little-endian? If you need to write over the network in little endian (if so your card is a bad netizen!) you will need to use a ByteBuffer, set its order to little-endian. Write your bytes to it, then write the bytes from your ByteBuffer to the DataOutputStream.
I would probably switch to a DataInputStream for your input stream too. readline() will only return once the newline character is seen. Is your card returning a newline in its response?
My last thought is that your println methods might have an error and you don't know it because PrintWriter doesn't throw exceptions. The PrintWriter JavaDocs says:
"Methods in this class never throw I/O exceptions, although some of its constructors may. The client may inquire as to whether any errors have occurred by invoking checkError()."
Hopefully something in my long rambling response will help you.
I'm trying to read in a large (700GB) file and incrementally process it, but the network I'm working on will occasionally go down, cutting off access to the file. This throws a java.io.IOException telling me that "The specified network name is no longer available". Is there a way that I can catch this exception and wait for, say, fifteen minues, and then retry the read, or is the Reader object fried once access to the file is lost?
If the Reader is rendered useless once the connection is lost, is there a way that I can rewrite this in such a way as to allow me to "save my place" and then begin my read from there without having to read and discard all the data before it? Even just munching data without processing it takes a long time when there's 500GB of it to get through.
Currently, the code looks something like this (edited for brevity):
class Processor {
BufferedReader br;
Processor(String fname) {
br = new BufferedReader(new FileReader("fname"));
}
void process() {
try {
String line;
while((line=br.readLine)!=null) {
...code for processing the line goes here...
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Thank you for your time.
You can keep track of read bytes in a variable. For example here I keep track in a variable called read, and buff is char[]. Not sure if this is possible using the readLine method.
read+=br.read(buff);
Then if you need to restart, you can skip that many bytes
br.skip(read);
Then you can keep processing away. Good luck
I doubt that the underlying fd will still be usable after this error, but you would have to try it. More probably you will have to reopen the file and skip to where you were up to.