I'm writing a program that executes another Java program by using the Class and Method classes to invoke the main method. This other program then tries to read from System.in. In order to pass arguments to the program, I set System.in to a PipedInputStream that is connected to a PipedOutputStream. I pass the arguments the other program requests to the PipedOutputStream, then invoke the main method.
However, as soon as the method is invoked, the program deadlocks. Why is that? Theoretically, the other program should have access to the arguments, since they're already available in the PipedInputStream.
I can't change the way the other program reads the input, so this solution wouldn't work.
Here some example code:
The part where I assign the PipedStreams
PipedInputStream inputStream = new PipedInputStream();
PipedStringOutputStream stringStream = new PipedStringOutputStream(); // custom class
try {
stringStream.connect(inputStream);
} catch (IOException e2) {
e2.printStackTrace();
}
System.setIn(inputStream);
The part where I invoke the method:
Class main = classes.get(className);
try {
Method m = main.getMethod("main", String[].class);
// write all parameters to System.in
String[] params = getParams(); // custom method, works (params is not empty)
for(int j = 0; j < params.length; j++) {
stringStream.write(params[j]);
}
params = null;
m.invoke(null, (Object) params); // this is were the program stops
} catch(Exception e) {}
The PipedStringOutputStream class:
public class PipedStringOutputStream extends PipedOutputStream {
public void write(String output) throws IOException {
this.write((output + "\n").getBytes());
flush();
}
}
My test program that reads from System.in:
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
while(sc.hasNext()) {
System.out.println(sc.nextLine());
}
}
So what is the problem? Do I have to start the Streams in Threads? Why doesn't the other program read the input from the PipedInputStream?
The javadoc of PipedInputStream explicitely says:
Typically, data is read from a PipedInputStream object by one thread and data is written to the corresponding PipedOutputStream by some other thread. Attempting to use both objects from a single thread is not recommended, as it may deadlock the thread.
(emphasis mine)
Write your input to a byte array using a ByteArrayOutputStream. Then construct a ByteArrayInputStream from this byte array, and set System.in to this ByteArrayInputStream.
Related
I have a piece of code
...
InputStream inputStream = new BufferedInputStream(new ByteArrayInputStream("test".getBytes()));
...
and this line makes string "test" an input for an InputStream, however this is a static InputStream.
is there any way without a Scanner, System.in or user external input to make this InputStream dynamic
what I need is something like this
...
InputStream inputStream = new BufferedInputStream(new
ByteArrayInputStream(generateContinuousDynamicString().getBytes()));
// So, basically input stream will be blocked until generateContinuousDynamicString()
// returns a result?
...
I've tried something like this
private static byte[] generateContinuousDynamicString(String s) {
String t = "";
// here comes the realization
// that the source for an input stream
// cannot be generated dynamically on the
// fly it only can be read from already
// existing (fully generated and available
// resource). Am I right? Otherwise how
// can I adjust this method in such a way that
// input stream would continuously have a new
// string to read from?
for (int i = 0; i < 1000; i++){
t += "<str>"+s+i+"</str>";
}
return ("<test>"+t+"</test>").getBytes();
}
So, if we have
...
InputStream inputStream = new BufferedInputStream(readFromADatabaseStream());
...
This is also not dynamic input stream as a resource is already in a database.
You want a pipe. Specifically, you want one of the following pairs of classes:
PipedInputStream and PipedOutputStream
PipedReader and PipedWriter
Your question asks for an InputStream, but since you’re dealing with text, you probably should use a Reader, which is intended for characters. In particular, note that getBytes() will return different values on Windows systems compared to non-Windows systems, for any String with non-ASCII characters. Using a Reader and Writer will remove the need to worry about that.
Either way, the approach is the same: create the readable end of the pipe, then create and feed the writable end of the pipe in another thread.
Using a PipedReader and PipedWriter:
PipedReader pipedReader = new PipedReader();
Reader reader = new BufferedReader(pipedReader);
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<?> pipeFeeder = executor.submit(
() -> generateContinuousDynamicString(pipedReader));
// ...
private Void generateContinuousDynamicString(PipedReader pipedReader)
throws IOException {
try (Writer writer = new PipedWriter(pipedReader)) {
writer.write("<test>");
for (int i = 0; i < 1000; i++) {
writer.write("<str>" + i + "</str>");
}
writer.write("</test>");
}
return null;
}
Using a PipedInputStream and PipedOutputStream:
PipedInputStream pipedInputStream = new PipedInputStream();
InputStream inputStream = new BufferedInputStream(pipedInputStream);
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<?> pipeFeeder = executor.submit(
() -> generateContinuousDynamicString(pipedInputStream));
// ...
private Void generateContinuousDynamicString(PipedInputStream pipedInputStream)
throws IOException {
Charset charset = StandardCharsets.UTF_8;
try (Writer writer = new OutputStreamWriter(
new PipedInputStream(pipedinputStream),
StandardCharsets.UTF_8)) {
writer.write("<test>");
for (int i = 0; i < 1000; i++) {
writer.write("<str>" + i + "</str>");
}
writer.write("</test>");
}
return null;
}
Sure. But you have a bit of an issue: Whatever code is generating the endless stream of dynamic data cannot just be in the method that 'returns the inputstream' just by itself, that's what your realisation is about.
You have two major options:
Threads
Instead, you could fire off a thread which is continually generating data. Note that whatever it 'generates' needs to be cached; this is not a good fit if, say, you want to dynamically generate an inputstream that just serves up an endless amount of 0 bytes, for example. It's a good fit if the data is coming from, say, a USB connected arduino that from time to time sends information about a temperature sensor that it's connected to. Note that you need the thread to store the data it receives someplace, and then have an inputstream that will 'pull' from this queue of data you're making. To make an inputstream that pulls from a queue, see the next section. As this will involve threads, use something from java.util.concurrent, such as ArrayBlockingQueue - this has the double benefit that you won't get infinite buffers, either (the act of putting something in the buffer will block if the buffer is full).
subclassing
What you can also do is take the code that can generate new values, but, put it in an envelope - a thing you can pass around. You want to make some code, but not run it - you want to run that later, when the thing you hand the inputstream to, calls .read().
One easy way to do that, is to extend InputStream - and then implement your own zero method. Looks something like this:
class InfiniteZeroesInputStream extends InputStream {
public int read() {
return 0;
}
}
It's that simple. Given:
try (InputStream in = new InfiniteZeroesInputStream()) {
in.read(); // returns 0.. and will always do so.
byte[] b = new byte[65536];
in.read(b); // fills the whole array with zeroes.
}
I need to dialogue with an external c++ console program (read output and write input). I read from the application with a Thread (and it works), but when it needs input, it works only the first time, then the stream probably remains empty, and it doesn't receive the second input (and external program closes).
The application i'm using is a simple .exe wrote in c++ that:
print "Insert first input"
scan input1
print input1
print "Insert second input"
scan input2
print input2
Main class:
import java.io.*;
import java.util.Scanner;
public class ExampleCom {
public static Communication com = new Communication();
public static void main(String[] args)
{
Scanner in = new Scanner(System.in);
String s;
com.read();
while(true)
{
s = in.nextLine();
com.write(s);
}
}
Communication class:
public class Communication
{
Process p;
OutputStream writer;
public InputStream reader = null;
Read r; //Class that with a loop read all exe input
Communication()
{
try{
p = Runtime.getRuntime ().exec ("C:\\esempio.exe");
writer = p.getOutputStream();
reader = p.getInputStream();
}catch(Exception e){}
}
public void read()
{
r = new Read();
Thread threadRead = new Thread(r);
threadRead.start();
}
public void write(String s)
{
try{
writer.write(s.getBytes());
writer.flush();
writer.close();
}catch(Exception e){}
}
}
How can I send my string (like "writer.write('hello')") when the external application needs it?
The problem is that in your write() method, you have the line
writer.close();
which means that after calling it the first time, you are closing the input stream to your C++. As far as it is concerned, it sees the "end of file" marker after your first input.
What you should do is put the close() in a separate method, and call that method only when you are done working with that process.
Now, as your target program expects text input and will only interpret the input if it gets an end-of-line (as per your answer to the question in my comment), you should supply that end-of-line to it.
Instead of doing raw byte-writes, I think a better approach would be to use a PrintWriter for that output stream, and use as naturally as you use System.out.println(). It can also save you on the flush() part.
You are interpreting it incorrectly when you see that your program is not reading the input until you close(). It's not waiting - it sends it as soon as you call flush(). But the C++ waits for either an end-of-file or an end-of-line, and since you are not giving it an end-of-line, then only close(), that sends it end-of-file, causes it to accept the input. But then you can no longer send any further data.
So the solution is, first, to define your writer as a PrintWriter. Instead of
OutputStream writer;
Use
PrintWriter writer;
And instead of
writer = p.getOutputStream();
Use
writer = new PrintWriter(p.getOutputStream(), true);
The true there will give you auto-flush whenever you use the println() command.
Now, your write method should be:
public void write(String s)
{
writer.println(s);
}
Note that a PrintWriter doesn't produce exceptions, so if you care about errors, you have to check for them using checkError().
And of course, have the close() in a separate method, as I mentioned before.
Because the write() method might throw an IOException, it is advisable to call the close() method inside a finally block.Place the writer.close() method outside the try clause:
finally {
if(writer != null) {
writer.close();
}
I'm writing a program that needs to read lines from a very large file (400K+ lines) and send the data in each line on to a web service. I decided to try threading and am seeing some behavior I did not expect, it appears like my BufferedReader starts reusing lines it's already given me when I call readline() on it.
My program is made up of two classes. A "Main" class that kicks off the threads and holds a static reference to the BufferedReader and has a static sychronized "readNextLine()" method that the threads can use to basically call readLine() on the BufferedReder. And the "Runnable" class that calls readNextLine() and makes a webservice call with the data from each readNextLine() call. I made the BufferedReader and readNextLine() static just because that's the only way I could think of for the threads to share the reader aside from passing an instance of my main class into the threads, I wasn't sure which was better.
After about 5 minutes, I start seeing errors in my web service saying that it's processing a line it's already processed. I'm able to verify lines are indeed being sent multiple times, minutes apart.
Does anyone have any ideas as to why the BufferedReader seems to be giving the threads lines it already read? I was under the impression readline() was sequential and all I needed to do was make sure the calls to readline() were synchronized.
I'll show some of the Main class code below. The runnable is essentially a while loop that calls readNextLine() and processes each line until there are no more lines left.
Main class:
//showing reader and thread creation
inputStream = sftp.get(path to file);
reader = new BufferedReader(new InputStreamReader(inputStream));
ExecutorService executor = Executors.newFixedThreadPool(threads);
Collection<Future> futures = new ArrayList<Future>();
for(int i=0;i<threads;i++){
MyRunnable runnable = new MyRunnable(i);
futures.add(executor.submit(runnable));
}
LOGGER.debug("futures.get()");
for(Future f:futures){
f.get(); //use to wait until all threads are done
}
public synchronized static String readNextLine(){
String results = null;
try{
if(reader!=null){
results = reader.readLine();
}
}catch(Exception e){
LOGGER.error("Error reading from file");
}
return results;
}
I'm testing what you said, but I found you get an error logic in your readNextLine() method, how can reader.readLine() be invoked as the results is null and the if condition is it is not null?
Now I finished my demo, and it seems it works well, the following is the demo, no re-read line happened:
static BufferedReader reader;
public static void main(String[] args) throws FileNotFoundException, ExecutionException, InterruptedException {
reader = new BufferedReader(new FileReader("test.txt"));
ExecutorService service = Executors.newFixedThreadPool(3);
List<Future> results = new ArrayList<Future>();
for (int i = 0; i < 3; i++) {
results.add(service.submit(new Runnable() {
#Override
public void run() {
try {
String line = null;
while ((line = readNextLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
}
}));
}
}
public synchronized static String readNextLine() throws IOException {
return reader.readLine();
}
I have this simple code:
public class Example {
public Example() {
Scanner scanner = new Scanner(System.in);
int row = scanner.nextInt(); // exception at this line
scanner.close();
}
public static void main(String[] args) {
Example ex1 = new Example(); // this line successfully operate
Example ex2 = new Example(); // exception : no such element exception at above line
}
}
I don't know why I always receive this Exception, when code run to ex2.
The problem is because you close the Scanner which in turn closes the underlying InputStream (in this case stdin). When you try to use stdin in again the Scanner is unable to retrieve any data because stdin has been closed.
If running directly from the commandlne then the correct way to provide access to stdin is to use the Console class. The console class provides a Reader wrapped around stdin that has a no-op close method. eg.
public class Example {
public Example() {
Scanner scanner = new Scanner(System.console().reader());
// note change on above line
int row = scanner.nextInt();
scanner.close();
}
}
Note, if you access stdin other than via the Console class then you'll likely cause problems for yourself. And if you invoke your java program other than directly from the command line then you will not get access to the console. For instance, the following will invokations cause problems.
echo 2 3 | java Example
or
Process p = new ProcessBuilder("java", "Example").start();
// write data to process
You Should add if(Scanner.hasNext()) before invoking scanner.nextInt();
You have the exception because no int found to be read.
I have a function "a()" that calls another function "b()" that writes to stdout. I cannot modify "b()", but I want to be able to read what "b" is writing and write back to stdout for "b" to read, meaning:
public void a() {
// start a thread that listens to stdout.
// the thread should print a name to stdout after "b" print "Please enter your name"
b();
}
public void b() { // I cannot modify this function
System.out.println("Welcome! The time is " + System.currentTimeMillis());
System.out.println("Please enter your name");
String name = ...
// ... b reads here the name that the thread from function a() will write
// ...
System.out.println("This is the name that was entered: " + name);
}
I thought about starting "b" in a new process but I wasn't sure how unless I wrap "b" in a main function and run it using a command line - I'd be happy for suggestions.
If it's not a process, I'm not sure how to implement the thread that will be activated by "a()".
I tried using:
BufferedReader stdin = new BufferedReader(new InputStreamReader(System.in));
String line;
while ((line = stdin.readLine()) != null) {
...
}
but it doesn't catch what "b" is writing.
Thanks for the help
You can run b() in another process but you don't need to do so.
System.out is a PrintStream. If you read the javadoc carefully you will notice System.setOut method. With it you can replace System.out with another PrintStream.
Example (not tested):
PrintStream originalOut = System.out; // To get it back later
ByteArrayOutputStream baos = new ByteArrayOutputStream();
PrintStream newOut = new PrintStream(baos);
System.setOut(newOut);
b();
System.out.flush();
System.setOut(originalOut); // So you can print again
ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
// Now you can read from bais what b() wrote to System.out
This solution has the problem of not being thread safe. If any other thread write to System.out when it is 'changed' the output will get redirected too. To get rid of this problem I think you need to run b() on another JVM or use a PrintStream that split (deMux) the output depending on the thread or context.
Unfortunately there is not easy way of doing this in Java. The biggest problem is that System.out and System.in are two separate files, created from FileDescriptor.out and FileDescriptor.in respectively. They are not connected in any way, and therefore you can't write to System.out and expect to see it in System.in.
Your options are:
Run b() in external process somehow. Yes, you'll need to put it in a class with main() function and do lots of complicated process setup, like getting the path to java.exe and setting up classpaths etc. The good part is that writing to and reading from the process will work as you expect.
Create two custom Input and Output streams that can duplicate all traffic to another in/out stream, as well as sending it to System.{in,out}, and set them using System.set{In,Out}. This way you can monitor those streams without affecting other code that might by using System.{in,out}.
As an example of the custom OutputStream mentioned in 2, try something like this:
class CopyOutputStream extends OutputStream {
private final OutputStream str1;
private final OutputStream str2;
public CopyOutputStream(OutputStream str1, OutputStream str2) {
this.str1 = str1;
this.str2 = str2;
}
#Override
public void write(int b) throws IOException {
str1.write(b);
str2.write(b);
}
// ...
#Override
public void close() throws IOException {
try {
str1.close();
} finally {
str2.close();
}
}
}
// then in a() do
public void a(){
//Create a pipe to capture data written to System.out
final PipedInputStream pipeIn = new PipedInputStream();
final PipedOutputStream pipeOut = new PipedOutputStream(pipeIn);
OutputStream out = new CopyOutputStream(System.out, pipeOut);
//From now on everything written to System.out will be sent to
// System.out first and then copied to pipeOut and will be available
// to read from pipeIn.
System.setOut(new PrintStream(out));
// In another thread: read data from System.out
BufferedReader reader = new BufferedReader(new InputStreamReader(pipeIn));
String name = reader.readLine();
}
Unfortunately you will have to repeat the above process for System.in, which means more crazy code, but I don't think it will get easier than this.
If you are ready for some really crazy action, maybe you can get hold of some java library (most likely with native code), that can give you the Process object of the currently running JVM, and then use get{Input,Output}Stream() methods to do the job.
How about setting System.out with System.setOut