I'm using proc_open in php to call java application, pass it text to be processed and read output text. Java execution time is quite long and I found the reason for that is reading input takes most of the time. I'm not sure whether it's php's or java's fault.
My PHP code:
$process_cmd = "java -Dfile.encoding=UTF-8 -jar test.jar";
$env = NULL;
$options = ["bypass_shell" => true];
$cwd = NULL;
$descriptorspec = [
0 => ["pipe", "r"], //stdin is a pipe that the child will read from
1 => ["pipe", "w"], //stdout is a pipe that the child will write to
2 => ["file", "java.error", "a"]
];
$process = proc_open($process_cmd, $descriptorspec, $pipes, $cwd, $env, $options);
if (is_resource($process)) {
//feeding text to java
fwrite($pipes[0], $input);
fclose($pipes[0]);
//reading output text from java
$output = stream_get_contents($pipes[1]);
fclose($pipes[1]);
$return_value = proc_close($process);
}
My java code:
public static void main(String[] args) throws Exception {
long start;
long end;
start = System.currentTimeMillis();
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
String in;
String input = "";
br = new BufferedReader(new InputStreamReader(System.in));
while ((in = br.readLine()) != null) {
input += in + "\n";
}
end = System.currentTimeMillis();
log("Input: " + Long.toString(end - start) + " ms");
start = System.currentTimeMillis();
org.jsoup.nodes.Document doc = Jsoup.parse(input);
end = System.currentTimeMillis();
log("Parser: " + Long.toString(end - start) + " ms");
start = System.currentTimeMillis();
System.out.print(doc);
end = System.currentTimeMillis();
log("Output: " + Long.toString(end - start) + " ms");
}
I'm passing to java html file of 3800 lines (~200KB in size as a standalone file). These are broken down execution times in the log file:
Input: 1169 ms
Parser: 98 ms
Output: 12 ms
My question is this: why does input take 100 times longer than output? Is there a way to make it faster?
Inspect your read block in the Java program: Try to use a StringBuilder to concat the data (instead of using += on a String):
String in;
StringBuilder input = new StringBulider();
br = new BufferedReader(new InputStreamReader(System.in));
while ((in = br.readLine()) != null) {
input.append(in + "\n");
}
Details are covered here: Why using StringBuilder explicitly
Generally speaking, to make it faster, consider using an application server (or a simple socket based server), to have a permanently running JVM. There is always some overhead when you start a JVM, on top of it the JIT needs some time as well to optimize your code. This effort is lost, after the the JVM exits.
As for the PHP program: Try to feed the Java program from the shell, just use cat to pipe the data (on a UNIX system like Linux). As an alternative, rewrite your Java program to accept a command line parameter for the file as well. Then you can judge, if your PHP code pipes the data fast enough.
As for the Java program: If you do performance analysis, consider the recommendations in How do I write a correct micro-benchmark in Java
Related
I written a program in C++ where multithreading is used. Call this program using a bash script or something like that is no problem, everything is working fine. Now I want to call this program in Java. This is the Java code sample I tried:
Process v2j = new ProcessBuilder("./src/main/cpp/vcd_converter",
"-vcd", vcdfilePath, "-out", logfilePath).start();
InputStream is = v2j.getInputStream();
InputStream es = v2j.getErrorStream();
for(int i = 0; i < is.available(); i++) {
System.out.println("Input: " + br.readLine());
}
for(int i = 0; i < es.available(); i++) {
System.out.println("Error: " + ebr.readLine());
}
v2j.waitFor();
So if I run this code, following will happen: It seems like the waitFor() instruction will not realy wait for "all" threads, maybe it will wait only for one. However, the java program executed with a error because it can't find files which are generated by the C++ program. So the waitFor() don't wait until my c++ program is finished and has generated the files.
The output is really weird too. I only get one output from std::cout. Here's an example:
std::cout << out_folder_path << "\n" << vcd_filename << "\n";
std::cout << std::flush;
This will print only the out_folder_path output. A problem is: In the section of the std::cout I don't even use multi-threading. All other std::cout in the program are don't printed too.
So my questions: How can I wait for really ALL threads? And how can I get the real output of my program?
Your program doesn't get all output from the C++ program because it doesn't read until the input streams are depleted. It jumps to waitFor if the C++ program has not produced any output when your program reaches is.available().
Here's an alternative approach doing reading until the input streams are closed. It uses an InputStreamReader and a BufferedReader and calls the BufferedReader's readline() until the input stream closes.
Disclaimer. I haven't written any java programs before so I'm not sure I've gotten everything correct. The general idea should work though.
var v2j = new ProcessBuilder("./src/main/cpp/vcd_converter", "-vcd",
vcdfilePath, "-out", logfilePath).start();
InputStream is = v2j.getInputStream();
try(var reader = new BufferedReader(new InputStreamReader(is))) {
String line;
while ((line = reader.readLine()) != null) {
System.out.println("Input: " + line);
}
}
InputStream es = v2j.getErrorStream();
try(var reader = new BufferedReader(new InputStreamReader(es))) {
String line;
while ((line = reader.readLine()) != null) {
System.out.println("Error: " + line);
}
}
v2j.waitFor();
I'm using mencoder to split files and I'd like to turn this into an Object Oriented approach, if possible, using Java or similar, for example. But I'm not sure the best way, so I leave it in the open. Here is what I need:
I have an excel file with start times and end times, and I need to extract out the appropriate clips from a video file. In the terminal (I'm on Mac OS X) I've had success using, for example:
mencoder -ss 0 -endpos 10 MyVideo.avi -oac copy -ovc copy -o Output.avi
Which creates the video Output.avi by clipping the first 10 seconds of the video MyVideo.avi.
But, like I said, I want to make it so that a program reads in from an excel file, and calls this mencoder command multiple times (over 100) for each of the start times and end times.
I know how to read in the excel file in Java, but I'm not sure it is best to call this command from Java. Plus, I'd like to be able to see the output of mencoder (because it prints out a nice percentage so you know about how much longer a single command will take). Is this type of thing feasible to do in a shell script? I would really like to use Java if possible, since I have many years of experience in Java and no experience in shell scripting.
UPDATE
Here is what I've tried in Java, but it freezes at in.readLine()
File wd = new File("/bin");
System.out.println(wd);
Process proc = null;
try {
proc = Runtime.getRuntime().exec("/bin/bash", null, wd);
}
catch (IOException e) {
e.printStackTrace();
}
if (proc != null) {
BufferedReader in = new BufferedReader(new InputStreamReader(proc.getInputStream()));
PrintWriter out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(proc.getOutputStream())), true);
out.println("cd ..");
out.println("pwd");
String video = "/Users/MyFolder/MyFile.avi";
String output = "/Users/MyFolder/output.avi";
int start = 0;
int end = 6;
String cmd = "mencoder -ss " + start +
" -endpos " + end +
" " + video + " -oac copy -ovc copy -o " + output;
out.println(cmd);
try {
String line;
System.out.println("top");
while ((line = in.readLine()) != null) {
System.out.println(line);
}
System.out.println("end");
proc.waitFor();
in.close();
out.close();
proc.destroy();
}
catch (Exception e) {
e.printStackTrace();
}
}
I'm not quite sure about mencoders multicore-capabilities, but I think with Java you can use Multiple Threads to get the maximal power of all cpu-cores.
You shouldn't use Runtime like your using it.
When using Runtime, you should not run bash and send commands via inputstream like when you are typing commands on a terminal.
Runtime.getRuntime().exec("mencoder -ss " + start +
" -endpos " + end +
" " + video + " -oac copy -ovc copy -o " + output);
To get the Output, you can use the inputStream
http://docs.oracle.com/javase/1.4.2/docs/api/java/lang/Runtime.html#exec%28java.lang.String,%20java.lang.String[],%20java.io.File%29
With this command you can also set the Workingdirectory where your command is executed.
I also prefer the version with the String[] as parameters. It's much more readable, than the a concatenated String.
hello everyone i created a java file,in that java i written a code for starting,stoping and restarring a windows service ,in that i want to create a log file and write output of window service as a console please if any one knows give suggestion
i used code for stoping service
public static void stopService(String serviceName) throws IOException,
InterruptedException {
String executeCmd = "cmd /c net stop \"" + serviceName + "\"";
Process runtimeProcess = Runtime.getRuntime().exec(executeCmd);
int processComplete = runtimeProcess.waitFor();
System.out.println("processComplete: " + processComplete);
if (processComplete == 1) {// if values equal 1 process failed
System.out.println("Service failed");
}
else if (processComplete == 0) {
System.out.println("Service Success");
}
}
Probably the best way to achieve this is Log4J. If you haven't used it before it can seem overbearing, but there are a lot of good tutorials out there to cherry pick from, should you need to. Put the LogFactory in your class definition and then pop lines like log.info.println("service starting"). The other keyword to keep an eye our for is FileAppender. (Sorry no better references, writing from my phone!)
The best solution is to use some logging API have a look at http://www.vogella.com/articles/Logging/article.html tutorial.
Or the other trivial option is to redirect the output of executing the command in some text File. Change the command
String executeCmd = "cmd /c net stop \" + serviceName + "\"+" >> "+ fileName".
In order to get the output of a command you can do this:
String lineSeparator = System.getProperty("line.separator");
stderr = runtimeProcess.getErrorStream();
stdout = runtimeProcess.getInputStream();
// clean up if any output in stdout
BufferedReader brCleanUp =
new BufferedReader(new InputStreamReader(stdout));
StringBuilder output = new StringBuilder();
String line;
while ((line = brCleanUp.readLine()) != null) {
output.append(line).append(lineSeparator);
}
brCleanUp.close();
// clean up if any output in stderr
brCleanUp =
new BufferedReader(new InputStreamReader(stderr));
StringBuilder error = new StringBuilder();
while ((line = brCleanUp.readLine()) != null) {
error.append(line).append(lineSeparator);
}
brCleanUp.close();
And now in output and error String you have the "standard output" and "standard error" of the process you executed.
First, this is my code :
import java.io.*;
import java.util.Date;
import com.banctecmtl.ca.vlp.shared.exceptions.*;
public class PowershellTest implements Runnable {
public static final String PATH_TO_SCRIPT = "C:\\Scripts\\ScriptTest.ps1";
public static final String SERVER_IP = "XX.XX.XX.XXX";
public static final String MACHINE_TO_MOD = "MachineTest";
/**
* #param args
* #throws OperationException
*/
public static void main(String[] args) throws OperationException {
new PowershellTest().run();
}
public PowershellTest(){}
#Override
public synchronized void run() {
String input = "";
String error = "";
boolean isHanging = false;
try {
Runtime runtime = Runtime.getRuntime();
Process proc = runtime.exec("powershell -file " + PATH_TO_SCRIPT +" "+ SERVER_IP +" "+ MACHINE_TO_MOD);
proc.getOutputStream().close();
InputStream inputstream = proc.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
proc.waitFor();
String line;
while (!isHanging && (line = bufferedreader.readLine()) != null) {
input += (line + "\n");
Date date = new Date();
while(!bufferedreader.ready()){
this.wait(1000);
//if its been more then 1 minute since a line has been read, its hanging.
if(new Date().getTime() - date.getTime() >= 60000){
isHanging = true;
break;
}
}
}
inputstream.close();
inputstream = proc.getErrorStream();
inputstreamreader = new InputStreamReader(inputstream);
bufferedreader = new BufferedReader(inputstreamreader);
isHanging = false;
while (!isHanging && (line = bufferedreader.readLine()) != null) {
error += (line + "\n");
Date date = new Date();
while(!bufferedreader.ready()){
this.wait(1000);
//if its been more then 1 minute since a line has been read, its hanging.
if(new Date().getTime() - date.getTime() >= 60000){
isHanging = true;
break;
}
}
}
inputstream.close();
proc.destroy();
} catch (IOException e) {
//throw new OperationException("File IO problem.", e);
} catch (InterruptedException e) {
//throw new OperationException("Script thread problem.",e);
}
System.out.println("Error : " + error + "\nInput : " + input);
}
}
I'm currently trying to run a powershell script that will start/stop a vm (VMWARE) on a remote server. The script work from command line and so does this code. The thing is, I hate how I have to use a thread (and make it wait for the script to respond, as explained further) for such a job. I had to do it because both BufferedReader.readline() and proc.waitFor() hang forever.
The script, when ran from cmd, is long to execute. it stall for 30 sec to 1 min from validating authentification with the server and executing the actual script. From what I saw from debugging, the readline hang when it start receiving those delays from the script.
I'm also pretty sure it's not a memory problem since I never had any OOM error in any debugging session.
Now I understand that Process.waitFor() requires me to flush the buffer from both the error stream and the regular stream to work and so that's mainly why I don't use it (I need the output to manage VM specific errors, certificates issues, etc.).
I would like to know if someone could explain to me why it hangs and if there is a way to just use the typical readline() without having it to hang so hard. Even if the script should have ended since a while, it still hang (I tried to run both the java application and a cmd command using the exact same thing I use in the java application at the same time, left it runingfor 1h, nothing worked). It is not just stuck in the while loop, the readline() is where the hanging is.
Also this is a test version, nowhere close to the final code, so please spare me the : this should be a constant, this is useless, etc. I will clean the code later. Also the IP is not XX.XX.XX.XXX in my code, obviously.
Either explanation or suggestion on how to fix would be greatly appreciated.
Ho btw here is the script I currently use :
Add-PSSnapin vmware.vimautomation.core
Connect-VIServer -server $args[0]
Start-VM -VM "MachineTest"
If you need more details I will try to give as much as I can.
Thanks in advance for your help!
EDIT : I also previously tested the code with a less demanding script, which job was to get the content of a file and print it. Since no waiting was needed to get the information, the readline() worked well. I'm thus fairly certain that the problem reside on the wait time coming from the script execution.
Also, forgive my errors, English is not my main language.
Thanks in advance for your help!
EDIT2 : Since I cannot answer to my own Question :
Here is my "final" code, after using threads :
import java.io.*;
public class PowershellTest implements Runnable {
public InputStream is;
public PowershellTest(InputStream newIs){
this.is = newIs;
}
#Override
public synchronized void run() {
String input = "";
String error = "";
try {
InputStreamReader inputstreamreader = new InputStreamReader(is);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
String line;
while ((line = bufferedreader.readLine()) != null) {
input += (line + "\n");
}
is.close();
} catch (IOException e) {
//throw new OperationException("File IO problem.", e);
}
System.out.println("Error : " + error + "\nInput : " + input);
}
}
And the main simply create and start 2 thread (PowerShellTest instances), 1 with the errorStream and 1 with the inputStream.
I believe I made a dumb error when I first coded the app and fixed it somehow as I reworked the code over and over. It still take a good 5-6 mins to run, which is somehow similar if not longer than my previous code (which is logical since the errorStream and inputStream get their information sequentially in my case).
Anyway, thanks to all your answer and especially Miserable Variable for his hint on threading.
First, don't call waitFor() until after you've finished reading the streams. I would highly recommend you look at ProcessBuilder instead of simply using Runtime.exec, and split the command up yourself rather than relying on Java to do it for you:
ProcessBuilder pb = new ProcessBuilder("powershell", "-file", PATH_TO_SCRIPT,
SERVER_IP, MACHINE_TO_MOD);
pb.redirectErrorStream(true); // merge stdout and stderr
Process proc = pb.start();
redirectErrorStream merges the error output into the normal output, so you only have to read proc.getInputStream(). You should then be able to just read that stream until EOF, then call proc.waitFor().
You are currently waiting to complete reading from inputStream before starting to read from errorStream. If the process writes to its stderr before stdout maybe you are getting into a deadlock situation.
Try reading from both streams from concurrently running threads. While you are at it, also remove proc.getOutputStream().close();. It shouldn't affect the behavior, but it is not required either.
I want to run a C/C++ program's exe file using java.......and handle its input and output......
my code is
import java.io.*;
class run2 {
public static void main(String[] args) throws java.io.IOException {
String[] command = new String[3];
command[0] = "cmd";
command[1] = "/C";
// command[2] = "java Run1";
command[2] = "start C:\\WE.EXE";
Process p = Runtime.getRuntime().exec(command);
String i = "20";
BufferedReader stdInput = new BufferedReader(new InputStreamReader(
p.getInputStream()));
BufferedWriter st = new BufferedWriter(new OutputStreamWriter(
p.getOutputStream()));
String s = null;
System.out.println("Here is the standard output of the command:\n");
s = stdInput.readLine();
System.out.println(s);
st.write(i);
st.newLine();
st.flush();
while ((s = stdInput.readLine()) != null) {
System.out.println("Stdout: " + s);
}
try {
System.out.println("Exit status = " + p.waitFor());
}
catch (InterruptedException e) {
}
stdInput.close();
}
}
i am getting an error which says pipes is closed
do help me out.....
Well, first of all, if there isn't a WE.EXE in C:/, that could be an issue. If no process is ever launched, of course you can't do anything with its input/output pipes.
However, presuming you have a WE.EXE, your error is probably at:
st.flush();
Your application is opening up WE.EXE in command prompt, or cmd.exe, who will take care of both standard input and standard output. Your call stdInput.readLine(); will wait until WE.EXE, and therefore cmd.exe, terminates, at which point the output stream will be closed (and you obviously can't write onto a closed pipe).
So if you want to handle input and output yourself, you should launch WE.exe directly, like:
Process p = Runtime.getRuntime().exec("C://WE.EXE");
Additionally, you may consider using ProcessBuilder instead of Runtime.exec.
Small detail, but consider using Java's naming conventions--for example, your class name would be Run2 (or something more descriptive) instead of run2.
You are trying to read from a stream (stdInput) that does not exist yet.
It won't exist until the WE.EXE program writes something to it.
Just wait until you send the commands to the program.
In other words, take out the first input line, and it will work fine.
//s = stdInput.readLine();
System.out.println(s);
st.write(i);
st.newLine();
st.flush();
while ((s = stdInput.readLine()) != null)
{ System.out.println("Stdout: " + s); }