I use Runtime.getRuntime().exec to execute command tail -f filename | grep str which is based on OS pipe. I managed to achieve my business logic. But still there is a problem I must solve:
When using pipe, the Process will for another process for tail command:
$ ps -ef | grep test.log
admin 6953 32721 0 16:32 ? 00:00:00 /bin/sh -c tail -f /home/admin
/test.log | unbuffer -p grep '1444379575648'
admin 6957 6953 0 16:32 ? 00:00:00 tail -f /home/admin/test.log
Process.destroy() method destroys itself(pid:6953) only.How can I destroy its subprocess(pid:6957) in my Java program?
Don't use Runtime.getRuntime().exec(...), use 2 ProcessBuilders to explicitly build the individual processes, and connect their inputs and outputs together to do the equivalent of the piping.
Then you will a separate Process object for each, and can kill them as you please.
i found another way:
public static final String getPid() {
try {
RuntimeMXBean runtimeBean = ManagementFactory.getRuntimeMXBean();
String name = runtimeBean.getName();
int k = name.indexOf('#');
if (k > 0)
return name.substring(0, k);
} catch (Exception ex) {
}
return null;
}
This works on linux,win,mac.
Related
My goal is to get the names of x .conf files in a directory called "conf". My code looks like this:
commandOutput = sshCommand remote: getRemote(), command: getPathCmd(type) + "cd conf; ls | grep '.conf' | grep -Eo '^[a-zA-Z0-9_-]+'"
def instances = commandOutput.split("\n") as String[]
return instances
If i print the variable commandOutput (with two files in the directory) it shows me:
name1
name2
But when i make a for loop to print every slot of the array, i get this:
name1
*empty line*
and
name2
I checked commandOutput[0] with .endsWith("\n") and returned false so i dont know where the new line is coming from.
Is there an obvious reason for that behaviour?
Edit: It also happens here:
command = getPathCmd(type) + getScript(type, "status") + "| grep -Eo '[0-9]{4,5}' | sort -u | grep -v '${params.xyID}\\|" + globalID + "'"
try{
commandOutput = sshCommand remote: getRemote(), command: command
def PIDs = commandOutput.split("\n") as String[]
return PIDs
}
catch(Exception e){
println("XY")
return "XY"
}
Print of commandOutput:
1234
5678
For loop print of PIDs:
1234
*empty line*
and
5678
stringXY = stringXY.replaceAll("[\\\r\\\n]+",""); did the trick. It replaces all new lines.
tr '\n' '\0' as suggested in this thread wouldn't work here, because it would be harder to find out where one name ends and the next one starts.
I have this R script:
palindrome <- function(p) {
for(i in 1:floor(nchar(p)/2) ) {
r <- nchar(p) - i + 1
if ( substr(p, i, i) != substr(p, r, r) ) return(FALSE)
}
return(TRUE)
}
that I am calling from Java using the following code:
connection.serverSource("C:\\Users\\x\\Desktop\\R Script\\Palindrome.R");
the connection is of type RConnection is created as follows:
public void startConnection() {
PATH_TO_R = SystemUtils.IS_OS_UNIX ? "R" : "C:\\Program Files\\R\\R-3.6.1\\bin\\x64\\R.exe";
try {
String cmd = PATH_TO_R + " -e " + "\"library(Rserve);Rserve(port=" + 6311+ ")\"";
Runtime.getRuntime().exec(cmd);
RConnection connection = new RConnection("localhost", 6311);
} catch (IOException | RserveException e) {
e.printStackTrace();
}
}
The problem is I am having this error thrown and have no idea how to fix it:
org.rosuda.REngine.Rserve.RserveException: serverSource failed, request status: control pipe to master process is closed/broken
any help will be much appreciated!
The function serverSource (just like serverEval and serverShutdown) directly works on the main R+Rserve process, this feature is off by default. You can enable it in /etc/Rserv.conf adding the line control enabled (config reference).
If you don't already use an Rserve config file, you can simply create it in the default location /etc/Rserv.conf (linux/macOS) or create it anywhere (also windows) and pass the location as parameter when starting Rserve: R CMD Rserve --RS-conf <your path here> (command line arguments at the bottom of the page).
Issue:
When executing the following command through Runtime.exec(...), it fails with an unexpected EOF while looking for a matching quote character.
One oddity is that the error message has a grave character followed by two single quotes.
However, when I execute the command that prints out in the logs through putty, it works fine.
Command:
bin/sh -c 'ps -eo uname,pid,ppid,nlwp,pcpu,pmem,psr,start_time,tty,time,args | fgrep IAAPC | fgrep /f1/f2/a00-a/f3/server/server_1/env_1/javadriver | fgrep -v fgrep'
Resulting error:
-eo: -c: line 0: unexpected EOF while looking for matching `''
-eo: -c: line 1: syntax error: unexpected end of file
Java Code (Java 1.6 ... Don't Judge):
String driverHome = trimToEmpty(System.getProperty("batchdriver.home"));
String cmd = "/bin/sh -c 'ps -eo uname,pid,ppid,nlwp,pcpu,pmem,psr,start_time,tty,time,args | fgrep "+jobName+" | fgrep "+driverHome+" | fgrep -v fgrep'";
String out = null, err = null;
Process proc = null;
try {
proc = Runtime.getRuntime().exec(cmd);
out = fullyRead(proc.getInputStream());
err = fullyRead(proc.getErrorStream());
int exitVal = proc.waitFor();
if(logger.isDebugEnabled()) {
logger.debug("Process Information: "+out);
}
if (isNotEmpty(err)) {
logger.error(failedCommandMessage(cmd, out, err));
this.processId = null;
this.processDesc = PROCESS_NOT_FOUND;
return;
}
String[] processes = StringUtils.split(out, "\r?\n");
if (processes == null || processes.length == 0) {
this.processDesc = PROCESS_NOT_FOUND;
}
else if (processes.length == 1) {
String[] processInfo = processes[0].split("\\s+");
this.processId = processInfo[1];
if (!isNumeric(this.processId)) {
this.processId = null;
}
this.processDesc = out;
}
else {
this.processDesc = out;
}
if (logger.isDebugEnabled()) {
logger.debug("Call to the OS completed with exit value: " + exitVal);
}
} catch (Exception e) {
try {out = fullyRead(proc.getInputStream());} catch (Exception e1) {}
try {err = fullyRead(proc.getErrorStream());} catch (Exception e1) {}
this.processId = null;
this.processDesc = PROCESS_NOT_FOUND;
logger.error(failedCommandMessage(cmd, out, err), e);
}
Related but not quite dupe: Pass a string with multiple contiguous spaces as a parameter to a jar file using Windows command prompt called from a java program
The Runtime.exec methods that take a String break it into tokens at whitespace only so this actually runs the program /bin/sh (a shell) with the following arguments:
-c
'ps
-eo
uname,pid,ppid,nlwp,pcpu,pmem,psr,start_time,tty,time,args
|
fgrep
...
The shell interprets these arguments like this:
-c 'ps -- the script to run consists of the apostrophe character, p, s (and nothing more)
-eo -- the name of the command being run is -eo
uname,pid,.... -- the first argument to the script is this
| -- the second argument to the script is this
fgrep -- the third argument to the script is this
...
-- but the script ignores the arguments and doesn't use them
Thus you get
-eo: -c: unexpected EOF while looking for matching `''
# the script named -eo, with the option -c having value 'ps,
# tried to find a closing ' to match the opening ' and it's not there
This shell is apparently (GNU) bash; many GNU programs that put a data string in an error message surround it by backquote and apostrophe because these were sort of matching quotes in one interpretation of ASCII popular decades ago.
Instead use the String[] overload of exec to give the shell the two arguments that it gets when your above command line is parsed by a shell instead of StringTokenizer:
String[] cmdary = {"/bin/sh", "-c", "ps -eo stuff | fgrep this | fgrep that | fgrep -v fgrep"};
... Runtime.getRuntime().exec(cmdary);
But instead of running three fgrep's, you could just run the ps and read the inputstream as lines and test them in Java using String.contains or similar. Also most of the columns you ask ps for will never be used for either your matching nor result, so that's just a waste of effort and clutter.
In my embedded Selenium/PhantomJSDriver driver it seems resources are not being cleaned up. Running the client synchronously causes millions of open files and eventually throws a "Too many files open" type exception.
Here is some output I gathered from lsof while the program is running for ~1 minute
$ lsof | awk '{ print $2; }' | uniq -c | sort -rn | head
1221966 12180
34790 29773
31260 12138
20955 8414
17940 10343
16665 32332
9512 27713
7275 19226
5496 7153
5040 14065
$ lsof -p 12180 | awk '{ print $2; }' | uniq -c | sort -rn | head
2859 12180
1 PID
$ lsof -p 12180 -Fn | sort -rn | uniq -c | sort -rn | head
1124 npipe
536 nanon_inode
4 nsocket
3 n/opt/jdk/jdk1.8.0_60/jre/lib/jce.jar
3 n/opt/jdk/jdk1.8.0_60/jre/lib/charsets.jar
3 n/dev/urandom
3 n/dev/random
3 n/dev/pts/20
2 n/usr/share/sbt-launcher-packaging/bin/sbt-launch.jar
2 n/usr/share/java/jayatana.jar
I don't understand why using the -p flag on lsof has a smaller result set. But it appears most of the entries are pipe and anon_inode.
The client is very simple at ~100 lines, and at the end of usage calls driver.close() and driver.quit(). I experimented with caching and reusing clients but it did not alleviate the open files
case class HeadlessClient(
country: String,
userAgent: String,
inheritSessionId: Option[Int] = None
) {
protected var numberOfRequests: Int = 0
protected val proxySessionId: Int = inheritSessionId.getOrElse(new Random().nextInt(Integer.MAX_VALUE))
protected val address = InetAddress.getByName("proxy.domain.com")
protected val host = address.getHostAddress
protected val login: String = HeadlessClient.username + proxySessionId
protected val windowSize = new org.openqa.selenium.Dimension(375, 667)
protected val (mobProxy, seleniumProxy) = {
val proxy = new BrowserMobProxyServer()
proxy.setTrustAllServers(true)
proxy.setChainedProxy(new InetSocketAddress(host, HeadlessClient.port))
proxy.chainedProxyAuthorization(login, HeadlessClient.password, AuthType.BASIC)
proxy.addLastHttpFilterFactory(new HttpFiltersSourceAdapter() {
override def filterRequest(originalRequest: HttpRequest): HttpFilters = {
new HttpFiltersAdapter(originalRequest) {
override def proxyToServerRequest(httpObject: HttpObject): io.netty.handler.codec.http.HttpResponse = {
httpObject match {
case req: HttpRequest => req.headers().remove(HttpHeaders.Names.VIA)
case _ =>
}
null
}
}
}
})
proxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT)
proxy.start(0)
val seleniumProxy = ClientUtil.createSeleniumProxy(proxy)
(proxy, seleniumProxy)
}
protected val driver: PhantomJSDriver = {
val capabilities: DesiredCapabilities = DesiredCapabilities.chrome()
val cliArgsCap = new util.ArrayList[String]
cliArgsCap.add("--webdriver-loglevel=NONE")
cliArgsCap.add("--ignore-ssl-errors=yes")
cliArgsCap.add("--load-images=no")
capabilities.setCapability(CapabilityType.PROXY, seleniumProxy)
capabilities.setCapability("phantomjs.page.customHeaders.Referer", "")
capabilities.setCapability("phantomjs.page.settings.userAgent", userAgent)
capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS, cliArgsCap)
new PhantomJSDriver(capabilities)
}
driver.executePhantomJS(
"""
|var navigation = [];
|
|this.onNavigationRequested = function(url, type, willNavigate, main) {
| navigation.push(url)
| console.log('Trying to navigate to: ' + url);
|}
|
|this.onResourceRequested = function(request, net) {
| console.log("Requesting " + request.url);
| if (! (navigation.indexOf(request.url) > -1)) {
| console.log("Aborting " + request.url)
| net.abort();
| }
|};
""".stripMargin
)
driver.manage().window().setSize(windowSize)
def follow(url: String)(implicit ec: ExecutionContext): List[HarEntry] = {
try{
Await.result(Future{
mobProxy.newHar(url)
driver.get(url)
val entries = mobProxy.getHar.getLog.getEntries.asScala.toList
shutdown()
entries
}, 45.seconds)
} catch {
case e: Exception =>
try {
shutdown()
} catch {
case shutdown: Exception =>
throw new Exception(s"Error ${shutdown.getMessage} cleaning up after Exception: ${e.getMessage}")
}
throw e
}
}
def shutdown() = {
driver.close()
driver.quit()
}
}
I tried several versions of Selenium in case there was a bugfix. The build.sbt:
libraryDependencies += "org.seleniumhq.selenium" % "selenium-java" % "3.0.1"
libraryDependencies += "net.lightbody.bmp" % "browsermob-core" % "2.1.2"
Also, I tried PhantomJS 2.0.1, and 2.1.1:
$ phantomjs --version
2.0.1-development
$ phantomjs --version
2.1.1
Is this a PhantomJS or Selenium problem? Is my client using the API improperly?
The resource usage is caused by BrowserMob. To close the proxy and clean-up its resources, one must call stop().
For this client that means modifying the shutdown method
def shutdown() = {
mobProxy.stop()
driver.close()
driver.quit()
}
Another method, abort, offers immediate termination of the proxy server and does not wait for traffic to cease.
In my opinion it seems a problem of PhantomJS. You can try the following alternatives:
Use phantomjs 2.5.0-beta. It has been recently released. I'm not sure if this upgrade solves your problem, but at least it is worth to give a try. According to the changelog, the new features of this version are:
Upgrade QtWebKit to QtWebKitNG
Upgraded Qt to 5.7.1
Clean the phantomjs processes after closing webdriver. You can implement your own cleaner to force that phantomjs is actually closed after driver.close() (invoking killall -9 phantomjs or similar).
I tried to ban an ip-adress temporarly if someone tryes to not join over the proxy to the minecraft server.
String st = e.getRealAddress() + "";
if(!st.startsWith("/x.x.x.x")) {
Runtime runtime = Runtime.getRuntime();
runtime.exec("iptables -A INPUT -s " + st.replace("/","") + " -j DROP");
runtime.exec("sleep 180 ; iptables -D INPUT -s " + st.replace("/","") + " -j DROP");
}
But for some reason to only executes the -A command and after I've waited three minutes I wasn't able to connect. What did I have done wrong?
you can:
decompose yourself the orders in java: iptables , wait, sleep, iptables, ...
then you must wait first iptables to finish:
use waitFor :
Process p = Runtime.getRuntime().exec("your command iptables");
p.waitFor();