We Keep Coding

Issue executing a piped ps unix command through Runtime.exec()

Issue:
When executing the following command through Runtime.exec(...), it fails with an unexpected EOF while looking for a matching quote character.
One oddity is that the error message has a grave character followed by two single quotes.
However, when I execute the command that prints out in the logs through putty, it works fine.
Command:
bin/sh -c 'ps -eo uname,pid,ppid,nlwp,pcpu,pmem,psr,start_time,tty,time,args | fgrep IAAPC | fgrep /f1/f2/a00-a/f3/server/server_1/env_1/javadriver | fgrep -v fgrep'
Resulting error:
-eo: -c: line 0: unexpected EOF while looking for matching `''
-eo: -c: line 1: syntax error: unexpected end of file
Java Code (Java 1.6 ... Don't Judge):
String driverHome = trimToEmpty(System.getProperty("batchdriver.home"));
String cmd = "/bin/sh -c 'ps -eo uname,pid,ppid,nlwp,pcpu,pmem,psr,start_time,tty,time,args | fgrep "+jobName+" | fgrep "+driverHome+" | fgrep -v fgrep'";
String out = null, err = null;
Process proc = null;
try {
proc = Runtime.getRuntime().exec(cmd);
out = fullyRead(proc.getInputStream());
err = fullyRead(proc.getErrorStream());
int exitVal = proc.waitFor();
if(logger.isDebugEnabled()) {
logger.debug("Process Information: "+out);
}
if (isNotEmpty(err)) {
logger.error(failedCommandMessage(cmd, out, err));
this.processId = null;
this.processDesc = PROCESS_NOT_FOUND;
return;
}
String[] processes = StringUtils.split(out, "\r?\n");
if (processes == null || processes.length == 0) {
this.processDesc = PROCESS_NOT_FOUND;
}
else if (processes.length == 1) {
String[] processInfo = processes[0].split("\\s+");
this.processId = processInfo[1];
if (!isNumeric(this.processId)) {
this.processId = null;
}
this.processDesc = out;
}
else {
this.processDesc = out;
}
if (logger.isDebugEnabled()) {
logger.debug("Call to the OS completed with exit value: " + exitVal);
}
} catch (Exception e) {
try {out = fullyRead(proc.getInputStream());} catch (Exception e1) {}
try {err = fullyRead(proc.getErrorStream());} catch (Exception e1) {}
this.processId = null;
this.processDesc = PROCESS_NOT_FOUND;
logger.error(failedCommandMessage(cmd, out, err), e);
}

Related but not quite dupe: Pass a string with multiple contiguous spaces as a parameter to a jar file using Windows command prompt called from a java program
The Runtime.exec methods that take a String break it into tokens at whitespace only so this actually runs the program /bin/sh (a shell) with the following arguments:
-c
'ps
-eo
uname,pid,ppid,nlwp,pcpu,pmem,psr,start_time,tty,time,args
|
fgrep
...
The shell interprets these arguments like this:
-c 'ps -- the script to run consists of the apostrophe character, p, s (and nothing more)
-eo -- the name of the command being run is -eo
uname,pid,.... -- the first argument to the script is this
| -- the second argument to the script is this
fgrep -- the third argument to the script is this
...
-- but the script ignores the arguments and doesn't use them
Thus you get
-eo: -c: unexpected EOF while looking for matching `''
# the script named -eo, with the option -c having value 'ps,
# tried to find a closing ' to match the opening ' and it's not there
This shell is apparently (GNU) bash; many GNU programs that put a data string in an error message surround it by backquote and apostrophe because these were sort of matching quotes in one interpretation of ASCII popular decades ago.
Instead use the String[] overload of exec to give the shell the two arguments that it gets when your above command line is parsed by a shell instead of StringTokenizer:
String[] cmdary = {"/bin/sh", "-c", "ps -eo stuff | fgrep this | fgrep that | fgrep -v fgrep"};
... Runtime.getRuntime().exec(cmdary);
But instead of running three fgrep's, you could just run the ps and read the inputstream as lines and test them in Java using String.contains or similar. Also most of the columns you ask ps for will never be used for either your matching nor result, so that's just a waste of effort and clutter.

java - Writing to a file does not recognize \n and \t as escape characters

I have records in database that contains \n.
NAME | ROW | COL
-------------------------------
name | 1 | 1
address | 1 | 2
tel\n no | 1 | 3
employeed\n id | 1 | 4
Then, I also save my delimiter , for text writing, in Database (\t)
DELIMITER | FILENAME_LOCATIO
----------------------
\t | SAMPLETEXT
Then, when I already want to write those values(first table) to a text file, it prints like this..it does not recognized the escape characters, instead it printed as ordinary text
name\taddress\ttelno\n no\temployeed\n id
Text file should look like this..
name address telno
no employeed
id
Here is the code im using..
FileWriter fw = new FileWriter(config.get("FILENAME_LOCATION").toString(), true);
BufferedWriter w = new BufferedWriter(fw);
List<Map<String, Object>> headers = sampleMapper.selectHeader();
for (Iterator<Map<String, Object>> iterator = headers.iterator(); iterator.hasNext();) {
Map<String, Object> map = (Map<String, Object>) iterator.next();
w.write(map.get("NAME").toString());
w.write(config.get("DELIMITER").toString()); // returns \t in String format
}
w.close();

It might depend on the OS you are running your application on, at least for the line delimiter.
try (BufferedWriter w = new BufferedWriter(new FileWriter("c:\\test.txt", false)))
{
w.write("hello");
w.write("\t");
w.write("hello");
w.write(System.getProperty("line.separator"));
w.write("world");
w.write("\t");
w.write("world");
}
catch (IOException e)
{
e.printStackTrace();
}
This code runs fine on my Windows, as the system property will provide you with the correct delimiter for the machine you're on (on Windows, it is \r\n).
According to this question, \t should work, and it does for me. It might be that your config does not give you a tab, but "\\t", which means a literal \ and a t. In that case, you could store the ascii char for tab in the database instead of \t.

How to destroy the subprocess of Java Process

I use Runtime.getRuntime().exec to execute command tail -f filename | grep str which is based on OS pipe. I managed to achieve my business logic. But still there is a problem I must solve:
When using pipe, the Process will for another process for tail command:
$ ps -ef | grep test.log
admin 6953 32721 0 16:32 ? 00:00:00 /bin/sh -c tail -f /home/admin
/test.log | unbuffer -p grep '1444379575648'
admin 6957 6953 0 16:32 ? 00:00:00 tail -f /home/admin/test.log
Process.destroy() method destroys itself(pid:6953) only.How can I destroy its subprocess(pid:6957) in my Java program?

Don't use Runtime.getRuntime().exec(...), use 2 ProcessBuilders to explicitly build the individual processes, and connect their inputs and outputs together to do the equivalent of the piping.
Then you will a separate Process object for each, and can kill them as you please.

i found another way:
public static final String getPid() {
try {
RuntimeMXBean runtimeBean = ManagementFactory.getRuntimeMXBean();
String name = runtimeBean.getName();
int k = name.indexOf('#');
if (k > 0)
return name.substring(0, k);
} catch (Exception ex) {
}
return null;
}
This works on linux,win,mac.

Need to filter, parse and sort multiple log files

I have a need to collect a subset of info from log files that reside on one-to-many log file servers. I have the following java code that does the initial data collection/filtering:
public String getLogServerInfo(String userName, String password, String hostNames, String id) throws Exception{
int timeout = 5;
String results = "";
String[] hostNameArray = hostNames.split("\\s*,\\s*");
for (String hostName : hostNameArray) {
SSHClient ssh = new SSHClient();
ssh.addHostKeyVerifier(new PromiscuousVerifier());
try {
Utils.writeStdOut("Parsing server: " + hostName);
ssh.connect(hostName);
ssh.authPassword(userName, password);
Session s = ssh.startSession();
try {
String sh1 = "cat /logs/en/event/event*.log | grep \"" + id + "\" | grep TYPE=ERROR";
Command cmd = s.exec(sh1);
results += IOUtils.readFully(cmd.getInputStream()).toString();
cmd.join(timeout, TimeUnit.SECONDS);
Utils.writeStdOut("\n** exit status: " + cmd.getExitStatus());
} finally {
s.close();
}
} finally {
ssh.disconnect();
ssh.close();
}
}
return results;
}
The results string variable looks something like this:
TYPE=ERROR, TIMESTAMP=10/03/2015 07:14:31 253 AM, HOST=server1, APPLICATION=app1, FUNCTION=function1, STATUS=null, GUID=null, etc. etc.
TYPE=ERROR, TIMESTAMP=10/03/2015 07:14:59 123 AM, HOST=server1, APPLICATION=app1, FUNCTION=function1, STATUS=null, GUID=null, etc. etc.
TYPE=ERROR, TIMESTAMP=10/03/2015 07:14:28 956 AM, HOST=server2, APPLICATION=app1, FUNCTION=function2, STATUS=null, GUID=null, etc. etc.
I need to accomplish the following:
What do I need to do to be able to sort results by TIMESTAMP? It is unsorted right now, because i am enumerating one to many files, and appending results to end of a string.
I only want a subset of "columns" returned, such as TYPE, TIMESTAMP, FUNCTION. I thought i could REGEX it in the grep, but maybe arrays would be better?
Results are simply being printed to console/report, as this is only printed for failed tests, and is there for troubleshooting purposes only.

I took the list of output that you provided and put it in a file, named test.txt, making sure that each "TYPE=ERROR etc. etc" was in a new line (I guess it's the same in your output, but it isn't clear).
Then I used cat test.txt | cut -d',' -f1,2,5 | sort -k2 to do what you want.
cut -d',' -f1,2,5 basically splits by comma and only reports tokens number 1,2,5 (TYPE,TIMESTAMP,FUNCTION). If you want more, you can add more numbers depending on what token you want
sort -k2 sorts according to the 2nd column (TIMESTAMP)
The output I get is:
TYPE=ERROR, TIMESTAMP=10/03/2015 07:14:28 956 AM, FUNCTION=function2
TYPE=ERROR, TIMESTAMP=10/03/2015 07:14:31 253 AM, FUNCTION=function1
TYPE=ERROR, TIMESTAMP=10/03/2015 07:14:59 123 AM, FUNCTION=function1
So what you should try and do, is to further pipe your command with |cut -d',' -f1,2,5 | sort -k2
I hope it helps.

After working on this some more, i come to find that one of the key/value pairs allows commas in the values, thus cut will not work. Here is the finished product:
My grep command stays the same, collecting data from all servers:
String sh1 = "cat /logs/en/event/event*.log | grep \"" + id + "\" | grep TYPE=ERROR";
Command cmd = s.exec(sh1);
results += IOUtils.readFully(cmd.getInputStream()).toString();
Put the string into an array, so i can process them line by line:
String lines[] = results.split("\r?\n");
I then used regex to get the data i needed, repeating the below for each line in the array, and for as many columns as needed. It's a bit of a hack, I probably could have done it better by simply replacing the comma in the offending key/value pair, then using SPLIT() and comma as delimeter, then looping for the fields i want.
lines2[i] = "";
Pattern p = Pattern.compile("TYPE=(.*?), APPLICATION=.*");
Matcher m = p.matcher(lines[i]);
if (m.find()) {
lines2[i] += ("TYPE=" + m.group(1));
}
Finally, this will sort by Timestamp, since it is 2nd column:
Arrays.sort(lines2);

Parse a task list

A file contains the following:
HPWAMain.exe 3876 Console 1 8,112 K
hpqwmiex.exe 3900 Services 0 6,256 K
WmiPrvSE.exe 3924 Services 0 8,576 K
jusched.exe 3960 Console 1 5,128 K
DivXUpdate.exe 3044 Console 1 16,160 K
WiFiMsg.exe 3984 Console 1 6,404 K
HpqToaster.exe 2236 Console 1 7,188 K
wmpnscfg.exe 3784 Console 1 6,536 K
wmpnetwk.exe 3732 Services 0 11,196 K
skypePM.exe 2040 Console 1 25,960 K
I want to get the process ID of the skypePM.exe. How is this possible in Java?
Any help is appreciated.

Algorithm
Open the file.
In a loop, read a line of text.
If the line of text starts with skypePM.exe then extract the number.
Repeat looping until all lines have been read from the file.
Close the file.
Implementation
import java.io.*;
public class T {
public static void main( String args[] ) throws Exception {
BufferedReader br = new BufferedReader(
new InputStreamReader(
new FileInputStream( "tasklist.txt" ) ) );
String line;
while( (line = br.readLine()) != null ) {
if( line.startsWith( "skypePM.exe" ) ) {
line = line.substring( "skypePM.exe".length() );
int taskId = Integer.parseInt( (line.trim().split( " " ))[0] );
System.out.println( "Task Id: " + taskId );
}
}
br.close();
}
}
Alternate Implementation
If you have Cygwin and related tools installed, you could use:
cat tasklist.txt | grep skypePM.exe | awk '{ print $2; }'

To find the Process Id of the application SlypePM..
Open the file
now read lines one by one
find the line which contains SkypePM.exe in the beginning
In the line containing SkypePM.exe parse the line to read the numbers after the process name leaving the spaces.
You get process id of the process
It is all string operations.
Remember the format of the file should not change after you write the code.

If you really want to parse the output, you may need a different strategy. If your output file really is the result of a tasklist execution, then it should have some column headers at the top of it like:
Image Name PID Session Name Session# Mem Usage
========================= ======== ================ =========== ============
I would use these, in particular the set of equal signs with spaces, to break any subsequent strings using a fixed-width column strategy. This way, you could have more flexibility in parsing the output if needed (i.e. maybe someone is looking for java.exe or wjava.exe). Do keep in mind the last column may not be padded with spaces all the way to the end.
I will say, in the strictest sense, the existing answers should work for just getting the PID.

Implementation in Java is not a good way. Shell or other script languages may help you a lot. Anyway, JAWK is a implementation of awk in Java, I think it may help you.