We Keep Coding

processBuilder not finding the main class

The command that is built is working fine on the command prompt but it not working in eclipse . This is how this looks :
String javaHome = System.getProperty("java.home");
cmds.add(javaHome+"/bin/java");
cmds.add("-cp");
cmds.add(_devInstallConfig.getProperty("CP"));
// CP = ("D:\Perforce\depot\nginst\src13920\nginst-install\repo-bootstrap-classpath-13.9.2.0-170213.1854.jar");
cmds.add("com.oracle.cie.repository.stager.CarbStager");
cmds.add("-repoURL "+repoURL);
cmds.add("-props");
cmds.add(_devInstallConfig.getProperty("PROPERTIES_FILE_LOCATION"));
//PROPERTIES_FILE_LOCATION=("C:\Users\bpurana.ORADEV\Desktop\carb.properties");
cmds.add("-repoBaseDir");
cmds.add(_ngInstallLocation.getParentFile().getAbsolutePath());
//repoBaseDir=("D:\Perforce\depot\nginst\src13920\nginst-install");
cmds.add("-PUBLISH_TYPE=BOTH");
cmds.add("-CARB_OUTPUT_DIR="+temp_file);
cmds.add("-NGINST_VERSION");
cmds.add(_devInstallConfig.getProperty("NGINST_VERSION"));
ProcessBuilder pb = new ProcessBuilder(cmds);
pb.directory(_ngInstallLocation.getParentFile());
Process process = pb.start();
The error that I am seeing is this :
Error: Could not find or load main class com.oracle.cie.repository.stager.CarbStager
Edit : I have updated the code snippet with the actual code ( and sharing sample examples of what each value means)

Ensure Install4j Uses only its bundled jre and never Java found by path

question
How can I ensure my install4j installer always finds only its java?
Can I create a top level installer which installs JRE to tmp, sets env variables and then starts the actual installer?
Can I load a vm file during installation?
problem
Install4j finds java 1.7 during install which impacts custom code preventing successful installation. I see found java7 prior to file deployment - ok expected given the JRE hasn't yet been unpacked.
evidence
I created a simple installer and see the following:
BEFORE
PATH=/opt/tools/Java/jdk1.7.0_79/bin:...
JAVA_HOME=/opt/tools/Java/jdk1.7.0_79
...
ENV [JAVA_HOME] /opt/tools/Java/jdk1.7.0_79
ENV [PATH] /opt/tools/Java/jdk1.7.0_79/bin:...
installer details
envTest.install4j
Optional customer install script reporting found java prior at execution start
echo BEFORE
echo PATH=$PATH
echo JAVA_HOME=$JAVA_HOME
echo Version: java -version
Run script reporting env after installer deployed jre
`
import java.util.Map;
import java.util.Set;
import java.util.SortedMap;
import java.util.TreeMap;
Map<String, String> envMap = System.getenv();
SortedMap<String, String> sortedEnvMap = new TreeMap<String, String>(envMap);
Set<String> keySet = sortedEnvMap.keySet();
for (String key : keySet) {
String value = envMap.get(key);
Util.logInfo(this,"ENV [" + key + "] " + value);
}
return true;

Actually, this turned our to be a problem with my custom code. The custom code launches an install4j generated executable via java. When launched on command line with wrong java found first, the launcher uses only its own java. When launched from my extension it fails.
Solution - set java in my extension:
private File getInstalledJREDir() {
return new File(installationDir, "jre");
}
private String addJREToFrontOfPathVar() {
File jreBinDir = new File(getInstalledJREDir(), "bin");
String path = System.getenv().get("PATH");
if (null == path) {
path = jreBinDir.getAbsolutePath();
} else {
path = jreBinDir.getAbsolutePath() + File.pathSeparator + path;
}
return path;
}
/**
* Start Laucnher and block until it starts or timeout reached
* #throws AutoRunException
*/
public void run() throws AutoRunException, IOException, InterruptedException {
notifier.setPhase("Starting Agent");
// Set Directories
File dataDir = new File(installationDir.getParentFile(), "data-agent");
File agentLog = new File(logDir,"agent.log");
if (! isWindows()) {
File agent = new File(installationDir, "bin/launcherExecutable");
CmdExecutor ce = new CmdExecutor(agent, agentLog);
// Ensure our installed JRE found 1st - PLAT-38833
ce.updateEnvironmentVariable("JAVA_HOME", getInstalledJREDir().getAbsolutePath());
ce.updateEnvironmentVariable("PATH", addJREToFrontOfPathVar());
ce.setWorkingDir(installationDir);
ce.setArgLine(String.format("--datadir %s", dataDir.getAbsolutePath()));
notifier.logInfo("Starting " + agent + " with " + ce.getArgLine());
if (! ce.run(true) ) {
throw new AutoRunException("Agent failed to start " + ce.getOutput());
}

Java runtime exec not handling string array well

I have a tomcat servlet which calls a jar function with parameters. The first parameter sometimes contains space. So I tried to use a String array, but it doesn't work at all.
What am I doing wrong?
requestParm = "java -classpath c:\\j\\test.jar test.connect " + fileName + " new";
requestParmarray =new String[]{"java -classpath c:\\j\\test.jar test.connect ",fileName , " new"};
requestParmarrayNew =new String[]{"java -classpath c:\\j\\test.jar test.connect "+fileName+" new"};
// This line works.but can not handle space well
Process ls_proc = Runtime.getRuntime().exec(requestPar);
// Does not call the function at all
Process ls_proc = Runtime.getRuntime().exec(requestParmarray );
// Does not call the function at all
Process ls_proc = Runtime.getRuntime().exec(requestParmarrayNew );
// Does not call the function at all
Process ls_proc = new ProcessBuilder("java -classpath c:\\j\\test.jar test.connect ",fileName, "new" ).start();

You're creating the array incorrectly: Each individual argument must be in its own entry:
String[] requestParmArray = new String[] {
"java",
"-classpath",
"c:\\j\\test.jar",
"test.connect",
fileName,
"new"
};
Process ls_proc = Runtime.getRuntime().exec(requestParmArray);
Also note that I removed the space you had after test.connect; the spaces you put on the command line are just to separate arguments, but in the above, they're separated by being separate entries in the array.

You should make the array in exec() have each parameter as a separate array entry like:
String[] requestPar = new String[]{"java", "-classpath", "c:\\j\\test.jar", "test.connect ", fileName, "new"};
And use it:
Process ls_proc = Runtime.getRuntime().exec(requestPar);

Java code or Oozie

I'm new to Hadoop, so I have some doubts what to do in the next case.
I have an algorithm that includes multiple runs of different jobs and sometimes multiple runs of a single job (in a loop).
How should I achieve this, using Oozie, or using Java code? I was looking through Mahout code and in ClusterIterator function function found this:
public static void iterateMR(Configuration conf, Path inPath, Path priorPath, Path outPath, int numIterations)
throws IOException, InterruptedException, ClassNotFoundException {
ClusteringPolicy policy = ClusterClassifier.readPolicy(priorPath);
Path clustersOut = null;
int iteration = 1;
while (iteration <= numIterations) {
conf.set(PRIOR_PATH_KEY, priorPath.toString());
String jobName = "Cluster Iterator running iteration " + iteration + " over priorPath: " + priorPath;
Job job = new Job(conf, jobName);
job.setMapOutputKeyClass(IntWritable.class);
job.setMapOutputValueClass(ClusterWritable.class);
job.setOutputKeyClass(IntWritable.class);
job.setOutputValueClass(ClusterWritable.class);
job.setInputFormatClass(SequenceFileInputFormat.class);
job.setOutputFormatClass(SequenceFileOutputFormat.class);
job.setMapperClass(CIMapper.class);
job.setReducerClass(CIReducer.class);
FileInputFormat.addInputPath(job, inPath);
clustersOut = new Path(outPath, Cluster.CLUSTERS_DIR + iteration);
priorPath = clustersOut;
FileOutputFormat.setOutputPath(job, clustersOut);
job.setJarByClass(ClusterIterator.class);
if (!job.waitForCompletion(true)) {
throw new InterruptedException("Cluster Iteration " + iteration + " failed processing " + priorPath);
}
ClusterClassifier.writePolicy(policy, clustersOut);
FileSystem fs = FileSystem.get(outPath.toUri(), conf);
iteration++;
if (isConverged(clustersOut, conf, fs)) {
break;
}
}
Path finalClustersIn = new Path(outPath, Cluster.CLUSTERS_DIR + (iteration - 1) + Cluster.FINAL_ITERATION_SUFFIX);
FileSystem.get(clustersOut.toUri(), conf).rename(clustersOut, finalClustersIn);
}
So, they have a loop in which they run MR jobs. Is this a good approach? I know that Oozie is used for DAGs, and can be used with another components, such Pig, but should I consider using it for something like this?
What if I want to run clustering algorithm multiple times, let's say for clustering (using specific driver), should I do that in a loop, or using Oozie.
Thanks

If you are looking to run map reduce jobs only then you can consider following ways
chain MR jobs using Map reduce job Control API.
http://hadoop.apache.org/docs/r2.5.0/api/org/apache/hadoop/mapreduce/lib/jobcontrol/JobControl.html
Submit multiple MR jobs from a single driver class.
Job job1 = new Job( getConf() );
job.waitForCompletion( true );
if(job.isSuccessful()){
//start another job with different Mapper.
//change config
Job job2 = new Job( getConf() );
}
If you have a complex DAG or involving multiple ecosystem tools like hive,pig then Oozie suits well.

Scala - Fails to execute a process through terminal in a particular scenario

I'm using graphviz to generate graphs based on the messages passed in a scala program.
To invoke the graphviz application from inside the scala program, I'm using the exec() method (similar to Java). It successfully executed the command and created the graph when I used the below code snippet:
var cmd: String = "dot -Tpng Graph.dot -o Graph.png"
var run: Runtime = Runtime.getRuntime() ;
var pr: Process = run.exec(cmd) ;
However It fails to execute after changing the path of the input and output files (I just included a directory inside which the input file and output file resides as shown below)
def main(args: Array[String]): Unit = {
var DirectoryName: String = "Logs"
var GraphFileName: String = DirectoryName + File.separator + "Graph.dot"
val GraphFileObj: File = new File(GraphFileName)
// var cmd: String = "dot -Tpng Graph.dot -o Graph.png"
var cmd: String = "dot -Tpng \"" + GraphFileObj.getAbsolutePath + "\" -o \"" + DirectoryName + File.separator + "Graph.png\"" ;
println(cmd)
var run: Runtime = Runtime.getRuntime() ;
var pr: Process = run.exec(cmd) ;
}
The same command when executed through terminal gives proper output. Can you please help me to find what I'm missing?

exec is not a shell...e.g. quoting won't work as you expect, and thus your path (which may contain spaces, etc) will not be processed as you expect. The command will be broken apart using StringTokenizer, and your literal quotes will be...well..literal.
Use the form of exec that takes an array instead, so you can tokenize the command correctly.
val args = Array[String]("dot", "-Tpng", GraphFileObj.getAbsolutePath, ...);
run.exec(args)