We Keep Coding

Java code manually Triggering kubernetes cronjob from the cluster

I'm trying to trigger cronjob manually(not scheduled) using fabric8 library
but getting the following error:
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://172.20.0.1:443/apis/batch/v1/
namespaces/engineering/jobs. Message: Job.batch "app-chat-manual-947171" is invalid: spec.template.spec.containers[0].name: Re
quired value. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.template.spec.co
ntainers[0].name, message=Required value, reason=FieldValueRequired, additionalProperties={})], group=batch, kind=Job, name=ap
p-chat-manual-947171, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Job.batch "app-chat-man
ual-947171" is invalid: spec.template.spec.containers[0].name: Required value, metadata=ListMeta(_continue=null, remainingItemCount=
null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
my code is running at the cluster:
maven dependency:
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-client</artifactId>
<version>6.3.1</version>
</dependency>
java code:
public static void triggerCronjob(String cronjobName, String applicableNamespace) {
KubernetesClient kubernetesClient = new KubernetesClientBuilder().build();
final String podName = String.format("%s-manual-%s", cronjobName.length() > 38 ? cronjobName.substring(0, 38) : cronjobName,
new Random().nextInt(999999));
System.out.println("triggerCronjob method invoked, applicableNamespace: " + applicableNamespace
+ ", cronjobName: " + cronjobName + ", podName: " + podName);
Job job = new JobBuilder()
.withApiVersion("batch/v1")
.withNewMetadata()
.withName(podName)
.endMetadata()
.withNewSpec()
.withBackoffLimit(4)
.withNewTemplate()
.withNewSpec()
.addNewContainer()
.withName(podName)
.withImage("perl")
.withCommand("perl", "-Mbignum=bpi", "-wle", "print bpi(2000)")
.endContainer()
.withRestartPolicy("Never")
.endSpec()
.endTemplate()
.endSpec().build();
kubernetesClient.batch().v1().jobs().inNamespace(applicableNamespace).createOrReplace(job);
kubernetesClient.close();
System.out.println("CronJob triggered: applicableNamespace: " + applicableNamespace + ", cronjob name: " + cronjobName);
}
the code executed at the kubernetes cluster, but not form the application, it's an external program that's running in the cluster.
my goal is to trigger given job in a given namespace.

If you want to trigger an already existing CronJob, you need to provide ownerReference for the existing CronJob in Job:
// Get already existing CronJob
CronJob cronJob = kubernetesClient.batch().v1()
.cronjobs()
.inNamespace(namespace)
.withName(cronJobName)
.get();
// Create new Job object referencing CronJob
Job newJobToCreate = new JobBuilder()
.withNewMetadata()
.withName(jobName)
.addNewOwnerReference()
.withApiVersion("batch/v1")
.withKind("CronJob")
.withName(cronJob.getMetadata().getName())
.withUid(cronJob.getMetadata().getUid())
.endOwnerReference()
.addToAnnotations("cronjob.kubernetes.io/instantiate", "manual")
.endMetadata()
.withSpec(cronJob.getSpec().getJobTemplate().getSpec())
.build();
// Apply job object to Kubernetes Cluster
kubernetesClient.batch().v1()
.jobs()
.inNamespace(namespace)
.resource(newJobToCreate)
.create();

Could not initialize English Chunker

I have included LanguageTool coding in my Java Maven project as below;
Java Code
List<Language> realLanguages = Languages.get();
for (Language language : realLanguages) {
System.out.println(language.getName() + " ==> " + language.getShortName());
if (language.getName().startsWith("English (US)")) {
JLanguageTool langTool = new JLanguageTool(language);
PatternRuleLoader patternRuleLoader = new PatternRuleLoader();
List<PatternRule> abstractPatternRuleList = new ArrayList<PatternRule>();
abstractPatternRuleList = patternRuleLoader.getRules(new File(LTPath + "/CustomGrammar.xml"));
System.out.println("\n\nDefault Active Rules: " + langTool.getAllActiveRules().size());
<-- More coding goes here -->
and it works absolutely fine when the module's jar is invoked from one project (on server 'A'), but the same throws the below attached exception, "Could not initialize English chunker" when invoked from another (on server 'B').
Dependency
<dependency>
<groupId>org.languagetool</groupId>
<artifactId>language-en</artifactId>
<version>3.1</version>
</dependency>
Exception
Please help !

Calling R script function from Java using rJava

My Requirement -
I need to deploy a Java webservice in a server which internally executes a R scipt file. I googled about various solutions for calling R from Java and the best were rJava and Rserve. Using Rserve I can call R function BUT as I am running this in Windows it can not handle multiple requests at a time and I dont want to switch to Linux.
[Edit]
What I tried -
I have used rJava to call a R function :
String[] args = new String[3];
args[0] = "--quiet"; // Don't print startup message
args[1] = "--no-restore"; // Don't restore anything
args[2] = "--no-save";
String rFilePath = "D:/Dataset_Info/AI-KMS_v2.0/tika/src/main/resources/HSConcordance.R";
Rengine engine = new Rengine(args, false, null);
if (!engine.waitForR()) {
System.out.println("Cannot load R");
}
System.out.print("JRI R-Engine call: ");
engine.eval("source(\"" + rFilePath + "\")");
REXP value = engine.eval("as.integer(a<-simple())");
int a = value.asInt();
System.out.println(a);
Maven dependency -
<dependency>
<groupId>com.github.lucarosellini.rJava</groupId>
<artifactId>JRI</artifactId>
<version>0.9-7</version>
</dependency>
<dependency>
<groupId>com.github.lucarosellini.rJava</groupId>
<artifactId>REngine</artifactId>
<version>0.9-7</version>
</dependency>
<dependency>
<groupId>com.github.lucarosellini.rJava</groupId>
<artifactId>JRIEngine</artifactId>
<version>0.9-7</version>
</dependency>
My R script file -
simple<-function(){
a=1
return(a)
}
Output - JRI R-Engine call: 1
and then it hangs. I debugged it and found that it got stuck in Thread.class
Any kind of help will be greatly appreciated.

The issue was when I am acessing the webservice for the 2nd time it got hanged because we already have an instance of Rengine present which was created at first call.
Rengine re = Rengine.getMainEngine();
if(re == null){
re=new Rengine (new String [] {"--vanilla"}, false, null);
if (!re.waitForR())
{
System.out.println ("Cannot load R");
return "failure";
}
}
re.eval("source(\"" + rFilePath + "\")");
re.eval("copyfile(\""+filePath+"\")");
re.end();
Few points to note -
Check if any instance of Rengine is already present by Rengine re = Rengine.getMainEngine();
Shut down R in the end by re.end();
It may be helpful. thanks.

Getting Spring-XD and the hdfs sink to work for maprfs

This is a question about spring-xd release 1.0.1 working together with maprfs, which is officially not yet supported. Still I would like to get it to work.
So this is what we did:
1) adjusted the xd-shell and xd-worker and xd-singlenode shell scripts to accept the parameter --hadoopDistro mapr
2) added libraries to the new directory $XD_HOME/lib/mapr
avro-1.7.4.jar jersey-core-1.9.jar
hadoop-annotations-2.2.0.jar jersey-server-1.9.jar
hadoop-core-1.0.3-mapr-3.0.2.jar jetty-util-6.1.26.jar
hadoop-distcp-2.2.0.jar maprfs-1.0.3-mapr-3.0.2.jar
hadoop-hdfs-2.2.0.jar protobuf-java-2.5.0.jar
hadoop-mapreduce-client-core-2.2.0.jar spring-data-hadoop-2.0.2.RELEASE-hadoop24.jar
hadoop-streaming-2.2.0.jar spring-data-hadoop-batch-2.0.2.RELEASE-hadoop24.jar
hadoop-yarn-api-2.2.0.jar spring-data-hadoop-core-2.0.2.RELEASE-hadoop24.jar
hadoop-yarn-common-2.2.0.jar spring-data-hadoop-store-2.0.2.RELEASE-hadoop24.jar
3) run bin/xd-singlenode --hadoopDistro mapr and shell/bin/xd-shell --hadoopDistro mapr.
When creating and deploying a stream via stream create foo --definition "time | hdfs" --deploy, data is written to a file tmp/xd/foo/foo-1.txt.tmp on maprfs. Yet when undeploying the stream, the following exceptions appears:
org.springframework.data.hadoop.store.StoreException: Failed renaming from /xd/foo/foo-1.txt.tmp to /xd/foo/foo-1.txt; nested exception is java.io.FileNotFoundException: Requested file /xd/foo/foo-1.txt does not exist.
at org.springframework.data.hadoop.store.support.OutputStoreObjectSupport.renameFile(OutputStoreObjectSupport.java:261)
at org.springframework.data.hadoop.store.output.TextFileWriter.close(TextFileWriter.java:92)
at org.springframework.xd.integration.hadoop.outbound.HdfsDataStoreMessageHandler.doStop(HdfsDataStoreMessageHandler.java:58)
at org.springframework.xd.integration.hadoop.outbound.HdfsStoreMessageHandler.stop(HdfsStoreMessageHandler.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201)
at com.sun.proxy.$Proxy120.stop(Unknown Source)
at org.springframework.integration.endpoint.EventDrivenConsumer.doStop(EventDrivenConsumer.java:64)
at org.springframework.integration.endpoint.AbstractEndpoint.stop(AbstractEndpoint.java:100)
at org.springframework.integration.endpoint.AbstractEndpoint.stop(AbstractEndpoint.java:115)
at org.springframework.integration.config.ConsumerEndpointFactoryBean.stop(ConsumerEndpointFactoryBean.java:303)
at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:229)
at org.springframework.context.support.DefaultLifecycleProcessor.access$300(DefaultLifecycleProcessor.java:51)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.stop(DefaultLifecycleProcessor.java:363)
at org.springframework.context.support.DefaultLifecycleProcessor.stopBeans(DefaultLifecycleProcessor.java:202)
at org.springframework.context.support.DefaultLifecycleProcessor.stop(DefaultLifecycleProcessor.java:106)
at org.springframework.context.support.AbstractApplicationContext.stop(AbstractApplicationContext.java:1186)
at org.springframework.xd.module.core.SimpleModule.stop(SimpleModule.java:234)
at org.springframework.xd.dirt.module.ModuleDeployer.destroyModule(ModuleDeployer.java:132)
at org.springframework.xd.dirt.module.ModuleDeployer.handleUndeploy(ModuleDeployer.java:111)
at org.springframework.xd.dirt.module.ModuleDeployer.undeploy(ModuleDeployer.java:83)
at org.springframework.xd.dirt.server.ContainerRegistrar.undeployModule(ContainerRegistrar.java:261)
at org.springframework.xd.dirt.server.ContainerRegistrar$StreamModuleWatcher.process(ContainerRegistrar.java:884)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: java.io.FileNotFoundException: Requested file /xd/foo/foo-1.txt does not exist.
at com.mapr.fs.MapRFileSystem.getMapRFileStatus(MapRFileSystem.java:805)
at com.mapr.fs.MapRFileSystem.delete(MapRFileSystem.java:629)
at org.springframework.data.hadoop.store.support.OutputStoreObjectSupport.renameFile(OutputStoreObjectSupport.java:258)
... 29 more
I had a look at the OutputStoreObjectSupport.renameFile() function. When a file on hdfs is finished, this method tries to rename the file /xd/foo/foo-1.txt.tmp to xd/foo/foo1.txt. This is the relevant code:
try {
FileSystem fs = path.getFileSystem(getConfiguration());
boolean succeed;
try {
fs.delete(toPath, false);
log.info("Renaming path=[" + path + "] toPath=[" + toPath + "]");
succeed = fs.rename(path, toPath);
} catch (Exception e) {
throw new StoreException("Failed renaming from " + path + " to " + toPath, e);
}
if (!succeed) {
throw new StoreException("Failed renaming from " + path + " to " + toPath + " because hdfs returned false");
}
}
When the target file does not exist on hdfs, maprfs seems to throw an exception when fs.delete(toPath, false) is called. Yet throwing an exception in this case does not make sense. I assume that other Filesystem implementations behave differently, but this is a point I still need to verify. Unfortuntately I cannot find the sources for MapRFileSystem.java. Is this closed source? This would help me to better understand the issue. Has anybody experience with writing from spring-xd to maprfs? Or renaming files on maprfs with spring-data-hadoop?
Edit
I managed to reproduce the issue outside of spring XD with a simple test case (see below). Note that this exception is only thrown if the inWritingSuffix or the inWritingPrefix is set. Otherwise spring-hadoop will not attempt to rename the file. So this is the still somehow unsatisfactory workaround for me: refrain from using inWritingPrefixes and inWritingSuffixes.
#ContextConfiguration("context.xml")
#RunWith(SpringJUnit4ClassRunner.class)
public class MaprfsSinkTest {
#Autowired
Configuration configuration;
#Autowired
FileSystem filesystem;
#Autowired
DataStoreWriter<String >storeWriter;
#Test
public void testRenameOnMaprfs() throws IOException, InterruptedException {
Path testPath = new Path("/tmp/foo.txt");
filesystem.delete(testPath, true);
TextFileWriter writer = new TextFileWriter(configuration, testPath, null);
writer.setInWritingSuffix("tmp");
writer.write("some entity");
writer.close();
}
#Test
public void testStoreWriter() throws IOException {
this.storeWriter.write("something");
}
}

I created a new branch for spring-hadoop which supports maprfs:
https://github.com/blinse/spring-hadoop/tree/origin/2.0.2.RELEASE-mapr
Building this release and using the resulting jar works fine with the hdfs sink.

How to run soapUI tests from Java

I need to run SoapUI test by Java. Could you please advise me useful links? And I would be happy if you can show me how to load/run tests (code examples).
I also found only one link which can be applicable for my project - http://pritikaur23.wordpress.com/2013/06/16/saving-a-soapui-project-and-sending-requests-using-soapui-api/ .
But when I try to do the same I faced below errors -
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.xmlbeans.XmlBeans.typeSystemForClassLoader(Ljava/lang/ClassLoader;Ljava/lang/String;)Lorg/apache/xmlbeans/SchemaTypeSystem;
It's weird because i added all needed jar files. Also I even tried different versions of a xmlbeans.
Thank in advance.

I found the way how to run soapUI test by code.
The small explanation:
Firstly - I created a maven project and added dependencies to pom.xml instead of a including .jar directly. For the SoapUI tests was needed to add following dependencies:
<dependency>
<groupId>com.github.redfish4ktc.soapui</groupId>
<artifactId>maven-soapui-extension-plugin</artifactId>
<version>4.6.4.0</version>
</dependency>
Secondly - I also added a few dependencies because I got an exceptions
java.lang.NoSuchMethodError
The needed dependencies:
<dependency>
<groupId>net.java.dev.jgoodies</groupId>
<artifactId>looks</artifactId>
<version>2.1.4</version>
</dependency>
<dependency>
<groupId>net.sf.squirrel-sql.thirdparty-non-maven</groupId>
<artifactId>com-fifesoft-rsyntaxtextarea</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.karaf.eik.plugins</groupId>
<artifactId>org.apache.commons.collections</artifactId>
<version>3.2.1</version>
</dependency>
After preparing of the environment I was able to write a code. I show you an example of a code what able to run all test suites and test cases in a specified soapUI project by Java.
// method for running all Test Suites and test cases in the project
public static void getTestSuite() throws Exception {
String suiteName = "";
String reportStr = "";
// variables for getting duration
long startTime = 0;
long duration = 0;
TestRunner runner = null;
List<TestSuite> suiteList = new ArrayList<TestSuite>();
List<TestCase> caseList = new ArrayList<TestCase>();
SoapUI.setSoapUICore(new StandaloneSoapUICore(true));
// specified soapUI project
WsdlProject project = new WsdlProject("your-soapui-project.xml");
// get a list of all test suites on the project
suiteList = project.getTestSuiteList();
// you can use for each loop
for(int i = 0; i < suiteList.size(); i++){
// get name of the "i" element in the list of a test suites
suiteName = suiteList.get(i).getName();
reportStr = reportStr + "\nTest Suite: " + suiteName;
// get a list of all test cases on the "i"-test suite
caseList = suiteList.get(i).getTestCaseList();
for(int k = 0; k < caseList.size(); k++){
startTime = System.currentTimeMillis();
// run "k"-test case in the "i"-test suite
runner = project.getTestSuiteByName(suiteName).getTestCaseByName(caseList.get(k).getName()).run(new PropertiesMap(), false);
duration = System.currentTimeMillis() - startTime;
reportStr = reportStr + "\n\tTestCase: " + caseList.get(k).getName() + "\tStatus: " + runner.getStatus() + "\tReason: " + runner.getReason() + "\tDuration: " + duration;
}
}
// string of the results
System.out.println(reportStr);
}
Output:
Test Suite: TS_ONE
TestCase: TC_ONE Status: FAILED Reason: Cancelling due to failed test step Duration: 1549
TestCase: TC_TWO Status: FINISHED Reason: {} Duration: 1277
...
TestCase: TC_N Status: FAILED Reason: Cancelling due to failed test step Duration: 1282
Test Suite: TS_TWO
TestCase: TC_BlaBla Status: FINSHED Reason: {} Duration: 1280
...
I hope the information above will help someone.

Using a continuous integration server (eg Hudson is perfect for this) it is possible to run unit tests automatically JUnit format. Below is an example of integrating SoapUI project in a JUnit test.
public void testRunner() throws Exception
{
SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
runner.setProjectFile( "src/dist/sample-soapui-project.xml" );
runner.run();
}
more information here.

Currentyl only SoapUI dependency is needed for #HeLL provided code
<dependency>
<groupId>com.smartbear.soapui</groupId>
<artifactId>soapui</artifactId>
<version>5.1.3</version>
<scope>test</scope>
</dependency>

I had the same issue, but I fixed it by using:
<dependency>
<groupId>com.smartbear.soapui</groupId>
<artifactId>soapui</artifactId>
<version>4.6.1</version>
</dependency>