We Keep Coding

How to get an Initial Contex from Wildfly 8

ADDED 7/23.
Many views: Not even a "that's dumb" question in response. Can anyone at least tell me why such an embarrassingly trivial question seems to have no answer anywhere.
Q:
--- Have Wildfly 8 running on local machine localhost:9990.
--- Have a Java program that need's Wildfly's IntialContext.
--- Every reference says use: "Context ctx = new InitialContext(env);"
--- Yet a week of searching turns up no set of properties that returns one.
And no example of a java program that gets one.
Does no one ever do this? Really need help
Original Msg Below
I know many people have asked how to get an Initial context from Wildfly 8. But I have yet to find a simple answer with a simple example.
Therefore, I hope someone can tell my why this doesn’t work.
I start Wildfly with standalone-full.xml
The three sections below have
A - Code summary of my test Class whose only purpose is to secure an Initial Context. (I only removed a lot of printing code that produced the next section.]
B - The Eclipse console output for a failure.
C - Cut and paste code. Just in case anyone can help me get this to work. I’d like to leave behind something the next new WF user can cut and past and run. The only difference from 1 above is that this version has all the static methods I used to format the output. NOTE: I know the comments I inserted about the less than sign sound dumb. BUT ... they are true.
A Code Summary
import java.util.Properties;
import javax.naming.CommunicationException;
import javax.naming.Context;
import javax.naming.InitialContext;
public class JmsTestGetJNDIContext {
//members
final private Properties env = new Properties() {
private static final long serialVersionUID = 1L;
{
/* These are Properties used by a standalone JavaClient to secure a WIldFly InitialContext()*/
put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
put(Context.PROVIDER_URL,"http-remoting://localhost:9990");
put(Context.SECURITY_PRINCIPAL,"userGLB");
put(Context.SECURITY_CREDENTIALS,"Open");
put("jboss.naming.client.ejb.context", true);
/*The above URL, ID and PW successfully open Wildfly's Admin Console*/
}
};
//constructor
private JmsTestGetJNDIContext (){
/*print "beg"*/
/*print "env"*/
try {
/*print "Requesting InitialContext"*/
Context ctx = new InitialContext(this.env);
/*print "JNDI Context: " + ctx)*/
/*print "end");
} catch (CommunicationException e) {
/* print "You forgot to start WildFly dummy!"*/
} catch (Exception e) {
/* print"caught: " + e.getClass().getName()*/
/*print e.getMessage()*/
/* "end")*/
}
static public void main (String[] args) {
/*print "beg"*/
JmsTestGetJNDIContext client = new JmsTestGetJNDIContext ();
/*print "end"*/
}
}
B - Console Output
JmsTestGetJNDIContext.main () beg
JmsTestGetJNDIContext.<init> () beg
JmsTestGetJNDIContext.<init> () These are Properties used to obtain IntialContext
Key: java.naming.provider.url
Value: http-remoting://localhost:9990
Key: java.naming.factory.initial
Value: org.jboss.naming.remote.client.InitialContextFactory
Key: jboss.naming.client.ejb.context
Value: true
Key: java.naming.security.principal
Value: userGLB
Key: java.naming.security.credentials
Value: Open
JmsTestGetJNDIContext.<init> () Requesting InitialContext
JmsTestGetJNDIContext.<init> () caught: javax.naming.NamingException
JmsTestGetJNDIContext.<init> () Failed to create remoting connection
JmsTestGetJNDIContext.<init> () end
JmsTestGetJNDIContext.main () end
Cut and Paste Code
package org.america3.gotest.xtra;
import java.util.Properties;
import javax.naming.CommunicationException;
import javax.naming.Context;
import javax.naming.InitialContext;
public class JmsTestGetJNDIContext {
//members
final private Properties env = new Properties() {
/**
* Properties used by a standalone JavaClient to secure
* a WIldFly InitialContext()*/
private static final long serialVersionUID = 1L;
{
put(Context.INITIAL_CONTEXT_FACTORY,"org.jboss.naming.remote.client.InitialContextFactory");
put(Context.PROVIDER_URL, "http-remoting://localhost:9990");
put(Context.SECURITY_PRINCIPAL, "userGLB");
put(Context.SECURITY_CREDENTIALS, "Open");
// The above URL, ID and PW successfully open Wildfly's Admin Console
put("jboss.naming.client.ejb.context", true);
}
};
//constructor
private JmsTestGetJNDIContext (){/*ignore*/String iAm = JmsTestGetJNDIContext.getIAm(" ", Thread.currentThread().getStackTrace());
P (iAm, "beg");
pProps(iAm, env);
try {
P (sp + iAm, "Requesting InitialContext");
Context ctx = new InitialContext(this.env);
P (sp + iAm, "JNDI Context: " + ctx);
P (sp + iAm, "end");
} catch (CommunicationException e) {
P (sp + iAm, "You forgot to start WildFly dummy!");
} catch (Exception e) {
P (sp + iAm, "caught: " + e.getClass().getName());
P (sp + iAm, e.getMessage());
P (iAm, "end");
}
}
static public void main (String[] args) {/*ignore*/String iAm = JmsTestGetJNDIContext.getIAm("",Thread.currentThread().getStackTrace());
P (iAm, "beg");
JmsTestGetJNDIContext client = new JmsTestGetJNDIContext ();
P (iAm , "end");
}
/*The remaining static methods are just to facilitate printing.
* They are normally in a Untility package I add to my projects.
* I put them here so this code would run for anyone.*/
static private void pProps (String leader, Properties p) {
StringBuffer sb = new StringBuffer ();
String s = JmsTestGetJNDIContext.padRight(leader, 45, ' ');
s = " " + s + "These are Properties used to obtain IntialContext"+"\n";
sb.append(s);
String skip = "";
for (Object key: p.keySet()) {
sb.append(skip + " " + JmsTestGetJNDIContext.padRight("\""
+ (String)key + "\"", 40, ' ')
+ " \"" + p.get(key) + "\"");
skip = "\n";
}
System.out.println(sb);
}
static private void P (String s, String s2) {
System.out.println(s + s2);
}
static public String getClassMethodName (StackTraceElement[] elements) {
String className = null;
for (int i = 0; i * elements.length; i++]i ) {
/* You need to type in a less than sign for the '*'
* because when I do, the editor will not show any code
* that comes after it.
* I have no idea why, but I've spent over an hour trying,
* and every time I type a less than sign all the following
* code dissappears!*/
className = elements[i].getClassName ();
if (className.startsWith ("org.america3")) {
int end = className.lastIndexOf ('.');
return className.substring (end + 1) + "." + elements[i].getMethodName ();
} else {
continue;
}
}
return "no project method found in elements beginning with org.america3" ;
}
static private String getIAm (String indent, StackTraceElement[] elements) {
StringBuffer sb = new StringBuffer ();
sb.append(JmsTestGetJNDIContext.getClassMethodName(elements));
sb.append(" ()");
return indent + JmsTestGetJNDIContext.padRight (sb.toString(), 45, ' ') ;
}
static public String padRight(String s, int width, char c){
if (s == null) return "Null String";
if(s.length() ** width){
/* You need to type in a greater than or equal sign for
* the '**'see above.*/
return s;
} else {
StringBuffer sb = new StringBuffer();
sb.append (s);
for(int i = 0; i *** (width - s.length()); i++){
/*You need to type in a less than sign the '***'. Again see above*/
sb.append(c);
}
return sb.toString();
}
}
static public String sp = " ";
}

A while ago I also struggled with remote EJBs in my CLI application. I excavated a small example project that I wrote then. It gets an InitialContext and calls a remote EJB named AddBrackets:
import java.util.Properties;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import de.dnb.test.ejb.AddBrackets;
public final class Application {
public static void main(String[] args) throws NamingException {
final Properties jndiProperties = initJndiProperties();
final AddBrackets addBrackets = getEjb(jndiProperties);
System.out.println(addBrackets.processText("Hello World"));
}
private static Properties initJndiProperties() {
final Properties jndiProperties = new Properties();
jndiProperties.put(Context.INITIAL_CONTEXT_FACTORY,
"org.jboss.naming.remote.client.InitialContextFactory");
jndiProperties.put("jboss.naming.client.ejb.context", true);
jndiProperties.put(Context.PROVIDER_URL, "http-remoting://localhost:8080/");
//jndiProperties.put(Context.SECURITY_PRINCIPAL, "test");
//jndiProperties.put(Context.SECURITY_CREDENTIALS, "test");
return jndiProperties;
}
private static AddBrackets getEjb(Properties jndiProps)
throws NamingException {
final Context jndiContext = new InitialContext(jndiProps);
final String interfaceName = AddBrackets.class.getName();
return (AddBrackets) jndiContext.lookup(
"ejbtest-app-1.0-SNAPSHOT/ejbtest-ejb-1.0-SNAPSHOT/AddBracketsBean!"
+ interfaceName);
}
}
I built this program as a Maven project which had a dependency on
<dependency>
<groupId>org.wildfly</groupId>
<artifactId>wildfly-ejb-client-bom</artifactId>
<version>8.2.1.Final</version>
<type>pom</type>
</dependency>
This dependency brings in Wildfly's remote client EJB implementation and adds the following jars to the class path (links are to Maven Central):
jboss-logging-3.1.4.GA.jar
jboss-marshalling-1.4.9.Final.jar
jboss-marshalling-river-1.4.9.Final.jar
jboss-remoting-4.0.7.Final.jar
jboss-sasl-1.0.4.Final.jar
jboss-ejb-api_3.2_spec-1.0.0.Final.jar
jboss-transaction-api_1.2_spec-1.0.0.Final.jar
xnio-api-3.3.0.Final.jar
xnio-nio-3.3.0.Final.jar
jboss-ejb-client-2.0.1.Final.jar
jboss-remote-naming-2.0.1.Final.jar
wildfly-build-config-8.2.1.Final.jar
I did no special configuration on Wildfly to run this example. I simply downloaded a vanilla Wildfly 8.2.1, unzipped it, set up an admin user with the add-user.sh script and deployed my EJB in an EAR. As you can see above access is granted without a username and a password.
You can find the complete project including the AddBrackets EJB on my bitbucket account.
When I tried to get my head around remote EJBs with Wildfly, I found the article JBoss EAP / Wildfly – Three ways to invoke remote EJBs really helpful. It clearly describes the three different methods to access remote EJBs on Wildfly.

According to your own answer the following jars are on your classpath:
jboss-remote-naming-1.0.7.final.jar
jboss-logging.jar
xnio-api-3.0.7.ga.jar
jboss-remoting-3.jar
jboss-ejb-client-1.0.19.final.jar
You write that the application throws the following exception:
java.lang.NoSuchMethodError:
org.jboss.remoting3.Remoting.createEndpoint(Ljava/lang/String;Lorg/xnio/OptionMap;)Lorg/jboss/remoting3/Endpoint;]
This exception is thrown when org.jboss.naming.remote.client.EndpointCache which is part of the jboss-remote-naming jar tries to call Remoting.createEndpoint() which is contained in the jboss-remoting jar.
As you explain in your answer the reason for this is that the Remoting class declares a 3-parameter version of the createEndpoint() method while the EndpointCache class tries to call a 2-parameter version which does not exist.
I checked the commit histories and declared dependencies of the jboss-remote-naming and the jboss-remoting projects to find out what is going wrong there. This is what I found out:
The 2-parameter version of createEndpoint() was only added in version 3.2 of jboss-remoting. The pom.xml for jboss-remote-naming-1.0.7.final says it depends on jboss-remoting 3.2.7.GA.
As there is no version number on your jboss-remoting-3.jar, I guess it is an older version. You should be able to check this by looking for a pom.xml in META-INF folder of your jboss-remoting-3.jar. This should contain the version number.
To solve your problem, I suggest to replace your jboss-remoting-3.jar with jboss-remoting-3.2.7ga.jar or to use the set of jars I listed in my other answer.

I’ve decided the problem isn’t coding or the JNDI InititialContext Properties.
I mean the fatal error is a NoSuchMethodError. Therefore, as I confirmed in the WildFly server logs, my main method never even tries to connect.
Here’s what I think explains the real problem.
And I think it explains why there are so many calls for help with this error:
java.lang.NoSuchMethodError:
org.jboss.remoting3.Remoting.createEndpoint(Ljava/lang/String;Lorg/xnio/OptionMap;)Lorg/jboss/remoting3/Endpoint;]
Also why none of those calls for help ever get a conclusive answer. Just people suggesting different jars.
And since all those answers fixed on jars, this is how I tested the Build Path I was using:
First I removed all jars from the Build Path. Then I ran my one line main program till all ClassNotFoundException were gone.
First Error
java.lang.ClassNotFoundException:
org.jboss.naming.remote.client.InitialContextFactory]
Added jboss-remote-naming-1.0.7.final.jar to class path
Next Error
java.lang.NoClassDefFoundError:
org/jboss/logging/Logger
Added jboss-logging.jar
Next Error
java.lang.NoClassDefFoundError:
org/xnio/Options
Added xnio-api-3.0.7.ga.jar
Next Error
java.lang.NoClassDefFoundError:
org/jboss/remoting3/spi/ConnectionProviderFactory
Added jboss-remoting-3.jar
Next Error
java.lang.NoClassDefFoundError:
org/jboss/ejb/client/EJBClientContextIdentifier
Added jboss-ejb-client-1.0.19.final.jar
FATAL ERROR (note: All NoClassDefFoundError have been cleared)
java.lang.NoSuchMethodError:
org.jboss.remoting3.Remoting.createEndpoint(Ljava/lang/String;Lorg/xnio/OptionMap;)Lorg/jboss/remoting3/Endpoint;]
Then I used Eclipse’s Project Explorer to verify:
That jboss-remoting3.jar has the org.jboss.remoting3.Remoting Class. It does. That’s why there is no NoClassDefFoundError left above.
And verified it had this method:
public Endpoint createEndpoint (String, Executor, OptionMap) note: 3 parameters.
BUT the above Error indicates something is calling:
public Endpoint createEndpoint (String, OptionMap) note: 2 parameters.
That’s why the program throws a NoSuchMethodError. It is looking for a 2 paramater version of org.jboss.remoting3.Remoting.createEndpoint(). And the Remoting Class I have only has a 3 parameter version.`
I know this sounds impossible but the only thing I can think is there is an inconsistency in the Java API???
Clearly something is calling org.jboss.remoting3.Remoting.createEndpoint with 2 parameters.
But my org.jboss.remoting3.Remoting Class only has a 3 parameter version of the createEndpoint() Method.
So I’m going to clean this all up and repost a question asking how to explain the existence of a Class calling for a 2 paramter org.jboss.remoting3.Remoting.createEndpoint Method when I have a jar whose org.jboss.remoting3.Remoting only offers a 3-parameter.

Here is your obligatory "that's a dumb question." Does the wildfly remote quickstart github repo answer the question for you? Their code, from RemoteEJB.java
final Hashtable<String, String> jndiProperties = new Hashtable<>();
jndiProperties.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");
final Context context = new InitialContext(jndiProperties);
return (RemoteCalculator) context.lookup("ejb:/ejb-remote-server-side/CalculatorBean!" + RemoteCalculator.class.getName());

No suitable native library found. native.libpath.* vs java.library.path

I encounter "No suitable native library found" when running some library (HDF5).
Full trace is follows:
java.lang.ExceptionInInitializerError
at ch.systemsx.cisd.hdf5.hdf5lib.HDF5Constants.javaToC(HDF5Constants.java:1938)
at ch.systemsx.cisd.hdf5.hdf5lib.HDF5Constants.<clinit>(HDF5Constants.java:982)
at ch.systemsx.cisd.hdf5.CharacterEncoding.<clinit>(CharacterEncoding.java:29)
at ch.systemsx.cisd.hdf5.HDF5BaseReader.<init>(HDF5BaseReader.java:137)
at ch.systemsx.cisd.hdf5.HDF5BaseWriter.<init>(HDF5BaseWriter.java:147)
at ch.systemsx.cisd.hdf5.HDF5WriterConfigurator.writer(HDF5WriterConfigurator.java:133)
at ch.systemsx.cisd.hdf5.HDF5FactoryProvider$HDF5Factory.open(HDF5FactoryProvider.java:48)
at ch.systemsx.cisd.hdf5.HDF5Factory.open(HDF5Factory.java:47)
at tests.jhdf5.TestHDF5.testReaderWriter(TestHDF5.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
...
Caused by: java.lang.UnsupportedOperationException: No suitable HDF5 native library found for this platform.
at ch.systemsx.cisd.hdf5.hdf5lib.H5.<clinit>(H5.java:41)
... 30 more
As you see, library is called from inside HDF5 java library.
Java library is attached as bunch of jars in global libraries:
As you see, native library is included.
The question is whether this setting is sufficient? May be I require to add some directories to the PATH variable?
UPDATE
I found, that HDF5 code expects library inside native.libpath.jhdf5, the code is from ch.systemsx.cisd.base.utilities.NativeLibraryUtilities#loadNativeLibrary():
public static boolean loadNativeLibrary(String libraryName) {
String linkLibNameOrNull = System.getProperty("native.libpath." + libraryName);
if(linkLibNameOrNull != null) {
return loadLib(linkLibNameOrNull);
} else {
String linkLibPathOrNull = System.getProperty("native.libpath");
if(linkLibPathOrNull != null) {
linkLibNameOrNull = getLibPath(linkLibPathOrNull, libraryName);
return loadLib(linkLibNameOrNull);
} else {
linkLibNameOrNull = tryCopyNativeLibraryToTempFile(libraryName);
return linkLibNameOrNull != null?loadLib(linkLibNameOrNull):loadSystemLibrary(libraryName);
}
}
}
While IntelliJ sets java.library.path:
public static void main(String[] args) {
System.out.println("native.libpath.jhdf5 = " + System.getProperty("native.libpath.jhdf5") );
System.out.println("java.library.path = " + System.getProperty("java.library.path") );
prints:
native.libpath.jhdf5 = null
java.library.path = D:\Users\Dims\Design\!Lib\sis-jhdf5-SNAPSHOT-r32355\sis-jhdf5\lib\nativejar\hdf5-windows-intel.jar
Why?

You may have the same problem as me.
The problem was that i had include the jar for the 3 differents platforms (linux, windows and macOS) thus there was a conflict and this problem happen
If it does not solved the case (as it has do in mine) i suggest to consult the jhdf5 faq https://wiki-bsse.ethz.ch/display/JHDF5/JHDF5+FAQ

Getting Spring-XD and the hdfs sink to work for maprfs

This is a question about spring-xd release 1.0.1 working together with maprfs, which is officially not yet supported. Still I would like to get it to work.
So this is what we did:
1) adjusted the xd-shell and xd-worker and xd-singlenode shell scripts to accept the parameter --hadoopDistro mapr
2) added libraries to the new directory $XD_HOME/lib/mapr
avro-1.7.4.jar jersey-core-1.9.jar
hadoop-annotations-2.2.0.jar jersey-server-1.9.jar
hadoop-core-1.0.3-mapr-3.0.2.jar jetty-util-6.1.26.jar
hadoop-distcp-2.2.0.jar maprfs-1.0.3-mapr-3.0.2.jar
hadoop-hdfs-2.2.0.jar protobuf-java-2.5.0.jar
hadoop-mapreduce-client-core-2.2.0.jar spring-data-hadoop-2.0.2.RELEASE-hadoop24.jar
hadoop-streaming-2.2.0.jar spring-data-hadoop-batch-2.0.2.RELEASE-hadoop24.jar
hadoop-yarn-api-2.2.0.jar spring-data-hadoop-core-2.0.2.RELEASE-hadoop24.jar
hadoop-yarn-common-2.2.0.jar spring-data-hadoop-store-2.0.2.RELEASE-hadoop24.jar
3) run bin/xd-singlenode --hadoopDistro mapr and shell/bin/xd-shell --hadoopDistro mapr.
When creating and deploying a stream via stream create foo --definition "time | hdfs" --deploy, data is written to a file tmp/xd/foo/foo-1.txt.tmp on maprfs. Yet when undeploying the stream, the following exceptions appears:
org.springframework.data.hadoop.store.StoreException: Failed renaming from /xd/foo/foo-1.txt.tmp to /xd/foo/foo-1.txt; nested exception is java.io.FileNotFoundException: Requested file /xd/foo/foo-1.txt does not exist.
at org.springframework.data.hadoop.store.support.OutputStoreObjectSupport.renameFile(OutputStoreObjectSupport.java:261)
at org.springframework.data.hadoop.store.output.TextFileWriter.close(TextFileWriter.java:92)
at org.springframework.xd.integration.hadoop.outbound.HdfsDataStoreMessageHandler.doStop(HdfsDataStoreMessageHandler.java:58)
at org.springframework.xd.integration.hadoop.outbound.HdfsStoreMessageHandler.stop(HdfsStoreMessageHandler.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201)
at com.sun.proxy.$Proxy120.stop(Unknown Source)
at org.springframework.integration.endpoint.EventDrivenConsumer.doStop(EventDrivenConsumer.java:64)
at org.springframework.integration.endpoint.AbstractEndpoint.stop(AbstractEndpoint.java:100)
at org.springframework.integration.endpoint.AbstractEndpoint.stop(AbstractEndpoint.java:115)
at org.springframework.integration.config.ConsumerEndpointFactoryBean.stop(ConsumerEndpointFactoryBean.java:303)
at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:229)
at org.springframework.context.support.DefaultLifecycleProcessor.access$300(DefaultLifecycleProcessor.java:51)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.stop(DefaultLifecycleProcessor.java:363)
at org.springframework.context.support.DefaultLifecycleProcessor.stopBeans(DefaultLifecycleProcessor.java:202)
at org.springframework.context.support.DefaultLifecycleProcessor.stop(DefaultLifecycleProcessor.java:106)
at org.springframework.context.support.AbstractApplicationContext.stop(AbstractApplicationContext.java:1186)
at org.springframework.xd.module.core.SimpleModule.stop(SimpleModule.java:234)
at org.springframework.xd.dirt.module.ModuleDeployer.destroyModule(ModuleDeployer.java:132)
at org.springframework.xd.dirt.module.ModuleDeployer.handleUndeploy(ModuleDeployer.java:111)
at org.springframework.xd.dirt.module.ModuleDeployer.undeploy(ModuleDeployer.java:83)
at org.springframework.xd.dirt.server.ContainerRegistrar.undeployModule(ContainerRegistrar.java:261)
at org.springframework.xd.dirt.server.ContainerRegistrar$StreamModuleWatcher.process(ContainerRegistrar.java:884)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: java.io.FileNotFoundException: Requested file /xd/foo/foo-1.txt does not exist.
at com.mapr.fs.MapRFileSystem.getMapRFileStatus(MapRFileSystem.java:805)
at com.mapr.fs.MapRFileSystem.delete(MapRFileSystem.java:629)
at org.springframework.data.hadoop.store.support.OutputStoreObjectSupport.renameFile(OutputStoreObjectSupport.java:258)
... 29 more
I had a look at the OutputStoreObjectSupport.renameFile() function. When a file on hdfs is finished, this method tries to rename the file /xd/foo/foo-1.txt.tmp to xd/foo/foo1.txt. This is the relevant code:
try {
FileSystem fs = path.getFileSystem(getConfiguration());
boolean succeed;
try {
fs.delete(toPath, false);
log.info("Renaming path=[" + path + "] toPath=[" + toPath + "]");
succeed = fs.rename(path, toPath);
} catch (Exception e) {
throw new StoreException("Failed renaming from " + path + " to " + toPath, e);
}
if (!succeed) {
throw new StoreException("Failed renaming from " + path + " to " + toPath + " because hdfs returned false");
}
}
When the target file does not exist on hdfs, maprfs seems to throw an exception when fs.delete(toPath, false) is called. Yet throwing an exception in this case does not make sense. I assume that other Filesystem implementations behave differently, but this is a point I still need to verify. Unfortuntately I cannot find the sources for MapRFileSystem.java. Is this closed source? This would help me to better understand the issue. Has anybody experience with writing from spring-xd to maprfs? Or renaming files on maprfs with spring-data-hadoop?
Edit
I managed to reproduce the issue outside of spring XD with a simple test case (see below). Note that this exception is only thrown if the inWritingSuffix or the inWritingPrefix is set. Otherwise spring-hadoop will not attempt to rename the file. So this is the still somehow unsatisfactory workaround for me: refrain from using inWritingPrefixes and inWritingSuffixes.
#ContextConfiguration("context.xml")
#RunWith(SpringJUnit4ClassRunner.class)
public class MaprfsSinkTest {
#Autowired
Configuration configuration;
#Autowired
FileSystem filesystem;
#Autowired
DataStoreWriter<String >storeWriter;
#Test
public void testRenameOnMaprfs() throws IOException, InterruptedException {
Path testPath = new Path("/tmp/foo.txt");
filesystem.delete(testPath, true);
TextFileWriter writer = new TextFileWriter(configuration, testPath, null);
writer.setInWritingSuffix("tmp");
writer.write("some entity");
writer.close();
}
#Test
public void testStoreWriter() throws IOException {
this.storeWriter.write("something");
}
}

I created a new branch for spring-hadoop which supports maprfs:
https://github.com/blinse/spring-hadoop/tree/origin/2.0.2.RELEASE-mapr
Building this release and using the resulting jar works fine with the hdfs sink.

WEKA - Error on assigning Instances to kmeans.buildClusterer

I'm pretty new in Weka framework. So far i find it pretty simple and easy to use and understand but i'm facing some problems i cannot understand. I'm trying to cluster a dataset from an csv file. I got a class that clusters the dataset using kmeans and as far as i read in the Weka's wiki (http://weka.wikispaces.com/) the class works fine. The only problem was that it was reading an arff file and i needed to read from a csv file. So far so good. The problem is when i assign the instances from the dataset to kmeans.buildClusterer(data) method. Then i get the following exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/pentaho/packageManagement/PackageManager
at weka.core.WekaPackageManager.<clinit>(WekaPackageManager.java:86)
at weka.core.Utils.readProperties(Utils.java:142)
at weka.core.Capabilities.<init>(Capabilities.java:261)
at weka.clusterers.AbstractClusterer.getCapabilities(AbstractClusterer.java:179)
at weka.clusterers.SimpleKMeans.getCapabilities(SimpleKMeans.java:289)
at weka.clusterers.SimpleKMeans.buildClusterer(SimpleKMeans.java:441)
at weka_examples.ClusteringDemo.<init>(ClusteringDemo.java:48)
at weka_examples.ClusteringDemo.main(ClusteringDemo.java:70)
Caused by: java.lang.ClassNotFoundException: org.pentaho.packageManagement.PackageManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 8 more
Java Result: 1
The code is the following:
import weka.core.Instances;
import weka.clusterers.DensityBasedClusterer;
import weka.clusterers.ClusterEvaluation;
import java.io.File;
import weka.clusterers.SimpleKMeans;
import weka.core.converters.CSVLoader;
/**
* An example class that shows the use of Weka clusterers from Java.
*
* #author FracPete
*/
public class ClusteringDemo {
/**
* Run clusterers
*
* #param filename the name of the ARFF file to run on
*/
public ClusteringDemo(String filename) throws Exception {
ClusterEvaluation eval;
Instances data;
String[] options;
DensityBasedClusterer cl;
String Origem = filename;
// data = new Instances(new BufferedReader(new FileReader(filename)));
// load CSV
CSVLoader loader = new CSVLoader();
loader.setSource(new File(Origem));
data = loader.getDataSet();
SimpleKMeans kmeans = new SimpleKMeans();
kmeans.setSeed(10);
// This is the important parameter to set
kmeans.setPreserveInstancesOrder(true);
kmeans.setNumClusters(5);
kmeans.buildClusterer(data);
// This array returns the cluster number (starting with 0) for each instance
// The array has as many elements as the number of instances
int[] assignments = kmeans.getAssignments();
int i = 0;
for (int clusterNum : assignments) {
System.out.printf("Instance %d -> Cluster %d", i, clusterNum);
i++;
}
}
/**
* usage: ClusteringDemo arff-file
*/
public static void main(String[] args) throws Exception {
/*if (args.length != 1) {
System.out.println("usage: " + ClusteringDemo.class.getName() + " <arff-file>");
System.exit(1);
}*/
new ClusteringDemo("Teste/Query1.csv");
}
}
I included the binary file: weka-dev-3.7.10
Am i missing something important?
Kind regards

I finally managed to answer this question. It turns out that, from version 3.7.2, weka core is much more compact, meaning that some packages i need were not present in the weka-dev.
The solution is simple: add to the classpath the weka.jar and not the weka-dev package. Alternatively the missing packages can also be installed http://weka.wikispaces.com/How+do+I+use+the+package+manager%3F
Best regards

PHP/Java bridge problem

I am using tomcat 6 on windows. Here is the code I am testing.
import java.io.ByteArrayOutputStream;
import java.io.Closeable;
import java.io.StringReader;
import javax.script.Invocable;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
/**
* Create and run THREAD_COUNT PHP threads, concurrently accessing a
* shared resource.
*
* Create 5 script engines, passing each a shared resource allocated
* from Java. Each script engine has to implement Runnable.
*
* Java accesses the Runnable script engine using
* scriptEngine.getInterface() and calls thread.start() to invoke each
* PHP Runnable implementations concurrently.
*/
class PhpThreads {
public static final String runnable = new String("<?php\n" +
"function run() {\n" +
" $out = java_context()->getAttribute('sharedResource', 100);\n" +
" $nr = (string)java_context()->getAttribute('nr', 100);\n" +
" echo \"started thread: $nr\n\";\n" +
" for($i=0; $i<100; $i++) {\n" +
" $out->write(ord($nr));\n" +
" java('java.lang.Thread')->sleep(1);\n" +
" }\n" +
"}\n" +
"?>\n");
static final int THREAD_COUNT = 5;
public static void main(String[] args) throws Exception {
ScriptEngineManager manager = new ScriptEngineManager();
Thread threads[] = new Thread[THREAD_COUNT];
ScriptEngine engines[] = new ScriptEngine[THREAD_COUNT];
ByteArrayOutputStream sharedResource = new ByteArrayOutputStream();
StringReader runnableReader = new StringReader(runnable);
// create THREAD_COUNT PHP threads
for (int i=0; i<THREAD_COUNT; i++) {
engines[i] = manager.getEngineByName("php-invocable");
if (engines[i] == null)
throw new NullPointerException ("php script engine not found");
engines[i].put("nr", new Integer(i+1));
engines[i].put("sharedResource", sharedResource);
engines[i].eval(runnableReader);
runnableReader.reset();
// cast the whole script to Runnable; note also getInterface(specificClosure, type)
Runnable r = (Runnable) ((Invocable)engines[i]).getInterface(Runnable.class);
threads[i] = new Thread(r);
}
// run the THREAD_COUNT PHP threads
for (int i=0; i<THREAD_COUNT; i++) {
threads[i].start();
}
// wait for the THREAD_COUNT PHP threads to finish
for (int i=0; i<THREAD_COUNT; i++) {
threads[i].join();
((Closeable)engines[i]).close();
}
// print the output generated by the THREAD_COUNT concurrent threads
String result = sharedResource.toString();
System.out.println(result);
// Check result
Object res=manager.getEngineByName("php").eval(
"<?php " +
"exit((int)('10011002100310041005'!=" +
"#system(\"echo -n "+result+"|sed 's/./&\\\n/g'|sort|uniq -c|tr -d ' \\\n'\")));" +
"?>");
System.exit(((Number)res).intValue());
}
}
I have added all the libraries. When I run the file I get the following error -
run:
Exception in thread "main" javax.script.ScriptException: java.io.IOException: Cannot run program "php-cgi": CreateProcess error=2, The system cannot find the file specified
at php.java.script.InvocablePhpScriptEngine.eval(InvocablePhpScriptEngine.java:209)
at php.java.script.SimplePhpScriptEngine.eval(SimplePhpScriptEngine.java:178)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:232)
at PhpThreads.main(NewClass.java:53)
Caused by: java.io.IOException: Cannot run program "php-cgi": CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
at java.lang.Runtime.exec(Runtime.java:593)
at php.java.bridge.Util$Process.start(Util.java:1064)
at php.java.bridge.Util$ProcessWithErrorHandler.start(Util.java:1166)
at php.java.bridge.Util$ProcessWithErrorHandler.start(Util.java:1217)
at php.java.script.CGIRunner.doRun(CGIRunner.java:126)
at php.java.script.HttpProxy.doRun(HttpProxy.java:63)
at php.java.script.CGIRunner.run(CGIRunner.java:111)
at php.java.bridge.ThreadPool$Delegate.run(ThreadPool.java:60)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(ProcessImpl.java:81)
at java.lang.ProcessImpl.start(ProcessImpl.java:30)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
... 8 more
What am I missing?

Just add this to your command line:
-Dphp.java.bridge.php_exec=/usr/bin/php
Problem solved!

copy the correct VERSION of the PHP of php5ts.dll and php-cgi.exe to "WEB-INF\cgi\amd64-windows" directory. then restart Tomcat. good luck.

...php-cgi...The system cannot find the file specified
I'm guessing that manager.getEngineByName("php-invocable") should return a wrapper around a system call to run PHP - but that wrapper doesn't know where to find the PHP executable.
A quick glance at the website for the PHP/Java bridge, and I infer that the path is hard coded in the Java - "For further information please see the INSTALL.J2EE file from the documentation download"
The Javadoc is decidedly vague on the topic.
You need to specifically make the -cgi version of PHP at compile time, assuming you've done that, and it is called php-cgi, then as a quick hack you could perpper your filesystem with links named "php-cgi" (its probably expected to be in /bin, /usr/bin/, /usr/local/bin, or the Java may be smart enough to check $PATH)
C.

when you get error like
Fatal Error: Failed to start PHP ["php-cgi", "-v"], reason: java.io.IOException:
Cannot run program ""php-cgi"" (in directory "C:\Documents and Settings\Adminis
trator"): CreateProcess error=2, The system cannot find the file specified
Could not start FCGI server: java.io.IOException: PHP not found. Please install
php-cgi. PHP test command was: [php-cgi, -v]
php.java.bridge.http.FCGIConnectException: Could not connect to server
at php.java.bridge.http.NPChannelFactory.test(NPChannel`enter code here`Factory.java:64)
at php.java.bridge.http.FCGIConnectionPool.<init>(FCGIConnectionPool.jav
a:175)
at php.java.bridge.http.FCGIConnectionPool.<init>(FCGIConnectionPool.jav
a:189)
at php.java.servlet.ContextLoaderListener.createConnectionPool(ContextLo
aderListener.java:541)
at php.java.servlet.ContextLoaderListener.contextInitialized(ContextLoad
erListener.java:185)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContex
t.java:4135)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4
630)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase
.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:77
1)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:546)
at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.jav
a:1041)
at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.j
ava:964)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502
)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java
:321)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Lifecycl
eSupport.java:119)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:785)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:445
)
at org.apache.catalina.core.StandardService.start(StandardService.java:5
19)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710
)
at org.apache.catalina.startup.Catalina.start(Catalina.java:581)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.io.IOException: File \\.\pipe\C:\Documents and Settings\Administ
rator\Desktop\softwares\apache-tomcat-6.0.29\temp\JavaBridge3144995283109409611.
socket not writable
at php.java.bridge.http.FCGIConnectException.<init>(FCGIConnectException
.java:37)
... 29 more
Caused by: java.io.IOException: PHP not found. Please install php-cgi. PHP test
command was: [php-cgi, -v]
at php.java.bridge.Util$Process.start(Util.java:1145)
at php.java.servlet.fastcgi.FCGIProcess.start(FCGIProcess.java:68)
at php.java.bridge.http.NPChannelFactory.doBind(NPChannelFactory.java:94
)
at php.java.bridge.http.FCGIConnectionFactory.runFcgi(FCGIConnectionFact
ory.java:88)
at php.java.bridge.http.FCGIConnectionFactory$1.run(FCGIConnectionFactor
y.java:109)
with the JavaBridge.war deployment in (windows, tomcat)
please specify the path for the php installation in your environment variable.