Please note: Although there are several questions on SO that paste in a similar exception & stack trace, this question is definitely not a dupe of any of them, as I'm trying to understand where my classloading is going awry.
Hi, Java 8/Groovy 2.4.3/Eclipse Luna here. I'm using the BigIP iControl Java client (for controlling a powerful load balancer programmatically) which in turn uses Apache Axis 1.4. In its use of Axis 1.4 I am getting the following stacktrace (from Eclipse console):
Caught: javax.xml.parsers.FactoryConfigurationError: Provider for javax.xml.parsers.DocumentBuilderFactory cannot be found
javax.xml.parsers.FactoryConfigurationError: Provider for javax.xml.parsers.DocumentBuilderFactory cannot be found
at org.apache.axis.utils.XMLUtils.getDOMFactory(XMLUtils.java:221)
at org.apache.axis.utils.XMLUtils.<clinit>(XMLUtils.java:83)
at org.apache.axis.configuration.FileProvider.configureEngine(FileProvider.java:179)
at org.apache.axis.AxisEngine.init(AxisEngine.java:172)
at org.apache.axis.AxisEngine.<init>(AxisEngine.java:156)
at org.apache.axis.client.AxisClient.<init>(AxisClient.java:52)
at org.apache.axis.client.Service.getAxisClient(Service.java:104)
at org.apache.axis.client.Service.<init>(Service.java:113)
at iControl.LocalLBPoolLocator.<init>(LocalLBPoolLocator.java:21)
at iControl.Interfaces.getLocalLBPool(Interfaces.java:351)
at com.me.myapp.F5Client.run(F5Client.groovy:27)
Hmmm, let's have a look at that XMLUtils.getDOMFactory method:
private static DocumentBuilderFactory getDOMFactory() {
DocumentBuilderFactory dbf;
try {
dbf = DocumentBuilderFactory.newInstance();
dbf.setNamespaceAware(true);
}
catch( Exception e ) {
log.error(Messages.getMessage("exception00"), e );
dbf = null;
}
return( dbf );
}
OK, LN 221 is that call to DocumentBuilderFactory.newInstance() so let's have a look at it:
public static DocumentBuilderFactory newInstance() {
return FactoryFinder.find(
/* The default property name according to the JAXP spec */
DocumentBuilderFactory.class, // "javax.xml.parsers.DocumentBuilderFactory"
/* The fallback implementation class name */
"com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl");
}
The plot thickens! Now let's take a final look at FactoryFinder.find:
static <T> T find(Class<T> type, String fallbackClassName)
throws FactoryConfigurationError
{
final String factoryId = type.getName();
dPrint("find factoryId =" + factoryId);
// lots of nasty cruft omitted for brevity...
// Try Jar Service Provider Mechanism
T provider = findServiceProvider(type);
if (provider != null) {
return provider;
}
if (fallbackClassName == null) {
throw new FactoryConfigurationError(
"Provider for " + factoryId + " cannot be found"); // <<-- Ahh, here we go
}
dPrint("loaded from fallback value: " + fallbackClassName);
return newInstance(type, fallbackClassName, null, true);
}
So if I'm interpreting this right, it's throwing the FactoryConfigurationError because it can't find the main "provider class" (whatever that means) and no fallback has been specified. But hasn't it?!? The call to FactoryFinder.find included the non-null "com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl" string argument. This has me suspicious that something is really wonky with my classpath, and that I have a rogue DocumentBuilderFactory (not the one defined in rt.jar/javax/xml/parsers) somewhere in my code that is passing a NULL arg to this finder method.
But that doesn't make sense either, because Axis 1.4 doesn't appear (at least according to Maven repo) to have any dependencies...which means the only "provider" for javax.xml.* would be the rt.jar. Unless, perhaps, Eclipse is mucking things up somehow? I'm so confused, please help :-/
Update
This is definitely an Eclipse issue. If I package my app as an executable JAR and run it from the command line I don't get this exception.
Related
I'm currently trying to add a URLStreamHandler so I can handle URLs with custom protocols. This works fine when run locally. When deployed to AWS Lambda I get:
java.net.MalformedURLException: unknown protocol: baas
I'm following the "Handler" approach to registering the URLStreamHandler.
I even went as far as copying the code from URL.getURLStreamHandler(String) and added logging into my own code that is run by Lambda:
(Note: this is from the Java 8 source - I realise now that this might not be representative because AWS Lambda uses a Java 11 runtime).
URLStreamHandler handler = null;
String packagePrefixList = null;
packagePrefixList
= java.security.AccessController.doPrivileged(
new sun.security.action.GetPropertyAction(
"java.protocol.handler.pkgs",""));
if (packagePrefixList != "") {
packagePrefixList += "|";
}
// REMIND: decide whether to allow the "null" class prefix
// or not.
packagePrefixList += "sun.net.www.protocol";
LOG.debug("packagePrefixList: " + packagePrefixList);
StringTokenizer packagePrefixIter =
new StringTokenizer(packagePrefixList, "|");
while (handler == null &&
packagePrefixIter.hasMoreTokens()) {
String packagePrefix =
packagePrefixIter.nextToken().trim();
try {
String clsName = packagePrefix + "." + "baas" +
".Handler";
Class<?> cls = null;
LOG.debug("Try " + clsName);
try {
cls = Class.forName(clsName);
} catch (ClassNotFoundException e) {
ClassLoader cl = ClassLoader.getSystemClassLoader();
if (cl != null) {
cls = cl.loadClass(clsName);
}
}
if (cls != null) {
LOG.debug("Instantiate " + clsName);
handler =
(URLStreamHandler)cls.newInstance();
}
} catch (Exception e) {
// any number of exceptions can get thrown here
LOG.debug(e);
}
}
This prints (in Cloudwatch logs):
packagePrefixList: com.elsten.bliss|sun.net.www.protocol (BaasDriver.java:94, thread main)
Try com.elsten.bliss.baas.Handler (BaasDriver.java:108, thread main)
Instantiate com.elsten.bliss.baas.Handler (BaasDriver.java:118, thread main)
com.elsten.bliss.baas.Handler constructor (Handler.java:55, thread main)
So, when run from my own code, in Lambda, it works.
However, the very next line of logging:
java.lang.IllegalArgumentException: URL is malformed: baas://folder: java.lang.RuntimeException
java.lang.RuntimeException: java.lang.IllegalArgumentException: URL is malformed: baas://folder
...
Caused by: java.net.MalformedURLException: unknown protocol: baas
at java.base/java.net.URL.<init>(Unknown Source)
at java.base/java.net.URL.<init>(Unknown Source)
at java.base/java.net.URL.<init>(Unknown Source)
So it seems odd the same code is failing when run in URL. The main difference I can think of is the parent classloader used to load URL and my code are different, and so there's some sort of class loading issue.
The SPI approach can't be used because Lambda doesn't extract META-INF folders!
Initially I thought the old URL.setURLStreamHandlerFactory(URLStreamHandlerFactory) approach was to be avoided, but it turns out this has been improved in recent Java versions, and so I have fallen back to that.
Specifically, a default fallback URLStreamHandlerFactory which is capable of handling streams to http, https, file et al is used as a fallback if the custom one provided cannot handle a stream.
This is a workaround though - it would be interesting to know why the class cannot be loaded.
ADDED 7/23.
Many views: Not even a "that's dumb" question in response. Can anyone at least tell me why such an embarrassingly trivial question seems to have no answer anywhere.
Q:
--- Have Wildfly 8 running on local machine localhost:9990.
--- Have a Java program that need's Wildfly's IntialContext.
--- Every reference says use: "Context ctx = new InitialContext(env);"
--- Yet a week of searching turns up no set of properties that returns one.
And no example of a java program that gets one.
Does no one ever do this? Really need help
Original Msg Below
I know many people have asked how to get an Initial context from Wildfly 8. But I have yet to find a simple answer with a simple example.
Therefore, I hope someone can tell my why this doesn’t work.
I start Wildfly with standalone-full.xml
The three sections below have
A - Code summary of my test Class whose only purpose is to secure an Initial Context. (I only removed a lot of printing code that produced the next section.]
B - The Eclipse console output for a failure.
C - Cut and paste code. Just in case anyone can help me get this to work. I’d like to leave behind something the next new WF user can cut and past and run. The only difference from 1 above is that this version has all the static methods I used to format the output. NOTE: I know the comments I inserted about the less than sign sound dumb. BUT ... they are true.
A Code Summary
import java.util.Properties;
import javax.naming.CommunicationException;
import javax.naming.Context;
import javax.naming.InitialContext;
public class JmsTestGetJNDIContext {
//members
final private Properties env = new Properties() {
private static final long serialVersionUID = 1L;
{
/* These are Properties used by a standalone JavaClient to secure a WIldFly InitialContext()*/
put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
put(Context.PROVIDER_URL,"http-remoting://localhost:9990");
put(Context.SECURITY_PRINCIPAL,"userGLB");
put(Context.SECURITY_CREDENTIALS,"Open");
put("jboss.naming.client.ejb.context", true);
/*The above URL, ID and PW successfully open Wildfly's Admin Console*/
}
};
//constructor
private JmsTestGetJNDIContext (){
/*print "beg"*/
/*print "env"*/
try {
/*print "Requesting InitialContext"*/
Context ctx = new InitialContext(this.env);
/*print "JNDI Context: " + ctx)*/
/*print "end");
} catch (CommunicationException e) {
/* print "You forgot to start WildFly dummy!"*/
} catch (Exception e) {
/* print"caught: " + e.getClass().getName()*/
/*print e.getMessage()*/
/* "end")*/
}
static public void main (String[] args) {
/*print "beg"*/
JmsTestGetJNDIContext client = new JmsTestGetJNDIContext ();
/*print "end"*/
}
}
B - Console Output
JmsTestGetJNDIContext.main () beg
JmsTestGetJNDIContext.<init> () beg
JmsTestGetJNDIContext.<init> () These are Properties used to obtain IntialContext
Key: java.naming.provider.url
Value: http-remoting://localhost:9990
Key: java.naming.factory.initial
Value: org.jboss.naming.remote.client.InitialContextFactory
Key: jboss.naming.client.ejb.context
Value: true
Key: java.naming.security.principal
Value: userGLB
Key: java.naming.security.credentials
Value: Open
JmsTestGetJNDIContext.<init> () Requesting InitialContext
JmsTestGetJNDIContext.<init> () caught: javax.naming.NamingException
JmsTestGetJNDIContext.<init> () Failed to create remoting connection
JmsTestGetJNDIContext.<init> () end
JmsTestGetJNDIContext.main () end
Cut and Paste Code
package org.america3.gotest.xtra;
import java.util.Properties;
import javax.naming.CommunicationException;
import javax.naming.Context;
import javax.naming.InitialContext;
public class JmsTestGetJNDIContext {
//members
final private Properties env = new Properties() {
/**
* Properties used by a standalone JavaClient to secure
* a WIldFly InitialContext()*/
private static final long serialVersionUID = 1L;
{
put(Context.INITIAL_CONTEXT_FACTORY,"org.jboss.naming.remote.client.InitialContextFactory");
put(Context.PROVIDER_URL, "http-remoting://localhost:9990");
put(Context.SECURITY_PRINCIPAL, "userGLB");
put(Context.SECURITY_CREDENTIALS, "Open");
// The above URL, ID and PW successfully open Wildfly's Admin Console
put("jboss.naming.client.ejb.context", true);
}
};
//constructor
private JmsTestGetJNDIContext (){/*ignore*/String iAm = JmsTestGetJNDIContext.getIAm(" ", Thread.currentThread().getStackTrace());
P (iAm, "beg");
pProps(iAm, env);
try {
P (sp + iAm, "Requesting InitialContext");
Context ctx = new InitialContext(this.env);
P (sp + iAm, "JNDI Context: " + ctx);
P (sp + iAm, "end");
} catch (CommunicationException e) {
P (sp + iAm, "You forgot to start WildFly dummy!");
} catch (Exception e) {
P (sp + iAm, "caught: " + e.getClass().getName());
P (sp + iAm, e.getMessage());
P (iAm, "end");
}
}
static public void main (String[] args) {/*ignore*/String iAm = JmsTestGetJNDIContext.getIAm("",Thread.currentThread().getStackTrace());
P (iAm, "beg");
JmsTestGetJNDIContext client = new JmsTestGetJNDIContext ();
P (iAm , "end");
}
/*The remaining static methods are just to facilitate printing.
* They are normally in a Untility package I add to my projects.
* I put them here so this code would run for anyone.*/
static private void pProps (String leader, Properties p) {
StringBuffer sb = new StringBuffer ();
String s = JmsTestGetJNDIContext.padRight(leader, 45, ' ');
s = " " + s + "These are Properties used to obtain IntialContext"+"\n";
sb.append(s);
String skip = "";
for (Object key: p.keySet()) {
sb.append(skip + " " + JmsTestGetJNDIContext.padRight("\""
+ (String)key + "\"", 40, ' ')
+ " \"" + p.get(key) + "\"");
skip = "\n";
}
System.out.println(sb);
}
static private void P (String s, String s2) {
System.out.println(s + s2);
}
static public String getClassMethodName (StackTraceElement[] elements) {
String className = null;
for (int i = 0; i * elements.length; i++]i ) {
/* You need to type in a less than sign for the '*'
* because when I do, the editor will not show any code
* that comes after it.
* I have no idea why, but I've spent over an hour trying,
* and every time I type a less than sign all the following
* code dissappears!*/
className = elements[i].getClassName ();
if (className.startsWith ("org.america3")) {
int end = className.lastIndexOf ('.');
return className.substring (end + 1) + "." + elements[i].getMethodName ();
} else {
continue;
}
}
return "no project method found in elements beginning with org.america3" ;
}
static private String getIAm (String indent, StackTraceElement[] elements) {
StringBuffer sb = new StringBuffer ();
sb.append(JmsTestGetJNDIContext.getClassMethodName(elements));
sb.append(" ()");
return indent + JmsTestGetJNDIContext.padRight (sb.toString(), 45, ' ') ;
}
static public String padRight(String s, int width, char c){
if (s == null) return "Null String";
if(s.length() ** width){
/* You need to type in a greater than or equal sign for
* the '**'see above.*/
return s;
} else {
StringBuffer sb = new StringBuffer();
sb.append (s);
for(int i = 0; i *** (width - s.length()); i++){
/*You need to type in a less than sign the '***'. Again see above*/
sb.append(c);
}
return sb.toString();
}
}
static public String sp = " ";
}
A while ago I also struggled with remote EJBs in my CLI application. I excavated a small example project that I wrote then. It gets an InitialContext and calls a remote EJB named AddBrackets:
import java.util.Properties;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import de.dnb.test.ejb.AddBrackets;
public final class Application {
public static void main(String[] args) throws NamingException {
final Properties jndiProperties = initJndiProperties();
final AddBrackets addBrackets = getEjb(jndiProperties);
System.out.println(addBrackets.processText("Hello World"));
}
private static Properties initJndiProperties() {
final Properties jndiProperties = new Properties();
jndiProperties.put(Context.INITIAL_CONTEXT_FACTORY,
"org.jboss.naming.remote.client.InitialContextFactory");
jndiProperties.put("jboss.naming.client.ejb.context", true);
jndiProperties.put(Context.PROVIDER_URL, "http-remoting://localhost:8080/");
//jndiProperties.put(Context.SECURITY_PRINCIPAL, "test");
//jndiProperties.put(Context.SECURITY_CREDENTIALS, "test");
return jndiProperties;
}
private static AddBrackets getEjb(Properties jndiProps)
throws NamingException {
final Context jndiContext = new InitialContext(jndiProps);
final String interfaceName = AddBrackets.class.getName();
return (AddBrackets) jndiContext.lookup(
"ejbtest-app-1.0-SNAPSHOT/ejbtest-ejb-1.0-SNAPSHOT/AddBracketsBean!"
+ interfaceName);
}
}
I built this program as a Maven project which had a dependency on
<dependency>
<groupId>org.wildfly</groupId>
<artifactId>wildfly-ejb-client-bom</artifactId>
<version>8.2.1.Final</version>
<type>pom</type>
</dependency>
This dependency brings in Wildfly's remote client EJB implementation and adds the following jars to the class path (links are to Maven Central):
jboss-logging-3.1.4.GA.jar
jboss-marshalling-1.4.9.Final.jar
jboss-marshalling-river-1.4.9.Final.jar
jboss-remoting-4.0.7.Final.jar
jboss-sasl-1.0.4.Final.jar
jboss-ejb-api_3.2_spec-1.0.0.Final.jar
jboss-transaction-api_1.2_spec-1.0.0.Final.jar
xnio-api-3.3.0.Final.jar
xnio-nio-3.3.0.Final.jar
jboss-ejb-client-2.0.1.Final.jar
jboss-remote-naming-2.0.1.Final.jar
wildfly-build-config-8.2.1.Final.jar
I did no special configuration on Wildfly to run this example. I simply downloaded a vanilla Wildfly 8.2.1, unzipped it, set up an admin user with the add-user.sh script and deployed my EJB in an EAR. As you can see above access is granted without a username and a password.
You can find the complete project including the AddBrackets EJB on my bitbucket account.
When I tried to get my head around remote EJBs with Wildfly, I found the article JBoss EAP / Wildfly – Three ways to invoke remote EJBs really helpful. It clearly describes the three different methods to access remote EJBs on Wildfly.
According to your own answer the following jars are on your classpath:
jboss-remote-naming-1.0.7.final.jar
jboss-logging.jar
xnio-api-3.0.7.ga.jar
jboss-remoting-3.jar
jboss-ejb-client-1.0.19.final.jar
You write that the application throws the following exception:
java.lang.NoSuchMethodError:
org.jboss.remoting3.Remoting.createEndpoint(Ljava/lang/String;Lorg/xnio/OptionMap;)Lorg/jboss/remoting3/Endpoint;]
This exception is thrown when org.jboss.naming.remote.client.EndpointCache which is part of the jboss-remote-naming jar tries to call Remoting.createEndpoint() which is contained in the jboss-remoting jar.
As you explain in your answer the reason for this is that the Remoting class declares a 3-parameter version of the createEndpoint() method while the EndpointCache class tries to call a 2-parameter version which does not exist.
I checked the commit histories and declared dependencies of the jboss-remote-naming and the jboss-remoting projects to find out what is going wrong there. This is what I found out:
The 2-parameter version of createEndpoint() was only added in version 3.2 of jboss-remoting. The pom.xml for jboss-remote-naming-1.0.7.final says it depends on jboss-remoting 3.2.7.GA.
As there is no version number on your jboss-remoting-3.jar, I guess it is an older version. You should be able to check this by looking for a pom.xml in META-INF folder of your jboss-remoting-3.jar. This should contain the version number.
To solve your problem, I suggest to replace your jboss-remoting-3.jar with jboss-remoting-3.2.7ga.jar or to use the set of jars I listed in my other answer.
I’ve decided the problem isn’t coding or the JNDI InititialContext Properties.
I mean the fatal error is a NoSuchMethodError. Therefore, as I confirmed in the WildFly server logs, my main method never even tries to connect.
Here’s what I think explains the real problem.
And I think it explains why there are so many calls for help with this error:
java.lang.NoSuchMethodError:
org.jboss.remoting3.Remoting.createEndpoint(Ljava/lang/String;Lorg/xnio/OptionMap;)Lorg/jboss/remoting3/Endpoint;]
Also why none of those calls for help ever get a conclusive answer. Just people suggesting different jars.
And since all those answers fixed on jars, this is how I tested the Build Path I was using:
First I removed all jars from the Build Path. Then I ran my one line main program till all ClassNotFoundException were gone.
First Error
java.lang.ClassNotFoundException:
org.jboss.naming.remote.client.InitialContextFactory]
Added jboss-remote-naming-1.0.7.final.jar to class path
Next Error
java.lang.NoClassDefFoundError:
org/jboss/logging/Logger
Added jboss-logging.jar
Next Error
java.lang.NoClassDefFoundError:
org/xnio/Options
Added xnio-api-3.0.7.ga.jar
Next Error
java.lang.NoClassDefFoundError:
org/jboss/remoting3/spi/ConnectionProviderFactory
Added jboss-remoting-3.jar
Next Error
java.lang.NoClassDefFoundError:
org/jboss/ejb/client/EJBClientContextIdentifier
Added jboss-ejb-client-1.0.19.final.jar
FATAL ERROR (note: All NoClassDefFoundError have been cleared)
java.lang.NoSuchMethodError:
org.jboss.remoting3.Remoting.createEndpoint(Ljava/lang/String;Lorg/xnio/OptionMap;)Lorg/jboss/remoting3/Endpoint;]
Then I used Eclipse’s Project Explorer to verify:
That jboss-remoting3.jar has the org.jboss.remoting3.Remoting Class. It does. That’s why there is no NoClassDefFoundError left above.
And verified it had this method:
public Endpoint createEndpoint (String, Executor, OptionMap) note: 3 parameters.
BUT the above Error indicates something is calling:
public Endpoint createEndpoint (String, OptionMap) note: 2 parameters.
That’s why the program throws a NoSuchMethodError. It is looking for a 2 paramater version of org.jboss.remoting3.Remoting.createEndpoint(). And the Remoting Class I have only has a 3 parameter version.`
I know this sounds impossible but the only thing I can think is there is an inconsistency in the Java API???
Clearly something is calling org.jboss.remoting3.Remoting.createEndpoint with 2 parameters.
But my org.jboss.remoting3.Remoting Class only has a 3 parameter version of the createEndpoint() Method.
So I’m going to clean this all up and repost a question asking how to explain the existence of a Class calling for a 2 paramter org.jboss.remoting3.Remoting.createEndpoint Method when I have a jar whose org.jboss.remoting3.Remoting only offers a 3-parameter.
Here is your obligatory "that's a dumb question." Does the wildfly remote quickstart github repo answer the question for you? Their code, from RemoteEJB.java
final Hashtable<String, String> jndiProperties = new Hashtable<>();
jndiProperties.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");
final Context context = new InitialContext(jndiProperties);
return (RemoteCalculator) context.lookup("ejb:/ejb-remote-server-side/CalculatorBean!" + RemoteCalculator.class.getName());
I'm trying to connect to SAP ECC 6.0 using JCo. I'm following this tutorial. However, there is a Note saying:
For this example the destination configuration is stored in a file that is called by the program. In practice you should avoid this for security reasons.
And that is reasonable and understood. But, there is no explenation how to set up secure destination provider.
I found solution in this thread that created custom implementation of DestinationDataProvider and that works on my local machine. But when I deploy it on Portal I get an error saying that there is already registered DestinationDataProvider.
So my question is:
How to store destination data in SAP Java EE application?
Here is my code to further clarify what I'm trying to do.
public static void main(String... args) throws JCoException {
CustomDestinationProviderMap provider = new CustomDestinationProviderMap();
com.sap.conn.jco.ext.Environment.registerDestinationDataProvider(provider);
Properties connectProperties = new Properties();
connectProperties.setProperty(DestinationDataProvider.JCO_ASHOST, "host.sap.my.domain.com");
connectProperties.setProperty(DestinationDataProvider.JCO_SYSNR, "00");
connectProperties.setProperty(DestinationDataProvider.JCO_CLIENT, "100");
connectProperties.setProperty(DestinationDataProvider.JCO_USER, "user");
connectProperties.setProperty(DestinationDataProvider.JCO_PASSWD, "password");
connectProperties.setProperty(DestinationDataProvider.JCO_LANG, "en");
provider.addDestination(DESTINATION_NAME1, connectProperties);
connect();
}
public static void connect() throws JCoException {
String FUNCTION_NAME = "BAPI_EMPLOYEE_GETDATA";
JCoDestination destination = JCoDestinationManager.getDestination(DESTINATION_NAME1);
JCoContext.begin(destination);
JCoFunction function = destination.getRepository().getFunction(FUNCTION_NAME);
if (function == null) {
throw new RuntimeException(FUNCTION_NAME + " not found in SAP.");
}
//function.getImportParameterList().setValue("EMPLOYEE_ID", "48");
function.getImportParameterList().setValue("FSTNAME_M", "ANAKIN");
function.getImportParameterList().setValue("LASTNAME_M", "SKYWALKER");
try {
function.execute(destination);
} catch (AbapException e) {
System.out.println(e.toString());
return;
}
JCoTable table = function.getTableParameterList().getTable("PERSONAL_DATA");
for (int i = 0; i < table.getNumRows(); i++) {
table.setRow(i);
System.out.println(table.getString("PERNO") + '\t' + table.getString("FIRSTNAME") + '\t' + table.getString("LAST_NAME")
+'\t' + table.getString("BIRTHDATE")+'\t' + table.getString("GENDER"));
}
JCoContext.end(destination);
}
Ok, so I got this up and going and thought I'd share my research.
You need to add your own destination in Portal. To achieve that you need to go to NetWeaver Administrator, located at: host:port/nwa. So it'll be something like sapportal.your.domain.com:50000/nwa.
Then you go to Configuration-> Infrastructure-> Destinations and add your destination there. You can leave empty most of the fields like Message Server. The important part is Destination name as it is how you will retrieve it and destination type which should be set to RFC Destination in my case. Try pinging your newly created destination to check if its up and going.
Finally you should be able to get destination by simply calling: JCoDestination destination = JCoDestinationManager.getDestination(DESTINATION_NAME); as it is added to your Portal environment and managed from there.
Take a look at the CustomDestinationDataProvider in the JCo examples of the Jco connector download. The important parts are:
static class MyDestinationDataProvider implements DestinationDataProvider
...
com.sap.conn.jco.ext.Environment.registerDestinationDataProvider(new MyDestinationDataProvider());
Then you can simply do:
instance = JCoDestinationManager.getDestination(DESTINATION_NAME);
Btw. you may also want to check out http://hibersap.org/ as they provide nice ways to store the config as well.
This is a question about spring-xd release 1.0.1 working together with maprfs, which is officially not yet supported. Still I would like to get it to work.
So this is what we did:
1) adjusted the xd-shell and xd-worker and xd-singlenode shell scripts to accept the parameter --hadoopDistro mapr
2) added libraries to the new directory $XD_HOME/lib/mapr
avro-1.7.4.jar jersey-core-1.9.jar
hadoop-annotations-2.2.0.jar jersey-server-1.9.jar
hadoop-core-1.0.3-mapr-3.0.2.jar jetty-util-6.1.26.jar
hadoop-distcp-2.2.0.jar maprfs-1.0.3-mapr-3.0.2.jar
hadoop-hdfs-2.2.0.jar protobuf-java-2.5.0.jar
hadoop-mapreduce-client-core-2.2.0.jar spring-data-hadoop-2.0.2.RELEASE-hadoop24.jar
hadoop-streaming-2.2.0.jar spring-data-hadoop-batch-2.0.2.RELEASE-hadoop24.jar
hadoop-yarn-api-2.2.0.jar spring-data-hadoop-core-2.0.2.RELEASE-hadoop24.jar
hadoop-yarn-common-2.2.0.jar spring-data-hadoop-store-2.0.2.RELEASE-hadoop24.jar
3) run bin/xd-singlenode --hadoopDistro mapr and shell/bin/xd-shell --hadoopDistro mapr.
When creating and deploying a stream via stream create foo --definition "time | hdfs" --deploy, data is written to a file tmp/xd/foo/foo-1.txt.tmp on maprfs. Yet when undeploying the stream, the following exceptions appears:
org.springframework.data.hadoop.store.StoreException: Failed renaming from /xd/foo/foo-1.txt.tmp to /xd/foo/foo-1.txt; nested exception is java.io.FileNotFoundException: Requested file /xd/foo/foo-1.txt does not exist.
at org.springframework.data.hadoop.store.support.OutputStoreObjectSupport.renameFile(OutputStoreObjectSupport.java:261)
at org.springframework.data.hadoop.store.output.TextFileWriter.close(TextFileWriter.java:92)
at org.springframework.xd.integration.hadoop.outbound.HdfsDataStoreMessageHandler.doStop(HdfsDataStoreMessageHandler.java:58)
at org.springframework.xd.integration.hadoop.outbound.HdfsStoreMessageHandler.stop(HdfsStoreMessageHandler.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201)
at com.sun.proxy.$Proxy120.stop(Unknown Source)
at org.springframework.integration.endpoint.EventDrivenConsumer.doStop(EventDrivenConsumer.java:64)
at org.springframework.integration.endpoint.AbstractEndpoint.stop(AbstractEndpoint.java:100)
at org.springframework.integration.endpoint.AbstractEndpoint.stop(AbstractEndpoint.java:115)
at org.springframework.integration.config.ConsumerEndpointFactoryBean.stop(ConsumerEndpointFactoryBean.java:303)
at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:229)
at org.springframework.context.support.DefaultLifecycleProcessor.access$300(DefaultLifecycleProcessor.java:51)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.stop(DefaultLifecycleProcessor.java:363)
at org.springframework.context.support.DefaultLifecycleProcessor.stopBeans(DefaultLifecycleProcessor.java:202)
at org.springframework.context.support.DefaultLifecycleProcessor.stop(DefaultLifecycleProcessor.java:106)
at org.springframework.context.support.AbstractApplicationContext.stop(AbstractApplicationContext.java:1186)
at org.springframework.xd.module.core.SimpleModule.stop(SimpleModule.java:234)
at org.springframework.xd.dirt.module.ModuleDeployer.destroyModule(ModuleDeployer.java:132)
at org.springframework.xd.dirt.module.ModuleDeployer.handleUndeploy(ModuleDeployer.java:111)
at org.springframework.xd.dirt.module.ModuleDeployer.undeploy(ModuleDeployer.java:83)
at org.springframework.xd.dirt.server.ContainerRegistrar.undeployModule(ContainerRegistrar.java:261)
at org.springframework.xd.dirt.server.ContainerRegistrar$StreamModuleWatcher.process(ContainerRegistrar.java:884)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: java.io.FileNotFoundException: Requested file /xd/foo/foo-1.txt does not exist.
at com.mapr.fs.MapRFileSystem.getMapRFileStatus(MapRFileSystem.java:805)
at com.mapr.fs.MapRFileSystem.delete(MapRFileSystem.java:629)
at org.springframework.data.hadoop.store.support.OutputStoreObjectSupport.renameFile(OutputStoreObjectSupport.java:258)
... 29 more
I had a look at the OutputStoreObjectSupport.renameFile() function. When a file on hdfs is finished, this method tries to rename the file /xd/foo/foo-1.txt.tmp to xd/foo/foo1.txt. This is the relevant code:
try {
FileSystem fs = path.getFileSystem(getConfiguration());
boolean succeed;
try {
fs.delete(toPath, false);
log.info("Renaming path=[" + path + "] toPath=[" + toPath + "]");
succeed = fs.rename(path, toPath);
} catch (Exception e) {
throw new StoreException("Failed renaming from " + path + " to " + toPath, e);
}
if (!succeed) {
throw new StoreException("Failed renaming from " + path + " to " + toPath + " because hdfs returned false");
}
}
When the target file does not exist on hdfs, maprfs seems to throw an exception when fs.delete(toPath, false) is called. Yet throwing an exception in this case does not make sense. I assume that other Filesystem implementations behave differently, but this is a point I still need to verify. Unfortuntately I cannot find the sources for MapRFileSystem.java. Is this closed source? This would help me to better understand the issue. Has anybody experience with writing from spring-xd to maprfs? Or renaming files on maprfs with spring-data-hadoop?
Edit
I managed to reproduce the issue outside of spring XD with a simple test case (see below). Note that this exception is only thrown if the inWritingSuffix or the inWritingPrefix is set. Otherwise spring-hadoop will not attempt to rename the file. So this is the still somehow unsatisfactory workaround for me: refrain from using inWritingPrefixes and inWritingSuffixes.
#ContextConfiguration("context.xml")
#RunWith(SpringJUnit4ClassRunner.class)
public class MaprfsSinkTest {
#Autowired
Configuration configuration;
#Autowired
FileSystem filesystem;
#Autowired
DataStoreWriter<String >storeWriter;
#Test
public void testRenameOnMaprfs() throws IOException, InterruptedException {
Path testPath = new Path("/tmp/foo.txt");
filesystem.delete(testPath, true);
TextFileWriter writer = new TextFileWriter(configuration, testPath, null);
writer.setInWritingSuffix("tmp");
writer.write("some entity");
writer.close();
}
#Test
public void testStoreWriter() throws IOException {
this.storeWriter.write("something");
}
}
I created a new branch for spring-hadoop which supports maprfs:
https://github.com/blinse/spring-hadoop/tree/origin/2.0.2.RELEASE-mapr
Building this release and using the resulting jar works fine with the hdfs sink.
I am getting following exception when trying to run my command line application:
java.lang.ExceptionInInitializerError
at org.hibernate.validator.engine.ConfigurationImpl.<clinit>(ConfigurationImpl.java:52)
at org.hibernate.validator.HibernateValidator.createGenericConfiguration(HibernateValidator.java:43)
at javax.validation.Validation$GenericBootstrapImpl.configure(Validation.java:269)
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -2
at java.lang.String.substring(String.java:1937)
at org.hibernate.validator.util.Version.<clinit>(Version.java:39)
... 34 more
Am I doing anything wrong? Please suggest.
This is strange. I pasted the relevant parts of the static initialization block of o.h.v.u.Version in a class with a main and added some poor man's logging traces:
public class VersionTest {
public static void main(String[] args) {
Class clazz = org.hibernate.validator.util.Version.class;
String classFileName = clazz.getSimpleName() + ".class";
System.out.println(String.format("%-16s: %s", "classFileName", classFileName));
String classFilePath = clazz.getCanonicalName().replace('.', '/') + ".class";
System.out.println(String.format("%-16s: %s", "classFilePath", classFilePath));
String pathToThisClass = clazz.getResource(classFileName).toString();
System.out.println(String.format("%-16s: %s", "pathToThisClass", pathToThisClass));
// This is line 39 of `org.hibernate.validator.util.Version`
String pathToManifest = pathToThisClass.substring(0, pathToThisClass.indexOf(classFilePath) - 1)
+ "/META-INF/MANIFEST.MF";
System.out.println(String.format("%-16s: %s", "pathToManifest", pathToManifest));
}
}
And here the output I get when running it:
classFileName : Version.class
classFilePath : org/hibernate/validator/util/Version.class
pathToThisClass : jar:file:/home/pascal/.m2/repository/org/hibernate/hibernate-validator/4.0.2.GA/hibernate-validator-4.0.2.GA.jar!/org/hibernate/validator/util/Version.class
pathToManifest : jar:file:/home/pascal/.m2/repository/org/hibernate/hibernate-validator/4.0.2.GA/hibernate-validator-4.0.2.GA.jar!/META-INF/MANIFEST.MF
In your case, the StringIndexOutOfBoundsException: String index out of range: -2 suggests that:
pathToThisClass.indexOf( classFilePath )
is returning -1, making the pathToThisClass.substring(0, -2) call indeed erroneous.
And this means that org/hibernate/validator/util/Version.class is somehow not part of the pathToThisClass that you get. I don't have a full explanation but this must be related to the fact that you're using One-Jar.
Could you run the above test class and update your question with the output?
So, as you use One-JAR, the problem probably is in incompatibility between One-JAR and Hibernate Validator. However, in the latest version of One-JAR (0.97) it works fine, therefore use the latest version.