Hadoop Configuration Error - java

I am attempting to start up my hadoop application, however upon startup i am seeing this in the log files, does anyone have a clue as to what the problem is?
Creating filesystem for hdfs://10.170.4.141:9000
java.io.IOException: config()
at org.apache.hadoop.conf.Configuration.(Configuration.java:229)
at org.apache.hadoop.conf.Configuration.(Configuration.java:216)
at org.apache.hadoop.security.SecurityUtil.(SecurityUtil.java:60)
at org.apache.hadoop.net.NetUtils.makeSocketAddr(NetUtils.java:188)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:168)
at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:198)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:88)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1413)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:68)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1431)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:256)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:125)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:240)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:372)
at org.blismedia.VolumeReportGenerateUpdates.main(VolumeReportGenerateUpdates.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:187)

I think you're running into HADOOP-2851. This "error" can safely be ignored.
Apparently, Configuration's constructor logs an exception to the debug log, despite no exception actually being thrown. Why? Your guess is as good as mine. But the issue is resolved in their project as won't fix. "It's a feature, not a bug."
public Configuration(boolean loadDefaults) {
if (LOG.isDebugEnabled()) {
LOG.debug(StringUtils.stringifyException(new IOException("config()")));
}
// ...
}

Related

Getting "java.lang.NoClassDefFoundError: javax/security/jacc/PolicyContext" when run node.js

I am trying to get a node.js+node-java app that calls an API from a commercial product, and have gotten it to the point it almost runs, but now, when I run it, it is giving me a "java.lang.NoClassDefFoundError".
Here's the ending of the output when it gets this error:
THIS IS CONSOLE.LOG!!
Finished STARTING TO DO MyFactoryImplClass.getPepRequestFactory()...
About to do pepReqF1.newPepRequest()...
/apps/Oracle/OESCLIENT/oes_sm_instances/smtest8-java-PULL-NODEJS/testOES.js:97
var pepReq1 = pepReqF1.newPepRequestSync("Administrators", "GET" , "test/foo2/fooresource" );
^
Error: Error running instance method
java.lang.NoClassDefFoundError: javax/security/jacc/PolicyContext
at oracle.security.jps.runtime.AppSecurityContext$2.run(AppSecurityContext.java:223)
at oracle.security.jps.runtime.AppSecurityContext$2.run(AppSecurityContext.java:221)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.runtime.AppSecurityContext.getApplicationID(AppSecurityContext.java:221)
at oracle.security.jps.internal.api.runtime.AppSecurityContext.getApplicationID(AppSecurityContext.java:66)
at oracle.security.jps.openaz.pep.SubjectObjMapper$2.run(SubjectObjMapper.java:234)
at oracle.security.jps.openaz.pep.SubjectObjMapper$2.run(SubjectObjMapper.java:231)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.openaz.pep.SubjectObjMapper.mapStringSubject(SubjectObjMapper.java:231)
at oracle.security.jps.openaz.pep.SubjectObjMapper.mapToJpsObject(SubjectObjMapper.java:173)
at oracle.security.jps.openaz.pep.PepRequestImpl.setAccessSubject(PepRequestImpl.java:429)
at oracle.security.jps.openaz.pep.PepRequestFactoryImpl.newPepRequest(PepRequestFactoryImpl.java:202)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at Error (native)
at Object.<anonymous> (/apps/Oracle/OESCLIENT/oes_sm_instances/smtest8-java-PULL-NODEJS/testOES.js:97:24)
at Module._compile (module.js:556:32)
at Object.Module._extensions..js (module.js:565:10)
at Module.load (module.js:473:32)
at tryModuleLoad (module.js:432:12)
at Function.Module._load (module.js:424:3)
at Module.runMain (module.js:590:10)
at run (bootstrap_node.js:394:7)
at startup (bootstrap_node.js:149:9)
[root#nodejs smtest8-java-PULL-NODEJS]#
I found that PolicyContext class is in jacc-api.jar, so I put a copy of jacc-api.jar in my directory and then added:
enter code herejava.classpath.push("/apps/node-v6.6.0-linux-x64/code/jacc-api.jsr");
However, even after doing that, I am still getting the same error when I run my node.js+node-java app.
I am guessing that when I call out to the commercial product's API, that THAT is looking for this PolicyContext class, so I tried exporting CLASSPATH set to the path that the jacc-api.jar file, but even then I get the same error.
Where can I add that JAR to the CLASSPATH so that I can eliminate this error?
Thanks,
Jim

ClassNotFoundException when trying to start topology

This is a strange problem that I'm hoping someone's encountered and overcome.
I have a relatively simple topology:
Spout -> BoltA -> BoltB
-> BoltC
If I use bolts A and B, it works. A and C works too. Enabling all three though gives me this:
16:05:20.627 [main] WARN backtype.storm.daemon.nimbus - Topology submission exception. (topology name='traffic') #<RuntimeException java.lang.RuntimeException: java.lang.ClassNotFoundException: backtype.storm.daemon.nimbus$normalize_conf$get_merged_conf_val__3916$fn__3917>
16:05:20.634 [main] ERROR o.a.zookeeper.server.NIOServerCnxn - Thread Thread[main,5,main] died
java.lang.RuntimeException: java.lang.ClassNotFoundException: backtype.storm.daemon.nimbus$normalize_conf$get_merged_conf_val__3916$fn__3917
at backtype.storm.utils.Utils.deserialize(Utils.java:88) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.daemon.nimbus$read_storm_conf.invoke(nimbus.clj:89) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.daemon.nimbus$start_storm.invoke(nimbus.clj:724) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.daemon.nimbus$eval3974$exec_fn__1459__auto__$reify__3987.submitTopologyWithOpts(nimbus.clj:962) ~[na:na]
at backtype.storm.daemon.nimbus$eval3974$exec_fn__1459__auto__$reify__3987.submitTopology(nimbus.clj:971) ~[na:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_55]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_55]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_55]
I'm running it locally using the code below:
LocalCluster cluster = new LocalCluster();
cluster.submitTopology(topoName, conf, buildTopology(source, service));
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
cluster.killTopology(topoName);
cluster.shutdown();
I'm running against storm-core-0.9.1-incubating.jar
All I could find was this discussion about it "being a classpath issue". The fact that it only happens after adding a third bolt, but it doesn't matter which bolt, really confuses me. Is there something that Storm does once it has a "sufficiently complex topology" that kicks in at 3 bolts or 4 components?
EDIT
Well, it gets weirder. After finding someone with the same problem I tried their solution and added to my logback.xml:
<logger name="backtype.storm.daemon.nimbus" level="INFO" />
which fixed the problem. Anything higher (WARN, ERROR) brings it back. Lower (DEBUG) is fine. Removing this line and setting the root logger to DEBUG brings the problem back, so I'm really confused. If something in the code path depends on logger.isInfoEnabled() shouldn't it act the same for rootLogger=INFO and backtype.storm.daemon.nimbus=INFO?

Java 7 update 25 makes our java web start application fail with no logging

Since the java 7 update 25 launched by Oracle our application no longer functions.
Initially we got some warning about codebase & sercurity tags missing in the Manifest file, which we fixed.
The problem we now end up with is that in the Console we only get the following lines:
#### Java Web Start Error:
#### null
We also get an application Error dialog with the message: Unable to launch the application.
The details button gives the following details in the Exception:
java.lang.NullPointerException
at com.sun.jnlp.JNLPClassLoader.getPermissions(Unknown Source)
at java.security.SecureClassLoader.getProtectionDomain(SecureClassLoader.java:206)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at com.sun.jnlp.JNLPClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at desktop.DesktopProxySelector.<init>(DesktopProxySelector.java:24) <- code smippet below
at desktop.Main.main(Main.java:139) <- code smippet below
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.javaws.Launcher.executeApplication(Unknown Source)
at com.sun.javaws.Launcher.executeMainClass(Unknown Source)
at com.sun.javaws.Launcher.doLaunchApp(Unknown Source)
at com.sun.javaws.Launcher.run(Unknown Source)
at java.lang.Thread.run(Thread.java:724)
The relevant code parts are:
Desktop.Main.main
/**
* Main method, starts the application
*/
public static void main(String[] args) {
System.setProperty("java.net.useSystemProxies", "true");
//Logger.getLogger("httpclient.wire.header.level").setLevel(Level.FINEST);
//Logger.getLogger("org.apache.commons.httpclient.level").setLevel(Level.FINEST);
java.net.ProxySelector.setDefault(new DesktopProxySelector(java.net.ProxySelector.getDefault()));
(The last line is line number 139)
desktop.DesktopProxySelector:
public class DesktopProxySelector extends ProxySelector {
public DesktopProxySelector(ProxySelector defaultSelector) {
URI httpsUri = new CentralConfigurationService().getCentralLocation();
(The last line is line number 24 where the exception occures)
Can someone give us some clues hints (or better a solution) for this new behaviour of java caused by this 'minor' update.
When we run the application straight from the cli using java -jar Desktop.jar the application wil run file, so the issue has clearly something todo with the changes in java web start.
#trashgod: the error clearly has something to do with the Permissions change in 7u25, since the NullPointerException occurs in com.sun.jnlp.JNLPClassLoader.getPermissions.
Just to explain what I think happens (I am a colleague of Wouter):
desktop.Main instantiates a desktop.DesktopProxySelector (our class),
desktop.DesktopProxySelector instantiates desktop.configuration.CentralConfigurationService
desktop.configuration.CentralConfigurationService instantiates a java.net.URI.
On the first line of the DesktopProxySelector init where the CentralConfigurationService is instantiated the getPermissions method, called by the JNLPClassLoader, throws the NullPointerException. So something is going wrong while loading the CentralConfigurationService class by java webstart with getting the permissions for the class. Could that have anything to do with the fact that a URI class is instantiated, which requires extra permissions (a connection to a remote uri is setup)?
Eventually the problem was solved.
The problem was caused between a mismatch in the included jar files in the main MANIFEST.MF file vs the jar files mentioned in the launch.jnlp.
Apperently it is now required to have all jar files that will be used also be present in the launch.jnlp file.
(In the past it was decided to keep this file manually in sink, which obviously was not always maintained in a propper way. Now this process is automated, so the problem should no longer happen to us.)

Error while making webservice call

I am accessing a webservice exposed by some other guy as a client using axis2 and I have made a wrapper method for abcStub .java (Stub java file made by running some batch) and tried to instantiate the Stub file inside the wrapper java file. But i get following errors:
INFO: ERROR org.apache.struts.actions.DispatchAction - Dispatch[/remittanceTransaction] to method 'remTransApprove' returned an exception
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
Caused by: java.lang.NoSuchMethodError: org.apache.neethi.PolicyEngine.getPolicy(Ljava/lang/Object;)Lorg/apache/neethi/Policy;
at org.tempuri.MIRSRemittanceServicesStub.getPolicy(MIRSRemittanceServicesStub.java:3558)
at org.tempuri.MIRSRemittanceServicesStub.populateAxisService(MIRSRemittanceServicesStub.java:57)
at org.tempuri.MIRSRemittanceServicesStub.(MIRSRemittanceServicesStub.java:268)
at org.tempuri.MIRSRemittanceServicesStub.(MIRSRemittanceServicesStub.java:257)
at org.tempuri.MIRSRemittanceServicesStub.(MIRSRemittanceServicesStub.java:305)
INFO: g.tempuri.MIRSRemittanceServicesStub.(MIRSRemittanceServicesStub.java:297)
at com.mtmrs.webservice.MIRSRemittanceTransactionManagerImpl.getAppDeclWrapper(MIRSRemittanceTransactionManagerImpl.java:201)
at com.mtmrs.business.backoffice.impl.RemittanceTransactionManagerImpl.remTransApprove(RemittanceTransactionManagerImpl.java:1417)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at com.mtmrs.application.backoffice.action.RemittanceTransactionAction.remTransApprove(RemittanceTransactionAction.java:1409)
... 78 more
INFO: WARN org.apache.struts.action.RequestProcessor - Unhandled Exception thrown: class javax.servlet.ServletException
WARNING: StandardWrapperValve[action]: PWC1406: Servlet.service() for servlet action threw exception
java.lang.NoSuchMethodError: org.apache.neethi.PolicyEngine.getPolicy(Ljava/lang/Object;)Lorg/apache/neethi/Policy;
at org.tempuri.MIRSRemittanceServicesStub.getPolicy(MIRSRemittanceServicesStub.java:3558)
at org.tempuri.MIRSRemittanceServicesStub.populateAxisService(MIRSRemittanceServicesStub.java:57)
at org.tempuri.MIRSRemittanceServicesStub.(MIRSRemittanceServicesStub.java:268)
I think there must be problem with this
org.apache.neethi.PolicyEngine.getPolicy(Ljava/lang/Object;)Lorg/apache/neethi/Policy;
or may be im wrong. Either way, dunno how to fix. Would be of much help if anyone knew of faced/solved this before.
I got this while making following call:
public String getAppDeclWrapper(String session, String parentAgent, String mirsTxnNo, String strReason, String isApprove, String modifiedBY){
String returnString = null;
try {
mtmrsLogger.info("Entering into Approve Decline procedure, Starting to send Status to MIRS");
MIRSRemittanceServicesStub stub = new MIRSRemittanceServicesStub(); //Error occurs at this line
ApproveDecline approvedEcline = new ApproveDecline();
ApproveDeclineResponse approveResponse = new ApproveDeclineResponse();
approvedEcline.setCParentAgent(parentAgent);
approvedEcline.setMirsSession(session);
approvedEcline.setModifiedBy(modifiedBY);
approvedEcline.setRefNo(mirsTxnNo);
approvedEcline.setStrReason(strReason);
approvedEcline.setIsApprove(isApprove);
My guess is you have multiple versions (or invalid version) of PolicyEngine on your classpath. I think you are running inside a servlet engine. Try following code to determine where this version is stored and fix it:
org.apache.neethi.PolicyEngine.class.getProtectionDomain().getCodeSource().getLo‌​cation()

How do I get a Java WebService to actually show me exceptions from the proxy object?

I'm chasing what should be a simple NullPointerException but a webservice proxy keeps swallowing the error.
The WS is exposed through a simple client library, that client then calls the WS. Somewhere in the WS theres an NPE and I can't find it because the stack trace only shows "$Proxy.someMethod" instead of the cause of the issue.
How can I get the stack trace from the proxy object? Is there a good strategy to log or handle these exceptions?
...
Caused by: javax.xml.ws.soap.SOAPFaultException: java.lang.NullPointerException
at com.sun.xml.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:187)
at com.sun.xml.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:116)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:254)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:224)
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:117)
******** I need the trace from this line *********
at $Proxy43.myMethod(Unknown Source)
*****************
at com.mypackage.client.MyClient.aMethod(MyClient.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:161)
... 17 more
I've done some reading and it seems like this has to do with checked vs. unchecked exceptions. So if I were throwing an IOException I'd get the stack trace, but since there is a NullPointerException bubbling up its not giving me the trace.
Wrap your service in a try/catch block and throw the exception that's defined as the fault element of the called service in your WSDL. That way the client will get a useable stacktrace.

Categories