I am trying to run example from this particular site:
https://github.com/larsgeorge/hbase-book/tree/master/ch07
Now.. it compiles succesfully
But when I try to run an example, it gives me the following exception:
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
... 7 more
Caused by: java.lang.NoClassDefFoundError: org/json/simple/parser/JSONParser
at mapreduce.ParseJson2$ParseMapper.<init>(ParseJson2.java:48)
... 12 more
Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.JSONParser
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
... 13 more
Now from the pom in this file, I can see that simple json is included but I am not sure why is it throwing this issue.?
I would say that the problem is that when run under map-reduce the jar you need is not in scope (i.e. not found). One way to fix that is to create a jar with all the dependencies and submit that as the job. Other options are to make sure it is available in each node and deploying the depdndecies to Hadoops' distribtued cache (but the first is the simplest - though it makes for slower job starts since the jar is bigger)
Related
After using java -jar tracker-server.jar "configuration.xml"
I got:
Exception in thread "main" java.lang.NoClassDefFoundError: org/jboss/netty/bootstrap/Bootstrap
at org.traccar.Main.main(Main.java:26)
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.bootstrap.Bootstrap
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
... 1 more
I found only its some about third party library. But I'm not friendly with Java.
The ClassNotFoundException is caused by there soem jar file missing from the classpath. Search for the jar what includes the missing class and make it available for the enviroment.
See this thread: java.lang.NoClassDefFoundError: org.jboss.netty.bootstrap.ClientBootstrap
I'm attempting to make a very simple Kafka Producer and am currently following the producer example except my producer does not have a partitioner class.
After exporting required files into a jar I transfer them to my Linux image and try to run it.
I get this exception:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.NoClassDefFoundError: org/apache/kafka/common/utils/Utils
at kafka.client.ClientUtils$$anonfun$parseBrokerList$1.apply(ClientUtils.scala:103)
at kafka.client.ClientUtils$$anonfun$parseBrokerList$1.apply(ClientUtils.scala:102)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:194)
at scala.collection.mutable.ArrayBuffer.map(ArrayBuffer.scala:44)
at kafka.client.ClientUtils$.parseBrokerList(ClientUtils.scala:102)
at kafka.producer.BrokerPartitionInfo.<init>(BrokerPartitionInfo.scala:32)
at kafka.producer.async.DefaultEventHandler.<init>(DefaultEventHandler.scala:41)
at kafka.producer.Producer.<init>(Producer.scala:60)
at kafka.javaapi.producer.Producer.<init>(Producer.scala:26)
at producers.HelloWorldProducer.main(HelloWorldProducer.java:20)
... 5 more
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.utils.Utils
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 19 more
After looking at the kafka jar I see that the utils is its own package now and not located within common.
What would be the best way to solve this issue?
The answer ended up being real silly ... I needed to use the kafka-clients-0.8.2.0.jar instead.
I have a simple java file packaged as a Jar on a server called by a PHP script. I have the "json-simple-1.1.1" library included in my java program. However, when I run it gives me the below error. I've added the json-simple jar to my project and import the correct classes. From what I've read it doesn't seem to be adding the json-simple library to my manifest file. Any ideas on how to fix this. I'm using eclipse if that helps.
Exception in thread "main" java.lang.NoClassDefFoundError: org/json/simple/parser/ParseException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2570)
at java.lang.Class.getMethod0(Class.java:2813)
at java.lang.Class.getMethod(Class.java:1663)
at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.ParseException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
Your library is probably not been loaded by classloader, so, if it is not in WEB-INF/lib, right click it and Add to build Path.
I have an Infinispan Server, version 6.0.x, the one derived from JBoss 7.2, and it is working fine for caching.
However, when I try to monitor it by JMX I can't. This is URL I type in jconsole.bat :
service:jmx:remoting-jmx://MY.IP.ADDRESS.HERE:9999
But I get this error :
Exception in thread "VMPanel.connect" java.util.ServiceConfigurationError: javax.management.remote.JMXConnectorProvider: Provider org.jboss.remotingjmx.RemotingConnectorProvider could not be instantiated: java.lang.NoClassDefFoundError: org/jboss/logging/Logger
at java.util.ServiceLoader.fail(ServiceLoader.java:224)
at java.util.ServiceLoader.access$100(ServiceLoader.java:181)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:377)
at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
at javax.management.remote.JMXConnectorFactory.getConnectorAsService(JMXConnectorFactory.java:472)
at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:341)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
at sun.tools.jconsole.ProxyClient.tryConnect(ProxyClient.java:370)
at sun.tools.jconsole.ProxyClient.connect(ProxyClient.java:313)
at sun.tools.jconsole.VMPanel$2.run(VMPanel.java:292)
Caused by: java.lang.NoClassDefFoundError: org/jboss/logging/Logger
at org.jboss.remotingjmx.RemotingConnectorProvider.<clinit>(RemotingConnectorProvider.java:42)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:374)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:373)
... 7 more
Caused by: java.lang.ClassNotFoundException: org.jboss.logging.Logger
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 14 more
Exception in thread "VMPanel.connect" java.util.ServiceConfigurationError: javax.management.remote.JMXConnectorProvider: Provider org.jboss.remotingjmx.RemotingConnectorProvider could not be instantiated: java.lang.NoClassDefFoundError: Could not initialize class org.jboss.remotingjmx.RemotingConnectorProvider
at java.util.ServiceLoader.fail(ServiceLoader.java:224)
at java.util.ServiceLoader.access$100(ServiceLoader.java:181)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:377)
at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
at javax.management.remote.JMXConnectorFactory.getConnectorAsService(JMXConnectorFactory.java:472)
at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:341)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
at sun.tools.jconsole.ProxyClient.tryConnect(ProxyClient.java:357)
at sun.tools.jconsole.ProxyClient.connect(ProxyClient.java:313)
at sun.tools.jconsole.VMPanel$2.run(VMPanel.java:292)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.jboss.remotingjmx.RemotingConnectorProvider
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:374)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:373)
... 7 more
Maybe I should create some managed beans somewhere? How can I fix this?
Remark 1 : In the documentation they include the necearry configuration between <global></global>, but in my case the settings should respect JBoss Schema...
Remark 2 : I tried with JBoss 7.2, and it could show me graphs and so on, and by the way I used the same URL. The difference is that I used jconsole.bat of JBoss instead of Infinispan.
Please, any idea about the necessary configuration?
Thank you.
I'm a Windows 7 user, and this is the fix that worked for me.
Open jconsole.bat for editting.
At the top, turn on the echoing.
#echo on
Then go down and find this line
call :SearchForJars "%JBOSS_MODULEPATH%\system\layers\base\org\jboss\logging\main"
Insert the following line before it:
echo "%JBOSS_MODULEPATH%\system\layers\base\org\jboss\logging\main"
I'm not sure why that worked, but it did. If you get different ClassNotFound exceptions, try echo statements above the other jar searches.
I am trying to join two datasets.. One of which is json..
I am relying on json-simple library to parse that json..
I am trying to use libjars.. So far .. for simple data processing.. the approach has worked.. but now i am getting the following error
Exception in thread "main" java.lang.NoClassDefFoundError: org/json/simple/parser/ParseException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
at org.apache.hadoop.mapreduce.lib.input.MultipleInputs.getMapperTypeMap(MultipleInputs.java:141)
at org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:60)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1024)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1041)
at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at org.select.Driver.run(Driver.java:130)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.select.Driver.main(Driver.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.ParseException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
I think I have implemented toolrunner.
hadoop jar domain_gold.jar org.select.Driver \
-libjars json-simple-1.1.1.jar $INPUT1 $INPUT2 $OUTPUT
The code
http://pastebin.com/7XnyVnkv
Place your third party JAR file on one of hadoop's default lib folder. For example, $HADOOP_HOME/share/hadoop/common/lib.