Typically I start by Googling for a solution, but this error does not seem to have occurred before.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/Shell
at org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:906)
at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:237)
at org.apache.hive.jdbc.HiveConnection.isHttpTransportMode(HiveConnection.java:221)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:138)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:123)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at com.merck.ghh.ingestion.HiveTableSetup.tableSetup(HiveTableSetup.java:31)
at com.merck.ghh.ingestion.HiveTableSetup.main(HiveTableSetup.java:546)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.Shell
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 10 more
The following code causes this error, specifically on the Connection declaration line:
try {
Class.forName(driverName);
Connection connection = DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive", "");
Statement statement = connection.createStatement();
} catch (ClassNotFoundException exception) {
exception.printStackTrace();
}
Other questions seem to point to this being related to not including hadoop-core in the dependencies, but hadoop-core does not seem to exist in the Hadoop 2.X. I am specifically working in Hadoop 2.1.0.2.0.5.0-67 with Hive 0.12.0. Before this error I was having issues bc my Hive dependencies had gotten destroyed at some point, but I put those back in and this error happened next. I'm wondering if it's not something as simple as forgetting a dependency, but I'm not finding which dependency I might be missing.
Any help with this is greatly appreciated.
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.Shell
As you are using Hadoop 2.X. you need to download hadoop-common-2.1.0-beta.jar OR
hadoop-common-2.0.0-cdh4.4.0.jar and to class path to avoid ClassNotFoundException.
You are right earlier hadoop-core-0.20.2-737.jar was used.
For more visit http://grepcode.com/
Try copying the JDBC connector jar into the $HIVE_HOME/lib/ folder. I faced a similar issue and copying the jar file worked for me.
Related
I am trying to get rid of some NoClassDefFoundError due to some jars not found at run time. So I put in my hdfs system some lib and I call and I put this
String lib = "/path/to/lib";
Path hdfsJar = new Path(lib);
DistributedCache.addFileToClassPath(hdfsJar, conf);
Now, I am still getting the error. However, if I set the jars in the $HADOOP_CLASSPATH. Am I doing wrong with the DistributedCache call ?
edit :
java.lang.RuntimeException: java.lang.NoClassDefFoundError: gov/nih/nlm/nls/metamap/MetaMapApi
at org.apache.hadoop.mapreduce.lib.chain.Chain.joinAllThreads(Chain.java:526)
at org.apache.hadoop.mapreduce.lib.chain.ChainMapper.run(ChainMapper.java:169)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NoClassDefFoundError: gov/nih/nlm/nls/metamap/MetaMapApi
at org.avrosation.metamap.ChainMetaProcess$TokenizerMapper.map(ChainMetaProcess.java:25)
at org.avrosation.metamap.ChainMetaProcess$TokenizerMapper.map(ChainMetaProcess.java:16)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapreduce.lib.chain.Chain$MapRunner.run(Chain.java:321)
Caused by: java.lang.ClassNotFoundException: gov.nih.nlm.nls.metamap.MetaMapApi
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
Try providing fully qualified HDFS path. try below code:
Make sure you upload jar to HDFS (any location on hdfs, i am assuming /tmp).
hadoop fs -copyFromLocal my.jar /tmp
Then edit your java code like :
String lib = "hdfs://localhost:9000/tmp/my.jar";
Path hdfsJar = new Path(lib);
DistributedCache.addFileToClassPath(hdfsJar, conf);
This doc detail about distributed cache : https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/filecache/DistributedCache.html
In fact, I suspected from the beginning some problem with my ide IntelliJ Idea 14 because I had to deal with a major refactoring of the code which then made the code reveal the issue. To begin I tried to clean the building with no success, then I simply created an other project and copy-pasted the classes and libraries' import and that made the trick !
I would execute a maven project but my classpath is not correct...
I tried many possibilities without find any solution.
The compilation is successfull, but when I start the program, I have this following error :
[root#sandbox kafka-log-appender]# java -cp "target/kafka-log-appender-1.0-SNAPSHOT.jar:log4j-1.2.17.jar" com.log.kafka.RogueApplication
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at com.log.kafka.RogueApplication.<clinit>(RogueApplication.java:11)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 1 more
Thanks
Add slf4j.jar given in the below path:
http://www.java2s.com/Code/JarDownload/slf4j/slf4j.jar.zip
If the issue is not solved try with other slf4j jars suggested in the same page.
Look for slf4j-api.jar and any slf4j jar that references log4j and add those as well. There are two or three drivers for log4j. I don't know which one Kafka ships with, but if you see multiple, only put one on there as multiple can make slf4j have problems.
for those who've managed to use this java implementation of leveldb https://github.com/dain/leveldb/
I'm getting the following stacktrace when I try to execute the code in the readme:
NoClassDefFoundError occurs when runing the following line:
DB db = factory.open(new File("example"), options);
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Lists
at org.iq80.leveldb.impl.DbImpl.(DbImpl.java:95)
at org.iq80.leveldb.impl.Iq80DBFactory.open(Iq80DBFactory.java:59)
at com.test.LevelDBDain.main(LevelDBDain.java:23)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Lists
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
I confirmed that new File("example") returns a non-null value; same as the variable options
I import two jars built from the source into my library:
1.leveldb-api.jar
2.leveldb.jar
I've tried this with releases 0.6, 0.7 and 0.8-SNAPSHOT and always get the same error.
Could it be something I'm doing wrong ...or missing?
com.google.common.collect.util from guava is a dependency of the leveldb library but for some reason adding it explicitly as a dependency for my java project solved the problem.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/protocol/HttpContext
at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:119)
at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:103)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:362)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:344)
at Gethtml_fromS3.main(Gethtml_fromS3.java:16)
Caused by: java.lang.ClassNotFoundException: org.apache.http.protocol.HttpContext
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 5 more
Can any one give me a solution for the above error?
Please check if you have httpcore jar. I got rid of this error, after adding httpcore-4.2.jar to my lib.
It's difficult to pinpoint the exact reason why your application raises this exception, but a few common pointers that might help you.
The exception is raised when you are using a class in your code that isn't available when it runs. In this case, it seems you are unable to use org.apache.http.protocol.HttpContext. Most likely, you created an application that does include the Amazon AWS jar, but lacks the Apache HTTPClient jar.
If you used gradle/maven to create your application, include it in your dependencies. It it's a plain Java program, ensure the required JARs are available in your classpath (e.g. the same place as you currently have the Amazon jar located).
When I try to simulate my little project, I decided to use jme3 library and browsed the jar file through Configure-> JDK Profiles-> Add Archieve-> JME3-core.jar and ran the project sample code. The code was written in a website and I got the exception as:
Mar 25, 2014 10:53:44 AM com.jme3.system.JmeSystem checkDelegate
SEVERE: No JmeSystemDelegate specified, cannot instantiate default JmeDesktopSystem:
{0}
java.lang.ClassNotFoundException: com.jme3.system.JmeDesktopSystem
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at com.jme3.system.JmeSystem.checkDelegate(JmeSystem.java:125)
at com.jme3.system.JmeSystem.showSettingsDialog(JmeSystem.java:96)
at com.jme3.app.SimpleApplication.start(SimpleApplication.java:129)
at Main.main(Main.java:12)
Exception in thread "main" java.lang.NullPointerException
at com.jme3.system.JmeSystem.showSettingsDialog(JmeSystem.java:97)
at com.jme3.app.SimpleApplication.start(SimpleApplication.java:129)
at Main.main(Main.java:12)
Process completed.
Is the problem about the native libraries kind of issue which is also need to be dealt with J3D libraries? I mean, do I need to follow some instructions about installing and importing JME3 libraries? If so, can anyone tell me the instructions or where to download the correct .jar file?
find a similar discussion here ..this might be relevant ...
more on the same http://hub.jmonkeyengine.org/forum/topic/latest-jme3-sdk-update-trashed-13-of-my-installations/