I am trying to get rid of some NoClassDefFoundError due to some jars not found at run time. So I put in my hdfs system some lib and I call and I put this
String lib = "/path/to/lib";
Path hdfsJar = new Path(lib);
DistributedCache.addFileToClassPath(hdfsJar, conf);
Now, I am still getting the error. However, if I set the jars in the $HADOOP_CLASSPATH. Am I doing wrong with the DistributedCache call ?
edit :
java.lang.RuntimeException: java.lang.NoClassDefFoundError: gov/nih/nlm/nls/metamap/MetaMapApi
at org.apache.hadoop.mapreduce.lib.chain.Chain.joinAllThreads(Chain.java:526)
at org.apache.hadoop.mapreduce.lib.chain.ChainMapper.run(ChainMapper.java:169)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NoClassDefFoundError: gov/nih/nlm/nls/metamap/MetaMapApi
at org.avrosation.metamap.ChainMetaProcess$TokenizerMapper.map(ChainMetaProcess.java:25)
at org.avrosation.metamap.ChainMetaProcess$TokenizerMapper.map(ChainMetaProcess.java:16)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapreduce.lib.chain.Chain$MapRunner.run(Chain.java:321)
Caused by: java.lang.ClassNotFoundException: gov.nih.nlm.nls.metamap.MetaMapApi
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
Try providing fully qualified HDFS path. try below code:
Make sure you upload jar to HDFS (any location on hdfs, i am assuming /tmp).
hadoop fs -copyFromLocal my.jar /tmp
Then edit your java code like :
String lib = "hdfs://localhost:9000/tmp/my.jar";
Path hdfsJar = new Path(lib);
DistributedCache.addFileToClassPath(hdfsJar, conf);
This doc detail about distributed cache : https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/filecache/DistributedCache.html
In fact, I suspected from the beginning some problem with my ide IntelliJ Idea 14 because I had to deal with a major refactoring of the code which then made the code reveal the issue. To begin I tried to clean the building with no success, then I simply created an other project and copy-pasted the classes and libraries' import and that made the trick !
Related
I have a large java application running on a Red Hat Enterprise Linux server. This application has around 100 jars. On most RHEL systems the application runs fine, however, on certain RHEL systems (all RHEL versions are the same) I get a NoClassDefFoundError on startup (before my application gets a chance to do anything). If I rename the JAR file which contains the class (e.g. com-test-1.3.4.jar -> com-test-1.3.4-2.jar) everything starts up great even though the renameing doesn't change the order of the JAR in my directory listing. If I add the JAR file to the beginning of the classpath it works great. On other systems with the exact same OS/classpath/java/jars it works great. If I try to run the class that java says is missing (before renaming), java reports that the class does not have a main method so I know the class itself is loadable/findable. I have tried the latest version of Java 1.7 with no luck. The classpath is specified using the wildcard operator (e.g. CLASSPATH=/home/test/*).
The actual exception:
Exception in thread "main" java.lang.NoClassDefFoundError: com/test/testpath/TestClass
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
at java.lang.Class.getMethod0(Class.java:2774)
at java.lang.Class.getMethod(Class.java:1663)
at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: com.test.testpath.TestClass
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 6 more
I am trying to use JDBC to access the MySql database on my computer. I get this error message
java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:191)
at FunctionClass.special_function(FunctionClass.java:72)
at FunctionClass.<init>(FunctionClass.java:25)
at EvaluateFunctions.main(EvaluateFunctions.java:12)
I've seen on the other posts that this is due to the fact that the driver is not in the lib directory of the JDK. I tried that and it still doesn't work. I added the *.jar file to my "/usr/lib/default-java/lib/" folder on Xubuntu. I am not using any kind of servers. Here is my code that connects to the database:
Class.forName("com.mysql.jdbc.Driver");
Does anyone know what I am missing here?
Thanks for all the help in advance.
If you are using eclipse or some ide you can add the jar files to your build path.
It can be done by checking this link for eclipse. https://stackoverflow.com/a/27085441/4083590
If you are using command prompt you should add the jar files to your CLASSPATH. It can be done as shown below.
Linux:
CLASSPATH=$CLASSPATH;{Current working directory};{Direct Path to .jar file}
CLASSPATH=$CLASSPATH;/home/users/xyz/workspace/;/home/users/xyz/workspace/xyz.jar
Note: ./ can be used to signify current working directory
Windows:
set CLASSPATH=%CLASSPATH%:{Current working directory}:{Direct path to .jar file}
set CLASSPATH=%CLASSPATH%:C:\users\xyz\workspace:C:\users\xyz\workspace\xyz.jar
After doing this you would be able to compile your program.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/protocol/HttpContext
at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:119)
at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:103)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:362)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:344)
at Gethtml_fromS3.main(Gethtml_fromS3.java:16)
Caused by: java.lang.ClassNotFoundException: org.apache.http.protocol.HttpContext
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 5 more
Can any one give me a solution for the above error?
Please check if you have httpcore jar. I got rid of this error, after adding httpcore-4.2.jar to my lib.
It's difficult to pinpoint the exact reason why your application raises this exception, but a few common pointers that might help you.
The exception is raised when you are using a class in your code that isn't available when it runs. In this case, it seems you are unable to use org.apache.http.protocol.HttpContext. Most likely, you created an application that does include the Amazon AWS jar, but lacks the Apache HTTPClient jar.
If you used gradle/maven to create your application, include it in your dependencies. It it's a plain Java program, ensure the required JARs are available in your classpath (e.g. the same place as you currently have the Amazon jar located).
I'm having a lot of trouble getting Lucene to work on Mac OS 10.7.5
I downloaded the binaries from http://lucene.apache.org/core/2_9_4/demo.html.
I changed my classpath
$ echo $CLASSPATH
/Users/me/Downloads/lucene-4.5.1/demo/lucene-demo-4.5.1.jar:/Users/me/Downloads/lucene-4.5.1/core/lucene-core-4.5.1.jar
Now I'm trying to run it.
$java org.apache.lucene.demo.IndexFiles /Users/me/Downloads/lucene-4.5.1/src
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/lucene/analysis/standard/StandardAnalyzer
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
at java.lang.Class.getMethod0(Class.java:2774)
at java.lang.Class.getMethod(Class.java:1663)
at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.apache.lucene.analysis.standard.StandardAnalyzer
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 6 more
It's not working. Can someone give me a step-by-step guide to getting up and running with Lucene. I have a simple task I want to do achieve, which is searching for text in a directory of files in a more efficient way than grep. Any help would be appreciated.
You'll also need to add two more jars to your classpath: lucene-analyzers-common-{version}.jar to fix this problem, and lucene-queryparser-{version}.jar to fix the next one. More recent demo documentation makes this clear (the documentation you linked to is for version 2.9.4)
I am trying to run a java program and I am getting the following run time error.The error is shown below.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/params/SyncBasicHttpParams
at org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:157)
at org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
at org.apache.http.impl.client.AbstractHttpClient.createClientConnectionManager(AbstractHttpClient.java:309)
at org.apache.http.impl.client.AbstractHttpClient.getConnectionManager(AbstractHttpClient.java:466)
at org.apache.http.impl.client.AbstractHttpClient.createHttpContext(AbstractHttpClient.java:286)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:851)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at net.floodlightcontroller.core.internal.PacketStreamerClient.registerForPackets(PacketStreamerClient.java:90)
at net.floodlightcontroller.core.internal.PacketStreamerClient.main(PacketStreamerClient.java:51)
Caused by: java.lang.ClassNotFoundException: org.apache.http.params.SyncBasicHttpParams
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 10 more
The error is java.lang.NoClassDefFoundError
Now the Obvious reason of NoClassDefFoundError is that a particular class is not available in Classpath, so we need to add that into Classpath or we need to check why it’s not available in Classpath if we are expecting it to be.
Now the files that I have added to the classpath are the following.
export CLASSPATH=$(JARS=(./lib/*.jar); IFS=:; echo "${JARS[*]}")
export CLASSPATH=$CLASSPATH:~/.m2/repository/org/apache/httpcomponents/httpclient/4.0.1/httpclient-4.0.1.jar
export CLASSPATH=$CLASSPATH:~/.m2/repository/org/apache/httpcomponents/httpcore/4.0.1/httpcore-4.0.1.jar
export CLASSPATH=$CLASSPATH:~/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar
export CLASSPAHT=$CLASSPATH:~/ms_thesis/ONOS/httpcore-4.1.jar
#export CLASSPATH=$CLASSPATH:~/ms_thesis/ONOS/lib/httpclient-4.2.jar
export CLASSPATH=$CLASSPATH:~/google-gson-2.2.4/gson-2.2.4.jar
What jar file should I add to find "org/apache/http/params/SyncBasicHttpParams" I am very new to java and don't know how to debug this issue.
According to the official documentation, the class that you seek is only in httpcore since 4.1. You'll want to pull a new JAR.
org/apache/http/params/SyncBasicHttpParams class is under httpcore-4.1-alpha1.jar , which can be downloaded from
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/httpcomponents/httpcore/4.1-alpha1/httpcore-4.1-alpha1.jar
In future if you need to find a jar for a class, then you can use this link, this is generally helpful:
http://www.findjar.com/
In your case it seems to br you missing out some necessory jar files to include in your project
otherwise you any one of your linked is giving an exception mainly because of static method invocation or intialization
Here's the link to get the jar file httpcore-4.1