I have an existing Hadoop client (hadoop-0.20.2) at $HADOOP_HOME. With this version of Hadoop all the client configuration files are placed in the directory: $HADOOP_HOME/conf
To get Pig to work, I have set $PIG_CLASSPATH=$HADOOP_HOME/conf. I can then simply run Pig from it's home directory without issues.
I tried to set up a new Hadoop client (hadoop-2.5.0), setting $HADOOP_HOME to point at this directory. For this version of Hadoop I have placed the client configuration at $HADOOP_HOME/etc/hadoop
I then set $PIG_CLASSPATH=$HADOOP_HOME/etc/hadoop. However when I try to run pig now I get the error below:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
at org.apache.pig.Main.run(Main.java:587)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 7 more
Are there other locations that need to be added to the PIG_CLASSPATH to get this up and running?
Managed to find the cause. I was copying over config to $HADOOP_HOME/etc/hadoop that had been sent to me. Inside hadoop-env.sh there was a line
export HADOOP_MAPRED_HOME=....
that was updating the location of the relevant jars to a non-existent location.
Related
I created a Java project in Eclipse and created a simple class with a main method.
Problem: when I run the program(without any server etc), I get the following error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/codec/DecoderException
at autoPEC.main(autoPEC.java:6)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.codec.DecoderException
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 1 more
What I tried to solve the problem: included path to commons-codec.jar in the environment variables JAVA_HOME by putting a semicolon after the jdk path
Result: class still does not run
I think that the problem could be that I am not correctly including the classpath. How should I do this?
Google the missing jar and include it in the Java Build path
Reason for this exception is missing of a jar.
please add the jar(Commons-codec.jar) file needed for this org.apache.commons.codec.DecoderException
Download it from here
http://commons.apache.org/proper/commons-codec/download_codec.cgi
then put it in your classpath
java.lang.ClassNotFoundException: org.apache.catalina.core.AccessLogAdapter
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.catalina.security.SecurityClassLoad.loadCorePackage(SecurityClassLoad.java:56)
at org.apache.catalina.security.SecurityClassLoad.securityClassLoad(SecurityClassLoad.java:40)
at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:231)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:426)
I don't understand why tomcat will not launch. I'm still configuring it, but I have catalina.jar inside $TOMCAT_HOME/lib. What am I doing wrong here?
Edit: Sorry, forgot to mention I am running on RHL
UPDATE: Okay so after fiddling with catalina.properties and catalina.sh for the past few days, I think I may have discovered the source of the issue. I think my class path is set up correctly, so I went through and double checked the libraries that they were pointed towards and I saw that the catalina.jar file was corrupted. I changed it out and now I'm getting the following message as opposed to the one above:
catalina
at org.apache.catalina.security.SecurityClassLoad.securityClassLoad(SecurityClassLoad.java:40)
at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:231)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:426)
Caused by: java.lang.ClassNotFoundException: javax.servlet.AsyncContext
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 15 more
This class also resides within the catalina.jar file. What is the deal here! Could all of my .jar files be corrupted?
I'm trying to run a custom jar EMR job on Amazon and I have a reference to a lucene jar file in my jar file. I have the jar file in a lib directory on s3 and my Jar arguments look like this:
MyMainClass -libjars s3n://mybucket/lib/lucene-core-3.6.1.jar s3n://mybucket/myinput s3n://mybucket/myoutput
The job fails and I keep getting these errors:
java.lang.NoClassDefFoundError: org/apache/lucene/analysis/Analyzer
at java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:247) at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:861)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:906)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:932)
at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:959)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more Caused by: java.lang.ClassNotFoundException:
org.apache.lucene.analysis.Analyzer at
java.net.URLClassLoader$1.run(URLClassLoader.java:202) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at
java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 21 more
It doesn't seem to find the lucene jar file... What am I missing?
If your runner class implements the Tool interface, then Hadoop will invoke GenericOptionsParser in the background which will pick up your -libjars parameter automatically.
I tried to connect java with My MS SQL express, I downloaded the sqljdbc4.jar also
When I don't bother with the CLASSPATH and all and I try to execute my program( even with sqljdbc in
C:\Program Files (x86)\Java\jre6\lib
I get the following error
java.lang.ClassNotFoundException: com.microsoft.jdbc.sqlserver.SQLServerDriver
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at Connect.getConnection(Connect.java:24)
at Connect.displayDbProperties(Connect.java:42)
at Connect.main(Connect.java:78)
Error Trace in getConnection() : com.microsoft.jdbc.sqlserver.SQLServerDriver
Error: No active Connection
But If I try and set the CLASSPATH variable to point to sqljdbc4.jar then ClassNotHoundExcepetion is occurring with identifying my classname.
Need immediate help. kindly respond.
I use Command line mode of invoking it, I tried setting the classpath in the system variables dialog box of windows 7. and when I do that or use set CLASSPATH="C:\temp\sqljdbc4.jar" the jre fails to recogonize the main class of my program and throws
Exception in thread "main" java.lang.NoClassDefFoundError: Connect
Caused by: java.lang.ClassNotFoundException: Connect
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: Connect. Program will exit.
So I am at a loss here.
Your problem occur because classloader has not found driver jar, how you start your program?
Have you put -cp or -classpath if it command line ? Or add to lib folder if web application ?
You should set the classpath to include the sqljdbc4.jar file. See details here.
I am using Ubuntu 11.04, when i start RubyMine3.1 it gave this error.
Exception in thread "main" java.lang.NoClassDefFoundError: com/intellij/util/lang/UrlClassLoader
at com.intellij.idea.Main.main(Main.java:55)
Caused by: java.lang.ClassNotFoundException: com.intellij.util.lang.UrlClassLoader
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 1 more
I tried every thing, i also have sun-java installed, set it default by update-java-alternatives, also my JAVA_HOME, JDK_HOME, RUBYMINE_JDK points to the sun-java directory, but still no success.
Does any one have any idea about it?
Damaged download or some classes left from older version? Try to download again and unpack into an empty directory.
Did you install any third-party plug-ins? In this case you should try to delete them from the plugins directory.
The class which fails to load is located in RubyMine/lib/util.jar. Maybe you are using an old version of rubymine.sh which doesn't add util.jar to the classpath?