We Keep Coding

Wrong classpath declaration gets a java.lang.NoClassDefFoundError

I have a small project which in my local IDE is running just fine, a screenshot will show the project structure:
I have highlighted the jewelcli jar because on that class i get the exception mentioned in the header.
While running locally all goes fine, on a testing ubuntu server i get:
java -verbose -classpath "/correct/path/jarName.jar:lib/jars/*" correct.package.and.main.Class
***all bootstrapping class from RT.JAR***
java.lang.NoClassDefFoundError: uk/co/flamingpenguin/jewel/cli/ArgumentValidationException
at correct.package.and.main.Class.main(Class.java:31)
Caused by: java.lang.ClassNotFoundException: uk.co.flamingpenguin.jewel.cli.ArgumentValidationException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 1 more
I dont think this is actually related at all to that jewelcli jar in particular, because i double checked and its there:
unzip -l jarname.jar | grep jewelcli
435937 11-20-11 01:22 lib/jars/jewelcli-0.7.6.jar
But it happens for that Jar because it is the very first jar that the program needs (after all the bootstrapping classes in rt.jar).
This is where i am basing my command line from
But i also see:
Subdirectories are not searched recursively. For example, foo/* looks for JAR files only in foo, not in foo/bar, foo/baz, etc.
I thought that by specifying in the command line the jar file containing all the libraries, java would automaticaly, and recursively, scan all subdirectories, but it does not work:
java -verbose -classpath "/correct/path/jarName" correct.package.and.main.Class
Any tip on how can i get java to scan the whole jar i specify in the -classpath option?

I thought that by specifying in the command line the jar file containing all the libraries, java would automaticaly, and recursively, scan all subdirectories
It depends. The standard Java classloaders do not know how to load classes from JARs that embedded inside other JARs.
And that's what you appear to have done ... if I am reading your question correctly.
There are a few ways to address this:
Copy all of the dependent JARs to the server, put them into a directory, and add the directory to the classpath; e.g.
java -classpath "/correct/path/jarName.jar:/correct/path/lib/*" \
correct.package.and.main.Class
Build an UberJAR or shaded JAR by unpacking the dependent JARs into a directory, adding your classes and producing a JAR from the tree. There are maven plugins for building such JAR files.
Use something like Spring Boot which uses a classloader that understands how to deal with JARs withing JARs.

How to run a jar file in hadoop?

I have created a jar file using the java file from this blog using following statements
javac -classpath /usr/local/hadoop/hadoop-core-1.0.3.jar -d /home/hduser/dir Dictionary.java
/usr/lib/jvm/jdk1.7.0_07/bin/jar cf Dictionary.jar /home/hduser/dir
Now i have tried running this jar in hadoop by hit and trial of various commands
1hduser#ubuntu:~$ /usr/local/hadoop/bin/hadoop jar Dictionary.jar
Output:
Warning: $HADOOP_HOME is deprecated.
RunJar jarFile [mainClass] args...
2.hduser#ubuntu:~$ /usr/local/hadoop/bin/hadoop jar Dictionary.jar Dictionary
Output:
Warning: $HADOOP_HOME is deprecated.
Exception in thread "main" java.lang.ClassNotFoundException: Dictionary
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
How can i run the jar in hadoop? I have the right DFS Locations as per needed by my program.

I was able to reproduce your problem. The problem is where you are creating the jar.
Basically, the directory that you are packaging into the jar is confusing the jar file in locating the main class file. Instead if you try doing :
/usr/lib/jvm/jdk1.7.0_07/bin/jar cf Dictionary.jar /home/hduser/dir/Dictionary.class
i.e. package the class file specifically into the jar and then run:
/usr/local/hadoop/bin/hadoop jar Dictionary.jar Dictionary
It just works fine provided that you have a main function in your class called Dictionary.
The problem is when you package a full directory inside a jar then the jar also needs to be aware of the directory structure to locate the class file. For this, we need to have a well defined package hierarchy to define the class location. So, when you are packaging /home/hduser/dir/ into the jar, the jar is not aware of the location of the class file which is located deep inside this directory structure. For this you need to add a package name to your .java file according to the directory structure , for example home.hduser.dir and while running the hadoop jar command specify the class name with the package structure, for example home.hduser.dir.Dictionary.

Use the below command to run the hadoop jar file from the CLI.
hadoop jar <jarFileName> <mainClassname> <AnyCommandLineArguements>

I also ran into the same problem and the console does not show much information but just
RunJar jarFile [mainClass] args...
Please check the jar for the package folders location, as a straightforward approach please try is the package starts with com.company...
The "com" folder should be the first level folder when the jar is unpacked

Run jar file with out specify supporting jar file

I create a new jar file. when i run the jar using
java -jar UpdateAvailable.jar com.dao.mysql.UpdateAvailable
I got the following exception.
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/jettison/json/JSONException
Caused by: java.lang.ClassNotFoundException: org.codehaus.jettison.json.JSONException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
But I copied all the supporting jars inside the UpdateAvailable.jar. Now what my requirement is run the jar file with out specifying supporting jars, like
java -jar UpdateAvailable.jar com.dao.mysql.UpdateAvailable

Here are some points to remember while you are working with JAR files and ClassPath: -
For making your JVM find class, your classpath should be set till the directory containing the class. In this case you have JAR.
So, you need to give the path till your Jar File Name
Now, since you have your Jar files inside your currently running Jar file, you can't set your classpath till the folder inside a Jar, so basically you won't be able to set classpath for inner Jar Files..
So, clearly JVM won't be able to see classes inside your inner Jar..
So, make sure that your required JAR files are outside of all the JAR files (I mean to say - avoid Nested JAR Files)..

I'm not sure how you are packaging your jar file, but it would sound like you need something like One-Jar to help you package your file and all your supporting libraries into a single jar. One-Jar comes with it's own classloader that allows it to load all the included jars in your executable jar file.
Configuration is fairly straight forward and easy to use. If you are using Maven, you can use the one-jar maven plugin to help you configure which libs you want included.
There are also other similar packages you can use instead of One-Jar, such as JarJar and ProGuard. Also take a quick look at this SO question.

In executable jar couldn't reads any supporting jar files inside a jar. So keep all supporting jars in a folder and place the executable jar in the same directory. And add "Class-Path" as "Supporting jar files separate with space" in "MANIFEST.MF" File. Now run the jar as
java -jar UpdateAvailable.jar com.dao.mysql.UpdateAvailable

java.lang.NoClassDefFoundError in Hadoop-1.0.1

i am working with Hadoop-1.0.1 in Eclipse. I am trying to run wordcount application but when i run my WordCount.java, Eclipse shows me following error
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:143)
at test.WordCount.main(WordCount.java:56)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
... 2 more
I have hadoop-core-1.0.1.jar in my classpath. Should the application require more jar

It seems you are getting this error because either two important logging classes commons-logging-1.1.1.jar and commons-logging-api-1.0.4.jar which are related with common logging is either not available from your classpath or you just don't have it.
In most cases these files are in \lib folder with other Hadoop Jar so when you set class path these jar are accessible. Please verify that you do have these files in your machine and are set in your classpath so they are accessible.
Also when you run "hadoop version" the version comes up as result, the same hadoop-core--SNAPSHOT.jar must be in your $HADOOP_HOME location.
You mentioned that you are running WordCount.java instead you should say that you are running WordCount job because you are always using the compiled application as jar not java.
$ bin/hadoop jar /usr/jboss/wordcount.jar org.myhadoop.WordCount /usr/jboss/wordcount/input /usr/jboss/wordcount/output

hadoop-core-1.0.1 is probably not in your classpath: commons-logging is actually included several times in the hadoop jar!

It seems that you did not include the common-logging jar file so Download Here the jar file depends on version, Then will included it in your program and run it THANK yoU

Why edit to Manifest file provides solution that CLASSPATH does not

I'm writing a short utility that performs a short task on a remote Oracle instance. The utility uses the Oracle jdbc driver found in ojdbc6.jar.
The Oracle jar is properly included in my classpath. When I export my project to a jar and run the jar on the command line using "java -jar ", I get:
Exception in thread "main" java.lang.ClassNotFoundException: oracle.jdbc.OracleDriver
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
After moving the Oracle jar into the same directory as my runnable jar file, I changed my Manifest file to include the the Oracle jar in the classpath. This fixed the problem, and my jar ran fine.
Can anyone explain why including the Oracle jar in my classpath didn't solve the problem, and why editing the Manifest file did?
Thanks!

From Sun (Oracle) documentation for the java -jar command:
Execute a program encapsulated in a JAR file. The first argument is
the name of a JAR file instead of a startup class name. In order for
this option to work, the manifest of the JAR file must contain a line
of the form Main-Class: classname. Here, classname identifies the
class having the public static void main(String[] args) method that
serves as your application's starting point. See the Jar tool
reference page and the Jar trail of the Java Tutorial for information
about working with Jar files and Jar-file manifests. When you use this
option, the JAR file is the source of all user classes, and other user
class path settings are ignored.