I am trying to run a java class in Hadoop, just like in the book "Hadoop.The.Definitive.Guide", like this:
hadoop URLCat hdfs://localhost/user/hamza/sometext.txt
Where it is supposed to print the contents of sometext.txt to the terminal.
What happens is when I type that command it gives me this error:
Error: Could not find or load main class URLCat
URLCat is a Java class, but I have no idea why it's not working.
I tried converting it to a jar file using IntelliJ but then when I run it, like this:
hadoop jar Hadoop_example.jar URLCat hdfs://localhost/user/hamza/sometext.txt
Note: Hadoop_example.jar is the jar file name and the class that has main method is URLCat.
It gives me this error:
Exception in thread "main" java.lang.ClassNotFoundException: URLCat
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
So how can I run Java classes in Hadoop.
I found a similar question on Stack overflow but the answers were mostly to convert it into a jar file, but that didn't work for me either.
hadoop jar Hadoop_example.jar URLCat hdfs://localhost/user/hamza/sometext.txt
This is the correct command line to start a program on hadoop. Why it does not start depends on how you made the jar.
This POST will be useful to you.
Related
Whenever I execute the command : hadoop jar wc.jar WordCount text.txt output
It gives me this error
Exception in thread "main" java.lang.ClassNotFoundException: WordCount
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.util.RunJar.run(RunJar.java:316)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Even though everything seems to be fine
Can anyone help me out with this ??
first of all you need to make sure you have a main method inside the WordCount class.
You can also look at this answer right here: How to run a jar file in hadoop?
As the answers suggest there's no problem when running the jar, but when creating the jar.
So for example, you need to run:
JAVA_HOME/bin/jar cf WordCount.jar WordCount.class
and then run
hadoop jar WordCount.jar WordCount
Hope this helps.
I am not particularly well experienced with Java at all and try to get a jar file running on my Ubuntu machine (https://sites.google.com/site/communitydetectionslpa/home).
However once I run the jar file with the command suggested by developers I receive the following error:
java -jar GANXiSw.jar -i test.ipairs
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/collections/map/MultiKeyMap
at Net.<init>(Net.java:38)
at SLPAw.<init>(SLPAw.java:146)
at SLPAw.main(SLPAw.java:2050)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.collections.map.MultiKeyMap
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
Apparently java is not able to import the org.apache.commons class properly. After some researching I checked if I have libcommons-collections3-java installed, which however is the case.
I read something about adding the library explicitly to my CLASSPATH, which however I also read to be not good practice.
What is the best approach to fix my issue?
I am trying to run a simple junit test using OS X and terminal. I have placed the Junit jar file inside my java folder. I was able to compile all of the files using:
javac -cp .:"/Library/java/junit.jar" *.java
It compiles just fine, with no errors. However when I try to run the command:
java TestRunner
It gives the error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/junit/runner/JUnitCore
at TestRunner.main(TestRunner.java:7)
Caused by: java.lang.ClassNotFoundException: org.junit.runner.JUnitCore
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
I cannot seem to find what I am doing incorrectly. Any help solving this would be greatly appreciated.
Like #ToddSewell said, the external libraries should be in your classpath for the execution too. Try this:
java -cp .:"/Library/java/junit.jar" TestRunner
I made a java class using the library Jaudiotagger, and it runs without problems. I then made a jar out of it and I got NoClassDefFoundError. Here's how it went:
I put the main class id3tag.java and the library jaudiotagger-2.0.3.jar in a folder and compiled using the command line. The program ran smoothly without problems.
javac -cp .;jaudiotagger-2.0.3.jar id3tag.java
java -cp .;jaudiotagger-2.0.3.jar id3tag
I then created the manifest and the jar file.
echo Main-Class: id3tag >manifest.txt
jar cvfm id3tag.jar manifest.txt id3tag.class jaudiotagger-2.0.3.jar
I got the following output:
added manifest
adding: id3tag.class(in = 5952) (out= 2997)(deflated 49%)
adding: jaudiotagger-2.0.3.jar(in = 811441) (out= 740599)(deflated 8%)
I then ran the jar file, and got "A Java Exception has occurred.". I also tried:
java -jar id3tag.jar
And I got the output:
Exception in thread "main" java.lang.NoClassDefFoundError: org/jaudiotagger/tag/
FieldDataInvalidException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2451)
at java.lang.Class.getMethod0(Class.java:2694)
at java.lang.Class.getMethod(Class.java:1622)
at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.jaudiotagger.tag.FieldDataInval
idException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 6 more
I then replaced all (both) error classes used from Jaudiotagger with Exception and recreated the jar. Now the same thing happens as for seemingly all other jar files when I run them (with Java(TM) Platform SE binary, or through writing id3tag.jar in the command prompt) - nothing. java -jar, however, works and gave me a runtime error.
Exception in thread "main" java.lang.NoClassDefFoundError: org/jaudiotagger/audi
o/AudioFileIO
at id3tag.tagSong(id3tag.java:112)
at id3tag.tagAlbum(id3tag.java:82)
at id3tag.tagArtist(id3tag.java:40)
at id3tag.main(id3tag.java:170)
Caused by: java.lang.ClassNotFoundException: org.jaudiotagger.audio.AudioFileIO
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 4 more
It seems like the entire library just doesn't do shit for me. How do I fix this, and how do I get jar files to run without java -jar? (Also, how do I get the full pile'o'errors in case someone needs it, rather than just having it say "x more"?)
I'm not very savvy with this kinda shit so the more specific the answer, the better. Thanks.
I'm on windows 8 and latest java (1.7.0_17).
this is informing you that you have no main class, as you might or might not already know all programs in almost all non web based programming languages must have a main class to start from. In java it looks like this
public static int main(String[] args){
}
The message is informing you that this class is no longer there, this could be a compiler error or something entirely different.
please try downloading the library again, and downloading eclipse ide for java ee developers.
this ide has a built in compiler that will work better than a command line sometimes.
i hope this helped.
I worked around it by including the org folder from jaudiotagger in my .jar instead of the actual .jar file, and I then used a .bat file to run it instead of running the .jar directly. Feel free to add your own answer if you find a better solution, and I'll check back.
I extracted jaudiotagger-2.0.3.jar with winrar so that the org folder was in the same folder as my main class. Then I could compile and run the main class simply by:
javac id3tag.java
java id3tag
I then created the manifest file the same way as before, and created the jar:
jar cvfm id3tag.jar manifest.txt id3tag.class org
The jar file worked with java -jar id3tag.jar, but simply writing id3tag.jar still did nothing. Turns out it's because jar files are by default run by the javaw.exe file, so you have to right click the jar file -> open with -> choose default program and navigate to the java.exe file in your java folder (just search the folder). Running the .jar then gets you "Error: Could not find or load main class path\id3tag.jar". I worked around this by using a .bat file. I entered
java -jar path\id3tag.jar
into Notepad and saved as whatevername.bat (save as type: all files).
I created a new question for getting the .jar file to work properly. See .jar error - could not find or load main class.
I am trying to run a java program and I am getting the following run time error.The error is shown below.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/params/SyncBasicHttpParams
at org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:157)
at org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
at org.apache.http.impl.client.AbstractHttpClient.createClientConnectionManager(AbstractHttpClient.java:309)
at org.apache.http.impl.client.AbstractHttpClient.getConnectionManager(AbstractHttpClient.java:466)
at org.apache.http.impl.client.AbstractHttpClient.createHttpContext(AbstractHttpClient.java:286)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:851)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at net.floodlightcontroller.core.internal.PacketStreamerClient.registerForPackets(PacketStreamerClient.java:90)
at net.floodlightcontroller.core.internal.PacketStreamerClient.main(PacketStreamerClient.java:51)
Caused by: java.lang.ClassNotFoundException: org.apache.http.params.SyncBasicHttpParams
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 10 more
The error is java.lang.NoClassDefFoundError
Now the Obvious reason of NoClassDefFoundError is that a particular class is not available in Classpath, so we need to add that into Classpath or we need to check why it’s not available in Classpath if we are expecting it to be.
Now the files that I have added to the classpath are the following.
export CLASSPATH=$(JARS=(./lib/*.jar); IFS=:; echo "${JARS[*]}")
export CLASSPATH=$CLASSPATH:~/.m2/repository/org/apache/httpcomponents/httpclient/4.0.1/httpclient-4.0.1.jar
export CLASSPATH=$CLASSPATH:~/.m2/repository/org/apache/httpcomponents/httpcore/4.0.1/httpcore-4.0.1.jar
export CLASSPATH=$CLASSPATH:~/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar
export CLASSPAHT=$CLASSPATH:~/ms_thesis/ONOS/httpcore-4.1.jar
#export CLASSPATH=$CLASSPATH:~/ms_thesis/ONOS/lib/httpclient-4.2.jar
export CLASSPATH=$CLASSPATH:~/google-gson-2.2.4/gson-2.2.4.jar
What jar file should I add to find "org/apache/http/params/SyncBasicHttpParams" I am very new to java and don't know how to debug this issue.
According to the official documentation, the class that you seek is only in httpcore since 4.1. You'll want to pull a new JAR.
org/apache/http/params/SyncBasicHttpParams class is under httpcore-4.1-alpha1.jar , which can be downloaded from
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/httpcomponents/httpcore/4.1-alpha1/httpcore-4.1-alpha1.jar
In future if you need to find a jar for a class, then you can use this link, this is generally helpful:
http://www.findjar.com/
In your case it seems to br you missing out some necessory jar files to include in your project
otherwise you any one of your linked is giving an exception mainly because of static method invocation or intialization
Here's the link to get the jar file httpcore-4.1