We Keep Coding

Configuring Pig-0.12.1 with Hadoop-2.5.0

I have an existing Hadoop client (hadoop-0.20.2) at $HADOOP_HOME. With this version of Hadoop all the client configuration files are placed in the directory: $HADOOP_HOME/conf
To get Pig to work, I have set $PIG_CLASSPATH=$HADOOP_HOME/conf. I can then simply run Pig from it's home directory without issues.
I tried to set up a new Hadoop client (hadoop-2.5.0), setting $HADOOP_HOME to point at this directory. For this version of Hadoop I have placed the client configuration at $HADOOP_HOME/etc/hadoop
I then set $PIG_CLASSPATH=$HADOOP_HOME/etc/hadoop. However when I try to run pig now I get the error below:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
at org.apache.pig.Main.run(Main.java:587)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 7 more
Are there other locations that need to be added to the PIG_CLASSPATH to get this up and running?

Managed to find the cause. I was copying over config to $HADOOP_HOME/etc/hadoop that had been sent to me. Inside hadoop-env.sh there was a line
export HADOOP_MAPRED_HOME=....
that was updating the location of the relevant jars to a non-existent location.

Tomcat 7.0.47 will not launch

java.lang.ClassNotFoundException: org.apache.catalina.core.AccessLogAdapter
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.catalina.security.SecurityClassLoad.loadCorePackage(SecurityClassLoad.java:56)
at org.apache.catalina.security.SecurityClassLoad.securityClassLoad(SecurityClassLoad.java:40)
at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:231)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:426)
I don't understand why tomcat will not launch. I'm still configuring it, but I have catalina.jar inside $TOMCAT_HOME/lib. What am I doing wrong here?
Edit: Sorry, forgot to mention I am running on RHL
UPDATE: Okay so after fiddling with catalina.properties and catalina.sh for the past few days, I think I may have discovered the source of the issue. I think my class path is set up correctly, so I went through and double checked the libraries that they were pointed towards and I saw that the catalina.jar file was corrupted. I changed it out and now I'm getting the following message as opposed to the one above:
catalina
at org.apache.catalina.security.SecurityClassLoad.securityClassLoad(SecurityClassLoad.java:40)
at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:231)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:426)
Caused by: java.lang.ClassNotFoundException: javax.servlet.AsyncContext
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 15 more
This class also resides within the catalina.jar file. What is the deal here! Could all of my .jar files be corrupted?

Google Developer Tools Java FATAL ERROR

Thanks in advance for your help.
I am new to the Google developer tools and I have been messing around trying to figure out how to get one of their features to work.
This one most specifically:
https://developers.google.com/appengine/docs/java/tools/eclipse#Running_the_Project
So this is how far I've gotten.
I updated my Java package to 1.7. I installed the Google plug in for eclipse.,
I created a new project named "Guestbook", made sure the boxes were checked, run->DebugAs-> WebApplication, and then I get this huge error:
Exception in thread "main" FATAL ERROR in native method: processing of
-javaagent failed java.lang.UnsupportedClassVersionError: com/google/appengine/tools/development/agent/AppEngineDevAgent :
Unsupported major.minor version 51.0 at
java.lang.ClassLoader.defineClass1(Native Method) at
java.lang.ClassLoader.defineClassCond(ClassLoader.java:637) at
java.lang.ClassLoader.defineClass(ClassLoader.java:621) at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at
java.net.URLClassLoader.access$000(URLClassLoader.java:58) at
java.net.URLClassLoader$1.run(URLClassLoader.java:197) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at
java.lang.ClassLoader.loadClass(ClassLoader.java:247) at
sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:280)
at
sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:338)
I don't know what all this means.
I am still fairly new to java, so any help would be excellent.
Hope all is well

Elastic MapReduce libjars argument to reference s3 jar files

I'm trying to run a custom jar EMR job on Amazon and I have a reference to a lucene jar file in my jar file. I have the jar file in a lib directory on s3 and my Jar arguments look like this:
MyMainClass -libjars s3n://mybucket/lib/lucene-core-3.6.1.jar s3n://mybucket/myinput s3n://mybucket/myoutput
The job fails and I keep getting these errors:
java.lang.NoClassDefFoundError: org/apache/lucene/analysis/Analyzer
at java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:247) at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:861)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:906)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:932)
at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:959)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more Caused by: java.lang.ClassNotFoundException:
org.apache.lucene.analysis.Analyzer at
java.net.URLClassLoader$1.run(URLClassLoader.java:202) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at
java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 21 more
It doesn't seem to find the lucene jar file... What am I missing?

If your runner class implements the Tool interface, then Hadoop will invoke GenericOptionsParser in the background which will pick up your -libjars parameter automatically.

Parse JSON string in Java

I'm trying to parse a JSON string in Java but I cant figure out how!
I'm using lib-JSON right now:
import net.sf.json.*;
.
String jsonStr = "{'string':'JSON', 'integer': 1, 'double': 2.0, 'boolean': true}";
JSONObject jsonObj = (JSONObject) JSONSerializer.toJSON(jsonStr); // this line crashes
I get no compilation errors but when i run the program it says:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/exception/NestableRuntimeException
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at ircbot.plugins.SiteTitle.<init>(SiteTitle.java:28)
at ircbot.Main.main(Main.java:10)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.exception.NestableRuntimeException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 14 more
Java Result: 1
BUILD SUCCESSFUL (total time: 0 seconds)
It sounds like I'm missing some dependencies, but I don't know how to find out if or which!
Thanks in advance!

It seems you have not imported need jar file for class "org.apache.commons.lang.exception.NestableRuntimeException"
There are many classes that we use in our Java program and which don't come with Java. So we need to put Suitable Jar file for them.

I recomend grepcode
Just enter the class name and you get a list of the known versions of jars
Example: here

You are missing jar file. No jar file in your class path.
see this Reading JSON file error

org/apache/commons/lang -> http://commons.apache.org/lang/
Basically, use Google and get the respective jar.

Your JSON String also not valid, This the valid JSON,
{"string": "JSON","integer": "1","double": "2.0","boolean": "true" }

These are the dependencies -
jakarta commons-lang 2.5
jakarta commons-beanutils 1.8.0
jakarta commons-collections 3.2.1
jakarta commons-logging 1.1.1
ezmorph 1.0.6
are you sure you added all of them in your classpath.