We Keep Coding

Class not found even when in Jar in the classpath

I am trying to deploy an application that is using Jackson, JUnit, and Commons-IO. I have the following Jars in my application's classpath:
commons-io-2.4.jar
jackson-databind-2.7.0.jar
jackson-annotations-2.7.0.jar
log4j-api-2.4.1.jar
wsdiscovery-0.2.jar
jackson-core-2.7.0.jar
log4j-core-2.4.1.jar
This application works within my development environment, and I have deployed all of the above Jars with the main application jar. I can run the application without problems, but every time I try to use it I get the following failure:
Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
at com.oncam.hware.app.OnvifApp.formatOutput(OnvifApp.java:356)
at com.oncam.hware.app.OnvifApp.dispatchCommand(OnvifApp.java:271)
at com.oncam.hware.app.OnvifApp.loopSocket(OnvifApp.java:130)
at com.oncam.hware.app.OnvifApp.useSocket(OnvifApp.java:216)
at com.oncam.hware.app.OnvifApp.main(OnvifApp.java:473)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.ObjectMapper
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 5 more
The ObjectMapper class is in the jackson-databind-2.7.0.jar file. Furthermore, I have no problems accessing the classes in the other jar files (including the JUnit jars!). For some reason, it is as if the classloader is loading every Jar except jackson-databind-2.7.0.jar.
Does anyone know what is causing this and how I can fix it?
Someone please advise...

I figured out what was wrong.
It turns out that the environment I am using (Eclipse!) does not properly update the manifest file when you export your code to a JAR file. Without the proper manifest entries, the application cannot "find" the dependent jar files.
This is, in my opinion, a serious oversight on the part of the Eclipse folks -- especially when you have an application that depends on a lot of jar files. In order to make my application run, I had the following choices:
Create a script that runs the jvm and has a list of parameters pointing to every needed jar file, or:
2: Manually enter each required jar file into the Manifest file
To my knowledge, there is no way to automatically update the manifest file. This is a serious PITA (Pain In The A**)...
Anyway, sorry for bothering people about this problem. Hopefully, posting this answer will help others avoid similar problems...

Parso Java Library Issue

I am trying to use the Parso Java Library in order to read in a .sas7bdat file. My goal is to convert it to a CSV, and then manipulate it further afterwards. Directions I am following for setting it up are here. I have the parso .jar file imported into Eclipse successfully. However, when I try to instantiate as so:
InputStream streamIn = new FileInputStream(sasFile);
SasFileReader sasFileReader = new SasFileReader(streamIn);
I get this exception:
Exception in thread "main"
java.lang.NoClassDefFoundError:org/slf4j/LoggerFactory
at com.ggasoftware.parso.SasFileReader.<clinit>(SasFileReader.java:30)
at sas7bdatFileConverter.convert(sas7bdatFileConverter.java:25)
at sas7bdatFileConverter.main(sas7bdatFileConverter.java:11)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
I would greatly appreciate any hints as to what I might be missing here. Thank you!

The page you cited says you should be using a Maven dependency (which will automatically download all the jar files needed), but it seems like you are directly importing the jar file.
Because you're not using Maven, you also need to include a slf4j jar file (and that's either slf4j-api which doesn't actually do anything, or at least slf4j-simple which logs to the console).
Simply go to http://www.slf4j.org/ and download the newest package, then take out slf4j-api-<version>.jar (and slf4j-simple-<version>.jar if you want to see the Parso library logging into your console).
In the long run, definitely learn how to use Maven, it will immensely ease your pain with library management.

Using MessagePack with Android

Has someone tried to use MessagePack with an Android app?
Is it possible? I have tried to use the Jar from msgpack-java and received the following Exception:
Caused by: java.lang.ExceptionInInitializerError
at org.msgpack.Packer.pack(Packer.java:532)
at org.msgpack.MessagePack.pack(MessagePack.java:31)
... 15 more
Caused by: java.lang.ExceptionInInitializerError
at org.msgpack.template.TemplateRegistry.<clinit>(TemplateRegistry.java:38)
... 17 more
Caused by: java.lang.VerifyError: org.msgpack.template.BeansFieldEntryReader
at org.msgpack.template.builder.BeansTemplateBuilder.<init (BeansTemplateBuilder.java:42)
at org.msgpack.template.builder.BuilderSelectorRegistry.initForJava(BuilderSelectorRegistry.java:73)
at org.msgpack.template.builder.BuilderSelectorRegistry.<clinit>(BuilderSelectorRegistry.java:38)
... 18 more
The code that I use is very simple
PrintWriter out = new PrintWriter(socket.getOutputStream());
Message msg = new Message();
msg.body = "asdasdasd";
msg.from = "qwe";
msg.to = "ttt";
byte[] bytes = MessagePack.pack(msg);
out.print(bytes);
out.flush();
I have javassist.jar, msgpack-0.5.2.jar, slf4j-api-1.6.2.jar and slf4j-jdk14-1.6.2.jar in my lib directory.
In my server application this code works fine with the same libraries.

(Hopefully) FINAL UPDATE
msgpack : 0.6.8 works on Android without any problems
msgpack-rpc : 0.7.0 works on Android with one caveat.
Specifically, you need to add the following to onCreate for API Level 8 (Android 2.2.1), and possibly lower:
java.lang.System.setProperty("java.net.preferIPv4Stack", "true");
java.lang.System.setProperty("java.net.preferIPv6Addresses", "false");
due to this bug.
If you want to see a simple example, here's a pair of projects set up for this purpose:
https://github.com/mikkoz/msgpack-android-test-server/tree/master/msgpack-android-test-server
https://github.com/mikkoz/msgpack-android-test-client/tree/master/msgpack-android-test-client
Previous Versions
UPDATE: as of 0.6.7 msgpack should be compatible with Android (there is a small dependency exclusion issue). Check the text below for msgpack-rpc (which also might be adapted in the future).
NOTE: If you're also using msgpack-rpc, you need to do the following steps:
Download the msgpack-rpc source from git://github.com/msgpack/msgpack-rpc.git (specifically, the "java" folder).
Change the main msgpack artifact version to the one you've built.
In org.msgpack.rpc.loop.netty.NettyEventLoop, change the NioClientSocketChannelFactory to OioClientSocketChannelFactory(getWorkerExecutor()).
Build the MessagePack-RPC in the same way as in the case of the main MessagePack JAR (see Step 11 above).
The NettyEventLoop replacement is due to this issue:
http://markmail.org/message/ypa3nrr64kzsyfsa .
Important: I've only tested synchronous communication. Asynchronous might not work.
And here's the reason for msgpack not working with Android prior to 0.6.7:
The reason for the error is that MessagePack uses several java.beans classes that are not included in the Android SDK. You're probably using the MessagePackBeans annotation.
This is a similar problem to the one described here, for which the general solution is outlined here. Unfortunately, in our case it requires a rebuild of msgpack. Here's what I did (you can almost certainly skip Steps 5 and 8, but I haven't tried it that way) :
Download the MessagePack source from https://github.com/msgpack/msgpack-java.git.
Import the MessagePack source as a project in your IDE.
Download the Apache Harmony source for the relevant packages from http://svn.apache.org/repos/asf/harmony/enhanced/java/trunk/classlib/modules/beans/src/main/java .
Copy these packages into your MessagePack project's src/main/java folder:
java.beans
java.beans.beancontext
org.apache.harmony.beans
org.apache.harmony.beans.internal.nls
In your MessagePack project, remove the following classes:
PropertyChangeListener
IndexedPropertyChangeEvent
PropertyChangeEvent
PropertyChangeListenerProxy
PropertyChangeSupport
Rename the java.beans packages to something different, e.g. custom.beans .
Change all java.beans references to the renamed ID, so again e.g. custom.beans. This applies especially to BeansFieldEntryReader (this class is the reason for the original error).
Change the custom.beans references for the five classes you removed in Step 5 back to java.beans.
In the org.apache.harmony.beans.internal.nls.Messages class, comment out the method setLocale, and remove the imports associated with it.
Remove all classes that still have errors, except Encoder. In that class, comment out all references to the classes you've removed. You should now have an error-free project.
Build the MessagePack JAR:
If you're using Maven, change the version in the pom.xml to something unique, run Maven build with the install goal, then add the dependency in your Android project with that version.
If you're not using Maven, you have to run the jar goal for Ant with the included build.xml. Replace the msgpack JAR in your Android project with this one.
If you're publishing your app, remember to include the relevant legal notice for Apache Harmony. It's an Apache License, just like MessagePack.
That should do it. Using your example code, and my own data class, I was successfully able to pack and unpack data.
The entire renaming ritual is due to the fact that the DEX compiler complains about java.* package naming.

There is a critical msgpack bug saying data packed with msgpack will get corrupted on the Dalvik VM. http://jira.msgpack.org/browse/MSGPACK-51

There is an ongoing effort by #TheTerribleSwiftTomato and the MessagePack core team to get MessagePack working on Android, please see the related GitHub issue. The fix mentioned in #TheTerribleSwiftTomato's answer is to be found here.
Update
I've managed to get it at least running on Android by (painstakingly) adding all the necessary javassist Classes which are currently required for the build to succeed. An extra 600KB gain in size, yet at least it seems to work. All in all, it appears to be working to some extent on Android, eventually check out the lesser-known resources about Message Pack such as its User Group and its Wiki for more information.
On a side-note, be sure to use a HTTP Request Library (such as LoopJ's Android Async HTTP or Apache's HttpClient) which can handle binary data.
Last but not least you can ping me if there is interest in this jar which makes MessagePack seemingly work on Android – credits go out of course to #TheTerribleSwiftTomato who supplied the fix above!

I suggest you write this in the main proguard-rules file-
-dontwarn org.msgpack.**
-keep class org.msgpack.** { *; }

Tomcat cannot find class "InaccessibleWSDLException", glassfish can

I am working with webservices in Java, and right after opening the port and calling the service, I try to catch a "com.sun.xml.ws.wsdl.parser.InaccessibleWSDLException".
This worked fine on my dev Glassfish v2 server, but when I shifted my WAR file to Tomcat, I got a NoClassDefFoundError:
java.lang.NoClassDefFoundError: com/sun/xml/ws/wsdl/parser/InaccessibleWSDLException
I did some Googled it and found out that the JAR jaxws-rt.jar had the InaccessibleWSDLException class. But including this JAR in my WAR only broke more things.
I would appreciate any help in eliminating this error.
Thanks.
Additional Info:
After adding jaxws-rt.jar, I get this on all services, irrespective of whether I catch InaccessibleWSDLException or not:
java.lang.ClassNotFoundException: com.sun.xml.stream.buffer.XMLStreamBuffer

The problem is probably that you're importing a class starting with com.sun.. This is generally bad: Why Developers Should Not Write Programs That Call 'sun' Packages
This class is not documented as part of the JAX-WS specification, so I expect this class is part of the JAX-WS implementation in Glassfish; it is unlikely to be present in any other implementation and you should not reference it if you want your code to be portable. I suggest you catch WebServiceException instead (which I believe is the parent type).

You need to add all WSDL dependencies, I'm not sure what they are but stream buffer looks like it is, jars can be found here:
http://download.java.net/maven/1/com.sun.xml.stream.buffer/jars/
It works in glassfish because it has all those jars installed by default while Tomcat does not.

How to unit test JSON parsing

I'm working on an Android app in which I download JSON data from a web service. The class parsing the data looks something like this:
public class JsonCourseParser implements CourseParser {
public Course parseCourse(String courseData) {
Course result;
try {
JSONObject jsonObject = new JSONObject(courseData);
...
result = new Course("foo", "bar");
} catch (JSONException e) {
result = null;
}
return result;
}
}
This builds just fine and it works when I call parseCourse() from within my app, but when I try to test this method in a unit test I get this exception:
java.lang.NoClassDefFoundError: org/json/JSONException
at JsonCourseParserTest.initClass(JsonCourseParserTest.java:15)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.ClassNotFoundException: org.json.JSONException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 16 more
It seems that the classes of the org.json package which come with the Android Framework are not available in Java by default.
Is there a way I can fix this so I can unit test the classes which parse JSON?

All you need to do is add the following line to the dependency section in your build.gradle:
testImplementation 'org.json:json:20180813'
Note that you also need to use Android Studio 1.1 or higher and at least build tools version 22.0.0 or above for this to work!
testImplementation means that the dependency is included as a test dependency will only be used when your project is being compiled for executing unit tests. The supplied jar will not be included in your APK and will only be used to replace the missing classes in the org.json package.

Edit: see updated answer at the end
I've found the answer to this one. It doesn't solve my problem, but at least it explains why there is a problem.
First, I made the faulty assumption that the JSONObject (imported from the org.json package) was included as a part of the JRE. It isn't - in an Android project, this resides in android.jar (classic "duh" moment).
This discovery boosted my self confidence a bit. This could easily be solved by adding a reference to the android.jar in my unit test project - or at least so I thought for a brief moment. Doing so only gave me another error when running my test:
java.lang.RuntimeException: Stub!
at org.json.JSONObject.<init>(JSONObject.java:8)
...
At least this gave me something more to google for. What I found however, wasn't really encouraging (yet another classic "duh" moment)...
This blog describes the problem pretty well: Why Android isn’t ready for TDD, and how I tried anyway. If you don't bother to read the whole thing, the brief explanation is as follows:
The problem here is that the
android.jar supplied with the SDK is
stubbed out with no implementation
code. The solution that seems to be
expected is that you should run your
unit tests on the emulator or a real
phone.
When further doing some googling with this in mind, I found several articles, blogs and also questions here on SO regarding the issue. I'll add a few links here at the end, for those that might be looking:
android.jar In The SDK Only Has The API And Not The Implementation?
Can I use android.os.* libraries in a standalone project?
AndroidTesting
Eclipse + Android + JUnit test references android.os class = NoClassDefFoundError (not sure why on earth I didn't find this one when I initially wrote my question, I must have been completely lost...)
And there are plenty more if you look around.
There are several suggestions/solution/alternative approaches to unit testing in Android in many of those links, but I won't bother to try to make a good answer based on that (as there is obviously way too much I still don't know about Android development). If anybody has any nice tips though, I'll be glad to hear about them :)
UPDATE:
After experimenting a bit more, I actually managed to find a working solution to this specific problem. The reason why I didn't try this in the first place, was that I thought I had read somewhere that it would be problematic to include "normal" java libraries in my Android app. I was trying so many different things to get around my problem, so I thought I'd just give this a try as well - and it actually worked! Here is how:
I downloaded the source for the "real" org.json package from here: http://www.json.org/java/index.html
Next I compiled the source code and packed it together in my own json.jar
I added the newly created json.jar to the build path of my project (the main project of my Android application, not the test project)
No changes to my code, no changes to my test, only adding this library. And everything works, both my Android app and my unit tests.
I also tested stepping through the code in debug mode, and when debugging the Android app the JSONObject in my JsonCourseParser is fetched from the Android SDK (android.jar), but when debugging my unit test it is fetched from my own json.jar. Not sure if this means that the json.jar isn't included when my app is built, or if the runtime intelligently selects the correct library to use. Also not sure if this extra library might have any other effects on my final app. I guess only time will tell...

I'm using Björn Quentin's Unmock Gradle plugin, which allows you to replace stubbed classes in the default android.jar with real versions from another android.jar (e.g. Robolectric).

I had the same problem but I found a better way. The framework Roboelectric does the job. At first I just added the roboelectric.jar as external library (because I don't use Maven) to the build path of my Java JUnit test project and as "class annotation" I added #RunWith(RobolectricTestRunner.class). But then I got a NoClassDefFoundError: android/net/Uri. After I still added the android.jar itself used by the corresponding Android project (although the Android library with its android.jar was already a part of my build path), it works.

I had this exact problem in my JSON tests. Your updated answer led me to try a simpler approach, which worked for my project. It allows me to run JUnit tests, using the 'real' org.json JAR, on my non-Android classes from within Eclipse:
Download org.json's json.jar from here (or wherever): http://mvnrepository.com/artifact/org.json/json
Add a folder 'libs-test' to your project, copy the json.jar file to it
In Eclipse open: Run / Run Configurations, select JUnit / YourTestClass
On the Classpath tab, remove "Google APIs" from Bootstrap entries
Click "Add JARs", navigate to /libs-test/json.jar and add it.
Click "Close"
Run your tests