We Keep Coding

How to deal with two versions of an API

My Question: What is a good way to deal with two different versions of an API? (Sub-question: Is there any way to avoid classpath reference problems if you include two libraries with the same class-names?)
Description of question: I have a project that uses an API. I have spent the last few months developing for the old version and I'm about to add features for a newer version. So far as I can tell there's only one critical difference. However, I don't have the API yet (waiting on someone to get me the jar) so I can't be sure whether there are more differences.
Description of subquestion: I'm worried that there may be class reference inconsistencies between the two APIs (like I said, I don't have the jar yet to be sure).
I realize you may want some code to look at, but this is a design question, not a coding issue. I'm hoping to get some best practices out of this. Thanks!
Not a duplicate: I've looked at a few questions which may appear to be duplicates, but they didn't really address my issue :)

Avoid, or shade.
If you have full control of the classpath you can use ordering: this is brittle and non-intuitive.
If you don't, you're at the mercy of whatever is, hence shading, and its related headaches.

There are also classloader isolation strategies you can use. For example, if you're in a container environment (Java EE, OSGi) you can put the different versions of the libraries in different classloader contexts (i.e., different EJB jars, or different web applications) and they won't interfere with each other.
OSGi can do the same sort of thing; deploy library A.1 and A.2 into your OSGi container, and deploy project.uno (which uses A.1) and project.dos (which uses A.2) into the same JVM, and the OSGi container handles resolution of the classpaths.

Should I be concerned with large number of dependencies?

I was just about to include the HtmlUnit library in a project. I unpacked the zip-file and realised that it had no less than 12 dependencies.
I've always been concerned when it comes to introducing dependencies. I suppose I have to ship all these dependencies together with the application (8.7 mb in this particular case). Should I bother checking for, say, security updates for these libraries? Finally (and most importantly, actually what I'm most concerned about): What if I want to include another library which depends on the same libraries as this library, but with different versions? That is, what if for instance HtmlUnit depends on one version of xalan and another library I need, depends on a different version of xalan?
The task HtmlUnit solves for me could be solved "manually" but that would probably not be as elegant.
Should I be concerned about this? What are the best practices in situations like these?
Edit: I'm interested in the general situation, not particularly involving HtmlUnit. I just used it here as an example as that was my current concern.

Handle your dependencies with care. They can bring you much speed, but can be a pain to maintain down the road. Here are my thoughts:
Use some software to maintain your dependencies. Maven is what I would use for Java to do this. Without it you will very soon loose track of your dependencies.
Remember that the various libraries have different licenses. It is not granted that a given license works for your setting. I work for a software house and we cannot use GPL based libraries in any of the software we ship, as the software we sell are closed source. Similarly we should avoid LGPL as well if we can (This is due to some intricate lawyer reasoning, don't ask me why)
For unit testing I'd say go all out. It is not the end of the world if you have to rewrite your tests in the future. It might even be then that that part of the software is either extremely stable or maybe not even maintained no more. Loosing those is not that big of a deal as you already had a huge gain of gaining speed when you got it.
Some libraries are harder to replace later than others. Some are like a marriage that should last the life of the software, but some other are just tools that are easily replaceable. (Think Spring versus an xml library)
Check out how the community support older versions of the library. Are they supporting older versions? What happens when life continues and you are stuck at a version? Is there an active community or do you have the skill to maintain it yourself?
For how long are your software supposed to last? Is it one year, five year, ten year or beyond? If the software has short time span, then you can use more to get where you are going as it is not that important to be able to keep up with upgrading your libraries.

It could be a serious issue if there isn't a active community which does maintain the libraries on long term. It is ok to use libraries, but to be honest you should care to get the sources and put them into your VCS.

Should I bother checking for, say, security updates for these libraries?
In general, it is probably a good idea to do this. But then so should everyone upstream and downstream of you.
In your particular case, we are talking about test code. If potential security flaws in libraries used only in testing are significant, your downstream users are doing something strange ...
Finally (and most importantly, actually what I'm most concerned about): What if I want to include another library which depends on the same libraries as this library, but with different versions? That is, what if for instance HtmlUnit depends on one version of xalan and another library I need, depends on a different version of xalan?
Ah yes. Assuming that you are building your own classpaths, etc by hand, you need to make a decision about which version of the dependent libraries you should use. It is usually safe to just pick the most recent of the versions used. But if the older version is not backwards incompatible with the new (for your use case) then you've got a problem.
Should I be concerned about this?
IMO, for your particular example (where we are talking about test code), no.
What are the best practices in situations like these?
Use Maven! It explicitly exposes the dependencies to the folks who download your code, making it possible for them to deal with the issue. It also tells you when you've got dependency version conflicts and provides a simple "exclude" mechanism for dealing with it.
Maven also removes the need to create distributions. You publish just your artifacts with references to their dependents. The Maven command then downloads the dependent artifacts from wherever they have been published.
EDIT
Obviously, if you are using HtmlUnit for production code (rather than just tests), then you need to pay more attention to security issues.

A similar thing has happened to me actually.
Two of my dependencies had the same 'transitive' dependency but a different version.
My favorite solution is to avoid "dependency creep" by not including too many dependencies. So, the simplest solution would be to remove the one I need less, or the one I could replace with a simple Util class, etc.
Too bad, it's not always that simple. In unfortunate cases where you actually need both libraries, it is possible to try to sync their versions, i.e. downgrade one of them so that dependency versions match.
In my particular case, I manually edited one of the jars, deleted the older dependency from it, and hoped it would still work with new version loaded from other jar. Luckily, it did (i.e. maintainers of the transitive dependency didn't remove any classes or methods that library used).
Was it ugly - Yes (Yuck!), but it worked.

I try to avoid introducing frivolous dependencies, because it is possible to run into conflicts.
One interesting technique I have seen used to avoid conflicts: renaming a library's package (if its license allows you to -- most BSD-style licenses do.) My favorite example of this is what Sun did when they built Xerces into the JRE as the de-facto JAXP XML parser: they renamed the whole of Xerces from org.apache.xerces to com.sun.org.apache.xerces.internal. Is this technique drastic, time consuming, and hard to maintain? Yes. But it gets the job done, and I think it is an important possible alternative to keep in mind.
Another possibility is -- license terms abided -- copying/renaming single classes or even single methods out of a library.
HtmlUnit can do a lot, though. If you are really using a large portion of its functionality on a lot of varied input data, it would be hard to make a case for spending the large amount of time it would take to re-write the functionality from scratch, or repackage it.
As for the security concerns -- you might weigh the chances of a widely used library having problems, vs. the likelihood of your hand-written less-tested code having some security flaw. Ultimately you are responsible for the security of your programs, though -- so do what you feel you must.

Alternative to servicewrapper for java?

I've used ServiceWrapper a few times in the past, however, the dual license is somewhat complex for commercial products (generally you have to pay them). Are there are fully FOSS alternatives with similar functionality?

YAJSW seems to be the best alternative, though I have not yet completed my evaluation.

Try Apache Procrun. It's what Tomcat uses on Windows. There is also Apache jsvc for Unix; they are both part of a parent project called Apache Commons Daemon.

Which functionality do you need exactly?
In their table I didn't see anything non-trivial. But it would help if you narrow down your requirements.

We use the open source JavaService package.
Edit: We use an older version of this package (when it was being maintained by the original developer, Alexandria Software Consulting). I have not had a chance to look at the improvements made since then.

What is classpath hell and is/was it really a problem for Java?

What is classpath hell and is/was it really a problem for Java?

Classpath hell is an unfortunate consequence of dynamic linking of the kind carried out by Java.
Your program is not a fixed entity but rather the exact set of classes loaded by a JVM in a particular instance.
It is very possible to be in situations where the same command line on different platforms or even on the same one would result in completely different results because of the resolution rules.
There could be differences in standard libraries (very common). Libraries could be hidden by one another (an an older version may even be used instead of a newer one). The directory structure could mess resolution. A different version of the same class may appear in multiple libraries and the first one encountered will be used, etc. Since Java, by specification, uses a first-encountered policy, unknown ordering dependencies can lead to problems. Of course, since this is the command line and it is part of the spec, there are no real warnings.
It is very much still a problem. For example on Mac OS the horrible support from Apple means that you machine ends up with several JVMs and several JREs, and you can never easily poart things from place to place. If you have multiple libraries that were compiled against specific but different versions of other libraries, you coulld have problems, etc.
However, this problem is not inherent in Java. I remember my share of DLL hell situations while programming windows in the 90s. Any situation where you have to count on something in the file system to assemble your program rather than having a single well defined executable is a problem.
However, the benefits of this model are still great, so I'm willing to tolerate this hell. There are also steps in the right direction on the side of Sun. For example, Java6 allows you to simply specify a directory with jars rather than have to enumerate them.
BTW: Classpaths are also a problem if you are using an environment that uses a non-default classloader. For example, I have had a lot of problems running things like Hibernate or Digester under Eclipse because the classloaders were incompatible.

Classpath/jar-hell has a couple of escape hatches if they make sense for your project:
OSGi
JarJarLinks
NetBeans Module System - Not sure if this is usable outside of NetBeans
Others?

I think "classpath hell" refers to the time when the classpath of a Java app could only be set by using the CLASSPATH environment variable. This led to many applications requiring changes to the global system configuration (different for each OS), version conflicts between applications, and general confusion.

This is a somewhat more concrete example:
When two libraries (or a library
and the application) require different versions of the same third
library. If both versions of the third library use the same class
names, there is no way to load both versions of the third library with
the same classloader.
Take a loot at http://en.wikipedia.org/wiki/Java_Classloader#JAR_hell for more examples.

There's lot of good stuff here http://mindprod.com/jgloss/classpath.html and http://java.sun.com/javase/6/docs/technotes/tools/windows/classpath.html
I've only had issues with classpaths when I am not setting is myself using -cp. Trying to figure out how your third-party software sets their classpaths can be a pain at times.

OpenJDK7: What essential ADTs are not Implemented in OpenJDK7?

What Abstract Datatypes should be added to OpenJDK7 ?

If you look at these two links you will see what some people feel is missing:
http://commons.apache.org/collections/
http://code.google.com/p/google-collections/
My favourite missing one is Bag... it is easy to implement, but annoying to have to implement it.
Mulimaps would also nice to have in the standard packages.

I think they should stop adding new libraries to the JDK. You can easily get what you need from third-party, open-source libraries. Hopefully, with version 7 the JVM and its standard libraries will become more modular.