In my office, the mere mention of the word Xerces is enough to incite murderous rage from developers. A cursory glance at the other Xerces questions on SO seem to indicate that almost all Maven users are "touched" by this problem at some point. Unfortunately, understanding the problem requires a bit of knowledge about the history of Xerces...
History
Xerces is the most widely used XML parser in the Java ecosystem. Almost every library or framework written in Java uses Xerces in some capacity (transitively, if not directly).
The Xerces jars included in the official binaries are, to this day, not versioned. For example, the Xerces 2.11.0 implementation jar is named xercesImpl.jar and not xercesImpl-2.11.0.jar.
The Xerces team does not use Maven, which means they do not
upload an official release to Maven Central.
Xerces used to be released as a single jar (xerces.jar), but was split into two jars, one containing the API (xml-apis.jar) and one containing the implementations of those APIs (xercesImpl.jar). Many older Maven POMs still declare a dependency on xerces.jar. At some point in the past, Xerces was also released as xmlParserAPIs.jar, which some older POMs also depend on.
The versions assigned to the xml-apis and xercesImpl jars by those who deploy their jars to Maven repositories are often different. For example, xml-apis might be given version 1.3.03 and xercesImpl might be given version 2.8.0, even though both are from Xerces 2.8.0. This is because people often tag the xml-apis jar with the version of the specifications that it implements. There is a very nice, but incomplete breakdown of this here.
To complicate matters, Xerces is the XML parser used in the reference implementation of the Java API for XML Processing (JAXP), included in the JRE. The implementation classes are repackaged under the com.sun.* namespace, which makes it dangerous to access them directly, as they may not be available in some JREs. However, not all of the Xerces functionality is exposed via the java.* and javax.* APIs; for example, there is no API that exposes Xerces serialization.
Adding to the confusing mess, almost all servlet containers (JBoss, Jetty, Glassfish, Tomcat, etc.), ship with Xerces in one or more of their /lib folders.
Problems
Conflict Resolution
For some -- or perhaps all -- of the reasons above, many
organizations publish and consume custom builds of Xerces in their
POMs. This is not really a problem if you have a small application and are only using Maven Central, but it quickly becomes an issue for enterprise software where Artifactory or Nexus is proxying multiple repositories (JBoss, Hibernate, etc.):
For example, organization A might publish xml-apis as:
<groupId>org.apache.xerces</groupId>
<artifactId>xml-apis</artifactId>
<version>2.9.1</version>
Meanwhile, organization B might publish the same jar as:
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>1.3.04</version>
Although B's jar is a lower version than A's jar, Maven does not know
that they are the same artifact because they have different
groupIds. Thus, it cannot perform conflict resolution and both
jars will be included as resolved dependencies:
Classloader Hell
As mentioned above, the JRE ships with Xerces in the JAXP RI. While it would be nice to mark all Xerces Maven dependencies as <exclusion>s or as <provided>, the third-party code you depend on may or may not work with the version provided in JAXP of the JDK you're using. In addition, you have the Xerces jars shipped in your servlet container to contend with. This leaves you with a number of choices: Do you delete the servlet version and hope that your container runs on the JAXP version? Is it better to leave the servlet version, and hope that your application frameworks run on the servlet version? If one or two of the unresolved conflicts outlined above manage to slip into your product (easy to happen in a large organization), you quickly find yourself in classloader hell, wondering which version of Xerces the classloader is picking at runtime and whether or not it will pick the same jar in Windows and Linux (probably not).
Solutions?
We've tried marking all Xerces Maven dependencies as <provided> or as an <exclusion>, but this is difficult to enforce (especially with a large team) given that the artifacts have so many aliases (xml-apis, xerces, xercesImpl, xmlParserAPIs, etc.). Additionally, our third party libs/frameworks may not run on the JAXP version or the version provided by a servlet container.
How can we best address this problem with Maven? Do we have to exercise such fine-grained control over our dependencies, and then rely on tiered classloading? Is there some way to globally exclude all Xerces dependencies, and force all of our frameworks/libs to use the JAXP version?
UPDATE: Joshua Spiewak has uploaded a patched version of the Xerces build scripts to XERCESJ-1454 that allows for upload to Maven Central. Vote/watch/contribute to this issue and let's fix this problem once and for all.
There are 2.11.0 JARs (and source JARs!) of Xerces in Maven Central since 20th February 2013! See Xerces in Maven Central. I wonder why they haven't resolved https://issues.apache.org/jira/browse/XERCESJ-1454...
I've used:
<dependency>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
<version>2.11.0</version>
</dependency>
and all dependencies have resolved fine - even proper xml-apis-1.4.01!
And what's most important (and what wasn't obvious in the past) - the JAR in Maven Central is the same JAR as in the official Xerces-J-bin.2.11.0.zip distribution.
I couldn't however find xml-schema-1.1-beta version - it can't be a Maven classifier-ed version because of additional dependencies.
Frankly, pretty much everything that we've encountered works just fine w/ the JAXP version, so we always exclude xml-apis and xercesImpl.
You could use the maven enforcer plugin with the banned dependency rule. This would allow you to ban all the aliases that you don't want and allow only the one you do want. These rules will fail the maven build of your project when violated. Furthermore, if this rule applies to all projects in an enterprise you could put the plugin configuration in a corporate parent pom.
see:
http://maven.apache.org/plugins/maven-enforcer-plugin/
http://maven.apache.org/enforcer/enforcer-rules/bannedDependencies.html
I know this doesn't answer the question exactly, but for ppl coming in from google that happen to use Gradle for their dependency management:
I managed to get rid of all xerces/Java8 issues with Gradle like this:
configurations {
all*.exclude group: 'xml-apis'
all*.exclude group: 'xerces'
}
I guess there is one question you need to answer:
Does there exist a xerces*.jar that everything in your application can live with?
If not you are basically screwed and would have to use something like OSGI, which allows you to have different versions of a library loaded at the same time. Be warned that it basically replaces jar version issues with classloader issues ...
If there exists such a version you could make your repository return that version for all kinds of dependencies. It's an ugly hack and would end up with the same xerces implementation in your classpath multiple times but better than having multiple different versions of xerces.
You could exclude every dependency to xerces and add one to the version you want to use.
I wonder if you can write some kind of version resolution strategy as a plugin for maven. This would probably the nicest solution but if at all feasible needs some research and coding.
For the version contained in your runtime environment, you'll have to make sure it either gets removed from the application classpath or the application jars get considered first for classloading before the lib folder of the server get considered.
So to wrap it up: It's a mess and that won't change.
You should debug first, to help identify your level of XML hell. In my opinion, the first step is to add
-Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl
-Djavax.xml.transform.TransformerFactory=com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl
to the command line. If that works, then start excluding libraries. If not, then add
-Djaxp.debug=1
to the command-line.
There is another option that hasn't been explored here: declaring Xerces dependencies in Maven as optional:
<dependency>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
<version>...</version>
<optional>true</optional>
</dependency>
Basically what this does is to force all dependents to declare their version of Xerces or their project won't compile. If they want to override this dependency, they are welcome to do so, but then they will own the potential problem.
This creates a strong incentive for downstream projects to:
Make an active decision. Do they go with the same version of Xerces or use something else?
Actually test their parsing (e.g. through unit testing) and classloading as well as not to clutter up their classpath.
Not all developers keep track of newly introduced dependencies (e.g. with mvn dependency:tree). This approach will immediately bring the matter to their attention.
It works quite well at our organization. Before its introduction, we used to live in the same hell the OP is describing.
Every maven project should stop depending on xerces, they probably don't really. XML APIs and an Impl has been part of Java since 1.4. There is no need to depend on xerces or XML APIs, its like saying you depend on Java or Swing. This is implicit.
If I was boss of a maven repo I'd write a script to recursively remove xerces dependencies and write a read me that says this repo requires Java 1.4.
Anything that actually breaks because it references Xerces directly via org.apache imports needs a code fix to bring it up to Java 1.4 level (and has done since 2002) or solution at JVM level via endorsed libs, not in maven.
What would help, except for excluding, is modular dependencies.
With one flat classloading (standalone app), or semi-hierarchical (JBoss AS/EAP 5.x) this was a problem.
But with modular frameworks like OSGi and JBoss Modules, this is not so much pain anymore. The libraries may use whichever library they want, independently.
Of course, it's still most recommendable to stick with just a single implementation and version, but if there's no other way (using extra features from more libs), then modularizing might save you.
A good example of JBoss Modules in action is, naturally, JBoss AS 7 / EAP 6 / WildFly 8, for which it was primarily developed.
Example module definition:
<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.1" name="org.jboss.msc">
<main-class name="org.jboss.msc.Version"/>
<properties>
<property name="my.property" value="foo"/>
</properties>
<resources>
<resource-root path="jboss-msc-1.0.1.GA.jar"/>
</resources>
<dependencies>
<module name="javax.api"/>
<module name="org.jboss.logging"/>
<module name="org.jboss.modules"/>
<!-- Optional deps -->
<module name="javax.inject.api" optional="true"/>
<module name="org.jboss.threads" optional="true"/>
</dependencies>
</module>
In comparison with OSGi, JBoss Modules is simpler and faster. While missing certain features, it's sufficient for most projects which are (mostly) under control of one vendor, and allow stunning fast boot (due to paralelized dependencies resolving).
Note that there's a modularization effort underway for Java 8, but AFAIK that's primarily to modularize the JRE itself, not sure whether it will be applicable to apps.
Apparently xerces:xml-apis:1.4.01 is no longer in maven central, which is however what xerces:xercesImpl:2.11.0 references.
This works for me:
<dependency>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
<version>2.11.0</version>
<exclusions>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xml-apis</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>1.4.01</version>
</dependency>
My friend that's very simple, here an example:
<dependency>
<groupId>xalan</groupId>
<artifactId>xalan</artifactId>
<version>2.7.2</version>
<scope>${my-scope}</scope>
<exclusions>
<exclusion>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
</exclusion>
</dependency>
And if you want to check in the terminal(windows console for this example) that your maven tree has no problems:
mvn dependency:tree -Dverbose | grep --color=always '(.* conflict\|^' | less -r
Related
I understand that PROVIDED dependencies are "provided" by container and application don't need generate this JAR.
1) So, I'm using JBOSS EAP 7.0.0.GA and this have the following jar in this module folder: hibernate-core-5.0.9.Final-redhat-1.jar.
In my project i'm using the following dependency:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.0.9.Final-redhat-1</version>
</dependency>
It works fine, without errors. But i understand that i should use "PROVIDED" scope because this jar is provided by container. Why it works ?
2) I have another example. In Jboss Eap 7.0.0.GA i have the following jar: jboss-servlet-api_3.1_spec-1.0.0.Final-redhat-1.jar. But in my project i have the following:
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.0.1</version>
<scope>provided</scope>
<optional>true</optional>
</dependency>
It works fine too, but i don't understand why. To me the correct dependency should be servlet-api_3.1_spec-1.0.0.Final-redhat-1 with provided. Why it works too ?
At build time, Maven will resolve the dependencies and make the relevant packages available to the compiler. Your appserver may provide, for example, JARs that contain the classes and interfaces in javax.servlet, but these classes won't necessarily be useful to the compiler, because it doesn't know where they are. By providing the dependencies to Maven, you're having Maven find its own implementations of these dependencies, just for use at compile time.
At run time, if you've marked the dependencies as provided, then your application will use the versions provided by the appserver, rather than the versions known to Maven. This is potentially a bad thing, but it often works because the compiler really needs to know only the method signatures, not their implementations. The method signatures of classes that are controlled by specifications, such as those in javax.servlet, change only infrequently, so the mismatch between the compile-time JARs and run-time JARs may go unnoticed. Unlike OSGi-compliant JARs, JARs built for JEE do not contain meta-data that specifies particular compatible dependency versions -- JEE classloaders will use what they find, for better or worse.
You can be caught out, however -- particularly if the mismatch is substantial. Problems may be very obvious at runtime, such as exceptions related to missing classes or methods, but they can be subtle.
It's therefore often best, where practicable, to use the same compile-time versions of dependencies as the versions that will be available at runtime. For EAP, I recall that Red Hat distributes a Maven bill-of-materials (BOM) file that specifies all the versions of all the EAP JARs for specific EAP releases.
On various projects I've been working on, I've seen diferent ways of specifying dependencies versions. On some projects, the package version is written on the same dependency declaration:
<dependency>
<groupId>org.apache.myfaces.extensions.validator.validation-modules</groupId>
<artifactId>myfaces-extval-property-validation</artifactId>
<version>2.0.7</version>
<scope>compile</scope>
</dependency>
On others, a property is used, as in:
<dependency>
<groupId>org.apache.myfaces.extensions.validator.validation-modules</groupId>
<artifactId>myfaces-extval-property-validation</artifactId>
<version>${versions.extval}</version>
<scope>compile</scope>
</dependency>
For multimodule projects, I can see a clear benefit in declaring versions on the parent pom to avoid duplication (and the potential confusion and errors that come with it), but on single module applications, would there be a benefit to use such a level of indirection?
What would be a best practice for this and why?
Thanks a lot :)
With a version property you can override it on the command line whereas with a fixed version you cannot.
So you can recompile your project with a newer version just by specifying it on the command line.
mvn -Dversions.extval=2.0.8 clean package
Or something.
apart from that, mostly used on multi-projects, and although there you have the dependency management section as well for versions.
I use the maven-enforcer-plugin to check for dependency convergence issues. A typical output would be:
[WARNING] Rule 1: org.apache.maven.plugins.enforcer.DependencyConvergence failed
with message:
Failed while enforcing releasability the error(s) are [
Dependency convergence error for junit:junit:3.8.1 paths to dependency are:
+-foo:bar:1.0-SNAPSHOT
+-ca.juliusdavies:not-yet-commons-ssl:0.3.9
+-commons-httpclient:commons-httpclient:3.0
+-junit:junit:3.8.1
and
+-foo:bar:1.0-SNAPSHOT
+-junit:junit:4.11
]
Seeing this message, I would normally "solve" it by excluding the transitive dependency, e.g.
<dependency>
<groupId>ca.juliusdavies</groupId>
<artifactId>not-yet-commons-ssl</artifactId>
<version>0.3.9</version>
<exclusions>
<!-- This artifact links to another artifact which stupidly includes
junit in compile scope -->
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
I'd like to understand whether this is truly a fix and the risks involved in excluding libraries in this fashion. As I see it:
The "fix" is normally safe, provided I'm choosing to use the newer version. This relies on the library authors maintaining backwards compatibility.
There is typically no impact on the Maven build (since the nearer definition wins), however by excluding the dependency I'm telling Maven that I know about this problem and thus appeasing the maven-enforcer-plugin.
Are my thoughts correct and is there an alternative way of handling this issue? I'm interested in answers that focus on the general case - I realise the junit example above is a little strange.
We all agree that JUnit should never be set to another scope than test. Generally speaking I don't think either that there is another solution than excluding the unwanted dependency, so we all agree that your are right to do it.
A SIMPLE CASE :
As Andreas Krueger says, there may be a risk with versions (I actually encountered it). Let say that the project's dependencies are the following:
+-foo:bar:1.0-SNAPSHOT
+-group1:projectA:2.0
+-group2:projectB:3.8.1
+-group2:projectB:4.11
Note that it is only a mere simplification of your case. Seeing this dependency tree, you would exclude the dependency projectB given by projectA :
<dependency>
<groupId>group1</groupId>
<artifactId>projectA</artifactId>
<version>2.0</version>
<exclusions>
<exclusion>
<groupId>group2</groupId>
<artifactId>projectB</artifactId>
</exclusion>
</exclusions>
</dependency>
After packaging the project with maven, the remaining dependency would be group2-someProjectB-4.11.jar, version 4.11 and not 3.8.1. Everything would be fine and the project would run without encountering any problem at all.
Then, a while after, let say that you decide to upgrade to the next version of project A, version 3.0 which adds new great features :
<dependency>
<groupId>group1</groupId>
<artifactId>projectA</artifactId>
<version>3.0</version>
<exclusions>
<exclusion>
<groupId>group2</groupId>
<artifactId>projectB</artifactId>
</exclusion>
</exclusions>
</dependency>
The problem is that you are not aware yet that projectA version 3.0 also have upgraded its dependency projectB to version 5.0 :
+-foo:bar:1.0-SNAPSHOT
+-group1:projectA:3.0
+-group2:projectB:5.0
+-group2:projectB:4.11
In that case, the exclusion you would have made a while ago excludes projectB version 5.0.
However, projectA version 3.0 needs the improvements from project B version 5.0. Because of the exclusion, after packaging the project with maven, the remaining dependency would be group2-someProjectB-4.11.jar, version 4.11 and not 5.0. At the moment you use any of projectA's new features, the program wouldn't run correctly.
WHAT WAS THE SOLUTION ?
I encountered this problem in a Java-EE project.
A team developped database services. They packaged it as projectA. Each time they updated the services, they also updated a file listing all their current dependencies and the current versions.
ProjectA was a dependency for the Java-EE project I was working on. Each time the service-team updated ProjectA, I also checked the versions' updates.
In fact, there is no harm in excluding a dependency. But each time you update a dependency where an exclusion has been set, You have to check :
if this exclusion still makes sense.
if you need to upgrade the version of the excluded dependency in your own project.
I guess maven exclusions are like kitchen knifes. It's sharp, cuts vegetables with no effort, but requires care when handling it...
If JUnit as an artifact is coming through as a dependency in compile scope, it is a bug of one of your libraries, here: ca.juliusdavies.
JUnit should always be included in test scope. Thus, it is not packed into the produced .jar, .war or .ear file, on successful build.
Generally speaking, there is no harm in excluding already included dependencies, as when library 1 and library 2 share one common dependency.
The only problem, of course, that can occur, is when library 1 and library 2 include different versions of the same dependent artifact. This can cause run-time errors, when the features of the library have changed.
Fortunately, this is not often the case, unless the difference in the version numbers is great. In general, it is advisable to include the latest dependency version and exlude the older one. This is most of the time viable.
If not, check wheter there are updates to the first-level dependencies of your project.
Assume you have set of web apps that use various versions of a common library like Spring. I have a business logic library that also uses this common library. So far no problem, but along the way a version of the common library changed an abstract class definition and broke the business logic library.
So I end up with a compatible version table that looks like this...
business-lib-version | common-lib-version
1.0 | 1.0
1.1 | 2.0
I don't want the business lib version to drive the common lib version in the consuming application. Rather I would like to pick the correct version of the business lib based on the common lib. I'm pretty sure that's not possible so I move on to the main question.
Is there an elegant way to detect version incompatibilities? Ideally I would like a build time solution, otherwise an early run-time solution would be ok.
I've tried using version ranges in Maven but this has caused us many problems due to how Maven sorts versions in non-standard version formats and we also had various issues in resolving ranges correctly at build time.
The Ning dependency-versions-check Maven plugin will fail your build if a dependency version has accidentally been held back by another dependency. It won't fix it for you, but it'll at least tell you!
Is there an elegant way to detect version incompatibilities? Ideally I would like a build time solution, otherwise an early run-time solution would be ok.
I don't know of any way to do this well -- either at runtime or even at build time. There is no standard mechanism for determining the version of packages without investigating the manifests or the jar file names -- which would be a hack at best. Maybe there is a maven plugin that I don't know about that does this automagically but I don't know of one.
We just fix the versions of what libraries we need for our code and upgrade when we need to either because we need additional functionality or another dependency does. Typically we are ahead of other dependencies so we put a typical exclusion marker in our dependency definitions in the pom.xml:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>${spring-version}</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
This allows us to depend on a later version of commons-logging.
If, however, you are using 1.0 of a library but one of your dependencies is using 2.0 then you will have to try the exclusion and see if the dependency runs with 1.0 -- or even compiles. If not then you will be forced to either upgrade your code to work with 2.0 or downgrade the dependency.
Sorry that I couldn't be more help.
I have lately become a big fan of Maven for controlling the build cycle for my application. However I've encountered some rough edges with Maven's dependency management. I'm wondering if these are limitations of the tool and paradigm, necessary evils of dependancy management, or if I"m just using the tool incorrectly.
First is the matter of transitive dependencies. As I understand it, if you provide a dependency, Maven will in turn find any dependencies of that dependency. That is great, but for many of my dependencies, this has not worked. For example, including Hibernate in my project:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>3.3.2.GA</version>
</dependency>
Results in a missing dependency of slf4j. I need to manually add this dependency which I assumed would be Maven's job. The same goes for Spring. If I add Spring-MVC as a dependency, shouldn't all of the basic servlet dependencies be added for me (because Spring-MVC would need this stuff)? I'm referring to the servlet, jsp, jstl libraries.
Second is the management of repositories. Maven comes shipped with a default main repository, but I've found that in many cases this repository is not up to date. For example, ifyou want spring3, you have to manually add the springsource repository, and if you want hibernate 3.5+ you have to add the jboss repository. It seems to defeat the point of automatic dependency management when you have to hunt down the correct repositories yourself. This hunting soon gets complicated. For example to add Spring3, you may want the spring release repo, the spring externals repo and the spring milestone repo.
Closely related to number 2 is ensuring you have the correct version of an artifact. I have been burned several times by including the wrong versions of dependent artifacts for a given artifact. For example the wrong version of the servlet/jsp/jstl apis for spring3, or the wrong version of persistence / annotation apis for hibernate. The repositories are filled with many versions, some with confusing names like productx-3.ga, productx-3-rc1, productx-3-SNAPSHOT, productx-3-cr, product-3-beta, etc. Some of these are obvious (rc= release candidate), but it can be confusing trying to determine the order of these versions.
Finally, the issue of the type a dependency. I probably just don't understand this well enough, but many repo artifacts are of type "pom" not "jar". Several times i have added a dependency jar to my project only to find out at build time that the repo jar does not actually exist (example is org.hibernate ejb3-persistence in the jboss repo).
With some experimenting, I can usually get a build to work, but is dependency management in general this complicated? I still prefer this approach to manually adding jar files to my project, but I would be interested to learn how to improve my maven dependency management skills.
Can't answer all parts of the question, but about some of them:
Some transitive dependecies are marked optional, so people who don't need these features wouldn't download them, but people, who need, have to set them explicitly in their poms.
Maven Central repository contains only releases. Therefore, it doesn't contain Hibernate 3.5 (which is beta) as well as it hadn't contained Spring 3 until it was released (by the way, you don't need to specify special Spring repository for Spring 3 any more - release is already in Maven Central)
slf4j is a very special kind of dependency - its runtime behavior depends on which implementation of slf4j you use. Therefore, to control its behavior you have to specify slf4j implementation explicitly
About management skills: to get useful information for maintaining your pom.xml you can use mvn dependency:tree (especially with -Dverbose=true) and mvn dependency:analyze. It can be also useful to check pom file of your dependency to look for optional dependencies.
First is the matter of transitive dependencies. As I understand it, if you provide a dependency, Maven will in turn find any dependencies of that dependency. That is great, but for many of my dependencies, this has not worked. (...)
As already pointed out, some dependencies may be marked as optional (and are not pulled transitively). The idea is that some dependencies are only used for certain features and will not be needed if that feature isn't used. If a user wants to use functionality related to an optional dependency, they will have to redeclare that optional dependency in their own project. So this just works as designed :)
Second is the management of repositories. Maven comes shipped with a default main repository, but I've found that in many cases this repository is not up to date. (...)
Even if the idea behind the concept of a central repo is noble, you can't objectively expect it to contain all jars in the world. One of the most obvious reason is that uploading artifacts to the Central Repository just takes time and resources are not infinite. And because companies like RedHat JBoss or SpringSource or Sun or even me need flexibility, reactivity (in one word, control), it's not surprising that they use their own repository. And, actually, I'm pretty happy that they expose them. But indeed, projects need to document where to find their artifacts if they are not available in central. Just in case, you may find this How to find dependencies on public Maven repositories? helpful. In a corporate environment, the best way to handle this would be to setup a centralized (corporate) proxying repository. See this page for such solutions.
Closely related to number 2 is ensuring you have the correct version of an artifact. (...)
Sorry but you need a bit to know what you're are doing. A project can't guess for you what JSTL version you are going to use. Then, regarding the various versions of artifacts, the naming convention used by projects has nothing to do with maven, this is a project/vendor choice (except for SNAPSHOT that maven handles specially). FWIW, common used schemes include: M1 = Milestone 1, RC1 = Release Candidate 1, GA = General Availability (final release), CR = Customer Release (often a bug-fix release). You may also see alpha, beta. This really depends on the project lifecycle and convention (nothing really unusual here though).
Finally, the issue of the type a dependency. I probably just don't understand this well enough, but many repo artifacts are of type "pom" not "jar". (...)
I think that you're indeed lacking of experience. You seem to be fighting with dependencies while things just go smoothly for me :) Maybe using a repository search engine will help.
You can also exclude some transitive dependencies, like the following example:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.15</version>
<exclusions>
<exclusion>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
</exclusion>
<exclusion>
<groupId>javax.jms</groupId>
<artifactId>jms</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jdmk</groupId>
<artifactId>jmxtools</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jmx</groupId>
<artifactId>jmxri</artifactId>
</exclusion>
</exclusions>
</dependency>
I don't remember the details, but apparently some other dependencies were being discovered when we wanted to use log4j, and we had no interest (or need) in them in our case, so my cow orker just said "no thanks to these" and it works.
Transitive dependencies are only there if they have been encoded explicitly in your direct dependencies' POM's. Sometimes you actually do not want them, either because there are alternative implementations or you're using only part of a library and you don't want to drag in dependencies for things you're actually not using.
You are using a repository manager such as Nexus, aren't you? I advise you to setup one even if you code alone.
I find that Maven actually helps with this problem. I would hate having to perform this trial-and-error from ftp sites and download pages.
Yes, I hate that too :-)
Read the book! I actually learned most of what I know from this other book, but given it comes from the same people, I expect them to be rather interchangeable.