I have a Maven Java project. I don't want my project dependencies to be satisfied by chance through a chain of subdependencies when compiling the project. It is OK for me when building the final war when maven must check all used dependencies and add necessary libs to the war, but when compiling the code I want to be sure that only direct dependencies are used. Why?
Let's say I have two dependencies:
<dependency>
<groupId>com.package</groupId>
<artifactId>module-1</artifactId>
</dependency>
<dependency>
<groupId>com.package</groupId>
<artifactId>module-2</artifactId>
</dependency>
For our project module-1 and module-2 serve completely different purposes, but somewhere in the dependency tree of module-2, module-1 is used. I delete module-1 dependency, but maven continue to build my project without compilation errors, because it resolves module-1 from module-2 sub-dependencies. This change goes unnoticed.
After sometime we decide to remove module-2, because we don't need it. Strange enough but we can not any more compile classes which were using imports from module-1 and which are not connected to module-2 logic.
This is a simple case, but in big project this can make quite a dependency mess.
You can use the Maven dependency plugin goal "dependency:analyze" to give you a report of all used dependencies which are not declared on the current module (included transitively). That way Maven will still use transitive dependencies (no way around that I guess), but you can force yourself via the plugin to make sure these are also declared. It will also warn you of unnecessary dependencies. Mind, the plugin analyzes the compiled classes. At times, you may need to configure the plugin, because occasionally it may not detect that a dependency is required at compile time but not at runtime, e.g. because a constant was inlined.
If you really need to do this then you can setup exclusions in the pom.
e.g. here's an example of an exclusion in one of my poms where I don't want it to automatically get commons-logging because I'm using a different logging provider.
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${org.springframework-version}</version>
<exclusions>
<!-- Exclude Commons Logging in favor of SLF4j -->
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
You could do something like this (untested)
<dependency>
<groupId>com.package</groupId>
<artifactId>module-2</artifactId>
<exclusions>
<exclusion>
<groupId>com.package</groupId>
<artifactId>module-1</artifactId>
</exclusion>
</exclusions>
</dependency>
I wouldn't necessarily recommend this though. It makes sense in the case of my logging exclusion because I'm using slf4j instead of commons logging. I've seen other examples where this is used to exclude spring 2 if the project as a whole is using spring 3.
It's a bit difficult to tell from your example because it's so vague. In general you should keep your dependencies to a minimum. If module-2 depends on module-1 then it implies that your application won't compile or run without module-1. If in fact it can live happily without it then it's not really dependent.
As a side note it's a bit alarming that you don't have a version number against the dependencies. You'll probably find maven warns you about this. It's good practice to always include a version number. If you're dependent on a module which is currently in development then you should use the .SNAPSHOT suffix on the version to get the latest build for that version.
There seems to be no way to tell maven not to resolve dependency transitively: How to exclude all transitive dependencies of a Maven dependency. One of the reason's I think, is that the user can soon run into runtime troubles, when he finds that some of the artifacts are not being resolved at runtime or there are artifact versions problems. However, if you check the link out, you can make each of the deps 'standalone' with a wildcard exclusion pattern.
One other option is to use <optional> dependency for each of your module-X sub-dependencies. This will make sure the project compiles and non of your module-X would be resolved transitively. Like:
<dependency>
<groupId>com.package</groupId>
<artifactId>module-1</artifactId>
<optional>true</optional>
</dependency>
Still, analyzing the dependency tree might be the most safe and predictable choice.
It does sound a bit strange what you plan to do. In a way you sabotage the dependency management you want to use.
If your module-2 depends on module-1 and has a dependency to it, then any module that depends on module-2 only need to define that one.
You may be able to restrict the depth of the resolution using exclusions: Exclude all transitive dependencies of a single dependency
Newer versions of maven allow wildcards in those.
But: you will need to re-add the ones you actually need, this is by repeating the dependencies you have an other modules. This duplicates the work.
If there are artifacts that cause weirdness it may be possible to define a scope: http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html so it is not propagated to dependant modules as well.
Related
I have a large Maven project with many modules and many pom.xml files. The project has changed and I suspect the pom's contain some unnecessary dependencies. Is there is a command which removes any unused dependencies from a pom?
The Maven Dependency Plugin will help, especially the dependency:analyze goal:
dependency:analyze analyzes the dependencies of this project and determines which are: used and declared; used and undeclared; unused and declared.
Another thing that might help to do some cleanup is the Dependency Convergence report from the Maven Project Info Reports Plugin.
You can use dependency:analyze -DignoreNonCompile
This will print a list of used undeclared and unused declared dependencies (while ignoring runtime/provided/test/system scopes for unused dependency analysis.)
## Be careful while using this, some libraries used at runtime are considered unused
For more details refer to this link
As others have said, you can use the dependency:analyze goal to find which dependencies are used and declared, used and undeclared, or unused and declared. You may also find dependency:analyze-dep-mgt useful to look for mismatches in your dependencyManagement section.
You can simply remove unwanted direct dependencies from your POM, but if they are introduced by third-party jars, you can use the <exclusions> tags in a dependency to exclude the third-party jars (see the section titled Dependency Exclusions for details and some discussion). Here is an example excluding commons-logging from the Spring dependency:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>2.5.5</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
Have you looked at the Maven Dependency Plugin ? That won't remove stuff for you but has tools to allow you to do the analysis yourself. I'm thinking particularly of
mvn dependency:tree
I had similar kind of problem and decided to write a script that removes dependencies for me. Using that I got over half of the dependencies away rather easily.
http://samulisiivonen.blogspot.com/2012/01/cleanin-up-maven-dependencies.html
You can use DepClean https://github.com/castor-software/depclean/
DepClean is a tool to automatically remove dependencies that are included in your Java dependency tree but are not actually used in the project's code.
You can use dependency_cleaner https://github.com/junaidbs/dependency_cleaner
This jar will help to identify and remove unwanted dependency from pom.
It will automate the process of Removing a dependency and run then check whether the dependency needful
If you are using eclipse, right-click on the jar in Maven Dependencies:
Select Maven -> Exclude Maven Artifact...
I use the maven-enforcer-plugin to check for dependency convergence issues. A typical output would be:
[WARNING] Rule 1: org.apache.maven.plugins.enforcer.DependencyConvergence failed
with message:
Failed while enforcing releasability the error(s) are [
Dependency convergence error for junit:junit:3.8.1 paths to dependency are:
+-foo:bar:1.0-SNAPSHOT
+-ca.juliusdavies:not-yet-commons-ssl:0.3.9
+-commons-httpclient:commons-httpclient:3.0
+-junit:junit:3.8.1
and
+-foo:bar:1.0-SNAPSHOT
+-junit:junit:4.11
]
Seeing this message, I would normally "solve" it by excluding the transitive dependency, e.g.
<dependency>
<groupId>ca.juliusdavies</groupId>
<artifactId>not-yet-commons-ssl</artifactId>
<version>0.3.9</version>
<exclusions>
<!-- This artifact links to another artifact which stupidly includes
junit in compile scope -->
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
I'd like to understand whether this is truly a fix and the risks involved in excluding libraries in this fashion. As I see it:
The "fix" is normally safe, provided I'm choosing to use the newer version. This relies on the library authors maintaining backwards compatibility.
There is typically no impact on the Maven build (since the nearer definition wins), however by excluding the dependency I'm telling Maven that I know about this problem and thus appeasing the maven-enforcer-plugin.
Are my thoughts correct and is there an alternative way of handling this issue? I'm interested in answers that focus on the general case - I realise the junit example above is a little strange.
We all agree that JUnit should never be set to another scope than test. Generally speaking I don't think either that there is another solution than excluding the unwanted dependency, so we all agree that your are right to do it.
A SIMPLE CASE :
As Andreas Krueger says, there may be a risk with versions (I actually encountered it). Let say that the project's dependencies are the following:
+-foo:bar:1.0-SNAPSHOT
+-group1:projectA:2.0
+-group2:projectB:3.8.1
+-group2:projectB:4.11
Note that it is only a mere simplification of your case. Seeing this dependency tree, you would exclude the dependency projectB given by projectA :
<dependency>
<groupId>group1</groupId>
<artifactId>projectA</artifactId>
<version>2.0</version>
<exclusions>
<exclusion>
<groupId>group2</groupId>
<artifactId>projectB</artifactId>
</exclusion>
</exclusions>
</dependency>
After packaging the project with maven, the remaining dependency would be group2-someProjectB-4.11.jar, version 4.11 and not 3.8.1. Everything would be fine and the project would run without encountering any problem at all.
Then, a while after, let say that you decide to upgrade to the next version of project A, version 3.0 which adds new great features :
<dependency>
<groupId>group1</groupId>
<artifactId>projectA</artifactId>
<version>3.0</version>
<exclusions>
<exclusion>
<groupId>group2</groupId>
<artifactId>projectB</artifactId>
</exclusion>
</exclusions>
</dependency>
The problem is that you are not aware yet that projectA version 3.0 also have upgraded its dependency projectB to version 5.0 :
+-foo:bar:1.0-SNAPSHOT
+-group1:projectA:3.0
+-group2:projectB:5.0
+-group2:projectB:4.11
In that case, the exclusion you would have made a while ago excludes projectB version 5.0.
However, projectA version 3.0 needs the improvements from project B version 5.0. Because of the exclusion, after packaging the project with maven, the remaining dependency would be group2-someProjectB-4.11.jar, version 4.11 and not 5.0. At the moment you use any of projectA's new features, the program wouldn't run correctly.
WHAT WAS THE SOLUTION ?
I encountered this problem in a Java-EE project.
A team developped database services. They packaged it as projectA. Each time they updated the services, they also updated a file listing all their current dependencies and the current versions.
ProjectA was a dependency for the Java-EE project I was working on. Each time the service-team updated ProjectA, I also checked the versions' updates.
In fact, there is no harm in excluding a dependency. But each time you update a dependency where an exclusion has been set, You have to check :
if this exclusion still makes sense.
if you need to upgrade the version of the excluded dependency in your own project.
I guess maven exclusions are like kitchen knifes. It's sharp, cuts vegetables with no effort, but requires care when handling it...
If JUnit as an artifact is coming through as a dependency in compile scope, it is a bug of one of your libraries, here: ca.juliusdavies.
JUnit should always be included in test scope. Thus, it is not packed into the produced .jar, .war or .ear file, on successful build.
Generally speaking, there is no harm in excluding already included dependencies, as when library 1 and library 2 share one common dependency.
The only problem, of course, that can occur, is when library 1 and library 2 include different versions of the same dependent artifact. This can cause run-time errors, when the features of the library have changed.
Fortunately, this is not often the case, unless the difference in the version numbers is great. In general, it is advisable to include the latest dependency version and exlude the older one. This is most of the time viable.
If not, check wheter there are updates to the first-level dependencies of your project.
I'm new to Maven and I'm trying to build a project for the first time. I want to write some code that depends on apache lucene. Here's a list of artifacts in maven that I'm trying to get.
Is there any way instead of explicitly listing each artifact, I could simply depend on all artifacts of a given version? I tried this:
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>*</artifactId>
<version>3.6.1</version>
</dependency>
which gave me the error
'dependencies.dependency.artifactId' for org.apache.lucene::jar with value '' does not match a valid id pattern. # line 19, column 19
I can verify that I can download dependencies when I explicitly state them. IE this works fine:
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>3.6.1</version>
</dependency>
I realize depending on everything in lucene is probably sub-optimal, but for doing something quick-and-dirty I'd hate to have to manually populate all these little lucene libraries. What is the typical practice for getting a large set of related dependencies in maven?
Short answer: you can't. Remember you just do this once and later you can simply copy-paste dependencies (not very DRY though). Also consider creating an archetype that will quickly create a skeleton with all required dependencies (for quick and dirty projects).
Longer answer: well, you can work around that. Create a separate pom.xml with:
<packaging>pom</packaging>
and declare all Lucene dependencies there manually, one after another. Once and for all. Later you can simply add a dependency to your pom.xml (that is to groupId/artifactId/version defined there) which will transitively include all dependencies of that pom.xml.
Talking about transitivity: if you depend on a JAR in maven and that JAR has other dependencies, you get that transitive dependencies implicitly. Examine Lucene poms, maybe it's enough to import few of them and rely on transitive dependencies?
Inside a single dependency for a groupId add different artifactId's
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<artifactId>spring-context</artifactId>
<artifactId>spring-beans</artifactId>
<version>4.3.7.RELEASE</version>
</dependency>
My project is made of 5 sub projects. One is a War, and the other 4 are jars. Basically the war project needs all 4 jar projects, and their dependencies.
I can strip down the dependencies to have something like war->A->B->C->D. Every sub project add their share of external dependencies (spring, struts, hibernate) so that in the end the war gets everything needed to run.
This looks pretty well organised and square, but then I ask myself if this is very practical to make changes.
Imagine I have to change one line of code in project D, without changing anything to its Maven dependencies. I would have to re-release project D obviously, but then I have to re-release projects C, B, A, and the war just to reflect this change in their pom files. This can be long and annoying, especially if you have to quickly release a new version to fix something in production.
I could make the war depend on all 4 projects, so then I just have to change project D version number in the war pom file. But then I have project A depending indirectly on project D 1.0 and the war specifying project D 1.1. I think that the war direct dependency would win in that case wouldn't it ?
This would make the new war release quicker, but it would also mess my sub projects dependencies, as they would be outdated.
What would be an acceptable way to handle this situation ?
There is no simple answer to your problem.
If you indeed do have a chain of transitive dependencies (A->B->C->D), then releasing each modules up the chain independently is not a bad option. Although it is tedious, there is a good chance your nested dependencies are simple lib jars and will not see changes too often. Hopefully you will not be forced to go through that process frequently. Pretend it would be the same situation as if log4j was updated and all of your modules needed to be updated as well.
Another thing to consider is your WAR's dependencies. Yes, Maven will pull dependencies in automatically for you but it is often a good practice to declare your known dependencies explicitly so you can specify a version number yourself for each module. This would mean A depends on D and the others directly. Unfortunately, if you have conflicting version numbers, as you've described, then you are looking for trouble on your classpath. If you really need to do this though, maven does allow you exclude transitive dependencies explicitly:
<project>
...
<dependencies>
<dependency>
<groupId>my.project</groupId>
<artifactId>module-B</artifactId>
<version>1.0</version>
<exclusions>
<exclusion>
<groupId>my.project</groupId>
<artifactId>module-C</artifactId>
</exclusion>
<exclusion>
<groupId>my.project</groupId>
<artifactId>module-D</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>my.project</groupId>
<artifactId>module-C</artifactId>
<version>1.0</version>
<exclusions>
<exclusion>
<groupId>my.project</groupId>
<artifactId>module-D</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>my.project</groupId>
<artifactId>module-D</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
...
</project>
Here is the documentation describing these optional dependencies and exclusions.
Do you actually need to release B, C, and D independently? If not, consider using an Aggregator pom.xml file at the root of your modules. This will allow you to use SNAPSHOT versions throughout your modules and then release the bunch at once. This is the way our team manages our multi-module project. Using SNAPSHOT dependencies ensures you use the version that was JUST built when those artifacts are needed.
Do you actually release any of projects A to D independently, without the WAR? If not, I don't see any problems with your current setup. You should absolutely use the same version of any module throughout the project. Otherwise you open the door to classloader hell - believe me, you don't want to get there :-(
To make releases easier, the maven-release-plugin may help you.
The best answer these days is now use gradle which is best of ant and maven. I never really liked maven but gradle took alot of the common concepts but made it more like ant in that it is flexible so that there is no an easy answer to your question in gradle ;).
I added some jars as dependencies in pom.xml, but it seems that some of them are useless because those jars were already downloaded using dependencies mechanism...
Is there a way to see those "built-in" dependencies, so that I could add only the needed dependencies in my pom.xml?
For example if I add a hibernate dependency in pom.xml one for cglib is not needed.
Don't do that - list every dependency of your code, but not of the libraries you use; Maven will do its transitive dependency thing and take care of them.
you can run mvn dependency:tree to get the whole tree including the transient dependencies that get included in your project.
There you can start looking
Hope that helped
First, check out Transitive Dependencies:
Transitive dependencies are a new
feature in Maven 2.0. This allows you
to avoid needing to discover and
specify the libraries that your own
dependencies require, and including
them automatically.
Then, a good dependency analyzer will help...
The mvn command-line is your first aid:
mvn dependency:tree
Sometimes you have to figure out where the version numbers came from (See also: Dependency Management). Here you'll have to unveil the parent relationships, and the 'effective-pom' command can help with that:
mvn help:effective-pom
Tool support is helpful as well...
m2eclipse has a Dependency Tree tab that shows how the different hierarchies collapse::
alt text http://www.sonatype.com/books/m2eclipse-book/reference/figs/web/eclipse_pom-editor-depend-tree.png
IntelliJ has another interesting view that lets you detect the conflicts:
(source: jetbrains.com)
If you are using Eclipse then you are see a visual dependency tree which is automatically generated. While adding a dependency if you do not want it to automatically pull some transitive dependencies use the exclusions tag like so
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate</artifactId>
<version>${hibernate.version}</version>
<exclusions>
<exclusion>
<groupId>asm</groupId>
<artifactId>asm</artifactId>
</exclusion>
<exclusion>
<groupId>asm</groupId>
<artifactId>asm-attrs</artifactId>
</exclusion>
<exclusion>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
</exclusion>
</exclusions>
</dependency>
There are some known conflicts when using certain versions of hibernate and AOP due to cglib.
Is there a way to see those "built-in" dependencies, so that I could add only the needed dependencies in my pom.xml?
There are no built-in dependencies. However, when declaring a dependency on a given artifact, Maven will also retrieve the dependencies of this dependency, transitively. Such dependencies are called 3.4.4. Transitive Dependencies:
A transitive dependency is a dependency of a dependency. If project-a depends on project-b, which in turn depends on project-c, then project-c is considered a transitive dependency of project-a. If project-c depended on project-d, then project-d would also be considered a transitive dependency of project-a.
So if you need a dependency in your project, just declare it (and the dependencies of this dependency will come transitively).
To visualize the dependency tree of a project, the best tool is mvn dependency:tree (or any fronted offered by your favorite IDE). This is a must use tool to analyze your dependencies and check them for proper convergence and potential conflicts resulting in expected version being used.
For example if I add a hibernate dependency in pom.xml one for cglib is not needed.
Actually, this is a bad example, cglib is an optional dependency of Hibernate Core which declares in its pom.xml:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-cglib-repack</artifactId>
<version>2.1_3</version>
<optional>true</optional><!-- think of it as "excluded by default" -->
</dependency>
Hibernate gives you the choice between javassist and cglib, it's up to you to decide which one to use and to declare it explicitly, hence the optional status.
See also
Introduction to the Dependency Mechanism
The whole section 3.4. Project Dependencies in the Maven Reference Guide