I have a parent POM and a normal Maven project.
Both define BOMs in their dependencyManagement. In some cases, these BOMs may overlap, e.g. both specify a version for log4j.
From my tests it seems that:
For overlapping BOMs in the parent POM, the first one wins, i.e. supplies the version for the artifact.
If a BOM from the child and from the parent overlap, then the version from the BOM in the child wins.
Unfortunately, I did not find any documentation about this.
Am I right and can I rely on this behaviour?
Logging framework traditionally lives "next to" your actual code, so it is a bit unclear how to handle this.
I found that separating the build phase dependencies from the deployment phase dependencies works for me.
The basic idea is that you write the code only requiring a dependency to the API of the logging framework (slf4j used to be a natural choice), and then you have several deployment Maven configurations (one for JBoss, one for WebSphere, one for running in your IDE etc) where you add the dependencies relevant to that deployment.
So my suggestion is to change your codebase accordingly to only have API dependencies for the logging framework, and then create a new Maven project for each actual deployment type. You might also want to bake in deployment specific configuration files if needed.
Related
why use Maven when you have such quantity of local jars?
So we have a client that have a lot of private jars and custom jars.
For example commons-langMyCompanyCustom.jar which is commons-lang.jar with 10 more classes in it.
So on their environment we use 100% Maven without local dependencies.
But on our site we have the jars for development in Eclipse and have Maven build with the public ones, but we do not have permission to add their jars in our organizational repository.
So we want to use the Maven good things like: compile,test, build uber-jar, add static code analysis, generate java-docs, sources-jars etc. not to do this thinks one by one with the help of Eclipse.
So we have 70 jar some of them are public if I get the effective pom on their environment I found 50 of them in Maven Central, but the other 20 are as I called "custom" jars. I searched for decision of course but found this:
<dependency>
<groupId>sample</groupId>
<artifactId>com.sample</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>${project.basedir}/src/main/resources/yourJar.jar</systemPath>
</dependency>
So for all 20 of them I have to add this in the development maven profile??
Is there a easy way like in Gradle where you can add all folder with its dependencies to the existing ones?
Also installing one by one in every developer's repo is not acceptable.
Please forget the system scope as mentioned before! Too problematic...
Ideally:
Ideally, all your developers have access to Repository Manager in your or their organization (if possible).
A central environment for your System Integration Testing, maybe?
Alternatively, you may have a central environment for testing where all the dependencies are provided. This approach can be used to simulate how a compilation would work as if it's in your client's environment. Plus you only setup jars once.
So on their environment we use 100% Maven without local dependencies.
But on our site we have the jars for development in Eclipse and have
Maven build with the public ones, but we do not have permission to add
their jars in our organizational repository.
According to what you're saying in the above-quoted excerpt I believe you want to have set in your build's pom.xml assuming that in the client setup the dependencies will be present.
Especially, as you indicate that the organization doesn't give you permission to add their jars in your repository, I would use the provided scope.
As stated in the Maven docs, the definition of a provided dependency is as followed:
This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime. For example, when building a web application for the Java Enterprise Edition, you would set the dependency on the Servlet API and related Java EE APIs to scope provided because the web container provides those classes. This scope is only available on the compilation and test classpath, and is not transitive.
So basically you assume that these dependencies will be present at your client's setup. However, this has some limitations. Meaning you can build solutions independently but cannot test it locally because you won't have the dependencies on your workstation.
If you won't even have access to the jars to configure your central environment ask if your client can provide a DEV/SIT environment.
None of the above? Inherit a parent pom.
To avoid the whole constant copy-paste process for every single (related) project, maven has the tools to centralize dependency and plugin configurations, one of such is by inheriting the configuration of a parent pom. As is explaining in the following documentation it is quite simple:
First you create a project with just a pom.xml where you define everything you wish to centralize (watch out, certain items have slight differences in their constructs);
Use as property of packaging tag the option pom: <packaging>pom</packaging>;
In the pom's that have to inherit these configurations set the parent configuration tags in <parent> ... </parent> (documentation is very clear with this);
Now everytime you update any "global" pom configuration only the parent version has to be updated on every project. As a result of this, you only need to configure everything once.
You can also apply this together with the abovementioned solutions, this way combining to find a solution that fits best to your needs.
But there is a big Maven world out there, so I advise a good read in its doc's to further acknowledge your possibilities. I remembered these situations because I've been in a similar situation you seem to be now.
Good luck!
Another alternative is the project RepoTree.
This one creates a Maven repository directory (not a server) from another directory which contains just the .jars. In other words, it creates the necessary .pom files and directory structure. It takes into account only the precise information from metadata contained in the archives (MANIFEST.MF, pom.xml).
Utility to recursively install artifacts from a directory into a local
Maven repository Based on Aether 1.7
This is 5 years old, but still should work fine.
TL;DR: MavenHoe creates a Maven repository server (not a directory) which serves the artefacts from a directory, guessing what you ask for if needed. The purpose is to avoid complicated version synchronizing - it simply takes whatever is closest to the requested G:A:V.
I have moved the MavenHoe project, which almost got lost with the decline of Google Code, to Github. Therefore I put it here for availability in the form of a full answer:
One of the options you have when dealing with conditions like that is to take whatever comes in form of a directory with .jar's and treat it as a repository.
Some time ago I have written a tool for that purpose. My situation was that we were building JBoss EAP and recompiled every single dependency.
That resulted in thousands of .jars which were most often the same as their Central counterpart (plus security and bug fixes).
I needed the tests to run against these artifacts rather than the Central ones. However, the Maven coordinates were the same.
Therefore, I wrote this "Maven repository/proxy" which provided the artifact if it found something that could be it, and if not, it proxied the request to Central.
It can derive the G:A:V from three sources:
MANIFEST.MF
META-INF/.../pom.xml
Location of the file in the directory, in combination with a configuration file like this:
jboss-managed.jar org/jboss/man/ jboss-managed 2.1.0.SP1 jboss-managed-2.1.0.SP1.jar
getopt.jar gnu-getopt/ getopt 1.0.12-brew getopt-1.0.12-brew.jar
jboss-kernel.jar org/jboss/microcontainer/ jboss-kernel 2.0.6.GA jboss-kernel-2.0.6.GA.jar
jboss-logging-spi.jar org/jboss/logging/ jboss-logging-spi 2.1.0.GA jboss-logging-spi-2.1.0.GA.jar
...
The first column is the filename in the .zip; Then groupId (with either slashes or dots), artifactId, version, artifact file name, respectively.
Your 70 files would be listed in this file.
See more information at this page:
https://rawgit.com/OndraZizka/MavenHoe/master/docs/README.html
The project is available here.
Feel free to fork and push further, if you don't find anything better.
I have several huge legacy applications that I am now working on. After months of testing, we finally reached deployment only to have a "failed to load webapplicationcontext" which foiled the whole endeavor. That specific failure was due to a name space conflict between two transitive dependencies. i.e., both jars had a class to load as: org.something.somethingelse.ClassName.
There are ~100 jars pulled in via maven for this single project. Several explicit, most transitive. Ideally, I would like to know every single jar I'm putting on my classpath. Practically, though, I don't have enough experience or time to look through every one of them for potential issues.
Is there a tool, technique, or eclipse/intelliJ feature that I can use to scan a set of jars for similar namespaces?
You can try with enforcer plugin. In a maven project, it's very usefull when you need to detect different jar depenndencies of same artifact with different version.
You can read this post too.
So there were a couple of different solutions here. I ended up using jhades (http://jhades.github.io/) to identify conflicts within the war, and then tattletale (a utility provided by JBOSS support) to identify conflicts between the war and the container.
I added 'exclude *' tags to all the explicit dependencies to prevent any transitive dependencies from loading. I added explicit dependencies for anything that still wasn't present. After ensuring that all compiled dependencies played nicely, I set any libraries identified by tattletale to provided and added the necessary module to standalone.xml. These things like hibernate, apache libs, servlet APIs etc.
The other thing I discovered which made this so difficult to identify in the first place is that JBOSS's classloader indexes libraries according to how the hosting file system orders them. On Windows, which is where we do 90% of our development, they are always loaded alphabetically. On linux, where we do our production deployments, the order is pseudo random. Our production servers are built from the same images, so a RHEL 3.4 server will load in the same order as another 3.4, but a 3.5 will load entirely differently. Thus, we did not see a failure until the stars aligned and we deployed to a 3.6 server. In production.
Hope this helps someone.
Well here is an interesting experience i had since last couple of weeks structuring my maven multi module project.
When i decided to use maven for my build life cycle management i had couple of reason that i wished to choose maven.
a. Mostly development teams are divided so that each team can work on separate Module within the project like Team-A to work on User Management System, Team-B to work on Authorization System, Team-C to work on Document Management System...and so on. Each team has java developers, testers, UI experts etc.
So the maven Project structure should be such that each team can independently work on their respective modules. They must be able to code, compile, build, test, deploy their module without having to compile, test modules belonging to other teams.
And thus i came to conclusion that each development module of the maven multi-module project must represent a Functional Module
After some discussions on forums i found people suggesting me to follow layered approach were child modules must be layers like controller-layer,service-layer,dao-layer etc. I did not pay heed to this advice because this not solving my purpose of teams working on individual module. This way for large project the build and deployment time for each team during development increases which does impact the project time-lines. sometimes the build and deploy time is upto 30 minutes say if there are 10 to 11 modules in the project.
But i did pay heed to a suggestion that keeping DAO layer separate for each module is not a good idea as DAO is highly granular and reused by other modules. and so the dependency of one module on other would would any how become greater.
I found a solution to this problem by creating a common module and moving DAOs and DOMAIN to the common module which will be inherited as a dependency by each module. And this seems to be a more viable option. Now the Project Structure looks like this.
Now when i build the project and run the webapp on server, It complains 404, Resource Not Found. I found that this is because the WEB-INF/classes folder is missing, src/main/java is missing in web-app module. I searched and found couple of links that suggested it is Deployment Assembly issue in Eclipse. So i need to manually create these folders and add in the deployment assembly because maven does not do it.
But the bigger questions are
do i need to move the Controller classes like com.mycompany.usermgmtsys.controller.UserMgmtController etc.. to src/main/java Or maven should find the controllers from the module jars included as dependency in WEB-INF/lib.
I dont want to do this i.e. putting java file in web-app. i want all the controllers should be available to the web-app as dependency for example WEB-INF/lib/usermgmtsystem.jar. But then wouldnt the Tomcat be looking for controllers in classes folder.
I dont know what should i do ? Any suggestions would be appreciated.
Its the way the eclipse render maven based project. It generally creates two structure. One based on master pom (parent project) and others based on individual module pom. however doing changes in any structure will reflect in the other one. As a practice I do changes in individual module folder structures and is more easy to read too.
Personally I try to avoid multi-module projects as, if you're using the Maven Release Plugin, you are locked into releasing all your modules together.
While this may sound like a convenience the problem arises when you need to do bug fix release to one of the modules - you end up releasing all the modules, not just the module with the bug fix, incrementing their version even though they haven't changed.
You also take a hit if you're running CI with multi-module projects - you're build typically runs over all modules from you root pom but if you're working in a particular module, you end up taking the hit of building those that haven't changed, in effect losing some of the benefits that the modularization was meant to provide.
So, go with independent modules but, and this is the important bit, create a common 'dependency' pom used by each.
A 'dependency' pom is a pom that standardizes all the dependencies across your projects and is different in that those dependencies are specified in the dependencyManagement section rather than the dependencies section (it also sets up standard plugin config, etc). This allows your project poms to specify the dependency pom as their parent and then declare the dependencies they need minus the versions, which are picked up from the 'dependency' pom and thus standardized across your projects.
If you are still concerned about being able to built everything, this can be achieved with a simple batch-file.
This is a good question. There are many aspects that must be considered for a useful project layout. I'd like to try to answer one which you didn't mention. Is your app extensible by users? If it is, then consider creating a separate module for your public API layer (service interfaces, DTOs used by those services, and Exceptions thrown by the services).
In our app, we have several maven modules per functional area. The idea is that a group worked on a feature within just one functional area and this isolation kept them messing with sources being modified by another group. Each functional area is broken down further in maven sub-modules we call "api", "domain", and "service" - we don't lump services/controllers, domain, and exceptions into a single module. The api module contains those classes we want to expose to customers for their customizations. Our service layer is the implementation of those interfaces. Further, we do not allow one module's service to call another module's service as this would bypass our service orchestration layer where customer can attach extensions to our services. Using separate maven modules per functional area helps enforce this.
We have other modules (internal-api, web, adapter) but they don't really add to this topic.
I figured out the issue. Controllers are presentation-layer components. The dispatcher expects the presentation layer components in the WEB-INF/classes folder in the target rather than looking for it in the lib. I am not sure if this is valid only for maven based structuring in eclipse. So finally these are the changes i have made
a. Created a src/main/java source folder in web-app. It is not generated by default in web-app module.
b. Add packages and respective controllers in the src/main/java folder.
So the final structure that i have (i am not pasting exact eclipse snapshot, this is generalized view)
In effort to customize the release of a project I came across the following article:
http://www.sonatype.com/people/2011/01/using-the-maven-release-plugin-things-to-know/
In essence the following assumptions are made for using the default release plugin:
Your codebase is going to be versioned and released as a “unit”.
You are using an SCM tool and a repository manager.
You are performing your release from a single, “versionable” unit in SCM.
You are using standard version numbers
You are publishing artifacts to a repository
In mostly all these cases, our project does not meet these assumptions. We want to use a custom version schema (independent of SCM, maven, etc). Deploy the artifacts to a filesystem (not a repository). Not have maven mess around with SCM at all. Etc.
As recommended, we should probably define our own release lifecycle. Therefore I am assuming we would need to override the maven default lifecycle release phase to run our plugin. I guess I am missing the location of the required documentation. Is this even possible?
What you're doing may be too radical to be worth doing within Maven, you may find yourself effectively writing a parallel build system simply triggered from Maven.
Assuming the effort is worth it - I would start by configuring the various plugins (most of Maven's functionality is delivered through plugins, triggered by phases in the lifecycle) and have them execute at a different goals within different phases. You may need to write Maven plugins to do whatever you want that doesn't already exist; which I expect you'll need to do in such a radical case.
http://maven.apache.org/guides/plugin/guide-java-plugin-development.html
I'm trying to work out the best way to setup our multi-module Apache Maven project in a way that allows for disparate release cycles of modules, and doesn't introduce dependency issues when debugging the project.
We currently have a setup along the lines of:
bigsystem#1.2
parent-1.1-SNAPSHOT
module a#1.4-SNAPSHOT
parented by parent#1.1-SNAPSHOT
module b#1.3-SNAPSHOT
parented by parent#1.1-SNAPSHOT
depends on a#1.1
module c#1.1-SNAPSHOT
parented by parent#1.1-SNAPSHOT
depends on a#1.2
depends on b#1.1
The dependencies declared in modules b and c contain the minimum version required to compile the module, which isn't necessarily the current version of the module, or the version of the module being deployed.
From a build perspective this works well, each module can be released/updated as needed, however when trying to debug the deployed application under IntelliJ IDEA (versions 8 and 9 EAPs) having opened the top level pom, IDEA decides that since we declared a dependency on a#1.2, that anytime we step into one of a's classes, it should open it from a-1.2-sources.jar rather than the current a#1.4 sources in the project. This is further confused by the fact that stepping into any of b's classes takes us to b#1.1 rather than b#1.3.
My initial attempt to work around this was to declare the version numbers in the parent pom's dependencyManagement section and just have the sub-modules inherit the version. This worked to the degree of solving the IDEA debug issue as the dependencyManagement section can point everyone to the current -SNAPSHOT versions.
This unfortunately causes a problem when doing a maven release due to having to release the parent pom before releasing the module, but as the parent may refer to multiple in-development -SNAPSHOTS it can't be released and we end up adding version references back to the modules pom to satisfy the release.
It would seem that using maven's dependencyManagement section would only really work well if we were releasing ALL bundles at the same time, regardless of if they changed, but as we're wanting to manage releases of each sub module only when needed this model doesn't seem to fit.
I have a suspicion I'm missing something, and that a combination of dependencyManagement and version ranges might satisfy out requirements although I've yet to see version ranges work properly.
Is there a better way? A proper way?
I would recommend not making them modules, but make their POMs independent. That way you do not have to worry about trying to satisfy parent POM dependencies. Since they are released independently, they really should have independent project object models. Think of Apache Commons as a template.
I think the problem with IDEA arises because you are using the root POM in your source structure to do two things that are usually mutually exclusive in Maven. You are first using the POM as a location to store common configuration information for unrelated (from a build perspective) Maven projects. Secondly you are using the POM as an aggregator for your build. You can do each of these without doing the other.
Like Rob said, remove your module a, b, etc. projects from the modules section of your parent POM. Secondly, move your parent POM down into its own directory as it is really a sibling of the other modules with respect to your build and release process. The way you have it now, it is more of a parent/aggregator.
The way you have it now also doesn't lend itself to tagging and releasing each module individually as a tag of your parent POM would likely needlessly include all of the module sub-folders.
Your file structure would look like:
parent
pom.xml
module a
pom.xml
module X
pom.xml
As for the thing you are missing, dependencyManagement isn't really well suited to manage versions for intra-project dependencies. That is dependencies between modules within an aggregated build. It is more well suited for declaring global versions for external dependencies.
The final/working solution we ended up using was fairly similar to what we started with. The actual project structure remains the same:
bigsystem#1.2
parent-1.1-SNAPSHOT
module a#1.4-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
module b#1.3-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
o depends on a#1.1
module c#1.1-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
o depends on a#1.2
o depends on b#1.1
distribution a#1.2-SNAPSHOP
However the main differences are that:
parent module does not include any versions of project artifacts
individual modules fully declare their project dependencies and specify a version range, i.e. [1.0.0,1.1.0)
all modules start there version number cycles from .1, i.e 1.0.1-SNAPSHOT, this allows the version range to satisfied by initial snapshots (1.0.0-SNAPSHOT is earlier than 1.0.0 final, so not included).
distribution pom (not initially shown in question) identifies the exact version to be deployed/included in a specific release.
delete all project -SNAPSHOTS from local maven repository when releasing so that ranges pickup releases only ( or use -Dmaven.repo.local=/tmp/sometemprepo for a fresh local repo)
This makes each module more standalone and gives us the freedom to release and deploy new versions of our project artifacts with minimal fuss.
They certainly seem like separate modules. What benefits are you gaining by smashing them together if they have different dependencies, even within the multi-module project?