I appologize for the length of this post, but I had trouble making it more concise without presenting the picture. I've recently inherited the job of build-master for a maven 3.0 multi-module project. The problem is that the structure of the project/modules is a disaster. From the way things are currently stored in Source Control (we use RTC) to the pom structure of the modules, I'm tearing my hair out trying to get a full build cycle completed each time.
As a project hierarchy goes, all the modules are stored "flat"; ie: everything is at the same level. I have a parent pom, and all modules depend on the parent. However, the parent is at the same level as all my other modules.
Ex:
c:\dev\MyEarProject
+ parent-pom
- pom.xml
+ module1
- pom.xml (depends on parent-pom)
- src
- main
- ...
+ module2
- pom.xml (depends on parent-pom)
- src
- main
- ...
+ module3
- pom.xml (depends on parent-pom)
- src
- main
- ...
The parent pom defines all the modules required to build the project, as well as a bunch of properties for artifact version numbers used throughout the different submodules:
<modules>
<module>../module1</module>
<module>../module2</module>
<module>../module3</module>
</modules>
<properties>
<org.springframework.version>3.0.5.RELEASE</org.springframework.version>
<slf4j.version>1.6.4</slf4j.version>
<repositoryAddress>${snapshots.repo.url}</repositoryAddress>
<my.hibernate-module.dao.impl>1.2.3</my.hibernate-module.dao.impl>
<my.hibernate-module.dao.api>1.2.3</my.hibernate-module.dao.api>
</properties>
Each module's pom, in turn, depends on the parent pom via the pom's artifact number:
<parent>
<groupId>com.cws.cs.lendingsimulationservice</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.6</version>
</parent>
To make things even more confusing, the actual artifact name may, or may not (depending on the module), match the module path. For example, module1 may be located in path c:\dev\MyEarProject\module1 but have artifact name hibernate-module. However, due to the way it is stored in RTC, the directory is called module1 when it is checked-out.
The easiest way to build everything, of course, is to go into c:\dev\MyEarProject\parent-pom\ and run mvn clean deploy. This works fine when in SNAPSHOT mode as the SNAPSHOT repo allows for multiple deployments of the same artifact version. But in release mode, this fails.
This structure is causing 2 problems for me.
Everytime I need to make a version change to a property in the parent, I have to update the parent-pom version number, and all the child modules parent pom's version, and all the child modules version themselves (since the parent changed).
Whenever I need to deploy a release cycle, mvn will throw an error if one of the modules has not changed since the last cycle and consequently cannot be redeployed to the same repo (the repo does not allow overwriting existing artifacts)
So I'm looking for the best way to restructure this project to avoid these problems. For the parent pom, I know I can use a relative path to point to the parent instead. However, given the "flat" structure of the modules, is this a recommended approach (ie: the parent pom relative path would be ../parent-pom/pom.xml - seems a little odd to me)? Additionally, given that the versioning control of the parent is independent of the modules, would using a relative path not just open the door to additional confusion (ie: there would be no way to know which version of the parent pom is associated with which version of the submodule).
Secondly, how can I build the entire ear without encountering the deploy errors I am having? Since the artifact already exists in the repo, I don't need to rebuild and redeploy it. I tried using --projects but with the number of modules involved, it gets extremely difficult to manage.
The first thing I really recommend is to restructure the projects folders ...which means to have the projects folder represent the structure which means NOT flatten the structure.
+-- parent-pom (pom.xml)
+--- module1 (pom.xml)
+--- module2 (pom.xml)
+--- module3 (pom.xml)
As a result of that the modules section your parent will be simplified like this:
<modules>
<module>module1</module>
<module>module2</module>
<module>module3</module>
</modules>
Furthermore the parent entries in your modules can be simplified as well like this:
<parent>
<groupId>com.cws.cs.lendingsimulationservice</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.6</version>
</parent>
...which brings me to the next point:
If all your current project define their parent as above this is simply wrong, cause will try to find the parent within the repository and not in a upper level folder. In other words this is causing of much of your problems with releasing etc.
If we would fix this problem it has to look like this which I can't recommend:
<parent>
<groupId>com.cws.cs.lendingsimulationservice</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.6</version>
<relativePath>../parent-pom/pom.xml</relativePath>
</parent>
An other thing which I observe is that you don't use SNAPTSHOT's which will be replaced by the release plugin during the release phase. And in relationship to that it will automatically change all versions in the appropriate parents etc.
In ideal case your modules should look like this:
<parent>
<groupId>com.cws.cs.lendingsimulationservice</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.6</version>
</parent>
<artifactId>module-1</artifactId>
<!-- No Version or groupId -->
Cause all modules will inherit the version and groupId from their parent. Sometimes it's useful or needed to change a modules groupId but it's an exception.
On thing I reread is about the separate versioning of the parent. This simply does not make sense, cause it's the parent of their modules so put it into the same structure and of course the same VCS.
If you want to make some configuration/plugins versions, dependencies which should be used for other projects as well than make a separate corporate pom.xml which is a separate project and will be separately released etc.
After you finished your structure changes you can simply go into the parent-pom directory and do mvn clean package or mvn release:prepare release:perform from that folder and everything will simpler.
If you're publishing your POM, you'll have to release any updates but you don't need to modify POM versions by hand - you can update versions automatically using the versions plugin or the release plugin. I tend to prefer the release plugin as it'll commit to SCM for you too.
mvn versions:set
http://mojo.codehaus.org/versions-maven-plugin/
mvn release:prepare release:perform
http://maven.apache.org/plugins/maven-release-plugin/
Your repository manager may also allow overwriting an existing version but it's better practice to just release a new version.
I tend to prefer the flat module structure as it allows use of the parent folder to store common files e.g. checkstyle configuration. I also find it useful to share the group id across modules then name the module directory the same as the artifactId.
You are presenting contradicting requirements. You want to restructure your project but can't move things around. You want to simplify your deployment and release cycles but do not want to use a single version.
Given that changes in one module will inevitably affect all of the dependent modules, I would use a simple version'ing scheme where all the sub-modules inherit their parent's version. maven release:prepare and release cycles become simple. Use a release notes to track your changes and justify skipping unnecessary testing of unchanged modules (changes to a version do not change the build/binary output of the build process so you can use that as your primary argument).
Good luck with your project.
Related
I hope this is the right place and right discussion to start.
What I want
I am looking for a good/the right way to handle multiple maven projects, that share a common configuration, which is in my case
Set the maven-compiler-plugin version with source/target compilation version
Set the maven-source-plugin to attach sources to artifacts
Define local repository server
Define common dependencies (via dependencyManagement and dependencies).
What I did
My first approach was to create a multi module project. But as the modules are actually independant, when it comes to release (so not the same release cycle), it seemed to be more a burden as a gain. I also failed to make it that I can independently work with a submodule in IntelliJ and have for each submodule it's own Jenkins job. For me it was always all or nothgin
The second approach was/is to have seperate projects. This makes coding in IntelliJ simple and Jenkins can have jobs for each project.
But now changes to parent pom, which affects the children, are tedious to delegate as all other projects must adopt to the change to the new parent.
It is doable and might be the way to work with this, but as I am afraid I misuse the parent concept here, I was wondering, whether there is a better way to use common settings in multiple maven projects ?
Thanks for reading and I hope this was understandable
You can re-use dependencies management section from different pom using <scope>import</scope>, described here, but that would work only for dependencies and you cannot do that for properties, repos and etc.
I'm afraid there is no better way to solve you problem then the parent-pom. Actually this is widely adopted approach. Spring boot, for example, uses that for a long time already.
One thing you want to do to is a good release management for that parent pom. Be sure that you are not dependent on SNAPSHOT version to not brake existing projects.
Just create a parent POM with the common settings and keep the rest separate.
Publish that POM to the (local) repository separately as a standalone entity with pom packaging (no submodules at this level).
Have a look at the Spring Boot parent POM as an example.
The only change in your projects would then be to use your new POM as the parent POM.
<project ... >
<parent>
<groupId>my-group</groupId>
<artifactId>my-parent</artifactId>
<version>1</version>
</parent>
...
Note however that it is generally not a good idea to hardcode dependencies in the parent POM (defining versions via dependencyManagement is OK, but let each project specify explicitly which dependency it actually needs).
Background: Most of my Java project have a lot of boiler-plate configuration that I end up writing into my POM. (For example the sourceEncoding is always UTF-8, the organization details never change). To save having to copy-paste blocks of configuration I've created my own standard 'parent' POM I can extend for each of my projects.
I also have a number of tools that I use in every Java project I work on (e.g. checkstyle, PMD etc) so I've added the plug-ins for each tool into my standard parent pom.
PMD (and a few other tools) have an interesting problem - they requires a set of configuration files to operate correctly. Since Maven prefers to work with the concept of 'one deployable resource per module' I've created the following (simplified) structure:
<My Template>
|--- Config Files
\--- Parent Pom
My Template: is a maven controlled project with two modules
Config Files: is a maven module that bundles the PMD config files into a ZIP and deploys them to a maven repo
Parent Pom: is all my boiler plate code. It has a dependency on "Config Files" and is set up to extract the configuration from the ZIP
So far so good....
The Requirment: "Parent Pom" and "Config Files" are always built and deployed together - that is "Parent Pom" version 1.2.3 will only ever work with "Config Files" version 1.2.3, so when I write the dependency in "Parent POM" I need to do something like:
<dependency>
<groupId>org.my-org</groupId>
<artifactId>config-files</artifactId>
<version>${project.version}</version>
<type>zip</type>
<scope>test</scope>
</dependency>
The Problem: When I come to start my application (let's say version 0.0.1) I use "Parent Pom" as my parent. Maven will calculate the effective POM for the project and ${project.version} will be interpreted as 0.0.1 (the application version) rather than 1.2.3 (the parent POM version). Maven then fails to find the config files.
The Question: How can I tell Maven to "Give me version for POM xxx"?
What I really don't to do is creating additional properties that are in lockstep with ${project.version} because I can guarantee that we'll forget to update it when we Bump the POMs for a release!
There are a few things you can do.
You can use ${project.parent.version} instead of ${project.version}.
But probably a better way would be to define pluginManagement and/or dependencyManagement in your parent pom - with appropriate versions and configuration. In you child projects you just "use" the "managed" version/configurations in dependencies or build/plugins without specification of concrete versions or configuration.
For example see the org.sonatype.oss:oss-parent POM which is widely used in open-source projects.
We have a very modular Maven pom set up, with common jars and specific jars all being bundled in to a war and ear. As there is so much reuse amongst the 70+ modules we do not use multi-module and each module can and does have its own lifecycle and can be released independently of any other module.
All modules inherit from various parent poms and ultimately every pom inherits from a master POM where all the external versions such as spring and the common local module versions are defined.
This works ok until we come to do a release. If the master POM requires a change, which it does occasionally, ALL poms need to be updated one way or another. I am aware that the maven versions plugin can update a specific POM with the latest SNAPSHOT versions etc, but this only works at the individual POM level.
We would like to be able to alter all poms once a release has been completed, iteratively.
We do not use multi-module POMs and are not able to change our build process to use this mechanism.
I have read around SO and the nearest to the problem is here.
https://stackoverflow.com/a/3615417/1279002
Writing a shell script seems to be one solution, but we have a Windows and Linux mix of development and build systems. I am sure other will have solved this issue. Can anybody advise how they have solved this?
In a similar setup, I have all my parent POMs always stay at 1.0.0-SNAPSHOT and setup various properties in the parent POMs to track internal module version numbers (so this setup now centralizes both dependency management versions AND custom module versions [via properties] into the parent POMs).
So if I need to update the reference to some com.myco:module-x, I can do this:
Edit the appropriate parent POM and set the <module-x.version>1.2.3</module-x.version> property to the new value
Rebuild/install the parent POM
Rebuild the target end-application (ear, war, jar app etc).
Where in module-x's POM it's definition may be something like this:
<groupId>com.myco</groupId>
<artifactId>module-x</artifactId>
<version>${module-x.version}</version>
And any POMs which reference com.myco:module-x refernece it via ${module-x.version} as well.
At this point, the build of the application will pickup the changes in the parent POM and thus any references it has to any properties defined in the parent POM.
There's some subtle nuances in doing this in terms of when/how the "middle man" modules need to get rebuilt...
But I really don't believe there is any silver bullet here.
The approach we've taken works pretty well, coupled with Jenkins to automate rebuilds of modules with interdependencies whenever parent POMs change.
The benefit here is that you seldom need to modify anything but the parent POMs, ever. The middle-man modules and application POMs don't need to be updated to get new version numbers, etc.
The biggest caveat though is that two rebuilds of a given module at the same version could result in a different artifact, for example:
module-x has a dependency on module-y:1.2.3
module-x is built (jar is created with a MANIFEST referencing module-y:1.2.3)
parent POM is modified to set <module-y.version>1.2.4</module-y.version>
module-y is rebuilt to create the 1.2.4 artifact
module-x is built (jar is created with a MANIFEST referencing module-y:1.2.4)
But note that #2 and #5 both built module-x with the same version for module-x, but with two different embedded MANIFEST's referencing different module-y versions.
We overcome this nuance by automating all the dependent modules with our Jenkins CI server
I have a maven project
Parent Project (with the following modules) (package type pom)
API Project (package type jar)
Packaging Project (has a dependency on API Project) (package type custom)
I want to be able to deploy the API project to Nexus repositories so others can leverage that code. The Packaging Project is more of a supporting utility project for a smaller set of deployment use-cases. The Parent Project wraps it all together for me.
When I deploy the API project to the nexus repo, it deploys fine. If i try to make a brand new project that has a depdency on API, it finds the API dependency in nexus but then also wants the Parent project as well. Is there any way to get around publishing the parent project as its really isnt necessary for use of the API lib when used via the nexus repo?
Any tips on how to organize my maven proj to support this?
When you add a <parent> reference to a Maven project what you are doing is saying: "Take all the configuration from that parent and inject it into my model, then override with the following"
Therefore, in order for Maven to build the model of your project, it is necessary for Maven to retrieve the parent itself. In other words, adding a <parent> tag creates an explicit hard dependency between the parent and the child.
The good news is that Inheritance does not have to follow Aggregation. What does that exactly mean?
Aggregation is when you list <modules> in your pom. It tells Maven that the reactor (i.e. the set of projects that Maven builds) should also include the following (sub)projects.
Inheritance is when you set a project's <parent>.
Nowhere does Maven enforce that a project's <parent> has to list its children as <modules> and nowhere does Maven enforce that a projects <modules> must list the project as a <parent>.
Some people will set up their project like so
ROOT/
+- pom.xml
+- parent/
| \- pom.xml
+- api/
| \- pom.xml
\- packaging/
\- pom.xml
where the parent of ROOT, api and packaging is actually a child of ROOT. Or sometimes ROOT will be a standalone project with no parent [In fact this is a pattern I use a lot myself. When I am working on several related project I will throw together an aggregating pom.xml on my local disk and open that with my IDE and that way all the related code is available as one single "project", even though the actual modules may come from different sources]
So in your case the solution would be to remove the <parent> tag from your "API" module.
Now! There is a downside. When you remove the <parent> tag from your "API" module you have removed all the defaults that your parent project is providing, so you will need to copy those defaults that are relevant to the "API" project or else you may find subtle changes in behaviour. For example, you should definately copy over the pinning of plugin versions, and any <dependencyManagement> that is relevant to the "API" dependencies. There are other bits you may have to copy, but you should use the Maven command mvn help:effective-pom before and after removing the <parent> tag as an aid to seeing the effective differences
for my developments I use uber-pom maven plugin which merges information in maven project hierarchy and provides independent pom as result and I publish artifacts in maven central without any extra links to parents
I'm trying to work out the best way to setup our multi-module Apache Maven project in a way that allows for disparate release cycles of modules, and doesn't introduce dependency issues when debugging the project.
We currently have a setup along the lines of:
bigsystem#1.2
parent-1.1-SNAPSHOT
module a#1.4-SNAPSHOT
parented by parent#1.1-SNAPSHOT
module b#1.3-SNAPSHOT
parented by parent#1.1-SNAPSHOT
depends on a#1.1
module c#1.1-SNAPSHOT
parented by parent#1.1-SNAPSHOT
depends on a#1.2
depends on b#1.1
The dependencies declared in modules b and c contain the minimum version required to compile the module, which isn't necessarily the current version of the module, or the version of the module being deployed.
From a build perspective this works well, each module can be released/updated as needed, however when trying to debug the deployed application under IntelliJ IDEA (versions 8 and 9 EAPs) having opened the top level pom, IDEA decides that since we declared a dependency on a#1.2, that anytime we step into one of a's classes, it should open it from a-1.2-sources.jar rather than the current a#1.4 sources in the project. This is further confused by the fact that stepping into any of b's classes takes us to b#1.1 rather than b#1.3.
My initial attempt to work around this was to declare the version numbers in the parent pom's dependencyManagement section and just have the sub-modules inherit the version. This worked to the degree of solving the IDEA debug issue as the dependencyManagement section can point everyone to the current -SNAPSHOT versions.
This unfortunately causes a problem when doing a maven release due to having to release the parent pom before releasing the module, but as the parent may refer to multiple in-development -SNAPSHOTS it can't be released and we end up adding version references back to the modules pom to satisfy the release.
It would seem that using maven's dependencyManagement section would only really work well if we were releasing ALL bundles at the same time, regardless of if they changed, but as we're wanting to manage releases of each sub module only when needed this model doesn't seem to fit.
I have a suspicion I'm missing something, and that a combination of dependencyManagement and version ranges might satisfy out requirements although I've yet to see version ranges work properly.
Is there a better way? A proper way?
I would recommend not making them modules, but make their POMs independent. That way you do not have to worry about trying to satisfy parent POM dependencies. Since they are released independently, they really should have independent project object models. Think of Apache Commons as a template.
I think the problem with IDEA arises because you are using the root POM in your source structure to do two things that are usually mutually exclusive in Maven. You are first using the POM as a location to store common configuration information for unrelated (from a build perspective) Maven projects. Secondly you are using the POM as an aggregator for your build. You can do each of these without doing the other.
Like Rob said, remove your module a, b, etc. projects from the modules section of your parent POM. Secondly, move your parent POM down into its own directory as it is really a sibling of the other modules with respect to your build and release process. The way you have it now, it is more of a parent/aggregator.
The way you have it now also doesn't lend itself to tagging and releasing each module individually as a tag of your parent POM would likely needlessly include all of the module sub-folders.
Your file structure would look like:
parent
pom.xml
module a
pom.xml
module X
pom.xml
As for the thing you are missing, dependencyManagement isn't really well suited to manage versions for intra-project dependencies. That is dependencies between modules within an aggregated build. It is more well suited for declaring global versions for external dependencies.
The final/working solution we ended up using was fairly similar to what we started with. The actual project structure remains the same:
bigsystem#1.2
parent-1.1-SNAPSHOT
module a#1.4-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
module b#1.3-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
o depends on a#1.1
module c#1.1-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
o depends on a#1.2
o depends on b#1.1
distribution a#1.2-SNAPSHOP
However the main differences are that:
parent module does not include any versions of project artifacts
individual modules fully declare their project dependencies and specify a version range, i.e. [1.0.0,1.1.0)
all modules start there version number cycles from .1, i.e 1.0.1-SNAPSHOT, this allows the version range to satisfied by initial snapshots (1.0.0-SNAPSHOT is earlier than 1.0.0 final, so not included).
distribution pom (not initially shown in question) identifies the exact version to be deployed/included in a specific release.
delete all project -SNAPSHOTS from local maven repository when releasing so that ranges pickup releases only ( or use -Dmaven.repo.local=/tmp/sometemprepo for a fresh local repo)
This makes each module more standalone and gives us the freedom to release and deploy new versions of our project artifacts with minimal fuss.
They certainly seem like separate modules. What benefits are you gaining by smashing them together if they have different dependencies, even within the multi-module project?