We Keep Coding

Integration testing of Maven Plugin submodule

I have Maven project with multiple submodules. One of these submodules, let's say submodule X depends on all other submodules, as it is Maven Plugin that integrates everything else.
Now I want to use this Maven Plugin during verify stage in my parent POM to do real run of it over entire project (kind of eating own dogfood).
Unfortunatelly I'm getting error:
The projects in the reactor contain a cyclic reference
So how can I make such dogfood integration test for a Maven Plugin submodule?

I would review aggregation and inheritance: they are indeed two different concepts in Maven often used in combination but which can however be used in a complete separated manner as well.
In this case, your plugin submodule is certainly defined as a module in the aggregator pom (that is, it is a submodule). But I also presume the plugin submodule also has as parent pom the aggregator pom as well (that is, the aggregator pom is also the parent pom, which is a normal approach, but not always required).
Is the latter required? You could keep on having the multi-module/aggregator approach without necessarely having the aggregator as parent of the plugin submodule. As such, the plugin subModule would still be a module of the aggregator pom but it would not have as a parent the aggregator pom, decoupling it from it and as such breaking the cyclic dependencies.
Possible drawback: in the aggregator pom you were also defining common things (dependencies management, properties) required also in the plugin submodule. If the case, you would then need to duplicate/review these common settings only for the concerned submodule.

Small suggestion from my side. Cyclic means you have some transitive dependency in your modules. Can please check the dependency tree of your modules and resolve the circular dependency first.

How to update version number in Maven POMs

We have a very modular Maven pom set up, with common jars and specific jars all being bundled in to a war and ear. As there is so much reuse amongst the 70+ modules we do not use multi-module and each module can and does have its own lifecycle and can be released independently of any other module.
All modules inherit from various parent poms and ultimately every pom inherits from a master POM where all the external versions such as spring and the common local module versions are defined.
This works ok until we come to do a release. If the master POM requires a change, which it does occasionally, ALL poms need to be updated one way or another. I am aware that the maven versions plugin can update a specific POM with the latest SNAPSHOT versions etc, but this only works at the individual POM level.
We would like to be able to alter all poms once a release has been completed, iteratively.
We do not use multi-module POMs and are not able to change our build process to use this mechanism.
I have read around SO and the nearest to the problem is here.
https://stackoverflow.com/a/3615417/1279002
Writing a shell script seems to be one solution, but we have a Windows and Linux mix of development and build systems. I am sure other will have solved this issue. Can anybody advise how they have solved this?

In a similar setup, I have all my parent POMs always stay at 1.0.0-SNAPSHOT and setup various properties in the parent POMs to track internal module version numbers (so this setup now centralizes both dependency management versions AND custom module versions [via properties] into the parent POMs).
So if I need to update the reference to some com.myco:module-x, I can do this:
Edit the appropriate parent POM and set the <module-x.version>1.2.3</module-x.version> property to the new value
Rebuild/install the parent POM
Rebuild the target end-application (ear, war, jar app etc).
Where in module-x's POM it's definition may be something like this:
<groupId>com.myco</groupId>
<artifactId>module-x</artifactId>
<version>${module-x.version}</version>
And any POMs which reference com.myco:module-x refernece it via ${module-x.version} as well.
At this point, the build of the application will pickup the changes in the parent POM and thus any references it has to any properties defined in the parent POM.
There's some subtle nuances in doing this in terms of when/how the "middle man" modules need to get rebuilt...
But I really don't believe there is any silver bullet here.
The approach we've taken works pretty well, coupled with Jenkins to automate rebuilds of modules with interdependencies whenever parent POMs change.
The benefit here is that you seldom need to modify anything but the parent POMs, ever. The middle-man modules and application POMs don't need to be updated to get new version numbers, etc.
The biggest caveat though is that two rebuilds of a given module at the same version could result in a different artifact, for example:
module-x has a dependency on module-y:1.2.3
module-x is built (jar is created with a MANIFEST referencing module-y:1.2.3)
parent POM is modified to set <module-y.version>1.2.4</module-y.version>
module-y is rebuilt to create the 1.2.4 artifact
module-x is built (jar is created with a MANIFEST referencing module-y:1.2.4)
But note that #2 and #5 both built module-x with the same version for module-x, but with two different embedded MANIFEST's referencing different module-y versions.
We overcome this nuance by automating all the dependent modules with our Jenkins CI server

Maven parent-child relationship

I have encountered the below "pattern" for maven parent-child relationships:
http://yuml.me/3f8dd366
In this example we have a module with 2 sub modules. The module has a parent pom "Parent for building the Module" which knows the two sub-modules as it's children.
The sub-modules how ever have no idea that this parent knows them, and they think their parent is the one named "Parent for dependency management". Which has common configurations like dependency management, plugin configuration, common properties etc.
My question:
Is this a "good" pattern? Meaning does it have advantages/disadvantages as the seemingly more intuitive patter of child<->parent relationship

One interesting thing to look at is the aggregator pom.
It's a pom that groups project by module, without having a "parent-child" relationship. The aggregator pom doesn't have dependencies management. It only manages the build.
Having both parent(s) pom and aggregator pom is quite a powerful feature of maven.
You can find more information here.
This maven page also has valuable insight on how to set up pom for complex projects.

Hum... I think I disagree with the terms on your picture. Here is how I see this:
This way of organizing modules can be confusing for lot of developpers, but it is a legal way of doing things.
Anyway, I don't recommend this approach because it's confusing. But sometimes, their is no alternatives.
When using this configuration ?
One (or more) of the sub module already have a parent (i.e. was developed in another project but you need to rebuild it). Note that the <module> entry in multi-module project is a relative path so you can have something like:
<modules>
<module>../../somedir/othermodule</module>
...
</modules>
When possible, I recommend to use the multi-module also as the parent because:
less confusing
clean and readable <modules> and <parent> sections (no need to use ugly relative paths to specify the parent or the submodules)
you can organize your modules in a clean hierarchical structure under SCM (so that maven-release-plugin will be happy) (I know that eclipse don't like hierarchical projects, but that's another problem)

This is actually good to have a company wide configuration pom which doesn't care about it's children and contains stuff like properties, dependecyManagement, repositories, pluginManagement etc. which all of them will inherit
Maybe this can help you: Company wide parent pom

How to restructure Maven multi-module project?

I appologize for the length of this post, but I had trouble making it more concise without presenting the picture. I've recently inherited the job of build-master for a maven 3.0 multi-module project. The problem is that the structure of the project/modules is a disaster. From the way things are currently stored in Source Control (we use RTC) to the pom structure of the modules, I'm tearing my hair out trying to get a full build cycle completed each time.
As a project hierarchy goes, all the modules are stored "flat"; ie: everything is at the same level. I have a parent pom, and all modules depend on the parent. However, the parent is at the same level as all my other modules.
Ex:
c:\dev\MyEarProject
+ parent-pom
- pom.xml
+ module1
- pom.xml (depends on parent-pom)
- src
- main
- ...
+ module2
- pom.xml (depends on parent-pom)
- src
- main
- ...
+ module3
- pom.xml (depends on parent-pom)
- src
- main
- ...
The parent pom defines all the modules required to build the project, as well as a bunch of properties for artifact version numbers used throughout the different submodules:
<modules>
<module>../module1</module>
<module>../module2</module>
<module>../module3</module>
</modules>
<properties>
<org.springframework.version>3.0.5.RELEASE</org.springframework.version>
<slf4j.version>1.6.4</slf4j.version>
<repositoryAddress>${snapshots.repo.url}</repositoryAddress>
<my.hibernate-module.dao.impl>1.2.3</my.hibernate-module.dao.impl>
<my.hibernate-module.dao.api>1.2.3</my.hibernate-module.dao.api>
</properties>
Each module's pom, in turn, depends on the parent pom via the pom's artifact number:
<parent>
<groupId>com.cws.cs.lendingsimulationservice</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.6</version>
</parent>
To make things even more confusing, the actual artifact name may, or may not (depending on the module), match the module path. For example, module1 may be located in path c:\dev\MyEarProject\module1 but have artifact name hibernate-module. However, due to the way it is stored in RTC, the directory is called module1 when it is checked-out.
The easiest way to build everything, of course, is to go into c:\dev\MyEarProject\parent-pom\ and run mvn clean deploy. This works fine when in SNAPSHOT mode as the SNAPSHOT repo allows for multiple deployments of the same artifact version. But in release mode, this fails.
This structure is causing 2 problems for me.
Everytime I need to make a version change to a property in the parent, I have to update the parent-pom version number, and all the child modules parent pom's version, and all the child modules version themselves (since the parent changed).
Whenever I need to deploy a release cycle, mvn will throw an error if one of the modules has not changed since the last cycle and consequently cannot be redeployed to the same repo (the repo does not allow overwriting existing artifacts)
So I'm looking for the best way to restructure this project to avoid these problems. For the parent pom, I know I can use a relative path to point to the parent instead. However, given the "flat" structure of the modules, is this a recommended approach (ie: the parent pom relative path would be ../parent-pom/pom.xml - seems a little odd to me)? Additionally, given that the versioning control of the parent is independent of the modules, would using a relative path not just open the door to additional confusion (ie: there would be no way to know which version of the parent pom is associated with which version of the submodule).
Secondly, how can I build the entire ear without encountering the deploy errors I am having? Since the artifact already exists in the repo, I don't need to rebuild and redeploy it. I tried using --projects but with the number of modules involved, it gets extremely difficult to manage.

The first thing I really recommend is to restructure the projects folders ...which means to have the projects folder represent the structure which means NOT flatten the structure.
+-- parent-pom (pom.xml)
+--- module1 (pom.xml)
+--- module2 (pom.xml)
+--- module3 (pom.xml)
As a result of that the modules section your parent will be simplified like this:
<modules>
<module>module1</module>
<module>module2</module>
<module>module3</module>
</modules>
Furthermore the parent entries in your modules can be simplified as well like this:
<parent>
<groupId>com.cws.cs.lendingsimulationservice</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.6</version>
</parent>
...which brings me to the next point:
If all your current project define their parent as above this is simply wrong, cause will try to find the parent within the repository and not in a upper level folder. In other words this is causing of much of your problems with releasing etc.
If we would fix this problem it has to look like this which I can't recommend:
<parent>
<groupId>com.cws.cs.lendingsimulationservice</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.6</version>
<relativePath>../parent-pom/pom.xml</relativePath>
</parent>
An other thing which I observe is that you don't use SNAPTSHOT's which will be replaced by the release plugin during the release phase. And in relationship to that it will automatically change all versions in the appropriate parents etc.
In ideal case your modules should look like this:
<parent>
<groupId>com.cws.cs.lendingsimulationservice</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.6</version>
</parent>
<artifactId>module-1</artifactId>
<!-- No Version or groupId -->
Cause all modules will inherit the version and groupId from their parent. Sometimes it's useful or needed to change a modules groupId but it's an exception.
On thing I reread is about the separate versioning of the parent. This simply does not make sense, cause it's the parent of their modules so put it into the same structure and of course the same VCS.
If you want to make some configuration/plugins versions, dependencies which should be used for other projects as well than make a separate corporate pom.xml which is a separate project and will be separately released etc.
After you finished your structure changes you can simply go into the parent-pom directory and do mvn clean package or mvn release:prepare release:perform from that folder and everything will simpler.

If you're publishing your POM, you'll have to release any updates but you don't need to modify POM versions by hand - you can update versions automatically using the versions plugin or the release plugin. I tend to prefer the release plugin as it'll commit to SCM for you too.
mvn versions:set
http://mojo.codehaus.org/versions-maven-plugin/
mvn release:prepare release:perform
http://maven.apache.org/plugins/maven-release-plugin/
Your repository manager may also allow overwriting an existing version but it's better practice to just release a new version.
I tend to prefer the flat module structure as it allows use of the parent folder to store common files e.g. checkstyle configuration. I also find it useful to share the group id across modules then name the module directory the same as the artifactId.

You are presenting contradicting requirements. You want to restructure your project but can't move things around. You want to simplify your deployment and release cycles but do not want to use a single version.
Given that changes in one module will inevitably affect all of the dependent modules, I would use a simple version'ing scheme where all the sub-modules inherit their parent's version. maven release:prepare and release cycles become simple. Use a release notes to track your changes and justify skipping unnecessary testing of unchanged modules (changes to a version do not change the build/binary output of the build process so you can use that as your primary argument).
Good luck with your project.

How do I properly setup a multi-module Maven project with sliding release cycles

I'm trying to work out the best way to setup our multi-module Apache Maven project in a way that allows for disparate release cycles of modules, and doesn't introduce dependency issues when debugging the project.
We currently have a setup along the lines of:
bigsystem#1.2
parent-1.1-SNAPSHOT
module a#1.4-SNAPSHOT
parented by parent#1.1-SNAPSHOT
module b#1.3-SNAPSHOT
parented by parent#1.1-SNAPSHOT
depends on a#1.1
module c#1.1-SNAPSHOT
parented by parent#1.1-SNAPSHOT
depends on a#1.2
depends on b#1.1
The dependencies declared in modules b and c contain the minimum version required to compile the module, which isn't necessarily the current version of the module, or the version of the module being deployed.
From a build perspective this works well, each module can be released/updated as needed, however when trying to debug the deployed application under IntelliJ IDEA (versions 8 and 9 EAPs) having opened the top level pom, IDEA decides that since we declared a dependency on a#1.2, that anytime we step into one of a's classes, it should open it from a-1.2-sources.jar rather than the current a#1.4 sources in the project. This is further confused by the fact that stepping into any of b's classes takes us to b#1.1 rather than b#1.3.
My initial attempt to work around this was to declare the version numbers in the parent pom's dependencyManagement section and just have the sub-modules inherit the version. This worked to the degree of solving the IDEA debug issue as the dependencyManagement section can point everyone to the current -SNAPSHOT versions.
This unfortunately causes a problem when doing a maven release due to having to release the parent pom before releasing the module, but as the parent may refer to multiple in-development -SNAPSHOTS it can't be released and we end up adding version references back to the modules pom to satisfy the release.
It would seem that using maven's dependencyManagement section would only really work well if we were releasing ALL bundles at the same time, regardless of if they changed, but as we're wanting to manage releases of each sub module only when needed this model doesn't seem to fit.
I have a suspicion I'm missing something, and that a combination of dependencyManagement and version ranges might satisfy out requirements although I've yet to see version ranges work properly.
Is there a better way? A proper way?

I would recommend not making them modules, but make their POMs independent. That way you do not have to worry about trying to satisfy parent POM dependencies. Since they are released independently, they really should have independent project object models. Think of Apache Commons as a template.

I think the problem with IDEA arises because you are using the root POM in your source structure to do two things that are usually mutually exclusive in Maven. You are first using the POM as a location to store common configuration information for unrelated (from a build perspective) Maven projects. Secondly you are using the POM as an aggregator for your build. You can do each of these without doing the other.
Like Rob said, remove your module a, b, etc. projects from the modules section of your parent POM. Secondly, move your parent POM down into its own directory as it is really a sibling of the other modules with respect to your build and release process. The way you have it now, it is more of a parent/aggregator.
The way you have it now also doesn't lend itself to tagging and releasing each module individually as a tag of your parent POM would likely needlessly include all of the module sub-folders.
Your file structure would look like:
parent
pom.xml
module a
pom.xml
module X
pom.xml
As for the thing you are missing, dependencyManagement isn't really well suited to manage versions for intra-project dependencies. That is dependencies between modules within an aggregated build. It is more well suited for declaring global versions for external dependencies.

The final/working solution we ended up using was fairly similar to what we started with. The actual project structure remains the same:
bigsystem#1.2
parent-1.1-SNAPSHOT
module a#1.4-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
module b#1.3-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
o depends on a#1.1
module c#1.1-SNAPSHOT
o parented by parent#1.1-SNAPSHOT
o depends on a#1.2
o depends on b#1.1
distribution a#1.2-SNAPSHOP
However the main differences are that:
parent module does not include any versions of project artifacts
individual modules fully declare their project dependencies and specify a version range, i.e. [1.0.0,1.1.0)
all modules start there version number cycles from .1, i.e 1.0.1-SNAPSHOT, this allows the version range to satisfied by initial snapshots (1.0.0-SNAPSHOT is earlier than 1.0.0 final, so not included).
distribution pom (not initially shown in question) identifies the exact version to be deployed/included in a specific release.
delete all project -SNAPSHOTS from local maven repository when releasing so that ranges pickup releases only ( or use -Dmaven.repo.local=/tmp/sometemprepo for a fresh local repo)
This makes each module more standalone and gives us the freedom to release and deploy new versions of our project artifacts with minimal fuss.

They certainly seem like separate modules. What benefits are you gaining by smashing them together if they have different dependencies, even within the multi-module project?