I have four projects, which have several modules in common.
When I try to run a SonarQube complete analysis, only the first project gets successfully analyzed. Starting with the second project, I get an error message along the lines of "XXX module is already present in the server".
Each of my four projects contains an aggregator POM. This is what I use to build them and to run the Maven sonar:sonar goal.
What would be the best approach to solve this issue?
Select a master project in which to analyze the shared modules; analyze that project first.
In the other (subsequent) projects, exclude the shared modules from SonarQube analysis by adding the following to the pom.xml of the shared modules:
<properties>
<sonar.skip>true</sonar.skip>
</properties>
If by shared modules, you mean that the module is present in both Project A and Project B then yes, It will create a problem because SONAR only allows one instance of a Module to be present.
What you can do is run SONAR on your Project 1 then clear the workspace and again run it for Project 2.. and So on...
That way you would have only one instance of a module in SONAR at a time.
you can probably write a script for executing the SONAR jobs one by one and create a new job for that in Jenkins.
It seems you're breaking component/module/library conventions or mis used it. From Configuration Management principles you're not supposed to "include" (build in the case of Binary Configuration Management) your component multiple times.
Having a library which have many other ones which depends on and last ones are independant is just a dependancy that you have to manage appart like any other framework.
You're just supposed to install/deploy this component on his own and then just refer to it. Same principles apply on SonarQube Configuration Manager. You can't have shared components analyzed through different "products".
They must have their own life cycle. How do you manage evolution of shared librairies when having the four projects living in parallel of each others ?
Related
Never found a really satisfactory solution to this. How do you do it? I am looking for inspiration for new approaches.
For context, assume I write a generator that takes a project resource and generates a code file. But it could be any other project support tool - validator, converter, deployer etc. Often manually triggered actions that are not running as part of normal build.
Such tools typically require a few dependencies that are not required by the project itself at runtime.
Strategies that I have applied or considered in the past:
add tool dependency to project anyway, and either mark it "provided" or filter it out during the packaging process (this is what I usually do, but now I am in danger of adding normal project code that uses the tool dependency, potentially resulting in an error that only manifests during runtime)
use a script (trying hard to avoid scripts and their hidden dependencies and complexities)
create separate support projects (trying hard to avoid project explosion, especially for seemingly small tasks that are handled by a few lines of code)
subprojects / modules (only vaguely aware of this option, never really tried it)
maven plugin that is run with a profile with separate dependencies (trying to avoid the separate project required to maintain the custom plugin)
Inspiration from answers and comments
separate tools project shared by multiple projects
I just realized that maven and eclipse already solved exactly this problem for a very specific "tool": test code.
Test code often needs additional dependencies not used by the application itself.
People obviously invested quite a bit to keep the "test / tool" infrastructure within the same project, as opposed to creating a separate test-project:
separate source locations (src/main/java, src/test/java)
separate resource locations (src/main/resources, src/test/resources)
a full-blown separate maven dependency scope "test", complete with transitive resolution
separate compilation phases (compile / test) with separate dependency trees
eclipse supports special junit launch configurations that are able to correctly resolve the test dependencies
probably more stuff that I am not aware of currently
So, I am strongly considering to program all my supporting tools as "junit test cases".
I am planning to create and commit shared junit launch configs for the team that execute just one specific "test case", which will run the tool logic instead of testing.
The problem I have to solve is to avoid running these dummy tests during the normal maven test phase.
Also, writing this, I realize that there is even another such system already in place: the maven plugin infrastructure, that also has a separate dependency resolution mechanism. Although, so far it seems necessary or normal to create separate projects to create plugins. I will look into ways of writing and building project specific maven plugins without needing to create separate projects. I am thinking about generating the pom.xml needed for plugin compilation on the fly, and including all the test dependencies.
I'm using rpm-maven-plugin to create an RPM that installs my java project. I need to build RPMs for both CentOS 5 and CentOS 7. I'm already using profiles to set properties that contain the names of some RPM dependencies that differ between the two, then referencing those properties in the rpm plugin.
I now need to also change what files the RPM puts in place based on the cent5 vs cent7 profiles (cent 5 still uses init scripts, cent 7 has gone to unit files with systemd). I can't think of an easy way to do that by setting properties, I need to include different file mappings in each. Do I need to duplicate my (long) RPM plugin configuration in two profiles to get the changes? That feels dangerous because the plugin configuration is long, and duplication will inevitably lead to developers making future changes in one and missing the other. Is there a way to just add to the plugin configuration with the profile?
What is the best way to manage a small change of a complicated plugin configuration across maven profiles?
Use Maven's aggregation (with sub-modules):
Inheritance and aggregation create a nice dynamic to control builds through a single, high-level POM. You will often see projects that are both parents and aggregators.
and inheritance (with a parent POM):
A parent project is one that passes its values to its children. A multimodule project simply manages a group of other subprojects or modules.
See also this answer to Maven and Profiles: ....
Well here is an interesting experience i had since last couple of weeks structuring my maven multi module project.
When i decided to use maven for my build life cycle management i had couple of reason that i wished to choose maven.
a. Mostly development teams are divided so that each team can work on separate Module within the project like Team-A to work on User Management System, Team-B to work on Authorization System, Team-C to work on Document Management System...and so on. Each team has java developers, testers, UI experts etc.
So the maven Project structure should be such that each team can independently work on their respective modules. They must be able to code, compile, build, test, deploy their module without having to compile, test modules belonging to other teams.
And thus i came to conclusion that each development module of the maven multi-module project must represent a Functional Module
After some discussions on forums i found people suggesting me to follow layered approach were child modules must be layers like controller-layer,service-layer,dao-layer etc. I did not pay heed to this advice because this not solving my purpose of teams working on individual module. This way for large project the build and deployment time for each team during development increases which does impact the project time-lines. sometimes the build and deploy time is upto 30 minutes say if there are 10 to 11 modules in the project.
But i did pay heed to a suggestion that keeping DAO layer separate for each module is not a good idea as DAO is highly granular and reused by other modules. and so the dependency of one module on other would would any how become greater.
I found a solution to this problem by creating a common module and moving DAOs and DOMAIN to the common module which will be inherited as a dependency by each module. And this seems to be a more viable option. Now the Project Structure looks like this.
Now when i build the project and run the webapp on server, It complains 404, Resource Not Found. I found that this is because the WEB-INF/classes folder is missing, src/main/java is missing in web-app module. I searched and found couple of links that suggested it is Deployment Assembly issue in Eclipse. So i need to manually create these folders and add in the deployment assembly because maven does not do it.
But the bigger questions are
do i need to move the Controller classes like com.mycompany.usermgmtsys.controller.UserMgmtController etc.. to src/main/java Or maven should find the controllers from the module jars included as dependency in WEB-INF/lib.
I dont want to do this i.e. putting java file in web-app. i want all the controllers should be available to the web-app as dependency for example WEB-INF/lib/usermgmtsystem.jar. But then wouldnt the Tomcat be looking for controllers in classes folder.
I dont know what should i do ? Any suggestions would be appreciated.
Its the way the eclipse render maven based project. It generally creates two structure. One based on master pom (parent project) and others based on individual module pom. however doing changes in any structure will reflect in the other one. As a practice I do changes in individual module folder structures and is more easy to read too.
Personally I try to avoid multi-module projects as, if you're using the Maven Release Plugin, you are locked into releasing all your modules together.
While this may sound like a convenience the problem arises when you need to do bug fix release to one of the modules - you end up releasing all the modules, not just the module with the bug fix, incrementing their version even though they haven't changed.
You also take a hit if you're running CI with multi-module projects - you're build typically runs over all modules from you root pom but if you're working in a particular module, you end up taking the hit of building those that haven't changed, in effect losing some of the benefits that the modularization was meant to provide.
So, go with independent modules but, and this is the important bit, create a common 'dependency' pom used by each.
A 'dependency' pom is a pom that standardizes all the dependencies across your projects and is different in that those dependencies are specified in the dependencyManagement section rather than the dependencies section (it also sets up standard plugin config, etc). This allows your project poms to specify the dependency pom as their parent and then declare the dependencies they need minus the versions, which are picked up from the 'dependency' pom and thus standardized across your projects.
If you are still concerned about being able to built everything, this can be achieved with a simple batch-file.
This is a good question. There are many aspects that must be considered for a useful project layout. I'd like to try to answer one which you didn't mention. Is your app extensible by users? If it is, then consider creating a separate module for your public API layer (service interfaces, DTOs used by those services, and Exceptions thrown by the services).
In our app, we have several maven modules per functional area. The idea is that a group worked on a feature within just one functional area and this isolation kept them messing with sources being modified by another group. Each functional area is broken down further in maven sub-modules we call "api", "domain", and "service" - we don't lump services/controllers, domain, and exceptions into a single module. The api module contains those classes we want to expose to customers for their customizations. Our service layer is the implementation of those interfaces. Further, we do not allow one module's service to call another module's service as this would bypass our service orchestration layer where customer can attach extensions to our services. Using separate maven modules per functional area helps enforce this.
We have other modules (internal-api, web, adapter) but they don't really add to this topic.
I figured out the issue. Controllers are presentation-layer components. The dispatcher expects the presentation layer components in the WEB-INF/classes folder in the target rather than looking for it in the lib. I am not sure if this is valid only for maven based structuring in eclipse. So finally these are the changes i have made
a. Created a src/main/java source folder in web-app. It is not generated by default in web-app module.
b. Add packages and respective controllers in the src/main/java folder.
So the final structure that i have (i am not pasting exact eclipse snapshot, this is generalized view)
In a nutshell, what I am trying to do is build a bunch of libraries and applications, all Maven projects, all at once. From what I understand a way to accomplish this just in one command line run of mvn package would be to create a multimodule project that will list each module that I would like to build, throw them in the Maven reactor, and build.
Following examples in the Maven book it seems that normally a multimodule pom sits in a directory above the individual modules. However it is also normally the case that a parent pom sits in a directory above the modules, which raises the question, is it normally the case that a multimodule build should also be a parent? I think not; however I wonder why I am running into this funny design quirk.
So, I'm wondering the right way to set this up. I see the following conventions / requirements:
The multimodule pom must have knowledge of where the other modules live on disc. Since it is actually doing the build from source it can't simply rely on already installed versions (since it's installing them!)
The parent doesn't actually have to be a physical directory up although that would be preferable. I see this as the convention best to break.
Really the individual libraries/application shouldn't even need to know they are being built as part of a multimodule build.
How is this usually set up in a multimodule build? Is there a simpler way to manage building multiple Maven projects all at once?
I put all the individual modules within the root module. Some software has trouble with multiple layers of hierarchy.
To make a child module refer to it's parent on the same level:
<parent>
<groupId>com.domain</groupId>
<artifactId>xyz</artifactId>
<version>0.0.1-SNAPSHOT</version>
<relativePath>../xyz/pom.xml</relativePath>
</parent>
I suggest you do not put anything in the multimodule (e.g. properties) that individual modules need to inherit. If you do, you won't be able to build the other modules independently.
I would go so far as to say that this is the fuzzy part of the "conventions". The documentation, and common sense, both suggest that project aggregation ( aka multi-module builds ) and inheritance are two different mechanisms provided to handle different use cases.
At the same time, it seems that there is a de facto convention ( yeah, I know ) of combining both the project aggregation and inheritance parent roles into a single pom. In fact, both the element of the parent declaration and the module element of the project aggegration mechanism seem to steer the use toward this combination.
Personally, I find it very usefull to separate the parent pom out on a regular basis. And I also find it useful ot locate parent pom in a totally separation location in my source control, and thus my folder structure. However, it rarely seems useful to locate builds that are a part of the same multi-module build structure in source control / folder structure. Perhaps this is even a good measure of whether something should be included in the same aggregate build; if it seems to deserve collocation in the source folder structure, then perhaps its a strong candidate for aggregation.
The only thin I am sure of is that these things are worth sorting out a head of time. And it's probably better to error on the side of not creating monolithic build structures . . . it's very hard to deal with a huge lump of aggregated, parent child build modules that isn't really necessary. On the other hand, aggregating individual builds to run together is a functionality provide at higher levels, such as the CI build server. So, I guess I might suggest erroring on the side of more independence.
What are the main possible reasons of breaking down a Maven project to sub-modules?
Are you looking something more than the benefits of Modularization? The sub-modules should ideally be representing a single concept/feature so that they are functionally cohesive.
Pom file inheritence
You can use the and sections of the root poms to keep consistent version numbers and configurations across all child projects. So if I have an aggregator project that lists 300 projects, all that use apache commons-io and I want to upgrade them all the the latest version of commons-io, I can just change the version in the dependencyManagement section of the root pom. None of the child projects need specify a version.
build profiles
In the above example, if I have 300 sub projects, an individual developer is probably not regularly working on all (or even many) of the 300 sub-projects. You can create a build profile that specifies only the modules you work on regularly, and if you have a continuous integration server that deploys artifacts to an artifact repository, you'll get all the changes the developers on your team make too, without having to build all 300 modules.
General organization/clarity
While waiting for an answer to my comment.
A reason to split a Java EE based maven project into sub modules is so you can build the JAR/RAR/WAR/EAR/whatever independently of eachother.
For regular Java apps, you might split out the functionality into separate JARs, again each of these could be a sub-module under the overall project and again you can build them independently, run separate goals/phases/reports etc.