I'm trying to organise my maven project.
Let's say my project is called "awesome". "awesome" has several artifact, each of them built differently (e.g., some of them may be built with some plugin, others are built with some other plugins): in general thse build-configurations are finite and limited (let's say there are at most 3 different ways to build an artifact), however, each artifact can only be built with exactly one build (e.g., the utility artifact is built with maven-jar-plugin configured in a particular way, while the artifact client-ui is built with maven-war-plugin configured in a particular way).
Now, I know I could organize the maven project as follows:
awesome-root
|---jars
| |--- utility
| |--- client-model
| |--- task-model
| |--- supplier-model
| |--- client-logic
| |--- task-logic
| ---- supplier-logic
|---wars
|--- client-ui
|--- task-ui
---- supplier-ui
This way, each particular configuration build can be put inside the build --> plugins section of the projects jars and wars, while general properties/dependency management/plugin management can be put in awesome-root.
Problem:
I quickly realized that the developers generates artifacts closely related with eachother but with different builds. In the previous example, we can notice that the artifacts can be grouped in this other way:
awesome-root
|--- tasks
| |--- task-model
| |--- task-logic
| ---- task-ui
|--- clients
| |--- client-model
| |--- client-logic
| ---- client-ui
|--- supplier
| |--- supplier-model
| |--- supplier-logic
| ---- supplier-ui
|--- others
|--- utility
The main advantage of this grouping is that tasks, clients and suppliers are 3 different, independent software sectors. When the developer needs to make a change in the, let's say, client sectors, she has everything she needs in a small part of the file system (or in the project explorer tab in an IDE, like Eclipse). Viceversa, in the first mapping, the clients software sector is scrambled all over in the project repository.
While this may not be a big deal, if "awesome" project starts to get really big, with a lot of artifacts and so on, finding all the related parts of clients sectors start to be annoying (not impossible, IDEs offer searches for this purpose).
I'd say the second structure is much better, developer wise.
However, It's seems difficult to implement this strategy in maven: the main difficulty is to where to put the different build configurations for each artifacts (e.g., *-ui needs to be built in a different way of *-model).
One may be tempted to put such configurations in client-ui, client-logic, client-model, but this would mean duplicate configuration build everywhere (e.g, client-ui, supplier-ui, task-ui has the same build configuration): if a build configuration needs to be changed, you need to change all the other copies;
Another solution might be to declare plugins management in awesome-root and them write the plugin definition in each artifactId: while this seems better, it still suffer from the same duplication problem of option 1;
Use archetype to generate poms with the correct build configuration: same as above;
Profiles: profiles are not inherited and they depend only on system properties, not maven's one;
My questions are:
Is the second structure impossible to achieve in Maven? Is there a way?
If not, do I need to bite the bullet and set on the first structure?
Is there any alternative? (I'm trying not to propose a XY problem, any alternative is appreciated);
Additional information:
OS: Ubuntu 18.04.3 (bionic), 64 bit
java version: openjdk 11.0.4 2019-07-16
IDE: Eclipse 4.10.0
m2e plugin: 1.10.0.20181127-2120
Thanks for any kind reply
Related
I'm building an Automation Tool using Java Automation Framework under the hood, to which users will not have access to the main POM.xml file. Sometimes users require to add a custom Java function which requires additional depedencies / repositories. Presently I have to make changes to the main POM file to accommodate the user request. User has access only to "src/test/java/com/script/custom" folder to write custom scripts / functions. I have explored options like Parent/Child POM, Plugin Management, Profile, etc. but examples are mainly for multiple projects. I'm a NodeJs/Angular person, so I'm a beginner at Java.
Project
|
|--src/test/java/com/script/custom
| |
| custom_code.java
| |
| custom_pom.xml
|
--pom.xml
Users should only enter additional dependencies / repos in custom_pom.xml. Parent pom.xml will still hold the main dependencies/repos of the project.
Running code (apart from tests) is against the core concept of Maven as a build tool. There are ways, however, to excute arbitrary code at build time:
Exec Maven Plugin
without an additional plugin (and with cleanly separated projects):
+- project
| +- pom.xml
+- custom
+- src/main/java/com/script/custom
| +- CustomCode.java ... convention for Java class names is CamelCase
+- src/test/java/com/script/custom
| +- CustomCodeTest.java ... instantiates and runs CustomCode
+- pom.xml ... containing <parent><relativePath>../project
For <parent> see Introduction to the POM #Project Inheritance. See also Maven: Lifecycle vs. Phase vs. Plugin vs. Goal for further basics.
I inherited a .Net/Java Project combo, in which there are different "modules" some modules are dependent on others, some are independent. It is not a true multi module project (No aggregrator POM).
I am using Intellij (community) to compile/debug these and I am unable to do so, Earlier these were made using Eclipse (.classpath , .project) and now the team (Management) wants to move to Intellij and VSTS CI.
Some more Info
Web application is packed as war and deployed to Tomcat using WIX on windows
Application some custom jars (no source available).
I have been asked to avoid using local repositories.
Multi module Project on Maven website / Github do not compile/ wants
me to use mvn clean install for every dependent java project
ALL java projects are maven projects with POM.XML
Questions :
How to create a Multi Module Project using this structure (Which can
also be imported in intellij/Eclipse).
How to compile the resulting multi module project with Maven/Intellij compile/debug .
Is Gradle more suitable here.
As a new Java Programmer , Its gets really difficult to change and deploy an already existing setup when you cannot connect/talk to the person who made it :( .
Any help/direction/suggestions will be really helpful.
Project Structure (JAVA)
C:\SomeProj\
├── Jars
| ├── Java
| | └── extJar2.jar
| | └── extJar3.jar
| | └── extJar4.jar
| | └── extJar5.jar
| | └── javax.servlet-3.0.jar (Why is this externally required)
| └── SomeFolder1
| | └── extJar1.jar
├── Source
| ├── Common
| | ├── DAL (JAVA - jar) [Depends On Logger]
| | ├── Logger (JAVA - Jar)
| ├── API
| | ├── DataInput
| | | ├── InfoPuller (JAVA - Jar) [Depends On Logger]
| | ├── Info
| | | └── MyWebApi (JAVA-WAR) [Depends On DAL and Logger]{Application to Build/Run/Deploy from Intellij/MAVEN}
| | └── Util
| | └── JobManager (JAVA - Jar) [Depends On Logger]
| └── WebApp (Js App) {FrontEnd}
I think your best bet is to create an aggregator project, in the most standard way possible (for example following the recommendations from here : https://maven.apache.org/guides/mini/guide-multiple-modules.html);
If you're new to maven, it might look scary, but this can be as little as a single file (which can even be in a subfolder in one of your projects, although I don't recommend this, and suggest to have instead its own project). Having an aggregator will help a lot with modules that depend on other modules, etc.
The resulting aggregated project can easily be imported in Eclipse/IntelliJ (Eclipse for example has a feature where it knows to import/create project files out of a pom.xml; IntelliJ has a similar feature).
The resulting project (if you set it up correctly) can be compiled/packaged via the default maven lifecycle, either with
mvn compile or mvn package. Obviously you can also run stuff via IDE, but you might need a little spadework (setting up runtimes/servers, etc).
For example I build my projects for CI via maven, but I am also running them via eclipse, for development purposes. I'm using my pom.xmls as the Single Source of Truth.
You've just described what I believe to be a classic use-case for multi-module projects; You're asking for an opinionated answer which in the end boils down to "it depends what you prefer". I personally don't know Gradle, but I don't see elements in your description that look strange. So I think you can use Gradle if members of your team have better knowledge of Gradle vs Maven, etc.
Additional things to consider:
On Application custom jars: If they do not exist in Maven Central repository, or in the provider's repository somewhere, and if you want to avoid local repositories, you can include a repository along with your project; this is basically just a flavor of local maven repository, with the distinct advantage that it can be packaged along with your projects, and other devs don't need to execute additional steps (i.e. they don't need to execute an extra step of adding artifacts to local repo). Some people don't like this approach, but if your constraints of "no local repo; no company repo" are unmovable, then I don't see any other way
I don't understand very well what you say here:
Multi module Project on Maven website / Github do not compile/ wants
me to use mvn clean install for every dependent java project.
I am going to assume that you mean that you've found some multi-module sample somewhere and it didn't worked for you. Without a concrete example, I can't really comment, except to mention that I have used multi-module builds in the past (and in the present), and they work fine for me.
Regarding your comment about mvn clean install - that's a maven command that install your project's/module's artifacts into local repository, and is not REALLY needed, unless these artifacts need to be consumed by projects external to your current multi-module build.
I am working on my little OSS project in which I am using Maven as build tool. I split the project into smaller sub-projects to simplify development. Thus I have following structure:
project
+-- main-module
| |
| +- pom.xml
|
+-- submodule1
| |
| +- pom.xml
|
+ pom.xml
My thought was that main-module should provide interfaces which each submodule should be implementing in order to be plugged into whole application. Therefore submodule1/pom.xml contains compile time dependency reference to main-module. In its turn I also need to be able to test whole application and thus main-module/pom.xml contains test scope dependency reference to submodule1. As the result maven refuses to compile projects saying that they contain cyclic references.
My thought was that maven could first compile classes of main-module as it does not require any compile time dependency on any of submodules, then it using compiled classes of main-module could compile classes of submodule1 and after that compile test classes of main-module (to be able run tests). But seems that maven compiler does not take in account the scope of dependency and I somehow need to work around that.
The only solution I can see is to move away tests from main-module, which doesn't really make sense for me as only that module provides main logic.
My question - is there any other way around this issue except for moving away tests? Or maybe something is wrong with my understanding of how maven-reactor-plugin should work?
Instead of moving your tests away, you could move all of your API into it's own module. Then your main module would contain the application and you can freely distribute your application's API to allow others to access it. If they want to develop new functionality they do not necessarily need the sources of your app.
I consider this a much better style, because sub modules with specific functionality can now clearly separate between what is your applications API and what is the code your application needs to startup/shutdown, etc.
This is in my mind the intended way of how maven projects should look like. It also sticks with the single responsibility principle. The main modules responsibility is to startup/shutdown, etc your application. The API modules responsibility is to show how other developers can access your application. And the other submodule provide specific functionality for your application.
I know this is more of a comment, but might provide you with (not so pretty) solution:
You can seet your submodule1 as a dependency of maven-surefire-plugin (so that reactor is forced to build it) and then play with its settings... i.e. childDelegation or additionalClasspathElements
I have a multi-module maven project structured in something like this:
parent
|
|-presentation
|+services
| |-services-api
| |-services-impl
|+data-access
| |-data-access-api
| |-data-access-impl
|-+connector
| |-connector-api
| |-connector-implA
| |-connector-implB
|-...
The presentation module is packaged in a war and it depends only on the api modules.
When i run the install goal the only dependencies that the war installs are the api modules. To choose wich impl modules to install in the presentation module i'm using profiles that add the dependency to the impl modules at build time depending on the profiles selected.
From what i've been reading i don't think that this is correct usage for the maven profiles.
What is the best way to tell maven to add a chosen impl to the presentation module?
I have the same usage of profiles but only for specific changes (dependencies mostly).
You do not have to put everything in profiles. Most of the implementation dependencies are common and are therefore declare directly without profiles.
Depending on the targeted application server I use profiles to override properties, add specific dependencies (CommonJ for Websphere for instance), ...
I got a solution from the maven users mailing list that i think is the right way to use maven in my scenario.
I use runtime dependencies for the impl modules and one war project for each implementation of the api. Using war overlays it merges the resources and enables me to have the application running with the correct module implementations depending on the war i run.
There seem to be several ways to structure parent poms in a multiproject build and I wondering if anyone had any thoughts on what the advantages / drawbacks are in each way.
The simplest method of having a parent pom would be putting it in the root of a project i.e.
myproject/
myproject-core/
myproject-api/
myproject-app/
pom.xml
where the pom.xml is both the parent project as well as describes the -core -api and -app modules
The next method is to separate out the parent into its own subdirectory as in
myproject/
mypoject-parent/
pom.xml
myproject-core/
myproject-api/
myproject-app/
Where the parent pom still contains the modules but they're relative, e.g. ../myproject-core
Finally, there's the option where the module definition and the parent are separated as in
myproject/
mypoject-parent/
pom.xml
myproject-core/
myproject-api/
myproject-app/
pom.xml
Where the parent pom contains any "shared" configuration (dependencyManagement, properties etc.) and the myproject/pom.xml contains the list of modules.
The intention is to be scalable to a large scale build so should be scalable to a large number of projects and artifacts.
A few bonus questions:
Where is the best place to define the various shared configuration as in source control, deployment directories, common plugins etc. (I'm assuming the parent but I've often been bitten by this and they've ended up in each project rather than a common one).
How do the maven-release plugin, hudson and nexus deal with how you set up your multi-projects (possibly a giant question, it's more if anyone has been caught out when by how a multi-project build has been set up)?
Edit: Each of the sub projects have their own pom.xml, I've left it out to keep it terse.
In my opinion, to answer this question, you need to think in terms of project life cycle and version control. In other words, does the parent pom have its own life cycle i.e. can it be released separately of the other modules or not?
If the answer is yes (and this is the case of most projects that have been mentioned in the question or in comments), then the parent pom needs his own module from a VCS and from a Maven point of view and you'll end up with something like this at the VCS level:
root
|-- parent-pom
| |-- branches
| |-- tags
| `-- trunk
| `-- pom.xml
`-- projectA
|-- branches
|-- tags
`-- trunk
|-- module1
| `-- pom.xml
|-- moduleN
| `-- pom.xml
`-- pom.xml
This makes the checkout a bit painful and a common way to deal with that is to use svn:externals. For example, add a trunks directory:
root
|-- parent-pom
| |-- branches
| |-- tags
| `-- trunk
| `-- pom.xml
|-- projectA
| |-- branches
| |-- tags
| `-- trunk
| |-- module1
| | `-- pom.xml
| |-- moduleN
| | `-- pom.xml
| `-- pom.xml
`-- trunks
With the following externals definition:
parent-pom http://host/svn/parent-pom/trunk
projectA http://host/svn/projectA/trunk
A checkout of trunks would then result in the following local structure (pattern #2):
root/
parent-pom/
pom.xml
projectA/
Optionally, you can even add a pom.xml in the trunks directory:
root
|-- parent-pom
| |-- branches
| |-- tags
| `-- trunk
| `-- pom.xml
|-- projectA
| |-- branches
| |-- tags
| `-- trunk
| |-- module1
| | `-- pom.xml
| |-- moduleN
| | `-- pom.xml
| `-- pom.xml
`-- trunks
`-- pom.xml
This pom.xml is a kind of "fake" pom: it is never released, it doesn't contain a real version since this file is never released, it only contains a list of modules. With this file, a checkout would result in this structure (pattern #3):
root/
parent-pom/
pom.xml
projectA/
pom.xml
This "hack" allows to launch of a reactor build from the root after a checkout and make things even more handy. Actually, this is how I like to setup maven projects and a VCS repository for large builds: it just works, it scales well, it gives all the flexibility you may need.
If the answer is no (back to the initial question), then I think you can live with pattern #1 (do the simplest thing that could possibly work).
Now, about the bonus questions:
Where is the best place to define the various shared configuration as in source control, deployment directories, common plugins etc. (I'm assuming the parent but I've often been bitten by this and they've ended up in each project rather than a common one).
Honestly, I don't know how to not give a general answer here (like "use the level at which you think it makes sense to mutualize things"). And anyway, child poms can always override inherited settings.
How do the maven-release plugin, hudson and nexus deal with how you set up your multi-projects (possibly a giant question, it's more if anyone has been caught out when by how a multi-project build has been set up)?
The setup I use works well, nothing particular to mention.
Actually, I wonder how the maven-release-plugin deals with pattern #1 (especially with the <parent> section since you can't have SNAPSHOT dependencies at release time). This sounds like a chicken or egg problem but I just can't remember if it works and was too lazy to test it.
From my experience and Maven best practices there are two kinds of "parent poms"
"company" parent pom - this pom contains your company specific information and configuration that inherit every pom and doesn't need to be copied. These informations are:
repositories
distribution managment sections
common plugins configurations (like maven-compiler-plugin source and target versions)
organization, developers, etc
Preparing this parent pom need to be done with caution, because all your company poms will inherit from it, so this pom have to be mature and stable (releasing a version of parent pom should not affect to release all your company projects!)
second kind of parent pom is a multimodule parent. I prefer your first solution - this is a default maven convention for multi module projects, very often represents VCS code structure
The intention is to be scalable to a large scale build so should be scalable to a large number of projects and artifacts.
Mutliprojects have structure of trees - so you aren't arrown down to one level of parent pom. Try to find a suitable project struture for your needs - a classic exmample is how to disrtibute mutimodule projects
distibution/
documentation/
myproject/
myproject-core/
myproject-api/
myproject-app/
pom.xml
pom.xml
A few bonus questions:
Where is the best place to define the various shared configuration as in source control, deployment directories, common plugins etc. (I'm assuming the parent but I've often been bitten by this and they've ended up in each project rather than a common one).
This configuration has to be wisely splitted into a "company" parent pom and project parent pom(s). Things related to all you project go to "company" parent and this related to current project go to project one's.
How do the maven-release plugin, hudson and nexus deal with how you set up your multi-projects (possibly a giant question, it's more if anyone has been caught out when by how a multi-project build has been set up)?
Company parent pom have to be released first. For multiprojects standard rules applies. CI server need to know all to build the project correctly.
An independent parent is the best practice for sharing configuration and options across otherwise uncoupled components. Apache has a parent pom project to share legal notices and some common packaging options.
If your top-level project has real work in it, such as aggregating javadoc or packaging a release, then you will have conflicts between the settings needed to do that work and the settings you want to share out via parent. A parent-only project avoids that.
A common pattern (ignoring #1 for the moment) is have the projects-with-code use a parent project as their parent, and have it use the top-level as a parent. This allows core things to be shared by all, but avoids the problem described in #2.
The site plugin will get very confused if the parent structure is not the same as the directory structure. If you want to build an aggregate site, you'll need to do some fiddling to get around this.
Apache CXF is an example the pattern in #2.
There is one little catch with the third approach. Since aggregate POMs (myproject/pom.xml) usually don't have parent at all, they do not share configuration. That means all those aggregate POMs will have only default repositories.
That is not a problem if you only use plugins from Central, however, this will fail if you run plugin using the plugin:goal format from your internal repository. For example, you can have foo-maven-plugin with the groupId of org.example providing goal generate-foo. If you try to run it from the project root using command like mvn org.example:foo-maven-plugin:generate-foo, it will fail to run on the aggregate modules (see compatibility note).
Several solutions are possible:
Deploy plugin to the Maven Central (not always possible).
Specify repository section in all of your aggregate POMs (breaks DRY principle).
Have this internal repository configured in the settings.xml (either in local settings at ~/.m2/settings.xml or in the global settings at /conf/settings.xml). Will make build fail without those settings.xml (could be OK for large in-house projects that are never supposed to be built outside of the company).
Use the parent with repositories settings in your aggregate POMs (could be too many parent POMs?).