Our project badly needs to move to Flexmojos4 to get a fix, but this requires Maven 3. Our project makes extensive use of Maven and we really love it, but have configured it very heavily. Between a dozen modules we probably have 50+ pages of XML configuration.
We also use Eclipse and make heavy use of the M2Eclipse plug-in. We also use the following Maven plug-ins:
Resources
BuildNumber
SQL
Hibernate3
Flexmojos
Assembly
Jetty
Cargo
JAR/WAR
and several others. Reading this blog makes me feel like Eclipse Indigo is when it'll all work together. When should we invest the time to make the move?
In fact "the move" should actually consist of simply upgrading the maven installation / m2eclipse only. Maven 3 is almost completely backward-compatible with Maven 2.
Check the compatibility notes to make sure you are not breaking something that will need a lot to fix.
(...) Makes me feel like Eclipse Indigo is when it'll all work together. When should we invest the time to make the move?
No, you can use Maven 3 and Eclipse 3.5 / 3.6 now (I don't even remember when I started to use Maven 3).
First, Maven 3 is backward compatible (see Maven 3 - Worth it?) so, as I commented in Should I upgrade to Maven 3?:
why don't you just try it? There is nothing to "upgrade", just install Maven 3 along your Maven 2 install, change your PATH settings and try it on an existing POM.
Second, m2eclipse 0.10 uses Maven 3 as embedded version for a long time (for dependency resolution even if you declare an "external" install) and is thus already Maven 3 ready. Just add the final version of Maven 3.0 as external install and there you go.
To sum up: Maven 3 is totally usable, I almost had nothing to change in my poms (only a few things to fix them thanks to the better reporting of Maven 3), it just works inside Eclipse and it builds faster. Just try it.
PS: I use the script attached to MNG-2730 to switch between the maven versions I have on my machine, if required.
Related
As example, i have program with version 0.0.1. Maven must create separate folder for it - "target/0.0.1/" instead of "target/". It must be done for version "0.0.2", "0.0.3", etc.
I use Eclipse & it's Maven:
Version: Oxygen.3a Release (4.7.3a)
Build id: 20180405-1200
JDK 1.8.0_172
Maven doesn't work that way, and trying to do something like that will lead to a path of suffering. Options I see include
Creating a separate assembly (and output Jar) for each version (see Maven Assembly Plugin)
Create a multi-project reactor with a separate output configuration for every project. Keep common code in one project that you link as dependency from the others. Possibly use the maven-shade-plugin to re-link the packages in your common project into the individual output projects
As you can see, both of these approaches are pretty hacky and require advanced Maven skills. It would be much easier to have parameterized builds where you pass in the output version. But that would make sense on a CI server like Jenkins.
I have a web application where we deploy to production whenever a feature is ready, sometimes that can be a couple of times a day, sometimes it can be a couple of weeks between releases.
Currently, we don't increment our version numbers for our project, and everything has been sitting at version 0.0.1-SNAPSHOT for well over a year.
I am wondering what is the Maven way for doing continuous delivery for a web apps. It seems overkill to bump up the version number on every commit, and never bumping the version number like we are doing now, also seems wrong.
What is the recommend best practice for this type of Maven usage?
The problem is actually a two-fold one:
Advancing project version number in individual pom.xml file (and there can be many).
Updating version number in all dependent components to use latest ones of each other.
I recommend the following presentation that discusses the practical realities of doing continuous delivery with Maven:
You tube presentation on CD with Maven
Slides
The key takeaway is each build is a potential release, so don't use snapshots.
This is my summary based on the video linked by Mark O'Connor's answer.
The solution requires a DVCS like git and a CI server like Jenkins.
Don't use snapshot builds in the Continuous Delivery pipeline and don't use the maven release plugin.
Snapshot versions such as 1.0-SNAPSHOT are turned into real versions such as 1.0.buildNumber where the buildNumber is the Jenkins job number.
Algorithm steps:
Jenkins clones the git repo with the source code, and say the source code has version 1.0-SNAPSHOT
Jenkins creates a git branch called 1.0.JENKINS-JOB-NUMBER so the snapshot version is turned into a real version 1.0.124
Jenkins invokes the maven versions plugin to change the version number in the pom.xml files from 1.0-SNAPSHOT to 1.0.JENKINS-JOB-NUMBER
Jenkins invokes mvn install
If the mvn install is a success then Jenkins will commit the branch 1.0.JENKINS-JOB-NUMBER and a real non-snapshot version is created with a proper tag in git to reproduce later. If the mvn install fails then Jenkins will just delete the newly created branch and fail the build.
I highly recommend the video linked from Mark's answer.
Starting from Maven 3.2.1 continuous delivery friendly versions are supported out of the box : https://issues.apache.org/jira/browse/MNG-5576
You can use 3 predefined variables in version:
${changelist}
${revision}
${sha1}
So what you basically do is :
Set your version to e.g. 1.0.0-${revision}. (You can use mvn versions:set to do it quickly and correctly in multi-module project.)
Put a property <revision>SNAPSHOT</revision> for local development.
In your CI environment run mvn clean install -Drevision=${BUILD_NUMBER} or something like this or even mvn clean verify -Drevision=${BUILD_NUMBER}.
You can use for example https://wiki.jenkins-ci.org/display/JENKINS/Version+Number+Plugin to generate interesting build numbers.
Once you find out that the build is stable (e.g. pass acceptance tests) you can push the version to Nexus or other repository. Any unstable builds just go to trash.
There are some great discussions and proposals how to deal with the maven version number and continuous delivery (CD) (I will add them after my part of the answer).
So first my opinion on SNAPSHOT versions. In maven a SNAPSHOT shows that this is currently under development to the specific version before the SNAPSHOT suffix. Because of this, tools like Nexus or the maven-release-plugin has a special treatment for SNAPSHOTS. For Nexus they are stored in a separate repository and its allowed to update multiple artefacts with the same SNAPSHOT release version. So a SNAPSHOT can change without you knowing about it (because you never increment any number in your pom). Because of this I do not recommend to use SNAPSHOT dependencies in a project especially in a CD world since the build is not reliable any more.
SNAPSHOT as project version would be a problem when your project is used by other ones, because of the above reasons.
An other problem of SNAPSHOT for me is that is not really traceable or reproducibly any more. When I see a version 0.0.1-SNAPSHOT in production I need to do some searching to find out when it was build from which revision it was build. When I find a releases of this software on a filesystem I need to have a look at the pom.properties or MANIFEST file to see if this is old garbage or maybe the latest and greatest version.
To avoid the manual change of the version number (especially when you build multiple builds a day) let the Build Server change the number for you. So for development I would go with a
<major>.<minor>-SNAPSHOT
version but when building a new release the Build Server could replace the SNAPSHOT with something more unique and traceable.
For example one of this:
<major>.<minor>-b<buildNumber>
<major>.<minor>-r<scmNumber>
So the major and minor number can be used for marketing issues or to just show that a new great milestone is reached and can be changed manually when ever you want it. And the buildNumber (number from your Continuous Integration server) or the scmNumber (Revision of SUbversion or GIT) make each release unique and traceable. When using the buildNumber or Subversion revision the project versions are even sortable (not with GIT numbers). With the buildNumber or the scmNumber is also kinda easy to see what changes are in this release.
An other example is the versioning of stackoverflow which use
<year>.<month>.<day>.<buildNumber>
And here the missing links:
Versioning in a Pipeline
Continuous Delivery and Maven
DON'T DO THIS!
<Major>.<minor>-<build>
will bite you in the backside because Maven treats anything after a hyphen as LEXICAL. This means version 1 will be lexically higher than 10.
This is bad as if you're asking for the latest version of something in maven, then the above point wins.
The solution is to use a decimal point instead of a hyphen preceding the build number.
DO THIS!
<Major>.<minor>.<build>
It's okay to have SNAPSHOT versions locally, but as part of a build, it's better to use
mvn versions:set -DnewVersion=${major}.${minor}.${build.number}
There are ways to derive the major/minor version from the pom, eg using help:evaluate and pipe to a environment variable before invoking versions:set. This is dirty, but I really scratched my head (and others in my team) to make it simpler, and (at the time) Maven wasn't mature enough to handle this. I believe Maven 2.3.1 might have something that go some way in helping this, so this info may no longer be relevant.
It's okay for a bunch of developers to release on the same major.minor version - but it's always good to be mindful that minor changes are non-breaking and major version changes have some breaking API change, or deprecation of functionality/behaviour.
From a Continuous Delivery perspective every build is potentially releasable, therefore every check-in should create a build.
At my work for web apps we currently use this versioning pattern:
<jenkins build num>-<git-short-hash>
Example: 247-262e37b9.
This is nice because it it gives you a version that is always unique and traceable back to the jenkins build and git revision that produced it.
In Maven 3.2.1+ they finally killed the warnings for using a ${property} as a version so that makes it really easy to build these. Simply change all your poms to use <version>${revision}</version> and build with -Drevision=whatever. The only issue with that is that in your released poms the version will stay at ${revision} in the actual pom file which can cause all sorts of weird issues. To solve this I wrote a simple maven plugin (https://github.com/jeffskj/cd-versions-maven-plugin) which does the variable replacement in the file.
As a starting point you may have a look at Maven: The Complete Reference. Project Versions.
Then there is a good post on versioning strategy.
i have multiple Elipse plugin projects that depend on each other. Several questions concerning that:
1) If i change the version number of one plugin, is it somehow possible to automatically update the plugin.xml files of the dependent plugins to reflect the version number change?
2) The same for Maven pom's: can i somehow automatically update the pom's when i change version numbers?
3) If i want to keep plugin.xml and pom.xml version numbers consistent, is there an automatic way to do so?
4) and last: If i have multiple projects that should have the same version numbers, can i somehow sync the version numbers?
Using the Maven Release Plugin - and especially the update-versions goal - may solve at least some of the issues you have addressed.
From the documentation at http://maven.apache.org/:
In some situations you may want an easy way to update the version numbers in each POM of a multi-module project. The update-versions goal is designed to accomplish this.
To update the version numbers in your POMs, run:
mvn release:update-versions
You will be prompted for the version number for each module of the project. If you prefer that each module version be the same as the parent POM, you can use the option autoVersionSubmodules.
mvn release:update-versions -DautoVersionSubmodules=true
In this case you will only be prompted for the desired version number once.
As pointed out by khmarbaise, there is Tycho, which is a maven plugin that works as bridge between maven and eclipse plugins.
Those two links provide the information needed to achieve what i asked in my question:
https://docs.sonatype.org/display/M2ECLIPSE/Staging+and+releasing+new+M2Eclipse+release
http://software.2206966.n2.nabble.com/Incrementing-updating-release-version-of-plugins-while-generating-p2-site-td5929658.html
It seems like you need to use the command
mvn -Dtycho.mode=maven org.eclipse.tycho:tycho-versions-plugin:set-version -DnewVersion=<version>
which should set all versions of the parent pom, the modules, and the plugin.xml 's to the given version.lik
I'm new to Maven, using the m2e plugin for Eclipse. I'm still wrapping my head around Maven, but it seems like whenever I need to import a new library, like java.util.List, now I have to manually go through the hassle of finding the right repository for the jar and adding it to the dependencies in the POM. This seems like a major hassle, especially since some jars can't be found in public repositories, so they have to be uploaded into the local repository.
Am I missing something about Maven in Eclipse? Is there a way to automatically update the POM when Eclipse automatically imports a new library?
I'm trying to understand how using Maven saves time/effort...
You picked a bad example. Portions of the actual Java Library that come with the Java Standard Runtime are there regardless of Maven configuration.
With that in mind, if you wanted to add something external, say Log4j, then you would need to add a project dependency on Log4j. Maven would then take the dependency information and create a "signature" to search for, first in the local cache, and then in the external repositories.
Such a signature might look like
groupId:artifactId:version
or perhaps
groupId:artifactId:version:classifier
This identifies a maven "module" which will then be downloaded and configured into your system. Once in place it adds all of the classes within the module to your configured project.
Maven principally saves time in downloading and organizing JAR files in your build. By defining a "standard" project layout and a "standard" build order, Maven eliminates a lot of the guesswork in the "why isn't my project building" sweepstakes. Also, you can use neat commands like "mvn dependency:tree" to print out a list of all the JARs your project depends on, recursively.
Warning note: If you are using the M2E plugin and Eclipse, you may also run into problems with the plugin itself. The 1.0 version (hosted at eclipse.org) was much less friendly than the previous 0.12 version (hosted at Sonatype). You can get around this to some extent by downloading and installing the "standalone" version of Maven from apache (maven.apache.org) and running Maven from the command line. This is actually much more stable than trying to run Maven inside Eclipse (in my personal experience) and may save you some pain as you try to learn about Maven.
I am working in a small team (3 persons) on several modules (about 10 currently). The compilation, integration and management of build versions is becoming more and more tedious.
I am looking for a good build / integration tool to replace / complete Ant.
Here is the description of our current development environment :
- Several modules depending on each over and on third party JARs
- Some may export JARS, some export WARS, some export standalone, runnable JARS (with Fat-Jar)
- Javadoc for all of them
- We work with eclipse
- Custom Ant script for each module. Many redundant information between the eclipse configuration and Ant scripts. For example, for the standalone Fat-JAR, we have listed all the recursive dependencies, whereas ideally, it could clearly be imported from the eclipse configuration.
- The source code is versioned using SVN
Here is what I would like a perfect integration tool to do for me :
Automatize the releases and versioning of modules. Ideally, the integration tool should detect if a new version is needed. For example, if I want to release a project A that depends on a project B, and if I have made small changes on the project B locally, then the integration tool should first release a new version of B as well and make A based on it.
Integrate strongly with eclipse, so that it could get the dependencies between modules and third party libs from its configuration. BTW, I would like to continue to configure build path with eclipse without updating some other ".xml" stuff. I saw that Gradle can generate eclipse project files from its configuration, but the counterpart would be great.
Enable a "live" and transparent development on local projects. I mean that I often make small changes on the core / common projects while developing the main / "leaf" projects. I would like to have my changes on core projects immediately available to leaf projects without the need of publishing (even locally) the JARs of my core projects.
Store all versions of the releases of my module on an external server. The simplest (shares folder / Webdav) would be the best. A nice web page with list of modules and delivered artifacts would be great too.
I have looked around for many things. From Ant4eclipse (to integrate the Eclipse configuration into my Ant script), to the Maven / Ivy / Gradle tools.
I am a bit confused.
Here is what I have understood so far:
- Maven is a great / big tool, but is somewhat rigid and obliges you to bend to its structure and concepts. It is based on description rather than on scripting. If you go out of the path, you have to develop you own plugins.
- Ivy is less powerful than maven, it handles less stuff but is more flexible.
- Gradle is in-between. It is general purpose. It enables scripting as well as "convention based" configuration. It integrates Ant and extends it.
So at this point I am looking for actual testimonials from real users.
What tools do you use ? How ? Do you have the same needs as me ?
Does it ease your life or get into the way ?
Are there sample some use cases, or workspace skeletons out there that I could use as a starting point to see what these tools are capable of ?
Sorry for the length of this message.
And thanks in advance for you advice.
Kind regards,
Raphael
Automatize the releases and versioning of modules (...)
The concepts of versioning and repository are built-in with Maven and they could fit here.
Maven supports SNAPSHOT dependencies. When using a snapshot, Maven will periodically try to download the latest available snapshot from a repository when you run a build. SNAPSHOT are typically used when a project is under active development.
Maven 2 also supports version ranges (I do not really recommend them but that's another story) which allow for example to configure A to depend on version [4.0,) of B (any version greater than or equal to 4.0). If you build and release a new version of B, A would use it.
Integrate strongly with eclipse
The m2eclipse plugin provides bi-directional synchronization with Eclipse.
Enable a "live" and transparent development on local projects.
The m2eclipse plugin supports "workspace resolution": if project A depend on project B and if project B is in the workspace, you can configure A to depend on B sources and not on B.jar (that's the default mode if I'm not wrong). So a change on B sources would be directly visible, without the need to build B.jar.
Store all versions of the releases of my module on an external server.
As mentioned earlier, this is actually a central concept of Maven (you don't even have the choice) and deploying through file:// or dav:// are both supported.
To sum up, Maven is (probably) not the only candidate but I'm sure it would fit:
Your project isn't that exotic or complex, there is nothing scaring from your description (some refactoring of the structure will probably be required but this shouldn't be a big deal).
Maven also brings a workflow based on best practices.
m2eclipse provides strong integration with the IDE.
But Maven has some learning curve.
CI tools? To me, there's only one: the Hudson CI.
I've setup a software development environment for Java once, with the components:
eclipse IDE
mercurial
bugzilla
maven
Nexus
Hudson CI
and some apache, mysql, php, perl, python, .. for integration.
The hudson was not integrated with eclipse and that was on purpose, because I wanted to build on a separate server. for all the other tools I had a perfect cross integration (like: mylyn on eclipse to talk with bugzilla, m2eclipse for using maven eclipse, a lot of plugins for hudson, ...)
We've been starting to integrate Gradle into our build process, and I can add to the answers posted already that Gradle would also work. Your assumptions are mostly correct, gradle is more off the cuff, but is powerful and allows for scripting and such within the build itself. It seems that most things maven can do, gradle does as well.
Now for your individual points:
Versioning: gradle supports dependency maps, versioning, and if you add in a CI server, you can trigger automated/dependent builds. For example, almost all of our 'deliverables' are .wars, but we have several code libs (.jars) and one executable .jar in development. One configuration is to make the the wars and the "fat-jar" dependent on the shared code libs. Then, when the shared libs are updated, bump the versions on the shared libs, test the consuming projects, then use Hudson's ability to fire dependent projects to redeploy those. There are other ways, but that seems to work best for us, for now.
Integrate strongly with eclipse: You're right, gradle can generate the eclipse files. We tend to only use the eclipseCp (to update .classpath) task once we get going, as only classpath needs changed. It's kind of quirky (grabs your default JRE, so make sure it's right, doesn't add exported="true" if you need it), but gets you 99% of the way there.
Enable a "live" and transparent development on local projects: This is one I'm not sure about. I've only hacked around gradle in this case; by removing the artifact in the consuming project and marked the shared project as such in eclipse, then reverted afterwards.
Store all versions of the releases of my module on an external server: simple and many approaches are supported, similar to Maven.
As far as examples, the docs for gradle are good, as well as the example projects that come with the full zip. They'll get you up and running fairly quickly.
Have a look at Ant Ivy. http://ant.apache.org/ivy/
There are no silver bullets, but in my experience Maven is a great project management tool. Personally, I like to use a comibnation of subversion (for version control), maven (for project/build management) and hudson (for continuous build/integration).
I find the convention brought by maven is really useful for context switching, and great for dependency management. It can be frustrating if jars aren't in the repositories, but you can install them locally and when you're ready you can host your own private repository which mirrors other places. I have had a good experience using sonar.nexus by http://www.sonatype.com/ . They also provide an excellenmt free book to get you started.
It might seem like overkill now, but setting up a good build / test / integrate / release environment now, will pay dividends later. It's is always harder to retro-fit, and it's something you can replicate easily.
Lastly, I happen to prefer Netbeans integration for maven, but that's just me :)
Some of your topics are part of deployment and release management.
You could check out a product like: Xebia DeployIt
(with an personal edition which is free)