I have a multi module java project. Maven takes almost around 40 secs to build it. I have tried maven with multi threaded builds too by specifying -T and -C args for no of threads and cores to be used. But I haven't seen any significant improvement in wall time of my builds.
I am using maven 3.2.3 and sometimes I need to build my project very frequently.
I know that clean goal take a lot of time but I can not omit it.
Suggestions please....
EDIT:
Note: In my case clean is not taking much time. It finishes in 1 sec. install is taking rest of the time.
Note: First thing is AFAIK, No other in built options available in maven apart from the all answers here.
Running maven build with Multiple threads works for me to speed up the builds.
For example :
mvn clean install -T100
where -T is for specifying how many threads you want based on your
hardware.
Below are the variants from wiki
Maven 3.x has the capability to perform parallel builds. The command
is as follows:
mvn -T 4 clean install Builds with 4 threads
mvn -T 1C clean install 1 thread per cpu core
mvn -T 1.5C clean install 1.5 thread per cpu core
How Execution is evaluated(See Parallel builds in Maven 3)?
Each node in the graph represents a module in a multi-module build, the "levels" simply indicate the distance to the first module in the internal reactor dependency graph. Maven calculates this graph based on declared inter-module dependencies for a multi-module build. Note that the parent maven project is also a dependency, which explains why there is a single node on top of most project graphs. Dependencies outside the reactor do not influence this graph.
Finally if you want to skip test execution you can also use -DskipTests as well.
Caution : Some of your plugins may not be compatible for multithreaded builder, it may work. but it will give below warning message. you may need to see plugin documentation for multithreading support.
[WARNING] *****************************************************************
[WARNING] * Your build is requesting parallel execution, but project *
[WARNING] * contains the following plugin(s) that have goals not marked *
[WARNING] * as #threadSafe to support parallel building. *
[WARNING] * While this /may/ work fine, please look for plugin updates *
[WARNING] * and/or request plugins be made thread-safe. *
[WARNING] * If reporting an issue, report it against the plugin in *
[WARNING] * question, not against maven-core *
[WARNING] *****************************************************************
[WARNING] The following plugins are not marked #threadSafe in test-project:
[WARNING] de.dentrassi.maven:rpm:0.9.2
[WARNING] Enable debug to see more precisely which goals are not marked #threadSafe.
[WARNING] *****************************************************************
On my actual project :
mvn clean install [INFO] Total time: 01:05 h
mvn clean install -DskipTests [INFO] Total time: 18:35 min
mvn clean install -Dmaven.test.skip -DskipTests [INFO] Total time: 10:58 min
mvn -T 1C clean install -Dmaven.test.skip -DskipTests [INFO] Total time: 04:00 min
We can also skip the javadoc to be generated as Archmed commented by adding -Dmaven.javadoc.skip=true mvn -T 1C clean install -Dmaven.test.skip -DskipTests -Dmaven.javadoc.skip=true
Don't use * imports, on IntelliJ, choose > Analyze > Run inspection by name > * imports , to find all * imports and correct it.
Remove all unused imports in your project > on Intellij > Analyze > Run inspection by name > unused imports
Remove all unused code (classes, variable, field, parameter, etc..), on Intellij : Analyze > run inspection by name > unused declaration.
Upgrade to last JAVA VERSION
I have found that the task mvn clean, is taking 2 minutes to clean the TARGET folder before building. I did create a new task called quickclean, and I am using it instead of clean, this way mvn -T 1C quickclean install -Dmaven.test.skip -DskipTests . This new task quickclean is only renaming the build folder from TARGET to TARGET-yyyy-MM-dd-HH-mm(what is VERY FAST). So now, every time you make a new mvn quickclean install..., you have a folder with the time of the build. The inconvenient, it's that this may take a lot of space on the hard disk, so you have to clean all these directories sometimes. So for that I have created another task called: trashclean, to put all this folder to trash. I am running this tasks maybe on time per week. or month, depending on my work mvn trashclean.
Here is what you need to add to your pom.xml if you want to use this concept
<properties>
<timestamp>${maven.build.timestamp}</timestamp>
<maven.build.timestamp.format>yyyy-MM-dd-HH-mm</maven.build.timestamp.format>
<trashdir>trash/target-${maven.build.timestamp}</trashdir>
</properties>
<profile>
<id>quickclean</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>rename_target</id>
<phase>pre-clean</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<move todir="${trashdir}" failonerror="false">
<fileset dir="target/"/>
</move>
</tasks>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>trashclean</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>clean_trash</id>
<phase>clean</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<delete dir="trash/" failonerror="false"/>
</tasks>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
You can optimize the build by using of some small-2 trick like
If you have written Junit Test and dont want to run test case every time you can use -DskipTests=true
Locally Install Nexus or Repositories
You can adjust memory configurations to optimum for eg: add this line to mvn.bat set MAVEN_OPTS=-Xmx512m -XX:MaxPermSize=256m
For more information you can check How to Speed Your Maven build
You can do a hacky way if you have some extra RAM as I do, and you don't need SSD, because it's sensitive for SSD to do a lot of read-writes. Even if you disable +a bit (MacOS - I don't remember if Win have similar issue). But
create RAM drive (number of ways depends on your OS) and mount it to /Users/Username/volatile folder
locate maven-module-builder-X.X.X.jar in MVN_HOME/lib
in JAR modify file org/apache/maven/model/pom-4.0.0.xml by modifying
<build><directory>/Users/Username/volatile/${project.groupId}/${project.artifactId}/${project.version}/target
From now on all project compilations will be done in memory. And it's a difference even in comparison with SSD.
Yes, its hack and some maven plugins can stop working BTW.
If using commandline you can check how many cores your machine has and use all of them, if you also want to skip your tests, you can add -DskipTests
For example, I have 8 core processor:
mvn -T 8C clean install -DskipTests
Apart from parallel run which many of the folks have already mentioned, try these maven options which would speed up build
-XX:+TieredCompilation -XX:TieredStopAtLevel=1
You can try turbo-maven-plugin to help speed up your multi module Maven builds locally (I am one of the plugin authors)?
https://github.com/sparebank1utvikling/turbo-maven-plugin
It works by analyzing what modules actually have code changes, and build only those and the modules dependent on them. More info about how it works here:
https://medium.com/sparebank1-digital/speed-up-your-multi-module-maven-builds-with-turbo-maven-plugin-4be0eb2a2601
Low-Level speed using Ramdisk (Windows)
My additional trick is using a ramdisk
Create a Ramdisk of 2GB (in example D:)
Install IDE, Maven, JDK and the .m2/repository to that Ramdisk (D:/m2/).
Edit the Maven's D:/maven/conf/settings.xml to use that Ramdisked repo (<localRepository>D:/m2</localRepository>)
Put the project into the ramdisk.
Run the project having its temp-folder in the Ramdisk.
On my machine I turned
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.954 s (Wall Clock)
[INFO] Finished at: 2021-02-17T13:07:51+01:00
[INFO] Final Memory: 55M/642M
[INFO] ------------------------------------------------------------------------
Into
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.805 s (Wall Clock)
[INFO] Finished at: 2021-11-11T10:48:14Z
[INFO] ------------------------------------------------------------------------
Both results are high-end (-DskipTests -T 12)
Warning: Do not forget to persist the ramdisk before shutdown or your work is lost.
Look at this beast:
If you are using spring boot with maven and you feel like dependencies are coming slow to your system, try following setting in pom.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.0</version>
<relativePath />
</parent>
I experienced that version 2.7.0 downloads the dependencies faster than other versions.
Related
I am testing out some migration in Maven from a Spotify dockerfile plugin with Fabric8 to just using Eclipse's JKube plugin. I have been hitting errors with what I thought would be a simple enough build using a Dockerfile.
The POM config is like so:
<plugin>
<groupId>org.eclipse.jkube</groupId>
<artifactId>kubernetes-maven-plugin</artifactId>
<version>1.5.1</version>
<configuration>
<images>
<image>
<name>${docker.image.prefix}/${docker.image.name}</name>
<build>
<dockerFile>${project.basedir}/Dockerfile</dockerFile>
</build>
</image>
</images>
</configuration>
</plugin>
The Dockerfile is (per config above) in the root of the project. It is very simple
FROM adoptopenjdk/openjdk11-openj9:x86_64-ubi-jre-11.0.12_7_openj9-0.27.0
ARG JAR_FILE
ADD target/${JAR_FILE} microservice.jar
ENTRYPOINT ["java", "-jar", "microservice.jar"]
Running mvn clean install k8s:build ends with the following error message:
[INFO] --- kubernetes-maven-plugin:1.5.1:build (default-cli) # persistence-svc ---
[INFO] k8s: Running in Kubernetes mode
[INFO] k8s: Building Docker image in Kubernetes mode
[ERROR] k8s: Failed to execute the build [Error while trying to build the image:<PROJ>\target\docker\<PREFIX>\<IMAGE>\build\maven\.git\objects\pack\pack-c35818936460f0bb0c1b903466f7bf1a17f22cc8.idx]
I am not sure what I am missing. I tried to add a jKube ignore file but that made no difference.
Any pointers would be much appreciated.
Looks like I was on the correct track with the ignore file.
The syntax I had was incorrect. I had just .git/ and needed .git/**.
.git/**
src/**
target/classes/**
target/persistence-svc.jar
target/persistence-svc-sources.jar
It seems a little crazy that this should cause problems. Anyways I am sure there is a nicer solution overall but this gets me going without changing anything other than the plugins.
EDIT:
Indeed there is a better way for my use-case. Use .jkube-dockerinclude instead. Example here.
I have a project document-scanner-aggregator which runs fine locally when run with Maven 3.1.1, i.e. ~/apache-maven-3.1.1/bin/mvn clean install, but doesn't on travis-ci.org where it detects Maven 3.2.5 which suspiciously is the version provided by travis-ci.org. However it's not supposed to do that, because the Maven version which runs the build is supposed to be enforced, right?
The failure is
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireMavenVersion failed with message:
Detected Maven Version: 3.2.5 is not in the allowed range (,3.2).
...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4:enforce (enforce-versions) on project javaocr-parent: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
and the configuration is
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<id>enforce-versions</id>
<goals>
<goal>enforce</goal>
</goals>
<configuration>
<rules>
<requireMavenVersion>
<!--different rules for different issues-->
<!--3.3.x causes `java.lang.NoClassDefFoundError: org/eclipse/aether/spi/connector/Transfer$State` which is caused by certain maven versions, see https://cwiki.apache.org/confluence/display/MAVEN/AetherClassNotFound for details-->
<version>(,3.3)</version>
<!--3.2.x causes `No implementation for org.eclipse.aether.connector.wagon.WagonConfigurator was bound.`-->
<version>(,3.2)</version>
</requireMavenVersion>
</rules>
</configuration>
</execution>
</executions>
</plugin>
in javaocr.
The .travis.yml is
language: java
install:
- wget http://mirrors.ae-online.de/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz && tar xf apache-maven-3.1.1-bin.tar.gz
- apache-maven-3.1.1/bin/mvn clean install
You're misunderstanding the role of the Enforcer Plugin:
The Enforcer plugin provides goals to control certain environmental constraints such as Maven version, JDK version and OS family along with many more standard rules and user created rules.
Its goal is not to do magic so that the configured rules are verified, but it is to fail the build when one of the rules is not verified. Put another way, it checks the list of configured rules, so that the build cannot continue if one of those isn't verified. The reason is that if one of the pre-requisite for the build is not met, it is better to fail as early as possible.
In your case, the requireMavenVersion rule enforces that the build is done with a Maven version strictly lower than 3.2. It will not spawn a new Maven instance if you're running the build with a Maven 3.2.5 so that the rule is verified; which version should it use, and where would it get it? Instead, it detects that the build is done with a Maven version that is disallowed, and fails accordingly. Since the Travis build is configured to use Maven 3.2.5, it is expected to fail, and you need to install a Maven version lower than 3.2 for your Travis build.
To use a different Maven version, like 3.1.1, on Travis CI, you can have the following in .travis.yml:
before_install:
- wget http://mirrors.ae-online.de/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz
- tar xf apache-maven-3.1.1-bin.tar.gz
- export M2_HOME=$PWD/apache-maven-3.1.1
- export PATH=$M2_HOME/bin:$PATH
install: /bin/true
script: mvn clean install
As a side-note, your current configuration of
<requireMavenVersion>
<version>(,3.3)</version>
<version>(,3.2)</version>
</requireMavenVersion>
is completely equivalent to
<requireMavenVersion>
<version>(,3.2)</version>
</requireMavenVersion>
The range (,3.2) means that it matches version strictly lower than 3.2. Therefore, requiring that it is also strictly lower than 3.3 is redundant (since 3.2 is before 3.3).
I'm trying to assemble a project using maven and the OpenImaj library, I been following the instructions on this page http://www.openimaj.org/tutorial/getting-started-with-openimaj-using-maven.html but some of the process seems to be different from the one outline in the tutorial, I have tried this on two different computers (mac and pc) and received the following errors/steps, any idea of where/what I am doing wrong will be helpful.
after running the mvn -DarchetypeCatalog=http://maven.openimaj.org/archetype-catalog.xml archetype:generate line I am prompt to "Choose a number or apply filter" where the default is 284.
Once I select 284 (should I be selecting anything else?), I'm being prompt to select "maven-archetype-quickstart version" where I choose the latest 6:1.1, following by the groupId, artifactId, version, package, and Y confirmation, this result with 'Build Success' and create the directory as well as the pom.xml file on my computer.
When I navigate to the project folder "cd projectName" and run the mvn assembly:assembly command, I first see that a few of the packages are being collected, then I see a "Build Failure" notification -"Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-5.... No assembly descriptors found.
any idea what I am doing wrong and how I can get the OpenImaj lib integrated into a project, should I be downloading the SVN version and attempt to set the projects from local libraries.
Many thanks in advance!
Just add this plugin to your pom.xml. This solved the problem for me:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
Are you sure you're entering the maven command in step 1 correctly? You should only see three options (just confirmed this is working on both OSX, debian & ubuntu):
abe:~ jon$ mvn -DarchetypeCatalog=http://maven.openimaj.org/archetype-catalog.xml archetype:generate
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> maven-archetype-plugin:2.2:generate (default-cli) # standalone-pom >>>
[INFO]
[INFO] <<< maven-archetype-plugin:2.2:generate (default-cli) # standalone-pom <<<
[INFO]
[INFO] --- maven-archetype-plugin:2.2:generate (default-cli) # standalone-pom ---
[INFO] Generating project in Interactive mode
[INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype:
1: http://maven.openimaj.org/archetype-catalog.xml -> org.apache.maven.archetypes:maven- archetype-quickstart (An archetype which contains a sample Maven project.)
2: http://maven.openimaj.org/archetype-catalog.xml -> org.openimaj:openimaj-quickstart-archetype (Maven quickstart archetype for OpenIMAJ)
3: http://maven.openimaj.org/archetype-catalog.xml -> org.openimaj:openimaj-subproject-archetype (Maven archetype for creating OpenIMAJ subprojects with the most of the standard configuration completed automatically)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains):
You then need to pick the org.openimaj:openimaj-quickstart-archetype option from the list (probably #2).
I had faced similar issues and finally worked it out. There are two things to note here. One is the network we are using and second is the maven tool we are using.
In your eclipse IDE go to window -> preferences.
Under maven tab go to installations sub-tab. Instead of using the embedded maven add filepath to maven installed on your system (The one command Line uses).
Under General Tab go to Network connections tab. Change the active provider.
Neither of the above will completly build the project successfully in one go. But each combination of network and maven will download some jars. Once you try two to three combinations you will have all the jars and the project will build successfully.
I want to know the version of plugin installed. Which command can do it?
mvn -Dplugin=<groupId>:<artifactId> help:describe
detailed description of the plugin - including the version
If you would to know which version of plugins (incl. plugins provided through maven master pom) your build use, try:
mvn help:effective-pom
I don't know what you mean by 'version of plugin installed' but the Maven help plugin enables you to get the desciption of a plugin by giving the groupId and artifactId,
mvn -Dplugin=<groupId>:<artifactId> help:describe
You will get a detailed description of the plugin - including the version (although I must admit that I don't know the strategy of version number resolving).
Example for the maven-dependency-plugin
mvn -Dplugin=org.apache.maven.plugins:maven-dependency-plugin help:describe
Output:
Name: Maven Dependency Plugin
Description: Provides utility goals to work with dependencies like copying,
unpacking, analyzing, resolving and many more.
Group Id: org.apache.maven.plugins
Artifact Id: maven-dependency-plugin
Version: 2.2
Goal Prefix: dependency
This plugin has 21 goals:
dependency:analyze
Description: Analyzes the dependencies of this project and determines which
are: used and declared; used and undeclared; unused and declared. This goal
is intended to be used standalone, thus it always executes the test-compile
phase - use the dependency:analyze-only goal instead when participating in
the build lifecycle.
dependency:analyze-dep-mgt
Description: This mojo looks at the dependencies after final resolution and
looks for mismatches in your dependencyManagement section. In versions of
maven prior to 2.0.6, it was possible to inherit versions that didn't match
your dependencyManagement. See MNG-1577 for more info. This mojo is also
useful for just detecting projects that override the dependencyManagement
directly. Set ignoreDirect to false to detect these otherwise normal
conditions.
dependency:analyze-duplicate
Description: Analyzes the <dependencies/> and <dependencyManagement/> tags
in the pom.xml and determines the duplicate declared dependencies.
... and much more
Add this to your pom.xml file and you will get the result on mvn clean install:
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>versions-maven-plugin</artifactId>
<version>2.1</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>display-dependency-updates</goal>
<goal>display-plugin-updates</goal>
<goal>display-property-updates</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
If you just want to do it once:
mvn versions:display-plugin-updates
You can add an "-X" parameter when maven is packaged(mvn clean compile * -X), and then search for "artifactId" to see the exact version number.
Im having a strange problem when im trying to let the wagon plugin upload files during the site-deploy lifecycle when i'm invoking the release:perform goal.
It seems wagon uploads the files correctly when im invoking mvn site-deploy but it just responds with
Nothing to upload
when calling mvn release:perform which is supposed to invoke the phases site site-deploy as stated in the documentation.
this is the plugin config for wagon.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>wagon-maven-plugin</artifactId>
<version>1.0-beta-3</version>
<executions>
<execution>
<id>upload-jars</id>
<phase>deploy site-deploy</phase>
<goals>
<goal>upload</goal>
</goals>
<configuration>
<fromDir>target/checkout/target</fromDir>
<includes>*.jar</includes>
<url>scpexe://nohost.com</url>
<toDir>/var/www/projects/test</toDir>
<serverId>server - projects</serverId>
</configuration>
</execution>
</executions>
</plugin>
maven tells me the right goals were started:
[INFO] Executing goals 'deploy site-deploy'...
[INFO] [INFO] Scanning for projects...
but wagon doesn't upload anything:
[INFO] [INFO] --- wagon-maven-plugin:1.0-beta-3:upload (default) # exp4j ---
[INFO] [INFO] Nothing to upload.
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] BUILD SUCCESS
Does anyone spot my problem that causes maven to work as expected when invoking site-deploy but failing when doing a release:perform ?
This plugin does not do what you think it does. Trust me, I've been there.
The underlying wagon protocol is only intended for talking to Maven Repositories, not arbitrary directories. If the stuff you are pushing doesn't have files and directories in the pattern of a repo, the plugin will decide that there's nothing for it to do.
I spent hours and hours and hours on this, and read the code, and reached the conclusion that this plugin is not intended to be useful for pushing arbitrary files to arbitrary places, and in fact does not work for that purpose.
I had the same issue until I've found at that the "includes" tag must contains "/*" to recursively include files and subdirectories.
See comments of that blog post
<includes>*/**</includes>