We Keep Coding

'mvn clean' vs.'mvn clean install' (when a plugin execution is attached to 'clean')

Let's say I want to install project-local dependencies (jar files) to my local maven repository (~/.m2) prior to compiling the project so I can reference them in my POM just like I would reference any dependency from Maven Central. Currently, I'm using Maven install plugin's install-file goal attached to the 'clean' phase (because my IDE uses it), like so:
<dependencies>
<dependency>
<groupId>group.id</groupId>
<artifactId>artifact.id</artifactId>
<version>artifact.version</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
<executions>
<execution>
<id>install-my-local-dependency</id>
<phase>clean</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>group.id</groupId>
<artifactId>artifact.id</artifactId>
<version>artifact.version</version>
<file>${project.basedir}/lib/local-dep.jar</file>
<packaging>jar</packaging>
<generatePom>true</generatePom>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
When I execute mvn clean (or its equivalent in the IDE), everything runs as I'd expect:
--- maven-clean-plugin:2.5:clean (default-clean) # MyProject ---
--- maven-install-plugin:2.5.2:install-file (install-...) # MyProject ---
Installing ${project.basedir}/lib/local-dep.jar to ~/.m2/repository/group.id/artifact.id/local-dep.jar
But when I execute mvn clean install instead (or its equivalent in the IDE), I get the following error:
Failed to execute goal on project MyProject: Could not resolve dependencies for project com.example.MyProject:jar:1.0: Could not find artifact group.id:artifact.id:jar:artifact.version in central (https://repo.maven.apache.org/maven2) -> [Help 1]
For some reason, Maven install plugin's install-file goal either does not run in this case, or doesn't run soon enough. Why? The other StackOverflow answers I found explain differences between both commands but in my eyes, they do not explain this particular difference as my project has no modules.
Is there a better way of doing the same thing cross-platform, even on build servers (e.g. Jenkins) and with at least one other dependent project?
Should it be any help, I have the following Maven versions:
CLI: 3.6.0
IDE: 3.3.9

Maven first analyses the pom.xml and then calls the goals/phases. The analysis itself is complicated and has different depths, so I guess that calling clean alone will not make Maven analyse the dependencies, but calling clean install does so. Note that the analysis of the POM only happens once, not again for every goal/phase.
Generally, your approach cannot be recommended. Usually, you put project dependencies into remote Maven repositories, so that they can be resolved through them. If you work inside a company, you should set up a Nexus/Artifactory server that handles your artifacts.
If you want people outside your company to build the artifact, you need to find a provider for Maven repositories. I guess that github/gitlab can help you here. Then you need to add those repositories to the POM.

Docker multi-staged maven build split dependency resolve and compilation

I'm trying to setup a multi-staged dockerfile for my maven based java project. Here's what I'm using so far:
### STAGE 1: Build ###
FROM maven:3.6.1-jdk-8 as build
WORKDIR /usr/src/app
# Install app dependencies
COPY pom.xml .
RUN mvn dependency:go-offline
# Bundle app source
COPY src src
RUN mvn package
### STAGE 2: Production Environment ###
FROM jboss/wildfly:17.0.0.Final
COPY --from=build /usr/src/app/target/Appname.war /opt/jboss/wildfly/standalone/deployments/Appname.war
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin123 --silent
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]
The question is about the mvn commands. I see that mvn dependency:go-offline downloads some dependencies, which is great. But then mvn package downloads some more packages. Why? How can I have two steps:
one for resolving and downloading all dependencies
one for compiling the source code and creating the .war file
EDIT
using mvn package -o, I get the following package not being found:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:2.6:resources (default-resources) on project monolith: Execution default-resources of goal org.apache.maven.plugins:maven-resources-plugin:2.6:resources failed: Plugin org.apache.maven.plugins:maven-resources-plugin:2.6 or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.apache.maven:maven-profile:jar:2.0.6, org.apache.maven:maven-repository-metadata:jar:2.0.6, org.apache.maven:maven-plugin-registry:jar:2.0.6, classworlds:classworlds:jar:1.1-alpha-2: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven:maven-profile:jar:2.0.6 has not been downloaded from it before. -> [Help 1]
I think it's related to the build plugin in pom.xml. Any thoughts?
<build>
<finalName>Monolith</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.3</version>
<!--<configuration>-->
<!--<webXml>src\main\webapp\WEB-INF\web.xml</webXml>-->
<!--</configuration>-->
</plugin>
</plugins>
</build>

You are most likely affected by MDEP-82 go-offline / resolve-plugins does not resolve all plugin dependencies bug which is currently unresolved.
As per this comment in the MDEP-82 issue this can potentially be fixed by specifying maven-dependency-plugin version instead of using the default one:
then define explicitiely the maven-dependency-plugin version used for go-offline to not depend on default version defined inside Maven:
mvn -s settings.xml org.apache.maven.plugins:maven-dependency-plugin:2.8:go-offline

The maven dependency plugin has some limitations and so it doesn't download all the dependencies. This is mostly OK when you work locally, because a single mvn package command would do the trick.
But this is not well suited for maven running inside Docker container. After a lot of searching, today I stumbled upon the go-offline-maven-plugin. I tried it with the small project I'm using to test this and it worked.
Here is how to include it in your pom.xml:
<!-- ... -->
<build>
<plugins>
<plugin>
<groupId>de.qaware.maven</groupId>
<artifactId>go-offline-maven-plugin</artifactId>
<version>1.2.1</version>
<configuration>
<dynamicDependencies>
<DynamicDependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit4</artifactId>
<version>2.20.1</version>
<repositoryType>PLUGIN</repositoryType>
</DynamicDependency>
</dynamicDependencies>
</configuration>
</plugin>
<!-- ... -->
</plugins>
<!-- ... -->
</build>
<!-- ... -->
The dynamicDependencies are dependencies, which are not really present in any pom.xml, but are instead hard-coded in the plugins. The most common case is the surefire plugin, which loads the needed dependency depending on your tests (e.g. JUnit4/5, TestNG, etc.).
Lastly, you need to run this plugin from your Dockerfile. This should work:
### STAGE 1: Build ###
FROM maven:3.6.1-jdk-8 as build
WORKDIR /usr/src/app
# Install app dependencies
COPY pom.xml .
RUN mvn -e -B de.qaware.maven:go-offline-maven-plugin:resolve-dependencies
# Bundle app source
COPY src src
RUN mvn -o -e -B package
### STAGE 2: Production Environment ###
FROM jboss/wildfly:17.0.0.Final
COPY --from=build /usr/src/app/target/Appname.war /opt/jboss/wildfly/standalone/deployments/Appname.war
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin123 --silent
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]

How resolve conflict with maven

i have pom file: https://gist.github.com/anonymous/2d2abdc47d868250e8f47d74bdd643c2
i build with command: clean compile assembly:single
but i get warning:
[WARNING] 'dependencies.dependency.systemPath' for
com.xxx.backtesting:client:jar should not point at files within the
project directory,
${project.basedir}/lib/client-0.1-jar-with-dependencies.jar will be
unresolvable by dependent projects # line 18, column 25
and this library doesnt exist in jar file:
When i run my jar file, i got:
java -jar backtestingCandlesDownloader-0.1-jar-with-dependencies.jar 1440672480000 1441025280000 60000
task: startDate = 1440672480000, endDate = 1441025280000, period = 60000
Exception in thread "main" java.lang.NoClassDefFoundError: com/xxx/backtesting/client/model/Server
at com.xxx.backtestingCandlesDownloader.Main.main(Main.java:33)
Caused by: java.lang.ClassNotFoundException: com.xxx.backtesting.client.model.Server
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
i dont know how include client-0.1-jar-with-dependencies.jar in jar file.

Your warning is referring to line 16 and 18 of your POM:
<scope>system</scope>
...
<systemPath>${project.basedir}/lib/client-0.1-jar-with-dependencies.jar</systemPath>
These lines identify a systemPath for a library and scope lets Maven know it is provided by the system instead of within the project [1].
Your best approach would be to include it in your local Maven Repository by installing it as part of your build. This will install the client jar file as a maven artifact in your local Maven repository during the clean phase making it available to dependency projects. Installing as part of the lifecycle ensures the artifact is available automatically for all future developers/builds.
<build>
<plugins>
<!-- Installs new artifact in the local repository -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
<executions>
<execution>
<id>install-artifact</id>
<phase>clean</phase>
<configuration>
<file>${project.basedir}/lib/client-0.1-jar-with-dependencies.jar</file>
<repositoryLayout>default</repositoryLayout>
<groupId>com.xxx.backtesting</groupId>
<artifactId>client</artifactId>
<version>0.1</version>
<packaging>jar</packaging>
<generatePom>true</generatePom>
</configuration>
<goals>
<goal>install-file</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Once installed, you simply reference it as a dependency as you would any other Maven dependency:
<dependencies>
...
<dependency>
<groupId>com.xxx.backtesting</groupId>
<artifactId>client</artifactId>
<version>0.1</version>
</dependency>
...
</depencencies>
Alternatively, you could install the artifact via command line, but would have to fully qualify the location of your jar file [2]. After installing using the command line method, you would be able to use the library as you would any other dependency in Maven:
mvn install:install-file -Dfile=/path/to/lib/client-0.1-jar-with-dependencies.jar -DgroupId=com.xxx.backtesting -DartifactId=client -Dversion=1.0 -Dpackaging=jar
https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html

First make sure you have lib/client-0.1-jar-with-dependencies.jar in your project directory,then replace ${project.basedir} with ${pom.basedir}

If I run a plugin from the command line at the root pom, what does it do when traversing modules?

I'm trying to use the plugin: http://gatling.io/docs/2.0.1/extensions/maven_plugin.html
I'm trying to run this command at the root pom:
mvn package gatling:execute
This multi module project only has this plugin defined in one child pom like so:
<plugins>
<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.1</version>
<configuration>
<args>
<arg>-deprecation</arg>
<arg>-feature</arg>
<arg>-language:postfixOps</arg>
</args>
</configuration>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
But when I run this command at the root, it gives this error:
[INFO] ------------------------------------------------------------------------
[INFO] Building root-pom 2.5.210-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- gatling-maven-plugin:2.1.7:execute (default-cli) # root-pom ---
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at io.gatling.mojo.MainWithArgsInFile.runMain(MainWithArgsInFile.java:50)
at io.gatling.mojo.MainWithArgsInFile.main(MainWithArgsInFile.java:33)
Caused by: java.lang.RuntimeException: Can't find the jar matching (.*scala-library.*\.jar)$
at io.gatling.compiler.ZincCompiler$$anonfun$jarMatching$1$2.apply(ZincCompiler.scala:88)
at io.gatling.compiler.ZincCompiler$$anonfun$jarMatching$1$2.apply(ZincCompiler.scala:88)
at scala.Option.getOrElse(Option.scala:120)
at io.gatling.compiler.ZincCompiler$.jarMatching$1(ZincCompiler.scala:88)
at io.gatling.compiler.ZincCompiler$.setupZincCompiler(ZincCompiler.scala:91)
at io.gatling.compiler.ZincCompiler$delayedInit$body.apply(ZincCompiler.scala:106)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
at scala.App$class.main(App.scala:71)
at io.gatling.compiler.ZincCompiler$.main(ZincCompiler.scala:35)
at io.gatling.compiler.ZincCompiler.main(ZincCompiler.scala)
... 6 more
This is not a scala project, so this is definitely related to the plugin. Running mvn package without gatling:execute works fine.
I do not know enough about maven to be able to troubleshoot this problem because I can't really tell what maven is trying to do here. If it is trying to run gatling at the root, why? That plugin is not defined in the root pom anywhere. My question is:
When you run a plugin at the root of a multi-module project, how does it traverse my project, look for plugins and run them? I'm asking this question abstractly. You don't have to answer in terms of gatling. I just want to understand maven plugins better.
How do I troubleshoot this issue? I think I could add scala to the root pom, but since it would be the first dependency at that level, it seems like it's the wrong approach.
I could also cd into the one child pom that has this plugin and run the command there, but that means I'll only be running package on a piece of the project. Ideally I'd like to run package on the whole project before I run the plugin.

Running maven with a plugin goal like that will run that goal for all project modules, the same as mvn install will install each module. You don't have to define the plugin anywhere - if you specify it on the commandline, it is run.
You correctly bound the execution of that plugin to one or more lifecycle phases, in this case process-resources and process-test-resources. If you run mvn compile, the the process-resources phase will be executed and the add-sources and compile goals will be run for the gatling plugin. Similarly, running mvn test (which is also run when you mvn install) will run the testCompile goal of the gatling plugin.
What you'll want to do is to also bind the execute goal as is exemplified
on the page you linked. If you want everyting packaged first, bind it to the
integration test phase:
<execution>
<phase>integration-test</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
....
</configuration>
</execution>
Note that integration-test comes after package (and somewhere before install), so to run it, execute mvn integration-test or mvn install.

Building with Maven on travis-ci: javadoc plugin won't fail

I'm trying out Travis-CI to build a recently-released Open source Java project. This is my .travis.yml file:
language: java
script: mvn clean install -Dmaven.javadoc.skip=false
I tried that last flag because I read in the docs that that flag defaults to true, and I do see it in the build logs on travis-ci:
$ mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V
However, when the next line in the log is run, the build succeeds:
$ mvn clean install -Dmaven.javadoc.skip=false
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
I know this is incorrect because the build fails on my personal machine (on a javadoc build error), running maven 3.2.5, and the build also fails when building it on a bamboo server with the same arguments, even with just mvn clean install:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.9.1:jar (attach-javadocs) on project luces: MavenReportException: Error while creating archive:
[ERROR] Exit code: 1 - <filepath>: error: tag not allowed here: <li>
[ERROR] * <li>0 for int / long</li>
[ERROR] ^
here's the relevant part of the Pom that's using the javadoc plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9.1</version>
<executions>
<execution>
<id>attach-javadocs</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
I checked and Travis-CI says they're using Maven 3.2.x as well. Is this a bug in Travis-CI?

I fixed the problem. The issue was I was running JDK8 on the environments where it was failing, and travis-ci was running on JDK7 by default. Setting the jdk to oraclejdk8 on travis-ci caused the same failure, as expected.