We Keep Coding

Docker multi-staged maven build split dependency resolve and compilation

I'm trying to setup a multi-staged dockerfile for my maven based java project. Here's what I'm using so far:
### STAGE 1: Build ###
FROM maven:3.6.1-jdk-8 as build
WORKDIR /usr/src/app
# Install app dependencies
COPY pom.xml .
RUN mvn dependency:go-offline
# Bundle app source
COPY src src
RUN mvn package
### STAGE 2: Production Environment ###
FROM jboss/wildfly:17.0.0.Final
COPY --from=build /usr/src/app/target/Appname.war /opt/jboss/wildfly/standalone/deployments/Appname.war
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin123 --silent
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]
The question is about the mvn commands. I see that mvn dependency:go-offline downloads some dependencies, which is great. But then mvn package downloads some more packages. Why? How can I have two steps:
one for resolving and downloading all dependencies
one for compiling the source code and creating the .war file
EDIT
using mvn package -o, I get the following package not being found:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:2.6:resources (default-resources) on project monolith: Execution default-resources of goal org.apache.maven.plugins:maven-resources-plugin:2.6:resources failed: Plugin org.apache.maven.plugins:maven-resources-plugin:2.6 or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.apache.maven:maven-profile:jar:2.0.6, org.apache.maven:maven-repository-metadata:jar:2.0.6, org.apache.maven:maven-plugin-registry:jar:2.0.6, classworlds:classworlds:jar:1.1-alpha-2: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven:maven-profile:jar:2.0.6 has not been downloaded from it before. -> [Help 1]
I think it's related to the build plugin in pom.xml. Any thoughts?
<build>
<finalName>Monolith</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.3</version>
<!--<configuration>-->
<!--<webXml>src\main\webapp\WEB-INF\web.xml</webXml>-->
<!--</configuration>-->
</plugin>
</plugins>
</build>

You are most likely affected by MDEP-82 go-offline / resolve-plugins does not resolve all plugin dependencies bug which is currently unresolved.
As per this comment in the MDEP-82 issue this can potentially be fixed by specifying maven-dependency-plugin version instead of using the default one:
then define explicitiely the maven-dependency-plugin version used for go-offline to not depend on default version defined inside Maven:
mvn -s settings.xml org.apache.maven.plugins:maven-dependency-plugin:2.8:go-offline

The maven dependency plugin has some limitations and so it doesn't download all the dependencies. This is mostly OK when you work locally, because a single mvn package command would do the trick.
But this is not well suited for maven running inside Docker container. After a lot of searching, today I stumbled upon the go-offline-maven-plugin. I tried it with the small project I'm using to test this and it worked.
Here is how to include it in your pom.xml:
<!-- ... -->
<build>
<plugins>
<plugin>
<groupId>de.qaware.maven</groupId>
<artifactId>go-offline-maven-plugin</artifactId>
<version>1.2.1</version>
<configuration>
<dynamicDependencies>
<DynamicDependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit4</artifactId>
<version>2.20.1</version>
<repositoryType>PLUGIN</repositoryType>
</DynamicDependency>
</dynamicDependencies>
</configuration>
</plugin>
<!-- ... -->
</plugins>
<!-- ... -->
</build>
<!-- ... -->
The dynamicDependencies are dependencies, which are not really present in any pom.xml, but are instead hard-coded in the plugins. The most common case is the surefire plugin, which loads the needed dependency depending on your tests (e.g. JUnit4/5, TestNG, etc.).
Lastly, you need to run this plugin from your Dockerfile. This should work:
### STAGE 1: Build ###
FROM maven:3.6.1-jdk-8 as build
WORKDIR /usr/src/app
# Install app dependencies
COPY pom.xml .
RUN mvn -e -B de.qaware.maven:go-offline-maven-plugin:resolve-dependencies
# Bundle app source
COPY src src
RUN mvn -o -e -B package
### STAGE 2: Production Environment ###
FROM jboss/wildfly:17.0.0.Final
COPY --from=build /usr/src/app/target/Appname.war /opt/jboss/wildfly/standalone/deployments/Appname.war
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin123 --silent
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]

If I run a plugin from the command line at the root pom, what does it do when traversing modules?

I'm trying to use the plugin: http://gatling.io/docs/2.0.1/extensions/maven_plugin.html
I'm trying to run this command at the root pom:
mvn package gatling:execute
This multi module project only has this plugin defined in one child pom like so:
<plugins>
<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.1</version>
<configuration>
<args>
<arg>-deprecation</arg>
<arg>-feature</arg>
<arg>-language:postfixOps</arg>
</args>
</configuration>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
But when I run this command at the root, it gives this error:
[INFO] ------------------------------------------------------------------------
[INFO] Building root-pom 2.5.210-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- gatling-maven-plugin:2.1.7:execute (default-cli) # root-pom ---
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at io.gatling.mojo.MainWithArgsInFile.runMain(MainWithArgsInFile.java:50)
at io.gatling.mojo.MainWithArgsInFile.main(MainWithArgsInFile.java:33)
Caused by: java.lang.RuntimeException: Can't find the jar matching (.*scala-library.*\.jar)$
at io.gatling.compiler.ZincCompiler$$anonfun$jarMatching$1$2.apply(ZincCompiler.scala:88)
at io.gatling.compiler.ZincCompiler$$anonfun$jarMatching$1$2.apply(ZincCompiler.scala:88)
at scala.Option.getOrElse(Option.scala:120)
at io.gatling.compiler.ZincCompiler$.jarMatching$1(ZincCompiler.scala:88)
at io.gatling.compiler.ZincCompiler$.setupZincCompiler(ZincCompiler.scala:91)
at io.gatling.compiler.ZincCompiler$delayedInit$body.apply(ZincCompiler.scala:106)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
at scala.App$class.main(App.scala:71)
at io.gatling.compiler.ZincCompiler$.main(ZincCompiler.scala:35)
at io.gatling.compiler.ZincCompiler.main(ZincCompiler.scala)
... 6 more
This is not a scala project, so this is definitely related to the plugin. Running mvn package without gatling:execute works fine.
I do not know enough about maven to be able to troubleshoot this problem because I can't really tell what maven is trying to do here. If it is trying to run gatling at the root, why? That plugin is not defined in the root pom anywhere. My question is:
When you run a plugin at the root of a multi-module project, how does it traverse my project, look for plugins and run them? I'm asking this question abstractly. You don't have to answer in terms of gatling. I just want to understand maven plugins better.
How do I troubleshoot this issue? I think I could add scala to the root pom, but since it would be the first dependency at that level, it seems like it's the wrong approach.
I could also cd into the one child pom that has this plugin and run the command there, but that means I'll only be running package on a piece of the project. Ideally I'd like to run package on the whole project before I run the plugin.

Running maven with a plugin goal like that will run that goal for all project modules, the same as mvn install will install each module. You don't have to define the plugin anywhere - if you specify it on the commandline, it is run.
You correctly bound the execution of that plugin to one or more lifecycle phases, in this case process-resources and process-test-resources. If you run mvn compile, the the process-resources phase will be executed and the add-sources and compile goals will be run for the gatling plugin. Similarly, running mvn test (which is also run when you mvn install) will run the testCompile goal of the gatling plugin.
What you'll want to do is to also bind the execute goal as is exemplified
on the page you linked. If you want everyting packaged first, bind it to the
integration test phase:
<execution>
<phase>integration-test</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
....
</configuration>
</execution>
Note that integration-test comes after package (and somewhere before install), so to run it, execute mvn integration-test or mvn install.

maven debugging in Vagrant - HADOOP YARN application

I have installed Vagrant as a VM (single node) from this repository and then I installed maven inside so that I can create easily my HADOOP YARN jar files. When I am trying to build my jar mvn package I am getting an error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile (default-compile) on project YarnTestClient: Compilation failure: Compilation failure:
[ERROR] /vagrant/openlab_ship_proto/YARN/src/main/java/ru/yandex/cern/yarntest/NMCallbackHandler.java:[16,25] generics are not supported in -source 1.3
[ERROR] (use -source 5 or higher to enable generics)
I have searched the whole internet but nothing solved my problem. Had anyone faced the same problem before? Thanks for your help.

Add this snippet to your pom.xml, if build tag didn't exist in pom.xml, maven would build java with source 1.3
<build>
<plugins>
<!-- compiler plugin -->
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.version}</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
<encoding>${project-sourceEncoding}</encoding>
</configuration>
</plugin>
</plugins>
</build>

maven plugin install:install-file error

I use install:install-file to install jar to my local repository.My pom.xml is written as follow:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>2.3.1</version>
<executions>
<execution>
<id>install-paho</id>
<phase>generate-resources</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<file>${basedir}/lib/paho.jar</file>
<groupId>org.eclipse</groupId>
<artifactId>paho</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
</configuration>
</execution>
</executions>
</plugin>
You can find that I binding it to phase 'generate-resources'.And then,I use order mvn eclipse:eclipse.It works very well and the jar was copied to my local repository.But when I use order mvn install:install-file I got the error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.3.1:install-file (default-cli) on project xxx:
The parameters 'file' for goal org.apache.maven.plugins:maven-install-plugin:2.3.1:install-file are missing or invalid -> [Help 1]
The error messages when use mvn compile
[ERROR] Failed to execute goal on project android-engine: Could not resolve dependencies for project com.youku.wireless:android-engine:jar:1.0-SNAPSHOT: Could not find artifact org.eclipse:paho:jar:1.0.0 in spring-milestone (http://maven.springframework.org/milestone) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

Since you have bound install:install-file goal to the generate-sources phase, you should run mvn compile or mvn install or such to use the defined configurations.
mvn eclipse:eclipse works because maven runs the generate-sources phase prior to invoking eclipse:eclipse.
Edit: From the comments it looks like you want to use the locally available paho.jar in your project by first installing it to your local repo in the generate-sources phase and thereafter use it as a dependency in your project.
This is not going to work since maven checks for availability of dependencies before it starts executing its life-cycle goals.
You could manually install it one-time using mvn install:install-file outside the context of the pom. Better still you could deploy it to a repository manager and then access it like any other dependency.
However, if you still want to go down this path, an alternate approach would be to specify the dependency with a system scope providing the path of the jar. Refer to this.
<project>
...
<dependencies>
<dependency>
<groupId>org.eclipse</groupId>
<artifactId>paho</artifactId>
<version>1.0.0/version>
<scope>system</scope>
<systemPath>${basedir}/lib/paho.jar</systemPath>
</dependency>
</dependencies>
...
</project>

Output failed test details to stdout using maven surefire

When I run my build using maven 2
mvn clean install
my tests are run by surefire plug-in. In case test failed I get the following output:
Results :
Failed tests:
test1(com.my.MyClassTest)
Tests run: 3, Failures: 1, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.
Please refer to /home/user/myproject/mymodule/target/surefire-reports for the individual test results.
To get the details about the problem I have to go and check surefire reports folder. Doing this every time my tests fail becomes annoying. Is there any way I can get those details (assert message + exception + stack trace) right here on the stdout ?

I find there's way too much output produced on stdout to be useful. Try leaving the HTML report open in your browser. After running your tests just refresh the page. Have a look at target/surefire-reports/index.html.
To output test results to stdout rather than a file use the following command:
mvn test -Dsurefire.useFile=false
Or to configure in your pom.xml add the following to your plugins section.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.10</version>
<configuration>
<useFile>false</useFile>
</configuration>
</plugin>

It's possible you may be using an older version of Surefire. I have found that newer versions produce more useful output on the console.
If you only want to see failing tests or tests with errors and only see errors in the build, you can pass the -q argument to your Maven build command.

please check your version of jacoco plugin is compatible with jdk or not:
i was getting similar issue , test cases were passing but it was failing while building project. so upgraded version worked for me:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.4</version>
<executions>
<execution>
<id>jacoco-initialize</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>jacoco-site</id>
<phase>package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>

try using bellow maven command
mvn clean install -DskipTests
-DskipTests compiles the tests, but skips running them