I have the following multi-stage dockerfile that attempts to copy each module of my multi-module java maven project and build my ear file using mvn clean install.
dockerfile:
# Copy files from local to maven image and build ear
FROM maven:3.5-jdk-8 AS build
COPY module1 /usr/src/app/src
COPY module2 /usr/src/app/src
COPY module3 /usr/src/app/src
COPY pom.xml /usr/src/app
RUN mvn -f /usr/src/app/pom.xml clean install
# Create server image + rest of docker file (working ok)
The error that I am getting is as follows:
Step 8/20 : RUN mvn -f /usr/src/app/pom.xml clean install
---> Running in cf9d8c1ef9ed
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for com.company.web:api:21.01-SNAPSHOT:
Could not find artifact com.company.parent:jee6:pom:1.0.1-SNAPSHOT and
'parent.relativePath' points at wrong local POM # line 8, column 10
#
[ERROR] The build could not read 1 project -> [Help 1]
The section in the pom.xml that corresponds to this error:
<parent>
<groupId>com.company.parent</groupId>
<artifactId>jee6</artifactId>
<version>1.0.1-SNAPSHOT</version>
</parent>
I assume this issue because when trying to run the command in the maven docker image it cannot see my local .m2 folder? And the dependency that it is looking for is a private dependency on my local machine.
Would also copying my maven settings.xml help?
How can I resolve this? I do not want someone to have to have Maven installed on their machine in order to run this dockerfile.
Dependency com.company.parent:jee6:pom:1.0.1-SNAPSHOT seems to be private, your Maven command inside Docker build needs to be able to either download it from private repository or have it readily accessible.
I assume this issue because when trying to run the command in the maven docker image it cannot see my local .m2 folder?
Yes, it then cannot see your settings.xml with private repository config, or local dependency if it's already available locally.
Would also copying my maven settings.xml help?
It's better not to: your settings.xml (and eventual secrets within) may be available to anyone using your image later. Using a secret mount with BuildKit would be a better solution (see below)
You have multiple solutions:
Mount settings.xml as secret during build
This solution assumes you have a settings.xml configured with proper credentials to access private registry.
Use Docker BuildKit with --mount=secret to load settings.xml as secret with a Dockerfile such as:
# syntax=docker/dockerfile:1.2
# Required comment at top of Dockerfile for using BuildKit
FROM maven:3.5-jdk-8 AS build
COPY module1 /usr/src/app/src
COPY module2 /usr/src/app/src
COPY module3 /usr/src/app/src
COPY pom.xml /usr/src/app
# Use your secret settings.xml
RUN --mount=type=secret,id=mvnsettings,target=/root/.m2/settings.xml \
mvn -f /usr/src/app/pom.xml clean install
And build command such as:
DOCKER_BUILDKIT=1 docker build --secret id=mvnsettings,src=$HOME/.m2/settings.xml .
Maven should then be able to download parent dependency during build.
Note: this is NOT COPYing the settings.xml in image, as the secret settings.xml will only be made available for the specified build step and won't be persisted in final image.
Copy com.company.parent:jee6 pom.xml during build
This solution is less practical and may not solve problem entirely:
It would require to have com.company.parent:jee6:pom:1.0.1-SNAPSHOT pom.xml file available in build context
Your parent pom.xml may refer to other private dependencies. You would have to include them the same way.
... But it still may be worth a try.
You can do something like:
FROM maven:3.5-jdk-8 AS build
# Copy and install parent pom
COPY parent-pom.xml /tmp/parent/pom.xml
RUN mvn -f /tmp/parent/pom.xml clean install
COPY module1 /usr/src/app/src
COPY module2 /usr/src/app/src
COPY module3 /usr/src/app/src
COPY pom.xml /usr/src/app
RUN mvn -f /usr/src/app/pom.xml clean install
There are 2 different parts to the error: Cannot find the parent, and the parent.relativePath element is wrong. I think the 2nd part might be causing the 1st.
Since your parent element doesn't specify a relativePath element, the default is the module parent path (aka ..). Your modules are not in a child of the parent folder (/usr/src/app) but rather in the (/usr/src/app/src subfolder).
Try changing your copy commands to :
COPY module1 /usr/src/app
COPY module2 /usr/src/app
COPY module3 /usr/src/app
COPY pom.xml /usr/src/app
You should then see:
/usr/src/app
/usr/src/app/module1
/usr/src/app/module2
/usr/src/app/module3
Related
I am working on a multi module project and trying to deploy them to JFROG.
Project Structure goes like this:
project
module-1
pom.xml
module-2
pom.xml
In the parent pom:
<modules>
module-1
module-2
<modules>
I have added <distributionManagement> and added the required details to connect to my artifactory.
Verified that there are no connectivity issues to Artifactory.
Module-1 gives com/abc/def/project/module-1/module-1.pom and com/abc/def/project/module-1/module-1.jar
Module-2 gives com/abc/def/project/module-2/module-2.pom and com/abc/def/project/module-2/module-2.jar
Requirement: I have to push the above pom and jar of both the sub modules to JFROG.
I am using goal - mvn clean deploy from my jenkins pipeline. However it is not working.
The best way to do that is to use the JFrog CLI. With that, you don't have to configure any resolution or deployment repositories in your pom.xml/settings.xml.
You can do that in 4 simple steps:
# Step 1: Install JFrog CLI
curl -fL https://install-cli.jfrog.io | sh
# Step 2: Configure the JFrog platform:
jf c add internal --url=https://acme.jfrog.io --user=jfrog_user --password=jfrog_password
# Step 3: Configure the Maven project
jf mvnc --repo-resolve-releases libs-release --repo-resolve-snapshots libs-snapshot --repo-deploy-snapshots libs-snapshot-local --repo-deploy-releases libs-release-local
# Step 4: Install and publish your Maven project
jf mvn clean install
Additionally, it is recommended to save the build-info in Artifactory.
To do that, set the JFROG_CLI_BUILD_NAME and JFROG_CLI_BUILD_NUMBER environment variables before step 4 and run jf rt build-publish in the end.
Read more:
QuickStart Guide: Maven and Gradle
Running Maven Builds
JFrog CLI Documentation
I have to build a docker image and package java application using maven in the docker container but when I run the build process all is going fine but all maven dependencies downloading from maven remote repo.
That is my docker file:
FROM ubuntu_img
CMD ./mvnw -s .mvn/settings.xml --batch-mode clean package
How can I configure docker or maven for downloading dependencies from maven local repository located on my laptop?
At first, you need to attach the directory of your existing local Maven repository into the Docker container:
VOLUME ["/home/<user>/.m2", "/root/.m2"]
Then you need to tell Maven (inside your container) to use this directory as a local repository.
setting.xml
<settings ...>
<localRepository>/root/.m2</localRepository>
...
</settings>
use volume,like this:
VOLUME ["/home/test/.m2", "/root/.m2"]
My project structure (multi module) is like this
parent
projectA
projectB
... other modules
parent also actually has a parent (Spring Boot).
I have set up a Jenkins jobs to compile & test on every commit, so it runs:
mvn -f pom.xml clean install
And that all works fine. ProjectB depends on ProjectA (which is like a common classes type of project) and is a Spring boot application. So the dependency information is the regular:
<dependency>
<groupId>Group</groupId>
<artifactId>ProjectA</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
ProjectB has a separate job in Jenkins to build the deployable jar file and deploy it to server. So the command there is:
mvn -f ProjectB/pom.xml clean install antrun:run
This fails with a message like:
[WARNING] The POM for Group:ProjectB:1.0-SNAPSHOT is missing, no dependency information available
...
[ERROR] Failed to execute goal on project host-api: Could not resolve dependencies for project Group:ProjectB:1.0-SNAPSHOT: The following artifacts could not be resolved: Group:ProjectA:jar:1.0-SNAPSHOT...
Now I can resolve this by doing a mvn install in the ProjectA directory - I've tested this and it does resolve the issue.
But my question is why should I have to? Shouldn't Maven figure out it should be installing the jar in the local repo?
Thanks in advance
TL;DR Tell maven about the structure of your project.
When you run the command
mvn -f pom.xml clean install
Then maven uses the reactor to work out the order of the modules, something like the following is output:
[INFO] Reactor build order:
[INFO] ProjectA
[INFO] ProjectB
So Maven first builds project A, then builds project B.
When you run the command:
mvn -f ProjectB/pom.xml clean install antrun:run
Then you have a problem; maven isn't starting from the parent - it's starting from a child. It's not told about the hierarchy of projects needed to be built first.
If you want to build a single project from a maven multimodule project, along with dependencies you should use:
mvn -f pom.xml -pl ProjectB -am install antrun:run
Where:
-pl ProjectB is the "project list" option, it tells maven to build these specific projects
-am is the "also make" option, it tells maven to build any projects that the projects in pl are dependant on
Specify the dependencies build part when you run the build:
Guide to Working with Multiple Modules
So I suppose the rule is, always build the parent project first and then run goals on subprojects afterwards.
The fix for me was to run clean install on the parent project first and then have a second build configuration in Jenkins that ran -f ProjectB/pom.xml antrun:run
Is there a way to run mvn archetype:generate and target the current directory instead of creating a directory from the artifactId? The plugin supposedly takes a basedir parameter, but passing in "-Dbasedir=." doesn't do the trick.
For additional context, I've got 2 Git repositories setup. The first contains the source of a Maven archetype for generating custom web services, and the second is a sample service generated from the archetype. I've got a Jenkins job that builds my archetype by essentially just running "mvn clean install". I'm trying to setup a second Jenkins job as part of our CI workflow generates a test service using "mvn archetype:generate", builds the service with mvn clean install, spins up the service, runs some integration tests, and then pushes the source for the test service into a second repository if the tests pass. Both Jenkins jobs use the Maven 2/3 build job type, and I've specified our Git repo information in the SCM section of the job configuration, so the jobs start by doing a "git clone".
For the second job, my current workflow looks like this:
// Clean out the existing source and commit locally.
git rm -r .
git commit -m "Cleaning out previous version." .
// Generate the new source from the archetype.
mvn archetype:generate ...
// Big hack that I'd like to remove.
mv <artifactId>/* .
rm -rf <artifactId>
// Add the new generated source and commit locally.
git add -A .
git commit -m "Committing new version." .
// Build and test.
mvn integration-test
// Assuming that passed, commit the changes.
git push origin master
The hack is there because I'm not sure how to tell the archetype plugin to use my current directory instead of creating a nested directory with the name of the artifactId, so I have to move everything back into my local repository root directory and delete the directory that was created. Not the end of the world, but it feels ugly. Note that I'm open to suggestions on how to better accomplish my overall goal in addition to answers to my initial question. :) Thanks in advance!
It is not possible without some kind of workaround, because of 2 reasons:
The basedir property doesn't work, see this issue.
The archetype plugin always adds artifactId as the last child directory. You can see this in the sources.
Your options
keep your hack, it is not so horrible ;-)
is it important that the project is in the root of the repository? if not just do cd <artifactId> before the build/integration tests
if building from root of the repository is required then use root pom.xml of type pom, with the generated project as child module (I think this is what other post suggests as #3)
provide a PR to add desired functionality to maven archetype plugin, this might take some time, but is probably the cleanest solution
I know it's old topic but I had similar issue:
The archetype:generate goal has a parameter outputDirectory, you can set it to root folder (../). When generating artifact from archetype it's important that your artifact name is same as current directory.
mvn archetype:generate -DarchetypeGroupId=[archetype-group] -DarchetypeArtifactId=[archetype-id] -DoutputDirectory=../ -DartifactId=[current-folder-name] -DgroupId=[whatever-you-want] -Dversion=[ex: 0.1] -Dpackage=[jar|pom|etc.] -B
I have three ideas:
You can try to use an absolute path for basedir but I think the property is overwritten by Maven when it sees a pom.xml file.
Use a script and pushd .. to change to the parent folder before you run mvn archetype:generate. popd will get you back.
Use a maven module build. That way, you have a root POM which builds the archetype and you can run mvn archetype:generate to generate a module inside of the project:
demo-project/
pom.xml
archetype/
pom.xml
I don't have the reputation to comment so I'll have to make an answer instead.
It's not a good idea to have these lines in your script:
mv <artifactId>/* .
rm -rf <artifactId>
If someone decides to fill in an empty (or just '/') artifactId you end up trying to move the whole disk into this directory, only to then delete the entire disk. Instead you can do this:
mv ./<artifactId>/* .
rmdir ./<artifactId>
The move can't hurt and the rmdir can only delete empty directories.
I have copied the JARs into src\main\webapp\WEB-INF\lib.
I use eclipse. If I add the jars one-by-one to Project-> Java Build Path-> Add jars, then I do Project-> Maven-> Update Project Configuration, they are removed by Maven. And Eclipse shows errors, which contain " xxx cannot be resolved".
Env:
Eclipse Java EE IDE for Web Developers.
Version: Indigo Service Release 1
Build id: 20110916-0149
m2e - Maven Integration for Eclipse 1.0.100.20110804-1717
Note: I don't want to create my own Maven repository. It will be just used once.
How should I proceed ?
Not all libraries can be found in a public Maven repository, for example your own libraries or proprietary libraries. Anyway first you can search the Maven Repository, maybe some of them are there.
In case they are really not there, and you don't want to install Nexus or Artifactory, you can choose one of these two approaches:
install the jars in your local repository cache, no need to create a special repository. See the instructions on mkyong.
another approach is using system dependencies, you just put a path to reach the Jar in the filesystem. It's a less recommended approach, but if you really want it will work. Here is the official documentation.
Add this an a Systems Dependency.
In case you want to add this (this uses jquantlib as example) to the maven local repo use:
mvn install:install-file -Dfile=./jquantlib-0.2.4.jar -DgroupId=org.jquantlib -DartifactId=jquantlib -Dversion=0.2.4 -Dpackaging=jar
In order to do this I use a batch file
#echo off
cd lib
CMD /C "mvn install:install-file -Dfile=./jquantlib-0.2.4.jar -DgroupId=org.jquantlib -DartifactId=jquantlib -Dversion=0.2.4 -Dpackaging=jar"
CMD /C "mvn install:install-file -Dfile=./jquantlib-helpers-0.2.4.jar -DgroupId=org.jquantlib -DartifactId=jquantlib-helpers -Dversion=0.2.4 -Dpackaging=jar"
CMD /C "mvn install:install-file -Dfile=./jquantlib-samples-0.2.4-ubber.jar -DgroupId=org.jquantlib -DartifactId=jquantlib-ubber -Dversion=0.2.4 -Dpackaging=jar"
CMD /C "mvn install:install-file -Dfile=./jquantlib-samples-0.2.4.jar -DgroupId=org.jquantlib -DartifactId=jquantlib-samples -Dversion=0.2.4 -Dpackaging=jar"
CMD /C "mvn install:install-file -Dfile=./ta-lib-0.4.0.jar -DgroupId=com.tictactec -DartifactId=ta-lib -Dversion=0.4.0 -Dpackaging=jar"
You can use a similar script file on other systems.
I believe the system dependency approach shouldn't be used unless you don't have any other choice, and that's because you're loosing the whole 'build portability thing' here.
Of course you can store your jars in your source control system together with your project's source files, but I don't think its a good approach neither...
Using only install:install-file is not good enough - this would indeed deploy the jars in the proper format into your local repository, but what happens when you'll move to another computer and start to build your project there?
You will need to make this once more.
So, If you don't want to install nexus/artifactory (which is the best solution, I believe), you probably should create an another repository (just in a file system on some of your servers), and deploy the jars there (you can use mvn install:install-file as was suggested here, and then just copy the whole tree). Now you can configure apache web server and access the directory with all your jars via http. I don't believe its better then nexus/artifactory approach, but it can be a little be easier to do if you're familiar with apache web server.
In order to get your maven aware about this new repository you'll need to edit the %MAVEN_HOME%\conf\settings.xml file
There are atleast three approaches in which 3rd party JARs can be added to Maven projects.
Install manually using mvn install command
Adding the location of jar file in pom dependency with the the following tag system
Creating a 'dummy' maven repository pointing to jar location.
While approach 1 and 2 has been suggested above, I will focus on third approach which I find more cleaner and does not require any mvn command and works out of box from any IDE.
Step 1: Add the location of local 'dummy' repository in pom.xml
<repositories>
<repository>
<id>repo</id>
<name>repo</name>
<url>file:${project.basedir}/src/main/resources/lib</url>
</repository>
</repositories>
Here the 'dummy' repository location is the 'lib' folder of my project directory
Step 2 : Add the jar dependency into your pom.xml
<dependency>
<groupId>com.cloudera.impala</groupId>
<artifactId>impala-frontend</artifactId>
<version>0.1-SNAPSHOT</version>
</dependency>
choose any groupId but make sure that artifactId and version is of the format <artifactId>-<version>.jar ( Name of 3rd party jar)
Step 3 : Create the folder structure as per the groupId,artifactId and version mentioned in the Step 2 in your local 'dummy' repository. So in this case the folder struction would be /src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/
Place your jar in the version folder and build your project.
You will get the following output which treats your 'dummy' repository to be the provider of your 3rd party jar.
[INFO] Downloading from repo: file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/maven-metadata.xml
[INFO] Downloading from repo: file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.pom
[WARNING] The POM for com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT is missing, no dependency information available
[INFO] Downloading from repo: file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.jar
[WARNING] Could not validate integrity of download from file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.jar: Checksum validation failed, no checksums available
[WARNING] Checksum validation failed, no checksums available from repo for file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.jar
[INFO] Downloaded from repo: file:C:\Users\skumar\eclipse-workspace\chdQueryBuilder/src/main/resources/lib/com/cloudera/impala/impala-frontend/0.1-SNAPSHOT/impala-frontend-0.1-SNAPSHOT.jar (7.0 MB at 79 MB/s)
[INFO]
To add external JAR files not in the local repository simply right click on your main source folder and from the build path menu select: "configure build path", then navigate to the libraries tab and click "add external JAR files". next, locate the JAR (or zip) file you would like to add as a library and click ok.
Congratulations, you have now successfully added an external JAR (or zip) to your build path and you can now import any classes from that JAR file in your project without throwing an errorPicture How-To :) http://hostthenpost.org/uploads/541be8420657320c74489ff8d456ad08.png