I am attempting to build a Maven Build Pipeline in Azure Devops.
I have one project that has files shared across several others (as set of micro service projects), so the maven compile for that project creates a .jar file ( which is added to my local maven repository .m2) when I do the build on my laptop. We can call it shared.jar for now.
Then when I compile the micro services on my laptop, they each have a dependency on the jar file in my .m2 repo. And they pick up the dependency from there and the Maven build (mvn clean install).
Unfortunately, The code build pipeline does not maintain a local .m2 repo. So when I try to do a “mvn clean install” on each micro service they cannot locate shared.jar, so the build fails.
I have been able to successfully add the .jar file to a Feed and Artifact under Azure DevOps, but I can't seem to figure out how to pull it into the Micro Services Build.
HOW do I pick up the dependency for a local jar file in my CI/CD Maven build pipeline and get it into a maven repo that my build will find?
I have attempted to follow this link: https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-maven?view=azure-devops
We do this by having a separate azure pipeline to build the dependency which deploys it to an Azure feed:
variables:
- name: mavenRepoURL
value: 'https://myorg.pkgs.visualstudio.com/myproject/_packaging/myfeed/maven/v1'
- task: DownloadSecureFile#1
name: mvnSettings
displayName: 'Download Maven settings'
inputs:
secureFile: 'maven-azuredevops-settings.xml'
- task: MavenAuthenticate#0
displayName: Maven Authenticate Artifacts
inputs:
artifactsFeeds: 'myfeed'
- task: Maven#3
inputs:
mavenPomFile: 'pom.xml'
options: '-X -B -s $(mvnSettings.secureFilePath) -DWHERE="AzureDevops" -DremoteRepositories=$(mavenRepoUrl) clean deploy -U'
mavenAuthenticateFeed: true
publishJUnitResults: true
testResultsFiles: '**/TEST-*.xml'
Note that its important that your personal access token is not stored in version control for security reasons hence the complication of having to upload it as a secure file and download it in the pipeline.
The dependency is added to the pom for the project using it as normal but with the Azure feed configured as well
<dependency>
<groupId>com.foobar.blah</groupId>
<artifactId>artifactId</artifactId>
<version>2.0.2</version>
</dependency>
<profile>
<id>AzureDevops</id>
<activation>
<property>
<name>WHERE</name>
<value>AzureDevops</value>
</property>
</activation>
<properties>
</properties>
<distributionManagement>
<repository>
<id>myfeedname</id>
<url>https://myorg.pkgs.visualstudio.com/_packaging/myfeed/maven/v1</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</distributionManagement>
</profile>
Caveat: I am not that familiar with maven or Azure so some things may not be right. For example, I have issues with project rather than organisation scoped feeds. See What are the equivalent maven commands and settings to upload and download azure artifacts?
I just had to do this myself to create a docker image whit the final .jar and got inspired by Bruce Adams answer and the documentation about maven
you have to add to your pipeline.yml
- task: DownloadSecureFile#1
name: mvnSettings
displayName: 'Download Maven settings'
inputs:
secureFile: 'maven-azuredevops-settings.xml'
- task: MavenAuthenticate#0
displayName: Maven Authenticate Artifacts
inputs:
artifactsFeeds: 'yourFeed'
- task: Maven#3
inputs:
mavenPomFile: 'pom.xml'
goals: 'deploy'
options: '-s $(mvnSettings.secureFilePath)'
mavenAuthenticateFeed: true
publishJUnitResults: true
testResultsFiles: '**/TEST-*.xml'
javaHomeOption: 'JDKVersion'
jdkVersionOption: '1.11'
mavenVersionOption: 'Default'
and in your pom.xml you need to add the configuration to connect to your artifacts feed
<repositories>
<repository>
<id>yourFeed</id>
<url>https://yourOrganization.pkgs.visualstudio.com/yourProject/_packaging/yourFeed/maven/v1</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<distributionManagement>
<repository>
<id>bithaus</id>
<url>https://yourOrganization.pkgs.visualstudio.com/yourProject/_packaging/yourFeed/maven/v1</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</distributionManagement>
and create the file setting.xml on your {user.home}/.m2/ folder, and upload it to the secure files on your pipeline.
Hope it helps someone.
please check following itemes:
Make sure that when you run "mvn clean install" command, maven read
configurations from settings.xml file in ${user.home}/.m2 .
Check id, username and password whitin <server></server> tags.
Please before running "mvn clean install" remove all files and
folder in .m2 folder.
enable debugging option when run maven install command by add -X or
--debug options.
Related
I wanted to create a maven project using "mvn archetype:generate ..." to create a maven project Skeleton. I looked up online and come up with this:
in settings.xml
<profile>
<id>PlatMigr</id>
<repositories>
<repository>
<id>internal repo</id>
<name>IT Maven Central</name>
<url>https://<internal maven repo></url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</profile>
From command line:
mvn archetype:generate -DarchetypeCatalog=internal -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId=com.mycompany.myproj -DartifactId=ProjName
However maven still tried to download artifacts from official central maven repo:
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom
We have a firewall installed to stop internet access, that's why I struggle to use internal repo in the first place. Anyone can advise on how to setup and use internal repo to create maven projects?
Edit you settings.xml in m2_home, add mirror to you nexus and create & set to active profile without central repositories
I am working on Maven project and I have a jar app-client.jar which have dependency on app-core.jar. So I have a pom.xml for app-client.jar and that pom.xml has dependency of app-core so we added dependency of app-core in this pom.xml.
Now I wanna use the app-client.jar in my main project. Because this jar is build locally and not available at remote repository. So I did add the app-client and also specify the location repository where it will located.
as following..
<repositories>
<repository>
<id>repo</id>
<releases>
<enabled>true</enabled>
<checksumPolicy>ignore</checksumPolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
<url>file://${project.basedir}/../lib</url>
</repository>
<repositories>
<dependencies>
<dependency>
<groupId>com.sample</groupId>
<artifactId>app-client</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
and also I put my jar as following
[My Module] / [com] / [sample] /[app-client] /[1.0]/app-client-1.0.jar
When I run mvn clean install I got error app-client's pom.xml not found. and build get failed. Usually when I use single jar then its working fine, but if I use jar having dependency with other jar it getting failed.
So how can I build my app-client jar and their pom so that it behave normal and also deploy app-core.jar too.
firstly when you are building app-client.jar build a fat jar which includes app-core.jar dependency.
Next copying app-client-1.0.jar into the specified location of local repo doesn't work, to add this jar into your local repo use this command mvn install:install-file -Dfile=<path-to-file> -DgroupId=<group-id> -DartifactId=<artifact-id> -Dversion=<version> -Dpackaging=<packaging>.
If you build app-client.jar with mvn clean install, then it will be installed into your Maven local repository (.../.m2/repository). Then any other project on the same computer can reference it in its pom without further information. So no <repositories> entry necessary.
If you want to work with multiple people on the same project, use a Nexus/Artifactory server to share the jars.
Solutions with lib folders and system paths are deprecated and cause trouble.
My problem is that I can't create a new Maven project after adding my Nexus repository to the .m2 settings.xml.
I've installed Nexus Repositoy Manager OSS 3.0.2 as my local Maven repository. I have a machine which is in offline mode and can not connect to the internet. What I can do is transfer data from an online machine to it thought. So I can deploy all necessary libraries on the online machine and just switch the /data folder later on.
Error after creating a new Maven project with Eclipse:
Could not calculate build plan: Plugin org.apache.maven.plugins:maven- >resources-plugin:2.6 or one of its dependencies could not be resolved: >Failed to read artifact descriptor for org.apache.maven.plugins:maven- >resources-plugin:jar:2.6
The maven-resources-plugin-2.6.jar is deployed/available on my Nexus Repository: Path org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.jar
My Maven settings.xml (partly):
<mirrors>
<mirror>
<!--This sends everything else to /public -->
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://localhost:8081/repository/maven-test/</url>
</mirror>
</mirrors>
...
<profile>
<id>nexus</id>
<repositories>
<repository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>http://central</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
...
<server>
<id>nexus</id>
<username>admin</username>
<password>admin123</password>
</server>
As you can see my repository is named "maven-test". Is it normal that the Nexus repositories are not browseable by their URLs (http://localhost:8081/repository/NAME_OF_REPO/)?
Is there any documentation which Maven libraries are mandatory to create a simple Maven project. The minimum amount of .jars?
What I've tried so far:
created a simple Maven project on a internet connected machine and downloaded all necessary Libraries with:
mvn dependency:go-offline. After that I deployed all .jars from my local m2 repository to Nexus (about 160 Jars - luckily I used a Shell script for the deployment). For the offline simulation I deleted my local repository and went offline. Now the error occurs after creating a new Maven project in Eclipse.
I'm using Maven 3.3.9 - Java 1.8 - Eclipse 4.4 - Mac OS
Please any help would be appreciated!
Thanks
[Edit] forgot to upload the .pom files to Nexus. Clean install worked for a existing project. Still got the error after creating a new Maven project
Run a mvn clean install on the project without using the nexus tool.
This will download all the required jars and poms to your local repository as specified in your maven settings.xml
Copy all of these files to your offline nexus installation.
The caveat with any new projects you create each time is that they may require new jars and poms that weren't used by the first project and so were never downloaded and copied to your Nexus install which will give you issues.
Our internal repository (Artifactory) now contains both the stable builds as well as SNAPSHOT versions of our internal libraries.
For stable builds there has never been a problem of downloading anything from the repository.
However, when I add a -SNAPSHOT, Maven claims to be unable to find the dependency, even though it is most definitely in the repository.
If I build and deploy the dependency locally (i.e. into my local repo) all works normally.
Basically, this works:
<dependency>
<groupId>com.example</groupId>
<artifactId>ourlibrary</artifactId>
<version>1.0.0</version>
</dependency>
and this doesn't:
<dependency>
<groupId>com.example</groupId>
<artifactId>ourlibrary</artifactId>
<version>1.0.1-SNAPSHOT</version>
</dependency>
Even though both versions were built the same way and deployed (as far as I can possibly tell) correctly to the repository.
The error:
Missing:
----------
1) com.example:ourlibrary:jar:1.0.1-SNAPSHOT,
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=com.example -DartifactId=ourlibrary -Dversion=1.0.1-SNAPSHOT, -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=com.example -DartifactId=ourlibrary -Dversion=1.0.1-SNAPSHOT, -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1) com.example:product:war:2.0.0-SNAPSHOT
2) com.example:ourlibrary:jar:1.0.1-SNAPSHOT,
While this sounds similar to this question, the resolution arrived at there does not apply to my case.
Any insights into this issue would be greatly appreciated.
Edit
Running with -X (as John V. suggested) revealed the following:
[DEBUG] Skipping disabled repository central
[DEBUG] ourlibrary: using locally installed snapshot
[DEBUG] Skipping disabled repository central
[DEBUG] Using mirror: http://repo.example.com/repo (id: repo.example.com)
[DEBUG] Artifact not found - using stub model: Unable to download the artifact from any repository
com.example:ourlibrary:pom:1.0.1-SNAPSHOT
from the specified remote repositories:
repo.example.com (http://repo.example.com/repo)
[DEBUG] Using defaults for missing POM com.example:ourlibrary:pom:1.0.1-SNAPSHOT:compile
[DEBUG] com.example:ourlibrary:jar:1.0.1-SNAPSHOT:compile (selected for compile)
Two thoughts come to mind:
The path structure in your internal
repository for your artifact is
incorrect. I suggest running the
maven command with -X parameter. It
will display the maven's attempt at
downloading the files. Get the line
that has your repository as the url
and try and look for it yourself.
The path should look like
/com/example/ourlibrary/1.0.1/ourlibrary-1.0.1-SNAPSHOT.jar
You didnt include your repository as
a repository in your pom.xml
Typically you have a separate snapshots url from releases url. Just different paths in the same repository, but listed as separate repositories in the pom. The one for snapshots needs to have snapshots enabled, and the one for releases has snapshots disabled:
<repositories>
<repository>
<id>central</id>
<url>
http://<releases-url>
</url>
**<snapshots>
<enabled>false</enabled>
</snapshots>**
</repository>
<repository>
<id>snapshots</id>
<url>
http://<snapshots-url>
</url>
<snapshots>
**<enabled>true</enabled>**
<!-- never, daily, interval:X (where X is in minutes) or always -->
<!--<updatePolicy>daily</updatePolicy> -->
</snapshots>
<releases>
<enabled>false</enabled>
</releases>
</repository>
</repositories>
I have just checked out some projects and need to build them, however I installed Maven quite some time ago (6 months maybe?) and really haven't used it since - the pom.xml for the project I have doesn't have this "http://repo1.maven.org/myurlhere" anywhere in it - it has the absolute url where the Maven repo is for the project, but Maven is still trying to download from the general Maven repo:
Macintosh:trunk$ mvn clean install
[INFO] Scanning for projects...
Downloading: http://repo1.maven.org/url/project/project/x.x/project-x.x.pom
[INFO] Unable to find resource 'url.project:project:pom:x.x' in repository central (http://repo1.maven.org/)
[INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.
GroupId: url.project
ArtifactId: project
Version: x.x
Reason: Unable to download the artifact from any repository
url.project:project:pom:x.x
from the specified remote repositories:
central (http://repo1.maven.org/)
Can anyone help me with what I'm not doing right?
Basically, I have just checked the projects out from the command line, cd-ed into the directory and ran mvn clean install - nothing else.
Any help is greatly appreciated.
the pom.xml for the project I have doesn't have this "http://repo1.maven.org/myurlhere" anywhere in it
All projects have http://repo1.maven.org/ declared as <repository> (and <pluginRepository>) by default. This repository, which is called the central repository, is inherited like others default settings from the "Super POM" (all projects inherit from the Super POM). So a POM is actually a combination of the Super POM, any parent POMs and the current POM. This combination is called the "effective POM" and can be printed using the effective-pom goal of the Maven Help plugin (useful for debugging).
And indeed, if you run:
mvn help:effective-pom
You'll see at least the following:
<repositories>
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>Maven Repository Switchboard</name>
<url>http://repo1.maven.org/maven2</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<releases>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>Maven Plugin Repository</name>
<url>http://repo1.maven.org/maven2</url>
</pluginRepository>
</pluginRepositories>
it has the absolute url where the maven repo is for the project but maven is still trying to download from the general maven repo
Maven will try to find dependencies in all repositories declared, including in the central one which is there by default as we saw. But, according to the trace you are showing, you only have one repository defined (the central repository) or maven would print something like this:
Reason: Unable to download the artifact from any repository
url.project:project:pom:x.x
from the specified remote repositories:
central (http://repo1.maven.org/),
another-repository (http://another/repository)
So, basically, maven is unable to find the url.project:project:pom:x.x because it is not available in central.
But without knowing which project you've checked out (it has maybe specific instructions) or which dependency is missing (it can maybe be found in another repository), it's impossible to help you further.
By default, Maven will always look in the official Maven repository, which is http://repo1.maven.org.
When Maven tries to build a project, it will look in your local repository (by default ~/.m2/repository but you can configure it by changing the <localRepository> value in your ~/.m2/settings.xml) to find any dependency, plugin or report defined in your pom.xml. If the adequate artifact is not found in your local repository, it will look in all external repositories configured, starting with the default one, http://repo1.maven.org.
You can configure Maven to avoid this default repository by setting a mirror in your settings.xml file:
<mirrors>
<mirror>
<id>repoMirror</id>
<name>Our mirror for Maven repository</name>
<url>http://the/server/</url>
<mirrorOf>*</mirrorOf>
</mirror>
</mirrors>
This way, instead of contacting http://repo1.maven.org, Maven will contact your entreprise repository (http://the/server in this example).
If you want to add another repository, you can define a new one in your settings.xml file:
<profiles>
<profile>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<repositories>
<repository>
<id>foo.bar</id>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
<url>http://new/repository/server</url>
</repository>
</repositories>
You can see the complete settings.xml model here.
Concerning the clean process, you can ask Maven to run it offline. In this case, Maven will not try to reach any external repositories:
mvn -o clean
tl;dr
All maven POMs inherit from a base Super POM.
The snippet below is part of the Super POM for Maven 3.5.4.
<repositories>
<repository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
I think what you have missed here is this:
https://maven.apache.org/settings.html#Servers
The repositories for download and deployment are defined by the repositories and distributionManagement elements of the POM. However, certain settings such as username and password should not be distributed along with the pom.xml. This type of information should exist on the build server in the settings.xml.
This is the prefered way of using custom repos. So probably what is happening is that the url of this repo is in settings.xml of the build server.
Once you get hold of the url and credentials, you can put them in your machine here: ~/.m2/settings.xml like this:
<settings ...>
.
.
.
<servers>
<server>
<id>internal-repository-group</id>
<username>YOUR-USERNAME-HERE</username>
<password>YOUR-PASSWORD-HERE</password>
</server>
</servers>
</settings>
EDIT:
You then need to refer this repository into project POM. The id internal-repository-group can be used in every project. You can setup multiple repos and credentials setting using different IDs in settings xml.
The advantage of this approach is that project can be shared without worrying about the credentials and don't have to mention the credentials in every project.
Following is a sample pom of a project using "internal-repository-group"
<repositories>
<repository>
<id>internal-repository-group</id>
<name>repo-name</name>
<url>http://project.com/yourrepourl/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</snapshots>
</repository>
</repositories>
Basically, all Maven is telling you is that certain dependencies in your project are not available in the central maven repository. The default is to look in your local .m2 folder (local repository), and then any configured repositories in your POM, and then the central maven repository. Look at the repositories section of the Maven reference.
The problem is that the project that was checked in didn't configure the POM in such a way that all the dependencies could be found and the project could be built from scratch.