How does Maven interacts with different repositories like Nexus and Artifactory?
Is it done using their REST APIs? It is possible to resolve dependent artifacts to actual locations to get them from using these APIs, but the two APIs are somewhat different form each other (as far as i see [Artifactory] [Nexus]) and the repository definition element in Maven pom files does not contain a property to state the type of the repository.
So how does Maven resolves artifact's groupId, artifactId and version to an actual file in different repositories? Does these repositories implement some other standardized API that enables Maven to request files from them in an repository-agnostic way?
Yoy define repositories to search artifacts in 2 ways:
1) in your pom.xml in project section
<project>
....
<repositories>
<repository>
...
<url>...</url>
</repository>
</repositories>
</project>
2) and/or in file $USER_HOME$/.m2/settings.xml (there are plenty examples on WEB)
Maven downloads artifacts from by constructing URL of artifact using repository URL and appending path according to artifact group, id, version, type and classifier. And converts dots in groupId into '/'
RepoUrl/groupId/artifactId/version/artifactId-version-classifier.type
Examples:
https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.4/commons-io-2.4.jar
https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.4/commons-io-2.4-sources.jar
https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.4/commons-io-2.4-javadoc.jar
This is convention and it's independent from Artifactory or Nexus. This is Maven. Everything is transfered through the HTTP or HTTPS. Additionally Maven handles MD5 and SHA1 files for security purposes.
Once downloaded, artifact stored in local repository on your workstation at $USER_HOME$/.m2/repositories (it have a structure similar to remote repository) that acts as a cache and proxy.
Typically Artifactory or Nexus are kind of local cache and proxy at company level with some functionality to manage it. Surely they host maven artifacts published by company developers.
Looks like a 3 levels of repositories: local - local company - global
Maven uses some kind of naming convention. In pom.xml you define a root URL of repository (e.g. http://download.java.net/maven/2/) and then maven is able to resolve an artifact by constructing a URL:
<root URL>/${group id where dots are replaced by slashes}/${artifact id}/${version}
So for the following dependency
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>2.0.0</version>
Maven will try to find poms and jar at http://download.java.net/maven/2/org/apache/lucene/lucene-core/2.0.0
That means every URI which follows this convention may serve as Maven repo.
maven uses normal http calls to interact with repositories. you don't need anything but an http server to serve a maven repository. maven relies on a specific directory hierarchy and a few metadata files. (the local on disk repository where maven caches files it has downloaded is the exact same format).
Related
Since Maven 3.5.0, it's possible to use some variables inside the <version> tag :
https://maven.apache.org/maven-ci-friendly.html
Let's say I have <version>${revision}</version>, in a library project (it produces a jar to be used in other projects, it's not just a web app or a batch application).
When I build and publish my library v1.0.0 (mvn deploy -Drevision=1.0.0-release) the artifact is named "my-library-1.0.0-release.jar", but the pom.xml metadata inside the jar is still at <version>${revision}</version>, are there some use cases where this will make my library unusable ?
EDIT : same question if my library is published in a SNAPSHOT repository and used from there as a dependency to other projects.
There will be problems with using your library as a dependency and publishing to shared artifact repository (e.g. Maven Central) because your pom.xml doesn't match the artifact version. Perhaps some artifact repositories will work (e.g. local Artifactory proxy with custom config) but it's asking for problems.
This is mentioned in the Maven CI Friendly Versions you linked to, under "Install/Deploy" chapter which suggests to use flatten-maven-plugin:
If you like to install or deploy artifacts by using the above setup you have to use the flatten-maven-plugin otherwise you will install/deploy artifacts in your repository which will not be consumable by Maven anymore.
Had this same issue. Solved with this maven extension (in an extension.xml file):
<extensions>
<!-- this extension ensures ${revision} gets replaced with the proper value in the output pom files-->
<extension>
<groupId>fr.jcgay.maven.extension</groupId>
<artifactId>unique-revision-maven-filtering</artifactId>
<version>1.2</version>
</extension>
</extensions>
It's possible to set the maven local repository in settings.xml:
<localRepository>${user.home}/.m2/repository</localRepository>
And it's possible to set the maven local repository on the command line:
mvn clean install -Dmaven.repo.local=repository
Is it possible to specify within the pom.xml itself?
Note: I'd like a way to specify, in the pom.xml, where maven initially searches for artifacts (by default, ~/.m2/repository) and where maven installs artifacts via mvn install (by default, ~/.m2/repository).
According to the Maven POM Reference and the Guide to using multiple repositories, you can specify repositories in pom.xml too.
There are two different ways that you can specify the use of multiple repositories. The first way is to specify in a POM which repositories you want to use
And according to Introduction to repositories, you can use the file:// protocol in <url>.
Remote repositories refer to any other type of repository, accessed by a variety of protocols such as file:// and http://.
So the following works:
<project>
...
<repositories>
<repository>
<id>example-repo</id>
<name>Example Repository</name>
<url>file://path/to/your/local/repository</url>
</repository>
</repositories>
</project>
Edit:
Based on your comment and edit, you need to override the default repository and Maven home directory in pom.xml.
I've found a topic about disabling central repository, and tried out the answers, but Maven still uses the values from settings.xml. This answer in another thread explains why:
settings.xml allows you to override definitions in pom.xml, not the other way round.
So it's seems it is not possible to override the default mechanism from pom.xml, Maven will search for dependencies in repositories configured in settings.xml and will install to Maven home specified in that file.
If the problem is having to write the options every time when running maven, you can use a feature available since version 3.3.1 that allows you to set command line options in your project (or parent project of your module). Read #Brice's answer: https://stackoverflow.com/a/48583079
So with this feature in mind, you can achieve a similar result by setting up a new settings.xml with the <localRepository> pointing to the location you desire and use maven.config in your project to make maven use the new settings.xml, which by the way can be anywhere you want.
I want to use this jar in a Maven project.
https://github.com/downloads/2checkout/2checkout-java/twocheckout-java-latest.jar
I know that the proper way is to add this jar into my local repository but I can't do this into every development machine. Is there any Maven plugin that can download this jar file and add it into my project?
There are only 3 options in a case like this:
Convince the people of the project to put their releases in Maven Central. See Guide to uploading artifacts to the Central Repository for more information on that.
Install a Maven proxy (e.g. Sonatype Nexus, see http://www.sonatype.org/nexus/) and upload the artifact manually. Each developer on the project can point to that proxy and will get the artifact.
Use the maven-install plugin to have each developer install the jar on his own local repository.
You can add below dependency into your pom.xml
<dependency>
<groupId>com.twocheckout</groupId>
<artifactId>twocheckout-java</artifactId>
<version>0.1.0</version>
</dependency>
The usual way to deal with this sort of situation is to use a shared repository such as nexus or artifactory. You configure the nexus repository to serve the locally-uploaded artifacts and you configure your pom to point to your nexus repository as one of the repositories where the artifact may be found. Then, you install the problematic artifact to the nexus repository (rather than every developer's local repository).
If desired, you can also configure your nexus to be a proxy for Maven Central (and all other repositories that you use) and have your pom configured to look only there. This results in a cache of all the artifacts that you use being held locally, which can improve performance and availability for your team (if they are co-located). This can be especially important if you have a dependency on a SNAPSHOT version that is stored in a remote repository.
The best way is to have a proxy repository installed in your corporate LAN and deploy these kind of jars in to "hosted" repositories. Then editing your settings.xml to have this as your repository.
The choice of local proxy varies from using a NAS based shared drive to using repository managers like 'Nexus' or 'Artefactory'.
This way you can easily distribute the artefacts across developers and all other users.
I'm trying to build the project from this site http://www.joptimizer.com/usage.html. I downloaded the sources jar file, unpacked it and ran maven package in the root folder. Maven fails at the last minute saying it couldn't resolve the dependency..
could not find artifact seventytwomiles:architecture-rules:jar:3.0.0-M1 in central repo - repo.maven.apache.org/maven2 ..
I have a feeling I might need to change something in the pom.xml file for this to work, but have no idea what. Googling for this missing dependency lead me no where. In general, how would one know what to do to handle such errors (and also please help with this specific case).
Specifically
According to the Building notes on http://www.joptimizer.com/usage.html:
JOptimizer is build on maven 3.0. Before building it, you must resolve
(in pom.xml) the external dependency on Colt and other dependencies
that aren't in public repositories. Please refer to the "Dependencies"
report for a complete treatment. For ease of use a boundle with
these external libraries is provided (visit "Download"): extract the
boundle in a folder and run the "maven-install.cmd" (translate it in
your own shell language), and you will get the artifacts in your local
repository.
To get the bundle for this, go to http://sourceforge.net/projects/cvxopt/files/, and download the appropriate version of joptimizer-3.X.X-dependencies.zip. Unzip in your own folder, and run mvn install:install-file -DgroupId=seventytwomiles -DartifactId=architecture-rules -Dversion=3.0.0-M1 -Dpackaging=jar -Dfile=architecture-rules-3.0.0-M1.jar -DpomFile=architecture-rules-3.0.0-M1.pom
Generally
Use a tool like http://mavenrepository.com to search for another version of the missing dependency and update your POM with the proper version. If MVNRepository doesn't know about it, you can install the dependency yourself. If you are working with a group of developers, as Eric Jablow mentions, an artifact repository like Nexus or Artifactory is great for sharing non-public dependencies. If it's just you, you can install the artifact in your local repo as described here: How to manually install an artifact in Maven 2?
You should add your own repository manager like Nexus or Artifactory. Then, find out where this dependency is kept; there are repositories other than central. If it's kept on another repository, have your repository mirror that too.
Otherwise, Nexus or Artifactory have commands to enter the dependency manually. Create a local repository called "Third-party" and add it there.
Finally, change your settings.xml file to refer everything to your repository manager.
The most common case for this is when a company refuses to license their products to be held at the central repository. For example, Microsoft won't let its sqljdbc.jar file be distributed through Central. So, you need to add it by hand.
Change the dependency as follows
<dependency>
<groupId>org.architecturerules</groupId>
<artifactId>architecture-rules</artifactId>
<version>3.0.0-rc1</version>
<scope>test</scope>
</dependency>
Add the repository in pom
<repositories>
<repository>
<id>architecturerules.googlecode.com</id>
<url>http://architecturerules.googlecode.com/svn/maven2/</url>
</repository>
</repositories>
I am working in a private network which doesn't have internet proxy.
I can not create a local repository as well which involves the bureaucracy, management won't allow it. I may be a long term fix but not the solution for the question I asked.
I can not keep maven as a build tool as it requires the direct or indirect internet connection.
I HAVE to use ANT for building the project hence using maven in offline mode also not an option for me.
But I still want to use the maven dependency management for collecting all the jars in a one archive smartly.
My plan is to generate a ZIP file containing dependencies resolved using maven. And then we will share this ZIP file to all developers working inside a private network which doesn't have internet connection.
To do so I will get a temporary access to a computer which is having internet connection and from there I will define a dummy POM with all the dependencies required.
Now the question is how do I generate a ZIP file ( not a single jar ) using maven which contains all the dependencies defined in POM.
while what youre suggesting is technically possible, it is (in my opinion) not the best solution to your problem.
your statement that
it requires the direct or indirect internet connection
is not accurate. what maven requires is a maven repository (or a set of them) to fetch stuff from. the best solution to your problem would be to install a local maven repository inside your organization's network. the 2 most popular choices for a loaclly-run maven repository seem to be nexus and artifactory - both offer free open source versions and paid supported pro versions.
once you set up a maven repository inside your organization's network and populate it with the artifacts you require you can simply configure all of your project's pom files to go to those repositories. for example, to configure maven to use your repo instead of maven central, you can do this:
<repositories>
<!-- override central -->
<repository>
<id>central</id>
<url>http://your.repo.location</url>
</repository>
</repositories>
you will need to map a plugin repository in a similar fashion.
its also possible to achieve this by configuring the maven settings.xml file in each user's home directory if you dont want this in the pom files but from my experience its less error-prone this way
use following command to build Maven project offline.
mvn -o package
Refer this and this for more information.
I ended up with a smart hack which lets me do dependency resolution and archiving!
I am creating a dummy maven web project with all the dependencies defined in pom xml.
Now the war packaging mode is used by default for web applications.
I simply install the maven project from internet facing machine.
I get all the dependencies and transitive dependencies in war file's "lib" directory with dependency naming version remaining unchanged !!!!
Copying and adding those files into an ANT project is a trivial task then..!