I am trying to set up CI & CD in a project I am working for my company. I am using Maven as the build tool and Nexus Repository for storing the artifacts. I want to know the actual concept for setting up maven with nexus so that build time is fast and maven doesn't always connect to central maven repository on the internet for downloading the dependencies. I came across the proxy repository concept from nexus documentation and implemented the same. So now, my nexus maven central proxy repository acts as the central repository for maven and the ~/.m2 as its local repository. Is there any way or the right concept to make the nexus central proxy repository as maven's local repository so that my builds will be faster and no storage will be occupied in the host server where maven is running?
No, but you can delete the local repository on the CI server regularly if you get into disk space trouble.
Related
I am setting up a maven repository for my workplace. The main reason for setting up a new private repository is that, our original nexus repo is hidden behind a VPN network. So anyone who wished to pull the dependencies. He needs to have a VPN connection in the working machine. Goal is to make the VPN secured artefacts under the newly created repo to make it available for CI pipeline.
Right now I am doing it as below, listing all the dependencies using following command:
mvn -Dmdep.copyPom=true dependency:copy-dependencies
Which send all the jars under target/dependency.
And then I using the following maven command:
mvn deploy:deploy-file
Which pushes all the dependencies in my newly created nexus, issue is that during setting deploy-file command I am passing a group id which is a generic one. And all the jars are pushed under that groupid. Which is different than my local name namespace/groupid, as different packages are under different group id and pushing all of them under a same group id makes them unusable as it messes with the namespace.
I am new to maven, and the approach I took feels like a hack. Can I use maven native functionality to solve the problem? What would be the standard way to solve the issue?
Nexus has a concept of proxy repositories , for the open source dependencies, you can create proxy repository to mirror the maven central repo and provide in the proxy information is maven settings and run a mvn install, during the resource generation phase of maven the nexus will automatically pull artifacts from the central repo and cache them in your proxy repository for future use.
I want to use this jar in a Maven project.
https://github.com/downloads/2checkout/2checkout-java/twocheckout-java-latest.jar
I know that the proper way is to add this jar into my local repository but I can't do this into every development machine. Is there any Maven plugin that can download this jar file and add it into my project?
There are only 3 options in a case like this:
Convince the people of the project to put their releases in Maven Central. See Guide to uploading artifacts to the Central Repository for more information on that.
Install a Maven proxy (e.g. Sonatype Nexus, see http://www.sonatype.org/nexus/) and upload the artifact manually. Each developer on the project can point to that proxy and will get the artifact.
Use the maven-install plugin to have each developer install the jar on his own local repository.
You can add below dependency into your pom.xml
<dependency>
<groupId>com.twocheckout</groupId>
<artifactId>twocheckout-java</artifactId>
<version>0.1.0</version>
</dependency>
The usual way to deal with this sort of situation is to use a shared repository such as nexus or artifactory. You configure the nexus repository to serve the locally-uploaded artifacts and you configure your pom to point to your nexus repository as one of the repositories where the artifact may be found. Then, you install the problematic artifact to the nexus repository (rather than every developer's local repository).
If desired, you can also configure your nexus to be a proxy for Maven Central (and all other repositories that you use) and have your pom configured to look only there. This results in a cache of all the artifacts that you use being held locally, which can improve performance and availability for your team (if they are co-located). This can be especially important if you have a dependency on a SNAPSHOT version that is stored in a remote repository.
The best way is to have a proxy repository installed in your corporate LAN and deploy these kind of jars in to "hosted" repositories. Then editing your settings.xml to have this as your repository.
The choice of local proxy varies from using a NAS based shared drive to using repository managers like 'Nexus' or 'Artefactory'.
This way you can easily distribute the artefacts across developers and all other users.
Env : Maven 3.3.9
Hi,
I have a maven jar module with version as 1.0-SNAPSHOT. When i do mvn install, the jar installed into local repository (~/.m2/repository) does not have timestamp. I agree that its not a common scenario where you would like to have timestamped jars in your ~/.m2/repository repo. But still couple of questions:
Does mvn install append timestamp automatically? Looking for some
configuration way rather than use ${timestamp} in final name.
Does automatic timestamp mechanism applies to mvn deploy?
Is there a way to tell maven to use ~/.m2/repository
itself for mvn deploy?
Thanks,
Rakesh
The mvn install will only put your project into your local cache which means it makes it available for other projects on your machine.
mvn deploy will deploy the artifacts into your remote repository which is usually a corporate repository manager.
In case of a SNAPSHOT this means this artifact has not been finalized and is under developer. This means you can create several states of the same version like 1.0.0-SNAPSHOT. The time stamp which is created during the deployment to a repository manager is intended to have different artifacts available for development. You can control via -U option if you like to use the most recent version of the SNAPSHOT's.
After you feel ready you make a so called release which will set the version to something like 1.0.0 (without SNAPSHOT) which is deployed as well but into a release repository which is immutable.
Furthermore having timestamps in your local cache $HOME/.m2/repository does not make really sense, cause you can control when you install an artifact there and no one else and you should prevent using finalName change cause this is only intended for your target folder and not for your local cache.
And finally using the local cache for mvn deploy does not make sense, cause what is the idea behind that? Best is to start using a repository manager like Nexus, Artifactory or Archiva in particular if your are working in a corporate environment.
In addition to #khmarbaise's answer see the following references:
Maven / Introduction to Repositories:
There are strictly only two types of repositories: local and remote. The local repository refers to a copy on your own installation that is a cache of the remote downloads, and also contains the temporary build artifacts that you have not yet released.
Remote repositories refer to any other type of repository, [...]
Maven: The Complete Reference, 15.2.8. Repositories:
Repositories are remote collections of projects from which Maven uses to populate the local repository of the build system.
Repository - SNAPSHOT Handling reads:
This documentation was targetted at Maven 2.0 alpha 1. It is here only for historical reference and to be updated and integrated into the Maven documentation.
But I didn't find any latest documentation where this has been integrated. (#khmarbaise?)
Timestamped files are not created on install in the local repository for reasons of disk space preservation. However, when a SNAPSHOT is resolved and downloaded, it is saved with its timestamp version number (eg: 0.15-20050401.150432-2).
Understanding Maven Version Numbers
Maven Dependency Resolution - A Repository Perspective
Long story short:
The same snapshot version can be deployed to a remote repository from different hosts, so they have to be distinguished there somehow. And they are distinguished by timestamps (and a build number).
There will be an artifact with a timestamp (and build number) in the local repository only if Maven resolved and, hence, downloaded it from remote.
So:
No.
Yes.
Would break Maven's repositories handling of local vs. remote.
One of the most time consuming task Jenkins makes during every build is to download the artifacts into his local repository which it deletes.
While deleting my artifacts is fine. I don't understand the necessity in deleting 3rd party artifacts which were previously downloaded into it's local maven repository(.m2).
Is there any way to prevent Jenkins from deleting the local repository before build.
Thanks
You should install a Maven repository manager (MRM) like Sonatype Nexus, JFrog Artifactory or Apache Archiva and the downloads will be local to your network and very fast. Using a MRM is pretty much considered a necessity for any useful usage of Maven or any build tool with declarative dependency management since it allows you to cache artifacts as well as upload your own libraries and share them acros you developers as well as you CI builds.
If that is still not enough you can disable the private repository deletion per build or even use one shared repository per build, but that reduces the stability of the build since you are now mixing between builds and therefore introducing interdependencies.
While I agree with Manfred's recommendation to use a Maven repository manage I'd also recommend looking at how you manage the Maven local repository:
Prevent Jenkins from Installing Artifact to Local Maven Repository
When is it safe to delete the local Maven repository?
Ivy, Ant, Jenkins - Is it good idea to to a <ivy:cleancache> on Jenkins builds?
Maven does not normally purge the local repository, I'm guessing you have a periodic task that does this.
What I want to do is to force Maven to download the 'maven-metadata.xml' for each artifact that I have in my local repository.
The default Maven behaviour is to download only metadata from remote repositories (see this question).
Why I want to do that:
Currently I have a remote repository running in a build machine. By remote repository I mean a directory located in the build machine that contains all dependencies that I need to build my Maven projects. Note that I'm not using a repository manager like Nexus, the repository is just a copy of a local repository that I have uploaded to my build machine.
However, since my local repository did not contain the 'maven-metadata.xml' files, these metadata files are also missing in the build machine repository. If I could retrieve the metadata files from the central repository, then it would be possible to upload a working remote repository to my build machine.
You don't want to get the metadata from the public repositories, it will contain all the versions available of a given artifact, whereas your repository will have some subset of the releases.
It's worth pointing out that you really would be better off with a repository manager. The following steps allow you to generate your artifact metadata once. But if your needs change, you'll have to repeat the process or update the files manually, so why not just use a manager? Nexus can run standalone and has a very small footprint.
Even if you're not planning on using Nexus for a repository manager, you can still use it to generate your metadata.
First install Nexus.
Locate the nexus work directory (by default ${user.home}/sonatype-work.
Copy your local repository contents to the nexus-work/releases sub-directory.
Start Nexus and connect to the Nexus home page in the browser (by default http://localhost:8081/nexus)
Log in using the admin account (password admin123)
Click on the repositories link on the left hand side.
Select the Releases repository, right-click and click Rebuild Metadata
In a few moments you should have the metadata generated for all the artifacts. You can then copy them to wherever you need them to be and uninstall Nexus.
The default repositories are defined in the super pom.xml that all poms inherit from by default.
If by local you mean you want to only use ~/.m2/repos/* then work in offline mode. Add <offline>true</offline> to your settings.xml
If by local you mean your local server, you could install a repository manager like Nexus, modify your settings file to use nexus under "mirrors" like this:
<mirror>
<id>central-proxy</id>
<mirrorOf>central</mirrorOf>
<url>my/local/nexus/server</url>
</mirror>
And disable remote repositories you don't want in Nexus.
One thing I found is someone was doing an initial search of jarvana and had placed this within the pom, causing a metadata error message to occur. This ...
<!-- <repository>
<id>jarvana</id>
<url>http://www.jarvana.com/jarvana/browse/</url>
</repository> -->
..fixed it and the error went away.
Repository manager is great but I don't want install and run it on my laptop. I just want to save resource.
So, my solution is below:
install repository as artifactory (https://www.jfrog.com/artifactory)
run materialize thru this server (localhost:8081)
run materialize thru remote server (center.maven.org)
download cache content of artifactory
merger to folder: folder download from step 4 and folder at ~/.m2/repository.
Now I have a repository offline with full content (binary, source, xml, md5, sha1)