I recently set up a password protected Artifactory server. A Jenkins server is deploying my artifacts to a maven repository on that Artifactory server.
Here is the problem:
The new builds are nor available at runtime (in IntelliJ) neither in Maven (when building). To get maven to update, I've to delete the artifacts from my local repo.
How can I tell Maven/IntelliJ to always check for the newest version?
Even that wasn't doing anything:
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
There are the cases to deal with:
Releases. Not Maven nor Artifactory will never check for a newer binary if a release binary of the same version exists in the local cache (~/.m2/repository cache for Maven or remote repository cache for Artifactory). The only way to force Artifactory and Maven to re-fetch a release artifact is to delete it from the cache. Do not use the same version for different blob, ever.
Snapshots. Although there is a difference between unique (file names end with timestamps) and non-unique (file names end with -SNAPSHOT and thus they violate the rule I just wrote, so don't use them), their retrieval policy is the same: Maven consider them expirable resources, it will occasionally check for a new blob, posted under the same version. Artifactory does the same: occasionally the expirable resource is declared "expired", so next time Maven asks for this resource, Artifactory will fetch to the remote repository to check for a new blob, posted under the same version. In both tools you can control the cache period and "zap the cache" manually.
In Maven <update-policy/> controls how often Maven checks for new snpashots, and the --update-snapshots flag forces Maven to check immediately.
In Artifactory the Retrieval Cache Period parameter in remote repository configuration controls how often Artifactory checks for new snapshots, and the Zap Caches action on the remote repository in the artifact browser screen makes sure that Artifactory will check for new snapshots on the next request.
Related
I'm trying to use jackson at a json project. And Maven is ignoring my local repository (where I have the library already set, from a previous project) and downloading from codehaus repo.
The catch is that my workplace has blocked access to external repos, so I must use local repository for this. How do I force Maven to look first at local and use the library there?
Thanks in advance
Please set the Local Repository Path in eclipse that maps to your local repo path..
Go to Windows->Preferences->Maven->User Settings and change the settings.xml and your local repository path..
If these both are correctly located then it should check to the local repo first
Having to move my development environment offsite in response to COVID-19, my environment had not been established to directly connect to various public repositories since we were mirroring those repos in our server at the office. However, my Maven was still contacting public repositories for our in-house releases. This was odd, because I already had our in-house releases in my local repo.
The solution was to update the Maven configuration, in my settings.xml. Define profiles, a profile for the office and a profile for offsite. In the offsite profile, in addition to the public repos we were already mirroring at the office, I also defined a repository that matched the name, id, and URL of our server at the office where we stored our in-house releases - identically. This allowed Maven to see that the library it was looking for did in-fact come from that repository and didn't need to go looking for it because it was already in the local repo. It also didn't matter that Maven wasn't going to be able to connect to the office server, it just needed to be defined.
Before:
Downloading from central: https://repo.maven.apache.org/maven2/com/mycompany/test-core/1.1.0/test-core-1.1.0.pom
Downloading from jboss-releases: https://repository.jboss.org/nexus/content/repositories/releases/com/mycompany/test-core/1.1.0/test-core-1.1.0.jar
Downloading from repository-apache-org: https://repository.apache.org/content/groups/public/com/mycompany/test-core/1.1.0/test-core-1.1.0.pom
[WARNING] The POM for com.mycompany:test-core:jar:1.1.0 is missing, no dependency information available
This was a library we had produced and released ourselves, definitely not at Maven Central. Maven was just hunting around for it - lost with no direction.
I defined a new repo in the offsite profile, a repo that my environment definitely would not be able to connect to due to existing company firewall rules:
<repository>
<id>leroy-releases</id>
<name>Development Release Repository</name>
<url>http://leroyhost:8080/repository/maven-releases/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>interval:15</updatePolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
Please note, the profile section includes its own repositories section in which the repository section above must be defined in. You also need to activate the offsite profile:
<activeProfiles>
<!--activeProfile>main-developer-profile</activeProfile-->
<activeProfile>offsite-snapshots</activeProfile>
</activeProfiles>
I named the profile snapshots because without access to our office repository, no one is doing any releases out "in the field".
After:
Maven no longer searched the public repos for our in-house release artifacts. It consulted my local repo and found all of our releases there.
When I run an integration test for my project, it is trying to search in a repo that is being pulled transitively and I need to wait for time out. I blacklisted that repo like this.
<repository>
<id>seacrh-snapshots</id>
<name>Exodus Snapshot Repository</name>
<url>http://repo URL/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
but it is still pulling this repo when I run the IT test:
Downloading: URL repo/maven-metadata.xml
Is there anything else that need to be changed?
I strongly recommend to use a Maven Repository Manager such as Nexus. It will help you to get stable and reproducible builds with Maven. You can disable Repositories there (or Nexus can handle this for you as well).
http://www.sonatype.com/books/nexus-book/reference/maven-sect-single-group.html
http://maven.apache.org/repository-management.html
Normally, when maven runs it goes to fetch artifacts in the following order
it check your local .m2/repositories folder
if it can't find the artifact then it reads your pom / parent pom/ super Pom / user level setting.xml/ global setting.xml - in that order to find external repositories to download the artifacts from, usually these repo are either directly from the internet (like the pre-configured default maven repositories) or local private repo managers(like nexus, Artifactory).
Maven will execute the repo as in the order in which they are declared.
First of all, check your project effective-pom that nothing else is overriding your configuration. either in the terminal by typing mvn help:effective-pom on the project root dir or easily in eclipse "effective pom" view of your pom.
Otherwise, it's a good use-case for you to consider installing local repository manager as you would not always need to fetch externally your third party dependencies everytime you run maven.
i would recommend Artifactory much better than the nexus.
Does maven require a connection to the internet at some point to be able to use it? Meaning specifically getting the internal maven plugins for compiling, cleaning, packaging, etc?
You can run Maven in "offline" mode using the -o or -offline option (e.g. mvn -o install). Of course any artifacts not available in your local repository will fail. Maven is not predicated on distributed repositories, but they certainly make things more seamless. It's for this reason that many shops use internal mirrors that are incrementally synced with the central repos.
In addition, the mvn dependency:go-offline can be used to ensure you have all of your dependencies installed locally before you begin to work offline.
If you have a PC with internet access in your LAN, you should install a local Maven repository.
I recommend Artifactory Open Source. This is what we use in our organization, it is really easy to setup.
Artifactory acts as a proxy between your build tool (Maven, Ant, Ivy, Gradle etc.) and the outside world.
It caches remote artifacts so that you don’t have to download them over and over again.
It blocks unwanted (and sometimes security-sensitive) external requests for internal artifacts and controls how and where artifacts are deployed, and by whom.
After setting up Artifactory you just need to change Maven's settings.xml in the development machines:
<?xml version="1.0" encoding="UTF-8"?>
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<mirrors>
<mirror>
<mirrorOf>*</mirrorOf>
<name>repo</name>
<url>http://maven.yourorganization.com:8081/artifactory/repo</url>
<id>repo</id>
</mirror>
</mirrors>
<profiles>
<profile>
<repositories>
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>libs-release</name>
<url>http://maven.yourorganization.com:8081/artifactory/libs-release</url>
</repository>
<repository>
<snapshots />
<id>snapshots</id>
<name>libs-snapshot</name>
<url>http://maven.yourorganization.com:8081/artifactory/libs-snapshot</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>plugins-release</name>
<url>http://maven.yourorganization.com:8081/artifactory/plugins-release</url>
</pluginRepository>
<pluginRepository>
<snapshots />
<id>snapshots</id>
<name>plugins-snapshot</name>
<url>http://maven.yourorganization.com:8081/artifactory/plugins-snapshot</url>
</pluginRepository>
</pluginRepositories>
<id>artifactory</id>
</profile>
</profiles>
<activeProfiles>
<activeProfile>artifactory</activeProfile>
</activeProfiles>
</settings>
We used this solution because we had problems with internet access in our development machines and some artifacts downloaded corrupted files or didn't download at all. We haven't had problems since.
You have two options for this:
1.) make changes in the settings.xml add this in first tag
<localRepository>C:/Users/admin/.m2/repository</localRepository>
2.) use the -o tag for offline command.
mvn -o clean install -DskipTests=true
mvn -o jetty:run
Maven needs the dependencies in your local repository. The easiest way to get them is with internet access (or harder using other solutions provided here).
So assumed that you can get temporarily internet access you can prepare to go offline using the maven-dependency-plugin with its dependency:go-offline goal. This will download all your project dependencies to your local repository (of course changes in the dependencies / plugins will require new internet / central repository access).
Sadly dependency:go-offline hasn't worked for me as it didn't cached
everything, ie. POMs files and other implicitly mention dependencies.
The workaround has been to specify a local repository location, either within settings.xml file with <localRepository>...</localRepository> or by running mvn with -Dmaven.repo.local=... parameter.
After initial project build, all necessary artifacts should be cached, and then you can reference repository location the same ways, while running Maven build in offline mode (mvn -o ...).
Before going offline you have to make sure that everything is in your local repo, which is required while working offline. Running "mvn dependency:go-offline" for the project(s)/pom(s), you intend to work on, will reduce the efforts to achieve this.
But it´s usually not the whole story, because dependency:go-offline will only download the "bare build" plugins (go-offline / resolve-plugins does not resolve all plugin dependencies). So you have to find a way to download deploy / test / site plugins (and maybe others) and their dependencies into your repo.
Furthermore dependency:go-offline does not download the pom´s artifact itself, so you have to dependency:copy it if required.
Sometimes - as MaDa wrote - you do not know, what you will need, while being offline, which makes it pretty impossible to have a "sufficient" repo.
Anyway having a properly filled repo you only have to add "<offline>true</offline>" to Maven´s settings.xml to go offline.
Do not change the Maven profile (id) you used to fill your repo, while being offline. Maven recognizes the downloaded artifacts in its metadata with an "identity", which is bound to the profile id.
If you're using IntelliJ, you can simply go to Preferences -> Build, Execution, Deployment -> Build Tools -> Maven and check/uncheck Work offline.
Does this work for you?
http://jojovedder.blogspot.com/2009/04/running-maven-offline-using-local.html
Don't forget to add it to your plugin repository and point the url to wherever your repository is.
<repositories>
<repository>
<id>local</id>
<url>file://D:\mavenrepo</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>local</id>
<url>file://D:\mavenrepo</url>
</pluginRepository>
</pluginRepositories>
If not, you may need to run a local server, e.g. apache, on your machines.
(source: jfrog.com)
or
Just use Maven repository servers like Sonatype Nexus http://www.sonatype.org/nexus/ or JFrog Artifactory https://www.jfrog.com/artifactory/.
After one developer builds a project, build by next developers or Jenkins CI will not require Internet access.
Maven repository server also can have proxies configured to access Maven Central (or more needed public repositories), and they can have cynch'ed list of artifacts in remote repositories.
My experience shows that the -o option doesn't work properly and that the go-offline goal is far from sufficient to allow a full offline build:
The solution I could validate includes the use of the --legacy-local-repository maven option rather than the -o (offline) one and
the use of the local repository in place of the distribution repository
In addition, I had to copy every maven-metadata-maven2_central.xml files of the local-repo into the maven-metadata.xml form expected by maven.
See the solution I found here.
A new plugin has appeared to fix shortcomings of mvn dependency:go-offline:
https://github.com/qaware/go-offline-maven-plugin
Add it in your pom, then run mvn -T1C de.qaware.maven:go-offline-maven-plugin:resolve-dependencies. Once you've setup all dynamic dependencies, maven won't try to download anything again (until you update versions).
Answering your question directly: it does not require an internet connection, but access to a repository, on LAN or local disk (use hints from other people who posted here).
If your project is not in a mature phase, that means when POMs are changed quite often, offline mode will be very impractical, as you'll have to update your repository quite often, too. Unless you can get a copy of a repository that has everything you need, but how would you know? Usually you start a repository from scratch and it gets cloned gradually during development (on a computer connected to another repository). A copy of the repo1.maven.org public repository weighs hundreds of gigabytes, so I wouldn't recommend brute force, either.
Here's a clear, straightforward way to cache Maven dependencies for offline development (based on #luka5z and others' comments):
While you have internet access, cache dependencies locally:
mvn -Dmaven.repo.local=dependencies install
Disconnect from the internet, verify that offline mode compilation succeeds:
mvn clean
mvn -o -Dmaven.repo.local=dependencies package
Continue developing offline as long as needed.
In preparation before working offline just run
mvn dependency:go-offline
<offline> false </offline>
<localRepository>${user.home}/.m2/repository</localRepository>
to
<offline> true <offline>
<localRepository>${user.home}/.m2/repository</localRepository>
Change the offline tag from false to true .
will download from repo online
I believe there's a maven command you can execute to download and install a plugin in the local repository. Something like:
mvn plugin:download -DartifactId=maven-war-plugin
-DgroupId=org.apache.maven.plugins -Dversion=2.1.1
-Dmaven.repo.remote=http://www.ibiblio.org/maven,http://maven-plugins.sourceforge.net/repository
I know that this should normally happen when you build a project whose pom.xml references this plugin, but the security policy where I work is abnormal, so I need to manually install plugins.
The syntax above doesn't seem to work, does anyone know how to do this under Maven 3.0.2?
The above syntax is for Maven 1.0, which operated very differently from Maven 2.0 and Maven 3.0. You should continue to reference the plugin as you would - via the POM in this case or via the command-line directly for some types of commands. To strictly control what artifacts get downloaded, you can use a repository manager (such as Apache Archiva, Artifactory or Nexus) to intervene (as well as add several interesting features).
Here is how to force Maven to use the managed repository instead: http://archiva.apache.org/docs/1.3.4/userguide/using-repository.html
You can either add only the artifacts you want to use to that, or configure rules about what can be retrieved externally (e.g. http://archiva.apache.org/docs/1.3.4/adminguide/proxy-connectors.html)
The optimal way to deal with plugin and other artifact installation is to implement an instance of a Maven repository on your LAN. Products such as Nexus are open-source and easy to set up. Once up and running, upload your Plugin manually to Nexus. Then add your own local Nexus instance as a plugin repository inside of the settings.xml file for the developers who need it:
<pluginRepositories>
<pluginRepository>
<id>mycorp-plugin-release</id>
<name>My Companys Nexus repository for plugin artifact releases</name>
<url>https://intranet.mycorp.com/nexus/content/repositories/releases</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
Seriously...it'll take a day to set up initially, but will make your life easier and shouldn't violate ANY security policy. It'll also help to ensure that all developers are using the exact same version of the plugin.
I'm working on a fairly big Maven project. We have probably around 70 or so individual artifacts, which are roughly split into two libraries of shared code and maybe ten applications which use them. All of these items live in the namespace com.mycompany.*.
Most of the time we're running against snapshot builds. So to do a full build of an application, I might first build the library projects so that they are installed to my local repository (as, say, mycompany-libname-2.4-SNAPSHOT.jar).
The problem is that when I then go build the applications. For some reason, Maven wants to check the main two public repositories (maven-net-repo and java-net-repo) for updates for all of the mycompany-*-SNAPSHOT.jar artifacts. Of course, they aren't found there, and everything eventually resolves back to the versions I just built to my local repository, but I'd like Maven to stop doing this because (a) it makes me feel like a bad net.citizen for constantly checking these repositories for things that will never be there, and (b) it adds some unnecessary and annoying network latency into my build process.
I've taken to running maven in offline mode most of the time to work around this, but that's not ideal since occasionally a dependency on a public library will be updated. So what I'm looking for is a solution which will cause Maven not to check for updates from given repositories for artifacts which meet certain criteria - in this case, I'd be happy if Maven would ignore either SNAPSHOT versions or artifacts which were in the com.mycompany namespace.
Also, you can use -o or --offline in the mvn command line which will put maven in "offline mode" so it won't check for updates. You'll get some warning about not being able to get dependencies not already in your local repo, but no big deal.
Something that is now available in maven as well is
mvn goal --no-snapshot-updates
or in short
mvn goal -nsu
The updatePolicy tag didn't work for me. However Rich Seller mentioned that snapshots should be disabled anyways so I looked further and noticed that the extra repository that I added to my settings.xml was causing the problem actually. Adding the snapshots section to this repository in my settings.xml did the trick!
<repository>
<id>jboss</id>
<name>JBoss Repository</name>
<url>http://repository.jboss.com/maven2</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
Update: I should have probably started with this as your projects are SNAPSHOTs. It is part of the SNAPSHOT semantics that Maven will check for updates on each build. Being a SNAPSHOT means that it is volatile and subject to change so updates should be checked for. However it's worth pointing out that the Maven super POM configures central to have snapshots disabled, so Maven shouldn't ever check for updates for SNAPSHOTs on central unless you've overridden that in your own pom/settings.
You can configure Maven to use a mirror for the central repository, this will redirect all requests that would normally go to central to your internal repository.
In your settings.xml you would add something like this to set your internal repository as as mirror for central:
<mirrors>
<mirror>
<id>ibiblio.org</id>
<name>ibiblio Mirror of http://repo1.maven.org/maven2/</name>
<url>http://path/to/my/repository</url>
<mirrorOf>central</mirrorOf>
</mirror>
</mirrors>
If you are using a repository manager like Nexus for your internal repository. You can set up a proxy repository for proxy central, so any requests that would normally go to Central are instead sent to your proxy repository (or a repository group containing the proxy), and subsequent requests are cached in the internal repository manager. You can even set the proxy cache timeout to -1, so it will never request for contents from central that are already on the proxy repository.
A more basic solution if you are only working with local repositories is to set the updatePolicy for the central repository to "never", this means Maven will only ever check for artifacts that aren't already in the local repository. This can then be overridden at the command line when needed by using the -U switch to force Maven to check for updates.
You would configure the repository (in your pom or a profile in the settings.xml) as follows:
<repository>
<id>central</id>
<url>http://repo1.maven.org/maven2</url>
<updatePolicy>never</updatePolicy>
</repository>
Very simple :
In your Super POM parent or setting.xml, use
<repository>
<id>central</id>
<releases>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<updatePolicy>never</updatePolicy>
</snapshots>
<url>http://repo1.maven.org/maven2</url>
<layout>legacy</layout>
</repository>
It's my tips
I had some trouble similar to this,
<repository>
<id>java.net</id>
<url>https://maven-repository.dev.java.net/nonav/repository</url>
<layout>legacy</layout>
</repository>
<repository>
<id>java.net2</id>
<url>https://maven2-repository.dev.java.net/nonav/repository</url>
</repository>
Setting the updatePolicy to "never" did not work. Removing these repo was the way I solved it.
ps: I was following this tutorial about web services (btw, probably the best tutorial for ws for java)