We Keep Coding

Use Gradle to upload archives to GitLab Maven repository

I use Gradle and Gradle Maven plugin to upload archives to Maven Central. This works fine. I want to upload the same archives also to the GitLab Maven repository. Upload to this repo requires authentication with special HTTP header values. But I don't know how to do that with the Gradle Maven plugin. The GitLab documentation describes a Maven based build process using pom.xml and settings.xml. What I currently have is this:
Files created with Gradle:
build/libs
├── wms-1.2.0.jar
├── wms-1.2.0.jar.asc
├── wms-1.2.0-javadoc.jar
├── wms-1.2.0-javadoc.jar.asc
├── wms-1.2.0-sources.jar
└── wms-1.2.0-sources.jar.asc
build/poms
└── pom-default.xml
Files according to the GitLab documentation:
pom.xml
settings.xml
When I execute
mvn deploy -s settings.xml
then pom-default.xml must be "mixed into" pom.xml and build/libs/* must be used as archive location. How can I do this?

Please note that the maven-publish plugin “is now the preferred option for publishing [Maven] artifacts” with Gradle (see also the note at the top of the page you linked). Doing what you need with the maven plugin would be more cumbersome as it’s both more intricate and not as well documented, IMHO. Hence, I hope you won’t mind when I answer your question with the maven-publish plugin in mind.
As for the authentication with special HTTP header values, you should be able to solve this as follows (via):
publishing {
repositories {
maven {
url "http://repo.mycompany.com/maven2"
credentials(HttpHeaderCredentials) {
name = "Private-Token"
value = "REPLACE_WITH_YOUR_PERSONAL_ACCESS_TOKEN"
}
authentication {
header(HttpHeaderAuthentication)
}
}
}
}
I you really want to use the old maven plugin then I’d suggest to start researching from this section of the documentation on how to solve the authentication issue.
BTW: I have mainly answered your question on how to authenticate with GitLab here. If you should have trouble with configuring the publication, then I’d recommend to post this as a separate question with more details on your setup and on maybe what you’ve tried already.
Edited to add: if you need to use different header credentials depending on certain command line options, you could change the credentials configuration above to something like the following:
credentials(HttpHeaderCredentials) {
if (project.hasProperty('jobToken')) {
name = "Job-Token"
value = project.property('jobToken')
} else {
name = "Private-Token"
value = "REPLACE_WITH_YOUR_PERSONAL_ACCESS_TOKEN"
}
}
In this example, you could run ./gradlew -PjobToken=REPLACE_WITH_YOUR_JOB_TOKEN … in GitLab CI jobs while leaving out the -PjobToken=… part when working locally.

gradle: downloading specific file from ivy repository

We use an internal ivy repository and are in the process from moving away from ant / ivy and tasking everything in gradle. I have my ivy repository set up in gradle as so:
repositories {
ivy {
url "${ivy_repository_url}"
layout "pattern", {
ivy "repository/[organisation]/[module]/[revision]/[artifact].[ext]"
artifact "repository/[organisation]/[module]/[revision]/[artifact].[ext]"
m2compatible = true
}
credentials {
username "${ivy_repository_username}"
password "${ivy_repository_password}"
}
}
}
and upon executing a task it resolves as expected, however, in some instances it pulls down everything in the repository if the naming convention doesn't match so I end up with a lot of extra stuff, javadocs.zip, sources.zip and everything else.
To get around this I want to download the specific jar files to a temp folder first and then compile them from local but I have no clue how to tell gradle to download a file that is named differently from the module name.
Example:
ivyFiles 'net.sourceforge.jtidy:jtidy:r938#jar'
downloads just jtidy-r938.jar from the net.sourceforge.jtidy repository but something like
ivyFiles 'com.gargoylesoftware:htmlunit:2.7#jar'
would pull htmlunit-2.7.jar but not the file htmlunit-core-js-2.7.jar.
If I omit the #jar it reverts to calling the ivy.xml file and I am left with all the junk + dependencies which I am trying to avoid. I have tried the following with no success
ivyFiles ('com.gargoylesoftware:htmlunit:2.7'){
artifact{
name = 'htmlunit-core-js'
type = 'jar'
}
}
There must be a way to do this.
Thank you

I figured it out. This is what I ended up doing
ivyFiles ('com.gargoylesoftware:htmlunit:2.7'){
transitive = false
artifact {
name = 'htmlunit-core-js'
extension = 'jar'
type = 'jar'
}
}
Now only the htmlunit-core-js-2.7.jar file was downloaded

How to get the next build number in Gradle

Is there any way to get the next version when publishing to a repository in gradle?
For e.g. if I have the version 3.0.1 in my repository I want the published version to be 3.0.2.
ivy has a task for ant named buildnumber which does exactly that:
<project xmlns:ivy="antlib:org.apache.ivy.ant"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<target name="ivyBuildNumber" description="Use ivy get the next build number">
<ivy:buildnumber
resolver="url-chain"
organisation="${ivy.organisation}"
module="${ivy.module}"
revision="${version.base}"/>
<echoproperties prefix="ivy.new."/>
</target>
Is there a way to do so in gradle? if not how can I access ivy tasks from gradle's ant?
In my build.gradle I calling to the ant
ant.importBuild 'build.xml'

I don't think there is support in Gradle, but you can try to use the Ant task.
https://docs.gradle.org/current/userguide/ant.html#sec:import_ant_build
Another way to do this is to use some sort of plugin, or customized task for managing the version.
Plugin: https://github.com/researchgate/gradle-release
Custom task: https://www.tikalk.com/devops/increment-version-numbers-in-gradle/

Yes, you can access ivy tasks from the ant script by importing ant's build.xml file to gradle's build.gradle file. Following is the syntax to do so.
ant.importBuild 'build.xml'
Please refer : https://docs.gradle.org/current/userguide/ant.html#sec:import_ant_build

I recommend you to use ResearchGate release plugin
https://github.com/researchgate/gradle-release
It has a pretty documentation. Easy to read.
Also, check out how I used it in my personal project.
https://github.com/vatolinrp/bitcoin-esb/blob/master/build.gradle
It would be a nice example for you.

After a long work, I managed to do that.
In my build.gradle I added this following code
ant.importBuild 'build.xml'
task getNextBuild(dependsOn : ivyBuildNumber) {
doLast{
def nextVersion = ant.properties['ivy.new.revision']
println nextVersion
}
}
I imported my ant build file, and created a task that calls the ivy buildnumber task.
There is my build.xml
<project xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="ivyBuildNumber">
<path id="ivy.classpath" path="lib/ivy.jar" />
<typedef resource="org/apache/ivy/ant/antlib.xml" uri="antlib:org.apache.ivy.ant" classpathref="ivy.classpath" />
<ivy:buildnumber
organisation="daniel"
module="hello"/>
<echoproperties prefix="ivy.new."/>
</target>
</project>
Because my IDE (Intellij), didn't have ivy.jar in the content,
I imported the ivy.jar from my root dir (lib/ivy.jar)

For this exact behavior, Ivy buildnumber task can be invoked using pure Gradle without importing the Ant build:
configurations {
antTasks // define a new configuration
}
repositories {
mavenCentral()
}
dependencies {
antTasks("org.apache.ivy:ivy:2.4.0") // add Ivy library to it
}
ext {
// define the Ivy task, using the extra configuration as classpath extension
ant.taskdef(name: "ivyBuildNumber",
classname: "org.apache.ivy.ant.IvyBuildNumber",
classpath: configurations.antTasks.asPath)
ant.ivyBuildNumber(organisation: "daniel", module: "hello")
nextVersion = ant.properties["ivy.new.revision"]
}
task demo {
doLast {
println nextVersion
}
}
In general, Gradle doesn't have any bundled equivalent to Maven Release Plugin, so one has to rely on plugins. One solid plugin is gradle-release by ResearchGate, the other is axion by Allegro Tech. The former is classic Maven-style versioning, the latter takes SCM itself as the only source of truth, eliminating the versioning in the build files. But neither of these plugins does provide the exact requested behavior.
My personal take on the versioning problem was initially to use some plugins. Since I use Bamboo as CI server at work, literally everything I did with release plugins using Gradle crashed on CI server sooner or later. It might have worked for some weeks, but every server update brought some problems. I ended up using SCM-less approach with a simple convention: use branch name as base version, concatenate it with build number (both values are provided by the CI server):
ext {
branch = System.getProperty("branch", "develop")
buildNumber = System.getProperty("buildNumber", "latest")
isRelease = System.getProperty("isRelease", "false").toBoolean()
artifactVersion = "${branch}${(isRelease ? ".$buildNumber" : "-SNAPSHOT")}"
}
CI server then can be set up for executing the following command
./gradlew -DisRelease=true -Dbranch=${git.branch} -DbuildNumber=${build.number} mavenPublish
when 'Release' button is pushed. For example, build 12 of the 3.0 branch will produce version 3.0.12 in the binary repository.
The advantages are:
+ the version comes for free, assuming the branches are named accordingly
+ the auto-incremented build number also comes for free
+ one can easily publish custom revisions
+ no plugins means no problems with Gradle version updates
+ this approach is dead simple and always works
The downsides are:
- additional script tasks are required for tags
- some build numbers will be skipped, obviously (e.g. next version after 3.5.76 can be 3.5.84)

How to feed Antlr goal from local maven repository?

I am trying to build my project on an offline machine (this is a requirement). I have created local maven repository (it's just a folder with appropriate structure) and successfully build all other things.
I do the following way:
1) Run gradle installl (this maven plugin's goal)
then checking errors by hand. If I see some library not found, then
2) I take it's maven coordinates and copy in to this machine by hand from my machine.
It works except of Antlr. I getting the following message:
>gradle install
:generateGrammarSource
FAILURE: Build failed with an exception.
* What went wrong:
Could not resolve all dependencies for configuration ':antlr'.
> Could not download antlr4.jar (org.antlr:antlr4:4.5)
> Could not get resource 'http://repo.maven.apache.org/maven2/org/antlr/antlr4/4.5/antlr4-4.5.jar'
> Could not HEAD 'http://repo.maven.apache.org/maven2/org/antlr/antlr4/4.5/antlr4-4.5.jar'.
> Connection to http://repo.maven.apache.org refused
error message is the same as always, but this time putting jar into local maven repository does not help.
How to overcome? How to configure Antlr to eat from local maven repo?
UPDATE
File is present in
MYHOME\.gradle\caches\modules-2\files-2.1\org.antlr\antlr4\4.5\af4a530e3cd7fa03636645d8077145eefac12907\antlr4-4.5.jar
and in
MYHOME\.m2\repository\org\antlr\antlr4\4.5\antlr4-4.5.jar
In maven case accompaning files are also present.
UPDATE 2
Note that it says
Could not resolve all dependencies for configuration ':antlr'.
and Antlr dependency is added by
antlr "org.antlr:antlr4:4.5" // use ANTLR version 4
i.e. not compile and not testCompile. May be this is the clue? May be it is a way to configure repositores specifically to antlr configuration?
UPDATE 3
I those cases I was resolving successfully it was writing:
Could not resolve all dependencies for configuration ':compile'.
> Could not resolve net.coobird:thumbnailator:0.4.8.
Required by:
com.cireca.overlaywidget:OverlayWidget:1.0-SNAPSHOT
> Could not resolve net.coobird:thumbnailator:0.4.8.
> Could not get resource 'https://repo1.maven.org/maven2/net/coobird/thumbnailator/0.4.8/thumbnailator-0.4.8.pom'.
> Could not GET 'https://repo1.maven.org/maven2/net/coobird/thumbnailator/0.4.8/thumbnailator-0.4.8.pom'.
> Connection to https://repo1.maven.org refused
UPDATE 4
Strange thing. I noticed that my config looks redundant:
repositories {
maven { url "http://repo.maven.apache.org/maven2" }
maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
maven { url "http://maven-eclipse.github.io/maven" }
mavenLocal()
flatDir {
dirs 'lib'
}
maven { url "http://repo.maven.apache.org/maven2" }
maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
maven { url "http://maven-eclipse.github.io/maven" }
}
I changed this to
repositories {
mavenLocal()
flatDir {
dirs 'lib'
}
maven { url "http://repo.maven.apache.org/maven2" }
maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
maven { url "http://maven-eclipse.github.io/maven" }
}
And after that it started to claim different liraries. First it claimed
org.antlr:antlr4-runtime:4.5
and I fed it successfully, but then it claimed
org.antlr:antlr-runtime:3.5.2
and I can't feed it (same situation).

Maven contacts the repositories as specified in your settings.xml. If you want to avoid external repositories, you need to mirror everything.
Actually, the best approach to work in environment not connected to the internet is to set up your own Nexus/Artifactory. And the easiest way to fill up your repository for offline use to connect it to the internet once, then build everything and disconnect. Then you have a local copy of everything that is relevant for you.

I have removed remote repositories references completely from that machine (before this the remote repositories just had lower priority).
After that the Gradle started to report what it miss in the form of explicit local paths and I found, that it had lack of some parent (?) artifacts.
For example, for Guava, it was wishing com.google.guava:guava-parent and for antlr it was org.antlr:antlr-master. These names were not reported when remote repositories were present in the config.
If anybody could explain what happened in more detailed way, I will accept his/her answer.

Combining/Resolving Two Repository Types in Gradle

In my Gradle project, I need to define two types of repositories: a flat directory and a Maven repository (a local Nexus server).
I can get both working separately, but can't get them to play nicely together. Ideally, I would have Gradle look at the local directory first, then at the Maven repository.
Current Setup
I have defined the repositories in my build.gradle like this:
repositories {
flatDir dirs: "${rootProject.projectDir}/libs"
flatDir dirs: "${rootProject.projectDir}/test.libs"
maven {
credentials {
username nexus_username
password nexus_password
}
url "http://nexus-server/nexus/content/groups/public"
}
}
In the libs (and test.libs) directory, the jar file names may or may not have versioning (but when using a flatDir repository, I believe that is irrelevant):
libs\activation.jar
libs\imap.jar
....
libs\<closed_source_framework>.jar
....
libs\gson-2.2.4.jar
libs\stax-utils.jar
.....
The reason I can't use our local Nexus server for everything is because of <closed_source_framework>.jar; most of the dependancies in the libs folder come packaged with that distribution, and I can't reliably get the version information to pull them from Nexus.
Now, one of the other teams is publishing their jars to the Nexus server and I'd like to be able to pull their jars from Nexus, so I have (re)defined my dependencies in my build.gradle:
dependencies {
// Grab other-team jar files from Nexus server
compile "other-team-group:other-team-jar-1:version"
compile "other-team-group:other-team-jar-2:version"
compile "other-team-group:other-team-jar-3:version"
// Grab everything else from 'flatDir'
compile name: 'activation'
compile name: 'imap'
...
compile name: 'gson-2.2.4'
compile name: 'stax-utils'
.....
}
The Problem
So now comes my problem. I had expected Gradle would search the repositories in the order specified in my build.gradle; meaning that it would look to the local libs folder first and then go to Nexus if it can't find it locally. What I'm seeing instead is that Gradle is looking at the Nexus server for the jar files already in the local libs folder. Obviously, this is slowing down my build (I have ~30 dependencies defined).
Some Info
Output from gradle properties command, to show repository information:
.....
repositories: [org.gradle.api.internal.artifacts.repositories.DefaultFlatDirArtifactRepository_Decorated#18814b1b, org.gradle.api.internal.artifacts.repositories.DefaultMavenArtifactRepository_Decorated#6dff028]
.....
Output from gradle --info compileJava, to show that Gradle is doing a lookup to Nexus:
.....
// Successfully find the other team jar files in Nexus, this is okay
Download http://nexus-server/nexus/content/groups/public/other-team-group/other-team-jar-1/version/other-team-jar-1-version.pom
Download http://nexus-server/nexus/content/groups/public/other-team-group/other-team-jar-2/version/other-team-jar-2-version.pom
Download http://nexus-server/nexus/content/groups/public/other-team-group/other-team-jar-3/version/other-team-jar-3-version.pom
.....
// Continues looking in Nexus for jar files that should be found in local libs folder
Resource missing. [HTTP GET: http://nexus-server/nexus/content/groups/public//activation//activation-.pom]
Resource missing. [HTTP HEAD: http://nexus-server/nexus/content/groups/public//activation//activation-.jar]
Resource missing. [HTTP GET: http://nexus-server/nexus/content/groups/public///imap//imap-.pom]
Resource missing. [HTTP HEAD: http://nexus-server/nexus/content/groups/public//imap//imap-.jar]
.....
Bottom Line
How can I get Gradle to stop looking at the Maven repository for jar files that I know it will only find locally?

I also posted this question over on the Gradle forums. I have copy/pasted the solution below.
Gradle will prefer an artifact with an pom/ivy descriptor over an
artifact without. I think this is why gradle continues searching after
it finds a match in the flatDir repository. This may or may not solve
your problem, but you could use a FileCollectionDependency instead of
a ModuleDependency.
Eg:
ext {
libs = "${rootProject.projectDir}/libs"
testLibs = "${rootProject.projectDir}/test.libs"
}
dependencies {
compile files("${libs}/activation.jar", "${libs}/imap.jar")
compile files("${testLibs}/gson-2.2.4.jar")
...
}