We Keep Coding

How to write Gradle plugin to cut down build.gradle file

A bit of Context first:
I am working on migrating my companies projects to be built by Gradle.
One thing, that this results in, is having redundancy in my build.gradle files,
as I am configuring the same Skeleton over and over again.
This includes:
Setting the java-,maven-publish- and org.sonarcube-plugin
Configuring the repositories to be mavenCentral and our private Artifactory Repo
Configuring the publishing block, that is all the same, except for the artifactId
Building a Manifest inside the Jar block (using helper Methods, to correctly build the Manifests classpath)
Helper Methods
two Tasks
two dependsOn statements
build.gradle file as of now:
plugins {
id 'io.spring.dependency-management' version '1.0.12.RELEASE'
id "org.sonarqube" version "3.2.0"
id 'maven-publish'
id 'java'
}
group = 'group'
version = 'version'
sourceCompatibility = '11'
ext.artifactName = 'ProjectName'
// Where to look for dependencies:
repositories {
mavenCentral()
maven{
credentials{
username = "${artifactory_user}"
password = "${artifactory_password}"
}
url "${artifactory_contextUrl}"
allowInsecureProtocol = true
}
}
// Where to publish what Artifacts to:
publishing {
publications {
maven(MavenPublication) {
groupId = 'group'
artifactId = 'ProjectName'
String buildEnvVar = System.env.BUILDENVIRONMENT
if(buildEnvVar == null){
version = 'LOCAL BUILD'
}else{
version = 'version'
}
from components.java
}
}
repositories {
maven {
// change to point to your repo, e.g. http://my.org/repo
name = "gradle-dev"
url = "${artifactory_contextUrl}"
allowInsecureProtocol = true
credentials{
username = "${artifactory_user}"
password = "${artifactory_password}"
}
}
}
}
dependencies {...}
jar {
// configuration of variables
String dateString = new Date().format("yyyy-MM-dd HH:mm:ss ")
String localBuild = "LOCAL BUILD by " + System.getProperty("user.name") + " on " + dateString
String buildEnvVar = System.env.BUILDENVIRONMENT
String buildEnvironment
String classpath = createCP()
if(buildEnvVar == null){
buildEnvironment = localBuild
archiveName = "ProjectName"
}else{
buildEnvironment = buildEnvVar
archiveFileName= "ProjectName_" + version + ".jar"
delete fileTree("build/libs") {
include('*')
}
}
manifest {
attributes (
"Main-Class": "org.example.foo",
"Specification-Title" : "ProjectName",
"Specification-Vendor" : "blab",
"Specification-Version" : "Spec-version",
"Implementation-Title" : "$System.env.JOB_NAME",
"Implementation-Version" : "Impl-version",
"Implementation-Vendor" : "blub",
"Implementation-Vendor-Id" : "blob",
"Implementation-Url" : "bleb",
"Build-By" : buildEnvironment,
'Class-Path': classpath
)
}
}
String createCP () {
// super secret can't share
}
// will suffix the jars with release or debug, depending on it being compiled with or without debug-information
project.gradle.taskGraph.whenReady{
boolean isDebug = project.gradle.taskGraph.getAllTasks().join(' ').contains('debugJar')
compileJava.options.debug = isDebug
String suffix = isDebug? "debug" : "release"
String fullJarName = "$artifactName-$suffix" + ".jar"
jar.setProperty('archiveName', fullJarName)
}
tasks.named('test') {
useJUnitPlatform()
}
task debugJar() {}
debugJar.dependsOn(jar)
//Downloads all Jars the project depends on, and saves them in buildDirectory/output/libs if the gradle build command is executed.
task copyToLib(type: Copy) {
into "${buildDir}/output/libs"
from configurations.runtimeClasspath
}
build.dependsOn(copyToLib)
what I want to achive:
plugins {
id 'io.spring.dependency-management' version '1.0.12.RELEASE'
id "org.sonarqube" version "3.2.0"
id 'maven-publish'
id 'java'
id 'mySuperPlugin'
}
// Configure mySuperPlugin
mySuperPlugin {
artifactId = 'xyz'
mainClass = 'org.example.foo'
version = 'version'
stuffFromOtherTasks = ...
}
// Where to look for dependencies:
repositories {
mavenCentral()
maven{
credentials{
username = "${artifactory_user}"
password = "${artifactory_password}"
}
url "${artifactory_contextUrl}"
allowInsecureProtocol = true
}
}
dependencies {...}
Most of the values are the same.
The ones that aren't are passed in via Environment-Variables (Jenkins-JobName,...),
or get determined through helper Methods.
I reckon, that i will most likely not end up with a buildfile like the one above,
but atleast some of the buildfile must be outsourceable.
I know as of now, that i can create seperate Tasks in a plugin, like comparing two files, that have been passed. What I didn't find a solution to yet:
Can I modify the Jar Task of the project applying the plugin, inside the plugin?
How do I pass Outputs from other Tasks into my plugins tasks?
How do I access the applying projects data (i.e. the runtimeClasspath)
Is a plugin even what i want to do, or is there another way of cutting down the build.gradle file?
I am relatively unexperienced with gradle. I have read through quite a bit of the docs and other postings, but chances are i just overlooked some best-practice way of doing certain things.
Therefore, feel free to criticize my buildfile aswell as my approach!

You can do this in a couple of ways. And this comes down to if your project is composed of multiple sub-projects or if they are stand alone projects. For stand alone projects you can do the following in your settings.gradle file:
includeBuild("../common-project/build.gradle")
The common project would just house the common build.gradle file, and you'd declare all of the items you want to share in there. It would look like a normal build.gradle file.
That would require that each project share the common configuration from another project, but wouldn't require any additional projects be checked out. Just the common project and the project itself. For more details see:
https://docs.gradle.org/current/userguide/composite_builds.html
If you have more of a multi-project build like say many micro-services or subprojects that make up a larger project then using multi-project setup and declare the common pieces in the allprojects closure in the root build.gradle:
allprojects {
plugin: java
plugin: web
repositories {
....
}
dependencies {
....
}
}
In the multi-project case you'd have to check out the top level project and all subprojects. Your folder layout might look like this:
- my-project
- service-1
src
build.gradle
- service-2
src
build.gradle
- service-3
src
build.gradle
build.gradle
settings.gradle
In the multi-project setup service1, service2, and service3 would be declared in the settings.gradle file using include like so:
rootProject.name = 'my-project'
include `service1`
include `service2`
include `service3`
In a multi-project setup you'd typically house that in a single source repository as oppose to using includeBuild where each project would belong to a separate source code repo. The former way forces you to checkout the appropriate number of projects in the multi-project case. But, in the includeBuild case the developer would have to know to check out minimum of 2 projects.

gradle does not import local java libraries

I have the following folder structure:
-bin
-build/build.gradle (the gradle script)
-lib/[*.jar] (libraries that the project is using)
-src/folder/folder/[*.java] (project's source code)
The following content for the build.gradle script:
plugins {
id 'java'
id 'groovy'
}
buildDir = new File('../bin')
sourceCompatibility = JavaVersion.VERSION_1_8
sourceSets {
main {
java {
allJava.srcDirs = [ '../src/folder/folder/ ']
compileClasspath = fileTree('../bin')
}
}
}
repositories {
flatDir {
dirs '../lib'
}
}
dependencies {
implementation fileTree('../lib')
}
tasks.register('javac', JavaCompile) {
println 'Call javac'
source.forEach { e -> println e}
classpath = sourceSets.main.compileClasspath
destinationDirectory = file('../bin')
source sourceSets.main.allJava.srcDirs
includes.add('*.java')
sourceCompatibility = JavaVersion.VERSION_1_8
}
When running gradle javac I got the error: error: cannot find symbol import com...
The documentations clearly says:
dependencies {
.
.
.
//putting all jars from 'libs' onto compile classpath
implementation fileTree('libs')
}
I'm using Gradle 7.3.1

Allow me to give you some general advice first.
I can strongly recommend using the Kotlin DSL instead of the Groovy DSL.
You instantly get strongly typed code in the build scripts and much better IDE support.
Also you should imho consider changing your project layout to be more like most other Java projects out there and especially not use a libs directory, but use normal dependencies in a repository where transitive dependencies are then handled automatically and so on.
But to answer your actual question, this is the build complete build script in Groovy DSL that you want:
plugins {
id 'java'
}
buildDir = '../bin'
java {
toolchain {
languageVersion.set(JavaLanguageVersion.of(8))
}
}
sourceSets {
main {
java {
srcDirs = ['../src/folder/folder']
}
}
}
dependencies {
implementation fileTree('../lib')
}
And this is the matching Kotlin DSL version:
plugins {
java
}
setBuildDir("../bin")
java {
toolchain {
languageVersion.set(JavaLanguageVersion.of(8))
}
}
sourceSets {
main {
java {
setSrcDirs(listOf("../src/folder/folder"))
}
}
}
dependencies {
implementation(fileTree("../lib"))
}

How do I consume given dependency with multiple classifiers and extensions in Gradle?

I am trying to figure out how to consume the *.{so,jar} files listed in https://repo1.maven.org/maven2/org/sosy-lab/javasmt-solver-z3/4.8.10/ (see also the corresponding entry on mvnrepository.com) as exemplified (with Maven) here, with Gradle 6.8.3 and a Kotlin configuration. Relevant parts of my code are
repositories {
jcenter()
mavenCentral()
}
dependencies {
implementation(group = "org.sosy-lab", name = "javasmt-solver-z3", version = "4.8.10", classifier = "com.microsoft.z3", ext = "jar")
implementation(group = "org.sosy-lab", name = "javasmt-solver-z3", version = "4.8.10", classifier = "libz3", ext = "so")
implementation(group = "org.sosy-lab", name = "javasmt-solver-z3", version = "4.8.10", classifier = "libz3java", ext = "so")
}
The output I get is
Could not resolve org.sosy-lab:javasmt-solver-z3:4.8.10.
I already tried (1.) commenting out any two out of the three dependencies, (2.) reordering the repositories. The output is the same. What am I doing wrong here?

Gradle searches for a pom.xml per default and it fails because some of the JavaSMT artifacts don't have any (or no useful ones).
You can however tell Gradle to search for artifacts directly.
To do this you need to add a metadataSource to your repo like so:
repositories {
jcenter()
mavenCentral {
metadataSources {
artifact()
}
}
}

Problem when auto generating gRPC stub files using Gradle

I've been asked to implement some gRPC classes for a college course, and have run into some problems when generating the java classes from one source proto file.
Some background first: it's a fairly basic service, with a simple method that receives an id and returns a phone and an email. This is the proto file (BuscarData means FetchData, sorry for the leftover non translation!):
syntax = 'proto3';
option java_multiple_files=true;
option java_generic_services= true;
package uy.edu.um.sd20;
message DataRequest {
int32 id = 1;
}
message DataResponse {
string phone = 1;
string email = 2;
}
service DataRepo {
rpc BuscarData (DataRequest) returns (DataResponse);
}
The idea I had was to generate the classes with gradle plugins. My build.gradle:
plugins {
id 'java'
id "com.google.protobuf" version '0.8.8'
}
apply plugin: 'java'
group 'org.example'
version '1.0-SNAPSHOT'
repositories {
mavenCentral()
}
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.12'
compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.11.4'
implementation 'io.grpc:grpc-netty-shaded:1.29.0'
implementation 'io.grpc:grpc-protobuf:1.29.0'
implementation 'io.grpc:grpc-stub:1.29.0'
}
sourceSets {
main {
proto {
srcDir 'src/main/proto'
}
java {
srcDirs 'src/main/java', 'generated-sources/main/java'
}
}
}
protobuf {
protoc {
artifact = 'com.google.protobuf:protoc:3.11.0'
}
plugins {
grpc {
artifact = 'io.grpc:protoc-gen-grpc-java:1.29.0'
}
}
generateProtoTasks.generatedFilesBaseDir = 'generated-sources'
generateProtoTasks {
all().each { task ->
// Here you can configure the task
}
ofSourceSet('main')
}
}
From what I understood, everything's there: the grpc and protoc dependencies, and the plugin which enables protoc to compile grpc (protoc-gen-grpc), and where to deposit the generated files.
However, there are two problems:
the generated-sources are not marked as source or anything like that, meaning they cannot be imported from other classes
if I'm not mistaken, the generated-sources should generate a skeleton of DataRepoImpl so that I can add the code needed for BuscarData. However, it didn't. Or maybe I should create it, extending from DataRepo.java, but I couldn't test it, due to problem n°1.
I've added a screenshot of the project file structure:
img
As you can see, quite a lot (if not all) of the gradle settings are copy-pasted and scavenged from many different web-sites. I hope I was careful enough not to repeat any imports. There are similar questions, and I tried the solutions there, but no luck there. One example, with which I knew I had to include the gen grpc plugin: another SO question
Any tip regarding anything else is welcome! I'm new to stackoverflow question-asking, so I may have made mistakes regarding the question specificity or aim of the question.
Thanks!
Franri.

For 1), the plugin should put the generated files are the input for java compile tasks even if you do not explicitly add 'generated-sources/main/java' in the sourceSets configuration. Version 0.8.8 has been a while, you can try with newer versions. There might have been some minor fixes for things you may hit.
For 2), you did not add grpc plugin to each generateProto task. It should be
generateProtoTasks {
all().each { task ->
task.plugins { grpc{} }
}
ofSourceSet('main')
}

Gradle Jaxb Plugin Issue

I am trying to generate classes from multiple .xsds in gradle. Tried like below, but it does not work and i get error that file not exists.
buildscript {
ext {
springBootVersion = '1.5.4.RELEASE'
}
repositories {
jcenter()
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
classpath("com.github.jacobono:gradle-jaxb-plugin:1.3.6")
}
}
apply plugin: 'java'
apply plugin: 'eclipse-wtp'
apply plugin: 'org.springframework.boot'
apply plugin: 'war'
apply plugin: 'com.github.jacobono.jaxb'
version = '0.0.1-SNAPSHOT'
repositories {
mavenCentral()
}
configurations.all {
exclude module: 'spring-boot-starter-logging'
}
configurations {
jaxb
}
task createDirs {
file("$buildDir/generated-sources").mkdirs()
}
xjc.dependsOn createDirs
jaxb {
xsdDir = "src/main/resources/xsd"
xjc {
destinationDir = "$buildDir/generated-sources"
taskClassname = "org.jvnet.jaxb2_commons.xjc.XJC2Task"
generatePackage = "com.test.generated1"
args = ["-Xinheritance", "-Xannotate"]
}
}
compileJava {
dependsOn(xjc)
}
compileTestJava {
dependsOn(xjc)
}
dependencies {
compile('org.springframework.boot:spring-boot-starter-web')
compile('org.springframework.boot:spring-boot-starter-log4j2')
compile('org.springframework.boot:spring-boot-starter-actuator')
jaxb('org.jvnet.jaxb2_commons:jaxb2-basics-ant:0.6.5')
jaxb('org.jvnet.jaxb2_commons:jaxb2-basics:0.6.4')
jaxb('org.jvnet.jaxb2_commons:jaxb2-basics-annotate:0.6.4')
jaxb('com.sun.xml.bind:jaxb-xjc:2.2.7-b41')
jaxb('com.sun.xml.bind:jaxb-impl:2.2.7-b41')
}
If i remove destinationDir = file("build/generated-sources"), it generates classes inside src/main/java which i do not want.
I get error .../build/generated-sources: non-existent directory
Any help is appreciated :)
NOTE: Gradle Version 3.2.1

Try using a variable for build directory "$buildDir/generated-sources"
Like vampire noticed you need to create the directory first.
I would suggest creating a task for it with doFirst() which will execute sooner on the execution phase.
task createDirs {
doFirst{
file("$buildDir/generated-sources").mkdirs()
}
}
and then run if before xjc:
xjc.dependsOn createDirs
And in this task particular task try to just pass the raw /build path because it seems like destinationDir already points to the root. Not sure what is going on here. Other than that I strongly sugest usign the variables $buildDir
jaxb {
xsdDir = "src/main/resources/xsd"
xjc {
destinationDir = "build/generated-sources"
taskClassname = "org.jvnet.jaxb2_commons.xjc.XJC2Task"
generatePackage = "com.test.generated1"
args = ["-Xinheritance", "-Xannotate"]
}
}
Another thing is you could watch on the directory changes if you don't want to run expensive xjc task all the time.
xjc {
inputs.dir ("$projectDir/src/main/resources/xsd")
outputs.dir ("$buildDir/generated-sources")
}

One point is that you should use the variable buildDir instead of using hard-coded 'build'.
That's not your problem though. Your problem is, that the plugin you are using does not create a non-existent target directory. So add the creation action to the task like
jaxb {
doFirst {
file("$buildDir/generated-sources").mkdirs()
}
}
Or if you prefer, create a separate task that does only the directory creation and then add a dependency from jaxb to your directory creation task.