We Keep Coding

Gradle: compile and run a single test

I have a separate task to run the test that generates open-api.yml specification for my application. Here is the Gradle configuration below:
task generateOpenApiYml(type: Test) {
group = 'verification'
useJUnitPlatform {
includeTags 'openApi'
}
testLogging {
showExceptions true
showStandardStreams = false
showCauses true
showStackTraces true
exceptionFormat "full"
events("skipped", "failed", "passed")
}
}
So, I have one test with openApi JUnit tag. It works very well, but there is a slight thing I want to approve.
The whole idea of this test is that the result open-api.yml file is used to generate Java client to invoke REST endpoints. This client is used in other tests in the project.
I want to put those generated Java classes to .gitgnore because they are generated anyway and there is no need to index those files. The problem is that the generateOpenApiYml task compiles all the tests in src/test/java directory. Therefore, some of them use generated classes. Which leads to compiling errors.
Is it possible to tune the generateOpenApiYml task to make it compile and run only the single test with openApi JUnit tag? Then I could easily put generated Java classes to .gitignore and don't bother about their temporary absence, because other tests won't be compiled.

I figured out the solution. Firstly, I installed the gradle-testsets-plugin.
Then I configured a separate source set like this:
testSets {
generateOpenApiYml
}
So, generateOpenApiYml is the new Gradle task that looks for sources in src/generatedOpenApiYml/java directory.
Afterwards, we need to tune all tasks of test type to bind them with JUnit 5 platform.
tasks.withType(Test) {
group = 'verification'
useJUnitPlatform()
testLogging {
showExceptions true
showStandardStreams = false
showCauses true
showStackTraces true
exceptionFormat "full"
events("skipped", "failed", "passed")
}
}
generateOpenApiYml.outputs.upToDateWhen { false }
I put the upToDateWhen option for convenience to make sure the generateOpenApiYml task is always run on demand and never cached.
Then I have the open-api-generator.
openApiGenerate {
inputSpec = "$buildDir/classes/java/generateOpenApiYml/open-api.json".toString()
outputDir = "$buildDir/generated/openapi".toString()
apiPackage = "..."
invokerPackage = "..."
modelPackage = "..."
configOptions = [
dateLibrary : "java8",
openApiNullable: "false",
]
generatorName = 'java'
groupId = "..."
globalProperties = [
modelDocs: "false"
]
additionalProperties = [
hideGenerationTimestamp: true
]
}
tasks.named('openApiGenerate') {
dependsOn generateOpenApiYml
}
Finally, I just need to run two commands to build and run tests for my whole project.
./gradlew openApiGenerate
./gradlew build
The first one creates the open-api.yml file and generates Java client according to the provided specification. The second one runs tests and build the project normally. The tests running during the build phase uses classes generated by openApiGenerate task. Therefore, I can put them to .gitignore safely.
Hope this will be helpful.

gradle Jigsaw module not found

I try to run a very simple gradle project which uses java 9 modules, but i receive the following error.
/home/vadim/IdeaProjects/test_modules/src/main/java/module-info.java:2: error: module not found: HdrHistogram
requires HdrHistogram;
^
Here is it https://github.com/vad0/test_modules.
The main class does basically nothing.
package app;
import org.HdrHistogram.Histogram;
public class RunHdr {
public static void main(String[] args) {
final Histogram histogram = new Histogram(5);
System.out.println(histogram);
}
}
It uses only one dependency: HdrHistogram. I included this magic command in build.gradle according to official gradle tutorial https://docs.gradle.org/current/samples/sample_java_modules_multi_project.html.
java {
modularity.inferModulePath = true
}
The whole build.gradle looks like this.
plugins {
id 'java'
}
group 'org.example'
version '1.0-SNAPSHOT'
repositories {
mavenCentral()
}
java {
modularity.inferModulePath = true
}
dependencies {
compile group: 'org.hdrhistogram', name: 'HdrHistogram', version: '2.1.12'
testCompile group: 'junit', name: 'junit', version: '4.12'
}
module.info looks like this
module test.modules.main {
requires HdrHistogram;
}
I have already read a number of tutorials on Jigsaw and a whole bunch of stackoverflow questions related to it, but still can't make this simple example work. How do i fix it?
Thank you

Unfortunately, gradle does not treat every jar as a module (in simple words). If you want to find out how exactly is gradle building the module-path (as opposed to class-path), you probably want to start from here, specifically at the isModuleJar method. It's pretty easy to understand (though it took me almost two days to set-up gradle and debug the problem out) that the dependency that you are trying to use : gradle says that it is not a module (it isn't wrong, but I am not sure it is correct either). To make it very correct, gradle will add your dependency to the CLASSPATH, but in the very next line: it will not add your dependency to the module-path, because if fails the filter in isModuleJar.
I do not know if this is a bug or not, or may be this is on purpose, but the solution is easy:
plugins.withType(JavaPlugin).configureEach {
java {
modularity.inferModulePath = true
}
tasks.withType(JavaCompile) {
doFirst {
options.compilerArgs = [
'--module-path', classpath.asPath,
]
classpath = files()
}
}
}
you add it to the path, on purpose. I will flag this as a defect and let's see what they have to say.
EDIT
Even better, use a plugin that is written by a gradle commiter:
plugins {
id 'java'
id 'de.jjohannes.extra-java-module-info' version "0.1"
}
And the easiest option on your case is to do :
extraJavaModuleInfo {
automaticModule("HdrHistogram-2.1.12.jar", "HdrHistogram")
}

Problem when auto generating gRPC stub files using Gradle

I've been asked to implement some gRPC classes for a college course, and have run into some problems when generating the java classes from one source proto file.
Some background first: it's a fairly basic service, with a simple method that receives an id and returns a phone and an email. This is the proto file (BuscarData means FetchData, sorry for the leftover non translation!):
syntax = 'proto3';
option java_multiple_files=true;
option java_generic_services= true;
package uy.edu.um.sd20;
message DataRequest {
int32 id = 1;
}
message DataResponse {
string phone = 1;
string email = 2;
}
service DataRepo {
rpc BuscarData (DataRequest) returns (DataResponse);
}
The idea I had was to generate the classes with gradle plugins. My build.gradle:
plugins {
id 'java'
id "com.google.protobuf" version '0.8.8'
}
apply plugin: 'java'
group 'org.example'
version '1.0-SNAPSHOT'
repositories {
mavenCentral()
}
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.12'
compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.11.4'
implementation 'io.grpc:grpc-netty-shaded:1.29.0'
implementation 'io.grpc:grpc-protobuf:1.29.0'
implementation 'io.grpc:grpc-stub:1.29.0'
}
sourceSets {
main {
proto {
srcDir 'src/main/proto'
}
java {
srcDirs 'src/main/java', 'generated-sources/main/java'
}
}
}
protobuf {
protoc {
artifact = 'com.google.protobuf:protoc:3.11.0'
}
plugins {
grpc {
artifact = 'io.grpc:protoc-gen-grpc-java:1.29.0'
}
}
generateProtoTasks.generatedFilesBaseDir = 'generated-sources'
generateProtoTasks {
all().each { task ->
// Here you can configure the task
}
ofSourceSet('main')
}
}
From what I understood, everything's there: the grpc and protoc dependencies, and the plugin which enables protoc to compile grpc (protoc-gen-grpc), and where to deposit the generated files.
However, there are two problems:
the generated-sources are not marked as source or anything like that, meaning they cannot be imported from other classes
if I'm not mistaken, the generated-sources should generate a skeleton of DataRepoImpl so that I can add the code needed for BuscarData. However, it didn't. Or maybe I should create it, extending from DataRepo.java, but I couldn't test it, due to problem n°1.
I've added a screenshot of the project file structure:
img
As you can see, quite a lot (if not all) of the gradle settings are copy-pasted and scavenged from many different web-sites. I hope I was careful enough not to repeat any imports. There are similar questions, and I tried the solutions there, but no luck there. One example, with which I knew I had to include the gen grpc plugin: another SO question
Any tip regarding anything else is welcome! I'm new to stackoverflow question-asking, so I may have made mistakes regarding the question specificity or aim of the question.
Thanks!
Franri.

For 1), the plugin should put the generated files are the input for java compile tasks even if you do not explicitly add 'generated-sources/main/java' in the sourceSets configuration. Version 0.8.8 has been a while, you can try with newer versions. There might have been some minor fixes for things you may hit.
For 2), you did not add grpc plugin to each generateProto task. It should be
generateProtoTasks {
all().each { task ->
task.plugins { grpc{} }
}
ofSourceSet('main')
}

Exclude package from Jacoco coverage in Jenkins

Trying to exclude packages from coverage report as my Jenkins pipeline fail. I have a sub project with all POJO:s. I don't want to write uittest for all these. Hence, they will drag down branch/line coverage som that coverage will be below threshold and fail my build.
It should be possible to exclude some packages, but I cant get it to work.
I have the following jacoco.gradle file:
apply plugin: 'jacoco'
apply plugin: 'java'
jacoco {
toolVersion = "0.8.2"
}
jacocoTestReport {
reports {
xml.enabled true
csv.enabled false
html.enabled true
}
afterEvaluate {
classDirectories = files(classDirectories.files.collect {
fileTree(dir: it, exclude: '**xxx/yyy/zzz/**')
})
}
task coverage { dependsOn 'jacocoTestReport' }
check.dependsOn 'jacocoTestReport'
The following in my sonar.gradle file:
apply plugin: 'org.sonarqube'
sonarqube {
properties {
property "sonar.forceAnalysis", "true"
property "sonar.forceAuthentication", "true"
property "sonar.java.coveragePlugin", "jacoco"
property "sonar.login", ""
property "sonar.password", ""
}
}
subprojects {
sonarqube {
properties {
property "sonar.jacoco.reportPaths",
"$buildDir/reports/jacoco/allTests.exec"
property "sonar.junit.reportsPath", "$buildDir/test-results/test"
}
}
}
task sonar { dependsOn 'sonarqube' }
In my build.gradle:
apply from: 'gradle/sonar.gradle'
...
apply plugin: 'java'
...
subprojects {
apply from: '../gradle/jacoco.gradle'
...
}
And finally from my Jenkins file:
step([$class: 'JacocoPublisher', changeBuildStatus: false,
exclusionPattern: '**/*Test*.class', inclusionPattern:
'**/*.class', minimumBranchCoverage: '80', sourcePattern: '**/src'])
try {
dir(BUILD_DIR) {
sh './gradlew sonar'
}
But still the xxx.yyy.zzz Is generated in the coverage report in Jenkins!

Finally got it working! The key is the JacocoPuplisher.
step([$class: 'JacocoPublisher', changeBuildStatus: false, exclusionPattern:
'**/xxx/yyy/zzz/**/*.class, **/*Test*.class', inclusionPattern: '**/*.class',
minimumBranchCoverage: '80', sourcePattern: '**/src'])
This is the only way to I get it to work with Jenkins.
Setting the sonar.coverage.exclusion or using the afterEvalueate stuff above has no effect.

I was able to accomplish file/package exclusions from the jacoco() Jenkins pipeline plugin ( which in our standard Jenkinsfile gets called in pipeline { ... post { ... always { ) by just adding directly to the plugin invocation arguments like so:
...
post {
always {
...
// Change the exclusion for the jacoco() jenkins plugin to exclude testutils package.
jacoco(exclusionPattern: '**/testutils/**,**/*Test*.class')
It seems one of the big confusions in this whole topic is that there is a jacocoTestReport gradle target, which has its own exclusion syntax, and then this jacoco() Jenkins pipeline task which seems totally independent. Sonar coverage exclusion looks like maybe a 3rd thing?

Gradle with non-standard named source sets - how do I make them available to the test classes?

I have a (legacy) Java project which I'm re-configuring to build with Gradle. The build should, eventually, output a number of jar archives, so I've divided to split up the source into source sets that correspond to the subdivision of the output into archives.
The file tree looks something like this:
- project root
- src
- setOne
- java
- ...
- resources
- ...
- setTwo
- java
- ...
- setThree
- java
- ...
- resources
- ...
- test
- java
- ...
- build.gradle
and in build.gradle I have the following:
apply plugin: 'java'
sourceSets {
setOne
setTwo {
compileClasspath += setOne.output
}
setThree
test
}
dependencies {
setOne group: 'foo', name: 'bar', version: '1.0'
}
// Setup dependencies so that compileJava compiles all source sets
// while avoiding cyclic dependencies for main and test
sourceSets.matching { ss ->
!(ss.getName().contains('test') || ss.getName().contains('main'))
}.each { ss ->
compileJava.dependsOn ss.getCompileJavaTaskName()
}
When I try to build this, I get a bunch of warnings from the compileSetTwoJava target about some classes not being resolved - referring to classes from the foobar library imported to setOne. I assume this is because Gradle only gives setTwo access to the classes built in setOne - but why isn't it able to resolve the dependencies for the classes imported from setOne in the process?
How do I set up this correctly?

It turns out this was just as easy as I thought it ought to be - if you just start out the right way. Definining
setTwo {
classPath += setOne.runtimeClasspath // instead of setOne.output
}
solves the problem.
UPDATE:
It seems even better to do it among the dependencies, where one can do
dependencies {
setTwo sourceSets.setOne.output
}
and get it working.