We Keep Coding

Gradle/Java: generate resources with executable Java class

Situation is as follows:
during the gradle build, I download and unzip resources from a dependency
one of the classes in the project will process these resources, and generate new resources to be included in the project (as generated resources)
I managed to come up with a bit of a hacky solution, which runs the generator after the 'classes' task, and writes the resources to $buildDir/resources/main/schema:
ext {
schemaGenerator = "org.something.JsonSchemaGenerator"
serviceContractsDir = file("$buildDir/service-contracts")
schemaOutputDir = file("$buildDir/resources/main/schema")
}
task jsonSchemas(type: Exec) {
dependsOn classes
dependsOn serviceContracts
commandLine "java", "-classpath", sourceSets.main.runtimeClasspath.getAsPath(), schemaGenerator, serviceContractsDir, schemaOutputDir
}
test.dependsOn jsonSchemas
assemble.dependsOn jsonSchemas
This works well, and the generated resources are then included in the JAR.
But when uploading the artefacts, it will upload the JAR without the generated resources.
I made an attempt to do it the proper way:
ext {
schemaOutputDir = file("$buildDir/generated-resources")
}
sourceSets.main.output.dir schemaOutputDir, builtBy: jsonSchemas
But then I end up with a circular dependency, as the resources are required by Gradle to build the classes, and the task to generate those resources also depends on the classes:
FAILURE: Build failed with an exception.
What went wrong:
Circular dependency between the following tasks:
:classes
--- :jsonSchemas
    --- :classes (*)
Is there a way to add them to the proper source sets (as generated-resources) so they are
visible to tests
included in the generated JAR?

Adding a new SourceSet that is independent of the main SourceSet should do the trick:
task jsonSchemas(type: Exec) {
// same as above
}
sourceSets {
schema {
output.dir(schemaOutputDir, builtBy: 'jsonSchemas')
}
test {
resources.srcDirs += [sourceSets.schema.output]
}
}
jar {
from sourceSets.schema.output
}

How can I exclude a java file conditionally with the Spring Boot Gradle Plugin and bootJar task?

I have some code that I do not want included in the jar file based on a condition.
My build script looks like
plugins {
id 'java'
id 'org.springframework.boot' version '2.0.0.RELEASE'
}
sourceSets {
main {
java {
if (project.environment == 'prod') {
exclude '**/dangerous/**'
}
forEach {
println it.absolutePath
}
}
}
}
Now, when I run the script with gradlew clean build bootJar -Penvironment=prod the absolute paths of everything but the dangerous java files is printed, but they are still included in the jar.
If I remove the boot plugin and run the jar task, the dangerous class files are still included in the jar.
gradlew clean build jar -Penvironment=prod
plugins {
id 'java'
}
sourceSets {
main {
java {
if (project.environment == 'prod') {
exclude '**/dangerous/**'
}
forEach {
println it.absolutePath
}
}
}
}
If I add an exclude clause to the jar task, the dangerous files are not printed, and they are not included in the jar.
gradlew clean build jar -Penvironment=prod
plugins {
id 'java'
}
sourceSets {
main {
java {
if (project.environment == 'prod') {
exclude '**/dangerous/**'
}
forEach {
println it.absolutePath
}
}
}
}
jar {
exclude '**/dangerous/**'
}
If I enable the boot plugin, and use the bootJar task (which inherits from the Jar task) (gradlew clean build bootJar -Penvironment=prod), I do not see the dangerous files printed, but the files are still included in the jar.
plugins {
id 'java'
id 'org.springframework.boot' version '2.0.0.RELEASE'
}
sourceSets {
main {
java {
if (project.environment == 'prod') {
exclude '**/dangerous/**'
}
forEach {
println it.absolutePath
}
}
}
}
bootJar {
exclude '**/dangerous/**'
}
How can I exclude a java file conditionally with the Spring Boot Gradle Plugin and bootJar task?

I was having same issue when i was using 2.0.1.RELEASE. I created jar using bootJar option. Add exclude inside it with file patterns which you want to exclude from executable jar.
This worked fine with spring 2.0.4.RELEASE version.
bootJar {
exclude("**/dangerous/*")
}

I narrowed down the problem. I didn't put in all of the plugins up above, because I thought the only important ones were java and spring boot. However, my actual code also uses the protobuf plugin. If I remove the configuration property generatedFilesBaseDir, then it successfully excludes the dangerous directory.
However, this opens up a new question of, what the hell is happening?
I was specifying the generated files base dir property so I could reference the generated classes in my source code, but I think I may need to create a different project just for the proto, and add that project as a reference to my main module.
Edit
Making a separate project for the protobuf files and referencing it as a project seems to be a viable workaround for this issue.

Add a 2nd source folder on a Gradle project which uses my Gradle plugin

I created a Gradle plugin which generates some Java code we want.
I tested my plugin on a test project and it worked perfectly, the files were correctly generated.
Now I have to apply my plugin on the project I work on, for doing this I added the following lines to the build.gradle of the project :
buildscript {
repositories {
[...]
mavenLocal() <--
}
dependencies {
[...]
classpath 'my.plugin.path:app:1.0' <-- the maven local path of my plugin
}
}
apply plugin: 'myPlugin' <--
Then, the files that my plugin generates are placed into the src folder. In that case the project build correctly.
BUT, I would like to place the generated files into the target folder.
I tried two ways to resolve it :
Add a sourceSets block into the build.gradle like :
sourceSets {
main {
java {
srcDirs = ['src/main/java', 'target/java']
}
}
}
And when I try to gradle build, the :compileDebugJavaWithJavac task fail, because some code in the src/main/java folder needs the code I generate. But the code in the target folder seems to not be compiled.
Add the source folder through the plugin code
Here is my plugin class (in Groovy) :
import com.android.build.gradle.AppPlugin
import org.gradle.api.Plugin
import org.gradle.api.Project
import org.gradle.api.Task
class MyPlugin implements Plugin<Project> {
#Override
void apply(Project project) {
def hasAppPlugin = project.plugins.hasPlugin AppPlugin
project.afterEvaluate({
def variants = hasAppPlugin ? project.android.applicationVariants : project.android.libraryVariants
variants.all { variant ->
def compileJavaTask = variant.javaCompile
[...] // Adds tasks to the plugins
compileJavaTask.source = project.files("${project.rootDir}/target/java", output)
compileJavaTask.dependsOn javaTask
}
})
}
}
The 2 last lines should be enough, but I have the same problem than in case 1.
I actually have no idea why this doesn't work, do someone have any idea ?

+= instead of = on the line :
compileJavaTask.source += project.files("${project.rootDir}/target/java", output)

Gradle eclipse task not including all sourcesets

I have a java project in gradle where I include sourceSets from another location in addition to sourceSets included in this project. The build and test tasks run all appropiate unittests from both src directories, but the eclipse task is missing sourceSets from one or the other. test sourceset is included from one and the main is the sourceSet from another. How do I make the eclipse task include both sourceSets?
sourceSets {
main {
java {
srcDir '../CommonSource/src/main/java'
}
}
test {
java {
srcDir '../CommonSource/src/test/java'
}
}
}
//I also have the implied src in the current project under the gradle convention src/main/java, and src/main/java.

Gradle multiple jars from single source folder

As for now we have a project structure with single source folder named src, which contains source code for three modules. What I want to do is:
1) Compile source code. This is easily done with sourceSets definition:
sourceSets {
main {
java {
srcDir 'src'
}
}
}
2) Put compilation results into three jars. I am doing this via three 'jar' type tasks:
I am doing this now via three separate tasks:
util.jar
task utilJar(type: Jar) {
from(sourceSets.main.output) {
include "my/util/package/**"
}
}
client.jar
task clientJar(type: Jar) {
from(sourceSets.main.output) {
include "my/client/package/**"
}
}
server.jar
task serverJar(type: Jar) {
from(sourceSets.main.output) {
include "**"
}
excludes.addAll(utilJar.includes)
excludes.addAll(clientJar.includes)
}
The thing is that server.jar should contain all classes that are not contained within client.jar and util.jar. In ant build script we solve this problem by using difference ant task. How this can be done in gradle (my current approach doesn't work)?
Maybe my approach is completely wrong. Please advice.
P.S. as for now we CAN NOT change the project source code folder structure.

I will post my working solution here as an answer (I've got a hint on gradle's forum).
The scopes in gradle are very strange thing :) I thought that every task definition creates an object of some 'Task' class, which is something like 'JarTask' in this particular case. Then I can access any property of the class from anywhere in my build.gradle script. However, I found the only place where I can see the patterns, which are included in jar file - inside a from block of a task. So my working solution for now is to:
1) Define a project-level collection to contain patterns to be excluded from server.jar
2) Exclude all patterns in from block of serverJar task.
Please see final version below
sourceSets {
main {
java {
srcDir 'src'
}
}
}
// holds classes included into client.jar and util.jar, so they are to be excluded from server.jar
ext.serverExcludes = []
// util.jar
task utilJar(type: Jar) {
from(sourceSets.main.output) {
include "my/util/package/**"
project.ext.serverExcludes.addAll(includes)
}
}
// client.jar
task clientJar(type: Jar) {
from(sourceSets.main.output) {
include "my/client/package/**"
project.ext.serverExcludes.addAll(includes)
}
}
// server.jar
task serverJar(type: Jar) {
from(sourceSets.main.output) {
exclude project.ext.serverExcludes
}
}

I think the approach is wrong. I recommend making a project with 3 sub projects.
project
- util
- server (depends on util)
- client (depends on util)
If for some reason you cannot change the class structure use this kind of build files:
settings.gradle
include 'util', 'client', 'server'
build.gradle
subprojects {
apply plugin: 'java'
}
project(':util') {
sourceSets {
main {
java {
srcDir '../src'
include 'util/**'
}
}
}
}
project(':server') {
sourceSets {
main {
java {
srcDir '../src'
include 'server/**'
}
}
}
dependencies {
compile project(':util')
}
}
project(':client') {
sourceSets {
main {
java {
srcDir '../src'
include 'client/**'
}
}
}
dependencies {
compile project(':util')
}
}
You still need directories for subprojects but the sources are in one place as you wanted.
When you run gradle assemble you will have 3 jars with separate set of classes. The advantage of this solution is that we make a proper Gradle multi module project with correct dependencies, not just tasks for building jars.
Please read Multi-Project Builds.

We have the same problem at my company, ie. legacy code that is difficult to migrate into a "good" project structure, and the need to build several jars from the same codebase. We decided to define different sourceSets and build each of the sourceSets using standard Gradle.
We then use iterators to add jar- and javadoc-tasks for each sourceSet:
sourceSets.all { SourceSet sourceSet ->
Task jarTask = tasks.create("jar" + sourceSet.name, Jar.class)
jarTask.from(sourceSet.output)
// Configure other jar task properties: group, description, manifest etc
Task javadocTask = tasks.create("javadoc" + sourceSet.name, Javadoc.class)
javadocTask.setClasspath(sourceSet.output + sourceSet.compileClasspath)
javadocTask.setSource(sourceSet.allJava)
// Extra config for the javadoc task: group, description etc
Task javadocJarTask = tasks.create("javadocJar" + sourceSet.name, Jar.class)
javadocJarTask.setClassifier("javadoc") // adds "-javadoc" to the name of the jar
javadocJarTask.from(javadocTask.outputs)
// Add extra config: group, description, manifest etc
}

I agree in principal with the accepted answer too.
I found a project where the client requires two JAR essentially of the same file except the Manifest is different only by the Class-Path key.
jar {
manifest {
attributes(
"Main-Class": platformMainClass,
"Implementation-Title": platformDisplayName,
"Implementation-Description": platformDescription,
"Platform-Version": platformVersion,
"Implementation-Version": version,
"Build-Assembly-User": System.getProperty("user.name"),
"Build-Assembly-Date": new java.util.Date().toString(),
"Class-Path": configurations.compile.collect { "lib/"+it.getName() }.join(' ')
)
}
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
exclude( [ 'log4j*.properties', 'uk/gov/acme/secret/product/server/**' ])
}
The same manifest and the source code then is:
task applicationClientJar(type: Jar, description: "Creates the Application Client JAR file.") {
dependsOn compileJava
manifest {
attributes(
"Main-Class": platformMainClass,
"Implementation-Title": platformDisplayName,
"Implementation-Description": platformDescription,
"Platform-Version": platformVersion,
"Implementation-Version": version,
"Assembly-Date": new java.util.Date().toString()
)
}
archiveName = "acme-client-${platformVersion}.jar"
destinationDir = file("${buildDir}/libs")
from sourceSets.main.output
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
exclude( [ 'log4j*.properties', 'uk/gov/acme/secret/product/server/**' }
So Grzegorz notation is correct, because the Gradle should know there are two different JAR with GAVs. Multi-module is the preferred option.
compile "uk.gov.acme.secret:acme:1.0" // CORE
compile "uk.gov.acme.secret:acme-client:1.0"
The only way to configure for this is to use the Multi-Module Gradle project and then add a compile and/or deploy dependency to the core / main project.
project(':common:acme-micro-service-webapp') {
dependencies {
compile project(':common:acme-core')
}
}
Inside the 'acme-micro-service-webapp' project, this ensures that the dependent 'common:acme-core' is compiled first.
PS: I am still trying to figure out a better solution.
PS PS: If you are using Maven as well as, it may be possible to hook on the `install' task.