We Keep Coding

Can IntelliJ import non-standard source directories from Gradle?

I am converting over to using IntelliJ (version 2019.1). The multi-project directory structure used has the standard src/main/java and src/test/java for each project, but additionally has some non-standard ones such as: src/testsupport/java.
Gradlew (using the internal/recommended gradlew packaged within IntelliJ) is used to import the projects. The Gradle build files include both:
apply plugin: 'idea'
apply plugin: 'java'
Edited to improve clarity
Every project imports fine. Interproject references work to the standard directories. However, when I am in Project B, but need access to src/generated/java or src/testsupport/java from Project A, those are not imported (import statements that compile fine from the gradle command line show up as unresolvable within IntelliJ). Is there a configuration change or something needed to make these take effect?
Currently, I have:
subprojects {
idea {
module {
testSourceDirs += project.sourceSets.generated.java.srcDirs
testSourceDirs += project.sourceSets.testsupport.java.srcDirs
}
}
}

You need help Gradle out by creating a source set for the custom sources your projects define. So from your question, something like:
(using Kotlin DSL)
allprojects {
apply {
plugin("idea")
plugin("java-library")
}
repositories {
mavenCentral()
}
configure<SourceSetContainer> {
create("generated") {
compileClasspath += project.the<SourceSetContainer>()["main"].output
runtimeClasspath += project.the<SourceSetContainer>()["main"].output
}
create("testsupport") {
compileClasspath += project.the<SourceSetContainer>()["main"].output
runtimeClasspath += project.the<SourceSetContainer>()["main"].output
}
}
val api by configurations
val testImplementation by configurations
val testRuntimeOnly by configurations
dependencies {
api(platform("org.junit:junit-bom:5.5.1"))
testImplementation("org.junit.jupiter:junit-jupiter-api")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
}
val test by tasks.getting(Test::class) {
useJUnitPlatform()
}
}
The above will give you:
So now you want to use projectA in projectB, so projectB's Gradle file would include a dependency on projectA:
dependencies {
implementation(":projectA")
}
This should hopefully get you started. Keep in mind, the examples given above use the Kotlin DSL which you should be able to convert back to Groovy.
References:
https://docs.gradle.org/current/userguide/java_plugin.html#source_sets
https://docs.gradle.org/current/userguide/java_testing.html#sec:configuring_java_integration_tests

Include scripts with Gradle Kotlin DSL

I'm trying to start using Kotlin DSL with gradle in the project with the following restrictions:
Project has different modules (moreover: sometimes these modules use different plugins, however if two projects uses the same plugin then version of the plugins are the same).
Project has internal corporate repositories only (e.g. we don't use jcenter directly, we use proxy for it).
What we have with Groovy:
Some common configurations items are excluded to the separate scripts. Please check the example below.
Gradle modules include these files.
As a result (just based on my example):
We don't need to add the same code lines into the each module.
The most of projects have difference just with dependency list.
I tried to reproduce the same with Gralde KTS and received the following difficulties:
I'm unable to apply plugin in the include file and use it in the module. In this case I receive compilation error (because plugin types are not added into the module script).
I'm unable to extract constants to the something common to use them in the each scripts (root build.gradle.kts inclusive). With Groovy I can just use variable like springBootVersion, however with Kotlin Script I have to create the same property in the each module file.
Precompiled script plugins does not work without public repositories access (e.g. I'm unable to configure common script file with idea "just use default embedded Kotlin Script version, download all dependencies from these urls: ...".
Include file sample:
apply plugin: 'kotlin'
compileKotlin {
kotlinOptions {
jvmTarget = "1.8"
}
}
compileTestKotlin {
kotlinOptions {
jvmTarget = "1.8"
}
}
Gradle module sample:
apply from: "${rootDir}/gradle/include/kotlin-common-include.gradle"
dependencies {
compile project(':my.project.libraries.common')
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web', version: springBootVersion
}
Questions:
How can I put all common constants (such as dependency versions) to the separate file to include them just by using something like springBootVersion or Constants.springBootVersion with compile-time checks?
How can I extract plugin applying to the include scripts (to avoid Gradle module scripts overload)?
How can I use precompiled script plugins without public global repositories access?
Additional links:
Issue for Gradle KTS for plugin applying.
Issue for Gradle KTS for shared constants extracting.

There are limitations in Kotlin DSL currently (5.3) that prevents to have compile-time checks everywhere. In order for Kotlin DSL to work it has to add extensions on top of the Gradle API and it can't do it magically. First of all you need to go through Kotlin DSL Primer page to understand it.
How can I extract plugin applying to the include scripts (to avoid Gradle module scripts overload)?
The one way to do it is to use precompiled build scripts with Kotlin DSL Plugin. In order to do it you need to move your script into $rootDir/buildSrc project. Here how it might look like:
// $rootDir/buildSrc/build.gradle.kts
plugins {
`kotlin-dsl`
}
repositories {
maven {
url = uri("http://host:8082/artifactory/...")
}
}
Then put your common script like that:
// $rootDir/buildSrc/src/main/kotlin/common.gradle.kts
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
plugins {
kotlin("jvm") version "1.3.21"
}
tasks.compileKotlin {
kotlinOptions.jvmTarget = "1.8"
}
tasks.compileTestKotlin {
kotlinOptions.jvmTarget = "1.8"
}
Then you can apply this script as to a plugin like that:
// $rootDir/build.gradle.kts
subprojects {
apply(id = "common")
}
How can I use precompiled script plugins without public global repositories access?
Configuring custom repositories for pre-compiled scripts plugin is no different that your usual build script. Do it like that:
// $rootDir/buildSrc/settings.gradle.kts
pluginManagement {
repositories.maven {
url = uri("http://host:8082/artifactory/...")
}
}
The other way around that if you don't want to use precompiled plugins is to configure extensions explicitly. You can do it like that:
// $rootDir/gradle/common.gradle.kts
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
tasks.withType<KotlinCompile> {
kotlinOptions.jvmTarget = "1.8"
}
And in main script:
// $rootDir/build.gradle.kts
subprojects {
apply {
plugin(KotlinPlatformJvmPlugin::class)
from("common.gradle.kts")
}
}
How can I put all common constants (such as dependency versions) to the separate file to include them just by using something like springBootVersion or Constants.springBootVersion with compile-time checks?
There is no good way to do it currently. You can use extra properties, but it won't guarantee compile time checks. Something like that:
// $rootDir/dependencies.gradle.kts
// this will try to take configuration from existing ones
val compile by configurations
val api by configurations
dependencies {
compile("commons-io:commons-io:1.2.3")
api("some.dep")
}
// This will put your version into extra extension
extra["springBootVersion"] = "1.2.3"
And you can use it like this:
// $rootDir/build.gradle.kts
subprojects {
apply {
plugin<JavaLibraryPlugin>()
from("$rootDir/dependencies.gradle.kts")
}
And in your module:
// $rootDir/module/build.gradle.kts
// This will take existing dependency from extra
val springBootVersion: String by extra
dependencies {
compile("org.spring:boot:$springBootVersion")
}

How can I exclude a java file conditionally with the Spring Boot Gradle Plugin and bootJar task?

I have some code that I do not want included in the jar file based on a condition.
My build script looks like
plugins {
id 'java'
id 'org.springframework.boot' version '2.0.0.RELEASE'
}
sourceSets {
main {
java {
if (project.environment == 'prod') {
exclude '**/dangerous/**'
}
forEach {
println it.absolutePath
}
}
}
}
Now, when I run the script with gradlew clean build bootJar -Penvironment=prod the absolute paths of everything but the dangerous java files is printed, but they are still included in the jar.
If I remove the boot plugin and run the jar task, the dangerous class files are still included in the jar.
gradlew clean build jar -Penvironment=prod
plugins {
id 'java'
}
sourceSets {
main {
java {
if (project.environment == 'prod') {
exclude '**/dangerous/**'
}
forEach {
println it.absolutePath
}
}
}
}
If I add an exclude clause to the jar task, the dangerous files are not printed, and they are not included in the jar.
gradlew clean build jar -Penvironment=prod
plugins {
id 'java'
}
sourceSets {
main {
java {
if (project.environment == 'prod') {
exclude '**/dangerous/**'
}
forEach {
println it.absolutePath
}
}
}
}
jar {
exclude '**/dangerous/**'
}
If I enable the boot plugin, and use the bootJar task (which inherits from the Jar task) (gradlew clean build bootJar -Penvironment=prod), I do not see the dangerous files printed, but the files are still included in the jar.
plugins {
id 'java'
id 'org.springframework.boot' version '2.0.0.RELEASE'
}
sourceSets {
main {
java {
if (project.environment == 'prod') {
exclude '**/dangerous/**'
}
forEach {
println it.absolutePath
}
}
}
}
bootJar {
exclude '**/dangerous/**'
}
How can I exclude a java file conditionally with the Spring Boot Gradle Plugin and bootJar task?

I was having same issue when i was using 2.0.1.RELEASE. I created jar using bootJar option. Add exclude inside it with file patterns which you want to exclude from executable jar.
This worked fine with spring 2.0.4.RELEASE version.
bootJar {
exclude("**/dangerous/*")
}

I narrowed down the problem. I didn't put in all of the plugins up above, because I thought the only important ones were java and spring boot. However, my actual code also uses the protobuf plugin. If I remove the configuration property generatedFilesBaseDir, then it successfully excludes the dangerous directory.
However, this opens up a new question of, what the hell is happening?
I was specifying the generated files base dir property so I could reference the generated classes in my source code, but I think I may need to create a different project just for the proto, and add that project as a reference to my main module.
Edit
Making a separate project for the protobuf files and referencing it as a project seems to be a viable workaround for this issue.

How to override a gradle build with dev/prod configuration

I'm evaluating gradle for replacing an ant build script and I can't manage to find a solution for creating a standard build script that correctly manages dev/prod environment.
Than ant script (it's for a java project, not android) is structured in this way:
a common script with the standard tasks (compile, build-jar, build-war)
a specific project script that includes the first one and through some properties it defines where the war task should pick the correct files
Our project structure/taks allows to override entire directories in the final war. Let consider this example:
the dev configuration is the standard one and lays int the dir webcontent
there are multiple prod conf (one of each specific installation, we do not have more that 10 different prod configs) all under the prod dir (i.e. *prod/conf1*m prod/conf2, etc)
The ant build has the dev_build task as the prod_conf1_build one, the prod_conf2_build one ,etc
the XXX_build task do the same things:
specify the parent (it's a project property) dir that contains the env dir/files
call the same ant taks that build the war using the property specified in the calling task
I'm trying to do the same in gradle but it seems that even calling a taks from another one it creates some problem (i.e. the task is always up to date)
Here is the script (it's a working draft, I'm learning gradle) that tries to do the same but it's not working when I call war_prod the taks does nothing since it reports up-to-date
apply plugin: 'java'
apply plugin: 'war'
apply plugin: 'eclipse'
project.ext.envdir = ""
eclipse {
jdt {
sourceCompatibility = 1.8
targetCompatibility = 1.8
javaRuntimeName = "jdk-1.8.x"
}
}
// In this section you declare where to find the dependencies of your project
repositories {
maven {
url 'http://artifactory.zzzz.priv/artifactory/libs-release'
url 'http://artifactory.zzzz.priv/artifactory/libs-snapshot'
credentials {
username 'xxxx'
password 'yyyy'
}
}
}
// In this section you declare the dependencies for your production and test code
dependencies {
// The production code uses the SLF4J logging API at compile time
compile 'org.slf4j:slf4j-api:1.7.18'
// Declare the dependency for your favourite test framework you want to use in your tests.
// TestNG is also supported by the Gradle Test task. Just change the
// testCompile dependency to testCompile 'org.testng:testng:6.8.1' and add
// 'test.useTestNG()' to your build script.
testCompile 'junit:junit:4.12'
}
task war_prod {
project.ext.envdir='prod/conf1'
project.ext.envdir=project.ext.envdir.replaceAll('\\\\',File.pathSeparator)
project.ext.envdir=project.ext.envdir.replaceAll('/',File.pathSeparator)
tasks.war.execute()
}
war {
eachFile {
println 'endir' + project.ext.envdir
println 'evaluating' + it
FileTree tree = fileTree(dir: project.ext.envdir)
tree.visit { FileVisitDetails file->
if (!file.file.isDirectory()) {
println '\tFileVisitDetails relpath ' + file.relativePath
println '\tsourcepath ' + it.file.getAbsolutePath()
println '\tcontains ' + it.file.getAbsolutePath().contains(project.ext.envdir)
if (it.relativePath == file.relativePath && !it.file.getAbsolutePath().contains(project.ext.envdir)) {
it.exclude()
println '\texcluding ' + it
} else {
if (it!=null) {
//println '\tincluding ' + it
}
}
}
}
}
from 'prod/conf1'
}
Can anyone point me in the right direction for creating a correct gradle script?
Is there a specific gradle way to build war files with prod/dev configurations (where the configuration is represented by some dir and files)?

In such scenarios task rules might be very useful. Basic idea is to keep configurations in a structured way and use a general task to build a war file with a configuration defined. Please have a look at build.gradle below:
apply plugin: 'war'
repositories {
mavenCentral()
}
tasks.addRule("Pattern: buildWar<ENV>") { String taskName ->
if (taskName.startsWith('buildWar')) {
def env = (taskName - 'buildWar').toLowerCase()
if (env in ['dev', 'prod',]) {
task(taskName, type: War) {
println "Configuring env: $env"
from("src/main/conf/$env") {
into("conf")
}
}
} else {
println "Invalid env: $env, skipping."
}
}
}
The buildWarENV rule defined here is pretty self descriptive. It accepts two environments dev and prod and prepares war file by taking configuration from appropriate folder. You can find a demo here. In case of questions, just ask.
P.S. Gradle has a bit different working model than ant, start with the basics. And what's important, never run a task from within other task.

Missing project dependencies in eclipse

I'm currently in the situation of editing to gradle projects( "database" & "masterdata"). Masterdata depends on the database-project. Database is published to a Nexus server from where it is loaded by masterdata as a dependency.
The build.gradle of masterdata
import org.gradle.plugins.ide.eclipse.model.SourceFolder
apply plugin: "java"
apply plugin: "eclipse"
sourceCompatibility = 1.7
version = '0.1-SNAPSHOT'
group = "net.example"
def nexusHost = "http://nexus:8081"
repositories {
logger.lifecycle("Configuration: Repositories")
maven {
url nexusHost + "/nexus/content/groups/public"
}
}
dependencies {
logger.lifecycle("Configuration: Dependencies")
compile 'net.example:database:0.1-SNAPSHOT' // project where the changes happen
compile 'com.google.guava:guava:14.0.1'
testCompile 'ch.qos.logback:logback-classic:1.0.13'
testCompile 'org.testng:testng:6.8.5'
testCompile 'org.dbunit:dbunit:2.4.9'
testCompile 'org.mockito:mockito-all:1.9.5'
testCompile 'org.easytesting:fest-assert-core:2.0M10'
testCompile 'org.hsqldb:hsqldb:2.2.9'
}
eclipse.classpath.file {
beforeMerged { classpath ->
classpath.entries.clear()
logger.lifecycle("Classpath entries cleared")
}
whenMerged { cp ->
cp.entries.findAll { it instanceof SourceFolder && it.path.startsWith("src/main/") }*.output = "bin/main"
cp.entries.findAll { it instanceof SourceFolder && it.path.startsWith("src/test/") }*.output = "bin/test"
cp.entries.removeAll { it.kind == "output" }
logger.lifecycle("Classpath entries modified")
}
}
When I change something in the database project it needs a complete build, publish, etc. till I see the changes in the masterdata project. In the company where I previously worked we had a similar setup using maven. There I saw changes in dependencies immediately with out publishing them first. Is this also possible with gradle? Maybe via Multi-Project Builds?
Basically a the following entry is missing from .classpath:
<classpathentry combineaccessrules="false" kind="src" path="/database"/>
Is there a way to automate the generation of it.
Update: As a workaround I add the entry manually to the .classpath

I did some additional searching and currently this is only possible with multi-project builds. Basically you need all your projects in one gigantic multi-project build. There you can reference them as you like and get the correct dependencies in eclipse as it seems.
There is a jira issue with a feature request to make that possible without a multi-project build. Custom logic for eclipse will only help with builds in eclipse, because in a gradle build it would use the dependency from the repository, where the changes are missing. You need to make sure that all changed dependencies are build and published before building the main project.
Eclipse workaround:
eclipse.classpath.file {
whenMerged { cp ->
// remove library dependencies
def toBeRemoved = cp.entries.findAll { it instanceof Library
&& ((Library) it).library.path.contains('someProject') }
//configure the project dependencies:
def toBeAdded = [ new ProjectDependency('/someProject', null)]
cp.entries -= toBeRemoved
cp.entries += toBeAdded
}
}
This would still fail when doing a manual gradle build, but if you using a CI-system with a good build order you should be fine.
Solution Multi-Project Build:
Creating a Multi-Project build is easier than I thought and it is also possible when the "subproject" is on the same level.
settings.gradle:
includeFlat 'database'
build.gradle:
dependencies {
...
compile project(':database')
...
}
This works well with gradle builds and with eclipse. The only disadvantage is that you always have to checkout the subproject everytime you checkout the project that depends on it. I'm sure someone can build some fancy groovy logic to fix that.