We Keep Coding

How do I resolve UnsatisfiedLinkError with JavaCPP native libraries while using JPMS Java modules?

I have a Java 17 project using Gradle with a multitude of JavaCPP libraries. I started with a simple demo project that I cloned from the JavaCPP Github repo. This sample project incorporates several native libs such as OpenCV, ffmpeg, etc., so I thought it'd be a good test. And, no surprise, it worked just fine. It brought up my camera, did the face detection, etc. All very cool.
My aim - I want to modularize my JavaCPP projects to be JPMS compliant.
Not easy. So, to troubleshoot, I figure that I would start with good test code, which is why I'm working with the official JavaCPP Gradle demo program.
So, I did the following to convert it to be JPMS compliant:
Created a module-info.java placed down src/main/java. I added the appropriate requires statements (see below).
Modified build.gradle to add several *-platform dependencies and a few other plugins, including JavaFX.
The TL;DR - I got it to work (though the camera appears at an angle in the app window, which is just weird, but I'm assuming I'm still missing a library in module-info.java). The problem is that it only worked after I not only specified numerous additional *-platform dependencies in build.gradle, but also needed to list the actual native platform libraries in module-info.java. So, for instance, I need to add the following statement:
requires org.bytedeco.opencv.macosx.x86_64;
If I do not do that, then I get the following error:
Exception in thread "main" java.lang.UnsatisfiedLinkError: no jniopencv_core in java.library.path: /Users/brk009/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
My main question - How can I make a modular JavaCPP project build and execute properly without hard coding the platform dependent native libraries in module-info.java? I thought that just specifying the *-platform libraries in module-info.java would do it, but nope.
If this was just a project for my own system, then fine - I'd live with it. However, I want to pass some of my example code off to my students. It'd be fine if they all ran Macs. However, my students have quite a heterogeneous platform base (i.e. a mix of Mac, Windows, and Linux users.) Ideally, it'd be great to have a platform-independent codebase and let my program build and run regardless of the platform. Heck, I'd even be happy if I only needed to specify the platform as a parameter for gradlew as a command-line argument, such as indicated here, where I could just specify -PjavacppPlatform=linux-x86_64. But that did not work either.
I did verify that Loader.Detector.getPlatform() returns the correct platform string, and Loader.getCacheDir() returns ~/.javacpp/cache as you would expect.
Any help/guidance would be immensely appreciated! Thank you kindly.
module-info.java
module HelloJavaCPP {
requires java.base;
requires java.desktop;
requires org.bytedeco.javacpp;
requires org.bytedeco.javacpp.macosx.x86_64; // I do NOT WANT to hard code any platform!
requires org.bytedeco.javacv;
requires org.bytedeco.opencv;
requires org.bytedeco.opencv.macosx.x86_64;
requires org.bytedeco.ffmpeg;
requires org.bytedeco.ffmpeg.macosx.x86_64;
requires org.bytedeco.openblas;
requires org.bytedeco.openblas.macosx.x86_64;
}
build.gradle
plugins {
id 'application'
id 'java'
id 'java-library'
id 'org.openjfx.javafxplugin' version '0.0.12'
id 'org.javamodularity.moduleplugin' version '1.8.10'
id 'org.bytedeco.gradle-javacpp-platform' version '1.5.7'
}
group = 'org.hello'
version = '1.5.7'
repositories {
mavenLocal()
mavenCentral()
maven { url 'https://oss.sonatype.org/content/repositories/snapshots/' }
}
javafx {
version = "17.0.2"
modules = [ 'javafx.graphics','javafx.controls', 'javafx.fxml' ]
}
dependencies {
api "org.bytedeco:javacv-platform:1.5.7"
api 'org.bytedeco:opencv-platform:4.5.5-1.5.7'
// api "org.bytedeco:opencv-platform-gpu:4.5.5-$version"
api "org.bytedeco:ffmpeg-platform-gpl:5.0-$version"
api 'org.bytedeco:openblas-platform:0.3.19-1.5.7'
testImplementation 'junit:junit:4.13.2'
}
application {
mainModule = "$moduleName"
mainClass = "org.hello.Demo"
}
settings.gradle
I'm including this just incase.
pluginManagement {
repositories {
mavenLocal()
mavenCentral()
maven { url 'https://oss.sonatype.org/content/repositories/snapshots/' }
gradlePluginPortal()
}
}
rootProject.name = 'HelloJavaCPP'
gradle.rootProject { ext.javacppVersion = '1.5.7' }

Those links posted by Samuel above were immensely helpful. It turns out there are some modularity peculiarities with JavaFX that can wreck havoc when using JavaFX with non-JavaFX modules such as those in JavaCPP. See here.
The key part that was important:
We can either use the run goal of the JavaFX Maven plugin, the java goal of the Exec Maven plugin, or manually launch java with a module path computed from Maven dependencies and option --add-modules ALL-MODULE-PATH.
Once I figured out how to add a JVM argument in Gradle, I was able to remove all hard-coded architecture requires statements, and I can now use gradlew run, let JavaCPP's Loader class do all the work of discovering the architecture itself and loading the appropriate native libraries!
The two most important files that need to change:
module-info.java
Notice how much simpler this becomes, and it has NO hard-coded platform system architecture information, which is exactly what we want:
module HelloJavaCPP {
requires java.base;
requires java.desktop;
requires org.bytedeco.javacpp;
requires org.bytedeco.javacv;
requires org.bytedeco.opencv;
requires org.bytedeco.ffmpeg;
requires org.bytedeco.openblas;
}
build.gradle
The most important change I needed to make (which I got from information posted here was to add a run configuration, and specify the JVM argument `--add-modules
plugins {
id 'application'
id 'java'
id 'java-library'
id 'org.openjfx.javafxplugin' version '0.0.12'
id 'org.javamodularity.moduleplugin' version '1.8.10'
id 'org.bytedeco.gradle-javacpp-platform' version "$javacppVersion"
}
group = 'org.hello'
version = '1.5.7'
repositories {
mavenCentral()
}
ext {
// javacppPlatform - should be autodetected, but can also specify on cmd line
// as -PjavacppPlatform=macosx-x86_64
// javacppPlatform = 'linux-x86_64,macosx-x86_64,windows-x86_64,etc' // defaults to Loader.getPlatform()
javacppPlatform = 'macosx-x86_64' // defaults to Loader.getPlatform()
}
javafx {
version = "17.0.2"
modules = [ 'javafx.graphics','javafx.controls', 'javafx.fxml' ]
}
dependencies {
api "org.bytedeco:javacpp-platform:$javacppVersion"
api "org.bytedeco:javacv-platform:$javacppVersion"
api "org.bytedeco:opencv-platform:4.5.5-1.5.7"
api "org.bytedeco:ffmpeg-platform-gpl:5.0-1.5.7"
api "org.bytedeco:openblas-platform:0.3.19-1.5.7"
testImplementation 'junit:junit:4.13.2'
}
application {
mainModule = "$moduleName"
mainClass = "org.hello.Demo"
}
// THIS WAS THE PRIMARY CHANGE:
run {
jvmArgs = ['--add-modules', 'ALL-MODULE-PATH']
}
It's worth noting that I was able to remove the ext configuration in build.gradle, which allows Loader.getPlatform() to do the work of determining the platform at runtime, and it worked just fine! (I left it in place just for reference purposes.)
I hope this helps others. I did NOT test out building an image, as judging from what I read, that is quite an additional level of complexity. We'll tackle that another time.
Thank you again.

How to shared gradle tasks across many projects in different git repositories

We have lots of libraries for which we have the following snippets to configure the respective plugins. We would want to avoid code duplication and rather want to pull these definitions from a base repository which can be shared across all projects. How should this be configured?
checkstyle {
showViolations = false
ignoreFailures = true
..
}
pmd {
..
}
license {
..
}
spotless {
..
}
artifactory {
..
}

With an easy approach you can import build script plugins into other build scripts even if they are located in a different location / different repository. The only requirement that I think is necessary that on the machine where you will build the project at the end you have access to those repositories.
Imagine you have GitHub repository that contains main configuration.
https://github.com/user/shared-gradle-config/blob/master/scriptPlugin.gradle
That contains:
checkstyle {
showViolations = false
ignoreFailures = true
..
}
pmd {
..
}
...
Now be sure that you use raw resources in other scripts. Every repository service will have different kind of link. Below example for GitHub:
https://raw.githubusercontent.com/user/shared-gradle-config/master/scriptPlugin.gradle
At the beginning of the Gradle build script for the actual projects you have to include application of the remote script plugin:
// Application of shared remote Gradle script plguin with common configuration
// With this line all the checkStyle, PMD configuration and whatever you have declared in the script will be applied to current project
apply from: "https://raw.githubusercontent.com/user/shared-gradle-config/master/scriptPlugin.gradle"
// Whatever custom logic below
wrapper {
version = "7.0.0"
distributionType = Wrapper.DistributionType.ALL
}
...
I made a prototype project miself as I tried to understand how to share build configuration this way. Maybe it can help you, https://github.com/rivancic/gradle/tree/master/script-plugin. Note its not in final version, still improving it..

Firebase Performance Plugin causing slow build time

When using Firebase Performance in Android Studio the gradle task app:transformClassesWithFirebasePerformancePluginForDebug is taking significantly longer than any other task and is therefore dramatically slowing down my gradle build times.
Slow Build shown in Profiler

Firebase in our project caused 40% build time increase. To speed up debug builds we added a possibility to switch it on/off using build parameters in app/build.gradle and root build.gradle files:
app:
if (!project.hasProperty("disable-performance-plugin")) {
apply plugin: 'com.google.firebase.firebase-perf'
}
root/buildscript/dependencies:
if (!project.hasProperty("disable-performance-plugin")) {
classpath('com.google.firebase:firebase-plugins:1.1.5') {
exclude group: 'com.google.guava', module: 'guava-jdk5'
}
}
when running from the command line use
./gradlew your-task -Pdisable-performance-plugin
when working from Android Studio, add the flag to compiler options:

All of the existing answers are valid, but they all miss something.
To deal with this issue, you have 2 major options to choose from.
1. Use firebasePerformanceInstrumentationEnabled property
This is the official way provided by the SDK itself to disable it during the build process.
What this does:
Reduces transformClassesWithFirebasePerformancePluginFor* task execution time to ~5-10s.
Disables automatic traces and request monitoring, but leaves custom traces enabled. You can control the latter with AndroidManifest <meta-data> tags and calls to FirebasePerformance.getInstance().setPerformanceCollectionEnabled(). More info in the docs.
How to do this:
I think it's much easier to only enable plugin in those rare cases when we need it (usually it will be only when we publish the app) rather than disable it in all other cases.
Note: Of course, with manual builds you might forget to enable it. So if you don't have CI, it might be worth adding some other automatic scripting in Gradle, or sticking to the opposite approach that is used in other answers.
In general though, we only need two steps:
Add the following line to gradle.properties file:
firebasePerformanceInstrumentationEnabled=false
Use the following command in your CI config or manual builds:
./gradlew assembleRelease -PfirebasePerformanceInstrumentationEnabled=true
Pros:
Only one property to set up.
Cons:
Plugin still adds additional ~5-15s to the build time.
2. Use custom Gradle project property to avoid applying firebase-perf Gradle plugin
What this does:
transformClassesWithFirebasePerformancePluginFor* task is not executed at all. Also we save some additional ~5–10s overhead that is present when using the first solution.
Same as the first method – disables automatic traces and request monitoring, but leaves custom traces enabled. You can control the latter with AndroidManifest <meta-data> tags and calls to FirebasePerformance.getInstance().setPerformanceCollectionEnabled(). More info in the docs.
How to do this:
This approach has similar points and warnings, and also includes two steps:
Modify your app module's build.gradle file:
if (project.hasProperty('useFirebasePerf')) {
apply plugin: 'com.google.firebase.firebase-perf'
}
Note: you don't need to apply the same check to your project-level build.gradle:
classpath "com.google.firebase:firebase-plugins:$firebase_plugins_version"
This declaration won't be used in any way by Gradle when the plugin itself is not enabled.
And you don't need to exclude guava-jdk5 dependency there, if you're using firebase-plugins v1.1.1 or later as stated in the docs.
Use the following command in your CI config or manual builds:
./gradlew assembleRelease -PuseFirebasePerf
Pros:
Completely eliminates time expenses associated with Firebase Performance Gradle plugin.
Cons:
Introduces conditional check for applying plugin in your Gradle script, some might argue that it's not an idiomatic approach.
* (Bonus option) Use custom Gradle project property to exclude firebase-perf SDK
If you don't use custom traces or any other features from Firebase Performance SDK and only rely on automatic monitoring (that is, you don't have any dependencies on SDK in your code), then you can exclude this dependency for non-production builds.
How to do this:
All you need to do is update your app module's build.gradle file:
If you chose to use the first option, then change your dependency like this:
if (project.property('firebasePerformanceInstrumentationEnabled') == 'true') {
implementation "com.google.firebase:firebase-perf:${firebase_perf_version}"
}
If you chose the second one:
if (project.hasProperty('useFirebasePerf')) {
implementation "com.google.firebase:firebase-perf:${firebase_perf_version}"
}
Advantage:
This might save you some additional ~5-10s, spent on configuring dependency and
"ProGuarding" it.
Drawbacks:
Your production APK size will be larger than debug one by ~0.5mb. This might disrupt your reports or predictions, so you need to be aware of it.
If you were close to surpassing 64K method count limit, you might suddenly step over it on production builds and find yourself in the MultiDex zone. And that means extra work to do and tests to run. All because Firebase Performance brings a formidable number of almost 5K method references (after applying ProGuard with optimizations).
You can also check out my article where I expand a bit more on this topic.

Firebase Performance has released a new version of perf-plugin (v1.3.0). This would enable disabling the Firebase Performance Monitoring Gradle plugin for a specific build variant (including buildTypes or productFlavors).
An example below:
android {
// ...
debug {
FirebasePerformance {
// Set this flag to 'false' to disable #AddTrace annotation processing and
// automatic HTTP/S network request monitoring
// for a specific build variant at compile time.
instrumentationEnabled false
}
}
}
Reference to release notes:
https://firebase.google.com/support/release-notes/android#2019-07-10

All comments in this thread are valid. I want to suggest a very simple way to disable that for debug builds:
if (getGradle().getStartParameter().getTaskRequests().toString().contains("Release")) {
apply plugin: 'com.google.firebase.firebase-perf'
}

For newer versions of the Firebase perf plugin (1.3.0 and up) with Kotlin DSL you'll need to add the following:
android {
...
buildTypes {
...
all {
with((this as ExtensionAware).extensions["FirebasePerformance"] as FirebasePerfExtension) {
setInstrumentationEnabled(!isDebuggable)
}
}
...
}
}
For the Groovy version you can check out the Firebase documentation.

Just to give another option to disable transformClassesWithFirebasePerformancePluginForDebug, here's my recipe:
In main build.gradle folder:
if (!project.gradle.startParameter.taskNames.any { taskName ->
taskName.toLowerCase().contains('assemble') && taskName.toLowerCase().contains('debug') }) {
classpath("com.google.firebase:firebase-plugins:$firebasePluginVersion") {
exclude group: 'com.google.guava', module: 'guava-jdk5'
}
}
In the build.gradle app file:
if (!project.gradle.startParameter.taskNames.any { taskName ->
taskName.toLowerCase().contains('assemble') && taskName.toLowerCase().contains('debug') }) {
apply plugin: 'com.google.firebase.firebase-perf'
}

I ran into this problem as well. Originally we had been using a variant of the answer provided by R. Zagórski, but based on a similar thread from the Gradle forums it seems like conditionally applying a plugin to a project isn't the right way to go:
Plugins can’t be applied to only “part of your project”. They are either applied or not.
Conditionally applying plugins does seem to work if you can do it right, but it's not an officially supported feature. Further down the same thread, another point is made:
But the plugin should allow you to configure it at a finer grained level.
Sure enough, there actually is a property exposed by the Firebase plugin that lets you toggle instrumentation on or off (therefore toggling the increased build time). Using this property is tricky, though, since you have to apply it at exactly the right time during the building process, but once you've got that then you can essentially pivot it on whatever you want.
The following code snippet is how we're pivoting instrumentation based on Debug vs. Non-Debug build variants. It's written in Kotlin, but I imagine it would translate to Groovy just as well:
plugins {
...
id ("com.google.firebase.firebase-perf")
}
...
android {
...
applicationVariants.all {
val variant = this
val isFirebaseEnabled = !variant.javaCompiler.name.contains("Debug", true)
gradle.taskGraph.whenReady {
if (this.hasTask(variant.javaCompiler))
{
project.FirebasePerformance.isInstrumentationEnabled = isFirebaseEnabled
}
}
}
...
}
Note that with this in place, the transformClassesWithFirebasePerformancePluginFor* task will still always run for every build variant, but it will complete almost immediately for a variant that doesn't have instrumentation enabled.

I think the clearest way is this with kotlin DSL gradle 👇
in app level build.gradle
buildTypes {
getByName(BuildType.DEBUG) {
extensions.configure<com.google.firebase.perf.plugin.FirebasePerfExtension>{
setInstrumentationEnabled(false)
}
}
}
in dependendencies part:
dependencies {
releaseImplementation(Dependencies.FIREBASE_PERFORMANCE) //implementation for only release mode (you can vary for other variants)
}

I have simplified option 2 of this answer https://stackoverflow.com/a/53270530/1635488
Define a property in gradle.properties
useFirebasePerf=false
Disable perf plugin
if (useFirebasePerf.toBoolean()) {
apply plugin: 'com.google.firebase.firebase-perf'
}
Remove dependency
if (useFirebasePerf.toBoolean()) {
implementation 'com.google.firebase:firebase-perf:16.2.3'
}
Enable Performance Monitoring only for CI builds (i would recommend only for release builds)
gradlew assembleRelease -PuseFirebasePerf=true

A much cleaner way in kotlin DSL
buildTypes {
//My custom extension
forName("debug") {
roject.ext.set("firebasePerformanceInstrumentationEnabled", "false")
}
}
Implementation of forName
fun <T> NamedDomainObjectContainer<T>.forName(name: String, action: T.() -> Unit) {
this.maybeCreate(name)
this.getByName(name, object: Action<T>{
override fun execute(t: T) {
t.action()
}
})
}

Property 'sonar.jacoco.reportPath' is deprecated. Please use 'sonar.jacoco.reportPaths' instead

Property 'sonar.jacoco.reportPath' is deprecated. Please use
'sonar.jacoco.reportPaths' instead.
I keep getting this message when running SonarQube through Gradle and the phrase "reportPath" does not even appear even once in the entire multi-module project. I even put the sonarqube property under allprojects to override any defaults that may be there. Any tips on how I can get rid of this error?
I am using:
allprojects {
sonarqube {
properties {
property "sonar.jacoco.reportPaths", "${project.buildDir}/jacoco/test.exec"
}
}
}
EDIT 1:
Gradle wrapper 3.1
Am using this in the root of build.gradle
plugins {
id "jacoco"
id "org.sonarqube" version "2.5"
}
And tried your suggestion with
allprojects {
sonarqube {
properties {
property "sonar.jacoco.reportPath", ""
property "sonar.jacoco.reportPaths", "${project.buildDir}/jacoco/test.exec"
}
}
}
No dice, what do you think?

The question is, which version of the sonarQube gradle plugin you are using:
https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Gradle
The sonarqube gradle plugin sets some values per default, eg. if you use JaCoCo, which is probably the case, it automatically adds that field, besides the groovy one too.
So generally speaking, you need to wait for an update of the sonarqube gradle plugin, which gets rid of this, and is using the other config value.
Maybe you can also try to override the setting, by setting it to empty like sonar.jacoco.reportPath=

Publishing artifact to remote maven repository on s3 using Gradle

I've been researching and trying to find examples for the above and found the following:
For Android Projects:
Hosting a Private Maven Repo on Amazon S3
gradle maven push project
These plugin however expects certain Android properties specified and throws errors like Could not find property 'android' on task ':androidJavadocs'.
For Remote repositories over SFTP, there is maven-publish.
Besides, it seems to be possible to do this in Maven.
Would be great if anyone can point me to an example of how to do this.

Gradle has a plugin to do this now however it is in incubating state and yet to be released.
https://docs.gradle.org/current/userguide/publishing_maven.html

I've slightly modified the code from your second link (here) for Java-based projects. The only problematic sections are at the bottom of the file, where the source and javadoc jars are being created since he is directly referencing Android. If you replace the final tasks (line 95 onwards) with the following it should resolve your problem:
task sourcesJar(type: Jar) {
classifier = 'sources'
from sourceSets.main.allSource
}
// Makes the Javadocs
task javadocs(type: Javadoc) {
// I set this to false here since I reference third party classes in my Javadocs
// which fails the Javadoc generation, but this can be removed
failOnError false
source = sourceSets.main.java.srcDirs
}
task javadocJar(type: Jar, dependsOn: javadocs) {
classifier = 'javadoc'
from javadoc.destinationDir
}
artifacts {
archives sourcesJar
archives javadocJar
}