I'm working on a new project using Gradle and JavaFX.
I have previously been able to create a cross-platform build using Maven, with the following pom dependencies:
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-graphics</artifactId>
<version>13</version>
<classifier>win</classifier>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-graphics</artifactId>
<version>13</version>
<classifier>linux</classifier>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-graphics</artifactId>
<version>13</version>
<classifier>mac</classifier>
</dependency>
How do i go about doing something similar with Gradle?
I have tried the following, with no success.
Is the classifier syntax incorrect?
compile group: 'org.openjfx', name: 'javafx-graphics', version: '11.0.2:win'
compile group: 'org.openjfx', name: 'javafx-graphics', version: '11.0.2:linux'
compile group: 'org.openjfx', name: 'javafx-graphics', version: '11.0.2:mac'
Any hints would be greatly appreciated!
The solution turned out to be the following gradle build dependencies:
implementation group: 'org.openjfx', name: 'javafx-base', version: javaFxVersion, classifier: 'win'
implementation group: 'org.openjfx', name: 'javafx-base', version: javaFxVersion, classifier: 'mac'
implementation group: 'org.openjfx', name: 'javafx-base', version: javaFxVersion, classifier: 'linux'
implementation group: 'org.openjfx', name: 'javafx-controls', version: javaFxVersion, classifier: 'win'
implementation group: 'org.openjfx', name: 'javafx-controls', version: javaFxVersion, classifier: 'mac'
implementation group: 'org.openjfx', name: 'javafx-controls', version: javaFxVersion, classifier: 'linux'
implementation group: 'org.openjfx', name: 'javafx-fxml', version: javaFxVersion, classifier: 'win'
implementation group: 'org.openjfx', name: 'javafx-fxml', version: javaFxVersion, classifier: 'mac'
implementation group: 'org.openjfx', name: 'javafx-fxml', version: javaFxVersion, classifier: 'linux'
implementation group: 'org.openjfx', name: 'javafx-graphics', version: javaFxVersion, classifier: 'win'
implementation group: 'org.openjfx', name: 'javafx-graphics', version: javaFxVersion, classifier: 'mac'
implementation group: 'org.openjfx', name: 'javafx-graphics', version: javaFxVersion, classifier: 'linux'
This allows a javafx gradle build to be run on any OS.
The syntax is indeed incorrect in Gradle. Documentation will show you the different supported syntax.
The two most common are the map style you used and the : separated notation.
So taking one of your dependency as an example, you need to use either:
compile group: 'org.openjfx', name: 'javafx-graphics', version: '11.0.2', classifier: 'win'
compile 'org.openjfx:javafx-graphics:11.0.2:win'
Note that you should also stop using the compile configuration and instead use implementation. See this explanation to understand more on this topic.
Related
I am using selenium "4.1.2" with chrome 97. While selecting value from drop down using select class, getting exception:
java.lang.NoSuchMethodError: org.openqa.selenium.WebElement.getDomAttribute(Ljava/lang/String;)Ljava/lang/String;
Below are dependencies in my project:
ext {
selenium = '4.1.2'
webdrivermanager = '5.0.3'
}
dependencies {
compile("org.seleniumhq.selenium:selenium-java:${selenium}")
// compile("io.github.bonigarcia:webdrivermanager:${webdrivermanager}")
testImplementation "org.seleniumhq.selenium:selenium-chrome-driver:${selenium}"
testImplementation "org.seleniumhq.selenium:selenium-firefox-driver:${selenium}"
testImplementation "org.seleniumhq.selenium:selenium-ie-driver:${selenium}"
testImplementation "org.seleniumhq.selenium:selenium-edge-driver:${selenium}"
testImplementation "org.seleniumhq.selenium:selenium-safari-driver:${selenium}"
testImplementation "org.seleniumhq.selenium:selenium-remote-driver:${selenium}"
testImplementation "org.seleniumhq.selenium:selenium-support:${selenium}"
testImplementation('org.junit.jupiter:junit-jupiter:5.5.1')
testImplementation 'org.hamcrest:hamcrest:2.1'
testImplementation 'org.hamcrest:hamcrest-library:2.1'
testCompile("org.junit.jupiter:junit-jupiter-api:5.7.2")
testRuntime 'org.junit.jupiter:junit-jupiter-engine:5.7.2'
testRuntime("org.junit.platform:junit-platform-launcher:1.7.2")
testCompile('io.github.bonigarcia:selenium-jupiter:3.3.4')
compile group: 'io.qameta.allure', name: 'allure-junit5', version: '2.11.0'
compile group: 'org.apache.pdfbox', name: 'pdfbox', version: '2.0.16'
implementation group: 'org.apache.logging.log4j', name: 'log4j-api', version: '2.17.1'
implementation group: 'org.apache.logging.log4j', name: 'log4j-core', version: '2.17.1'
compile group: 'io.qameta.allure', name: 'allure-gradle', version: '2.7.0'
compile 'org.apache.maven.plugins:maven-surefire-plugin:2.21.0'
compile('com.assertthat:selenium-shutterbug:1.5')
compile 'org.slf4j:slf4j-nop:1.7.25'
implementation group: 'javax.mail', name: 'mail', version: '1.4.7'
implementation group: 'javax.mail', name: 'javax.mail-api', version: '1.6.2'
runtimeClasspath group: 'javax.mail', name: 'javax.mail-api', version: '1.6.2'
compile group: 'net.lightbody.bmp', name: 'browsermob-core', version: '2.1.4'
compile group: 'org.postgresql', name: 'postgresql', version: '42.2.14'
compile group: 'org.mongodb', name: 'mongo-java-driver', version: '3.12.4'
compile group: 'ru.yandex.qatools.ashot', name: 'ashot', version: '1.5.4'
implementation group: 'org.json', name: 'json', version: '20201115'
implementation group: 'com.googlecode.json-simple', name: 'json-simple', version: '1.1.1'
testImplementation group: 'org.junit.platform', name: 'junit-platform-launcher', version: '1.7.2'
testImplementation group: 'org.junit.platform', name: 'junit-platform-runner', version: '1.7.2'
testImplementation group: 'org.junit.platform', name: 'junit-platform-surefire-provider', version: '1.3.2'
}
Any help in resolving this issue is highly appreciated.
This error message...
java.lang.NoSuchMethodError: org.openqa.selenium.WebElement.getDomAttribute(Ljava/lang/String;)Ljava/lang/String;
...is the result of dependency version conflict.
Deep Dive
As per #titusfortner comment in the discussion you need to crosscheck that everything related to Selenium is set to 4.x and that nothing that requires Selenium 3.x (e.g., Appium 7) is included in your dependencies as #asolntsev in the comment mentions:
All implementations of WebElement do override method getDomAttribute(). It never throws UnsupportedOperationException in real life.
Solution
You need to execute mvn dependency:tree or gradle dependencies and you will be able to trace the problem as described in this example.
I would try forcing below dependency in build.gradle
implementation (group: 'org.seleniumhq.selenium', name: 'selenium-api', version: '4.1.2') { force = true }
NoSuchMethod error occurs when dependency version conflict happens. If you are using lower version Appium, upgrading to following version,
implementation 'io.appium:java-client:8.0.0'
may resolve errors.
I have some Android code that needs to find contours and outline them with rectangles. I have no problems with OpenCV initialization (all other functions work fine). But when I run Imgproc.boundingRect(contours.get(i)); error java.lang.NoSuchMethodError occurs.
2021-09-03 18:24:24.331 23081-23081/? E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.fern.mangoeye, PID: 23081
java.lang.NoSuchMethodError: No static method boundingRect(Lorg/opencv/core/Mat;)Lorg/opencv/core/Rect; in class Lorg/opencv/imgproc/Imgproc; or its super classes (declaration of 'org.opencv.imgproc.Imgproc' appears in /data/app/com.fern.mangoeye-jmLx1kABmB7GFFie5xaCyA==/base.apk!classes9.dex)
at com.fern.mangoeye.OpenCVHandler.feedNewYUVData(OpenCVHandler.java:132)
at com.fern.mangoeye.CameraView.onPreviewFrame(CameraView.java:140)
at android.hardware.Camera$EventHandler.handleMessage(Camera.java:1221)
at android.os.Handler.dispatchMessage(Handler.java:107)
at android.os.Looper.loop(Looper.java:237)
at android.app.ActivityThread.main(ActivityThread.java:8167)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
OpenCV initialization has OpenCVLoader.initDebug() as well as BaseLoaderCallback
OpenCV SDK version: 3.4.1
compileSdk: 31
I solved my problem by specifying exclude group: 'org.bytedeco', module: 'opencv' for implementation (group: 'org.bytedeco', name: 'javacv', version: '1.5.5') in build.gradle.
So now my dependencies looks like this:
dependencies {
implementation 'androidx.appcompat:appcompat:1.2.0'
implementation 'com.google.android.material:material:1.3.0'
implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
testImplementation 'junit:junit:4.+'
androidTestImplementation 'androidx.test.ext:junit:1.1.2'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'
implementation project(path: ':opencv')
implementation 'org.jcodec:jcodec:0.2.5'
implementation (group: 'org.bytedeco', name: 'javacv', version: '1.5.5'){
exclude group: 'org.bytedeco', module: 'opencv'
}
implementation group: 'org.bytedeco', name: 'openblas', version: '0.3.13-1.5.5', classifier: 'android-arm'
implementation group: 'org.bytedeco', name: 'openblas', version: '0.3.13-1.5.5', classifier: 'android-arm64'
implementation group: 'org.bytedeco', name: 'openblas', version: '0.3.13-1.5.5', classifier: 'android-x86_64'
implementation group: 'org.bytedeco', name: 'openblas', version: '0.3.13-1.5.5', classifier: 'android-x86'
implementation group: 'org.bytedeco', name: 'ffmpeg', version: '4.3.2-1.5.5', classifier: 'android-arm'
implementation group: 'org.bytedeco', name: 'ffmpeg', version: '4.3.2-1.5.5', classifier: 'android-arm64'
implementation group: 'org.bytedeco', name: 'ffmpeg', version: '4.3.2-1.5.5', classifier: 'android-x86_64'
implementation group: 'org.bytedeco', name: 'ffmpeg', version: '4.3.2-1.5.5', classifier: 'android-x86'
}
Also I created jni folder and copy folders from sdk\native\libs to it. So, the problem was with the initialization of OpenCV
I am getting java.lang.NoSuchMethodError: org.apache.hadoop.security.ProviderUtils.excludeIncompatibleCredentialProviders exception while submitting my spark job using spark-submit
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.security.ProviderUtils.excludeIncompatibleCredentialProviders(Lorg/apache/hadoop/conf/Configuration;Ljava/lang/Class;)Lorg/apache/hadoop/conf/Configuration;
at org.apache.hadoop.fs.s3a.S3AUtils.getAWSAccessKeys(S3AUtils.java:740)
at org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider.<init>(SimpleAWSCredentialsProvider.java:58)
at org.apache.hadoop.fs.s3a.S3AUtils.createAWSCredentialProviderSet(S3AUtils.java:600)
at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:260)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
Spark version Installed: 2.4.5
My Configuration: build.gradle:
buildscript {
repositories {
maven {
url "https://*********/****/content/repositories/thirdparty"
credentials {
username ****User
password ****Pwd
}
}
}
}
plugins {
id 'java'
id 'com.github.johnrengelman.shadow' version '1.2.3'
}
group 'com.felix'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
repositories {
mavenCentral()
mavenLocal()
}
dependencies {
compile group: 'org.apache.spark', name: 'spark-hadoop-cloud_2.11', version: '2.4.2.3.1.3.0-79'
// https://mvnrepository.com/artifact/org.apache.spark/spark-sql
compileOnly group: 'org.apache.spark', name: 'spark-sql_2.11', version: '2.4.5'
// https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind
compileOnly group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.6.7.3'
// https://mvnrepository.com/artifact/org.apache.parquet/parquet-column
compileOnly group: 'org.apache.parquet', name: 'parquet-column', version: '1.10.1'
// https://mvnrepository.com/artifact/org.apache.parquet/parquet-hadoop
compileOnly group: 'org.apache.parquet', name: 'parquet-hadoop', version: '1.10.1'
// https://mvnrepository.com/artifact/org.apache.spark/spark-sketch
compileOnly group: 'org.apache.spark', name: 'spark-sketch_2.11', version: '2.4.5'
// https://mvnrepository.com/artifact/org.apache.spark/spark-core
compileOnly group: 'org.apache.spark', name: 'spark-core_2.11', version: '2.4.5'
// https://mvnrepository.com/artifact/org.apache.spark/spark-catalyst
compileOnly group: 'org.apache.spark', name: 'spark-catalyst_2.11', version: '2.4.5'
// https://mvnrepository.com/artifact/org.apache.spark/spark-tags
compileOnly group: 'org.apache.spark', name: 'spark-tags_2.11', version: '2.4.5'
compileOnly group: 'org.apache.spark', name: 'spark-avro_2.11', version: '2.4.5'
// https://mvnrepository.com/artifact/org.apache.spark/spark-hive
compileOnly group: 'org.apache.spark', name: 'spark-hive_2.11', version: '2.4.5'
// https://mvnrepository.com/artifact/org.apache.xbean/xbean-asm6-shaded
compile group: 'org.apache.xbean', name: 'xbean-asm7-shaded', version: '4.15'
// https://mvnrepository.com/artifact/org.codehaus.janino/commons-compiler
compileOnly group: 'org.codehaus.janino', name: 'commons-compiler', version: '3.0.9'
// https://mvnrepository.com/artifact/org.codehaus.janino/janino
compileOnly group: 'org.codehaus.janino', name: 'janino', version: '3.0.9'
//HIVE Metastore
compile group: 'org.postgresql', name: 'postgresql', version: '42.2.9'
compile 'com.google.guava:guava:22.0'
compile group: 'org.apache.hadoop', name: 'hadoop-common', version: '3.2.1'
compile group: 'org.apache.hadoop', name: 'hadoop-aws', version: '3.2.1'
compile group: 'org.apache.hadoop', name: 'hadoop-client', version: '3.2.1'
compile group: 'com.amazonaws', name: 'aws-java-sdk-bundle', version: '1.11.271'
compile group: 'io.delta', name: 'delta-core_2.11', version: '0.5.0'
compile group: 'joda-time', name: 'joda-time', version: '2.10.5'
compile group: 'org.projectlombok', name: 'lombok', version: '1.16.6'
}
shadowJar {
zip64 true
}
Spark job:
df.distinct()
.withColumn("date", date_format(col(EFFECTIVE_PERIOD_START), "yyyy-MM-dd"))
.repartition(col("date"))
.write()
.format(fileFormat)
.partitionBy("date")
.mode(SaveMode.Append)
.option("fs.s3a.committer.name", "partitioned")
.option("fs.s3a.committer.staging.conflict-mode", "append")
.option("spark.sql.sources.commitProtocolClass", "org.apache.spark.internal.io.cloud.PathOutputCommitProtocol")
.option("spark.sql.parquet.output.committer.class", "org.apache.spark.internal.io.cloud.BindingParquetOutputCommitter")
.option("compression", compressionCodecName.name().toLowerCase())
.save(DOWNLOADS_NON_COMPACT_PATH);
Executed script:
spark-submit --class com.felix.DataIngestionApplication --master local DataIngestion-1.0-SNAPSHOT-all.jar
From what I understand is the hadoop version is creating the issue All the hadoop-* JARs need to be 100% matching on versions. So I have ensured that all org.apache.hadoop dependencies are of the same version (3.2.1). But still it's giving this error.
I want to use hadoop version 3 or newer since that provides newer S3A committers like "PartitionedStagingCommitter". How does everybody using this with Spark 2.4.5?
How can I force/override hadoop version to use as 3.2.1 instead of hadoop versions in Spark/jars? When I looked at /usr/local/Cellar/apache-spark/2.4.5/libexec/jars/ I could see hadoop-common-2.7.3.jar, hadoop-client-2.7.3.jar etc. So how do we force hadoop newer version and therefore I could leverage new S3A comitters.?
Note: If I don't using spark-submit and instead run the application from IntelliJ with all dependencies as compile, then the app starts and executes without exception. I could see the data getting inserted in S3.
I have got this working by installing Spark distribution without Hadoop (user provided Hadoop version option) Then install hadoop version 3.2.1 (brew install hadoop) and create spark-env.sh from spark-env.sh.template file and add following line in spark-env.sh (/usr/local/spark-2.4.5/conf/):
export SPARK_DIST_CLASSPATH=$(hadoop classpath)
Now when I run spark-submit, the job executed without any issues
You can use force option in gradle.
compile(group: 'org.apache.hadoop', name: 'hadoop-aws', version: '3.1.1') {
force = true
}
compile(group: 'org.apache.hadoop', name: 'hadoop-common', version: '3.1.1') {
force = true
}
I am trying to run OpenBLAS preset in the DL4J library but i am having a problem in SetNumThreads.allocate()
the library implementation in build.gradle:
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-arm"
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-arm64"
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-x86"
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-x86_64"
and multiDexEnabled true.
When i run this method in my android app:
INDArray i = Nd4j.zeros(height, width, channels);
I get this error message:
Caused by: java.lang.UnsatisfiedLinkError: No implementation found for void org.bytedeco.openblas.presets.openblas_nolapack$SetNumThreads.allocate()
(tried Java_org_bytedeco_openblas_presets_openblas_1nolapack_00024SetNumThreads_allocate and Java_org_bytedeco_openblas_presets_openblas_1nolapack_00024SetNumThreads_allocate__)
So in my build.gradle file I have this dependencies added.
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.12'
// https://mvnrepository.com/artifact/org.hibernate/hibernate-core
compile group: 'org.hibernate', name: 'hibernate-core', version: '5.3.6.Final'
// https://mvnrepository.com/artifact/mysql/mysql-connector-java
compile group: 'mysql', name: 'mysql-connector-java', version: '8.0.12'
// https://mvnrepository.com/artifact/javax.xml.bind/jaxb-api
compile group: 'javax.xml.bind', name: 'jaxb-api', version: '2.3.0'
// https://mvnrepository.com/artifact/javax.xml/jaxb-impl
compile group: 'javax.xml', name: 'jaxb-impl', version: '2.1'
// https://mvnrepository.com/artifact/org.openjfx/javafx-controls
compile group: 'org.openjfx', name: 'javafx-controls', version: '11'
}
The class of other dependencies were added except for javafx classes. It wasn't downloaded all I get is just the meta-inf directory.
I am using JDK 11, so I really need openjfx repo to use JavaFX. because in JDK 11, javafx is decoupled.
Update: I am using Intellij Idea
Each supported platform has its own version of JavaFx module artifacts, so you need to specify it too:
compile group: 'org.openjfx', name: 'javafx-controls', version: '11', classifier: 'linux' //'win', 'mac'
Additionally, you will need to explicitly add all transitive dependencies of the included javafx modules, because gradle cannot resolve platform specific modules by itself:
compile group: 'org.openjfx', name: 'javafx-base', version: '11', classifier: 'linux'
compile group: 'org.openjfx', name: 'javafx-graphics', version: '11', classifier: 'linux'
Check official documentation: https://openjfx.io/openjfx-docs/#gradle