I'm working on migrating our project to Bazel.
WORKSPACE
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
RULES_JVM_EXTERNAL_TAG = "2.10"
RULES_JVM_EXTERNAL_SHA = "1bbf2e48d07686707dd85357e9a94da775e1dbd7c464272b3664283c9c716d26"
http_archive(
name = "rules_jvm_external",
strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
sha256 = RULES_JVM_EXTERNAL_SHA,
url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)
load("#rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
name = "maven",
artifacts = [
"com.foo:bar:1.0.0-SNAPSHOT"
"org.apache.commons:commons-lang3:3.9",
],
repositories = [
"https://our-maven-repo",
],
resolve_timeout = 200,
fail_on_missing_checksum = False,
fetch_sources = True
)
BUILD
java_library(
name = "our-name",
srcs = glob([
"src/main/java/**/*.java"
]),
resources = glob([
"src/main/resources/**",
]),
deps = [
"#maven//:com.foo_bar",
"#maven//:org.apache.commons_commons-lang3"
],
)
When I run:
PS> bazel fetch //:our-name
Output
INFO: Call stack for the definition of repository 'maven' which is a coursier_fetch (rule definition at C:/users/name/_bazel_name/73nyktky/external/rules_jvm_external/coursier.bzl:620:18):
- C:/users/name/_bazel_name/73nyktky/external/rules_jvm_external/defs.bzl:89:5
- C:/project/WORKSPACE:19:1
INFO: Repository 'maven' used the following cache hits instead of downloading the corresponding file.
* Hash '8f35f92fb8e021f96b3aa8145c66c3b2e29295baabb28ff50569e613438afcbd' for https://github.com/coursier/coursier/releases/download/v2.0.0-RC3-4/coursier.jar
If the definition of 'maven' was updated, verify that the hashes were also updated.
ERROR: An error occurred during the fetch of repository 'maven':
Error while obtaining the sha256 checksum of v1/https/our-repo/prod/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar: java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("python" C:/users/name/_bazel_name/73nyktky/external/bazel_tools/tools/build_defs/hash/sha256.py C:/users/name/_bazel_name/73nyktky/external/maven/v1/https/ourrepo/prod/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar artifact.sha256): The system cannot find the file specified.
(error: 2)
ERROR: C:/project/BUILD:1:1: no such package '#maven//': Error while obtaining the sha256 checksum of v1/https/our-repo/prod/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar: java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("python" C:/users/name/_bazel_name/73nyktky/external/bazel_tools/tools/build_defs/hash/sha256.py C:/users/name/_bazel_name/73nyktky/external/maven/v1/https/ourrepo/prod/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar artifact.sha256): The system cannot find the file specified.
(error: 2) and referenced by '//:our-name'
ERROR: Evaluation of query "deps(//:our-name)" failed: errors were encountered while computing transitive closure
Loading: 0 packages loaded
org.apache.commons:commons-lang3:3.9 jar does get download, along with a sha1 and md5 hash. The com.foo:bar:1.0.0-SNAPSHOT jar does NOT get downloaded. The sha1 and md5 do get downloaded.
I think my issue is that our repo does not have any sha256 hashes to download so the fetch (or build) fail with that error. However when i look at the actual rules_jvm_external https://github.com/bazelbuild/rules_jvm_external#checksum-verification it seems that sha256 is not mandatory?
Any ideas on what I'm doing wrong?
Bazel 1.1.0
Windows 10 Enterprise, version 1803, OS build 17134.1069
rules_jvm_external maintainer here.
ERROR: C:/project/BUILD:1:1: no such package '#maven//': Error while obtaining the sha256 checksum of v1/https/our-repo/prod/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar: java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("python" C:/users/name/_bazel_name/73nyktky/external/bazel_tools/tools/build_defs/hash/sha256.py C:/users/name/_bazel_name/73nyktky/external/maven/v1/https/ourrepo/prod/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar artifact.sha256): The system cannot find the file specified.
This is the real error where Windows' CreateProcessW is complaining that python is not available. This is previously reported in this issue as well. We added SHA256 checking in 2.3, and unfortunately this depends on Python.
Do you have python installed? There is an outstanding PR that drops the dependency here.
Related
My CustomTest.java has this import:
com.google.protobuf.Timestamp
I'm using java_test_suite to run tests in my BUILD file like so:
java_test_suite(
name = "all-tests",
srcs = glob(["src/test/java/**/*.java"]),
runner = "junit5",
test_suffixes = ["Test.java"],
runtime_deps = JUNIT5_DEPS,
deps = [
":mylib",
"#com_google_protobuf//:timestamp_proto",
artifact("org.junit.jupiter:junit-jupiter-api"),
artifact("org.junit.jupiter:junit-jupiter-params"),
] + deps,
)
However when I run tests on it using:
bazel test //:all-tests
I'm getting this error:
src/test/java/com/x/CustomTest.java:75: error: [strict] Using type com.google.protobuf.Timestamp from an indirect dependency (TOOL_INFO: "#com_google_protobuf//:timestamp_proto wrapped in java_proto_library"). See command below **
private static Timestamp timestampFromMilli(long milli) {
^
** Please add the following dependencies:
#com_google_protobuf//:timestamp_proto to //:src/test/java/com/x/CustomTest
** You can use the following buildozer command:
buildozer 'add deps #com_google_protobuf//:timestamp_proto' //:src/test/java/com/x/CustomTest
What do I need to do exactly? I tried using the buildozer command but all I got was:
rule 'src/test/java/com/x/CustomTest' not found
Where do I need to add this #com_google_protobuf//:timestamp_proto?
Looking at protobuf's build files, it looks like timestamp_proto is a plain proto_library:
https://github.com/protocolbuffers/protobuf/blob/main/BUILD.bazel#L70-L74
https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/BUILD.bazel#L64-L68
and so per the advice here:
https://github.com/protocolbuffers/protobuf/blob/main/BUILD.bazel#L19-L25
you might just need to use java_proto_library to make the java version of the proto:
java_proto_library(
name = "timestamp_java_proto",
deps = ["#com_google_protobuf//:timestamp_proto"],
)
and then use that in the deps of your java_test_suite instead of the timestamp_proto.
Just a guess, but the error message is not very helpful maybe because there happens to be a Timestamp java class in the deps of the plain proto library, and Strict deps is finding that one in the test's indirect dependencies. Might be worth filing a bug about it on https://github.com/bazelbuild/bazel/issues
I am a beginner in Bazel and I need to migrate from sbt. I use Scala Rules to build my app.
I use the following dependencies with following aliases (to prevent typos):
Alias
Group
Artifact
Version
borer_core
io.bullet
borer-core_2.12
1.6.3
borer_derivation
io.bullet
borer-derivation_2.12
1.6.3
scala_logging
com.typesafe.scala-logging
scala-logging_2.12
3.9.2
logback
ch.qos.logback
logback-classic
1.2.3
tagging
com.softwaremill.common
tagging_2.12
2.2.1
ujson
com.lihaoyi
ujson_2.12
1.2.2
All these dependencies will be installed by JVM External Rules. It looks like so in Workspace:
dp_deps = [
borer_core,
borer_derivation,
scala_logging,
logback,
tagging,
ujson,
]
maven_install(
name = "maven",
artifacts = dp_deps,
repositories = ["http://repo1.maven.org/maven/"],
fetch_sources = True
)
Then I try to build a jar with scala_library in my BUILD file. It looks so:
scala_library(
name = "some_lib",
srcs = glob(["some_lib/src/main/**/*.scala"]),
unused_dependency_checker_mode = 'warn',
deps = [
"#maven//:io_bullet_borer_core_2_12",
"#maven//:io_bullet_borer_derivation_2_12",
"#maven//:com_typesafe_scala_logging_scala_logging_2_12",
"#maven//:ch_qos_logback_logback_classic",
"#maven//:com_softwaremill_common_tagging_2_12",
"#maven//:com_lihaoyi_ujson_2_12"
]
)
Then I try to build it with the following command:
bazel build //test-dir:some_lib --verbose_failures
and I receive the following Error with Log:
Error:
ERROR: /Users/<edited>/test-dir/BUILD.bazel:136:14: scala //test-dir:some_lib failed: Worker process did not return a WorkResponse:
Log:
---8<---8<--- Start of log snippet, file at /private/var/tmp/_bazel_<edited>/07059176926c1842b8d4e633b0ddf1f4/bazel-workers/worker-7-Scalac.log ---8<---8<---
[... truncated ...]
scala.tools.nsc.typechecker.Macros$MacroExpander.expand(Macros.scala:622)
at scala.tools.nsc.typechecker.Macros$MacroExpander.apply(Macros.scala:609)
at scala.tools.nsc.typechecker.Macros.standardMacroExpand(Macros.scala:784)
at scala.tools.nsc.typechecker.Macros.standardMacroExpand$(Macros.scala:782)
at scala.tools.nsc.Global$$anon$4.standardMacroExpand(Global.scala:480)
at scala.tools.nsc.typechecker.AnalyzerPlugins$$anon$10.default(AnalyzerPlugins.scala:457)
at scala.tools.nsc.typechecker.AnalyzerPlugins$$anon$10.default(AnalyzerPlugins.scala:454)
at scala.tools.nsc.typechecker.AnalyzerPlugins.invoke(AnalyzerPlugins.scala:411)
at scala.tools.nsc.typechecker.AnalyzerPlugins.pluginsMacroExpand(AnalyzerPlugins.scala:454)
at scala.tools.nsc.typechecker.AnalyzerPlugins.pluginsMacroExpand$(AnalyzerPlugins.scala:454)
at scala.tools.nsc.Global$$anon$4.pluginsMacroExpand(Global.scala:480)
at scala.tools.nsc.typechecker.Macros.macroExpand(Macros.scala:773)
at scala.tools.nsc.typechecker.Macros.macroExpand$(Macros.scala:766)
at scala.tools.nsc.Global$$anon$4.macroExpand(Global.scala:480)
at scala.tools.nsc.typechecker.Typers$Typer.vanillaAdapt$1(Typers.scala:1164)
at scala.tools.nsc.typechecker.Typers$Typer.adapt(Typers.scala:1227)
at scala.tools.nsc.typechecker.Typers$Typer.adapt(Typers.scala:1200)
at scala.tools.nsc.typechecker.Typers$Typer.adapt(Typers.scala:1220)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5747)
at scala.tools.nsc.typechecker.Typers$Typer.typedValDefImpl(Typers.scala:5949)
at scala.tools.nsc.typechecker.Typers$Typer.typedValDef(Typers.scala:2056)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5651)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5733)
at scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:5797)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$10(Typers.scala:3357)
at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3357)
at scala.tools.nsc.typechecker.Typers$Typer.typedTemplate(Typers.scala:2017)
at scala.tools.nsc.typechecker.Typers$Typer.typedModuleDef(Typers.scala:1883)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5654)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5733)
at scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:5797)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$10(Typers.scala:3357)
at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3357)
at scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5363)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5656)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5733)
at scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.apply(Analyzer.scala:115)
at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:451)
at scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.run(Analyzer.scala:104)
at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1501)
at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1485)
at scala.tools.nsc.Global$Run.compileSources(Global.scala:1478)
at scala.tools.nsc.Global$Run.compile(Global.scala:1604)
at scala.tools.nsc.Driver.doCompile(Driver.scala:47)
at scala.tools.nsc.MainClass.doCompile(Main.scala:32)
at scala.tools.nsc.Driver.process(Driver.scala:67)
at io.bazel.rulesscala.scalac.ScalacWorker.compileScalaSources(ScalacWorker.java:226)
at io.bazel.rulesscala.scalac.ScalacWorker.work(ScalacWorker.java:70)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:92)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:46)
at io.bazel.rulesscala.scalac.ScalacWorker.main(ScalacWorker.java:37)
Caused by: java.lang.ClassNotFoundException: io.bullet.borer.deriver.Deriver
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 75 more
---8<---8<--- End of log snippet, 2207 chars omitted ---8<---8<---
Target //test-dir:some_lib failed to build
How can I resolve this error? The io.bullet.borer.deriver.Deriver is in dependencies and classpath but JVM doesn't see it. Cannot log classpath of the sandbox, because bazel cannot get WorkerResponce. Please help me 😅.
I have found the problem. Default scala_toolchain has direct mode. So it sees only dependencies, that are defined in the deps filed of scala_library or scala_macro_library. So there are two options to solve this problem:
Add all needed direct dependencies to the deps array.
Or define own scala_toolchain - docs - example
So for the current example, we need to define all direct dependencies. By the way, they are already downloaded when you do maven_install. Now we need only reference them:
For borer additional dependencies will be:
#maven//:io_bullet_borer_deriver_2_12
For scala_logging we need to add:
#maven//:org_slf4j_slf4j_api
And for ujson we need:
#maven//:com_lihaoyi_geny_2_12
#maven//:com_lihaoyi_upickle_core_2_12
All fixes for the Github example repository are available in the repository under fix branches.
Borer fix
Spark fix
I am trying to build a simple java project with Bazel, using rules_jvm_external. Some of the dependencies are kept in a private maven repository.
My WORKSPACE looks like this:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
RULES_JVM_EXTERNAL_TAG = "3.2"
RULES_JVM_EXTERNAL_SHA = "82262ff4223c5fda6fb7ff8bd63db8131b51b413d26eb49e3131037e79e324af"
http_archive(
name = "rules_jvm_external",
strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
sha256 = RULES_JVM_EXTERNAL_SHA,
url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)
load("#rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
name = "maven_deps",
artifacts = [
"org.projectlombok:lombok:1.18.12",
"mygroup:myartifact:version"
],
repositories = [
"https://repo1.maven.org/maven2",
"https://username:password#my.repo.io/artifactory/repo",
],
)
My BUILD looks like this:
java_library(
name = "mylib",
srcs = glob([
"proj/src/main/java/**/*.java"
]),
deps = [
"#maven_deps//:org_projectlombok_lombok",
"#maven_deps//:mygroup_myartifact",
],
)
When i run bazel build //:mylib the fetching of mygroup:myartifact:version from the private maven repository fails with http error code 403. I hardcoded the username and password for simplicity. The username used is an email so i encoded it, e.g.: me%40gmail.com.
I am using bazel version 3.1.0.
Passing the username and password through env vars produced the same error.
Fetching the same jar using curl works great:
curl -O 'https://me%40gmail:PASSWORD#my.repo.io/artifactory/repo/mygroup/myartifcat-version.jar'
Can anyone see what the problem is?
Thank you in advance!
If I recall correctly, Bazel's maven_install from rules_jvm_external relies on Coursera Coursier* for fetching dependencies. Where I work at, we rely on a property file containing the credentials at the correct location for your OS.
Try setting this:
simple.username=<username>
simple.password=<password>
simple.host=my.repo.io
In either /.config/coursier/credentials.properties (if you are on Linux) or ~/Library/Preferences/Coursier/credentials.properties on OS X.
(*) https://github.com/bazelbuild/rules_jvm_external/blob/master/docs/api.md mentions Coursier indeed
I currently have a very trivial JavaFX "Hello, World!" application that I am trying to build and run with Bazel. I am using the maven_install() rule to install the JavaFX dependencies in my WORKSPACE files like so:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
RULES_JVM_EXTERNAL_TAG = "3.2"
RULES_JVM_EXTERNAL_SHA = "82262ff4223c5fda6fb7ff8bd63db8131b51b413d26eb49e3131037e79e324af"
http_archive(
name = "rules_jvm_external",
strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
sha256 = RULES_JVM_EXTERNAL_SHA,
url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)
load("#rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
artifacts = [
"org.openjfx:javafx-controls:mac:11.0.1",
"org.openjfx:javafx-graphics:mac:11.0.1",
"org.openjfx:javafx-base:mac:11.0.1",
],
repositories = [
"https://repo1.maven.org/maven2",
],
)
And then I try to build a java_binary in the BUILD file like so:
java_binary(
name = "app",
srcs = glob(["src/**/*.java"]),
main_class = "com.dylanpowers.Main",
deps = [
"#maven//:org_openjfx_javafx_controls_mac",
"#maven//:org_openjfx_javafx_graphics_mac",
"#maven//:org_openjfx_javafx_base_mac"
]
)
In this case, Main.java is actually the only file in my application, as I am trying to just get the program to run. The build seems to work fine with bazel build :app, but when I try to run it with bazel run :app, I get the following error:
Error: JavaFX runtime components are missing, and are required to run this application
Can somebody please help me resolve this?
https://github.com/deepinthink-pumpkin/pumpkin-chat-jfx/blob/master/main/BUILD.bazel#L21
Create another main class as javafx application entrance.
I am trying to get data from SQL Server database table and show it as part of choice parameter as part of a Jenkins Job Build Parameters that I am trying to setup.
I am trying to figure out how to use Extensible Choice for this.
The Choice provider I used is "System Groovy Choice Parameter"
import groovy.sql.Sql
import com.microsoft.sqlserver.jdbc.SQLServerDriver
def output = []
def configuration = [
'dbInstance' : 'servername',
'dbPort' : 0000,
'dbName' : 'dbName',
'dbUser' : 'dbUser',
'dbPass' : 'dbPass'
]
def sql = Sql.newInstance("jdbc:sqlserver://${configuration.dbInstance}:${configuration.dbPort};"
+ "databaseName=" + configuration.dbName,
configuration.dbUser, configuration.dbPass,
'com.microsoft.sqlserver.jdbc.SQLServerDriver')
String sqlString = "SELECT * FROM dbTable"
sql.eachRow(sqlString){ row -> output.push(row[0])
}
return output.sort()
Below is the error I see. Which I understand I see because the jdbc driver is not present. I downloaded the driver from the link below:
https://www.microsoft.com/en-us/download/details.aspx?id=11774
I followed the instructions as to where it should be unzipped to as mentioned in the instructions.
I saw that the CLASSPATH variable is missing, so i went ahead and created the Environment variable with path: "C:\Program Files\sqljdbc_6.0\enu\sqljdbc.jar"
Error: unable to resolve class com.microsoft.sqlserver.jdbc.SQLServerDriver
How do i make sure that the script runs successfully and returns all the data to Extensible Choice. If there is anyother way to do this, I am open to suggestions.
Thank you very much
To resolve the issue I had to copy the "sqljdbc4.jar" file to the following location "C:\Java\jdk1.8.0_92\jre\lib\ext" since that is the path where the JAVA searches for the external jars. Use 4th version for the file which will have 4 in the file name as above as that is version Jenkins supports.