I want to use a specific Azul Zulu JDK for my Java builds. Therefore I have stored it in my repository locally e.g. under tools/zulu19.30.11-ca-jdk19.0.1-macosx_x64. Now I want to configure a java toolchain target such that I can pass it via --java_toolchain=//tools:my_custom_java_toolchain. I don't want to depend on some remote repositories.
What are the required steps to achieve this?
I have found this repository: https://github.com/salesforce/bazel-jdt-java-toolchain/blob/main/jdt/defs.bzl which defines a target of type default_java_toolchain but I can not derive something useful for my use case. I don't know e.g. what the field header_compiler means. My naive assumption is that I just have to pass some paths to the required tools (such as bin/javac) for java compilation.
My current approach uses the rules java_toolchain and java_runtime. My BUILD file looks like this:
java_runtime(
name = "zulu19.30.11-ca-jdk19.0.1-macosx_x64",
srcs = glob(["zulu19.30.11-ca-jdk19.0.1-macosx_x64/**"]),
java_home = "zulu19.30.11-ca-jdk19.0.1-macosx_x64",
)
java_toolchain(
name = "zulu-19",
source_version = "19",
target_version = "19",
java_runtime = ":zulu19.30.11-ca-jdk19.0.1-macosx_x64",
javabuilder = "",
ijar = "",
singlejar = "",
genclass = "",
)
I am trying to execute the command: bazel build --extra_toolchains="//tools:zulu-19" //:ProjectRunner and it complains about the missing mandatory attributes javabuilder, ijar, singlejar and genclass but I have no idea which are the correct paths or values.
I'm just wondering how bazel knows how to compile the java code with these few information. Why I don't have to specify javac for example?
Basically I came up with the following solution which seems to solve my problem and is inspired by the rule default_java_toolchain. I used the existing targets for javabuilder, ijar, singlejar and genclass which are defined in #bazel_tools//tools/jdk. Note that I had to use BASE_JDK9_JVM_OPTS for a successful build.
load("#bazel_tools//tools/jdk:default_java_toolchain.bzl", "BASE_JDK9_JVM_OPTS")
java_runtime(
name = "zulu19.30.11-ca-jdk19.0.1-macosx_x64",
srcs = glob(["zulu19.30.11-ca-jdk19.0.1-macosx_x64/**"]),
java_home = "zulu19.30.11-ca-jdk19.0.1-macosx_x64",
)
config_setting(
name = "zulu-19-runtime_version_setting",
values = {"java_runtime_version": "19"},
visibility = ["//visibility:private"],
)
toolchain(
name = "zulu-19_runtime_toolchain_definition",
target_settings = ["zulu-19-runtime_version_setting"],
toolchain_type = "#bazel_tools//tools/jdk:runtime_toolchain_type",
toolchain = "zulu19.30.11-ca-jdk19.0.1-macosx_x64",
)
java_toolchain(
name = "zulu-19",
source_version = "19",
target_version = "19",
java_runtime = ":zulu19.30.11-ca-jdk19.0.1-macosx_x64",
jvm_opts = BASE_JDK9_JVM_OPTS,
javabuilder = ["#bazel_tools//tools/jdk:javabuilder"],
ijar = ["#bazel_tools//tools/jdk:ijar"],
singlejar = ["#bazel_tools//tools/jdk:singlejar"],
genclass = ["#bazel_tools//tools/jdk:genclass"],
)
config_setting(
name = "zulu-19_version_setting",
values = {"java_language_version": "19"},
visibility = ["//visibility:private"],
)
toolchain(
name = "zulu-19_toolchain_definition",
toolchain_type = "#bazel_tools//tools/jdk:toolchain_type",
target_settings = ["zulu-19_version_setting"],
toolchain = "zulu-19",
)
The following command now successfully runs without any errors. System.out.println(Runtime.version()); also prints the correct version.
bazel run --extra_toolchains="//tools:zulu-19_toolchain_definition,//tools:zulu-19_runtime_toolchain_definition" --java_language_version="19" --java_runtime_version="19" //:ProjectRunner
But I still have a couple of questions, e.g. why javabuilder, ijar, singlejar and genclass are needed and what is the purpose of those targets? How does the toolchain target know about my javac binary for compiling the java code? Why do I have to use BASE_JDK9_JVM_OPTS for a successful build and can I omit some settings?
Related
My CustomTest.java has this import:
com.google.protobuf.Timestamp
I'm using java_test_suite to run tests in my BUILD file like so:
java_test_suite(
name = "all-tests",
srcs = glob(["src/test/java/**/*.java"]),
runner = "junit5",
test_suffixes = ["Test.java"],
runtime_deps = JUNIT5_DEPS,
deps = [
":mylib",
"#com_google_protobuf//:timestamp_proto",
artifact("org.junit.jupiter:junit-jupiter-api"),
artifact("org.junit.jupiter:junit-jupiter-params"),
] + deps,
)
However when I run tests on it using:
bazel test //:all-tests
I'm getting this error:
src/test/java/com/x/CustomTest.java:75: error: [strict] Using type com.google.protobuf.Timestamp from an indirect dependency (TOOL_INFO: "#com_google_protobuf//:timestamp_proto wrapped in java_proto_library"). See command below **
private static Timestamp timestampFromMilli(long milli) {
^
** Please add the following dependencies:
#com_google_protobuf//:timestamp_proto to //:src/test/java/com/x/CustomTest
** You can use the following buildozer command:
buildozer 'add deps #com_google_protobuf//:timestamp_proto' //:src/test/java/com/x/CustomTest
What do I need to do exactly? I tried using the buildozer command but all I got was:
rule 'src/test/java/com/x/CustomTest' not found
Where do I need to add this #com_google_protobuf//:timestamp_proto?
Looking at protobuf's build files, it looks like timestamp_proto is a plain proto_library:
https://github.com/protocolbuffers/protobuf/blob/main/BUILD.bazel#L70-L74
https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/BUILD.bazel#L64-L68
and so per the advice here:
https://github.com/protocolbuffers/protobuf/blob/main/BUILD.bazel#L19-L25
you might just need to use java_proto_library to make the java version of the proto:
java_proto_library(
name = "timestamp_java_proto",
deps = ["#com_google_protobuf//:timestamp_proto"],
)
and then use that in the deps of your java_test_suite instead of the timestamp_proto.
Just a guess, but the error message is not very helpful maybe because there happens to be a Timestamp java class in the deps of the plain proto library, and Strict deps is finding that one in the test's indirect dependencies. Might be worth filing a bug about it on https://github.com/bazelbuild/bazel/issues
I have a script that works perfectly when I'm not using Renv. However, when running it in a project with Renv enabled, the last command line returns the following message:
> r5r_core <- setup_r5(data_path = data_path, verbose = FALSE)
Error in rJava::.jinit() : Unable to create a Java class loader.
Just run the code below inside a renv project to have a reproducible example:
options(java.parameters = "-Xmx2G")
library(r5r)
library(rJava)
data_path <- system.file("extdata/poa", package = "r5r")
list.files(data_path)
poi <- fread(file.path(data_path, "poa_points_of_interest.csv"))
head(poi)
points <- fread(file.path(data_path, "poa_hexgrid.csv"))
points <- points[ c(sample(1:nrow(points), 10, replace=TRUE)), ]
head(points)
# Indicate the path where OSM and GTFS data are stored
r5r_core <- setup_r5(data_path = data_path, verbose = FALSE)
My Java version is compatible with the one used in this package, but it looks like R is having a hard time communicating with Java in Renv. Could anyone tell me?
I've switched (or, am in the process of switching) to using bazel, though I'm doing so on Windows.
I'm interested in calling into my Go code from Java, so I started with this tutorial.
I was able to make that work using the same code as on their Github example and everything works fine. I tried adapting that to my bazel build. If I take the awesome.so file generated by go build -o awesome.so -buildmode=c-shared awesome.go and include it as a resource to my java_library, I can make everything work.
Relevant files shown below.
Ideally, however, I'd like to have everything generated through bazel, but despite all my attempts thus far my go_binary rule always outputs awesome.a (and awesome.x). If I switch to using //go:awesome as the resource from java:client_lib I am able to successfully see the awesome.a output as a resource, which suggests that getting my go_binary to output awesome.so is the last piece of the puzzle, but the correct combination of flags has thus far eluded me.
Basically I just want to make my go_binary rule have the same behavior as running go build -o awesome.so --buildmode=c-shared awesome.go.
In theory I'm ok if I need another rule to bridge the gap, but since I'm on windows and bash has been hit or miss thus far, using genrule as the intermediate doesn't currently look promising.
Please advise, and thanks!
WORKSPACE
...
# bazelbuild/rules_go for golang support.
http_archive(
name = "io_bazel_rules_go",
sha256 = "b725e6497741d7fc2d55fcc29a276627d10e43fa5d0bb692692890ae30d98d00",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.24.3/rules_go-v0.24.3.tar.gz",
"https://github.com/bazelbuild/rules_go/releases/download/v0.24.3/rules_go-v0.24.3.tar.gz",
],
)
load("#io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
go_rules_dependencies()
go_register_toolchains()
...
go/awesome.go is copied from the article.
go/BUILD
load("#io_bazel_rules_go//go:def.bzl", "go_binary", "go_library")
package(default_visibility = ["//visibility:public"])
go_binary(
name = "awesome",
srcs = glob(["*.go"]),
cgo = True,
copts = [
"-fPIC", # I tried adding this after some other reading about .a->.so
],
gc_linkopts = [
"-shared", # I think this is equivalent to the linkmode=c-shared below, but... <shrug>
],
linkmode = "c-shared",
static = "off",
)
# This one uses the pre-built awesome.so, and this works.
filegroup(
name = "prebuilt_awesome_resource",
srcs = ["awesome.so"],
)
java/Client.java is copied from the github repo linked in the article (with slight tweaks to the library's location).
java/BUILD
package(default_visibility = ["//visibility:public"])
java_import(
name = "jna",
jars = ["jna.jar"],
)
java_library(
name = "client_lib",
srcs = glob(["*.java"]),
resources = [
# "//go:awesome", # I'd rather use this one.
"//go:prebuilt_awesome_resource",
],
deps = [
":jna",
],
)
java_binary(
name = "client",
main_class = "Client",
runtime_deps = [
":client_lib",
],
)
and also, since it was important getting go stuff to run:
%programdata%/basel.bazelrc
startup --output_user_root="C:/_bazel_out"
build --compiler=mingw-gcc
Well, I guess I need to go sit in the shame cube for a while.
Of all the options I was looking at for the compiler I missed checking other attributes on go_binary. Specifically, the obvious one, out. The one that actually corresponds to the -o flag on go build
I added out = "awesome.so" to my go_binary rule and, sure enough, everything works.
Well that's a few hours wasted. Thanks Jay for trying to help and sorry for asking a dumb question.
This might not answer your question exactly, but I can give an example of calling a Go shared library from a C program on macOS. Hopefully that gets you most of the way there.
For the go_binary, you just need linkmode = "c-shared". You'll also need cgo = True for each package that either contains cgo code or has exported definitions. You don't need -shared, -fPIC, or static = "off".
Exported definitions should be marked with an //export comment.
There's an implicitly declared target with the suffix .c_hdrs that builds a header file for the Go library. It's :go_hello.c_hdrs in the example below. The actual header file name is go_hello.h, matching the target name.
You need to wrap the generated files with a cc_import rule to make it usable as a C/C++ dependency. #2433 is an open issue to streamline that process, but it's only recently possible in Bazel.
Anything that can consume a cc_library can consume the cc_import target the same way. So you should be able to call Go functions via JNI, though I've never tried that out.
BUILD.bazel
load("#io_bazel_rules_go//go:def.bzl", "go_binary")
go_binary(
name = "go_hello",
srcs = ["hello.go"],
cgo = True,
linkmode = "c-shared",
)
cc_import(
name = "c_hello",
hdrs = [":go_hello.c_hdrs"],
shared_library = ":go_hello",
)
cc_binary(
name = "use",
srcs = ["use.c"],
deps = [":c_hello"],
)
hello.go
package main
import "fmt"
import "C"
//export SayHello
func SayHello() {
fmt.Println("hello")
}
func main() {}
use.c
#include "go_hello.h"
int main() {
SayHello();
return 0;
}
I am trying to build a simple java project with Bazel, using rules_jvm_external. Some of the dependencies are kept in a private maven repository.
My WORKSPACE looks like this:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
RULES_JVM_EXTERNAL_TAG = "3.2"
RULES_JVM_EXTERNAL_SHA = "82262ff4223c5fda6fb7ff8bd63db8131b51b413d26eb49e3131037e79e324af"
http_archive(
name = "rules_jvm_external",
strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
sha256 = RULES_JVM_EXTERNAL_SHA,
url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)
load("#rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
name = "maven_deps",
artifacts = [
"org.projectlombok:lombok:1.18.12",
"mygroup:myartifact:version"
],
repositories = [
"https://repo1.maven.org/maven2",
"https://username:password#my.repo.io/artifactory/repo",
],
)
My BUILD looks like this:
java_library(
name = "mylib",
srcs = glob([
"proj/src/main/java/**/*.java"
]),
deps = [
"#maven_deps//:org_projectlombok_lombok",
"#maven_deps//:mygroup_myartifact",
],
)
When i run bazel build //:mylib the fetching of mygroup:myartifact:version from the private maven repository fails with http error code 403. I hardcoded the username and password for simplicity. The username used is an email so i encoded it, e.g.: me%40gmail.com.
I am using bazel version 3.1.0.
Passing the username and password through env vars produced the same error.
Fetching the same jar using curl works great:
curl -O 'https://me%40gmail:PASSWORD#my.repo.io/artifactory/repo/mygroup/myartifcat-version.jar'
Can anyone see what the problem is?
Thank you in advance!
If I recall correctly, Bazel's maven_install from rules_jvm_external relies on Coursera Coursier* for fetching dependencies. Where I work at, we rely on a property file containing the credentials at the correct location for your OS.
Try setting this:
simple.username=<username>
simple.password=<password>
simple.host=my.repo.io
In either /.config/coursier/credentials.properties (if you are on Linux) or ~/Library/Preferences/Coursier/credentials.properties on OS X.
(*) https://github.com/bazelbuild/rules_jvm_external/blob/master/docs/api.md mentions Coursier indeed
I have lots of SQLite tables that has now become hard to manage at the app side because of multiple DAO classes. I am using Bazel as my build system but I can't figure out how to use Room DB with Bazel build system.
If you use a Maven artifact resolver like rules_jvm_external, it'll look something like this.
In your WORKSPACE file, add the dependency on the Room compiler:
load("#rules_jvm_external//:specs.bzl", "maven")
maven_install(
name = "maven",
artifacts = [
"androidx.room:room-runtime:2.1.0-alpha04",
"androidx.room:room-compiler:2.1.0-alpha04",
"com.google.guava:guava:28.1-android",
maven.artifact("com.google.auto", "auto-common", "0.10", neverlink = True),
# .. other artifacts
],
repositories = [
"https://maven.google.com",
"https://jcenter.bintray.com",
],
)
In a BUILD file (e.g. <project root>/BUILD), create the java_plugin target to expose the annotation processor for Room:
java_plugin(
name = "androidx_room_room_compiler_plugin",
processor_class = "androidx.room.RoomProcessor",
deps = ["#maven//:androidx_room_room_compiler"],
neverlink = True,
)
java_library(
name = "androidx_room_room_compiler_library",
exports = [
"#maven//:androidx_room_room_compiler",
],
exported_plugins = [
":androidx_room_room_compiler_plugin"
],
)
Finally, in your app's BUILD file, depend on the Room compiler and runtime:
android_library(
name = "lib_prod",
# ...
deps = [
"#maven//:androidx_room_room_runtime",
"//:androidx_room_room_compiler_library",
],
)
I have ported an Android sample app that uses the Room and Lifecycle libraries to build with Bazel here: https://github.com/jin/BasicRxJavaSample-Bazel