I'm trying to migrate my build tool from maven to bazel. I have a java/grpc with a proto file, if I build it with maven all is good, but with bazel I keep facing a problem that happens on the grpc. Maybe my bazel BUILD/WORKSPACE file is not configured properly, I'm not too sure. Any help would be appreciated!
Here is my WORKSPACE file:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
RULES_JVM_EXTERNAL_TAG = "3.0"
RULES_JVM_EXTERNAL_SHA = "62133c125bf4109dfd9d2af64830208356ce4ef8b165a6ef15bbff7460b35c3a"
http_archive(
name = "rules_jvm_external",
strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
sha256 = RULES_JVM_EXTERNAL_SHA,
url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)
http_archive(
name = "bazel_skylib",
sha256 = "97e70364e9249702246c0e9444bccdc4b847bed1eb03c5a3ece4f83dfe6abc44",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
],
)
load("#bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()
http_archive(
name = "rules_proto",
sha256 = "73ebe9d15ba42401c785f9d0aeebccd73bd80bf6b8ac78f74996d31f2c0ad7a6",
strip_prefix = "rules_proto-2c0468366367d7ed97a1f702f9cd7155ab3f73c5",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_proto/archive/2c0468366367d7ed97a1f702f9cd7155ab3f73c5.tar.gz",
"https://github.com/bazelbuild/rules_proto/archive/2c0468366367d7ed97a1f702f9cd7155ab3f73c5.tar.gz",
],
)
load("#rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()
http_archive(
name = "com_google_protobuf",
sha256 = "e11f901c62f6a35e295b7e9c49123a96ef7a47503afd40ed174860ad4c3f294f",
strip_prefix = "protobuf-3.10.0",
url = "https://github.com/protocolbuffers/protobuf/releases/download/v3.10.0/protobuf-all-3.10.0.tar.gz",
)
http_archive(
name = "googleapi",
sha256 = "51849d3ef693c88eb7692875eb444ef7131502e3fa200f25fc0a37b1e7e55ab5",
strip_prefix = "googleapis-a111a53c0c6722afcd793b64724ceef7862db5b9",
url = "https://github.com/googleapis/googleapis/archive/a111a53c0c6722afcd793b64724ceef7862db5b9.zip",
)
load("#com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()
load("#rules_jvm_external//:specs.bzl", "maven")
load("#rules_jvm_external//:defs.bzl", "artifact", "maven_install")
maven_install(
artifacts = [
"com.google.cloud:google-cloud-pubsub:1.92.0",
"com.google.cloud:google-cloud-storage:1.98.0",
"io.grpc:grpc-netty-shaded:1.23.0",
"io.grpc:grpc-protobuf:1.23.0",
"io.grpc:grpc-stub:1.23.0",
"mysql:mysql-connector-java:8.0.17",
"com.google.cloud.sql:mysql-socket-factory-connector-j-8:1.0.15",
"com.zaxxer:HikariCP:3.3.1",
"com.google.inject:guice:4.2.2",
"jaxen:jaxen:1.2.0",
"org.dom4j:dom4j:2.1.1",
"org.slf4j:slf4j-simple:2.0.0-alpha1",
"org.slf4j:slf4j-api:2.0.0-alpha1",
"log4j:log4j:1.2.17",
"com.google.code.gson:gson:2.8.6",
"junit:junit:4.13-beta-3",
"com.google.protobuf:protobuf-java:3.0.0",
"com.google.protobuf:protobuf-java-util:3.0.0",
],
override_targets = {
"com.google.protobuf:protobuf-java": "#com_google_protobuf//:protobuf_java",
},
maven_install_json = "//:maven_install.json",
generate_compat_repositories = True,
repositories = [
"https://jcenter.bintray.com",
],
)
load("#maven//:defs.bzl", "pinned_maven_install")
pinned_maven_install()
load("#maven//:compat.bzl", "compat_repositories")
compat_repositories()
maven_install(
artifacts = [
maven.artifact(
group = "com.google.api.grpc",
artifact = "proto-google-common-protos",
version = "1.16.0",
exclusions = [
maven.exclusion(
group = "com.google.protobuf",
artifact = "protobuf-java"
)
]
)
],
maven_install_json = "//:maven_install.json",
generate_compat_repositories = True,
repositories = [
"https://maven.google.com",
"https://repo1.maven.org/maven2",
],
)
load("#maven//:defs.bzl", "pinned_maven_install")
pinned_maven_install()
load("#maven//:compat.bzl", "compat_repositories")
compat_repositories()
Here is my BUILD file:
package(default_visibility = ["//visibility:public"])
load("#rules_proto//proto:defs.bzl", "proto_library")
proto_library(
name = "account",
srcs = glob(["src/main/proto/account.proto"]),
deps = ["#googleapi//google/api:annotations_proto"],
)
java_proto_library(
name = "account_proto_java",
deps = [":account"],
)
java_library(
name = "AccountService",
srcs = glob(["src/main/java/**/*.java"]),
resources = glob(["src/main/resources/**"]),
deps = [
"#maven//:com_google_cloud_google_cloud_pubsub",
"#maven//:com_google_cloud_google_cloud_storage",
"#maven//:io_grpc_grpc_netty_shaded",
"#maven//:io_grpc_grpc_protobuf",
"#maven//:io_grpc_grpc_stub",
"#maven//:mysql_mysql_connector_java",
"#maven//:com_google_cloud_sql_mysql_socket_factory_connector_j_8",
"#maven//:com_zaxxer_HikariCP",
"#maven//:com_google_inject_guice",
"#maven//:jaxen_jaxen",
"#maven//:org_dom4j_dom4j",
"#maven//:org_slf4j_slf4j_simple",
"#maven//:org_slf4j_slf4j_api",
"#maven//:log4j_log4j",
"#maven//:com_google_code_gson_gson",
"#maven//:junit_junit",
"#maven//:com_google_guava_guava",
"#maven//:io_grpc_grpc_api",
"#maven//:com_google_protobuf_protobuf_java",
"#maven//:com_google_protobuf_protobuf_java_util",
":account_proto_java",
],
)
I finally run that command: docker run --rm -v /${PWD}:${PWD} -w /${PWD} l.gcr.io/google/bazel:latest build ///...
Some issues that occur:
I always seem to get an error when it's downloading the resources. Here is an example: WARNING: Download from https://maven.google.com/org/checkerframework/checker-compat-qual/2.5.5/checker-compat-qual-2.5.5.jar failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
For some reason, it doesn't stop there when the 404 appears, towards the end when it comes to build the proto section. I get this error:
ERROR: /c/Users/<username>/projects/Ecommerce-Account-Microservice/BUILD:16:1: Building libAccountService-class.jar (198 source files) failed (Exit 1)
src/main/java/com/app/account/GRPCClient.java:7: error: cannot find symbol
import com.app.account.services.AccountServiceGrpc;
^
symbol: class AccountServiceGrpc
location: package com.app.account.services
src/main/java/com/app/account/GRPCClient.java:20: error: package AccountServiceGrpc does not exist
private final AccountServiceGrpc.AccountServiceBlockingStub blockingStub;
^
src/main/java/com/app/account/server/services/AccountService.java:24: error: cannot find symbol
import com.app.account.services.AccountServiceGrpc;
^
symbol: class AccountServiceGrpc
location: package com.app.account.services
src/main/java/com/app/account/server/services/AccountService.java:39: error: package AccountServiceGrpc does not exist
public class AccountService extends AccountServiceGrpc.AccountServiceImplBase {
^
src/main/java/com/app/account/GRPCClient.java:28: error: cannot find symbol
blockingStub = AccountServiceGrpc.newBlockingStub(channel);
^
symbol: variable AccountServiceGrpc
location: class GRPCClient
src/main/java/com/app/account/server/GrpcServer.java:28: error: no suitable method found for addService(AccountService)
.addService(AccountService.getInstance())
^
method ServerBuilder.addService(ServerServiceDefinition) is not applicable
(argument mismatch; AccountService cannot be converted to ServerServiceDefinition)
method ServerBuilder.addService(BindableService) is not applicable
(argument mismatch; AccountService cannot be converted to BindableService)
where CAP#1 is a fresh type-variable:
CAP#1 extends ServerBuilder<CAP#1> from capture of ?
src/main/java/com/app/account/server/services/AccountService.java:57: error: method does not override or implement a method from a supertype
#Override
^
src/main/java/com/app/account/server/services/AccountService.java:65: error: method does not override or implement a method from a supertype
#Override
^
src/main/java/com/app/account/server/services/AccountService.java:223: error: method does not override or implement a method from a supertype
#Override
^
src/main/java/com/app/account/server/services/AccountService.java:273: error: method does not override or implement a method from a supertype
#Override
^
INFO: Elapsed time: 373.098s, Critical Path: 44.04s
INFO: 358 processes: 353 processwrapper-sandbox, 5 worker.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
I'm not too sure what is going on or what I'm doing wrong at that point, that's why I'm showing all the code.
Thanks a lot!
Mark
If you want to use multiple maven_install declarations, you need unique names for each of them. See documentation here.
I always seem to get an error when it's downloading the resources. Here is an example: WARNING: Download from https://maven.google.com/org/checkerframework/checker-compat-qual/2.5.5/checker-compat-qual-2.5.5.jar failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
It looks like you don't actually use maven.google.com for any of the artifacts. Remove that from your maven_install.repositories lists and re-run artifact pinning for them.
For the missing GRPC classes, it looks like you're missing a java_grpc_library between java_library and java_proto_library. Example.
I want to be able to write a vim snippet that automatically turns into the required package.
E.g. expanding pkg while inside of .../com/theonlygust/project/Main.java would become
package com.theonlygusti.project;
I think the ways to do this are to either: read up the directory tree until seeing a TLD directory name (com, io, net, etc.) and then use the encountered directory names to build the package string, or to look up the directory tree for the pom.xml and find the package from there.
I learned about python interpolation.
I'm now trying this:
snippet pkg "Create package" b
package `!p
import os
from xml.etree import ElementTree
def get_package_name(pom_file_path):
namespaces = {'xmlns' : 'http://maven.apache.org/POM/4.0.0'}
tree = ElementTree.parse(pom_file_path)
root = tree.getroot()
groupId = root.find(".//xmlns:groupId", namespaces=namespaces)
artifactId = root.find(".//xmlns:artifactId", namespaces=namespaces)
return groupId.text + '.' + artifactId.text
def find_nearest_pom():
absolute_path = os.path.abspath(os.path.dirname('__file__')).split("/")
pom_dir_index = -1
# Find index of 'base_dir_name' element
while not os.path.isfile('/'.join(absolute_path[:pom_dir_index]) + '/pom.xml'):
pom_dir_index -= 1
return '/'.join(absolute_path[:pom_dir_index]) + '/pom.xml'
snip.rv = get_package_name(find_nearest_pom())`;
endsnippet
But I get the error
Name __file__ does not exist
And os.getcwd() doesn't work because that returns the directory from which vim was opened, not the directory that contains the current buffer.
I had a look at the snip object because I know it provides snip.fn to get the filename, but I couldn't find out if it provides the current file's directory.
Nevermind, finally learned that UltiSnips sets a global variable "path"
UltiSnips stores Java snippets in java.snippets, which on my machine is ~/.vim/bundle/vim-snippets/UltiSnips/java.snippets (I am usinghonza/vim-snippets` as well).
Snippets for Java are implemented using Python, so I have implemented the snippet below using Python as well (you can do it using multiple languages in UltiSnips).
There is already a snippet for package, which adds simply "package" word followed with a placeholder:
snippet pa "package" b
package $0
endsnippet
Let's create a pad snippet that will automatically insert package name, based on the directory chain, instead of the $0 placeholder:
snippet pad "package" b
package `!p
def get_package_string(base_dir_name):
import os
# Get absolute path of the package (without filename)
absolute_path = os.getcwd().split("/")
src_dir_index = 0
# Find index of 'base_dir_name' element
while absolute_path[src_dir_index] != base_dir_name:
src_dir_index+=1
# Create a 'package ' string, joining with dots
package_string = ".".join(absolute_path[src_dir_index+1:])
return package_string
# snip.rv is UltiSnips' return value we want to paste between ``
snip.rv = get_package_string("java")`
endsnippet
Note that this solution is based on the fact that in many Java projects, there is an src directory with main/java and test/java directories in it and you are editing one of the files in java directory (e.g. for src/main/com/google/common it will return com.google.common). You may need to modify this to be more flexible.
You can find more information about creating snippets in screencasts linked in its README.
I use a combination of the file path and the groupId and the artifactId from the nearest pom.xml (upwards)
global !p
import os
from xml.etree import ElementTree
def get_package_name(pom_file_path):
namespaces = {'xmlns' : 'http://maven.apache.org/POM/4.0.0'}
tree = ElementTree.parse(pom_file_path)
root = tree.getroot()
groupId = root.find(".//xmlns:groupId", namespaces=namespaces)
artifactId = root.find(".//xmlns:artifactId", namespaces=namespaces)
return groupId.text + '.' + artifactId.text
def find_nearest_pom():
current_file_dir = '/'.join((os.getcwd() + ('/' if os.getcwd()[-1] != '/' else '') + path).split('/')[:-1])
absolute_path = current_file_dir.split("/")
pom_dir_index = -1
if os.path.isfile('/'.join(absolute_path) + '/pom.xml'):
return '/'.join(absolute_path) + '/pom.xml'
# Find index of 'base_dir_name' element
while not os.path.isfile('/'.join(absolute_path[:pom_dir_index]) + '/pom.xml'):
pom_dir_index -= 1
return '/'.join(absolute_path[:pom_dir_index]) + '/pom.xml'
def get_file_package():
current_file_location = '.'.join((os.getcwd() + ('/' if os.getcwd()[-1] != '/' else '') + path).split('/')[:-1])
package = get_package_name(find_nearest_pom())
return package + current_file_location.split(package)[1]
endglobal
snippet pkg "package" b
package `!p snip.rv = get_file_package()`;
endsnippet
I have 3 .proto files under same folder, I plan to add LowDataBalanceRequest
and MobileRequest into `EventRequest
Command to generate Java sources:
protoc --java_out=/home/haifzhan/myproject/src/main/java/com/example/util star_event.proto ldb_event.proto generic_event.proto
star_event.proto and ldb_event.proto can generate Java source properly with no error, but generic_event.proto has errors as(my point is to show Errors, so I attache images other than source code):
One error says Descriptor instance might not be initialized
Another one is complaining bracket is expected.
Here's my generic_event.proto
import "star_event.proto";
import "ldb_event.proto";
message EventRequest {
optional LowDataBalanceRequest ldbRequest = 1;
optional MobileRequest mobileRequest = 2;
}
I am using version 2.6.1
Can anyone help?
I have figured it out.
A related issue on GithubJava: Invalid code generated when referencing a file with no package statement
The error is gone after I add option java_package in all .proto files:
option java_package = "com.example.util";
Here's my full .proto files:
option java_package = "com.example.util";
ldb_event.proto
option java_package = "com.example.util";
enum RequestTrigger {
LDB = 1;
OTHER = 2;
}
message LowDataBalanceRequest {
required string probe_id = 1;
required int64 unix_time = 2;
}
generic_event.proto
import "star_event.proto";
import "ldb_event.proto";
option java_package = "com.example.util";
message EventRequest {
optional LowDataBalanceRequest ldbRequest = 1;
optional MobileRequest mobileRequest = 2;
}
I have a child java project which has groovy files added in classpath using eclipse. Parent java project triggers some functionality in child which uses Groovy library to run the scripts. So import works fine in eclipse environment with opened child project but if I run it from command line or if I close child project then I get groovy compilation error at import statement. How can I resolve this ? I want to avoid using evaluate() method.
Following is my master groovy:
package strides_business_script
abstract class Business_Script extends Script {
//some stuff
}
Following is the another groovy:
import static strides_business_script.StridesBusiness_Script.*;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
String Key = Part_Product_PartDetails
boolean containsData = checkIncomingMessage(Key)
if(containsData) {
def edgeKeyList = [PPR]
JSONArray partDetails = appendEdgeValueToMsg(edgeKeyList,Key,vertex,messageIterator);
//deleteMessages(Key);
JSONObject jsonObject = constructInfoWithPropertyJSON("NAME,PRODUCTTYPE,FGTYPE,UOM,ITEMCLASSIFICATIONBYMARKET");
jsonObject.put("PARTS",partDetails);
send(Product_AggPO_ProductDetails,convertJSONToString(jsonObject));
}
Edit:
My master script Business_Script.groovy resides in scripts/strides_business_script/ folder. All other scripts are in scripts/StridesComputationScripts/ folder and they import the Business_Script.groovy.
I run the application with remote debugging enabled like this:
java -cp "./lib/*:./scripts/strides_business_script/Business_Script.groovy" -Xdebug -Xrunjdwp:transport=dt_socket,address=6969,server=y -Dhibernate.cfg.xml.path=./conf/hibernate.cfg.xml -Dlog4j.configuration=file:./conf/log4j.properties com.biglabs.dataExtractor.dataDump.DataDumpDriver 7
and here I am trying to parse all computation scripts.
for (String scriptName : files) {
Script script = groovyShell.parse(new File(
SCRIPT_PLACED_AT + Constants.SLASH
+ SCRIPT_FILE_FOLDER + Constants.SLASH
+ scriptName));
scriptMping.put(scriptName, script);
}
It throws following exception while parsing using groovy shell:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/home/manoj/strides/release/strides/scripts/StridesComputationScripts/PRODUCT-script.groovy: 2: unable to resolve class strides_business_script.StridesBusiness_Script
# line 2, column 1.
import static strides_business_script.Business_Script.*;
^
/home/manoj/strides/release/strides/scripts/StridesComputationScripts/PRODUCT-script.groovy: 2: unable to resolve class strides_business_script.StridesBusiness_Script
# line 2, column 1.
import static strides_business_script.Business_Script.*;
^
2 errors
Fixed it by adding script path in comiler configuration:
CompilerConfiguration compilerConfiguration = new CompilerConfiguration();
String path = SCRIPT_PLACED_AT;
if(!SCRIPT_PLACED_AT.endsWith("/")){
path = path+ "/";
}
compilerConfiguration.setClasspath(path);
GroovyShell groovyShell = new GroovyShell(
compilerConfiguration);
for (String scriptName : files) {
Script script = groovyShell.parse(new File(
SCRIPT_PLACED_AT + Constants.SLASH
+ SCRIPT_FILE_FOLDER + Constants.SLASH
+ scriptName));
scriptMping.put(scriptName, script);
}
I have a project in Java. This project has a class com.xyz.api.base.models.mongo.Member.
I want to import this Java project to a Scala project to use Member class.
However, I got this error (the library is already downloaded to Scala dependencies):
java.lang.RuntimeException: java.lang.ClassNotFoundException: models.mongo.Member
The strange thing is that there is not compilation error. The error above only happens at runtime. Furthermore, the error message does not mention com.xyz.api.base as the base package of models.mongo.Member.
My code:
import com.redmart.api.base.models.mongo.Member
import com.redmart.api.base.utils.RedisCacheImpl
import redis.RedisClient
object Redis extends App {
implicit val akkaSystem = akka.actor.ActorSystem()
val host: String = "127.0.0.1"
val port: Int = 6379
val db: Int = 0
val timeout: Long = 10000L
val key = "a2IxSE5kdW9HRHZUe"
var redisCacheImpl: RedisCacheImpl = _
try {
RedisCacheImpl.configRedis(host, port, db, timeout)
redisCacheImpl = RedisCacheImpl.getInstance()
val obj = redisCacheImpl.get(key)
val member = obj.asInstanceOf[Member]
println(s"member id ${member.getMemberId}")
}
Thank you for your help.
In this case spring-boot's version 1.2.3.RELEASE use mongo-java-driver 2.12.5. for more details go through this documentation :Link