We Keep Coding

Execute bazel-buildfarm in a production-ready way without "bazel run"

The readme of Bazel-buildfarm says:
In general do not execute server binaries with bazel run, since bazel does not support running multiple targets.
This sounds like bazel run is designed for local testing-purpose, not for productive servers where multiple jobs could run.
But I couldn't find a documentation showing how to run the bazel server instead. Older Dockerfiles run bazel build and then execute the generated jar-file with java. I tried this on my test machine and got the following error:
[vagrant#localhost bazel-buildfarm]$ java -Djava.util.logging.config.file=/config/logging.properties -jar bazel-bin/src/main/java/build/buildfarm/buildfarm-server.jar
no main manifest attribute, in bazel-bin/src/main/java/build/buildfarm/buildfarm-server.jar
The build before seems working:
[vagrant#localhost bazel-buildfarm]$ bazel build //src/main/java/build/buildfarm:buildfarm-server
Starting local Bazel server and connecting to it...
[...]
INFO: Analyzed target //src/main/java/build/buildfarm:buildfarm-server (94 packages loaded, 1623 targets configured).
INFO: Found 1 target...
INFO: Deleting stale sandbox base /home/vagrant/.cache/bazel/_bazel_vagrant/277990f61896f4bac21fb997fb976b70/sandbox
Target //src/main/java/build/buildfarm:buildfarm-server up-to-date:
bazel-bin/src/main/java/build/buildfarm/buildfarm-server.jar
bazel-bin/src/main/java/build/buildfarm/buildfarm-server
INFO: Elapsed time: 11.180s, Critical Path: 0.61s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
And the jar file exists:
[vagrant#localhost bazel-buildfarm]$ l bazel-bin/src/main/java/build/buildfarm/buildfarm-server
-r-xr-xr-x. 1 vagrant vagrant 17K Nov 11 11:27 bazel-bin/src/main/java/build/buildfarm/buildfarm-server
I just installed bazel using CentOS repo with all dependencies (gcc gcc-c++) and cloned the git repo of bazel-buildfarm. With
bazel run //src/main/java/build/buildfarm:buildfarm-server $(pwd)/examples/server.config.example
the server could at least start successfull:
INFO: Analyzed target //src/main/java/build/buildfarm:buildfarm-server (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //src/main/java/build/buildfarm:buildfarm-server up-to-date:
bazel-bin/src/main/java/build/buildfarm/buildfarm-server.jar
bazel-bin/src/main/java/build/buildfarm/buildfarm-server
INFO: Elapsed time: 0.325s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Nov 12, 2019 3:08:13 PM build.buildfarm.server.BuildFarmServer <init>
INFO: buildfarm-server-c6735a2a-2c1b-473e-9d3e-14fe5d01c27d initialized
What am I doing wrong? Or is there another way how to run bazel in a production-ready way?
Platform: CentOS 7.7.1908 with Bazel 1.1.0 on Vagrant

While I don't know anything specific about buildfarm, you're probably missing a double dash (--) between the target you're running with Bazel and its arguments. Your command line should probably be this:
bazel run //src/main/java/build/buildfarm:buildfarm-server -- $(pwd)/examples/server.config.example

How to enable fluentd pluggins

I am using EFK.
Could someone help here on how to enable fluentd plugins.
I have installed the following 3 plugins as part of my work.
gem install fluent-plugin-concat
gem install fluent-plugin-detect-exceptions
gem install fluent-plugin-detect-exceptions-with-error
gem install fluent-plugin-record-modifier
And I am seeing the list of gems installed here:
opsmxuser#opsmx-ind-setup:~/docker-compose-efk$ fluent-gem list
*** LOCAL GEMS ***
.
.
fluent-plugin-concat (2.4.0)
fluent-plugin-detect-exceptions (0.0.12)
fluent-plugin-detect-exceptions-with-error (0.0.3)
fluent-plugin-record-modifier (2.0.1)
fluentd (1.6.2, 0.14.25)
.
.
But these 3 plugins are not enabled. When we are using these configurations into fluent.conf file getting following error messages into fluentd container logs.
2019-08-05 12:34:55 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2019-08-05 12:34:55 +0000 [error]: config error file="/etc/fluent/fluent.conf" error_class=Fluent::ConfigError error="Unknown filter plugin 'concat'. Run 'gem search -rd fluent-plugin' to find plugins"
2019-08-02 12:46:23 +0000 [error]: config error file="/fluentd/etc/fluent.conf" error_class=Fluent::ConfigError error="Unknown output plugin 'detect_exceptions'. Run 'gem search -rd fluent-plugin' to find plugins"
2019-08-05 12:34:55 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2019-08-05 12:34:55 +0000 [error]: config error file="/fluentd/etc/fluent.conf" error_class=Fluent::ConfigError error="Unknown filter plugin 'record_modifier'. Run 'gem search -rd fluent-plugin' to find plugins"

Add to fluentd config #type copy directive.
Example below:
<match **>
#type copy
<store>
#type exec
#id load2clickhouse
command bash /usr/local/bin/insert_ch.sh {{ template "fluentdInsertTable" $ }}
format json
<buffer>
#type memory
chunk_limit_size 32m
queue_limit_length 32
flush_at_shutdown true
flush_interval 15s
flush_thread_count 4
</buffer>
</store>
<store>
#type elasticsearch
host ip
port 5044
logstash_format true
</store>
</match>

I also encountered the same problem today.
I follow the fluent github issue, build a image with plugin in it
build your own image
it works for me

sonarqube 5.0 giving Error getting generated key or setting result to parameter object message

I am trying to use SonarQube 5.0 with Hudson 3.2.1.
Sonar runner version 2.4.
Once Hudson invokes a sonar analysis, it ends with a build failure.
SonarQube is configured with Oracle 12c database using ojdbc7.jar driver.
The log output -
[workspace] $ /opt/sonar-runner-2.4/bin/sonar-runner -Dsonar.jdbc.driver=oracle.jdbc.xa.client.OracleXADataSource -Dsonar.jdbc.url=jdbc:oracle:thin:#//host:port/URL id pass -Dsonar.host.url=http://host:port id pass -Dsonar.projectBaseDir=/basedir -Dsonar.projectName=prjName -Dsonar.projectVersion=1.0 -Dsonar.projectKey=org:prj -Dsonar.sources=.
SonarQube Runner 2.4
Java 1.7.0_75 Oracle Corporation (64-bit)
Linux 2.6.18-308.el5 amd64
INFO: Runner configuration file: /opt/sonar-runner-2.4/conf/sonar-runner.properties
INFO: Project configuration file: NONE
INFO: Default locale: "en_US", source code encoding: "UTF-8" (analysis is platform dependent)
INFO: Work directory: /basedir
INFO: SonarQube Server 5.0
04:04:54.792 INFO - Load global referentials...
04:04:55.100 INFO - Load global referentials done: 311 ms
04:04:55.105 INFO - User cache: /var/lib/hudson/.sonar/cache
04:04:55.116 INFO - Install plugins
04:04:55.196 INFO - Install JDBC driver
04:04:55.205 INFO - Create JDBC datasource for jdbcURL
04:04:59.095 INFO - Initializing Hibernate
04:05:01.906 INFO - Load project referentials...
04:05:02.242 INFO - Load project referentials done: 336 ms
04:05:02.242 INFO - Load project settings
04:05:02.594 INFO - Loading technical debt model...
04:05:02.616 INFO - Loading technical debt model done: 21 ms
04:05:02.622 INFO - Apply project exclusions
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
Total time: 9.050s
Final Memory: 16M/242M
INFO: ------------------------------------------------------------------------
ERROR: Error during Sonar runner execution
ERROR: Unable to execute Sonar
ERROR: Caused by: Error getting generated key or setting result to parameter object. Cause: java.lang.NullPointerException
ERROR:
ERROR: To see the full stack trace of the errors, re-run SonarQube Runner with the -e switch.
ERROR: Re-run SonarQube Runner using the -X switch to enable full debug logging.
Finished: FAILURE
I have replaced the actual parameters like host,port, jdbcurl with placeholders.

Chances are that you are running Oracle 12c version 12.1.0.1.0 and using the JDBC drivers that came with it. The Oracle 12c JDBC driver that comes with 12.1.0.1.0 has a bug with the oracle.jdbc.driver.AutoKeyInfo.initMetaDataColumnIndexes() method, which is now fixed with the 12.2.0 patch. I am guessing that your error "getting generated key" is related to this issue. You should enable logging within SonarQube so that you can verify the root cause.
Full post is here.

jasmine_node failed while running mean-seed

Having trouble running a project generated by yeoman generator called mean-seed. I've been tinkering with it for a few days now and tried a couple things. Where I am having trouble is running the "jasmine_node" task:
Running "jasmine_node" task
>> Error: Unable to access jarfile node_modules/protractor/selenium/selenium-server-standalone-2.39.0.jar
Warning: Done, with errors. Use --force to continue.
Aborted due to warnings.
So I first tried identifying missing npm packages to install
744 npm install && bower update && bower install
755 npm install protractor
760 npm install npm install selenium-standalone
761 npm install -g protractor
765 npm install protractor-tester
767 npm install protractor-selenium-server-vagrant
768 npm install selenium-standalone#2.39.0-2.8.0-2
none of these actually created the file selenium-server-standalone-2.39.0.jar in the node_modules directory. I ran a find to check, however several other jars were install.
$ find . -name "*.jar" -print
./node_modules/grunt-jasmine-node-coverage-validation/node_modules/jasmine-node/node_modules/jasmine-reporters/ext/jline.jar
./node_modules/grunt-jasmine-node-coverage-validation/node_modules/jasmine-node/node_modules/jasmine-reporters/ext/js.jar
./node_modules/selenium-standalone/.selenium/2.39.0/server.jar
I found the selenium server download so I tried downloading and copying it over to the referenced directory which I had to create
https://code.google.com/p/selenium/downloads/detail?name=selenium-server-standalone-2.39.0.jar
I run grunt again. It goes through tasks then runs the jasmine_node task
Running "jasmine_node" task
TEST configFile: /Users/jgs/Projects/mean/app/test/../configs/config.test.json
configFile: ./app/configs/config.test.json
info - socket.io started
waiting for server to be running..
[success] connected to db at localhost:27017/test_temp
>> Feb 01, 2014 3:03:54 PM org.openqa.grid.selenium.GridLauncher main
>> INFO: Launching a standalone server
>> Setting system property webdriver.chrome.driver to node_modules/protractor/selenium/chromedriver
------------------
[ERROR]
Run `node run.js config=test`
Server not connected. Ensure you have a node server running with the `config=test` command line option so this server connects to the TEST database - the same one used here for the tests. Do NOT connect to the live database for doing tests!
------------------
[success] connected to db at localhost:27017/test_temp
all tests - 2526 ms
should test everything - 2525 ms
Failures:
1) all tests should test everything
Message:
Expected 'ERROR - check the logs above to fix the problem then try again' to be false.
Stacktrace:
Error: Expected 'ERROR - check the logs above to fix the problem then try again' to be false.
at /Users/jgs/Projects/mean/app/test/all.spec.js:6:2935
at _rejected (/Users/jgs/Projects/mean/node_modules/q/q.js:808:24)
at /Users/jgs/Projects/mean/node_modules/q/q.js:834:30
at Promise.when (/Users/jgs/Projects/mean/node_modules/q/q.js:1079:31)
at Promise.promise.promiseDispatch (/Users/jgs/Projects/mean/node_modules/q/q.js:752:41)
at /Users/jgs/Projects/mean/node_modules/q/q.js:574:44
at flush (/Users/jgs/Projects/mean/node_modules/q/q.js:108:17)
at process._tickCallback (node.js:415:13)
Finished in 2.53 seconds
1 test, 1 assertion, 1 failure, 0 skipped
Warning: Task "jasmine_node" failed. Use --force to continue.
Aborted due to warnings.
$
Any idea why jasmine task is still failing?

Run ./node_modules/protractor/bin/webdriver-manager update, see here for more info:
https://github.com/jackrabbitsgroup/generator-mean-seed/issues/5
I hadn't yet updated the documentation everywhere yet for the Protractor upgrade, sorry about that!

Here is my solution.
First you need to download both the selenium-server-standalone
https://code.google.com/p/selenium/downloads/detail?name=selenium-server-standalone-2.39.0.jar
and the chromedriver
http://chromedriver.storage.googleapis.com/index.html
copy them to
node_modules/protractor/selenium/
you will want another terminal tab or window to start the node server by runing
node run.js
then you can run the default grunt task
grunt
You will then see the output of the selenium tests in the console and chrome should open and perform the tests automatically.

Multiple 500 errors when google caliper tries to upload benchmark results to microbenchmarks.appspot.com

I'm seeing multiple HTTP 500 errors when Caliper tries to upload the JSON result files for the benchmarks I'm running. The logs are filled with exceptions similar to the one below:
Dec 19, 2013 3:29:20 PM com.google.caliper.runner.ResultsUploader logUploadFailure
SEVERE: Could not upload trial c0e4f478-7408-4050-88d3-7446e21bb7a1. Consider uploading it manually.
com.sun.jersey.api.client.UniformInterfaceException: POST https://microbenchmarks.appspot.com/data/trials?key={my-valid-api-key} returned a response status of 500 Internal Server Error
at com.sun.jersey.api.client.WebResource.voidHandle(WebResource.java:697)
at com.sun.jersey.api.client.WebResource.access$400(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:548)
at com.google.caliper.runner.ResultsUploader.processTrial(ResultsUploader.java:104)
at com.google.caliper.runner.ExperimentingCaliperRun.run(ExperimentingCaliperRun.java:231)
at com.google.caliper.runner.CaliperMain.exitlessMain(CaliperMain.java:135)
at com.google.caliper.runner.CaliperMain.main(CaliperMain.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
at java.lang.Thread.run(Thread.java:695)
However, a report does show up on the /runs page under my account, so it seems it's able to upload at least some of them. Doing a curl POST to the specified URL with any of the failed trials results in the same error.
Has anyone else encountered these errors? Is there something I'm missing in my configuration?
Caliper config.properties:
# Caliper config file
# Run with --print-config to see all of the options being applied
# INSTRUMENT CONFIG
# instrument.micro.options.warmup=10s
# instrument.micro.options.timingInterval=500ms
# instrument.micro.options.reportedIntervals=7
# instrument.micro.options.maxRuntime=10s
# VM CONFIG
vm.args=-Xmx3g -Xms3g
# See the Caliper webapp to get a key so you can associate results with your account
results.upload.options.key={my-valid-api-key}
Caliper version:
<dependency>
<groupId>com.google.caliper</groupId>
<artifactId>caliper</artifactId>
<version>1.0-beta-SNAPSHOT</version>
</dependency>
System properties:
Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 08:22:22-0700)
Maven home: ~/maven3/apache-maven-3.1.1
Java version: 1.6.0_65, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.9.1", arch: "x86_64", family: "mac"
Also, the error logs suggest uploading the failed results manually. However, I don't see any option on the Microbenchmarks Appspot page to upload results.
Edit 1:
Benchmark launch command:
mvn exec:java -Dexec.mainClass="com.google.caliper.runner.CaliperMain" -Dexec.args="my.benchmarkclass"