The remote worker guide of bazel (here) explains how to start the remote-worker locally and then run bazel against it.
I tried it and indeed that worked (with bugs that reported in GH)
Another attempt was to create run the remote worker on a virtual separate machine, by running it inside docker container and running bazel against it. But it failed in a different way - and I think this time I'm using it wrong.
Here's my docker file:
FROM openjdk:8
# install release bazel from apt
RUN echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list
RUN curl https://bazel.build/bazel-release.pub.gpg | apt-key add -
RUN apt-get update && apt-get install -y zip bazel
# compile dev bazel from sources
RUN mkdir -p /usr/src/bazel
# "bazel" has the latest development code of bazel from github
COPY bazel /usr/src/bazel
WORKDIR /usr/src/bazel
RUN bazel build src/bazel
# compile remote_worker using latest development bazel
RUN bazel-bin/src/bazel build //src/tools/remote_worker
# prepare cache folder
RUN mkdir -p /tmp/test
# Run remote-worker
CMD ["bazel-bin/src/tools/remote_worker/remote_worker","--work_path=/tmp/test","--listen_port=3030"]
After building it I simply ran the docker binding the port to the localhost:
$ docker build -t bazel-worker .
$ docker run -p 3030:3030 bazel-worker
Then ran bazel java test to run using the remote worker:
(Can check out my test repo here)
$ bazel --host_jvm_args=-Dbazel.DigestFunction=SHA1 test \
--spawn_strategy=remote \
--remote_executor=localhost:3030 \
--remote_cache=localhost:3030 \
--strategy=Javac=remote \
--remote_local_fallback=false \
--remote_timeout=600 \
//src/main/java/com/example/...
But I got this weird error message:
____Loading package: src/main/java/com/example
____Loading package: #bazel_tools//tools/cpp
____Loading package: #local_jdk//
____Loading package: #local_config_xcode//
____Loading package: #local_config_cc//
____Loading complete. Analyzing...
____Loading package: tools/defaults
____Loading package: #bazel_tools//third_party/java/jdk/langtools
____Loading package: #junit//jar
____Found 1 test target...
____Building...
____[0 / 2] BazelWorkspaceStatusAction stable-status.txt
____[2 / 4] Creating source manifest for //src/main/java/com/example:my_test
____From Extracting interface #junit//jar:jar:
/tmp/test/build-80057300-ffd2-49ea-a20b-3f234d9963db/external/bazel_tools/tools/jdk/ijar/ijar: 1: /tmp/test/build-80057300-ffd2-49ea-a20b-3f234d9963db/external/bazel_tools/tools/jdk/ijar/ijar: �����0��!H__PAGEZEROx__TEXTpp__text__TEXT/��__stubs__TEXT0p�__stub_helper__TEXT���__gcc_except_tab__TEXT�: not found
/tmp/test/build-80057300-ffd2-49ea-a20b-3f234d9963db/external/bazel_tools/tools/jdk/ijar/ijar: 2: /tmp/test/build-80057300-ffd2-49ea-a20b-3f234d9963db/external/bazel_tools/tools/jdk/ijar/ijar: Syntax error: word unexpected (expecting ")")
ERROR: /private/var/tmp/_bazel_ors/719f891d5db9fd5e73ade25b0c847fd1/external/junit/jar/BUILD.bazel:2:1: output 'external/junit/jar/_ijar/jar/external/junit/jar/junit-4.12-ijar.jar' was not created.
ERROR: /private/var/tmp/_bazel_ors/719f891d5db9fd5e73ade25b0c847fd1/external/junit/jar/BUILD.bazel:2:1: not all outputs were created or valid.
____Building complete.
Target //src/main/java/com/example:my_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
____Elapsed time: 13.614s, Critical Path: 0.21s
Am I doing anything wrong? Do I need to run it differently when running the remote worker on an actual (or virtual) remote machine (vs. just running it locally)?
Important to mention: my machine is mac osx sierra. , I believe that docker openjdk:8 is ubuntu based, I'm running locally bazel development version (sha 956810b6ee24289e457a4b8d0a84ff56eb32c264).
Running the remote worker on a different architecture / OS combination than Bazel itself isn't working yet. We still have a couple of places in Bazel where we inspect the local machine - they were added as temporary measures, but haven't been fixed yet.
Edit: It may work in some cases, especially for platform-independent code (e.g., Java or Scala).
If your build is test-heavy, you could try only running tests remotely with --test_strategy=remote; I'm not sure if the default Jvm configuration will work, though.
If you want to run the entire build remotely, then you need to tell Bazel what kind of machines / OS it's executing on. Right now, that'd require setting --host_cpu, and probably --crosstool_top / --host_crosstool_top to configure a C++ compiler for that platform.
Also, some combinations of platforms are more and some less likely to work. In particular, combining MacOS and Linux or different flavors of Linux are much more likely to work than Windows in any combination.
Related
I'm trying to write a dockerfile to run a rust test suite that requires a jar to be running on a separate port. I've downloaded the jar into the project and would like to start with the openjdk docker image and download rust as well. That way I can use the java command to run the jar then cargo to run the tests.
# Start with java
FROM openjdk:latest
# Add rust
RUN <one line command to download rust>
# Install production dependencies and build a release artifact.
RUN cargo build --release
# Start chromedriver, geckodriver, and selenium standalone binary
RUN ./thirtyfour/tests/binaries_and_jars/chromedriver
RUN ./thirtyfour/tests/binaries_and_jars/geckodriver
RUN java -jar ./thirtyfour/tests/binaries_and_jars/selenium.jar standalone --port 1234
# Run the tests
RUN cargo test
I was hoping to download rust using the curl command provided on the website, passing all the options as cli arguments, but I still get the prompt. Running
curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain stable --default-host x86_64-unknown-linux-gnu --profile default
still produces the confirmation prompt.
I'm open to any solution here, including
A one-line command to install rust
Starting from the rust image + a one-line command to install java.
You can skip the confirmation prompt by adding the -y argument. This can be found in the help page of installer script.
The installer for rustup
USAGE:
rustup-init [FLAGS] [OPTIONS]
FLAGS:
-v, --verbose Enable verbose output
-q, --quiet Disable progress output
-y Disable confirmation prompt.
--no-modify-path Don't configure the PATH environment variable
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
--default-host <default-host> Choose a default host triple
--default-toolchain <default-toolchain> Choose a default toolchain to install
--default-toolchain none Do not install any toolchains
--profile [minimal|default|complete] Choose a profile
-c, --component <components>... Component name to also install
-t, --target <targets>... Target name to also install
We have a Java Spring Boot application that runs in a Docker container. It is based on openjdk:13-jdk-alpine. We deploy it to Linux machines, but we are also able to run it locally on Windows machines, as well as on an Intel-based iMac.
We have found, though, that it cannot run properly on an ARM-based MacBook Pro. The exceptions we get are basic Java errors like "Can't find symbol Java.class[]," and other things that look like the JVM is off.
Is there a way to build a Docker image that will work on all these platforms, including the M1 MacBook Pro?
I have a lot of problems with Java containers too on my M1 macbook. For your problem, maybe you need to create your own docker image:
Dockerfile
FROM --platform=linux/arm64/v8 ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
EXPOSE 8080
RUN apt update \
&& apt upgrade -y \
&& apt install -y openjdk-13-jre git \
&& apt clean
RUN mkdir -pv /app && cd /app && \
git clone https://github.com/spring-guides/gs-spring-boot.git && \
cd /app/gs-spring-boot/initial && ./gradlew build
WORKDIR /app/gs-spring-boot/initial
ENTRYPOINT [ "./gradlew", "bootRun" ]
Build image
docker build -t test .
Run container
docker run --rm -p 8080:8080 test
Go to http://localhost:8080/ on your browser and your Spring-Boot application is running without Rosetta 2.
Disclaimer: I'm not a Java developer and my Dockerfile is for Proof of Concept purpose.
Remember that your Docker image is builded to ARM64 architecture. If you wanna run this container on a Intel/AMD processor, you have to change FROM --platform=linux/amd64 ubuntu:20.04 on your Dockerfile.
I made it work with the following image.
I pulled the image with
docker pull bellsoft/liberica-openjdk-alpine-musl:17
My Dockerfile:
FROM bellsoft/liberica-openjdk-alpine-musl:17
ADD build/libs/app-0.0.1-SNAPSHOT-plain.jar app.jar
ENTRYPOINT ["java","-jar","app.jar"]
Now the docker build command worked
Build your images with multiarch support to get rid of all possible architecture failures in the future. To do this cleanly, avoid using anything related to the platform in your Dockerfile, just old-school Dockerfiles are ok.
If you are using github and github-actions, you may check this to build your images and push them into your image repository. This can be also used for building images which work on RaspberryPi like SBCs.
it's because image is not supported for m1 yet, you can build it for cross platform and run it
docker build --platform=linux/arm64 -t image:latest .
I have a fairly standard Java image with some dependencies added in that I'm writing this Dockerfile on top of. Here's where I'm having a problem in my Dockerfile:
RUN which java
RUN ["/usr/bin/java", "-jar", "tzupdater.jar", "-v", "-l"]
I thought that every Dockerfile command was run as root? But when I try to build from this Dockerfile, I'm running into this permissions issue which also happened in my local environment when I forgot to run this JAR as sudo:
$ docker build -t container-w-tz-update .
Uploading context 1.122 GB
Uploading context
Step 0 : FROM company/java-img-with-dependencies:1.0
---> 0101010101
...
Step 3 : RUN which java
---> Running in 0101010101
/usr/bin/java
---> 0101010101
Step 4 : RUN ["/usr/bin/java", "-jar", "tzupdater.jar", "-v", "-l"]
---> Running in 0101010101
java.vendor: Sun Microsystems Inc.
java.version: 1.6.0_45
tzupdater version 2.1.1-b01
Downloaded file to /tmp/tz.tmp/tzdata.tar.gz
Downloaded file to /tmp/tz.tmp/sha512hash
failed.
Cant rename {0} to {1}.
com.sun.tools.tzupdater.TzRuntimeException: com.sun.tools.tzupdater.TzRuntimeException: Cant rename {0} to {1}.
Caused by: com.sun.tools.tzupdater.TzRuntimeException: Cant rename {0} to {1}.
at com.sun.tools.tzupdater.TimezoneUpdater.update(TimezoneUpdater.java:301)
at com.sun.tools.tzupdater.TimezoneUpdater.run(TimezoneUpdater.java:249)
at com.sun.tools.tzupdater.TimezoneUpdater.main(TimezoneUpdater.java:643)
2017/04/05 22:20:23 The command [/usr/bin/java -jar tzupdater.jar -v -l] returned a non-zero code: 1
I was able to run this utility as sudo on my local with the exact same version of Java with no issues. When I tried to run it locally without sudo, I received the same error. Thoughts?
(This is more like a comment but the message length is beyond the comment limit, so I leave it here as an answer so that other people can maybe give more useful info based on what I've done.)
Seems it's really an unresolved issue, I reproduced your problem by image enoniccloud/java6 running with root user, I also tried docker run -it --privileged ... to run it manually but it does not help. I also tried jdk8 with base image alpine:3.3 but also failed with:
Downloaded file to /tmp/tz.tmp/sha512hash
Renaming /opt/jdk1.8.0_91/jre/lib/tzdb.dat to /opt/jdk1.8.0_91/jre/lib/tzdb.dat.tzdata2016a failed.
Cant rename {0} to {1}.
Validating for : tzdata2017b
Validation complete
JRE updated to version : tzdata2017b
I searched and the only info I found is as follows:
An open issue: https://github.com/docker/hub-feedback/issues/896
Related but not the same: https://forums.docker.com/t/update-docker-container-jre-to-set-timezone-correctly/24426
Before you find the root cause for this issue, I think an alternative way to build your image could be:
Download a JDK tarball for your platform
Untar it, setup JAVA_HOME and PATH, run java -jar tzupdater.jar -v -l on your host
Build your base image based on this updated JDK, ADD to image and setup environment variable like JAVA_HOME and PATH
Hope this could be helpful to you :-)
docker build will complete RUN steps as the last USER set in the Dockerfile.
To reset:
RUN whoami
USER root
RUN ["/usr/bin/java", "-jar", "tzupdater.jar", "-v", "-l"]
USER "whatever whoami reported"
I got a workaround for that. Adding some mv commands (that have no real effect) made it work:
FROM openjdk:7u211-jdk-alpine3.9
ADD tzupdater.jar tzupdater.jar
RUN mv /usr/lib/jvm/java-1.7-openjdk/jre/lib/zi /usr/lib/jvm/java-1.7-openjdk/jre/lib/zi.tzdata2018g && \
mv /usr/lib/jvm/java-1.7-openjdk/jre/lib/zi.tzdata2018g /usr/lib/jvm/java-1.7-openjdk/jre/lib/zi && \
java -jar tzupdater.jar -v -l https://.../tzdata-latest.tar.gz
I have no idea why it works, I got it by trial and error.
Notice the two mv commands do like mv a b && mv b a, so nothing really changes.
The names of the files (dirs, actually) used in the mv are the ones it outputs when it errors (before the workaround), so change it accordingly.
Mac 10.10.5 here, using docker-machine to create a VirtualBox host VM for my local Docker. I have a project that builds an executable JVM located at build/libs/myapp-SNAPSHOT.jar. My Dockerfile, which is located in the root of the project, looks like:
FROM frolvlad/alpine-oraclejdk8:slim
VOLUME /tmp
ADD build/libs/myapp-SNAPSHOT.jar myapp.jar
RUN sh -c 'touch /myapp.jar'
ENTRYPOINT ["java","-jar","/myapp.jar"]
Please note, I don't wish to push my images to any registry, just keep/run them locally (for now). When I run:
docker build -t myorg/myapp .
I get the following console output:
myuser#mymachine:~/sandbox/myapp$docker build -t myorg/myapp .
Sending build context to Docker daemon 42.69 MB
Step 1 : FROM frolvlad/alpine-oraclejdk8:slim
slim: Pulling from frolvlad/alpine-oraclejdk8
d0ca440e8637: Downloading [=================================================> ] 2.295 MB/2.32 MB
0f86278f6be1: Downloading [=================================================> ] 3.149 MB/3.172 MB
c704a6161dca: Download complete
And then the command-line just hangs after printing that "Download complete" message. I've waited for as long as 30 minutes (!!!) and nothing happens.
Any ideas where I'm going awry?
The VM is probably hanging. Try the following: https://github.com/docker/machine/issues/1819#issuecomment-138981139
docker-machine rm -f default
rm -fv ~/.docker/machine
docker-machine -D create -d virtualbox default
There are more issues about this on OSX.
I think the best practice is to setup a Linux native build box if you are doing any serious development. That way you can run docker without any VM overhead(which is ironically one of the major pain points docker is trying to solve)
There's also a Docker Beta program which runs on libcontainer natively on OSX and Windows.
I am getting this strange error at the end of the process of creating a docker image from a Dockerfile:
/bin/sh: 1: gradle: not found
INFO[0003] The command [/bin/sh -c gradle test jar] returned a non-zero code: 127
The relevant part of the Dockerfile:
FROM debian:jessie
[...]
RUN curl -L https://services.gradle.org/distributions/gradle-2.4-bin.zip -o gradle-2.4-bin.zip
RUN apt-get install -y unzip
RUN unzip gradle-2.4-bin.zip
RUN echo 'export GRADLE_HOME=/app/gradle-2.4' >> $HOME/.bashrc
RUN echo 'export PATH=$PATH:$GRADLE_HOME/bin' >> $HOME/.bashrc
RUN /bin/bash -c "source $HOME/.bashrc"
RUN gradle test jar
[...]
The command I am using is: docker build -t java_i .
The strange thing is that if:
I run a container from the previous image commenting out RUN gradle test jar (command: docker run -d -p 9093:8080 -p 9094:8081 --name java_c -i -t java_i),
then I log into that container (command: docker exec -it java_c bash),
then I manually check the gradle environment variables finding them,
then I manually run that commented out command from within the running container (gradle test jar):
I eventually get the expected output (the compiled java code in the build folder).
I am using Docker version 1.6.2
I solved the problem using the ENV docker instructions (link to the documentation).
ENV GRADLE_HOME=/app/gradle-2.4
ENV PATH=$PATH:$GRADLE_HOME/bin
This command /bin/bash -c "source $HOME/.bashrc" means that you create a new non-interactive process and run a command in it to set environment variables there. Which does not affect the parent process. As soon as variables are set, process exits. You can check this by running something like this:
RUN /bin/bash -c "source $HOME/.bashrc; env"
RUN env
What should be working is this option:
RUN source ~/.bashrc
And the reason why it works when you log in, is because the new process reads already updated ~/.bashrc.
I was trying to install same version with JDK 11.0.7 but gradle-2.4 does not work. and got below error
FAILURE: Build failed with an exception.
* What went wrong:
Could not determine java version from '11.0.7'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
I install later version to fix the above issue after installation.
Posting as an answer might help someone else.
FROM openjdk:11.0.7-jdk
RUN apt-get update && apt-get install -y unzip
WORKDIR /gradle
RUN curl -L https://services.gradle.org/distributions/gradle-6.5.1-bin.zip -o gradle-6.5.1-bin.zip
RUN unzip gradle-6.5.1-bin.zip
ENV GRADLE_HOME=/gradle/gradle-6.5.1
ENV PATH=$PATH:$GRADLE_HOME/bin
RUN gradle --version
You can use multi-stage builds and the Gradle Docker image (no need to install Gradle...) to build the application then use the result in the runtime container:
# Build
FROM gradle AS build
WORKDIR /appbuild
COPY . /appbuild
RUN gradle --version
# here goes your build code
Once the Gradle build is done, switch to the runtime container:
# Runtime
FROM openjdk:8-jre-alpine
# more stuff here...
COPY --from=0 appbuild/<somepath>/some.jar application.jar
# more stuff here...
The COPY command copies the build artifacts from the build phase to the runtime container (in this case a jar file).