I'm fairly new to Docker and am struggling with JAVA_HOME not being seen in a Dockerfile. I get the titular error; which includes Please set the JAVA_HOME variable in your environment to match the location of your Java installation. & executor failed running [/bin/sh -c /opt/Android/cmdline-tools/latest/bin/sdkmanager --update]: exit code: 1 when it runs RUN /opt/Android/cmdline-tools/latest/bin/sdkmanager --update.
I feel like I'm at a loss but I think my issue is not knowing where the JDK is being installed to or knowing how to find it from a Dockerfile; I've tried echoing JAVA_HOME thinking I could see it while the image built but, again, no luck. Any help with this would be greatly appreciated. I've been pulling my hair out Googling & trying things. Thank you.
FROM node:12.12.0
ARG CMDLINE_TOOLS_VERSION=7583922
ARG ANDROID_BUILD_TOOLS=30.0.3
RUN apt-get -qqy update \
&& apt-get -qqy install \
python-dev \
--no-install-recommends
RUN apt-get install -y software-properties-common gcc
RUN apt-get update && apt-get install -y python3-pip
RUN pip3 install awscli
RUN apt-get install -y jq
RUN mkdir -p /usr/share/man/man1 /usr/share/man/man2
RUN apt-get update && apt-get install -y --no-install-recommends openjdk-8-jdk && apt-get clean;
ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
ENV PATH=$PATH:$JAVA_HOME/bin
RUN wget -q https://services.gradle.org/distributions/gradle-4.5.1-bin.zip && unzip gradle-4.5.1-bin.zip -d /opt && rm gradle-4.5.1-bin.zip
ENV GRADLE_HOME=/opt/gradle-4.5.1
ENV PATH=$PATH:/opt/gradle-4.5.1/bin
RUN wget https://dl.google.com/android/repository/commandlinetools-linux-${CMDLINE_TOOLS_VERSION}_latest.zip
RUN mkdir -p /opt/Android/cmdline-tools
RUN unzip commandlinetools-linux-7583922_latest.zip -d /opt/Android/cmdline-tools
RUN mv /opt/Android/cmdline-tools/cmdline-tools /opt/Android/cmdline-tools/latest
ENV ANDROID_HOME=/opt/Android
ENV PATH="$ANDROID_HOME/emulator:$ANDROID_HOME/tools:$ANDROID_HOME/tools/bin \
:$ANDROID_HOME/cmdline-tools/latest:$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/platform-tools:$PATH"
RUN /opt/Android/cmdline-tools/latest/bin/sdkmanager --update
RUN /opt/Android/cmdline-tools/latest/bin/sdkmanager --list
RUN /opt/Android/cmdline-tools/latest/bin/sdkmanager --list | grep build-tools
RUN echo y | /opt/Android/cmdline-tools/latest/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS}" "platform-tools" "platforms;android-30" "tools" >/dev/null
RUN yes | /opt/Android/cmdline-tools/latest/bin/sdkmanager --licenses
CMD ["yarn", "start"]
I'd suggest to use another base image. Java 11 is required to build for newer API levels:
FROM openjdk:11-jdk as builder
...
And then install Python3 and AWS CLI.
Working example: cloudbuild-android.
Or if you want to continue with your's, RUN which java would tell you where it's actually installed.
In a comment, you mentioned:
when I run [RUN ls -lart /usr/lib/jvm/java-8-openjdk-amd64], I get cannot access '/usr/lib/jvm/java-8-openjdk-amd64': No such file or directory
Maybe I'm missing something, but... doesn't that mean that the directory does not exist?
Either you got the path wrong, or (as another answer suggested) there's something in your Dockerfile, probably line endings, that's mangling your lines.
To discard the bit about line endings, try the same command but switching the parameters, i.e.:
RUN ls /usr/lib/jvm/java-8-openjdk-amd64 -lart
If now it whines about unrecognized options, then it's probably a line ending issue (which now affects the t option instead of the directory path).
However, if it still says No such file or directory, then you definitely should check your Java installation path.
I think your problem is in the line termination characters of your Dockerfile.
Your Dockerfile works in my computer, and when it comes to weird errors, I have been there.
My experience with Dockerfiles is that sometimes they are very picky in the Windows/Unix/Mac line terminators, so please make sure you save the Dockerfile with line terminators adequate for Unix machines.
I have used the method in this other answer to successfully replace Windows line terminators for Unix's on Windows using Powershell. I hope this helps!
The exact location of $JAVA_HOME can be scripted. If you don't feel like hardcoding that directory is a safe bet, replace your ENV line with this RUN: RUN export JAVA_HOME=$(readlink -f $(which java) | sed "s:/bin/java::"). This is how I set up my environment in all my Linux machines, and what it does is the following:
which java searches for java and will output /usr/bin/java, or something like that, which is a symbolic link.
readlink -f over the above will get you to the destination of the symbolic link.
piping such destination over to | sed "s:/bin/java::" will strip the trailing bin/java and get you the right value, no matter the version.
I also tried your Dockerfile with this change, and it does work too.
It looks like you had some network issues during your first docker build run. It failed to install java, but cached the layer with the attempt. I recommend you to run your build again without caching: docker build -t name --no-cache . and check logging of network operations.
I'm very much new to docker. I have a node application that process/read a java file. I'm trying to create a docker image that runs node and java. I node configs are working well but I'm always getting errors when building the docker file because of java - it shows
failed to compute cache key: "/japp.jar" not found: not found"
My Dockerfile contains:
FROM node:14.18.1
RUN apt-get update && apt-get -y upgrade
WORKDIR /code
EXPOSE 8443
COPY package.json /code/package.json
RUN npm install
COPY . /code
# Jave Config
FROM eclipse-temurin:11
RUN mkdir /opt/app
COPY japp.jar /opt/app
ENTRYPOINT ["java", "-jar", "/opt/app/japp.jar"]
CMD ["node","app.js"]
I was trying to follow the docs here but no luck.
In the past, I was using the following for Java config and that was working fine, but it shows some vulnerability so I need to switch to eclipse-temurin:11
# RUN echo 'deb http://ftp.debian.org/debian stretch-backports main' | tee /etc/apt/sources.list.d/stretch-backports.list
# RUN apt-get update && apt-get -y upgrade && \
# apt-get install -y openjdk-11-jre-headless && \
# apt-get clean;
I successfully created and started this container :
https://github.com/puckel/docker-airflow
by running :
docker build --rm --build-arg AIRFLOW_DEPS="datadog,dask" --build-arg PYTHON_DEPS="flask_oauthlib>=0.9" -t puckel/docker-airflow .
docker-compose -f docker-compose-CeleryExecutor.yml up -d
Then i just wanted to add java to images :
I added at the end :
https://github.com/puckel/docker-airflow/blob/master/Dockerfile#L83
USER airflow
WORKDIR ${AIRFLOW_USER_HOME}
ENTRYPOINT ["/entrypoint.sh"]
CMD ["webserver"] # set default arg for entrypoint
# my new code :
USER root
RUN mkdir -p /usr/share/man/man1/
# Install OpenJDK-8
RUN apt-get update && \
apt-get install -y openjdk-8-jdk && \
apt-get install -y ant && \
apt-get clean;
# Fix certificate issues
RUN apt-get update && \
apt-get install ca-certificates-java && \
apt-get clean && \
update-ca-certificates -f;
# Setup JAVA_HOME -- useful for docker commandline
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
After building and running the container,
i'm trying to connect like i did before instaling java :
docker exec -it docker-airflow_worker_1 /bin/bash
But i'm automaticaly disconnected. And java is installed
root#5475d28fb5f5:/usr/local/airflow# java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-8u232-b09-1~deb9u1-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
root#5475d28fb5f5:/usr/local/airflow# %
And when i log the container :
User information: uid=0 euid=0 gid=0 egid=0
[2019-10-31 12:07:40,312] {{settings.py:213}} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=1
/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
""")
[2019-10-31 12:07:40,621] {{__init__.py:51}} INFO - Using executor CeleryExecutor
Running a worker with superuser privileges when the
worker accepts messages serialized with pickle is a very bad idea!
If you really want to continue then you have to set the C_FORCE_ROOT
environment variable (but please think about this before you do).
User information: uid=0 euid=0 gid=0 egid=0
Can anyone help ?
Thanks
So you changed user to root in Dockerfile and Airflow now refuses to start. Try to move your new code in Dockerfile before USER airflow line. Also you can delete USER root line after that.
I need both java and python in my docker container to run some code.
This is my dockerfile:
It works perpectly if I don't add the FROM openjdk:slim
#get python
FROM python:3.6-slim
RUN pip install --trusted-host pypi.python.org flask
#get openjdk
FROM openjdk:slim
COPY . /targetdir
WORKDIR /targetdir
# Make port 81 available to the world outside this container
EXPOSE 81
CMD ["python", "test.py"]
And the test.py app is in the same directory:
from flask import Flask
import os
app = Flask(__name__)
#app.route("/")
def hello():
html = "<h3>Test:{test}</h3>"
test = os.environ['JAVA_HOME']
return html.format(test = test)
if __name__ == '__main__':
app.run(debug=True,host='0.0.0.0',port=81)
I'm getting this error:
D:\MyApps\Docker Toolbox\Docker Toolbox\docker.exe: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"python\": executable file not found in $PATH": unknown.
What exactly am I doing wrong here? I'm new to docker, perhaps I'm missing a step.
Additional details
My goal
I have to run a python program that runs a Java file. The python library I'm using requires the path to JAVA_HOME.
My issues:
I do not know Java, so I cannot run the file properly.
My entire code is in Python, except this Java bit
The Python wrapper runs the file in a way I need it to run.
An easier solution to the above issue is to use multi-stage docker containers where you can copy the content from one to another. In the above case you can have openjdk:slim as the base container and then use content from a python container to be copied over into this base container as follows:
FROM openjdk:slim
COPY --from=python:3.6 / /
...
<normal instructions for python container continues>
...
This feature is available as of Docker 17.05 and there are more things you can do using multi-stage build as in copying only the content you need from one to another.
Reference documentation
OK it took me a little while to figure it out. And my thanks go to this answer.
I think my approach didn't work because I did not have a basic version of Linux.
So it goes like this:
Get Linux (I'm using Alpine because it's barebones)
Get Java via the package manager
Get Python, PIP
OPTIONAL: find and set JAVA_HOME
Find the path to JAVA_HOME. Perhaps there is a better way to do this, but I did this running the running the container, then I looked inside the container using docker exec -it [COINTAINER ID] bin/bash and found it.
Set JAVA_HOME in dockerfile and build + run it all again
Here is the final Dockerfile ( it should work with the python code in the question) :
### 1. Get Linux
FROM alpine:3.7
### 2. Get Java via the package manager
RUN apk update \
&& apk upgrade \
&& apk add --no-cache bash \
&& apk add --no-cache --virtual=build-dependencies unzip \
&& apk add --no-cache curl \
&& apk add --no-cache openjdk8-jre
### 3. Get Python, PIP
RUN apk add --no-cache python3 \
&& python3 -m ensurepip \
&& pip3 install --upgrade pip setuptools \
&& rm -r /usr/lib/python*/ensurepip && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache
### Get Flask for the app
RUN pip install --trusted-host pypi.python.org flask
####
#### OPTIONAL : 4. SET JAVA_HOME environment variable, uncomment the line below if you need it
#ENV JAVA_HOME="/usr/lib/jvm/java-1.8-openjdk"
####
EXPOSE 81
ADD test.py /
CMD ["python", "test.py"]
I'm new to Docker, so this may not be the best possible solution. I'm open to suggestions.
UPDATE: COMMON ISUUES
Difficulty using python packages
As Joabe Lucena pointed out here, Alpine can have issues certain python packages.
I recommend that you use a Linux distro that works best for you, e.g. centos.
Another alternative is to simply use docker-java-python image from docker hub. https://hub.docker.com/r/rappdw/docker-java-python
FROM rappdw/docker-java-python:openjdk1.8.0_171-python3.6.6
RUN java -version
RUN python --version
I found Sunny Pal's answer very useful but I made the copy more specific and added the necessary environment variables and update-alternatives lines so that Java was accessible from the command line in the Python container.
FROM python:3.9-slim
COPY --from=openjdk:8-jre-slim /usr/local/openjdk-8 /usr/local/openjdk-8
ENV JAVA_HOME /usr/local/openjdk-8
RUN update-alternatives --install /usr/bin/java java /usr/local/openjdk-8/bin/java 1
...
Oh, let me add my five cents. I took python slim as a base image. Then I found open-jdk-11 (Note, open-jdk-10 will fail because it is not supported) base image code!... And copy-pasted it into my docker file.
Note, copy-paste driven development is cool... ONLY when you understand each line you use in your code!!!
And here it is!
<!-- language: shell -->
FROM python:3.7.2-slim
# Do your stuff, install python.
# and now Jdk
RUN rm -rf /var/lib/apt/lists/* && apt-get clean && apt-get update && apt-get upgrade -y \
&& apt-get install -y --no-install-recommends curl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
ENV JAVA_VERSION jdk-11.0.2+7
COPY slim-java* /usr/local/bin/
RUN set -eux; \
ARCH="$(dpkg --print-architecture)"; \
case "${ARCH}" in \
ppc64el|ppc64le) \
ESUM='c18364a778b1b990e8e62d094377af48b000f9f6a64ec21baff6a032af06386d'; \
BINARY_URL='https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.1%2B13/OpenJDK11U-jdk_ppc64le_linux_hotspot_11.0.1_13.tar.gz'; \
;; \
s390x) \
ESUM='e39aacc270731dadcdc000aaaf709adae7a08113ccf5b4a045bc87fc13458d71'; \
BINARY_URL='https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11%2B28/OpenJDK11-jdk_s390x_linux_hotspot_11_28.tar.gz'; \
;; \
amd64|x86_64) \
ESUM='d89304a971e5186e80b6a48a9415e49583b7a5a9315ba5552d373be7782fc528'; \
BINARY_URL='https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.2%2B7/OpenJDK11U-jdk_x64_linux_hotspot_11.0.2_7.tar.gz'; \
;; \
aarch64|arm64) \
ESUM='b66121b9a0c2e7176373e670a499b9d55344bcb326f67140ad6d0dc24d13d3e2'; \
BINARY_URL='https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.1%2B13/OpenJDK11U-jdk_aarch64_linux_hotspot_11.0.1_13.tar.gz'; \
;; \
*) \
echo "Unsupported arch: ${ARCH}"; \
exit 1; \
;; \
esac; \
curl -Lso /tmp/openjdk.tar.gz ${BINARY_URL}; \
sha256sum /tmp/openjdk.tar.gz; \
mkdir -p /opt/java/openjdk; \
cd /opt/java/openjdk; \
echo "${ESUM} /tmp/openjdk.tar.gz" | sha256sum -c -; \
tar -xf /tmp/openjdk.tar.gz; \
jdir=$(dirname $(dirname $(find /opt/java/openjdk -name javac))); \
mv ${jdir}/* /opt/java/openjdk; \
export PATH="/opt/java/openjdk/bin:$PATH"; \
apt-get update; apt-get install -y --no-install-recommends binutils; \
/usr/local/bin/slim-java.sh /opt/java/openjdk; \
apt-get remove -y binutils; \
rm -rf /var/lib/apt/lists/*; \
rm -rf ${jdir} /tmp/openjdk.tar.gz;
ENV JAVA_HOME=/opt/java/openjdk \
PATH="/opt/java/openjdk/bin:$PATH"
ENV JAVA_TOOL_OPTIONS="-XX:+UseContainerSupport"
Now references.
https://github.com/AdoptOpenJDK/openjdk-docker/blob/master/11/jdk/ubuntu/Dockerfile.hotspot.releases.slim
https://hub.docker.com/_/python/
https://hub.docker.com/r/adoptopenjdk/openjdk11/
I used them to answer this question, which may help you sometime.
Running Python and Java in Docker
I believe that by adding FROM openjdk:slim line, you tell docker to execute all of your subsequent commands in openjdk container (which does not have python)
I would approach this by creating two separate containers for openjdk and python and specify individual sets of commands for them.
Docker is made to modularize your solutions and mashing everything into one container is usually a bad practice.
I tried pajamas's anwser which worked very well for creating this image. However, when trying to install packages like gensim, pandas or else, I faced some errors like: don't know how to compile Fortran code on platform 'posix'. I searched and tried this, this and that but none worked for me.
So, based on pajamas's anwser I decided to convert his image from Alpine to Centos which worked very well. So here's a Dockerfile that might help someone who's may be struggling in this scenario like I was:
# Get Linux
FROM centos:7
# Install Java
RUN yum update -y \
&& yum install java-1.8.0-openjdk -y \
&& yum clean all \
&& rm -rf /var/cache/yum
# Set JAVA_HOME environment var
ENV JAVA_HOME="/usr/lib/jvm/jre-openjdk"
# Install Python
RUN yum install python3 -y \
&& pip3 install --upgrade pip setuptools wheel \
&& if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi \
&& if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi \
&& yum clean all \
&& rm -rf /var/cache/yum
CMD ["bash"]
you should have one FROM in your dockerfile
(unless you use multi-stage build for the docker)
I think i found easiest way to mix java jdk 17 and python3. I is not working on python2
FROM openjdk:17.0.1-jdk-slim
RUN apt-get update && \
apt-get install -y software-properties-common && \
apt-get install -y python3-pip
Software Commons have python3 lightweight version. (3.9.1 version)
U can also install some libraries like that.
RUN python3 -m pip install --upgrade pip && \
python3 -m pip install numpy && \
python3 -m pip install opencv-python
OR
RUN apt-get update && \
apt-get install -y ffmpeg
Easiest is to just start from a Python image and add the OpenJDK. Note that FROM openjdk has been deprecated and replaced with eclipse-temurin
FROM python:3.10
ENV JAVA_HOME=/opt/java/openjdk
COPY --from=eclipse-temurin:17-jre $JAVA_HOME $JAVA_HOME
ENV PATH="${JAVA_HOME}/bin:${PATH}"
RUN pip install --trusted-host pypi.python.org flask
See How to use this Image - Using a different base Image section of https://hub.docker.com/_/eclipse-temurin for details.
Instead of using FROM openjdk:slim you can separately install Java, please refer below example:
# Install OpenJDK-8
RUN apt-get update && \
apt-get install -y openjdk-8-jdk && \
apt-get install -y ant && \
apt-get clean;
# Fix certificate issues
RUN apt-get update && \
apt-get install ca-certificates-java && \
apt-get clean && \
update-ca-certificates -f;
# Setup JAVA_HOME -- useful for docker commandline
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JAVA_HOME
I am building a virtual machine for some tests that needs to run among other things DynamoDb web service. VM is based on stock Ubuntu 16.04 64bit.
This is the part of provisioning script that installs DynamoDB:
# Install local instance of DynamoDB
DARCHFILE='/tmp/dynamodb_local_latest.zip'
wget -P /tmp/ -nv https://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.zip
sudo unzip -q "$DARCHFILE" -d /usr/local/lib/dynamodb
sudo mkdir -pv /var/lib/dynamodb/
sudo mkdir -pv /var/log/dynamodb/
sudo apt-get install -y openjdk-8-jdk
This is the script that executes DynamoDb:
#!/bin/bash
# -*- mode: sh -*-
# vi: set ft=sh :
java -Djava.library.path=/usr/local/lib/dynamodb/DynamoDBLocal_lib/ \
-jar /usr/local/lib/dynamodb/DynamoDBLocal.jar \
-dbPath /var/lib/dynamodb/ \
-optimizeDbBeforeStartup \
-port 8000 > /var/log/dynamodb/trace.log 2> /var/log/dynamodb/error.log &
echo $! > /var/run/dynamodb.pid
Now if I try to stop the process using the "nice one" SIGINT (equivalent of Ctrl+C) it does nothing. Only sending SIGTERM works.
So this does nothing:
sudo kill -s SIGINT $(< /var/run/dynamodb.pid)
As we can confirm by executing this:
sudo ps -x | grep $(< /var/run/dynamodb.pid)
While this works just fine:
sudo kill -s SIGTERM $(< /var/run/dynamodb.pid)
On the other hand if I start dynamodb manually and don't send it to the background Ctrl+C does work.
So what gives? Is it my mistake or behavior by design?
Thanks