We Keep Coding

Adding GDAL to openjdk-alpine based container

I have Spring boot application running on container. The Dockerfile for it is like this:
FROM openjdk:8-jdk-alpine
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} app.jar
CMD ["java", "-jar", "/app.jar"]
In the application there is ogr2ogr command so I want to install GDAL package to container.
I have no idea how I can do this.I feel that I approach the solution the most here.
In this solution, proj was in testing branch of Alpine but now it is in community branch as I understand. So, I added the dependencies of proj and modified Dockerfile like this:
FROM openjdk:8-jdk-alpine
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} app.jar
ENV ROOTDIR /usr/local
ENV LD_LIBRARY_PATH /usr/local/lib
ENV GDAL_VERSION 3.1.2
ENV OPENJPEG_VERSION 2.3.0
# Load assets
WORKDIR $ROOTDIR/
RUN mkdir -p $ROOTDIR/src
RUN wget -qO- \
http://download.osgeo.org/gdal/${GDAL_VERSION}/gdal-${GDAL_VERSION}.tar.gz | \
tar -xzC $ROOTDIR/src/
RUN wget -qO- \
https://github.com/uclouvain/openjpeg/archive/v${OPENJPEG_VERSION}.tar.gz | \
tar -xzC $ROOTDIR/src/
RUN set -ex \
# Install runtime dependencies
&& apk add --update --no-cache \
git \
bash-completion \
python-dev \
python3-dev \
libffi-dev \
jpeg-dev \
openjpeg-dev \
libpng-dev \
linux-headers \
curl-dev \
musl \
libjpeg-turbo \
libcurl \
zlib \
libgcc \
libstdc++ \
musl \
sqlite-libs \
tiff \
&& apk add --no-cache \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/community \
proj \
proj-dev \
# Install common build dependencies
&& apk add --no-cache --virtual .build-deps \
gcc \
cmake \
build-base \
make \
swig \
apache-ant \
# Compile and install OpenJPEG
&& cd src/openjpeg-${OPENJPEG_VERSION} \
&& mkdir build && cd build \
&& cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$ROOTDIR \
&& make -j3 && make -j3 install && make -j3 clean \
&& cd $ROOTDIR && rm -Rf src/openjpeg* \
# Compile and install GDAL
&& cd src/gdal-${GDAL_VERSION} \
&& ./configure --with-python --with-curl --with-java=${JAVA_HOME} --with-openjpeg=$ROOTDIR \
&& make -j3 && make -j3 install \
# Compile Python and Java bindings for GDAL
&& cd $ROOTDIR/src/gdal-${GDAL_VERSION}/swig/java && make -j3 && make -j3 install \
&& cd $ROOTDIR/src/gdal-${GDAL_VERSION}/swig/python \
&& python3 setup.py build \
&& python3 setup.py install \
&& cd $ROOTDIR && rm -Rf src/gdal* \
# Remove build dependencies
&& apk del .build-deps \
CMD ["java", "-jar", "/app.jar"]
Then the output of docker build is:
15 176.1 checking for ld used by GCC... /usr/x86_64-alpine-linux-musl/bin/ld -m elf_x86_64
#15 176.1 checking if the linker (/usr/x86_64-alpine-linux-musl/bin/ld -m elf_x86_64) is GNU ld... yes
#15 176.1 checking for shared library run path origin... /bin/sh: can't open './config.rpath': No such file or directory
#15 176.1 done
#15 176.1 checking for iconv... yes
#15 176.2 checking for working iconv... yes
#15 176.2 checking for iconv declaration...
#15 176.2 extern size_t iconv (iconv_t cd, char * *inbuf, size_t *inbytesleft, char * *outbuf, size_t *outbytesleft);
#15 176.3 using ICONV_CPP_CONST=""
#15 176.3 configure: Bash completions not requested
#15 176.3 checking for PROJ >= 6 library... checking for proj_create_from_wkt in -lproj... no
#15 176.3 checking for internal_proj_create_from_wkt in -lproj... no
#15 176.4 checking for internal_proj_create_from_wkt in -linternalproj... no
#15 176.4 configure: error: PROJ 6 symbols not found
------
executor failed running [/bin/sh -c set -ex && apk add --update --no-cache git bash-completion python-dev python3-dev libffi-dev jpeg-dev openjpeg-dev
libpng-dev linux-headers curl-dev musl libjpeg-turbo libcurl zlib libgcc libstdc++ musl sqlite-libs tiff && apk add --no-cache
--repository http://dl-cdn.alpinelinux.org/alpine/edge/community proj proj-dev && apk add --no-cache --virtual .build-deps gcc cmake build-base make
swig apache-ant && cd src/openjpeg-${OPENJPEG_VERSION} && mkdir build && cd build && cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$ROOTDIR && make -j3 && make -j3 install &
& make -j3 clean && cd $ROOTDIR && rm -Rf src/openjpeg* && cd src/gdal-${GDAL_VERSION} && ./configure --with-python --with-curl --with-java=${JAVA_HOME} --with-openjpeg=$ROOTDIR && make -j3 && m
ake -j3 install && cd $ROOTDIR/src/gdal-${GDAL_VERSION}/swig/java && make -j3 && make -j3 install && cd $ROOTDIR/src/gdal-${GDAL_VERSION}/swig/python && python3 setup.py build && python3 setup.p
y install && cd $ROOTDIR && rm -Rf src/gdal* && apk del .build-deps CMD ["java", "-jar", "-Dlogging.config=/logback.xml", "/app.jar"]]: exit code: 1
Let me emphasize that I do not know about Alpine Linux and GDAL. I tried to do something based on a solution I found.

Quarkus docker build is failing

We have been trying to deploy quarkus build image to ECR but recently been hitting
error: Error -1 running transaction
The command '/bin/sh -c microdnf install curl ca-certificates ${JAVA_PACKAGE} && microdnf update && microdnf clean all && mkdir /deployments && chown 1001 /deployments && chmod "g+rwX" /deployments && chown 1001:root /deployments && curl https://repo1.maven.org/maven2/io/fabric8/run-java-sh/${RUN_JAVA_VERSION}/run-java-sh-${RUN_JAVA_VERSION}-sh.sh -o /deployments/run-java.sh && chown 1001 /deployments/run-java.sh && chmod 540 /deployments/run-java.sh && echo "securerandom.source=file:/dev/urandom" >> /etc/alternatives/jre/lib/security/java.security' returned a non-zero code: 1
With the command docker build -f /src/main/docker/Dockerfile.jvm -t app:latest .
Here is the docker file
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.1
ARG JAVA_PACKAGE=java-11-openjdk-headless
ARG RUN_JAVA_VERSION=1.3.8
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en'
# Install java and the run-java script
# Also set up permissions for user `1001`
RUN microdnf install curl ca-certificates ${JAVA_PACKAGE} \
&& microdnf update \
&& microdnf clean all \
&& mkdir /deployments \
&& chown 1001 /deployments \
&& chmod "g+rwX" /deployments \
&& chown 1001:root /deployments \
&& curl https://repo1.maven.org/maven2/io/fabric8/run-java-sh/${RUN_JAVA_VERSION}/run-java-sh-${RUN_JAVA_VERSION}-sh.sh -o /deployments/run-java.sh \
&& chown 1001 /deployments/run-java.sh \
&& chmod 540 /deployments/run-java.sh \
&& echo "securerandom.source=file:/dev/urandom" >> /etc/alternatives/jre/lib/security/java.security
# Configure the JAVA_OPTIONS, you can add -XshowSettings:vm to also display the heap size.
ENV JAVA_OPTIONS="-Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
COPY target/lib/* /deployments/lib/
COPY target/*-runner.jar /deployments/app.jar
EXPOSE 8080
USER 1001
ENTRYPOINT [ "/deployments/run-java.sh" ]

Got the same error couple of days ago on a working project w/o changing any configurations. Just started failing. On my side managed to fix the problem after updating
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.1
to
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.3
in the Dockerfile.jvm

#isolona's solution worked on our project too. Upgrading from ubi-minimal:8.1 to ubi-minimal:8.3 for the base image fixed it.

How to download specific version in dockerfile?

I'm new at writing dockers. I need to download specific java version for it:
FROM alpine:3.9
RUN apk add bash && apk add openjdk8 && apk add R && apk add perl
this is working, however it downloads the latest version of java - 1.8.0_212.
I need the specific version 1.8.0_171, so I tried the below and it didn't work:
RUN apk add bash && apk add openjdk8=1.8.0_171 && apk add R && apk add perl
Anyone know how to get the specific version?

Try this out
ARG JAVA_VERSION_MAJOR=8
ARG JAVA_VERSION_MINOR=131
ARG JAVA_VERSION_BUILD=11
ARG JAVA_PACKAGE=server-jre
ARG JAVA_SHA256_SUM=a80634d17896fe26e432f6c2b589ef6485685b2e717c82cd36f8f747d40ec84b
ARG JAVA_URL_ELEMENT=d54c1d3a095b4ff2b6607d096fa8016
# Download and unarchive Java
RUN apk add --update curl && \
mkdir -p /opt
RUN wget -c --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-b${JAVA_VERSION_BUILD}/${JAVA_URL_ELEMENT}/${JAVA_PACKAGE}-${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-linux-x64.tar.gz
RUN mv ${JAVA_PACKAGE}-${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-linux-x64.tar.gz java.tar.gz
RUN gunzip -c java.tar.gz | tar -xf - -C /opt && rm -f java.tar.gz && \
ln -s /opt/jdk1.${JAVA_VERSION_MAJOR}.0_${JAVA_VERSION_MINOR} /opt/jdk
RUN apk add unzip
RUN curl -L -C - -b "oraclelicense=accept-securebackup-cookie" -O http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip && \
unzip jce_policy-8.zip -d /tmp && \
cp /tmp/UnlimitedJCEPolicyJDK8/*.jar /opt/jdk/jre/lib/security/ && \
rm -rf jce_policy-8.zip /tmp/UnlimitedJCEPolicyJDK8 && \
apk del curl && \
rm -rf /var/cache/apk/*
ENV JAVA_HOME /opt/jdk
ENV PATH ${PATH}:${JAVA_HOME}/bin

this worked for me :
ENV JAVA_VERSION 10.0.2 # your java version
ENV JAVA_HOME /java
RUN apt-get update
RUN apt-get install -y wget cron vim unzip bzip2 && apt-get clean
RUN wget -c --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-b${JAVA_VERSION_BUILD}/${JAVA_URL_ELEMENT}/${JAVA_PACKAGE}-${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-linux-x64.tar.gz \
&& mkdir -p $JAVA_HOME \
&& tar -xvzf jdk-10.0.2_linux-x64_bin.tar.gz -C $JAVA_HOME

Dockerfile: Multi-stage builds not working and any python3 + alpine + java image readily available

I am containerizing a component. It requires Linux + Python and Java.
Linux + Python - I am using Alpine 3.5 & Python 3.5 image.
Here is my Docker file.
Dockerfile:
#
# Health Monitoring Docker File
#
#ARG JAVA_BASE=1.0
#FROM csf-docker-delivered.repo.lab.pl.alcatel-lucent.com/java_base:17.6-1
#WORKDIR /etc/alternatives
#RUN CGO_ENABLED=0
FROM registry1-docker-io.repo.lab.pl.alcatel-lucent.com/python:3.5-alpine
#COPY --from=build /etc/alternatives/* .
#RUN rm -rf /var/cache/apk/* && \
# rm -rf /tmp/*
#
#RUN apk update
#
#RUN apk add --update --no-cache\
# build-base \
# openjdk8-jre \
# && pip install virtualenv \
# && rm -rf /var/cache/apk/*
#RUN apk update && apk upgrade && \
# apk add openjdk8 && \
# mkdir /tmp/tmprt && \
# cd /tmp/tmprt && \
# apk add zip && \
# unzip -q /usr/lib/jvm/default-jvm/jre/lib/rt.jar && \
# apk add zip && \
# zip -q -r /tmp/rt.zip . && \
# apk del zip && \
# cd /tmp && \
# mv rt.zip /usr/lib/jvm/default-jvm/jre/lib/rt.jar && \
# rm -rf /tmp/tmprt /var/cache/apk/* bin/jjs bin/keytool bin/orbd bin/pack200 bin/policytool \
# bin/rmid bin/rmiregistry bin/servertool bin/tnameserv bin/unpack200
COPY ./jdk-8u201-linux-x64.tar.gz /
RUN tar xf /jdk-8u201-linux-x64.tar.gz
ENV JAVA_HOME=/jdk1.8.0_201/bin
ENV NGDB_HOME /opt/nsn/ngdb
#RUN yum -y install openjdk-8-jdk-headless
#RUN apt-get install openjdk-8-jdk-headless
#COPY ./openjdk-8_8u181-b13.orig.tar.gz /
#RUN tar xf /openjdk-8_8u181-b13.orig.tar.gz
RUN mkdir -p /opt/nsn/ngdb/monitoring/scripts
RUN mkdir -p /opt/nsn/ngdb/monitoring/utils
RUN mkdir -p /var/local/monitoring/output
RUN mkdir -p /var/local/monitoring/work
RUN for directory in boundaryStatus postgresUsersCount backlogHadoop tableCount_Usage Dimension_Count tableCount_Day tableCount_Week tableCount_Month sendSummaryReport; do mkdir -p $directory;done
COPY ./utils/* /opt/nsn/ngdb/monitoring/utils/
COPY ./scripts/* /opt/nsn/ngdb/monitoring/scripts/
COPY ./conf/* /opt/nsn/ngdb/monitoring/conf/
COPY ./postgresql-9.2-1004.jdbc4.jar /opt/nsn/ngdb/monitoring/utils/
RUN mkdir -p /opt/nsn/ngdb/monitoring/python-dependencies
COPY ./html3-1.17.tar.gz /opt/nsn/ngdb/monitoring/python-dependencies
COPY ./py4j-0.10.8.1.zip /opt/nsn/ngdb/monitoring/python-dependencies
#RUN tar xf /opt/nsn/ngdb/monitoring/python-dependencies/html3-1.17.tar.gz
#WORKDIR /html3-1.17/
#RUN python /html3-1.17/setup.py install
RUN unzip /opt/nsn/ngdb/monitoring/python-dependencies/py4j-0.10.8.1.zip
WORKDIR /py4j-0.10.8.1
RUN python /py4j-0.10.8.1/setup.py install
But if i try to add Java Image & Python Alpine Image using "FROM", I am able to get only one thing in my container.
When i searched, i came across Multi stage builds, but when i did like below it is throwing a error "Unknown flag: from"
FROM csf-docker-delivered.repo.lab.pl.alcatel-lucent.com/java_base:17.6-1
WORKDIR /etc/alternatives
FROM registry1-docker-io.repo.lab.pl.alcatel-lucent.com/python:3.5-alpine
COPY --from=0 /etc/alternatives/* .
Can some one please help me out?
And addition to this,
I am trying to install py4j as a external module through which i am calling java classes.
i have set WORKDIR and then installation of py4j module goes smooth.
Later if i try to set WORKDIR to other module ex: html, and when i try
RUN python setup.py install
It gives error stating "No Such file or Directory"
Can you please help on this?

Setting Up DB2 in Airflow

I am trying to connect DB2 as a data connection for airflow which is residing in a Docker Container (realizing that this is not nativly supported). I am developing on a Mac
I have added the connectionas seen in the below screenshot where the URL is host:port/databse.
I am then going to the Data Profiling > Ad Hoc Query to try and test the connection and I get the below.
In order to make sure the drivers were available, I mounted the folder where the jdbc driver is located to /usr/local/airflow/drivers in the docker-compose file.
I also made sure to include the below packages in my requirements.txt as these were required when I query from a jupyter notebook.
sasl
thrift_sasl
jaydebeapi
jpype1
ibm_db
ibm_db_sa
I can't figure out what I'm missing.
I've been through:
Airflow documentation
Unable to setup a DB2 / DashDB JDBC Connection in Apache Airflow
Apache Airflow db2 Connection
Apache Airflow db2 Connection
https://github.com/puckel/docker-airflow
Lots of GitHub issues
and many more resources but can't find anything that solves this
Here is my current Dockerfile. As indicated in the comments, JVM isn't installed in the Dockerfile so that may be the issue.
# VERSION 1.10.1
# AUTHOR: Matthieu "Puckel_" Roisil
# DESCRIPTION: Basic Airflow container
# BUILD: docker build --rm -t puckel/docker-airflow .
# SOURCE: https://github.com/puckel/docker-airflow
FROM python:3.6-slim
LABEL maintainer="Puckel_"
# Never prompts the user for choices on installation/configuration of packages
ENV DEBIAN_FRONTEND noninteractive
ENV TERM linux
# Airflow
ARG AIRFLOW_VERSION=1.10.1
ARG AIRFLOW_HOME=/usr/local/airflow
ARG AIRFLOW_DEPS=""
ARG PYTHON_DEPS=""
ENV AIRFLOW_GPL_UNIDECODE yes
# Define en_US.
ENV LANGUAGE en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LC_ALL en_US.UTF-8
ENV LC_CTYPE en_US.UTF-8
ENV LC_MESSAGES en_US.UTF-8
# Java
RUN apt-get update && apt-get install -y openjdk-7-jre-headless wget \
&& apt-get clean
ENV JAVA_HOME /usr/lib/jvm/java-7-openjdk-amd64
RUN set -ex \
&& buildDeps=' \
freetds-dev \
libkrb5-dev \
libsasl2-dev \
libssl-dev \
libffi-dev \
libpq-dev \
git \
' \
&& apt-get update -yqq \
&& apt-get upgrade -yqq \
&& apt-get install -yqq --no-install-recommends \
$buildDeps \
freetds-bin \
build-essential \
default-libmysqlclient-dev \
apt-utils \
curl \
rsync \
netcat \
locales \
&& sed -i 's/^# en_US.UTF-8 UTF-8$/en_US.UTF-8 UTF-8/g' /etc/locale.gen \
&& locale-gen \
&& update-locale LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 \
&& useradd -ms /bin/bash -d ${AIRFLOW_HOME} airflow \
&& pip install -U pip setuptools wheel \
&& pip install pytz \
&& pip install pyOpenSSL \
&& pip install ndg-httpsclient \
&& pip install pyasn1 \
&& pip install apache-airflow[crypto,celery,postgres,hive,jdbc,mysql,ssh${AIRFLOW_DEPS:+,}${AIRFLOW_DEPS}]==${AIRFLOW_VERSION} \
&& pip install 'redis>=2.10.5,<3' \
&& if [ -n "${PYTHON_DEPS}" ]; then pip install ${PYTHON_DEPS}; fi \
&& apt-get purge --auto-remove -yqq $buildDeps \
&& apt-get autoremove -yqq --purge \
&& apt-get clean \
&& rm -rf \
/var/lib/apt/lists/* \
/tmp/* \
/var/tmp/* \
/usr/share/man \
/usr/share/doc \
/usr/share/doc-base
COPY script/entrypoint.sh /entrypoint.sh
COPY config/airflow.cfg ${AIRFLOW_HOME}/airflow.cfg
COPY requirements.txt ${AIRFLOW_HOME}/requirements.txt
RUN pip install --upgrade pip && pip install -r requirements.txt
RUN chown -R airflow: ${AIRFLOW_HOME}
EXPOSE 8080 5555 8793
USER airflow
WORKDIR ${AIRFLOW_HOME}
ENTRYPOINT ["/entrypoint.sh"]
CMD ["webserver"] # set default arg for entrypoint

The jpype1 module requires a JVM, accessible on your $PATH -- try installing one and try again.

Finally figured it out. Below is what I ended up using to get Java installed. then I just mounted the folder with my driver in it.
# Java
RUN mkdir -p /usr/share/man/man1 && \
(echo "deb http://http.debian.net/debian stretch main" > /etc/apt/sources.list.d/backports.list) && \
apt-get update -y \
&& apt-get install --no-install-recommends -y build-essential libkrb5-dev libsasl2-dev libffi-dev default-libmysqlclient-dev vim-tiny gosu krb5-user openjdk-8-jre openjdk-8-jdk-headless openjdk-8-jdk openjdk-8-jre-headless \
&& apt-get clean
RUN apt-get install unzip -y && \
apt-get autoremove -y