We Keep Coding

Connect to MSSQL using python jaydebeapi from inside a Docker Image

I have a Python code that's running properly on my system (Mac OS Catalina) but is failing when I am using it in my docker image. I am open to having a completely new dockerfile as well if that can work.
import pandas as pd
import jaydebeapi
import argparse
import json
from datetime import datetime
import os
def read_data():
MSSQL_DRIVER = "net.sourceforge.jtds.jdbc.Driver"
host = 'server_name'
port = '1433'
user = 'user'
password = 'password'
db_url = f"jdbc:jtds:sqlserver://{host}:{port};"
connection_properties = {
"user": user,
"password": password
}
jar_path = './jtds-1.3.1.jar'
connection = jaydebeapi.connect(MSSQL_DRIVER, db_url, connection_properties, jar_path)
query = 'SELECT TOP 10 * FROM table_name;'
data = pd.read_sql_query(query,connection)
print(data)
connection.close()
if __name__ == "__main__":
read_data()
I have the jar file next to my code so it can be picked up properly.
Here is my dockerfile:
FROM alpine:3.7
RUN apk update \
&& apk upgrade \
&& apk add --no-cache bash \
&& apk add --no-cache --virtual=build-dependencies unzip \
&& apk add --no-cache curl \
&& apk add --no-cache openjdk8-jre
RUN apk add --no-cache python3 \
&& python3 -m ensurepip \
&& pip3 install --upgrade pip setuptools \
&& rm -r /usr/lib/python*/ensurepip && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache
RUN apk add make automake gcc g++ subversion python3-dev
RUN pip install --trusted-host pypi.python.org flask
ENV JAVA_HOME="/usr/lib/jvm/java-1.8-openjdk"
EXPOSE 8000
WORKDIR /usr/src/app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY jtds-1.3.1.jar .
COPY server.py .
CMD ["python", "server.py"]
The error that I am getting is:
Error occurred during initialization of VM
Unable to load native library: Error loading shared library libjvm.so: No such file or directory (needed by /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libjava.so)
Please suggest me better dockerfile that I can use. Thanks for the help :)

Found the solution, all I needed to do was to add the following line to my dockerfile.
ENV LD_LIBRARY_PATH="/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server"
The rest is the same and it worked like a charm! I tried working with pymssql, pyodbc (FreeTDS one) and nothing seemed to work for me.

Are you running this on an apple silicon mac?
When I build your image, (i'm using an apple silicon mac), I am able to verify that /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libjava.so does not exist. It's instead in /usr/lib/jvm/java-1.8-openjdk/jre/lib/aarch64.
However, if I build the image and set the platform to amd like
FROM --platform=linux/x86_64 alpine:3.7
RUN apk update \
&& apk upgrade \
...
the file is there. I suspect this is your issue as well
alex#laptop ~/r/_/docker_test> docker run --entrypoint /bin/sh -it <image>
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
/ # ls /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64
jli libj2gss.so libjawt.so libnpt.so
jvm.cfg libj2krb5.so libjdwp.so libsplashscreen.so
libattach.so libj2pcsc.so libjsdt.so libsunec.so
libawt.so libj2pkcs11.so libjsig.so libunpack.so
libawt_headless.so libj2sctp.so libjsound.so libverify.so
libawt_xawt.so libjaas_unix.so libjsoundalsa.so libzip.so
libdt_socket.so libjava.so libmanagement.so server
libfontmanager.so libjava_crw_demo.so libmlib_image.so
libhprof.so libjavajpeg.so libnet.so
libinstrument.so libjavalcms.so libnio.so

How to download specific version in dockerfile?

I'm new at writing dockers. I need to download specific java version for it:
FROM alpine:3.9
RUN apk add bash && apk add openjdk8 && apk add R && apk add perl
this is working, however it downloads the latest version of java - 1.8.0_212.
I need the specific version 1.8.0_171, so I tried the below and it didn't work:
RUN apk add bash && apk add openjdk8=1.8.0_171 && apk add R && apk add perl
Anyone know how to get the specific version?

Try this out
ARG JAVA_VERSION_MAJOR=8
ARG JAVA_VERSION_MINOR=131
ARG JAVA_VERSION_BUILD=11
ARG JAVA_PACKAGE=server-jre
ARG JAVA_SHA256_SUM=a80634d17896fe26e432f6c2b589ef6485685b2e717c82cd36f8f747d40ec84b
ARG JAVA_URL_ELEMENT=d54c1d3a095b4ff2b6607d096fa8016
# Download and unarchive Java
RUN apk add --update curl && \
mkdir -p /opt
RUN wget -c --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-b${JAVA_VERSION_BUILD}/${JAVA_URL_ELEMENT}/${JAVA_PACKAGE}-${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-linux-x64.tar.gz
RUN mv ${JAVA_PACKAGE}-${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-linux-x64.tar.gz java.tar.gz
RUN gunzip -c java.tar.gz | tar -xf - -C /opt && rm -f java.tar.gz && \
ln -s /opt/jdk1.${JAVA_VERSION_MAJOR}.0_${JAVA_VERSION_MINOR} /opt/jdk
RUN apk add unzip
RUN curl -L -C - -b "oraclelicense=accept-securebackup-cookie" -O http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip && \
unzip jce_policy-8.zip -d /tmp && \
cp /tmp/UnlimitedJCEPolicyJDK8/*.jar /opt/jdk/jre/lib/security/ && \
rm -rf jce_policy-8.zip /tmp/UnlimitedJCEPolicyJDK8 && \
apk del curl && \
rm -rf /var/cache/apk/*
ENV JAVA_HOME /opt/jdk
ENV PATH ${PATH}:${JAVA_HOME}/bin

this worked for me :
ENV JAVA_VERSION 10.0.2 # your java version
ENV JAVA_HOME /java
RUN apt-get update
RUN apt-get install -y wget cron vim unzip bzip2 && apt-get clean
RUN wget -c --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-b${JAVA_VERSION_BUILD}/${JAVA_URL_ELEMENT}/${JAVA_PACKAGE}-${JAVA_VERSION_MAJOR}u${JAVA_VERSION_MINOR}-linux-x64.tar.gz \
&& mkdir -p $JAVA_HOME \
&& tar -xvzf jdk-10.0.2_linux-x64_bin.tar.gz -C $JAVA_HOME

Dockerfile: Multi-stage builds not working and any python3 + alpine + java image readily available

I am containerizing a component. It requires Linux + Python and Java.
Linux + Python - I am using Alpine 3.5 & Python 3.5 image.
Here is my Docker file.
Dockerfile:
#
# Health Monitoring Docker File
#
#ARG JAVA_BASE=1.0
#FROM csf-docker-delivered.repo.lab.pl.alcatel-lucent.com/java_base:17.6-1
#WORKDIR /etc/alternatives
#RUN CGO_ENABLED=0
FROM registry1-docker-io.repo.lab.pl.alcatel-lucent.com/python:3.5-alpine
#COPY --from=build /etc/alternatives/* .
#RUN rm -rf /var/cache/apk/* && \
# rm -rf /tmp/*
#
#RUN apk update
#
#RUN apk add --update --no-cache\
# build-base \
# openjdk8-jre \
# && pip install virtualenv \
# && rm -rf /var/cache/apk/*
#RUN apk update && apk upgrade && \
# apk add openjdk8 && \
# mkdir /tmp/tmprt && \
# cd /tmp/tmprt && \
# apk add zip && \
# unzip -q /usr/lib/jvm/default-jvm/jre/lib/rt.jar && \
# apk add zip && \
# zip -q -r /tmp/rt.zip . && \
# apk del zip && \
# cd /tmp && \
# mv rt.zip /usr/lib/jvm/default-jvm/jre/lib/rt.jar && \
# rm -rf /tmp/tmprt /var/cache/apk/* bin/jjs bin/keytool bin/orbd bin/pack200 bin/policytool \
# bin/rmid bin/rmiregistry bin/servertool bin/tnameserv bin/unpack200
COPY ./jdk-8u201-linux-x64.tar.gz /
RUN tar xf /jdk-8u201-linux-x64.tar.gz
ENV JAVA_HOME=/jdk1.8.0_201/bin
ENV NGDB_HOME /opt/nsn/ngdb
#RUN yum -y install openjdk-8-jdk-headless
#RUN apt-get install openjdk-8-jdk-headless
#COPY ./openjdk-8_8u181-b13.orig.tar.gz /
#RUN tar xf /openjdk-8_8u181-b13.orig.tar.gz
RUN mkdir -p /opt/nsn/ngdb/monitoring/scripts
RUN mkdir -p /opt/nsn/ngdb/monitoring/utils
RUN mkdir -p /var/local/monitoring/output
RUN mkdir -p /var/local/monitoring/work
RUN for directory in boundaryStatus postgresUsersCount backlogHadoop tableCount_Usage Dimension_Count tableCount_Day tableCount_Week tableCount_Month sendSummaryReport; do mkdir -p $directory;done
COPY ./utils/* /opt/nsn/ngdb/monitoring/utils/
COPY ./scripts/* /opt/nsn/ngdb/monitoring/scripts/
COPY ./conf/* /opt/nsn/ngdb/monitoring/conf/
COPY ./postgresql-9.2-1004.jdbc4.jar /opt/nsn/ngdb/monitoring/utils/
RUN mkdir -p /opt/nsn/ngdb/monitoring/python-dependencies
COPY ./html3-1.17.tar.gz /opt/nsn/ngdb/monitoring/python-dependencies
COPY ./py4j-0.10.8.1.zip /opt/nsn/ngdb/monitoring/python-dependencies
#RUN tar xf /opt/nsn/ngdb/monitoring/python-dependencies/html3-1.17.tar.gz
#WORKDIR /html3-1.17/
#RUN python /html3-1.17/setup.py install
RUN unzip /opt/nsn/ngdb/monitoring/python-dependencies/py4j-0.10.8.1.zip
WORKDIR /py4j-0.10.8.1
RUN python /py4j-0.10.8.1/setup.py install
But if i try to add Java Image & Python Alpine Image using "FROM", I am able to get only one thing in my container.
When i searched, i came across Multi stage builds, but when i did like below it is throwing a error "Unknown flag: from"
FROM csf-docker-delivered.repo.lab.pl.alcatel-lucent.com/java_base:17.6-1
WORKDIR /etc/alternatives
FROM registry1-docker-io.repo.lab.pl.alcatel-lucent.com/python:3.5-alpine
COPY --from=0 /etc/alternatives/* .
Can some one please help me out?
And addition to this,
I am trying to install py4j as a external module through which i am calling java classes.
i have set WORKDIR and then installation of py4j module goes smooth.
Later if i try to set WORKDIR to other module ex: html, and when i try
RUN python setup.py install
It gives error stating "No Such file or Directory"
Can you please help on this?

How to split docker file from monolithic deployment

In a single docker file I use to run Node and Java for the following:
run test script
run java executable (to bundle screenshots)
run my application
My understanding from the documentation is that this is bad practice, in the sense that this is a 'monolithic' deployment. I am supposed to split this in separate images based on the individual tasks. So, presumably, I would have 3 docker files.
Dockerfile 1: test script
FROM node:8
RUN node --version
RUN apt-get update && apt-get install -yq libgconf-2-4
# Note: this installs the necessary libs to make the bundled version of Chromium work
RUN apt-get update && apt-get install -y wget --no-install-recommends \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-unstable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst ttf-freefont \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get purge --auto-remove -y curl \
&& rm -rf /src/*.deb
# project repo
RUN git clone ...
WORKDIR /myApp/
RUN npm install
RUN npm i puppeteer
CMD ["node", "test.ts"]
Dockerfile 2: Java executable (bundle images)
FROM openjdk
RUN git clone -b ...
WORKDIR /myApp/
CMD ["java -jar", "ImageTester.jar"]
Dockerfile 3: Run app
FROM node:8
RUN node --version
RUN git clone -b ...
WORKDIR /myApp/
RUN npm install
EXPOSE 9999
CMD ["npm", "start"]
The question is, how does one exactly do this? How is a non monolithic deployment implemented in my case? How does one run 3 docker images inside one project?

Use docker-compose. In your docker-compose.yml file, you'll have 3 services pointing to the 3 different dockerfiles.
Ex:
version: '3'
services:
test:
build:
context: .
dockerfile: Dockerfile.test
images:
build:
context: .
dockerfile: Dockerfile.images
app:
build:
context: .
dockerfile: Dockerfile.app

Jenkins Slave can't read settings.xml

I have created a jenkins slave image for Docker, which I want to use to build all of my Java projects, however, I can't work out how to reference the .m2/settings.xml file to tell it where to pull from.
My Dockerfile is:
FROM openjdk:8
MAINTAINER Chris Hudson <chudson#amelco.co.uk>
RUN apt-get -qqy update && \
apt-get -y install openssh-server sudo
RUN useradd -m -u 1000 -s /bin/bash jenkins && \
mkdir -p /home/jenkins/.ssh && \
mkdir -p /home/jenkins/.m2 && \
echo jenkins:jenkins | chpasswd && \
mkdir -p /etc/sudoers.d/ && \
echo "jenkins ALL=(root) NOPASSWD: ALL" > /etc/sudoers.d/jenkins && \
chmod 440 /etc/sudoers.d/jenkins
COPY id_rsa.pub /home/jenkins/.ssh/authorized_keys
COPY settings.xml /home/jenkins/.m2/
RUN chown -R jenkins:jenkins /home/jenkins
RUN mkdir -p /var/run/sshd
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
But when I run the build, it attempts to pull from maven central, and not our local Artifactory instance, which is configured in the settings file.
This works when I run it on Jenkins Master, but I want to offload the builds to the slaves, but I can't work out how to configure Maven correctly.

I think that your workspace is mounted from the slave. and it's not read the .m2 from your container
you can try use his plugin - https://wiki.jenkins-ci.org/display/JENKINS/Config+File+Provider+Plugin to create the settings.xml and configure your MVN build step to use it.