We Keep Coding

Nifi stuck at "Launched Apache NiFi with Process ID 41"

I have created a docker image for Nifi 1.14.0 with Alpine OS and jdk-8. I have been able to successfully build it but when I execute "./nifi.sh run" command in docker to run Nifi but it gets stuck at "Launched Apache NiFi with Process ID 41" and doesn't move beyond this unless I do Ctrl+C which shutdowns Nifi. Upon checking the nifi-app.log, it is seen that nar files have been unpacked and no errors are shown. Nifi-bootstrap.log and nifi-user.log don't show any errors eithers. What could be the possible reason and solution for this such that Nifi launches properly?
Thanks in advance!!
My Dockerfile is:
#getting base image Alpine
FROM alpine
ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk/jre
ENV PATH $JAVA_HOME/bin:$PATH
ENV NIFI_VERSION 1.14.0
ENV NIFI_HOME /opt/nifi
RUN apk --update add bash git wget ca-certificates sudo openssh rsync openjdk8 zip && \
rm -rf /var/cache/apk/* && \
rm -rf /opt && \
mkdir -p /opt
RUN wget https://dlcdn.apache.org/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz && \
tar xzf nifi-$NIFI_VERSION-bin.tar.gz -C /opt/ && \
ln -s /opt/nifi-$NIFI_VERSION $NIFI_HOME && \
rm nifi-$NIFI_VERSION-bin.tar.gz
RUN apk update --no-cache && apk upgrade --no-cache && apk add --no-cache bash libstdc++ libc6-compat
VOLUME ["$NIFI_HOME/conf"]
EXPOSE 8080
WORKDIR $NIFI_HOME

Unable to run jmeter test in docker, getting class not found exception

I am trying to run jmeter through docker and I am getting a class not found exception.
The maven dependency is there and I am able to run it locally with command line.
Here is the exception I am getting:
2021-02-13 17:48:35,315 ERROR o.a.j.JMeter: Uncaught exception in thread Thread[Thread Group 1-2,5,main]
java.lang.NoClassDefFoundError: Could not initialize class com.amazonaws.auth.DefaultAWSCredentialsProviderChain
at Utils.TestParameters.<init>(TestParameters.java:62) ~[Performance-1.0-SNAPSHOT.jar:?]
at InitTest.InitTest(InitTest.java:96) ~[Performance-1.0-SNAPSHOT.jar:?]
at InitTest.runTest(InitTest.java:33) ~[Performance-1.0-SNAPSHOT.jar:?] at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:197) ~[ApacheJMeter_java.jar:5.4.1] at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:635) ~[Performance-1.0-SNAPSHOT.jar:?] at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558) ~[Performance-1.0-SNAPSHOT.jar:?] at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489) ~[Performance-1.0-SNAPSHOT.jar:?] at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256) ~[Performance-1.0-SNAPSHOT.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
As I said, I am able to run this locally but for the sake of this ticket, I am including
the above dependency that is in the projects pom.xml:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>1.11.924</version>
</dependency>
My dockerfile includes the following command which copies my jar to the lib location (this jar includes the above dependency):
COPY Performance-1.0-SNAPSHOT.jar /opt/apache-jmeter-5.4.1/lib/Performance-1.0-SNAPSHOT.jar
I am new to docker so apologies if I am missing something basic here.
EDIT:
Here is my dockerfile:
FROM ubuntu:latest
# setup jmeter version to use
ARG JMETER_VERSION="5.4.1"
ARG JMETER_PLUGINS_MANAGER_VERSION="1.3"
ARG CMDRUNNER_VERSION="2.2"
ENV JMETER_HOME /opt/apache-jmeter-${JMETER_VERSION}
ENV JMETER_BIN ${JMETER_HOME}/bin
ENV MIRROR_HOST https://archive.apache.org/dist/jmeter
ENV JMETER_DOWNLOAD_URL ${MIRROR_HOST}/binaries/apache-jmeter-${JMETER_VERSION}.tgz
ENV JMETER_PLUGINS_DOWNLOAD_URL https://repo1.maven.org/maven2/kg/apc
ENV JMETER_PLUGINS_FOLDER ${JMETER_HOME}/lib/ext/
ENV PATH $PATH:$JMETER_BIN
# Install Everything.
RUN \
sed -i -e 's/# \(.*multiverse$\)/\1/g' /etc/apt/sources.list && \
apt-get update && \
apt-get install -y build-essential && \
apt-get install -y software-properties-common && \
apt-get install -y byobu openjdk-8-jre curl git htop man unzip vim wget python3-pip && \
mkdir -p /tmp/dependencies && \
curl -L --silent ${JMETER_DOWNLOAD_URL} > /tmp/dependencies/apache-jmeter-${JMETER_VERSION}.tgz && \
mkdir -p /opt && \
tar -xzf /tmp/dependencies/apache-jmeter-${JMETER_VERSION}.tgz -C /opt && \
rm -rf /tmp/dependencies && \
rm -rf /var/lib/apt/lists/*
# Install jmeter lib and dependency jars
RUN curl -L --silent ${JMETER_PLUGINS_DOWNLOAD_URL}/jmeter-plugins-manager/${JMETER_PLUGINS_MANAGER_VERSION}/jmeter-plugins-manager-${JMETER_PLUGINS_MANAGER_VERSION}.jar -o ${JMETER_PLUGINS_FOLDER}/jmeter-plugins-manager-${JMETER_PLUGINS_MANAGER_VERSION}.jar
RUN curl -L --silent ${JMETER_PLUGINS_DOWNLOAD_URL}/cmdrunner/${CMDRUNNER_VERSION}/cmdrunner-${CMDRUNNER_VERSION}.jar -o ${JMETER_HOME}/lib/cmdrunner-${CMDRUNNER_VERSION}.jar && \
java -cp ${JMETER_PLUGINS_FOLDER}/jmeter-plugins-manager-${JMETER_PLUGINS_MANAGER_VERSION}.jar org.jmeterplugins.repository.PluginManagerCMDInstaller && \
PluginsManagerCMD.sh install jpgc-cmd=2.2,jpgc-dummy=0.4,jpgc-filterresults=2.2,jpgc-synthesis=2.2,jpgc-graphs-basic=2.0 \
&& jmeter --version \
&& PluginsManagerCMD.sh status \
RUN ln -nsf /usr/bin/pip3 /usr/bin/pip
RUN ln -nfs /usr/bin/python3 /usr/bin/python
RUN pip3 install awscli && pip3 install xmltodict
# sheel script has script to convert JTL to CSV
COPY run.sh /
COPY performance.jmx /performance.jmx
COPY Performance-1.0-SNAPSHOT.jar /opt/apache-jmeter-5.4.1/lib/Performance-1.0-SNAPSHOT.jar
# Set environment variables.
ENV HOME /root
# Final cleanup
RUN apt-get --purge autoremove
RUN ["chmod", "+x", "/run.sh"]
# Define working directory.
WORKDIR /
CMD /run.sh
Here is how I start jmeter:
echo "START Running Jmeter on `date`"
JVM_ARGS="-Xms2048m -Xmx8192m" jmeter -n -t /performance.jmx -l /jmeter.jtl 2>&1
java -jar /opt/apache-jmeter-5.4.1/lib/cmdrunner-2.2.jar --tool Reporter --plugin-type AggregateReport --input-jtl /jmeter.jtl --generate-csv /results/results.csv 2>&1
cat /results/results.csv
echo "END Running Jmeter on `date`"

I haven't worked with Docker much, but my suspicion is that the Apache Commons Logging JAR is somehow missing or not on the classpath.
The error java.lang.NoClassDefFoundError: Could not initialize class <SomeClassName> doesn't mean that the class SomeClassName couldn't be found. It means that the JVM found the class but encountered an error performing static initialization. Furthermore, this exception is only thrown if there has already been a previous failure to perform static initialization on this class. You may or may not see a different error earlier on in your logs.
Static initialization of the class DefaultAWSCredentialsProviderChain consists of creating an INSTANCE field using instances of various classes that appear to be in the same package. Of these, EC2ContainerCredentialsProviderWrapper uses an Apache Commons Logging logger, but all the rest seem to only use built-in Java classes or classes in packages under com.amazonaws.
Static initialization for the superclass AWSCredentialsProviderChain also needs to take place, and that also includes a logger from Apache Commons Logging.

Your "dependency" declaration doesn't necessarily mean that your Performance-1.0-SNAPSHOT.jar will contain aws-java-sdk-core-1.11.924.jar (as well as its dependencies)
So I would recommend checking your Performance-1.0-SNAPSHOT.jar (.jar files are normal .zip archives) and ensure that it contains all the dependencies of the AWS Java SDK your test needs and if it doesn't - you will need to amend your pom.xml to build it as a fat (uber) jar
Alternative option is using JMeter Maven Plugin inside your docker file, it will automatically download JMeter, plugins, dependencies, etc. so you won't have to do this manually.

As #luke-woodward suggested, the issue was eventually not with SomeClassName but with the JVM having an issue instantiating it. There was a version mismatch between different dependencies that had to be addressed. My suggestion to anyone who encounters this problem is to look for an underlying reason why SomeClassName cannot be instantiated.

create a dockerfile to run python and groovy app

I am working on a project which is using both python and groovy to scrape data from websites and do some engineering on that data.
I want to create a dockerfile which should have a python(3.6.5) as base image and java8 and groovy should be installed on it to run my code.
the dockerfile I have right now is working for all the python codes(image : FROM python:3.6.5) but failing for groovy script and I cant find a solution which I can use to install groovy in dockerfile.
is there anyone who has a dockerfile solving this part problem ?
##########docker file below#############
FROM python:3.6.5
RUN sh -c "ls /usr/local/lib"
RUN sh -c "cat /etc/*-release"
# Contents of requirements.txt each on a separate line for incremental builds
RUN pip install SQLAlchemy==1.2.7
RUN pip install pandas==0.23.0
RUN pip uninstall bson
RUN pip install pymongo
RUN pip install openpyxl==2.5.3
RUN pip install joblib
RUN pip install impyla
RUN sh -c "mkdir -p /src/dateng"
ADD . /src/dateng
RUN sh -c "ls /src/dateng"
WORKDIR /src/dateng/
ENTRYPOINT ["python", "/src/dateng/_aws/trigger.py"]

You don't need to use sh -c command, just RUN command and we should not use a RUN instruction per command, intead we should group them in only one RUN, because each RUN is a separated layer in the docker image, thus increasing the final size of it.
Possible Solution
Inspired in this Dockerfile I use for a Python demo:
FROM python:3.6.5
ARG CONTAINER_USER="python"
ARG CONTAINER_UID="1000"
# Will not prompt for questions
ENV DEBIAN_FRONTEND=noninteractive \
CONTAINER_USER=python \
CONTAINER_UID=1000
RUN apt update && \
apt -y upgrade && \
apt -y install \
ca-certificates \
locales \
tzdata \
inotify-tools \
python3-pip \
groovy && \
locale-gen en_GB.UTF-8 && \
dpkg-reconfigure locales && \
#https://github.com/guard/listen/wiki/Increasing-the-amount-of-inotify-watchers
printf "fs.inotify.max_user_watches=524288\n" >> /etc/sysctl.conf && \
useradd -m -u ${CONTAINER_UID} -s /bin/bash ${CONTAINER_USER}
ENV LANG=en_GB.UTF-8 \
LANGUAGE=en_GB:en \
LC_ALL=en_GB.UTF-8
USER ${CONTAINER_USER}
RUN pip3 install \
fSQLAlchemy==1.2.7 \
pandas==0.23.0 \
pymongo \
openpyxl==2.5.3 \
joblib \
impyla && \
pip3 uninstall bson
# pip install will put the executables under ~/.local/bin
ENV PATH=/home/"${CONTAINER_USER}"/.local/bin:$PATH
WORKDIR /home/${CONTAINER_USER}/workspace
ADD . /home/${CONTAINER_USER}/dataeng
EXPOSE 5000
ENTRYPOINT ["python", "/home/python/dateng/_aws/trigger.py"]
NOTE: I am behind a corporate firewall, therefore I cannot test building this image as it is now, because I would need to add stuff to it that you don't need. Let me know if something doesn't work for you and I will work it out from home.

How to split docker file from monolithic deployment

In a single docker file I use to run Node and Java for the following:
run test script
run java executable (to bundle screenshots)
run my application
My understanding from the documentation is that this is bad practice, in the sense that this is a 'monolithic' deployment. I am supposed to split this in separate images based on the individual tasks. So, presumably, I would have 3 docker files.
Dockerfile 1: test script
FROM node:8
RUN node --version
RUN apt-get update && apt-get install -yq libgconf-2-4
# Note: this installs the necessary libs to make the bundled version of Chromium work
RUN apt-get update && apt-get install -y wget --no-install-recommends \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-unstable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst ttf-freefont \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get purge --auto-remove -y curl \
&& rm -rf /src/*.deb
# project repo
RUN git clone ...
WORKDIR /myApp/
RUN npm install
RUN npm i puppeteer
CMD ["node", "test.ts"]
Dockerfile 2: Java executable (bundle images)
FROM openjdk
RUN git clone -b ...
WORKDIR /myApp/
CMD ["java -jar", "ImageTester.jar"]
Dockerfile 3: Run app
FROM node:8
RUN node --version
RUN git clone -b ...
WORKDIR /myApp/
RUN npm install
EXPOSE 9999
CMD ["npm", "start"]
The question is, how does one exactly do this? How is a non monolithic deployment implemented in my case? How does one run 3 docker images inside one project?

Use docker-compose. In your docker-compose.yml file, you'll have 3 services pointing to the 3 different dockerfiles.
Ex:
version: '3'
services:
test:
build:
context: .
dockerfile: Dockerfile.test
images:
build:
context: .
dockerfile: Dockerfile.images
app:
build:
context: .
dockerfile: Dockerfile.app

Jenkins Slave can't read settings.xml

I have created a jenkins slave image for Docker, which I want to use to build all of my Java projects, however, I can't work out how to reference the .m2/settings.xml file to tell it where to pull from.
My Dockerfile is:
FROM openjdk:8
MAINTAINER Chris Hudson <chudson#amelco.co.uk>
RUN apt-get -qqy update && \
apt-get -y install openssh-server sudo
RUN useradd -m -u 1000 -s /bin/bash jenkins && \
mkdir -p /home/jenkins/.ssh && \
mkdir -p /home/jenkins/.m2 && \
echo jenkins:jenkins | chpasswd && \
mkdir -p /etc/sudoers.d/ && \
echo "jenkins ALL=(root) NOPASSWD: ALL" > /etc/sudoers.d/jenkins && \
chmod 440 /etc/sudoers.d/jenkins
COPY id_rsa.pub /home/jenkins/.ssh/authorized_keys
COPY settings.xml /home/jenkins/.m2/
RUN chown -R jenkins:jenkins /home/jenkins
RUN mkdir -p /var/run/sshd
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
But when I run the build, it attempts to pull from maven central, and not our local Artifactory instance, which is configured in the settings file.
This works when I run it on Jenkins Master, but I want to offload the builds to the slaves, but I can't work out how to configure Maven correctly.

I think that your workspace is mounted from the slave. and it's not read the .m2 from your container
you can try use his plugin - https://wiki.jenkins-ci.org/display/JENKINS/Config+File+Provider+Plugin to create the settings.xml and configure your MVN build step to use it.