We Keep Coding

ERROR: JAVA_HOME is set to an invalid directory: /usr/lib/jvm/java-8-openjdk-amd64

I'm fairly new to Docker and am struggling with JAVA_HOME not being seen in a Dockerfile. I get the titular error; which includes Please set the JAVA_HOME variable in your environment to match the location of your Java installation. & executor failed running [/bin/sh -c /opt/Android/cmdline-tools/latest/bin/sdkmanager --update]: exit code: 1 when it runs RUN /opt/Android/cmdline-tools/latest/bin/sdkmanager --update.
I feel like I'm at a loss but I think my issue is not knowing where the JDK is being installed to or knowing how to find it from a Dockerfile; I've tried echoing JAVA_HOME thinking I could see it while the image built but, again, no luck. Any help with this would be greatly appreciated. I've been pulling my hair out Googling & trying things. Thank you.
FROM node:12.12.0
ARG CMDLINE_TOOLS_VERSION=7583922
ARG ANDROID_BUILD_TOOLS=30.0.3
RUN apt-get -qqy update \
&& apt-get -qqy install \
python-dev \
--no-install-recommends
RUN apt-get install -y software-properties-common gcc
RUN apt-get update && apt-get install -y python3-pip
RUN pip3 install awscli
RUN apt-get install -y jq
RUN mkdir -p /usr/share/man/man1 /usr/share/man/man2
RUN apt-get update && apt-get install -y --no-install-recommends openjdk-8-jdk && apt-get clean;
ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
ENV PATH=$PATH:$JAVA_HOME/bin
RUN wget -q https://services.gradle.org/distributions/gradle-4.5.1-bin.zip && unzip gradle-4.5.1-bin.zip -d /opt && rm gradle-4.5.1-bin.zip
ENV GRADLE_HOME=/opt/gradle-4.5.1
ENV PATH=$PATH:/opt/gradle-4.5.1/bin
RUN wget https://dl.google.com/android/repository/commandlinetools-linux-${CMDLINE_TOOLS_VERSION}_latest.zip
RUN mkdir -p /opt/Android/cmdline-tools
RUN unzip commandlinetools-linux-7583922_latest.zip -d /opt/Android/cmdline-tools
RUN mv /opt/Android/cmdline-tools/cmdline-tools /opt/Android/cmdline-tools/latest
ENV ANDROID_HOME=/opt/Android
ENV PATH="$ANDROID_HOME/emulator:$ANDROID_HOME/tools:$ANDROID_HOME/tools/bin \
:$ANDROID_HOME/cmdline-tools/latest:$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/platform-tools:$PATH"
RUN /opt/Android/cmdline-tools/latest/bin/sdkmanager --update
RUN /opt/Android/cmdline-tools/latest/bin/sdkmanager --list
RUN /opt/Android/cmdline-tools/latest/bin/sdkmanager --list | grep build-tools
RUN echo y | /opt/Android/cmdline-tools/latest/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS}" "platform-tools" "platforms;android-30" "tools" >/dev/null
RUN yes | /opt/Android/cmdline-tools/latest/bin/sdkmanager --licenses
CMD ["yarn", "start"]

I'd suggest to use another base image. Java 11 is required to build for newer API levels:
FROM openjdk:11-jdk as builder
...
And then install Python3 and AWS CLI.
Working example: cloudbuild-android.
Or if you want to continue with your's, RUN which java would tell you where it's actually installed.

In a comment, you mentioned:
when I run [RUN ls -lart /usr/lib/jvm/java-8-openjdk-amd64], I get cannot access '/usr/lib/jvm/java-8-openjdk-amd64': No such file or directory
Maybe I'm missing something, but... doesn't that mean that the directory does not exist?
Either you got the path wrong, or (as another answer suggested) there's something in your Dockerfile, probably line endings, that's mangling your lines.
To discard the bit about line endings, try the same command but switching the parameters, i.e.:
RUN ls /usr/lib/jvm/java-8-openjdk-amd64 -lart
If now it whines about unrecognized options, then it's probably a line ending issue (which now affects the t option instead of the directory path).
However, if it still says No such file or directory, then you definitely should check your Java installation path.

I think your problem is in the line termination characters of your Dockerfile.
Your Dockerfile works in my computer, and when it comes to weird errors, I have been there.
My experience with Dockerfiles is that sometimes they are very picky in the Windows/Unix/Mac line terminators, so please make sure you save the Dockerfile with line terminators adequate for Unix machines.
I have used the method in this other answer to successfully replace Windows line terminators for Unix's on Windows using Powershell. I hope this helps!
The exact location of $JAVA_HOME can be scripted. If you don't feel like hardcoding that directory is a safe bet, replace your ENV line with this RUN: RUN export JAVA_HOME=$(readlink -f $(which java) | sed "s:/bin/java::"). This is how I set up my environment in all my Linux machines, and what it does is the following:
which java searches for java and will output /usr/bin/java, or something like that, which is a symbolic link.
readlink -f over the above will get you to the destination of the symbolic link.
piping such destination over to | sed "s:/bin/java::" will strip the trailing bin/java and get you the right value, no matter the version.
I also tried your Dockerfile with this change, and it does work too.

It looks like you had some network issues during your first docker build run. It failed to install java, but cached the layer with the attempt. I recommend you to run your build again without caching: docker build -t name --no-cache . and check logging of network operations.

How to bundle tesseract-ocr with a serverless Java application built for Azure Functions?

I am adding Apache Tika for extracting text out of documents and images (with TikaOcr) to an already existing service in the Azure Functions based on top of AppService. Now, Apache Tika requires tesseract to be installed in the machine locally. To overcome that, I used apt-get to set up (by ssh-ing) into the server but (from what I understand) the setup is performed on the base AppService layer. As a result, invocation of concurrent OCR commands really slow down my functions. Since there are no official binaries of Tesseract, I was wondering if any of the following is possible:
Bundle Tesseract with my Functions app
Build a docker image with Tesseract.
Build a multi-container docker app with a tesseract runtime image (tesseract-shadow/tesseract-ocr-re)
I have tried to build docker image (following instructions from here) with tesseract with the following dockerfile but Apache Tika fails to perform OCR with this.
ARG JAVA_VERSION=11
# This image additionally contains function core tools – useful when using custom extensions
#FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION-core-tools AS installer-env
FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION-build AS installer-env
RUN apt-get update && apt-get install -y tesseract-ocr
COPY . /src/functions-tika-extraction
RUN cd /src/functions-tika-extraction && \
mkdir -p /home/site/wwwroot && \
mvn clean package && \
cd ./target/azure-functions/ && \
cd $(ls -d */|head -n 1) && \
cp -a . /home/site/wwwroot
# This image is ssh enabled
FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION-appservice
# This image isn't ssh enabled
#FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION
ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
AzureFunctionsJobHost__Logging__Console__IsEnabled=true
COPY --from=installer-env ["/home/site/wwwroot", "/home/site/wwwroot"]
I'm fairly new to Docker and Azure Platform so I may be missing something here, but how can I get my Azure Functions to work with Tesseract using Docker or any other method?

After reading through the docker docs and getting to know some basics about docker, I could finally figure out that tesseract was in fact installed, below Azure AppService layer which somehow does not allow a container to access it. Tesseract can be made available to Azure Functions if installed in the uppermost layer by including it in the bottom of the Dockerfile as follows:
ARG JAVA_VERSION=11
FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION-build AS installer-env
# remove this line
# RUN apt-get update && apt-get install -y tesseract-ocr
COPY . /src/functions-tika-extraction
RUN cd /src/functions-tika-extraction && \
mkdir -p /home/site/wwwroot && \
mvn clean package && \
cd ./target/azure-functions/ && \
cd $(ls -d */|head -n 1) && \
cp -a . /home/site/wwwroot
# This image is ssh enabled
FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION-appservice
# add the line here
RUN apt-get update && apt-get install -y tesseract-ocr
ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
AzureFunctionsJobHost__Logging__Console__IsEnabled=true
COPY --from=installer-env ["/home/site/wwwroot", "/home/site/wwwroot"]
While it does satisfy my requirement of bundling tesseract-ocr with Azure Functions Java application, the invocation is still very slow unfortunately.

Run Maven project inside Docker Container with run time parameters

I have a Testng Selenium Project that is build using Maven. I am running this maven project using Maven Surefire plugin like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<forkMode>never</forkMode>
<useFile>true</useFile>
<testFailureIgnore>true</testFailureIgnore>
<!-- Suite testng xml file to consider for test execution-->
<suiteXmlFiles>
<suiteXmlFile>${suiteXmlFile}</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
What I need to do?
I need to run this Selenium project inside Docker Container. I need to move the complete source code to the container and run it from there. While running, I will be passing the path of testng xml file, that particular test alone should run. Post run, I need to take the result from docker image to local system (which we can do using docker cp ...).
What I have done so far?
I have created a docker image with maven, chrome, chromedriver. At run time, I am passing the testng XML file path and as expected that particular test case alone is running. But....
Once the program gets completed, the docker container is getting closed. docker ps shows no running containers. So, am not able to see the report.
What I want?
So, I want a way to avoid container from getting closed after the execution so that I can go into the docker container and see the report.
My Dockerfile:
FROM kshivaprasad/java
RUN apt-get update
RUN apt-get upgrade --fix-missing -y
RUN apt-get install -y curl
RUN apt-get install -y p7zip \
p7zip-full \
unace \
zip \
unzip
# Install Chrome for Selenium
RUN curl http://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_83.0.4103.116-1_amd64.deb -o /chrome.deb
RUN dpkg -i /chrome.deb || apt-get install -yf
RUN rm /chrome.deb
# Install chromedriver for Selenium
RUN mkdir -p /app/bin
RUN curl https://chromedriver.storage.googleapis.com/83.0.4103.39/chromedriver_linux64.zip -o /tmp/chromedriver.zip \
&& unzip /tmp/chromedriver.zip -d /app/bin/ \
&& rm /tmp/chromedriver.zip
ARG MAVEN_VERSION=3.6.3
# 2- Define a constant with the working directory
ARG USER_HOME_DIR="/root"
# 3- Define the SHA key to validate the maven download
ARG SHA=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
# 4- Define the URL where maven can be downloaded from
ARG BASE_URL=http://apachemirror.wuchna.com/maven/maven-3/${MAVEN_VERSION}/binaries
# 5- Create the directories, download maven, validate the download, install it, remove downloaded file and set links
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& echo "Downlaoding maven" \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
\
&& echo "Checking download hash" \
&& echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c - \
\
&& echo "Unziping maven" \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
\
&& echo "Cleaning and setting links" \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
# 6- Define environmental variables required by Maven, like Maven_Home directory and where the maven repo is located
ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
COPY src /app/src
COPY pom.xml /app
COPY testng /app/testng
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /app/bin/chromedriver
#RUN mvn -f /app/pom.xml clean package
ENTRYPOINT ["/entrypoint.sh"]
That entrypoint.sh file is used to send the argument at run time. It consists of:
#!/bin/sh
mvn -f /app/pom.xml clean install -DsuiteXmlFile=$1
How I run this?
docker build -t my_image .
docker run -it my_image module/testng.xml

This process would be very time consuming when it comes to the quick time to market. I would suggest use zalenium https://opensource.zalando.com/zalenium/ in order to run your selenium script in the docker container and it has various features too like viewing the ongoing execution and retrieving the reports.

I just need to need bash command in the shell file (entrypoint.sh). It worked fine for me.
#!/bin/sh
mvn -f /app/pom.xml clean install -DsuiteXmlFile=$1
/bin/bash

How do I setup liquibase on Heroku with NodeJs?

All,
Attempting to install liquibase on Heroku so that we can update our database as part of our NodeJs deployments but running into (I'm guessing) classpath errors.
app structure
bower_components
liquibase
- install
- update
node_modules
src
package.json
...
Heroku can run a postinstall script where we run the liquibase install
package.json
"scripts": {
"postinstall": "./liquibase/install && ./liquibase/update && ./node_modules/bower/bin/bower install && ./node_modules/grunt-cli/bin/grunt bundle --force",
}
liquibase install script. Downloads the postgresql.jar and the liquibase executable and puts them in the liquibase folder.
#!/usr/bin/env bash
wget https://github.com/liquibase/liquibase/releases/download/liquibase-parent-3.5.3/liquibase-3.5.3-bin.tar.gz
mkdir -p ~/liquibase
tar -zx -C ~/liquibase -f liquibase-3.5.3-bin.tar.gz
wget https://jdbc.postgresql.org/download/postgresql-42.1.1.jar
mkdir -p ~/lib
mv postgresql-42.1.1.jar ~/lib/postgresql.jar
After the install, we attempt to run the liquibase update (./liquibase/update)
liquibase \
--logLevel="info" \
--driver="org.postgresql.Driver" \
--classpath="$~/lib/postgresql.jar" \
--changeLogFile="liquibase.xml" \
--url="jdbc:postgresql://$HOST:$PORT/$DATABASE" \
--username="$USERNAME" \
--password="$PASSWORD" \
update
But I get the error
liquibase: command not found
That makes me think liquibase isn't on the path
So I do this
export PATH=${PATH}:~/liquibase
Which gives me this error
Error: Could not find or load main class null
Yay for Java :( So no Java or classpath isn't set?
java -v
java version "1.7.0_151"
OpenJDK Runtime Environment (IcedTea 2.6.11) (7u151-2.6.11-0ubuntu1.14.04.1)
OpenJDK 64-Bit Server VM (build 24.151-b01, mixed mode)
echo $JAVA_HOMENothing
which java/usr/bin/java
Maybe I should set $JAVA_HOME=/usr/bin/java
Again, nothing.
At this point, I've got no clue on how to proceed. Any help would be appreciated.
* SOLVED *
Adding the buildpack helped. Also needed to modify the update script
HOME=~
java -jar $HOME/liquibase/liquibase.jar \
--logLevel="info" \
--driver="org.postgresql.Driver" \
--classpath="$HOME/lib/postgresql.jar" \
--changeLogFile="liquibase.xml" \
--url="jdbc:postgresql://$HOST:$PORT/$DATABASE" \
--username="$USERNAME" \
--password="$PASSWORD" \
update
and then the postinstall script
"scripts": {
"postinstall": "cd liquibase && ./install && ./update && cd .. && ./node_modules/bower/bin/bower install && ./node_modules/grunt-cli/bin/grunt bundle --force"
}

First, you'll want to add the JVM buildpack to your app:
$ heroku buildpacks:add -i 1 heroku/jvm
This will install JDK 8 (instead of the default JDK 7), set JAVA_HOME correctly, and even set JDBC_DATABASE_URL (which you can use directly in your --url option).
The message "Error: Could not find or load main class null" suggests that the java command created by the liquibase script (the one you are running) is either malformed, or incomplete. I think this may be due to the option --classpath="$~/lib/postgresql.jar", which looks odd. Or the location of the liquibase.jar relative to the script.
I think you want your classpath option to look like:
--classpath="/app/path/to/classes:/app/lib/postgresql.jar"
If you still have trouble, I would try running the liquibase.jar directly instead of using the script, like:
java -jar liquibase.jar \
--logLevel="info" \
--driver="org.postgresql.Driver" \
--classpath="/app/path/to/classes:/app/lib/postgresql.jar" \
--changeLogFile="liquibase.xml" \
--url="$JDBC_DATABASE_URL" \
--username="$JDBC_DATABASE_USERNAME" \
--password="$JDBC_DATABASE_PASSWORD" \
update

Jenkins Slave can't read settings.xml

I have created a jenkins slave image for Docker, which I want to use to build all of my Java projects, however, I can't work out how to reference the .m2/settings.xml file to tell it where to pull from.
My Dockerfile is:
FROM openjdk:8
MAINTAINER Chris Hudson <chudson#amelco.co.uk>
RUN apt-get -qqy update && \
apt-get -y install openssh-server sudo
RUN useradd -m -u 1000 -s /bin/bash jenkins && \
mkdir -p /home/jenkins/.ssh && \
mkdir -p /home/jenkins/.m2 && \
echo jenkins:jenkins | chpasswd && \
mkdir -p /etc/sudoers.d/ && \
echo "jenkins ALL=(root) NOPASSWD: ALL" > /etc/sudoers.d/jenkins && \
chmod 440 /etc/sudoers.d/jenkins
COPY id_rsa.pub /home/jenkins/.ssh/authorized_keys
COPY settings.xml /home/jenkins/.m2/
RUN chown -R jenkins:jenkins /home/jenkins
RUN mkdir -p /var/run/sshd
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
But when I run the build, it attempts to pull from maven central, and not our local Artifactory instance, which is configured in the settings file.
This works when I run it on Jenkins Master, but I want to offload the builds to the slaves, but I can't work out how to configure Maven correctly.

I think that your workspace is mounted from the slave. and it's not read the .m2 from your container
you can try use his plugin - https://wiki.jenkins-ci.org/display/JENKINS/Config+File+Provider+Plugin to create the settings.xml and configure your MVN build step to use it.