Hi I need to dockerize a system. the way I have to do this like below
steps:
up dynamodb local instance ( just for up ).
run a custom script to create tables ( have to go through this to create the tables ).
then run the system.
I wrote a compose file also. the way I did that was, like below
version: "3"
services:
dynamodb:
image: amazon/dynamodb-local
ports:
- "8000:8000"
networks:
- custom-network
volumes:
- "db-data:/home/dynamodblocal/data"
app:
container_name: my-app
build:
context: .
dockerfile: Dockerfile
args:
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
URL: ${URL}
env_file:
- docker.env
depends_on:
- dynamodb
networks:
- custom-network
volumes:
db-data:
networks:
custom-network:
docker file as below. ( sorry had to hide sensitive details )
FROM debian:buster
ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY
ARG URL
RUN echo "deb http://ftp.us.debian.org/debian sid main" >> /etc/apt/sources.list
RUN apt-get update
RUN apt-get install openjdk-8-jdk maven awscli -y
RUN aws s3 cp ${URL} db-updater.jar
RUN echo local > input
# there are few lines of configs that wrote to input file
RUN cat input | java -jar db-updater.jar http://dynamodb:8000
WORKDIR /opt/app
COPY . .
RUN mvn package
EXPOSE 8080
CMD ["java","-cp","./app/target/app-1.0.0.jar:./app/target/lib/*"]
my problem is looks like dynamodb do not start before the script run. so script throws a error as can't connect to server.
if I could write a custom a dynamodb with executed script that is also great. please help
Commands in a Dockerfile can never interact with other Docker containers. The general pattern is that an image is built once and reused, so you could delete and recreate your DynamoDB container, or run the same image on a different system, and the database setup wouldn't have happened. Mechanically, the Dockerfile runs in an environment where it's not connected to the Compose networking system, so attempts at connecting between containers will generally fail with a "no such host" error.
A typical pattern is to use an entrypoint script to do first-time setup when the container launches. For example, you could write a simple shell script:
#!/bin/sh
# Seed the database
java -jar db-updater.jar http://dynamodb:8000 < input
# Run whatever the main container command is
exec "$#"
You can then include this in your Dockerfile:
COPY entrypoint.sh . # probably included in the `COPY . .` line
ENTRYPOINT ["./entrypoint.sh"] # must be JSON-array syntax
# replaces `RUN java -jar db-updater.jar`
CMD ["java", "-cp", ...] # as in the current Dockerfile
If you only need this to run once when you first set up the container stack, you could also seed the data on your host.
# Outside Docker
aws s3 cp ... db-updater.jar
./make-seed-data.sh > input
# Start the DynamoDB container (only)
docker-compose up -d dynamodb
# Load the seed data
java -jar db-updater.jar -url http://localhost:8000 < input
# Now start the rest of the application
docker-compose up -d
This would let you remove the code to build the input file and download the updater tool from your Dockerfile. It would also let you remove the AWS credentials from the build sequence (very important: it may be possible to find them in plain text looking at the image's docker history).
Related
I'm trying to create a CI/CD Pipeline for a simple java/maven project.
The runner that I'm using is a docker runner.
I'm using a dockerfile to create a container which installs maven/java/etc.. and in this container the program should be tested.
Sorry for the question but I am new to CI/CD Pipelines in GitLab.
GitHub works just fine have a look: https://github.com/ni920/CICD-Test
Thank you
Here are the CI logs
...
Executing "step_script" stage of the job script
$ docker build --build-arg JAVA_VERSION=openjdk7
/bin/sh: eval: line 95: docker: not found
Cleaning up file based variables
ERROR: Job failed: exit code 127
Thats the .gitlab-ci.yml
stages:
- java7
# - java11
# - deploy
java7:
stage: java7
script:
- docker build --build-arg JAVA_VERSION=openjdk7
# tags:
# - docker
#java11:
# stage: java11
# script:
# - docker build --build-arg JAVA_VERSION=openjdk11
# tags:
# - docker
Thats the dockerfile
# Pull base image.
FROM alpine as build
ARG MAVEN_VERSION=3.6.1
ARG USER_HOME_DIR="/root"
ARG JAVA_VERSION=openjdk7
ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
ENV HTTP_PROXY=#comment
ENV HTTPS_PROXY=#comment
# Install Java.
RUN apk --update --no-cache add JAVA_VERSION curl
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
# Define working directory.
WORKDIR /data
# Define commonly used JAVA_HOME variable
ENV JAVA_HOME /usr/lib/jvm/default-jvm/
# Define default command.
CMD ["mvn", "--version"]
Running your pipelines using the Docker executor means that your jobs will run in a Docker container, but not that you will be able to execute docker commands.
If you need to run docker commands inside a GitLab CI job (read "inside a container") you will need Docker-in-Docker (often abbreviated DinD). It is a vast topic on itself but you can get started with GitLab CI's documentation: Use Docker to build Docker images
I always use DinD and have a minimal setup in my gitlab-ci.yml.
Using a docker image as a default:
image: docker:19.03.13
Define a default variable for TLS certificates:
variables:
DOCKER_TLS_CERTDIR: "/certs"
Then use a docker image as a service to enable DinD:
services:
- name: docker:19.03.13-dind
alias: docker
I wrote a few posts about using Docker-in-Docker on GitLab CI that you may find useful, but I still recommend to extensively read GitLab's documentation before reading them.
I am trying to set up a very simple Docker container, that will execute a Jar file. I want an image that will run a simple Jar. I dont need anything else special.
So far my docker-compose.yml looks like this, but it doesn't start correctly:
version: "3.3"
services:
myapp:
image: openjdk:8
container_name: "myapp"
restart: always
ports:
- 8091:8091
volumes:
- "./meanwhile-in-hell.jar:/app.jar"
command: ['java', '-jar', '/app.jar']
STATUS
Restarting (1) 2 seconds ago
Using the command: ['java', '-jar', '/app.jar'] option, I see this in the Docker logs:
Error: Invalid or corrupt jarfile /app.jar
If I change to use entrypoint: [ "sh", "-c", "java -jar /app.jar" ], I see the same error.
The Jar file is absolutely fine and not corrupt. I have run it manually on another tomcat:8-alpine container I have running and it starts up successfully.
I want to know how to configure a docker setup (docker-compose) in order to supply a configuration file which is consumed by my Spring boot application.
The configuration file is called services.xml which resides in the applications's /lib/conf directory. The file is deployed with the default configuration, however I want the file in host so that whenever I need to change the configuration, I should edit in host and the container would read the updated file.
The docker-compose.yml
version: '3.1'
services:
my-app:
image: my-app
container_name: my-app
# restart: always
ports:
- 8443:8443
volumes:
- ./my-app/conf:/opt/lib/my-app/lib/conf:rw
Expected results after running: docker-compose up
I expect this should create the directory, copy the default services.xml (along with all other files in /opt/lib/my-app/lib/conf) in container into this directory to make it available for me to edit.
Actual results
After running docker-compose, it creates an empty directory inside the my-app directory. The my-app fails to read the services.xml file and app doesn't start (as it depends on this file).
I expect this should create the directory, copy the default services.xml (along with all other files in /opt/lib/my-app/lib/conf) in container into this directory to make it available for me to edit.
From you said above, if your aim is to let the contents in container be pop to host & let you have chance to modify them, then I suggest you to use named volumes. But, the folder in host will be managed by docker itself, so you need to find where they are located.
A minimal example for your reference:
docker-compose.yaml(In my example it located in the folder 77):
version: '3'
services:
frontend:
image: alpine
command: "tail -f /dev/null"
volumes:
- my_data:/etc
volumes:
my_data:
Start the service:
shubuntu1#shubuntu1:~/77$ docker-compose up -d
Creating network "77_default" with the default driver
Creating volume "77_my_data" with default driver
Creating 77_frontend_1 ... done
Check the location of named volume in host:
shubuntu1#shubuntu1:~/77$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6635aba545c9 alpine "tail -f /dev/null" 14 minutes ago Up 14 minutes 77_frontend_1
shubuntu1#shubuntu1:~/77$ docker inspect 77_frontend_1 | grep Source
"Source": "/var/lib/docker/volumes/77_my_data/_data",
Check the content of original /etc/profile in container:
shubuntu1#shubuntu1:~/77$ docker exec 77_frontend_1 cat /etc/profile
export CHARSET=UTF-8
export LANG=C.UTF-8
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export PAGER=less
export PS1='\h:\w\$ '
umask 022
for script in /etc/profile.d/*.sh ; do
if [ -r $script ] ; then
. $script
fi
done
Modify the script from host:
shubuntu1#shubuntu1:~/77$ sudo -s -H
root#shubuntu1:/home/shubuntu1/77# cd /var/lib/docker/volumes/77_my_data/_data
root#shubuntu1:/var/lib/docker/volumes/77_my_data/_data# echo 'echo "hello"' >> profile
Check again the /etc/profile in container after we made changes on host:
shubuntu1#shubuntu1:~/77$ docker exec 77_frontend_1 cat /etc/profile
export CHARSET=UTF-8
export LANG=C.UTF-8
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export PAGER=less
export PS1='\h:\w\$ '
umask 022
for script in /etc/profile.d/*.sh ; do
if [ -r $script ] ; then
. $script
fi
done
echo "hello"
We can see echo "hello" which we add on host already be seen in container.
I'm building a postgres+java container, and I'd like to open a shell into the java "service". That service exits immediately after starting, how can I do to open a shell into it?
I see it in docker ps -a but it has already exited.
The file I'm using is this .yaml with docker-compose
version: '3.1'
services:
db:
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: postgres
volumes:
- datavolume:/var/lib/postgresql
java:
image: openjdk:8
volumes:
datavolume:
A Docker container generally runs a single process. In the same way that just running a JVM without an application attached to it isn't really meaningful, running a Docker container with a JVM but no actual application added to it isn't that useful.
You should write a Dockerfile that adds your application's jar file to a base Java image; for instance
FROM openjdk:8
COPY app.jar /
CMD ["java", "-jar", "/app.jar"]
and then your docker-compose.yml file can have instructions to build and run this image
services:
java:
build: .
If you just want a shell in a copy of the image to poke around and see what's there, you can generally run
docker run --rm -it openjdk:8 sh
The standard openjdk Dockerfile doesn't explicitly declare any specific ENTRYPOINT or CMD so it will exit immediately when run. (It probably inherits a default /bin/sh, but with no command to run, that will also exit immediately.) You can declare some other command: in the Dockerfile to cause the "service" to not exit, but it's not really doing anything useful for you.
I have the following docker file to run my java application
FROM gidikern/rhel-oracle-jre
RUN mkdir /application
WORKDIR /application
CMD "java -Dspring.profiles.active=sprofileName -jar my.war --spring.config.location=./application.properties > app.log > 2>&1"
and i'm running using docker-compose:
backend_app:
restart: always
image: my-app-runner:latest
ports:
- "8080:8080"
volumes:
- ./app/my.war:/application/my.war:Z
- ./app/application.properties:/application/application.properties:Z
- /srv/docker/backend_app/logs:/application/my.log:Z
tty: true
however when i start i get that my app exited with code 0 constantly.
I can't find what is wrong.
The problem was actually with my base image gidikern/rhel-oracle-jre
I tested and even basic commands like ls didn't work..
Switched to openjdk for now and it is fine.
Very probably there's an error in your application. To debug I suggest you to run the container in shell and to execute the java command manually to see what's happening.
In other words:
docker run -it --name app-debug my-app-runner:latest /bin/bash
java -Dspring.profiles.active=sprofileName -jar my.war --spring.config.location=./application.properties