So i am quite new to the whole CI topic and im trying to run tests after my merge request in my pipeline. Except that when my command try to run the container i have an error.
So this is my gitlab-ci file
image: docker:18
variables:
REPO_URL: "registry.gitlab.com/xxxxx/xxxxxxx"
PROD_SRV: "xxx"
DEV_SRV: "xxx"
services:
- docker:dind
stages:
- build jar
- build docker image
- tests
maven-build:
only:
- work
- test
image: maven:3-jdk-8
stage: build jar
script: "mvn -Dmaven.test.skip=true package -B"
artifacts:
paths:
- target/*.jar
expire_in: 7 days
docker build:
stage: build docker image
only:
- work
- test
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login --username "$CI_REGISTRY_USER" "$CI_REGISTRY" --password-stdin
- chmod a+x ./mvnw
script:
- docker info
- docker build -t $REPO_URL:$CI_COMMIT_SHA .
- docker images
- docker push $REPO_URL:$CI_COMMIT_SHA
after_script:
- docker logout registry.gitlab.com 2>/dev/null
- rm /root/.docker/config.json 2>/dev/null
tests job branch test:
stage: tests
only:
- test
variables:
GIT_STRATEGY: none
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login --username "$CI_REGISTRY_USER" "$CI_REGISTRY" --password-stdin
script:
- docker pull $REPO_URL:$CI_COMMIT_SHA
- docker run -t $REPO_URL:$CI_COMMIT_SHA .
- docker push $REPO_URL:$CI_COMMIT_SHA
after_script:
- docker logout registry.gitlab.com 2>/dev/null
- rm /root/.docker/config.json 2>/dev/null
and this is my dockerfile
# syntax=docker/dockerfile:1
FROM openjdk:16-alpine3.13 as base
WORKDIR /amsn-counter
COPY .mvn/ .mvn
COPY mvnw pom.xml ./
RUN ./mvnw dependency:go-offline
COPY src ./src
FROM base as test
RUN ["./mvnw", "test"]
FROM base as development
CMD ["./mvnw", "spring-boot:run", "-Dspring-boot.run.profiles=mysql", "-Dspring-boot.run.jvmArguments='-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000'"]
FROM base as build
RUN ./mvnw package
FROM openjdk:11-jre-slim as production
EXPOSE 8080
ADD /target/linkedin-0.0.1-SNAPSHOT.jar app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
and the error in the pipeline as well:
If you need any precision i will gladly answer, and like i said im new so don't hesitate to point if something is weird in the files or in my description in general
Edit: My branch are named {work}(local branch) and {test}
It looks as if there is no target specified when you are running docker build. Try this:
script:
- docker info
- image_name="$REPO_URL:$CI_COMMIT_SHA"
- docker build
--target test
--tag $image_name
.
- docker push $image_name
Related
To use a integration test library https://www.testcontainers.org/" I need a image with java a docker installed at same time.
I'm trying o use this stage:
test:
stage: test
image: gradle:jdk16
services:
- docker:latest
script:
- docker --version
- chmod +x ./gradlew
- export GRADLE_USER_HOME=`pwd`/.gradle
- ./gradlew test --stacktrace
rules:
- !reference [.rules_merge_request, rules]
But It does not work:
$ docker --version
/scripts-33119345-2089057982/step_script: line 154: docker: command not found
Any help?
The image gradle:jdk16 does not include the docker client. You'll have to install it in your job. Additionally, you'll need to use the service docker:dind in your services: configuration (not docker:latest)
test:
stage: test
image: gradle:jdk16
services:
- docker:dind # use the docker-in-docker image
before_script: # install docker
- apt update && apt install --no-install-recommends -y docker.io
script:
- docker --version
Running this on gitlab.com runners, you should see an output like this:
Docker build works on my laptop but on GitLab I get
Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain
Tried a lot of different setups but nothing works...fails at gradlew build...Any ideas are welcome
My .gitlab-ci.yaml
....variables here
publish:
image:
name: amazon/aws-cli
entrypoint: [""]
services:
- docker:dind
before_script:
- amazon-linux-extras install docker
- aws --version
- docker --version
- export GRADLE_USER_HOME=`pwd`/gradle
- export CLASSPATH=`pwd`/gradle/wrapper
cache:
paths:
- gradle/wrapper
- .gradle/wrapper
- .gradle/caches
script:
- docker build -t $DOCKER_REGISTRY/$APP_NAME:$CI_PIPELINE_IID .
My Dockerfile
FROM openjdk:11
ENV wdir=code
ENV MY_SERVICE_PORT=8080
WORKDIR /$wdir
COPY . /code
RUN echo "Running build"
RUN ["/code/gradlew", "build"]
EXPOSE $MY_SERVICE_PORT
# Run the service
CMD ["java", "-jar", "build/libs/code-1.0-SNAPSHOT.jar"]
Created a workaround by installing gradle...works now:
FROM openjdk:11
ENV wdir=code
ENV MY_SERVICE_PORT=8080
WORKDIR /code
# Install Gradle
RUN wget -q https://services.gradle.org/distributions/gradle-6.5-bin.zip \
&& unzip gradle-6.5-bin.zip -d /opt \
&& rm gradle-6.5-bin.zip
ENV GRADLE_HOME /opt/gradle-6.5
ENV PATH $PATH:/opt/gradle-6.5/bin
# Prepare by downloading dependencies
ADD build.gradle /code/build.gradle
ADD src /code/src
RUN echo "Running build"
RUN cd /code
RUN gradle --no-daemon build
EXPOSE $MY_SERVICE_PORT
# Run the service
CMD ["java", "-jar", "build/libs/code-1.0-SNAPSHOT.jar"]
I'm trying to create a CI/CD Pipeline for a simple java/maven project.
The runner that I'm using is a docker runner.
I'm using a dockerfile to create a container which installs maven/java/etc.. and in this container the program should be tested.
Sorry for the question but I am new to CI/CD Pipelines in GitLab.
GitHub works just fine have a look: https://github.com/ni920/CICD-Test
Thank you
Here are the CI logs
...
Executing "step_script" stage of the job script
$ docker build --build-arg JAVA_VERSION=openjdk7
/bin/sh: eval: line 95: docker: not found
Cleaning up file based variables
ERROR: Job failed: exit code 127
Thats the .gitlab-ci.yml
stages:
- java7
# - java11
# - deploy
java7:
stage: java7
script:
- docker build --build-arg JAVA_VERSION=openjdk7
# tags:
# - docker
#java11:
# stage: java11
# script:
# - docker build --build-arg JAVA_VERSION=openjdk11
# tags:
# - docker
Thats the dockerfile
# Pull base image.
FROM alpine as build
ARG MAVEN_VERSION=3.6.1
ARG USER_HOME_DIR="/root"
ARG JAVA_VERSION=openjdk7
ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
ENV HTTP_PROXY=#comment
ENV HTTPS_PROXY=#comment
# Install Java.
RUN apk --update --no-cache add JAVA_VERSION curl
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
# Define working directory.
WORKDIR /data
# Define commonly used JAVA_HOME variable
ENV JAVA_HOME /usr/lib/jvm/default-jvm/
# Define default command.
CMD ["mvn", "--version"]
Running your pipelines using the Docker executor means that your jobs will run in a Docker container, but not that you will be able to execute docker commands.
If you need to run docker commands inside a GitLab CI job (read "inside a container") you will need Docker-in-Docker (often abbreviated DinD). It is a vast topic on itself but you can get started with GitLab CI's documentation: Use Docker to build Docker images
I always use DinD and have a minimal setup in my gitlab-ci.yml.
Using a docker image as a default:
image: docker:19.03.13
Define a default variable for TLS certificates:
variables:
DOCKER_TLS_CERTDIR: "/certs"
Then use a docker image as a service to enable DinD:
services:
- name: docker:19.03.13-dind
alias: docker
I wrote a few posts about using Docker-in-Docker on GitLab CI that you may find useful, but I still recommend to extensively read GitLab's documentation before reading them.
The question is very straightforward
What is the best way to execute liquibase migration in gitlab pipelines
what i have so far
but seems gitlab services immediately executes docker run, and docker run already requires db migration parameters
image: docker:19.03.1
stages:
- build
- db-migration
- deploy
services:
- docker:19.03.1-dind
- liquibase/liquibase:latest
variables:
DOCKER_TLS_CERTDIR: "/certs"
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_IMAGE_PATH: $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
before_script:
- apk add --update python python-dev py-pip
- pip install awscli --upgrade
- $(aws ecr get-login --no-include-email --region $AWS_REGION | tr -d '\r')
build:
stage: build
script:
- docker build --tag $AWS_IMAGE_PATH/$CI_PROJECT_NAME:$CI_COMMIT_SHA --tag $AWS_IMAGE_PATH/$CI_PROJECT_NAME:latest .
- docker push $AWS_IMAGE_PATH/$CI_PROJECT_NAME:$CI_COMMIT_SHA
- docker push $AWS_IMAGE_PATH/$CI_PROJECT_NAME:latest
db-migration:
stage: db-migration
script:
- liquibase --changeLogFile=/src/main/resources/db/changelog/psql/changelog.yaml
--url="jdbc:postgresql://host:5432/db"
--username username --password $DB_PASSWORD update
deploy:
stage: deploy
script:
- echo "Deployed"
db-migration:
image: openjdk:8-jre-alpine
stage: db-migration
script:
- INIT_PATH=`pwd`
- apk add bash
- mkdir /liquibase
- mkdir /Downloads
- cd /Downloads
- wget "https://github.com/liquibase/liquibase/releases/download/liquibase-parent-3.7.0/liquibase-3.7.0-bin.zip"
- wget "https://repo1.maven.org/maven2/org/postgresql/postgresql/42.2.8/postgresql-42.2.8.jar"
- unzip liquibase-3.7.0-bin.zip -d /liquibase -q
- mv postgresql-42.2.8.jar /liquibase/lib/
- export PATH=$PATH:/liquibase
- liquibase --changeLogFile=$INIT_PATH/src/main/resources/db/changelog/psql/changelog.yaml
--url="jdbc:postgresql://host:port/db"
--username username --password $DB_PASSWORD update
only:
- master
I've built a gitlab-ci yaml file, which works well. However the gradle version used in the image is different to that on my local machine. This is causing some unusual side effects, such as only some Java tests being run.
Here's my yaml file:
image: java:8-jdk
before_script:
- echo `pwd`
- export GRADLE_USER_HOME=`pwd`/.gradle
- rm -f .gradle/caches/modules-2/modules-2.lock
- rm -fr .gradle/caches/*/plugin-resolution/
cache:
paths:
- .gradle/wrapper
- .gradle/caches
build:
script:
- ./gradlew build
test:
stage: test
script:
- ./gradlew test
- cat build/jacocoHtml/index.html | grep -o 'Total[^%]*%'
artifacts:
paths:
- build/jacocoHtml
#deploy test coverage
pages:
stage: deploy
dependencies:
- test
script:
- mkdir public
- mkdir public/jacoco
- mv build/jacocoHtml/* public
artifacts:
paths:
- public
only:
- master
Currently, the build is being run on 4.10.3 but I need to update this to 5.1.1 to match my local setup.
Thanks in advance,
Sam
Use gradle docker image instead of java:8-jdk
For the version 5.1.1 with jdk8, use :
image: gradle:5.1.1-jdk8