The question is very straightforward
What is the best way to execute liquibase migration in gitlab pipelines
what i have so far
but seems gitlab services immediately executes docker run, and docker run already requires db migration parameters
image: docker:19.03.1
stages:
- build
- db-migration
- deploy
services:
- docker:19.03.1-dind
- liquibase/liquibase:latest
variables:
DOCKER_TLS_CERTDIR: "/certs"
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_IMAGE_PATH: $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
before_script:
- apk add --update python python-dev py-pip
- pip install awscli --upgrade
- $(aws ecr get-login --no-include-email --region $AWS_REGION | tr -d '\r')
build:
stage: build
script:
- docker build --tag $AWS_IMAGE_PATH/$CI_PROJECT_NAME:$CI_COMMIT_SHA --tag $AWS_IMAGE_PATH/$CI_PROJECT_NAME:latest .
- docker push $AWS_IMAGE_PATH/$CI_PROJECT_NAME:$CI_COMMIT_SHA
- docker push $AWS_IMAGE_PATH/$CI_PROJECT_NAME:latest
db-migration:
stage: db-migration
script:
- liquibase --changeLogFile=/src/main/resources/db/changelog/psql/changelog.yaml
--url="jdbc:postgresql://host:5432/db"
--username username --password $DB_PASSWORD update
deploy:
stage: deploy
script:
- echo "Deployed"
db-migration:
image: openjdk:8-jre-alpine
stage: db-migration
script:
- INIT_PATH=`pwd`
- apk add bash
- mkdir /liquibase
- mkdir /Downloads
- cd /Downloads
- wget "https://github.com/liquibase/liquibase/releases/download/liquibase-parent-3.7.0/liquibase-3.7.0-bin.zip"
- wget "https://repo1.maven.org/maven2/org/postgresql/postgresql/42.2.8/postgresql-42.2.8.jar"
- unzip liquibase-3.7.0-bin.zip -d /liquibase -q
- mv postgresql-42.2.8.jar /liquibase/lib/
- export PATH=$PATH:/liquibase
- liquibase --changeLogFile=$INIT_PATH/src/main/resources/db/changelog/psql/changelog.yaml
--url="jdbc:postgresql://host:port/db"
--username username --password $DB_PASSWORD update
only:
- master
Related
So i am quite new to the whole CI topic and im trying to run tests after my merge request in my pipeline. Except that when my command try to run the container i have an error.
So this is my gitlab-ci file
image: docker:18
variables:
REPO_URL: "registry.gitlab.com/xxxxx/xxxxxxx"
PROD_SRV: "xxx"
DEV_SRV: "xxx"
services:
- docker:dind
stages:
- build jar
- build docker image
- tests
maven-build:
only:
- work
- test
image: maven:3-jdk-8
stage: build jar
script: "mvn -Dmaven.test.skip=true package -B"
artifacts:
paths:
- target/*.jar
expire_in: 7 days
docker build:
stage: build docker image
only:
- work
- test
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login --username "$CI_REGISTRY_USER" "$CI_REGISTRY" --password-stdin
- chmod a+x ./mvnw
script:
- docker info
- docker build -t $REPO_URL:$CI_COMMIT_SHA .
- docker images
- docker push $REPO_URL:$CI_COMMIT_SHA
after_script:
- docker logout registry.gitlab.com 2>/dev/null
- rm /root/.docker/config.json 2>/dev/null
tests job branch test:
stage: tests
only:
- test
variables:
GIT_STRATEGY: none
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login --username "$CI_REGISTRY_USER" "$CI_REGISTRY" --password-stdin
script:
- docker pull $REPO_URL:$CI_COMMIT_SHA
- docker run -t $REPO_URL:$CI_COMMIT_SHA .
- docker push $REPO_URL:$CI_COMMIT_SHA
after_script:
- docker logout registry.gitlab.com 2>/dev/null
- rm /root/.docker/config.json 2>/dev/null
and this is my dockerfile
# syntax=docker/dockerfile:1
FROM openjdk:16-alpine3.13 as base
WORKDIR /amsn-counter
COPY .mvn/ .mvn
COPY mvnw pom.xml ./
RUN ./mvnw dependency:go-offline
COPY src ./src
FROM base as test
RUN ["./mvnw", "test"]
FROM base as development
CMD ["./mvnw", "spring-boot:run", "-Dspring-boot.run.profiles=mysql", "-Dspring-boot.run.jvmArguments='-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000'"]
FROM base as build
RUN ./mvnw package
FROM openjdk:11-jre-slim as production
EXPOSE 8080
ADD /target/linkedin-0.0.1-SNAPSHOT.jar app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
and the error in the pipeline as well:
If you need any precision i will gladly answer, and like i said im new so don't hesitate to point if something is weird in the files or in my description in general
Edit: My branch are named {work}(local branch) and {test}
It looks as if there is no target specified when you are running docker build. Try this:
script:
- docker info
- image_name="$REPO_URL:$CI_COMMIT_SHA"
- docker build
--target test
--tag $image_name
.
- docker push $image_name
To use a integration test library https://www.testcontainers.org/" I need a image with java a docker installed at same time.
I'm trying o use this stage:
test:
stage: test
image: gradle:jdk16
services:
- docker:latest
script:
- docker --version
- chmod +x ./gradlew
- export GRADLE_USER_HOME=`pwd`/.gradle
- ./gradlew test --stacktrace
rules:
- !reference [.rules_merge_request, rules]
But It does not work:
$ docker --version
/scripts-33119345-2089057982/step_script: line 154: docker: command not found
Any help?
The image gradle:jdk16 does not include the docker client. You'll have to install it in your job. Additionally, you'll need to use the service docker:dind in your services: configuration (not docker:latest)
test:
stage: test
image: gradle:jdk16
services:
- docker:dind # use the docker-in-docker image
before_script: # install docker
- apt update && apt install --no-install-recommends -y docker.io
script:
- docker --version
Running this on gitlab.com runners, you should see an output like this:
Docker build works on my laptop but on GitLab I get
Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain
Tried a lot of different setups but nothing works...fails at gradlew build...Any ideas are welcome
My .gitlab-ci.yaml
....variables here
publish:
image:
name: amazon/aws-cli
entrypoint: [""]
services:
- docker:dind
before_script:
- amazon-linux-extras install docker
- aws --version
- docker --version
- export GRADLE_USER_HOME=`pwd`/gradle
- export CLASSPATH=`pwd`/gradle/wrapper
cache:
paths:
- gradle/wrapper
- .gradle/wrapper
- .gradle/caches
script:
- docker build -t $DOCKER_REGISTRY/$APP_NAME:$CI_PIPELINE_IID .
My Dockerfile
FROM openjdk:11
ENV wdir=code
ENV MY_SERVICE_PORT=8080
WORKDIR /$wdir
COPY . /code
RUN echo "Running build"
RUN ["/code/gradlew", "build"]
EXPOSE $MY_SERVICE_PORT
# Run the service
CMD ["java", "-jar", "build/libs/code-1.0-SNAPSHOT.jar"]
Created a workaround by installing gradle...works now:
FROM openjdk:11
ENV wdir=code
ENV MY_SERVICE_PORT=8080
WORKDIR /code
# Install Gradle
RUN wget -q https://services.gradle.org/distributions/gradle-6.5-bin.zip \
&& unzip gradle-6.5-bin.zip -d /opt \
&& rm gradle-6.5-bin.zip
ENV GRADLE_HOME /opt/gradle-6.5
ENV PATH $PATH:/opt/gradle-6.5/bin
# Prepare by downloading dependencies
ADD build.gradle /code/build.gradle
ADD src /code/src
RUN echo "Running build"
RUN cd /code
RUN gradle --no-daemon build
EXPOSE $MY_SERVICE_PORT
# Run the service
CMD ["java", "-jar", "build/libs/code-1.0-SNAPSHOT.jar"]
I am building an node mongo project.
I am using docker compose for the project.
here is my dockerFile
FROM node:carbon
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD [ "npm", "start" ]
Here is docker-compose.yml
version: "2"
services:
app:
container_name: app
restart: always
build: .
ports:
- "3000:3000"
links:
- mongo
mongo:
container_name: mongo
image: mongo
volumes:
- ./data:/data/db
ports:
- "27017:27017"
Here I also want to install java using docker-compose. As I will need java for elastic search and for other purposes. So can any one help about how to install java using docker-compose in this project.
Docker-compose is a tool used to launch multiple containers from a single .yaml file.
Add this line to your Dockerfile to install Java:
RUN apt-get update && \
apt-get install -y openjdk-8-jdk && \
apt-get install -y ant && \
apt-get clean;
I've built a gitlab-ci yaml file, which works well. However the gradle version used in the image is different to that on my local machine. This is causing some unusual side effects, such as only some Java tests being run.
Here's my yaml file:
image: java:8-jdk
before_script:
- echo `pwd`
- export GRADLE_USER_HOME=`pwd`/.gradle
- rm -f .gradle/caches/modules-2/modules-2.lock
- rm -fr .gradle/caches/*/plugin-resolution/
cache:
paths:
- .gradle/wrapper
- .gradle/caches
build:
script:
- ./gradlew build
test:
stage: test
script:
- ./gradlew test
- cat build/jacocoHtml/index.html | grep -o 'Total[^%]*%'
artifacts:
paths:
- build/jacocoHtml
#deploy test coverage
pages:
stage: deploy
dependencies:
- test
script:
- mkdir public
- mkdir public/jacoco
- mv build/jacocoHtml/* public
artifacts:
paths:
- public
only:
- master
Currently, the build is being run on 4.10.3 but I need to update this to 5.1.1 to match my local setup.
Thanks in advance,
Sam
Use gradle docker image instead of java:8-jdk
For the version 5.1.1 with jdk8, use :
image: gradle:5.1.1-jdk8