I am trying to setup my development environment using docker for a sprint boot application.
I am using intellij idea.
Here is the dockerfile.
FROM gradle:7.4.2-jdk18-alpine AS build
COPY --chown=gradle:gradle ./ /home/gradle/src
WORKDIR /home/gradle/src
RUN gradle build --no-daemon --debug
FROM openjdk:19-slim
EXPOSE 5097
EXPOSE 5005
RUN mkdir /app
COPY --from=build /home/gradle/src/build/libs/*.jar /app/spring-boot-application.jar
ENTRYPOINT ["java", "-Xdebug", "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005", "-jar", "/app/spring-boot-application.jar"]
and docker-compose.yml is:
version: '3.7'
services:
hmis-config-service:
build:
context: .
dockerfile: Dockerfile
image: 'hmis-config-service:0.0.0.1'
ports:
- "9060:5080"
- "8091:5005"
volumes:
- myapp:/home/gradle/src
environment:
db.url: 'jdbc:postgresql://host.docker.internal:5432/hms'
db.username: 'postgres'
db.password: 'pgsroot'
GRADLE_HOME: /usr/local/gradle
JAVA_HOME: /usr/lib/jvm/java
M2: /usr/local/apache-maven/bin
M2_HOME: /usr/local/apache-maven
volumes:
myapp:
Every time I try to build it, it takes a long time because it downloads all gradle dependencies. Hence even a small change takes a long time.
If project is started using
docker-compose up # without the --build flag
it starts instantaneously but the changes made aren't in the container.
I have tried to mount the volume /home/gradle/src to keep the dependencies synced but that didn't do anything.
What can be done to improve the build time and cache the dependencies?
Is there a reason you need to build the app in the docker context? If you build it outside (locally), and only copy the artefacts, you can leverage build caching from the (local) gradle daemon, which should increase the speeds significantly.
Essentially, you are building the project from scratch on each build/invocation.
i.e, don't build inside the Dockerfile, only copy the already built artefacts.
PS: If you have to build inside the docker context (for instance, if you don't have a JDK locally), you can try to figure out where gradle keeps its local cache and also sync/copy that over.
As you mentioned the issue is the build takes long time because it downloads dependencies each time.
If that is the case, you can try using multi-stage builds.
Basically you download the dependencies once then you just copy them each time. This also reduces the size of the image.
References:
https://docs.docker.com/develop/develop-images/multistage-build/
https://spring.io/guides/topicals/spring-boot-docker/
https://gist.github.com/msauza/6a906e879549e218c54868d81161afcb
https://hashedin.com/blog/a-guide-to-optimize-build-time-in-docker/
P.S: I have not tried this for Spring Boot applications, but they work well for applications based on Go.
Related
I want to build an application. For testing it uses testcontainers. The build will run on CI and on the developers' machines. The Dockerfile is more or less:
FROM amazoncorretto:17-alpine as builder
add . .
run ./gradlew build
from amazoncorretto:17-alpine
copy --from=builder build/libs/*.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
And I run the build using docker build .
Part of the ./gradlew build runs tests with Testscontainers and uses
val sftpDocker = GenericContainer(DockerImageName.parse("atmoz/sftp:alpine"))
And it returns
java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
I know that:
Testcontainers has its own docker API client and doesn't requires installed docker inside the Alpine container 3
Someone made it using "docker:20.10.14-dind" image. But I don't know how it fits in my problem 4
I can mount the /var/run/docker.sock during docker run ... but I'm using RUN command inside dockerfile and docker build ... instead
I can expose DOCKER_HOST and testcontainers should use the default gateway's IP address. But it's way less secure than using socket
So is there a way to use a socket in this setup? If not, how should I run my host Docker to expose TCP instead of a socket?
I'm trying to work with gitlab CI/CD. I'm using Ubuntu server and Spring Boot with Maven. All is fine, runner starts pipeline jobs but it gets lots of errors with pattern "warning: failed to remove target/..." even if i call simple echo 'something' in .yaml pipeline script gitlab-ci.yaml. I found that if i remove /home/gitlab-runner/builds then all starts to work fine until /builds generated again. What am i doing wrong? I already tried to reinstall runner, making gitlab-user, different variations of script^ nothing works until i manually remove builds folder. However, there is also js frontend which is also on gitlab ci/cd and everything works fine there. Help me please!
Here is the error i get trying to get my java spring boot maven pipeline work:
enter image description here
gitlab-ci.yaml code here:
stages:
- test
- package
- deploy
# - sonar
test:
stage: test
only:
- master
- merge_requests
except:
- tags
script:
- echo 'test are running i swear!!!!!!'
- sudo mvn clean
- sudo systemctl stop socnet.service
package:
stage: package
only:
- master
except:
- tags
script:
- sudo mvn package -Dmaven.test.skip=true
deploy_to_server:
stage: deploy
only:
- master
except:
- tags
script:
- sudo systemctl restart socnet.service
Remove sudo from your .gitlab-ci.yml.
Using sudo there will execute mvn package as root user, hence all generated files have root as the owner.
When gitlab-runner picks up a job and proceeds to clean up previously generated files, it is still unprivileged and hence will fail to remove files owned by root.
You might want to add the following variables into your .gitlab-ci.yml file in order to change the location for Maven dependencies cache to inside the project directory:
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Maven.gitlab-ci.yml
I try to deploy the spring-boot project with GitLab CI/CD. But can't find any information that can help me.
So, I try to figure out how I can do it and created an AWS instance and run the gitlub-runner on it.
I created .gitlab-ci.yml and can copy files to the S3-bucket:
variables:
S3_BUCKET_NAME: "awesome-proj"
deploy:
script: aws s3 cp ./ s3://awesome-proj/ --acl bucket-owner-full-control --no-sign-request --recursive --exclude "*" --include "*.html"
tags:
- Tag1
I figure out I can use the console in a similar way and clone my project to the AWS server. I decided to clone the project from the git repo and build him on the server with 'maven'.
So I try to clone the project:
S3_BUCKET_NAME: "awesome-proj"
deploy:
script:
- sudo cd /home/ec2-user/aweproject
- sudo git init
- sudo git clone https://gitlab.com/lTer/cicdtestdev.git
tags:
- Tag1
These scripts even copied some data once.
For the second time, I got the error, but it is doesn't matter now. The problem is that I haven't any files after git clone.... So it not works how I hoped.
How to correctly use the console with GitLab-runner or deploy the project (spring-boot) with Gitlab ci/cd to the AWS?
In principle, you don't need to clone your project. When your CI pipeline runs, it already has the appropriate clone (of a branch, or the master, of whatever the CI has been instructed to run on).
The only thing you need is to compile and then deploy. For instance:
stages:
- compile
- deploy
compile:
stage: compile
script: mvn ...
artifacts:
paths:
- build/
expire_in: 1 week
deploy:
stage: deploy
script: aws s3 sync build/ s3://somebucket/some_place/build/
Of course, in real life, you may want to have a specific stage for testing etc.
I am building an image from a dockerfile for a spring boot sample app, using the COPY command to copy a local file (which i need to pass as a vm argument) from host into my container.
The docker is running on windows 10 and with linux container.
When I try to run the image it shows error same as
Could not find agent library test/libaegean.dll in absolute path, with error: Error loading shared library test/libaegean.dll: No such file or directory.
Here is my Dockerfile
FROM openjdk:8-jdk-alpine
RUN mkdir test
RUN chmod +x test
COPY libaegean.dll test
COPY dellicence.lic test
COPY application-0.0.1-SNAPSHOT.jar test
WORKDIR test
EXPOSE 6070
ENTRYPOINT ["java","-agentpath:test/libaegean.dll","-jar","application-0.0.1-SNAPSHOT.jar"]
This is how I am building the image
docker image build --file=Dockerfile --tag=sample .
This is how I am running the container
docker container run --name=sample sample
please look at the picture to have a clear idea.
Thank you for the help in advance.
docker build image success
docker run failure
I'm trying to implement the example from this tutorial:
https://spring.io/guides/gs/spring-boot-docker/
I successfully compiled the package:
C:\Users\Desktop\rest_api>docker build -t springio/gs-spring-boot-docker .
Sending build context to Docker daemon 105.6MB
Step 1/5 : FROM openjdk:13-alpine
---> c4b0433a01ac
Step 2/5 : EXPOSE 8080
---> Using cache
---> 010600c5a7d0
Step 3/5 : ARG JAR_FILE=target/rest_api.jar
---> Running in 8ba2e28e0870
Removing intermediate container 8ba2e28e0870
---> b453cd05cbd2
Step 4/5 : ADD ${JAR_FILE} app.jar
---> dade5dd3eff2
Step 5/5 : ENTRYPOINT ["java","-jar","/app.jar"]
---> Running in e8a1f985f0fd
Removing intermediate container e8a1f985f0fd
---> cfa353eb23c5
Successfully built cfa353eb23c5
Successfully tagged springio/gs-spring-boot-docker:latest
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
C:\Users\Desktop\rest_api>
It's not clear for me where is the compiled package located? Can you guide me where it's located and how to mount it into Docker?
Docker file:
FROM openjdk:13-alpine
EXPOSE 8080
ARG JAR_FILE=target/rest_api.jar
ADD ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
In order to get the "compiled package" (spring boot jar) you need to invoke mvn package first.
This command will compile the sources and create a JAR in the target directory of your project.
Since you're working with spring boot, you'll need to configure spring boot maven plugin (make sure it appears in the pom.xml) This plugin creates a special jar of spring boot applications with all the dependencies inside. It stores it in the target directory
So, after invoking mvn package command and before running docker build command go to target directory and make sure that you have a relatively big JAR of the application.
This explanation, I assume, answers the part of the question "where is the compiled package located?"
Now as for "how to mount to Docker" part of the question:
In the Dockerfile you use ADD command. This command takes the jar and "adds" it to the docker image (like into the filesystem of the container), so It will be available under /app.jar (because you also rename the artifact on the way)
At build time, the jar file has been copied from your computer into the container and is since located at /app.jar. No need to mount it.