We Keep Coding

GitLab CI/CD runner fails to execute

I'm trying to work with gitlab CI/CD. I'm using Ubuntu server and Spring Boot with Maven. All is fine, runner starts pipeline jobs but it gets lots of errors with pattern "warning: failed to remove target/..." even if i call simple echo 'something' in .yaml pipeline script gitlab-ci.yaml. I found that if i remove /home/gitlab-runner/builds then all starts to work fine until /builds generated again. What am i doing wrong? I already tried to reinstall runner, making gitlab-user, different variations of script^ nothing works until i manually remove builds folder. However, there is also js frontend which is also on gitlab ci/cd and everything works fine there. Help me please!
Here is the error i get trying to get my java spring boot maven pipeline work:
enter image description here
gitlab-ci.yaml code here:
stages:
- test
- package
- deploy
# - sonar
test:
stage: test
only:
- master
- merge_requests
except:
- tags
script:
- echo 'test are running i swear!!!!!!'
- sudo mvn clean
- sudo systemctl stop socnet.service
package:
stage: package
only:
- master
except:
- tags
script:
- sudo mvn package -Dmaven.test.skip=true
deploy_to_server:
stage: deploy
only:
- master
except:
- tags
script:
- sudo systemctl restart socnet.service

Remove sudo from your .gitlab-ci.yml.
Using sudo there will execute mvn package as root user, hence all generated files have root as the owner.
When gitlab-runner picks up a job and proceeds to clean up previously generated files, it is still unprivileged and hence will fail to remove files owned by root.
You might want to add the following variables into your .gitlab-ci.yml file in order to change the location for Maven dependencies cache to inside the project directory:
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Maven.gitlab-ci.yml

Error: Unable to access jarfile (Kubernetes)

kubectl get pods -n abc
NAME READY STATUS RESTARTS AGE
abc-v2-78b59ccc4f-85xgr 0/1 CrashLoopBackOff 27 129m
Facing below error:
> ➜ ~ kubectl logs -f abc-v2-78b59ccc4f-85xgr -n
Error: Unable to access jarfile abc.jar
I am assuming either jar is not present or required access is missing.
Pls guide me here, how to proceed.
Edit 1: As suggested by #howard-roark, Jar is available inside container, getting the same error message.
Edit 2: Check results now with .jar in java command
Edit 4: Ideally there should be only one instance with running status.

Kubernetes is a Container Orchestrator. It runs your images as Containers. The error you are showing looks like an application error and not a Kubernetes error. The way I would approach this is:
Check if the jar file your application calls is in your image. You can do this locally by running your image and exec'ing in to see if your jar file that your application runs is there.
docker run -it <image> /bin/bash
Or you can do a similar command via Kubernetes to exec into your pod:
kubectl run -i --tty testjavacontainer --image=<image> -- /bin/bash
If it is there, then I would confirm its path and make sure that my java command is correctly referencing that path. If it is not there, then I would adjust my Dockerfile to ensure it is at the path that my java command expects.
In short, I would approach this as a standard java error, with the nuance that it happens to run in a container.

Where to find produced Docker image

I'm trying to implement the example from this tutorial:
https://spring.io/guides/gs/spring-boot-docker/
I successfully compiled the package:
C:\Users\Desktop\rest_api>docker build -t springio/gs-spring-boot-docker .
Sending build context to Docker daemon 105.6MB
Step 1/5 : FROM openjdk:13-alpine
---> c4b0433a01ac
Step 2/5 : EXPOSE 8080
---> Using cache
---> 010600c5a7d0
Step 3/5 : ARG JAR_FILE=target/rest_api.jar
---> Running in 8ba2e28e0870
Removing intermediate container 8ba2e28e0870
---> b453cd05cbd2
Step 4/5 : ADD ${JAR_FILE} app.jar
---> dade5dd3eff2
Step 5/5 : ENTRYPOINT ["java","-jar","/app.jar"]
---> Running in e8a1f985f0fd
Removing intermediate container e8a1f985f0fd
---> cfa353eb23c5
Successfully built cfa353eb23c5
Successfully tagged springio/gs-spring-boot-docker:latest
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
C:\Users\Desktop\rest_api>
It's not clear for me where is the compiled package located? Can you guide me where it's located and how to mount it into Docker?
Docker file:
FROM openjdk:13-alpine
EXPOSE 8080
ARG JAR_FILE=target/rest_api.jar
ADD ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

In order to get the "compiled package" (spring boot jar) you need to invoke mvn package first.
This command will compile the sources and create a JAR in the target directory of your project.
Since you're working with spring boot, you'll need to configure spring boot maven plugin (make sure it appears in the pom.xml) This plugin creates a special jar of spring boot applications with all the dependencies inside. It stores it in the target directory
So, after invoking mvn package command and before running docker build command go to target directory and make sure that you have a relatively big JAR of the application.
This explanation, I assume, answers the part of the question "where is the compiled package located?"
Now as for "how to mount to Docker" part of the question:
In the Dockerfile you use ADD command. This command takes the jar and "adds" it to the docker image (like into the filesystem of the container), so It will be available under /app.jar (because you also rename the artifact on the way)

At build time, the jar file has been copied from your computer into the container and is since located at /app.jar. No need to mount it.

running war from command line without maven jetty

I am trying to put a piece of open source software in a docker container (https://github.com/att/XACML) but in this container I can not use maven. The documentation for running this service says to use mvn jetty, which does work fine, but in order to get this in a container I don't want to include a build step (maven).
Instead, I'd like a way to compile the a war, so I can copy just the war into the container and execute it from there.
I have tried many attempts to get the war running without maven jetty but none of them work.
java -jar /path/to/jar
no main manifest attribute error. There is no main class, it extends an HttpServlet
using jetty-runner
when I launch the war with jetty-runner through the command line I do not get any errors, but it boots up to a page showing the directory of files, and not the actual project.
Making an 'uber-jar' to package all deps
same issue as 1, get a no main manifest issue.
I can include more code if that would be helpful (pom files etc), but I don't want to add too much if it is unneeded. I am super unfamiliar with how java projects are packaged and deployed, so I am having a difficult time figuring out what needs to be done.
Thanks!

Minimal Dockerfile to work with your webapp / war file is ...
FROM jetty:9.4.18
ADD ROOT.war /var/lib/jetty/webapps/
This uses the official jetty docker image at
https://hub.docker.com/_/jetty
Managed at
https://github.com/eclipse/jetty.docker
The name ROOT.war is special, and will deploy your webapp in the "root" context path of "/"
Building Image
If you build it like this ...
$ docker build -t stackoverflow/jetty:latest .
Running Image
Interactively (so you can the logs)
$ docker run --interactive --tty --rm --publish 80:8080 stackoverflow/jetty:latest
As Daemon
$ docker run --detach --publish 80:8080 stackoverflow/jetty:latest
The server will be available on port 80 of the machine you ran the docker run command on.
Configuring Jetty Base
If you need to configure the jetty image you can use any of the standard start.jar commands.
Example:
FROM jetty:9.4.18
WORKDIR $JETTY_BASE
RUN java -jar $JETTY_HOME/start.jar --add-to-start=jsp
ADD ROOT.war /var/lib/jetty/webapps/
How This Works Without Maven
See the official image details ...
https://github.com/eclipse/jetty.docker/blob/master/9.4-jdk11/Dockerfile
The key commands are ...
WORKDIR $JETTY_BASE
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["java","-jar","/usr/local/jetty/start.jar"]

How do I install Spring Boot CLI in Ubuntu?

I am trying to install Spring Boot CLI in Ubuntu. I am new to spring boot and was trying to learn it. I have already installed Gradle and groovy but did not find anything online to install Spring boot CLI.

please follow the code
sudo apt install unzip zip
curl -s https://get.sdkman.io | bash
source "/home/username/.sdkman/bin/sdkman-init.sh"=
sdk install springboot
spring version

The Spring boot CLI install guide link
Spring boot CLI

You can read the docs for installation steps:
Manual installation:
Download the Spring CLI distribution: you can find the latest version link here.
Extract the downloaded zip file to any path of your choice.
Prerequisite: Make sure you have Java JDK v1.8 or above (check INSTALL.txt file from extracted archive in Step-1). You can check your JDK version using command: java --version.
Add the spring CLI script's path to $PATH environment variable:
For example, you can add below lines to your .bashrc file in *nix systems:
export SPRING_HOME=/path/to/your/extracted-spring-cli/spring-2.4.0
export PATH=$SPRING_HOME/bin:$PATH
After saving the .bashrc file, run $ source .bashrc to reload the environment variables you just added.
(optional; and not for Windows user) Shell auto-completion scripts are provided for BASH and ZSH with the extracted archive in Step-1. Add symlinks to the appropriate
location for your environment. For example, something like:
ln -s /path/to/your/extracted-spring-cli/spring-2.4.0/shell-completion/bash/spring /etc/bash_completion.d/spring
ln -s /path/to/your/extracted-spring-cli/spring-2.4.0/shell-completion/zsh/_spring /usr/local/share/zsh/site-functions/_spring
To check if you are using BASH or ZSH, run the command: $ echo $SHELL.
DONE: To test if you have successfully installed the CLI you can run the following command: spring --version.

Here is detailed instruction on how to do it, much more useful than official one, at least for manual installation. One more thing - to make path change permanent you might need to logout - login, as "source /etc/profile" will be effective only for currently opened terminal.

Use this link to get a detailed instruction on how to download the Spring framework successfully, It really helped me a lot after long time of making research.
https://www.decodejava.com/download-and-install-spring-framework.htm