name: demo
on:
push:
branches:
- 'pipeline-dev'
jobs:
conditional_step:
runs-on: 'ubuntu-20.04'
steps:
- uses: actions/checkout#v3
with:
repository: hello-world-with-gradle
- run: cd /home/runner/work/test-actions/test-actions
- uses: actions/setup-java#v3
with:
distribution: temurin
java-version: 8
- name: Setup Gradle
uses: gradle/gradle-build-action#v2
- name: Execute Gradle build
run: ./gradlew build
The above workflow gives a JAR as as a build for Application. Now do we have the capabilities to run this JAR in the workflow itself?
like
- run: Java -jar hello-world-0.1.0.jar // This gives an error
In our real use case we want to run the utility in pipeline and send its output result to other google bucket.
Tried out various java actions plugins but not able to run JAR yet. If anyone know how to run a JAR or any other alternative It would be helpful.
check if the case matters, use java, not Java:
- run: java -jar hello-world-0.1.0.jar
^^^
That way, the GitHub runner should find and execute java after applying actions/setup-java.
Related
I'm trying to work with gitlab CI/CD. I'm using Ubuntu server and Spring Boot with Maven. All is fine, runner starts pipeline jobs but it gets lots of errors with pattern "warning: failed to remove target/..." even if i call simple echo 'something' in .yaml pipeline script gitlab-ci.yaml. I found that if i remove /home/gitlab-runner/builds then all starts to work fine until /builds generated again. What am i doing wrong? I already tried to reinstall runner, making gitlab-user, different variations of script^ nothing works until i manually remove builds folder. However, there is also js frontend which is also on gitlab ci/cd and everything works fine there. Help me please!
Here is the error i get trying to get my java spring boot maven pipeline work:
enter image description here
gitlab-ci.yaml code here:
stages:
- test
- package
- deploy
# - sonar
test:
stage: test
only:
- master
- merge_requests
except:
- tags
script:
- echo 'test are running i swear!!!!!!'
- sudo mvn clean
- sudo systemctl stop socnet.service
package:
stage: package
only:
- master
except:
- tags
script:
- sudo mvn package -Dmaven.test.skip=true
deploy_to_server:
stage: deploy
only:
- master
except:
- tags
script:
- sudo systemctl restart socnet.service
Remove sudo from your .gitlab-ci.yml.
Using sudo there will execute mvn package as root user, hence all generated files have root as the owner.
When gitlab-runner picks up a job and proceeds to clean up previously generated files, it is still unprivileged and hence will fail to remove files owned by root.
You might want to add the following variables into your .gitlab-ci.yml file in order to change the location for Maven dependencies cache to inside the project directory:
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Maven.gitlab-ci.yml
I am trying to build a Java Maven application and run some tests against a Postgres db.
However, the java app uses JDK 13 and the Azure hosted agent ubuntu doesn't have this default installed. As such, I use the script task to install it, then use it during the Maven build.
However, I tried several config's and in all Maven keeps complaining that it can not found any JDK 13 installed.
The last config I tried is listed below, in which I install it through a script and then use the JavaToolInstaller task to make it available (ensuring the Java_home is set and the java can be found in the path.
I then get the error
##[error]Java 13 is not preinstalled on this agent
I also tried it without the JavaToolInstaller task and then export JAVA_HOME and modify the PATH in the script, but then Maven complaints it can not find JDK 13...
Please some help how to use JDK 13 on an ubuntu agent during the maven build?
Azure pipeline snippet:
variables:
MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository
MAVEN_OPTS: "-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)"
JAVA_HOME : "/usr/lib/jvm/openjdk-13-jdk"
PATH: $(JAVA_HOME)/bin:$(PATH)
service_name: backend
mygetUsername: myUserName
mygetPassword: myPassword
resources:
containers:
- container: postgres
image: postgres:11.6-alpine
ports:
- 5432:5432
env:
POSTGRES_DB: default
POSTGRES_USER: default
POSTGRES_PASSWORD: default
POSTGRES_HOST_AUTH_METHOD: trust
stages:
- stage: create_artifact
displayName: Create artifact
jobs:
- job: build
displayName: Build, test and publish artifact
services:
postgres: postgres
steps:
- script: |
sudo apt-get install openjdk-13-jdk
displayName: Installing JDK 13
- task: JavaToolInstaller#0
displayName: Using JDK 13
inputs:
versionSpec: "13"
jdkArchitectureOption: x64
jdkSourceOption: "PreInstalled"
- task: Cache#2
displayName: Cache Maven local repo
inputs:
key: 'maven | "$(Agent.OS)" | backend/pom.xml'
restoreKeys: |
maven | "$(Agent.OS)"
maven
path: $(MAVEN_CACHE_FOLDER)
- task: Maven#3
name: maven_package
displayName: Maven package
inputs:
goals: "package"
mavenPomFile: "backend/pom.xml"
options: '--settings backend/.mvn/settings.xml -DmygetUsername=$(mygetUsername) -DmygetPassword=$(mygetPassword)'
mavenOptions: "-Xmx3072m $(MAVEN_OPTS)"
javaHomeOption: "JDKVersion"
jdkVersionOption: "1.13"
mavenAuthenticateFeed: true
"PreInstalled" feature allows you to use Java versions that are pre-installed on the Microsoft-hosted agent. You can find available pre-installed versions of Java in Software section:
I think your script does not work on hosted machine somehow. And, the JDK version you specify is not present on your host machine. I would suggest you to set your Host machine as per the above table (as per pre-defined JDK installed).
Consequently, the other two options can be leveraged:
Here's an example of getting the archive file from a local directory on Linux. The file should be an archive (.zip, .gz) of the JAVA_HOME directory so that it includes the bin, lib, include, jre, etc. directories.
- task: JavaToolInstaller#0
inputs:
versionSpec: "11"
jdkArchitectureOption: x64
jdkSourceOption: LocalDirectory
jdkFile: "/builds/openjdk-11.0.2_linux-x64_bin.tar.gz"
jdkDestinationDirectory: "/builds/binaries/externals"
cleanDestinationDirectory: true
Here's an example of downloading the archive file from Azure Storage. The file should be an archive (.zip, .gz) of the JAVA_HOME directory so that it includes the bin, lib, include, jre, etc. directories.
- task: JavaToolInstaller#0
inputs:
versionSpec: '6'
jdkArchitectureOption: 'x64'
jdkSourceOption: AzureStorage
azureResourceManagerEndpoint: myARMServiceConnection
azureStorageAccountName: myAzureStorageAccountName
azureContainerName: myAzureStorageContainerName
azureCommonVirtualFile: 'jdk1.6.0_45.zip'
jdkDestinationDirectory: '$(agent.toolsDirectory)/jdk6'
cleanDestinationDirectory: false
The problem was that I was pointing to the wrong installed directory.
The above location of JAVA_HOME I found on internet, after ouputing the tree content of the /usrl/lib/jvm I changed it and added this to the variable section and maven found the jdk and uses it 😊
JAVA_HOME_13_X64 : "/usr/lib/jvm/java-13-openjdk-amd64"
I try to deploy the spring-boot project with GitLab CI/CD. But can't find any information that can help me.
So, I try to figure out how I can do it and created an AWS instance and run the gitlub-runner on it.
I created .gitlab-ci.yml and can copy files to the S3-bucket:
variables:
S3_BUCKET_NAME: "awesome-proj"
deploy:
script: aws s3 cp ./ s3://awesome-proj/ --acl bucket-owner-full-control --no-sign-request --recursive --exclude "*" --include "*.html"
tags:
- Tag1
I figure out I can use the console in a similar way and clone my project to the AWS server. I decided to clone the project from the git repo and build him on the server with 'maven'.
So I try to clone the project:
S3_BUCKET_NAME: "awesome-proj"
deploy:
script:
- sudo cd /home/ec2-user/aweproject
- sudo git init
- sudo git clone https://gitlab.com/lTer/cicdtestdev.git
tags:
- Tag1
These scripts even copied some data once.
For the second time, I got the error, but it is doesn't matter now. The problem is that I haven't any files after git clone.... So it not works how I hoped.
How to correctly use the console with GitLab-runner or deploy the project (spring-boot) with Gitlab ci/cd to the AWS?
In principle, you don't need to clone your project. When your CI pipeline runs, it already has the appropriate clone (of a branch, or the master, of whatever the CI has been instructed to run on).
The only thing you need is to compile and then deploy. For instance:
stages:
- compile
- deploy
compile:
stage: compile
script: mvn ...
artifacts:
paths:
- build/
expire_in: 1 week
deploy:
stage: deploy
script: aws s3 sync build/ s3://somebucket/some_place/build/
Of course, in real life, you may want to have a specific stage for testing etc.
I'm building a simple hello world application in java (based on spring) which I launch to AWS through a pipeline.
The buildspec.yml is defined as follows:
version: 0.2
phases:
install:
runtime-versions:
java: openjdk8
build:
commands:
- mvn package
artifacts:
files:
- '**/*'
with the appspec.yml as follows:
version: 0.0
os: linux
files:
- source: target/helloworld-1.0-SNAPSHOT.jar
destination: /tmp
hooks:
ApplicationStart:
- location: codedeploy/ApplicationStart.sh
timeout: 60
runas: root
The file codedeploy/ApplicationStart.sh:
#!/usr/bin/env bash
JAR_FILE_HOME='/tmp/helloworld-1.0-SNAPSHOT.jar'
java -jar JAR_FILE_HOME
Weirdly enough the deployment fails with the following error:
Script at specified location: codedeploy/ApplicationStart.sh run as
user root failed with exit code 127
Output log:
[stderr]/opt/codedeploy-agent/deployment-root/5092b759-ecc4-44cb-859a-9823734abc04/d-GVQ6R854B/deployment-archive/codedeploy/ApplicationStart.sh:
line 9: java: command not found
This seems very counter-intuitive since I've installed java in the buildspec.yml. Do I need to install java manually again within the ApplicationStart script or am I doing something else wrong?
CodeBuild doesn't have a link with your application instance instead it only create run time when it receives artifacts for build event.
You don't need to install JAVA run time every time with appspec.yml. I would recommend you to install JAVA run time on an EC2 instance then create an AMI, as a reference base image for future use or you can proceed with Elasticbeanstalk which has prebuilt environments.
The other answer also suggests this but just to clarify:
CodeBuild (specified in buildspec.yml) does the artifact creation. Simply put it takes your code and creates the jar. You defined here the java version. However this has nothing to do with you instance where it will deployed. It is an image where the build happens.
CodeDeploy (specified in appspec.yml) is responsible for deploying the artifact on the instances defined and owned by you. If you created the target instance manually you need to make java available there. As matesio suggested above you could simplify / automate the instance creation with proper java env but that is your responsibility as that is your instance (not like the env used for the build which is configured by AWS in the background)
I've a virtual server running a gitlab runner.
I've now added some GUI unit tests which run nicely on my pc but not on the virtual server.
It always exits with:
java.awt.HeadlessException:
No X11 DISPLAY variable was set, but this program performed an operation which requires it.
Any idea how to get this running with gitlab-ci?
Edit:
My virtual server is running centos 7
This im my current .gitlab-ci.yml
image: kaiwinter/docker-java8-maven
#maven:3-jdk-7
before_script:
- "Xvfb :99 &"
- "export DISPLAY=:99"
maven_build:
stage: build
script:
- "mvn clean package"
- "cat target/site/jacoco/index.html"
Now, the headlessexception is gone but basically all unit tests are failing due to awt exceptions like:
Could not initialize class java.awt.image.IndexColorModel
Edit2:
I've added the headless option as suggested:
image: kaiwinter/docker-java8-maven
#maven:3-jdk-7
before_script:
- "Xvfb :99 &"
- "export DISPLAY=:99"
maven_build:
stage: build
script:
- "mvn clean package -Djava.awt.headless=true"
- "cat target/site/jacoco/index.html"
Now I get the headless exceptions again...
You try using xvfb program like in this post http://elementalselenium.com/tips/38-headless.
I have used xvfb to run browser from text terminal. Your case is basically the same.
The problem is that your program expects to be run in window environment, but you are running it from text terminal.
Finally I've found a solution!
I've created a DOCKER image which is prepared for GUI testing (using xvfb, thanks vbuhlev):
https://github.com/thorstenwagner/docker-java8-maven
In the .gitlab-ci.yml I've added the following lines:
before_script:
- "Xvfb :99 &"
- "export DISPLAY=:99"
You need to enable Headless Mode:
maven_build:
stage: build
script:
- "mvn clean package -Djava.awt.headless=true"