I have a git repository with 2 modules in it. One is SpringBoot based backend module and another one is VueJS based frontend module.
app-root
- backend
- frontend
I have a declarative style Jenkinsfile to build my backend module with relevant maven commands. I want to execute all those maven commands from inside backend directory.
One option is to use dir("backend") { ....} for all commands separately, which looks ugly to me.
Is there any other option to instruct Jenkins to execute the entire pipeline from inside a sub-directory?
I ended up with a "prepare project" stage that puts the subdirectory content into the root.
It's also probably a good idea to remove all the root contents (stage "clean") to be absolutely sure there are no leftovers from previous builds.
node {
def dockerImage
stage('clean') {
sh "rm -rf *"
}
stage('checkout') {
checkout scm
}
// need to have only 'backend' subdir content on the root level
stage('prepare project') {
// The -a option is an improved recursive option, that preserve all file attributes, and also preserve symlinks.
// The . at end of the source path is a specific cp syntax that allow to copy all files and folders, included hidden ones.
sh "cp -a ./backend/. ."
sh "rm -rf ./backend"
// List the final content
sh "ls -la"
}
stage('build docker image') {
dockerImage = docker.build("docker-image-name")
}
stage('publish docker image') {
docker.withRegistry('https://my-private-nexus.com', 'some-jenkins-credentials-id') {
dockerImage.push 'latest'
}
}
}
You can have jenkinsfiles inside backend and frontend modules and just point to them on each pipeline, eg:
and on the pipeline itself you just cd to the submodule and execute its commands:
pipeline {
agent any
stages {
stage('build') {
steps {
sh 'cd backend'
sh 'mvn clean package'
}
}
//other stages
}
}
if you don't want to cd to a sub module you can use sparse checkouts but in this case you have to change the path to the jenkinsfile accordingly because it will be on the root folder.
Related
I have a Java project with some Cucumber tests and some regular JUnit tests, managed by Maven.
I want to run the tests in Jenkins, using Docker, so I wrote this Jenkinsfile:
pipeline {
agent {
docker {
image 'maven:3.6.1'
}
}
stages {
stage('build') {
steps {
sh 'mvn clean verify -Dmaven.test.failure.ignore=true'
}
post {
success {
junit 'target/surefire-reports/**/*.xml'
}
}
}
}
}
When I run the build, the regular tests pass but the Cucumber tests fail with:
Error Message
URI has a query component
Stacktrace
java.lang.IllegalArgumentException: URI has a query component
It's not until I disable trimStackTrace for maven-surefire-plugin that I also get the details:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<trimStackTrace>false</trimStackTrace>
</configuration>
</plugin>
java.lang.IllegalArgumentException: URI has a query component
at java.base/java.io.File.<init>(File.java:427)
at cucumber.runtime.io.ZipResourceIterator.<init>(ZipResourceIterator.java:22)
at cucumber.runtime.io.ZipResourceIteratorFactory.createIterator(ZipResourceIteratorFactory.java:24)
at cucumber.runtime.io.ZipThenFileResourceIteratorFactory.createIterator(ZipThenFileResourceIteratorFactory.java:22)
at cucumber.runtime.io.DelegatingResourceIteratorFactory.createIterator(DelegatingResourceIteratorFactory.java:49)
at cucumber.runtime.io.ClasspathResourceIterable.iterator(ClasspathResourceIterable.java:35)
at cucumber.runtime.io.ResourceLoaderClassFinder.getDescendants(ResourceLoaderClassFinder.java:25)
at cucumber.runtime.Reflections.instantiateSubclasses(Reflections.java:34)
at cucumber.runtime.BackendModuleBackendSupplier.loadBackends(BackendModuleBackendSupplier.java:52)
at cucumber.runtime.BackendModuleBackendSupplier.get(BackendModuleBackendSupplier.java:39)
at cucumber.runner.ThreadLocalRunnerSupplier.createRunner(ThreadLocalRunnerSupplier.java:42)
at cucumber.runner.ThreadLocalRunnerSupplier.access$000(ThreadLocalRunnerSupplier.java:13)
at cucumber.runner.ThreadLocalRunnerSupplier$1.initialValue(ThreadLocalRunnerSupplier.java:22)
at cucumber.runner.ThreadLocalRunnerSupplier$1.initialValue(ThreadLocalRunnerSupplier.java:19)
at java.base/java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:195)
at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:172)
at cucumber.runner.ThreadLocalRunnerSupplier.get(ThreadLocalRunnerSupplier.java:38)
at cucumber.api.junit.Cucumber$RunCucumber.evaluate(Cucumber.java:146)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
This is quite puzzling because:
when running mvn verify locally, it works
when using agent any in the Dockerfile, instead of the maven Docker image, it works
when using maven-surefire-plugin version 2.22.2 instead of 3.0.0-M3, it works
I logged in as Jenkins and looked in the workspace:
$ sudo su - jenkins
$ ls /var/lib/jenkins/workspace/question-mark-dir_master
'?' Jenkinsfile LICENSE pom.xml README.md src target
There's a question mark (?) directory in there, and it contains the .m2 directory, with all the artifacts needed by the project.
I suspect this could be the cause of the exception, because then there are question marks in the classpath, and question marks are the ones who introduce the query component of URIs.
For the moment I can downgrade to maven-surefire-plugin version 2.22.2 as a workaround.
But what is the actual problem here?
Could it be a bug in the new maven-surefire-plugin? Judging by the stack trace I wouldn't bet on that.
Is it a problem in Jenkins or in the Maven docker image (which one?), that causes a directory to have such an unfortunate name?
Am I doing something wrong?
If you want to try this out, I have a MVCE.
In my local Jenkins installation, I created a new project of type "Multibranch Pipeline" and in "Branch Sources" I added as the git project repository my local git project. No other changes.
By combining TYY's comment to the question and the documentation of the maven docker image, I was able to configure the Jenkinsfile in a way that avoids creating a question mark directory.
We need to inform Maven of the user's home directory, and map that to a directory outside the container.
pipeline {
agent {
docker {
image 'maven:3.6.1'
args '-v /var/lib/jenkins:/usr/src/mymaven -w /usr/src/mymaven'
}
}
stages {
stage('build') {
steps {
sh 'MAVEN_OPTS="-Duser.home=/usr/src/mymaven" mvn clean verify -Dmaven.test.failure.ignore=true'
}
// ...
}
}
}
-v /var/lib/jenkins:/usr/src/mymaven maps the /var/lib/jenkins directory on the host to the /usr/src/mymaven directory inside the container
-w /usr/src/mymaven sets the working directory inside the container
MAVEN_OPTS="-Duser.home=/usr/src/mymaven" sets the user.home Java property for Maven
Proof that it works, on a branch of the MCVE.
Hello I'm new to jenkins and getting this issue. I'm using jenkins in windows azure
mvn clean package /var/lib/jenkins/workspace/vcc#tmp/durable-b5407f14/script.sh: 2:
/var/lib/jenkins/workspace/vcc#tmp/durable-b5407f14/script.sh: mvn:
not found.
Jenkinsfiles:
node {
stage('init') {
checkout scm
}
stage('build') {
sh '''
mvn clean package
cd target
cp ../src/main/resources/web.config web.config
cp todo-app-java-on-azure-1.0-SNAPSHOT.jar app.jar
zip todo.zip app.jar web.config
'''
}
stage('deploy') {
azureWebAppPublish azureCredentialsId: env.AZURE_CRED_ID,
resourceGroup: env.RES_GROUP, appName: env.WEB_APP, filePath: "**/todo.zip"
}
}
can any body help me how can I resolve this mvn issue.
P.S I'm following this tutorial
https://learn.microsoft.com/en-us/azure/jenkins/tutorial-jenkins-deploy-web-app-azure-app-service
You may try to add maven tool to your pipeline:
tools {
maven 'M3'
}
stages {
stage('init') {
checkout scm
}
stage('build') {
sh '''
mvn clean package
cd target
cp ../src/main/resources/web.config web.config
cp todo-app-java-on-azure-1.0-SNAPSHOT.jar app.jar
zip todo.zip app.jar web.config
'''
}
stage('deploy') {
azureWebAppPublish azureCredentialsId: env.AZURE_CRED_ID,
resourceGroup: env.RES_GROUP, appName: env.WEB_APP, filePath: "**/todo.zip"
}
}
I add this line right before sh command in the build stage : def mvnHome = tool name: 'Apache Maven 3.6.0', type: 'maven'
and instead of mvn you should use ${mvnHome}/bin/mvn
thank this youtube film to help me.
pipeline{
stage('com'){
def mvnHome = tool name: 'Apache Maven 3.6.0', type: 'maven'
sh "${mvnHome}/bin/mvn -B -DskipTests clean package"
}
}
You may wanna check if Jenkins has the pipeline-maven plugin installed.
If you don't have it, search and install the pipeline-maven plugin.
Once the plugin is installed, you can use maven as follows
node{
stage('init'){
//init sample
}
stage('build'){
withMaven(maven: 'mvn') {
sh "mvn clean package"
}
}
}
I am successfully able to run the WSDLToJava class from the command line to generate JaxB classes from a WSDL.
java -Xmx128M
-cp "C:\cxf\apache-cxf-3.1.6\lib\cxf-manifest.jar;
C:\java\jdk1.7.0_80\lib\tools.jar"
-Djava.util.logging.config.file="C:\cxf\apache-cxf-3.1.6
\etc\logging.properties"
org.apache.cxf.tools.wsdlto.WSDLToJava
-d generated -frontend jaxws21 -b C:\Project\jaxb-bindings.xml
C:\Project\Service.wsdl
How can I run the same command line from a "build.gradle" file? I am completely new to Gradle.
Thank you in advance for your help.
Pete
While there are some Gradle plugins that implement this, I think it's just as easy to make the call directly via a JavaExec task. Add the following to your build.gradle:
ext.cxfVersion = "3.1.6"
configurations {
wsdl2java
}
dependencies {
wsdl2java("org.apache.cxf:cxf-tools-wsdlto-core:${cxfVersion}")
wsdl2java("org.apache.cxf:cxf-tools-wsdlto-frontend-jaxws:${cxfVersion}")
wsdl2java("org.apache.cxf:cxf-tools-wsdlto-databinding-jaxb:${cxfVersion}")
}
task wsdl2java(type: JavaExec) {
main = "org.apache.cxf.tools.wsdlto.WSDLToJava"
classpath = configurations.wsdl2java
// Uncomment to add JVM arguments if necessary (e.g. for cert-based security)
//jvmArgs = [
// "-Djavax.net.ssl.keyStore=${keystorePath}",
// "-Djavax.net.ssl.keyStorePassword=${keystorePassword}"
//]
args = [
"-d", "src/gen/java",
"example.wsdl"
// Uncomment and remove the previous line to run for multiple WSDL files
//"-wsdlList, "wsdls.txt"
]
}
// If you want to be able to debug the wsdl2java task from Eclipse, add the following
//eclipse {
// classpath {
// plusConfiguration += [configurations.wsdl2java]
// }
//}
Execute the task by running gradlew wsdl2java.
I am trying to deploy my Java build to CentOS using Gradle ssh plugin
remotes {
ftc {
role 'masterNode'
host = '173.199.123.42'
user = 'root'
password = 'myPass'
}
}
ssh.settings {
knownHosts = allowAnyHosts
}
task deploy {
doLast {
ssh.run {
session(remotes.ftc) {
execute 'pkill -f \'java.*chat\'', ignoreError: true
remove 'build/libs/chat.jar'
put from: file('build/libs/chat.jar'), into: '/root/test'
execute 'nohup java -jar /root/test/chat.jar &'
}
}
}
}
It works except it never finish, it get stuck on execute nohup java -jar /root/test/chat.jar &
How can I make it to run in background?
Using nohup usually involves also redirection of the IO to some file so the file descriptors in the connections can be closed:
execute 'nohup java -jar /root/test/chat.jar & &> /tmp/log'
I have a maven project which will generate a JAR file, if you build it as Maven Install.
Inside this project, I have a code like this -
public class GameBot extends GameController {
public int botNumber = 1
Static String botName = "Bot1";
...// Rest of the code..
}
I want to create around 150 JAR files from this project with incremented botNumbers. For example - JAR File 1 should contain botNumber = 1 and botName = Bot1
JAR file 2 should contain botNumber = 2 and BotName = Bot2 .. So on till 150.
I used to build each of these manually and now it looks cumbersome to build each time this way. Because, if I make any small change in the code, I have to build all 150 Files again.
Does any one have some suggestions as in Can I automate the process and get all 150 JAR in a folder built for me?
Thanks.
Why dont you externalize the botNumber to a property file and using shell script or batch file create the property file and build the maven project.
Java Code
Properties properties = new Properties();
properties.load(this.getClass().getResourceAsStream("/bottest.properties"));
botNumber = Integer.parseInt(properties.getProperty("botnumber"));
botName = "Bot"+botNumber;
Shell Script
#!/bin/sh
mvn clean;
for i in {1..5}
do
echo "botnumber=$i" > ./src/main/resources/bottest.properties;
mvn install -Dmaven.test.skip;
cp target/bottest-1.0-SNAPSHOT.jar target/bootest$i.jar
done
Batch File
echo off
call mvn clean
for /l %%x in (1, 1, 5) do (
echo botnumber=%%x > .\src\main\resources\bottest.properties
call mvn install -Dmaven.test.skip
copy target\bottest-1.0-SNAPSHOT.jar target\bootest%%x.jar
)
The above example is for 5 times.