I am trying to use semantic-release within a GitLab CI pipeline. I have the prepare stage working fine, but the publish stage always fails when I use anything other than mvn jar:jar deploy:deploy, but when i use those commands it deploys a jar that is 3kb big instead of a jar that is 10mb. So i can only assume that it is not gathering dependencies. There was a WARNING message about no files being marked for inclusion and the jar being empty. So I tried to package the project before calling deploy. It did not work.
The pipeline fails with no reason as to why. It just show that line as the culprit.
commands I have tried:
mvn clean install
mvn clean package deploy
mvn jar:jar deploy:deploy
mvn clean deploy:deploy
.. you get the idea.
Here is the prepare section that works:
verifyConditions:
- "#semantic-release/changelog"
- "#semantic-release/gitlab"
- "#semantic-release/git"
verifyRelease:
- path: "#semantic-release/exec"
cmd: echo -e "VERSION=${nextRelease.version}\nNEW_RELEASE=true" > RELEASE.env
prepare:
- path: "#semantic-release/exec"
cmd: if [ ! -d ".m2" ]; then mkdir .m2; cd .m2; touch settings.xml; echo $MVN_SETTINGS | base64 -d > 'settings.xml'; cd ..; fi; mvn versions:set -DnewVersion=${nextRelease.version} -B -gs .m2/settings.xml;
- "#semantic-release/changelog"
And here is the publish section that only works with jar:jar deploy:deploy but does not create the correct jar.
publish:
- "#semantic-release/gitlab"
- path: "#semantic-release/exec"
cmd: if [ ! -d ".m2" ]; then mkdir .m2; cd .m2; touch settings.xml; echo $MVN_SETTINGS | base64 -d > 'settings.xml'; cd ..; fi; mvn versions:set -DnewVersion=${nextRelease.version} -DremoveSnapshot=true clean deploy -B -gs .m2/settings.xml;
I'm extremely new to this, and I cannot see why:
1) trying clean deploy is causing this to fail and jar:jar deploy:deploy doesn't
2) how I can get semantic-release to create a jar with all dependencies for upload to our repository.
I should note that both Maven Shade plugin and Maven Deploy plugin are present in my pom.
This is an older run, but they all are formatted like this and tell you nothing about WHY it failed. Just that it did:
stderr: '/bin/sh: line 1: 425 Killed mvn clean deploy -B -gs .m2/settings.xml\n',
failed: true,
signal: null,
cmd: '/bin/sh -c mvn $MAVEN_CLI_OPTS versions:set -DremoveSnapshot; mvn clean deploy -B -gs .m2/settings.xml',
timedOut: false,
killed: false,
pluginName: '#semantic-release/exec' }ERROR: Job failed: command terminated with exit code 1
First of all, for deployment use mvn clean deploy. The other combinations you presented do not produce sensible output.
If you want to package the dependencies into your jar, you need to properly configure the Maven shade plugin (configuring the deploy plugin is usually not necessary). Without your pom.xml, I can only guess, but I would say that the error is probably in that configuration.
BTW: Only put the dependencies into the jar if the jar is meant to run standalone. If, on the other hand, you write a Java library to be used as dependency, don't do it.
Related
I am using the sh file which contains all the maven configuration, secret keys, maven command and all, I I want to execute this sh file from inside the Dockerfile so that when I run the container sh file will execute.
this is how my Docker file looks like:
#using alpine jdk
FROM somewhere.docker.hub/build/alpine:latest AS fetch
#MVN as build tool
FROM somewhere.docker.hub/build/maven:3.5.3-jdk-8 AS build
#Settings.xml for downloading dependencies from nexus repository
ARG MVN_SETTINGS=settings.xml
#Defining the current working repository
WORKDIR /usr/src/app
#Coping the pom.xml into working directory
COPY pom.xml /usr/src/app
COPY ${MVN_SETTINGS} /usr/src/app/settings.xml
# Download the package and make it cached in docker image
#RUN mvn -B -f ./pom.xml -s settings.xml dependency:resolve
#Coping the source code into the working repository
COPY . /usr/src/app
#Run the sh file
#RUN chmod a+x /usr/src/app/DockerTest.sh
# Not using the full path of DockerTest.sh file because it is already in the work directory as defined above
RUN chmod a+x DockerTest.sh
DockerTest.sh file looks like this.
#!/bin/sh
mvn clean install
mvn test
when I create the docker image and run the image, it shows this error.
I have tried lots of things it is still not working, it shows this error
No goals have been specified for this build. You must specify a valid lifecycle phase or a goal in the format : or :[:]:. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy. -> [Help 1]
First of all you cannot combine two Docker images by using multiple FROM lines one after another. The reason it's even possible to specify more than one is to enable multi-stage builds.
Second, if you're not using this line:
RUN mvn -B -f ./pom.xml -s settings.xml dependency:resolve
you may as well comment this one too:
COPY pom.xml /usr/src/app
since your POM will be copied along with the rest of the source code.
Third, judging from your Dockerfile, you have several settings.xml files in your working directory and you want to choose one base on a build argument. However, if you have say settings.xml and settings.prod.xml and you set MVN_SETTINGS to settings.prod.xml, then /usr/src/app/settings.xml will be overwritten by this line:
COPY . /usr/src/app
with settings.xml.
Fourth, mvn clean install runs tests as well, there is no need to do mvn test separately.
Finally, for the issue you have noticed. I'm not sure what you expect to happen when you run the image, but you did not specify any entrypoint or command to be run in your Dockerfile, so your image inherits the default one from its base. Now assuming that by somewhere.docker.hub/build/maven:3.5.3-jdk-8 you actually mean docker.io/library/maven:3.5.3-jdk-8 then the command is mvn and the entrypoint is /usr/local/bin/mvn-entrypoint.sh (see Dockerfile for a slightly newer image here).
With all that said, assuming you want to run DockerTest.sh when your image is run, your Dockerfile should look like this:
FROM maven:3.5.3-jdk-8
ARG MVN_SETTINGS=settings.xml
WORKDIR /usr/src/app
COPY . .
COPY ${MVN_SETTINGS} settings.xml
RUN chmod a+x DockerTest.sh
CMD ./DockerTest.sh
Also, if you actually want to use your settings.xml during build you need to change your DockerTest.sh to:
#!/bin/sh
mvn --settings /usr/src/app/settings.xml clean install
You could also remove unnecessary copying the same file twice by doing:
FROM maven:3.5.3-jdk-8
ARG MVN_SETTINGS=/usr/src/app/settings.xml
ENV MVN_SETTINGS=${MVN_SETTINGS}
WORKDIR /usr/src/app
COPY . .
RUN chmod a+x DockerTest.sh
CMD ./DockerTest.sh
and
#!/bin/sh
mvn --settings "$MVN_SETTINGS" clean install
I have downloaded Apache spark and trying to build it with MAVEN as suggested here. http://spark.apache.org/docs/1.0.0/building-with-maven.html
But I am not able to resolve the error after running the command -
build/mvn -DskipTests clean package run , the error is- build/mvn: No such file or directory .
I checked by running mvn -v and also JAVA_HOME is set to the JDK.(screen shot attached ) .
Please help to resolve the problem.command promt output
mvn -DskipTests clean package run
You don't need to use build/mvn. I am assuming you have mvn installed somewhere within the system.
I am working on a Java project. In my project multiple dependency projects are these. So I created a bat file to build all projects one by one. Please see the logic I used to achieve this.
set x=proj1-prx
set y=proj2-prx
set z=proj3-prx
set LIST=(%x% %y% %z% )
echo Checkout and deploy started
for %%G in %LIST% do (
set _module=%%G
set _value=!%%G!
echo Checkout module - %%G
svn checkout %SVNHOST%/%%G/%REPO% %WORKSPACE%\%%G\%REPO% --username %USER% --password %PASSWORD%
echo Install module to AEM - %%G
mvn clean install -Dskiptests -f %WORKSPACE%\%%G\%REPO%\pom.xml -l output.log
#ECHO OFF
)
echo Checkout and deploy finished
This file is executing well , Log file also creating but each time for loop building a project the build result overrides in to log file. I want build result of all project. Please help me friends
I want build result of all project. Please help me friends
You cannot depend on mavens logger implementation. Use the OS stdout/stderr redirection:
at the beginning of your batch file delete the old log content:
echo > output.log
then change the maven call:
mvn clean install -Dskiptests -f %WORKSPACE%\%%G\%REPO%\pom.xml >> output.log 2>&1
I am getting this strange error at the end of the process of creating a docker image from a Dockerfile:
/bin/sh: 1: gradle: not found
INFO[0003] The command [/bin/sh -c gradle test jar] returned a non-zero code: 127
The relevant part of the Dockerfile:
FROM debian:jessie
[...]
RUN curl -L https://services.gradle.org/distributions/gradle-2.4-bin.zip -o gradle-2.4-bin.zip
RUN apt-get install -y unzip
RUN unzip gradle-2.4-bin.zip
RUN echo 'export GRADLE_HOME=/app/gradle-2.4' >> $HOME/.bashrc
RUN echo 'export PATH=$PATH:$GRADLE_HOME/bin' >> $HOME/.bashrc
RUN /bin/bash -c "source $HOME/.bashrc"
RUN gradle test jar
[...]
The command I am using is: docker build -t java_i .
The strange thing is that if:
I run a container from the previous image commenting out RUN gradle test jar (command: docker run -d -p 9093:8080 -p 9094:8081 --name java_c -i -t java_i),
then I log into that container (command: docker exec -it java_c bash),
then I manually check the gradle environment variables finding them,
then I manually run that commented out command from within the running container (gradle test jar):
I eventually get the expected output (the compiled java code in the build folder).
I am using Docker version 1.6.2
I solved the problem using the ENV docker instructions (link to the documentation).
ENV GRADLE_HOME=/app/gradle-2.4
ENV PATH=$PATH:$GRADLE_HOME/bin
This command /bin/bash -c "source $HOME/.bashrc" means that you create a new non-interactive process and run a command in it to set environment variables there. Which does not affect the parent process. As soon as variables are set, process exits. You can check this by running something like this:
RUN /bin/bash -c "source $HOME/.bashrc; env"
RUN env
What should be working is this option:
RUN source ~/.bashrc
And the reason why it works when you log in, is because the new process reads already updated ~/.bashrc.
I was trying to install same version with JDK 11.0.7 but gradle-2.4 does not work. and got below error
FAILURE: Build failed with an exception.
* What went wrong:
Could not determine java version from '11.0.7'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
I install later version to fix the above issue after installation.
Posting as an answer might help someone else.
FROM openjdk:11.0.7-jdk
RUN apt-get update && apt-get install -y unzip
WORKDIR /gradle
RUN curl -L https://services.gradle.org/distributions/gradle-6.5.1-bin.zip -o gradle-6.5.1-bin.zip
RUN unzip gradle-6.5.1-bin.zip
ENV GRADLE_HOME=/gradle/gradle-6.5.1
ENV PATH=$PATH:$GRADLE_HOME/bin
RUN gradle --version
You can use multi-stage builds and the Gradle Docker image (no need to install Gradle...) to build the application then use the result in the runtime container:
# Build
FROM gradle AS build
WORKDIR /appbuild
COPY . /appbuild
RUN gradle --version
# here goes your build code
Once the Gradle build is done, switch to the runtime container:
# Runtime
FROM openjdk:8-jre-alpine
# more stuff here...
COPY --from=0 appbuild/<somepath>/some.jar application.jar
# more stuff here...
The COPY command copies the build artifacts from the build phase to the runtime container (in this case a jar file).
I am new to hadoop and maven. I will like to compile the hadoop 2.0.3 from the source and install it. I am following instructions from
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html
So far, i have managed to download hadoop source code and from the source directory issued "mvn clean install -Pnative"
Next i tried to execute mvn assembly:assembly, but i get following error:
Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.3:assembly (default-cli) on project hadoop-main: Error reading assemblies: No assembly descriptors found. -> [Help 1]
Please help so that i can move forward.
Also, the above mentioned install link, does not mention what should be the value of "$HADOOP_COMMON_HOME/$HADOOP_HDFS_HOME"
I compiled 1.0.4 just as an academic exercise. Not sure if it will be valid for 2.0.3
This should be done (on Ubuntu) before you start compilation to make sure that all needed stuff is there:
sudo apt-get -y install maven build-essential protobuf-compiler autoconf automake libtool cmake zlib1g-dev pkg-config libssl-dev
I did not had subversion so I did this too:
sudo apt-get install subversion
After that I checked out the code:
svn checkout http://svn.apache.org/repos/asf/hadoop/common/tags/release-1.0.4/ hadoop-common-1.0.4
Then went to newly created folder “hadoop-common-1.0.4″ and gave command:
ant clean package
You can refer to my blog for the whole story:
http://hadoopmagic.wordpress.com/