I have a Maven project which includes in the resources a set of database scripts relating to features which have been implemented.
I've created a separate Maven project which produces a jar file used to create a diff between the database scripts and what has been implemented in the database. This executable takes a command line argument which specifies the database details and the output path for the diff file.
What I'd like to do (and I'm open to suggestions on whether this is the best way to tackle this) is to include the diff jar in my project and to execute a goal from Jenkins which executes the jar and stores the diff file as an artifact of the build.
Could anybody suggest a way to do this as I'm not too familiar with Jenkins.
If this is a jar you can execute, you can just use the java exec plugin.
Once you've got that working to generate the output then you need to set up maven to run the command line that generates that output. This is pretty straight forward. Just add it as a build step.
Then to make it available as an artifact, tell Jenkins to archive the artifact by pointing it to the path of the diff.
Related
I have a build setup in Jenkins, which compiles a simple Git hosted Maven project into a jar and publishes the jar into Artifactory.
How can I setup the build job to tag the sources which were used for the build with e.g. a build number (or similar), in order to later be able to identify which sources went exactly into this particular build. This build tag should also be visible in git's remote repository, not just in the local version on the Jenkins build server.
Ideally I'd also like to package this "build tag" into the jar (I suppose in a file inside the jar) so I can always correlate the jar to which source files were used to create this jar.
Your help is much appreciated.
Just use the Jenkins Artifactory Plugin or the Maven Artifactory Plugin.
Both generate a BuildInfo metadata that include the information you need and much, much more. All this information is attached to the artifacts and make the artifacts traceble without need to embed this info in the filenames or into the archives themselves.
Please take some time to watch this screencast. It explains in detail what you can get and why it's important.
How can I setup the build job to tag the sources which were used for the build with e.g. a build number (or similar), in order to later be able to identify which sources went exactly into this particular build. This build tag should also be visible in git's remote repository, not just in the local version on the Jenkins build server.
This can be done using the standard Git plugin. Underneath where you configure the repository to clone, click "Add" on "Additional Behaviours", and choose "Create a tag for every build"
Ideally I'd also like to package this "build tag" into the jar (I suppose in a file inside the jar) so I can always correlate the jar to which source files were used to create this jar.
This tag should then be available as an environment variable during the build steps (I think it's called GIT_TAG) - you can have your existing build step put this value into the file or add a pre-build "Execute Shell" step which does something like this:
$ echo $GIT_TAG > git.tag
I have a Spark Application which I initially created using maven. I converted my maven project into an Eclipse project, and I am now working on it via Eclipse.
I also have my own version of the Spark source code to which I have made some modifications and added additional methods. I am able to build my version of Spark and obtain the jar file for spark-core.
In my Eclipse project I now replace the spark-core that is in the build path with the newly built spark-core jar. I am able to utilize the new methods I created and verify this by running the application through Eclipse.
Now, I am trying to submit my application through the command line. I use the spark-submit command from the Spark project I have modified. When I submit my jar, I get an error saying the method I created does not exist. I'm not sure what I'm doing wrong. I would appreciate some insight on what I should be doing.
UPDATE
I now understand that there are certain jars that I need to have from my custom build. Exactly which jars are these? I understand the spark-assembly jar is one. Are there any others?
I am trying to run an executable jar file through jenkins. What I am trying to achieve here is I have an executable jar file created and is residing locally in my machine. I want to put it in Git and then run it in jenkins. Is there some how I can do this? I am totally new to Jenkins and if someone can help it will save me a lot of time.
You can do nearly everything you want with Jenkins since you can have it run arbitrary scripts / executables.
If you want to put this .jar in git, then the Jenkins jobs will retrieve it when it fetches the repo. Then you just need to add a script shell step to the build, and to put your command line java my.jar my.class ....
On the other hand, it's sometimes frowned up to put binary files in git. It's true in particular if the .jar will need to be often updated. Hence you might want to provide the .jar to the Jenkins job using alternative methods. For example using a maven repository.
I am having trouble adding dependancies that are located in my working directory, to my .jar. When I try to run the .jar from command line I get errors saying that files I am attempting to access do not exist. Why does netbeans not include the working directory in the build dependancies, seems like a pretty easy and obvious thing to do....
I am simply trying to read a file, located at res/settings/settings.txt. I have made my working directory the res folder so that in netbeans I access that file successfully through 'settings/settings.txt'. I attempted to make the settings folder a library so it would be added to the build, however it refused to copy it because it was a directory.
How can I add this file to the build so i can run it from command line?
Why should it? The working directory is simply a "test" location for your application. The build process is not a "packaging" process.
Netbeans provides you with a build.xml file which can you can modify to perform custom actions, but consider this, each time you build your application, these custom build actions/targets could be executed. That could add considerable time to your build process. If you're just making small changes, this might not be desirable.
Two solutions then come to mind.
You could create custom build targets within the build.xml file which you would need to trigger by right-clicking the build.xml file (in the files view) and selecting Run Target and choosing the required target from the cascading menu...from experience this is not as convenient as it sounds. You could also run the build.xml file from the command line, specifying the targets you want...
Make another ant script which packaged your application separately. This is similar to the first comment except that you completely divorce the build and package processes.
I work on a very large application, which was taking upwards of 15 mins to build (completely). We wrote a utility which went through all the Netbeans project properties and built it's own dependency map, which then generated an Ant build script. We then included this in our own packaging script, so we could build the application and package it within a single pass when we wanted to create a release. This reduced the build down to something like 3 minutes (don't ask why, it just did), it also meant that we could remove the immediate dependency on Netbeans from our build process.
So. The basic answer is. Netbeans is a builder, not a packager. In order to supply this support, you're going to have to write something yourself. The easiest thing to do would be to simply write some Ant script to perform this aciton. Where you do this is up to you.
So I've been pigeon-holed into writing some Jython code. I've been using the latest version of Eclipse IDE with the PyDev plugin for development. Up until now, things have been moderately tolerable. I've gotten all my Python scripts working and I'm successfully including a couple of JAR files and the class directory of another Java project as external dependencies. Everything seems to run fine through the Eclipse IDE.
Now I need to package everything up and deploy it. From what I can gather, the best way to do this would be to package everything up in a JAR file. The Jython documentation suggests starting out with the jython.jar file and adding to it. OK. So I modify my main python module and start adding all my python source to the JAR.
It executes but of course can't find all the external dependencies.
How is one supposed to add the external JAR files so that they are correctly seen by the Jython interpreter? How is one supposed to manage more complex dependencies in a setup like this?
Is there a plugin for Eclipse or maybe something like Ant or Maven that can handle all of these steps for me with the push of a button?
I can't be the first person that has needed to deploy Jython code with complex dependencies can I?
I've made some headway on getting this all working so I thought I would put some notes here in case they help anyone else out. I'd still like to hear from others on their experiences trying to put together something like this.
It turns out that Eclipse as of 3.5 has a project export option for Java -> Runnable JAR File. If you use this option, you can point to a Java main class in the export wizard. You also have the option to have it repackage all the JARs that you are dependent on in your new JAR file. Make sure to check the box to save the export as an ANT build so that you can repeat the process quickly. NOTE that the first time you do this through the interface, it may fail, but it will still have created a JAR file.
Now here's where it gets strange. To track all the dependencies, I am still using a mostly incomplete Maven build in my project. I create the Maven .POM file. And I told Maven what my external JAR dependency was. I then told Maven to do a dependency update for me. It pulled everything into my Maven repository as expected.
Now when I do my ANT build, it appears that it is getting its list of JARs to include in the final build from Maven. I'm not really sure if it is supposed to work that way. I'm also not 100% sure that it is working. I guess I'll find out when I have to add another external JAR to the project.
Anyways, if you follow this question you'll see that you can take the latest builds of Jython and pull the org.python.util.JarRunner.java file out and use it in your own project. This is you Java.main class that you will need to point your ANT build at. From there, convert your main Python/Jython script to be the run script that was talked about in that question.
Next, build up another copy of the Jython JAR file in your Jython directory. This one should have the /Lib directory pulled into the JAR. Save that off and then point your Eclipse IDE Jave Build option for your PyDev project at that JAR as an external dependency. Your JarRunner will now work and execute the run.py file under Jython.
If all that works, you should then be able to rerun the ANT exported build from earlier and you will end up with a single JAR file that you can execute on the command line as:
java -jar {yourjar} args
And distribute to your customers without any additional dependencies.
If that all seems a little bit convoluted, it is. If anyone has a better way of setting this all up using Eclipse, please let me know.
Make your life easier and just use Maven and the mavenjython compile plugin.
See my answer of this question: Using Jython with Maven
You get full automation of the build and deploy process, and the result is a jar that includes jython and all other dependencies.