We Keep Coding

Using Jenkins and/or Ant, how can I rename a directory after a build step?

I am managing a java build process in Jenkins and after a number of projects are built I want to rename the distribution/release folder with something like the SVN revision number.
I can probably do this in a shell script from jenkins, but can I do it in the Ant config/build file as well? I think I can populate the env vars with the svn/jenkins details, but I don;t know how to reference those from an ant xml build file
For example, after I do a build I have something like:
/var/lib/jenkins/workspace/MyProject/dist/
which contains a bunch of stuff that was packaged.
I'd like to rename the leaf directory 'dist' to something like 'release.x' or even 'release.datetime'
Alternatively, what is considered best practice int he java world as far as "labeling" builds after they are done so one can tell them apart and manage them for release mgmt activities?

It turns out to be pretty simple - though I did not rename, rather I named the dist directory when i create it. here is the relevant bit in the ant build file:
> ... <property environment="env"/> <property name="dist"
> location="dist.${env.SVN_REVISION}"/>
Jenkins publishes the env vars when it spawns the ant tasks. I will probably come up with a better naming scheme, but this is the first step in making unique distributions/versions.

Configure Eclipse to pre-bundle App Engine classes into a single JAR for faster warm-up

After some discussion with a colleague from another company that also uses App Engine, he told me that he managed to cut down his app warm up time from ~15 seconds to ~5 seconds using the following steps:
Configure Eclipse to bundle classes produced during compilation into a single JAR file.
Configure Eclipse to upload this single JAR file instead of hundreds (or thousands) of separate Java classes during App Engine deployment.
He argues that during instance warm up, since the instance need to load only a single bundled JAR file instead of thousands of separate classes, the warm up would be significantly faster. Any thoughts or opinions about this?
I would definitely like to try it by myself, but I don't have enough Eclipse-mojo to know how to configure such steps. Does anyone know how to configure Eclipse or the Google Plugin for Eclipse to do the steps outlined above? (And have the deployed apps successfully runs in App Engine, of course)
Thank You,

What we did in Eclipse was:
In Project Properties > Java Build Path (Source tab) change output folder from war/WEB-INF/classes to something else, e.g. bin (I believe Eclipse complained about this once)
In Project Properties > Builders add a new builder (I used type "program builder" and pointed it to the jar executable in my JDK, but as others mentioned an Ant builder would work too). Obviously you should configure this builder to take its input from wherever you decided to write your class files to, and output to something like war/WEB-INF/lib/myclasses.jar
You can configure the jar builder to execute automatically as class files change (which in turn are usually automatically recompiled as your source files change).
There are some downsides, though. For some reason the google eclipse plugin gets confused by your changing the output directory of the java compiler. This means you will have to manually point to the war directory when deploying, and I believe you'll have to manually copy some GAE jars into the war/WEB-INF/lib folder.

I don't know how (or if) you can integrate it into eclipse, but it's fairly trivial to do with ant:
<import file="${appengine.sdk.dir}/config/user/ant-macros.xml" />
<target name="deploy">
<delete dir="${staging.dir}" />
<mkdir dir="${staging.dir}" />
<copy todir="${staging.dir}">
<fileset dir="war">
<exclude name="WEB-INF/classes/**" />
<exclude name="WEB-INF/appengine-generated/**" />
</fileset>
</copy>
<jar destfile="${staging.dir}/WEB-INF/lib/classes.jar" basedir="${classes.dir}" />
<appcfg action="update" war="${staging.dir}" />
</target>
I will add that I did not experience a 3X reduction in app startup time. I posted some experimental numbers in this thread:
https://groups.google.com/d/msg/google-appengine/dStBW4wIemY/K69f9ufDiN0J
What I found is that instead of varying wildly from 20-45s, it made my app consistently load in 20s. It has not subsequently remained this consistent, but I still jar my classes as a standard part of deployment now.

One way this can be achieved if by doing the deployment through Ant, as described in: https://developers.google.com/appengine/docs/java/tools/ant
Next you can modify the ant build.xml file to call the ant command for building the jar file. Just before the actual deployment you can either delete or move the compiled artifacts away. The build jar-file should be placed in the WAR/WEB-INF/lib folder.
Drawback of this solution is that you have to deploy through the build.xml, i.s.o. through the appengine eclipse plugin.

As stated in an earlier answer, the App Engine SDK supports packaging WEB-INF/classes into a jar file, which will end up in WEB-INF/lib/_ah_webinf_classes-0000.jar. You can activate this
using the appcfg tool with the option --enable_jar_classes.
using the Google Plugin for Eclipse by configuring the properties of either your WAR or EAR project: Project properties > Google App Engine > Deployment > "Package WEB-INF/classes as a jar"
For me, on App Engine 1.9.4, this resulted in only a minor improvement in instance spin-up (about 5-10 %, if any).
Note that this will package all files in WEB-INF/classes (not only .class ones). Following the change, I got an error message during instantiation about not being able to read the logging.properties file anymore; probably because the new jar file hasn't been read at that time:
Unable to read the java.util.logging configuration file, WEB-INF/classes/logging.properties
As a workaround, I changed the path in appengine-web.xml to WEB-INF/logging.properties and configured the maven-war-plugin to copy the file to that location:
<webResources>
<resource>
<directory>lib</directory>
<targetPath>WEB-INF/lib</targetPath>
</resource>
<resource>
<!-- Workaround: During GAE deployment, all files in WEB-INF/classes will be packaged into WEB-INF/lib/_ah_webinf_classes-0000.jar,
which prevents the logging.properties referenced in appengine-web.xml from being read. -->
<directory>src/main/resources</directory>
<includes>
<include>logging.properties</include>
</includes>
<targetPath>WEB-INF</targetPath>
</resource>
</webResources>

Note that as of version 1.7.4:
You can now package all the WEB-INF/classes/* classes into jar files. This can be done via the new
--enable_jar_classes option in the appcfg tools. By default, this option is not set.
http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes

I think the easiest way to do this would be to have a separate project that you write all the code in as well as store any other dependencies. When you're ready to deploy, Eclipse's export to runnable JAR file can repack all the dependencies and your classes into a single JAR file, which you would then use as your one dependency in your deployable Google App Engine project which would basically consist of any config files in the WEB-INF folder as well as other static resources + the one big JAR file.
Alternatively to the eclipse export, you can set that up as a Maven or Ant build which can also do the repacking.

He argues that during instance warm up, since the instance need to load only a
single bundled WAR file instead of thousands of separate classes, the warm up
would be significantly faster. Any thoughts or opinions about this?
I doubt it. WAR is just a ZIP file, which gets unpacked when it's deployed on the server. So there is an additional step, meaning the process can be equally fast (if unpacked when uploaded) or slower (if unpacked when instance is spun up).

Updating files in a WAR/JAR reguardless of their date

I have an ANT task that uses the jar task to update a few files inside of a previously built war. [The files are processed between compilation of a WAR and deliverable.] How can I get ANT to update all of the files I've specified to be updated? There is an attribute for jar called update = "[...]" that will allow you to either force a new creation of the Jar file or (add new files/"update existing ones if deemed necessary").
An example:
Java class(es) are compiled
Jars are created
A .war is created
A script runs to modify the Jars
The .war needs to update the Jars that were modified [the Jars are a subset of all of the files in the war]
The problem I'm running into is that the Jars that are being instructed to be updated aren't getting updated in the final step. The log claims that the just processed Jars are "out of date." Is there a way I can force the update to happen? If I switch "jar [...] update" back to false (which its by default) the correct Jar files get placed there, but the rest of the files in the war don't.
Note I realize that this could be moved to produce the WAR after everything is done. But this is not an option for me.
The ant task in question:
<target name="(the 'Rewaring task')" depends="step-3">
<echo>Adding modifed jars to war</echo>
<jar destfile="${output.war.dir}/existing.war" update="true">
<zipfileset dir="${output.jar.dir}/modded-jars" prefix="folder" />
</jar>
</target>
More specifically, the problem I'm having is with the "update" behavior.

It looks like the Jar task is correct.
Are you sure the files being updated have the correct Date modified time - that is what is actually compared in the zipfileset/update option?
I've seen this issue when signing Jars [and had the preservelastmodified option set to "true" (default : false)].
To fix it, you should only need to change the signing option.

Build project into a JAR automatically in Eclipse

I have an Eclipse project where I want to keep my Java project built into a JAR automatically. I know I have an option to export the project into a JAR; if I do a right click; but what I am really looking for is, that like Eclipse automatically builds a project's .class files and put them in target folder; it should also build a JAR automatically and copy the latest JAR at some or a specific location.
Is there a option to configure Eclipse in such a way, to build JARs automatically?
Just to make it clear for guys, patient enough to answer my question; I am not looking at ANT as solution; as I already use it, but what I would like it something that gets initiated automatically either with a time based trigger or immediate build with change.

You want a .jardesc file. They do not kick off automatically, but it's within 2 clicks.
Right click on your project
Choose Export > Java > JAR file
Choose included files and name output JAR, then click Next
Check "Save the description of this JAR in the workspace" and choose a name for the new .jardesc file
Now, all you have to do is right click on your .jardesc file and choose Create JAR and it will export it in the same spot.

Create an Ant file and tell Eclipse to build it. There are only two steps and each is easy with the step-by-step instructions below.
Step 1
Create a build.xml file and add to package explorer:
<?xml version="1.0" ?>
<!-- Configuration of the Ant build system to generate a Jar file -->
<project name="TestMain" default="CreateJar">
<target name="CreateJar" description="Create Jar file">
<jar jarfile="Test.jar" basedir="." includes="*.class" />
</target>
</project>
Eclipse should looks something like the screenshot below. Note the Ant icon on build.xml.
Step 2
Right-click on the root node in the project.
- Select Properties
- Select Builders
- Select New
- Select Ant Build
- In the Main tab, complete the path to the build.xml file in the bin folder.
Check the Output
The Eclipse output window (named Console) should show the following after a build:
Buildfile: /home/<user>/src/Test/build.xml
CreateJar:
[jar] Building jar: /home/<user>/src/Test/Test.jar
BUILD SUCCESSFUL
Total time: 152 milliseconds
EDIT: Some helpful comments by #yeoman and #betlista
#yeoman I think the correct include would be /.class, not *.class, as most
people use packages and thus recursive search for class files makes
more sense than flat inclusion
#betlista I would recomment to not to have build.xml in src folder

Check out Apache Ant
It's possible to use Ant for automatic builds with eclipse, here's how

This is possible by defining a custom Builder in eclipse (see the link in Peter's answer). However, unless your project is very small, it may slow down your workspace unacceptably. Autobuild for class files happens incrementally, i.e. only those classes affected by a change are recompiled, but the JAR file will have to be rebuilt and copied completely, every time you save a change.

Regarding to Peter's answer and Micheal's addition to it you may find How Do I Automatically Generate A .jar File In An Eclipse Java Project useful. Because even you have "*.jardesc" file on your project you have to run it manually. It may cools down your "eclipse click hassle" a bit.

Using Thomas Bratt's answer above, just make sure your build.xml is configured properly :
<?xml version="1.0" ?>
<!-- Configuration of the Ant build system to generate a Jar file -->
<project name="TestMain" default="CreateJar">
<target name="CreateJar" description="Create Jar file">
<jar jarfile="Test.jar" basedir="bin/" includes="**/*.class" />
</target>
</project>
(Notice the double asterisk - it will tell build to look for .class files in all sub-directories.)

Creating a builder launcher is an issue since 2 projects cannot have the same external tool build name. Each name has to be unique. I am currently facing this issue to automate my build and copy the JAR to an external location.
I am using IBM's Zip Builder, but that is just a help but not doing the real.
People can try using IBM ZIP Creation plugin.
http://www.ibm.com/developerworks/websphere/library/techarticles/0112_deboer/deboer2.html#download

Classpath including JAR within a JAR

Is it possible to specify a Java classpath that includes a JAR file contained within another JAR file?

If you're trying to create a single jar that contains your application and its required libraries, there are two ways (that I know of) to do that. The first is One-Jar, which uses a special classloader to allow the nesting of jars. The second is UberJar, (or Shade), which explodes the included libraries and puts all the classes in the top-level jar.
I should also mention that UberJar and Shade are plugins for Maven1 and Maven2 respectively. As mentioned below, you can also use the assembly plugin (which in reality is much more powerful, but much harder to properly configure).

You do NOT want to use those "explode JAR contents" solutions. They definitely make it harder to see stuff (since everything is exploded at the same level). Furthermore, there could be naming conflicts (should not happen if people use proper packages, but you cannot always control this).
The feature that you want is one of the top 25 Sun RFEs: RFE 4648386, which Sun, in their infinite wisdom, has designated as being of low priority. We can only hope that Sun wakes up...
In the meanwhile, the best solution that I have come across (which I wish that Sun would copy in the JDK) is to use the custom class loader JarClassLoader.

After some research I have found method that doesn't require maven or any 3rd party extension/program.
You can use "Class-Path" in your manifest file.
For example:
Create manifest file MANIFEST.MF
Manifest-Version: 1.0
Created-By: Bundle
Class-Path: ./custom_lib.jar
Main-Class: YourMainClass
Compile all your classes and run jar cfm Testing.jar MANIFEST.MF *.class custom_lib.jar
c stands for create archive
f indicates that you want to specify file
v is for verbose input
m means that we will pass custom manifest file
Be sure that you included lib in jar package. You should be able to run jar in the normal way.
based on: http://www.ibm.com/developerworks/library/j-5things6/
all other information you need about the class-path do you find here

Use the zipgroupfileset tag (uses same attributes as a fileset tag); it will unzip all files in the directory and add to your new archive file.
More information: http://ant.apache.org/manual/Tasks/zip.html
This is a very useful way to get around the jar-in-a-jar problem -- I know because I have googled this exact StackOverflow question while trying to figure out what to do. If you want to package a jar or a folder of jars into your one built jar with Ant, then forget about all this classpath or third-party plugin stuff, all you gotta do is this (in Ant):
<jar destfile="your.jar" basedir="java/dir">
...
<zipgroupfileset dir="dir/of/jars" />
</jar>

If you are building with ant (I am using ant from eclipse), you can just add the extra jar files
by saying to ant to add them...
Not necessarily the best method if you have a project maintained by multiple people but it works for one person project and is easy.
for example my target that was building the .jar file was:
<jar destfile="${plugin.jar}" basedir="${plugin.build.dir}">
<manifest>
<attribute name="Author" value="ntg"/>
................................
<attribute name="Plugin-Version" value="${version.entry.commit.revision}"/>
</manifest>
</jar>
I just added one line to make it:
<jar ....">
<zipgroupfileset dir="${external-lib-dir}" includes="*.jar"/>
<manifest>
................................
</manifest>
</jar>
where
<property name="external-lib-dir"
value="C:\...\eclipseWorkspace\Filter\external\...\lib" />
was the dir with the external jars.
And that's it...

Not without writing your own class loader. You can add jars to the jar's classpath, but they must be co-located, not contained in the main jar.

You need to build a custom class-loader to do this or a third-party library that supports this. Your best bet is to extract the jar from the runtime and add them to the classpath (or have them already added to the classpath).

I use maven for my java builds which has a plugin called the maven assembly plugin.
It does what your asking, but like some of the other suggestions describe - essentially exploding all the dependent jars and recombining them into a single jar

If you have eclpise IDE, you just need to export your JAR and choose "Package Required libraries into generated JAR". eclipse will automatically add the required dependant JARs into the generated JAR as well as generated some eclipse custom class loader that load these JARs automatically.

I was about to advise to extract all the files at the same level, then to make a jar out of the result, since the package system should keep them neatly separated.
That would be the manual way, I suppose the tools indicated by Steve will do that nicely.

Winstone is pretty good http://blog.jayway.com/2008/11/28/executable-war-with-winstone-maven-plugin/. But not for complex sites. And that's a shame because all it takes is to include the plugin.

Well, there is a very easy way if you're using Eclipse.
Export your project as a "Runnable" Jar file (right-click project folder from within Eclipse, select "Export..."). When you configure the export settings, be sure to select "Extract required libraries into generated Jar." Keep in mind, select "Extract..." and not "Package required libraries...".
Additionally: You must select a run-configuration in your export settings. So, you could always create an empty main( ) in some class and use it for your run configuration.
Anyway, it isn't guaranteed to work 100% of the time - as you will notice a pop-up message telling you to make sure you check the licenses of the Jar files you're including and something about not copying signature files. However, I have been doing this for years and have never encountered a problem.

Extracting into an Uber-dir works for me as we s should all be using root:\java and have outlets code in packages with versioning. Ie ca.tecreations-1.0.0. Signing is okay because the jars are intact from their downloaded location. 3rd party signatures intact, extract to c:\java. There’s my project dir. run from launcher so java -cp c:\java Launcher

In case you are using Spring Boot, you may want to have a look at this documentation: The Executable Jar Format
Java does not provide any standard way to load nested jar files (that
is, jar files that are themselves contained within a jar). This can be
problematic if you need to distribute a self-contained application
that can be run from the command line without unpacking.
To solve this problem, many developers use “shaded” jars. A shaded jar
packages all classes, from all jars, into a single “uber jar”. The
problem with shaded jars is that it becomes hard to see which
libraries are actually in your application. It can also be problematic
if the same filename is used (but with different content) in multiple
jars. Spring Boot takes a different approach and lets you actually
nest jars directly.
The Spring documentation also lists some alternative single Jar solutions:
Apache Maven Shade Plugin
JDotSoft JarClassLoader
One-JAR
Shadow Plugin (Gradle)

I would advise to use one jar and many libraries in separate jars, not in a single jar. Use separate jar from jar libraries.
Suppose you have such a folder structure:
path/yourApp/yourApp.jar
path/yourApp/lib/lib1.jar
path/yourApp/lib/megalib1.jar
path/yourApp/lib/supermegalib1.jar
All you have to do, add in MANIFEST.MF each of used jar.
Manifest-Version: 1.0
Main-Class: com.company.MyProgram
Class-Path: ./lib/lib1.jar ./lib/megalib1.jar ./lib/supermegalib1.jar
From within the manifest, you grant usage to each library.
Single all in one jar file might be easier to share and distribute, but in fact this doesn't give significant advantages over distributing as an archive and unpack it in some folder where you want to deploy. This will not make your program easier to maintain, faster. It will not make significant hdd usage difference.