We Keep Coding

"Override" properties of an imported Ant file

I´d like to override some properties of an imported Ant-File which handles most of the basic stuff to deploy an application. First of all I know that Properties in Ant are immutable. But I can think of two ways to "override" properties for my needs and I´d like to know, which one should be the prefered approach, and/or if there are any things to consider by doing it one way or the other.
Imagine we have following master-build-script, which I like to import in my build-script:
<project name="Application" default="build" basedir=".">
<property name="overridden" value="false" />
<target name="build">
<echo message="Value of overridden is ${overridden}" />
</target>
</project>
Now I want to override the overridden property. In both cases I import the master-build-file. I use import over include, because I also want to override some targets. I know of two possible solutions to override them:
1) Since Properties are immutable I can just define the properties I´d like to "override" by defining them before I actually import the master-file:
<project name="MyApplication" default="buildApplication" basedir=".">
<property name="overridden" value="true" />
<import file="master-build.xml" />
<target name="buildApplication">
<antcall target="build" />
</target>
</project>
2) I define Params within the Ant call which seems to redefine a properties value as well:
<project name="MyApplication" default="buildApplication" basedir=".">
<import file="master-build.xml" />
<target name="buildApplication">
<antcall target="build" >
<param name="overridden" value="true"/>
</antcall>
</target>
</project>
In both cases the result is:
build:
[echo] Value of overridden is true
BUILD SUCCESSFUL
Would anyone be so kind and could explain me the difference and might also explain why I should use one way over the other, or even should use a different approach to get the same result?

The main difference between your first and second examples is that in the second case the property is only defined for the sub-build, which might sometimes be what you want, rather than globally for the build.
Other ways you can preemptively define properties globally...
Pass as JVM args when you execute ant, e.g.
ant -Doverridden=true
Define the properties in a file and load at start of your build, e.g.
build.properties:
overridden=true
build.xml:
<project name="MyApplication" default="buildApplication" basedir=".">
<property file="build.properties/>
A useful variant is to load user definable properties from the user home directory, e.g. something like:
<property file="${user.home}/MyApplication/build.properties/>
The advantage of both of these methods over the examples you gave are that the properties are overridden without changing the build file.

Ant versions 1.8 and later come with the <local/> task which allows you to declare a property as local. This might be what you need.
Another choice is to use <macrodef> which allows you to define your own macro (which is better than using <antcall> since <antcall> can break Ant's ability to create a execution matrix).

Programmatically executing ANT script in Java

I've built a swing gui in eclipse that is supposed to run a bunch of code previously developed, part of which involves running ant to build the code. When I run the any script outside of the GUI (in the original project) the ant executes correctly and builds the project. However when I try and run ant programmatically it throws errors that look like the project doesn't have the necessary .jars. The code, top of the build.xml, and errors are listed below.
Code Block
File buildFile = new File("lib\\ePlay\\build.xml");
Project p = new Project();
p.setUserProperty("ant.file", buildFile.getAbsolutePath());
DefaultLogger consoleLogger = new DefaultLogger();
consoleLogger.setErrorPrintStream(System.err);
consoleLogger.setOutputPrintStream(System.out);
consoleLogger.setMessageOutputLevel(Project.MSG_INFO);
p.addBuildListener(consoleLogger);
try {
p.fireBuildStarted();
p.init();
ProjectHelper helper = ProjectHelper.getProjectHelper();
p.addReference("ant.projectHelper", helper);
helper.parse(p, buildFile);
p.executeTarget(p.getDefaultTarget());
p.fireBuildFinished(null);
} catch (BuildException e) {
p.fireBuildFinished(e);
}
Build.xml
<?xml version="1.0" encoding="UTF-8"?>
<project name="EPlay" xmlns:ivy="antlib:org.apache.ivy.ant" default="resolve">
<dirname file="${ant.file}" property="ant.dir" />
<property name="scm.user" value="scmrel"/>
<property name="scm.user.key" value="/.ssh/scmrel/id_rsa"/>
<property name="ivy.jar" value="ivy-2.0.0.jar" />
<property name="ivy.settings.dir" value="${ant.dir}/ivysettings" />
<property name="VERSION" value="LATEST" />
<property name="tasks.dir" value="${ant.dir}/.tasks" />
<property name="deploy.dir" value="${ant.dir}/deploy" />
...
<!-- retrieve the dependencies using Ivy -->
<target name="resolve" depends="_loadantcontrib,_getivy" description=" retrieve the dependencies with Ivy">
<ivy:settings file="${ivy.settings.dir}/ivysettings.xml" />
<ivy:resolve file="${ant.dir}/ivy.xml" transitive="false" />
<ivy:retrieve pattern="${deploy.dir}/[conf]/[artifact].[ext]"/>
</target>
And the error
resolve:
BUILD FAILED
H:\eclipse\CLDeploySwing\lib\ePlay\build.xml:66: Problem: failed to create task or type antlib:org.apache.ivy.ant:settings
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any <presetdef>/<macrodef> declarations have taken place.
No types or tasks have been defined in this namespace yet
This appears to be an antlib declaration.
Action: Check that the implementing library exists in one of:
-ANT_HOME\lib
-the IDE Ant configuration dialogs
Total time: 0 seconds
I've looked through my ant installation and it appears everything is there. Like I said, the original project builds successfully if build.xml is run outside of this application.

I would suggest that this is causde because your java program does not have the same classpath, where it is running, as does the normal ant build - and thus the ANT_HOME isn't the right one.
You can make sure that this is correct by passing the right enviornmental variables into the JVM, or simply a call to System.getProperty("ANT_HOME"), to see where your ANT_HOME actually is residing.

I think ${basedir} for your project is not properly calculated.
Add this line to your build.xml
<echo message="basedir='${basedir}'/>
Looking at this line
File buildFile = new File("lib\\ePlay\\build.xml");
It may be that it's 2 levels up ( I know the documentation says that it should be a parent directory of build.xml, but you are not running from the command line ).

rather than load the ivy library using the new method of using the namespace in the project declaration go old school on it.
<path id="ivy.lib.path">
<fileset dir="path/to/dir/with/ivy/jar" includes="*.jar"/>
</path>
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>
I am doing something in Gradle that requires this
ant.taskdef(name: 'ivy-retrieve', classname: 'org.apache.ivy.ant.IvyRetrieve', classpath: '...path to ivy jar.../ivy-2.2.0.jar')
which in ant would be something like
<taskdef name="ivy-retrieve" classname="org.apache.ivy.ant.IvyRetrieve"/>
I know it's more klunky and simply not as nice as including the namespace declaration but it does remove some of the confusion regarding which libraries on which classpath are being loaded.

How to validate properties/custom configuration files?

My application has many properties/text configuration files. Is there a common way to validate these kind of files? May be some tools or something like xsd?
The files are large (more 1000 rows), so I frequently make mistake. Information in file in majority of cases - is path to different directories. So it will be good if they exist and are consistent. E.g.
if I have
mydata.root="c:\data"
and after I have:
myreports=${mydata.root}/reports
that will be good check that c:\data and c:\data\report exist
and not written (some hundreds rows down) e.g.
myreports=${mdata.root}/reports

You could do this validation in your build file.
For example, the following build file defines a macrodef validate-file-property which validates that a specified property is defined and that it exists as a file or dir in the file system.
<project default="init">
<property file="test.properties"/>
<target name="init">
<validate-file-property property="program.files" type="dir"/>
<validate-file-property property="mydata.root" type="dir"/>
<validate-file-property property="foo"/>
</target>
<macrodef name="validate-file-property">
<attribute name="property"/>
<attribute name="type" default="file"/>
<sequential>
<fail unless="#{property}" message="The property '#{property}' is undefinded."/>
<available file="${#{property}}" property="#{property}.exists" type="#{type}"/>
<fail unless="#{property}.exists" message="The #{type} '#{property}' with value '${#{property}}' does not exist."/>
</sequential>
</macrodef>
</project>
You need to decide when to validate the properties, and you need to validate them explicitly - as shown in the init target in this example.
BTW, if you used a standardized pattern to name properties which refer to files or directories - e.g. my.special.file, my.build.dir - then you could use a shell script to discover all relevant properties and write all your validate-file-property elements . Something like this:
awk -F= '/\.(file|dir)/{ printf "<validate-file-property property=\"%s\" type=\"%s\"/>\n", $1, gensub(/.*\.(file|dir)/, "\\1", "g", $1) }' *.properties
You could paste the output into your build file.

I once offered a bounty for information on configuration languages other than XML that provide schema validation. Unfortunately, such configuration languages are few and far between.
Here is a link to the bountied question, in case you want to see the sparseness of the responses.

Filtering sources before compilation in NetBeans & Ant

I need to filter java files before compilation, leaving the original sources unchanged and compiling from filtered ones (basically, I need to set build date and such).
I'm using NetBeans with its great Ant build-files.
So, one day I discovered the need to pre-process my source files before compilation, and ran into a big problem. No, I did not run to SO at once, I did some research, but failed. So, here comes my sad story...
I found the "filter" option of "copy" task, overrided macrodef "j2seproject3:javac" in build-impl.xml file and added filter in the middle of it. I got the desired result, yes, but now my tests are not working, since they use that macrodef too.
Next, I tired to overriding "-do-compile" target, copying&filtering files to directory build/temp-src, and passing an argument of new source directory to "j2seproject3:javac":
<target depends="init,deps-jar,-pre-pre-compile,-pre-compile, -copy-persistence-xml,
-compile-depend,-prepare-sources"
if="have.sources" name="-do-compile">
<j2seproject3:javac gensrcdir="${build.generated.sources.dir}" srcdir="build/temp-src"/>
<copy todir="${build.classes.dir}">
<fileset dir="${src.dir}" excludes="${build.classes.excludes},${excludes}" includes="${includes}"/>
</copy>
</target>
And now Ant says to me, that macrodef in question does not exist!
The prefix "j2seproject3" for element "j2seproject3:javac" is not bound.
That's strange, since build-impl.xml contains that macrodef, and build-impl.xml is imported into main build file.
And, by the way, I cannot edit build-impl.xml directly, since NetBeans rewrites it on every other build.
So, my question is: how can I automatically filter sources before compiling in NetBeans, and do not break the build process?

Looking at the default build.xml, it contains a comment that reads (in part):
There exist several targets which are by default empty and which can be
used for execution of your tasks. These targets are usually executed
before and after some main targets. They are:
-pre-init: called before initialization of project properties
-post-init: called after initialization of project properties
-pre-compile: called before javac compilation
-post-compile: called after javac compilation
-pre-compile-single: called before javac compilation of single file
-post-compile-single: called after javac compilation of single file
-pre-compile-test: called before javac compilation of JUnit tests
-post-compile-test: called after javac compilation of JUnit tests
-pre-compile-test-single: called before javac compilation of single JUnit test
-post-compile-test-single: called after javac compilation of single JUunit test
-pre-jar: called before JAR building
-post-jar: called after JAR building
-post-clean: called after cleaning build products
So, to inject some pre-compile processing, you would provide a definition for -pre-compile.
FWIW, the error you got is because the j2seprojectX prefix is defined on the project tag of build-impl.xml, and the code in build.xml is outside of that tag.

Since I found an answer, and because it seems that nobody knows the answer, I'll post my solution.
build.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Needed to add xmlns:j2seproject3 attribute, to be able to reference build-impl.xml macrodefs -->
<project name="Parrot" default="default" basedir="." xmlns:j2seproject3="http://www.netbeans.org/ns/j2se-project/3">
<import file="nbproject/build-impl.xml"/>
<target depends="init,deps-jar,-pre-pre-compile,-pre-compile, -copy-persistence-xml, -compile-depend,-prepare-sources" if="have.sources" name="-do-compile">
<j2seproject3:javac gensrcdir="${build.generated.sources.dir}" srcdir="build/temp-src"/>
<copy todir="${build.classes.dir}">
<fileset dir="${src.dir}" excludes="${build.classes.excludes},${excludes}" includes="${includes}"/>
</copy>
</target>
<!-- target to alter sources before compilation, you can add any preprocessing actions here -->
<target name="-filter-sources" description="Filters sources to temp-src, setting the build date">
<delete dir="build/temp-src" />
<mkdir dir="build/temp-src" />
<tstamp>
<format property="build.time" pattern="yyyy-MM-dd HH:mm:ss"/>
</tstamp>
<filter token="build-time" value="${build.time}" />
<copy todir="build/temp-src" filtering="true">
<fileset dir="src">
<filename name="**/*.java" />
</fileset>
</copy>
</target>
</project>

Is there a way to generalize an Apache ANT target?

We have an Apache ANT script to build our application, then check in the resulting JAR file into version control (VSS in this case). However, now we have a change that requires us to build 2 JAR files for this project, then check both into VSS.
The current target that checks the original JAR file into VSS discovers the name of the JAR file through some property. Is there an easy way to "generalize" this target so that I can reuse it to check in a JAR file with any name? In a normal language this would obviously call for a function parameter but, to my knowledge, there really isn't an equivalent concept in ANT.

I would suggest to work with macros over subant/antcall because the main advantage I found with macros is that you're in complete control over the properties that are passed to the macro (especially if you want to add new properties).
You simply refactor your Ant script starting with your target:
<target name="vss.check">
<vssadd localpath="D:\build\build.00012.zip"
comment="Added by automatic build"/>
</target>
creating a macro (notice the copy/paste and replacement with the #{file}):
<macrodef name="private-vssadd">
<attribute name="file"/>
<sequential>
<vssadd localpath="#{file}"
comment="Added by automatic build"/>
</sequential>
</macrodef>
and invoke the macros with your files:
<target name="vss.check">
<private-vssadd file="D:\build\File1.zip"/>
<private-vssadd file="D:\build\File2.zip"/>
</target>
Refactoring, "the Ant way"

It is generally considered a bad idea to version control your binaries and I do not recommend doing so. But if you absolutely have to, you can use antcall combined with param to pass parameters and call a target.
<antcall target="reusable">
<param name="some.variable" value="var1"/>
</antcall>
<target name="reusable">
<!-- Do something with ${some.variable} -->
</target>
You can find more information about the antcall task here.

Take a look at Ant macros. They allow you to define reusable "routines" for Ant builds. You can find an example here (item 15).

Also check out the subant task, which lets you call the same target on multiple build files:
<project name="subant" default="subant1">
<property name="build.dir" value="subant.build"/>
<target name="subant1">
<subant target="">
<property name="build.dir" value="subant1.build"/>
<property name="not.overloaded" value="not.overloaded"/>
<fileset dir="." includes="*/build.xml"/>
</subant>
</target>
</project>

You can use Gant to script your build with groovy to do what you want or have a look at the groovy ant task.