I'm trying to set up my ant build so that the targets which run Ivy functions are completely separated from the ones that the continuous build and most developers run. I want one target to download updated dependencies, which I'll check in. Other targets will set up their classpath by including *.jar from the relevant directory.
I have two configurations:
<configurations>
<conf name="compile" />
<conf name="test" />
</configurations>
I have some dependencies:
<dependency
org="my.org"
name="some-lib"
rev="latest.release"
conf="compile->default" />
<dependency
org="my.org"
name="some-test-lib"
rev="latest.release"
conf="test->default" />
And I download those dependencies and their transitive dependencies using ivy:retrieve:
<ivy:retrieve
pattern="lib/[conf]/[type]/[artifact]-[revision].[ext]"
sync="true"
file="ivy.xml" />
The problem is that I'm seeing some duplicates jars between the compile and the test directories, which come from transitive dependencies. Since I want to check in all these jars and use them for creating classpaths, I'd like to avoid duplicates. Is this possible?
lib/compile/jar/some-lib-1.0.jar
lib/compile/jar/slf4j-api-1.5.11.jar
lib/test/jar/some-test-lib-1.0.jar
lib/test/jar/junit-4.7.jar
lib/test/jar/slf4j-api-1.5.11.jar
This is not duplication, each configuration is a separate set of jars and the ivy restrieve task is faithly creating each set....
Perhaps it would make more sense to create the classpaths directly, rather than populating a local lib directory.
Here's a snippet of my ANT build files:
<target name="get-dependencies">
<ivy:resolve/>
<ivy:cachepath pathid="compile.path" conf="compile" />
<ivy:cachepath pathid="test.path" conf="test" />
</target>
<target name="compile" depends="get-dependencies">
<javac srcdir="src" destdir="build/classes" classpathref="compile.path"/>
</target>
I normally only use the ivy retrieve task when I need to create a local copy of a set of jars, for example assembling a web app's directory:
<ivy:retrieve pattern="build/WEB_INF/lib/[artifact].[ext]" conf="runtime"/>
Update
Another alternative is to instruct ivy to exclude the slf4j module when downloading transient dependencies, as follows:
<dependency org="my.org" name="some-lib" rev="latest.release" conf="compile->default">
<exclude module="slf4j-api"/>
</dependency>
<dependency org="my.org" name="some-test-lib" rev="latest.release" conf="test->default">
<exclude module="slf4j-api"/>
</dependency>
If I can do this through Ivy directly, that would be best. For now I've solved the problem by just deleting duplicates using ant.
Try the following. Your test should extend compile
<dependency
org="my.org"
name="some-test-lib"
rev="latest.release"
conf="**test->compile**" />
If i am right IVY should find that test extends compile and would download slf4j only once.
Related
Problem: I need to have a lib on the eclipse classpath that should not be deployed to Tomcat. (In a maven project it would be scope provided)
Explanation:
I've setup a project with some Ivy dependencies and had to externalize a configuration as JNI (mail/session) in order to do it I had to put the mail-1.4.7.jar inside the Tomcat lib folder.
The problem is that I have a dependency that add to my classpath the javax.mail-1.5.2.jar so I change it to:
<dependency org="org.apache.logging.log4j" name="log4j-core" rev="2.2">
<exclude org="com.sun.mail" name="javax.mail"/>
</dependency>
The problem now is that my project break (compilation errors) because of missing mail classes such as javax.mail.MessagingException
So I have to add the mail dependency but only to eclipse. I've tried some configurations as explained here from what I know from Maven behavior with no avail.
Keeping the mail dependency only in the project, breaks Tomcat, keeping it on both tomcat and project breaks project. When I manually remove it from my project lib folder (WEB-INF\lib), after deploy the project, it works properly.
Bottom line (after deploy):
tomcatFolder
|_lib
| |_...
| |_mail-1.4.7.jar
| |_...
|_webapps
|_myproject
|_WEB-INF
|_lib
|_...
|_javax.mail-1.5.2.jar //need to remove it at deploy time only
|_...
Can't change it to maven right now. But it is in process :)
This is really a duplicate of this question:
How to copy runtime libraries without the provided ones in IVY
But.. from your question I suspect you're not using ivy configuration mappings. This is unfortunate because this is the mechanism used by ivy to logically group dependencies into functional groupings, similar to how Maven maintains scopes. The following posting attempts to bridge this understanding
How are maven scopes mapped to ivy configurations by ivy
Furthermore you are also using Eclipse, which means that unless you're using the ivy plugin you effectively have two build mechanisms. (ivy and eclipse). I would recommend fixing your ANT build first and then look at how to maintain the Eclipse classpath second.
Example
The first section describes how configurations are declared and used in the ivy file and the second section explains how the ivy ANT tasks are used in the build logic.
ivy.xml
You should always declare ivy configurations and use these to control your classpaths. In my builds I always have at least three: compile, runtime and test. Notice how the extends attribute is used to create relationships between the configs, because runtime should also include the compile dependencies.
Adding an additional one for the provided scope jars is easy. Simple stand-alone configuration:
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
<conf name="provided" description="Needed for compile, but will be present on the target platform."/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="4.11" conf="test->default"/>
<!-- provided dependencies -->
<dependency org="org.apache.tomcat" name="servlet-api" rev="6.0.16" conf="provided->master"/>
</dependencies>
</ivy-module>
It's the configuration mappings that make things special. The simple explanation is that they fall into two basic types when pulling from a Maven repository:
conf="local_configuration->default"
conf="local_configuration->master"
The first means include the remote module and all its dependencies. The second means include the remote module and exclude it's dependencies. This means you don't need the following exclude trickery:
<dependency org="org.apache.logging.log4j" name="log4j-core" rev="2.2">
<exclude org="com.sun.mail" name="javax.mail"/>
</dependency>
You simply use the following, if all you want is the log4j-core jar:
<dependency org="org.apache.logging.log4j" name="log4j-core" rev="2.2" conf="provided->master"/>
Additional notes:
In ivy mapping to the remote "default" configuration will pull down only the jars you need. It will exclude optional dependencies and other stuff like javadocs.
Sometimes "excludes" are necessary when module authors get their dependencies wrong.
build.xml
The resolve target will pull down dependencies, generate a report and create the compile and test classpaths. Note the use of configurations to determine which jar groupings should be used:
<target name="resolve" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='${build.dir}/ivy-reports' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile,provided"/>
<ivy:cachepath pathid="test.path" conf="test,provided"/>
</target>
These classpath references are then used by the compile target as normal:
<target name="compile" depends="resolve,resources" description="Compile code">
<mkdir dir="${build.dir}/classes"/>
<javac srcdir="${src.dir}" destdir="${build.dir}/classes" includeantruntime="false" debug="true" classpathref="compile.path"/>
</target>
<target name="compile-tests" depends="compile" description="Compile tests">
<mkdir dir="${build.dir}/test-classes"/>
<javac srcdir="${test.src.dir}" destdir="${build.dir}/test-classes" includeantruntime="false" debug="true">
<classpath>
<path refid="test.path"/>
<pathelement path="${build.dir}/classes"/>
</classpath>
</javac>
</target>
And the test target:
<target name="test" depends="compile-tests" description="Run unit tests">
<mkdir dir="${build.dir}/test-reports"/>
<junit printsummary="yes" haltonfailure="yes">
<classpath>
<path refid="test.path"/>
<pathelement path="${build.dir}/classes"/>
<pathelement path="${build.dir}/test-classes"/>
</classpath>
<formatter type="xml"/>
<batchtest fork="yes" todir="${build.dir}/test-reports">
<fileset dir="${test.src.dir}">
<include name="**/*Test*.java"/>
<exclude name="**/AllTests.java"/>
</fileset>
</batchtest>
</junit>
</target>
Lastly the ivy retrieve task is used to build the war file. Only the "runtime" configuration jars are used:
<target name="package" depends="test" description="Create the WAR file">
<ivy:retrieve pattern="${build.dir}/lib/[artifact].[ext]" conf="runtime"/>
<war destfile="${war.file}" webxml="${resources.dir}/web.xml">
<fileset dir="${resources.dir}" excludes="web.xml"/>
<lib dir="${build.dir}/lib"/>
</war>
</target>
In conclusion the cachepath ivy task is used to create classpath references based on the ivy configurations and the retrieve task is used when assembling the war file.
I have a project where we store a local copy of some client API jars. These jars may or may not be the same as the final run-time versions used by the client. When I build the project locally for testing, I want to use the local copies of the jars. However, when I'm on-site with the client, I want to look at an external location from my project and check if the client updated jars are available. If they are we use them, if not we fall back on our local copies.
I have found a way to conditionally build a file-set containing the given jars, but I can't figure out how to get the file-set into my classpath.
Below is the script I have generated so far. I do not have access to the ant-contrib IF statements, so this is the only way I've found to conditionally build the file-set.
<condition property="externalFileExists">
<available file="[path to external file]"/>
</condition>
<target name="buildExternalFileset" if="externalFileExists">
<fileset id="fileset" includes="[path to external library]"/>
</target>
<target name="buildInternalFileset" unless="externalFileExists">
<fileset id="fileset" includes="[path to internal library]"/>
</target>
<target name="buildFileset depends="buildExternalFileset, buildInternalFileSet>
</target>
Once I have this file-set built, how do I include it in the classpath?
<path id="classpath">
<!-- other filesets -->
<!-- line that includes the conditional fileset by ID -->
</path>
According to the documentation for path-like structures you can include a path within another path, but I don't see an example for including an external file-set. Perhaps I've answered my question - - should I be building paths instead of file-sets? Additionally, file-set documentation doesn't actually include the option to add IDs, do they work?
If anyone has a better way to do this I'd be more than happy to hear about it!
I would suggest using a dependency manager rather than try and building your own.
Apache ivy is able to download dependencies from configured locations. It also caches downloaded files removing the need to store them locally if they are universally available from Maven Central.
Example
This project has an ivy settings file that pulls dependencies from either the local "lib" directory or from Maven central.
├── build.xml
├── ivysettings.xml
├── ivy.xml
└── lib
├── hamcrest-core-1.3.jar
├── junit-4.11.jar
├── log4j-1.2.17.jar
├── slf4j-api-1.7.5.jar
└── slf4j-log4j12-1.7.5.jar
build.xml
<project name="demo" default="resolve" xmlns:ivy="antlib:org.apache.ivy.ant">
<property name="build.dir" location="target"/>
<available classname="org.apache.ivy.Main" property="ivy.installed"/>
<target name="install-ivy" unless="ivy.installed">
<mkdir dir="${user.home}/.ant/lib"/>
<get dest="${user.home}/.ant/lib/ivy.jar" src="http://search.maven.org/remotecontent?filepath=org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar"/>
<fail message="Ivy installed run build again"/>
</target>
<target name="resolve" depends="install-ivy" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='${build.dir}/ivy-reports' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
<target name="clean">
<delete dir="${build.dir}"/>
</target>
<target name="clean-all" depends="clean">
<ivy:cleancache/>
</target>
</project>
Notes:
Ivy does not come bundled with ANT. The "install-ivy" task ensures it gets installed on the first ant run
The "cachepath" tasks will create classpaths for use in your javac, junit and java tasks.
The "report" and "cleancache" tasks are useful additional tasks.
ivy.xml
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="4.11" conf="test->default"/>
</dependencies>
</ivy-module>
Notes:
Uses configurations to keep dependencies separated. Note how this is used by the "cachepath" tasks to control dependency classpaths
ivysettings.xml
<ivysettings>
<settings defaultResolver="my-resolvers"/>
<resolvers>
<chain name="my-resolvers" returnFirst="true">
<filesystem name="local-lib">
<artifact pattern="${ivy.settings.dir}/lib/[artifact]-[revision].[ext]"/>
</filesystem>
<ibiblio name="central" m2compatible="true"/>
</chain>
</resolvers>
</ivysettings>
Notes:
This file is optional. By default Ivy will retrieve from Maven Central
The special chain resolver controls which source of dependencies is used. In this case the local "lib" directory is favoured over Maven Cental.
I'm not very familiar with ant, but how about a build configuration on ant ?
Try this site for a tutorial on how to do it: http://ant.apache.org/easyant/history/trunk/howto/BuildConfigurations.html
Try something like this
<classpath>
<fileset dir="[external-path1]">
<include name="**/*.jar"/>
</fileset>
<fileset dir="[external-path2]">
<include name="**/*.jar"/>
</fileset>
<fileset dir="[external-path3]">
<include name="**/*.jar"/>
</fileset>
</classpath>
found here
https://ant.apache.org/manual/using.html#path
I have two jars that conflict in my libraries in the java build path. Both of them are resolved by ivy, so I see them inside an ivy.xml entry on the libraries tab. I can't delete anything inside the ivy.xml entry, eclipse only lets me delete the entire entry. Deleting the jar from the source folder is'nt viable either, since I don't have the access rights for that.
Could you give me some advice on how to solve this conflict?
Thanks for your time.
Use exclude in your ivy file, eg.:
<dependency org="log4j" name="log4j" rev="1.2.17" conf="default" >
<exclude module="javaee-api"/>
<exclude module="geronimo-jms_1.1_spec"/>
</dependency>
You can use following ant task to create dependency report:
<target name="report" >
<delete dir="report"/>
<mkdir dir="report" />
<ivy:resolve type="${ivy.resolve.types}"/>
<ivy:report todir="report" />
</target>
I am looking for the equivalent of a system scope dependency in Maven for Ivy. In Maven, declaring a dependency with system scope means that Maven will not include the dependency in the output, which is what I want. How can I achieve the same thing with Ivy?
I suspect you're talking about the Maven provided scope, not system.
provided This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime. For example, when building a web application for the Java Enterprise Edition, you would set the dependency on the Servlet API and related Java EE APIs to scope provided because the web container provides those classes. This scope is only available on the compilation and test classpath, and is not transitive.
Ivy's configurations are very flexible which means that are several ways to do this.
ivy.xml
I would model my configuration on the different types of jar that my build will use:
<configurations>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
<conf name="provided" description="Additional compile time dependencies, implementation provided by target platform"/>
</configurations>
The dependencies are then assigned to each logical grouping using a "conf" mapping:
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="compile->default"/>
Note this is where the "extends" attribute it useful, without it the following mapping would be required for a logging dependency:
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="compile,runtime,test->default"/>
Dependencies provided by the target platform are special. This is why I create a standalone configuration for them:
<!-- compile dependencies -->
<dependency org="my.target.platform" name="makeitgo-api" rev="1.0" conf="provided->default"/>
For even more details on ivy configuration mappings see:
Ivy, what is the master configuration and why is it not pulling jvyaml?
build.xml
It's here where the classpaths are actually managed. (We could try and model the set relationships in various ivy configurations, but I'd argue this approach is simpler and gets the job done)
<target name="resolve" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='${ivy.reports.dir}' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile,provided"/>
<ivy:cachepath pathid="test.path" conf="test,provided"/>
</target>
So just as the Maven documentation describes, add the provided dependencies to the compile and test path.
This means the "runtime" configuration only contains the dependencies that should be bundled:
<ivy:retrieve pattern="${build.dir}/WEB-INF/lib/[artifact]-[revision].[ext]" conf="runtime"/>
I guess that the only problem to deal with is to have a dependency which should be used at compile time but not at runtime, right ?
Here is an exemple of ivy.xml on how to deal with it:
<ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven">
<info organisation="com.acme" module="mymodule" />
<configurations>
<conf name="default" description="runtime" />
<conf name="runtime" description="Runtime configuration"/>
<conf name="compile" description="Used only for compilation" />
<conf name="test" extends="compile,runtime" description="Unit testing configuration" visibility="private" />
</configurations>
<dependencies defaultconf="runtime,compile->default">
<!-- Compile and runtime -->
<dependency org="commons-lang" name="commons-lang" rev="2.6" />
<!-- Only compile -->
<dependency org="javax.servlet" name="servlet-api" rev="2.5" conf="compile->default" />
</dependencies>
</ivy-module>
We recently started using ant-ivy for a java webapp and the journey has been good so far. I have specified transitive='false' for all dependencies and listed log4j version 1.2.16 as a dependency. I have to use a recent version of log4j as I rely on an enhanced pattern layout.
Main problem - when the project is built, there are multiple versions of log4j in the ivy-lib folder. When looking at the package explorer in Eclipse, I see the right version and seems like it downloads only 1.2.16 from ivycachepath.
Any pointers? Do I need to remove each dependency individually and see which one brings older versions?
Thanks for the help.
I've included an example ant build which downloads log4j as a dependency, into a local directory called "lib".
$ ant
$ find lib
lib
lib/log4j-1.2.16.jar
As you can see only one version of the jar is present. Hope this helps in diagnosing your issue.
build.xml
<project xmlns:ivy="antlib:org.apache.ivy.ant" name="hello-ivy" default="retrieve">
<property name="lib.dir" location="lib"/>
<property name="report.dir" location="report"/>
<target name="retrieve" description="Retrieve dependencies locally">
<ivy:retrieve pattern="${lib.dir}/[artifact]-[revision](-[classifier]).[ext]"/>
<ivy:report todir="${report.dir}" graph="false"/>
</target>
<target name="clean" description="Remove build directories">
<delete dir="${lib.dir}"/>
<delete dir="${report.dir}"/>
</target>
<target name="clean-all" depends="clean" description="clean ivy cache">
<ivy:cleancache />
</target>
</project>
ivy.xml
<ivy-module version="2.0">
<info organisation="org.demo" module="demo"/>
<configurations defaultconfmapping="compile->default">
<conf name="compile" description="jars used for compilation"/>
</configurations>
<dependencies>
<dependency org="log4j" name="log4j" rev="1.2.16"/>
</dependencies>
</ivy-module>