We Keep Coding

Ivy dependency as provided

Problem: I need to have a lib on the eclipse classpath that should not be deployed to Tomcat. (In a maven project it would be scope provided)
Explanation:
I've setup a project with some Ivy dependencies and had to externalize a configuration as JNI (mail/session) in order to do it I had to put the mail-1.4.7.jar inside the Tomcat lib folder.
The problem is that I have a dependency that add to my classpath the javax.mail-1.5.2.jar so I change it to:
<dependency org="org.apache.logging.log4j" name="log4j-core" rev="2.2">
<exclude org="com.sun.mail" name="javax.mail"/>
</dependency>
The problem now is that my project break (compilation errors) because of missing mail classes such as javax.mail.MessagingException
So I have to add the mail dependency but only to eclipse. I've tried some configurations as explained here from what I know from Maven behavior with no avail.
Keeping the mail dependency only in the project, breaks Tomcat, keeping it on both tomcat and project breaks project. When I manually remove it from my project lib folder (WEB-INF\lib), after deploy the project, it works properly.
Bottom line (after deploy):
tomcatFolder
|_lib
| |_...
| |_mail-1.4.7.jar
| |_...
|_webapps
|_myproject
|_WEB-INF
|_lib
|_...
|_javax.mail-1.5.2.jar //need to remove it at deploy time only
|_...
Can't change it to maven right now. But it is in process :)

This is really a duplicate of this question:
How to copy runtime libraries without the provided ones in IVY
But.. from your question I suspect you're not using ivy configuration mappings. This is unfortunate because this is the mechanism used by ivy to logically group dependencies into functional groupings, similar to how Maven maintains scopes. The following posting attempts to bridge this understanding
How are maven scopes mapped to ivy configurations by ivy
Furthermore you are also using Eclipse, which means that unless you're using the ivy plugin you effectively have two build mechanisms. (ivy and eclipse). I would recommend fixing your ANT build first and then look at how to maintain the Eclipse classpath second.
Example
The first section describes how configurations are declared and used in the ivy file and the second section explains how the ivy ANT tasks are used in the build logic.
ivy.xml
You should always declare ivy configurations and use these to control your classpaths. In my builds I always have at least three: compile, runtime and test. Notice how the extends attribute is used to create relationships between the configs, because runtime should also include the compile dependencies.
Adding an additional one for the provided scope jars is easy. Simple stand-alone configuration:
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
<conf name="provided" description="Needed for compile, but will be present on the target platform."/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="4.11" conf="test->default"/>
<!-- provided dependencies -->
<dependency org="org.apache.tomcat" name="servlet-api" rev="6.0.16" conf="provided->master"/>
</dependencies>
</ivy-module>
It's the configuration mappings that make things special. The simple explanation is that they fall into two basic types when pulling from a Maven repository:
conf="local_configuration->default"
conf="local_configuration->master"
The first means include the remote module and all its dependencies. The second means include the remote module and exclude it's dependencies. This means you don't need the following exclude trickery:
<dependency org="org.apache.logging.log4j" name="log4j-core" rev="2.2">
<exclude org="com.sun.mail" name="javax.mail"/>
</dependency>
You simply use the following, if all you want is the log4j-core jar:
<dependency org="org.apache.logging.log4j" name="log4j-core" rev="2.2" conf="provided->master"/>
Additional notes:
In ivy mapping to the remote "default" configuration will pull down only the jars you need. It will exclude optional dependencies and other stuff like javadocs.
Sometimes "excludes" are necessary when module authors get their dependencies wrong.
build.xml
The resolve target will pull down dependencies, generate a report and create the compile and test classpaths. Note the use of configurations to determine which jar groupings should be used:
<target name="resolve" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='${build.dir}/ivy-reports' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile,provided"/>
<ivy:cachepath pathid="test.path" conf="test,provided"/>
</target>
These classpath references are then used by the compile target as normal:
<target name="compile" depends="resolve,resources" description="Compile code">
<mkdir dir="${build.dir}/classes"/>
<javac srcdir="${src.dir}" destdir="${build.dir}/classes" includeantruntime="false" debug="true" classpathref="compile.path"/>
</target>
<target name="compile-tests" depends="compile" description="Compile tests">
<mkdir dir="${build.dir}/test-classes"/>
<javac srcdir="${test.src.dir}" destdir="${build.dir}/test-classes" includeantruntime="false" debug="true">
<classpath>
<path refid="test.path"/>
<pathelement path="${build.dir}/classes"/>
</classpath>
</javac>
</target>
And the test target:
<target name="test" depends="compile-tests" description="Run unit tests">
<mkdir dir="${build.dir}/test-reports"/>
<junit printsummary="yes" haltonfailure="yes">
<classpath>
<path refid="test.path"/>
<pathelement path="${build.dir}/classes"/>
<pathelement path="${build.dir}/test-classes"/>
</classpath>
<formatter type="xml"/>
<batchtest fork="yes" todir="${build.dir}/test-reports">
<fileset dir="${test.src.dir}">
<include name="**/*Test*.java"/>
<exclude name="**/AllTests.java"/>
</fileset>
</batchtest>
</junit>
</target>
Lastly the ivy retrieve task is used to build the war file. Only the "runtime" configuration jars are used:
<target name="package" depends="test" description="Create the WAR file">
<ivy:retrieve pattern="${build.dir}/lib/[artifact].[ext]" conf="runtime"/>
<war destfile="${war.file}" webxml="${resources.dir}/web.xml">
<fileset dir="${resources.dir}" excludes="web.xml"/>
<lib dir="${build.dir}/lib"/>
</war>
</target>
In conclusion the cachepath ivy task is used to create classpath references based on the ivy configurations and the retrieve task is used when assembling the war file.

Combining Ant and Maven projects

I'm working in very big java project, consisting of many smaller maven modules and one web module which is maintained using ant. Now, I need to access in my web module projects written in maven modules. Is it possible to somehow add maven modules to ant project?
I have tried to make jar files from maven modules and simple add them to ant project, but I received errors ClassNotFoundException and NoClassDefFoundError.

You can integrate maven dependencies with Apache Ant by using Maven Ant Tasks.

Use a Maven repository manager like Nexus, Artifactory or Archiva to store jars built by your Maven project.
The Apache ivy plugin for ANT can then be used to pull these as dependencies (Ivy can also be used to store jars in a Maven repo but that is another question).
Example
├── build.xml
├── ivysettings.xml
└── target
└── WEB-INF
└── lib
├── log4j-1.2.17.jar
├── slf4j-api-1.7.5.jar
└── slf4j-log4j12-1.7.5.jar
build.xml
<project name="demo" default="resolve" xmlns:ivy="antlib:org.apache.ivy.ant">
<property name="build.dir" location="target"/>
<available classname="org.apache.ivy.Main" property="ivy.installed"/>
<target name="resolve" depends="install-ivy" description="Use ivy to resolve classpaths">
<ivy:retrieve pattern="${build.dir}/WEB-INF/lib/[artifact]-[revision].[ext]">
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="default"/>
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" conf="default"/>
</ivy:retrieve>
</target>
<target name="install-ivy" unless="ivy.installed">
<mkdir dir="${user.home}/.ant/lib"/>
<get dest="${user.home}/.ant/lib/ivy.jar" src="http://search.maven.org/remotecontent?filepath=org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar"/>
<fail message="Ivy installed run build again"/>
</target>
<target name="clean">
<delete dir="${build.dir}"/>
</target>
<target name="clean-all" depends="clean">
<ivy:cleancache/>
</target>
</project>
Notes:
The "install-ivy" target is optional and my recommended way to ensure the ivy plugin jar is installed on your build server
The ivy retrieve task is used to resolve dependencies and place them within the project space
The ivy cleancache task purges the local ivy cache.
ivysettings.xml
<ivysettings>
<settings defaultResolver="my-resolvers"/>
<resolvers>
<chain name="my-resolvers" returnFirst="true">
<ibiblio name="central" m2compatible="true"/>
<ibiblio name="myrepo" root="http://mavenrepo/path/to/repo" m2compatible="true"/>
</chain>
</resolvers>
</ivysettings>
Note:
The ivy configuration file is optional and this example demonstrates how both Maven Central and a local Maven repository can be used as a source of dependencies.

approch for inter-project dependencies

I have a console-based build system that's using Ant and Ivy and would like to avoid using Maven. I fetch my project's external dependencies using Ivy. My question is how best to handle my internal project dependencies. I.e. I build a number of library files (JARs) on which my modules are based and which unlikely to be used in another context. So these internal project dependencies are simply captured by build.xml files directly identifying the location of the needed JARs via relative paths (since everything's under the same source tree in the repository). E.g. I have Ant "code" like this:
<path id = "compile.classpath">
<fileset dir="${internal-project-dependency-a.dir}">
<include name ="*.jar"/>
</fileset>
<fileset dir="${internal-project-dependency-b.dir}">
<include name ="*.jar"/>
</fileset>
<fileset dir="${internal-project-dependency-c.dir}">
<include name ="*.jar"/>
</fileset>
</path>
My question is: is the above approach acceptable (from a best-practice point of view) or should I instead package and publish even my internal project dependencies in some lightweight "local-only" Ivy repository (if there's such a thing)? The way I've setup my system anyone with Ivy and Ant can checkout my sources from github and build everything from the console without needing to configure any other kind of Ivy repository information and I'd like to keep that property.

Ivy supports different kinds or repositories, described in the documentation.
local
shared
public
Without customization these are located under your "~/.ivy2" directory. So, I would recommend configuring each builds to publish to your local repository and simply reference dependencies as normal. All you need to do is ensure that on a fresh machine the build order is correct, so that the local repo is populated in the correct order (See the buildlist task).
Using the local repo is not very different from referencing the jars in known relative locations. It does have the benefit of making each project decoupled from the other.
Example
├── build.xml
└── ivy.xml
Project publishes 3 files to the local repository:
$ find ~/.ivy2/local -type f
/home/mark/.ivy2/local/myorg/hello/1.0/ivys/ivy.xml.sha1
/home/mark/.ivy2/local/myorg/hello/1.0/ivys/ivy.xml
/home/mark/.ivy2/local/myorg/hello/1.0/ivys/ivy.xml.md5
/home/mark/.ivy2/local/myorg/hello/1.0/docs/English.txt.md5
/home/mark/.ivy2/local/myorg/hello/1.0/docs/Spanish.txt.sha1
/home/mark/.ivy2/local/myorg/hello/1.0/docs/English.txt.sha1
/home/mark/.ivy2/local/myorg/hello/1.0/docs/Irish.txt.sha1
/home/mark/.ivy2/local/myorg/hello/1.0/docs/Spanish.txt.md5
/home/mark/.ivy2/local/myorg/hello/1.0/docs/English.txt
/home/mark/.ivy2/local/myorg/hello/1.0/docs/Spanish.txt
/home/mark/.ivy2/local/myorg/hello/1.0/docs/Irish.txt.md5
/home/mark/.ivy2/local/myorg/hello/1.0/docs/Irish.txt
build.xml
<project name="demo" default="publish" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="init">
<ivy:resolve/>
</target>
<target name="build" depends="init">
<mkdir dir="build"/>
<echo file="build/English.txt">Hello world</echo>
<echo file="build/Irish.txt">Dia dhuit</echo>
<echo file="build/Spanish.txt">Hola mundo</echo>
</target>
<target name="publish" depends="clean,build">
<ivy:publish pubrevision="1.0" status="release" resolver="local" >
<artifacts pattern="build/[artifact].[ext]"/>
</ivy:publish>
</target>
<target name="clean" description="Cleanup build files">
<delete dir="build"/>
</target>
</project>
ivy.xml
<ivy-module version="2.0">
<info organisation="myorg" module="hello"/>
<publications>
<artifact name="English" ext="txt" type="doc"/>
<artifact name="Irish" ext="txt" type="doc"/>
<artifact name="Spanish" ext="txt" type="doc"/>
</publications>
</ivy-module>

ivy build - multiple artifacts of different versions

We recently started using ant-ivy for a java webapp and the journey has been good so far. I have specified transitive='false' for all dependencies and listed log4j version 1.2.16 as a dependency. I have to use a recent version of log4j as I rely on an enhanced pattern layout.
Main problem - when the project is built, there are multiple versions of log4j in the ivy-lib folder. When looking at the package explorer in Eclipse, I see the right version and seems like it downloads only 1.2.16 from ivycachepath.
Any pointers? Do I need to remove each dependency individually and see which one brings older versions?
Thanks for the help.

I've included an example ant build which downloads log4j as a dependency, into a local directory called "lib".
$ ant
$ find lib
lib
lib/log4j-1.2.16.jar
As you can see only one version of the jar is present. Hope this helps in diagnosing your issue.
build.xml
<project xmlns:ivy="antlib:org.apache.ivy.ant" name="hello-ivy" default="retrieve">
<property name="lib.dir" location="lib"/>
<property name="report.dir" location="report"/>
<target name="retrieve" description="Retrieve dependencies locally">
<ivy:retrieve pattern="${lib.dir}/[artifact]-[revision](-[classifier]).[ext]"/>
<ivy:report todir="${report.dir}" graph="false"/>
</target>
<target name="clean" description="Remove build directories">
<delete dir="${lib.dir}"/>
<delete dir="${report.dir}"/>
</target>
<target name="clean-all" depends="clean" description="clean ivy cache">
<ivy:cleancache />
</target>
</project>
ivy.xml
<ivy-module version="2.0">
<info organisation="org.demo" module="demo"/>
<configurations defaultconfmapping="compile->default">
<conf name="compile" description="jars used for compilation"/>
</configurations>
<dependencies>
<dependency org="log4j" name="log4j" rev="1.2.16"/>
</dependencies>
</ivy-module>

Get ivy:retrieve to not copy a jar into two configurations?

I'm trying to set up my ant build so that the targets which run Ivy functions are completely separated from the ones that the continuous build and most developers run. I want one target to download updated dependencies, which I'll check in. Other targets will set up their classpath by including *.jar from the relevant directory.
I have two configurations:
<configurations>
<conf name="compile" />
<conf name="test" />
</configurations>
I have some dependencies:
<dependency
org="my.org"
name="some-lib"
rev="latest.release"
conf="compile->default" />
<dependency
org="my.org"
name="some-test-lib"
rev="latest.release"
conf="test->default" />
And I download those dependencies and their transitive dependencies using ivy:retrieve:
<ivy:retrieve
pattern="lib/[conf]/[type]/[artifact]-[revision].[ext]"
sync="true"
file="ivy.xml" />
The problem is that I'm seeing some duplicates jars between the compile and the test directories, which come from transitive dependencies. Since I want to check in all these jars and use them for creating classpaths, I'd like to avoid duplicates. Is this possible?
lib/compile/jar/some-lib-1.0.jar
lib/compile/jar/slf4j-api-1.5.11.jar
lib/test/jar/some-test-lib-1.0.jar
lib/test/jar/junit-4.7.jar
lib/test/jar/slf4j-api-1.5.11.jar

This is not duplication, each configuration is a separate set of jars and the ivy restrieve task is faithly creating each set....
Perhaps it would make more sense to create the classpaths directly, rather than populating a local lib directory.
Here's a snippet of my ANT build files:
<target name="get-dependencies">
<ivy:resolve/>
<ivy:cachepath pathid="compile.path" conf="compile" />
<ivy:cachepath pathid="test.path" conf="test" />
</target>
<target name="compile" depends="get-dependencies">
<javac srcdir="src" destdir="build/classes" classpathref="compile.path"/>
</target>
I normally only use the ivy retrieve task when I need to create a local copy of a set of jars, for example assembling a web app's directory:
<ivy:retrieve pattern="build/WEB_INF/lib/[artifact].[ext]" conf="runtime"/>
Update
Another alternative is to instruct ivy to exclude the slf4j module when downloading transient dependencies, as follows:
<dependency org="my.org" name="some-lib" rev="latest.release" conf="compile->default">
<exclude module="slf4j-api"/>
</dependency>
<dependency org="my.org" name="some-test-lib" rev="latest.release" conf="test->default">
<exclude module="slf4j-api"/>
</dependency>

If I can do this through Ivy directly, that would be best. For now I've solved the problem by just deleting duplicates using ant.

Try the following. Your test should extend compile
<dependency
org="my.org"
name="some-test-lib"
rev="latest.release"
conf="**test->compile**" />
If i am right IVY should find that test extends compile and would download slf4j only once.