I'm trying to define a task that emits (using echo) a message when a target completes execution, regardless of whether that target was successful or not. Specifically, the target executes a task to run some unit tests, and I want to emit a message indicating where the results are available:
<target name="mytarget">
<testng outputDir="${results}" ...>
...
</testng>
<echo>Tests complete. Results available in ${results}</echo>
</target>
Unfortunately, if the tests fail, the task fails and execution aborts. So the message is only output if the tests pass - the opposite of what I want. I know I can put the task before the task, but this will make it easier for users to miss this message. Is what I'm trying to do possible?
Update: It turns out I'm dumb. I had haltOnFailure="true" in my <testng> task, which explains the behaviour I was seeing. Now the issue is that setting this to false causes the overall ant build to succeed even if tests fail, which is not what I want. The answer below using the task looks like it might be what I want..
You can use a try-catch block like so:
<target name="myTarget">
<trycatch property="foo" reference="bar">
<try>
<testing outputdir="${results}" ...>
...
</testing>
</try>
<catch>
<echo>Test failed</echo>
</catch>
<finally>
<echo>Tests complete. Results available in ${results}</echo>
</finally>
</trycatch>
</target>
According to the Ant docs, there are two properties that control whether the build process is stopped or not if the testng task fails:
haltonfailure - Stop the build process
if a failure has occurred during the
test run. Defaults to false.
haltonskipped - Stop the build process
if there is at least on skipped test.
Default to false.
I can't tell from the snippet if you're setting this property or not. May be worth trying to explicitly set haltonfailure to false if it's currently set to true.
Also, assuming you're using the <exec> functionality in Ant, there are similar properties to control what happens if the executed command fails:
failonerror - Stop the buildprocess if the command exits with a return code
signaling failure. Defaults to false.
failifexecutionfails - Stop the build if we can't start the program.
Defaults to true.
Can't tell based on the partial code snippet in your post, but my guess is that the most likely culprit is failonerror or haltonfailure being set to true.
The solution to your problem is to use the failureProperty in conjunction with the haltOnFailure property of the testng task like this:
<target name="mytarget">
<testng outputDir="${results}" failureProperty="tests.failed" haltOnFailure="false" ...>
...
</testng>
<echo>Tests complete. Results available in ${results}</echo>
</target>
Then, elsewhere when you want the build to fail you add ant code like this:
<target name="doSomethingIfTestsWereSuccessful" unless="tests.failed">
...
</target>
<target name="doSomethingIfTestsFailed" if="tests.failed">
...
<fail message="Tests Failed" />
</target>
You can then call doSomethingIfTestsFailed where you want your ant build to fail.
Although you are showing a fake task called "testng" in your example I presume you are using the junit target.
In this case, it is strange you are seeing these results because the junit target by default does NOT abort execution on a test failure.
There is a way to actually tell ant to stop the build on a junit failure or error by using the halt attributes, eg. haltonfailure:
<target name="junit" depends="junitcompile">
<junit printsummary="withOutAndErr" fork="yes" haltonfailure="yes">
However, both haltonfailure and haltonerror are by default set to off. I suppose you could check your build file to see if either of these flags have been set. They can even be set globally, so one thing you could try is to explicitly set it to "no" on your task to make sure it is overridden in case it is set in the global scope.
http://ant.apache.org/manual/Tasks/junit.html
Can you fork the testng task ? If yes, then, you might want to use that feature so that the testng task will run on a different JVM.
Related
I have test in java that run on Jenkins.
Is it no possible to debug that test and therefore I want to add dummy comment to the java and also to see values of variables
I want to see it in the console output of the Jenkins ?
How I can do it ?
System.out.println() did not work for me
If you use a java testing framework like JUnit, then any tests run will be logged by Jenkins, including any print statements that the tests execute.
Jenkins also has a plugin specifically for JUnit (found here), which will publish the results so you can easily access them in Jenkins, allowing you to see if tests fail, and then inspect anything printed out, as well as the stack trace, for any given test.
If your junit tests are being run by an ant task, then you can simply add a formatter to your task. For example:
<target name="unit-tests">
<junit>
...
<formatter type="plain" usefile="false" />
</junit>
</target>
Copied directly from this post
I have found that the following :
<junit haltonfailure="no" haltonerror="no" >
<!-- some tests guaranteed to call Assert.fail(); -->
</junit>
Will not prematurely halt (all tests will run).
HOWEVER ...... The following
<junit haltonfailure="no" haltonerror="yes" >
Will fail after the FIRST failed test. Note that in this case, I'm guaranteeing that the test is failing via Assert.fail();
This, I am assuming that either
1) Ant considers an ERROR to be a subclass of a FAILURE. or
2) There is a oddity in the way ant percieves failures - that is, failures are treated (erroneously) as errors when deciding to halt.
My Question : Why would ant would halt on a test failure (even when explicitly configured with haltonfailure="no"). Any thoughts could be very enlightening ! Thanks.
UPDATE
I now realize that this is clearly an RTFM question which is covered in the ant http://ant.apache.org/manual/Tasks/junit.html (see correct answer below).
Quote from junit ant task documentation:
haltonfailure: Stop the build process if a test fails (errors are considered failures as well).
So the answer is: ant junit considers failures as errors, hence the stop of the build
I receive the following warning:
[javac] build.xml:9: warning: 'includeantruntime' was not set,
defaulting to build.sysclasspath=last; set to false for repeatable builds
What does this mean?
Ant Runtime
Simply set includeantruntime="false":
<javac includeantruntime="false" ...>...</javac>
If you have to use the javac-task multiple times you might want to consider using PreSetDef to define your own javac-task that always sets includeantruntime="false".
Additional Details
From http://www.coderanch.com/t/503097/tools/warning-includeantruntime-was-not-set:
That's caused by a misfeature
introduced in Ant 1.8. Just add an
attribute of that name to the javac
task, set it to false, and forget it
ever happened.
From http://ant.apache.org/manual/Tasks/javac.html:
Whether to include the Ant run-time
libraries in the classpath; defaults
to yes, unless build.sysclasspath is
set. It is usually best to set this to
false so the script's behavior is not
sensitive to the environment in which
it is run.
As #Daniel Kutik mentioned, presetdef is a good option. Especially if one is working on a project with many build.xml files which one cannot, or prefers not to, edit (e.g., those from third-parties.)
To use presetdef, add these lines in your top-level build.xml file:
<presetdef name="javac">
<javac includeantruntime="false" />
</presetdef>
Now all subsequent javac tasks will essentially inherit includeantruntime="false". If your projects do actually need ant runtime libraries, you can either add them explicitly to your build files OR set includeantruntime="true". The latter will also get rid of warnings.
Subsequent javac tasks can still explicitly change this if desired, for example:
<javac destdir="out" includeantruntime="true">
<src path="foo.java" />
<src path="bar.java" />
</javac>
I'd recommend against using ANT_OPTS. It works, but it defeats the purpose of the warning. The warning tells one that one's build might behave differently on another system. Using ANT_OPTS makes this even more likely because now every system needs to use ANT_OPTS in the same way. Also, ANT_OPTS will apply globally, suppressing warnings willy-nilly in all your projects
Chet Hosey wrote a nice explanation here:
Historically, Ant always included its own runtime in the classpath made available to the javac task. So any libraries included with Ant, and any libraries available to ant, are automatically in your build's classpath whether you like it or not.
It was decided that this probably wasn't what most people wanted. So now there's an option for it.
If you choose "true" (for includeantruntime), then at least you know that your build classpath will include the Ant runtime. If you choose "false" then you are accepting the fact that the build behavior will change between older versions and 1.8+.
As annoyed as you are to see this warning, you'd be even less happy if your builds broke entirely. Keeping this default behavior allows unmodified build files to work consistently between versions of Ant.
The answer from Daniel works just perfect. Here is a sample snippet that I added to my build.xml:
<target name="compile">
<mkdir dir="${classes.dir}"/>
<javac srcdir="${src.dir}" destdir="${classes.dir}" includeantruntime="false">
<!-- ^^^^^^^^^^^^^^^^^^^^^^^^^ -->
<classpath>
<path id="application" location="${jar.dir}/${ant.project.name}.jar"/>
<path id="junit" location="${lib.dir}/junit-4.9b2.jar"/>
</classpath>
</javac>
</target>
If you like me work from commandline the quick answer is executing
export ANT_OPTS=-Dbuild.sysclasspath=ignore
And then run your ant script again.
Use <property name="build.sysclasspath" value="last"/> in your build.xml file
For more details search includeAntRuntime in Ant javac
Other possible values could be found here
i faced this same, i check in in program and feature. there was an update has install for jdk1.8 which is not compatible with my old setting(jdk1.6.0) for ant in eclipse.
I install that update.
right now, my ant project is build success.
Try it, hope this will be helpful.
The core of my question is that I am concerned that my Ant build file is missing something that will allow a test to finish and clean itself up. The details are below.
I have a suite of tests that always passes when I run it through Eclipse, but sometimes passes or fails when I run it using my Ant build. The tests use openCL via JOCL so I have limited memory on the GPU and it has to be managed correctly. I get this in my output sometimes when I run my Ant build,
[junit] Caused an ERROR
[junit] CL_MEM_OBJECT_ALLOCATION_FAILURE
[junit] org.jocl.CLException: CL_MEM_OBJECT_ALLOCATION_FAILURE
The problem can not be in the test itself. I think it is that my most memory hungry test is invoked at the end of the suite. When this last test is invoked, somehow the GPU is left in a bad state from my previous tests. This doesn't happen when I run the tests through Eclipse. It has never failed in my Ant build when I make the memory hungry test the first test in the suite. Is this a familiar case? Why does running the tests through Eclipse always work? Is there anything I can try?
Here is the testing target in my Ant build:
<target name="test" if="testing.enabled">
<mkdir dir="${test.bin.dir}" />
<javac srcdir="test" destdir="${test.bin.dir}" debug="true" classpathref="testclasspath" source="1.6"/>
<junit haltonerror="true" haltonfailure="true">
<classpath refid="testclasspath"/>
<formatter type="plain" usefile="false" />
<batchtest>
<fileset dir="test">
<include name="*Test.java"/>
</fileset>
</batchtest>
</junit>
</target>
If you are really sure no left-over cleanup is missed in your code, you can create a JUnit test suite and run that from both eclipse and ant. By creating a test suite you make yourself independent of the sequence of tests that eclipse (order within the project?) and ant (order within the filesystem?) use and determine the order yourself in both cases.
If you are not really really sure your code is issue-free you could make a test suite which starts of by calling Collections.shuffle() on the list of test classes to introduce unknown test execution order in both eclipse see if your tests still never fail.
The problem could be that you do not free memory in your test cases. JUnit instanciates all test classes when it is started and then runs them, as far as I know. If you have fields that reference objects in your test classes all fields will stay assigned through the whole testrun when you don't assign null to them in a tearDown() method. For example:
class Test extends TestCase {
private Data data;
public void setUp() {
data = new Data();
}
public void tearDown() {
data = null; // required to allow garbage collection of the Data object
}
}
Maybe Eclipse unreferences the Test instances after they are executed, so that the fields can be garbage collected. But using the standard JUnit TestRunner you will end up with a lot of objects that are not used anymore but that are still referenced and eat up all your memory.
If the tests are passing in Eclipse and failing elsewhere, then you're suffering from one of the many kinds of developer syndrome: "...but It works when I run it here...!"
You have managed to configure Eclipse to let you work with your code, the functionality is in, yet your code is, this far, not deployable, which means it's not done.
Shelve Eclipse for a while (stop blaming it), and drop to the command line (or use a different IDE) until things work. Try the code on a different computer, even!
Then go back to Eclipse, and repeat the above cycle until you're certain that any dependencies on Eclipse or your hard disk/setup have been removed. In the end, your code must be able to run on who-knows-which server.
Have you tried having a clean Eclipse installation (on a different computer) take a shot at a source-only snapshot of the code? It would be a good configurations management test that I'm quite sure your code won't pass as it stands.
Seriously try to get Eclipse doing its magic on a clean (virtual) machine. It won't work on a first run, but you'll learn what you did to make it work under your setup.
Let me google that for you:
Ant looks for an environment variable called ANT_OPTS which is use to set Java parameters. > Just set the environment variable and off you go. So I added the following to increase the > heap size:
export ANT_OPTS=-Xmx256m
When running withing Eclipse, the JVM (and therefore Ant) most likely already has more memory than default.
We're using the task within our master build to invoke targets in separate ant builds for each of our sub-projects. So far so good, we get things built, we can even run JUnit tasks within each one and everybody is happy. However...
We want to take it to the next level, we would like to have a single JUnit report generated from the JUnit test result XML for all of our sub-projects but if we terminate the build whenever any of the sub-projects has a unit test failure, we never get to the point where we can generate a unified report. So that suggests that we would somehow note that unit tests failed within one or more of the sub-projects and not fail immediately but wait until the end of the master build to fail.
What mechanism exists for that communication from the child builds up to the master build though? Properties are immutable and everything else we think of (properties files we update, files we touch, etc.) seem like horrible kludges. Is there a way to do this that fits in nicely with Ant and doesn't seem like something horrible we grafted on?
The junit task supports the haltonerror and haltonfailure attributes, which, if set to false, will cause the ant script to continue to run even if a test fails. There is also an errorproperty and failureproperty you can set instead. You can then copy your junit reports to a master directory (with all success and failures), and use the fail task to fail if either of those properties have been set.
Something along the lines of:
<target name="run-tests" >
<junit printsummary="on" fork="yes" haltonfailure="false" haltonerror="false" errorproperty="test.failed" failureproperty="tests.failed" showoutput="true" maxmemory="512m">
<classpath refid="classpath" />
<formatter type="xml" />
<batchtest todir="test/reports">
<fileset dir="${build.classes.dir}">
<patternset refid="testfiles" />
</fileset>
</batchtest>
</junit>
<copy todir="test/master/reports" dir="test/reports" />
<fail if="tests.failed"/>
</target>
OK, I never got an answer on this one that I liked that much but we did end up finding a good solution we like a lot. We switched from using Anthill OS as our build server to Hudson and after we did that we were able to take advantage of a Hudson feature where it will aggregate JUnit results from any number of locations to produce a single report that goes with each build (whether successful or not). So, in short, use Hudson. It rocks!