I have test in java that run on Jenkins.
Is it no possible to debug that test and therefore I want to add dummy comment to the java and also to see values of variables
I want to see it in the console output of the Jenkins ?
How I can do it ?
System.out.println() did not work for me
If you use a java testing framework like JUnit, then any tests run will be logged by Jenkins, including any print statements that the tests execute.
Jenkins also has a plugin specifically for JUnit (found here), which will publish the results so you can easily access them in Jenkins, allowing you to see if tests fail, and then inspect anything printed out, as well as the stack trace, for any given test.
If your junit tests are being run by an ant task, then you can simply add a formatter to your task. For example:
<target name="unit-tests">
<junit>
...
<formatter type="plain" usefile="false" />
</junit>
</target>
Copied directly from this post
Related
I need to cause my build to fail based only on the junit report. Is there a way to do this. I know how to fail the build using haltonfailure in junit tag, but in my build.xml I have access only to the junit report. I am using ant.
Try something like
<fail if="testFail" message="At least one unit test failed"/>
in your unit test target.
Refer to this: http://ant.apache.org/manual/Tasks/junit.html ,
try using failureproperty="failed.unit.test.property" inside junit ant task & check if its set & fail when your run completes with :
<fail if="failed.unit.test.property" message="Unit tests failed with more custom msg to your test class/suite"/>
I have found that the following :
<junit haltonfailure="no" haltonerror="no" >
<!-- some tests guaranteed to call Assert.fail(); -->
</junit>
Will not prematurely halt (all tests will run).
HOWEVER ...... The following
<junit haltonfailure="no" haltonerror="yes" >
Will fail after the FIRST failed test. Note that in this case, I'm guaranteeing that the test is failing via Assert.fail();
This, I am assuming that either
1) Ant considers an ERROR to be a subclass of a FAILURE. or
2) There is a oddity in the way ant percieves failures - that is, failures are treated (erroneously) as errors when deciding to halt.
My Question : Why would ant would halt on a test failure (even when explicitly configured with haltonfailure="no"). Any thoughts could be very enlightening ! Thanks.
UPDATE
I now realize that this is clearly an RTFM question which is covered in the ant http://ant.apache.org/manual/Tasks/junit.html (see correct answer below).
Quote from junit ant task documentation:
haltonfailure: Stop the build process if a test fails (errors are considered failures as well).
So the answer is: ant junit considers failures as errors, hence the stop of the build
The core of my question is that I am concerned that my Ant build file is missing something that will allow a test to finish and clean itself up. The details are below.
I have a suite of tests that always passes when I run it through Eclipse, but sometimes passes or fails when I run it using my Ant build. The tests use openCL via JOCL so I have limited memory on the GPU and it has to be managed correctly. I get this in my output sometimes when I run my Ant build,
[junit] Caused an ERROR
[junit] CL_MEM_OBJECT_ALLOCATION_FAILURE
[junit] org.jocl.CLException: CL_MEM_OBJECT_ALLOCATION_FAILURE
The problem can not be in the test itself. I think it is that my most memory hungry test is invoked at the end of the suite. When this last test is invoked, somehow the GPU is left in a bad state from my previous tests. This doesn't happen when I run the tests through Eclipse. It has never failed in my Ant build when I make the memory hungry test the first test in the suite. Is this a familiar case? Why does running the tests through Eclipse always work? Is there anything I can try?
Here is the testing target in my Ant build:
<target name="test" if="testing.enabled">
<mkdir dir="${test.bin.dir}" />
<javac srcdir="test" destdir="${test.bin.dir}" debug="true" classpathref="testclasspath" source="1.6"/>
<junit haltonerror="true" haltonfailure="true">
<classpath refid="testclasspath"/>
<formatter type="plain" usefile="false" />
<batchtest>
<fileset dir="test">
<include name="*Test.java"/>
</fileset>
</batchtest>
</junit>
</target>
If you are really sure no left-over cleanup is missed in your code, you can create a JUnit test suite and run that from both eclipse and ant. By creating a test suite you make yourself independent of the sequence of tests that eclipse (order within the project?) and ant (order within the filesystem?) use and determine the order yourself in both cases.
If you are not really really sure your code is issue-free you could make a test suite which starts of by calling Collections.shuffle() on the list of test classes to introduce unknown test execution order in both eclipse see if your tests still never fail.
The problem could be that you do not free memory in your test cases. JUnit instanciates all test classes when it is started and then runs them, as far as I know. If you have fields that reference objects in your test classes all fields will stay assigned through the whole testrun when you don't assign null to them in a tearDown() method. For example:
class Test extends TestCase {
private Data data;
public void setUp() {
data = new Data();
}
public void tearDown() {
data = null; // required to allow garbage collection of the Data object
}
}
Maybe Eclipse unreferences the Test instances after they are executed, so that the fields can be garbage collected. But using the standard JUnit TestRunner you will end up with a lot of objects that are not used anymore but that are still referenced and eat up all your memory.
If the tests are passing in Eclipse and failing elsewhere, then you're suffering from one of the many kinds of developer syndrome: "...but It works when I run it here...!"
You have managed to configure Eclipse to let you work with your code, the functionality is in, yet your code is, this far, not deployable, which means it's not done.
Shelve Eclipse for a while (stop blaming it), and drop to the command line (or use a different IDE) until things work. Try the code on a different computer, even!
Then go back to Eclipse, and repeat the above cycle until you're certain that any dependencies on Eclipse or your hard disk/setup have been removed. In the end, your code must be able to run on who-knows-which server.
Have you tried having a clean Eclipse installation (on a different computer) take a shot at a source-only snapshot of the code? It would be a good configurations management test that I'm quite sure your code won't pass as it stands.
Seriously try to get Eclipse doing its magic on a clean (virtual) machine. It won't work on a first run, but you'll learn what you did to make it work under your setup.
Let me google that for you:
Ant looks for an environment variable called ANT_OPTS which is use to set Java parameters. > Just set the environment variable and off you go. So I added the following to increase the > heap size:
export ANT_OPTS=-Xmx256m
When running withing Eclipse, the JVM (and therefore Ant) most likely already has more memory than default.
We're using the task within our master build to invoke targets in separate ant builds for each of our sub-projects. So far so good, we get things built, we can even run JUnit tasks within each one and everybody is happy. However...
We want to take it to the next level, we would like to have a single JUnit report generated from the JUnit test result XML for all of our sub-projects but if we terminate the build whenever any of the sub-projects has a unit test failure, we never get to the point where we can generate a unified report. So that suggests that we would somehow note that unit tests failed within one or more of the sub-projects and not fail immediately but wait until the end of the master build to fail.
What mechanism exists for that communication from the child builds up to the master build though? Properties are immutable and everything else we think of (properties files we update, files we touch, etc.) seem like horrible kludges. Is there a way to do this that fits in nicely with Ant and doesn't seem like something horrible we grafted on?
The junit task supports the haltonerror and haltonfailure attributes, which, if set to false, will cause the ant script to continue to run even if a test fails. There is also an errorproperty and failureproperty you can set instead. You can then copy your junit reports to a master directory (with all success and failures), and use the fail task to fail if either of those properties have been set.
Something along the lines of:
<target name="run-tests" >
<junit printsummary="on" fork="yes" haltonfailure="false" haltonerror="false" errorproperty="test.failed" failureproperty="tests.failed" showoutput="true" maxmemory="512m">
<classpath refid="classpath" />
<formatter type="xml" />
<batchtest todir="test/reports">
<fileset dir="${build.classes.dir}">
<patternset refid="testfiles" />
</fileset>
</batchtest>
</junit>
<copy todir="test/master/reports" dir="test/reports" />
<fail if="tests.failed"/>
</target>
OK, I never got an answer on this one that I liked that much but we did end up finding a good solution we like a lot. We switched from using Anthill OS as our build server to Hudson and after we did that we were able to take advantage of a Hudson feature where it will aggregate JUnit results from any number of locations to produce a single report that goes with each build (whether successful or not). So, in short, use Hudson. It rocks!
I'm trying to define a task that emits (using echo) a message when a target completes execution, regardless of whether that target was successful or not. Specifically, the target executes a task to run some unit tests, and I want to emit a message indicating where the results are available:
<target name="mytarget">
<testng outputDir="${results}" ...>
...
</testng>
<echo>Tests complete. Results available in ${results}</echo>
</target>
Unfortunately, if the tests fail, the task fails and execution aborts. So the message is only output if the tests pass - the opposite of what I want. I know I can put the task before the task, but this will make it easier for users to miss this message. Is what I'm trying to do possible?
Update: It turns out I'm dumb. I had haltOnFailure="true" in my <testng> task, which explains the behaviour I was seeing. Now the issue is that setting this to false causes the overall ant build to succeed even if tests fail, which is not what I want. The answer below using the task looks like it might be what I want..
You can use a try-catch block like so:
<target name="myTarget">
<trycatch property="foo" reference="bar">
<try>
<testing outputdir="${results}" ...>
...
</testing>
</try>
<catch>
<echo>Test failed</echo>
</catch>
<finally>
<echo>Tests complete. Results available in ${results}</echo>
</finally>
</trycatch>
</target>
According to the Ant docs, there are two properties that control whether the build process is stopped or not if the testng task fails:
haltonfailure - Stop the build process
if a failure has occurred during the
test run. Defaults to false.
haltonskipped - Stop the build process
if there is at least on skipped test.
Default to false.
I can't tell from the snippet if you're setting this property or not. May be worth trying to explicitly set haltonfailure to false if it's currently set to true.
Also, assuming you're using the <exec> functionality in Ant, there are similar properties to control what happens if the executed command fails:
failonerror - Stop the buildprocess if the command exits with a return code
signaling failure. Defaults to false.
failifexecutionfails - Stop the build if we can't start the program.
Defaults to true.
Can't tell based on the partial code snippet in your post, but my guess is that the most likely culprit is failonerror or haltonfailure being set to true.
The solution to your problem is to use the failureProperty in conjunction with the haltOnFailure property of the testng task like this:
<target name="mytarget">
<testng outputDir="${results}" failureProperty="tests.failed" haltOnFailure="false" ...>
...
</testng>
<echo>Tests complete. Results available in ${results}</echo>
</target>
Then, elsewhere when you want the build to fail you add ant code like this:
<target name="doSomethingIfTestsWereSuccessful" unless="tests.failed">
...
</target>
<target name="doSomethingIfTestsFailed" if="tests.failed">
...
<fail message="Tests Failed" />
</target>
You can then call doSomethingIfTestsFailed where you want your ant build to fail.
Although you are showing a fake task called "testng" in your example I presume you are using the junit target.
In this case, it is strange you are seeing these results because the junit target by default does NOT abort execution on a test failure.
There is a way to actually tell ant to stop the build on a junit failure or error by using the halt attributes, eg. haltonfailure:
<target name="junit" depends="junitcompile">
<junit printsummary="withOutAndErr" fork="yes" haltonfailure="yes">
However, both haltonfailure and haltonerror are by default set to off. I suppose you could check your build file to see if either of these flags have been set. They can even be set globally, so one thing you could try is to explicitly set it to "no" on your task to make sure it is overridden in case it is set in the global scope.
http://ant.apache.org/manual/Tasks/junit.html
Can you fork the testng task ? If yes, then, you might want to use that feature so that the testng task will run on a different JVM.