I'm trying to retrieve the total [unit]test execution time (surefire for now).
There are couple of places stating we should parse the surefire result (with some text processing) like here
generating the test execution time is not a big problem while tests are running sequentially, but how can I retrieve the test execution time when surefire is configure to run the tests in parallel
for example if I use something like this is pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<parallel>methods</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
<threadCount>10</threadCount>
</configuration>
</plugin>
<plugin>
how should I retrieve the overall [unit] test execution time?
Is there any flag added to the result which I can take advantage of?
Related
I have all the timeout properties set in my pom.xml.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<forkCount>${surefire.forkNumber}</forkCount>
<reuseForks>false</reuseForks>
<argLine>-Xmx2G -XX:MaxMetaspaceSize=2G </argLine>
<parallel>methods</parallel>
<threadCount>${surefire.threadCount}</threadCount>
<forkedProcessTimeoutInSeconds>40s</forkedProcessTimeoutInSeconds>
<forkedProcessExitTimeoutInSeconds>40s</forkedProcessExitTimeoutInSeconds>
<parallelTestsTimeoutInSeconds>30s</parallelTestsTimeoutInSeconds>
<parallelTestsTimeoutForcedInSeconds>30s</parallelTestsTimeoutForcedInSeconds>
</configuration>
</plugin>
But I still see a few tests are taking more time than 40s (one test is even taking 17 min). What could be the possible reason? How can I enforce 30s timeout with surefire?
Document is specifying that you have to enable option
parallel
You have to set
<parallelTestTimeoutForcedInSeconds>5</parallelTestTimeoutForcedInSeconds>
to force current running on timeout.
If you are using
parallelTestTimeoutInSeconds
only the queued threads will be stopped from executing
Refer: http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html
Is it possible to execute same life cycle of two maven plugin if one fails ?
Example:
Let's say I have below plugin configuration,
<plugins>
<plugin>
<groupId>smothing</groupId>
<artifactId>plugin-1</artifactId>
<version>2.2</version>
<executions>
<execution>
<id>doSomthing</id>
<phase>test</phase>
//...//
</plugin>
<plugin>
<groupId>something</groupId>
<artifactId>plugin-2</artifactId>
<version>2.5</version>
<executions>
<execution>
<id>doSomthingAgain</id>
<phase>test</phase>
//...
</plugin>
</plugins>
I would like to execute plugin-2 test phase even if the first plugin fails. I don't want to ignore or skip test cases.
I have below two plugin to be executed same phase even if one fails.
<groupId>com.thoughtworks.gauge.maven</groupId>
<artifactId>gauge-maven-plugin</artifactId>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
Basically, after the gauge tests I want to perform some cleanup activities through maven exec plugin. So is there any option for me to always execute maven exec plugin ? (No command line arguments, something which I am expecting in pom.xml )
I have seen these answers, but everything says to skip test cases.
How to run a maven goal when there is tests failures?
Maven reporting plugins do not execute if a unit test failure occurs
Any help much appreciated :)
If a plugin fails, it'll stop the execution of the lifecycle. So you shouldn't try to solve it by thinking of executing another plugin for some condition.
Based on your description the best approach seems to be writing an extension, see https://maven.apache.org/examples/maven-3-lifecycle-extensions.html . With https://maven.apache.org/ref/3.6.0/maven-core/apidocs/index.html?org/apache/maven/execution/AbstractExecutionListener.html you can see that you can do actions before or after any segment of the lifecycle, e.g. cleaning up after projectSucceeded+projectFailed
I have a test suite which needs to have some setup code to be executed before hand to make sure some data is correct in our database.
We're using the maven surefire plugin to run tests in parallel. ${tests.wildcard} is specified by the profile.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<forkCount>4</forkCount>
<reuseForks>false</reuseForks>
<includes>
<include>${tests.wildcard}</include>
</includes>
</configuration>
</plugin>
I'd like to be able to run a method only once per entire maven execution before surefire runs my tests in parallel. How can I do that?
You could have a special test case which executes your verification code (and fail if the case).
This test case would then be executed by a specific Maven Surefire execution (excluding other tests) and attached to a Maven phase occurring just before the test phase (like process-test-classes): hence, effectively being invoked once per Maven run and before any other test.
Then, the normal test phase would execute any other desired test, excluding the special init test.
An example of such a configuration would be:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<excludes>
<exclude>**/InitTest.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>test-init</id>
<phase>process-test-classes</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<test>InitTest</test>
</configuration>
</execution>
</executions>
</plugin>
Note the global configuration for any Surefire execution would exclude the special init test. Then an additional execution would be executed (before the test phase) and would run only the init test (using the <test> element, which takes priority over any other include/exclude).
As such you would have the following flow:
Execute special init test and verify data into database (and fail if required)
Execute any other test
Update
Note that you can achieve the same and perhaps in a more semantically correct way by overriding the default surefire test execution (having default-test execution id) to run the special test (and exclude the others) and then add another surefire execution for the rest (as described above as global configuration, this time as specific execution configuration).
With this approach everything would be attached to the test phase, that's why it would be semantically more correct, although a little bit more verbose in the pom.
I'm working on creating a reporting mechanism for tests and I want it to execute after all of the tests are run, and the resulting junit xml files are written out. Unfortunately, it doesn't appear that the executing continues after it fails in the test phase. Does anybody know what Maven configuration I could use to get the desired result if possible. Many Thanks.
May be this helps?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
We have our POM defining the maven-surefire-plugin as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14</version>
<configuration>
<reuseForks>false</reuseForks>
<forkCount>1</forkCount>
<argLine>-Xms64m -Xmx256m</argLine>
<includes>
<include>**/*Test.java</include>
</includes>
</configuration>
</plugin>
However, our Java tests (which involve some parallel tests and static singletons) only run properly when we run our test phase/build using:
mvn test -DforkMode=always
Strangely, even if we change our <configuration> to use (instead of the newer options):
<forkMode>always</forkMode>
And run:
mvn test
It will fail. But then if we run:
mvn test -DforkMode=always
It will pass. Instead of the newer options, it still will only work if we explicitly provide forkMode on the command line. We have tried this with multiple versions of the surefire plugin, to the same effect.
Are there any locations where this property could be overridden, or known issues in which the XML configuration is not properly used?
Rookie mistake. The configuration I was using was listed in a separate <profile> block that was not being executed. The profile with:
<activeByDefault>true</activeByDefault>
Did not include its own Surefire configuration at all (so it didn't show up in a search), and was using inherited values, which explains why the command-line system properties were able to affect its behavior.