I have all the timeout properties set in my pom.xml.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<forkCount>${surefire.forkNumber}</forkCount>
<reuseForks>false</reuseForks>
<argLine>-Xmx2G -XX:MaxMetaspaceSize=2G </argLine>
<parallel>methods</parallel>
<threadCount>${surefire.threadCount}</threadCount>
<forkedProcessTimeoutInSeconds>40s</forkedProcessTimeoutInSeconds>
<forkedProcessExitTimeoutInSeconds>40s</forkedProcessExitTimeoutInSeconds>
<parallelTestsTimeoutInSeconds>30s</parallelTestsTimeoutInSeconds>
<parallelTestsTimeoutForcedInSeconds>30s</parallelTestsTimeoutForcedInSeconds>
</configuration>
</plugin>
But I still see a few tests are taking more time than 40s (one test is even taking 17 min). What could be the possible reason? How can I enforce 30s timeout with surefire?
Document is specifying that you have to enable option
parallel
You have to set
<parallelTestTimeoutForcedInSeconds>5</parallelTestTimeoutForcedInSeconds>
to force current running on timeout.
If you are using
parallelTestTimeoutInSeconds
only the queued threads will be stopped from executing
Refer: http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html
Related
I got this error while running a junit-test in my application. Which I later found out because of declared field size
java.lang.IllegalArgumentException: Comparison method violates its general contract!
at java.util.TimSort.mergeHi(TimSort.java:899)
at java.util.TimSort.mergeAt(TimSort.java:516)
at java.util.TimSort.mergeCollapse(TimSort.java:441)
...
org.mockito.internal.configuration.injection.PropertyAndSetterInjection.orderedInstanceFieldsFrom(PropertyAndSetterInjection.java:125)
I found that probable solution would be adding this flag.
-Djava.util.Arrays.useLegacyMergeSort=true in VM args. But I wanted to add in pom.xml
I referred to this how to add VM args using pom xml but it refers mostly for -X flags, what would be an appropriate placement in here?
I updated the pom.xml with the surefire plugin and used argLine parameter as suggested here
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<argLine>-Djava.util.Arrays.useLegacyMergeSort=true</argLine>
</configuration>
</plugin>
Trere are 2 ways to configure [System Properties in surefire plugin] (https://maven.apache.org/surefire/maven-surefire-plugin/examples/system-properties.html)
Option 1: systemPropertyVariables
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<systemPropertyVariables>
<java.util.Arrays.useLegacyMergeSort>true</java.util.Arrays.useLegacyMergeSort>
</systemPropertyVariables>
</configuration>
</plugin>
Option 2: argLine
Some system properties must be set on the command line of the forked VM, and cannot be set after the VM has been started. These properties must be added to the argLine parameter of the Surefire plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<argLine>-Djava.util.Arrays.useLegacyMergeSort=true</argLine>
</configuration>
</plugin>
Upgrade your dependencies
On the other hand, I really think that the right solution is to upgrade Mockito to 2.1 or later
2.1 was released in October, 2016, and contains the fix to your issue: Make PropertyAndSetterInjection field sorting consistent #176
I'm trying to retrieve the total [unit]test execution time (surefire for now).
There are couple of places stating we should parse the surefire result (with some text processing) like here
generating the test execution time is not a big problem while tests are running sequentially, but how can I retrieve the test execution time when surefire is configure to run the tests in parallel
for example if I use something like this is pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<parallel>methods</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
<threadCount>10</threadCount>
</configuration>
</plugin>
<plugin>
how should I retrieve the overall [unit] test execution time?
Is there any flag added to the result which I can take advantage of?
I'm trying to get Maven surefire to run under JDK 11 but I keep getting these errors:
If I set reuseForks to true:
Error occurred in starting fork, check output in log
Process Exit Code: 1
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:670)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:283)
at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:246)
If I set it to false:
Execution default-test of goal org.apache.maven.plugins:maven-surefire- plugin:3.0.0-M1:test
failed: java.lang.ClassNotFoundException: org.apache.maven.plugin.surefire.StartupReportConfiguration
I've found this and this link that describe the same problem but they don't have any solution.
For replication of this bug I created this git repo
Seems like while using a modular project to test, you need to have forkCount set as 0 :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M1</version>
<configuration>
<forkCount>0</forkCount> <!-- changed this to 0 -->
<reuseForks>false</reuseForks>
<!-- <threadCount>1</threadCount> --> <!-- shall be used with 'parallel' -->
<printSummary>true</printSummary>
<!-- <skipTests>false</skipTests> --> <!-- defaults to false -->
<!-- run test in headless mode -->
<systemPropertyVariables>
<glass.platform>Monocle</glass.platform>
<monocle.platform>Headless</monocle.platform>
<prism.order>d3d</prism.order>
</systemPropertyVariables>
<argLine>
--add-exports javafx.graphics/com.sun.javafx.application=ALL-UNNAMED
--add-exports javafx.graphics/com.sun.glass.ui=ALL-UNNAMED
</argLine>
</configuration>
</plugin>
Quoting from this article
When module-info.java is present and fork process is enabled, surefire
creates a mixed classpath with modules and unnamed modules causing
module visibility issues and preventing the application to start.
Note: Disabling the forkCount and reuseForks configuration parameters, results in org.apache.maven.surefire.booter.SurefireBooterForkException being thrown, similar to the one reported in SUREFIRE-1528.
If this could help the developers at Maven community, the execution dump from the same run reads the following:
# Created at 2018-11-23T09:31:53.631
Corrupted STDOUT by directly writing to native stream in forked JVM 1. Stream 'Error occurred during initialization of boot layer'.
java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'Error occurred during initialization of boot layer'.
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:507)
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:210)
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:177)
at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:88)
at java.base/java.lang.Thread.run(Thread.java:834)
Here is the line of code in the source repository.
I found the solution here:
https://winterbe.com/posts/2018/08/29/migrate-maven-projects-to-java-11-jigsaw/
I had to add.
<argLine>
--illegal-access=permit
</argLine>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<argLine>
--illegal-access=permit
</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<argLine>
--illegal-access=permit
</argLine>
</configuration>
</plugin>
I have a test suite which needs to have some setup code to be executed before hand to make sure some data is correct in our database.
We're using the maven surefire plugin to run tests in parallel. ${tests.wildcard} is specified by the profile.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<forkCount>4</forkCount>
<reuseForks>false</reuseForks>
<includes>
<include>${tests.wildcard}</include>
</includes>
</configuration>
</plugin>
I'd like to be able to run a method only once per entire maven execution before surefire runs my tests in parallel. How can I do that?
You could have a special test case which executes your verification code (and fail if the case).
This test case would then be executed by a specific Maven Surefire execution (excluding other tests) and attached to a Maven phase occurring just before the test phase (like process-test-classes): hence, effectively being invoked once per Maven run and before any other test.
Then, the normal test phase would execute any other desired test, excluding the special init test.
An example of such a configuration would be:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<excludes>
<exclude>**/InitTest.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>test-init</id>
<phase>process-test-classes</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<test>InitTest</test>
</configuration>
</execution>
</executions>
</plugin>
Note the global configuration for any Surefire execution would exclude the special init test. Then an additional execution would be executed (before the test phase) and would run only the init test (using the <test> element, which takes priority over any other include/exclude).
As such you would have the following flow:
Execute special init test and verify data into database (and fail if required)
Execute any other test
Update
Note that you can achieve the same and perhaps in a more semantically correct way by overriding the default surefire test execution (having default-test execution id) to run the special test (and exclude the others) and then add another surefire execution for the rest (as described above as global configuration, this time as specific execution configuration).
With this approach everything would be attached to the test phase, that's why it would be semantically more correct, although a little bit more verbose in the pom.
I'm working on creating a reporting mechanism for tests and I want it to execute after all of the tests are run, and the resulting junit xml files are written out. Unfortunately, it doesn't appear that the executing continues after it fails in the test phase. Does anybody know what Maven configuration I could use to get the desired result if possible. Many Thanks.
May be this helps?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>