I run a JUnit-based test suite in Maven, some 200 tests in total. Many produce Log4J DEBUG or ERROR lines, and I see them all mashed into a single long file in my surefire-reports directory.
I want to see all this output divided up by test, so I can discover what test produced what output. How can I do this?
The most straightforward solution would be to a call to the following method to the beginning and end of each test:
static void printBeginning(PrintStream stream) {
String methodName = Thread.currentThread().getStackTrace()[0].getMethodName();
stream.format("---- Beginning Test: %s ----%n", methodName);
}
However, if you want to use surefire, the way to do this would be to write your own custom RunListener:
Basically, assign the RunListener in Maven, as in this documentation
<plugins>
[...]
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<properties>
<property>
<name>listener</name>
<value>com.mycompany.MyResultListener,com.mycompany.MyResultListener2</value>
</property>
</configuration>
</plugin>
[...]
</plugins>
Then the code will be run by Surefire.
You can read more about how to do this here: Using JUnit RunListener with Maven
Related
My goal: using parameters to switch environments in my tests for instance:
mvn test google -> Tests goes to google site
mvn test bing -> bing site
"I need to feed my tests which environment is the target and it should comes from the pom.xml and using them as arguments."
It would be very useful for teamcity/jenkins integration as well.
Apart from that, I need to use url as variables in my tests. How can I do that?
Profiles can be a solution in pom.xml?
<profiles>
<profile>
<id>google</id>
<properties>
<base.url>http://www.google.com</base.url>
</properties>
</profile>
<profile>
<id>bing</id>
<properties>
<base.url>http://www.bing.com</base.url>
</properties>
</profile>
</profiles>
From build section:
<configuration>
<systemProperties>
<base.url>${base.url}</base.url>
</systemProperties>
</configuration>
But How can I use system properties and overall the approach is good? Thanks!
You can configure the maven-surefire-plugin to include only specific test classes and the run mvn test. By default, mvn will run all these:
"**/Test*.java" - includes all of its subdirectories and all Java filenames that start with "Test".
"**/*Test.java" - includes all of its subdirectories and all Java filenames that end with "Test".
"**/*Tests.java" - includes all of its subdirectories and all Java filenames that end with "Tests".
"**/*TestCase.java" - includes all of its subdirectories and all Java filenames that end with "TestCase".
but you could specify the tests you want to include like this:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<includes>
<include>Sample.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
or exclude:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<excludes>
<exclude>**/TestCircle.java</exclude>
<exclude>**/TestSquare.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
Having said that, this is probably not the best design and generally, you should be using some testing framework which you then can configure according to your needs. Few examples are (or combination of): jUnit, TestNG, Cucumber, Spring.
In Cucumber, for example, you can have tags which then you can configure as a part of your test execution. If you use Jenkins, you might have something like this in your build fild:
clean install -Dcucumber.options="--tags #Google
or
clean install -Dcucumber.options="--tags #Bing
In Spring, you can have profiles that you can run like this as Jenkins job:
mvn clean test -Dspring.profiles.active="google"
EDIT
Alternatively, you can define a custom property in your pom like this:
<properties>
<myProperty>command line argument</myProperty>
</properties>
And then pass it from command line like this:
mvn install "-DmyProperty=google"
EDIT2
Providing a -D prefixed value in command line is a way of setting a system property. You can perform this action from Java code itself like this:
Properties props = System.getProperties();
props.setProperty("myPropety", "google");
or simply:
System.setProperty("myPropety", "google");
I am using JUnit's categories to split my tests into different categories and using maven to compile and run my test (surefire and failsafe).
Question is, how to I choose which category of tests are executed from command line?
something like: mvn clean install -DloadTests.
my failsafe plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${failsafe.plugin.version}</version>
<configuration>
<!--Exclude load tests by default-->
<excludedGroups>com.test.lib.categories.LoadTestCategory</excludedGroups>
</configuration>
</plugin>
You could do that with profiles and specify the plugin configuration in there.
<profiles>
<profile>
<id>noLoadTests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${failsafe.plugin.version}</version>
<configuration>
<!--Exclude load tests by default-->
<excludedGroups>com.test.lib.categories.LoadTestCategory</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
And then run maven
mvn test -PnoLoadTests
If you only ever need to exclude/include one specific category you could also define a property in the profile and use that in the . For more info you can look here
Edit: The other provided answer is the better one in this case, but profiles allow for various advanced configurations.
According to the documentation, user property is called groups. Therefore this should work:
mvn clean install -Dgroups=com.test.lib.categories.LoadTestCategory
I am using TestNG and Maven with surefire plugin to run tests.
I have:
#Test(groups={"groupA"})
TestA{}
#Test
TestB
I would like to have possibility to run:
mvn test
should invoke all test without any group
mvn test -Dgroups=groupA
should invoke only groupA tests ( this works by default but just added here to have it working with previous option )
I tried to configure surefire like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<excludedGroups>groupA</excludedGroups>
</configuration>
</plugin>
mvn test works as expected, but after
mvn test -Dgroups=groupA
no tests are executed
Edit
I found solution here:
https://labs.consol.de/blog/maven/citrus-and-testng-groups/
<!-- TestNG groups -->
<testGroups></testGroups>
<testGroupsExcluded>groupA</testGroupsExcluded>
<!-- /TestNG groups-->
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<groups>${testGroups}</groups>
<excludedGroups>${testGroupsExcluded}</excludedGroups>
</configuration>
</plugin>
...
<profiles>
<profile>
<id>a-testes</id>
<properties>
<testGroups>a</testGroups>
<testGroupsExcluded></testGroupsExcluded>
</properties>
</profile>
</profiles>
But there is one problem with this solution. It works fine when we want run just always one groups of tests, for example mvn test -P a-tests, but when we will add another group, let's say b-tests, then after mvn test -P a-tests, b-tests only one group will be executed because last defined profile will overide properties... Any ideas how to combine testGroupsExcluded, testGroups properties from multiple profiles?
Edit 2
I just ended with solution
<properties>
<testGroups>unit</testGroups>
</properties>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<groups>${testGroups}</groups>
</configuration>
</plugin>
But I had to assign all my tests explicitly to groups (all unassigned tests now are 'unit'), but now I can call:
mvn test To invoke all tests marked as unit
mvn test -DtestGroups=groupA, groupB To invoke any group tests...
Dude, have you checked http://testng.org/doc/documentation-main.html#beanshell ?
In surefire plugin attach testng suit config:
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
...
in testng.xml
<suite name="Emap test suite">
<test name="Emap tests">
<method-selectors>
<method-selector>
<script language="beanshell"><![CDATA[
addClassPath("target/test-classes" );
return com.yourpackage.shouldExecuteTest(groups, System.getProperty("groups"));
]]>
In static Java method shouldExecuteTest you can implement any rule you want!
Accoriding to doc, you can use those vars:
java.lang.reflect.Method method: the current test method.
org.testng.ITestNGMethod testngMethod: the description of the current test method
java.util.Map<String, String> groups: a map of the groups the current test method belongs to.
System.getProperty("groups") is just -Dgroups=xxx passed from mvn invocation.
Works like a charm!
I'm working on creating a reporting mechanism for tests and I want it to execute after all of the tests are run, and the resulting junit xml files are written out. Unfortunately, it doesn't appear that the executing continues after it fails in the test phase. Does anybody know what Maven configuration I could use to get the desired result if possible. Many Thanks.
May be this helps?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
We have our POM defining the maven-surefire-plugin as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14</version>
<configuration>
<reuseForks>false</reuseForks>
<forkCount>1</forkCount>
<argLine>-Xms64m -Xmx256m</argLine>
<includes>
<include>**/*Test.java</include>
</includes>
</configuration>
</plugin>
However, our Java tests (which involve some parallel tests and static singletons) only run properly when we run our test phase/build using:
mvn test -DforkMode=always
Strangely, even if we change our <configuration> to use (instead of the newer options):
<forkMode>always</forkMode>
And run:
mvn test
It will fail. But then if we run:
mvn test -DforkMode=always
It will pass. Instead of the newer options, it still will only work if we explicitly provide forkMode on the command line. We have tried this with multiple versions of the surefire plugin, to the same effect.
Are there any locations where this property could be overridden, or known issues in which the XML configuration is not properly used?
Rookie mistake. The configuration I was using was listed in a separate <profile> block that was not being executed. The profile with:
<activeByDefault>true</activeByDefault>
Did not include its own Surefire configuration at all (so it didn't show up in a search), and was using inherited values, which explains why the command-line system properties were able to affect its behavior.