I want to run single test class from command line using Maven and TestNG
Things that doesn't work:
mvn -Dtest=ClassName test
I have defined groups in pom.xml, and this class isn't in one of those groups. So it got excluded on those grounds.
mvn -Dgroups=skipped-group test
mvn -Dsurefire.groups=skipped-group test
when config is
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7.1</version>
<configuration>
<groups>functest</groups>
</configuration>
</plugin>
Parameters work fine in there are no groups defined in pom.xml.
Similarly, when surefire is configured with
<configuration>
<includes>
<include>**/*UnitTest.java</include>
</includes>
</configuration>
I can add another test with -Dtest parameter, but cannot add group. In any combination, I can narrow down tests to be executed with groups, but not expand them.
What's wrong with my configuration? Is there a way to run a single test or group outside of those defined in pom.xml?
Tried on Ubuntu 10.04 with Maven 2.2.1, TestNG 5.14.6 and Surefire 2.7.1
I didn't test with TestNG 5.12.1 but I can say that running a single test using the test parameter and tests from groups using the groups parameter works with TestNG 5.14.2 (and surefire 2.6) (groups doesn't work in TestNG 5.14)
Here is the pom.xml I'm using:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.stackoverflow</groupId>
<artifactId>Q4159948</artifactId>
<version>1.0-SNAPSHOT</version>
<name>Q4159948</name>
<url>http://maven.apache.org</url>
<properties>
<maven.compiler.source>1.6</maven.compiler.source>
<maven.compiler.target>1.6</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>5.14.2</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.6</version>
<configuration/>
</plugin>
</plugins>
</build>
</project>
With a simple AppTest as follow:
import org.testng.annotations.*;
public class AppTest {
#BeforeClass
public void setUp() {
// code that will be invoked when this test is instantiated
}
#Test(groups = { "fast" })
public void aFastTest() {
System.out.println("Fast test");
}
#Test(groups = { "slow" })
public void aSlowTest() {
System.out.println("Slow test");
}
}
Both
$ mvn test -Dtest=AppTest
and
$ mvn test -Dgroups=slow
produce the expected result.
As I've explained in question, any mention of groups either in pom.xml or on command line resulted in reduction of executed tests count. Only way I've managed to avoid this is by using mavens profiles like this:
<profiles>
<profile>
<id>test-slow</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<groups>slow</groups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
and then running tests with
mvn -P test-slow test
I would suggest to try something like
mvn test -Dincludes=rs/magrathea/TestClassName
although I haven't tested this myself.
In order to run a single test you need the following from official documentation
mvn -Dtest=MyFirstTest test
or
mvn -Dtest=MyFirstTest,MySecondTest test
This is tested (and working) on maven 3.
Then you can avoid using the profiles. I had the same problem as I needed to run load test in isolation and using profiler in parallel to get the real figures.
Note: Not sure why but make sure that the switches come before the phase i.e. "-Dtest=MyFirstTest" before "test" otherwise it is not working (Mac OSX)
Related
My goal: using parameters to switch environments in my tests for instance:
mvn test google -> Tests goes to google site
mvn test bing -> bing site
"I need to feed my tests which environment is the target and it should comes from the pom.xml and using them as arguments."
It would be very useful for teamcity/jenkins integration as well.
Apart from that, I need to use url as variables in my tests. How can I do that?
Profiles can be a solution in pom.xml?
<profiles>
<profile>
<id>google</id>
<properties>
<base.url>http://www.google.com</base.url>
</properties>
</profile>
<profile>
<id>bing</id>
<properties>
<base.url>http://www.bing.com</base.url>
</properties>
</profile>
</profiles>
From build section:
<configuration>
<systemProperties>
<base.url>${base.url}</base.url>
</systemProperties>
</configuration>
But How can I use system properties and overall the approach is good? Thanks!
You can configure the maven-surefire-plugin to include only specific test classes and the run mvn test. By default, mvn will run all these:
"**/Test*.java" - includes all of its subdirectories and all Java filenames that start with "Test".
"**/*Test.java" - includes all of its subdirectories and all Java filenames that end with "Test".
"**/*Tests.java" - includes all of its subdirectories and all Java filenames that end with "Tests".
"**/*TestCase.java" - includes all of its subdirectories and all Java filenames that end with "TestCase".
but you could specify the tests you want to include like this:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<includes>
<include>Sample.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
or exclude:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<excludes>
<exclude>**/TestCircle.java</exclude>
<exclude>**/TestSquare.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
Having said that, this is probably not the best design and generally, you should be using some testing framework which you then can configure according to your needs. Few examples are (or combination of): jUnit, TestNG, Cucumber, Spring.
In Cucumber, for example, you can have tags which then you can configure as a part of your test execution. If you use Jenkins, you might have something like this in your build fild:
clean install -Dcucumber.options="--tags #Google
or
clean install -Dcucumber.options="--tags #Bing
In Spring, you can have profiles that you can run like this as Jenkins job:
mvn clean test -Dspring.profiles.active="google"
EDIT
Alternatively, you can define a custom property in your pom like this:
<properties>
<myProperty>command line argument</myProperty>
</properties>
And then pass it from command line like this:
mvn install "-DmyProperty=google"
EDIT2
Providing a -D prefixed value in command line is a way of setting a system property. You can perform this action from Java code itself like this:
Properties props = System.getProperties();
props.setProperty("myPropety", "google");
or simply:
System.setProperty("myPropety", "google");
I have just upgraded my solution to use JUnit5. Now trying to create tags for my tests that have two tags: #Fast and #Slow. To start off I have used the below maven entry to configure which test to run with my default build. This means that when I execute mvn test only my fast tests will execute. I assume I can override this using the command line. But I can not figure out what I would enter to run my slow tests....
I assumed something like.... mvn test -Dmaven.IncludeTags=fast,slow which does not work
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<properties>
<includeTags>fast</includeTags>
<excludeTags>slow</excludeTags>
</properties>
</configuration>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.0.0-M3</version>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.0.0-M3</version>
</dependency>
</dependencies>
</plugin>
You can use this way:
<properties>
<tests>fast</tests>
</properties>
<profiles>
<profile>
<id>allTests</id>
<properties>
<tests>fast,slow</tests>
</properties>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<groups>${tests}</groups>
</configuration>
</plugin>
</plugins>
</build>
This way you can start with mvn -PallTests test all tests (or even with mvn -Dtests=fast,slow test).
Using a profile is a possibility but it is not mandatory as groups and excludedGroups are user properties defined in the maven surefire plugin to respectively include and exclude any JUnit 5 tags (and it also works with JUnit 4 and TestNG test filtering mechanism).
So to execute tests tagged with slow or fast you can run :
mvn test -Dgroups=fast,slow
If you want to define the excluded and/or included tags in a Maven profile you don't need to declare a new property to convey them and to make the association of them in the maven surefire plugin. Just use groups and or excludedGroups defined and expected by the maven surefire plugin :
<profiles>
<profile>
<id>allTests</id>
<properties>
<groups>fast,slow</groups>
</properties>
</profile>
</profiles>
You can omit profile and just use properties, it is more elastic way.
<properties>
<tests>fast</tests>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<groups>${tests}</groups>
</configuration>
</plugin>
</plugins>
</build>
Then you can runt fast tests by typing mvn test, all test by typing mvn test -Dtests=fast | slow or only slow by typing mvn test -Dtests=slow. When you have more test tags then you can also run all of them except the selected type by typing mvn test -Dtests="! contract".
It's possible to specify a tag for a test case in SOAP UI like this:
I'm using the SOAP UI Maven plugin to execute a functional test suite in different environments and it would be useful to be able to exclude some of the test cases by specifying a tag in the invocation.
It looks like there is no configuration parameter for the Maven plugin to specify a tag (so that only a subset of the tests cross cutting different test suites can be executed):
https://www.soapui.org/test-automation/maven/maven-2-x.html
However it is possible to specify a tag when running via the GUI or command line:
http://readyapi.smartbear.com/features/automation/testrunner/cli
You can see from the above link that it is possible to specify tests which are tagged by using the -T switch.
Is this just a limitation of the Maven plugin?
Is it possible to simulate specifying a tag by reading an environmental variable during the Groovy startup script execution and disabling test cases which don't have the specified tag?
The Maven invocation is as follows:
mvn test -Dmyenv="dev" com.smartbear.soapui:soapui-pro-maven-plugin:5.2.1:test
The pom.xml looks like this:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>testing</groupId>
<artifactId>testing</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<prerequisites>
<maven>3.0.5</maven>
</prerequisites>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<pluginRepositories>
<pluginRepository>
<id>SmartBearPluginRepository</id>
<url>http://www.soapui.org/repository/maven2/</url>
</pluginRepository>
<pluginRepository>
<id>mvnPluginRepository</id>
<url>http://mirrors.ibiblio.org/pub/mirrors/maven/mule/dependencies/maven2/</url>
</pluginRepository>
<pluginRepository>
<id>codehausPluginRepository</id>
<url>https://nexus.codehaus.org/content/groups/snapshots-group/org/codehaus/mojo/</url>
</pluginRepository>
</pluginRepositories>
<build>
<plugins>
<plugin>
<groupId>com.smartbear.soapui</groupId>
<artifactId>soapui-pro-maven-plugin</artifactId>
<version>5.2.1</version>
<dependencies>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.6</version>
</dependency>
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.1-901-1.jdbc4</version>
</dependency>
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections-maven</artifactId>
<version>0.9.9-RC1</version>
</dependency>
</dependencies>
<configuration>
<projectFile>${basedir}/my-project</projectFile>
<outputFolder>${basedir}/surefire-reports</outputFolder>
<printReport>true</printReport>
<junitReport>true</junitReport>
<exportAll>true</exportAll>
<reportFormat>HTML</reportFormat>
<testFailIgnore>true</testFailIgnore>
<coverage>true</coverage>
<environment>${myenv}</environment>
</configuration>
<executions>
<execution>
<phase>test</phase>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.10</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
Please correct me if I am wrong about any of this or if there is a better way as I am rather new to using SOAP UI. If you give a significantly better answer, I will accept yours. I really hope that Smartbear will support a way of specifying tags in the Maven plugin.
I have established that it is not possible to specify a tag via the Maven plugin and that the server version of SOAP UI costs a large amount of money, so it's not possible in my situation to use the command line method of specifying a tag.
I thought about simulating test tags/categories by moving all of the tests that I want to include/exclude for an environment to a number of test suites. Unfortunately it appears to be only possible to run either only one test suite (by specifying one the the "testSuite" parameter) or all of the test suites (by leaving "testSuite" blank).
I thought I might be able to pull out the tag for a test case or suite using a Groovy script and use that to determine whether it should be run or not, but as far as I can tell, it's not possible to get the tag information for a test case or testSuite (object model API documentation).
I settled on marking the test suites to be excluded with a custom property (testingOnly) and disabling those for a particular environment (Dev) in the project level "Setup Script":
// When running tests for the Dev environment, skip test suites with the property testingOnly=true
def disableSuitesWithProperty(def propertyName) {
project.testSuiteList.each { testSuite ->
def isPropertyTrue = testSuite.getProperty(propertyName)?.getValue()?.toBoolean() ?: false;
if(isPropertyTrue) {
log.info "[Project Setup Script] Will Skip Test Suite: ${testSuite.name}";
testSuite.setDisabled(true);
}
else {
log.info "[Project Setup Script] Will Execute Test Suite: ${testSuite.name}";
testSuite.setDisabled(false);
}
}
}
if ("Dev".equals(project.getActiveEnvironmentName())) {
disableSuitesWithProperty("testingOnly");
}
The suites are reenabled in the project level "TearDown Script":
// Reenable all test suites after all tests have run
for (testSuite in project.testSuiteList) {
log.info "[Project TearDown Script] Reenabling Test Suite: ${testSuite.name}";
testSuite.setDisabled(false);
}
Are you looking to specify an environment variable to the Surefire plugin? If so, this should work:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Dmyenv=dev</argLine>
</configuration>
</plugin>
I am using TestNG and Maven with surefire plugin to run tests.
I have:
#Test(groups={"groupA"})
TestA{}
#Test
TestB
I would like to have possibility to run:
mvn test
should invoke all test without any group
mvn test -Dgroups=groupA
should invoke only groupA tests ( this works by default but just added here to have it working with previous option )
I tried to configure surefire like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<excludedGroups>groupA</excludedGroups>
</configuration>
</plugin>
mvn test works as expected, but after
mvn test -Dgroups=groupA
no tests are executed
Edit
I found solution here:
https://labs.consol.de/blog/maven/citrus-and-testng-groups/
<!-- TestNG groups -->
<testGroups></testGroups>
<testGroupsExcluded>groupA</testGroupsExcluded>
<!-- /TestNG groups-->
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<groups>${testGroups}</groups>
<excludedGroups>${testGroupsExcluded}</excludedGroups>
</configuration>
</plugin>
...
<profiles>
<profile>
<id>a-testes</id>
<properties>
<testGroups>a</testGroups>
<testGroupsExcluded></testGroupsExcluded>
</properties>
</profile>
</profiles>
But there is one problem with this solution. It works fine when we want run just always one groups of tests, for example mvn test -P a-tests, but when we will add another group, let's say b-tests, then after mvn test -P a-tests, b-tests only one group will be executed because last defined profile will overide properties... Any ideas how to combine testGroupsExcluded, testGroups properties from multiple profiles?
Edit 2
I just ended with solution
<properties>
<testGroups>unit</testGroups>
</properties>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<groups>${testGroups}</groups>
</configuration>
</plugin>
But I had to assign all my tests explicitly to groups (all unassigned tests now are 'unit'), but now I can call:
mvn test To invoke all tests marked as unit
mvn test -DtestGroups=groupA, groupB To invoke any group tests...
Dude, have you checked http://testng.org/doc/documentation-main.html#beanshell ?
In surefire plugin attach testng suit config:
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
...
in testng.xml
<suite name="Emap test suite">
<test name="Emap tests">
<method-selectors>
<method-selector>
<script language="beanshell"><![CDATA[
addClassPath("target/test-classes" );
return com.yourpackage.shouldExecuteTest(groups, System.getProperty("groups"));
]]>
In static Java method shouldExecuteTest you can implement any rule you want!
Accoriding to doc, you can use those vars:
java.lang.reflect.Method method: the current test method.
org.testng.ITestNGMethod testngMethod: the description of the current test method
java.util.Map<String, String> groups: a map of the groups the current test method belongs to.
System.getProperty("groups") is just -Dgroups=xxx passed from mvn invocation.
Works like a charm!
I have a maven project that I want to load properties from a file when things happen. I've got the codehaus properties-maven-plugin and I need it to run automagically.
If I run mvn test or mvn compile, the task plugin load the properties file just fine.
I see this in the output:
[INFO] --- properties-maven-plugin:1.0-alpha-2:read-project-properties (default)...---
When I run mvn flyway:init, say, I do not see it run and the properties do not get loaded.
Here is the pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>net.foo</groupId>
<artifactId>workflow</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Foo Framework</name>
<description>Foo framework</description>
<properties>
<postgre.version>9.1-901.jdbc4</postgre.version>
<flyway.version>2.2.1</flyway.version>
</properties>
<dependencies>
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>${postgre.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>com.googlecode.flyway</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>2.2.1</version>
<configuration>
<url>${url}</url>
<user>${user}</user>
<password>${pass}</password>
<locations>
<location>classpath:sql/pgsql</location>
</locations>
</configuration>
<dependencies>
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>${postgre.version}</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>src/main/resources/db.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Can anyone suggest what I can do, or some documentation I can read to make this work? I have read plenty of maven documentation and cannot seem to understand it well enough to make this happen. I thought what I have here would make it run during the initialize phase, but evidently not...
You are close. There is a difference between running Maven using a phase, and using a goal. Start with a quick review of the Maven lifecycle documentation.
When you run mvn compile, Maven runs all goals bound to the compile phase, plus anything bound to earlier phases. You have bound the properties:read-project-properties goal to the initialize phase. initialize is one of the first phases, so read-project-properties executes when compile, or test, or any of the later phases are used in the command.
When you run mvn flyway:init, you are only running a single goal, not the entire lifecycle. Thus, the only thing that runs with that command is the flyway-maven-plugin's init goal, nothing else.
If you want those two goals to run together, you could try binding the flyway:init goal to the initialize phase (if that is not the default; I could not tell from reading the plugin's documentation). Then, mvn initialize will run both.
If you don't want to bind flyway:init to a phase, then you may run both goals explicitly with mvn flyway:init initialize.