Why Suite classes are not run but the other Tests - java

I have a set of integration tests which I need to run in specific order. So i created a BlahSuite.java inside the same package, and specified the order of classes there. And annotation as following
#RunWith(Suite.class)
#Suite.SuiteClasses({
And I added the plugin into pom as following
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.9</version>
<configuration>
<includes>
<include>**/*Suite.java</include>
</includes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
But still the tests are fired in different orders, feels like the Suite class is fully ignored. Any idea how to fix this ?

I found the answer at Stackoverflow question Run Junit Suite using Maven Command
So what my final setup is I just removed failsafe plugin and added following,
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/*Suite.class</include>
</includes>
</configuration>
</plugin>

Related

maven-surefire-plugin how to rerun failed TestNG tests? There is default rerun option for JUnit only

How can I run the failed TestNG tests with maven failsafe plugin?
I run my tests with 'mvn clean install'.
When I run the tests and I get failing tests, I need to rerun them.
I do not have a testng.xml file and I think that this is also an issue.
This my pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<configuration>
<parallel>methods</parallel>
<threadCount>${threads}</threadCount>
<systemProperties>
<browser>${browser}</browser>
<screenshotDirectory>${project.build.directory}/screenshots</screenshotDirectory>
<remoteDriver>${remote}</remoteDriver>
<gridURL>${seleniumGridURL}</gridURL>
<desiredPlatform>${platform}</desiredPlatform>
<desiredBrowserVersion>${browserVersion}</desiredBrowserVersion> <!--Set properties passed in by the driver binary downloader-->
<!--Set properties passed in by the driver binary downloader-->
<phantomjs.binary.path>${phantomjs.binary.path}</phantomjs.binary.path>
<webdriver.chrome.driver>${webdriver.chrome.driver}</webdriver.chrome.driver>
<webdriver.ie.driver>${webdriver.ie.driver}</webdriver.ie.driver>
<webdriver.opera.driver>${webdriver.opera.driver}</webdriver.opera.driver>
</systemProperties>
<includes>
<!-- All that is left now is to clean up the code in our basicTest class and change its name to BasicTestWD. You may have noticed that we added an <includes> configuration item to our POM. This is because Maven will use maven-surefireplugin to run files that have test at the start or end of their name. We don't want maven-surefire-plugin to pick up our tests; we want to use maven-failsafeplugin instead. -->
<include>**/*WD.java</include>
</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.lazerycode.selenium</groupId>
<artifactId>driver-binary-downloader-maven-plugin</artifactId>
<version>1.0.7</version>
<configuration>
<rootStandaloneServerDirectory>
${project.basedir}/src/test/resources/selenium_standalone_binaries
</rootStandaloneServerDirectory>
<downloadedZipFileDirectory>${project.basedir}/src/test/resources/selenium_standalone_zips
</downloadedZipFileDirectory>
<customRepositoryMap>${project.basedir}/src/test/resources/RepositoryMap.xml
</customRepositoryMap>
<overwriteFilesThatExist>${overwrite.binaries}</overwriteFilesThatExist>
</configuration>
<executions>
<execution>
<goals>
<goal>selenium</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

Sign all jars and wars maven project

How to sign all jars and wars that are generated in project when i clean and build with maven ? Is there such plugin and instruction how to use it. I found this http://maven.apache.org/plugins/maven-jarsigner-plugin/usage.html while searching the internet but there is no good instruction how to use or test it.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jarsigner-plugin</artifactId>
<version>1.2</version>
<executions>
<execution>
<id>sign</id>
<phase>install</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
<configuration>
<archiveDirectory>jars location</archiveDirectory>
<includes>
<include>**/*.jar</include>
</includes>
<keystore>/path/to/keystore.jks</keystore>
<alias>...</alias>
<storepass>...</storepass>
<keypass>...</keypass>
</configuration>
</plugin>
This code worked for me

How to run surefire tests in parallel while forcing some to run sequentially?

I'm working on maven project with thousands of tests that run very slowly (takes 2h to run all the tests). So I tried to run the tests in parallel by configuring surefire plugin as follow:
<configuration>
<failIfNoTests>false</failIfNoTests>
<reuseForks>false</reuseForks>
<reuseForks>true</reuseForks>
<forkCount>2C</forkCount>
<systemPropertyVariables>
<jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
</systemPropertyVariables>
<parallel>suites</parallel>
<threadCount>12</threadCount>
</configuration>
But then some of the tests fail in their #Before and #After methods where we initialize and clean up some resources (it seems to be related to ports conflicts).
I tried to add this annotation #net.jcip.annotations.NotThreadSafe to the failing tests as described in surefire documentation to run them sequentially and avoid conflicts. However this didn't work and these tests still fail!!!
Any pointer on how to force some surefire tests to run sequentially on same JVM process and same thread while the rest will run in parallel (potentially on different JVM processes)?
EDIT 1
Now, I tried to split the configuration of surefire into two: one for sequential tests and another one for parallel tests. However this approach does not seem to enhance the execution time as I still have tests that fail and it still takes 2h to run all tests.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.version}</version>
<!-- Sequential tests -->
<configuration>
<includes>**/*Sequential.java</includes>
<failIfNoTests>false</failIfNoTests>
<reuseForks>false</reuseForks>
<systemPropertyVariables>
<jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
</systemPropertyVariables>
</configuration>
<executions>
<!-- Parallel tests -->
<execution>
<id>parallel-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>**/*.java</includes>
<excludes>
<exclude>**/*Sequential.java</exclude>
</excludes>
<failIfNoTests>false</failIfNoTests>
<reuseForks>true</reuseForks>
<forkCount>2C</forkCount>
<parallel>suites</parallel>
<threadCount>12</threadCount>
</configuration>
</execution>
</executions>
</plugin>
I'd go with your "split the configuration of surefire into two" approach.
However your configuration seems to be faulty because you only have one execution defined.
Might want to try something like this (define 2 executions with their own configurations):
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.version}</version>
<executions>
<!-- Sequential tests -->
<execution>
<id>sequential-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>**/*Sequential.java</includes>
<failIfNoTests>false</failIfNoTests>
<reuseForks>false</reuseForks>
<systemPropertyVariables>
<jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
</systemPropertyVariables>
</configuration>
</execution>
<!-- Parallel tests -->
<execution>
<id>parallel-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>**/*.java</includes>
<excludes>
<exclude>**/*Sequential.java</exclude>
</excludes>
<failIfNoTests>false</failIfNoTests>
<reuseForks>true</reuseForks>
<forkCount>2C</forkCount>
<parallel>suites</parallel>
<threadCount>12</threadCount>
</configuration>
</execution>
</executions>
</plugin>
Here is my final maven configuration for running the tests as wanted:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.version}</version>
<!-- Sequential tests -->
<configuration>
<failIfNoTests>false</failIfNoTests>
<reuseForks>false</reuseForks>
<systemPropertyVariables>
<jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
</systemPropertyVariables>
</configuration>
<executions>
<!-- Default tests -->
<execution>
<id>default-test</id>
<configuration>
<skip>true</skip>
</configuration>
</execution>
<!-- Parallel tests -->
<execution>
<id>parallel-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>**/*.java</includes>
<excludes>
<exclude>**/*Sequential.java</exclude>
</excludes>
<failIfNoTests>false</failIfNoTests>
<reuseForks>true</reuseForks>
<forkCount>2C</forkCount>
<parallel>suites</parallel>
<threadCount>12</threadCount>
</configuration>
</execution>
<!-- Sequential tests -->
<execution>
<id>sequential-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>${skip.sequential.tests}</skip>
<includes>**/*Sequential.java</includes>
<reuseForks>false</reuseForks>
</configuration>
</execution>
</executions>
</plugin>

Instruct Jacoco and Coveralls to ignore some files from the report

I use Coveralls in a Java project with Maven to create a coverage report for my project and I want to avoid some classes from being included in the coverage computation.
Now, I think that I the build phase I can exlcude the class, e.g. MyClassTest.java as follows:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.5.201505241946</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
<configuration>
<excludes>
<exclude>**/*MyClassTest.java</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.eluder.coveralls</groupId>
<artifactId>coveralls-maven-plugin</artifactId>
<version>4.1.0</version>
</plugin>
Anyway, the coverage doesn't decrease, so I guess that the class MyClass is still computed from the Jacoco report.
In the travis file I call the report phase as follows:
- mvn clean test jacoco:report coveralls:report
Any idea?
My bad!
First the .java has to be removed and replace with .class as correctly suggested by Tunaki.
Second, I don't have to exclude the test class, but the class that I don't want to measure in the coverage!
So it is:
<excludes>
<exclude>**/*MyClass</exclude>
</excludes>
instead of
<excludes>
<exclude>**/*MyClassTest.java</exclude>
</excludes>

How to make PMD run at the start of the maven build rather than at the end of it?

I have the following configuration within my pom.xml that checks for PMD violations:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>${pmd.version}</version>
<configuration>
<linkXRef>true</linkXRef>
<sourceEncoding>UTF-8</sourceEncoding>
<minimumTokens>100</minimumTokens>
<targetJdk>1.7</targetJdk>
</configuration>
<executions>
<execution>
<goals>
<goal>check</goal>
<goal>cpd-check</goal>
</goals>
</execution>
</executions>
</plugin>
When I run a build using the command mvn clean install, the PMD checks are run as last step of the build process. Rather, I would want the PMD checks to run as the first step of the build.
Does anybody know how could I achieve this?
Add the phase element to your POM.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>${pmd.version}</version>
<configuration>
<linkXRef>true</linkXRef>
<sourceEncoding>UTF-8</sourceEncoding>
<minimumTokens>100</minimumTokens>
<targetJdk>1.7</targetJdk>
</configuration>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>check</goal>
<goal>cpd-check</goal>
</goals>
</execution>
</executions>
</plugin>
The validate phase is the first phase of the maven lifecycle: http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
Thanks for your answers #JamesB and #PetrMensik for letting me know about the phase element within the POM. It helped me solve my problem. I finally settled for this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>${pmd.version}</version>
<configuration>
<linkXRef>true</linkXRef>
<sourceEncoding>UTF-8</sourceEncoding>
<minimumTokens>100</minimumTokens>
<targetJdk>1.7</targetJdk>
</configuration>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>check</goal>
<goal>cpd-check</goal>
</goals>
</execution>
</executions>
</plugin>
I used the phase:compile, the reason being I have plenty of tests in my project which take up a lot of time to execute. And, its quite irritating to wait for those tests to finish and be notified about a PMD violation at the end of all the tests. I needed something just before the tests. Hence, I settled for compile.
Further suggestions are welcome. :)
You need to hook execution of this plugin to a different Maven lifecycle phase (validation comes as the first one in default lifecycle).
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>check</goal>
<goal>cpd-check</goal>
</goals>
</execution>
</executions>
See this the list of the available Maven phases for reference.
I set build configuration
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>${pmd.plugin.version}</version>
<configuration>
<failOnViolation>true</failOnViolation>
<printFailingErrors>true</printFailingErrors>
</configuration>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

Categories