i have spring project that consists of a parent and few child projects.
the build is maven-based and the continous integration server is hudson.
i need to choose the way the unit tests will run.
currently, there are several good and also quite few garbage junit tests.
i preffer not to mess with the test packages of the child projects since it would be time-consuming, i am not familiar with all the junits and the last but not least: i'm lazy.
my questions are:
should i use maven-surefire-plugin and do a heavy cleanup in the test package?
is it any way to tell hudson (not in pom.xml of the project being built) to run specific unit tests and ignore others?
should I create some other build - (ant?) and use it for running unit tests on hudson?
are there any other options good options in the market i am not aware of?
any piece of advice would be appreciated.
aviad
should i use maven-surefire-plugin
and do a heavy cleanup in the test
package?
is it any way to tell hudson (not in
pom.xml of the project being built)
to run specific unit tests and
ignore others?
If you just want to run unit tests in a single-module project, you can do
mvn test
That will run all maven lifecycle phases up to test, including compile and test-compile (as you can see in built-in lifecycle bindings, the surefire:test goal is executed in phase test). Now if you want to restrict the unit tests that are executed, you can configure the surefire:test plugin execution with the test parameter:
mvn test -Dtest=Funky*Test
(This will execute all test classes that start with Funky and end with Test)
Unfortunately, the command line configuration is limited, if you want multiple includes and excludes you will have to do some XML configuration. I'd suggest to use a dedicated profile for hudson:
<profiles>
<profile>
<id>hudson</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.6</version>
<configuration>
<includes>
<include>**/TestA.java</include>
<include>**/Funky*Test.java</include>
<include>**/Test**.java</include>
</includes>
<excludes>
<!-- overrides above include -->
<exclude>**/TestButNotMe.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Now you can let your hudson call maven like this:
mvn -Phudson test
1) should i use maven-surefire-plugin and do a heavy cleanup in the test package?
yes
2) is it any way to tell hudson (not in pom.xml of the project being built) to run specific unit tests and ignore others?
I don't know any if you use JUnit. But you could check the "hidden" features of version 4.7, 4.6, 4.5... (there are many, you can found it in the release notes). If there is nothing what you need, then you could program it by yourselfe. (checking for a system property and skip the test).
3) should I create some other build - (ant?) and use it for running unit tests on hudson?
This would be the same like sprecifiying it in the pom. But in my personal opining mixing Ant and Maven is one of the uggliest thing you can do.
4) are there any other options good options in the market i am not aware of?
Have a look at TestNG.
Related
I am using TestNG, Cucumber, but if there is no solution for TestNG, then JUnit is OK as well.
I know how to run tests in different forks using cucumber-jvm-parallel-plugin. This plugin will generate a bunch of runners for each feature file, and then maven-surefire-plugin will run runners in different forks if configured appropriately.
But cucumber-jvm-parallel-plugin github page says I should not use it anymore. So that means I can not generate runners for each feature file and I have only one runner that includes a bunch of feature files.
But then I can not find how to run those feature files in different forks. There is a possibility to run in different threads, but I was interested in forks...
So is it possible, and if yes, then how?
Thanks.
I am sure you must have found some way to create more than one runners and run it in forks.
But here is what we have used in our project if the question is still relevant.
Cucumber + Junit
Plugin : <groupId>com.trivago.rta</groupId>
<artifactId>cucable-plugin</artifactId>
<phase>generate-test-resources</phase>
In the configuration tag,
<generatedRunnerDirectory>${project.build.directory}/parallel/runners</generatedRunnerDirectory>
Plugin : <groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<phase>generate-test-sources</phase>
<goal>add-test-source</goal>
<source>${project.build.directory}/parallel/runners</source>
plugin : <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<goal>integration-test</goal>
<forkCount>${fork.count}</forkCount>
<reuseForks>true</reuseForks>
Let me know if this solves your problem.
I am relatively new to Maven. I have done a lot of research and digging on this topic, but I can't seem to find an answer, so I thought I would ask here.
Goal: I would like to run mvn clean install test while skipping integration tests, as well as one particular unit test class.
I have tried the following:
mvn clean install -DskipITs -Dtest=!MyTestClass test
mvn clean install -DskipITs&&test=!MyTestClass test
mvn clean install -DskipITs&test=!MyTestClass test
However, none of the above commands seem to work. The first command of the three above made the most sense to me, but it seems as though the integration tests are being run when using that command. This is where my knowledge and understanding of Maven has a gap; I'm not sure if that's the expected behavior, or if that is the appropriate way to pass multiple properties on the command line?
When I run this command: mvn clean install -DskipITs test, the integration tests are successfully skipped.
I am familiar with the Maven build life-cycle, but it is possible that I am misunderstanding something or missing a detail.
Integration tests with maven are normally run with maven-failsafe-plugin
To tell this plugin to skip integration tests (make sure your integration test class names follow the convention *IT.java, otherwise you need to include them with <inclusions>), you can do that in the plugin's configuration, or from the command line (official doc):
mvn test -DskipITs
Single tests can be skipped with:
mvn test -Dtest=!MyTestClass
So this should work:
mvn clean install -DskipITs -Dtest=!MyTestClass
What worked for me was the following command:
mvn clean install -DskipITs "-Dtest=!MyTestClass, !**/*IT.java" test
I am still learning Java, but here is what I think happened in my case.
There are two plugins pertaining to testing in Java (there are probably many more, but these two were relevant to my issue): one is called "maven-failsafe-plugin", while the other is "maven-surefire-plugin". As #hovanessyan and others have pointed out, maven-failsafe-plugin typically runs integration tests, while maven-surefire-plugin typically runs unit tests (Maven docs reference).
In my case, when I would run the command mvn clean install -DskipITs -Dtest=!MyTestClass test, upon further digging in the logs, the integration tests would fail and I would receive the following additional buried error message:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test)
The integration tests for the project are found in a directory called integrationtests, and are named according to the convention "MyIntegrationTestIT.java". What eventually led me down the right track was this: Surefire docs. These docs describe the "test" argument that you can pass with a Maven command.
It seems to me that when I passed the argument -Dtest=!MyTestClass, it's as if that instructed the Surefire plugin to "don't run MyTestClass, but do run every other test file." Meanwhile, the -DskipITs argument instructed the Failsafe plugin to skip integration tests (which it had been doing all along). When I explicitly called out the test files that I didn't want to run, in the form "-Dtest=!MyTestClass, !**/*IT.java", Surefire understood exactly what I wanted to do. The Surefire plugin ran every test with the exception of MyTestClass and the integration tests, and the Failsafe plugin skipped the integration tests.
I don't fully understand why, in my case, the Surefire plugin was running the integration tests in the first place. Maybe it has to do with some config setting in the codebase I'm working with, or the naming convention of the integration test files, or some annotation (I'm still learning a lot about these things). I am sure that this answer could be edited to include even more helpful information or context that I don't yet have. In any case, hopefully these learnings are helpful for some other folks experiencing this issue.
Finally, it helped a lot when debugging to run the command mvn help:effective-pom and pass -X along with my mvn clean install test command.
This configuration works for me:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>%regex[.*TestIT.*.class]</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<includes>
<include>%regex[.*TestIT.*.class]</include>
</includes>
</configuration>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
How can I run unit tests for spring boot application while building and deploying using spring boot:run command.
My expectation is to have all my unit tests executed before running application, but I dont want to make another maven command like mvn test before.
My problem:
I made a simple spring boot application and I could'd find a way to run unit tests while running application from intellij or from command line. Firstly I thought that maybe I have wrong configuration or wrong names of test classess or maybe wrong project structure. So I created spring boot application from intellij template. To my happiness it had already default test written so I simply run application. Unfortunatelly test was not executed.
This is a screenshot of project structure, pom.xml, main class and unit test created by intellij.Project created by intetelij
I changed the test runner and test to fail and tried again. Same result.
unit test changed to fail
I googled what is hidden underneath spring boot:run command here
http://docs.spring.io/spring-boot/docs/current/maven-plugin/run-mojo.html
I found something interesting at the top of manual: "Invokes the execution of the lifecycle phase test-compile prior to executing itself." So my understanding is that this command only compiles tests but not run them? If So, the question is - Is it possible to add "test" phase by adding some flag to the command?
Your problem here is to do with the maven lifecycle. According to the docs for the spring-boot:run, it binds to the lifecyle phase validate by default, and invokes the phase test-compile before executing.
What you're asking for is to execute the tests before running the application. You could do this with a custom maven profile in your POM - something like the following.
<project>
<profiles>
<profile>
<id>test-then-run</id>
<build>
<defaultGoal>verify</defaultGoal>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<id>spring-boot-run</id>
<phase>verify</phase>
<goals>
<goal>run</goal>
</goals>
<inherited>false</inherited>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
...
</profiles>
...
</project>
With this in your POM, you could then run the tests and start the app with:
mvn -P test-then-run
This binds the run goal to the verify phase instead of the validate phase, which means that the tests will be run first. You can see which order the phases are run here: https://maven.apache.org/ref/3.3.9/maven-core/lifecycles.html
I knew that I could use mvn test before running application. In fact it can be achieved by many ways in intellij. For me, I just add mvn goal test before launching application:
spring boot configuration in intellij.
But of course it may be done differently.
The thing is, I was just very curious whether I could somehow manipulate spring boot:run command by using some flag or sth else to get same results but its impossible.
olambert, thank You for Your answer too, it works very well.
In my project, we have both TestNG and JUnit (actually Spock) tests. So we need to run both, and get results from both. With surefire providers it seemd quite easy, so we did this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<threadCount>1</threadCount>
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-testng</artifactId>
<version>2.17</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.17</version>
</dependency>
</dependencies>
</plugin>
However, after some time I found out that junit tests are run twice. This is not good, because we have a lot of tests, and I'd like to limit the time they are running.
After a little research I found out that surefire-testng provider runs both TestNG and JUnit tests. So again it seemed simple - I removed surefire-junit47 provider.
Well, this was not a good solution either, because it turned out testng provider runs junit tests, but doesn't provide results in target/surefire-reports/TEST*.xml. I found only TestNG results there.
Is there any solution to have tests run only once, and results reported?
I use surefire-testng to run junit tests, but don't have any testng tests in the same run.
I extract the reports using:
**/surefire-reports/*.xml
You can configure surefire-testng to include or exclude any tests, perhaps you can exclude the junit tests by pattern, then have the junit47 provider run those?
surefire-reports can also be configured to collect junit and/or testng reports.
Later junit tests use annotations rather than naming conventions. testng seems to recognise this, but default behaviour of surefire-reports seems to be only collect data by naming convention.
I still use naming conventions as well as annotations in my junit tests (Test*/Test for unit and IT/*IT for integration), which allows me to control surefire and failsafe better.
Either utilise naming conventions to include tests in surefire plugins default behaviour, or configure the plugins to run specific tests and collect the data you want.
I have tests which run very slow, say a test which exports a large database to test the exporting code for memory leaks. I'd like to have these tests in my usual test suite but they should be ignored unless one of these conditions is fulfilled:
The test is running on a CI server
The user has selected this test in the IDE and runs it
The solution for #1 is simple: Use a property and skip the test if it's not set.
But how could I implement #2?
[EDIT] This is Java specific. I'm using JUnit 4 with Eclipse.
I have tests which run very slow, say a test which exports a large database to test the exporting code for memory leaks.
While useful, these are not unit tests, as denoted by the unit-test tag.
A test is not a unit test if:
It talks to the database
It communicates across the network
It touches the file system
It can't run at the same time as any of your other unit tests
You have to do special things to your environment (such as editing config files) to run it.
Tests that do these things aren't bad and are certainly worth writing. They can even be written in using unit test framework. However, it is important to be able to separate them from true unit tests so that you can keep a set of tests that you can run fast whenever you make changes, particularly in a TDD cycle where you want to go from failing test to passing test as quickly as possible.
If you are using a Makefile you could have a check target for true unit-tests and a livecheck target for these slow, system tests.
How to run each test individually will depend on your framework.
We use Maven for that, which integrates nicely with Eclipse. We use JUnit for
unit tests, which are prefixed or suffixed simply with Test and are run on every full build
integration tests, which are prefixed IntegrationTest. Those are only run when a special Maven profile is activated via a command line switch (-PintegrationTests).
The complete set of integration tests is run three times a day on the CI server or by a developer who explicitly specifies the above-mentioned profile (mvn test -PintegrationTests). A single integration test can of course be run by programmers in their IDE at any time.
This is the relevant part from pom.xml:
<profiles>
<profile>
<id>integrationTests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/Test*.java</include>
<include>**/*Test.java</include>
<include>**/IntegrationTest*.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>