Two tests fail in my build service that will not fail when ran locally.
What I've discovered is that when I execute mvn clean test locally the sort order of my test classes are alphabetical (package and class), but when ran in the build service they're seemingly random.
It is not my intention of needing a specific "order" so I'm definitely concerned that one test prior to another is hinting at an isolation issue. However, I don't think I can just tell maven to run classes in a certain order.
What can I do to try to reproduce?
EDIT
I cloned my build plan, but pointed at a forked repo (same code) and it ran successfully with no failed tests...
You should try to execute mvn -Dsurefire.runOrder=random clean test for random order locally.
But even if your build fails locally, random order is not the best option to reproduce failures because of test order. If the tests are all green when run in alphabetical order, you might be able to make your build consistently fail with mvn -Dsurefire.runOrder=reversealphabetical clean test
There are also a couple of more options -- see runOrder documentation
Related
I am trying run single integration test. I have a a lot of *IT class and I want to run only one test. I try this :
mvn -Dit.test=XControllerIT verify
Am I doing wrong? Is there another alternative to this? Maven is being used.
There are two main options depending on your project setup:
Integration Tests are run with a dedicated Failsafe plugin
Integration Tests are run with a regular surefire plugin
If you have a failsafe plugin (and you actually should, its a recommended approach), then use the following snippet:
mvn -Dit.test=MySampleIntegrationTest failsafe:integration-test
If you're on surefire, then run:
mvn -Dtest=MySampleUnitTest surefire:test
In both cases there is a direct plugin goal execution, bypassing the lifecycle like in your initial example (with mvn verify)
In maven it is possible to run the lifecycle, see Default Lifecycle Documentation for more information
Basically, the lifecycle is comprised of phases with plugins bound to each phase
So when you run the mvn verify all the phases before verify will also run.
As a consequence, the code will be compiled (compile phase with a maven compile plugin automatically attached to it will do the job), tests will run (surefire plugin), and so on.
If you don't have a compiled source code and code of tests, you can't use the presented approach because you have to compile the code first.
However, if you already have everything compiled, it makes sense to run only the one test without recompilation of the code, and in this case, depending on the plugin you can use the suggested solution.
Especially it can make sense for local debugging or for CI in some cases multi-step build setup (can be seen in fairly complicated projects)
I have a plugin that is supposed to send files to a database regardless of test failures. I tried setting the fail-at-end setting but the plugin still didn't run. ignoring failures altogether doesn't seem to be an option for me in this case because then faulty jars would be deployed to the artifact repository. I can use this plugin for as a separate command line script but it would be optimal if it could be put in the pom.
The tests need to be run before the plugin does, because the plugin gets the test report files.
I am aware that there are similar questions to this that have been asked before, but they don't seem to help me so I was wondering if someone has similar advice.
If the plugin has to run after the tests, the tests just can't fail the build.
Maven Surefire plugin has an option: maven.test.failure.ignore to run the tests but keep execution
See Here a full description of Test MOJO.
This should be the first bet.
Another option is to build just like this, but run the plugin "externally" (a kind of two steps build). Run maven install or whatever you need, and then (separately)
mvn <group>:<artifact>:<version>:<mojo of your plugin>
try mvn your-goal -DskipTests - will ignore testing.
We're migrating some ant build scripts to gradle and are diagnosing issues along the way. One problem that has popped up is that on the CI server (jenkins running gradle) we occasionally get test failures. We think the issue is related to test execution order because one of the tests that is failing uses thread local storage in some library code.
I would like to be able to reproduce the problem locally before fixing the broken tests. However. I can't reproduce the problem locally because gradle always runs the tests in an order that happens to work.
So, is there a way to force gradle to run test class X before test class Y? The tests need to run in the same JVM - one test right after the other.
In case it matters, the tests are JUnit tests.
Yes, it is possible. One of the possibilities is to create additional task for test Y run
task YTest(type: Test) {
include '**/Y.*'
}
test {
exclude '**/Y.*'
}
test.dependsOn YTest
Per Peter's response on the gradle forum, this is not possible.
http://forums.gradle.org/gradle/topics/can_gradle_run_two_tests_in_a_specific_desired_order_in_the_same_jvm?utm_content=reply_link&utm_medium=email&utm_source=reply_notification#reply_13187620
An alternative is to use a test suite.
testNG (a Junit alternative, also supported by gradle) also allows you to have control on test order. See this post for more details.
I have maven project imported in my eclipse. Now I need to start making changes to it and test it with the integration test (out of App server). Currently, the integration test is run out of server using openEJB container.
My basic question is, what is the regular process to compile, build and test with Maven?
mvn install
Maven -> Update Project.
Run my test from command line
Is it how it is done? I am specifically interested in knowing mvn install commands.
So should I do all three steps before I can test it?
Example: I just wanted to print something and see what is the output. For this I guess I need to do all these steps?
The openEJB container needs classes so it can load them.
There is a wonderful Maven quick-reference sheet at http://maven.apache.org/guides/MavenQuickReferenceCard.pdf
First, you should be aware that unit tests and integration tests are separate and are run from separate plugins and at separate parts of the maven lifecycles. Unit tests are run with surefire and integration tests are run with failsafe.
You want to run integration tests and the failsafe documentation says:
NOTE: when running integration tests, you should invoke maven with the (shorter to type too)
mvn verify
rather than trying to invoke the integration-test phase directly...
This is the best way to run integration tests directly in maven. It will run all the preceding steps necessary (eg: compile) in order to run the integration tests. It won't waste time doing an install because install happens immediately after verify.
But if you're running the tests locally, it may be a better idea to run your integration tests directly in your IDE. That will give you a much faster feedback loop.
If it is Eclipse project the most reasonable thing is to do everything not from command line but from Eclipse. Assuming you have m2e plugin installed, go to your_project->run as->Maven test and run it.
You dont need neither install nor package phase to run Maven tests, package will create a jar which is not needed for tests, install will copy this jar to local repo which is also useless. When Maven run tests it uses compiled classes from target dir and ignores project's jar if even it exists.
Yes, mvn isntall is the most popular option. It compiles, packages and tests your project.
I have a project with nearly 500 individual tests in around 200 test classes. Some of these tests don't do a great job of tearing down their own state after they're finished, and in Eclipse this results in some tests failing. The tests all pass when running the test suite from the command line via Ant.
Can I enable 'test isolation' somehow in Eclipse? I don't mind if it takes longer to run.
Long term, I will clean up the misbehaving tests, but in the short term I'd like to get the tests working.
If you use Ant in Eclipse, you can set the JUnit task to fork a new JVM process for each test, providing isolation.
http://ant.apache.org/manual/Tasks/junit.html
I also had similar needs and developed small maven plugin Jute which has been published in maven central. It starts external JVM process for each JUnit test method.