Integration test executed by cucumber tends to leave behind context that causes problems with subsequent tests. Obvious solution appeared to be Spring's #DirtiesContext, but instead of tearing down the context after all the cucumber features have been run, it does this after each and every scenario, thus making the test execution time rather lengthy.
Tried also with #TestExecutionListeners, but no luck.
#RunWith( SpringJUnit4ClassRunner.class )
#ContextConfiguration( classes = { MyApplication.class, MyTestComponent.class }, loader = SpringApplicationContextLoader.class )
#ActiveProfiles( { "test", "someotherprofile" } )
#DirtiesContext( classMode = DirtiesContext.ClassMode.AFTER_CLASS )
#WebIntegrationTest( randomPort = true )
public class StepDefs extends StepDefUtils {
// givens, whens, thens
Am I trying to use DirtiesContext in an unsupported way?
As previous answer said the scenarios get compiled and run as separate classes stopping DirtiesContext from working and there are no per feature hooks in cucumber for same reason.
Workaround is to put tags in scenarios and have a class with hook detect these and conditionally dirty the context during the afterTestClass method.
The tag lets you control when context gets dirtied for example if want each feature to have fresh context then mark last scenario with tag, or can have many time per feature as and when needed.
public class CucumberFeatureDirtyContextTestExecutionListener extends AbstractTestExecutionListener{
private static boolean dirtyContext = false;
#After("#DirtyContextAfter")
public void afterDirtyContext(){
dirtyContext = true;
}
#Override public void afterTestClass(TestContext testContext) throws Exception {
if (dirtyContext) {
testContext.markApplicationContextDirty(HierarchyMode.EXHAUSTIVE);
testContext.setAttribute(DependencyInjectionTestExecutionListener.REINJECT_DEPENDENCIES_ATTRIBUTE, TRUE);
dirtyContext = false;
}
}
}
Mark scenarios with tag
#DirtyContextAfter
Scenario: My scenario
On steps class register the listener with spring
#TestExecutionListeners(listeners = {DependencyInjectionTestExecutionListener.class, CucumberFeatureDirtyContextTestExecutionListener.class})
Make sure the listener is in cucumber glue so after hook is registerd
Could not get it working on beforeClass as the context is already set up so have to do on afterClass.
Cucumber test methods are compiled into different test classes so #DirtiesContext( classMode = DirtiesContext.ClassMode.AFTER_CLASS ) will be run after each test method.
Unfortunately I don't see any DirtiesContext mode which suits your needs.
I would search for some cucumber listener and manually make spring context dirty trough it.
Related
A set of tests should be run on every microservice. Current solution is to have an abstract class and extend in every service, providing the necessary properties in abstract getters.
public abstract class AbstractTest {
#LocalServerPort
protected int serverPort;
protected abstract String getPath();
#Test
void someTest() {}
#Test
void conditionalTest() {}
}
#SpringBootTest(
webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT,
classes = {...})
#ActiveProfiles(...) // etc
public class MyTest extends AbstractTest {
// ... implement getPath()
// tests from parent will be executed
}
The goal:
Ditch inheritance and have the AbstractTest's logic get executed automatically with conditional #Test execution based on beans/properties etc.
The possible solution:
A concrete class with all the tests or some sort of Configuration/TestFactory to create the necessary tests. It should take into account available properties and beans to determine which tests to run.
The problem:
How can those tests (created in runtime) be discovered and registered for execution?
How to inject all the properties that are part of the current context of the #SpringBootTest?
Failed attempts:
TestInstanceFactory extension doesn't seem to be the solution as it requires an instance of the class which it annotates.
Using the Launcher API seems overkill, and also doesn't seem to work, since the library class won't be created with the Spring context configs.
using cglib and a base class Spring Contract-style is not a desirable solution
Ideally I don't want the client of this lib to implement/create anything, so abstract String getPath(); would be a test.lib.path property, and if it's present, a test from the library which uses it will run.
Any thoughts on this would be great, because right now this just seems impossible to me.
What is the reason to have the inheritance for tests?
In case you need to share some common logic within the tests you may try JUnit features (custom rules/extensions), for example
For junit < 5.x.x #Rule functionality https://junit.org/junit4/javadoc/4.12/org/junit/rules/TemporaryFolder.html https://stackoverflow.com/a/34608174/6916890
For junit >= 5.x.x (jupiter) there is an extension API
https://junit.org/junit5/docs/current/user-guide/#writing-tests-built-in-extensions-TempDirectory
I have been trying to set up a Junit 5 extension to force every test to get a separate ClassLoader. I am able to do it quite easily in Junit4, creating my own BlockJUnit4ClassRunner. But, I fail to have it work now.
The purpose is to be able to test things such as static blocks or memorized fields in different states.
I have been trying to use the TestInstanceFactory without any success so far with something like that:
public class SeparateClassLoaderExtension implements TestInstanceFactory {
#SneakyThrows
#Override
public Object createTestInstance(TestInstanceFactoryContext factoryContext, ExtensionContext extensionContext) throws TestInstantiationException {
ClassLoader testClassLoader = new TestClassLoader();
final Class<?> testClass = Class.forName(factoryContext.getTestClass().getName(), true, testClassLoader);
Constructor<?> defaultConstructor = testClass.getDeclaredConstructor();
defaultConstructor.setAccessible(true);
return defaultConstructor.newInstance();
}
}
I get an exception from Junit saying that the class is not of the right type.
Someone any idea?
JUnit Jupiter does not support this, yet. Here's the related issue: https://github.com/junit-team/junit5/issues/201
Does Spring or the JUnit runner have something that I can use to run code before running ANY tests?
The closest I've found is the #Rule and #ClassRule, which work on a class level.
I have a Docker #ClassRule which hosts an empty database for empty integration testing. With the #ClassRule, it restarts the container every time.
I'd prefer instead to just start it once when starting the tests (regardless if it's all tests or just a single one), runs the tests, then kill the container.
I've searched around, but I haven't found anything other than the class rule. Apologizes if I'm missing something obvious.
It appears that Spring and JUnit don't directly have anything to do this. After some more googling, I found a few bits that lead to some inspiration.
Making use of a custom rule extending ExternalResource (from JUnit), I'm kind of bastardizing it, but it does what I want:
public class MyRule extends ExternalResource {
static private MyRule instance;
static public MyRule get() {
if (instance == null) {
instance = new MyRule();
}
return instance;
}
private MyRule() {
// do init stuff
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
// do shutdown stuff
});
}
}
The basic idea is that the rule is a singleton. In each class that might need it, I'd put an #ClassRule:
public class MyTest {
#ClassRule
private MyRule myRule = MyRule.get();
}
It'll lazy-initialize itself, which will do all of the setup needed. It'll also register a shutdown hook, which will then handle any after stuff.
With this pattern, it'll run code exactly once before any tests (that need this rule) run, and it'll perform shutdown code only at the very end after all tests have finished.
Note: It purposely doesn't override the before() and after() functions, because those are before and after each class. You could add things there if you wanted to do something in between classes as well.
Maybe what you are looking for are those 2 annotations :
#BeforeClass
#Before
#Before runs before each test while #BeforeClass runs only once
You can use it like this :
#Before
public void setUp(){
// start container here
}
You also have equivalent for after test : #After #AfterClass
You will find a great explanation here
Thanks
i'm looking for a solution that allow me to handle the setup and the cleanup of my test environment at the launch and the end of my test framework execution.
The setup is not a problem but the cleanup imply to know when the test framework has finished to work or the index of the current test in execution queue.
Has someone a solution to implement this?
You can use org.spockframework.runtime.extension.IGlobalExtension to achieve this, as Spock extensions have callbacks for both BEFORE all specs start, and AFTER all specs end.
public interface IGlobalExtension {
void start();
void visitSpec(SpecInfo spec);
void stop();
}
So, implement stop() in your case to do whatever you need to do.
Spock finds extensions via Java's ServiceLoader, so make sure to add a META-INF/services file (pre-Java9) or declare it in your module-info.java file (post Java9), as explained here: http://spockframework.org/spock/docs/1.1/extensions.html#_writing_custom_extensions
One solution is to create an abstract class that all your specs extend:
class AbstractSpec extends Specification
then inside AbstractSpec find out the classes that are going to be run(for example if you're using spring framework):
private static final synchronized TEST_CLASSES = new ClassPathScanningCandidateComponentProvider(false).with {
addIncludeFilter(new AssignableTypeFilter(AbstractSpec.class))
findCandidateComponents("com/example/test").findAll { it.beanClassName != AbstractSpec.class.canonicalName }
.collect { Class.forName(it.beanClassName) }
}
then inside AbstractSpec do the actual clean up after all classes are run:
def synchronized cleanupSpec() {
TEST_CLASSES.remove(getClass())
if (TEST_CLASSES.isEmpty()) {
// clean up here
}
}
The problem with this solution is that if you run a subset of tests or classes in your IDE; you might have all the test classes in the classpath so TEST_CLASSES won't be emptied so the clean up won't execute.
I am trying to setup TestNG so that it gives me new instances of my class variable for each test (basically like JUnit). I need this as I intend to parallelize my tests at the method level. I have been experimenting with both standalone Guice and the built in Guice functionality that TestNG provides to try to accomplish this but I have had no luck. I know that I can use ThreadLocal, but calling .get() for every variable in the test is pretty unappealing. I am weary of using GuiceBerry as it does not really have a lot of updates/activity and it's last release is not even acquirable via Maven. I am pretty set on TestNG as for all the inconvenience this is causing me it still does a lot of great things. I am open to things other tools though to accomplish my goal. Basically I want things setup so the below tests would work consistently. Any help would be greatly appreciated.
// just has a variable thats a class called child with a simple string variable
// with a value of "original
Parent p;
#Test
public void sometest1(){
p.child.value = "Altered";
Assert.assertTrue(p.child.value.equals("Altered"));
}
#Test
public void sometest2(){
Assert.assertTrue(p.child.value.equals("original"));
}
TestNG doesn't create a new instance for each test. If you want such a behavior than I recommend creating separate test classes. e.g.:
public class SomeTest1 {
Parent p;
#Test
public void something(){
p.child.value = "Altered";
Assert.assertTrue(p.child.value.equals("Altered"));
}
}
public class SomeTest2 {
Parent p;
#Test
public void something(){
Assert.assertTrue(p.child.value.equals("original"));
}
}
Note that TestNG can run JUnit 3 and JUnit 4 tests (you might maintain a mixed suite depending on the style you want to use in a given test class).