I have a class with several tests split into two groups. I want a strict ordering to when the grouped tests are run, such that the tests in group A run first, then a setup method for group B is executed, and then group B runs. For example:
#Test(groups="A")
public void a1() {
// ...
}
#Test(groups="A")
public void a2() {
// ...
}
#BeforeGroups(value="B", dependsOnGroups="A")
public void setupB() {
// ...
}
#Test(groups="B")
public void b1() {
// ...
}
#Test(groups="B")
public void b2() {
// ...
}
The problem I'm running into is that TestNG doesn't seem to be honoring the setupB method. Instead of the expected execution order:
a1/a2
a2/a1
setupB
b1/b2
b2/b1
It executes something like this:
a1
setupB
b1
a2
b2
Any idea what I'm doing wrong with this setup? Am I missing something conceptually about how TestNG's groups work?
Try to specify dependsOnGroups for test methods as well.
public class TestClass {
#Test(groups="B")
public void b1() {
System.out.println("b1");
}
#Test(groups="B")
public void b2() {
System.out.println("b2");
}
#Test(groups="A", dependsOnGroups="B")
public void a1() {
System.out.println("a1");
}
#Test(groups="A", dependsOnGroups="B")
public void a2() {
System.out.println("a2");
}
#BeforeGroups(value="A", dependsOnGroups="B")
public void setupA() {
System.out.println("before");
}
}
I may be wrong about it, but seems that if a test method that belongs to a group has been picked for execution and it does not depend on any groups or methods, it just causes #BeforeGroups-annotated method to be run (ignoring dependsOnGroups specified there). Note that TestNG does not guarantee the execution order without some explicit declaration, e.g. using "depends" or "priority" mechanisms.
Hopefully, Cedric Beust will pay this question a visit.
This might be a workaround for now if it works. Not sure if you can use both annotations together.
#Test(groups="A")
public void a1() {
// ...
}
#Test(groups="A")
public void a2() {
// ...
}
#BeforeGroups(value="B")
#AfterGroups(value="A")
public void setupB() {
// ...
}
#Test(groups="B")
public void b1() {
// ...
}
#Test(groups="B")
public void b2() {
// ..
.
}
As mentioned earlier, when a test method is invoked, it just checks whether the corresponding #BeforeGroups annotated method has been executed for the test group the method belongs to, and if it hasn't, TestNG just invokes the #BeforeGroups method.
The workaround is to add a custom IMethodInterceptor which would add dependencies on the groups from #BeforeGroups#dependsOnGroups for the test methods, related to the same group:
public class TestMethodInterceptor implements IMethodInterceptor {
#Override
public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
if (!(context instanceof TestRunner)) {
return methods;
}
TestRunner testRunner = (TestRunner) context;
Collection<ITestClass> testClasses = testRunner.getTestClasses();
Map<String, List<String>> groupDependencies = MethodGroupsHelper.findGroupsMethods(testClasses, true).entrySet().stream()
.collect(Collectors.toMap(
Map.Entry::getKey,
groupBeforeMethods -> groupBeforeMethods.getValue().stream()
.flatMap(m -> Arrays.stream(m.getGroupsDependedUpon()))
.collect(Collectors.toList())
));
return methods.stream()
.map(IMethodInstance::getMethod)
.map(method -> {
Set<String> methodGroupDependencies = Stream.ofNullable(method.getGroups())
.flatMap(Arrays::stream)
.flatMap(group -> groupDependencies.getOrDefault(group, List.of()).stream())
.collect(Collectors.toSet());
return methodGroupDependencies.isEmpty() ? method : addGroupDependencies(method, methodGroupDependencies);
})
.map(MethodInstance::new)
.collect(Collectors.toList());
}
private ITestNGMethod addGroupDependencies(ITestNGMethod method, Collection<String> groups) {
String[] methodGroupDependencies;
if (method.getGroupsDependedUpon() == null || method.getGroupsDependedUpon().length == 0) {
methodGroupDependencies = groups.toArray(new String[0]);
} else {
methodGroupDependencies = Stream.concat(Arrays.stream(method.getGroupsDependedUpon()), groups.stream())
.distinct()
.toArray(String[]::new);
}
return new WrappedTestNGMethod(method) {
#Override
public String[] getGroupsDependedUpon() {
return methodGroupDependencies;
}
};
}
}
The TestMethodInterceptor is a listener that can be added to the execution by means of #Listeners annotation.
Related
I have a Play Controller that has a couple of methods that are very similar. I wondered how I can reduce boilerplate:
public static Result foo() {
// boilerplate
if (!PREVIEW) {
return redirect(routes.Application.login());
}
// other code
...
return ok("...");
}
public static Result bar() {
// boilerplate
if (!PREVIEW) {
return redirect(routes.Application.login());
}
// other code
...
return ok("...");
}
PREVIEW is shorthand for a configuration setting.
I created an Action like this:
public class PreviewAction extends Action.Simple {
public F.Promise<Result> call(Http.Context ctx) throws Throwable {
if (!PREVIEW) {
return F.Promise.pure(redirect(routes.Application.login()));
}
return delegate.call(ctx);
}
}
Now I can use an annotation on my other actions and it works like it did before:
#With(PreviewAction.class)
public static Result foo() {
...
}
More information: https://www.playframework.com/documentation/2.4.x/JavaAsync
Thanks to Tunaki for pointing me in the right direction.
I want only some subset of my test methods to run on a production environment. I annotate such test methods with #ProdAllowed annotation. I also wrote a small custom JUnit runner, which overrides the runChild method so it runs only for #ProdAllowed methods while on "PROD" environment:
public class ProdAwareRunner extends BlockJUnit4ClassRunner {
public ProdAwareRunner(Class<?> klass) throws InitializationError {
super(klass);
}
#Override
protected void runChild(FrameworkMethod method, RunNotifier notifier) {
ProdAllowed annotation = method.getAnnotation(ProdAllowed.class);
String env = CreditCheckSuite.getEnv().trim();
if (annotation != null || "DEV".equalsIgnoreCase(env) || "UAT".equalsIgnoreCase(env)) {
super.runChild(method, notifier);
} else {
notifier.fireTestIgnored(null); // this probably needs to be changed
}
}
}
This works quite well, but I want a little more - to have this skipped test methods to be marked in Eclipse as ignored (right now they are marked as not run, which is not exactly what I want)
You could write a rule by extending TestWatcher
public class DoNotRunOnProd extends TestWatcher {
protected void starting(Description description) { {
ProdAllowed annotation = description.getAnnotation(ProdAllowed.class);
String env = CreditCheckSuite.getEnv().trim();
if ((annotation == null) && !"DEV".equalsIgnoreCase(env) && !"UAT".equalsIgnoreCase(env)) {
throw new AssumptionViolatedException("Must not run on production.")
}
}
}
and add it to your test
public class Test {
#Rule
public final TestRule doNotRunOnProd = new DoNotRunOnProd();
...
}
This is already implemented (and actively used) in TestNG Groups:
public class Test1 {
#Test(groups = { "dev", "uat" })
public void testMethod1() {
}
#Test(groups = {"uat", "prod"} )
public void testMethod2() {
}
#Test(groups = { "prod" })
public void testMethod3() {
}
}
Is there a way to verify if a methodOne is called before methodTwo in Mockito?
public class ServiceClassA {
public void methodOne(){}
}
public class ServiceClassB {
public void methodTwo(){}
}
public class TestClass {
public void method(){
ServiceClassA serviceA = new ServiceClassA();
ServiceClassB serviceB = new ServiceClassB();
serviceA.methodOne();
serviceB.methodTwo();
}
}
InOrder helps you to do that.
ServiceClassA firstMock = mock(ServiceClassA.class);
ServiceClassB secondMock = mock(ServiceClassB.class);
Mockito.doNothing().when(firstMock).methodOne();
Mockito.doNothing().when(secondMock).methodTwo();
//create inOrder object passing any mocks that need to be verified in order
InOrder inOrder = inOrder(firstMock, secondMock);
//following will make sure that firstMock was called before secondMock
inOrder.verify(firstMock).methodOne();
inOrder.verify(secondMock).methodTwo();
Note that you can also use the InOrder class to verify that various methods are called in order on a single mock, not just on two or more mocks.
Suppose I have two classes Foo and Bar:
public class Foo {
public void first() {}
public void second() {}
}
public class Bar {
public void firstThenSecond(Foo foo) {
foo.first();
foo.second();
}
}
I can then add a test class to test that Bar's firstThenSecond() method actually calls first(), then second(), and not second(), then first(). See the following test code:
public class BarTest {
#Test
public void testFirstThenSecond() {
Bar bar = new Bar();
Foo mockFoo = Mockito.mock(Foo.class);
bar.firstThenSecond(mockFoo);
InOrder orderVerifier = Mockito.inOrder(mockFoo);
// These lines will PASS
orderVerifier.verify(mockFoo).first();
orderVerifier.verify(mockFoo).second();
// These lines will FAIL
// orderVerifier.verify(mockFoo).second();
// orderVerifier.verify(mockFoo).first();
}
}
Yes, this is described in the documentation. You have to use the InOrder class.
Example (assuming two mocks already created):
InOrder inOrder = inOrder(serviceAMock, serviceBMock);
inOrder.verify(serviceAMock).methodOne();
inOrder.verify(serviceBMock).methodTwo();
For Kotlin users, you can go this way:
class MyTrackerTest {
private val trackEventUseCase: TrackEventUseCase = mock()
private val sut = MyTracker(trackEventUseCase)
#Test
fun `trackSomething SHOULD invoke tracker use case twice with correct event names WHEN called`() {
sut.trackSomething()
trackEventUseCase.inOrder {
verify().invoke("Is it August?")
verify().invoke("No!")
}
}
}
With BDD it's
#Test
public void testOrderWithBDD() {
// Given
ServiceClassA firstMock = mock(ServiceClassA.class);
ServiceClassB secondMock = mock(ServiceClassB.class);
//create inOrder object passing any mocks that need to be verified in order
InOrder inOrder = inOrder(firstMock, secondMock);
willDoNothing().given(firstMock).methodOne();
willDoNothing().given(secondMock).methodTwo();
// When
firstMock.methodOne();
secondMock.methodTwo();
// Then
then(firstMock).should(inOrder).methodOne();
then(secondMock).should(inOrder).methodTwo();
}
I think I might have found a bug in JMockit, but I would like some to confirm whether it's a bug or there's something I'm missing.
I have the following (very simple) class:
public class Dummy {
public void foo() {System.out.println("O");}
}
Now I have the following tests, where in each of them I try to mock the method 'foo' more than once (each test does it a little differently):
Test #1
#Test
public void test1() {
new MockUp<Dummy>() {
#Mock
public void foo(Invocation inv) {
System.out.println("A");
inv.proceed();
}
}
new MockUp<Dummy>() {
#Mock
public void foo(Invocation inv) {
System.out.println("B");
inv.proceed();
}
}
new Dummy().foo();
}
Test #2
#Test
public void test2() {
mock("A");
mock("B");
new Dummy().foo();
}
private void mock(final String s) {
new MockUp<Dummy>() {
#Mock
public void foo(Invocation inv) {
System.out.println(s);
inv.proceed();
}
}
}
The only difference between the tests is the extraction of the mock code to a different method. But the results are not the same...
Test #1 output:
B
A
B
O
This is odd, because I wouldn't expect A to appear at all. But anyway, here's test #2 output:
B
A
A
A
...ad infinitum
Test #2 will fail with a StackOverflowError.
Is this a bug or am I missing something?
Update (with the solution)
As #Rogério mentioned, this behavior is not acceptable.
Then how can the mock be overridden? like this:
private MockUp<Dummy> mock;
#Test
public void test3() {
mockCorrectly("A");
mockCorrectly("B");
new Dummy().foo();
}
private void mockCorrectly(final String s) {
if (mock != null) {
mock.tearDown();
}
mock = new MockUp<Dummy> {
#Mock
public void foo(Invocation inv) {
System.out.println(s);
inv.proceed();
}
}
}
And for the output:
B
O
Great :)
It's not clear what exactly happens here; apparently, at runtime some "chained mocking" is occurring.
The real problem is that both tests are doing something invalid with the MockUp API: they are mocking the same method in the same class twice in the same test. It is ok to have two different mock-ups for the same class in the same test, as long as they mock different methods/constructors.
The resulting behavior is undefined, as JMockit does not support multiple simultaneous mockings of the same method.
In junit4 I want to execute specific test methods from different classes i.e want create a test suite with specific test methods from different classes.
Lets say I have 2 classes:
public class Test_Login {
#Test
public void test_Login_001(){
System.out.println("test_Login_001");
}
#Test
public void test_Login_002(){
System.out.println("test_Login_002");
}
#Test
public void test_Login_003(){
System.out.println("test_Login_003");
}
}
public class Logout {
#Test
public void test_Logout_001(){
System.out.println("test_Logout_001");
}
#Test
public void test_Logout_002(){
System.out.println("test_Logout_002");
}
#Test
public void test_Logout_003(){
System.out.println("test_Logout_003");
}
}
From the above classes I want to execute test methods test_Login_001 , test_Login_003 , test_Logout_002 only.
How this can be achieved in junit4 ?
Since JUnit 4.8 introduced Categories there exists a clean solution, create a TestSuite:
#RunWith(Categories.class)
#IncludeCategory(MustHaveTests.class)
#SuiteClasses( { Test_Login.class, Test_Logout.class })
public class MustHaveTestsTestSuite {
public interface MustHaveTests { /* category marker */ }
}
And add the #Category(MustHaveTests.class) above every test you would like to run with the TestSuite, e.g.:
#Category(MustHaveTests.class)
#Test
public void test_Login_001(){
System.out.println("test_Login_001");
}
When running the TestSuite only the MustHaveTests-"tagged" tests will be executed. More Details on #Category: https://github.com/junit-team/junit4/wiki/categories
You need to create an org.junit.runner.Request and pass it to the JunitCore runner, or actually to any Runner.
JUnitCore junitRunner = new JUnitCore();
Request request = Request.method(Logout.class, "test_Logout_002");
Result result = junitRunner.run(request);
I actually created an Annotation and can search for methods with those annotations and dynamically create Request and run them
public class TestsSuite {
public static void main(String[] args) throws Exception {
Class annotation = MyTestAnnotation.class;
JUnitCore junitRunner = new JUnitCore();
Class testClass = Test_Login.class;
Method[] methods = testClass.getMethods();
for (Method method : methods) {
if (method.isAnnotationPresent(annotation)) {
if (method.isAnnotationPresent(org.junit.Test.class)) {
Request request = Request.method(testClass, method.getName());
Result result = junitRunner.run(request);
System.out.println(result.wasSuccessful());
}
}
}
}
}
This might not be the slickest implementation, but I solved a similar problem by created a new #SuiteMethods annotation as follows:
SuiteMethods.java
#Retention(RUNTIME)
#Target(TYPE)
public #interface SuiteMethods {
String[] value() default {""};
}
FilteredSuite.java
public class FilteredSuite extends Categories {
private static String[] TEST_METHODS_TO_RUN = {""}; // default behavior is to run all methods
private static Class<?> extractMethodNamesFromAnnotation(Class<?> clazz) {
SuiteMethods methodsAnnotation = clazz.getAnnotation(SuiteMethods.class);
if (methodsAnnotation != null) {
// if our MethodsAnnotation was specified, use it's value as our methods filter
TEST_METHODS_TO_RUN = methodsAnnotation.value();
}
return clazz;
}
public static Filter getCustomFilter() {
Filter f = new Filter() {
#Override
public boolean shouldRun(Description desc) {
String methodName = desc.getMethodName();
for (String subString : TEST_METHODS_TO_RUN) {
if (methodName == null || methodName.contains(subString)) {
return true;
}
}
return false;
}
#Override
public String describe() {
return null;
}
};
return f;
}
public FilteredSuite(Class<?> arg0, RunnerBuilder arg1) throws InitializationError {
super(extractMethodNamesFromAnnotation(arg0), arg1);
}
#Override
public void filter(Filter arg0) throws NoTestsRemainException {
// At test suite startup, JUnit framework calls this method to install CategoryFilter.
// Throw away the given filter and install our own method name filter
super.filter(getCustomFilter());
}
}
A Usage Example
#RunWith(FilteredSuite.class)
#SuiteClasses({
GroupRestTest.class,
ScenarioRestTest.class
})
#SuiteMethods({
"testReadOnlyFlag",
"testSheetWriteData",
"testAddScenarioMeta"
})
public class SubsetTestSuite {
}