We Keep Coding

EasyMock verifying calls to mock in tearDown method after verify finished

I am seeing inconsistent behaviour in EasyMock tests that I don't understand.
My first test passes..
public class MockATest {
private final AtomicLong aMock = createStrictMock(AtomicLong.class);
#Before
public void setUp() {
aMock.set(101L);
}
#After
public void tearDown() {
aMock.set(999L);
}
#Test
public void testA() {
reset(aMock);
replay(aMock);
// TODO : test stuff here
verify(aMock);
}
}
.. but my second test fails ...
public class MockBTest {
private final List<Long> bMock = createStrictMock(List.class);
#Before
public void setUp() {
bMock.add(101L);
}
#After
public void tearDown() {
bMock.add(999L);
}
#Test
public void testB() {
reset(bMock);
replay(bMock);
// TODO : test stuff here
verify(bMock);
}
}
The failure reason is
Unexpected method call List.add(999)
I have 2 questions really...
Why is the behaviour different for the 2 tests?
Why is the add(999L) that happens in the tearDown method is being verified after the verification in the testB method has already fully completed?
(I know I can make this work by adding another reset(bMock) in after the verify(bMock) but I am not sure whether this is just avoiding the issue)

Why is the behaviour different for the 2 tests?
Because AtomicLong.set is typed void AtomicLong.set(long) so it's a void method. The recording is fine. However, List.add is typed boolean List.add(E) so it's not a void method. The correct way to record a non-void method is to do expect(list.add(101L)).andReturn(true).
Why is the add(999L) that happens in the tearDown method is being verified after the verification in the testB method has already fully completed?
Because it never goes in testB(). EasyMock throws an error on the call to bMock.add(101L) in setUp() so it goes directly to the tearDown which fail as well and hides to exception from setUp().

Why do multiple JUnit TestWatchers not spawn multiple test method calls?

I'm using multiple JUnit TestWatchers on each of the tests in my test suite. I was worried that each TestWatcher was calling base.evaluate() and each test was actually being run multiple times (once by each call to base.evaluate()). That does not seem to be happening, which is great, but I'm confused as to why that is the case.
Why don't multiple JUnit TestWatchers in a single test result in multiple calls to the test method?
I think I have some fundamental misunderstanding of how these components (in particular base.evaluate()) interact, but haven't found any good explanations that have resolved this confusion on my part.
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
public class SimpleTest {
#Rule
public TestWatcher testWatcher1 = new TestWatcher() {
#Override
public Statement apply(Statement base, Description description) {
return new Statement() {
public void evaluate() throws Throwable {
try {
System.out.println("testWatcher1");
base.evaluate();
} catch (Throwable e) {
e.printStackTrace();
}
}
};
}
};
#Rule
public TestWatcher testWatcher2 = new TestWatcher() {
#Override
public Statement apply(Statement base, Description description) {
return new Statement() {
public void evaluate() throws Throwable {
try {
System.out.println("testWatcher2");
base.evaluate();
} catch (Throwable e) {
e.printStackTrace();
}
}
};
}
};
#Test
public void test() {
System.out.println("test");
}
}
Output:
testWatcher2
testWatcher1
test
Process finished with exit code 0
Upon further investigation:
Adding System.out.println(base.toString()); inside of each of those evaluate() calls lead to some really interesting output:
testWatcher2
com.glenpierce.Tests.base.SimpleTest$1$1#3c09711b
testWatcher1
org.junit.internal.runners.statements.InvokeMethod#5cc7c2a6
test
Process finished with exit code 0
It seems like each TestWatcher is looking at a different scope. Am I unintentionally nesting these things?

Rules in JUnit are designed so that they are linked to each other. base.evaluate() in one rule causes the next rule to run, until all the rules have run. Only if all the rules calls base.evaluate() then the test method is run.

Detect skipped test in #afterMethod with testng

I'm trying to detect skipped test in my #AfterMethod for reporting purpose.
I can catch failed or passed test but it doesn't seems to enter it when a test is skipped.
My tests :
#Test
public void method1 () {
Assert.fail("created failure");
}
#Test (dependsOnMethods = {"method1"})
public void method2 () {}
My AfterMethod :
#AfterMethod
protected void afterMethod(ITestResult result){
switch (result.getStatus()) {
case ITestResult.FAILURE:
...
break;
case ITestResult.SKIP:
...
break;
case ITestResult.SUCCESS:
...
break;
}
for example here i only retrieve the failure but the skip doesn't pass in the after method
Any idea on how to do it ?
Thank you !

I found a way of doing it by Implementing the ITestListener class you can find an example there :
https://github.com/khmarbaise/testng-example/blob/41861115eb0ea1d98eed97fcfeb7ff30e93e0925/src/test/java/com/soebes/testing/testng/IntegrationTestListener.java
basically what you do is creating a class that will catch all your ITestResult even the skipped one and redefine inside it what you want to do like that :
public class IntegrationTestListener implements ITestListener {
public void onTestSkipped(ITestResult result) {
// do your stuff here
}
}
and to use it in your test do it like that :
#Listeners ({IntegrationTestListener.class})
public class NotAScenario extends BasicScenario {
#Test
public void test1() {
}
}

For the #AfterMethod, when we know that the test has definitely run (thus not ITestResult.STARTED), there is a simple solution (tested and it works):
#AfterMethod(alwaysRun = true )
public void doStuff(ITestResult result){
Integer status = result.getStatus();
if(!status.equals(ITestResult.SUCCESS) || !status.equals(ITestResult.FAILURE) ) {
// do stuff
}
}
Much cleaner for some cases where you don't want to implement all of the methods in ITestListener

In testng, if a test case is skipped, then #AfterMethod would not be called, as the test case has not been executed. #AfterMethod will get executed only when the test case is executed.
Hope this helps.

kartik is correct, u can not get it through after method, but in report u will find the skipped test case.
but if u want to get the skipped test cases, u can add soft dependacies here like : #Test(alwaysRun = true, dependsOnMethods = { "method" })
This creates a soft dependency, i.e. the annotated test should be executed even if the tests it depends on failed/threw an exception and it will invoke in after method.

You can capture skip tests by doing the following:
public void afterMethod(ITestResult result)
throws Exception {
if (result.getStatus() == ITestResult.SKIP) {
//Some code
}

How do i control the order in which jUnit tests run [duplicate]

This question already has answers here:
How to run test methods in specific order in JUnit4?
(23 answers)
Closed 7 years ago.
I have this jUnit test class
public class TestRaavareBatch {
#Before
public void prep(){
try { new Connector(); }
catch (InstantiationException e) { e.printStackTrace(); }
catch (IllegalAccessException e) { e.printStackTrace(); }
catch (ClassNotFoundException e) { e.printStackTrace(); }
catch (SQLException e) { e.printStackTrace(); }
}
MySQLRaavareBatchDAO rvb = new MySQLRaavareBatchDAO();
#Test
public void testgetRaavareBatch() throws DALException{
RaavareBatchDTO rvbOBJ = rvb.getRaavareBatch(7);
assertEquals(7, rvbOBJ.getRaavareId());
assertEquals(100.0, rvbOBJ.getMaengde(),0.0);
assertEquals(7, rvbOBJ.getRbId());
}
#Test
public void testgetRaavareBatchList() throws DALException{
List<RaavareBatchDTO> rvbOBJ = rvb.getRaavareBatchList();
assertEquals(rvbOBJ.size(), 8);
}
#Test
public void testgetRaavareBatchListId() throws DALException{
List<RaavareBatchDTO> rvbOBJ = rvb.getRaavareBatchList(5);
assertEquals(rvbOBJ.size(), 2);
}
#Test
public void testcreateRaavareBatch() throws DALException{
RaavareBatchDTO test;
rvb.createRaavareBatch(test = new RaavareBatchDTO(8, 8, 200.0));
RaavareBatchDTO rvbOBJ = rvb.getRaavareBatch(8);
assertEquals(8, rvbOBJ.getRbId());
assertEquals(200.0, rvbOBJ.getMaengde(),0.0);
assertEquals(8, rvbOBJ.getRbId());
}
#Test
public void testupdateRaavareBatch() throws DALException{
RaavareBatchDTO test;
rvb.updateRaavareBatch(test = new RaavareBatchDTO(8, 7, 100.0));
RaavareBatchDTO rvbOBJ = rvb.getRaavareBatch(8);
assertEquals(7, rvbOBJ.getRaavareId());
assertEquals(100.0, rvbOBJ.getMaengde(),0.0);
}
}
It connects to a database with 7 rows, and after i run the last test "updateRaavareBatch" i have created a new row so the size of the list in testgetRaavareBatchList() will be 8. But it gives me an error because it counts the size before i create a new row..
How can i run testgetRaavareBatchList() after i create the new row and update it.

I once got something like that in testing queries, insertions and deletions in a database.
I ended with the following infra in order to ensure test independance :
prepare the database connection in a #Before method
rollback in #After
put some inserts in private not #Test annotated utility methods to avoid duplication
when needed the #Test methods called utility methods, and did their job with assertions
In another harder case, I created an embedded database in a #BeforeClass method and destroyed it in #AfterClass
But you should never rely on test order.

You can use #FixedMethodOrder annotation on your test class.
A simple example is the following:
import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;
//Running test cases in order of method names in ascending order
#FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class OrderedTestCasesExecution {
#Test
public void secondTest() {
System.out.println("Executing second test");
}
#Test
public void firstTest() {
System.out.println("Executing first test");
}
#Test
public void thirdTest() {
System.out.println("Executing third test");
}
}
Output:
Executing first test
Executing second test
Executing third test
Just one thing about your particular test scenario though. It is better in your case to have a proper #Before and #After methods to setup and rollback database tests. Later on, if your codebase is big enough you might run into cases where one test does not clean up properly and makes another random testcase fail.
References:
Simple TestCase source
JUnit Javadoc for the #FixMethodOrder
Another decent page on JUnit

How to run tearDown type method for a specific test in JUnit class with multiple tests?

I have a junit testCase class with multiple test methods in it ( As requirement , we don't want to create separate class for each test.)
I wanna create a tearDown type method for EACH test method , which will run specifically for that test. Not for ALL test.
My problem is , in many tests i Insert record in database, test it and delete it after test.
But, If a test fails mid way , control don't reaches till end my dummy record ain't deleting.
I think only ONE tearDown() is allowed for one class, and this tearDown() don't know what object/record i created or inserted and what to delete!!!
I want to create a tearDown() or #After method just for one specific test. Something like finally{} in java for each method.
For Eg:
public class TestDummy extends TestCase {
public void testSample1(){
InsertSomeData1();
assertFalse(true);
runTearDown1();
}
public void testSample2(){
InsertSomeData2();
assertFalse(true);
runTearDown2();
}
public void runTearDown1(){
deleteDummyDatafromTestSample1....
}
public void runTearDown2(){
deleteDummyDatafromTestSample2....
}
}
Here control will never go to runTearDown1() or runTearDown2() and I don't a one common tearDown() because it won't know what data I inserted and thats specific to each method.

It seems your test relies on a fixed database, and future tests will break if your current test breaks. What I'd recommend is not to focus on this particular problem (a test-specific tearDown method that runs for each test), but your main problem - borken tests. Before your test run, it should always work with a clean database, and this should be the case for each test. Right now, your first test has a relationship with the second (through the database).
What the right approach would be is that you recreate your database before each test, or at the very least reset it to a basic state. In this case, you'll want a test like this:
public class TestDummy {
// this code runs (once) when this test class is run.
#BeforeClass
public void setupDatabase() {
// code that creates the database schema
}
// this code runs after all tests in this class are run.
#AfterClass
public void teardownDatabase() {
// code that deletes your database, leaving no trace whatsoever.
}
// This code runs before each test case. Use it to, for example, purge the
// database and fill it with default data.
#Before
public void before() {
}
// You can use this method to delete all test data inserted by a test method too.
#After
public void after() {
}
// now for the tests themselves, we should be able to assume the database will
// always be in the correct state, independent from the previous or next test cases.
#Test
public void TestSample2() {
insertSomeData();
assertTrue(someData, isValid());
}
}
Disclaimer: JUnit 4 tests (using annotations), might not be the right annotations, might not even be the right answer(s).

You could have smth like this:
interface DBTest {
void setUpDB();
void test();
void tearDownDB();
}
class DBTestRunner {
void runTest(DBTest test) throws Exception {
test.setUpDB();
try {
test.test();
} finally {
test.tearDownDB();
}
}
}
public void test48() throws Exception {
new DBTestRunner().runTest(new DBTest() {
public void setUpDB() {...}
public void test() {...}
public void tearDownDB() {...}
});
}

#iluxa . Gr8.. Your solution is perfect!!! In one test class i created two tests test48 and test49 (same as required in my code above testSample1 and testSample2) and viola! every test method now gets its own setup() and tearDown. Only this solution looks little complicated as need to use DBTestRunner in each method, but I don't see any better solution. I was thinking Junit may have some direct solution. like #After or tearDown() with some parameter or something.
Tks a lot.

Use MethodRule:
public class MyRule implements MethodRule {
#Override
public Statement apply(final Statement base, FrameworkMethod method, Object target) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (AssertionError e) {
doFail();
} finally {
doAnyway();
}
}
};
}
}
Then declare it in your test class:
public class TestDummy{
public MethodRule rule = new MyRule();
......
}