We Keep Coding

bypassing #Test dependsOnMethods due to failure

I have a question regarding the #Test(dependOnMethods{""})..
I want my tests to run in a particular order so I hit every test that I have written. The best way to do this so far, atleast that I have found, is the dependsOnMethods! However, since the tests to come after requires the one before it to pass, I cant run all of my tests and see which ones failed. My program just exits! So here is what I'm working with..
#Test(dependsOnMethods = {"shouldSelectAmountOfDoors"})
public void shouldSelectExtColor() throws InterruptedException{
sycOptionalInfoPage.selectExtColor("GREEN");
}
#Test(dependsOnMethods = {"shouldSelectExtColor"})
public void shouldSelectIntColor() throws InterruptedException{
sycOptionalInfoPage.selectIntColor("GOLD");
}
#Test(dependsOnMethods = {"shouldSelectIntColor"})
public void shouldEnterAComment() throws InterruptedException{
sycOptionalInfoPage.enterComments("<(*-<) <(*-*)> (>-*)> woot!");
takeABreakYo();
}
Boom. Very easy to understand and trusty POM! But, if my shouldSelectIntColor() fails due to a changed id from the dev team, I want the shouldEnterAComment to still run! How can I continue to keep my tests chained in a line, but still run after a failure? Thanks :)

You can use priority instead of dependsOnMethods to achieve what you want:
#Test(priority = 1)
public void shouldSelectIntColor() throws InterruptedException{
}
#Test(priority = 2)
public void shouldEnterAComment() throws InterruptedException{
}
Here if your shouldSelectIntColor method fails, it will still execute shouldEnterAComment test method.

You can use priority as mentioned or you can use run-always=true. Its called soft dependancy. This way your methods will still run even if the method before failed.

How to use attributes in JUnit Tests?

I have been working with JUnit for several years and I have found many examples where isolation within the test was not fullfil.
Most of the tests I develop follow the same structure
class ClassToTestTest {
// Attributes
private ClassToTest objectToTest;
// Methods
#Before
public void setup() {
objectToTest = new ClassToTest();
}
#Test
public void test1() {
//do something
Assert...
}
#Test
public void test2() {
//do something
Assert...
}
}
The internal state of the test object (not the ClassToTest object but the object which perform the test) are in its attributes. If there is any flaw at the setup() or teardown() methods some internal state of objectToTest might sneak to other test.
Just wondering, would it be better to have no shared state? you have no attributes
class ClassToTestTest {
// Attributes
// No attributes, no internal state
// Methods
#Before
public void setup() {
objectToTest = new ClassToTest();
}
#Test
public void test1() {
localObjectToTest = createObjectToTest();
//do something
Assert...
}
#Test
public void test2() {
localObjectToTest = createObjectToTest();
//do something
Assert...
}
}
I know the first code do pretty much the same than the second, but in the first code you are tempted to do something like
// Methods
#Before
public void setup() {
objectToTest = objectToTest.reset();
}
or even worse, rely on the previous test in order to use the state of the objectToTest in the previous test to "save" time and you end up with an empty setup() method
Following the second code is much dificult to reach this point because there is no shared state, every object is local.
What are your thoughts? Make sense? Is it worthy?

JUnit creates a new ClassToTestTest object for each of your test methods. Therefore no field can sneak to another test. For details: http://martinfowler.com/bliki/JunitNewInstance.html

In my opinion I think it is worthy to create a ClassToTestTest object in each test because it is possible to modify the internal state and each test should be independent of the other ones.
A fussier developer could tell you that you are also testing the Constructor of the object in each test. Then, you can also use the #Before and #After annotations in the methods for creating and destroying the ClassToTestTest object.
But anyway, I would rather go to create the object in each test.
Edit: Adding bold font to put the text more readable

Mockito - how to verify that a mock was never invoked

I'm looking for a way to verify with Mockito, that there wasn't any interaction with a given mock during a test. It's easy to achieve that for a given method with verification mode never(), but I haven't found a solution for the complete mock yet.
What I actually want to achieve: verify in tests, that nothing get's printed to the console. The general idea with jUnit goes like that:
private PrintStream systemOut;
#Before
public void setUp() {
// spy on System.out
systemOut = spy(System.out);
}
#After
public void tearDown() {
verify(systemOut, never()); // <-- that doesn't work, just shows the intention
}
A PrintStream has tons of methods and I really don't want to verify each and every one with separate verify - and the same for System.err...
So I hope, if there's an easy solution, that I can, given that I have a good test coverage, force the software engineers (and myself) to remove their (my) debug code like System.out.println("Breakpoint#1"); or e.printStacktrace(); prior to committing changes.

Use this :
import static org.mockito.Mockito.verifyZeroInteractions;
// ...
private PrintStream backup = System.out;
#Before
public void setUp() {
System.setOut(mock(PrintStream.class));
}
#After
public void tearDown() {
verifyZeroInteractions(System.out);
System.setOut(backup);
}

verifyZeroInteractions(systemOut);
As noted in comments, this doesn't work with a spy.
For a roughly equivalent but more complete answer, see the answer by gontard to this question.

Since the original correct answer, verifyZeroInteractions has been deprecated, use verifyNoInteractions instead:
import org.junit.jupiter.api.Test;
import static org.mockito.Mockito.*;
public class SOExample {
#Test
public void test() {
Object mock = mock(Object.class);
verifyNoInteractions(mock);
}
}

You could try a slightly different tack:
private PrintStream stdout;
#Before public void before() {
stdout = System.out;
OutputStream out = new OutputStream() {
#Override public void write(int arg0) throws IOException {
throw new RuntimeException("Not allowed");
}
};
System.setOut(new PrintStream(out));
}
#After public void after() {
System.setOut(stdout);
}
If you preferred, you could switch the anonymous type for a mock and verify as Don Roby suggests.

One way of solving this problem is to refactor the class that you're testing, to allow for the injection of a PrintStream that can be used for output. This will let you unit test it, without relying on the behaviour of the System class. You could use a package-private constructor for this injection, since you'll only ever use it from the corresponding test class. So it might look something like this.
public class MyClass{
private PrintWriter systemOut;
public MyClass(){
this(System.out);
}
MyClass(PrintWriter systemOut){
this.systemOut = systemOut;
// ...any other initialisation processing that you need to do
}
}
and within the class itself, use the systemOut variable instead of System.out wherever you call the latter.
Now, within the test class, make a mock PrintStream, and pass it to the package-private constructor, to get the object that you're going to test. Now you can run any actions you like from your tests, and use verify to check their effects on your mock PrintStream.

Selenium RC : Only one browser for all the tests?

I'm quite stuck right now, I don't understand why my code doesn't work as I need to. The fact is that each time there is a new test, it closes firefox and reopens it. That makes my tests take ages to realise... Could you tell me what I'm doing wrong ?
public class SeleniumTestLoginEntry extends SeleneseTestCase {
private String login="scLocator=//DynamicForm[ID=\"formulaire_login\"]/item[index=0||Class=TextItem]/element";
private String passwd="scLocator=//DynamicForm[ID=\"formulaire_login\"]/item[index=1||Class=PasswordItem]/element";
static public DefaultSelenium selenium;
public void setUp() throws Exception {
selenium = new DefaultSelenium("localhost", 4500, "*firefox", "http://localhost:9091/");
selenium.start();
}
public void testFields() throws Exception {
selenium.open("/agepro-prototype/login/Login.html");
selenium.type(login, "Unlogin");
selenium.type(passwd, "Unpassword");
Assert.assertEquals("Unlogin", selenium.getValue(login));
Assert.assertEquals("Unpassword", selenium.getValue(passwd));
selenium.click("scLocator=//ImgButton[ID=\"bouton_login\"]/");
}
public void testSameWindow() throws Exception {
selenium.open("/agepro-prototype/login/Login.html");
selenium.type(login, "truc");
Assert.assertEquals("truc", selenium.getValue(login));
}
public void tearDown() throws Exception {
selenium.stop();
}
}
I tried to put the annotations #BeforeClass and #AfterClass before the setUp() and tearDown, it doesn't change anything. Before there was an #Test before each test but this doesn't helped at all. May I ask you some help ? =)
Edit : Oh also I tried the selenium.open(blablabla) in the setUp() and not in the tests directly, same issue, doesn't change a thing.

Without using annotations, setUp() is going to run before every test (ie, twice).
It's not pretty, but you can create an initial test (which only runs once) and move the instantiation out of setUp() into the new test(). Keep in mind, though, that if that fails to create your selenium instance, then all your subsequent tests will also fail. Not great :)

How to run tearDown type method for a specific test in JUnit class with multiple tests?

I have a junit testCase class with multiple test methods in it ( As requirement , we don't want to create separate class for each test.)
I wanna create a tearDown type method for EACH test method , which will run specifically for that test. Not for ALL test.
My problem is , in many tests i Insert record in database, test it and delete it after test.
But, If a test fails mid way , control don't reaches till end my dummy record ain't deleting.
I think only ONE tearDown() is allowed for one class, and this tearDown() don't know what object/record i created or inserted and what to delete!!!
I want to create a tearDown() or #After method just for one specific test. Something like finally{} in java for each method.
For Eg:
public class TestDummy extends TestCase {
public void testSample1(){
InsertSomeData1();
assertFalse(true);
runTearDown1();
}
public void testSample2(){
InsertSomeData2();
assertFalse(true);
runTearDown2();
}
public void runTearDown1(){
deleteDummyDatafromTestSample1....
}
public void runTearDown2(){
deleteDummyDatafromTestSample2....
}
}
Here control will never go to runTearDown1() or runTearDown2() and I don't a one common tearDown() because it won't know what data I inserted and thats specific to each method.

It seems your test relies on a fixed database, and future tests will break if your current test breaks. What I'd recommend is not to focus on this particular problem (a test-specific tearDown method that runs for each test), but your main problem - borken tests. Before your test run, it should always work with a clean database, and this should be the case for each test. Right now, your first test has a relationship with the second (through the database).
What the right approach would be is that you recreate your database before each test, or at the very least reset it to a basic state. In this case, you'll want a test like this:
public class TestDummy {
// this code runs (once) when this test class is run.
#BeforeClass
public void setupDatabase() {
// code that creates the database schema
}
// this code runs after all tests in this class are run.
#AfterClass
public void teardownDatabase() {
// code that deletes your database, leaving no trace whatsoever.
}
// This code runs before each test case. Use it to, for example, purge the
// database and fill it with default data.
#Before
public void before() {
}
// You can use this method to delete all test data inserted by a test method too.
#After
public void after() {
}
// now for the tests themselves, we should be able to assume the database will
// always be in the correct state, independent from the previous or next test cases.
#Test
public void TestSample2() {
insertSomeData();
assertTrue(someData, isValid());
}
}
Disclaimer: JUnit 4 tests (using annotations), might not be the right annotations, might not even be the right answer(s).

You could have smth like this:
interface DBTest {
void setUpDB();
void test();
void tearDownDB();
}
class DBTestRunner {
void runTest(DBTest test) throws Exception {
test.setUpDB();
try {
test.test();
} finally {
test.tearDownDB();
}
}
}
public void test48() throws Exception {
new DBTestRunner().runTest(new DBTest() {
public void setUpDB() {...}
public void test() {...}
public void tearDownDB() {...}
});
}

#iluxa . Gr8.. Your solution is perfect!!! In one test class i created two tests test48 and test49 (same as required in my code above testSample1 and testSample2) and viola! every test method now gets its own setup() and tearDown. Only this solution looks little complicated as need to use DBTestRunner in each method, but I don't see any better solution. I was thinking Junit may have some direct solution. like #After or tearDown() with some parameter or something.
Tks a lot.

Use MethodRule:
public class MyRule implements MethodRule {
#Override
public Statement apply(final Statement base, FrameworkMethod method, Object target) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (AssertionError e) {
doFail();
} finally {
doAnyway();
}
}
};
}
}
Then declare it in your test class:
public class TestDummy{
public MethodRule rule = new MyRule();
......
}