I am having a problem with EasyMock 2.5.2 and JUnit 4.8.2 (running through Eclipse). I have read all the similar posts here but have not found an answer. I have a class containing two tests which test the same method. I am using matchers.
Each test passes when run alone.
The first test always passes - this is true if I switch the order of the tests in the file.
Here is a simplified version of the test code:
private Xthing mockXthing;
private MainThing mainThing;
#Before
public void setUp() {
mockXthing = EasyMock.createMock(Xthing.class);
mainThing = new MainThing();
mainThing.setxThing(mockXthing);
}
#After
public void cleanUp() {
EasyMock.reset(mockXthing);
}
#Test
public void testTwo() {
String abc = "abc";
EasyMock.expect(mockXthing.doXthing((String) EasyMock.anyObject())).andReturn(abc);
EasyMock.replay(mockXthing);
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
assertEquals("abc", testResult);
EasyMock.verify(mockXthing);
}
#Test
public void testOne() {
String xyz = "xyz";
EasyMock.expect(mockXthing.doXthing((String) EasyMock.anyObject())).andReturn(xyz);
EasyMock.replay(mockXthing);
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
assertEquals("xyz", testResult);
EasyMock.verify(mockXthing);
}
The second (or last) test always fails with the following error:
java.lang.IllegalStateException: 1 matchers expected, 2 recorded
Any insight to this would be greatly appreciated.
Thanks,
Anne
I haven't looked meticulously closely yet, but this looks suspect:
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
anyObject() is a matcher and you're calling it after the replay. It's not used to produce any object. It's used to instruct EasyMock to allow any object. EasyMock is detecting that extra matcher but it is not harmful until the second test. At that point, the number of matchers that EasyMock has recorded but hasn't yet used (2) doesn't line up with the number of parameters expected for the second doXthing call (1).
You should be passing in real parameters to testCallingXthing (or a mock that is in replay mode). Try passing in null directly, or a real value like 2.
for me this failure (in my case 2 matchers expected, 4 recorded.) meant "you are mixing easymock and mockito in the same unit test, so accidentally calling easymock's notNull() method for a mockito argument. Which causes the failure but only if the tests are run in a certain order.
Try:
String testResult = mainThing.testCallingXthing(eq(EasyMock.anyLong()));
There are more refined matchers than anyObject(). These allow you to make type-based assertions about collaborators.
From the EasyMock documentation:
eq(X value)
Matches if the actual value is equals the expected value. Available for all primitive types and for objects.
anyBoolean(), anyByte(), anyChar(), anyDouble(), anyFloat(), anyInt(), anyLong(), anyObject(), anyShort()
You should reset mock after each test method to get rid of this problem. Adding below code will solve this problem.
#After
public void after(){
EasyMock.reset(mockXthing)
}
Related
I've read in lot of places that test methods should/must be void, but no one says what is the reason for this.
I found in MethodValidator the following check without comments/javadocs.
if (each.getReturnType() != Void.TYPE) {
errors.add(new Exception("Method " + each.getName()
+ " should be void"));
}
So why it should be void?
Ask you the reverse question : why JUnit test methods would need to be not void ?
No reason : because a test method is not designed to return a result that will be exploited by a client class.
The goal of an unit test is validating some assertions. A test runner invokes the test methods and and this is the runner that interprets any assertion failure or any exception thrown during the test execution to produce the test result.
We could wonder why tests don't return the assertion result.
But it would be a bad idea as it would be very cumbersome to write unit tests :
#Test
public AssertionResult foo(){
Bar actualBar = foo.doThat(...);
if (actualBar == null){
return AssertionResult.fail("actualBar == null");
}
}
Write something like that is really readable and straight :
#Test
public void foo(){
Bar actualBar = foo.doThat(...);
Assert.assertNotNull(actualBar);
}
We could also wonder why test methods cannot be invoked by other test methods such as :
#Test
public int foo(){
Bar actualBar = foo.doThat(...);
//...
return intValue;
}
#Test
public void fooWithComputedInt(){
Bar actualBar = foo.doThat(foo());
//...
}
But this also would be not a good idea as this would couple the tests execution while unit tests execution has to be isolated from others. And it also would make the tests be executed multiple times and unit tests have to be executed as fast as possible.
So really no value to make a test method return other thing than void.
This is purely a design choice. JUnit does not know about your code, so it could not do anything if your method would return something.
So either it should discard a return value, or require you to use "void" methods. The authors chose the latter option - you could argue that this slightly better because it's not confusing the reader.
Note that non-#Test methods are free to do whatever they want - they don't have this limitation.
I have a problem with returning list of desired objects from Spock Groovy mocked interface:
public interface SomeRepository {
List<SomeObject> getAll();
}
So I want to mock that in class:
#CompileStatic
class SomeProcessor {
private final SomeRepository repository
SomeProcessor(SomeRepository repository) {
this.repository = repository
}
List<SomeObject> getAll() {
return repository.all
}
}
And I have that test:
class SomeProcessorSpec extends Specification {
private final SomeRepository repository = Mock(SomeRepository)
#Subject private final SomeProcessor processor = new SomeProcessor(repository)
def 'should collect items from repository'() {
given:
List<SomeObject> expected = [new SomeObject(), new SomeObject()]
repository.all >> expected
when:
List<SomeObject> actual = processor.all
then:
assertEquals(expected, actual)
}
}
When I try to run that test I get an assertion error:
junit.framework.AssertionFailedError:
Expected :[com.example.SomeObject#1fa268de, com.example.SomeOjbect#4f6ee6e4]
Actual :null
So it means that from repository.all method it returned null and not my expected list, which is confusing me. And the question is: how to actually return list from mocked instance when testing with spock and groovy?
You can try moving stubbing part to interaction check phase, e.g.
def 'should collect items from repository'() {
given:
List<SomeObject> expected = [new SomeObject(), new SomeObject()]
when:
List<SomeObject> actual = processor.all
then:
1 * repository.all >> expected
and:
expected == actual
}
Also you don't have to use JUnit's assertEquals - Groovy allows you to compare both objects with == operator.
I have checked your example in simple Spock based app and it worked fine. I tested it with Spock 0.7-groovy-2.0, 1.0-groovy-2.4 and 1.2-groovy-2.4-SNAPSHOT, worked with all Spock versions. Anyway I had some similar issues in the past and stubbing as interaction check did the trick in those cases. Hope it helps.
According to white-box testing better to test exactly the same what is implemented. processor.all returns the result of repository.all as is. So, better to test this fact.
Based on the correct code provided by Szymon Stepniak the test may be simplified to:
def 'should collect items from repository'() {
given:
def expected = []
when:
def actual = processor.all
then:
1 * repository.all >> expected
and: 'make sure we got the same expected list instance'
actual.is(expected)
}
where by .is() we verify the same references.
As result it doesn't mater what the list has, it can be just empty.
I have a test method inside a test class where I want to verify a couple things, only fail after I soft assert in this specific test method.
But, I feel my test method is getting messy with failure handling. I haven't been able to find any best practices on this. Any ideas? If I move the asserts into the page object class, it will be a bit messy there too.
#Test
public void test() {
// steps here
// then asserts here
SoftAssert soft = new SoftAssert();
String expectedHeaderText = "foo";
soft.assertTrue(pageObjectClass.isHeaderPresent(), "Unable to find the Header page object.");
soft.assertTrue(pageObjectClass.getHeader().contains(expectedHeaderText),
String.format("Expected to find '%s'. Page actually shows '%s'", expectedHeaderText, pageObjectClass.getHeader()));
// more asserts
sa.assertAll();
}
Check below convention
#Test
public void test() {
// steps here
// then asserts here
SoftAssert soft = new SoftAssert();
String expectedHeaderText = "foo";
Boolean checkHeader=pageObjectClass.isHeaderPresent() //Change the method on POM pageObjectClass such that it returns the true or false
soft.assertTrue(checkHeader,true);
String checkHeaderContent=pageObjectClass.getHeader()//change method on POM pageObjectClass to return a string
soft.assertTrue(checkHeaderContent.contains(expectedHeaderText), String.format("Expected to find '%s'. Page actually shows '%s'", expectedHeaderText, checkHeaderContent));
// more asserts
sa.assertAll();
}
You can check QMetry Automation Framework which provided assertion and verification methods. For example:
//verify element present
firstName.verifyPresent();
firstName.assertPresent();
//verify Text of Element
firstName.verifyText("First User");
firstName.assertText("First User");
//verify Text of element with StringMatchers conditions
firstName.verifyText(StringMatcher.contains("First User"));
firstName.assertText(StringMatcher.contains("First User"),"Username Validation");
In case of assert method, your test will not continue on assert fail.
In case of any verify method, your test will continue even if verification failed and the final status of test will be failed if one or more verification failed.
It's always a dilemma: to have explicit checks and readable error messages, or omit something in order to make the code shorter or more generic to be reused.
Your example is a plain SoftAssert usage which is recommended by many tutorials.
It's a best practice to keep all assertions on Test Level, not in Page Objects.
But how to deal when you see, some assertions are huge and duplicated within several test methods?
I suggest following next rules:
Similar(duplicated) assert constructions can be moved to a new helper method within a current Test Class.
Try to split the tests into classes in the way, test classes contain only similar test methods.
If several test classes contain the same code, you can create a common parent for a group of classes and move some reusable code on this test-class-group level.
Do not try to fully avoid duplications on Test Level, tests are something that is changed and outdated rapidly. They should be still readable and easy to understand what happens in the code.
I don't use SoftAsserts, but can suggest some of this as a point for the extension:
import org.testng.asserts.SoftAssert
import static java.lang.String.format
public class ProjectSoftAssert extends SoftAssert {
public void assertElementVisibleAndContainsText(
boolean isVisible, String actualText, String expectedText, String elementName
) {
assertTrue(isVisible, format("Unable to find the '%s' page object.", elementName));
assertTrue(
actualText.contains(expectedText),
format(
"Wrong '%s' page object text. Expected to find '%s'. Page actually shows '%s'",
elementName, expectedText, actualText
)
);
}
}
And in your scenario:
#Test
public void test() {
// steps here
// then asserts here
ProjectSoftAssert soft = new ProjectSoftAssert();
soft.assertElementVisibleAndContainsText(
pageObjectClass.isHeaderPresent(), pageObjectClass.getHeader(), "foo", "Header"
);
// more asserts
soft.assertAll();
}
I'm experiencing some strange behavior with EasyMock. I've set up some expectations, but when I run my tests, EasyMock fails because my expectations execute as specified. Here is an example of the failure:
Unexpected method call MyClass.myMethod(en, EasyMock for interface com.google.common.collect.Multimap, EasyMock for interface java.util.concurrent.BlockingQueue, EasyMock for interface java.util.concurrent.BlockingQueue):
MyClass.myMethod(en, EasyMock for interface com.google.common.collect.Multimap, EasyMock for interface java.util.concurrent.BlockingQueue, EasyMock for interface java.util.concurrent.BlockingQueue): expected: 100, actual: 0
The test looks like this:
Multimap<String, String> multimap = createMock(Multimap.class);
BlockingQueue<MyType> myTypeQueue = createMock(BlockingQueue.class);
BlockingQueue<MyOtherType> myOtherTypeQueue = createMock(BlockingQueue.class);
MyClass myClass = createMock(MyClass.class);
QueueFactory queueFactory = createMock(QueueFactory.class);
expect(queueFactory.<MyType>getQueue(100)).andReturn(myTypeQueue).times(2);
expect(queueFactory.<MyOtherType>getQueue(100)).andReturn(myOtherTypeQueue).times(2);
expect(myClass.myMethod("en", multimap, myTypeQueue, myOtherTypeQueue)).andReturn(something).times(100);
replayAll();
// The actual method to test
TestClass myTestInstance = new TestClass(myClass);
myTestInstance.testMethod(queueFactory, multimap);
myTestInstance.testMethod(queueFactory, multimap);
Here is the method I am testing:
public class MyTestClass
private MyClass myClass;
public MyTestClass(MyClass myClass) {
this.myClass = myClass;
}
public void testMethod(QueueFactory queueFactory, Multimap<String, String> multimap) {
BlockingQueue<MyType> myTypeQueue = queueFactory.getQueue(100);
BlockingQueue<MyOtherType> myOtherTypeQueue = queueFactory.getQueue(100);
for (int i = 0; i < 50; ++i) {
myClass.myMethod("en", multimap, myTypeQueue, myOtherTypeQueue);
}
}
}
So far, I've tried replacing the mocks with instances of HashMultimap and ArrayBlockingQueue. I've also tried wrapping all of the method parameter's in EasyMock.eq. Neither of these approaches solved the issue.
Any ideas why this is happening?
Because of type erasure, there's nothing EasyMock can do to determine that
expect(queueFactory.<MyType>getQueue(100))
is different from
expect(queueFactory.<MyOtherType>getQueue(100))
Each expect call registers an expectation for a number of invocations, in order. So
expect(queueFactory.<MyType> getQueue(100)).andReturn(myTypeQueue).times(2);
states that the first two times you invoke getQueue(100), it will return myTypeQueue. If you follow that with
expect(queueFactory.<MyOtherType> getQueue(100)).andReturn(myOtherTypeQueue).times(1);
The next two calls to getQueue(100) will return myOtherTypeQueue.
In your testMethod, you invoke
BlockingQueue<MyType> myTypeQueue = queueFactory.getQueue(100);
BlockingQueue<MyOtherType> myOtherTypeQueue = queueFactory.getQueue(100);
which is invoking getQueue(100) twice, ie. the first two times. These will both return myTypeQueue, which breaks the expectation on myClass.myMethod.
It works when you only do it once, because the first expect returns myTypeQueue and the second returns myOtherTypeQueue which then matches your myClass.myMethod expectations.
You can fix it by registering each in order, once
expect(queueFactory.<MyType> getQueue(100)).andReturn(myTypeQueue).times(1);
expect(queueFactory.<MyOtherType> getQueue(100)).andReturn(myOtherTypeQueue).times(1);
expect(queueFactory.<MyType> getQueue(100)).andReturn(myTypeQueue).times(1);
expect(queueFactory.<MyOtherType> getQueue(100)).andReturn(myOtherTypeQueue).times(1);
I'm trying to get PowerMock to work with mockito, and I'm following the documentation here: http://code.google.com/p/powermock/wiki/MockitoUsage13.
To simplify a bit, lets say that I have a static method:
StaticObj.put(String key, String val) { ... }
And the class to be tested does something like this:
public class ClassToTest {
public void doSomething(Params p) {
if (StringUtils.isNotBlank(p.getK()) StaticObj.put("k1", p.getK());
if (StringUtils.isNotBlank(p.getX()) StaticObj.put("x1", p.getX());
}
}
In my unit test I'd like to verify that StaticObj.put is called for K and X when they are not blank or null, so I do something like this:
public void testNormalCase() {
// assume that mocking setup for statics already happened in some #Before function..
Params params = new Params("k", "x");
ClassToTest classToTest = new ClassToTest();
classToTest.doSomething(params);
// now I want to verify:
PowerMockito.verifyStatic(times(1));
StaticObj.put("k1", "k1");
PowerMockito.verifyStatic(times(1));
StaticObj.put("x1", "x");
}
This works, and it's what I'd expect. What doesn't work, is if I comment out the verification for K, then the verification for X fails! The error message indicates that ("x1", "x") is expected but got ("k1", "k"). Why is this? Am I not coding this correctly?
Also it leads me to believe that the following type of test, which passes, might pass for the wrong reason entirely:
public void testOtherCase() {
// assume that mocking setup for statics already happened in some #Before function..
Params params = new Params("k", null);
ClassToTest classToTest = new ClassToTest();
classToTest.doSomething();
// now I want to verify:
PowerMockito.verifyStatic(never());
StaticObj.put(eq("x1"), anyString());
}
E.g. I wonder if powermock sees "k1", decides that "x1" was never called, and passes. (?)
To state it generally, I have a static method that is called N times (where N changes depending on the input params). And I want to verify that it was called in the correct cases (which can be determined by input params). It seems like powermock doesn't handle this well, unless I misunderstand.
Thanks for any ideas!
I read this question and the issue carefully but not sure if I understood them clearly - From my understanding, it's correct that powermock raise the exception when you pass k and x but only verify k.
Because you are mocking the static method StaticObj.put, when you pass parameter k and x and verify it with
PowerMockito.verifyStatic(times(1));
StaticObj.put("k1", "k1");
PowerMockito.verifyStatic(times(1));
StaticObj.put("x1", "x");
This should work. And when you verify parameter k and x with verification for k is commented out.
// PowerMockito.verifyStatic(times(1));
// StaticObj.put("k1", "k1");
PowerMockito.verifyStatic(times(1));
StaticObj.put("x1", "x");
Powermock will get the invocation with put("k1"...) first apparently, so the verification of x will raise an error. Your verification process is sequenced.
I don't know as of which version, but PowerMockito.verifyStatic(VerificationMode) is deprecated. Just wanted to point that out to anyone else finding this years after the last post.