I have a strange problem with EasyMock
This is the call I'm making which throws an IllegalStateException : Matcher expected as expected
expect(this.mock.expectedOperation(gt(0l), MyClass.class)).andReturn(createClassObject());
If I replace the above call with
expect(this.mock.expectedOperation(gt(0l), createClass(MyClass.class))).andReturn(createClassObject());
#SuppressWarnings("unchecked")
public static <T> Class<T> createClass(Class<T> clazz)
{
return (Class<T>) EasyMock.anyObject();
}
Most times i don't get an error, but sometimes I do. It stays IllegalStateException : Matcher expected ..
Sometimes I get the IllegalStateException : 2 Matchers expected 1 recorder error for doing this:
MyClass object = createClassObject();
expect(this.mock.expectedOperation(anyLong(), anyLong()).andReturn(object);
public MyClass createClassObject() {
// Actually sets properties and then returns
return new MyClass();
}
But it runs when I do this:
expect(this.mock.expectedOperation(anyLong(), anyLong()).andReturn(createClassObject());
In the above example, sometimes the former runs and latter fails.
Sometimes this fails :
MyClass object = createClassObject();
expect(this.mock.expectedOperation(1, MyClass.class)).andReturn(object);
I have quadruple checked the reset, replay, verify calls. These tests run sometimes and sometimes fail.
If i run my unit test, it randomly fails at least once in one of the above listed situations. Why? How do I get it running?
EDIT : I'm using EasyMock version 3.1 and
MockedClass mock = EasyMock.createMock(MockedClass.class);
Found the problem. We can't use the gt(0) etc methods to pass parameters into the unit being tested.
In another test by my mistake had used :
service.performOperation(1, gt(0l));
service was not a mock, but the unit i was testing.
On checking the documentation I saw that gt(0l) returns 0, which was causing this test to pass by others to fail. (Dunno why?) Since tests are executed randomly, random conditions were failing.
Using the reset(mocks..) at the beginning of the tests did nothing to help.
Related
I am recently writing a JUnit test and I acknowledged that Mockito either needs all Raw values or using all Matchers but I find the following case still fails with all Matchers, error message is:
Invalid use of argument matchers!
0 matchers expected, 1 recorded:
I tested a little bit and it looks like something to do with using a stub method as value for the eq() Matcher. See example below:
I have a Class A very simple
public class A {
public void testMockitoMatcher(double a, String b){
}
}
Here is my test case
import org.junit.Test;
import static org.mockito.Mockito.*;
public class SomeUnitTest {
private A mockA = mock(A.class);
#Test
public void allMatchersDoesntWork() {
Object mockObject = mock(Object.class);
String someString = "Just some mocked String value to return";
when(mockObject.toString()).thenReturn(someString);
mockA.testMockitoMatcher(1312d, mockObject.toString());
verify(mockA, times(1)).testMockitoMatcher(anyDouble(), eq(someString)); //<- This works
verify(mockA, times(1)).testMockitoMatcher(anyDouble(), eq(mockObject.toString())); //<- This doesn't by using stub method toString() to return the String value as param to eq()
}
}
I also verified by using debug at 2nd verify statement that mockObject.toString() can still return the stubbed value for me. Also both verify use all Matchers, why mockito still gives me only 1 Matchers recorded instead of 2 at the 2nd verify?
Thanks!
Mockito expects your tests to be in three stages.
You set things up for the test, including stubbing any methods that need to be stubbed (the "arrange" stage).
You run the actual method that you're trying to test (the "act" stage).
You make assertions about the output of the test, possibly including verifying which methods got called on your mocks (the "assert" stage).
You need to do these three things, in this sequence, because Mockito tracks whether you're stubbing, verifying or whatever. It also has its own internal stack which stores Matchers, for use in stubbing or verification.
In your second verify call in your example, the call to anyDouble() puts the Matcher onto the stack. But this creates an invalid state for the call to mockObject.toString().
Don't do this. Once you're ready to run your assertions and verifications, you shouldn't be making any more calls to stubbed methods. Keep the "arrange", "act" and "assert" stages of each test entirely separate from each other.
I'm getting an error when I step through my test code with #MockBean.thenReturn() in a debugger (Intellij Java 17). It works fine when it is executed outside the debugger though.
Here is a simplified version:
#ExtendWith(SpringExtension.class)
public class MockitoBusted {
#MockBean
Target target;
#Test
void test() {
var call = Mockito.when(target.x());
// step over the below line and it fails.
call.thenReturn(Collections.emptyList());
}
public static class Target {
public List<String> x() {
return Collections.emptyList();
}
}
}
The failure occurs when stepping over the call.thenReturn(...) line. The error message is:
org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
EmptyList cannot be returned by toString()
toString() should return String
***
If you're unsure why you're getting above error read on.
Due to the nature of the syntax above problem might occur because:
1. This exception *might* occur in wrongly written multi-threaded tests.
Please refer to Mockito FAQ on limitations of concurrency testing.
2. A spy is stubbed using when(spy.foo()).then() syntax. It is safer to stub spies -
- with doReturn|Throw() family of methods. More in javadocs for Mockito.spy() method.
When using Mockito.mock() directly, this does not happen - everything works fine.
var target = Mockito.mock(Target.class);
var call = Mockito.when(target.x());
call.thenReturn(Collections.emptyList());
How is it possible that it fails when stepping over in the debugger, but works just fine when executed outside the debugger?
And is there a work-around for this?
I'm having issues with SonarQube raising issues with several of my unit tests, prompting the following issue:
Add at least one assertion to this test case.
Each test case resembles this format (where a number of assertions are delegated to a method with common assertions, to avoid duplication):
#Test
public void companyNameOneTooLong() throws Exception {
AddressFormBean formBean = getValidBean();
formBean.setCompanyNameOne("123456789012345678901234567890123456");
assertViolation(validator.validate(formBean), "companyNameOne", "length must be between 0 and 35");
}
private void assertViolation(Set<ConstraintViolation<AddressFormBean>> violations, String fieldname, String message) {
assertThat(violations, hasSize(1));
assertEquals(fieldname, violations.iterator().next().getPropertyPath().iterator().next().getName());
assertEquals(message, violations.iterator().next().getMessage());
}
Now, obviously I could just pull the three assertions out of the private method and put them in the test method - but I'm performing the same checks (on different fields) multiple times.
So, I thought I'd try to emulate the behaviour of the assertion methods, by (re) throwing an AssertionError:
private void assertViolation(Set<ConstraintViolation<AddressFormBean>> violations, String fieldname, String message) throws AssertionError {
try {
assertThat(violations, hasSize(1));
assertEquals(fieldname, violations.iterator().next().getPropertyPath().iterator().next().getName());
assertEquals(message, violations.iterator().next().getMessage());
} catch (AssertionError e) {
throw e;
}
}
Unfortunately, this approach does not work either.
What's special about the JUnit assert methods / what is SonarQube looking for specifically to check that an assertion has been made for each test?
Alternatively - are there other approaches to achieve the same end result (avoiding duplicating the shared assertion code over and over)?
The rule S2699 (Tests should include assertions) from the SonarQube Java Analyzer does not perform cross-procedural analysis and only explore body of methods being identified as test method (usually annotated with #Test).
Consequently, if the only assertions which will be called when executing the test method are done by a dedicated method (to avoid duplication), then the rule will raise an issue. This is a known limitation of the rule and we will deal with it only when we will be able to efficiently perform cross-procedural analysis.
Regarding the issues raised by SonarQube on such cases, you can safely mark them as Won't Fix.
Regarding the detected assertions, the rule consider as assertions the usual assert/fail/verify/expect methods from the following (unit test) frameworks :
JUnit
Fest (1.x & 2.x)
AssertJ
Hamcrest
Mockito
Spring
EasyMock
If you don't expect any exception to be throw from your test, this can be a workaround:
#Test(expected = Test.None.class /* no exception expected */)
Alternatively, you can suppress the warning for the test method/test class:
#SuppressWarnings("squid:S2699")
One thing I have done in the past is to have the helper method return true, and assert on that:
#Test
public void testSomeThings() {
Thing expected = // . . .
Thing actual = service.methodReturningThing(42);
assertTrue(assertViolation(expected, actual));
}
private boolean assertViolation(Thing expected, Thing actual) {
assertEquals(expected.getName(), actual.getName());
assertEquals(expected.getQuest(), actual.getQuest());
assertEquals(expected.getFavoriteColor(), actual.getFavoriteColor());
return true;
}
I hate this, but I hate duplicated code even more.
The other thing we've done at times, is to simply mark any such objections from SonarQube as Won't Fix, but I hate that, too.
Sometimes you don't need to have any code or assertation, for example, the test of load the context of spring boot successfully. In this case, to prevent Sonar issue when you don't expect any exception to be throw from your test, you can use this part of the code:
#Test
void contextLoads() {
Assertions.assertDoesNotThrow(this::doNotThrowException);
}
private void doNotThrowException(){
//This method will never throw exception
}
I am using JMockit 1.12 and want to verify that AccessController.doPrivileged() was called. This seems rather straightforward:
#Test(expected = MissingInvocation.class)
public void testFoo1() {
foo(false, true);
}
#Test
public void testFoo2() {
foo(false, false);
}
#Test
public void testFoo3() {
foo(true, true);
}
private void foo(boolean usePrivilegedAccess, boolean expectAccessControllerCall) {
new NonStrictExpectations(AccessController.class) {{
}};
if (usePrivilegedAccess) {
AccessController.doPrivileged((PrivilegedAction<String>) () -> "");
}
// verify AccessController.doPrivileged was called
if (expectAccessControllerCall) {
new Verifications() {{ AccessController.doPrivileged(withAny((PrivilegedAction<Object>) () -> null )); }};
}
}
Note that testFoo1() does not call AccessController.doPrivileged() yet performs the check anyway.
I added this method because I found that sometimes the Verifications block would pass even if AccessController.doPrivileged(). I am using Netbeans 8.0.1 and after a lot of testing, I found that if I run the test using "Run Focused Test Method" or "Debug Focused Test Method" (runs only 1 test) then it passes. If I use "Test File" (runs all tests) then testFoo1() fails because it does not throw MissingInvocation. If I use "Debug Test File" (runs all tests) then it always fails if I put in a breakpoint; it intermittently fails if I do not put in a breakpoint. Very strange.
Is my JMockit usage correct? I am new, so any pointers appreciated but please note that I want to run the exact same test code from 2 tests which only differ by a boolean flag. I do not want to copy/paste the test twice.
Is there something up with Netbeans?
Is it something to do with the CGLib injection somewhere in the pipeline?
What probably happens is that, during the test, AccessController.doPrivileged(...) gets called from somewhere else, perhaps from NetBeans or more likely from the JRE itself.
JMockit 1.x does not restrict the mocking of the AccessController class to just the foo(boolean,boolean) method; instead, it registers all invocations to its methods, regardless of where they come from. The test would have to implement a more restrictive verification, perhaps checking the exact PrivilegedAction instance passed to the mocked method, or even by checking the call stack to see where it comes from.
For JMockit 2, an API change is planned so that mocking gets scoped to #Tested classes only, avoiding situations like this.
I'm having a strange problem with Easymock 3.0 and JUnit 4.8.2.
The problem only occurs when executing the tests from Maven and not from Eclipse.
This is the unit test (very simple):
...
protected ValueExtractorRetriever mockedRetriever;
...
#Before
public void before() {
mockedRetriever = createStrictMock(ValueExtractorRetriever.class);
}
#After
public void after() {
reset(mockedRetriever);
}
#Test
public void testNullValueExtractor() {
expect(mockedRetriever.retrieve("PROP")).andReturn(null).once();
replay(mockedRetriever);
ValueExtractor retriever = mockedRetriever.retrieve("PROP");
assertNull(retriever);
assertTrue(true);
}
And I get:
java.lang.IllegalStateException: 1 matchers expected, 2 recorded.
The weird thing is that I'm not even using an argument matcher. And that is the only method of the test! and to make it even worst it works from Eclipse and fails from Maven!
I found a few links which didn't provide me with an answer:
Another StackOverflow post
Expected Exceptions in JUnit
If I change the unit test and add one more method (which does use an argument matcher):
#Test
public void testIsBeforeDateOk() {
expect(mockedRetriever.retrieve((String)anyObject())).andReturn(new PofExtractor()).anyTimes();
replay(this.mockedRetriever);
FilterBuilder fb = new FilterBuilder();
assertNotNull(fb);
CriteriaFilter cf = new CriteriaFilter();
assertNotNull(cf);
cf.getValues().add("2010-12-29T14:45:23");
cf.setType(CriteriaType.DATE);
cf.setClause(Clause.IS_BEFORE_THE_DATE);
CriteriaQueryClause clause = CriteriaQueryClause.fromValue(cf.getClause());
assertNotNull(clause);
assertEquals(CriteriaQueryClause.IS_BEFORE_THE_DATE, clause);
clause.buildFilter(fb, cf, mockedRetriever);
assertNotNull(fb);
Filter[] filters = fb.getFilters();
assertNotNull(filters);
assertEquals(filters.length, 1);
verify(mockedRetriever);
logger.info("OK");
}
this last method passes the test but not the other one. How is this possible!?!?!
Regards,
Nico
More links:
"bartling.blogspot.com/2009/11/using-argument-matchers-in-easymock-and.html"
"www.springone2gx.com/blog/scott_leberknight/2008/09/the_n_matchers_expected_m_recorded_problem_in_easymock"
"stackoverflow.com/questions/4605997/3-matchers-expected-4-recorded"
I had a very similar problem and wrote my findings in the link below.
http://www.flyingtomoon.com/2011/04/unclosed-record-state-problem-in.html (just updated)
I believe the problem in on another test that affects your current test. The problem is on another test class and it affects you test. In order to find the place of the real problem, I advice to disable the problematic tests one by one till you notify the failing test.
Actually this is what I did. I disabled the failing tests one by one till I found the problematic test. I found a test that throws an exception and catches by "#extected" annotation without stopping the recording.
We had this problem recently, and it only reared its head when we ran the entire test suite (1100+ test cases). Eventually, I found that I could put a breakpoint on the test that was blowing up, and then step back in the list of tests that Eclipse had already executed, looking for the previous test case that had set up a mock incorrectly.
Our problem turned out to be somebody using EasyMock.anyString() outside of an EasyMock.expect(...) statement. Sure enough, it was done two tests before the one that was failing.
So essentially, what was happening is that the misuse of a matcher outside of an expect statement was poisoning EasyMock's state, and the next time we tried to create a mock, EasyMock would blow up.
I believe the first error message
java.lang.IllegalStateException: 1
matchers expected, 2 recorded.
means your mockedRetriever methods called twice but test expects it was called once. So your Eclipse and Maven's configuration differs.
And I have no reason to reset mock after test. Just keep in mind JUnit creates new class instance for every single test method.
EDITED:
What about the reason why the last test method passed the answer is:
expect(mockedRetriever.retrieve((String)anyObject())).andReturn(new PofExtractor()).anyTimes();
But in your first test method it is:
expect(mockedRetriever.retrieve("PROP")).andReturn(null).once();
as equivalent of:
expect(mockedRetriever.retrieve("PROP")).andReturn(null);