I am using JMockit 1.12 and want to verify that AccessController.doPrivileged() was called. This seems rather straightforward:
#Test(expected = MissingInvocation.class)
public void testFoo1() {
foo(false, true);
}
#Test
public void testFoo2() {
foo(false, false);
}
#Test
public void testFoo3() {
foo(true, true);
}
private void foo(boolean usePrivilegedAccess, boolean expectAccessControllerCall) {
new NonStrictExpectations(AccessController.class) {{
}};
if (usePrivilegedAccess) {
AccessController.doPrivileged((PrivilegedAction<String>) () -> "");
}
// verify AccessController.doPrivileged was called
if (expectAccessControllerCall) {
new Verifications() {{ AccessController.doPrivileged(withAny((PrivilegedAction<Object>) () -> null )); }};
}
}
Note that testFoo1() does not call AccessController.doPrivileged() yet performs the check anyway.
I added this method because I found that sometimes the Verifications block would pass even if AccessController.doPrivileged(). I am using Netbeans 8.0.1 and after a lot of testing, I found that if I run the test using "Run Focused Test Method" or "Debug Focused Test Method" (runs only 1 test) then it passes. If I use "Test File" (runs all tests) then testFoo1() fails because it does not throw MissingInvocation. If I use "Debug Test File" (runs all tests) then it always fails if I put in a breakpoint; it intermittently fails if I do not put in a breakpoint. Very strange.
Is my JMockit usage correct? I am new, so any pointers appreciated but please note that I want to run the exact same test code from 2 tests which only differ by a boolean flag. I do not want to copy/paste the test twice.
Is there something up with Netbeans?
Is it something to do with the CGLib injection somewhere in the pipeline?
What probably happens is that, during the test, AccessController.doPrivileged(...) gets called from somewhere else, perhaps from NetBeans or more likely from the JRE itself.
JMockit 1.x does not restrict the mocking of the AccessController class to just the foo(boolean,boolean) method; instead, it registers all invocations to its methods, regardless of where they come from. The test would have to implement a more restrictive verification, perhaps checking the exact PrivilegedAction instance passed to the mocked method, or even by checking the call stack to see where it comes from.
For JMockit 2, an API change is planned so that mocking gets scoped to #Tested classes only, avoiding situations like this.
Related
When debugging a test I realized that something was interfering with Mockito well functioning. Somehow, the inclusion of breakpoints in specific classes leads to a different output.
I try to illustrate it with a simple example.
public class MockitoTrial {
#Test
public void simpleTest() {
var func = Mockito.mock(Function.class);
Entry<String, Integer> entry = new SimpleEntry<>("one", 1);
when(func.apply(eq(Entry.class))).thenReturn(entry);
assertThat(func.apply(Entry.class)).isEqualTo(entry);
}
}
If I set a breakpoint for instance in org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.interceptAbstract, and rerun it in debug-mode, the test fails.
It seems apparently unrelated to the IDE as it happens when debugging remotely as well.
The library versions I am using:
assertj-core-3.22.0
junit-jupiter-api-5.8.2
mockito-core-4.5.1
The question pointed out by Jonasz, gives a good clue.
If I instead set the breakpoint in org.mockito.internal.handler.MockHandlerImpl.handle, the test does already pass even in debug-mode.
Now, I add a watcher for the expression invocation.getMock().toString() and rerun it in debug-mode. Doing that, I got the test to fail again.
Mockito is very sensitive to calls to the mock methods in between its processing.
I'm getting an error when I step through my test code with #MockBean.thenReturn() in a debugger (Intellij Java 17). It works fine when it is executed outside the debugger though.
Here is a simplified version:
#ExtendWith(SpringExtension.class)
public class MockitoBusted {
#MockBean
Target target;
#Test
void test() {
var call = Mockito.when(target.x());
// step over the below line and it fails.
call.thenReturn(Collections.emptyList());
}
public static class Target {
public List<String> x() {
return Collections.emptyList();
}
}
}
The failure occurs when stepping over the call.thenReturn(...) line. The error message is:
org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
EmptyList cannot be returned by toString()
toString() should return String
***
If you're unsure why you're getting above error read on.
Due to the nature of the syntax above problem might occur because:
1. This exception *might* occur in wrongly written multi-threaded tests.
Please refer to Mockito FAQ on limitations of concurrency testing.
2. A spy is stubbed using when(spy.foo()).then() syntax. It is safer to stub spies -
- with doReturn|Throw() family of methods. More in javadocs for Mockito.spy() method.
When using Mockito.mock() directly, this does not happen - everything works fine.
var target = Mockito.mock(Target.class);
var call = Mockito.when(target.x());
call.thenReturn(Collections.emptyList());
How is it possible that it fails when stepping over in the debugger, but works just fine when executed outside the debugger?
And is there a work-around for this?
I'm having issues with SonarQube raising issues with several of my unit tests, prompting the following issue:
Add at least one assertion to this test case.
Each test case resembles this format (where a number of assertions are delegated to a method with common assertions, to avoid duplication):
#Test
public void companyNameOneTooLong() throws Exception {
AddressFormBean formBean = getValidBean();
formBean.setCompanyNameOne("123456789012345678901234567890123456");
assertViolation(validator.validate(formBean), "companyNameOne", "length must be between 0 and 35");
}
private void assertViolation(Set<ConstraintViolation<AddressFormBean>> violations, String fieldname, String message) {
assertThat(violations, hasSize(1));
assertEquals(fieldname, violations.iterator().next().getPropertyPath().iterator().next().getName());
assertEquals(message, violations.iterator().next().getMessage());
}
Now, obviously I could just pull the three assertions out of the private method and put them in the test method - but I'm performing the same checks (on different fields) multiple times.
So, I thought I'd try to emulate the behaviour of the assertion methods, by (re) throwing an AssertionError:
private void assertViolation(Set<ConstraintViolation<AddressFormBean>> violations, String fieldname, String message) throws AssertionError {
try {
assertThat(violations, hasSize(1));
assertEquals(fieldname, violations.iterator().next().getPropertyPath().iterator().next().getName());
assertEquals(message, violations.iterator().next().getMessage());
} catch (AssertionError e) {
throw e;
}
}
Unfortunately, this approach does not work either.
What's special about the JUnit assert methods / what is SonarQube looking for specifically to check that an assertion has been made for each test?
Alternatively - are there other approaches to achieve the same end result (avoiding duplicating the shared assertion code over and over)?
The rule S2699 (Tests should include assertions) from the SonarQube Java Analyzer does not perform cross-procedural analysis and only explore body of methods being identified as test method (usually annotated with #Test).
Consequently, if the only assertions which will be called when executing the test method are done by a dedicated method (to avoid duplication), then the rule will raise an issue. This is a known limitation of the rule and we will deal with it only when we will be able to efficiently perform cross-procedural analysis.
Regarding the issues raised by SonarQube on such cases, you can safely mark them as Won't Fix.
Regarding the detected assertions, the rule consider as assertions the usual assert/fail/verify/expect methods from the following (unit test) frameworks :
JUnit
Fest (1.x & 2.x)
AssertJ
Hamcrest
Mockito
Spring
EasyMock
If you don't expect any exception to be throw from your test, this can be a workaround:
#Test(expected = Test.None.class /* no exception expected */)
Alternatively, you can suppress the warning for the test method/test class:
#SuppressWarnings("squid:S2699")
One thing I have done in the past is to have the helper method return true, and assert on that:
#Test
public void testSomeThings() {
Thing expected = // . . .
Thing actual = service.methodReturningThing(42);
assertTrue(assertViolation(expected, actual));
}
private boolean assertViolation(Thing expected, Thing actual) {
assertEquals(expected.getName(), actual.getName());
assertEquals(expected.getQuest(), actual.getQuest());
assertEquals(expected.getFavoriteColor(), actual.getFavoriteColor());
return true;
}
I hate this, but I hate duplicated code even more.
The other thing we've done at times, is to simply mark any such objections from SonarQube as Won't Fix, but I hate that, too.
Sometimes you don't need to have any code or assertation, for example, the test of load the context of spring boot successfully. In this case, to prevent Sonar issue when you don't expect any exception to be throw from your test, you can use this part of the code:
#Test
void contextLoads() {
Assertions.assertDoesNotThrow(this::doNotThrowException);
}
private void doNotThrowException(){
//This method will never throw exception
}
I'm using Spock to write tests for a Java application. The test includes a JMockit MockUp for the class. When debugging a test (using IntelliJ) I would like to be able to step into the Java code. I realize using F5 to step into the code won't work, because of all the Groovy magic with reflection that goes on. Unfortunately, even if I set a breakpoint in the Java code, it still will not be hit, even though it runs through the code.
Here is the code to test:
public class TestClass {
private static void initializeProperties() {
// Code that will not run during test
}
public static boolean getValue() {
return true; // <=== BREAK POINT SET HERE WILL NEVER BE HIT
}
}
Here is the test code:
class TestClassSpec extends Specification {
void setup() {
new MockUp<TestClass>() {
#Mock
public static void initializeProperties() {}
}
}
void "run test"() {
when:
boolean result = TestClass.getValue()
then:
result == true
}
}
The test passes and everything works well, but I'm not able to debug into the Java code.
Note: I can debug if I do not use the JMockit MockUp, but that is required to test a private static method in my production code.
Any ideas on how to get the debugger to stop in the Java code from a Spock test that uses JMockit?
This answer has plagued me for days, and I have searched and searched for an answer. Fifteen minutes after posting for help, I find the answer . So for anyone else who runs into this, here's my solution - this is all thanks to this answer: https://stackoverflow.com/a/4852727/2601060
In short, the breakpoint is removed when the class is redefined by JMockit. The solution is to set a break point in the test, and once the debugger has stopped in the test THEN set the breakpoint in the Java code.
And there was much rejoicing...
I'm having a strange problem with Easymock 3.0 and JUnit 4.8.2.
The problem only occurs when executing the tests from Maven and not from Eclipse.
This is the unit test (very simple):
...
protected ValueExtractorRetriever mockedRetriever;
...
#Before
public void before() {
mockedRetriever = createStrictMock(ValueExtractorRetriever.class);
}
#After
public void after() {
reset(mockedRetriever);
}
#Test
public void testNullValueExtractor() {
expect(mockedRetriever.retrieve("PROP")).andReturn(null).once();
replay(mockedRetriever);
ValueExtractor retriever = mockedRetriever.retrieve("PROP");
assertNull(retriever);
assertTrue(true);
}
And I get:
java.lang.IllegalStateException: 1 matchers expected, 2 recorded.
The weird thing is that I'm not even using an argument matcher. And that is the only method of the test! and to make it even worst it works from Eclipse and fails from Maven!
I found a few links which didn't provide me with an answer:
Another StackOverflow post
Expected Exceptions in JUnit
If I change the unit test and add one more method (which does use an argument matcher):
#Test
public void testIsBeforeDateOk() {
expect(mockedRetriever.retrieve((String)anyObject())).andReturn(new PofExtractor()).anyTimes();
replay(this.mockedRetriever);
FilterBuilder fb = new FilterBuilder();
assertNotNull(fb);
CriteriaFilter cf = new CriteriaFilter();
assertNotNull(cf);
cf.getValues().add("2010-12-29T14:45:23");
cf.setType(CriteriaType.DATE);
cf.setClause(Clause.IS_BEFORE_THE_DATE);
CriteriaQueryClause clause = CriteriaQueryClause.fromValue(cf.getClause());
assertNotNull(clause);
assertEquals(CriteriaQueryClause.IS_BEFORE_THE_DATE, clause);
clause.buildFilter(fb, cf, mockedRetriever);
assertNotNull(fb);
Filter[] filters = fb.getFilters();
assertNotNull(filters);
assertEquals(filters.length, 1);
verify(mockedRetriever);
logger.info("OK");
}
this last method passes the test but not the other one. How is this possible!?!?!
Regards,
Nico
More links:
"bartling.blogspot.com/2009/11/using-argument-matchers-in-easymock-and.html"
"www.springone2gx.com/blog/scott_leberknight/2008/09/the_n_matchers_expected_m_recorded_problem_in_easymock"
"stackoverflow.com/questions/4605997/3-matchers-expected-4-recorded"
I had a very similar problem and wrote my findings in the link below.
http://www.flyingtomoon.com/2011/04/unclosed-record-state-problem-in.html (just updated)
I believe the problem in on another test that affects your current test. The problem is on another test class and it affects you test. In order to find the place of the real problem, I advice to disable the problematic tests one by one till you notify the failing test.
Actually this is what I did. I disabled the failing tests one by one till I found the problematic test. I found a test that throws an exception and catches by "#extected" annotation without stopping the recording.
We had this problem recently, and it only reared its head when we ran the entire test suite (1100+ test cases). Eventually, I found that I could put a breakpoint on the test that was blowing up, and then step back in the list of tests that Eclipse had already executed, looking for the previous test case that had set up a mock incorrectly.
Our problem turned out to be somebody using EasyMock.anyString() outside of an EasyMock.expect(...) statement. Sure enough, it was done two tests before the one that was failing.
So essentially, what was happening is that the misuse of a matcher outside of an expect statement was poisoning EasyMock's state, and the next time we tried to create a mock, EasyMock would blow up.
I believe the first error message
java.lang.IllegalStateException: 1
matchers expected, 2 recorded.
means your mockedRetriever methods called twice but test expects it was called once. So your Eclipse and Maven's configuration differs.
And I have no reason to reset mock after test. Just keep in mind JUnit creates new class instance for every single test method.
EDITED:
What about the reason why the last test method passed the answer is:
expect(mockedRetriever.retrieve((String)anyObject())).andReturn(new PofExtractor()).anyTimes();
But in your first test method it is:
expect(mockedRetriever.retrieve("PROP")).andReturn(null).once();
as equivalent of:
expect(mockedRetriever.retrieve("PROP")).andReturn(null);