Disabling certain aspects during unit test runs - java

I have integration tests (load context) and unit tests running together. My code does aspectj compile time weaving using spring.
My problem is that my declared advises also run during some of my unit tests. This kills the notion of a unit test, which is why I would like to disable them.
Is there something I can put on the pointcut declaration, some method I can call, some spring configuration, or maven command that disables these advises for something like all *UnitTest.java?
Thanks for the help.
example:
I have the following unit test:
#RunWith(MockitoJUnitRunner.class)
public class CompanyServiceImplTest {
#Test
public void createCampaignTest() throws Exception {
when(companyDaoMock.saveCompany(any(Campaign.class))).thenReturn(77L);
Long campaignId = companyService.createCampaign(campaignMock);
assertEquals(Long.valueOf(77L), Long.valueOf(campaignId));
}
}
and following service method:
#Override
#Transactional
#EventJournal(type = EventType.CAMPAIGN_CREATE, owner = EventOwner.TERMINAL_USER)
public Long createCampaign(Campaign campaign) {
return companyDao.saveCompany(campaign);
}
aspect:
#Aspect
public class EventJournalAspect {
#Autowired
private EventJournalService eventJournalService;
#Pointcut(value="execution(public * *(..))")
public void anyPublicMethod() {}
#Pointcut("within(com.terminal.service..*)")
private void inService() {}
#AfterReturning(pointcut = "anyPublicMethod() && inService() && #annotation(eventJournal) && args(entity,..)", returning = "id")
public void process(Object id, EventJournal eventJournal, AbstractDomainEntity entity)
throws Throwable {
if (eventJournal.type() != EventType.CAMPAIGN_PAYMENT || id != null) {
saveEvent(eventJournal, EventStatus.SUCCESS, entity, (Long) id);
}
}
#AfterThrowing(pointcut = "anyPublicMethod() && inService() && #annotation(eventJournal) && args(entity,..)", throwing="ex")
public void processException(EventJournal eventJournal, AbstractDomainEntity entity, Exception ex) throws Throwable {
saveEvent(eventJournal, EventStatus.FAILURE, entity, null);
}
private void saveEvent(EventJournal eventJournal, EventStatus status, AbstractDomainEntity entity, Long persistentId) {
EventType type = eventJournal.type();
EventOwner owner = eventJournal.owner();
eventJournalService.saveEvent(type, owner, EventStatus.SUCCESS, entity, persistentId);
}
}
When test executes - eventJournalService is null. Thus I see NullPointerException

The answer is simple: You want to use an if() pointcut expression.
Update (after the question has also been updated): The originally provided link above should contain enough information, but for what it is worth, a short explanation and a simple example:
An if() pointcut is a static aspect method returning a boolean. If the return value is true, it means that any combined pointcut like myPointcut() && if() matches as long as myPointcut() matches. For a return value of false the whole combined pointcut does not match, effectively deactivating any advice connected to the pointcut.
So what can you do in a static if() pointcut?
evaluate a static boolean member of some tool class like TestMode.ACTIVE which is only true during unit or integration testing
evaluate an environment variable which is only set during testing
evaluate a Java system property which is only set during testing
and many more things
If you want to do something fancier (and trickier) and performance is not so important, you can also try to dynamically determine whether the auto-wired aspect member variable equals null or not and only activate your pointcuts if the injected object is actually present. The only problem here is how to determine a member variable from a static method. I have no idea about Spring AOP, but in plain AspectJ there is the helper class Aspects with several overloaded methods named aspectOf(..). Assuming that your aspect is instantiated as a singleton, you could do something like this:
#Pointcut("if()")
public static boolean isActive() {
return Aspects.aspectOf(PerformanceMonitorAspect.class).eventJournalService != null;
}
// ...
#AfterReturning(pointcut = "isActive() && anyPublicMethod() && inService() && #annotation(eventJournal) && args(entity,..)", returning = "id")
// ...
#AfterThrowing(pointcut = "isActive() && anyPublicMethod() && inService() && #annotation(eventJournal) && args(entity,..)", throwing="ex")
// ...

I can only guess:
The first thing is to have a separate Spring applicationContext-test.xml,
without component-scan;
In maven you can add a phase runtime excluding weaving jars for test.

Compile time weaving would inline the advice calls in the targeted methods identified by the pointcuts that you have. I personally feel that it is good to unit test with the compile time weaving in place, because at runtime your unit does include the class with the advice inlined?
The thought I have to not include advice would be to have two different compile targets, one with compile time weaving, and one without, you should be able to do this through maven profiles, a dev profile not weaving advice in and a prod profile to weave the aspects in.

You can write a method that returns if current execution was launched using JUnit framework.
The method can check stack trace with Thread.currentThread().getStackTrace() and search for MockitoJUnitRunner presence.
I tested this solution using a SpringJUnit4ClassRunner but I think could work with MockitoJUnitRunner.
Also, you can have got a static boolean field like:
private static boolean TEST_ENVIRONMENT = false;
In a class present in your project (not in your tests) and check the value in the control method instead of use stack trace.
When you run your tests, you can use #BeforeClass annotation to set TEST_ENVIRONMENT = true.
This solution only gives you a way to know if your code is running from a test or not.

Related

Is there a Maven plugin for checking annotations presence in code?

I need to ensure that, in specific classes (e.g. all classes extending some other class), fields annotated with e.g. #Deprecated are also annotated with #ThisOtherAnnotationMustBeHere.
#Deprecated
#ThisOtherAnnotationMustBeHere // this must be present if #Deprecated is also present; otherwise build should fail
private String field;
I need in general something to check for the presence of annotations.
I guess I could write a JUnit test for this using reflection, but I was wondering if there was a Maven solution to this.
Following #khmarbaise suggestion (thanks!) I've used archunit.org to write a unit test for this. In my case I needed to verify that join fields in JPA entities were annotated with a specific custom JsonAdapter
class CodeChecksTest {
#ArchTest
public static final ArchRule persistenceIdAnnotationRule = fields().that()
.areDeclaredInClassesThat().areAnnotatedWith(Entity.class).and()
.areAnnotatedWith(OneToOne.class).or()
.areAnnotatedWith(OneToMany.class).or()
.areAnnotatedWith(ManyToOne.class).or()
.areAnnotatedWith(ManyToMany.class)
.should(beAnnotatedForMyCustomAdapter());
private static ArchCondition<? super JavaField> beAnnotatedForMyCustomAdapter() {
return new ArchCondition<JavaField>("annotated with #JsonAdapter(MyCustomAdapter.class)") {
#Override
public void check(JavaField item, ConditionEvents events) {
final Optional<JsonAdapter> annotation = item.tryGetAnnotationOfType(JsonAdapter.class);
final boolean satisfied = annotation.isPresent() && annotation.get().value() == MyCustomAdapter.class;
// createMessage is a utility method
String message = createMessage(item,
(satisfied ? "is " : "is not ") + getDescription());
events.add(new SimpleConditionEvent(item, satisfied, message));
}
};
}
}

Spring AOP custom annotation, getting Null with annotation arguments

I am using this custom annotation for logging execution time, annotation could be present on method or class in which all public methods have it. Everything works fine, except in case of method level "LogExecutionTime logExecutionTime" comes null. This throws an NPE.
#Around("#annotation(logExecutionTime) || #within(logExecutionTime)")
public Object logExecutionTime(ProceedingJoinPoint joinPoint, LogExecutionTime logExecutionTime) throws Throwable {
final Logger logger = LoggerFactory.getLogger(joinPoint.getTarget().getClass());
final String name = joinPoint.toShortString();
final StopWatch stopWatch = new StopWatch(name);
stopWatch.start(name);
try {
return joinPoint.proceed();
} finally {
stopWatch.stop();
if (logExecutionTime.value()) {
logger.info(joinPoint.getSignature().getName() + ".time=", stopWatch.getTotalTimeSeconds());
}
}
}
if I reverse the order-
#Around("#within(logExecutionTime) || #annotation(logExecutionTime)")
the behavior reverses and I get a valid object at method level and null at class level annotated methods.
I have worked around this by having 2 explicit methods and separating the two-
#Around("#within(logExecutionTime)")
public Object logExecutionTimeClassLevel(ProceedingJoinPoint joinPoint, LogExecutionTime logExecutionTime) throws Throwable {
return logExecutionTimeMethodLevel(joinPoint, logExecutionTime);
}
#Around("#annotation(logExecutionTime)")
public Object logExecutionTimeMethodLevel(ProceedingJoinPoint joinPoint, LogExecutionTime logExecutionTime) throws Throwable {
final Logger logger = LoggerFactory.getLogger(joinPoint.getTarget().getClass());
final String name = joinPoint.toShortString();
final StopWatch stopWatch = new StopWatch(name);
stopWatch.start(name);
try {
return joinPoint.proceed();
} finally {
stopWatch.stop();
if (logExecutionTime.value()) {
logger.info(joinPoint.getSignature().getName() + ".time=", stopWatch.getTotalTimeMillis());
}
}
Was hoping to understand this behavior, when we use OR '||' with two pointcuts.
class level
#LogExecutionTime
#Component
public class CleanUpService implements ICleanUpService { ... }
method level
#Scheduled(fixedDelay = 100)
#LogExecutionTime(false)
public void processMessageQueue() { ... }
I came to run you example, and reproduce the same example as yours, when it come to runtime expression is same weird because when you specify the annotation on class level and you write this expression
#Around(" #within(logExecutionTime) || #annotation(logExecutionTime) ")
The point cut will evaluate to true for you class (event you annotation its available in joinPoint.getTarget().getClass().getAnnotations(), )
Now when it come to binding the variable the compiler check all your expressions that mean binding #within(logExecutionTime) to variable logExecutionTime and #annotation(logExecutionTime) to the same variable , if the method is not annotated it will ge null, => override the initial with, that cause all senarios you mention.
Try to put this expression #within(logExecutionTime) || #annotation(logExecutionTime) || #within(logExecutionTime)
and you'll get you variable not null which prove what i said, last #within(logExecutionTime) override what precedent
The key here is that the logic applied to select the point cut matching not the same when it come context-binding
Now when it come to AOP point-cut you must be careful and follow best practice as the spring team they mention here to avoid weird runtime results
Cheers
This cannot work, it does not even compile with the AspectJ compiler. Maybe in your IDE and with Spring AOP you do not see any warnings or errors, but I see:
ambiguous binding of parameter(s) logExecutionTime across '||' in pointcut
This means that it is not clear which annotation should be selected if e.g. both the class and the method contain an instance of that annotation. It is, as the error message said, ambiguous. But ambiguous parameter bindings across || are not permitted. They can also happen if you try to bind values from different "or" branches to a single parameter in an args() list.
I had the same problem. What you want is exactly same as Spring #Transcriptional behaves (I mean, class level or method level annotation with parameters). I used your solution but to get the class level parameter value (as the annotation object received null), I used reflection. I know it is a dirty solution! But I tried other solutions and couldn't find!
Her is the full code. This will call the advice code either the annotation is used on a class or a method. If the annotation is placed on both (class and method), the method takes the precedence.
#Aspect
#Configurable
#Component
public class DynamicValueAspect {
#Around(" #within(dynamicValueAnnotation) || #annotation(dynamicValueAnnotation))")
public Object process(ProceedingJoinPoint joinPoint, DynamicValue dynamicValueAnnotation) throws Throwable {
String annotationParameter;
if (dynamicValueAnnotation == null) { //class level annotation
annotationParameter = extractClassLevelAnnotationParameterValue(joinPoint);
} else {
annotationParameter = dynamicValueAnnotation.myAnnotationParameter();
}
System.out.println(" " + annotationParameter);//get the annotation parameter value
return joinPoint.proceed();
}
private String extractClassLevelAnnotationParameterValue(ProceedingJoinPoint joinPoint) {
Annotation[] classAnnotations = joinPoint.getTarget().getClass().getAnnotations();
for (Annotation annotation : classAnnotations) {
if (annotation.annotationType() == DynamicValue.class) {
return ((DynamicValue) annotation).myAnnotationParameter();
}
}
throw new RuntimeException("No DynamicValue value annotation was found");
}
}
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.METHOD, ElementType.TYPE})
public #interface DynamicValue {
String myAnnotationParameter();
}
Let's know if you got a cleaner solution!
The problem with your workaround appears when you annotate both a class and a method with the annotation, resulting in triggering both of them.
To prevent it declare the class level advice as:
#Around("!#annotation(LogExecutionTime) && #within(logExecutionTime)")

How to unit test a function that has a condition for the state of something?

I'm trying to write a unit test for a function that has a condition for wifi state. When the wifi state is disconnected, the function returns with false, but I want to simulate a scenario when the wifi state is connected. How do I go about it? Should I make a setter for the state variable? Isn't that a bad approach?
The function I want to test:
public boolean performSomething() {
if (WIFI_STATE != "connected") {
return false;
}else{
....
}
}
I want to test the else part of the function above, but as you may have guessed, the function executes the if condition, and returns false because WIFI_STATE is "disconnected"
Without seeing any of your code, make sure your class uses a WifiState constructor parameter or injection (or something similar). In your test you can then provide a mock for this object and set the state accordingly to your testing needs.
Two approaches that I can think of:
1: Define a sensor's state capturing class say WifiStatus, which your production code initializes automatically through a static initializer.
For testing, you can load a dummy WifiStatus class though a test initializer or change the value of WifiStatus through instrumentation.
2: Use WifiStatus as a interface and then mock it for dependency injection. This approach is more common.
public interface WifiStatus{
boolean isConnected();//
}
public class Performer{
WifiStatus wifiStatusProvider;
public Performer(WifiStatus stateProvider){
this.wifiStatusProvider = stateProvider;
}
public boolean performSomething() {
//if (WIFI_STATE != "connected") {
if (wifiStatusProvider.isConnected() != true) {
return false;
}else{
....
}
}
}
For test class, you use as follows:
public class PerformerTest{
#Test
public void verifyPerformSomething(){
WifiStatus dummyWifiStatus = mock(WifiStatus.class);
doReturn(true).when(dummyWifiStatus).isConnected();
new Performer(dummyWifiStatus).performSomething();
}
}
Assuming you are talking about "Local Unit Tests" and not "Instrumented Unit Tests", you can use Robolectric: http://robolectric.org
http://robolectric.org/javadoc/3.0/org/robolectric/shadows/ShadowWifiManager.html
https://github.com/robolectric/robolectric/blob/master/robolectric/src/test/java/org/robolectric/shadows/ShadowWifiManagerTest.java
Local Unit Tests:
https://developer.android.com/training/testing/unit-testing/instrumented-unit-tests.html
Instrumented Unit Tests:
https://developer.android.com/training/testing/unit-testing/instrumented-unit-tests.html
Well, for testing, you must be sure for your input how your output looks like because you need to compare them. Make the boolean value of wifi state is injected from outside, so you can mock it how you like. For example:
public boolean method(boolean wifiState){
return !wifiState;
}
So now you can be sure that if your parameter is false you will got true and vice versa so you can mock that param and make your tests.
Chiming in with a code-based answer based on john16384's answer. You mention that the state is set with other functions within the same class, assuming those methods are public I propose something like this (assuming WifiChecker is your class):
private WifiChecker wifiChecker;
#Before public void setUp() {
wifiChecker = new WifiChecker();
}
#Test public void testWifiConnected() {
wifiChecker.setConnected()
assertTrue(wifiChecker.performSomething());
}
#Test public void testWifiDisconnected() {
wifiChecker.setDisconnected()
assertFalse(wifiChecker.performSomething());
}

What is the difference between a Seam and a Mock?

Its being a few months since I am working with java legacy code, this are some of the things I am dealing with:
0% test coverage.
Huge functions in occasions I even saw some with more than 300 lines of code.
Lots of private methods and in occasions static methods.
Highly tight coupled code.
At the beginning I was very confused, I found difficult to use TDD in the legacy. After doing katas for weeks and practicing my unit testing and mocking skills, my fear has decreased and I feel a bit more confident. Recently I discovered a book called: working effectivelly with legacy, I didn't read it, I just had a look at the table of contents and I discovered something that is new for me, The Seams. Apparently this is very important when working in the legacy.
I think that this Seams could help me alot in breaking dependencies and make my code testeable so I can increase the code coverage and make my unit testing more precise.
But I have a lot of doubts:
Can somebody explain me the difference between a seam and a mock?
Do Seams, break TDD rules in what regards not touching production code, before is tested?
Could you show me some simple example that compares a Seam and a Mock?
Below I would like to paste an example I did today where I tried to break a dependency with the goal of making the code testeable and finally increasing test coverage. I would appreciate if you could comment a bit if you see some mistakes?
This is how the legacy code looked like at the beginning:
public class ABitOfLegacy
{
private String sampleTitle;
String output;
public void doSomeProcessing(HttpServletRequest request) {
String [] values = request.getParameterValues(sampleTitle);
if (values != null && values.length > 0)
{
output = sampleTitle + new Date().toString() + values[0];
}
}
}
If I just add a unit test that calls that method and asserts that variable output, has a certain value after the call,then I would be making a mistake, because I am not unit testing, I would be doing integration testing. So what I need to do, Is get rid of the dependency I have in the parameter. To do So, I replace the parameter with an interface:
public class ABitOfLegacy
{
private String sampleTitle;
String output;
public void doSomeProcessing(ParameterSource request) {
String [] values = request.getParameters(sampleTitle);
if (values != null && values.length > 0)
{
output = sampleTitle + new Date().toString() + values[0];
}
}
}
This is how the interface looks like:
public interface ParameterSource {
String[] getParameters(String name);
}
The next thing I do, is create my own implementation of that interface but I include the HttpServletRequest as a global variable and I implement the method of the interface using the method/s of HttpServletRequest:
public class HttpServletRequestParameterSource implements ParameterSource {
private HttpServletRequest request;
public HttpServletRequestParameterSource(HttpServletRequest request) {
this.request = request;
}
public String[] getParameters(String name) {
return request.getParameterValues(name);
}
}
Until this point, I think that all the modifications on the production code were safe.
Now I create the Seam in my test package. If I understood well, now I am able to safely change the behavoir of the Seam. This is how I do it:
public class FakeParameterSource implements ParameterSource {
public String[] values = {"ParamA","ParamB","ParamC"};
public String[] getParameters(String name) {
return values;
}
}
And the final step, would be to get support from the Seam, to test the original behavoir of the method.
import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.*;
import static org.mockito.Mockito.*;
import code.ABitOfLegacyRefactored;
import static org.hamcrest.Matchers.*;
public class ABitOfLegacySpecification {
private ABitOfLegacy aBitOfLegacy;
private String EMPTY = null;
#Before
public void initialize() {
aBitOfLegacy = new ABitOfLegacy();
}
#Test
public void
the_output_gets_populated_when_the_request_is_not_empty
() {
FakeParameterSource fakeParameterSource = new FakeParameterSource();
aBitOfLegacy.doSomeProcessing(fakeParameterSource);
assertThat(aBitOfLegacy.output,not(EMPTY));
}
#Test(expected=NullPointerException.class)
public void
should_throw_an_exception_if_the_request_is_null
() {
aBitOfLegacy.doSomeProcessing(null);
}
}
This will give me 100% test coverage.
I appreciate your thoughts:
Did I break the dependency correctly?
Are the unit tests missing something?
What could be done better?
Is this example good enough to help me understand the difference between a Seam and a Mock?
How could a mock help me here if I don't use the Seam?
A seam is a place in the code that you can insert a modification in behavior. You created a seam when you setup injection of your dependency.
One way to take advantage of a seam is to insert some sort of fake. Fake's can be hand-rolled, as in your example, or be created with a tool, like Mockito.
So, a mock is a type of fake, and a fake is often used by taking advantage of a Seam.
As for your tests and the way you broke the dependency, that's pretty much how I would have done it.
Seams
A seam is a place that allows you to modify the behavior without modifying the code.
In your example, the following is an example of an Object seam (if i'm not mistaken). It allows you to pass in a different object without having to change the code. hence it is a type of seam.
public void doSomeProcessing(ParameterSource request) {..}
By making the parameter an abstract type (instead of a concrete class), you have introduced a seam. The seam now allows you to modify the behavior of the method without editing its code - i.e. at the place of invokation, I can pass in a different object and make the method do something else.
Mocks
Now instead of creating your custom fake (creating a subtype of the interface), you could using a Mock framework to do something like this
Mocks also support asserting whether specific methods were called on it, argument matching and other nifty functionality to be consumed by tests. Less test code to maintain. Mocks are primarily used to assert that a specific call is being made to a dependency. In your example, you seem to be in need of a Stub, you just want to return a canned value.
Pardon my rusty JMock..
#Test
public void
the_output_does_not_get_populated_when_the_request_is_empty
() {
Mockery context = new Mockery();
final ParameterSource mockSource = context.mock(ParameterSource.class)
context.checking(new Expectations(){{
oneOf(mockSource).getParameters();
will(returnValue(new string[]{"ParamA","ParamB","ParamC"} );
}});
aBitOfLegacy.populate(mockSource);
assertThat(aBitOfLegacy.output,not(EMPTY));
}
in .Net
var mockSource = new Mock<ParameterSource>();
mockSource.Setup(src => src.GetParameters())
.Returns(new []{"ParamA","ParamB","ParamC"});

Mark unit test as an expected failure in JUnit

How can I mark a test as an expected failure in JUnit 4?
In this case I want to continue to run this test until something is patched upstream. Ignoring the test goes a little too far, as then I might forget about it. I may be able to add an #expected annotation and catch the exception thrown by assertThat, but that also seems to lie about the expected behavior.
Here's what my current test looks like:
#Test
public void unmarshalledDocumentHasExpectedValue()
{
doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
final ST title = doc.getTitle();
assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}
That assert should succeed, but because of an upstream bug it doesn't. Yet, that test is correct; it should succeed. Virtually all the alternatives that I've found are misleading. Right now I think #Ignore("This test should pass once fixed upstream") is my best bet, but I still have to remember to come back to it. I'd prefer that the test run.
In Python I can use the expectedFailure decorator:
class ExpectedFailureTestCase(unittest.TestCase):
#unittest.expectedFailure
def test_fail(self):
self.assertEqual(1, 0, "broken")
With Qt's QTestLib in C++, you can use QEXPECT_FAIL:
QEXPECT_FAIL("", "Will be fixed next version", Continue);
QCOMPARE(i, 42);
In both cases above, the unit test runs which is what I'm hoping to have happen. Am I missing something in JUnit?
I'm not quite getting the specifics of your scenario, but here's how I generally test for expected failure:
The slick new way:
#Test(expected=NullPointerException.class)
public void expectedFailure() {
Object o = null;
o.toString();
}
for older versions of JUnit:
public void testExpectedFailure() {
try {
Object o = null;
o.toString();
fail("shouldn't get here");
}
catch (NullPointerException e) {
// expected
}
}
If you have a bunch of things that you want to ensure throw an exception, you may also want to use this second technique inside a loop rather than creating a separate test method for each case. If you were just to loop through a bunch of cases in a single method using expected, the first one to throw an exception would end the test, and the subsequent cases wouldn't get checked.
What about explicitly expecting an AssertionError?
#Test(expected = AssertionError.class)
public void unmarshalledDocumentHasExpectedValue() {
// ...
}
If you're reasonably confident that only the JUnit machinery within the test would raise AssertionError, this seems as self-documenting as anything.
You'd still run the risk of forgetting about such a test. I wouldn't let such tests into version control for long, if ever.
I'm assuming here that you want the test to pass if your assert fails, but if the assert succeeds, then the test should pass as well.
The easiest way to do this is to use a TestRule. TestRule gives the opportunity to execute code before and after a test method is run. Here is an example:
public class ExpectedFailureTest {
public class ExpectedFailure implements TestRule {
public Statement apply(Statement base, Description description) {
return statement(base, description);
}
private Statement statement(final Statement base, final Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable e) {
if (description.getAnnotation(Deprecated.class) != null) {
// you can do whatever you like here.
System.err.println("test failed, but that's ok:");
} else {
throw e;
}
}
}
};
}
}
#Rule public ExpectedFailure expectedFailure = new ExpectedFailure();
// actually fails, but we catch the exception and make the test pass.
#Deprecated
#Test public void testExpectedFailure() {
Object o = null;
o.equals("foo");
}
// fails
#Test public void testExpectedFailure2() {
Object o = null;
o.equals("foo");
}
}
First, note that the first method is marked as #Deprecated. I'm using this as a marker for the method for which I want to ignore any assertion failures. You can do whatever you like to identify the methods, this is just an example.
Next, in the ExpectedFailure#apply(), when I do the base.evaluate(), I'm catching any Throwable (which includes AssertionError) and if the method is marked with the annotation #Deprecated, I ignore the error. You can perform whatever logic you like to decide whether you should ignore the error or not, based on version number, some text, etc. You can also pass a dynamically determined flag into ExpectedFailure to allow it to fail for certain version numbers:
public void unmarshalledDocumentHasExpectedValue() {
doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
expectedFailure.setExpectedFailure(doc.getVersionNumber() < 3000);
final ST title = doc.getTitle();
assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}
For further examples, see ExternalResource, and ExpectedException
Ignoring an expected failure test rather than passing it
If you want to mark you tests as Ignored rather than Success, it becomes a bit more complex, because tests are ignored before they are executed, so you have to retrospectively mark a test as ignored, which would involve constructing your own Runner. To give you a start, see my answer to How to define JUnit method rule in a suite?. Or ask another question.
One option is mark the test as #Ignore and put text in there that is a bug perhaps and awaiting a fix. That way it won't run. It will then become skipped. You could also make use of the extensions to suit your need in a potentially different way.
I've taken Matthew's answer a step further and actually implemented an #Optional annotation you could use instead of the #Deprecated marker annotation he mentions in his answer. Although simple, I'll share the code with you, maybe it's of help for someone:
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface Optional {
/**
* Specify a Throwable, to cause a test method to succeed even if an exception
* of the specified class is thrown by the method.
*/
Class<? extends Throwable>[] exception();
}
With a simple alteration of Matt's ExpectedFailure class:
public class ExpectedFailure implements TestRule {
#Override
public Statement apply(final Statement base, final Description description) {
return statement(base, description);
}
private Statement statement(final Statement base, final Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable e) {
// check for certain exception types
Optional annon = description.getAnnotation(Optional.class);
if (annon != null && ArrayUtils.contains(annon.exception(), e.getClass())) {
// ok
} else {
throw e;
}
}
}
};
}
}
You can now annotate your test method with #Optional and it will not fail, even if the given type of exception is raised (provide one or more types you would like the test method to pass):
public class ExpectedFailureTest {
#Rule public ExpectedFailure expectedFailure = new ExpectedFailure();
// actually fails, but we catch the exception and make the test pass.
#Optional(exception = NullPointerException.class)
#Test public void testExpectedFailure() {
Object o = null;
o.equals("foo");
}
}
[UPDATE]
You could also rewrite your tests using JUnit's org.junit.Assume instead of the tradtional org.junit.Assert, if you want your tests to pass even if the assumption does not hold.
From Assume's JavaDoc:
A set of methods useful for stating assumptions about the conditions in which a test is meaningful.A failed assumption does not mean the code is broken, but that the test provides no useful information. The default JUnit runner treats tests with failing assumptions as ignored.
Assume is available since JUnit 4.4
Use mocked upstream class if possible. Stub it with correct result. Optionally, replace mock with real object after bug is fixed.

Categories