I need to simulate a test scenario in which I call the getBytes() method of a String object and I get an UnsupportedEncodingException.
I have tried to achieve that using the following code:
String nonEncodedString = mock(String.class);
when(nonEncodedString.getBytes(anyString())).thenThrow(new UnsupportedEncodingException("Parsing error."));
The problem is that when I run my test case I get a MockitoException that says that I can't mock a java.lang.String class.
Is there a way to mock a String object using mockito or, alternatively, a way to make my String object throw an UnsupportedEncodingException when I call the getBytes method?
Here are more details to illustrate the problem:
This is the class that I want to test:
public final class A {
public static String f(String str){
try {
return new String(str.getBytes("UTF-8"));
} catch (UnsupportedEncodingException e) {
// This is the catch block that I want to exercise.
...
}
}
}
This is my testing class (I'm using JUnit 4 and mockito):
public class TestA {
#Test(expected=UnsupportedEncodingException.class)
public void test(){
String aString = mock(String.class);
when(nonEncodedString.getBytes(anyString())).thenThrow(new UnsupportedEncodingException("Parsing error."));
A.f(aString);
}
}
The problem is the String class in Java is marked as final, so you cannot mock is using traditional mocking frameworks. According to the Mockito FAQ, this is a limitation of that framework as well.
How about just creating a String with a bad encoding name? See
public String(byte bytes[], int offset, int length, String charsetName)
Mocking String is almost certainly a bad idea.
If all you are going to do in your catch block is throw a runtime exception then you can save yourself some typing by just using a Charset object to specify your character set name.
public final class A{
public static String f(String str){
return new String(str.getBytes(Charset.forName("UTF-8")));
}
}
This way you aren't catching an exception that will never happen just because the compiler tells you to.
As others have indicated, you can't use Mockito to mock a final class. However, the more important point is that the test isn't especially useful because it's just demonstrating that String.getBytes() can throw an exception, which it can obviously do. If you feel strongly about testing this functionality, I guess you could add a parameter for the encoding to f() and send a bad value into the test.
Also, you are causing the same problem for the caller of A.f() because A is final and f() is static.
This article might be useful in convincing your coworkers to be less dogmatic about 100% code coverage: How to fail with 100% test coverage.
From its documentation, JDave can't remove "final" modifiers from classes loaded by the bootstrap classloader. That includes all JRE classes (from java.lang, java.util, etc.).
A tool that does let you mock anything is JMockit.
With JMockit, your test can be written as:
import java.io.*;
import org.junit.*;
import mockit.*;
public final class ATest
{
#Test(expected = UnsupportedOperationException.class)
public void test() throws Exception
{
new Expectations()
{
#Mocked("getBytes")
String aString;
{
aString.getBytes(anyString);
result = new UnsupportedEncodingException("Parsing error.");
}
};
A.f("test");
}
}
assuming that the complete "A" class is:
import java.io.*;
public final class A
{
public static String f(String str)
{
try {
return new String(str.getBytes("UTF-8"));
}
catch (UnsupportedEncodingException e) {
throw new UnsupportedOperationException(e);
}
}
}
I actually executed this test in my machine. (Notice I wrapped the original checked exception in a runtime exception.)
I used partial mocking through #Mocked("getBytes") to prevent JMockit from mocking everything in the java.lang.String class (just imagine what that could cause).
Now, this test really is unnecessary, because "UTF-8" is a standard charset, required to be supported in all JREs. Therefore, in a production environment the catch block will never be executed.
The "need" or desire to cover the catch block is still valid, though. So, how to get rid of the test without reducing the coverage percentage? Here is my idea: insert a line with assert false; as the first statement inside the catch block, and have the Code Coverage tool ignore the whole catch block when reporting coverage measures. This is one of my "TODO items" for JMockit Coverage. 8^)
Mockito can't mock final classes. JMock, combined with a library from JDave can. Here are instructions.
JMock doesn't do anything special for final classes other than rely on the JDave library to unfinalize everything in the JVM, so you could experiment with using JDave's unfinalizer and see if Mockito will then mock it.
You can also use PowerMock's Mockito extension to mock final classes/methods even in system classes such as String. However I would also advice against mocking getBytes in this case and rather try to setup your expectation so that a real is String populated with the expected data is used instead.
You will be testing code that can never be executed. UTF-8 support is required to be in every Java VM, see http://java.sun.com/javase/6/docs/api/java/nio/charset/Charset.html
It is a project requirement that the unit tests coverage percentage must but higher than a given value. To achieve such percentage of coverage the tests must cover the catch block relative to the UnsupportedEncodingException.
What is that given coverage target? Some people would say that shooting for 100% coverage isn't always a good idea.
Besides, that's no way to test whether or not a catch block was exercised. The right way is to write a method that causes the exception to be thrown and make observation of the exception being thrown the success criterion. You do this with JUnit's #Test annotation by adding the "expected" value:
#Test(expected=IndexOutOfBoundsException.class) public void outOfBounds() {
new ArrayList<Object>().get(1);
}
Have you tried passing an invalid charsetName to getBytes(String)?
You could implement a helper method to get the charsetName, and override that method within your test to a nonsense value.
Perhaps A.f(String) should be A.f(CharSequence) instead. You can mock a CharSequence.
If you can use JMockit, look at Rogério answer.
If and only if your goal is to get code coverage but not actually simulate what missing UTF-8 would look like at runtime you can do the following (and that you can't or don't want to use JMockit):
public static String f(String str){
return f(str, "UTF-8");
}
// package private for example
static String f(String str, String charsetName){
try {
return new String(str.getBytes(charsetName));
} catch (UnsupportedEncodingException e) {
throw new IllegalArgumentException("Unsupported encoding: " + charsetName, e);
}
}
public class TestA {
#Test(expected=IllegalArgumentException.class)
public void testInvalid(){
A.f(str, "This is not the encoding you are looking for!");
}
#Test
public void testNormal(){
// TODO do the normal tests with the method taking only 1 parameter
}
}
You can change your method to take the interface CharSequence:
public final class A {
public static String f(CharSequence str){
try {
return new String(str.getBytes("UTF-8"));
} catch (UnsupportedEncodingException e) {
// This is the catch block that I want to exercise.
...
}
}
}
That way, you can still pass in String, but you can mock any way you like.
instead of mocking string inject string value at setup method as below
public void setup() throws IllegalAccessException {
FieldUtils.writeField(yourTestClass, "stringVariableName", "value", true);
}
If you have a block of code that can never actually be run, and a managerial requirement to have 100% test coverage, then something's going to have to change.
What you could do is make the character encoding a member variable, and add a package-private constructor to your class that lets you pass it in. In your unit test, you could call the new constructor, with a nonsense value for the character encoding.
Related
I am new to writing tests in java, and seem to be unable to test if a method of a class is called.
I am sending metrics to datadog, and want to test in the code if a function of another class was called.
It says I need to mock first, but I couldn't get it to work.
MetricRecorder.java
import com.timgroup.statsd.StatsDClient;
import com.timgroup.statsd.NonBlockingStatsDClient;
import com.google.common.base.Preconditions;
public class MetricRecorder {
private final String namespace;
private final static StatsDClient metrics = new NonBlockingStatsDClient(
"my.prefix",
"localhost",
8125,
new String[] {"tag:value"}
);
public MetricRecorder(String namespace) {
Preconditions.checkNotNull(namespace);
this.namespace = namespace;
}
public void inc(String metricName) {
this.inc(metricName, 1);
}
public void inc(final String metricName, final long value) {
Preconditions.checkNotNull(metricName);
try {
metrics.recordHistogramValue(MetricRecorder.name(namespace, metricName), value);
} catch (Exception e) {
logger.warn("Unable to record metric {} due to :", metricName, e);
}
}
...
}
MetricRecorderTest.java
public class MetricsRecorderTest {
#Test
public void metricsRecorderTest() {
MetricRecorder recorder = new MetricRecorder("dev");
recorder.inc("foo", 1);
verify(recorder.metrics, times(1)).recordHistogramValue(eq("dev.foo"), 1);
}
}
When I run the test I get this => org.mockito.exceptions.misusing.NotAMockException:
Argument passed to verify() is of type NonBlockingStatsDClient and is not a mock!
Any idea of how I should be testing if recordHistogramValue was called, and if so with what arguments?
Since it looks like StatsDClient is an interface of some kind, it would make your testing effort easier to simply inject this dependency into your object. Even if you're not using an IoC container like Spring or Guice, you can still somewhat control this simply by passing an instance of it in through the constructor.
public MetricRecorder(String namespace, StatsDClient client) {
Preconditions.checkNotNull(namespace);
Preconditions.checkNotNull(client);
this.namespace = namespace;
this.client = client;
}
This will make your testing simpler since all you realistically need to do is mock the object passed in during test.
Right now, the reason it's failing is because you're newing up the instance, and Mockito (in this current configuration) isn't equipped to mock the newed instance. In all honesty, this set up will make testing simpler to conduct, and you should only need your client configured in one area.
#RunWith(MockitoJUnitRunner.class)
public class MetricsRecorderTest {
#Test
public void metricsRecorderTest() {
StatsDClient dClientMock = Mockito.mock(StatsDClient.class);
MetricRecorder recorder = new MetricRecorder("dev", dClientMock);
recorder.inc("foo", 1);
verify(recorder.metrics).recordHistogramValue(eq("dev.foo"), 1);
}
}
You are getting things wrong here. You don't use a mocking framework to test your "class under test".
You use the mocking framework to create mocked objects; which you then pass to your "class under test" within a test case. Then your "code under test" calls methods on the mocked object; and by controlling returned values (or by verifying what happens to your mock); that is how you write your testcases.
So, your testcase for a MetricRecorder doesn't mock a MetricRecorder; it should mock the StatsDClient class; and as Makoto suggests; use dependency injection to put an object of that class into MetricRecorder.
Besides: basically writing "test-able" code is something that needs to be practiced. I wholeheartedly recommend you to watch these videos if you are serious about getting in this business. All of them; really (worth each second!).
Trying to get Mockito and PowerMock to behave, but I'm getting an UnfinishedStubbingException when trying to run this code:
#RunWith(PowerMockRunner.class)
#PrepareForTest(FileIOHelper.class)
public class FileIOHelperTest {
#Test
public void testIOExceptionOnWrite() {
PowerMockito.mockStatic(FileIOHelper.class);
PowerMockito.doThrow(new IOException()).when(FileIOHelper.class);
PowerMockito.verifyStatic();
FileIOHelper.write(Mockito.anyString(), Mockito.anyString());
}
#After
public void validate() {
Mockito.validateMockitoUsage();
}
}
and this IO class
public final class FileIOHelper {
public static void write(final String file, String message, final boolean appendNewLine) {
if(checkArgs(file, message)) {
final Path path = Paths.get(file);
StandardOpenOption mode = StandardOpenOption.APPEND;
if(Files.notExists(path)) {
mode = StandardOpenOption.CREATE_NEW;
}
if(appendNewLine) {
message += System.getProperty("line.separator");
}
try {
Files.write(path, message.getBytes(), mode);
} catch(IOException e) {
handleException(e, "Problem writing to " + file);
}
}
}
private static boolean checkArgs(final String... args) {
if(args != null && args.length > 0) {
for(final String arg : args) {
if(arg == null || arg.isEmpty()) {
return false;
}
}
}
return true;
}
private static void handleException(final IOException e, final String errorMsg) {
handleException(e, errorMsg, true);
}
private static void handleException(final IOException e, final String errorMsg, final boolean printStace) {
checkArgs(errorMsg);
System.err.println(errorMsg);
System.err.println(e.getMessage());
if(printStace) {
e.printStackTrace();
}
}
}
What I want to do is somehow trigger the IOException so handleException can be tested. Why one might ask? I'm looking at my Jacoco report and I see this:
I've looked at:
How to mock a void static method to throw exception with Powermock?
Powermock/mockito does not throw exception when told to
PowerMockito mock static method which throws exception
https://github.com/jayway/powermock/wiki/MockitoUsage#how-to-stub-void-static-method-to-throw-exception
http://www.johnmullins.co/blog/2015/02/15/beginners-guide-to-using-mockito-and-powermockito-to-unit-test-java/
and I'm still completely lost. I have no idea if I need to trigger the IOException or if I can somehow verify the output of handleException without doThrow. Someone, help!
Error log:
org.mockito.exceptions.misusing.UnfinishedStubbingException:
Unfinished stubbing detected here:
-> at FileIOHelperTest.testIOExceptionOnWrite(FileIOHelperTest.java:8) // doThrow line
E.g. thenReturn() may be missing.
Examples of correct stubbing:
when(mock.isOk()).thenReturn(true);
when(mock.isOk()).thenThrow(exception);
doThrow(exception).when(mock).someVoidMethod();
Hints:
1. missing thenReturn()
2. you are trying to stub a final method, you naughty developer!
3: you are stubbing the behaviour of another mock inside before 'thenReturn' instruction if completed
My recommendation: forget about using PowerMock.
If you have to mock static methods, then build your own little wrapper class around that. Then, for testing, your wrapper can return something you control; and for production usage; your wrapper just calls the static method.
PowerMock looks like the solution to many problems; but rather sooner than later, it will be the cause of much more problems. It breaks coverage, it makes it harder to change the underlying JVM, and so on.
Seriously: if your design can only be tested with PowerMock, this is very often a clear indication that your design is bad. So: focus on reworking your code under test; instead of investing time into a tool like PowerMock that does more harm than good.
I have spent countless hours trying to resolve PowerMock problems; and since I started to instead write "better to test" production code ... I have written hundreds or thousands of tests without ever needing PowerMock again.
In your case: start by avoiding static all over the place. Basically you achieve that by (worst case) pulling little wrapper classes around the static calls you have to make. For testing, you can mock the wrapper object; and in production code, you use dependency injection to provide a (singleton/enum) wrapper object that simply makes the static call.
First of all, IOException is checked exception - it should be declared with throws in the method signature. But your method FileIOHelper.write does not have such. This may be the reason of the UnsutisfiedStubbingException.
I do not understand, what your are trying to test: if the FileIOHelper is a mock - handleException will be never called, since it is called by the real write method, not by mocked.
First, you have to mock the class 'Files', not 'FileIOHelper'. FileIOHelper is the tested class. Second, you didn't specified which method should throw IOException. The unit test method should be as follows (supposing the tested method catches and manage the IOException):
#RunWith(PowerMockRunner.class)
#PrepareForTest(Files.class)
public class FileIOHelperTest {
#Test
public void testIOExceptionOnWrite() {
PowerMockito.mockStatic(Files.class);
PowerMockito.doThrow(new IOException()).when(Files.class);
Files.write("path", "message", true);
FileIOHelper.write("path", "message", true);
PowerMockito.verifyStatic();
Files.write("path", "message", true);
}
}
I need to write unit tests against a pre-existing code base using TestNG, Mockito and now PowerMockito, to test private and static methods more easily. I am currently trying to write a test against a private void method in a class that we are testing, but am unable to figure it out. In the normal PowerMock API there are methods called replayAll(), verifyAll(), and expectLastCalled(), which are suitable for most purposes. I just can't find good docs that explain how to do it the PowerMockito way. Any explanations or insights on this would be much appreciated.
Method to test:
private void pVMethod(Type param) throws Exception {
param.setA(StaticClass.methodA().toString());
param.setB(StaticClass.methodB().toString());
// getMemo(String label) is in a public class in same package
param.setC(getMemo("memo.provider"));
param.setD(getMemo("memo.item"));
try {
param.setTimestamp(DataTypeFactory.newInstance().newXMLGregorianCalendar(newjava.util.GregorianCalendar()));
} catch (SomeException e) {
...
throw new Exception();
}
}
test attempt:
#Test(expectedExceptions = Exception.class)
public void pVMethod() throws Exception {
TestClass testMock = mock(TestClass.class);
Exception exception = mock(Exception.class);
// StaticClass staticClassMock = mock(StaticClass.class); ??
mockStatic(StaticClass.class);
// when(..) and thenReturn(..) are static imports from PowerMockito library
when(StaticClass.methodA()).thenReturn("stubStringA");
when(StaticClass.methodB()).thenReturn("stubStringB");
doThrow(exception).when(param).setTimestamp(Mockito.any(XMLGregorianCalendar.class));
// Docs say method name can be inferred via reflection
Whitebox.invokeMethod(tested, event);
// this is where things are hairy. testedSpy is defined at the top level
verifyPrivate(testedSpy).pVMethod(testMock);
}
Ok, here is the answer:
In PowerMockito, if you want to verify the behavior of a private void method you use the verifyPrivate() method, but you have to do it like this:
verifyPrivate(tested).invoke("privateMethodName", argument);
Notice the use of the invoke method, missing from the last line of the OP.
NOTE: You do not have to use a doNothing().when(mock.privateMethod()) statement, because void methods in mock objects don't do anything by default.
example taken from here
I have a class with a few methods advised through an input validation aspect (validates whether all input parameters are not-null/non-empty strings).
I am facing an issue while writing test case for them and want to verify if this is indeed a bad design issue.
Here's a very simplified version of my class:
public class A {
public String one(String word) {
// Some actions
String val = two(word2);
// Some more actions
}
protected String two(String word) {
// Some actions
}
}
Now while writing test cases for one() I use Mockito and want to mock calls to two(). So I use:
#Spy
A a;
#Test
void test() {
doReturn("Bye").when(A).two(Mockito.anyString());
a.one("hello");
// Some validations
}
This test fails as the: doReturn() line fails with input being empty for two().
Should I not mock two() or can I make this work somehow?
Edit:
Adding a more specific example related to the two methods being present in two different classes as requested:
Create a page through a WebService. This builds a putRequest, executes it and returns a response.
public class AUtility implements BaseUtility {
public Response create(Params params) {
try {
PutMethod putRequest = buildPUTRequest(params.getAttr1(), params.getAttr2());
return Utils.buildResponse(client.executeMethod(putRequest),
params.getAttr3(),
params.getAttr4());
} catch (Exception e) {
throw new AppException(e);
}
}
}
The put request marshals the data into a file to write it through the HttpClient
private PutMethod buildPUTRequest(final String url, final Object obj) throws IOException, JAXBException {
// Create a temp file to store the stream
File tempFile = File.createTempFile(APPLICATION_LABEL, XML_LABEL);
decoder.marshal(obj, tempFile);
// Build the put method
return putMethod;
}
XMLMarshaller
public interface XMLDecoder implement Decoder {
public void marshal(Object obj, File tempFile) throws IOException, JAXBException {
// Perform marshalling operations
}
}
The test fails on line2 with the inputs being null.
#Test
public void createPageParamsHttpException() throws HttpException, IOException, JAXBException {
expectedException.expect(AppException.class);
doNothing().when(decoder).marshal(Mockito.anyString(), Mockito.any(File.class));
doThrow(HttpException.class).when(client).executeMethod(Mockito.any(HttpMethod.class));
Params params = new Params(new Application(),
APPLICATION_URL_LABEL,
SITE_NAME_LABEL,
URL_WITHOUT_HTTP_N_HTML);
utility.createPage(params);
}
Any idea how should I proceed for the same?
You don't want to do this.
You are inherently changing the behavior of the class. If you change what two() does, how do you know that one() will do what it's supposed to do in production?
If you truly want to do this, you should extract the behavior of two() into another top level class, and then inject the dependency into A. Then you can mock this dependency and you don't have to worry about going to the trouble of creating a partial mock for A.
In a similar vein, if you must keep two in the same class (because it's behavior is part of the same responsibility that is assigned to A - see the Single Responsibility Principle - why is it public?
The reason you are having trouble is because you are violating the SRP, see my note above. You said this:
This builds a putRequest, executes it and returns a response.
You should not be trying to test the behavior of all three of those things at the same time. Ultimately, this method does not really do anything. The buildPUTRequest method does, and shouldn't be in a class called AUtility, it should be in a class RequestFactory. Then, you would want to test the Utils.buildResponse method, except that shouldn't be in a class called Utils, it should be in a class called Responder or something... and this method ABSOLUTELY should not be static.
Work on naming your classes better things, and if you can't come up with a good name, that means the class probably does too much and should be refactored. And a method that wraps the work in two other methods doesn't need to be unit tested. Integration tested, perhaps, but that's another story.
How can I mark a test as an expected failure in JUnit 4?
In this case I want to continue to run this test until something is patched upstream. Ignoring the test goes a little too far, as then I might forget about it. I may be able to add an #expected annotation and catch the exception thrown by assertThat, but that also seems to lie about the expected behavior.
Here's what my current test looks like:
#Test
public void unmarshalledDocumentHasExpectedValue()
{
doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
final ST title = doc.getTitle();
assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}
That assert should succeed, but because of an upstream bug it doesn't. Yet, that test is correct; it should succeed. Virtually all the alternatives that I've found are misleading. Right now I think #Ignore("This test should pass once fixed upstream") is my best bet, but I still have to remember to come back to it. I'd prefer that the test run.
In Python I can use the expectedFailure decorator:
class ExpectedFailureTestCase(unittest.TestCase):
#unittest.expectedFailure
def test_fail(self):
self.assertEqual(1, 0, "broken")
With Qt's QTestLib in C++, you can use QEXPECT_FAIL:
QEXPECT_FAIL("", "Will be fixed next version", Continue);
QCOMPARE(i, 42);
In both cases above, the unit test runs which is what I'm hoping to have happen. Am I missing something in JUnit?
I'm not quite getting the specifics of your scenario, but here's how I generally test for expected failure:
The slick new way:
#Test(expected=NullPointerException.class)
public void expectedFailure() {
Object o = null;
o.toString();
}
for older versions of JUnit:
public void testExpectedFailure() {
try {
Object o = null;
o.toString();
fail("shouldn't get here");
}
catch (NullPointerException e) {
// expected
}
}
If you have a bunch of things that you want to ensure throw an exception, you may also want to use this second technique inside a loop rather than creating a separate test method for each case. If you were just to loop through a bunch of cases in a single method using expected, the first one to throw an exception would end the test, and the subsequent cases wouldn't get checked.
What about explicitly expecting an AssertionError?
#Test(expected = AssertionError.class)
public void unmarshalledDocumentHasExpectedValue() {
// ...
}
If you're reasonably confident that only the JUnit machinery within the test would raise AssertionError, this seems as self-documenting as anything.
You'd still run the risk of forgetting about such a test. I wouldn't let such tests into version control for long, if ever.
I'm assuming here that you want the test to pass if your assert fails, but if the assert succeeds, then the test should pass as well.
The easiest way to do this is to use a TestRule. TestRule gives the opportunity to execute code before and after a test method is run. Here is an example:
public class ExpectedFailureTest {
public class ExpectedFailure implements TestRule {
public Statement apply(Statement base, Description description) {
return statement(base, description);
}
private Statement statement(final Statement base, final Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable e) {
if (description.getAnnotation(Deprecated.class) != null) {
// you can do whatever you like here.
System.err.println("test failed, but that's ok:");
} else {
throw e;
}
}
}
};
}
}
#Rule public ExpectedFailure expectedFailure = new ExpectedFailure();
// actually fails, but we catch the exception and make the test pass.
#Deprecated
#Test public void testExpectedFailure() {
Object o = null;
o.equals("foo");
}
// fails
#Test public void testExpectedFailure2() {
Object o = null;
o.equals("foo");
}
}
First, note that the first method is marked as #Deprecated. I'm using this as a marker for the method for which I want to ignore any assertion failures. You can do whatever you like to identify the methods, this is just an example.
Next, in the ExpectedFailure#apply(), when I do the base.evaluate(), I'm catching any Throwable (which includes AssertionError) and if the method is marked with the annotation #Deprecated, I ignore the error. You can perform whatever logic you like to decide whether you should ignore the error or not, based on version number, some text, etc. You can also pass a dynamically determined flag into ExpectedFailure to allow it to fail for certain version numbers:
public void unmarshalledDocumentHasExpectedValue() {
doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
expectedFailure.setExpectedFailure(doc.getVersionNumber() < 3000);
final ST title = doc.getTitle();
assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}
For further examples, see ExternalResource, and ExpectedException
Ignoring an expected failure test rather than passing it
If you want to mark you tests as Ignored rather than Success, it becomes a bit more complex, because tests are ignored before they are executed, so you have to retrospectively mark a test as ignored, which would involve constructing your own Runner. To give you a start, see my answer to How to define JUnit method rule in a suite?. Or ask another question.
One option is mark the test as #Ignore and put text in there that is a bug perhaps and awaiting a fix. That way it won't run. It will then become skipped. You could also make use of the extensions to suit your need in a potentially different way.
I've taken Matthew's answer a step further and actually implemented an #Optional annotation you could use instead of the #Deprecated marker annotation he mentions in his answer. Although simple, I'll share the code with you, maybe it's of help for someone:
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface Optional {
/**
* Specify a Throwable, to cause a test method to succeed even if an exception
* of the specified class is thrown by the method.
*/
Class<? extends Throwable>[] exception();
}
With a simple alteration of Matt's ExpectedFailure class:
public class ExpectedFailure implements TestRule {
#Override
public Statement apply(final Statement base, final Description description) {
return statement(base, description);
}
private Statement statement(final Statement base, final Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable e) {
// check for certain exception types
Optional annon = description.getAnnotation(Optional.class);
if (annon != null && ArrayUtils.contains(annon.exception(), e.getClass())) {
// ok
} else {
throw e;
}
}
}
};
}
}
You can now annotate your test method with #Optional and it will not fail, even if the given type of exception is raised (provide one or more types you would like the test method to pass):
public class ExpectedFailureTest {
#Rule public ExpectedFailure expectedFailure = new ExpectedFailure();
// actually fails, but we catch the exception and make the test pass.
#Optional(exception = NullPointerException.class)
#Test public void testExpectedFailure() {
Object o = null;
o.equals("foo");
}
}
[UPDATE]
You could also rewrite your tests using JUnit's org.junit.Assume instead of the tradtional org.junit.Assert, if you want your tests to pass even if the assumption does not hold.
From Assume's JavaDoc:
A set of methods useful for stating assumptions about the conditions in which a test is meaningful.A failed assumption does not mean the code is broken, but that the test provides no useful information. The default JUnit runner treats tests with failing assumptions as ignored.
Assume is available since JUnit 4.4
Use mocked upstream class if possible. Stub it with correct result. Optionally, replace mock with real object after bug is fixed.