I have a simple JUnit test. It fails, unless I have called the relevant function previously.
FAILS:
#Test public void isTotalSuccess(){
writer.calculateStats();
Assert.assertFalse(writer.isTotalSuccess());
}
PASSES and displays "TotalSuccess is false":
#Test public void isTotalSuccess(){
writer.calculateStats();
// One additional line here
System.out.println("TotalSuccess is " + writer.isTotalSuccess());
Assert.assertFalse(writer.isTotalSuccess());
}
I don't understand why this is happening, since usually nested methods/functions are evaluated first, before their enclosing function.
Please explain why the test fails unless the method being tested has already been called once. Is this behaviour specific to JUnit, or have I missed something really obvious?
More information:
These tests are part of WriterTest, which is testing Writer
writer is declared as a field of WriterTest with private static Writer writer = new Writer();
writer.isTotalSuccess() is a very simple method:
public boolean isTotalSuccess() {
if (total == success){
return true;
}
return false;
}
Related
I need to mock the constant variable in order to test one of my method. How can I do it with Mokito and Junit.
#Component( "mybean" )
#org.springframework.context.annotation.Scope( value="session" )
public class MyBean {
Public void methodToBeTested() {
if (!AppConst.SOME_CONST.equals(obj.getCostCode())) {
// some logic
}
}
}
AppConst class
#Configuration
public class AppConst
{
public static String SOME_CONST;
public static String HOST_URL;
#PostConstruct
public void postConstruct()
{
SOME_CONST = "My Code";
HOST_URL = "Some URL";
}
}
So, from my junit test class, how can I mock the AppConst and it's variables? Now, when I run it, i hit a nullpointer error.
Can this be done with powermock? if yes please give some sample
Mockito version I use.
compile "org.mockito:mockito-all:1.9.5"
compile "org.powermock:powermock-mockito-release-full:1.6.1"
Instead of mocking there would be another solution to be able to test it:
public void methodToBeTested(SomeObject obj) {
performLogic(AppConst.SOME_CONST, obj);
}
boolean performLogic(String check, SomeObject testObj) {
if (!check.equals(obj.getCostCode())) {
// some logic
return true;
}
return false;
}
That way you can test two things, both combined show you that your code works as intended:
public void testMethodToBeTested() {
MyBean mb = new MyBean() {
#Override
void performLogic(String check, SomeObject testObj) {
assertSame("check constant is passed", AppConst.SOME_CONST, check);
}
}
mb.methodToBeTested(new SomeObject());
mb = new MyBean();
SomeObject so = createSomeTestObject("My Code"); // not the actual constant but an equal String
assertFalse("check some logic not occurred", mb.performLogic("My Code", so));
so = createSomeTestObject("Not the constant");
assertFalse("check some logic not occurred", mb.performLogic("Not the constant", so));
assertTrue("check some logic occurred", mb.performLogic("My Code", so));
// additional tests covering the actual logic
}
Another solution could be putting the condition of the if-statement into its own method, e.g. shouldLogicOccurr(String check) and test that method individually.
In other words: Sometimes it's necessary to refactor your code to make tests easier or sometimes even possible at all. A good side effect is the next time you implement something you already have testability in mind and create your code suitable for this in the first place.
Mocking is a good way to get things under test that use third party libraries that can't be changed and have too many dependencies to be set up but if you end up using that for your own code you've got a design issue.
Suppose I have a code like below
public boolean checkForecastIfCityRaining(String name){
result = WeatherAPICallToSomeVendor(name)
if(result = rain)return true; else return false;
}
How would I unit test if the result data will change depending
on what the API vendor is providing?
Would i mock a fixed result of every scenario and then unit test
it like so?
A UNIT test should really only test a single method at a time (Isolate other functionality that might be invoked by that method). My current group might achieve that by writing our function like this:
public class WeatherCheck
{
private ForecastService fs;
public WeatherCheck(ForecastService fs)
{
forecastService = fs;
}
public boolean checkForecastIfCityRaining(String name){
result = forecastService.weatherAPICallToSomeVendor(name)
if(result = rain)return true; else return false;
}
This would allow us to pass a mock forecast service into the constructor. Dependency Injection would be better, but we don't do that yet.
Note: We differentiate between a Unit test and an Integration test. We still might write our Integration tests with Junit, but they have more of a tendency to go out and actually poke the service--this can give you advance warning of a failure. So you might write an integration test for ForecastService that simply calls the weatherAPICallToSomeVendor method of ForecastService and ensures a non-error result (Perhaps no exceptions or doesn't return null...).
I think the function needs to be rewritten as this:
public boolean checkForecastInCityCondition(String city, String condition){
result = WeatherAPICallToSomeVendor(city)
return result == condition;
}
Now you gain the advantage of exposing clients to care about arbitrary conditions and you can enhance with a new API as needed. From a testing perspective you can now safely write tests like this:
public void testRainingInLancaster() throws Exception{
//your code here
}
public void testSnowInRedding() throws Exception{
//your code here
}
And you can determine which pieces need to be mocked for testing.
How can I test if a method does nothing. For example I have a static method that throws an exception if the given string-argument is null or empty (it's meant for argument-validation). Right now my tests look like this:
#Test
public void notNullOrEmpty_doesNothingIfValueIsNotNullOrEmpty() {
Require.notNullOrEmpty(Generate.randomString());
assertTrue(true); // <- this looks very ugly
}
#Test(expected = IllegalArgumentException.class)
public void notNullOrEmpty_throwsExceptionIfValueIsNull() {
Require.notNullOrEmpty(null);
}
#Test(expected = IllegalArgumentException.class)
public void notNullOrEmpty_throwsExceptionIfValueIsEmpty() {
Require.notNullOrEmpty("");
}
How can I make the first test to pass without calling assertTrue(true), there is a Assert.fail() is there something like an Assert.pass()?
EDIT:
Added missing (expected = IllegalArgumentException.class) to 3rd test
You have just to remove the assert in the first method.
#Test
public void notNullOrEmpty_doesNothingIfValueIsNotNullOrEmpty() {
Require.notNullOrEmpty(Generate.randomString());
// Test has passed
}
If the test method runs completely then it means it pass with success. Look at Eclipse junit output:
Update: as an additional comment, if you use Mockito framework you can leverage verify method to verify that a method was called X times. For instance, I used something like this:
verify(cmAlertDao, times(5)).save(any(CMAlert.class));
In your case, since you are testing static methods, then you might find useful using PowerMock which allows you to verify static methods (since Mockito doesn't). And you can use verifyStatic(...).
You should add #Test(expected = YourException.class) annotation.
Try to add to the first test:
#Test
public void notNullOrEmpty_doesNothingIfValueIsNotNullOrEmpty() {
String str = Generate.randomString();
Require.notNullOrEmpty(str);
assertNotNull(str);
}
and probably to you have better to rename it to notNullOrEmpty_doesNothingIfValueIsNotNullOrNotEmpty because you are testing it for not empty value.
A unit test must assert which is expected in the behavior of the method.
If in your specification, when your call notNullOrEmpty()no exception must be thrown when the data is valid and an exception must be thrown when the data is not valid so in your unit test you must do no assertion when the data is valid since if it doesn't success, a exception will be thrown and the test will so be in failure.
#Test
public void notNullOrEmpty_doesNothingIfValueIsNotNullOrEmpty() {
Require.notNullOrEmpty(Generate.randomString());
}
Trying to get Mockito and PowerMock to behave, but I'm getting an UnfinishedStubbingException when trying to run this code:
#RunWith(PowerMockRunner.class)
#PrepareForTest(FileIOHelper.class)
public class FileIOHelperTest {
#Test
public void testIOExceptionOnWrite() {
PowerMockito.mockStatic(FileIOHelper.class);
PowerMockito.doThrow(new IOException()).when(FileIOHelper.class);
PowerMockito.verifyStatic();
FileIOHelper.write(Mockito.anyString(), Mockito.anyString());
}
#After
public void validate() {
Mockito.validateMockitoUsage();
}
}
and this IO class
public final class FileIOHelper {
public static void write(final String file, String message, final boolean appendNewLine) {
if(checkArgs(file, message)) {
final Path path = Paths.get(file);
StandardOpenOption mode = StandardOpenOption.APPEND;
if(Files.notExists(path)) {
mode = StandardOpenOption.CREATE_NEW;
}
if(appendNewLine) {
message += System.getProperty("line.separator");
}
try {
Files.write(path, message.getBytes(), mode);
} catch(IOException e) {
handleException(e, "Problem writing to " + file);
}
}
}
private static boolean checkArgs(final String... args) {
if(args != null && args.length > 0) {
for(final String arg : args) {
if(arg == null || arg.isEmpty()) {
return false;
}
}
}
return true;
}
private static void handleException(final IOException e, final String errorMsg) {
handleException(e, errorMsg, true);
}
private static void handleException(final IOException e, final String errorMsg, final boolean printStace) {
checkArgs(errorMsg);
System.err.println(errorMsg);
System.err.println(e.getMessage());
if(printStace) {
e.printStackTrace();
}
}
}
What I want to do is somehow trigger the IOException so handleException can be tested. Why one might ask? I'm looking at my Jacoco report and I see this:
I've looked at:
How to mock a void static method to throw exception with Powermock?
Powermock/mockito does not throw exception when told to
PowerMockito mock static method which throws exception
https://github.com/jayway/powermock/wiki/MockitoUsage#how-to-stub-void-static-method-to-throw-exception
http://www.johnmullins.co/blog/2015/02/15/beginners-guide-to-using-mockito-and-powermockito-to-unit-test-java/
and I'm still completely lost. I have no idea if I need to trigger the IOException or if I can somehow verify the output of handleException without doThrow. Someone, help!
Error log:
org.mockito.exceptions.misusing.UnfinishedStubbingException:
Unfinished stubbing detected here:
-> at FileIOHelperTest.testIOExceptionOnWrite(FileIOHelperTest.java:8) // doThrow line
E.g. thenReturn() may be missing.
Examples of correct stubbing:
when(mock.isOk()).thenReturn(true);
when(mock.isOk()).thenThrow(exception);
doThrow(exception).when(mock).someVoidMethod();
Hints:
1. missing thenReturn()
2. you are trying to stub a final method, you naughty developer!
3: you are stubbing the behaviour of another mock inside before 'thenReturn' instruction if completed
My recommendation: forget about using PowerMock.
If you have to mock static methods, then build your own little wrapper class around that. Then, for testing, your wrapper can return something you control; and for production usage; your wrapper just calls the static method.
PowerMock looks like the solution to many problems; but rather sooner than later, it will be the cause of much more problems. It breaks coverage, it makes it harder to change the underlying JVM, and so on.
Seriously: if your design can only be tested with PowerMock, this is very often a clear indication that your design is bad. So: focus on reworking your code under test; instead of investing time into a tool like PowerMock that does more harm than good.
I have spent countless hours trying to resolve PowerMock problems; and since I started to instead write "better to test" production code ... I have written hundreds or thousands of tests without ever needing PowerMock again.
In your case: start by avoiding static all over the place. Basically you achieve that by (worst case) pulling little wrapper classes around the static calls you have to make. For testing, you can mock the wrapper object; and in production code, you use dependency injection to provide a (singleton/enum) wrapper object that simply makes the static call.
First of all, IOException is checked exception - it should be declared with throws in the method signature. But your method FileIOHelper.write does not have such. This may be the reason of the UnsutisfiedStubbingException.
I do not understand, what your are trying to test: if the FileIOHelper is a mock - handleException will be never called, since it is called by the real write method, not by mocked.
First, you have to mock the class 'Files', not 'FileIOHelper'. FileIOHelper is the tested class. Second, you didn't specified which method should throw IOException. The unit test method should be as follows (supposing the tested method catches and manage the IOException):
#RunWith(PowerMockRunner.class)
#PrepareForTest(Files.class)
public class FileIOHelperTest {
#Test
public void testIOExceptionOnWrite() {
PowerMockito.mockStatic(Files.class);
PowerMockito.doThrow(new IOException()).when(Files.class);
Files.write("path", "message", true);
FileIOHelper.write("path", "message", true);
PowerMockito.verifyStatic();
Files.write("path", "message", true);
}
}
How can I mark a test as an expected failure in JUnit 4?
In this case I want to continue to run this test until something is patched upstream. Ignoring the test goes a little too far, as then I might forget about it. I may be able to add an #expected annotation and catch the exception thrown by assertThat, but that also seems to lie about the expected behavior.
Here's what my current test looks like:
#Test
public void unmarshalledDocumentHasExpectedValue()
{
doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
final ST title = doc.getTitle();
assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}
That assert should succeed, but because of an upstream bug it doesn't. Yet, that test is correct; it should succeed. Virtually all the alternatives that I've found are misleading. Right now I think #Ignore("This test should pass once fixed upstream") is my best bet, but I still have to remember to come back to it. I'd prefer that the test run.
In Python I can use the expectedFailure decorator:
class ExpectedFailureTestCase(unittest.TestCase):
#unittest.expectedFailure
def test_fail(self):
self.assertEqual(1, 0, "broken")
With Qt's QTestLib in C++, you can use QEXPECT_FAIL:
QEXPECT_FAIL("", "Will be fixed next version", Continue);
QCOMPARE(i, 42);
In both cases above, the unit test runs which is what I'm hoping to have happen. Am I missing something in JUnit?
I'm not quite getting the specifics of your scenario, but here's how I generally test for expected failure:
The slick new way:
#Test(expected=NullPointerException.class)
public void expectedFailure() {
Object o = null;
o.toString();
}
for older versions of JUnit:
public void testExpectedFailure() {
try {
Object o = null;
o.toString();
fail("shouldn't get here");
}
catch (NullPointerException e) {
// expected
}
}
If you have a bunch of things that you want to ensure throw an exception, you may also want to use this second technique inside a loop rather than creating a separate test method for each case. If you were just to loop through a bunch of cases in a single method using expected, the first one to throw an exception would end the test, and the subsequent cases wouldn't get checked.
What about explicitly expecting an AssertionError?
#Test(expected = AssertionError.class)
public void unmarshalledDocumentHasExpectedValue() {
// ...
}
If you're reasonably confident that only the JUnit machinery within the test would raise AssertionError, this seems as self-documenting as anything.
You'd still run the risk of forgetting about such a test. I wouldn't let such tests into version control for long, if ever.
I'm assuming here that you want the test to pass if your assert fails, but if the assert succeeds, then the test should pass as well.
The easiest way to do this is to use a TestRule. TestRule gives the opportunity to execute code before and after a test method is run. Here is an example:
public class ExpectedFailureTest {
public class ExpectedFailure implements TestRule {
public Statement apply(Statement base, Description description) {
return statement(base, description);
}
private Statement statement(final Statement base, final Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable e) {
if (description.getAnnotation(Deprecated.class) != null) {
// you can do whatever you like here.
System.err.println("test failed, but that's ok:");
} else {
throw e;
}
}
}
};
}
}
#Rule public ExpectedFailure expectedFailure = new ExpectedFailure();
// actually fails, but we catch the exception and make the test pass.
#Deprecated
#Test public void testExpectedFailure() {
Object o = null;
o.equals("foo");
}
// fails
#Test public void testExpectedFailure2() {
Object o = null;
o.equals("foo");
}
}
First, note that the first method is marked as #Deprecated. I'm using this as a marker for the method for which I want to ignore any assertion failures. You can do whatever you like to identify the methods, this is just an example.
Next, in the ExpectedFailure#apply(), when I do the base.evaluate(), I'm catching any Throwable (which includes AssertionError) and if the method is marked with the annotation #Deprecated, I ignore the error. You can perform whatever logic you like to decide whether you should ignore the error or not, based on version number, some text, etc. You can also pass a dynamically determined flag into ExpectedFailure to allow it to fail for certain version numbers:
public void unmarshalledDocumentHasExpectedValue() {
doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
expectedFailure.setExpectedFailure(doc.getVersionNumber() < 3000);
final ST title = doc.getTitle();
assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}
For further examples, see ExternalResource, and ExpectedException
Ignoring an expected failure test rather than passing it
If you want to mark you tests as Ignored rather than Success, it becomes a bit more complex, because tests are ignored before they are executed, so you have to retrospectively mark a test as ignored, which would involve constructing your own Runner. To give you a start, see my answer to How to define JUnit method rule in a suite?. Or ask another question.
One option is mark the test as #Ignore and put text in there that is a bug perhaps and awaiting a fix. That way it won't run. It will then become skipped. You could also make use of the extensions to suit your need in a potentially different way.
I've taken Matthew's answer a step further and actually implemented an #Optional annotation you could use instead of the #Deprecated marker annotation he mentions in his answer. Although simple, I'll share the code with you, maybe it's of help for someone:
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface Optional {
/**
* Specify a Throwable, to cause a test method to succeed even if an exception
* of the specified class is thrown by the method.
*/
Class<? extends Throwable>[] exception();
}
With a simple alteration of Matt's ExpectedFailure class:
public class ExpectedFailure implements TestRule {
#Override
public Statement apply(final Statement base, final Description description) {
return statement(base, description);
}
private Statement statement(final Statement base, final Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable e) {
// check for certain exception types
Optional annon = description.getAnnotation(Optional.class);
if (annon != null && ArrayUtils.contains(annon.exception(), e.getClass())) {
// ok
} else {
throw e;
}
}
}
};
}
}
You can now annotate your test method with #Optional and it will not fail, even if the given type of exception is raised (provide one or more types you would like the test method to pass):
public class ExpectedFailureTest {
#Rule public ExpectedFailure expectedFailure = new ExpectedFailure();
// actually fails, but we catch the exception and make the test pass.
#Optional(exception = NullPointerException.class)
#Test public void testExpectedFailure() {
Object o = null;
o.equals("foo");
}
}
[UPDATE]
You could also rewrite your tests using JUnit's org.junit.Assume instead of the tradtional org.junit.Assert, if you want your tests to pass even if the assumption does not hold.
From Assume's JavaDoc:
A set of methods useful for stating assumptions about the conditions in which a test is meaningful.A failed assumption does not mean the code is broken, but that the test provides no useful information. The default JUnit runner treats tests with failing assumptions as ignored.
Assume is available since JUnit 4.4
Use mocked upstream class if possible. Stub it with correct result. Optionally, replace mock with real object after bug is fixed.