We Keep Coding

Thinking about unit tests structure

I am thinking about how to write tests for my project. At the moment, tests structure is like this:
RealClass
{
method1;
method2;
...
}
and exactly same test class structure:
TestClass {
testMethod1;
testMethod2;
...
}
But, I do not like it, because I am putting too much test cases in one test method...
May be I should use structure like this:
TestClass {
testMethod1Opt1;
testMethod1Opt2;
...
testMethod2Opt1;
...}
How are you writing Unit tests?
Example of my test code: (Very simple test)
public void testIsAppUser() {
// My (Artem`s Zinnatullin) uId
final long artemZinnatullinUId = 172672179;
try {
assertTrue(usersApi.isAppUser(artemZinnatullinUId));
} catch (Exception e) {
fail(e.getMessage());
}
// Pavel`s Durov uId
final long durovUId = 1;
try {
assertFalse(usersApi.isAppUser(durovUId));
} catch (Exception e) {
fail(e.getMessage());
}
// By default uId == current user`s (who has authorized) uId
try {
assertTrue(usersApi.isAppUser(null));
} catch (Exception e) {
fail(e.getMessage());
}
}
What I am thinking about:
public void testIsAppUser1() {
// My (Artem`s Zinnatullin) uId
final long artemZinnatullinUId = 172672179;
try {
assertTrue(usersApi.isAppUser(artemZinnatullinUId));
} catch (Exception e) {
fail(e.getMessage());
}
}
public void testIsAppUser2() {
// Pavel`s Durov uId
final long durovUId = 1;
try {
assertFalse(usersApi.isAppUser(durovUId));
} catch (Exception e) {
fail(e.getMessage());
}
}
public void testIsAppUser3() {
// By default uId == current user`s (who has authorized) uId
try {
assertTrue(usersApi.isAppUser(null));
} catch (Exception e) {
fail(e.getMessage());
}
}
Give me advice please.

Comments:
Instead of try{} catch(){ fail() } just add throws Exception to the test method. JUnit will automatically fail the test for you and preserve the stack trace. This will make bug fixing much easier.
Create small test methods. That creates a name problem: How to come up with lots of good names? The solution here is to name the test after what it logically tests, not which methods it calls.
If you want to see what methods are called, use a code coverage tool like JaCoCo.
So the first test should be called testIsArtemsZinnatullinAppUser(). As a guideline: Whenever you feel like you need a comment to explain what a test does, the test name is wrong. Use whatever you'd write in the comment to create a test name.
The reason why you should have smaller tests is that JUnit stops for the first problem. So if you have 20 tests in one test case and the 3rd fails, 17 tests won't run. But these 17 tests could contain valuable information helping to figure out what is wrong.
If they all succeed, then this is probably a specific problem. If many tests fail, the problem must be in shared code.

Your second way of structuring the tests is a lot better. That way each test method covers a different way for the method to break, so you won't have the case where you fix one bug in a method, then have the test fail a little further down (so that one error prevents you from seeing others). It is a lot more important that the test methods be small and sharply-focused than that the test methods map to the methods of the object being tested.
Also, don't catch exceptions, JUnit will do that for you. Add throws Exception to the signature of each of your test methods. If you want to check that an exception is really thrown, then you can catch it in a test, like:
try {
objectUnderTest.doSomethingThatShouldThrowFooException();
fail("should've thrown an exception before getting here");
}
catch (FooException fooEx) {
// yay. my test passed
}
, but the boilerplate of:
} catch (Exception e) {
fail(e.getMessage());
}
is unnecessary.

I won't repeat what's in the other responses. But just adding this:
Avoid duplication of code construct in your test classes.
Don't hesitate to write explicit failure messages.
Here is something to illustrate what I mean:
public void testIsAppUser1() {
// My (Artem`s Zinnatullin) uId
assertAppUser(172672179,true,"artemZinnatullinUId");
}
public void testIsAppUser2() {
// Pavel`s Durov uId
assertAppUser(1,false,"Pavel`s Durov");
}
public void testIsAppUser3() {
// By default uId == current user`s (who has authorized) uId
assertAppUser(null,true,"current user");
}
private void assertAppUser(Long id, boolean isExpectedAppUser, String userName){
boolean isAppUser = usersApi.isAppUser(id);
if(isExpectedAppUser){
assertTrue("User with id:"+id+"and named:"+userName+" must be an appUser" ,isAppUser);
}else{
assertFalse("User with id:"+id+"and named:"+userName+" cannot be an appUser" ,isAppUser);
}
}
}

Only throw when you have an error that might happen because of an 'exception' don't necassarily throw because you can. The following assumes you are creating your own testing enviorment.
I don't know what your assert methods look like but really they should be the ones throwing if anything. You also don't need a try catch to throw an exception you can do the following:
throw new Exception("msg"); // Or another type of Exception
So implementation:
public void AssertEqual(Object obj1, Object obj2) throws Exception
{
if (!obj1.equals(obj2))
throw new Exception("Objects are not equal");
}

Java Writing unittest for exiting a program when user type quit in the console

I found really hard to write unit test for this method, it basically exits the program when user types a quit command.
SytemExit class:
public class SystemExit {
public void exit(int status) {
System.exit(status);
}
}
My static method:
public static void exitWhenQuitDetected() {
final SystemExit systemExit = new SystemExit();
final String QUIT = "quit";
String line = "";
try {
final InputStreamReader input = new InputStreamReader(System.in);
final BufferedReader in = new BufferedReader(input);
while (!(line.equals(QUIT))) {
line = in.readLine();
if (line.equals(QUIT)) {
System.out.println("You are now quiting the program");
systemExit.exit(1);
}
}
} catch (Exception e) {
System.err.println("Error: " + e.getMessage());
}
}
Something is not quite right here as I am struggling to unit test the method exitWhenQuitDetected (I am using Mockito for mocking). How would I mock the InputStreamReader and verify the SystemExit.exit method gets called when it sees a quit? Can you shed some lights on this please? Thanks.
Added the test I am working on at the moment, it's not working.
#Test
#Ignore
public void shouldExitProgramWhenTypeQuit() {
String quit = "quit";
SystemExit systemExit = mock(SystemExit.class);
try {
BufferedReader bufferedReader = mock(BufferedReader.class);
when(bufferedReader.readLine()).thenReturn(quit + "\n");
SomeClass.exitWhenQuitDetected();
verify(systemExit, times(1)).exit(1);
} catch (IOException e) {
e.printStackTrace();
}
}

You should include the PowerMockito Jars into your project rather than just vanilla Mockito. The Powermock library is designed for mocking Static and/or Final classes and methods.
The following this blog post contains example code describing a similar scenario to yours.
Essentially you require a test class similar to this...
#RunWith(PowerMockRunner.class)
#PrepareForTest({System.class, ClassToTest.class})
public class SystemExitTest {
#Test
public void shouldCallSystemExit() {
PowerMockito.mockStatic(System.class);
ClassToTest.methodToTest();
PowerMockito.verifyStatic();
System.exit(0);
System.out.println("If this message displays them System.exit() was mocked successfully");
}
}
Given this simple implementation class...
public class ClassToTest {
public static void methodToTest() {
// do some stuff
System.exit(0);
}
}

You've done 90% of the work already, by placing the actual exiting code off in a separate class with no logic of its own. Your difficulty is caused by your use of a static method.
I would advise making the exitWhenQuitDetected not static. Put it in a class that you can instantiate when you need it, and that you can create with a mocked SystemExit. Something like this.
public class SomeClass{
private final SystemExit exiter;
private final static String QUIT = "quit";
public SomeClass(){
this(new SystemExit());
}
SomeClass(SystemExit exiter){
this.exiter = exiter;
}
public static void exitWhenQuitDetected() {
String line = "";
try {
final InputStreamReader input = new InputStreamReader(System.in);
final BufferedReader in = new BufferedReader(input);
while (!(line.equals(QUIT))) {
line = in.readLine();
if (line.equals(QUIT)) {
System.out.println("You are now quiting the program");
exiter.exit(1);
}
}
} catch (Exception e) {
System.err.println("Error: " + e.getMessage());
}
}
// ...
}
Then, in your test, you can make a mock of SystemExit, and use the package-private constructor of SomeClass to create an object that will use your mock as its exiter. You can then run your test, and verify on your mock SystemExit.

There is no real way to test you SystemExit class since exercising it will cause the JVM to exit. You might be able to do something with a SecurityManager which detects and rejects the System.exit(), but that's going to be a whole lot of work to test a single line of code.
You've done the right thing - you've pulled the functionality into a small class. If I were you, I would put an interface on it and inject it via the interface into the parsing code. Then in your test you can inject a mock and test that your parsing code calls the exit() method on the mock with the right exit code.
The code in the SystemExit class is small and self-contained enough to look at and reason about without testing, IMHO.

Plenty of technical solutions were offered. I would like to point out another perspective:
This code should not really be unit tested.
The best gain you get out of unit tests is when applying them to complex business code, especially with a lot of branching.
In cases where the code is trivial, I would advise against writing unit tests around it since it simply just does not have a high enough return of investment. your case actually exacerbates my claim, think of the amount of effort it takes you to test simple code such as this and compare it with the gain.. does this really worth the effort. Does this really make you trust your own code more?

JUnit MethodRule only tests one line

I've made an MethodRule and #Rule-annotation to make my test-life a bit easier.
It checks if a specific exception had been thrown and checks if the exception-message equals or contains the given message.
Now when i run a testmethod with more lines to test, it only takes the first line and than is ready. How do I make so all my lines in the testmethod are tested?
This is my code:
Annotation:
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.METHOD})
public #interface ExpectedDomeinValidatieMessage {
String value() default "";
String contains() default "";
}
MethodRule:
#Override
public Statement apply(final Statement base, final FrameworkMethod method, final Object target) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
ExpectedDomeinValidatieMessage message = method.getAnnotation(ExpectedDomeinValidatieMessage.class);
if (message == null) {
base.evaluate();
} else {
try {
base.evaluate();
Assert.fail("DomeinValidatieException not thrown");
} catch (DomeinValidatieException e) {
if (StringUtils.isNotBlank(message.value())) {
if (!e.getMessage().equals(message.value())) {
throwException(e, "", message.value(), e.getMessage());
}
}
if (StringUtils.isNotBlank(message.contains())) {
if (!e.getMessage().contains(message.contains())) {
throwException(e, "Segment niet gevonden:", message.contains(), e.getMessage());
}
}
}
}
}
private void throwException(Throwable exception, String message, String expected, String actual) {
ComparisonFailure cf = new ComparisonFailure(message, expected, actual);
cf.setStackTrace(exception.getStackTrace());
throw cf;
}
};
Usage:
#Test
#ExpectedDomeinValidatieMessage("[Werkzaamheden] WerkzaamMetGevaarlijkeStoffen niet gevuld")
public void valideerWerkzaamMetGevaarlijkeStoffen() throws DomeinValidatieException {
aanvraag.getVerzekerde().getWerkzaamheden().setWerkzaamMetGevaarlijkeStoffen(null);
validator.valideer();
}
If I use it like this, it only tests the first test in the method:
#Test
#ExpectedDomeinValidatieMessage("[Werkzaamheden] WerkzaamMetGevaarlijkeStoffen niet gevuld")
public void valideerWerkzaamMetGevaarlijkeStoffen() throws DomeinValidatieException {
aanvraag.getVerzekerde().getWerkzaamheden().setWerkzaamMetGevaarlijkeStoffen(null);
validator.valideer(); //Only this one is tested
aanvraag.getVerzekerde().getWerkzaamheden().setWerkzaamMetGevaarlijkeStoffen("bla");
validator.valideer(); //This is NOT tested
}

Run the code through a debugger. My guess is that the first call to valideer() does indeed throw an exception even though you don't expect it.

The JUnit assertXXX methods work by throwing exceptions (specifically AssertionError). So when an exception is thrown (either by your code, or by an assert) control exits from the test method. There isn't any way to restart from the place where the exception is thrown.
You probably want Parameterized, which allows you to run the same tests multiple times with different parameters.
EDIT: I suspect that valideer() is throwing an Exception. To explain a bit further, let's paraphrase your code. When you define a rule, what you're effectively doing is the following:
try {
base.evaluate(); // this calls valideerWerkzaamMetGevaarlijkeStoffen()
Assert.fail("DomeinValidatieException not thrown");
} catch (DomeinValidatieException e) {
// evaluate whether or not the test has failed or not
}
This means that if your first call to valideer() throws an Exception, then control is transferred to the catch block above. There isn't a chance to continue executing the test, because the control has passed elsewhere. You can pass/fail the test as much as you like, but control has passed to the catch block above.
By the way, MethodRule has been deprecated in the later versions, you should be using TestRule instead.

Java: how to handle retries without copy-paste code?

I have multiple cases when I have to deal retrial for DB and networking operations. Everywhere I do it I have the following type of code:
for (int iteration = 1; ; iteration++) {
try {
data = doSomethingUseful(data);
break;
} catch (SomeException | AndAnotherException e) {
if (iteration == helper.getNumberOfRetries()) {
throw e;
} else {
errorReporter.reportError("Got following error for data = {}. Continue trying after delay...", data, e);
utilities.defaultDelayForIteration(iteration);
handleSpecificCase(data);
}
}
}
The issue is that this code pattern is copy-pasted all over my classes. Which is really bad. I can't figure out how to get rid of this for-break-catch copy-paste pattern, since I usually get different exception to handle, I want to log data I failed on (usually also different ways).
Is there a good way to avoid this copy-paste in Java 7?
Edit: I do use guice for dependency injection. I do have checked exceptions. There could be multiple variables instead of just one data and they are all of different type.
Edit2: AOP approach looks as the most promising for me.

Off-hand, I can think of two different approaches:
If the differences in exception handling can be expressed declaratively, you might use AOP to weave the exception handling code around your methods. Then, your business code could look like:
#Retry(times = 3, loglevel = LogLevel.INFO)
List<User> getActiveUsers() throws DatabaseException {
// talk to the database
}
The advantage is that it is really easy to add retry behaviour to a method, the disadvantage is the complexity of weaving the advice (which you only have to implement once. If you are using a dependency injection library, chances are it will offer method interception support).
The other approach is to use the command pattern:
abstract class Retrieable<I,O> {
private final LogLevel logLevel;
protected Retrieable(LogLevel loglevel) {
this.logLevel = loglevel;
}
protected abstract O call(I input);
// subclasses may override to perform custom logic.
protected void handle(RuntimeException e) {
// log the exception.
}
public O execute(I input) {
for (int iteration = 1; ; iteration++) {
try {
return call(input);
} catch (RuntimeException e) {
if (iteration == helper.getNumberOfRetries()) {
throw e;
} else {
handle();
utilities.defaultDelayForIteration(iteration);
}
}
}
}
}
The problem with the command pattern are the method arguments. You are restricted to a single parameter, and the generics are rather unwieldly for the caller. In addition, it won't work with checked exceptions. On the plus side, no fancy AOP stuff :-)

As already suggested, AOP and Java annotations is a good option. I would recommend to use a read-made mechanism from jcabi-aspects:
#RetryOnFailure(attempts = 2, delay = 10, verbose = false)
public String load(URL url) {
return url.openConnection().getContent();
}
Read also this blog post: http://www.yegor256.com/2014/08/15/retry-java-method-on-exception.html

I have implemented the RetryLogic class below which provides reusable retry logic and supports parameters because the code to be retried is in a delegate passed in.
/**
* Generic retry logic. Delegate must throw the specified exception type to trigger the retry logic.
*/
public class RetryLogic<T>
{
public static interface Delegate<T>
{
T call() throws Exception;
}
private int maxAttempts;
private int retryWaitSeconds;
#SuppressWarnings("rawtypes")
private Class retryExceptionType;
public RetryLogic(int maxAttempts, int retryWaitSeconds, #SuppressWarnings("rawtypes") Class retryExceptionType)
{
this.maxAttempts = maxAttempts;
this.retryWaitSeconds = retryWaitSeconds;
this.retryExceptionType = retryExceptionType;
}
public T getResult(Delegate<T> caller) throws Exception {
T result = null;
int remainingAttempts = maxAttempts;
do {
try {
result = caller.call();
} catch (Exception e){
if (e.getClass().equals(retryExceptionType))
{
if (--remainingAttempts == 0)
{
throw new Exception("Retries exausted.");
}
else
{
try {
Thread.sleep((1000*retryWaitSeconds));
} catch (InterruptedException ie) {
}
}
}
else
{
throw e;
}
}
} while (result == null && remainingAttempts > 0);
return result;
}
}
Below is a use example. The code to be retried is within the call method.
private MyResultType getDataWithRetry(final String parameter) throws Exception {
return new RetryLogic<MyResultType>(5, 15, Exception.class).getResult(new RetryLogic.Delegate<MyResultType> () {
public MyResultType call() throws Exception {
return dataLayer.getData(parameter);
}});
}
In case you want to retry only when a specific type of exception occurs (and fail on all other types of exceptions) the RetryLogic class supports an exception class parameter.

Make your doSomething implement an interface, e.g., Runable and create a method containing your code above with doSomething replaced with interface.run(data)

take a look at: this retry utility
this method should work for most use cases:
public static <T> T executeWithRetry(final Callable<T> what, final int nrImmediateRetries,
final int nrTotalRetries, final int retryWaitMillis, final int timeoutMillis)
you can eassily implement an aspect using this utility to do this with even less code.

Extending the approach discusssed already, how about something like this (no IDE on this netbook, so regard this as pseudocode...)
// generics left as an exercise for the reader...
public Object doWithRetry(Retryable r){
for (int iteration = 1; ; iteration++) {
try {
return r.doSomethingUseful(data);
} catch (Exception e) {
if (r.isRetryException(e)) {
if(r.tooManyRetries(i){
throw e;
}
} else {
r.handleOtherException(e);
}
}
}

One thing I would like to add. Most exceptions (99.999%) mean there is something very wrong with your code or environment that needs an admins attention. If your code can't connect to the database it's probably a misconfigured environment there is little point to retrying it just to find out it didn't work the 3rd, 4th, or 5th time either. If you're throwing an exception because the person didn't give a valid credit card number, retrying isn't going to magically fill in a credit card number.
The only situations that are remotely worth retrying is when a system is tremendously strained and things are timing out, but in this situation retry logic is probably going to cause more strain than less (3x for 3 retries on every transaction). But this is what systems do to back down demand (see the apollo lander mission story). When a system is asked to do more than it can it starts dropping jobs and timeouts are the signal the system is strained (or poorly written). You'd be in a far better situation if you just increased the capacity of your system (add more ram, bigger servers, more servers, better algorithms, scale it!).
The other situation would be if you're using optimistic locking and you can somehow recover and auto merge two versions of an object. While I have seen this before I'd caution this approach, but it could be done for simple objects that can be merged without conflicts 100% of the time.
Most exceptions logic should be catch at the appropriate level (very important), make sure your system is in a good consistent state (ie rollback transactions, close files, etc), log it, inform user it didn't work.
But I'll humor this idea and try to give a good framework (well because it's fun like crossword puzzle fun).
// client code - what you write a lot
public class SomeDao {
public SomeReturn saveObject( final SomeObject obj ) throws RetryException {
Retry<SomeReturn> retry = new Retry<SomeReturn>() {
public SomeReturn execute() throws Exception {
try {
// doSomething
return someReturn;
} catch( SomeExpectedBadExceptionNotWorthRetrying ex ) {
throw new NoRetryException( ex ); // optional exception block
}
}
}
return retry.run();
}
}
// framework - what you write once
public abstract class Retry<T> {
public static final int MAX_RETRIES = 3;
private int tries = 0;
public T execute() throws Exception;
public T run() throws RetryException {
try {
return execute();
} catch( NoRetryException ex ) {
throw ex;
} catch( Exception ex ) {
tries++;
if( MAX_RETRIES == tries ) {
throw new RetryException("Maximum retries exceeded", ex );
} else {
return run();
}
}
}
}

Is there a less-verbose way to silently ignore a specific nested exception in a Java unit test?

I have some unit tests which exercise code which makes calls out to a test server, in order to make sure that the requests are well-formed (i.e. we get valid data back in response). However, this means that the unit tests, and hence the build, can get blocked if this test server is down. This does not conform to good unit test practices, but as a thought experiment let's say I'm not allowed to delete these tests or change them so they don't actually call out to the server. I want to change them so that they will still pass if the server is down (i.e. trying to connect results in ConnectException), but fail if any other exception occurs. Making it more difficult, the code under test doesn't throw the ConnecException directly, but throws a wrapper exception that contains it. So initially, that means each test will go from looking like this:
#Test
public void testNumberOne() {
// body of test...
}
To this:
#Test
public void testNumberOne() {
try {
// body of test...
} catch (ThirdPartyWrapperException e) {
if (!(e.getRootCause() instanceof ConnectException) {
throw e;
}
}
}
Is there any way I can avoid having to paste that try/catch into each unit test?
I know I can refactor out at least some of it, ala:
#Test
public void testNumberOne() {
try {
// body of test...
} catch (ThirdPartyWrapperException e) {
handleException(e);
}
}
private void handleException(ThirdPartyWrapperException e)
throws ThirdPartyWrapperException {
if (!(e.getRootCause() instanceof ConnectException) {
throw e;
}
}
But is there anything further I can do?

I would add a line to the start to determine if the required resources are available
#Test
public void testNumberOne() {
if (!requiredServerAvailable()) return;