We Keep Coding

How to create JUnit tests for different input files as separate cases?

Right now, I have around 107 test input cases for my interpreter, and I have my JUnit tester set up to manually handle each case independently so as to not lump them all together. That is, if I use a loop to iterate over the test files as such
for (int i = 0; i < NUM_TESTS; i++) {
String fileName = "file_" + (i + 1) + ".in";
testFile(fileName);
}
JUnit will create one giant test result for all 107 tests, meaning if one fails, the entire test fails, which I don't want. As I said, right now I have something like
#Test
public static void test001() {
testFile("file1.in");
}
#Test
public static void test002() {
testFile("file2.in");
}
While this works, I imagine there's a much better solution to get what I'm after.

You can use #ParameterizedTest with #MethodSource annotations.
For exemple :
#ParameterizedTest
#MethodSource("fileNameSource")
void test(final String fileName) {
testFile(fileName);
}
private static Stream<String> fileNameSource() {
return IntStream.range(0,NUM_TESTS).mapToObj(i -> "file_" + (i + 1) + ".in");
}
Check the documentation at https://junit.org/junit5/docs/current/user-guide/#writing-tests-parameterized-tests
For each params returned by fileNameSource(), the corresponding test will be considered as a different case.

You have to define own structure based on your need,
one way to define your input to json file in like list of values as below.
{
[
"value1",
"value2"
]
}
Read this value when you test case execute with the help of object mapper.
objectMapper.readValue(fixture("filePathName.json"),CustomInput.class);
Where CustomInput would be something like below.
public class CustomInput {
List<String> values;
}
You can keep increase & decrease your inputs in json.

How to use correctly assertThrows on set of values?

I've been learning Java just for a bit, so please advise how exception throwing test should look like in this case?
I have following Gambling Machine Class. And then 2 tests for it. I do not really know what should follow the "Integer" in second method (shouldThrowWhenNumbersOutOfRange). Could you please advise as to the exact syntax?
public class GamblingMachine {
public int howManyWins(Set<Integer> userNumbers) throws InvalidNumbersException {
validateNumbers(userNumbers);
Set<Integer> computerNumbers = generateComputerNumbers();
int count = 0;
for (Integer number : userNumbers) {
if (computerNumbers.contains(number)) {
count++;
}
}
return count;
}
private void validateNumbers(Set<Integer> numbers) throws InvalidNumbersException {
if (numbers.size() != 6) {
throw new InvalidNumbersException();
}
if (numbers.stream().anyMatch(number -> number < 1 || number > 49)) { //anyMatch-function to check whether any element in list satisfy given condition
throw new InvalidNumbersException();
}
}
private Set<Integer> generateComputerNumbers() {
Set<Integer> numbers = new HashSet<>();
Random generator = new Random();
while(numbers.size() < 6) {
numbers.add(generator.nextInt(49) + 1);
}
return numbers;
}
}
private GamblingMachine machine = new GamblingMachine();
#ParameterizedTest
#NullAndEmptySource
public void shouldThrowWhenNumbersEmpty(Set<Integer> numbers) throws InvalidNumbersException {
Assertions.assertThrows(NumberFormatException.class, () -> {
Integer.parseInt(" ");
});
}
#ParameterizedTest
#CsvFileSource(resources ="/numbersOutOfRange.cvs", numLinesToSkip = 1)
public void shouldThrowWhenNumbersOutOfRange(Set<Integer> numbers) throws InvalidNumbersException {
Assertions.assertThrows(NumberFormatException.class, () -> {
Integer. //how code should look like here?
});
}

The point of a test is to, you know, test something. Your shouldThrowWhenNumbersEmpty test doesn't do that (well, it tests that Integer.parseInt(" ") throws something. It does, of course. You... don't have to test the core libraries).
In other words, your gambling machine tests need to be calling some stuff from your GamblingMachine class. The idea is to test GamblingMachine. Not to test Integer.parseInt.
It's also a bizarre test: Why in the blazes is shouldThrowWhenNumbersEmpty parameterized? I assume the point of that test is: "Ensure that the gambling machine works as designed when passing an empty set of numbers in, specifically, the part of the design that states that an InvalidNumbersException is thrown if you do that".
Which is done with something like:
#Test
public void shouldThrowWhenNumbersEmpty() {
Assertions.assertThrows(InvalidNumbersException.class, () -> {
Set<Integer> empty = Set.of();
machine.howManyWins(empty);
});
}
Parameterized tests are a fairly exotic concept. Your test setup appears to be falling into a trap: It appears to be set up that you repeat all the logic that is already in your gamblingmachine class, to then apply this logic to the incoming (parameterized) data, figure out what your gambling machine ought to be doing, and then double check its work.
That's not how you should write tests. Tests focus on a specific result. Parameterized tests can make sense, but only if the stuff you have to do for any given input is roughly the same. For example:
Good use of parameterized testing
You have a csv file containing a bunch of lines, each of which has 6 rolls + the correct answer. Your parameterized test treats each line the same: Call howManyWins using the 6 rolls as input, then check that howManyWins returns the expected value.
Bad use of parameterized testing
You have a csv file containing a bunch of lines, each of which has 6 rolls. Your parameterized test will calculate the right result for the rolls, then invoke gambling machine, and check that the gambling machine gives the same answer as what you calculated.
This is bad: You're just repeating the code. It also means your test code is itself doing more than the very basics (it's doing a bunch of business logic), thus raising the question: Who tests your test, then?
Both of your test methods seem like they should NOT be parameterized, unless that csv also contains results.

Bind Datapoints in JUnit Theory to a particular variable

I have following theory to test. In the code I want variable a to be Even and variable b to be odd
#RunWith(Theories.class)
public class TestJunit{
// add the error
#DataPoints
public static Integer[] integersOdd() {
return new Integer[]{1, 3, 5};
}
#DataPoints
public static Integer[] integersEven() {
return new Integer[]{2, 4, 6};
}
#Theory
public void testAdd(Integer a , Integer b) {
...
}
}
For now I am using assumeTrue and a validation function as in:
public boolean validateInput(Integer a, Integer b){
Set<Integer> even = new HashSet<Integer>(Arrays.asList(integersEven()));
Set<Integer> odd = new HashSet<Integer>(Arrays.asList(integersOdd()));
return (even.contains(a) && odd.contains(b));
}
Modified Theory:
#Theory
public void testAdd(Integer a , Integer b) {
Assume.assumeTrue(validateInput(a,b));
System.out.println("a="+a+", b="+b);
assertTrue(a+b>-1);
// add any test
}
It is a very dirty way as Java will pick all the combinations and will discard the inputs at assumeTrue. What If I have 10 theories with 10 datapoints? Java will try 100 combinations where I wanted only 10!
Is there neat way to do so? May be some annotation to tell JUnit to pick values for variables from which DataPoint?
Edit:
Another way I found is to use Test Generators. I am using JUnit-QuickCheck [Read Here] to generate random data according to the range required by my variables. Then I encapsulate them in a class and pass this object into my theory to test.

JUnit 4.12 allows for named data points in theories. Here's the original pull request, and here are the release notes for 4.12 - look for "Added mechanism for matching specific data points".

How do I run the same JUnit test multiple times with different test data each time?

I am just getting started with unit testing. I did the junit tutorial from a pdf from the tutorial points website. So my question is, I want to test my shunting yard algorithm and my RPNEvaluator.
The constructors (and any other variables to help you out with the context) look like this:
ShuntingYard.java:
private ArrayList<String> tokens = new ArrayList<String>();
public ShuntingYard(ArrayList<String> tokens) {
this.tokens = tokens;
}
RPNEvaluator.java:
private Queue<String> polishExpression;
public RPNEvaluator(Queue<String> exp) {
polishExpression = exp;
}
ShuntingYard.java has a method called toRpn() which will take an ArrayList and return a Queue after some processing.
RPNEvaluator has a method called evaluate which will take a Queue type and return a double after some processing.
With Junit I am trying to write some unit tests and I wanted to know if this start was the best way to go about it:
package testSuite;
import static org.junit.Assert.fail;
import java.util.ArrayList;
import org.junit.Before;
import org.junit.Test;
public class ExpressionEvaluationTest {
/**
* Initialise the lists to be used
*/
#Before
public void beforeTest() {
ArrayList<String> exprOne = new ArrayList<String>();
exprOne.add("3");
exprOne.add("+");
exprOne.add("4");
exprOne.add("*");
exprOne.add("2");
exprOne.add("/");
exprOne.add("(");
exprOne.add("1");
exprOne.add("-");
exprOne.add("5");
exprOne.add(")");
exprOne.add("^");
exprOne.add("2");
exprOne.add("^");
exprOne.add("3");
ArrayList<String> exprTwo = new ArrayList<String>();
exprTwo.add("80");
exprTwo.add("+");
exprTwo.add("2");
ArrayList<String> exprThree = new ArrayList<String>();
exprThree.add("2");
exprThree.add("/");
exprThree.add("1");
exprThree.add("*");
exprThree.add("4");
ArrayList<String> exprFour = new ArrayList<String>();
exprFour.add("11");
exprFour.add("-");
exprFour.add("(");
exprFour.add("2");
exprFour.add("*");
exprFour.add("4");
exprFour.add(")");
ArrayList<String> exprFive = new ArrayList<String>();
exprFive.add("120");
exprFive.add("/");
exprFive.add("(");
exprFive.add("10");
exprFive.add("*");
exprFive.add("4");
exprFive.add(")");
ArrayList<String> exprSix = new ArrayList<String>();
exprSix.add("600");
exprSix.add("*");
exprSix.add("2");
exprSix.add("+");
exprSix.add("20");
exprSix.add("/");
exprSix.add("4");
exprSix.add("*");
exprSix.add("(");
exprSix.add("5");
exprSix.add("-");
exprSix.add("3");
exprSix.add(")");
}
#Test
public void test() {
}
}
I was going to put this in the before() method:
ShuntingYard sy = new ShuntingYard(/arraylist here/);
And then in the test, pass the lists to the algorithm. My question is that I think I am going the long way around it, would it be better to have a parameterised annotation and pass those lists as a list of parameters?
and a further question: if a test for any of the ArrayLists passes then I am sure I can execute a subsequent test to the RPNEvaluator evaluate method. I hope I haven't been ambiguous.
Help would be very much appreciated.

I would come at it a little differently. Instead of just creating several sets of test data and calling the same test each time break it up in to something meaningful. Instead of writing one test called test() write several separate tests for each aspect of ShuntingYard. For example:
#Test public void
itDoesntDivideByZero()
{
ArrayList<String> divideByZeroExpression = Arrays.asList("5", "0", "/");
// Add code to call your method with this data here
// Add code to verify your results here
}
#Test public void
itCanAdd()
{
ArrayList<String> simpleAdditionExpression = Arrays.asList("1", "2", "+");
// Add code to call your method with this data here
// Add code to verify your results here
}
and so on. This will make your JUnit output much easier to read. When there's a failure you know that it failed while trying to add, or it failed while trying to evaluate an expression that would cause a divide by zero, etc. Doing it the way you have it in the original you'd only know that it failed in the test() method.
Each of the tests here does 3 things:
Arranges the test data
Performs some action with that data
Asserts that the results of the action are as expected
This Arrange, Assert, Act idiom is very common in automated testing. You may also see it called Given, When, Then as in, "Given these conditions, when I call this method, then I should get this result".
Try to get out of the mindset of writing one test to test an entire class or method. Write a test to test one part of a method. Consider this class:
public class Adder {
public int addOneTo(int someNumber) {
return someNumber + 1;
}
}
You might end up with a test suite that looks like:
#Test public void
itAddsOne()
{
int numberToAddTo = 1;
int result = new Adder().addOneTo(numberToAddTo);
assertEquals("One plus one is two", 2, result);
}
#Test(expected="NullPointerException.class") public void
itChokesOnNulls()
{
new Adder().addOneTo((Integer)null);
}
#Test public void
itDoesntOverflow()
{
int result = new Adder().addOneTo(Integer.MAX_VALUE);
// do whatever here to make sure it worked correctly
}
And so on.

The advise from Mike B is very good, try to separate your test thinking in one test per behavior/functionality.
For make your test more readable i probably write a static constructor for the class ShuntingYard that receives a string, then you can write:
ShuntingYard addition = ShuntingYard.createFromExpresion("2+2");
assertThat(addition.getRpn().evaluate(), is(4));
you can refactor a little more and ends with something like that:
assertThat(evaluate("2+2"), is(4))
That is easy to understand an and easy to read, and in addition write more test with diferent scenarios its one-line of code.
Other option its to write parametrized test, one example: http://www.mkyong.com/unittest/junit-4-tutorial-6-parameterized-test/, but in my opinion are really ugly. This test are normally called "data driven test" and are used when you want to test the same code with different input values.
For this data-driven test a much better option its to use something like spock, a groovy framework for testing that allows you to write incredible semantic test, and of course you can use for testing java code, check this out: http://docs.spockframework.org/en/latest/data_driven_testing.html

How do I write a unit test to verify that a function sorts its result? [duplicate]

This question already has answers here:
How to test the ordering of elements in a Collection in JUnit test?
(4 answers)
Closed 6 years ago.
I have a data source from which I can request a list of people that live in a (any) country, and a method which retrieves the people from that data source and sorts them by their name alphabetically. How should I write my unit test to make sure that the sorting part of my method works properly?
This is what my SUT looks like:
class PeopleStuff {
public IData data;
public List<Person> getSortedPeopleForCountry(String countryName) {
List<Person> people = data.getPeopleForCountry(countryName);
Comparator nameComparator = new PersonNameComparator();
Collections.sort(people, nameComparator);
return people;
}
}
And this is what my unit test looks like:
#Test public void testGetPeopleSortsByPeopleName() {
String COUNTRY = "Whatistan";
// set up test (the 3 lines below are actually in a #Before setup method)
PeopleStuff peopleStuff = new PeopleStuff();
IData mockData = createNiceMock(IData.class);
peopleStuff.data = mockData;
// set up data
List<PersonName> mockPeopleList = new ArrayList<PersonName>();
mockPeopleList.add(new Person(COUNTRY, "A"));
mockPeopleList.add(new Person(COUNTRY, "D"));
mockPeopleList.add(new Person(COUNTRY, "B"));
mockPeopleList.add(new Person(COUNTRY, "C"));
when(mockData.getPeopleForCountry(COUNTRY)).thenReturn(mockPeopleList);
// exercise
List<String> result = peopleStuff.getSortedPeopleForCountry(COUNTRY);
// assert
assertEquals("A", result.get(0).name);
assertEquals("B", result.get(1).name);
assertEquals("C", result.get(2).name);
assertEquals("D", result.get(3).name);
}
What I need to know is if the way I am stubbing the data, running the test and making the assertions is correct, or if there are better ways of doing this.
My application has a lot of methods to test and a lot of custom sorting algorithms; I implemented all tests to use some 4 values that I stub like that, in a "random" order which I choose when I write the test.
Should I just test if the comparators are called? That doesn't seem right to me, because I don't know if they're called for the right data or at the right time in the algorithm that's inside getSortedPeopleForCountry(). I want to detect situations like this:
public List<Person> getSortedPeopleForCountry(String countryName) {
List<Person> people = data.getPeopleForCountry(countryName);
Comparator nameComparator = new PersonNameComparator();
List<Person> sortedPeople = new ArrayList<Person>(people)
Collections.sort(sortedPeople, nameComparator);
return people; // oops!
}
Should I leave it like this and add mock comparators which use the real comparators but also verify that they're being called?
Am I doing it right?

I think your current test is very good - the tests are realistic, exercising all of the code, and you are mocking out the data source & using dependency injection to supply a mock data source. There is a lot of best practice going on in this test.
On the issue of whether you should look to mock the comparators (and therefore make the test on testGetPeopleSortsByPeopleName a pure unit test), you will definitely get two different opinions here:
A purist would argue that your test is technically an integration test, and that to have proper unit tests you need to adjust your test to use a mock comparator, and then test the comparator separately.
A pragmatist would argue that your test is already high quality, and that it doesn't matter that it isn't a unit test in the strictest sense. Furthermore, to split this into two separate unit tests may make the test less readable - which I imagine would be the case with the test above if you were to involve mock comparators.
My personal opinion is that you should leave it as it is, the fact that you have a high quality, readable test that exercises all the code and effectively asserts your requirements is far more important than worrying about having strictly pure unit tests.
The only way in which the test looks in need of improvement is the length of the test method - I think a little method extraction could help improve readability and make the test method more expressive. I would aim for something like this:
#Test public void testGetPeopleSortsByPeopleName() {
peopleStuff.data = buildMockDataSource(COUNTRY, "A", "D", "B", "C")
List<String> result = peopleStuff.getSortedPeopleForCountry(COUNTRY);
assertPersonList(result, "A", "B", "C", "D")
}
private IData buildMockDataSource(String country, String ... names) {
...
}
private void assertPersonList(List<Person> people, String ... names) {
...
}

ObjectA[] arr = objectAList.toArray(new ObjectA[objectAList.size()]);
for (int i = 0; i < objectAList.size() - 1; i++) {
int j = i + 1;
assertTrue(arr[i].getDate().compareTo(arr[j].getDate()) >= 0);
}
This code represents an example where ArrayList contaning ObjectA objects is sorted by field date in descending order. We are checking if the member of the list has smaller or equal date from his predecessor.

Separate the sorting logic from returning the list. So I'd have getPeopleForCountry(String countryName) only return a list whereas a sorted list would be returned from getSortedPeopleForCountry(List ). That way you can test how it works before and after sorting. Also, you might want to override the Equals() method to compare names if that is what you want to go with, but then you'd later want to compare with some other property. That's your call.