In Eclipse, how do I run a JUnit test case multiple times - java

I have a unit test that fails sometimes and debugging it is a pain because I don't know why it sometimes fails.
Is there a way inside Eclipse that I can run a JUnit test 5 times or 50 times or something?
Thanks.

I just found the following solution which doesn't require any additional depedency (Spring is required for one of the answers you got).
Run your test with the Parameterized runner:
#RunWith(Parameterized.class)
Then add the following method to provide a number of empty parameters equals to the number of times you want to run the test:
#Parameterized.Parameters
public static List<Object[]> data() {
return Arrays.asList(new Object[10][0]);
}
This way you don't even have to write a loop. IntelliJ and eclipse also group the results of every iteration together.

Have you tried something like this?
#Test
public void runMultipleTests() {
for (int i = 0; i < 10; i++) {
myTestMethod();
}
}

There is a test decorator for this. See Junit API at
http://junit.org/apidocs/junit/extensions/RepeatedTest.html
for example
#Test
#Repeat(10)
public void FailRandomlyNeedToKnowWhy() {
....
}

Inspired on this solution:
Use #Repeat annotation like this:
public class MyTestClass {
#Rule
public RepeatRule repeatRule = new RepeatRule();
#Test
#Repeat(10)
public void testMyCode() {
//your test code goes here
}
}
You'll only need these two classes:
Repeat.java:
import static java.lang.annotation.ElementType.ANNOTATION_TYPE;
import static java.lang.annotation.ElementType.METHOD;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
#Retention( RetentionPolicy.RUNTIME )
#Target({ METHOD, ANNOTATION_TYPE })
public #interface Repeat {
int value() default 1;
}
RepeatRule.java:
import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
public class RepeatRule implements TestRule {
private static class RepeatStatement extends Statement {
private final Statement statement;
private final int repeat;
public RepeatStatement(Statement statement, int repeat) {
this.statement = statement;
this.repeat = repeat;
}
#Override
public void evaluate() throws Throwable {
for (int i = 0; i < repeat; i++) {
statement.evaluate();
}
}
}
#Override
public Statement apply(Statement statement, Description description) {
Statement result = statement;
Repeat repeat = description.getAnnotation(Repeat.class);
if (repeat != null) {
int times = repeat.value();
result = new RepeatStatement(statement, times);
}
return result;
}
}
2016-10-25 Edit:
In order to use this solution when using #RunWith(PowerMockRunner.class), update to Powermock 1.6.5 (which includes this patch).

Related

How To Write a Java Unit Test for the Method inside Another Method?

Public Class DailyJob(){
public void runJob(ScheduleJob currentJob) {
try {
int employee = employeeService.getEmployeeNum();
JobPerformance jobPerformance = performanceService.searchJobPerformance(employee);
if(jobPerformance.size() >0 ) {
currentJob.setRecord("success");
}
else {
currentJob.setRecord("failed");
}
}
catch{
//catch error
}
}
}
By now I want to write a Junit test for runJob(). But how can I setup the value I prefer into 'employee' parameter in Junit Test while I test runJob()? Anyone know how to implement it?
You can use Mockito. You setup stubbing, which will return desired value. Also you can stub for multiple calls by
.thenReturn(9, 10, 7);
import org.junit.jupiter.api.Test;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class MyTest {
#Test
void test() {
EmployeeService employeeService = mock(EmployeeService.class);
when(employeeService.getEmployeeNum()).thenReturn(9);
}
}

Retry a specific piece of code in a TestNG code if the test fails, not the whole test

I want to retry some method specific code if the TestNG test fails, and not retry the whole test.
I have looked into custom test Listeners and Retry analyzers from here https://github.com/haojiwu/testng-retry-example. This will rerun the whole test if it fails I want to some method specific code.
#Test()
public void testStuff() {
String var1;
String var2;
/* Code specific to this test and should only be ran once. */
doSomething(param1)
/* Contains code looking for files that may or not be ready by the time the code is ran. */
/* I want to try this code then wait and retry up to a max of 3 retries. */
assertStuff(var1, var2);
}
I want to do the code in doSomething(param1), the try the stuff in assertStuff(var1, var2) if the asserts in that fail I want to wait like 5 seconds and then retry the assertStuff(var1, var2) code if the asserts pass then the test pass, else retry up to 2 more times.
You probably want to use TestNG retryAnalyzer feature, but only for the assertStuff(var1, var2) part.
Therefore you should move the doSomething(param1) into a separate method, annotated with #BeforeClass (or #BeforeMethod):
...
#BeforeClass
public void initSomething() {
doSomething(param1);
}
#Test(retryAnalyzer = ThreeRetries.class)
public void testStuff() {
String var1;
String var2;
assertStuff(var1, var2);
}
Alternatively you can use #Test annotation for mandatory part and declare dependency between tests, e.g.:
...
#Test
public void initSomething() {
doSomething(param1);
}
#Test(retryAnalyzer = ThreeRetries.class, dependsOnMethods = "initSomething")
public void testStuff() {
String var1;
String var2;
assertStuff(var1, var2);
}
An easier way of getting this done without having to depend on RetryAnalyzer (one issue with RetryAnalyzer if you are not using the latest beta version viz., 7.0.0-beta5 or higher is that you would need to take responsibility of pruning the duplicate test results that result from the retrial).
You could instead build a simple polling mechanism around your assertStuff() method and have it return true or false which you can assert.
Here's an example. For the sake of ease, I have leveraged using org.openqa.selenium.support.ui.FluentWait since its APIs are pretty concise
import java.time.Duration;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Function;
import java.util.function.Supplier;
import org.openqa.selenium.TimeoutException;
import org.openqa.selenium.support.ui.FluentWait;
import org.testng.Assert;
import org.testng.annotations.Test;
public class TestClass {
private static final int max = 5;
#Test
public void testStuff() {
doSomething();
boolean wasAsserted = assertStuff(2);
Assert.assertTrue(wasAsserted);
}
#Test
public void testStuffThatFails() {
doSomething();
boolean wasAsserted = assertStuff(10);
Assert.assertTrue(wasAsserted);
}
private boolean assertStuff(int retries) {
final AtomicInteger integer = new AtomicInteger(0);
Supplier<Integer> input = integer::getAndIncrement;
FluentWait<Supplier<Integer>> checker =
new FluentWait<>(input)
.pollingEvery(Duration.ofSeconds(2))
.withTimeout(Duration.ofSeconds(10));
try {
return checker.until(getCondition(retries));
} catch (TimeoutException e) {
return false;
}
}
private Function<Supplier<Integer>, Boolean> getCondition(int retries) {
return integerSupplier -> {
int current = integerSupplier.get();
return current == retries || current > max;
};
}
private void doSomething() {
System.err.println("I did something");
}
}
In simple, we try like this also
int iterate=5;
boolean status=false;
while(iterate >=1 && status ==false) {
try {
assertEquals("false", "true");
status=true;
}catch (AssertionError e) {
iterate--;
Thread.sleep(500);
}
}

Passing parameter to #Before setup in jUnit test [duplicate]

This question already has answers here:
Is it possible to use different #Before #After for each test case in JUnit?
(2 answers)
Closed 8 years ago.
Is there any way to avoid calling populateRandomData() method at the begining of each test without having a fixed parameter 100. I need to call the same method to setup data before execution of each test but I need to change the number of test data entries e.g. 100 in each case .
public class Tester
{
#Before
public void setUp() {
populateRandomData(100)
}
#Test
public void testMethod() {
}
private void populateRandomData(n){
//n times insert random data in table.
}
}
You can create Parameterized JUnit Test which allows you to add number of parameters you want to pass in unit test case. Have a look at example tutorial Create Parameterized Test Case.
OR
#Rule, using this annotations on your test methods to parameterize the execution of your rules makes it even more useful. Taken from JUnit 4.7 #Rules
EDIT :
Example of Using #Rule :
Below is the class which allows you to initialize different value of num variable which will be used in test method :
import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
public class Test1 implements TestRule {
private final int num;
public Test1(int num) {
this.num = num;
}
public int getNum() {
return num;
}
public class Test1Statement extends Statement {
private final Statement statement;
public Test1Statement(Statement statement, int num) {
this.statement = statement;
}
#Override
public void evaluate() throws Throwable {
statement.evaluate();
}
}
#Override
public Statement apply(Statement statement, Description description) {
return new Test1Statement(statement, num);
}
}
The class below is the actual test case class. It contains JUnit test cases & set value of num variable in test method.
import org.junit.Rule;
import org.junit.Test;
public class RuleNumberTester {
#Rule
public Test1 test = null;
#Rule
public Test1 test1 = null;
#Test
public void num1Test() {
test = new Test1(111);
System.out.println("Num 1 : " + test.getNum());
}
#Test
public void num2Test() {
test1 = new Test1(222);
System.out.println("Num 2 : " + test1.getNum());
}
}
Output :
Test cases are executed successfully & shows the values of num variable which was initialized in test methods on console.
Num 1 : 111
Num 2 : 222
I suppose you could use a #Rule to ensure populateRandomData() is called each time with the correct parameters.
However, this gets ugly quickly since you then need to maintain a list of test method names.
private static final Map<String, Integer> dataCounts = new HashMap<>();
static {
// list (or otherwise obtain) counts here
dataCounts.put("testMethod", 100);
}
#Rule
public TestWatcher watcher = new TestWatcher() {
#Override
protected void starting(Description description) {
Integer count = dataCounts.get(description.getMethodName());
assertNotNull(count);
populateRandomData(count.intValue());
};
};

JUnit #Ignore all other tests (#IgnoreOther ?)

I'm testing extensively with JUnit and sometimes - while debugging my code - I want (temporary) only run a single #Test of my #RunWith(Arquillian.class) test class. Currently I'm adding a #Ignore to all other tests and wondering if something like #IgnoreOther does exist.
Are there better solutions to ignore all other tests?
The simplest way is to replace all #Test to //###$$$#Test. Then when your debugging is finished replace //###$$$#Test to #Test.
Moreover typically IDEs allow running one test only. For example in Eclipse you can do it from Outline view.
Just my two cents. You can try to use Junit Rules as #srkavin suggested.
Here is an example.
package org.foo.bar;
import org.junit.rules.MethodRule;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.Statement;
public class SingleTestRule implements MethodRule {
private String applyMethod;
public SingleTestRule(String applyMethod) {
this.applyMethod = applyMethod;
}
#Override
public Statement apply(final Statement statement, final FrameworkMethod method, final Object target) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
if (applyMethod.equals(method.getName())) {
statement.evaluate();
}
}
};
}
}
package org.foo.bar;
import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
public class IgnoreAllTest {
#Rule
public SingleTestRule test = new SingleTestRule("test1");
#Test
public void test1() throws Exception {
System.out.println("test1");
}
#Test
public void test2() throws Exception {
Assert.fail("test2");
}
#Test
public void test3() throws Exception {
Assert.fail("test3");
}
}
Test rules (JUnit 4.7+) will help. For example, you can write a rule that ignores all #Test methods except one with a specific name.
The answer from srkavin (and mijer) is correct, but the code is deprecated from JUnit 4.9. The interface and the method signature have changed. I want to provide this for others interested in this issue.
public class IgnoreOtherRule implements TestRule
{
private String applyMethod;
public IgnoreOtherRule(String applyMethod){
this.applyMethod = applyMethod;
}
#Override
public Statement apply(final Statement statement, final Description description)
{
return new Statement()
{
#Override
public void evaluate() throws Throwable {
if (applyMethod.equals(description.getMethodName())) {
statement.evaluate();
}
}
};
}
}

Detect Failure or Error of Junit Test in #After method

Is there a way in JUnit to detect within an #After annotated method if there was a test failure or error in the test case?
One ugly solution would be something like that:
boolean withoutFailure = false;
#Test
void test() {
...
asserts...
withoutFailure = true;
}
#After
public void tearDown() {
if(!withoutFailuere) {
this.dontReuseTestenvironmentForNextTest();
}
}
This is ugly because one need to take care of the "infrastructure" (withoutFailure flag) in the test code.
I hope that there is something where I can get the test status in the #After method!?
If you are lucky enough to be using JUnit 4.9 or later, TestWatcher will do exactly what you want.
Share and Enjoy!
I extend dsaff's answer to solve the problem that a TestRule can not execute some code snipped between the execution of the test-method and the after-method. So with a simple MethodRule one can not use this rule to provide a success flag that is use in the #After annotated methods.
My idea is a hack! Anyway, it is to use a TestRule (extends TestWatcher). A TestRule will get knowledge about failed or success of a test. My TestRule will then scan the class for all Methods annotated with my new AfterHack annotations and invoke that methods with a success flag.
AfterHack annotation
import static java.lang.annotation.ElementType.METHOD;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;
#Retention(RUNTIME)
#Target(METHOD)
public #interface AfterHack {}
AfterHackRule
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
public class AfterHackRule extends TestWatcher {
private Object testClassInstance;
public AfterHackRule(final Object testClassInstance) {
this.testClassInstance = testClassInstance;
}
protected void succeeded(Description description) {
invokeAfterHackMethods(true);
}
protected void failed(Throwable e, Description description) {
invokeAfterHackMethods(false);
}
public void invokeAfterHackMethods(boolean successFlag) {
for (Method afterHackMethod :
this.getAfterHackMethods(this.testClassInstance.getClass())) {
try {
afterHackMethod.invoke(this.testClassInstance, successFlag);
} catch (IllegalAccessException | IllegalArgumentException
| InvocationTargetException e) {
throw new RuntimeException("error while invoking afterHackMethod "
+ afterHackMethod);
}
}
}
private List<Method> getAfterHackMethods(Class<?> testClass) {
List<Method> results = new ArrayList<>();
for (Method method : testClass.getMethods()) {
if (method.isAnnotationPresent(AfterHack.class)) {
results.add(method);
}
}
return results;
}
}
Usage:
public class DemoTest {
#Rule
public AfterHackRule afterHackRule = new AfterHackRule(this);
#AfterHack
public void after(boolean success) {
System.out.println("afterHack:" + success);
}
#Test
public void demofails() {
Assert.fail();
}
#Test
public void demoSucceeds() {}
}
BTW:
1) Hopefully there is a better solution in Junit5
2) The better way is to use the TestWatcher Rule instead of the #Before and #After Method at all (that is the way I read dsaff's answer)
#see
I don't know any easy or elegant way to detect the failure of a Junit test in an #After method.
If it is possible to use a TestRule instead of an #After method, one possibility to do it is using two chained TestRules, using a TestWatcher as the inner rule.
Example:
package org.example;
import static org.junit.Assert.fail;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExternalResource;
import org.junit.rules.RuleChain;
import org.junit.rules.TestRule;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
public class ExampleTest {
private String name = "";
private boolean failed;
#Rule
public TestRule afterWithFailedInformation = RuleChain
.outerRule(new ExternalResource(){
#Override
protected void after() {
System.out.println("Test "+name+" "+(failed?"failed":"finished")+".");
}
})
.around(new TestWatcher(){
#Override
protected void finished(Description description) {
name = description.getDisplayName();
}
#Override
protected void failed(Throwable e, Description description) {
failed = true;
}
})
;
#Test
public void testSomething(){
fail();
}
#Test
public void testSomethingElse(){
}
}

Categories