We Keep Coding

Cucumber Java: How to force JSON plugin generation

Overview: There are instances where in I want to stop the running cucumber test pack midway -- say for example when x number of tests failed.
I can do this just fine but I want the json file (plugin = {json:...}) to be generated when the test stops. Is this doable?
What I've tried so far:
Debug and see where the reporting / plugin generation happens. It seems to be when this line executes:
Cucumber.java: runtime.getEventBus().send.....
#Override
protected Statement childrenInvoker(RunNotifier notifier) {
final Statement features = super.childrenInvoker(notifier);
return new Statement() {
#Override
public void evaluate() throws Throwable {
features.evaluate();
runtime.getEventBus().send(new TestRunFinished(runtime.getEventBus().getTime()));
runtime.printSummary();
}
};
}
I was hoping to access the runtime field but it has a private modifier. I also tried accessing it via reflections but I'm not exactly getting what I need.

Found a quite dirty, but working solution and got what I need. Posting my solution here in case anyone might need.
Create a custom cucumber runner implementation to take the runtime instance.
public final class Foo extends Cucumber {
static Runtime runtime;
/**
* Constructor called by JUnit.
*
* #param clazz the class with the #RunWith annotation.
* #throws IOException if there is a problem
* #throws InitializationError if there is another problem
*/
public Foo(Class clazz) throws InitializationError, IOException {
super(clazz);
}
#Override
protected Runtime createRuntime(ResourceLoader resourceLoader, ClassLoader classLoader, RuntimeOptions runtimeOptions) throws InitializationError, IOException {
runtime = super.createRuntime(resourceLoader, classLoader, runtimeOptions);
return runtime;
}
}
Call the same line that generates the file depending on the plugin used:
public final class ParentHook {
#Before
public void beforeScenario(Scenario myScenario) {
}
#After
public void afterScenario() {
if (your condition to stop the test) {
//custom handle to stop the test
myHandler.pleaseStop();
Foo.runtime.getEventBus().send(new TestRunFinished(Foo.runtime.getEventBus().getTime()));
}
}
}
This will however require you to run your test via Foo.class eg:
#RunWith(Foo.class) instead of #RunWith(Cucumber.class)
Not so much value here but it fits what I need at the moment. I hope Cucumber provides a way to do this out of the box. If there's a better way, please do post it here so I can accept your answer once verified.

Why not quit?
import cucumber.api.Scenario;
import cucumber.api.java.After;
import cucumber.api.java.Before;
import cucumber.api.java.en.When;
public class StepDefinitions {
private static int failureCount = 0;
private int threshold = 20;
#When("^something$")
public void do_something() {
// something
}
#After
public void after(Scenario s) {
if (s.isFailed()) ++failureCount;
}
#Before
public void before() {
if (failureCount > threshold) {
if (driver !=null) {
driver.quit();
driver = null;
}
}
}

Testing ClassNotFound Exception

I'm trying to test that a class is not found with UnitTest on Android.
What's going on:
1. I'm writing an android library with transitive dependencies which are resolved in the host application
2. The developer may remove some dependencies for example remove all com.example.package
3. I have a Factory that will try to instantiate (using reflection) an Object and catch the ClassNotFoundException. If the developer remove the dependencies, the exception should be thrown.
4. I want to test this case, but all I found is issue with dependencies, not how to test for it.
Example code I want to test
try {
sNetworkResponseBuilderClass = OkHttpNetworkResponse.Builder.class;
} catch (Exception e){
// <<<< I want to test this case
new ClassNotFoundException("Unable to find OkHttpNetworkResponse.Builder.class").printStackTrace();
return null;
}
library used: hamcrast, mockito, JUnit 4.
Do you know how to do it?

So for me the first thing you need to do is to extract the part of the code that can throw a ClassNotFoundException in order to be able to easily mock it, something like:
public Class<? extends NetworkResponseBuilder> getNetworkResponseBuilderClass()
throws ClassNotFoundException {
// Your logic here
}
Then you can test a real factory instance using Mockito.spy to be able to redefine the behavior of the method getNetworkResponseBuilderClass() as next:
public void testFactoryIfNetworkResponseBuilderNotFound() {
Factory factory = spy(new Factory());
when(factory.getNetworkResponseBuilderClass()).thenThrow(
new ClassNotFoundException()
);
// The rest of your test here
}
public void testFactoryIfNetworkResponseBuilderFound() {
Factory factory = spy(new Factory());
when(factory.getNetworkResponseBuilderClass()).thenReturn(
OkHttpNetworkResponse.Builder.class
);
// The rest of your test here
}
More details about Mockito.spy.

Not quite sure if I understood your question correctly, but you can check with JUnit if an exception gets thrown:
#Test(expected=ClassNotFoundException.class)
public void testClassNotFoundException() {
// a case where the exception gets thrown
}

OkHttpNetworkResponse.Builder might be as follows:
package com.example.model;
public class OkHttpNetworkResponse {
public static class Builder {
}
}
I have a Factory that will try to instantiate (using reflection) an Object and catch the ClassNotFoundException. If the developer remove
the dependencies, the exception should be thrown.
Factory Class: which will create any object might be as follows:
package com.example.factory;
public class Factory {
public static Object getInstance(String className)
throws ClassNotFoundException, InstantiationException,
IllegalAccessException {
Class clazz = Class.forName(className);
return clazz.newInstance();
}
}
The developer may remove some dependencies for example remove all com.example.package
I want to test this case, but all I found is issue with dependencies, not how to test for it.
FactoryTest Class: which will test whether ClassNotFoundException is thrown or not might be as follows: N.B: please Check the comments carefully.
package com.example.factory;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import java.io.File;
import java.io.IOException;
import org.junit.Test;
public class FactoryTest {
Factory factory;
#Test(expected=ClassNotFoundException.class)
public void test() throws IOException, ClassNotFoundException, InstantiationException, IllegalAccessException {
ClassLoader loader = FactoryTest.class.getClassLoader();
String directory = loader.getResource(".").getPath() + "/com/example/model";
File dir = new File(directory);
//Checking directory already existed or not..
assertTrue("Directory:"+dir.getPath()+" not exist",dir.exists());
//Deleting directory
deleteDirectoryProgramatically(directory);
//Checking directory already deleted or not..
assertFalse("Directory:"+dir.getPath()+" still exist",dir.exists());
//Now getInstance Method will throw ClassNotFoundException because OkHttpNetworkResponse.Builder.class has been deleted programatically.
Factory.getInstance("OkHttpNetworkResponse.Builder.class");
}
private void deleteDirectoryProgramatically(String directory) {
File dir = new File(directory);
System.out.println(dir.getAbsolutePath());
String[] files = dir.list();
for (String f : files) {
File fl = new File(directory,f);
System.out.println(f+ " deleted?"+fl.delete());
}
System.out.println(dir+ " deleted?"+dir.delete());
}
}

It is very simple issue. JUnit4 exception unit testing is given below with an example. Hope it will clarify you.
MyNumber.java
public class MyNumber {
int number;
public MyNumber div(MyNumber rhs) {
if (rhs.number == 0) throw new IllegalArgumentException("Cannot divide by 0!");
this.number /= rhs.number;
return this;
}
}
MyNumberTest.java
public class MyNumberTest {
private MyNumber number1, number2; // Test fixtures
#Test(expected = IllegalArgumentException.class)
public void testDivByZero() {
System.out.println("Run #Test testDivByZero"); // for illustration
number2.setNumber(0);
number1.div(number2);
}
}
JUnit - Exceptions Test
To test if the code throws a desired exception, use annotation #Test(expected = exception.class), as illustrated in the previous example. For your case it will be
/**
* Check for class not found exception
**/
#Test(expected=ClassNotFoundException.class)
public void testClassNotFoundException() {
.....
}
For better understanding, you can go through this tutorial: Java Unit
Testing - JUnit & TestNG. It contains full running code example
step by step with explanation.

inside catch you can check the object with the instanceof operator as :
try {
sNetworkResponseBuilderClass = OkHttpNetworkResponse.Builder.class;
} catch (Exception e){
if(e instanceof ClassNotFoundException){
// here you can do the code you want in case of ClassNotFoundException thrown
}
}

it is your dictionary problem. in your dictionary in test class will not have . change your dictionary.

Use Class.forName("com.example.ClassName")
try {
Class.forName("com.example.OkHttpNetworkResponse.Builder");
} catch (ClassNotFoundException e) {
// This class was not found
}
See Class.forName(String className)

JTest 9.5. How to execute preparation actions before each unit test

I'm using JTest 9.5. I wanted to ask whether there is possibility to perform same preparation routine before each unit test, just like I did it in JUnit with #Before and #After annotations? If yes, then how? Let's say I have following unit test class in JTest:
public class TestArrayFileManager extends PackageTestCase {
FileManager fileMngr;
public TestArrayFileManager(String name)
{
super(name);
}
public Class getTestedClass()
{
return FileManager.class;
}
public void testFileManager1() throws Throwable
{
final String fileName = "InputFile.txt";
fileMngr = new FileManager(fileName);
fileMngr.doResetFile();
fileMngr.doReplaceNthElement(0, 3);
fileMngr.doReplaceNthElement(1, 9);
assertEquals(3, fileMngr.doReadNthElement(0L));
}
public void testFileManager2() throws Throwable
{
final String fileName = "InputFile.txt";
fileMngr = new FileManager(fileName);
fileMngr.doResetFile();
fileMngr.doReplaceNthElement(0, 3);
fileMngr.doReplaceNthElement(1, 9);
assertEquals(9, fileMngr.doReadNthElement(1L));
}
}
Notice how i keep repeating same preparation code. How can I perform it before each test?

JTest complements and extends JUnit which means that it is not there to provide the features of JUnit. You will have to use JTest with Junit in order to achieve what you want.
You can use existing JUnit test cases with JTest which will provide both setUp and tearDown methods using #Before and #After annotations.
If you want to use a JUnit Test Class with Jtest, you need to:
Include the junit.jar file on your CLASSPATH.
Make sure Jtest knows how to locate the Test Class (as described in Loading Test Classes below).
After you perform these steps, Jtest will use the JUnit Test Class when you run your test in the normal manner.

I just fount the methods I have to add in order to achieve preparation and ending tasks before each unit test. These are the methods I found in generated unit tests file and they are working:
public void setUp() throws Exception {
super.setUp();
/*
* Add any necessary initialization code here (e.g., open a socket).
* Call Repository.putTemporary() to provide initialized instances of
* objects to be used when testing.
*/
// jtest.Repository.putTemporary("name", object);
}
/**
* Used to clean up after the test. This method is called by JUnit after
* each of the tests have been completed.
*
* #see junit.framework.TestCase#tearDown()
* #author Parasoft Jtest 9.5
*/
public void tearDown() throws Exception {
try {
/*
* Add any necessary cleanup code here (e.g., close a socket).
*/
} finally {
super.tearDown();
}
}

No clue why I'm getting a illegal character error

For some reason in the testEmployee method I am getting weird error messages saying illegal character.
I've looked at the code but can't find anything syntactically wrong with it.
Any help would be great thanks!
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
import ttp.Cache;
import ttp.DataException;
import ttp.EmployeeDAO;
import ttp.Employee;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import static org.junit.Assert.*;
/**
*
* #author ericrea
*/
public class Tester {
public Tester() {
}
#BeforeClass
public static void setUpClass() throws Exception {
}
#AfterClass
public static void tearDownClass() throws Exception {
}
#Before
public void setUp() {
}
#After
public void tearDown() {
}
// TODO add test methods here.
// The methods must be annotated with annotation #Test. For example:
//
// #Test
// public void hello() {}
// /** Test the Skeleton BO/DAO */
// #Test
// public void TestSkeleton() throws Exception {
// Skeleton s = SkeletonDAO.getInstance().create(“skeleton1”);
// s.save();
//
// // since skeleton1 is in the Cache, this tests reading from the cache
// Skeleton s2 = SkeletonDAO.getInstance().read(“skeleton1”);
// assertSame(s, s2);
//
// // now clear the cache (you’d never do this in the real world)
// // then we can test reading from the database
// Cache.getInstance().clear();
// Skeleton s3 = SkeletonDAO.getInstance().read(“skeleton1”);
// assertEquals(s.getId(), s3.getId());
// //assertEquals(s.getSomething(), s3.getSomething());
//
// // keep testing more methods, BOs, and DAOs
// }//TestSkeleton
// //5. Right-click the file and run the file. You’ll see it test. It will obviously have errors
// //because Skeleton isn’t a real BO/DAO.
// //6. Note in the above code the “assertEquals” methods. That is how you test that two things are equals.
#Test
public void TestEmployee() throws DataException {
Employee e = EmployeeDAO.getInstance().create(“Employee1”);
e.save();
// since skeleton1 is in the Cache, this tests reading from the cache
Employee e2 = EmployeeDAO.getInstance().read(“Employee1”);
assertSame(e, e2);
// now clear the cache (you’d never do this in the real world)
// then we can test reading from the database
Cache.getInstance().clear();
Employee e3 = EmployeeDAO.getInstance().read(“Employee1”);
assertEquals(e.getId(), e3.getId());
assertEquals(e.getName1(), s]e3.getName1());
}

You seem to use 'fancy' quotes (“) instead of normal ones (")
PS If Matt is right, please do not use Word to edit your programs. There're lots of IDEs, but even Notepad would be easier.

The fragment
s]e3.getName1()
(last line) doesn't look right: remove the "s]" fragment.
Edit: You are also missing the closing parenthesis for the class at the end of the code.

Easy way of running the same junit test over and over?

Like the title says, I'm looking for some simple way to run JUnit 4.x tests several times in a row automatically using Eclipse.
An example would be running the same test 10 times in a row and reporting back the result.
We already have a complex way of doing this but I'm looking for a simple way of doing it so that I can be sorta sure that the flaky test I've been trying to fix stays fixed.
An ideal solution would be an Eclipse plugin/setting/feature that I am unaware of.

The easiest (as in least amount of new code required) way to do this is to run the test as a parametrized test (annotate with an #RunWith(Parameterized.class) and add a method to provide 10 empty parameters). That way the framework will run the test 10 times.
This test would need to be the only test in the class, or better put all test methods should need to be run 10 times in the class.
Here is an example:
#RunWith(Parameterized.class)
public class RunTenTimes {
#Parameterized.Parameters
public static Object[][] data() {
return new Object[10][0];
}
public RunTenTimes() {
}
#Test
public void runsTenTimes() {
System.out.println("run");
}
}
With the above, it is possible to even do it with a parameter-less constructor, but I'm not sure if the framework authors intended that, or if that will break in the future.
If you are implementing your own runner, then you could have the runner run the test 10 times. If you are using a third party runner, then with 4.7, you can use the new #Rule annotation and implement the MethodRule interface so that it takes the statement and executes it 10 times in a for loop. The current disadvantage of this approach is that #Before and #After get run only once. This will likely change in the next version of JUnit (the #Before will run after the #Rule), but regardless you will be acting on the same instance of the object (something that isn't true of the Parameterized runner). This assumes that whatever runner you are running the class with correctly recognizes the #Rule annotations. That is only the case if it is delegating to the JUnit runners.
If you are running with a custom runner that does not recognize the #Rule annotation, then you are really stuck with having to write your own runner that delegates appropriately to that Runner and runs it 10 times.
Note that there are other ways to potentially solve this (such as the Theories runner) but they all require a runner. Unfortunately JUnit does not currently support layers of runners. That is a runner that chains other runners.

With IntelliJ, you can do this from the test configuration. Once you open this window, you can choose to run the test any number of times you want,.
when you run the test, intellij will execute all tests you have selected for the number of times you specified.
Example running 624 tests 10 times:

With JUnit 5 I was able to solve this using the #RepeatedTest annotation:
#RepeatedTest(10)
public void testMyCode() {
//your test code goes here
}
Note that #Test annotation shouldn't be used along with #RepeatedTest.

I've found that Spring's repeat annotation is useful for that kind of thing:
#Repeat(value = 10)
Latest (Spring Framework 4.3.11.RELEASE API) doc:
org.springframework.test.annotation
Unit Testing in Spring

Inspired by the following resources:
blog post
this solution
commented version
Example
Create and use a #Repeat annotation as follows:
public class MyTestClass {
#Rule
public RepeatRule repeatRule = new RepeatRule();
#Test
#Repeat(10)
public void testMyCode() {
//your test code goes here
}
}
Repeat.java
import static java.lang.annotation.ElementType.ANNOTATION_TYPE;
import static java.lang.annotation.ElementType.METHOD;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
#Retention( RetentionPolicy.RUNTIME )
#Target({ METHOD, ANNOTATION_TYPE })
public #interface Repeat {
int value() default 1;
}
RepeatRule.java
import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
public class RepeatRule implements TestRule {
private static class RepeatStatement extends Statement {
private final Statement statement;
private final int repeat;
public RepeatStatement(Statement statement, int repeat) {
this.statement = statement;
this.repeat = repeat;
}
#Override
public void evaluate() throws Throwable {
for (int i = 0; i < repeat; i++) {
statement.evaluate();
}
}
}
#Override
public Statement apply(Statement statement, Description description) {
Statement result = statement;
Repeat repeat = description.getAnnotation(Repeat.class);
if (repeat != null) {
int times = repeat.value();
result = new RepeatStatement(statement, times);
}
return result;
}
}
PowerMock
Using this solution with #RunWith(PowerMockRunner.class), requires updating to Powermock 1.6.5 (which includes a patch).

Anything wrong with:
#Test
void itWorks() {
// stuff
}
#Test
void itWorksRepeatably() {
for (int i = 0; i < 10; i++) {
itWorks();
}
}
Unlike the case where you are testing each of an array of values, you don't particularly care which run failed.
No need to do in configuration or annotation what you can do in code.

This works much easier for me.
public class RepeatTests extends TestCase {
public static Test suite() {
TestSuite suite = new TestSuite(RepeatTests.class.getName());
for (int i = 0; i < 10; i++) {
suite.addTestSuite(YourTest.class);
}
return suite;
}
}

There's an Intermittent annotation in the tempus-fugit library which works with JUnit 4.7's #Rule to repeat a test several times or with #RunWith.
For example,
#RunWith(IntermittentTestRunner.class)
public class IntermittentTestRunnerTest {
private static int testCounter = 0;
#Test
#Intermittent(repition = 99)
public void annotatedTest() {
testCounter++;
}
}
After the test is run (with the IntermittentTestRunner in the #RunWith), testCounter would be equal to 99.

This is essentially the answer that Yishai provided above, re-written in Kotlin :
#RunWith(Parameterized::class)
class MyTest {
companion object {
private const val numberOfTests = 200
#JvmStatic
#Parameterized.Parameters
fun data(): Array<Array<Any?>> = Array(numberOfTests) { arrayOfNulls<Any?>(0) }
}
#Test
fun testSomething() { }
}

I build a module that allows do this kind of tests. But it is focused not only in repeat. But in guarantee that some piece of code is Thread safe.
https://github.com/anderson-marques/concurrent-testing
Maven dependency:
<dependency>
<groupId>org.lite</groupId>
<artifactId>concurrent-testing</artifactId>
<version>1.0.0</version>
</dependency>
Example of use:
package org.lite.concurrent.testing;
import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
import ConcurrentTest;
import ConcurrentTestsRule;
/**
* Concurrent tests examples
*/
public class ExampleTest {
/**
* Create a new TestRule that will be applied to all tests
*/
#Rule
public ConcurrentTestsRule ct = ConcurrentTestsRule.silentTests();
/**
* Tests using 10 threads and make 20 requests. This means until 10 simultaneous requests.
*/
#Test
#ConcurrentTest(requests = 20, threads = 10)
public void testConcurrentExecutionSuccess(){
Assert.assertTrue(true);
}
/**
* Tests using 10 threads and make 20 requests. This means until 10 simultaneous requests.
*/
#Test
#ConcurrentTest(requests = 200, threads = 10, timeoutMillis = 100)
public void testConcurrentExecutionSuccessWaitOnly100Millissecond(){
}
#Test(expected = RuntimeException.class)
#ConcurrentTest(requests = 3)
public void testConcurrentExecutionFail(){
throw new RuntimeException("Fail");
}
}
This is a open source project. Feel free to improve.

You could run your JUnit test from a main method and repeat it so many times you need:
package tests;
import static org.junit.Assert.*;
import org.junit.Test;
import org.junit.runner.Result;
public class RepeatedTest {
#Test
public void test() {
fail("Not yet implemented");
}
public static void main(String args[]) {
boolean runForever = true;
while (runForever) {
Result result = org.junit.runner.JUnitCore.runClasses(RepeatedTest.class);
if (result.getFailureCount() > 0) {
runForever = false;
//Do something with the result object
}
}
}
}