I'm trying to test that a class is not found with UnitTest on Android.
What's going on:
1. I'm writing an android library with transitive dependencies which are resolved in the host application
2. The developer may remove some dependencies for example remove all com.example.package
3. I have a Factory that will try to instantiate (using reflection) an Object and catch the ClassNotFoundException. If the developer remove the dependencies, the exception should be thrown.
4. I want to test this case, but all I found is issue with dependencies, not how to test for it.
Example code I want to test
try {
sNetworkResponseBuilderClass = OkHttpNetworkResponse.Builder.class;
} catch (Exception e){
// <<<< I want to test this case
new ClassNotFoundException("Unable to find OkHttpNetworkResponse.Builder.class").printStackTrace();
return null;
}
library used: hamcrast, mockito, JUnit 4.
Do you know how to do it?
So for me the first thing you need to do is to extract the part of the code that can throw a ClassNotFoundException in order to be able to easily mock it, something like:
public Class<? extends NetworkResponseBuilder> getNetworkResponseBuilderClass()
throws ClassNotFoundException {
// Your logic here
}
Then you can test a real factory instance using Mockito.spy to be able to redefine the behavior of the method getNetworkResponseBuilderClass() as next:
public void testFactoryIfNetworkResponseBuilderNotFound() {
Factory factory = spy(new Factory());
when(factory.getNetworkResponseBuilderClass()).thenThrow(
new ClassNotFoundException()
);
// The rest of your test here
}
public void testFactoryIfNetworkResponseBuilderFound() {
Factory factory = spy(new Factory());
when(factory.getNetworkResponseBuilderClass()).thenReturn(
OkHttpNetworkResponse.Builder.class
);
// The rest of your test here
}
More details about Mockito.spy.
Not quite sure if I understood your question correctly, but you can check with JUnit if an exception gets thrown:
#Test(expected=ClassNotFoundException.class)
public void testClassNotFoundException() {
// a case where the exception gets thrown
}
OkHttpNetworkResponse.Builder might be as follows:
package com.example.model;
public class OkHttpNetworkResponse {
public static class Builder {
}
}
I have a Factory that will try to instantiate (using reflection) an Object and catch the ClassNotFoundException. If the developer remove
the dependencies, the exception should be thrown.
Factory Class: which will create any object might be as follows:
package com.example.factory;
public class Factory {
public static Object getInstance(String className)
throws ClassNotFoundException, InstantiationException,
IllegalAccessException {
Class clazz = Class.forName(className);
return clazz.newInstance();
}
}
The developer may remove some dependencies for example remove all com.example.package
I want to test this case, but all I found is issue with dependencies, not how to test for it.
FactoryTest Class: which will test whether ClassNotFoundException is thrown or not might be as follows: N.B: please Check the comments carefully.
package com.example.factory;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import java.io.File;
import java.io.IOException;
import org.junit.Test;
public class FactoryTest {
Factory factory;
#Test(expected=ClassNotFoundException.class)
public void test() throws IOException, ClassNotFoundException, InstantiationException, IllegalAccessException {
ClassLoader loader = FactoryTest.class.getClassLoader();
String directory = loader.getResource(".").getPath() + "/com/example/model";
File dir = new File(directory);
//Checking directory already existed or not..
assertTrue("Directory:"+dir.getPath()+" not exist",dir.exists());
//Deleting directory
deleteDirectoryProgramatically(directory);
//Checking directory already deleted or not..
assertFalse("Directory:"+dir.getPath()+" still exist",dir.exists());
//Now getInstance Method will throw ClassNotFoundException because OkHttpNetworkResponse.Builder.class has been deleted programatically.
Factory.getInstance("OkHttpNetworkResponse.Builder.class");
}
private void deleteDirectoryProgramatically(String directory) {
File dir = new File(directory);
System.out.println(dir.getAbsolutePath());
String[] files = dir.list();
for (String f : files) {
File fl = new File(directory,f);
System.out.println(f+ " deleted?"+fl.delete());
}
System.out.println(dir+ " deleted?"+dir.delete());
}
}
It is very simple issue. JUnit4 exception unit testing is given below with an example. Hope it will clarify you.
MyNumber.java
public class MyNumber {
int number;
public MyNumber div(MyNumber rhs) {
if (rhs.number == 0) throw new IllegalArgumentException("Cannot divide by 0!");
this.number /= rhs.number;
return this;
}
}
MyNumberTest.java
public class MyNumberTest {
private MyNumber number1, number2; // Test fixtures
#Test(expected = IllegalArgumentException.class)
public void testDivByZero() {
System.out.println("Run #Test testDivByZero"); // for illustration
number2.setNumber(0);
number1.div(number2);
}
}
JUnit - Exceptions Test
To test if the code throws a desired exception, use annotation #Test(expected = exception.class), as illustrated in the previous example. For your case it will be
/**
* Check for class not found exception
**/
#Test(expected=ClassNotFoundException.class)
public void testClassNotFoundException() {
.....
}
For better understanding, you can go through this tutorial: Java Unit
Testing - JUnit & TestNG. It contains full running code example
step by step with explanation.
inside catch you can check the object with the instanceof operator as :
try {
sNetworkResponseBuilderClass = OkHttpNetworkResponse.Builder.class;
} catch (Exception e){
if(e instanceof ClassNotFoundException){
// here you can do the code you want in case of ClassNotFoundException thrown
}
}
it is your dictionary problem. in your dictionary in test class will not have . change your dictionary.
Use Class.forName("com.example.ClassName")
try {
Class.forName("com.example.OkHttpNetworkResponse.Builder");
} catch (ClassNotFoundException e) {
// This class was not found
}
See Class.forName(String className)
Related
I am learning Spock so this may be very basic.
public Random genRand() {
try {
return SecureRandom.getInstanceStrong();
}
catch (NoSuchAlgorithmException e) {
logger.debug(e.getMessage());
return new SecureRandom();
}
}
What I have tried so far is:
public void setup() {
mockClassName = spy(ClassName)
mockClassName.logger() >> mockLogger
}
def "exception test case"() {
given: "nothing"
when:"method call happens"
mockClassName.genRand()
then:"handle"
SecureRandom.getInstanceStrong()
}
This covers the try block only.
While trying:
public void setup() {
mockClassName = spy(ClassName)
mockClassName.logger() >> mockLogger
}
def "exception test case"() {
given: "nothing"
SecureRandom.getInstanceStrong() >> Exception
when:"method call happens"
mockClassName.genRand()
then:"catch"
NoSuchAlgorithmException e = thrown()
new SecureRandom()
}
This gives the error, expected exception of type java.security.NoSuchAlgorithmException, but no exception was thrown.
Is it possible to do both try and catch in one test case? Feel free to make two.
You should not try to test several branches of your code in one test, so doing it in two tests is actually the right approach. If you try to find a simple description of your test and have to describe several kinds of results, then that's a hint that your approach is not good.
A reminder, you should name your test methods with "It...", for example:
"It returns a default SecureRandom, if no strong instance of SecureRandom can be found"
Another point: You code never leaks a NoSuchAlgorithmException into Spec. This exception is caught in your production code, and then you retunr a valid SecureRandom. You can only use thrown() if your code actually throws an exception, meaning this is a hard expectation. Your test will fail if the exception is not thrown.
Quoting my own comment:
Why don't you want to change the application code in order to make it more testable? You could factor out the SecureRandom.getInstanceStrong() call into an extra method, calling it from genRand(), handling the exception there like before. Then you could stub that method in a spy for the error case, throwing an exception instead of returning a result. That would give you full coverage of all execution paths without the need to mock static methods of JDK classes.
I mean something like this:
package de.scrum_master.stackoverflow.q74797317;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.util.Random;
public class RandomProvider {
public Random genRand() {
try {
return getRandStrong();
}
catch (NoSuchAlgorithmException e) {
System.err.println(e.getMessage());
return new SecureRandom();
}
}
protected SecureRandom getRandStrong() throws NoSuchAlgorithmException {
return SecureRandom.getInstanceStrong();
}
}
package de.scrum_master.stackoverflow.q74797317
import spock.lang.Specification
import java.security.NoSuchAlgorithmException
class RandomProviderTest extends Specification {
def "happy path"() {
given:
RandomProvider randomProvider = new RandomProvider()
expect:
randomProvider.genRand()
}
def "exception test case"() {
given:
RandomProvider randomProvider = Spy() {
getRandStrong() >> { throw new NoSuchAlgorithmException("oops") }
}
expect:
randomProvider.genRand()
}
}
Try it in the Groovy Web Console.
In my IDE, the coverage view looks like this:
Overview: There are instances where in I want to stop the running cucumber test pack midway -- say for example when x number of tests failed.
I can do this just fine but I want the json file (plugin = {json:...}) to be generated when the test stops. Is this doable?
What I've tried so far:
Debug and see where the reporting / plugin generation happens. It seems to be when this line executes:
Cucumber.java: runtime.getEventBus().send.....
#Override
protected Statement childrenInvoker(RunNotifier notifier) {
final Statement features = super.childrenInvoker(notifier);
return new Statement() {
#Override
public void evaluate() throws Throwable {
features.evaluate();
runtime.getEventBus().send(new TestRunFinished(runtime.getEventBus().getTime()));
runtime.printSummary();
}
};
}
I was hoping to access the runtime field but it has a private modifier. I also tried accessing it via reflections but I'm not exactly getting what I need.
Found a quite dirty, but working solution and got what I need. Posting my solution here in case anyone might need.
Create a custom cucumber runner implementation to take the runtime instance.
public final class Foo extends Cucumber {
static Runtime runtime;
/**
* Constructor called by JUnit.
*
* #param clazz the class with the #RunWith annotation.
* #throws IOException if there is a problem
* #throws InitializationError if there is another problem
*/
public Foo(Class clazz) throws InitializationError, IOException {
super(clazz);
}
#Override
protected Runtime createRuntime(ResourceLoader resourceLoader, ClassLoader classLoader, RuntimeOptions runtimeOptions) throws InitializationError, IOException {
runtime = super.createRuntime(resourceLoader, classLoader, runtimeOptions);
return runtime;
}
}
Call the same line that generates the file depending on the plugin used:
public final class ParentHook {
#Before
public void beforeScenario(Scenario myScenario) {
}
#After
public void afterScenario() {
if (your condition to stop the test) {
//custom handle to stop the test
myHandler.pleaseStop();
Foo.runtime.getEventBus().send(new TestRunFinished(Foo.runtime.getEventBus().getTime()));
}
}
}
This will however require you to run your test via Foo.class eg:
#RunWith(Foo.class) instead of #RunWith(Cucumber.class)
Not so much value here but it fits what I need at the moment. I hope Cucumber provides a way to do this out of the box. If there's a better way, please do post it here so I can accept your answer once verified.
Why not quit?
import cucumber.api.Scenario;
import cucumber.api.java.After;
import cucumber.api.java.Before;
import cucumber.api.java.en.When;
public class StepDefinitions {
private static int failureCount = 0;
private int threshold = 20;
#When("^something$")
public void do_something() {
// something
}
#After
public void after(Scenario s) {
if (s.isFailed()) ++failureCount;
}
#Before
public void before() {
if (failureCount > threshold) {
if (driver !=null) {
driver.quit();
driver = null;
}
}
}
I can perform actions on test failure by using:
#After
public void afterTest(Scenario scenario) {
if (scenario.isFailed()) {
/*Do stuff*/
}
}
However some of the actions I need to perform depend on the Exception that was thrown and in what context it was thrown. Is there a way to get the Throwable that caused the test to fail? For example in JUnit I would do this by extending TestWatcher and add a rule to my tests:
#Override
protected void failed(Throwable e, Description description) {
/*Do stuff with e*/
}
However the cucumber-junit iplementation does not allow the use of rules, so this solution would not work with Cucumber.
I don't think I need to explain why accessing a thrown exception on test failure would be useful, however I will still provide an Example:
My test environment is not always stable, so my tests might fail unexpectedly at any moment (there's no specific place I can try to catch the exception since it could occur at any time). When this happens I need the test to reschedule for another attempt, and log the incident so that we can get some good statistical data on the environment instability (when, how frequent, how long etc.)
The problem with the work around suggested by Frank Escobar:
By using reflection to reach into a frameworks internals you're depending on implementation details. This is a bad practice, when ever the framework changes its implementation your code may break as you will observe in Cucumber v5.0.0.
Hooks in Cucumber are designed to manipulate the test execution context before and after a scenario. They're not made to report on the test execution itself. Reporting is cross cutting concern and best managed by using the plugin system.
For example:
package com.example;
import io.cucumber.plugin.ConcurrentEventListener;
import io.cucumber.plugin.event.EventPublisher;
import io.cucumber.plugin.event.Result;
import io.cucumber.plugin.event.Status;
import io.cucumber.plugin.event.TestCase;
import io.cucumber.plugin.event.TestCaseFinished;
public class MyTestListener implements ConcurrentEventListener {
#Override
public void setEventPublisher(EventPublisher publisher) {
publisher.registerHandlerFor(TestCaseFinished.class, this::handleTestCaseFinished);
}
private void handleTestCaseFinished(TestCaseFinished event) {
TestCase testCase = event.getTestCase();
Result result = event.getResult();
Status status = result.getStatus();
Throwable error = result.getError();
String scenarioName = testCase.getName();
String id = "" + testCase.getUri() + testCase.getLine();
System.out.println("Testcase " + id + " - " + status.name());
}
}
When using JUnit 4 and TestNG you can activate this plugin using:
#CucumberOptions(plugin="com.example.MyTestListener")
With JUnit 5 you add it to junit-platform.properties:
cucumber.plugin=com.example.MyTestListener
Or if you are using the CLI
--plugin com.example.MyTestListener
I've implemented this method using reflections. You can't access directly to steps errors (stack trace). I've created this static method which allows you to access to "stepResults" attribute and then you can iterate and get the error and do whatever you want.
import cucumber.runtime.ScenarioImpl;
import gherkin.formatter.model.Result;
import org.apache.commons.lang3.reflect.FieldUtils;
import java.lang.reflect.Field;
import java.util.ArrayList;
#After
public void afterScenario(Scenario scenario) {
if (scenario.isFailed())
logError(scenario);
}
private static void logError(Scenario scenario) {
Field field = FieldUtils.getField(((ScenarioImpl) scenario).getClass(), "stepResults", true);
field.setAccessible(true);
try {
ArrayList<Result> results = (ArrayList<Result>) field.get(scenario);
for (Result result : results) {
if (result.getError() != null)
LOGGER.error("Error Scenario: {}", scenario.getId(), result.getError());
}
} catch (Exception e) {
LOGGER.error("Error while logging error", e);
}
}
You can to this by writing your own custom implementation of Formatter & Reporter interface. The empty implementation of Formatter is the NullFormatter.java which you can extend. You will need to provide implementations for the Reporter interface.
The methods which would be of interest will be the result() of the Reporter interface and possibly the done() method of Formatter. The result() has the Result object which has the exceptions.
You can look at RerunFormatter.java for clarity.
Github Formatter source
public void result(Result result) {
//Code to create logs or store to a database etc...
result.getError();
result.getErrorMessage();
}
You will need to add this class(com.myimpl.CustomFormRep) to the plugin option.
plugin={"pretty", "html:report", "json:reports.json","rerun:target/rerun.txt",com.myimpl.CustomFormRep}
More details on custom formatters.
You can use the rerun plugin to get a list of failed scenarios to run again. Not sure about scheduling a run of failed tests, code to create a batch job or schedule one on your CI tool.
This is the workaround for cucumber-java version 4.8.0 using reflection.
import cucumber.api.Result;
import io.cucumber.core.api.Scenario;
import io.cucumber.core.logging.Logger;
import io.cucumber.core.logging.LoggerFactory;
import io.cucumber.java.After;
import org.apache.commons.lang3.ClassUtils;
import org.apache.commons.lang3.reflect.FieldUtils;
import java.io.IOException;
import java.lang.reflect.Field;
import java.net.URL;
import java.util.ArrayList;
#After
public void afterScenario(Scenario scenario) throws IOException {
if(!scenario.getStatus().isOk(true)){
logError(scenario);
}
}
private static void logError(Scenario scenario) {
try {
Class clasz = ClassUtils.getClass("cucumber.runtime.java.JavaHookDefinition$ScenarioAdaptor");
Field fieldScenario = FieldUtils.getField(clasz, "scenario", true);
fieldScenario.setAccessible(true);
Object objectScenario = fieldScenario.get(scenario);
Field fieldStepResults = objectScenario.getClass().getDeclaredField("stepResults");
fieldStepResults.setAccessible(true);
ArrayList<Result> results = (ArrayList<Result>) fieldStepResults.get(objectScenario);
for (Result result : results) {
if (result.getError() != null) {
LOGGER.error(String.format("Error Scenario: %s", scenario.getId()), result.getError());
}
}
} catch (Exception e) {
LOGGER.error("Error while logging error", e);
}
}
For cucumber-js https://www.npmjs.com/package/cucumber/v/6.0.3
import { After } from 'cucumber'
After(async function(scenario: any) {
const exception = scenario.result.exception
if (exception) {
this.logger.log({ level: 'error', message: '-----------StackTrace-----------' })
this.logger.log({ level: 'error', message: exception.stack })
this.logger.log({ level: 'error', message: '-----------End-StackTrace-----------' })
}
})
After a lot of experimentation I now removed the Before/After-Annotations and rely on Cucumber-Events instead. They contain the TestCase (which is what the Scenario-class wraps) and a Result where you can call getError(); to get the Throwable.
Here is a simple example to get it working
import io.cucumber.plugin.EventListener;
import io.cucumber.plugin.event.EventPublisher;
import io.cucumber.plugin.event.Result;
import io.cucumber.plugin.event.Status;
import io.cucumber.plugin.event.TestCase;
import io.cucumber.plugin.event.TestCaseFinished;
import io.cucumber.plugin.event.TestCaseStarted;
import org.openqa.selenium.WebDriver;
public class TestCaseListener implements EventListener {
#Override
public void setEventPublisher(final EventPublisher publisher) {
publisher.registerHandlerFor(TestCaseStarted.class, this::onTestCaseStarted);
publisher.registerHandlerFor(TestCaseFinished.class, this::onTestCaseFinished);
}
public void onTestCaseStarted(TestCaseStarted event) {
TestCase testCase = event.getTestCase();
System.out.println("Starting " + testCase.getName());
// Other stuff you did in your #Before-Method.
// ...
}
private void onTestCaseFinished(final TestCaseFinished event) {
TestCase testCase = event.getTestCase();
System.out.println("Finished " + testCase.getName());
Result result = event.getResult();
if (result.getStatus() == Status.FAILED) {
final Throwable error = result.getError();
error.printStackTrace();
}
// Other stuff you did in your #After-Method.
// ...
}
}
All that's left to do is to register this class as a Cucumber-Plugin.
I did this by modifying my #CucumberOptions-annotation:
#CucumberOptions(plugin = {"com.example.TestCaseListener"})
I find this much cleaner than all of this reflection-madness, however it requires a lot more code-changes.
Edit
I don't know why, but this caused a lot of tests to randomly fail in a multithreaded environment.
I tried to figure it out, but now also use the ugly reflections mentioned in this thread:
public class SeleniumUtils {
private static final Logger log = LoggerFactory.getLogger(SeleniumUtils.class);
private static final Field field = FieldUtils.getField(Scenario.class, "delegate", true);
private static Method getError;
public static Throwable getError(Scenario scenario) {
try {
final TestCaseState testCase = (TestCaseState) field.get(scenario);
if (getError == null) {
getError = MethodUtils.getMatchingMethod(testCase.getClass(), "getError");
getError.setAccessible(true);
}
return (Throwable) getError.invoke(testCase);
} catch (Exception e) {
log.warn("error receiving exception", e);
}
return null;
}
}
If you just want to massage the result being sent to the report then you can extend the CucumberJSONFormatter and override the result method like this:
public class CustomReporter extends CucumberJSONFormatter {
CustomReporter(Appendable out) {
super(out);
}
/**
* Truncate the error in the output to the testresult.json file.
* #param result the error result
*/
#Override
void result(Result result) {
String errorMessage = null;
if (result.error) {
errorMessage = "Error: " + truncateError(result.error);
}
Result myResult = new Result(result.status, result.duration, errorMessage);
// Log the truncated error to the JSON report
super.result(myResult);
}
}
Then set the plugin option to:
plugin = ["com.myimpl.CustomReporter:build/reports/json/testresult.json"]
Plz check following code.... class testError has been instantiated but still Class not found exception is generated... If that is true then why statement written in exception handler does not get printed??
class testError
{
void display()
{
System.out.println("This is testError Class");
}
}
class checkResult
{
public static void main(String[] args)
{
testError te = new testError();
te.display();// I hope the class has been created
Class cls = Class.forName("testError"); // will throw ClassNotFound exception
// Why??... Though the class has been
// instantiated
// if we try to put it in trycatch block it will work...Why??
try{ Class cls = Class.forName("testError");}
catch(ClassNotFoundException e)
{
System.out.println("Error found"); //"Error found" will not be printed
// as the class has been instantiated
}
}
}
I can't comment - as my reputation is too low, but your code runs and debugs fine - though I had to alter it a little bit to make it compile:
public static void main(String[] args) throws ClassNotFoundException {
testError te = new testError();
Class<?> cls = Class.forName("testError");
try {
cls = Class.forName("testError");
// If you got there then everything went fine
te.display();
}
catch (ClassNotFoundException e) {
System.out.println("Error found");
}
}
How do you run your code (from the command line, in an IDE)? What is the console output?
You have got to give more helpful information if you want people to investigate your issue.
Finally, Java convention specifies that classes name should begin with an uppercase character (CheckResult and TestError). Also you should avoid using classes in the default package, as those cannot be imported.
First of all Follow java naming convention
Make your main class public
Create some package(not good to create in default packag) like mypackage and put the classes inside them
and try to invoke the method this way
String name = packageName.className.class.getName();//get the name of the class
className o = (className)Class.forName(name)
.newInstance();
//will give an instance of type Object so cast it
o.display();// call the method
I have the following test:
#Test(expected=ArithmeticException.class)
public void divideByZero() {
int n = 2 / 1;
}
as seen here.
I would like to add a message that will print if this test fails.
For instance if I was doing an Assertion test, I would do the following to add a message:
#Test public void assertFail(){
Assert.fail("This is the error message I want printed.");
Assert.assertEquals(true, false);
}
The second example should print out "This is the error message I want printed.". How do I set the first example message text?
Maybe #Rule annotation should help. Into your unit test class add sth like this:
import org.junit.Rule;
import org.junit.rules.MethodRule;
import org.junit.runners.model.Statement;
import org.junit.runners.model.FrameworkMethod;
import org.junit.internal.runners.model.MultipleFailureException;
...
#Rule
public MethodRule failureHandler = new MethodRule()
{
#Override
public Statement apply(final Statement base, FrameworkMethod method, Object target)
{
return new Statement()
{
#Override
public void evaluate() throws Throwable
{
List<Throwable> listErrors = new ArrayList<Throwable>();
try
{
// Let's execute whatever test runner likes to do
base.evaluate();
}
catch (Throwable testException)
{
// Your test has failed. Store the test case exception
listErrors.add(testException);
// Now do whatever you need, like adding your message,
// capture a screenshot, etc.,
// but make sure no exception gets out of there -
// catch it and add to listErrors
}
if (listErrors.isEmpty())
{
return;
}
if (listErrors.size() == 1)
{
throw listErrors.get(0);
}
throw new MultipleFailureException(listErrors);
}
};
}
};
Instead of collecting all the exceptions in listErrors you may consider wrapping testException with your exception with additional message and just throwing it.
I recommend instead naming the test to make it obvious what the test is testing, so when some of your tests fail, they tell you what the problem is. Here's an example using the ExpectedException rule:
#RunWith(JUnit4.class)
public class CalculatorTest {
#Rule
public ExpectedException exception = ExpectedException.none();
#Test
public void divisionByZeroShouldThrowArithmeticException() {
Calculator calculator = new Calculator();
exception.expect(ArithmeticException.class);
calculator.divide(10, 0);
}
}
For details on ExpectedException, see this article and the ExpectedException JavaDoc
If you are willing to use catch-exception instead of JUnit's built-in exception handling mechanisms, then your problem can be easily solved:
catchException(myObj).doSomethingExceptional();
assertTrue("This is the error message I want printed.",
caughtException() instanceof ArithmeticException);
I don't think you can easily, but this guy seems to have partially worked his way around it.