We Keep Coding

How to monitor the invocation of methods in abstract class using java agent and ASM?

What I want to do is to monitor the invocation of JUnit 4 test methods. The reason I must do this by myself is: I need to record the executed classes during the execution of each test method. So I need to insert some instructions to the test method so that I know when the test start/end and those recorded classes are executed by which test entity. So I need to filter the test methods on my own.
Actually, the reason why I am doing this is not relevant to the question, as I did not mention "JUnit", "test" in the title. The problem can still be a problem in other similar cases.
The case I have is like this:
public abstract class BaseTest {
#Test
public void t8() {
assert new C().m() == 1;
}
}
public class TestC extends BaseTest{
// empty
}
I have also modified Surefire's member argLine so that my agent will be attached (premain mode) when Surefire launch a new JVM process to execute tests.
In my agent class:
public static void premain(String args, Instrumentation inst){
isPreMain = true;
agentArgs = args;
log("args: " + args);
parseArgs(args);
inst.addTransformer(new TestTransformer(), true);
}
My transformer class:
public class TestTransformer implements ClassFileTransformer {
public byte[] transform(ClassLoader loader, String className,
Class<?> classBeingRedefined, ProtectionDomain protectionDomain,
byte[] classfileBuffer) throws IllegalClassFormatException {
log("TestTransformer: transform: " + className);
...
ClassReader cr = new ClassReader(classfileBuffer);
ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_MAXS);
RecordClassAdapter mca = new RecordClassAdapter(cw, className);
cr.accept(mca, 0);
return cw.toByteArray();
}
}
In my ClassVisitor adapter class:
class RecordClassAdapter extends ClassVisitor {
...
#Override
public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) {
MethodVisitor mv = cv.visitMethod(access, name, desc, signature, exceptions);
mv = new RecordMethodAdapter (...);
return mv;
}
}
In my MethodVisitor adapter class:
class RecordMethodAdapter extends MethodVisitor {
public void visitCode() {
mv.visitCode();
if (isTestMethod){
mv.visitLdcInsn(methodName);
mv.visitMethodInsn(INVOKESTATIC, MyClass, "entityStarted",
"(Ljava/lang/String;)V", false);
}
}
}
Sadly, I found that the abstract class will not get into the transform method, thus I can not instrument the t8 method. TestC should be executed as a test class, but I can never monitor the invocation of TestC.t8.

There are several opportunities to inject logging into the test via the JUnit API. There is no need for instrumentation.
For a very simple setup:
public class BaseTest {
#Test
public void t8() {
System.out.println("Running test "+getClass().getName()+".t8() [BaseTest.t8()]");
}
#Test
public void anotherMethod() {
System.out.println("Running test "
+getClass().getName()+".anotherMethod() [BaseTest.anotherMethod()]");
}
}
public class TestC extends BaseTest {
#Rule
public TestName name = new TestName();
#Before
public void logStart() throws Exception {
System.out.println("Starting test "+getClass().getName()+'.'+name.getMethodName());
}
#After
public void logEnd() throws Exception {
System.out.println("Finished test "+getClass().getName()+'.'+name.getMethodName());
}
}
which will print
Starting test class TestC.t8
Running test TestC.t8() [BaseTest.t8()]
Finished test class TestC.t8
Starting test class TestC.anotherMethod
Running test TestC.anotherMethod() [BaseTest.anotherMethod()]
Finished test class TestC.anotherMethod
You can also implement your own rule. E.g. ad-hoc:
public class TestB extends BaseTest {
#Rule
public TestRule notify = TestB::decorateTest;
static Statement decorateTest(Statement st, Description d) {
return new Statement() {
#Override public void evaluate() throws Throwable {
System.out.println("Starting test "+d.getClassName()+"."+d.getMethodName());
st.evaluate();
System.out.println("Finished test "+d.getClassName()+"."+d.getMethodName());
}
};
}
}
Or as a reusable rule that can be inserted via a single-liner into a test class
public class LoggingRule implements TestRule {
public static final LoggingRule INSTANCE = new LoggingRule();
private LoggingRule() {}
#Override
public Statement apply(Statement base, Description description) {
Logger log = Logger.getLogger(description.getClassName());
log.setLevel(Level.FINEST);
Logger.getLogger("").getHandlers()[0].setLevel(Level.FINEST);
String clName = description.getClassName(), mName = description.getMethodName();
return new Statement() {
#Override
public void evaluate() throws Throwable {
log.entering(clName, mName);
String result = "SUCCESS";
try {
base.evaluate();
}
catch(Throwable t) {
result = "FAIL";
log.throwing(clName, mName, t);
}
finally {
log.exiting(clName, mName, result);
}
}
};
}
}
used as simple as
public class TestB extends BaseTest {
#Rule
public LoggingRule log = LoggingRule.INSTANCE;
}
A different approach is implementing a custom test runner. This allows to apply a behavior to an entire test suite, as test suites are implemented via runners as well.
public class LoggingSuiteRunner extends Suite {
public LoggingSuiteRunner(Class<?> klass, RunnerBuilder builder)
throws InitializationError {
super(klass, builder);
}
#Override
public void run(RunNotifier notifier) {
notifier.addListener(LOG_LISTENER);
try {
super.run(notifier);
} finally {
notifier.removeListener(LOG_LISTENER);
}
}
static final RunListener LOG_LISTENER = new RunListener() {
public void testStarted(Description d) {
System.out.println("Starting test "+d.getClassName()+"."+d.getMethodName());
}
public void testFinished(Description d) {
System.out.println("Finished test "+d.getClassName()+"."+d.getMethodName());
}
public void testFailure(Failure f) {
Description d = f.getDescription();
System.out.println("Failed test "+d.getClassName()+"."+d.getMethodName()
+": "+f.getMessage());
};
};
}
This may get applied to an entire test suite, i.e. still inheriting test methods from BaseTest, you may use
#RunWith(LoggingSuiteRunner.class)
#SuiteClasses({ TestB.class, TestC.class })
public class TestA {}
public class TestB extends BaseTest {}
public class TestC extends BaseTest {}
which will print
Starting test TestB.t8
Running test TestB.t8() [BaseTest.t8()]
Finished test TestB.t8
Starting test TestB.anotherMethod
Running test TestB.anotherMethod() [BaseTest.anotherMethod()]
Finished test TestB.anotherMethod
Starting test TestC.t8
Running test TestC.t8() [BaseTest.t8()]
Finished test TestC.t8
Starting test TestC.anotherMethod
Running test TestC.anotherMethod() [BaseTest.anotherMethod()]
Finished test TestC.anotherMethod
These are only pointers, to suggest studying the API which allows even more. Another point to consider, is that depending on the method you’re using for launching the tests (you mentioned a maven plugin), there might be support for adding a global RunListener right there, without the need to alter the test classes.

How do I test a util detect OS class

I create a class that detect the OS:
public class OperatingSystemProvider {
private final String osName;
public OperatingSystemProvider(){
this.osName = getOSName();
}
public boolean isWindowsOS(){
return osName.contains("windows");
}
public boolean isLinuxOS(){
return osName.contains("linux");
}
private String getOSName(){
return System.getProperty("os.name").toLowerCase();
}
}
And I created a class to test this:
public class OperatingSystemTests {
#Test
public void IsWindowsSystemInWidowsSystemTest(){
Assert.assertTrue(new OperatingSystemProvider().isWindowsOS());
}
#Test
public void IsLinuxSystemInWidowsSystemTest(){
Assert.assertFalse(new OperatingSystemProvider().isLinuxOS());
}
#Test
public void IsWindowsSystemInLinuxSystemTest(){
Assert.assertFalse(new OperatingSystemProvider().isWindowsOS());
}
#Test
public void IsLinuxSystemInLinuxSystemTest(){
Assert.assertTrue(new OperatingSystemProvider().isLinuxOS());
}
}
The test 2 tests passed in Windows and 2 tests passed in Linux.
I don't want to mock the the getOSName method, because this can lead to wrong assumptions (I think). So how can I test this class to passed in all tests in independent of the system?

The tests need to be executed on the corresponding OS. That's the best way to assure the tests will behavior exactly what should be. So to run tests according OS, the JUnit 5 has a feature called EnabledOnOS
#Test
#EnabledOnOs(WINDOWS)
public void IsLinuxSystemInWidowsSystemTest() {
Assertions.assertFalse(new OperatingSystemProvider().isLinuxOS());
}

JUnitParams - executing separate methods before test

#RunWith(JUnitParamsRunner.class)
public class MySimpleTest {
private MyRec rec;
private Matrix matrix;
#Before
public void createRecognizerBeforeEveryExecution() {
rec = new MyRec();
matrix = MatrixUtils.createMatrixWithValues();
}
public static Iterable<Object[]> data() {
return Arrays.asList(
new Object[]{"expectedvalue1", "input1"},
new Object[]{"expectedvalue2", "input2"}
);
}
#Test
#Parameters(method = "data")
public void test1(String output, String input) {
rec.fun1(matrix);
assertEquals(output, rec.someFunction(input));
}
public static Iterable<Object[]> data2() {
return Arrays.asList(
new Object[]{"expectedothervalue1", "input1"},
new Object[]{"expectedothervalue2", "input2"}
);
}
#Test
#Parameters(method = "data2")
public void test2(String output, String input) {
rec.fun1(matrix);
rec.fun2(matrix);
assertEquals(output, rec.someFunction(input));
}
}
I'm trying to find out what is the proper way to make this test. I'd like to use parametrized test, because it's really convenient way.
So as you can see, in every test function I call some function (fun1 and fun2). But I need to call it only once per every test (e.g. before each parametrized test execution).
Is there any way to tell JUnitParams that it should execute other function before executing all of parametrized tests?
I can't use #Before annotation, because as you can see in test1 I'm not using fun2. It think it should be executed by separate function.

Solution 1:
As fun[1|2] does not depend on internal test state, try to place their invocations inside data and data2 methods accordingly.
public static Iterable<Object[]> data() {
rec.fun1(matrix);
return Arrays.asList(
new Object[]{"expectedvalue1", "input1"},
new Object[]{"expectedvalue2", "input2"}
);
}
public static Iterable<Object[]> data2() {
rec.fun1(matrix);
rec.fun2(matrix);
return Arrays.asList(
new Object[]{"expectedvalue1", "input1"},
new Object[]{"expectedvalue2", "input2"}
);
}
Solution 2:
Spliting test cases is not a best practice. Your test are harder to maintain. Flow is far more complicated. There is also a risk your tests start depends on each other. Duplication in tests sometimes is simply better.
PS:
If you are using Strings as test method parameters it's better to pass them exactly like in 25th line of this file:
https://github.com/Pragmatists/JUnitParams/blob/master/src/test/java/junitparams/usage/SamplesOfUsageTest.java instead of special methods.
#Test
#Parameters({"AAA,1", "BBB,2"})
public void paramsInAnnotation(String p1, Integer p2) { }

I decided to use TestNG to resolve this problem (code just to show my train of thought):
import org.testng.Assert;
import org.testng.annotations.*;
import java.lang.reflect.Method;
public class TempTest {
private Integer number;
#BeforeMethod
public void init(Method m) {
number = 5;
switch(m.getName()) {
case "test2":
fun(10);
fun2(5);
break;
case "test1":
fun(10);
break;
}
}
public void fun(int value) {
number += value;
}
public void fun2(int value) {
number -= value;
}
#Test
public void test1() {
Assert.assertEquals(new Integer(15), number);
}
#Test
public void test2() {
Assert.assertEquals(new Integer(10), number);
}
#Test
public void test3() {
Assert.assertEquals(new Integer(5), number);
}
}

junit4 creating test suite with specific test methods

In junit4 I want to execute specific test methods from different classes i.e want create a test suite with specific test methods from different classes.
Lets say I have 2 classes:
public class Test_Login {
#Test
public void test_Login_001(){
System.out.println("test_Login_001");
}
#Test
public void test_Login_002(){
System.out.println("test_Login_002");
}
#Test
public void test_Login_003(){
System.out.println("test_Login_003");
}
}
public class Logout {
#Test
public void test_Logout_001(){
System.out.println("test_Logout_001");
}
#Test
public void test_Logout_002(){
System.out.println("test_Logout_002");
}
#Test
public void test_Logout_003(){
System.out.println("test_Logout_003");
}
}
From the above classes I want to execute test methods test_Login_001 , test_Login_003 , test_Logout_002 only.
How this can be achieved in junit4 ?

Since JUnit 4.8 introduced Categories there exists a clean solution, create a TestSuite:
#RunWith(Categories.class)
#IncludeCategory(MustHaveTests.class)
#SuiteClasses( { Test_Login.class, Test_Logout.class })
public class MustHaveTestsTestSuite {
public interface MustHaveTests { /* category marker */ }
}
And add the #Category(MustHaveTests.class) above every test you would like to run with the TestSuite, e.g.:
#Category(MustHaveTests.class)
#Test
public void test_Login_001(){
System.out.println("test_Login_001");
}
When running the TestSuite only the MustHaveTests-"tagged" tests will be executed. More Details on #Category: https://github.com/junit-team/junit4/wiki/categories

You need to create an org.junit.runner.Request and pass it to the JunitCore runner, or actually to any Runner.
JUnitCore junitRunner = new JUnitCore();
Request request = Request.method(Logout.class, "test_Logout_002");
Result result = junitRunner.run(request);
I actually created an Annotation and can search for methods with those annotations and dynamically create Request and run them
public class TestsSuite {
public static void main(String[] args) throws Exception {
Class annotation = MyTestAnnotation.class;
JUnitCore junitRunner = new JUnitCore();
Class testClass = Test_Login.class;
Method[] methods = testClass.getMethods();
for (Method method : methods) {
if (method.isAnnotationPresent(annotation)) {
if (method.isAnnotationPresent(org.junit.Test.class)) {
Request request = Request.method(testClass, method.getName());
Result result = junitRunner.run(request);
System.out.println(result.wasSuccessful());
}
}
}
}
}

This might not be the slickest implementation, but I solved a similar problem by created a new #SuiteMethods annotation as follows:
SuiteMethods.java
#Retention(RUNTIME)
#Target(TYPE)
public #interface SuiteMethods {
String[] value() default {""};
}
FilteredSuite.java
public class FilteredSuite extends Categories {
private static String[] TEST_METHODS_TO_RUN = {""}; // default behavior is to run all methods
private static Class<?> extractMethodNamesFromAnnotation(Class<?> clazz) {
SuiteMethods methodsAnnotation = clazz.getAnnotation(SuiteMethods.class);
if (methodsAnnotation != null) {
// if our MethodsAnnotation was specified, use it's value as our methods filter
TEST_METHODS_TO_RUN = methodsAnnotation.value();
}
return clazz;
}
public static Filter getCustomFilter() {
Filter f = new Filter() {
#Override
public boolean shouldRun(Description desc) {
String methodName = desc.getMethodName();
for (String subString : TEST_METHODS_TO_RUN) {
if (methodName == null || methodName.contains(subString)) {
return true;
}
}
return false;
}
#Override
public String describe() {
return null;
}
};
return f;
}
public FilteredSuite(Class<?> arg0, RunnerBuilder arg1) throws InitializationError {
super(extractMethodNamesFromAnnotation(arg0), arg1);
}
#Override
public void filter(Filter arg0) throws NoTestsRemainException {
// At test suite startup, JUnit framework calls this method to install CategoryFilter.
// Throw away the given filter and install our own method name filter
super.filter(getCustomFilter());
}
}
A Usage Example
#RunWith(FilteredSuite.class)
#SuiteClasses({
GroupRestTest.class,
ScenarioRestTest.class
})
#SuiteMethods({
"testReadOnlyFlag",
"testSheetWriteData",
"testAddScenarioMeta"
})
public class SubsetTestSuite {
}

Parameterized test case classes in JUnit 3.x

I have a JUnit 3.x TestCase which I would like to be able to parameterize. I'd like to parametrize the entire TestCase (including the fixture). However, the TestSuite.addTestSuite() method does not allow be to pass a TestCase object, just a class:
TestSuite suite = new TestSuite("suite");
suite.addTestSuite(MyTestCase.class);
I would like to be able to pass a parameter (a string) to the MyTestCase instance which is created when the test runs. As it is now, I have to have a separate class for each parameter value.
I tried passing it an anynomous subclass:
MyTestCase testCase = new MyTestCase() {
String getOption() {
return "some value";
}
}
suite.addTestSuite(testCase.getClass());
However, this fails with the assertion:
... MyTestSuite$1 has no public constructor TestCase(String name) or TestCase()`
Any ideas? Am I attacking the problem the wrong way?

If this is Java 5 or higher, you might want to consider switching to JUnit 4, which has support for parameterized test cases built in.

Rather than create a parameterized test case for the multiple/different backends you want to test against, I would look into making my test cases abstract. Each new implementation of your API would need to supply an implementing TestCase class.
If you currently have a test method that looks something like
public void testSomething() {
API myAPI = new BlahAPI();
assertNotNull(myAPI.something());
}
just add an abstract method to the TestCase that returns the specific API object to use.
public abstract class AbstractTestCase extends TestCase {
public abstract API getAPIToTest();
public void testSomething() {
API myAPI = getAPIToTest();
assertNotNull(myAPI.something());
}
public void testSomethingElse() {
API myAPI = getAPIToTest();
assertNotNull(myAPI.somethingElse());
}
}
Then the TestCase for the new implementation you want to test only has to implement your AbstractTestCase and supply the concrete implementation of the API class:
public class ImplementationXTestCase extends AbstractTestCase{
public API getAPIToTest() {
return new ImplementationX();
}
}
Then all of the test methods that test the API in the abstract class are run automatically.

Ok, here is a quick mock-up of how JUnit 4 runs parameterized tests, but done in JUnit 3.8.2.
Basically I'm subclassing and badly hijacking the TestSuite class to populate the list of tests according to the cross-product of testMethods and parameters.
Unfortunately I've had to copy a couple of helper methods from TestSuite itself, and a few details are not perfect, such as the names of the tests in the IDE being the same across parameter sets (JUnit 4.x appends [0], [1], ...).
Nevertheless, this seems to run fine in the text and AWT TestRunners that ship with JUnit as well as in Eclipse.
Here is the ParameterizedTestSuite, and further down a (silly) example of a parameterized test using it.
(final note : I've written this with Java 5 in mind, it should be trivial to adapt to 1.4 if needed)
ParameterizedTestSuite.java:
package junit.parameterized;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Collection;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
public class ParameterizedTestSuite extends TestSuite {
public ParameterizedTestSuite(
final Class<? extends TestCase> testCaseClass,
final Collection<Object[]> parameters) {
setName(testCaseClass.getName());
final Constructor<?>[] constructors = testCaseClass.getConstructors();
if (constructors.length != 1) {
addTest(warning(testCaseClass.getName()
+ " must have a single public constructor."));
return;
}
final Collection<String> names = getTestMethods(testCaseClass);
final Constructor<?> constructor = constructors[0];
final Collection<TestCase> testCaseInstances = new ArrayList<TestCase>();
try {
for (final Object[] objects : parameters) {
for (final String name : names) {
TestCase testCase = (TestCase) constructor.newInstance(objects);
testCase.setName(name);
testCaseInstances.add(testCase);
}
}
} catch (IllegalArgumentException e) {
addConstructionException(e);
return;
} catch (InstantiationException e) {
addConstructionException(e);
return;
} catch (IllegalAccessException e) {
addConstructionException(e);
return;
} catch (InvocationTargetException e) {
addConstructionException(e);
return;
}
for (final TestCase testCase : testCaseInstances) {
addTest(testCase);
}
}
private Collection<String> getTestMethods(
final Class<? extends TestCase> testCaseClass) {
Class<?> superClass= testCaseClass;
final Collection<String> names= new ArrayList<String>();
while (Test.class.isAssignableFrom(superClass)) {
Method[] methods= superClass.getDeclaredMethods();
for (int i= 0; i < methods.length; i++) {
addTestMethod(methods[i], names, testCaseClass);
}
superClass = superClass.getSuperclass();
}
return names;
}
private void addTestMethod(Method m, Collection<String> names, Class<?> theClass) {
String name= m.getName();
if (names.contains(name))
return;
if (! isPublicTestMethod(m)) {
if (isTestMethod(m))
addTest(warning("Test method isn't public: "+m.getName()));
return;
}
names.add(name);
}
private boolean isPublicTestMethod(Method m) {
return isTestMethod(m) && Modifier.isPublic(m.getModifiers());
}
private boolean isTestMethod(Method m) {
String name= m.getName();
Class<?>[] parameters= m.getParameterTypes();
Class<?> returnType= m.getReturnType();
return parameters.length == 0 && name.startsWith("test") && returnType.equals(Void.TYPE);
}
private void addConstructionException(Exception e) {
addTest(warning("Instantiation of a testCase failed "
+ e.getClass().getName() + " " + e.getMessage()));
}
}
ParameterizedTest.java:
package junit.parameterized;
import java.util.Arrays;
import java.util.Collection;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.parameterized.ParameterizedTestSuite;
public class ParameterizedTest extends TestCase {
private final int value;
private int evilState;
public static Collection<Object[]> parameters() {
return Arrays.asList(
new Object[] { 1 },
new Object[] { 2 },
new Object[] { -2 }
);
}
public ParameterizedTest(final int value) {
this.value = value;
}
public void testMathPow() {
final int square = value * value;
final int powSquare = (int) Math.pow(value, 2) + evilState;
assertEquals(square, powSquare);
evilState++;
}
public void testIntDiv() {
final int div = value / value;
assertEquals(1, div);
}
public static Test suite() {
return new ParameterizedTestSuite(ParameterizedTest.class, parameters());
}
}
Note: the evilState variable is just here to show that all test instances are different as they should be, and that there is no shared state between them.

a few details are not perfect, such as the names of the tests in the IDE being the same across parameter sets (JUnit 4.x appends [0], [1], ...).
To solve this you just need to overwrite getName() and change the constructor in your test case class:
private String displayName;
public ParameterizedTest(final int value) {
this.value = value;
this.displayName = Integer.toString(value);
}
#Override
public String getName() {
return super.getName() + "[" + displayName + "]";
}

For Android projects, we wrote a library called Burst for test parameterization. For example
public class ParameterizedTest extends TestCase {
enum Drink { COKE, PEPSI, RC_COLA }
private final Drink drink;
// Nullary constructor required by Android test framework
public ConstructorTest() {
this(null);
}
public ConstructorTest(Drink drink) {
this.drink = drink;
}
public void testSomething() {
assertNotNull(drink);
}
}
Not really an answer to your question since you're not using Android, but a lot of projects which still use JUnit 3 do so because Android's test framework requires it, so I hope some other readers will find this helpful.