We Keep Coding

Error when using the same string in different tests

I have the following scenario:
I perform several tests (#Test) and tests in Cucumber, in Selenium Webdriver, Java.
The tests are going well. However, I want to leave a string stored in one #Test (public void) in another #Test (public void). I cannot.
Could anyone help?
First test:
#Test
public void testDocuments() {
OneClass oneClass = new OneClass();
oneClass.one();
oneClass.two();
}
Second test:
#Test
public void testDocuments() {
OneClass oneClass = new OneClass();
oneClass.one();
oneClass.two();
}
Method one
public String one() {
if (this.cnpj == null) {
this.cnpj = add.cnpj(false);
} else {
}
return this.cnpj;
}
Both tests I want you to use the same generated string !!!!
I look forward and thanks in advance!

I'm not sure what your method one() does, but assuming you want to use the same value for two different tests, why not just do this:
OneClass oneClass = new OneClass();
String yourGeneratedString = oneClass.one();
// First test
#Test
public void testDocuments() {
yourFunction(yourGeneratedString);
}
// Second test
#Test
public void testDocuments2() {
yourOtherFunction(yourGeneratedString);
}

If I understand correctly, you need this.cnpj value to be available within the second test?
Each time you do new OneClass() , it creates a new instance of it.
So you can do one of the following:
Use singleton instance of OneClass
Make cnpj a static field within OneClass

If I understand it right, you want to share data from one test to second one. If you user testNG then you can do it this way.
import org.testng.ITestContext;
import org.testng.annotations.Test;
public class MyTest {
#Test
public void testOne(ITestContext context){
context.setAttribute("myKey", "myValue");
}
#Test
public void testTwo(ITestContext context){
String valueFromTestOne = (String) context.getAttribute("myKey");
System.out.println("My key = " + valueFromTestOne);
}
}

How to monitor the invocation of methods in abstract class using java agent and ASM?

What I want to do is to monitor the invocation of JUnit 4 test methods. The reason I must do this by myself is: I need to record the executed classes during the execution of each test method. So I need to insert some instructions to the test method so that I know when the test start/end and those recorded classes are executed by which test entity. So I need to filter the test methods on my own.
Actually, the reason why I am doing this is not relevant to the question, as I did not mention "JUnit", "test" in the title. The problem can still be a problem in other similar cases.
The case I have is like this:
public abstract class BaseTest {
#Test
public void t8() {
assert new C().m() == 1;
}
}
public class TestC extends BaseTest{
// empty
}
I have also modified Surefire's member argLine so that my agent will be attached (premain mode) when Surefire launch a new JVM process to execute tests.
In my agent class:
public static void premain(String args, Instrumentation inst){
isPreMain = true;
agentArgs = args;
log("args: " + args);
parseArgs(args);
inst.addTransformer(new TestTransformer(), true);
}
My transformer class:
public class TestTransformer implements ClassFileTransformer {
public byte[] transform(ClassLoader loader, String className,
Class<?> classBeingRedefined, ProtectionDomain protectionDomain,
byte[] classfileBuffer) throws IllegalClassFormatException {
log("TestTransformer: transform: " + className);
...
ClassReader cr = new ClassReader(classfileBuffer);
ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_MAXS);
RecordClassAdapter mca = new RecordClassAdapter(cw, className);
cr.accept(mca, 0);
return cw.toByteArray();
}
}
In my ClassVisitor adapter class:
class RecordClassAdapter extends ClassVisitor {
...
#Override
public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) {
MethodVisitor mv = cv.visitMethod(access, name, desc, signature, exceptions);
mv = new RecordMethodAdapter (...);
return mv;
}
}
In my MethodVisitor adapter class:
class RecordMethodAdapter extends MethodVisitor {
public void visitCode() {
mv.visitCode();
if (isTestMethod){
mv.visitLdcInsn(methodName);
mv.visitMethodInsn(INVOKESTATIC, MyClass, "entityStarted",
"(Ljava/lang/String;)V", false);
}
}
}
Sadly, I found that the abstract class will not get into the transform method, thus I can not instrument the t8 method. TestC should be executed as a test class, but I can never monitor the invocation of TestC.t8.

There are several opportunities to inject logging into the test via the JUnit API. There is no need for instrumentation.
For a very simple setup:
public class BaseTest {
#Test
public void t8() {
System.out.println("Running test "+getClass().getName()+".t8() [BaseTest.t8()]");
}
#Test
public void anotherMethod() {
System.out.println("Running test "
+getClass().getName()+".anotherMethod() [BaseTest.anotherMethod()]");
}
}
public class TestC extends BaseTest {
#Rule
public TestName name = new TestName();
#Before
public void logStart() throws Exception {
System.out.println("Starting test "+getClass().getName()+'.'+name.getMethodName());
}
#After
public void logEnd() throws Exception {
System.out.println("Finished test "+getClass().getName()+'.'+name.getMethodName());
}
}
which will print
Starting test class TestC.t8
Running test TestC.t8() [BaseTest.t8()]
Finished test class TestC.t8
Starting test class TestC.anotherMethod
Running test TestC.anotherMethod() [BaseTest.anotherMethod()]
Finished test class TestC.anotherMethod
You can also implement your own rule. E.g. ad-hoc:
public class TestB extends BaseTest {
#Rule
public TestRule notify = TestB::decorateTest;
static Statement decorateTest(Statement st, Description d) {
return new Statement() {
#Override public void evaluate() throws Throwable {
System.out.println("Starting test "+d.getClassName()+"."+d.getMethodName());
st.evaluate();
System.out.println("Finished test "+d.getClassName()+"."+d.getMethodName());
}
};
}
}
Or as a reusable rule that can be inserted via a single-liner into a test class
public class LoggingRule implements TestRule {
public static final LoggingRule INSTANCE = new LoggingRule();
private LoggingRule() {}
#Override
public Statement apply(Statement base, Description description) {
Logger log = Logger.getLogger(description.getClassName());
log.setLevel(Level.FINEST);
Logger.getLogger("").getHandlers()[0].setLevel(Level.FINEST);
String clName = description.getClassName(), mName = description.getMethodName();
return new Statement() {
#Override
public void evaluate() throws Throwable {
log.entering(clName, mName);
String result = "SUCCESS";
try {
base.evaluate();
}
catch(Throwable t) {
result = "FAIL";
log.throwing(clName, mName, t);
}
finally {
log.exiting(clName, mName, result);
}
}
};
}
}
used as simple as
public class TestB extends BaseTest {
#Rule
public LoggingRule log = LoggingRule.INSTANCE;
}
A different approach is implementing a custom test runner. This allows to apply a behavior to an entire test suite, as test suites are implemented via runners as well.
public class LoggingSuiteRunner extends Suite {
public LoggingSuiteRunner(Class<?> klass, RunnerBuilder builder)
throws InitializationError {
super(klass, builder);
}
#Override
public void run(RunNotifier notifier) {
notifier.addListener(LOG_LISTENER);
try {
super.run(notifier);
} finally {
notifier.removeListener(LOG_LISTENER);
}
}
static final RunListener LOG_LISTENER = new RunListener() {
public void testStarted(Description d) {
System.out.println("Starting test "+d.getClassName()+"."+d.getMethodName());
}
public void testFinished(Description d) {
System.out.println("Finished test "+d.getClassName()+"."+d.getMethodName());
}
public void testFailure(Failure f) {
Description d = f.getDescription();
System.out.println("Failed test "+d.getClassName()+"."+d.getMethodName()
+": "+f.getMessage());
};
};
}
This may get applied to an entire test suite, i.e. still inheriting test methods from BaseTest, you may use
#RunWith(LoggingSuiteRunner.class)
#SuiteClasses({ TestB.class, TestC.class })
public class TestA {}
public class TestB extends BaseTest {}
public class TestC extends BaseTest {}
which will print
Starting test TestB.t8
Running test TestB.t8() [BaseTest.t8()]
Finished test TestB.t8
Starting test TestB.anotherMethod
Running test TestB.anotherMethod() [BaseTest.anotherMethod()]
Finished test TestB.anotherMethod
Starting test TestC.t8
Running test TestC.t8() [BaseTest.t8()]
Finished test TestC.t8
Starting test TestC.anotherMethod
Running test TestC.anotherMethod() [BaseTest.anotherMethod()]
Finished test TestC.anotherMethod
These are only pointers, to suggest studying the API which allows even more. Another point to consider, is that depending on the method you’re using for launching the tests (you mentioned a maven plugin), there might be support for adding a global RunListener right there, without the need to alter the test classes.

Parameterized unit test suites

I am trying to set up some parameterized test suites, unfortunately without any luck so far.
I have two set of parameters, and I would like to run multiple test cases (they are in different classes) with all possible combinations. I tried to do it with JUnit4, but I am unable to set it up correctly. This would be my basic idea:
TestSuite1.class sets up one set of parameters, then it starts TestSuite2.class.
TestSuite2.class sets up the second set of parameters, then it starts the actual test(s) that will use both parameters.
Meanwhile it seems it is not possible to set up both Suite.class and Parameterized.class in the RunWith annotation at the same time (according to google, Parameterized extends Suite, I get usually "no runnable method found" message if I use.)
This is how my code looks like basically:
TestSuite1.class:
#RunWith(Parameterized.class)
#Parameterized.SuiteClasses({TestSuite2.class})
//I have tried with #RunWith(Suite.class) and
//#Suite.SuiteClasses({TestSuite2.class}) annotations also - all combinations
public class TestSuite1{
public TestSuite1(int number) {
Params.first = number;
}
#Parameters
public static Collection<Object[]> parameters(){
Object[][] data = new Object[][] { { 1 }, { 2 }, { 3 }, { 4 } };
return Arrays.asList(data);
}
}
TestSuite2.class looks the same as TestSuite1.class, except that I have added TestCase1.class to the suite instead of TestSuite2, and that it sets another variable in Params.
TestCase1.class:
public class TestCase1 {
#Test
public void test1(){
System.out.println("first: "+Params.first+" second: "+Params.second);
Assert.assertTrue(true);
}
}
I am open to all ideas - even with TestNG for example. I have tried it also (although today was the first time I saw it), but as I noticed the suites are a bit different than in JUnit. I would prefer not to set up XML files before testing, I would like to solve all set up programmatically.
Is what I am trying to achieve possible with any framework?
Update: With TestNG I have the following code:
Start.class:
public class Start {
public static void main(String[] args){
TestListenerAdapter tla = new TestListenerAdapter();
TestNG testng = new TestNG();
testng.setTestClasses(new Class[] { FirstTest.class, SecondTest.class });
testng.addListener(tla);
testng.run();
}
}
Params.class:
public class Params {
#DataProvider(name = "param")
public static Object[][] createData() {
Object[][] data = new Object[][] { { 1 }, { 2}, { 3}, { 4} };
return data;
}
}
FirstTest.class:
public class FirstTest {
#Test(dataProvider = "param", dataProviderClass = Params.class)
public static void printIt(int number){
System.out.println("FirstTest: "+number);
}
}
SecondTest.class is the same as FirstTest.class. If I run this, it runs FirstTest 4 times, then it runs SecondTest 4 times. I would like to run FirstTest one time, and SecondTest one time also with the first set of parameters. Then I would like to run FirstTest and SecondTest one time, with the second set of parameters, etc.
I have tried to set setPreserveOrder(true), and tried all setParallel options also. On this way however the results are in kind of random order.
(It would be some selenium test. I am aware that tests should not depend on each other, but still it would be my desired way for this)

Although Parameterized extends Suite, it behaves totally different - in disrespect of the Liskov substitution principle. This is because normally the constructor Suite(Class<?>, RunnerBuilder) processes the #SuiteClasses annotation. But Parameterized(Class<?>) replaces this behaviour with a processing of #Parameters.
If you want to combine the behaviour of Suite and Parameterized you have to look outside of JUnit 4. E.g. you could implement your own Runner like Adam Hawkes already mentioned in another post here.
I did the same by myself and cobbled a library together that provides you with a ParameterizedSuite Runner: https://github.com/PeterWippermann/parameterized-suite
A parameterized test suite looks like this:
#RunWith(ParameterizedSuite.class)
#SuiteClasses({OneTest.class, TwoTest.class})
public class MyParameterizedTestSuite {
#Parameters(name = "Parameters are {0} and {1}")
public static Object[] params() {
return new Object[][] {{'A',1}, {'B',2}, {'C',3}};
}

So basicly as far as I understand, what you want to do is to run a test with set of sets of parameters. That is possible with JUnit and that is why method annotated with #Parameters returns a Collection of arrays (in general a set of sets).
Look at this example:
import static org.junit.runners.Parameterized.Parameters;
#RunWith(Parameterized.class)
public class TestCase1 {
public TestCase1(int first, int second) {
Params.first = first;
Params.second = second;
}
#Parameters
public static Collection<Object[]> parameters(){
Object[][] data = new Object[][] { { 1, 11 }, { 2, 22 }, { 3, 33 }, { 4, 44 } };
return Arrays.asList(data);
}
#Test
public void test1(){
System.out.println("first: "+Params.first+" second: "+Params.second);
Assert.assertTrue(true);
}
}
Edit:
If you want to share parameters between multiple test you could use an abstraction in your test cases.
public class AbstractParametrizedTest {
public AbstractParametrizedTest(int first, int second) {
Params.first = first;
Params.second = second;
}
#Parameterized.Parameters
public static Collection<Object[]> parameters(){
Object[][] data = new Object[][] { { 1, 11 }, { 2, 22 }, { 3, 33 }, { 4, 44 } };
return Arrays.asList(data);
}
}
#RunWith(Parameterized.class)
public class TestCase1 extends AbstractParametrizedTest {
public TestCase1(int first, int second) {
super(first, second);
}
...
}
However the best way to do it i think, would be to use TestNGs data providers. Take a look at example at section 5.6.2 and usage of static data providers
http://testng.org/doc/documentation-main.html

To achieve the goal of executing all test cases with the same parameters in sequential order, you would need a different Runner as this behavior is held in that class. You're in luck, as this is available in the JUnit Toolbox Project with the ParallelParameterized class!

Here some other suggest that seems to be much more flexible: #RunWith(Enclosed.class)
In short:
Instead of #Suite.SuiteClasses(...), just use #RunWith(Enclosed.class) and extend your Test Classes
#RunWith(Enclosed.class)
public class FastTest {
public static class Test1FirstAppInit extends AppInitTest { }
public static class Test2Download extends DownloadTest{ }
public static class Test3OtherTest extends OtherTest { }
}
Now with Parameterized:
#RunWith(Enclosed.class)
public class FastTest {
private static Iterable<? extends Object> mAllLocale = Arrays.asList(Locale.ENGLISH, Locale.GERMAN);
private static Iterable<? extends Object> mSingleLocale = Arrays.asList(Locale.ENGLISH);
/*
Run test class for all Locale
*/
#RunWith(Parameterized.class)
public static class Test1FirstAppInit extends AppInitTest {
#Parameterized.Parameter
public Locale mLocale;
#Parameterized.Parameters
public static Iterable<? extends Object> data() {
return mAllLocale;
}
#Override
public Locale getLocale() {
return mLocale;
}
#Override
public void test001ResetAll {
assumeTrue(false); // skip test completedly
}
#Override
public void test002ClearAppData() {
// replace existing test
if (getLocale() != Locale.ENGLISH) {
/*
should run only on first Locale
skip test on following Parameter runs
*/
assumeTrue(false); // skip test
}
else {
super.test000ClearAppData();
}
}
}
/*
Run test class only for one Locale
*/
#RunWith(Parameterized.class)
public static class Test2Download extends DownloadTest{
#Parameterized.Parameter
public Locale mLocale;
#Parameterized.Parameters
public static Iterable<? extends Object> data(){
return mSingleLocale;
}
#Override
public Locale getLocale() {
return mLocale;
}
#Override
public void test900Delete() {
assumeTrue(false); // skip test
}
}
/*
Test not Parameterized
*/
public static class Test3OtherTest extends OtherTest { }
}
Your Test Classes for Parameterized tests look like this:
#RunWith(AndroidJUnit4.class)
#LargeTest
#FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class DownloadTest {
public Locale getLocale() {
// will be overwritten in #RunWith(Enclosed.class)
// but we are still able to run test class separatedly
return Locale.ENGLISH;
}
#Test
public void test900Delete() {
....
}
....
}
Matches exactly what I was searching for. I can create different Test scenarios (full test, fast test,...). Just create different #RunWith(Enclosed.class) classes and extend the tests that you want to include.
Only side point seems to be that Enclosed.class does not care about sort order (if important to you).
I solved it by replacing Enclosed:
public class SortedEnclosed extends Suite {
public SortedEnclosed(Class<?> klass, RunnerBuilder builder) throws Throwable {
super(builder, klass, filterAbstractClasses(klass.getClasses()));
}
protected static Class<?>[] filterAbstractClasses(final Class<?>[] classes) {
final List<Class<?>> filteredList= new ArrayList<Class<?>>(classes.length);
for (final Class<?> clazz : classes) {
if (!Modifier.isAbstract(clazz.getModifiers())) {
filteredList.add(clazz);
}
}
// this is new (there may be better way with own "#FixClassOrder"...):
Collections.sort(filteredList, new Comparator<Class<?>>() {
#Override
public int compare(Class<?> o1, Class<?> o2) {
return o1.getSimpleName().compareTo(o2.getSimpleName());
}
});
//
return filteredList.toArray(new Class<?>[filteredList.size()]);
}
}
And then use #RunWith(SortedEnclosed.class)

JMockit: How to override already mocked method with a new mock?

I think I might have found a bug in JMockit, but I would like some to confirm whether it's a bug or there's something I'm missing.
I have the following (very simple) class:
public class Dummy {
public void foo() {System.out.println("O");}
}
Now I have the following tests, where in each of them I try to mock the method 'foo' more than once (each test does it a little differently):
Test #1
#Test
public void test1() {
new MockUp<Dummy>() {
#Mock
public void foo(Invocation inv) {
System.out.println("A");
inv.proceed();
}
}
new MockUp<Dummy>() {
#Mock
public void foo(Invocation inv) {
System.out.println("B");
inv.proceed();
}
}
new Dummy().foo();
}
Test #2
#Test
public void test2() {
mock("A");
mock("B");
new Dummy().foo();
}
private void mock(final String s) {
new MockUp<Dummy>() {
#Mock
public void foo(Invocation inv) {
System.out.println(s);
inv.proceed();
}
}
}
The only difference between the tests is the extraction of the mock code to a different method. But the results are not the same...
Test #1 output:
B
A
B
O
This is odd, because I wouldn't expect A to appear at all. But anyway, here's test #2 output:
B
A
A
A
...ad infinitum
Test #2 will fail with a StackOverflowError.
Is this a bug or am I missing something?
Update (with the solution)
As #Rogério mentioned, this behavior is not acceptable.
Then how can the mock be overridden? like this:
private MockUp<Dummy> mock;
#Test
public void test3() {
mockCorrectly("A");
mockCorrectly("B");
new Dummy().foo();
}
private void mockCorrectly(final String s) {
if (mock != null) {
mock.tearDown();
}
mock = new MockUp<Dummy> {
#Mock
public void foo(Invocation inv) {
System.out.println(s);
inv.proceed();
}
}
}
And for the output:
B
O
Great :)

It's not clear what exactly happens here; apparently, at runtime some "chained mocking" is occurring.
The real problem is that both tests are doing something invalid with the MockUp API: they are mocking the same method in the same class twice in the same test. It is ok to have two different mock-ups for the same class in the same test, as long as they mock different methods/constructors.
The resulting behavior is undefined, as JMockit does not support multiple simultaneous mockings of the same method.

JUnit #Ignore all other tests (#IgnoreOther ?)

I'm testing extensively with JUnit and sometimes - while debugging my code - I want (temporary) only run a single #Test of my #RunWith(Arquillian.class) test class. Currently I'm adding a #Ignore to all other tests and wondering if something like #IgnoreOther does exist.
Are there better solutions to ignore all other tests?

The simplest way is to replace all #Test to //###$$$#Test. Then when your debugging is finished replace //###$$$#Test to #Test.
Moreover typically IDEs allow running one test only. For example in Eclipse you can do it from Outline view.

Just my two cents. You can try to use Junit Rules as #srkavin suggested.
Here is an example.
package org.foo.bar;
import org.junit.rules.MethodRule;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.Statement;
public class SingleTestRule implements MethodRule {
private String applyMethod;
public SingleTestRule(String applyMethod) {
this.applyMethod = applyMethod;
}
#Override
public Statement apply(final Statement statement, final FrameworkMethod method, final Object target) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
if (applyMethod.equals(method.getName())) {
statement.evaluate();
}
}
};
}
}
package org.foo.bar;
import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
public class IgnoreAllTest {
#Rule
public SingleTestRule test = new SingleTestRule("test1");
#Test
public void test1() throws Exception {
System.out.println("test1");
}
#Test
public void test2() throws Exception {
Assert.fail("test2");
}
#Test
public void test3() throws Exception {
Assert.fail("test3");
}
}

Test rules (JUnit 4.7+) will help. For example, you can write a rule that ignores all #Test methods except one with a specific name.

The answer from srkavin (and mijer) is correct, but the code is deprecated from JUnit 4.9. The interface and the method signature have changed. I want to provide this for others interested in this issue.
public class IgnoreOtherRule implements TestRule
{
private String applyMethod;
public IgnoreOtherRule(String applyMethod){
this.applyMethod = applyMethod;
}
#Override
public Statement apply(final Statement statement, final Description description)
{
return new Statement()
{
#Override
public void evaluate() throws Throwable {
if (applyMethod.equals(description.getMethodName())) {
statement.evaluate();
}
}
};
}
}