We Keep Coding

Eclipse listing but not running unit tests, how to fix this?

When running a specific junit test in the Apache Jackrabbit project, Eclipse goes through the motions of running the test, but the test is never run, debugger breakpoints are never fired, and the results are that zero out of eight tests were run, and no errors were reported.
https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-spi2dav/src/test/java/org/apache/jackrabbit/spi2dav/ConnectionTest.java
The confusion is that the tests, after the run, don't indicate whether they succeeded, failed, or were explicitly skipped, and I cannot see anything in the source to indicate they should be skipped.
What is Eclipse trying to tell me?

This behaviour was triggered by the unit test overriding Junit's run method as follows:
https://github.com/apache/jackrabbit/blob/ed3124e5fe223dada33ce6ddf53bc666063c3f2f/jackrabbit-jcr-server/src/test/java/org/apache/jackrabbit/webdav/server/WebDAVTestBase.java#L234
public void run(TestResult testResult) {
if (Boolean.getBoolean("jackrabbit.test.integration")) {
super.run(testResult);
}
}
This was causing the test to not run at all.
In this case, Eclipse was picking up the test methods that should have been run and listed them, but with the tests not having been run, the Eclipse UI remains in a "vanilla" state.
This unit test obviously needs to be fixed to follow maven's best practise of splitting unit and integration tests into surefire and failsafe respectively, and not relying on a private mechanism.

JUnit tests influence each other

I'm working with a lot of legacy code. There was a JUnit-TestSuite to begin with. When running all tests with gradle, they failed. When running the test in IntelliJ, they worked. We configured gradle to use the test suite.
Now someone reported tests working locally without gradle, but not with gradle. It's time we fix this mess.
Is there a smart way to figure out which test leaves some configuration behind or which tests relies on the other tests?

The most likely cause of this "bleed" from one test into another is mutable static values. By default, all tests are run by the same JVM so a static variable which is "mutated" by one test will be "dirty" in another test.
Mutable statics are evil! I'm working on a codebase currently with mutable statics everywhere and it's a mess. If possible you should refactor to use dependency injection and store mutable state in instances and not statics.
The best workaround is to find the tests which "dirty" the static mutable variables and do
#After
public void cleanup() {
SomeStatic.reset();
}
If you can't find the "dirty" test which is causing the issue, you might be forced to do the following in the "failing" test. This is not preferred, and a little hacky
#Before
public void cleanBefore() {
SomeStatic.reset();
}
But this has a slight code "smell". Better to find the offending test which "dirties" the mutable static
The "nuclear" option is to run each test in its own jvm. This is a total hack and should be avoided at all costs. It will drastically increase the time it takes to run your tests
test {
forkEvery = 1
}
See Test.forkEvery

I recently diagnosed a similar issue in a Gradle Java project, where a test was working when run individually, but not when run as part of the Gradle build.
In order to track down the offending test that was breaking the subsequent test, I first configured Gradle to print out the tests that were run using beforeTest, so I could tell what all was running prior to my test:
test {
beforeTest { TestDescriptor descriptor ->
logger.lifecycle("$descriptor.className#$descriptor.name")
}
}
This printed the tests in the order in which they ran:
$./gradlew test
> Task :test
com.example.Test1#firstTest()
com.example.Test1#secondTest()
com.example.Test2#firstTest()
com.example.Test2#secondTest()
com.example.Test3#firstTest()
com.example.Test3#secondTest()
com.example.BrokenTest#brokenTest()
BrokenTest > brokenTest() FAILED
java.lang.AssertionError at BrokenTest.java:34
com.example.Test4#firstTest()
com.example.Test4#secondTest()
Now that I knew which test classes were running before the broken test, I knew that one (or more) of those tests were causing the test to break.
Next, I added test filtering in my Gradle build to only run those tests that ran before the broken test, plus the broken test itself:
test {
filter {
includeTestsMatching 'com.example.Test1'
includeTestsMatching 'com.example.Test2'
includeTestsMatching 'com.example.Test3'
includeTestsMatching 'com.example.BrokenTest'
}
}
After doing a sanity check Gradle build to confirm that the test was still broken, I commented out groups of tests and reran the build until I was able to narrow it down to the test that caused the build to break:
test {
filter {
// includeTestsMatching 'com.example.Test1'
includeTestsMatching 'com.example.Test2'
// includeTestsMatching 'com.example.Test3'
includeTestsMatching 'com.example.BrokenTest'
}
}
In this example, the build broke when Test2 was run before BrokenTest, but not when Test1 or Test3 were. Armed with this information, I could dive deeper into what specifically about Test2 was affecting the system in such a way as to break the other test when it was run after it.

Running TestCases (from Maven) when TestCases are in src?

I have a project that itself is a set of test cases for an application. The test cases are tested itself using a mockup of the application to ensure the tests itself is correct. This is important to ensure the test cases justify some sort of specifications that are hard to follow and easy to mess up.
Now I want the final tests (those who are tested before) being in src in Maven. As expected mvn test just executes the test cases of the test cases not the test cases themself.
So basically how do I execute test cases that reside inside the 'src' folder using a maven goal?

Try this :
<testSourceDirectory>src</testSourceDirectory>

Why is JUnit 4 test outcome in Eclipse dependent on previous outcome of mvn test?

I am running Eclipse 4.4.1 with JUnit 4.12 as a Maven dependency (scope = test). I'm running the tests both with Eclipse and with Maven.
I have a project with a main class, Corpus2Json, which I would like to test. I noticed that unit tests weren't failing when I expected them to, so I cut it down to the most basic test which should fail.
package path.to.package;
import static org.junit.Assert.*;
import org.junit.Test;
public class Corpus2JsonTest {
#Test
public void test() {
fail();
}
}
It didn't fail in Eclipse, but it did in Maven. I checked again in Eclipse and it failed. I removed the fail() line and the test still failed in Eclipse.
So I created a new test class with identical code to the above except named Corpus2JsonsTest, and it failed as expected in Eclipse. I removed fail(), and it no longer failed. Once I ran mvn test, Eclipse began only returning the result of the previous Maven test.
There appears to be behavior that makes it so when Eclipse runs the test, it only has the result of the latest Maven test. Unless the test file did not exist since the previous Maven test, in which case it acts normally.

JUnit test, Unrooted Tests: initializationError

So I edited the name of a JUnit test and now it wont work. Instead I get Unrooted Tests: initializationError.
This is a simple test. Infact it is a test for JUnit tests as I am just starting to use it.
#Test
public void testRun()
String s = null;
assertNull(s);
}
and all i did was change it to testRun2(). Also when I run the file not the individual test, it still runs the old testRun(), not testRun2().
My project has Maven not sure if that is a factor. And I have updated the project

So it turned out that I needed to rebuild using Maven to update the classes. Now it works fine and I can add/modify test cases.

In my case, i changed the method name and it didn't update it automatically, so the above solution of Project> Clean worked well for me.

Another way this error would occur is forgetting the Test annotation. Encountered when right click method name in Eclipse and Run As -> Junit Test.