If I have the below classes (A and B) which are each in a separate module and I create a test for A.doSomething(), Sonar will complain about 0% coverage on B.doSomething() although it is being testing indirectly in TestA.
Using Ecclema coverage plugin for eclipse, I can see B.doSomething() is considered as covered.
Is there a reason for why Sonar acts this way? Is there a way to change the behavior of Sonar's code test coverage to include indirectly tested classes?
class A {
boolean doSomething() {
return new B().doSomething();
}
}
class B {
boolean doSomething() {
return true;
}
}
You can easily achieve what you want. This is a matter of how you configure JaCoCo so it has little to do with SonarQube.
For example in SonarQube Eclipse project we need that since unit tests are in a separate module.
So we have configured JaCoCo to use the same dump file for all modules (by default JaCoCo Maven plugin will use one separate dump per module):
https://github.com/SonarSource/sonar-eclipse/blob/master/pom.xml#L373
And then we configured SonarQube JaCoCo plugin to use the same report:
https://github.com/SonarSource/sonar-eclipse/blob/master/pom.xml#L357
Related
How do we make mvn build or Gradle build to take test cases from the main package if we have written test cases in the main package?
In gradle you just redefine the main and test sourceSets with filters in your build.gradle file to exclude / include the test files in the specific phase of the build.
For example the file named LibraryTest.java will be compiled and executed only during the test phase.
sourceSets {
main {
java {
srcDirs = ['src/main/java']
exclude '**/*Test.java'
}
}
test {
java {
srcDirs = ["src/main/java"]
include '**/*Test.java'
}
}
}
you can write your test cases where ever you want, BUT!
It is not a recommended practice, so if you are using maven/gradle - they will give you dedicated folder/path for writing test cases.
The reason why it is not recommended - is maven/gradle provides lot of plugins which will help you to run your test cases, generate reports for those test cases, control the build if test cases fails.
All these plugins will look up at the default path, so if you decide to use a different path rather than default - you need to change the path for test cases in all your plugin.
so if you choose to use your own path for test resources, you are just adding overhead of additional configuration changes.
Jacoco, as all other profiling capabilities in java I know of, depends on java agents/instrumentation.
Java instrumentation can only impact bytecode when a class is loaded.
I wonder if it is somehow possible to detect unused classes with Jacoco? Obviously, there might be classes that are "dead" and are in turn, never loaded by the ClassLoader.
When coverage output (HTML report, etc.) is generated after the test run has executed, JaCoCo scans additional classes in the runtime classpath that haven't been loaded during test execution, so they can also be instrumented and included in the output.
I implemented this same mechanism in my own code coverage tool (JMockit Coverage), which also relies on java.lang.instrument. It's the only way to have all relevant classes instrumented for coverage.
Yes, though it might be easier to specify -verbose:class if you are not already using Jacoco.
I just ran a test: I created a file called Delete.java that is never called by any other method. The generated Jacoco report shows the Delete class with 0% coverage.
I have a Gradle project with the JaCoCo plugin applied to it. When I run my tests and create a jacocoTestReport I get this classes not matching error
[ant:jacocoReport] Classes in bundle 'e-services' do no match with execution data. For report generation the same class files must be used as at runtime.
[ant:jacocoReport] Execution data for class eservices/model/persistence/Event does not match.
The classes should match as I'm doing a clean -> build -> test locally. I suspect the mismatch comes from the fact that I'm using jackson.map.ObjectMapper to create an object from a JSON and somehow this causes the classId stored in jacoco's .exec file not match the compiled class id.
My test uses the Event class extensively and still I get 0% coverage due to class mismatch:
import eservices.model.persistence.Event;
event = mapper.readValue(json, Event.class);
event.setTenId(TenIds.getInternalId());
Is there a way to get coverage from this scenario?
This is a JaCoCo Known limitation as JaCoCo relies on the checksum of the runtime bytecodes matching the checksum of the bytecodes it uses for report generation. Typically it happens when you have two libraries instrumenting bytecodes, like PowerMock and JaCoCo, or JPA and JaCoCo.
This is referenced in the following JaCoCo issue Coverage is missing a class that was in fact tested #193 and it is labeled as 'wontfix' 'known limitation'
I have a legacy application that has a unit test module that's separate from the application modules. I'm converting the project to use Gradle and the structure looks like this:
/root
/module1
/module2
...
/moduleN
/test
where the test module executes tests for module1 through moduleN (and depends on them). I know this isn't a very good practice as it kinda defeats the purpose of unit tests but as all know, legacy code is always a headache to work with.
So before I start refactoring the code so that each module has its own unit tests (which means disassembling the test module in a sensible way, i.e., a lot of work) I wanted to find a temporary solution to get the correct code coverage, i.e., have JaCoCo instrument all the classes from module1, ..., moduleN instead of just module test.
Is there a way to tell JaCoCo to instrument classes from other modules?
To include coverage results from subprojects "module*" in the "test" project, you might want to try something like this in your build.gradle from the test project:
// [EDIT] - 'afterEvaluate' NOK, use 'gradle.projectsEvaluated' instead (see comments)
// afterEvaluate {
gradle.projectsEvaluated {
// include src from all dependent projects (compile dependency) in JaCoCo test report
jacocoTestReport {
// get all projects we have a (compile) dependency on
def projs = configurations.compile.getAllDependencies().withType(ProjectDependency).collect{it.getDependencyProject()}
projs.each {
additionalSourceDirs files(it.sourceSets.main.java.srcDirs)
additionalClassDirs files(it.sourceSets.main.output)
}
}
}
I have some JUnit tests that contained in a .jar that is intended to be used as a library. The library contains some tests that should be run whenever the library is used in another project.
However when I create a new project using the library and run JUnit on it in Eclipse then the tests in the dependency .jar don't run / don't get detected by the JUnit test runner. I get the message:
No tests found with test runner 'JUnit 4'.
Is there a way I can configure the dependency .jar so that the tests will run alongside any tests that might be contained in the main project?
Basically I want the dependency .jar to "export" the tests to whatever projects it is used in.
I'm using Eclipse Juno, JUnit 4.10, and Maven for the dependency management.
EDIT:
The point of this library is to be able to help test projects that use it - i.e. it runs some specialised tests. This is why I want to be able to import the library .jar and have it contribute the extra tests to the importing project.
You can try Maven Surefire.
In some cases it would be useful to have a set of tests that run with various dependency configurations. One way to accomplish this would be to have a single project that contains the unit tests and generates a test jar. Several test configuration projects could then consume the unit tests and run them with different dependency sets. The problem is that there is no easy way to run tests in a dependency jar. The Surefire plugin should have a configuration to allow me to run all or a set of unit tests contained in a dependency jar.
This can be done as follows (Junit 3):
Ensure test jar contains a class which has a static suite() method
import junit.framework.Test;
import junit.framework.TestSuite;
public class AllTests {
public static Test suite()
{
TestSuite suite = new TestSuite( "All Tests");
suite.addTestSuite(TestOne.class);
suite.addTestSuite(TestTwo.class);
return suite;
}
}
Then in the project using the test-jar dependency:
create a TestCase:
package org.melati.example.contacts;
import org.melati.poem.AllExportedTests;
import junit.framework.Test;
import junit.framework.TestCase;
public class PoemTest extends TestCase {
public static Test suite()
{
return AllExportedTests.suite();
}
}
Now the tests will be found.
I think that making a library of unit tests (#Test annotated methods) is a bad idea. However, making a library of reusable test components is a good one. We've done this in a few open source projects, and you can take a look how it works.
One Maven module exports test components (we call them "mocks"), from src/mock/java directory. Exported artifact has -mock classifier. See rexsl/pom.xml (pay attention to highlighted lines).
Mock artifacts are being deployed to Maven Central, together with usual artifacts: http://repo1.maven.org/maven2/com/rexsl/rexsl-core/0.3.8/ (pay attention to ...-mock.jar files)
Modules that need that mocks can include them as usual artifacts, for example rexsl-core/pom.xml (see highlighted lines):
Then, in your unit tests just use the classes from that mock libraries, like regular builders of mocks, for example: BulkHttpFeederTest
That's how you can make your test artifacts reusable, in an elegant way. Hope it helps.
#Mikera,
I find that this may help you. Just extend the Testcase Class to one of your java classes in project and you can run that particular class to run it as a JUnit Test.
I am not sure that this is desirable - On the one hand, if you use a jar, its behaviour might be influenced by the external context, e.g. other libraries in the classpath. From inside the jar, there is no simple way to analyse this context and to adjust the tests accordingly. On the other hand, if you write and compile a library, you should test it before packaging it as a jar. You might even want to not include your tests.
If it is really important to you to run the tests again, I would be interested in what could make them fail without changing the jar. In that case, however, you might want to extend the testrunner. As far as I know it uses reflection. You can quite easily load jars in a classloader and go through all their classes. By reflection you can identify the test classes and assemble testsuites. You could look into the testrunner for an example. Still, you would need to start this process from outside, e.g. from inside one of your test classes in the client project. Here, QATest's approach might be helpful: By providing an overriden version of testsuite or testrunner, you could automate this - if the client uses your overridden API.
Let me know if this rather costly approach seems to be applicable in your scenario and I can provide code examples.
Why should the user of the jar run the test cases inside the jar!!! When the jar is packaged and delivered, it means that the unit tests are run successfully.
Typically, the jar itself should be either treated as a separate project or as one of the modules. In both the cases, unit test cases are run before its delivered.