I'm running Jacoco and Sonar on multimodule Java8 project.
I have unit tests in each of the modules and to save resources I collect all 'integration tests' into one "integration-tests-runner" and run them all there (wrapping them with before and after tests).
When measuring coverage UT generates exec file per module target/jacoco-ut.exec, while the IT generates one exec file: /target/jacoco-it.exec.
When I run sonar I reuse those exec files, giving path to the jacoco-it.exec.
I get a very weird image:
How can it be that overall coverage is lower?
I found the problem and the solution.
From Sonar website I see this:
By default, when no coverage report is found, the JaCoCo plugin will
not set any value for coverage metric. This behaviour can be
overriden to force coverage to 0% in case of a lack of report by
setting the following property :
sonar.jacoco.reportMissing.force.zero=true
This means that UT analysis was skipped for modules without any tests.
Since I've set the sonar.jacoco.itReportPath from parent pom then all modules got analyzed for integration tests coverage, and overall coverage.
Bottom line: setting the property sonar.jacoco.reportMissing.force.zero=true from parent pom fixed the numbers.
Why is that weird? The unit and integration tests execute code and certain executed code chunks overlap. With other words, the code which is covered by the unit and integration tests is not disjoint, thus you cannot simply add them up.
Related
I have a spring-based java project and I have my tests placed in /src/test/java folder. I tried to run the Sonar cube locally via
docker run -d --name sonarqube -p 9000:9000 sonarqube
in order to analyze the percentage of the test coverage. The test coverage shows 0% even though I have many tests in my project. And the Number of Unit tests count shows as 8 though there are more than that. Here is the screenshot
And here is my general Settings
Does anyone has any idea how to configure Sonar cube to reflect the test coverage?
I faced the same situation when trying to use Sonar with Docker.
What worked for me was:
Make sure you have run your tests [You can use mvn test]
Make sure JaCoCo has generated the coverage report [In my case reports were generated in folder target\jacoco-report]
Set the following property pointing to your XML report
Sonar property to be set
Sonarqube and sonar scanner do not provide tooling to generate code coverage reports. This should be done with tools like jacoco for java or opencover for .NET.
You can then add the output of jacoco to the sonar scanner with the sonar.coverage.jacoco.xmlReportPaths property.
so you will basically need the following steps for your analysis:
Sonar scanner begin
Restore packages
Build application
Use code coverage tool to test and calculate the coverage for your app
Sonar scanner end
I created a simple scenario with sources, tests, and configuration. I am not able to my current setup reach 100% code coverage in Sonar even it should be. (link to repository with sources - https://bitbucket.org/sloukam/gradle-multiproject-coverage-sonar/src )
It's a little bit complicated - it's Gradle multi-project which sources and tests mixed in java and groovy.
Tests are in the same module as source class and sometimes test are in "general" test module (let's call it integration test module) and corresponding sources are located in their module (in this simple case in module source_module_one and source_module_two.
Some sources are written in Java and covered by the Groovy test and vice versa.
After spending a lot of time I'm not able to see in my local Sonar 100% code coverage. I would really appreciate help to reach the target of 100% code coverage. Is it possible with my project structure? With any other coverage plugin?
First, you should certainly reconsider having tests in the same directory as sources. Just for future sanity's sake, I'd segregate tests in a different directory. Additionally, if your tests are in with your sources, then they'll be counted as source files. So you'll need to have tests of your tests, and test of your test-tests, ... ad infinitum.
Beyond that, though, 100% coverage is a Grail Quest. Holding yourself to high standards is a good thing, but at a certain point you hit the law of diminishing returns. Instead, I'd set the requirement to ... 90%? 92%? 95%?
basically what we are solving here is typical unit tests / integration tests scenario. you have unit tests inside your modules but then you have another module / project that runs integration tests and you want to see combined coverage
this can be really easily solved in SonarQube 6.2+ because it introduced new property sonar.jacoco.reportPaths - which takes multiple report files and merges them.
in lower versions of SonarQube, you probably have to merge both unit and integration tests report files into one, using JacocoMerge. In our project, I added this task:
task jacocoMerge(type: JacocoMerge) {
subprojects { subproject ->
executionData fileTree(subproject.buildDir).include("/jacoco/*.exec")
}
}
which generates report files that merges all reports over all modules into projectRoot/build/jacoco/jacocoMerge.exec. Now you can use absolute path to this file as sonar.jacoco.reportPath for all your modules you want to analyze.
I have big project and many ignored unit tests. Is there any way that I can run all ignored tests using surefire maven plugin and see list of passed tests?
Total number of ignored tests are >1000, so, manually it is impossible.
Currently I have Sonar setup to show unit test coverage which works great. Is there a way to enforce a certain percentage of unit test coverage for a project? For example if the coverage drops below 50%, I want it to be a sonar violation and for my build to fail.
Yes, it is possible. For example, if you use jenkins, you can configure and step in which you can choose the minimum coverage(JaCoCo Plugin).
In the jenkins job, you can pass the unit test, and take the coverage report of jacoco, and then, a step with the sonar plugin, but if the coverage is less than for instance 50%, the jenkins will show your build as failure.
You need to install this plugin Build Breaker the only purpose of this plugin is to break the build when new alerts raise in the analysis.
Imagine a multi-modules Maven project, such as the following one:
parent
+- core
+- main
main is dependent on the core module.
I now write a class CoreClass in core, with 2 methods: method1() and method2().
In core tests, I write a test class that will only test CoreClass.method1().
If I run a coverage tool (in my case Cobertura, using mvn sonar:sonar), I will find that I get 50% of test coverage on CoreClass (if we imagine that both methods have the same length).
Until now, everything is ok.
Now, in main project, I write a test class that will test the CoreClass.method2(). So normally, I would expect to have 100% of line coverage on CoreClass when I run an analysis on the whole project.
However, I still get my 50%.
I understand that this is a comprehensive behavior. Indeed, Cobertura will instrument CoreClass for coverage analysis only during the tests execution on the core module, and not on the main.
That explains why I still have 50% of code coverage.
However, my question is to know if there is a way to get the real code coverage of CoreClass when I am running the tests on all of my modules.
Thanks!
ps: I know that in a perfect world, it is not the concern of the main module to test the core classes. But as you may know, we are not in a perfect world :o)
Technical information: Java 1.6, JUnit 4.8.1, Maven 2.0.9 (will be upgraded to 2.2.1 soon, but I don't think it does really matter), Sonar 2.8
Use jacoco and sonar and have a single jacoco.exec file result for all modules.
Sonar will use this file and report the correct coverage for each module.
I have use it for a multi module project successfully with Sonar
Here you can find a solution for jacoco/sonar and here only for jacoco.