I wanted to get code coverage of regression tests,which tests java webserver endpoints.
Things I have
1. I am having jar file of source
2. Regression code written using TestNG framework.
So here, I will run this jar file to run webserver and then I will run my regression code which will test running webserver.Here I wanted to get code coverage report of Regression tests.Can anyone give any suggestion ?
You can use EMMA(A free code coverage tool). If you are using eclipse you can directly get it from market place. Also it supports running individual Java class file or jar file. Here is the link for more information.
http://emma.sourceforge.net/intro.html
If you're using Maven, you can have a look at qualinsight-maven-cobertura-mojo. There is a companion Github project that provides examples showing how to use it along with jetty (versions 7.6.x, 8.1.x, 9.2.x, 9.3.x). In the exmaples JUnit is being used, but it has no impact at all on the result, you can use TestNG if you want to.
The example shows how to use this Maven plugin and configure your project in order to:
Instrument your code for coverage with qualinsight-maven-mojo-core
run your Jetty server and deploy instrumented code
run your tests (you'll have to replace Junit tests by TestNG ones, and make sure they are run during the integration-test phase)
stop your Jetty server (this will dump coverage file to disk)
Generate coverage report
If needed coverage reports can then be imported in SonarQube using the Generic Coverage plugin (see documentation.)
Note that this plugin has some advantages and limitations compared to cobertura-maven-plugin (see its documentation), but given the description of your requirements, none of the limitations seems to be a blocker in your context.
Related
Our application is spring boot based project written with kotlin. There is great plugin for JetBrains IntelliJ IDEA for Allure support. The plugin provide a possibility to generate test reports, upload them to Allure TestOps and link tests from source code to test launch (tree of all tests) in Allure TestOps. Linking is achieved through adding #AllureId annotation to tests in source code.
My goal is automate this process inside build pipelines. Allure has allurectl tool which allows to upload test reports to Allure TestOps. But as far as I concerned it doesn't have a feature to assign #AllureId annotations.
Is there any option to assign #AllureId to tests from source code using command line?
You are generally right - allurectl does not have this feature. It is supported in the Allure IDEA plugin only.
The reason is that automated test-case linking feature in the plugin is implemented via the IntelliJ engine, so we'd have to write our own engine for the allurectl to make it work.
I've find out that adding a label's node with name "as_id" to generated json report works as a way to set allureId. Tested with upload to Allure TestOps, it works.
Some details are here and in related PR:https://github.com/Tinkoff/Allure.XUnit/issues/25
What would be a good tool (similar to jest for javascript) to report unit test coverage for just the changes files in Java projects? We are planning to build this tool into each pull request on GitHub at workplace!
Currently we are generating jacoco test reports on a daily-basis, but would be needing something that can be run on each build to target just the changes made. Appreciate the thoughts!
Is there a way to verify the coverage of jacoco in eclipse without run mvn test command? I'm using power mock, so Eclemma does not work in my case.
Try a plugin e.g. EclEmma
https://www.eclemma.org/
Fast develop/test cycle: Launches from within the workbench like JUnit test runs can directly be analyzed for code coverage.
Rich coverage analysis: Coverage results are immediately summarized and highlighted in the Java source code editors.
Non-invasive: EclEmma does not require modifying your projects or performing any other setup.
I want to know how to link production code and test code, i.e., I'd like to answer this question: which product codes are the targets of this test code?
I'd like to do this automatically.
My project uses maven and I have used SonarQube and the source codes are written in Java.
If it's needed, I will try any other tools.
How can I link production codes and test codes?
Please let me know how to do it.
What you want to do is effectively getting the Coverage of your tests, an answer to the question "What lines/branches of my code are covered by my tests?".
Maven and SonarQube are perfectly suited for this, the only thing you need to add into the mix is Jacoco. A good explanation for the configuration of Jacoco/Junit is here. Jacoco is an agent that gets added to the execution of your tests and which monitors them, analyzing which lines/branches have been executed (covered) and which have not.
The important part is to configure the jacoco plugin and the surefire/failsafe plugin(s) (last one is for integration tests) to use jacoco. This will generate jacoco report files, which then can/will be read by SonarCube during the sonar:sonar goal (you might have to set the path to these files either in your maven pom.xml as a sonar property or directly in the SonarQube server properties, both work fine).
You can test it step-by-step, first getting jacoco to run, since it already creates nice html reports. Reading the reports into SonarQube is the easier part then.
Our Hudson installation shows a very spartan version of the Selenium report:
Selenium Report Result
numTestPasses 2
numTestFailures 0
I've understood that it should be possible to view full reports via Hudson, and also track development of the tests. But how?
We configured Hudson to run Selenium through an Ant task. Then we added a post-build action to "Publish JUnit test result report." With this setup we can see each failed test name along with how long each test took and the ability to drill down into all the tests to see specifics.
You may be able to just add that post build action to your project using the Selenium plugin.
Not sure if this fits in your environment: I'm running selenium-rc via JUnit (and phpunit) test cases using ant in a Hudson freestyle Project.
Hudson collects and publishes JUnit and via xUnit Plugin various other test results in an optional post build step.
Maybe you just need to find out where the build process stores the actual xml report and instruct hudson to publish it?
I run Selenium tests in Hudson using this plugin. It runs my Selenium tests and publishes detailed test reports. Very easy to configure, maybe not perfect if you have a lot of test suites to execute.
We use the SeleniumHQ plugin. The results are saved in the workspace (this is done in the project config by using "Publish Selenium Report"). You can save the build reports by using "Archive the artifacts" in project config.
We have written the Selenium tests as normal JUnit tests, which gets executed with the rest of the tests, so the reports are embeded i the junit test report generated by hudson.