Our Hudson installation shows a very spartan version of the Selenium report:
Selenium Report Result
numTestPasses 2
numTestFailures 0
I've understood that it should be possible to view full reports via Hudson, and also track development of the tests. But how?
We configured Hudson to run Selenium through an Ant task. Then we added a post-build action to "Publish JUnit test result report." With this setup we can see each failed test name along with how long each test took and the ability to drill down into all the tests to see specifics.
You may be able to just add that post build action to your project using the Selenium plugin.
Not sure if this fits in your environment: I'm running selenium-rc via JUnit (and phpunit) test cases using ant in a Hudson freestyle Project.
Hudson collects and publishes JUnit and via xUnit Plugin various other test results in an optional post build step.
Maybe you just need to find out where the build process stores the actual xml report and instruct hudson to publish it?
I run Selenium tests in Hudson using this plugin. It runs my Selenium tests and publishes detailed test reports. Very easy to configure, maybe not perfect if you have a lot of test suites to execute.
We use the SeleniumHQ plugin. The results are saved in the workspace (this is done in the project config by using "Publish Selenium Report"). You can save the build reports by using "Archive the artifacts" in project config.
We have written the Selenium tests as normal JUnit tests, which gets executed with the rest of the tests, so the reports are embeded i the junit test report generated by hudson.
Related
Our application is spring boot based project written with kotlin. There is great plugin for JetBrains IntelliJ IDEA for Allure support. The plugin provide a possibility to generate test reports, upload them to Allure TestOps and link tests from source code to test launch (tree of all tests) in Allure TestOps. Linking is achieved through adding #AllureId annotation to tests in source code.
My goal is automate this process inside build pipelines. Allure has allurectl tool which allows to upload test reports to Allure TestOps. But as far as I concerned it doesn't have a feature to assign #AllureId annotations.
Is there any option to assign #AllureId to tests from source code using command line?
You are generally right - allurectl does not have this feature. It is supported in the Allure IDEA plugin only.
The reason is that automated test-case linking feature in the plugin is implemented via the IntelliJ engine, so we'd have to write our own engine for the allurectl to make it work.
I've find out that adding a label's node with name "as_id" to generated json report works as a way to set allureId. Tested with upload to Allure TestOps, it works.
Some details are here and in related PR:https://github.com/Tinkoff/Allure.XUnit/issues/25
We have created a Maven project and automated test scripts using Selenium, Java and TestNG framework.
The code is checked in tfs using TEE plugin and we are able to create maven build in tfs. Now we need to publish the results of the test scripts in TFS.
Please advise how could we publish the results in TFS and generate reports.
Generate a custom report from TestNG is pretty easy: just implements your own IReporter. See the related documentation: http://testng.org/doc/documentation-main.html#logging
Now, you have to find what is the best way for sending test reports to TFS.
It could be by generating a specific file or by using an API like https://www.visualstudio.com/en-us/docs/integrate/api/test/results#add-test-results-to-a-test-run
I wanted to get code coverage of regression tests,which tests java webserver endpoints.
Things I have
1. I am having jar file of source
2. Regression code written using TestNG framework.
So here, I will run this jar file to run webserver and then I will run my regression code which will test running webserver.Here I wanted to get code coverage report of Regression tests.Can anyone give any suggestion ?
You can use EMMA(A free code coverage tool). If you are using eclipse you can directly get it from market place. Also it supports running individual Java class file or jar file. Here is the link for more information.
http://emma.sourceforge.net/intro.html
If you're using Maven, you can have a look at qualinsight-maven-cobertura-mojo. There is a companion Github project that provides examples showing how to use it along with jetty (versions 7.6.x, 8.1.x, 9.2.x, 9.3.x). In the exmaples JUnit is being used, but it has no impact at all on the result, you can use TestNG if you want to.
The example shows how to use this Maven plugin and configure your project in order to:
Instrument your code for coverage with qualinsight-maven-mojo-core
run your Jetty server and deploy instrumented code
run your tests (you'll have to replace Junit tests by TestNG ones, and make sure they are run during the integration-test phase)
stop your Jetty server (this will dump coverage file to disk)
Generate coverage report
If needed coverage reports can then be imported in SonarQube using the Generic Coverage plugin (see documentation.)
Note that this plugin has some advantages and limitations compared to cobertura-maven-plugin (see its documentation), but given the description of your requirements, none of the limitations seems to be a blocker in your context.
I have arquillian set up here with some integration tests, but so far I'm running them with:
mvn clean test -Parq-jbossas-managed
I would like to run and see the errors/successes on the Testng eclipse plugin. How do I do that?
I have tried to add a testng running config, selection a class to test, but it didn't work.
Any ideas?
Running TestNG from its Eclipse plugin or from maven are two different things!
The TestNG Eclipse plugin comes with its own version of TestNG - afaik, this isn't possible (without starting eclipse processes from you build process)
If reporting matters to you, you should have a look at ReportNG
To start a test run with your Eclipse Plugin (so you can see it in Eclipse's dedicated view), you have to select your test-class (or your test-xml) and use Run As -> TestNG test.
Our pom.xml currently contains both the build settings, as well as execution of selenium using selenium-maven-plugin.
I would like to split it in to two pom files, one for the build and unit tests and the second one for executing selenium tests. (This way I could first build the project in Hudson, and after successful build execute Selenium tests using another project).
Is it possible to configure maven to only execute the selenium-maven-plugin?
The answer is yes. Put your functional (selenium) tests in a dedicated module with a dependency on the webapp.