Our application is spring boot based project written with kotlin. There is great plugin for JetBrains IntelliJ IDEA for Allure support. The plugin provide a possibility to generate test reports, upload them to Allure TestOps and link tests from source code to test launch (tree of all tests) in Allure TestOps. Linking is achieved through adding #AllureId annotation to tests in source code.
My goal is automate this process inside build pipelines. Allure has allurectl tool which allows to upload test reports to Allure TestOps. But as far as I concerned it doesn't have a feature to assign #AllureId annotations.
Is there any option to assign #AllureId to tests from source code using command line?
You are generally right - allurectl does not have this feature. It is supported in the Allure IDEA plugin only.
The reason is that automated test-case linking feature in the plugin is implemented via the IntelliJ engine, so we'd have to write our own engine for the allurectl to make it work.
I've find out that adding a label's node with name "as_id" to generated json report works as a way to set allureId. Tested with upload to Allure TestOps, it works.
Some details are here and in related PR:https://github.com/Tinkoff/Allure.XUnit/issues/25
Related
I have inherited a framework that uses Cucumber to use a series of browser-based tests with Selenium, in Java. The framework is currently built using Gradle and this works well.
I would very much like to integrate this whole thing with Browserstack, but the only Gradle plugin that I can find (here) is for Espresso, and so not applicable to my issue. I have also found another repository (here) which does exactly what I want, but does so through Maven.
Note I am unfamiliar with both Gradle and Maven and hence am inclined to stick with the solution that is currently running and wary of unnecessary change. Bearing this in mind, I have the following questions:
Is there a published way to integrate Cucumber (Java) with Browserstack using Gradle?
If not, is converting from Gradle to Maven really as trivial as this article makes me believe? Is there anything else I must be aware of?
Regarding your queries:
1- I am not aware of any published documentation to integrate Cucumber (Java) with Browserstack using Gradle
2- In the link that you have shared( https://dzone.com/articles/how-to-convert-maven-to-gradle-and-vice-versa), it says that you can convert maven to gradle in one step.
Run the command:
gradle init
In the directory containing the POM. This will convert the Maven build to a Gradle build, generating a settings.gradle file and one or more build.gradle files.
3-You have also shared the link: https://github.com/browserstack/cucumber-java-browserstack which uses maven.
You can follow the steps and easily convert maven to gradle
I have updated my pom.xml as per link.
But only xml files are getting created under folder 'allure-results'.
No index.html file is created for the executed test cases.
I was also searching for testng listeners for allure reporting. But unable to find the same.
Can anyone help on this.
You need to use one of the report generation tools to generate the report from xml results. In your case allure-maven-plugin is already configured in pom.xml. So simply run mvn site to generate the report. For more information see the docs.
We have created a Maven project and automated test scripts using Selenium, Java and TestNG framework.
The code is checked in tfs using TEE plugin and we are able to create maven build in tfs. Now we need to publish the results of the test scripts in TFS.
Please advise how could we publish the results in TFS and generate reports.
Generate a custom report from TestNG is pretty easy: just implements your own IReporter. See the related documentation: http://testng.org/doc/documentation-main.html#logging
Now, you have to find what is the best way for sending test reports to TFS.
It could be by generating a specific file or by using an API like https://www.visualstudio.com/en-us/docs/integrate/api/test/results#add-test-results-to-a-test-run
I wanted to get code coverage of regression tests,which tests java webserver endpoints.
Things I have
1. I am having jar file of source
2. Regression code written using TestNG framework.
So here, I will run this jar file to run webserver and then I will run my regression code which will test running webserver.Here I wanted to get code coverage report of Regression tests.Can anyone give any suggestion ?
You can use EMMA(A free code coverage tool). If you are using eclipse you can directly get it from market place. Also it supports running individual Java class file or jar file. Here is the link for more information.
http://emma.sourceforge.net/intro.html
If you're using Maven, you can have a look at qualinsight-maven-cobertura-mojo. There is a companion Github project that provides examples showing how to use it along with jetty (versions 7.6.x, 8.1.x, 9.2.x, 9.3.x). In the exmaples JUnit is being used, but it has no impact at all on the result, you can use TestNG if you want to.
The example shows how to use this Maven plugin and configure your project in order to:
Instrument your code for coverage with qualinsight-maven-mojo-core
run your Jetty server and deploy instrumented code
run your tests (you'll have to replace Junit tests by TestNG ones, and make sure they are run during the integration-test phase)
stop your Jetty server (this will dump coverage file to disk)
Generate coverage report
If needed coverage reports can then be imported in SonarQube using the Generic Coverage plugin (see documentation.)
Note that this plugin has some advantages and limitations compared to cobertura-maven-plugin (see its documentation), but given the description of your requirements, none of the limitations seems to be a blocker in your context.
Our Hudson installation shows a very spartan version of the Selenium report:
Selenium Report Result
numTestPasses 2
numTestFailures 0
I've understood that it should be possible to view full reports via Hudson, and also track development of the tests. But how?
We configured Hudson to run Selenium through an Ant task. Then we added a post-build action to "Publish JUnit test result report." With this setup we can see each failed test name along with how long each test took and the ability to drill down into all the tests to see specifics.
You may be able to just add that post build action to your project using the Selenium plugin.
Not sure if this fits in your environment: I'm running selenium-rc via JUnit (and phpunit) test cases using ant in a Hudson freestyle Project.
Hudson collects and publishes JUnit and via xUnit Plugin various other test results in an optional post build step.
Maybe you just need to find out where the build process stores the actual xml report and instruct hudson to publish it?
I run Selenium tests in Hudson using this plugin. It runs my Selenium tests and publishes detailed test reports. Very easy to configure, maybe not perfect if you have a lot of test suites to execute.
We use the SeleniumHQ plugin. The results are saved in the workspace (this is done in the project config by using "Publish Selenium Report"). You can save the build reports by using "Archive the artifacts" in project config.
We have written the Selenium tests as normal JUnit tests, which gets executed with the rest of the tests, so the reports are embeded i the junit test report generated by hudson.