I have a Maven-based Java webapp that has bunch of unit tests, integration tests, code coverage reports etc and some more technical details.
I would like generate project information which would contain all the above information aggregated in one place so that it could be seen by others.
What's are some of the tools available in order to achieve this?
Remote server upload can be done either via http/dav/scp etc.
You can maintain a roadmap via markdown/apt or other formats.
Current open issues (may be jira) via maven-changes-plugin ?
Technical details such as tools etc. can be documented by the above as well?
Unit tests results can be done via usual maven site generation (surefire reporting) Code coverage via cobertura the old way or via JaCoCo.
Related
I am working for a company that is using multiple Maven projects/modules to create what will eventually become one product. To help me explain, imagine a file structure similar to below:
- Parent Directory
- Project_1
- /src/
- /target/
- POM.xml
- Project_2
- /src/
- /target/
- POM.xml
Along the way we are using JUnit to unit test our code, and it is an important contractual requirement that we achieve above a certain percentage threshold of code coverage with our tests.
We are using JaCoCo to generate coverage reports in the form of a HTML website. JaCoCo itself is proving to be invaluable but one major problem we have is that this creates a single site under the /target/site/jacoco/ directory.
I have done some investigating myself and found that, unless I am mistaken, JaCoCo by default does not support the ability to converge multiple Maven projects into a single JaCoCo report.
So my question is, can anybody suggest an alternative solution - something that will allow us to converge multiple reports onto a single web server?
One option we have is to move all sites into individual folders on a web server and then have an index page linking them together, but it's "clumsy" at best. For example:
- Web Server
- index.html
- Project_1
- (Generated report files)
- Project_2
- (Generated report files)
Any better suggestions would be greatly appreciated.
JaCoCo does not provide a simple way to do this as of today. However, they do specify three alternatives that are described here: https://github.com/jacoco/jacoco/wiki/MavenMultiModule
Their most suitable approach involves creating a separate reporter module that contains dependencies on all the other modules (in the github article referred to as Strategy: Module with Dependencies).
The reporter module uses the jacoco:report-aggregate (http://www.eclemma.org/jacoco/trunk/doc/report-aggregate-mojo.html) maven goal to fetch all the individual reports and binds them together into one.
An example project:
https://prismoskills.appspot.com/lessons/Maven/Chapter_06_-_Jacoco_report_aggregation.jsp
There are many different approaches you can go with.
First of all you might want to consider something like Sonar, so that you'll compile all your modules and will run a Sonar that will inspect the coverage among other things. Sonar will take the results and upload to sonar server (with the database and everything) so that you'll be able to see in UI what went wrong
Another approach is just rolling your own Maven plugin (assuming you're using Maven). The reports generated by jacoco is also an XML report if I'm not mistaken. So it can be parsed pretty easily. So, one could write a Maven plugin that would identify all the reports like this, parse them and provide some unified view.
Yet another approach is to cause the whole build failure when the coverage doesn't reach some threshold. I know, it doesn't answer your question directly, but if you do it like this, you'll kind of guarantee the minimum level of coverage (that can be increased from time to time at the level of project).
I am trying to create the following setup :
A Selenium (Java) project that has a set of 10 automated test cases.
When this project is executed, it generates an HTML test execution report.
This project should be 'hosted' on an internal network.
Anyone who has access to the network should be able to 'invoke' this project, which in turn executes the test cases and passes the HTML report to the person who invoked it.
The project should be accessible ONLY for execution and the code should NOT be accessible.
My goal is that this implementation should be executable by any framework irrespective of the technology that the framework uses. I was thinking of creating the project as a WebService using Java (servlet).
My question is:
Can this implementation be accessed by any external automation framework ?
Are there any limitations to this implementation?
Is there a better way to implement this requirement?
Thanks in advance.
You can create a maven project and have your automated tests under maven test folder.Configure your tests to run through POM.xml(use maven surefire plugin).Configure a jenkins job to run the maven test.Anybody with access the jenkins can build/run this task!
Below link should give you a headstart
http://learn-automation.com/selenium-integration-with-jenkins/
As a matter of fact, it is something we did on one of our projects. As I cannot share specifics, I will give you overall architectural view of the project.
The core of all things was a service that could run JUnit tests on requests. It was a Soap web-service, but nothing stops you from making it REST. To implement this you need to implement your version of JUnit test runners (see for example: http://www.mscharhag.com/java/understanding-junits-runner-architecture or https://github.com/junit-team/junit4/wiki/Test-runners)
If you use JUnit as test framework for running your Selenuim tests this may be a great solution for you - JUnit will generate HTML reports for you if you configure it properly, it will hide actual test suite implementation from users and run test suite on demand. This solution is also great because it operates on JUnit level and does not care about what kind of tests it actually runs, so it can be also reused for any other kind of automated tests.
So to answer all your questions:
Can this implementation be accessed by any external automation
framework ? -> yes, it can be accessed by anybody who able send http
requests
Are there any limitations to this implementation? -> none that I am
aware of
Is there a better way to implement this requirement? -> well, I
didn't actually work with TestNG much so I don't know if it is
easier or more difficult to do it on Junit level. You can use
Jenkins or other CI tool as well to achieve same results - they can
run JUnit tests for you and almost always have API ready for this,
although those APIs may be not perfect.
So I'd say that if you need this only for one thing you can use CI tools for this purpose, if you don't have CI tools available then choice has been made for you. However, from our experience, having this kind of service was a great asset for a company and I really wonder why there's no such products available elsewhere yet.
We have a Java web app, and a number of developers working with it. Every developer is working with his/her own feature, in its own branch. When the feature is ready - we want to review it and visually test (after all unit and integration tests are passed, of course). We want to automate this process of deployment. Ideally, we would like to let our developers click just one button somewhere to make the application deployed to http://example.com/staging/branches/foo (where branches/foo is developer's path in SVN repository).
Then, the deployment is reviewed (by project sponsors mostly), merged into /trunk, and removed from the staging server.
I think that I'm not the first one who needs to implement such a scenario. What are the tools and technologies that may help me?
Typically, I would use a stage environment to test the "trunk" (ie all the individual branches for a release merged together). Several reasons for this:
Stakeholders and sponsors usually don't have time to test individual branches. They want to test the entire release. It also tend to get very confusing for people not inside the immediate team to keep track of different, changing URLs and understanding why feature X works on one URL and not the other. Always keep it simple for your sponsors.
It tends to become very messy and costly to maintain more than one instance of third-party dependencies (databases, service providers etc) for proper stage testing. Bear in mind that you want to maintain realistic test-data at all times.
Until you merge all individual branches together for a release, there will be collisions and integration bugs that will be missed. Assume that automated integration tests won't be perfect.
All that being said, there are lots of good tools for automatic build/deploy out there. Not knowing anything about your build setup and deployment environment, a standard setup could consist of a build-server, maven and tomcat. The build-server would execute the build and deploy the resulting appplication to the test-server. If you are using maven and tomcat, there is a plugin available for this task (http://mojo.codehaus.org/tomcat-maven-plugin/introduction.html). There are a number of good build-servers out there as well with good support for maven. Teamcity is popular, as is Hudson CI.
Basically you can use Hudson/Jenkins.
There are ways to manage have multiple deployments on one machine with some plugins, as stated on the following post on Jenkins Users, you'll just have to manage those multiple deployments to be the branches the developers are working on.
As #pap said, Hudson and other CI software build, test (if you have any tests in it) and deploy webapps, you'll just have to configure this procedure. Hope the link is helpful.
We've just started looking at using JBehave for acceptance tests and I was wondering how people that are using it are organising the writing of stories and the storage of story files. It's just development that are working on them at the moment so we have the story files stored in the resources folder alongside the Java code to implement them.
I guess my actual question is how and where are you storing your story files and how does this work with the product owner or QA writing stories?
#MrWiggles
as t0rx told you are lucky to have QA to write stories/scenarios. coming to your question:Behaviour-Driven Development encourages you to start defining the stories via scenarios that express the desired behavior in a textual format.
JBehave Stories you can run by configuring in Maven (pom.xml).
You can make a folder for storing your story files in your package structure, like below:
Your_Project
|
|
|--Source_Code
|
|--Stories
|
|--Testing
|
*pom.xml
By configuring your stories in maven, every time you build project it will give result with succeeded and failed stories/scenarios results.
QA will update the scenarios in the folder Stories, and the developer will implement the scenarios step by step by omitting existing steps (which are already developed and came in other scenarios).
QA simply run the scenario/story and he will find out the result in a textual (understandable) format.
Like below:
Behaviour-Driven Development in test levels.
Some of the JBehave features concentrate on easy organizing.
Annotation-based configuration and Steps class specifications
Dependency Injection support allowing both configuration and Steps instances composed via your favourite container (Guice, PicoContainer, Spring).
Extensible story reporting: outputs stories executed in different human-readable file-based formats (HTML, TXT, XML). Fully style-able view.
Auto-generation of pending steps so the build is not broken by a missing step, but has option to configure breaking build for pending steps.
Localisation of user stories, allowing them to be written in any language.
IDE integration: stories can be run as JUnit tests or other annotation-based unit test frameworks, providing easy integration with your favourite IDE.
Ant integration: allows stories to be run via Ant task
Maven integration: allows stories to be run via Maven plugin at given build phase
If you are lucky enough to have the product owner or QA writing stories then you probably want them in a specific area of your source code repository so you can control access independently from your main source (and also give you more flexibility with when CI builds are triggered if you're doing that).
You'll likely find a lot of back-and-forth to minimise the number of new steps the devs have to write (i.e. stop them using ten different ways to write the same step), so will also need to run with pending steps not failuring the scenario (which is the default out of the box).
An alternative approach is that QA/product owner send scenarios to the devs who then cleanse them before adding to source control, but this puts effort back on the devs.
is there such a thing in a standard manner?
including
Java Source Code - Test Code -
Ant or Maven
JUnit
Continuous Integration (possibly Cruise Control)
ClearCase Versioning Tool
Deploy to Application Server
in the end I 'd like to have an automatic Build and Integration Environment.
Sounds like a job for Hudson.
There are no end of possible solutions. Take a look at the continuous integration matrix, which details common solutions, and their associated features. Hopefully you should be able to make a decision based on that.
Take a look at Apache Continuum.
A tool set could be
IntelliJ
Ant
The ant files can be generated by intellij
Then you need to write some ant to run your tests!
Then you need to write some ant to package your application
Then you need to write some ant to deploy your application
including configuring queues, databases etc (dbdeploy might work for this)
TeamCity
Subversion - Its nice and easy. ClearCase is shocking don't go near.
If you're doing Java EE, then you'll probably have a web site, if so, you might want to use WebDriver, possibly in conjunction with WindowLicker
If you have a database or jms broker, make sure each developer HAS THEIR OWN! This is very important - makes sure everybody has a copy that they can do what they like with, and obviously the continuous integration (CI) environment must have its own copy too!
The pay back for such an environment can be huge. On my current project, we have a two-click-to-production automation coming straight out of TeamCity.
I would suggest below set:
Spring Tool Suite www.springsource.org/ - IDE with maven plugin for development and Accurev plugin for repo management.
Accurev www.accurev.com - For source code repository/version management.
Maven maven.apache.org- For build process.
Hudson hudson-ci.org/ - to Automate build and integration.
JIRA www.atlassian.com/JIRA- For bug/issue tracking.
Rally www.rallydev.com - For project management.