We've just started looking at using JBehave for acceptance tests and I was wondering how people that are using it are organising the writing of stories and the storage of story files. It's just development that are working on them at the moment so we have the story files stored in the resources folder alongside the Java code to implement them.
I guess my actual question is how and where are you storing your story files and how does this work with the product owner or QA writing stories?
#MrWiggles
as t0rx told you are lucky to have QA to write stories/scenarios. coming to your question:Behaviour-Driven Development encourages you to start defining the stories via scenarios that express the desired behavior in a textual format.
JBehave Stories you can run by configuring in Maven (pom.xml).
You can make a folder for storing your story files in your package structure, like below:
Your_Project
|
|
|--Source_Code
|
|--Stories
|
|--Testing
|
*pom.xml
By configuring your stories in maven, every time you build project it will give result with succeeded and failed stories/scenarios results.
QA will update the scenarios in the folder Stories, and the developer will implement the scenarios step by step by omitting existing steps (which are already developed and came in other scenarios).
QA simply run the scenario/story and he will find out the result in a textual (understandable) format.
Like below:
Behaviour-Driven Development in test levels.
Some of the JBehave features concentrate on easy organizing.
Annotation-based configuration and Steps class specifications
Dependency Injection support allowing both configuration and Steps instances composed via your favourite container (Guice, PicoContainer, Spring).
Extensible story reporting: outputs stories executed in different human-readable file-based formats (HTML, TXT, XML). Fully style-able view.
Auto-generation of pending steps so the build is not broken by a missing step, but has option to configure breaking build for pending steps.
Localisation of user stories, allowing them to be written in any language.
IDE integration: stories can be run as JUnit tests or other annotation-based unit test frameworks, providing easy integration with your favourite IDE.
Ant integration: allows stories to be run via Ant task
Maven integration: allows stories to be run via Maven plugin at given build phase
If you are lucky enough to have the product owner or QA writing stories then you probably want them in a specific area of your source code repository so you can control access independently from your main source (and also give you more flexibility with when CI builds are triggered if you're doing that).
You'll likely find a lot of back-and-forth to minimise the number of new steps the devs have to write (i.e. stop them using ten different ways to write the same step), so will also need to run with pending steps not failuring the scenario (which is the default out of the box).
An alternative approach is that QA/product owner send scenarios to the devs who then cleanse them before adding to source control, but this puts effort back on the devs.
Related
Working on Java projects, it is widely accepted that you should use a standard project layout. That is the standard template you find on many open source projects, i.e.
src/main/java
src/main/sass
src/main/test
src/main/resources
src/anotherModule/kotlin
Now, it is assumed that all tests fall under src/main/test. I have seen this everywhere.
Now suppose you want to separate unit from integration tests so that they will not run together in CI. Is there any widely accepted convention about project layouts for integration and/or automated E2E tests?
In general, is it correct to state that I should plan to store tests so that all those under the same folder will run in the same test execution? That means if I have multiple suites, each should have its own root folder. And again, is there any known and accepted convention that integrates with popular build tools and makes testing experience seamless?
I couldn't quickly identify any in most popular open source frameworks.
I am working for a company that is using multiple Maven projects/modules to create what will eventually become one product. To help me explain, imagine a file structure similar to below:
- Parent Directory
- Project_1
- /src/
- /target/
- POM.xml
- Project_2
- /src/
- /target/
- POM.xml
Along the way we are using JUnit to unit test our code, and it is an important contractual requirement that we achieve above a certain percentage threshold of code coverage with our tests.
We are using JaCoCo to generate coverage reports in the form of a HTML website. JaCoCo itself is proving to be invaluable but one major problem we have is that this creates a single site under the /target/site/jacoco/ directory.
I have done some investigating myself and found that, unless I am mistaken, JaCoCo by default does not support the ability to converge multiple Maven projects into a single JaCoCo report.
So my question is, can anybody suggest an alternative solution - something that will allow us to converge multiple reports onto a single web server?
One option we have is to move all sites into individual folders on a web server and then have an index page linking them together, but it's "clumsy" at best. For example:
- Web Server
- index.html
- Project_1
- (Generated report files)
- Project_2
- (Generated report files)
Any better suggestions would be greatly appreciated.
JaCoCo does not provide a simple way to do this as of today. However, they do specify three alternatives that are described here: https://github.com/jacoco/jacoco/wiki/MavenMultiModule
Their most suitable approach involves creating a separate reporter module that contains dependencies on all the other modules (in the github article referred to as Strategy: Module with Dependencies).
The reporter module uses the jacoco:report-aggregate (http://www.eclemma.org/jacoco/trunk/doc/report-aggregate-mojo.html) maven goal to fetch all the individual reports and binds them together into one.
An example project:
https://prismoskills.appspot.com/lessons/Maven/Chapter_06_-_Jacoco_report_aggregation.jsp
There are many different approaches you can go with.
First of all you might want to consider something like Sonar, so that you'll compile all your modules and will run a Sonar that will inspect the coverage among other things. Sonar will take the results and upload to sonar server (with the database and everything) so that you'll be able to see in UI what went wrong
Another approach is just rolling your own Maven plugin (assuming you're using Maven). The reports generated by jacoco is also an XML report if I'm not mistaken. So it can be parsed pretty easily. So, one could write a Maven plugin that would identify all the reports like this, parse them and provide some unified view.
Yet another approach is to cause the whole build failure when the coverage doesn't reach some threshold. I know, it doesn't answer your question directly, but if you do it like this, you'll kind of guarantee the minimum level of coverage (that can be increased from time to time at the level of project).
I have a Maven-based Java webapp that has bunch of unit tests, integration tests, code coverage reports etc and some more technical details.
I would like generate project information which would contain all the above information aggregated in one place so that it could be seen by others.
What's are some of the tools available in order to achieve this?
Remote server upload can be done either via http/dav/scp etc.
You can maintain a roadmap via markdown/apt or other formats.
Current open issues (may be jira) via maven-changes-plugin ?
Technical details such as tools etc. can be documented by the above as well?
Unit tests results can be done via usual maven site generation (surefire reporting) Code coverage via cobertura the old way or via JaCoCo.
I think this is a long shot, but...
We have such project structure:
common-library
- denmark
- application
- france
- application
- application-xxxxxx
- application
- integration-tests
Each application has different configuration, translations, different package names and so on, but in all they are almost the same. They have the same features, same user interface etc.
Only one of our applications is now tested with integration tests with Robotium. Is there a way to "share" same integration tests for other applications?
It would be perfect to have "common tests" and custom/specific tests for each application. Is it at all possible?
We're using maven and Jenkins for our needs.
Any other approaches or suggestions are welcome.
well, assuming you need to maintain one set of integration tests only, I would go in direction of having:
separate maven module holding integration tests only
and in it I'd introduce multiple maven profiles, where each one would have specified maven dependency on one of the modules to be tested only
build can later switch between the profiles to activate the particular build only
As some notes on integration testing options say (http://docs.codehaus.org/display/MAVENUSER/Maven+and+Integration+Testing) and I believe it would affect this approach as well:
The disadvantage of doing it this way is that it tends to separate the integration tests from the code they're attempting to test. As a result, you may find that no one "owns" the integration tests; typically you'll have some one person whose job it is to analyze the integration tests and find bugs. QA is hard, but it's even harder when it's unclear who "owns" test failures.
Another problem could be if you run your builds including integration tests (via Jenkins) on code change automatically. Module dependencies would not launch your integration tests automatically. Rather you might need to define one Jenkins job per profile and define the correct jobs sequence manually.
E.g.: If Jenkins built denmark app => build integration, with profile denmark, ...
We have a Java web app, and a number of developers working with it. Every developer is working with his/her own feature, in its own branch. When the feature is ready - we want to review it and visually test (after all unit and integration tests are passed, of course). We want to automate this process of deployment. Ideally, we would like to let our developers click just one button somewhere to make the application deployed to http://example.com/staging/branches/foo (where branches/foo is developer's path in SVN repository).
Then, the deployment is reviewed (by project sponsors mostly), merged into /trunk, and removed from the staging server.
I think that I'm not the first one who needs to implement such a scenario. What are the tools and technologies that may help me?
Typically, I would use a stage environment to test the "trunk" (ie all the individual branches for a release merged together). Several reasons for this:
Stakeholders and sponsors usually don't have time to test individual branches. They want to test the entire release. It also tend to get very confusing for people not inside the immediate team to keep track of different, changing URLs and understanding why feature X works on one URL and not the other. Always keep it simple for your sponsors.
It tends to become very messy and costly to maintain more than one instance of third-party dependencies (databases, service providers etc) for proper stage testing. Bear in mind that you want to maintain realistic test-data at all times.
Until you merge all individual branches together for a release, there will be collisions and integration bugs that will be missed. Assume that automated integration tests won't be perfect.
All that being said, there are lots of good tools for automatic build/deploy out there. Not knowing anything about your build setup and deployment environment, a standard setup could consist of a build-server, maven and tomcat. The build-server would execute the build and deploy the resulting appplication to the test-server. If you are using maven and tomcat, there is a plugin available for this task (http://mojo.codehaus.org/tomcat-maven-plugin/introduction.html). There are a number of good build-servers out there as well with good support for maven. Teamcity is popular, as is Hudson CI.
Basically you can use Hudson/Jenkins.
There are ways to manage have multiple deployments on one machine with some plugins, as stated on the following post on Jenkins Users, you'll just have to manage those multiple deployments to be the branches the developers are working on.
As #pap said, Hudson and other CI software build, test (if you have any tests in it) and deploy webapps, you'll just have to configure this procedure. Hope the link is helpful.