I think this is a long shot, but...
We have such project structure:
common-library
- denmark
- application
- france
- application
- application-xxxxxx
- application
- integration-tests
Each application has different configuration, translations, different package names and so on, but in all they are almost the same. They have the same features, same user interface etc.
Only one of our applications is now tested with integration tests with Robotium. Is there a way to "share" same integration tests for other applications?
It would be perfect to have "common tests" and custom/specific tests for each application. Is it at all possible?
We're using maven and Jenkins for our needs.
Any other approaches or suggestions are welcome.
well, assuming you need to maintain one set of integration tests only, I would go in direction of having:
separate maven module holding integration tests only
and in it I'd introduce multiple maven profiles, where each one would have specified maven dependency on one of the modules to be tested only
build can later switch between the profiles to activate the particular build only
As some notes on integration testing options say (http://docs.codehaus.org/display/MAVENUSER/Maven+and+Integration+Testing) and I believe it would affect this approach as well:
The disadvantage of doing it this way is that it tends to separate the integration tests from the code they're attempting to test. As a result, you may find that no one "owns" the integration tests; typically you'll have some one person whose job it is to analyze the integration tests and find bugs. QA is hard, but it's even harder when it's unclear who "owns" test failures.
Another problem could be if you run your builds including integration tests (via Jenkins) on code change automatically. Module dependencies would not launch your integration tests automatically. Rather you might need to define one Jenkins job per profile and define the correct jobs sequence manually.
E.g.: If Jenkins built denmark app => build integration, with profile denmark, ...
Related
I built a backend server (ready-to-serve) that can load jar files as plugins and use the methods in it to serve different functionalities.
I want to write tests for it but I'm not sure what kind of tests I should write.
You should have a look at the different kinds of tests. Unit tests, integration tests, end-to-end tests.
In case you are writing the code for the imported jars yourself, you could write unit tests for the helper functions and services inside. They should be small and pure (self contained).
I guess, that you have a generalised interface exposed with each jar, that your main application reuses to communicate with the jar.
You could write a general integration test, that imports one of the jars and calls a general method, to show, that the execution succeeded. Something like a health check. Then you could write more tests for other functions and the expected results, though they will probably become more focused on each separate jar.
When it comes to testing plugins having only Unit tests is not enough. You'll need integration tests too, otherwise not possible to test combinations of plugins and/or dependencies between them.
Backing to your github project: from what i see, you basically need to test only plugin API and maybe some shared resources: configuration file, datasources and so on.
Often some testing framework for automated testing - like Selenium - is used to continiuosly verify the integrity of a deployed application. These tests often cover real user scenarios and may also utilize a range of deployed applications in combination.
We would like to achieve some what the same for a "backend only" application - that is, an application (or more really) without frontend. We are currently building a series of batchjobs where one job produces input to the next.
We have a great unit-test suite that tests the individual jobs however we would really like to test the series of jobs when deployed to some environment.
Do you have any suggestions for such testing framework? The framework must be able to leverage other Java SDKs such as AWS SDK (e.g. to instruct startup of batchjob, inject data to queues etc.). Whether the framework with tests needs to be deployed as an application as well or run directly from CI is secondary.
If you already have backedn tests that can be run on the production all you need is to schedule running on those tests. Jenkins is fine for that (https://wiki.jenkins.io/display/JENKINS/Schedule+Build+Plugin)
You could have emails (or other alerts) for failed jobs. Jenkins will also care for test reports- exactly as it does for unit tests.
Technologies for scheduling test runs
You could schedule running your tests using any other technology- for example Amazon AWS instances, AWS Elastic Beanstalk Worker Environments (https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html) etc.
I find Jenkins most reasonable, because you have out-of-the box support for test reports, notifications, etc.
For any other technologies you would have to write reportting, notifications on your own.
Technologies for writing tests
I could write tests in any technology that is capable of making HTTP REST calls. For performance tests Jmeter or Gatling are good choices.
For acceptance test you could use RestEasy, TestRestTemplate from Spring, Apache HTTP client, etc.
As test running framework you could use Junit4,Juni5, TestNG or Spock (if you are fine with Groovy language). Test structure could be similar to those of ordinary tests. Well named independend methods that test one thing well, meaningfull assertions, etc.
For writing assertions my personal preference is AssertJ, but JavaHamcrest would also do.
Those tests can (and should) be written in src/test directory, in separate repository (or in same repository or different module).
For that test module you may write test-releated services in src/main directory, so src/test directory would contain only test scenarios. Test services may everyting you need- manage files, inspect database, etc.
You may consider writting test scenarios in BDD-style and tools like JBehave or Cucumber. Personally I see value in BDD tests only if business is interested in test scenarios. If those tests are to be used only by technical people then I find easier to maintain such tests in non-BDD technologies (Junit, AssertJ).
I am trying to create the following setup :
A Selenium (Java) project that has a set of 10 automated test cases.
When this project is executed, it generates an HTML test execution report.
This project should be 'hosted' on an internal network.
Anyone who has access to the network should be able to 'invoke' this project, which in turn executes the test cases and passes the HTML report to the person who invoked it.
The project should be accessible ONLY for execution and the code should NOT be accessible.
My goal is that this implementation should be executable by any framework irrespective of the technology that the framework uses. I was thinking of creating the project as a WebService using Java (servlet).
My question is:
Can this implementation be accessed by any external automation framework ?
Are there any limitations to this implementation?
Is there a better way to implement this requirement?
Thanks in advance.
You can create a maven project and have your automated tests under maven test folder.Configure your tests to run through POM.xml(use maven surefire plugin).Configure a jenkins job to run the maven test.Anybody with access the jenkins can build/run this task!
Below link should give you a headstart
http://learn-automation.com/selenium-integration-with-jenkins/
As a matter of fact, it is something we did on one of our projects. As I cannot share specifics, I will give you overall architectural view of the project.
The core of all things was a service that could run JUnit tests on requests. It was a Soap web-service, but nothing stops you from making it REST. To implement this you need to implement your version of JUnit test runners (see for example: http://www.mscharhag.com/java/understanding-junits-runner-architecture or https://github.com/junit-team/junit4/wiki/Test-runners)
If you use JUnit as test framework for running your Selenuim tests this may be a great solution for you - JUnit will generate HTML reports for you if you configure it properly, it will hide actual test suite implementation from users and run test suite on demand. This solution is also great because it operates on JUnit level and does not care about what kind of tests it actually runs, so it can be also reused for any other kind of automated tests.
So to answer all your questions:
Can this implementation be accessed by any external automation
framework ? -> yes, it can be accessed by anybody who able send http
requests
Are there any limitations to this implementation? -> none that I am
aware of
Is there a better way to implement this requirement? -> well, I
didn't actually work with TestNG much so I don't know if it is
easier or more difficult to do it on Junit level. You can use
Jenkins or other CI tool as well to achieve same results - they can
run JUnit tests for you and almost always have API ready for this,
although those APIs may be not perfect.
So I'd say that if you need this only for one thing you can use CI tools for this purpose, if you don't have CI tools available then choice has been made for you. However, from our experience, having this kind of service was a great asset for a company and I really wonder why there's no such products available elsewhere yet.
I have writting some JUnit integration tests that currently run from Maven (via the command line or out CI server). These integration tests automatically configure and startup the database and servlet container inside of Maven's 'pre-integration-test' lifecycle phase.
I would very much like to run these JUnit tests (like I do all our other tests) from within an Eclipse JUnit launch configuration. However, the JUnit launch configuration does not trigger the pre-integration-test executions and thus the environment to test is not properly established.
Whats the best way to get this to work?
You can setup Maven Run Configurations to run within Eclipse, using Run -> Run Configurations, assuming you have the Maven plugin for Eclipse. Using the goals clean verify will run unit tests and integration tests.
Also if you use the failsafe reports Maven plugin, you can create JUnit style reports, that you can open in Eclipse using the JUnit framework, to see which tests passed and failed.
One way to do this I just discovered is to use Eclipse Launch Groups as shown here:
http://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.cdt.debug.application.doc%2Freference%2Fcdt_u_run_dbg_launch_group.htm
I missed this because this capability, while language independent needs to be installed from the C/C++ side of Eclipse as shown here:
What installable component provides 'launch groups' in Eclipse?
Within the launch group the first job should run a maven task to process test resources, the next jobs launch the server(s), and then finally a job to execute the junit tests.
I don't know what database you want to use, but for unit tests its handy to use an in memory database and maybe also an embedded servlet container. Maybe you could use something like http://arquillian.org/
JUnit framework should be mainly used for Unit testing and not Integration testing.
Integration testing could be performed by QA team.
If you have lot of Integration plumbing code then you should considering Mocking of external elements, such as Databases and web services calls.
Concentrate mainly on testing code that you have written.Add negative, positive and exceptional test scenarios. That will give your code more credibility than full integration test.
For Database I suggest using DBUNIT > http://www.dbunit.org/ which is an excellent tool for testing database plumbing. This tool can create XML schema from your database and you can keep data as xml replicating the database.
This will avoid changing any dataset for your JUnit testing.
I have a doubt. Say I have a web application which is big and relies on Java/Java EE (JSP/Servlets).
Every time before a drop we test each and every functionality on GUI so that everything is working properly. Previously it was easy but now as the number of modules has increased exponentially, manually testing each and every GUI with required functionality is no more a feasible option.
I am on lookout for tools in which I can write my entire test case say about 1000 and then just run it once before the drop and it will list down all the test cases that have failed.
The tool preferably must be free to download and use it.
I dont know whether using
Arquilian
or
JUnit
in this regard will help or not but automating testing before the drop is really needed..
Please guide.
Use Junit together with a mock framework i.e Mockito to test units (service methods)
Use Arquillian to test on an integration level ( how different services, modules work together )
Use a database testing tool (i.e dbunit) to test your database / persistence layer)
Use Selenium to test your frontend
Test as much as possible.
Use Jenkins and Sonar to track your build process and your quality of tests and code
You should always test your application on different level. There is not just one solution.
Use unit testing to test small pieces of your application and to make refactoring as easy as possible.
Use integration test to check your modules still work together as expected.
Use GUI testing to check if your customers can work with your software.
If its relevant, think about performance testing (i.e. jmeter )
Definitively Selenium. Couple it with maven cause you will probably need to package your project specifically for testing purpose. Moreover maven allow you to launch a container during the integration-test phase and to close it automatically at the end. You can also configure this as a nightly build on jenkins / hudson so you will be quicly notified of any regression.