We would like to have a set of tests as part of our web application. The tests will be used for analyzing the health status of the application, so a support person or a scheduler can run the test to see if the application itself and various required remote systems are available.
I have seen this being done using some kind of webbased JUnit frontend, it allowed to run tests and reported the results as HTML. This would be great because the developers know JUnit, but I couldn't find the library in the intertubes.
Where can I find a library doing this?
You can use some free services to verify the availability of your system. Here are two that I've used:
mon.itor.us
pingdom
Another thing you can take a look at is JMeter, but it does not have a web UI.
Original answer:
Perhaps you mean functional tests (that can be run through JUnit). Take a look at
Selenium - it's web functional testing tool.
(Note that these are not unit tests. They don't test individual units of the code. Furthermore unit tests are executed at build time, not at runtime.)
Bozho is correct, these are not unit tests but I have done something similar. At my company I am not the one that ultimately deploys these things to our test environment or production environment. During development I create a couple of servlets that test things like it can get a valid database connection, it can hit our AD server etc. It than basically prints out a message and indicates success or failure.
That way when I have the code deployed to one of our environments, I can have the person deploying it hit the URL and make sure everything comes back OK. When I get ready to do the final deployment I just remove the servlet config.
If you already have a set of tests composed and ready to run, then Hudson can run those tests on a schedule and report on the results.
Update: If you're looking for a tool to check your servers and applications every few minutes for availability check out Nagios.
Maybe you mean some kind of acceptance test tool. If so, have a look at Fitnesse.
What you're probably looking for is CruiseControl.Net - it combines with NUnit/JUnit etc to make an automated testing framework with HTML reporting tools and a tray app for your desktop as well. I actually just downloaded it again an hour ago for a new role - it's really good.
It can be used to run anything from unit tests to getting files from source control, to kicking off compiler builds or rebooting servers (when used with NAnt - a .Net build tool).
You should look for a Continous Integration tool like Jenkin.
Related
We have Spring Boot application and we have integration tests using JUnit 4 and RestAssured Framework.
We have a lot of test related classes in our project.
Our integration tests take a long time to run locally, because each class bootstraps and brings the server - which runs the tests in the class and then terminates the server.
This happens for each class.
Bringing up the server and killing the server each time takes a lot of time - which makes running all the tests together really slow.
I would like the server to launch locally only once - and all tests to be run against it.
So basically my use case is when i am trying to run all tests at once.
Thanks in advance
Best Regards
Integration tests are typically slow. But to maintain test independence it is not always an option to move test preparation and cleaning outside of the test.
How about making the tests run in parallel? See for Gradle. This can be done in Maven as well I think.
Yes, that usually introduces a new set of troubles, like making sure that tests that run at the same time do not influence each other. Usually the ports used need to be test specific, usually providing 0 will imply: find a free port. During the test the actually used port can be retrieved.
The total speed up may be better this way, while maintaining testing independence.
We are generating a Web Service for deployment to Azure. This includes four pipeline stages for Dev, Test, Full UAT and production. On initial deployment to Dev I want to perform a set of Selenium smoke tests. Then when deployed to UAT, a full set of automated tests should be triggered.
Our test team are happier using Selenium through its Java route. After a couple of days it became clear that the process was to generate a UI agent (really important to anyone who hasn't done this yet, as ChromeDriver does run without a session, but will just hang, making you think it must be close to running), assign a SELENIUM_TEST agent property, and set this flag as a build dependency (this helps it to find the correct agent), and ensure that you set the required java and maven variables in the VSTS settings (apart from the path), rather than the local machine environment. Finally to use the clean, update and -X parameters to force the environment to be configured as part of the test process.
Now I have the problem of how do I trigger these tests from the deployment pipeline. I have searched and found articles on a large number sites and cant find anything on how this may be achieved using the Maven Java Selenium combination.
Can anyone help?
For build and deploy Java Selenium Tests in VSTS, you can refer the document Testing Java applications with VSTS for detail steps:
Besides, you can also refer the blog Continuous Testing of a Java Web App in VSTS using Selenium for build and deploy Java Web App in VSTS.
I am not posting this as a full answer, but I wanted to respond to the kind input from Michele and Marina. I am not sure that there isn't a better way of approaching this but with the assistance of both I was able to at least get closer to an answer. I did prepare images, but apparently you need a reputation to do so.
So this is what I actually ended up doing.
Step 1 – the MVC web app was generated and appropriate deployment slots set up to receive the web build artefacts.
Step 2 – Created a CI process purely to generate code I could deploy into the WebApp CD pipeline.
Step 3 – Generated an empty “Smoke Test” environment in the WebApp Deployment pipeline, and added the new Artefact from step 1 into this.
Configured the Smoke Tests item
Configured it to only receive the _AutoTest-CI artefact
Set it to use the “default” pipeline
Added the “Demand” that specifies the machine configured for Selenium tests.
Added the Maven task, and pointed it at the Maven POM
At this stage it succeeded to run through the configured tests. The Maven deployment step seems to have the idea that it can generate test results, but the output gives warnings that no test results were generated. It will generate the output, and reports a success or failure, so this is a semi-success. The missing last piece is to report the full test results, which I have yet to achieve.
You can trigger the Tasks inside of an Environment by configuring the Triggers with this tools in the Release management UI.
If the trigger contitions are met the process will start automatically. Inside of your process you can do whatever Task you need.
Reference
Microsoft VSTS Docs
Often some testing framework for automated testing - like Selenium - is used to continiuosly verify the integrity of a deployed application. These tests often cover real user scenarios and may also utilize a range of deployed applications in combination.
We would like to achieve some what the same for a "backend only" application - that is, an application (or more really) without frontend. We are currently building a series of batchjobs where one job produces input to the next.
We have a great unit-test suite that tests the individual jobs however we would really like to test the series of jobs when deployed to some environment.
Do you have any suggestions for such testing framework? The framework must be able to leverage other Java SDKs such as AWS SDK (e.g. to instruct startup of batchjob, inject data to queues etc.). Whether the framework with tests needs to be deployed as an application as well or run directly from CI is secondary.
If you already have backedn tests that can be run on the production all you need is to schedule running on those tests. Jenkins is fine for that (https://wiki.jenkins.io/display/JENKINS/Schedule+Build+Plugin)
You could have emails (or other alerts) for failed jobs. Jenkins will also care for test reports- exactly as it does for unit tests.
Technologies for scheduling test runs
You could schedule running your tests using any other technology- for example Amazon AWS instances, AWS Elastic Beanstalk Worker Environments (https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html) etc.
I find Jenkins most reasonable, because you have out-of-the box support for test reports, notifications, etc.
For any other technologies you would have to write reportting, notifications on your own.
Technologies for writing tests
I could write tests in any technology that is capable of making HTTP REST calls. For performance tests Jmeter or Gatling are good choices.
For acceptance test you could use RestEasy, TestRestTemplate from Spring, Apache HTTP client, etc.
As test running framework you could use Junit4,Juni5, TestNG or Spock (if you are fine with Groovy language). Test structure could be similar to those of ordinary tests. Well named independend methods that test one thing well, meaningfull assertions, etc.
For writing assertions my personal preference is AssertJ, but JavaHamcrest would also do.
Those tests can (and should) be written in src/test directory, in separate repository (or in same repository or different module).
For that test module you may write test-releated services in src/main directory, so src/test directory would contain only test scenarios. Test services may everyting you need- manage files, inspect database, etc.
You may consider writting test scenarios in BDD-style and tools like JBehave or Cucumber. Personally I see value in BDD tests only if business is interested in test scenarios. If those tests are to be used only by technical people then I find easier to maintain such tests in non-BDD technologies (Junit, AssertJ).
I am trying to create the following setup :
A Selenium (Java) project that has a set of 10 automated test cases.
When this project is executed, it generates an HTML test execution report.
This project should be 'hosted' on an internal network.
Anyone who has access to the network should be able to 'invoke' this project, which in turn executes the test cases and passes the HTML report to the person who invoked it.
The project should be accessible ONLY for execution and the code should NOT be accessible.
My goal is that this implementation should be executable by any framework irrespective of the technology that the framework uses. I was thinking of creating the project as a WebService using Java (servlet).
My question is:
Can this implementation be accessed by any external automation framework ?
Are there any limitations to this implementation?
Is there a better way to implement this requirement?
Thanks in advance.
You can create a maven project and have your automated tests under maven test folder.Configure your tests to run through POM.xml(use maven surefire plugin).Configure a jenkins job to run the maven test.Anybody with access the jenkins can build/run this task!
Below link should give you a headstart
http://learn-automation.com/selenium-integration-with-jenkins/
As a matter of fact, it is something we did on one of our projects. As I cannot share specifics, I will give you overall architectural view of the project.
The core of all things was a service that could run JUnit tests on requests. It was a Soap web-service, but nothing stops you from making it REST. To implement this you need to implement your version of JUnit test runners (see for example: http://www.mscharhag.com/java/understanding-junits-runner-architecture or https://github.com/junit-team/junit4/wiki/Test-runners)
If you use JUnit as test framework for running your Selenuim tests this may be a great solution for you - JUnit will generate HTML reports for you if you configure it properly, it will hide actual test suite implementation from users and run test suite on demand. This solution is also great because it operates on JUnit level and does not care about what kind of tests it actually runs, so it can be also reused for any other kind of automated tests.
So to answer all your questions:
Can this implementation be accessed by any external automation
framework ? -> yes, it can be accessed by anybody who able send http
requests
Are there any limitations to this implementation? -> none that I am
aware of
Is there a better way to implement this requirement? -> well, I
didn't actually work with TestNG much so I don't know if it is
easier or more difficult to do it on Junit level. You can use
Jenkins or other CI tool as well to achieve same results - they can
run JUnit tests for you and almost always have API ready for this,
although those APIs may be not perfect.
So I'd say that if you need this only for one thing you can use CI tools for this purpose, if you don't have CI tools available then choice has been made for you. However, from our experience, having this kind of service was a great asset for a company and I really wonder why there's no such products available elsewhere yet.
I have a doubt. Say I have a web application which is big and relies on Java/Java EE (JSP/Servlets).
Every time before a drop we test each and every functionality on GUI so that everything is working properly. Previously it was easy but now as the number of modules has increased exponentially, manually testing each and every GUI with required functionality is no more a feasible option.
I am on lookout for tools in which I can write my entire test case say about 1000 and then just run it once before the drop and it will list down all the test cases that have failed.
The tool preferably must be free to download and use it.
I dont know whether using
Arquilian
or
JUnit
in this regard will help or not but automating testing before the drop is really needed..
Please guide.
Use Junit together with a mock framework i.e Mockito to test units (service methods)
Use Arquillian to test on an integration level ( how different services, modules work together )
Use a database testing tool (i.e dbunit) to test your database / persistence layer)
Use Selenium to test your frontend
Test as much as possible.
Use Jenkins and Sonar to track your build process and your quality of tests and code
You should always test your application on different level. There is not just one solution.
Use unit testing to test small pieces of your application and to make refactoring as easy as possible.
Use integration test to check your modules still work together as expected.
Use GUI testing to check if your customers can work with your software.
If its relevant, think about performance testing (i.e. jmeter )
Definitively Selenium. Couple it with maven cause you will probably need to package your project specifically for testing purpose. Moreover maven allow you to launch a container during the integration-test phase and to close it automatically at the end. You can also configure this as a nightly build on jenkins / hudson so you will be quicly notified of any regression.