We have recently started TDD and we are writing JUnit test-cases for each of the RESTful webservices that we develop. Is it a good idea to run the JUnit test-cases on the production environment when a new version is released?
Background:
Our web-app is enterprise web-app and has complex business logic. The plan is to use #After and #AfterClass to cleanup the test-data generated by running test-cases.
The web-app is available as both SaaS and on-premise editions and we are planning to validate upgrades for both edition if it's feasible and a good practice. I understand that we will be having this test-cases run on staging environment but to make sure that we have not broken anything during upgrade/deployment or environment has not created any diverse affect.
Now question is, "Is it good practice? if no what is suggested?"
Your unit tests should just test the code. They shouldnt be modifying with external sources like databases and/or web services then they become integration tests.
Generally I would suggest running unit tests before building/deploying your project to production, these can be run on a build server or QA / staging server. Only if they all pass do you deploy to production, then you know the build is stable in that sense.
If you have integration tests that do speak to other services and may modify data etc, I wouldnt run these on production. I would run them on your QA/Staging server against the code that is about to be released to production.
This is an interesting question; I would say no you don't want to run a full unit test suite, however you might want to invest in maintaining a separate deployment smoke test suite.
Reason - it's possible your test suites could affect a prod database, e.g. bug in an actual test fails to rollback or something.
Also, you are not wanting to test your whole codebase in prod, just verify it deployed OK (assuming you got satsifactory results when you did run your whole test suite).
Related
I am a Java developer, not a QA engineer, so my question may seem trivial to some.
How do I run my integration tests post-deployment? In normal build and deployment process, all JUnit tests are executed prior to deployment. After deployment is done, testing (as I understand) should be done using other (black box) methods. But what if I want to periodically check my application using my nice suite of integration tests?
What would be a proper way to run IT tests against an existing deployment (including production)?
Many companies have two environments: production and UAT. The production environment is the live production site. UAT has the same build as production but can be used for testing, trying things out, and comparing if production has a problem.
Should you want to run JUnit tests on production (and like cjds i don't understand why you would), then UAT is the place to do it.
Is it possible to automate tests on a server using puppet ? I have done some research but cannot find any accurate resources. if its possible, can it be done without the puppet enterprise ?
I am trying to test how much load a software (java server) can handle under different configurations, is it possible to do this using puppet?
If someone can direct me to a good source that'll help a lot.
Thank You.
What kind of testing are you after for this
possible to automate tests on a server using puppet ?
As you know Puppet is about managing your (test) infrastructure, like getting the required this test environment to the desired state (fixtures). My guess is that you need to consider one of the most used CI servers - Jenkins, TeamCity or Bamboo. They all can support your testing process. As example I've used Jenkins and remote physical machine to run
API tests on it, and TeamCity and remote VM for my Selenium grid server.
UPDATE:
For your performance testing you can use some tool like JMeter, framework like Grinder or your own framework implementation with some client like google-http-java-client. And Puppet to tune this java server
under different configurations
per test execution.
Are you supposed to run automated integration tests against a QA server or are you supposed to somehow start an application server from your tests? Does anyone do option #2? How are you supposed to start an application server from tests?
I'm just running into the dilemma of not knowing where to point my selenium driver to. This is a spring java app.
Are you supposed to run automated integration tests against a QA
server or are you supposed to somehow start an application server from
your tests?
As a practical concept, at least the way I see it, the more your test environment(s) looks like your production environment(s), the better. It means that even hardware, location, operational system, etc, have to be considered.
It all comes down to how much "effort" the project is willing to invest on the quality of the product.
You are supposed to run automated integration tests based on your product and project contexts. There isn't a single and final answer to your question, because there are a lot of variables that have to be considered.
Does anyone do option #2?
Yes, I do use a embedded application server but I only used it for database integration but you can apply that for functional automated testing as well.
How are you supposed to start an application server from tests?
One option is to use embedded containers that you can manage with Maven profiles. I recommend you to follow this Arquillian Getting Started guide to understand how it works, and then you can apply the same concept for Selenium and Spring.
I usually go with option 2 -- I use the Maven Jetty Plugin to start an application server running the webapp (usually under a 'test' profile to swap out certain dependencies like the database) and then run Selenium against the locally hosted application. You can bind the Jetty plugin to pre-integration-tests, and stop it in post-integration-tests.
I typically also include the JaCoCo plugin to instrument the Jetty JVM so that I can check coverage from Selenium-style integration tests.
I'm trying to find a practical unit testing framework for JSF.
I know about JSFUnit, but this is very impractical to me. I need to include about 10 JARs to my project, and jump through many other hoops just to get it running.
I realize that -- due to the need to simulate a platform and a client -- unit testing web applications is difficult. But is there a better way?
Have you thought about doing integration testing with Selenium or another tool? Selenium allows you to record and run tests directly in the browser. You can also run tests in multiple browsers and on multiple platforms with Selenium Remote Control.
Writing unit tests is good, but it might provide more to create some functional integration tests rather than unit-testing the presentation layer code.
On the project I'm working on at the moment we dabbled with using selenium. We actually spent a lot of time writing these selenium tests, but found that they added little value because the UI changes so much and you just end up doubling your effort for very little return on investment.
Another problem with selenium is that it requires your code to be deployed, which means it doesn't play well with unit test frameworks eg maven.
What I would say is that writing really good unit tests for your managed beans is invaluable.
Have you taken a look at the jsfunitwar Ant task or alternatively the Maven plugin provided by JSFUnit? Both greatly reduce the complexity of generating the .war file to be tested. I'm using JSFUnit on my current project and find the combination of white box and black box testing capabilities to be very powerful. Because JSFUnit uses HtmlUnit under the covers, you can very easily and effectively examine the generated HTML, or conversely, verify the state of your internal JSF backing beans. I was able to incorporate the JSFUnit tests into my Continuous Integration process and have been quite pleased with the outcome.
HttpUnit can also be an alternative. It provides apis so you have a choice to automate the tests.
http://httpunit.sourceforge.net/index.html
Selenium is superficial, jsfunit is inward. I recommend that use jsfunit if project is not simple. Because team member can change jsf managedbean names or etc, you can catch that with jsfunit.
I'm with Paul on Selenium being very easy to setup and start working with. I use Selenium IDE in Firefox with some customization at that level, then you can export these to other platforms such as Java JUnit tests. It was quite easy to download and launch the selenium-server.jar, add the selenium-java-client-driver to my existing Eclipse Maven POM driver project; then launch the same exported JUnit test in Eclipse. I mainly wanted to use Java just for looping which the basic Selenium IDE didn't support.
I have configured JSF Unit for my project too which does require more time to configure... more importantly though with in-container tests like JSFUnit changes to the test require rebuilding the WAR, redeploying in the container and then executing from Eclipse or via a browser. So for quickly trying a small change this is time consuming. Of course with JSFUnit you have access to all the internals of the JSFSession etc so it depends what granularity of testing you need I guess.
I'd be interested if anybody knows a faster way to turnaround changes to a JSFUnit test and execute it. Definitely Selenium tests feel more like JUnit tests in that regard.
We're migrating our application into a Java EE container, and looking for tools to use for unit testing (and integration testing) our migrated app.
Our requirements include:
Ad-hoc testing: the ability to run tests manually, on demand (to be used by developers while developing code)
Batch testing: the ability to run a large (and growing) set of tests regularly
In-Container: integration tests that use EJBs as they are deployed in the container
Unit testing: Testing of classes not necessarily inside an EJB context
Nice to have: Simple to set up, integrates with ant/IDE
No requirement to test Servlets/JSPs - only POJOs and EJBs
What are you using to achieve testing in Java EE environment? What technologies/setup have you deployed?
My research have uncovered Cactus and JUnitEE: have you had success setting them up?
We use normal JUnit for both unit tests and integration testing. We switch between the two using a VM argument, and have the tests annotated with markers for direct vs. server. We do have a custom TestSuite class though that finds and runs the tests based on this information, as it was easier and less error prone than manually maintaining which tests to run.
In our case we use Spring remoting to talk to servlets and EJB's (via the servlets), and testing both cases is simply a separate launch configuration within Eclipse.
We used JunitEE a few years ago, but eventually gave up on it in favor of just using JUnit throughout. This enabled us to have developers do all their testing without a server at all and run both unit and what I would call low level integration tests in their IDE. Then we let the build machine run the same integration tests against the same code now deployed in the actual server. This makes the development cycle much faster as we rarely need to run the server and deploy service code.
JUnit(EE) is what every major project($100mil+) I've supported has used. It's really a fantastic tool and knowing how to use it is invaluable when/if you decide to look for other job opportunities.
I supported a government financial system that used no unit testing but after a lot of pushing we finally implemented JUnit. The system I work on now is a large government agency modernization and we use JUnit for all of our unit testing. The two large firms supporting the modernization have essential made JUnit the standard across all of the sub projects. We've got ~200 developers using it without a hitch.
It's quick and easy to set up and once you understand it and you can leverage the features it will demonstrate how invaluable it is.