I'd like to know if any of You have experience in automation UI testing of modular-like apps. The whole app is like all typical CRM-related apps, where based on Your personal client needs You just put together some of the available modules (that have been predefined earlier) in order to provide all necessary functionalities.
If there would be "static" app built of all these modules put together then we could test it in a quite easy way, just going through all defined test classes, because we would know the behaviour/interactions between all these modules.
But in case we would need to test app behaviour while putting some of its random pieces/modules together in order to check if they work well, we would need some other approach.
If there's a solution, some recommended architect pattern or anything that can help me to perform such automation tests (using i.e. Selenium WebDriver)? Or does this kind of tests are even possible to perform using WebDriver library?
I'd be grateful if You'll share any of Your thoughts and experiences in this area.
I am working in that area and had a similar situation, here's what I learned from it:
Avoid creating UI tests if you can. UI tests are intended to test the look of your application and that's it. Business logic (like when I change that setting, the displayed data should change, etc.) should be tested in unit tests which are much easier to implement. Interaction between the modules should be covered as much as possible in integration tests.
If you still have functionality left over that needs to be tested, create a config file that contains the information about what customer has which modules enabled. In your test, read that config and if a test is not supposed to run, abort it.
In case some further researcher will look for the know-how solution for this case, we can just set some different test suites for each of app modules, and then we can check each suits for some certain condition met. If some suit won't meet this condition then we'll just skip this test suites. I.e we can get the app bundles.json file, which will most likely contain all information concerning app modules, and then we can just process this file to search for modules which are unavailable in current deployed app.
Look this as nice reference on how to achieve this: Introducing to conditional test running in TestNG
The application I am currently working with is being worked on by 3 separate teams, each working away on different functional areas that come together at the end of the day. The difficulty is keeping the 3 teams always in sync and not having one team's issues affect another. I am looking for a way that I can stub out / mock the calls that are being made to some of these services provided by the other teams so that we can work separately most of the time, yet quickly switch back to integrated mode when needed.
Ideally I would like:
- during normal development, I could turn on a flag and those services will be mock services (for example, when I am just developing away on my part of the code and don't really care if the other team's service returns the right thing, just that it returns something)
- I don't want to have add code to check this flag everywhere in the code and if it is on, use the mock, else use the real thing... I just want it to automatically know to use the mock class when this flag is on
We are using Java 7 + CDI + Jboss. Is this possible to do with some kind of wiring or filters?
TIA.
Using Alternatives you can achieve this better ways, you can switch back to integrated mode when needed with disable alternative in beans.xml
CDI Alternatives are so good for the Mock service.
Instead of having to change the source code of your application, however, you can make the choice at deployment time by using alternatives.
Alternatives are commonly used for purposes like the following:
To handle client-specific business logic that is determined at runtime
To specify beans that are valid for a particular deployment scenario
(for example, when country-specific sales tax laws require
country-specific sales tax business logic)
To create dummy (mock) versions of beans to be used for testing
Alternatives allow you to overwrite this at execution time using the beans.xml file - a simple deployment artifact.
A typical scenario would be to use different beans.xml for different environments and thereby enable mock-alternatives for components that you don't want to execute on your local / integration environments.
Using Alternatives in CDI Applications
In Spring-based applications (using dependency injection), I've accomplished this by keeping two application contexts, one configured to use stubs and one configured to use real code. I wrote some wrapper code to load the appropriate application context at the right time, but you could manage it other ways, i.e. with separate run targets that use a different classpath.
In terms of the stubs, I've often created them by hand (i.e. written stubbed services that wrap a spreadsheet of canned data or generate random data), but Mockito could be useful when building certain kinds of stubs.
Depending on what kinds of resources you need to stub (and whether you have a budget), another option is service virtualization. I don't have any direct experience using service virtualization tools, but my current client is using the commercial LISA tool to stub out a SOA service layer. I gather that several companies sell similar tools.
I'm creating a Spring-based Webapplication where I use Spring to add the implementation of a service to the defined interface. So far, so standard. Works fine.
I want to allow a user to overwrite applications behavior at runtime for his session. For that I want spring to change the implementation behind an interface depending on the user-session.
A use case for that are automatic testcases that run on INT and should test the output of an email created by the system. On INT there is an email-serivce configured sending emails to our mail-server. I dont want the testcase to have to check mails using a mail-Protocol. I want in case automatic testcases are running to chenge the email-implementation to write the email as comments to the HTML, so my tests can easily check the result. And so there are some more such cases where it would be nice to change the implementing bean for special circumstances.
Is there a concept in spring that helps me to implement such a feature or do I have to create that on my own?
Additional information: It's all about automated acceptance tests. That tests run on systems that we share with maunual testers.
=> Manual testers want to get a real email for their tests
=> Automatic tests reduce complexity by not receiving emails, just checking the email-content with less dependencies.
There was no problem, if we had two systems, one configured for humans needs and one for automated tests needs. But thats not the case, so I need a way to change the systems behavior on runtime.
It is usually done in the following way:
set up tests using spring-test module which lets you create a spring context for testing and inject beans into your test classes,
use the same context files for tests as you do for production except create a separate spring profile where default mail service implementation is substituted with a mock,
write a test case where you simulate steps done by user programatically and finally you assert something like assertEquals("<expected_email_text>", mailServiceMock.getLastEmail()).
From your question it is not clear why you would deviate from the standard approach described above. If you explained your reasons perhaps it would be easier to come up with an appropriate answer.
Can I create a junit environment without file system and network access? I want to enforce stricter rules for our test cases.
Ideally, this would be configurable with maven and work for the default test phase.
Based on this answer https://stackoverflow.com/a/309427/116509, I think you could set a SecurityManager on test setup and restore the original on teardown.
However, IMHO, some unit tests should be allowed to touch the file system, if for example the class under test actually creates files as part of its contract. (You can use dependency injection to make sure the files are created in a temp directory). Likewise, a good unit test of a class that uses HTTP should test it against an HTTP endpoint. Otherwise you just end up mocking everything and your test becomes worthless. I suppose the default should be to deny access, and then a developer would need to specifically override the permissions for this kind of test.
The typical way to handle these dependancies on a file system/network access, is to mock them out in a test context. This way, the real code can go through the normal channels, but your tests don't have to rely on a file system or a network.
Look into mocking frameworks to help you do a lot of this. Enabling this kind of testing will also make your code cleaner, too. :)
You can use Ashcroft to prohibit access to file system and other resources from your tests. It uses Java Security manager to restrict access to certain resources.
Another approach would be to use AspectJ and implement several advices prohibiting calling certain APIs or packages.
I'm not sure what you mean by JUnit environment, but you should not need a file system, or network access to run unit tests. On the other hand, if you are testing code that uses network and filesystem APIs, you may have an issue. In that case, you may need to abstract your code into smaller testable chunks. You should not be testing weather the network and filesystem APIs are working in a unit test, these are integration tests. You should only be testing your code in unit tests.
I have a couple of design/architectural questions that always come up in our shop. I said "our", as opposed to "me" personally. Some of the decisions were made and made when J2EE was first introduced so there are some bad design choices and some good.
In a web environment, how do you work with filters. When should you use J2EE filters and when shouldn't you? Is it possible to have many filters, especially if you have too much logic in them. For example, there is a lot of logic in our authentication process. If you are this user, go to this site and if not go to another one. It is difficult to debug because one URL path could end up rendering different target pages.
Property resource bundle files for replacement values in JSP files: It seems that the consensus in the Java community is to use bundle files that contain labels and titles for a jsp parsing. I can see the benefit if you are doing development with many different languages and switching the label values based on locale. But what if you aren't working with multiple languages? Should every piece of static text in a JSP file or other template file really have to be put into a property file. Once again, we run into issues with debugging where text may not show up due to misspelling with property value keys or corrupt property files. Also, we have a process where graphic designers will send us html templates and then we convert them to jsp. It seems it more confusing to then remove the static text, add a key, add the key/value in a property file, etc.
E.g. A labels.properties file may contain the Username: label. That gets replaced by some key and rendered to the user.
Unit Testing for all J2EE development - we don't encourage unit testing. Some people do but I have never worked at shop that uses extensive unit testing. Once place did and then when crunch time hit, we stopped doing unit testing and then after a while the unit tests were useless and wouldn't ever compile. Most of the development I have done has been with servers, web application development, database connectivity. I see where unit testing can be cumbersome because you need an environment to unit test against. I think unit test manifestos encourage developers not to actually connect to external sources. But it seems like a major portion of the testing should be connecting to a database and running all of the code, not just a particular unit. So that is my question, for all types of development (like you see in CRUD oriented J2EE development) should we write unit tests in all cases? And if we don't write unit tests, what other developer testing mechanisms could we use?
Edited: Here are some good resources on some of these topics.
http://www.ibm.com/developerworks/java/library/j-diag1105.html
Redirection is a simpler way to handle different pages depending on role. The filter could be used simply for authentication, to get the User object and any associated Roles into the session.
As James Black said, if you had a central controller you could obviate the need to put this logic in the filters. To do this you'd map the central controller to all urls (or all non-static urls). Then the filter passes a User and Roles to the central controller which decides where to send the user. If the user tries to access a URL he doesn't have permission for, this controller can decide what to do about it.
Most major MVC web frameworks follow this pattern, so just check them out for a better understanding of this.
I agree with James here, too - you don't have to move everything there but it can make things simpler in the future. Personally, I think you often have to trade this one off in order to work efficiently with designers. I've often put the infrastructure and logic in to make it work but then littered my templates with static text while working with designers. Finally, went back and pulled all the static text out into the external files. Sure enough, found some spelling mistakes that way!
Testing - this is the big one. In my experience, a highly disciplined test-first approach can eliminate 90% of the stress in developing these apps. But unit tests are not quite enough.
I use three kinds of tests, as indicated by the Agile community:
acceptance/functional tests - customer defines these with each requirement and we don't ship til they all pass (look at FitNesse, Selenium, Mercury)
integration tests - ensure that the logic is correct and that issues don't come up across tiers or with realistic data (look at Cactus, DBUnit, Canoo WebTest)
unit tests - both defines the usage and expectations of a class and provides assurance that breaking changes will be caught quickly (look at JUnit, TestNG)
So you see that unit testing is really for the benefit of the developers... if there are five of us working on the project, not writing unit tests leads one of two things:
an explosion of necessary communication as developers try and figure out how to use (or how somebody broke) each other's classes
no communication and increased risk due to "silos" - areas where only one developer touches the code and in which the company is entirely reliant on that developer
Even if it's just me, it's too easy to forget why I put that little piece of special case logic in the class six months ago. Then I break my own code and have to figure out how... it's a big waste of time and does nothing to reduce my stress level! Also, if you force yourself to think through (and type) the test for each significant function in your class, and figure out how to isolate any external resources so you can pass in a mock version, your design improves immeasurably. So I tend to work test-first regardless.
Arguably the most useful, but least often done, is automated acceptance testing. This is what ensures that the developers have understood what the customer was asking for. Sometimes this is left to QA, and I think that's fine, but the ideal situation is one in which these are an integral part of the development process.
The way this works is: for each requirement the test plan is turned into a script which is added to the test suite. Then you watch it fail. Then you write code to make it pass. Thus, if a coder is working on changes and is ready to check in, they have to do a clean build and run all the acceptance tests. If any fail, fix before you can check in.
"Continuous integration" is simply the process of automating this step - when anyone checks code in, a separate server checks out the code and runs all the tests. If any are broken it spams the last developer to check in until they are fixed.
I once consulted with a team that had a single tester. This guy was working through the test plans manually, all day long. When a change took place, however minor, he would have to start over. I built them a spreadsheet indicating that there were over 16 million possible paths through just a single screen, and they ponied up the $10k for Mercury Test Director in a hurry! Now he makes spreadsheets and automates the test plans that use them, so they have pretty thorough regression testing without ever-increasing QA time demands.
Once you've begun automating tests at every layer of your app (especially if you work test-first) a remarkable thing happens. Worry disappears!
So, no, it's not necessary. But if you find yourself worrying about technical debt, about the big deployment this weekend, or about whether you're going to break things while trying to quickly change to meet the suddenly-urgent customer requirements, you may want to more deeply investigate test-first development.
Filters are useful to help move logic such as is the user authenticated, to properly handle this, since you don't want this logic in every page.
Since you don't have a central controller it sounds like your filters are serving this function, which is fine, but, as you mentioned, it does make debugging harder.
This is where unit tests can come in handy, as you can test different situations, with each filter individually, then with all the filters in a chain, outside of your container, to ensure it works properly.
Unit testing does require discipline, but, if the rule is that nothing goes to QA without a unit test then it may help, and there are many tools to help generate tests so you just have to write the test. Before you debug, write or update the unit test, and show that the unit test is failing, so the problem is duplicated.
This will ensure that that error won't return, and that you fixed it, and you have updated a unit test.
For resource bundles. If you are certain you will never support another language, then as you refactor you can remove the need for the bundles, but, I think it is easier to make spelling/grammar corrections if the text is actually in one place.
Filters in general are expected to perform smaller units of functionality and filter-chaining would be used to apply the filters as needed. In your case, maybe a refactoring can help to move out some of the logic to additional filters and the redirecting logic can be somewhat centralized through a controller to be easier to debug and understand.
Resource bundles are necessary to maintain flexibility, but if you know absolutely that the site is going to be used in a single locale, then you might skip it. Maybe you can move some of the work in maintaining the bundles to the designers i.e let them have access to the resource bundles, so that you get the HTML with the keys in place.
Unit testing is much easier to implement at the beginning of a project as opposed to building it into a existing product. For existing software, you may still implement unit tests for the new features. However, it requires a certain amount of insistence from team leads and the team needs to buy into the necessity of having unit tests. Code review for unit tests helps and a decision on what parts of the code need to be absolutely covered can help developers. Tools/plugins like Coverlipse can indicate the unit testing coverage, but they tend to look at every possible code path, some of which may be trivial.
At one of my earlier projects, unit tests were just compulsory and unit tests would be automatically kicked off after each check-in. However, this was not Test-driven development, as the tests were mostly written after the small chunks of code were written. TDD can result in developers writing code to just work with the unit tests and as a result, developers can lose the big picture of the component they are developing.
In a web environment, how do you work with filters. When should you use J2EE filters and when shouldn't you?
Filters are meant to steer/modify/intercept the actual requests/responses/sessions. For example: setting the request encoding, determining the logged-in user, wrapping/replacing the request or response, determining which servlet it should forward the request to, and so on.
To control the actual user input (parameters) and output (the results and the destination) and to execute actual business logic, you should use a servlet.
Property resource bundle files for replacement values in JSP files.
If you don't do i18n, just don't use them. But if you ever grow and the customer/users want i18n, then you'll be happy that you're already prepared. And not only that, it also simplifies the use of a CMS to edit the content by just using the java.util.Properties API.
Unit Testing for all J2EE development
JUnit can take care about it. You can also consider to "officially" do user tests only. Create several use cases and test it.