I recently managed to convince my mates in the project that we need testing (!). Due to the highly dynamic and flexible structure of our web application, with behavior depending of lots of parameters and permission relationships, they had rejected testing altogether, for the usual reasons (time consuming, test maintenance, etc.).
We will introduce testing at the service layer:
Web Browser -> GWT/RPC -> GWT Servlet -> RMI -> SessionEJB -> RMI -> Spring beans
Thus after the GWT Servlet.
Do people recommend to use junit? Or are there other test frameworks better suited? Any other general suggestions? Thanks
You can indeed use plain JUnit or TestNG with a mock framework to test your SessionEJB and individual Spring beans in isolation, i.e. proper Unit testing.
But since there is already a lot of code written, you'll probably find more bugs with less code using system testing or integration testing, i.e. test your complete SessionEJB and spring beans roundtrip in a test application context, with even a real database behind.
For integration and system testing, you can use DBUnit to have a fixture of test data in a database. And Spring also has a lot of test support utils. All of this things work with both JUnit and TestNG.
You should be able to JUnit your Servlets & EJBs. I suggest using some kind of mock framework (e.g. EasyMock) for your servlet context and if you are using any kind of JNDI resource or dependency injection.
As for a testing framework, I highly recommend TestNG (http://testng.org), with Mockito (code.google.com/p/mockito/). I love using both due to their ease of use. #DataProvider in TestNG helps me a lot, as well as other annotations for setting up a test before/after running. I was using JUnit before until I met TestNG at work and don't think I'll be going back anytime soon :)
Check them out, TestNG is definitely picking up some steam and gaining reputation.
Related
I am a Java developer. We want to use cucumber testing in our project. We are working mainly on creating APIs. I am good with unit testing and researching about cucumber.
I am thinking about testing persistence methods - CRUD operations as an starter. My questions is that what could be the scenerios in this testing.
Also should I mock the database by creating tables in the feature file. Should I use mockito with Cucumber to mock call to some other services which connects to database and server.
What should be the cucumber testing in these scenerios and whats the best way to create framework to use cucumber in our Java API's project.
Also, how to populate models if not using database
IMO Gherkin (the language you write Cucumber features in), is good for writing business readable, simple scenarios.
To answer quickly, I would say that Cucumber is not a good fit for testing methods, if it is what you want to do.
As you can see with the file naming convention, you write *.feature files, and I think these files must only contains feature-related descriptions.
However, if you do have features to test, you have to choose how to test them
disconnected, can be run quicky by your CI
you will have to mock everything that cannot start-up in the build lifecycle
and they are solutions to start almost anything using Docker, like Testcontainers
connected to a environment
you do not have to mock anything
your tests may be slower
your tests may break because of the environement (failed deployement, server down, etc.)
I am an entry level tester, mainly been doing manual testing for a company in the UK following scripts on a spreadsheet which I have written in the BDD format, however, I have been learning some automation on the side as that's what I want to move into full time. I have some questions though which are as follows.
I've been using Selenium web driver + java bindings to make simple tests such as logging in or filling out a registration form, i've also set up log4j but only basic to record low level recording. I have now come across testNG. My main question is this framework used by testers? or developers? Is testNG only for unit tests? or UI tests?
From what i've learnt so far the developer does the unit and component tests and the tester does the services/ui tests is this correct?
Unfortunately I was put into a team of developers and not testers as this is my first job outside of university. So I haven't had the chance to learn from other testers. There was no plan for me when I started just that I was going to be the first tester in this development team without any prior testing knowledge.
Which is why I need a bit of guidance on these issues.
My main question is this framework used by testers? or developers? Is
testNG only for unit tests? or UI tests?
TestNG can be used for both, developers and automation testers, it is a tool that can operate over and together with Junit, basically in some cases is being used to create the concept of test suite, that allows to split all the test cases based on specific criteria (time, module, complexity). Also this framework can be used in unit testing and integration testing as well as ui-testing.
TestNG also in some cases replaced Junit entirely, whit this approach you will have a framework with some out of the box capabilities as DataProviders, Multi threading support and other, you could check this link, consider this as and powerful option for Junit.
From what i've learnt so far the developer does the unit and component
tests and the tester does the services/ui tests is this correct?
Unit testing which I consider very similar as "component test" is being done by the developers. If you have web services or a REST API, developers sometimes are in charge of create some test using integration testing, basically verify that services are working as we expected, returning JSON/XML with the correct format and other kind of validations.
Testers also could check services, using tools such as Jmeter, SOAP-UI, they check more things related to the business logic.
Finally I would said UI test is being done in most of the places by the manual and automation testing team, in places where is no QA department this tasks also belongs to the DEV team.
In order to run tests you need to have a test runner it could be anything, most common in java world is JUnit and TestNG, with those frameworks you can run the tests which annotated by #Test tag, also you can group the tests the way you want it and run them in parallel.
Testers use it to run Selenium tests and do assertions, even though for assertions it is good to have knowledge of hamcrest matchers. Also it providing you reports after tests been completed.
Developers would use same frameworks for unit testing purposes.
Check out guys from toolsqa.com they have pretty comprehensive tutorials on using Selenium with TestNG.
TestNG is basically used by developers for doing unit testing, I agree. But it is also widely used by system test automation using Selenium. This framework is inspired by JUnit framework, and most of the automation test developers use this framework because of its advantages and more added features to support reporting.
I can say following advantages I got by using this framework:
1.Support for parameters.
2.Supports dependent methods testing.
3.Test configuration flexible. Supports powerful execution model.
4.Embeds BeanShell for further flexibility.
5.TestNG has a more elegant way of handling parameterized tests with the data-provider concept.
6.For the same test class TestNG support for multiple instances.
7.Extendibility of using different Tools and plug-ins like Eclipse, Maven, IDEA etc.
8.Default JDK functions for runtime and logging (no dependencies).
9.Supported different Annotations like #BeforeSuite, #AfterSuite, #BeforeClass, #AfterClass, #BeforeTest, #AfterTest, #BeforeGroups, #AfterGroups, #BeforeMethod, #AfterMethod, #DataProvider, #Factory, #Listeners, #Parameters, #Test.
The most beautiful part I found in testNG is, using data provider, i can easily read test inputs and expected results from excel. And I can able to see the Results of Pass/Fail and skip test cases in an emailable format.
For testing a system, we don't need any training/extra classes. Just if we know the system requirements, and this as a end user what they want from the system and start testing. If any deviations found in the system behavior and are not as per the expectations of user. Then mark it as an issue and raise a defect and track it until it get resolved. Retest the same and confirm that the system is working as per the expectations. even at the Unit test level this principle holds the same. But only the difference is that we can do Structure based testing there.
To your questions ..
1.My main question is this framework used by testers? or developers? Is testNG only for unit tests? or UI tests?
Answer = Test NG can be used for unit testing as well as UI testing. the advantage of test NG over JUNIT is that you dont need to write code for test result reporting.
I am confused between the following test framework/tools:
JUnit
Shale
FitNesse
I need a test framework which is lean and generic in a way that the test cases can be re-used later by any other web application.
Any suggestions and other testing tool for the same ?
JSF Testing Tools is not the latest article, but giving a nice overview.
It depends upon what you want to test. If you want functional end to end test then which framework you use to build the webapp doesn't matter much and my preference is Selinium 2/WebDriver. If your intention is to do white-box testing then I suggest JSFUnit or Arquillian.
I read article saying
Testing support baked-in : Testing is
a priority and first-class citizen in
Grails. Grails promotes testing; and
provides utilities to make testing
easier — from low level unit tests to
high level functional tests. In
addition, Grails is not married to a
particular testing framework. You can
use JUnit, Spock, EasyB, Geb,
Selenium, Canoo etc. Any testing
framework can be made to work with
Grails (by writing a plugin that hooks
testing framework with Grails testing
infrastructure).
Does this mean that I can test Grails just like any other Java EE framework? Is that block of text saying nothing(like Grails have integration with jUnit) or is there anything special about Grails testing?
EDIT:
How does it compare to SeamTest?
I would say that Grails supports testing by means of a folder structure that already contains folders for unit and integration tests, and its commands help out with test writing. When you create a domain class or controller, for instance, it automatically creates test stubs for you. It also has commands to run all tests, run unit/integration tests only or run individual tests - these create reports for you automatically in the test folder.
You can also find a lot of plugins that support testing - there is a good functional test plugin that uses HtmlUnit to test actual requests. There is also a Selenium plugin.
My overall experience with Grails has been very positive and I highly recommend it as a framework.
I hope this helps.
As Matthew pointed out, the testing infrastructure is all set up. The directory layout is defined and tests can be run through the grails script.
Overall, the testing environment of grails and SeamTest aren't that different. They both have unit tests sans database, and integration tests that has the whole stack. The differences are mostly of a java vs. groovy nature.
Just like SeamTest provides a layer over TestNG, grails has a layer over JUnit, that provides similar support. grails.test.GrailsUnitTestCase and groovy.util.GroovyTestCase are good starting points to see how they compare.
In my opinion, where grails really stands out is in its mocking support. It uses groovy to provide very flexible mocking. In particular, you can dynamically override methods with mock versions directly on classes and objects, so there's no need to create mock classes. The framework provides shortcuts for mocking out the whole ORM layer, which allows you easily test higher level components without the overhead of the database.
Take a look at the manual's chapter on testing for some concrete examples.
How do you automate integration testing? I use JUnit for some of these tests. This is one of the solutions or is totally wrong? What do you suggest?
I've used JUnit for doing a lot of integration testing. Integration testing can, of course, mean many different things. For more system level integration tests, I prefer to let scripts drive my testing process from outside.
Here's an approach that works well for me for applications that use http and databases and I want to verify the whole stack:
Use Hypersonic or H2 in in-memory mode as a replacement for the database (this works best for ORMs)
Initialize the database in #BeforeSuite or equivalent (again: easiest with ORMs)
Use Jetty to start an in-process web server.
#Before each test, clear the database and initialize with the necessary data
Use JWebUnit to execute HTTP requests towards Jetty
This gives you integration tests that can run without any setup of database or application server and that exercises the stack from http down. Since it has no dependencies on external resources, this test runs fine on the build server.
Here some of the code I use:
#BeforeClass
public static void startServer() throws Exception {
System.setProperty("hibernate.hbm2ddl.auto", "create");
System.setProperty("hibernate.dialect", "...");
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setJdbcUrl("jdbc:hsqldb:mem:mytest");
new org.mortbay.jetty.plus.naming.Resource(
"jdbc/primaryDs", dataSource);
Server server = new Server(0);
WebAppContext webAppContext = new WebAppContext("src/main/webapp", "/");
server.addHandler(webAppContext);
server.start();
webServerPort = server.getConnectors()[0].getLocalPort();
}
// From JWebUnit
private WebTestCase tester = new WebTestCase();
#Before
public void createTestContext() {
tester.getTestContext().setBaseUrl("http://localhost:" + webServerPort + "/");
dao.deleteAll(dao.find(Product.class));
dao.flushChanges();
}
#Test
public void createNewProduct() throws Exception {
String productName = uniqueName("product");
int price = 54222;
tester.beginAt("/products/new.html");
tester.setTextField("productName", productName);
tester.setTextField("price", Integer.toString(price));
tester.submit("Create");
Collection<Product> products = dao.find(Product.class);
assertEquals(1, products.size());
Product product = products.iterator().next();
assertEquals(productName, product.getProductName());
assertEquals(price, product.getPrice());
}
For those who'd like to know more, I've written an article about Embedded Integration Tests with Jetty and JWebUnit on Java.net.
JUnit works. There are no limitations that restrict it to being unit tests only. We use JUnit, Maven and CruiseControl to do CI.
There may be tools that are specific for integration testing, but I would think their usefulness is dependent on what type of system components you are integrating. JUnit will work fine for non UI type testing.
When using Maven to build a project, I've had a little more luck with TestNG because it has #BeforeSuite and #AfterSuite operations. Which are useful because Maven will not execute the 'post-integration-test` if any of the integration tests fail. Not a problem with Ant, so I just use jUnit out of preference with it.
In either case, segmenting out the tests as both TestNG and jUnit do is helpful with integration tests too.
In our work here, our integration testing solution has three major parts:
CruiseControl is the foundation of our continuous integration methodology.
Our CruiseControl configuration kicks off a quick-test build within 3 minutes of anyone's checkin to Subversion. The tests that happen here are "does everything still compile?" and "do the unit tests all still pass?". JUnit is obviously the major facilitator in answering the second questions.
Every hour, it kicks off a larger build that constructs the online help and installers that we use on our various deployment platforms. This step verifies the bigger questions of "do we still have a deployable product for each of our target platforms?"
The end result is that most people here never worry about integration testing: it just happens. Unit testing, on the other hand, is everyone's priority. JUnit makes it easy to construct tests, though good tests will always require thought and development time.
Yes, you may use junit for integration tests, but it depends on the type of integration test you need.
Testing a servlet:
setup the servlet context and config
do the tests using mock servlet requests (Spring has support for this, but you may also use EasyMock or your own mocks)
Testing a spring application:
use AbstractDependencyInjectionSpringContextTests to setup the context
test the wired beans
there are also subclasses of AbstractDependencyInjectionSpringContextTests supporting transaction handling when testing with a database.
But pure Junit has its limit. Testing user interfaces is a typical case. You may use selenium for web applications, soapui for webservices or other appropriate tools.
But whatever you use, it should be possible to integrate it in your continious build (cruise control, team city or whatever).
Definitely!
We use a combination of JUnit, ANT tasks to run them, and Hudson for continues integration tests. Works like a charm.
The suggestion depends on your application and your objective.
I've written integration tests in JUnit, but I've also seen people use HtmlUnit (JUnit extension), Selenium, Watir, Fit/Fitness, and even commercial tools like WinRunner and Silk.
So tell us a bit more about your domain and the objectives of your tests and you can probably get a better answer.
There is a very good extension for JUnit called Jitr.
Jitr is a JUnit Integration Test Runner and it allows your web application integration tests to easily run against a lightweight web container in the same JVM as your tests.
See their site for details: http://www.jitr.org/
Update for 2012: Whilst JUnit can be used (and benefits from CI support) JWebUnit and Selenium appear to be eating up the mindshare for Integration Testing.
I think automation and integration tests do not play well together. The very basic problem is environment setup before every test. The more integration-type test bigger setup is needed.
My thoughts on test automation on integration layer: http://blog.aplikacja.info/2012/03/whats-wrong-with-automated-integration-tests/