I am wondering what people have found their best practice to be for testing Hibernate mappings and queries ?
This cannot be done with Unit Testing, so my experience has been to write Integration Tests that solely tests the DAO layer downwards. This way I can fully test each Insert / Update / Delete / ReadQueries without testing the full end-to-end solution.
Whenever the Integration test suite is run it will:-
Drop and re-create the database.
Run an import SQL script that contains a subset of data.
Run each test in a Transactional context that rolls back the transaction. Therefore it can be run multiple times as an independent test, or and as part of a suite, and the same result is returned as the database is always in a known state.
I never test against a different "in memory" database, as there is always an equivalent development database to test against.
I have never had the need to use DBUnit.
Never use DbUnit for this. It's way too much overhead for this level of testing.
Especially if you're using Spring in your app, check out the Spring Test Framework to help manage your data-access tests, particularly the transaction management features.
An "equivalent development database" is fine, but an in-memory H2 database will blow away anything else for speed. That's important because, while the unit/integration status of these tests may be contested, they're tests you want to run a lot, so they need to be as fast as possible.
So my DAO tests look like this:
Spring manages the SessionFactory and TransactionManager.
Spring handles all transactions for test methods.
Hibernate creates the current schema in an in-memory H2 database.
Test all the save, load, delete, and find methods, doing field-for-field comparison on before and after objects. (E.g. create object foo1, save it, load it as foo2, verify foo1 and foo2 contain identical values.)
Very lightweight and useful for quick feedback.
If you don't depend on proprietary rdbms features (triggers, stored procedures etc) then you can easily and fully test your DAOs using JUnit and an in memory database like HSQLDB. You'll need some rudimentary hibernate.cfg.xml emulation via a class (to initialize hibernate with HSQLDB, load the hbm.xml files you want) and then pass the provided datasource to your daos.
Works well and provides real value to the development lifecycle.
The way I do it is pretty similar with your own, with the exception of actually using in-memory data-bases, like HSQLDB. It's faster and more portable than having a real database configured (one that runs in a standalone server). It's true that for certain more advanced features HSQLDB won't work as it simply does not support them, but I've noticed that I hardly run into those when just integration testing my data access layer. However if this is the case, I like to use the "jar" version of mysql, which allows me to start a fully functional MYSql server from java, and shut it down when I'm done. This is not very practical as the jar file is quite big :
http://dev.mysql.com/doc/refman/5.0/en/connector-mxj-configuration-java-object.html
but it's still useful in some instances.
Related
I'm having a Java EE application that runs on JBoss 7. In order to do TDD, I'm setting up embedded tests that use Arquillian with embedded Weld and a H2 embedded database.
This works fine, but the initial startup of Hibernate takes a considerable amount of time (5-10 seconds) and I haven't included all JPA entities yet. I have tried to use a persisted Oracle DB instead to avoid table creation, but it doesn't make much of a difference.
The main problem seems to be that Hibernate goes through all the entities and validates and prepares all the CRUD methods, named queries and so on.
Is there any way to tell Hibernate to do this lazily when needed (or not at all)? Most of the time, there will only be a subset of entities and queries involved in a test case, so I'd happily trade in execution time for start-up time while implementing.
Any ideas?
I know I could just use a subset of the entities, but it's sometimes difficult as they often have relations to other entities not needed in a test context. Or is there an easy way to 'deactivate' such relations to generate subsets of the database?
Clarification
It seams like it's not clear what my problem is, so I'll try to clarify:
I have set up a testing environment with Arquillian (embedded Weld) that sets up an embedded database (H2) to do JPA enabled testing
I would like to use this approach to do Test Driven Development (TDD), which means I will have the following workflow on my local developing machine:
Create test case
Run test case
If test case fails, implement necessary changes and go back to 2.
Normally, one will perform steps 2 and 3 a couple of times before finishing a feature which means that I will often run a single test from my IDE that has to set up the entire testing JVM with Arquillian, Weld, embedded DB and whatever to run just A SINGLE TEST.
So much for the scenario. Now I've noticed that running that single test takes around 10 seconds, which is not the end of the world, but rather long to do TDD. And when I further investigated, I've noticed that most of this time goes to Hibernate setting up its internal structures (it's not Weld, Arquillian, Schema creation or whatever, but Hibernate getting ready to provide an EntityManager).
So my question is: Is there a way to speed up hibernate initialization so I can drop these 10 seconds to maybe 1-2 seconds? I wouldn't care if it's sort of a hack (like keeping the testing JVM with hibernate alive during multiple manual test runs or deactivating some validations or optimizations of Hibernate). My only issue is the start up time for a single test. Consecutive tests run fine and fast, so I don't have a problem with full regression testing or with testing on my build server.
Hope that makes my case a bit clearer...
Let me guess, you are using junit library?
Database schema creation usually isn't the most time-consuming operation (although it depends on amount of entities).
Personally, if I were you, I would run all your JUnit tests with TestNG (yes, TestNG can run all JUnit tests out of the box) and would take a look at <your-module>/test-output/index.html (particularly its sections Chronological view and Times). Then you would know what operations are the most time-consuming. And with such information, you can go further.
Furthermore, a couple of items of advice:
Arquillian tests with a remote (or managed) containers are usually faster, because you start server only once.
H2 embedded database isn't a bad choice, really. Typically you don't have to abandon it (typically, because sometimes your application may use some of the target database features, that are not present in the H2).
There is Arquillian Suite Extension that lets you do deployment only once and reuse it across the test classes. In case of many tests, this extension can speed up tests execution significantly.
You can test your entities outside any container (so called standalone JPA with transaction-type="RESOURCE_LOCAL"). It is the fastest way, but suits well only for testing entities annotations, relations, queries and so on.
Edit
There is number of ways to do your task much better. Just a couple of points:
You don't have to re-create database every time. Even embedded databases (H2, HSQLDB, Derby) have server mode in which they last longer than JVM.
If Hibernate initialization bothers you, then do it once across the all tests (I mean keeping EntityManagerFactory between unit tests).
If you want to avoid a database creation, just use H2 in server mode and set hibernate to not do any changes (<property name="hibernate.hbm2ddl.auto" value="none" />)
I don't know how to apply unit tests on data access layer. I always wonder if the data access layer should be tested. In my company, we have stable database to store unit test data and test data access layer by running data access objects and check the data they get from the stable database.
In order to pass the unit tests, data in the stable database can not be modified anymore. I think there is a better solution to this. If I am not mistaken, the the mock object cannot perform tests on SQL statements and ResultSet mappings.
What is the best way to unit test the DAO? Is there a better way to do this with TDD?
First, by most definitions, "unit" tests do not depend on external systems like a database. You want to create what are called "functional" or "integration" tests. In practice these types of tests will be implemented in the same way as unit tests, using something like Junit, but you should separate them from unit tests, which should run very fast and not break when your database is down or the data has changed.
Second, try to keep most of your business logic out of DAOs and instead put it into a service POJO layer so that you can test biz logic without involving the database.
Next, the ideal way to set up testing for DAOs is to start with an empty database, and load it with test data (often using the DAOs themselves), and then run your DAO tests against a known, and writable, test dataset. If you're fortunate enough to have a read-only database, the stable database approach you outline will work, but most systems are read/write to the database.
Finally, it is valuable to test DAOs. Often the database queries are some of the most fragile parts of your system, and you don't want to wait until they are deployed to production to find out they are breaking.
Strictly speaking, you're writing a functional test. To do this you're going to need a test database of one sort or another. Let's talk about your options.
HSQL/in-memory DB. Small and fast. Trivial to setup and get rolling, and great performance on unit test sized data. The downside is, unless you are deploying using these environments then you risk having your unit test work but actual code fail. It also means you can't use any SQL constructs which are not support in both HSQL and your production DB. This can be mitigated to an extent by using Hibernate or similar. A good way to go if you only have very simple queries.
Mock out the DB calls entirely. Pointless unless you are doing too much heavy lifting in your DAO.
Use a test instance of your production DB. This will give you the best results in terms of accuracy or results. It will let you test to make sure that all your calls work as expected and lets you use non-portable SQL. You can use something like DBUnit to load database data or just use the DAO under test to do it. I would recommend this if you large and nasty queries. Ones with a lot of edge cases, roll-up views and subtle behavior. The downside is that real DBs will incur performance penalties since they'll be doing real work (transactions, index updates, rollback support).
Some comments/suggestions:
The DAO test is aimed at verifying whether the queries fired and the data retrieved are as per the expectations. There should hardly any business logic to test in DAO.
Since the prime objective is test the database interaction, mocking is not going to make it foolproof, specially the edge cases.
In light of this, the approach that you have right now is fair enough. I am not sure why you feel that its not good. A little elaboration will help.
If you are not convenient on using external database, then you can use Java's inbuilt javaDB for this. Please note that there will be overhead of creating the test data first before you run this test.
For JDBC based project, JDBC connection can be mocked, so that tests can be executed without live RDBMS, with each test case isolated (no data conflict).
It allow to verify, persistence code passes proper queries/parameters (e.g. https://github.com/playframework/playframework/blob/master/framework/src/anorm/src/test/scala/anorm/ParameterSpec.scala) and handle JDBC results (parsing/mapping) as expected.
Framework like jOOQ or my framework Acolyte can be used for: https://github.com/cchantep/acolyte .
At work we are trying to simplify an application that was coded with an overkill use of Spring Remoting. This is how it works today:
(Controllers) Spring MVC -> Spring Remoting -> Hibernate
Everything is deployed in a single machine, Spring Remoting is not needed (never will be needed) and adds complexity to code maintenance. We want it out.
How to ensure everything works after our changes? Today we have 0% code coverage! We thought on creating integration tests to our controllers so when we remove Spring Remoting they should behave exactly the same. We thought on using a mix of Spring Test framework in conjunction with DBUnit to bring up Oracle up to a known state every test cycle.
Has anyone tried a similar solution? What are the drawbacks? Would you suggest any better alternative?
It always depends on the ratio effort / benefit that you are willing to take. You can get an almost 100% code coverage if you are really diligent and thorough. But that might be overkill too, especially when it comes to maintaining those tests. But your idea is good. I've done this a couple of times before with medium applications. This is what you should do:
Be sure that you have a well known test data set in the database at the beginning of every test in the test suite (you mentioned that yourself)
Since you're using Hibernate, you might also try using HSQLDB as a substitute for Oracle. That way, your tests will run a lot faster.
Create lots of independent little test cases covering most of your most valued functionality. You can always allow yourself to have some minor bugs in some remote and unimportant corners of the application.
Make sure those test cases all run before the refactoring.
Make sure you have a reference system that will not be touched by the refactoring, in order to be able to add new test cases, in case you think of something only later
Start refactoring, and while refactoring run all relevant tests that could be broken by the current refactoring step. Run the complete test suite once a night using tools such as jenkins.
That should work. If your application is a web application, then I can only recommend selenium. It has a nice jenkins integration and you can create hundreds of test cases by just clicking through your application in the browser (those clicks are recorded and a Java/Groovy/other language test script is generated).
In our Spring MVC / Hibernate (using v3.4) web app we use an Oracle database for integration testing.
To ensure that our database is in a known state each time the test suites are run, we set the following property in our test suite's persistence.xml:
<property name="hibernate.hbm2ddl.auto" value="create"/>
This ensures that the db schema is created each time our tests are run based on the Hibernate annotations in our classes. To populate our database with a know data set, we added a file named import.sql to our classpath with the appropriate SQL inserts. If you have the above property set, Hibernate will run the statements in import.sql on your database after creating the schema.
I am trying to figure out the best way(s) to test Service and DAO layers. So, a few sub questions...
When testing a service layer, is it best to test against a mock DAO layer or a "live" DAO layer pointed at a testing environment?
How should SQL in the DAO layer be tested when the only test database is in a shared environment (Oracle/DB2)
How do you solve the paradox of any DAO writes/updates need to be tested with DAO reads which is something that also has to be tested?
I am looking for any good documentation, articles, or references in this area along with any tools to help automate the process. I already know about JUint for unit testing and Hudson for CI.
Get Growing Object-Oriented Software, Guided by Tests. It has some great tips about how to test database access.
Personally, I usually break the DAO tests in 2, a unit test with a mocked database to test functionality on the DAO, and an integration test, to test the queries against the DB. If your DAO only has database access code, you won't need a unit test.
One of the suggestions from the book that I took, is that the (integration) test has to commit the changes to the DB. I've learn to do this, after using hibernate and figuring out that the test was marked for rollback and the DB never got the insert statement. If you use triggers or any kind of validation (even FKs) I think this is a must.
Another thing, stay away from dbunit, it's a great framwork to start working, but it becomes hellish when a project becomes something more than tiny. My preference here, is to have a set of Test Data Builder classes to create the data, and insert it in the setup of the test or in the test itself.
And check dbmigrate, it's not for testing, but it will help you to manage scripts to upgrade and downgrade your DB schema.
In the scenario where the DB server is shared, I've creates one schema/user per environment. Since each developer has his own "local" environment, he also owns one schema.
Here are my answers :
Use mock DAOs to test your services. Much easier, mush faster. Use EasyMock or Mockito or any other mock framework to test the service layer.
Give each developer its own database schema to execute his tests. Such schemas are typically empty : the unit tests populate the database with a small test data set before running a test, and empties it once the test is completed. Use DBUnit for this.
If the reads work against a well-defined, static, test data set (which you should unit-test), then you can rely on them to unit-test the writes. But you can also use ad-hoc queries or even DBUnit to test that the writes work as expected. The fact that the tests are not necessarily run in this order doesn't matter. If everything passes, then everything is OK.
I have an application where many "unit" tests use a real connection to an Oracle database during their execution.
As you can imagine, these tests take too much time to be executed, as they need to initialize some Spring contexts, and communicate to the Oracle instance. In addition to that, we have to manage complex mechanisms, such as transactions, in order to avoid database modifications after the test execution (even if we use usefull classes from Spring like AbstractAnnotationAwareTransactionalTests).
So my idea is to progressively replace this Oracle test instance by an in-memory database. I will use hsqldb or maybe better h2.
My question is to know what is the best approach to do that. My main concern is related to the construction of the in-memory database structure and insertion of reference data.
Of course, I can extract the database structure from Oracle, using some tools like SQL Developer or TOAD, and then modifying these scripts to adapt them to the hsqldb or h2 language. But I don't think that's the better approach.
In fact, I already did that on another project using hsqldb, but I have written manually all the scripts to create tables. Fortunately, I had only few tables to create. My main problem during this step was to "translate" the Oracle scripts used to create tables into the hsqldb language.
For example, a table created in Oracle using the following sql command:
CREATE TABLE FOOBAR (
SOME_ID NUMBER,
SOME_DATE DATE, -- Add primary key constraint
SOME_STATUS NUMBER,
SOME_FLAG NUMBER(1) DEFAULT 0 NOT NULL);
needed to be "translated" for hsqldb to:
CREATE TABLE FOOBAR (
SOME_ID NUMERIC,
SOME_DATE TIMESTAMP PRIMARY KEY,
SOME_STATUS NUMERIC,
SOME_FLAG INTEGER DEFAULT 0 NOT NULL);
In my current project, there are too many tables to do that manually...
So my questions:
What are the advices you can give me to achieve that?
Does h2 or hsqldb provide some tools to generate their scripts from an Oracle connection?
Technical information
Java 1.6, Spring 2.5, Oracle 10.g, Maven 2
Edit
Some information regarding my unit tests:
In the application where I used hsqldb, I had the following tests:
- Some "basic" unit tests, which have nothing to do with DB.
- For DAO testing, I used hsqldb to execute database manipulations, such as CRUD.
- Then, on the service layer, I used Mockito to mock my DAO objects, in order to focus on the service test and not the whole applications (i.e. service + dao + DB).
In my current application, we have the worst scenario: The DAO layer tests need an Oracle connection to be run. The services layer does not use (yet) any mock objects to simulate the DAO. So services tests also need an Oracle connection.
I am aware that mocks and in-memory database are two separates points, and I will address them as soon as possible. However, my first step is to try to remove the Oracle connection by an in-memory database, and then I will use my Mockito knowledges to enhance the tests.
Note that I also want to separate unit tests from integration tests. The latter will need an access to the Oracle database, to execute "real" tests, but my main concern (and this is the purpose of this question) is that almost all of my unit tests are not run in isolation today.
Use an in-memory / Java database for testing. This will ensure the tests are closer to the real world than if you try to 'abstract away' the database in your test. Probably such tests are also easier to write and maintain. On the other hand, what you probably do want to 'abstract away' in your tests is the UI, because UI testing is usually hard to automate.
The Oracle syntax you posted works well with the H2 database (I just tested it), so it seems H2 supports the Oracle syntax better than HSQLDB. Disclaimer: I'm one of the authors of H2. If something doesn't work, please post it on the H2 mailing list.
You should anyway have the DDL statements for the database in your version control system. You can use those scripts for testing as well. Possibly you also need to support multiple schema versions - in that case you could write version update scripts (alter table...). With a Java database you can test those as well.
By the way, you don't necessarily need to use the in-memory mode when using H2 or HSQLDB. Both databases are fast even if you persist the data. And they are easy to install (just a jar file) and need much less memory than Oracle.
Latest HSQLDB 2.0.1 supports ORACLE syntax for DUAL, ROWNUM, NEXTVAL and CURRVAL via a syntax compatibility flag, sql.syntax_ora=true. In the same manner, concatenation of a string with a NULL string and restrictions on NULL in UNIQUE constraints are handled with other flags. Most ORACLE functions such as TO_CHAR, TO_DATE, NVL etc. are already built in.
At the moment, to use simple ORACLE types such as NUMBER, you can use a type definition:
CREATE TYPE NUMBER AS NUMERIC
The next snapshot will allow NUMBER(N) and other aspects of ORACLE type compatibility when the flag is set.
Download from http://hsqldb.org/support/
[Update:] The snapshot issued on Oct 4 translates most Oracle specific types to ANSI SQL types. HSQLDB 2.0 also supports the ANSI SQL INTERVAL type and date / timestamp arithmetic the same way as Oracle.
What are your unit tests for?
If they test the proper working of DDLs and stored procedures then you should write the tests "closer" to Oracle: either without Java code or without Spring and other nice web interfaces at all focusing on the db.
If you want to test the application logic implemented in Java and Spring then you may use mock objects/database connection to make your tests independent of the database.
If you want to test the working as a whole (what is against the modular development and testing principle) then you may virtualize your database and test on that instance without having the risk of doing some nasty irreversible modifications.
As long as your tests clean up after themselves (as you already seem to know how to set up), there's nothing wrong with running tests against a real database instance. In fact it's the approach I usually prefer, because you'll be testing something as close to production as possible.
The incompatibilities seem small, but really end up biting back not so long afterwards. In a good case, you may get away with some nasty sql translation / extensive mockery. In bad cases, parts of the system will be just impossible to test, which I think is an unacceptable risk for business-critical systems.