I've read the following posts:
Is there a way to run MySQL in-memory for JUnit test cases?
Stubbing / mocking a database in .Net
SQL server stub for java
They seem to address unit/component level testing (or have no answers), but I'm doing system testing of an application which has few test hooks. I have a RESTful web service backed by a database with JPA. I'm using NUnit to run tests against the API, but those tests often need complex data setup and teardown. To reduce the cost of doing this within a test via API calls, I would like to create (ideally in memory) databases which can be connected to via a DB provider using a connection string. The idea would be to have a test resource management service which builds databases of specific types, allowing a test to re-point the SUT to a new database with the expected data when it starts - one which can simply be dropped on teardown.
Is there a way, using Oracle or MSSQL, to create a database in memory (could be something as simple as a C# DataSet) which the web server can talk to as if it were a production database? Quick/cheap creation and disposal would be as good as in memory, to be honest.
I feel like this is a question that should have an answer already, but can't find it/ don't understand enough to know that I've found it.
Related
I have a simple REST app with MySQL database, everything works fine, but while testing do we need to create a dummy object and test on it, or test via Mock database?
The dummy object has quite large constructor and nested classes, which is a lot of work.
IMO, there's little point using a mock database, unless you're testing connectivity handling. For example, how does my application behave if the database connection is dropped etc.
For testing SQL, you will do no better than testing against the actual database you're going to use in production. If you use another database as a substitute, i.e. H2, make sure you understand that you are testing a DB driver and database that will be different to your production deployment and this means that you may not catch potential errors in your tests that use this setup.
For testing data handling, you could also use a mock of some kind but again, if you're always going to be better off using the actual database you will be using in production, whenever you can.
If you're using Hibernate as an ORM provider, as part of setting up your integration tests, you can have it execute DML scripts to load your data for testing purposes.
If you using spring boot, then H2 is one of the popular in memory databases. Spring Boot has very good integration for H2.
For integration tests, you should consider using a database in-memory, such as H2.
H2 supports compatibility modes for IBM DB2, Apache Derby, HSQLDB, Microsoft SQL Server, MySQL, Oracle and PostgreSQL. To use the MySQL mode, use the database URL as shown below (and refer to the documentation for further details):
jdbc:h2:~/test;MODE=MySQL;DATABASE_TO_LOWER=TRUE
After a lot of reading in internet, i found out that it seems to be a good practice to use a in-memory DB like H2 to unit test the DAO layer. The idea behind is to avoid working with the production DB.
Fine, so I set up a H2 DB and activated the H2 PostgreSQL Compatibility Mode, since my production DB is on Postgres. The problem I'm now facing: when I run on H2 the original SQL queries in order to build the test DB, this query is not accepted as valid by H2:
ALTER SEQUENCE MYERP.ecriture_comptable_id_seq OWNED BY MYERP.ecriture_comptable.id;
I guess that using the PostgreSQL Compatibility Mode is not a guarantee that all the Postgres specific syntax will be accepted by H2. And probably this would also happen with other DB such as MySQL.
So what's the point in using a h2 database in this case?
Did I miss something?
Unit testing should focus on testing a unit. PostgreSQL is external to your application and thus trivially not part of any DAO that uses it: no 'unit test' should be written that uses any such external database.
Using the database for integration tests may be fine but as you already noticed multiple problems may arise when you use a different database to execute your integration tests (I know very little applications that only use ANSI-SQL without any triggers or other vendor specific SQL extensions like UPSERT in PostgreSQLs case or the SQL in your post).
So what if your production base is in-memory too? That's quite difficult and I would argue an integration test would be fine, just don't call it unit test. All of this just highlights the pain points of doing business logic in your database (e.g. through triggers that you now can't test) or using vendor specific sql which has several advantages that don't play so well with changing the underlying database vendor.
While other interfaces are relatively easy to mock in my Java integration tests, I couldn't find a proper way of mocking Bigquery.
One possibility is to mock the layer I wrote on top of Bigquery itself, but I prefer mocking Bigquery in a more natural way. I'm looking for a limited, lightweight implementation, which allows defining the table contents, and supports queries using the standard API.
Is there such a library? If not, what alternative approaches are recommended?
In unit testing it is perfectly fine to mock all external dependencies, and as long you are using interfaces to abstract out access to BigQuery client, mocking should not be an issue.
With integration testing I would rather get all my 3rd parties dependencies tested
to the extend an application needs it.
For instance one case would be an ETL which streams data from external sources to BigQuery, in this case an integration test needs to verify that all data is in BigQuery as expected, which means that verification stage needs to take into account repeated, and nested messages as required.
Another case would an application that runs some business SQLs, in this case you would have populate BigQuery with some test data before applicaiton run,
then the applicaiton needs to publishe the SQL output either as view/new table/or stream out of data out of for verification.
There are already some libraries taking care of integration testing with datastores including BigQuery/NoSQL/SQL
They would provide an easy solution for the cases described above and full support for SQL, dynamic macro/predicate etc ....
Dsunit (go-lang)
JDsunit (java)
Endly(language agnostic)
See more how to use endly for ETL and BiqQuery testing
If datastore integration test library is not an option for you and you are looking for just testing BigQuery client, the good news is that the client uses REST, so using network sniffers you can easy record what is being send back and forth, then you can use it in replayer. In order to redirect BigQuery from
public BG endpoints to your replayer you would use http java proxy.
I am trying to find a way to unit test my MySQL dependant code. I know how I would like it to work but cannot find the solution that would work for me. I have looked into DBUnit, but it would seem (if I am not mistaken) that this would require a running database and just aids with the unit testing side of things. I would like some way to avoid running a mysql database when testing. What would work great would be some sort of MySQL spoof driver that actually stored data in memory, rather than needing to access a real persistent database.
In my code it is hard coded to access a MySQL database so I can't just inject some mock object. The way I would like it to work is that when my code calls:
DriverManager.getConnection("jdbc:mysql://" + host + ":" + port + "/" + database, username, password);
It actually gets some other local database that can either be configured via maven or in the setUp of the maven test. I have looked into memory based databases such as HSQLDB but can't find a way for it to spoof the MySQL driver.
Are there any tools that provide what I am looking for? Do you have any good methods for testing MySQL dependant code?
I have had several projects in which I had to do integrations test against a running MySql server. Instead of spending time setting it up every time, I developed a library that sets up a local running instance of MySQL every time you run your tests.
With that you get a test database that acts like the real thing (because it is) without having to set it up.
DBUnit is also a good alternative if you want to mock the database integration (as far as I know, there is no need for a real MySql server when using DBUnit).
I want to allow consumers of a web services layer (web services are written in Java) to create automated integration tests to validate that the version of the web services layer that the consumers will use will still work for the consumer (i.e. the web services are on a different release lifecycle than the consumers and their APIs or behavior might change-- they shouldn't change wihtout notifying the consumer, but the point of this automated test is to validate that they haven't changed)
What would I do if the web service actually executes a transaction (updates database tables). Is there a common practice for how to handle this without having to put logic into the web service itself to know its in a unit test and rollback the transaction once finished? (basically baking in the capability to deal with testing of the web service). Or is that the recommended way to do it?
The consumers are created by one development team at our company and the web services are created by a seperate team. The tests would run in an integration environment (the integration environment is one environment behind the test environment used by QA functional testers, one environment behind the prod environment)
The best approach to this sort of thing is dependency injection.
Put your database handling code in a service or services that are injected into the webservice, and create mock versions that are used in your testing environment and do not actually update a database, or in which you add capability of reset under test control.
This way your tests can exercise the real webservices but not a real database, and the tests can be more easily made repeatable.
Dependency injection in Java can be done using (among others) Spring or Guice. Guice is a bit more lightweight.
It may be sensible in your situation to have the injection decision made during application startup based on a system property as you note.
If some tests need to actually update a database to be valid, then your testing version of the database handling will need to use a real database, but should also provide a way accessible from tests to reset your database to a known (possibly empty) state so that tests can be repeated.
My choice would be to host the web services layer in production and in pre-production. Your customers can test against pre-production, but won't get billed for their transactions.
Obviously, this requires you to update production and pre-production at the same time.
Let the web services run unchanged and update whatever they need to in the database.
Your integration tests should check that the correct database records have been written/updated after each test step.
We use a soapUI testbed to accomplish this.
You can write your post-test assertion scripts in Groovy and Java, which can easily connect to the db using JDBC and check records.
People get worried about using the actual database - I wouldn't get hung up on this, it's actually a GOOD thing and makes for a really accurate testbed.
Regarding the "state" of the db, you can approach this in a number of ways:
Restore a db in a known state before the tests run
Get the test to cleanup after themselves
Let the db fill up as more tests run and clean it out occassionally
We've taken the last approach for now but may change in future if it becomes problematic.
I actually don't mind filling the db up with records as it makes it even more like a real customer database. It also is really useful when investigating test failures.
e.g. cxf allows you to change the transport layer. so you can just change the configuration and use localTransport. then you can have to objects: client and server without any network activity. it's great for testing (un)marhasling. everything else should be separated so the business logic is not aware of webservices so it can be tested as any other class