TLDR: Should you test a compiled JAR or is it enough to write a #Test method to run the main class?
I have a Spring Boot service that integrates with several other services.
Currently, in order to test the integration between the services I'm running the compiled JAR in sort of a staging environment while simulating the other services.
Is there a benefit of running the compiled JAR over running a unit test that runs the main application and mocks (e.g. using Mockito) the other services?
Or maybe, do I need both of these capabilities?
It is likely that you'll want both. This is a great guide on testing strategies for microservices
https://martinfowler.com/articles/microservice-testing/
The unit tests that test only a specific piece of your application with other parts of the internal application mocked. These can run as part of your CI/CD pipelines.
You'll also want to run integration tests in your codebase using mocks for external systems. Those can run as part of your pipelines and use the test annotation you mentioned. These will ensure that your application functions correctly as long as the contracts to the external systems are correct.
Then you may want to do some runtime smoke testing as well against a running application using external tools that invoke your APIs.
Related
Overview: I have a frontend service running in a Kubernetes pod that I want to run integration tests on to assure that the backend services do what is expected. The service uses junit tests that I have repurposed to be integration tests. And that runs fine in a development environment, but we need them to run in a set of (canary) test pods that mimic our eventual SaaS solution so that we can make them part of our CI/CD pipeline that needs to run on each merge request. That way people won't be able to check in code that "breaks the build".
So, what I need to find is how to have the minimal stuff to add to our front end (canary) test pod to allow it to run these junit tests.
I found junit5 via apt-get and in it is junit-platform-console-standalone-1.3.2.jar. That is the start of what I need, but the tests reference org.junit.jupiter.api.Test (and Disabled) and other files in our test classes (e.g. FooTestBaseClass.java) and our tests in the development world are in /foo/src/test/java/com/bar/foo/test with resources in /foo/src/test/resources.
So, I need to know where to "put" the org.junit.jupiter.api directory as well as what to put in my classpath to find the local java classes.
The stuff works by "magic" incantations in the development environment, e.g. I just type "gradle test", but I don't want to replicate that entire environment in the Kubernetes pod. I want to understand the "sufficiently advanced technology" so that it is no longer magic.
Note, I can't run the integration tests in the CICD pipeline outside the pod because the backend code is not in Java (it's mostly in Rust) and there is a hardware layer which is currently simulated in C++ and all the communication is done via GraphQL in yaml files. And the point of these tests are to make certain that the different implementations of the yaml (and the contents in the GraphQL) are speaking compatible dialects. Thus, I have to spin up 3 different servers.
Partial success. I found the 1.9.0 version (as a jar file) in the Maven Central repository. (It took a bit of looking to find that and how to get it as a jar file rather than just a Maven link in xml.) That satisfies the junit.jupiter packages.
I built a backend server (ready-to-serve) that can load jar files as plugins and use the methods in it to serve different functionalities.
I want to write tests for it but I'm not sure what kind of tests I should write.
You should have a look at the different kinds of tests. Unit tests, integration tests, end-to-end tests.
In case you are writing the code for the imported jars yourself, you could write unit tests for the helper functions and services inside. They should be small and pure (self contained).
I guess, that you have a generalised interface exposed with each jar, that your main application reuses to communicate with the jar.
You could write a general integration test, that imports one of the jars and calls a general method, to show, that the execution succeeded. Something like a health check. Then you could write more tests for other functions and the expected results, though they will probably become more focused on each separate jar.
When it comes to testing plugins having only Unit tests is not enough. You'll need integration tests too, otherwise not possible to test combinations of plugins and/or dependencies between them.
Backing to your github project: from what i see, you basically need to test only plugin API and maybe some shared resources: configuration file, datasources and so on.
Often some testing framework for automated testing - like Selenium - is used to continiuosly verify the integrity of a deployed application. These tests often cover real user scenarios and may also utilize a range of deployed applications in combination.
We would like to achieve some what the same for a "backend only" application - that is, an application (or more really) without frontend. We are currently building a series of batchjobs where one job produces input to the next.
We have a great unit-test suite that tests the individual jobs however we would really like to test the series of jobs when deployed to some environment.
Do you have any suggestions for such testing framework? The framework must be able to leverage other Java SDKs such as AWS SDK (e.g. to instruct startup of batchjob, inject data to queues etc.). Whether the framework with tests needs to be deployed as an application as well or run directly from CI is secondary.
If you already have backedn tests that can be run on the production all you need is to schedule running on those tests. Jenkins is fine for that (https://wiki.jenkins.io/display/JENKINS/Schedule+Build+Plugin)
You could have emails (or other alerts) for failed jobs. Jenkins will also care for test reports- exactly as it does for unit tests.
Technologies for scheduling test runs
You could schedule running your tests using any other technology- for example Amazon AWS instances, AWS Elastic Beanstalk Worker Environments (https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html) etc.
I find Jenkins most reasonable, because you have out-of-the box support for test reports, notifications, etc.
For any other technologies you would have to write reportting, notifications on your own.
Technologies for writing tests
I could write tests in any technology that is capable of making HTTP REST calls. For performance tests Jmeter or Gatling are good choices.
For acceptance test you could use RestEasy, TestRestTemplate from Spring, Apache HTTP client, etc.
As test running framework you could use Junit4,Juni5, TestNG or Spock (if you are fine with Groovy language). Test structure could be similar to those of ordinary tests. Well named independend methods that test one thing well, meaningfull assertions, etc.
For writing assertions my personal preference is AssertJ, but JavaHamcrest would also do.
Those tests can (and should) be written in src/test directory, in separate repository (or in same repository or different module).
For that test module you may write test-releated services in src/main directory, so src/test directory would contain only test scenarios. Test services may everyting you need- manage files, inspect database, etc.
You may consider writting test scenarios in BDD-style and tools like JBehave or Cucumber. Personally I see value in BDD tests only if business is interested in test scenarios. If those tests are to be used only by technical people then I find easier to maintain such tests in non-BDD technologies (Junit, AssertJ).
I am working on a Spring MVC application.
Unit tests are already writen (nice 85% code coverage and lots of assertions :)
Now I need to write integration tests. I already have a look at stackoverflow still I do have some questions
Right now I am using a standard maven structure with main and test directories, do you recommend to create another directory called integration-tests and write the unit tests there? If so, why?
Another alternative would be to write the integration tests in the "tests" directory, mixing unit and integration tests, and then using maven to run one or the other (maybe using different file suffixes depending on the test type)
In any case, the way I am planning to run the integration tests is essentially to (almost) reuse the unit tests of the controllers WITHOUT injecting mocks, that means that my current stack (Controller-Service-DAO) will contain no mocks (of course, in the unit tests, it does), by removing the mocks I will access to the real resources (Database and so), is this a good approach?
UPDATE: Just to clarify, the project has not JSP or any HTML-related views, output is XML which can be easily validated with XSDs
I am assuming that you have used JUNIT to create your unit tests to achieve the impressive 85% code coverage. Please notice that JUNIT is designed for unit testing only (thus the name JUNIT). Unit testing is done while the code is running in the development environment.
Integration testing can only be performed once the target code has been deployed in the target integration environment.
You have mentioned that you application isn't a web application. Is it SOAP/Rest Web Service? If so, you can use Soap UI [http://www.soapui.org/], to create and save automated regression/integration tests.
I have writting some JUnit integration tests that currently run from Maven (via the command line or out CI server). These integration tests automatically configure and startup the database and servlet container inside of Maven's 'pre-integration-test' lifecycle phase.
I would very much like to run these JUnit tests (like I do all our other tests) from within an Eclipse JUnit launch configuration. However, the JUnit launch configuration does not trigger the pre-integration-test executions and thus the environment to test is not properly established.
Whats the best way to get this to work?
You can setup Maven Run Configurations to run within Eclipse, using Run -> Run Configurations, assuming you have the Maven plugin for Eclipse. Using the goals clean verify will run unit tests and integration tests.
Also if you use the failsafe reports Maven plugin, you can create JUnit style reports, that you can open in Eclipse using the JUnit framework, to see which tests passed and failed.
One way to do this I just discovered is to use Eclipse Launch Groups as shown here:
http://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.cdt.debug.application.doc%2Freference%2Fcdt_u_run_dbg_launch_group.htm
I missed this because this capability, while language independent needs to be installed from the C/C++ side of Eclipse as shown here:
What installable component provides 'launch groups' in Eclipse?
Within the launch group the first job should run a maven task to process test resources, the next jobs launch the server(s), and then finally a job to execute the junit tests.
I don't know what database you want to use, but for unit tests its handy to use an in memory database and maybe also an embedded servlet container. Maybe you could use something like http://arquillian.org/
JUnit framework should be mainly used for Unit testing and not Integration testing.
Integration testing could be performed by QA team.
If you have lot of Integration plumbing code then you should considering Mocking of external elements, such as Databases and web services calls.
Concentrate mainly on testing code that you have written.Add negative, positive and exceptional test scenarios. That will give your code more credibility than full integration test.
For Database I suggest using DBUNIT > http://www.dbunit.org/ which is an excellent tool for testing database plumbing. This tool can create XML schema from your database and you can keep data as xml replicating the database.
This will avoid changing any dataset for your JUnit testing.