How do I write integration test for an application that interacts with website ?
More specifically I have an application that interacts with Flickr website.During the OAuth authorization process flickr website display's the verifier code which the user has to copy and paste into my application. Now how do I automate this process so that I can test the application automatically.I am using swing for GUI.
Writing automation that depends on external services can be tricky. For something like this, I would advise you to set up a mock service, or some other way of using canned responses.
I've had success doing this a couple of ways:
Writing an external mock service, using something like bottle.py. This has the advantage of requiring little to no modification to your existing codebase, but obviously requires a bit of work to ensure that this external process is managed correctly as part of your test suite, especially if you are running tests in a CI environment.
Using dependency injection, you can write mock network components, and swap the real network components for your mock components for testing. I recommend this approach, but it will require a bit of modification to your codebase.
Related
I have a rest controller for creating and getting employees. To test these endpoints I have two classes. In one I mock the service layer and do status code checks such as:
Mockito.when(employeeService.createEmployee(any(Employee.class))).then(returnsFirstArg());
this.employeeClient.createEmployee(this.testEmployee)
.andExpect(status().isCreated());
In the other I start up a docker postgres database and don't mock anything. These tests involve multiple endpoints e.g. "If I create an employee and then get all the employees, the created employee should be returned".
Both these classes create an application context - the first uses the WebMvcTest annotation, the second uses SpringBootTest. For this reason I see them both as integration tests.
So then what do I call these tests? I'm currently using EmployeeControllerTest and EmployeeContorllerIT (running IT with the failsafe plugin), but the name for the first is a little misleading because it's not really a unit test.
I was thinking of using EmployeControllerMockedIT for the first, but not convinced that's the right way to go.
If you're mocking responses, I'd say its a unit test as you're testing the module of code in isolation from a network and 3rd party systems like a database.
If you're running a test against (real) 3rd party code/systems you don't control, its an integration (or functional) test.
Functional tests will invoke your service as if your system is a black-box where we don't know the internal details (vs. white-box where we know, and have access to change, the internals and invoke code as if we're making a method call rather than sending an HTTP request).
Functional tests may run against real/fake servers but there will typically be a running application waiting for requests. The point being here the responses are not "mocked", the request is made for real, from your code, to the third party code/system, and that system is expected to respond as the real server would.
We do not fake responding in functional tests but arbitrary responses may be provided from the real/fake server to simulate the data we expect to receive. It could be we write a fake HTTP service to "mock" a real API. It will provide the exact same type of responses over the network, and be invoked over the network. Or we could just spin up a dockerised database and send queries to it and receive responses from it. The database will be a real database-server, rather than a mocked response we pre-specified in code.
Functional tests (of a web service) will begin from outside your application and ensure requests are answered over the network layer. i.e. you'll be making an HTTP request to invoke the service.
Integration tests of a web service will invoke the service at an application level (not over the network layer) so you may call a Controller class directly or invoke some code that is supposed to connect/integrate with a database from within the application. Integration tests will test your code is integrated correctly with third party code/a library/a database. Your tests should alert you to any situation where if the 3rd party code (code/system you do not control) changes in such a way that your software would break.
I want to add WebAuthN as an option for multi factor authentication to an Angular & Spring application. I use the WebAuthN java-webauthn-server library from Yubico.
What is the best way to integration test my WebAuthN server, without a hardware client? Is there any software that can handle the cryptography in an automated test? I want to run these tests automatically in the CI/CD pipeline (GitLab).
In a best case scenario I want to be able to test the whole process, creating credentials as well as logging in. An alternative scenario could be that I use known credentials in the backend and only log in with these.
My API is REST/JSON based, with relying party, user, challenge, pubKey etc...
My integration tests are Java based (spring boot starter test)
I am mainly interested in how to integration test the server without the client side. Are there utility programs or libraries that can handle authenticators and return the correct data/json objects?
I have looked at Testing WebAuthn via REST tool, however, I am not interested in testing the specification, since I am using a library, I only want to ensure that I applied the library correctly to my code.
If you are only interested in testing the server side, you can write a simple webpage with buttons that exercise your endpoints and call navigator.credentials.(create|get). You can then instrument a browser using Selenium 4+, set up Virtual Authenticators, and run tests against that webpage. Take a look at the selenium tests for an example. The code to set up the authenticators looks like this in java:
VirtualAuthenticatorOptions options = new VirtualAuthenticatorOptions();
options.setTransport(Transport.INTERNAL)
.setHasUserVerification(true)
.setIsUserVerified(true);
VirtualAuthenticator authenticator =
((HasVirtualAuthenticator) driver).addVirtualAuthenticator(options);
Pay attention to setting up the authenticator with the right settings to match your webauthn call. You should pick the right user verification support, resident keys support, and internal (i.e. platform) vs usb / nfc / ble (i.e. cross-platform) transport.
If you're using an older version of selenium, you'll have to manually define the commands yourself. The code should look like
browser.driver.getExecutor().defineCommand(
"AddVirtualAuthenticator", "POST", "/session/:sessionId/webauthn/authenticator");
// ...
Command addVirtualAuthCommand = new Command("AddVirtualAuthenticator");
addVirtualAuthCommand.setParameter("protocol", "ctap2");
addVirtualAuthCommand.setParameter("transport", "usb");
browser.driver.getExecutor().execute(addVirtualAuthCommand);
Running selenium tests might take a bit of work if you aren't already using it for integration testing. However, this implementation will very closely match reality. From the browser's perspective, the virtual authenticator is real hardware. The response from the authenticator will be processed by the browser as if it was real.
At the moment, only chromium based browsers support Virtual Authenticators.
I have two spring boot applications. The first one manages data in a PostgreSQL database. The other one exposes this data over REST.
In my first spring boot application I write a test that uses a test database. Now I want to write a test for the other application (REST), that test needs data inside the database.
How can I use the first spring boot application in my test for the second spring boot application?
Or can I setup that the test only can be run if the test from the first spring boot application?
There are different types of testing. The first is unit testing -- this confirms that your business logic works. The second form is integration testing, which is again split into two parts -- the first you test the component in isolation to confirm that it communicates the way you expect (sometimes called component testing), and the second you test the component against other, real, components.
You can easily do unit-tests in maven/spring-boot, and it's fairly easy to do component testing too. The integration testing however is usually a lot more complicated and usually needs to involve a mechanism outside the simple maven build system. The most common approach to this is to use a CI/CD tool, like Jenkins or CircleCI.
The usual pattern is to run the unit-tests first because they are the fastest, then component tests, then integration tests. The latter often requires an 'environment' to be created that contains all of the collaborating components that compose a service (the two spring-boot apps in your case).
For integration testing, we often find that the biggest problem is "Configuration Management", which is basically a description of which versions of which components work together. For your problem you need a database, data, and two spring-boot apps, along with their configuration and environment data.
First of all: you shouldn't start neither the first nor the second application in your tests. It will slower down your testing drastically. What's more you will be dependent on another application that in reality may be developed by another team - bad idea.
In fact, you've got something like 3 ways to do it:
Use Wiremock or some other dummy stub service- this approach will suite you if you're ready no to call the "real" application. The service should mimic your application (should expose the same interface i.e. url, HTTP method, the same response)
Use both the application in Docker containers and start it with Docker Compose - you can use here either a real database deployed somewhere or another container with predefined data
Deploy the first service somewhere along with the database and run either integration tests or system or end-to-end tests
Hope this helps!
UPDATE: I meant you shouldn't start apps in tests. Also, to my mind it is obvious enough that we're talking about integrating testing at least as the guy mentions testing of whole request trip starting with the REST app. Maybe that clarifies the things to downvoters
I have a service that calls out to a third-party endpoint using java.net.URLConnection. As part of an integration test that uses this service I would like to use a fake endpoint of my own construction.
I have made a Spring MVC Controller that simulates that behaviour of the endpoint I require. (I know this endpoint works as expected as I included it in my web app's servlet config and hit it from a browser once started).
I am having trouble figuring out how I can get this fake endpoint available for my integration test.
Is there some feature of Spring-Test that would help me here?
Do I somehow need to start up a servlet at the beginning of my test?
Are there any other solutions entirely?
It's a bad idea to use a Spring MVC controller as a fake endpoint. There is no way to simply have the controller available for the integration test and starting a servlet with just that controller alongside whatever you are testing requires a lot of configuration.
It is much better to use a mocking framework like MockServer (http://www.mock-server.com/) to create your fake endpoint. MockServer should be powerful enough to cover even complex responses from the fake endpoint, with relatively little setup.
Check out Spring MVC Test that was added to Spring in version 3.2.
Here are some tutorials: 1, 2, 3
First I think we should get the terminology right. There are two general groups of "fake" objects in testing (simplified): a mock, which returns predefined answers on predefined input and stubs which are a simplified version of the object the SUT (system under test) communicates with. While a mock basically does nothing than to provide a response, a stub might use a live algorithm, but not store it's results in a database or send them to customers via eMail for example. I am no expert in testing, but those two fake objects are rather to be used in unit and depending on their scope in acceptance tests.
So your sut communicates with a remote system during integration test. In my book this is the perfect time to actually test how your software integrates with other systems, so your software should be tested against a test version of the remote system. In case this is not possible (they might not have a test system) you are conceptually in some sort of trouble. You can shape your stub or mock only in a way you expect it to work, very much like the part of the software you have written to communicate with that remote service. This leaves out some important things you want to test with integration tests: Was the client side implemented correctly so that it will work with the live server. Do we have to develop work around as there are implementation errors on the server side? In which scale will the communication with the remote system affect our software's performance? Do our authentication credentials work? Does the authentication mechanism work? What are the technical and conceptual implications of this communication relationship no one has thought of so far? (Believe me, the latter will happen more often than you might expect!)
Generally speaking: What will happen if you do integration tests against a mock or a stub is that you test against your own understanding of how to implement the client and the server side of communication, and you do not test how your client works with the actual remote server or at least the best thing next to that, a test system. I can tell you from experience: never make assumptions on how a remote system should behave - test it. Even when talking of a JMS server: test it!
In case you are working for a company, testing against a provided test system is even more important: if you software works against a test system and you can prove it (selenium is a good helper here, as well as good logging, believe it or not) and your software does not work with a live version, you have a situation which I call "instablame": it is immediately obvious that it is not your fault the software isn't working. I myself hate fingerpointing to the bone, but most suits tend to ask "Who's fault was it?" even before "Can we fix that immediately?" and way before "How can we solve that problem?". And there is a special group of suits called lawyers, you know ... ;)
That being said: if you absolutely have to use those stubs during your integration tests, I would create an own project for them (let's say "MyProject-IT-Stubs" and build and run the latest version of MyProject-IT-Stubs before I run the IT of my main project. When using maven, you could create MyProject-IT-Stubs with war packaging, call it as a dependency during the pre-integration-test phase and fire up a jetty for this war in the same phase. Then your integration tests run, either successful or not and you can tear down the jetty in the post-integration-test phase.
The IMHO best way to organize your project with maven would be to have a project with three modules: MyProject,MyProject-IT-Stubs and MyProject-IT(declaring dependencies on MyProject and MyProject-IT-Stubs. This keeps your projects nice and tidy and the stubs do not pollute your project. You might want to think about organizing MyProject-IT-Stubs into modules as well, one for each remote system you have to talk to. As soon as you have test access, you can simply deactivate the according module in MyProject-IT-Stubs.
I am sure according options exist for InsertYourBuildToolHere.
I want to allow consumers of a web services layer (web services are written in Java) to create automated integration tests to validate that the version of the web services layer that the consumers will use will still work for the consumer (i.e. the web services are on a different release lifecycle than the consumers and their APIs or behavior might change-- they shouldn't change wihtout notifying the consumer, but the point of this automated test is to validate that they haven't changed)
What would I do if the web service actually executes a transaction (updates database tables). Is there a common practice for how to handle this without having to put logic into the web service itself to know its in a unit test and rollback the transaction once finished? (basically baking in the capability to deal with testing of the web service). Or is that the recommended way to do it?
The consumers are created by one development team at our company and the web services are created by a seperate team. The tests would run in an integration environment (the integration environment is one environment behind the test environment used by QA functional testers, one environment behind the prod environment)
The best approach to this sort of thing is dependency injection.
Put your database handling code in a service or services that are injected into the webservice, and create mock versions that are used in your testing environment and do not actually update a database, or in which you add capability of reset under test control.
This way your tests can exercise the real webservices but not a real database, and the tests can be more easily made repeatable.
Dependency injection in Java can be done using (among others) Spring or Guice. Guice is a bit more lightweight.
It may be sensible in your situation to have the injection decision made during application startup based on a system property as you note.
If some tests need to actually update a database to be valid, then your testing version of the database handling will need to use a real database, but should also provide a way accessible from tests to reset your database to a known (possibly empty) state so that tests can be repeated.
My choice would be to host the web services layer in production and in pre-production. Your customers can test against pre-production, but won't get billed for their transactions.
Obviously, this requires you to update production and pre-production at the same time.
Let the web services run unchanged and update whatever they need to in the database.
Your integration tests should check that the correct database records have been written/updated after each test step.
We use a soapUI testbed to accomplish this.
You can write your post-test assertion scripts in Groovy and Java, which can easily connect to the db using JDBC and check records.
People get worried about using the actual database - I wouldn't get hung up on this, it's actually a GOOD thing and makes for a really accurate testbed.
Regarding the "state" of the db, you can approach this in a number of ways:
Restore a db in a known state before the tests run
Get the test to cleanup after themselves
Let the db fill up as more tests run and clean it out occassionally
We've taken the last approach for now but may change in future if it becomes problematic.
I actually don't mind filling the db up with records as it makes it even more like a real customer database. It also is really useful when investigating test failures.
e.g. cxf allows you to change the transport layer. so you can just change the configuration and use localTransport. then you can have to objects: client and server without any network activity. it's great for testing (un)marhasling. everything else should be separated so the business logic is not aware of webservices so it can be tested as any other class