Can I create a junit environment without file system and network access? I want to enforce stricter rules for our test cases.
Ideally, this would be configurable with maven and work for the default test phase.
Based on this answer https://stackoverflow.com/a/309427/116509, I think you could set a SecurityManager on test setup and restore the original on teardown.
However, IMHO, some unit tests should be allowed to touch the file system, if for example the class under test actually creates files as part of its contract. (You can use dependency injection to make sure the files are created in a temp directory). Likewise, a good unit test of a class that uses HTTP should test it against an HTTP endpoint. Otherwise you just end up mocking everything and your test becomes worthless. I suppose the default should be to deny access, and then a developer would need to specifically override the permissions for this kind of test.
The typical way to handle these dependancies on a file system/network access, is to mock them out in a test context. This way, the real code can go through the normal channels, but your tests don't have to rely on a file system or a network.
Look into mocking frameworks to help you do a lot of this. Enabling this kind of testing will also make your code cleaner, too. :)
You can use Ashcroft to prohibit access to file system and other resources from your tests. It uses Java Security manager to restrict access to certain resources.
Another approach would be to use AspectJ and implement several advices prohibiting calling certain APIs or packages.
I'm not sure what you mean by JUnit environment, but you should not need a file system, or network access to run unit tests. On the other hand, if you are testing code that uses network and filesystem APIs, you may have an issue. In that case, you may need to abstract your code into smaller testable chunks. You should not be testing weather the network and filesystem APIs are working in a unit test, these are integration tests. You should only be testing your code in unit tests.
Related
I built a backend server (ready-to-serve) that can load jar files as plugins and use the methods in it to serve different functionalities.
I want to write tests for it but I'm not sure what kind of tests I should write.
You should have a look at the different kinds of tests. Unit tests, integration tests, end-to-end tests.
In case you are writing the code for the imported jars yourself, you could write unit tests for the helper functions and services inside. They should be small and pure (self contained).
I guess, that you have a generalised interface exposed with each jar, that your main application reuses to communicate with the jar.
You could write a general integration test, that imports one of the jars and calls a general method, to show, that the execution succeeded. Something like a health check. Then you could write more tests for other functions and the expected results, though they will probably become more focused on each separate jar.
When it comes to testing plugins having only Unit tests is not enough. You'll need integration tests too, otherwise not possible to test combinations of plugins and/or dependencies between them.
Backing to your github project: from what i see, you basically need to test only plugin API and maybe some shared resources: configuration file, datasources and so on.
I want to test the effects of a library call of my program with a real device. This call starts a service, that sends an HTTP request to a server whose URL that is hard-coded in the resources.
I want to verify that the request is sent correctly. So I set up a local HTTP server, but to be able to use it I have to change/override/mock the resource so it points to http://127.0.0.1 instead.
I want to do "end-to-end" testing; in this case it's important that the service makes an actual network request, although locally.
I've tried to override the value by creating a string resource with the same name in androidTest/res/values/strings.xml, but that resource is only visible in the test package, not in the application package.
Using the Instrumentation class only allows me to obtain the Context reference, but there's no way to replace it (or the return value of getResources()) with a mock or something similar.
How can I change a resource value of an Application under test?
You have a couple choices:
Dependency injection
Stubs/mocks
SharedPreferences
Scripts or gradle tasks
Dependency injection
Use a library like RoboGuice or Dapper. Inject an object that handles making the API requests. Then, in your test setup, you can replace the injection modules with testing versions instead. That way your test code runs instead of the original; that code can pass in different strings (either hard-coded or from the test strings.xml) instead.
DI libraries can be expensive to setup: high learning curve and can be performance problems if not used correctly. Or even can introduce hard to debug problems if the scope/lifetime of the objects isn't configured correctly. If testing is the only reason to use DI, it might not be worth it to you if you're not comfortable with a DI container.
Stubs/mocks
Wrap up your calls in something that implements a custom interface you write. Your main implementation then fills in the host URL and calls the API. Then, in tests, use a combination of stubs or mocks on that interface to replace the code that fills in the host URL part.
This is less of an integration test since the stubs or mocks will be replacing parts of the code. But is simpler than setting up a dependency injection framework.
SharedPreferences
Use the Android SharedPreferences system. Have it default to a certain endpoint (production). But allow the app to be started on the testing device, then some dialog or settings to let you change the host URL. Run the tests again and now they point to a different API URL.
Scripts or gradle tasks
Write some script or gradle task to modify the source before it is compiled in certain scenarios.
This can be fairly complicated and might even be too platform or system-dependent if not done right. Will probably be fairly brittle to changes in the system. Might introduce bugs if the wrong command is run to build the final packaged version and the wrong code goes out to the market.
Personal opinion
Which do I recommend? If you and/or your team is familiar with a DI library like RoboGuice or Dapper, I recommend that option. It is the most formal, type-safe and strict solution. It also maintains more of the integrity of the stack to test the whole solution.
If you're not familiar with a good DI library, stubs/mocks and interface wrappers are a good fall back solution. They partly have to be used in the DI solution anyway, and you can write enough tests around them to cover a good majority of the cases you need to test (and are in control of). It is close enough to the DI solution that I would recommend this to everyone who doesn't use DI in the project already.
The SharedPreferences solution works great for switching between staging and production environments for QA and support. However, I wouldn't recommend it for automated tests since the app will most likely be reinstalled/reset so often during development, it would get annoying resetting that URL that often. Also, first runs of tests would probably fail; headless tests on a CI server would fail, etc. (You could default the URL to the localhost, but then you run the risk of accidentally release that default to production sometime.)
I don't recommend scripts or the hacked-up gradle tasks. Too brittle, less clear to other developers that come behind you, and more complicated then they're worth, IMO.
In addition to Jon Adams's solutions, there's a further one:
Override resource in build type
By default, a library module is built in release mode when it's used by another module. The debug mode is only used for testing (unit tests and instrumented tests). Therefore, using the resource overriding it's possible to change the resource value for the instrumentation tests for that library only, and use the original value in the library's users.
This has some caveats though:
Instrumented/integration tests must stay on the library itself, not on the main application package;
The same resource values have to be shared across all tests (unless using product flavors)
I have a class used for handling a connection to an external system.
The class has some a few public methods, let's say:
close()
configure()
send()
connect()
And a handful of private methods.
The class is intended to hide most of the re-establishing, fail-checking and connection-handling in the internal works.
Now, I get a code coverage error on this, since there is no unit tests for this class, besides the configure-method.
Is there another way of writing unit tests for such classes, except for heavy mocking?
If so, isn't that a good proof the class should be tested at an intergration-test or system-test level rather than unit-test? Do communication-classes belong to unit-tests or system tests?
As long as you cannot instantiate the external system locally in an unit test, this task is a classical integration or system test. So the communication classes, typically do not belong to unit tests.
The coverage will be low for such classes, showing that there is
need to test them otherwise.
Further the coverage report, e.g in SonarCube can (hopefully) be filtered.
One could define exception filters, together with the responsible person (e.g Software Architect.)
It may help to move all such communication classes to its own package.
Or even to its own project or jar file. For that project the coverage would then not be executed.
Sometimes it makes sense to build a dummy external system, to work with.
If the external system is yet under development and when it changes its interfaces daily, you can lose much time in using the external system.
In such situations a dummy system can be used, which is maybe instantiable from the unit test, too.
That depends on the amount of code in the class. If it just configures another service (like a socket or a database driver), unit testing doesn't make sense since someone probably already tested the actual service.
If the code is very complex (error handling, data transformation), you should write unit tests for this part of the code and mock the service out.
We are using JUnit to execute integration tests and also the system integration tests which rely on external test systems (not necessarily maintained by our own company).
I wonder where to put the code that checks if the system is available prior to running the test cases? So I can determine if there is a network or other issue and not one with the test itself.
JUnit allows to setup some parts of the test in JUnit-Rules. Is it a good idea to setup the service that communicates with the external system within the rule and do some basic checks ("ping") to the external system within the rule? Or to store the state and within the Test use a JUnit assume(rule.isAvailable) to avoid having the test executed?
Or would it be smarter to put this verification code in a custom JUnit Runner?
Or is there even another way to do this? (simply create some utils?)
The goal is to skip the tests if some conditions are not met since it is obvious the tests will fail. I know this indicates a bad exception handling but there is a lot of legacy code I can't change altogether.
I tried to find some articles myself but it seems the search terms ("test", "external system" and so on) are a little thankless.
thanks!
Consider using org.junit.Assume.* as described here. When assumptions fail, your tests are ignored by default. You can write a custom runner to do something else when your assumptions fail.
The key thing though is that the tests don't fail when assumptions like the availability of your external services fail.
If the ping applies to every single test in the class, I would put the ping call in the #Before method. #Before will get executed before every single test method (i.e., #JUnit -annotated methods).
If the ping does not apply to all tests in the class, then you would have to hit ping explicitly from those methods.
I am using an external library in a java project but I am not sure how can I integration test it with my code.
For example: Let us say I am using a string encryption library to encrypt passwords.
Please enlighten.
Thanks
You are probably thinking of integration testing, not unit testing. Generally I don't unit test code that isn't my own. What I would do for integration testing is basically write tests similar to my unit tests for my code, but not mock out the external library -- i.e., use it directly. You may need to do some set up to create a test environment, including any data you want to use in the test, in which to carry this out. The integration tests may be less extensive than your unit tests since you really only need to test the paths that exercise the external functionality, not necessarily all paths through your code.