I've started using Redis in my project with the help of the Jedis library. All is working fine but now I have a problem that my functional tests requires Redis to be up which I want to avoid in my Continuous Integration. What is the best way to do this?
I've implemented a simple redis embedded runner for Java:
https://github.com/kstyrc/embedded-redis
Currently, it uses redis 2.6.14 for*nix and https://github.com/MSOpenTech/redis for Windows. However you can utilize RedisServer class to run your own run script.
I plan to extend the implementation to support RedisConf (bind, slaveof, port, dbfilename, etc). After this, I'll upload jar to clojars for mvn deps.
Here are few options you have for functional/integration testing:
Just start an instance of redis on you CI server. All tests will be responsible to do proper clean up after execution.
Try somehow to control redis process, i.e. have some shell script or job on CI server to start/stop it before/after running tests. At least some of the burden of setup/cleanup is removed from tests, as for each independent build you will have independent redis setup.
Control redis further by using some in-memory solution like the one you mention for cassandra(if it exists).
One thing to mention is that integration tests should not replace unit tests. Unit tests should probably be preferred and they can cover more cases whereas integration tests can be used just to check that all parts of application play nicely together. And i think this is the reason why a lot of people choose to go for option number one.
Here is a similar question about mongodb The answer has a link to the project which works for second option(controls mongodb process) If you follow some related links on the project page there's also something called nosql-unit. This one i think tries to cover option three. I didn't use it but looks like it has something for redis too.
You can start Redis server on an arbitrary port via the command line: redis-server --port 7777. So for the purposes of integration testing, you could start on Redis on an available (or random) port, making sure that Jedis is configured to use that port.
In this way, you've got a "fresh" instance of Redis that you know won't conflict with any other processes, including other test runs occurring at the same time. This is as close as I can think of to the analogue of running an in-memory/embedded database for integration testing.
For pre-loading Redis with "canned data," use the --dbfilename <file> switch: redis-server --port 7777 --dbfilename test.rdb.
try nosql-unit. It supports redis unit test with java.
I have tried EmbeddedRedis and found that many Jedis interfaces are not supported. Hence using EmbbededRedis is not a good idea, especially when you are using some advanced redis function like "pipeline".
I suggest using ManagedRedis for unit test:
download a redis source code from redis.io into your test resource
build a redis-server in the $(your-redis-dir)/src
write a unit test with ManagedRedis, here is an example. Note that "REDIS_HOME" is the dir where your redis code downloaded, ManagedRedis will find redis-server in ${REDIS_HOME}/src
run you unit test.
As #ksytrc mentioned in his answer I basically used his solution. It was working in this project.You just need to add embedded-redis dependency.
<dependency>
<groupId>com.github.kstyrc</groupId>
<artifactId>embedded-redis</artifactId>
<version>0.6</version>
<scope>test</scope>
</dependency>
then in test class define the redisServer
RedisServer redisServer;
#Before
public void setUp() throws IOException {
redisServer = new RedisServer();
redisServer.start();
}
Also define application.yml with below credentials.
spring:
redis:
host: localhost
port: 6379
The better way that I could handle this same problem was create a Spring service that handle the RedisTemplate. After that, I just use #MockBean to mock the service and abstract the lack of a Redis instance running during my tests.
Per example:
#Service
class RedisService {
#Autowired
private RedisTemplate<String, SomeClass> redisTemplate;
SomeClass get(String key) {
return redisTemplate.opsForValue().get(key);
}
}
And in the integration test:
class IntegrationTest {
#MockBean
private RedisService redisService;
#Before
public void setup() {
SomeClass someClass= new SomeClass();
when(redisService.get(anyString())).thenReturn(someClass);
}
}
I'm skeptical to use some redis memory database solution, because I understand that the actual alternatives is not "officially" recommended by the Spring team.
Related
I want to toggle the webEnvironment config inside SpringBootTest to support running the tests in a pipeline (where the tests needs to be able to boot the server themselves) and locally (where I want to use a standalone server for faster tests.
Figured having a Profile which I could set would be a good solution to it but SpringBootTest seems to flat out ignore whatever profile I attach at the same level, is it simply too early in Spring's lifecycle for it to pick up profiles? Is there a better way to do this?
#Profile("myProfile")
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT) // Starts regardless of what #Profile is
public class MyClass{
...
}
E: Related question:
SpringBootTest enable/disable webEnvironment based on input
E2: After some discussion down below I'm ditching the remote mode, not worth the hassle.
I was wondering why do you even need to rely on spring here? JUnit already has #EnabledIf annotation (documentation), so you can use it to not even attempt to run the test.
In my understanding it's even better, because this will work for all the tests that might even not run spring (unit tests for example). Also it should be better from the performance endpoint (you don't try to even run the Application Context, find/register beans, etc.)
I have some integration tests that create/delete entries in my database. The problem is if I run the tests with
spring.profiles.active=prod
my production database gets deleted (as the tests clear the database). Is there any way to prevent tests from running on this specific spring profile?
I have seen this thread: How to prevent running tests when specific spring profile is active? but there was no useful answer.
Thank you
There could be multiple solution to your problem.
Using in-memory databases like H2 for Sql and flapdoodle for no-sql for running tests. **preferred way
Create a separate properties file with clone of spring properties. Just change the database properties/spring profile or other things. Use this properties file with #testpropertysource on test class.
Use #dirtiescontext on tests to create/delete impacted rows only.
Another thing you can do is create stud classes your database layer to mock operations.
I was able to resolve the problem following this comment: https://stackoverflow.com/a/32892291/8679100. The solution is not perfect, as I need to verify if the prod profile is active in #BeforeAll and #AfterAll, but it does work. Furthermore, System.getProperty("spring.profiles.active", "");didn't actually work, butArrays.stream(environment.getActiveProfiles()).anyMatch(env -> (env.equalsIgnoreCase("prod")))``` did
You can use #IfProfileValue annotation to disable the test. it's not directly depend on the spring profile, but you can easily use config file to set the value you want based on the spring profile.
Having said that - it sounds very risky to run tests that delete entries from the db (or any other db transaction) on production DB. I support what #Sachin suggested above - run your test on tests environment, not production
There are several web spring boot java applications. I need to prepare several components for integration testing. My task is to mock all external behaviour such as other projects's components, db calls etc. I found a solution for this using #Profileannotation from spring framework. Here's an example. I can simply create new profile and declare two beans implementations for each profile: one for real usage, for production and another one for integration testing, for stubbing. It would look like this:
#Profile("PROD")
#Configuration
#EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {
}
#Profile("MOCK")
#Configuration
#EnableWebSecurity
public class SecurityMockConfig extends WebSecurityConfigurerAdapter {
}
But I have doubts about this design. It looks little bit messy for me. Does this solution considered acceptable for task I have?
Doing this, your mocks and their configuration will be probably packaged with the app running in production.
This seems very odd to me. Would you package your units tests in your deliverd Spring application ? I don't think so. So I would like to say this is a "bad" design since testing dependencies should not be embedded with production code.
However, Spring's documentation about #Profile annotation is using the exemple of environment segregation.
Now, there is a question which needs to be answered: what do you mean by "integration testing" ?
Is this automated integration test ? Or do you want to run your application in different modes for the testing teams ?
Is this is an automated integration test, then there is no reason to use #Profile annotation as automated tests and production code will not be packaged together.
However, if you want your users to make integration tests, then you could create standalone fake project which will be used to simulate the external dependencies you are calling (database, webservices, etc).
Then, #Profile can be used to switch from fake to production mode but only through configuration file: fake profile will make call on your fake external services whereas production will call the real external services.
I am trying to implement integration tests for my Tomcat application, but my issue is that the application is launched separately from the tests so the tests cannot access the application context and neither the database.
My idea is running the tests "within" the running application, so I can #Autowire EntityManager and check for instance the state of the database during testing or even create database entities for testing.
My only idea of doing this is to actually run the application programmatically from the tests as ClassPathXmlApplicationContext("applicationContext.xml") and the access the Context. This would work, but it would be very hard for debugging as we wouldn't be able to use Hotswapping during the testing. Also I guess the server would be stopped as soon as the tests would end. I guess that is not the best and correct solution.
EDIT:
My question was probably unclear, so I will try to clarify.
I have a Tomcat application with Spring and Hibernate. The Spring beans and Hibernate database connection is initialised when the Tomcat application is started. The issue is how to run the tests of the active Spring beans from methods annotated with #Test in src/test/java which are started separately.
Consider this class:
#Component
class MyRepository {
#Autowired
EntityManager em;
#Transactional
public void myMethod(MyEntity entity) {
// do some job with entity
...
em.flush();
}
}
This class will be initialised with Tomcat as a MyRepository bean.
To test it, I cannot just call new MyRepository().myMethod(...) - I need to access the bean. The issue is accessing the bean from the #Test method:
#Test
void testMyRepository() {
Item item = ...
// then use the repository to handle the entity
context.getBean(MyRepository.class).myMethod(item);
// then assert the state of the database
context.getBean(EntityManager.class).find(Item.class, ...) ...
}
I can probably get the context in the initialisation of the tests with
ApplicationContext context = ClassPathXmlApplicationContext("applicationContext.xml");
But it would mean launching the whole application each time the tests are started. The better solution would be if the application could run separately from the tests.
Hope my problem is more clear now.
I would suggest you to use the SpringRunner to start the Spring application context and perform your tests on that running instance. You can customize the context the way it doesn't contain parts you don't want to tests and you can create mocks for components that require some external resources (REST clients and such). Take a look at the Spring docs or Spring Boot docs.
If multiple tests use the same Spring context configuration, the context is started just once and reused. So it's good to have it's configuration in a parent class of your tests. You can autowire any Spring bean into your test and test it.
You can use an in-memory database (such as H2) instead of a production one, so your tests are not dependent on an external infrastructure. To initialize the database, use tools like Flyway or Liquibase. To clear the database before each test, you can use the #Sql annotation.
You can find many examples of projects with such tests, for example my own demo.
If you want to test an external system, I would suggest something like JMeter.
Unfortunately you cant mirror your classes and use them in your tests. Thats a big disadvantage of web services. They always depend on user / machine interaction. With a lot of effort you can extract the functionality of the essential classes or methods and construct test scenarios etc. with jUnit.
The Overview of your possibilities:
special drivers and placeholders
you can use a logger with detailed log-level and file output. Then you created scenarios with the expected result and compare it with your log files.
Capture replay tools. They record your exection and replay them for monitoring.
I can also recommend using Selenium for the frontend tests.
Hope it helped.
I'm using arquillian for functional tests only. Arquillian is not managing a container (standalone) and is not deploying an app (also done manually. Since there's no deployment I can't obtain deploymentUrl using #ArquillianResource.
Also, it's a maven project and I have a property with server hostname which is pretty much what I need to get in arquillian test.
My question is: what would be another option to acquire a url except for hard coding it?
If the tests are run in client JVM, you can probably use system properties. For example, with maven it could be:
$ mvn test -Ddeployment.url=http://whatever
And in test code:
String url = System.getProperty("deployment.url", "http://defaulturl");