I need to populate some data in a database managed by flyway with data from another database from an external system that we do not manage by flyway.
We use spring and I have a a DataSource bean available for the external system. Is there a way to inject beans into a flyway java migration in order to do this? Or will I have to create a static variable to access my spring ApplicationContext and just get the bean from there manually?
You could create a Java migration for that: http://flywaydb.org/documentation/migration/java.html
Related
Is there any way to Load the database schema from .sql or json or textfile to create the ORM mapping with JPA/Hibernate to database in spring-boot while starting up the server.
Spring Boot enables you to use database migration tools such as Liquibase and Flyway, you can read more about that on Spring's official documentation.
Edit: From the docs
85.5 Use a Higher-level Database Migration Tool
Spring Boot supports two higher-level migration tools: Flyway and Liquibase.
85.5.1 Execute Flyway Database Migrations on Startup
To automatically run Flyway database migrations on startup, add the org.flywaydb:flyway-core to your classpath.
The migrations are scripts in the form V__.sql (with an underscore-separated version, such as ‘1’ or ‘2_1’). By default, they are in a folder called classpath:db/migration, but you can modify that location by setting spring.flyway.locations. This is a comma-separated list of one or more classpath: or filesystem: locations. For example, the following configuration would search for scripts in both the default classpath location and the /opt/migration directory:
spring.flyway.locations=classpath:db/migration,filesystem:/opt/migration
You can also add a special {vendor} placeholder to use vendor-specific scripts. Assume the following:
spring.flyway.locations=classpath:db/migration/{vendor}
Rather than using db/migration, the preceding configuration sets the folder to use according to the type of the database (such as db/migration/mysql for MySQL). The list of supported databases is available in DatabaseDriver.
FlywayProperties provides most of Flyway’s settings and a small set of additional properties that can be used to disable the migrations or switch off the location checking. If you need more control over the configuration, consider registering a FlywayConfigurationCustomizer bean.
Spring Boot calls Flyway.migrate() to perform the database migration. If you would like more control, provide a #Bean that implements FlywayMigrationStrategy.
Flyway supports SQL and Java callbacks. To use SQL-based callbacks, place the callback scripts in the classpath:db/migration folder. To use Java-based callbacks, create one or more beans that implement Callback. Any such beans are automatically registered with Flyway. They can be ordered by using #Order or by implementing Ordered. Beans that implement the deprecated FlywayCallback interface can also be detected, however they cannot be used alongside Callback beans.
By default, Flyway autowires the (#Primary) DataSource in your context and uses that for migrations. If you like to use a different DataSource, you can create one and mark its #Bean as #FlywayDataSource. If you do so and want two data sources, remember to create another one and mark it as #Primary. Alternatively, you can use Flyway’s native DataSource by setting spring.flyway.[url,user,password] in external properties. Setting either spring.flyway.url or spring.flyway.user is sufficient to cause Flyway to use its own DataSource. If any of the three properties has not be set, the value of its equivalent spring.datasource property will be used.
There is a Flyway sample so that you can see how to set things up.
You can also use Flyway to provide data for specific scenarios. For example, you can place test-specific migrations in src/test/resources and they are run only when your application starts for testing. Also, you can use profile-specific configuration to customize spring.flyway.locations so that certain migrations run only when a particular profile is active. For example, in application-dev.properties, you might specify the following setting:
spring.flyway.locations=classpath:/db/migration,classpath:/dev/db/migration
With that setup, migrations in dev/db/migration run only when the dev profile is active.
85.5.2 Execute Liquibase Database Migrations on Startup
To automatically run Liquibase database migrations on startup, add the org.liquibase:liquibase-core to your classpath.
By default, the master change log is read from db/changelog/db.changelog-master.yaml, but you can change the location by setting spring.liquibase.change-log. In addition to YAML, Liquibase also supports JSON, XML, and SQL change log formats.
By default, Liquibase autowires the (#Primary) DataSource in your context and uses that for migrations. If you need to use a different DataSource, you can create one and mark its #Bean as #LiquibaseDataSource. If you do so and you want two data sources, remember to create another one and mark it as #Primary. Alternatively, you can use Liquibase’s native DataSource by setting spring.liquibase.[url,user,password] in external properties. Setting either spring.liquibase.url or spring.liquibase.user is sufficient to cause Liquibase to use its own DataSource. If any of the three properties has not be set, the value of its equivalent spring.datasource property will be used.
See LiquibaseProperties for details about available settings such as contexts, the default schema, and others.
There is a Liquibase sample so that you can see how to set things up.
Spring also supports a database initialization on its own, the official docs are here.
Spring Boot can automatically create the schema (DDL scripts) of your DataSource and initialize it (DML scripts). It loads SQL from the standard root classpath locations: schema.sql and data.sql, respectively.
Hello fellow developers,
i created a library using the Spring boot framework.
This library is creating a dynamic database connection using #Beans where i create a "data" Bean which holds the unlimited Datasource beans provided by a Postgresql db. At the end i wanted to have a dynamic db connection which could be triggered from outside to change the db i want to connect to. The information of the different databases where stored as mentioned inside of a postgres. This is loaded at the application start into this bean. My Problem is, that i'm not able to switch between the different Datasource beans. Spring boot is creating them, but it seems like it's not possible to change the bean started at runtime of the application which only holds one of the unlimited Datasources... So also after a retriggering of the creation of the original bean it still uses the old datasource.
Is there a way to use the beans from spring boot and change them on runtime?
Regards,
Andreas
I believe you are asking for DB multitenancy support where tenants information is stored in a Postgres DB.
Configuring the persistent layer for multi-tenancy support involves configuring:
Hibernate, JPA and Datasources properties
Datasources beans
Entity manager factory bean
Transaction manager bean
Spring Data JPA and annotation-driven transactions
I recently blog about Multi-tenant applications using Spring Boot, JPA, Hibernate and Postgres and although the tenants data is stored in a yml "properties" file, it shouldn't be difficult to convert it to read tenant data from a DB. I think it would be a starting point for what you would like to accomplish.
I use Flyway in my application and it is configured as Spring bean, which performs migration as init-method(example configuration can be found here in the bottom). Of course migration should be performed before any app's interaction with database.
That's why my datasource bean depend on Flyway bean. But i create Flyway bean conditionally using #Profile, because i do not want to create it using the same app's context in integration tests.
The problem is that when i use test profile which does not create Flyway bean, datasource instantiation fails with NoSuchBeanDefinitionException, because it depends-on="flyway" which is not created with current profile. Can i somehow make spring to ignore this depends-on dependency, if related bean does not exist? If no, what is the way to decouple Flyway bean from context when i use it under test profile?
I use liquibase to set up my database schema. I disable hibernate to create anything. Because of that, my import.sql is ignored. Is there any way to configure spring boot or liquibase or any other part to load test data after liquibase has created the tables?
If you need something crude (i.e, not for actual data migrations), you can use Spring's JDBC initializer in Spring Boot. In a nutshell, you'll need to:
create a data-test.sql to load your data, and place it in your src/main/resources directory. For different environments, just use the naming convention data-{platform}.sql
add applications-test.properties to src/main/resources with the following:
spring.datasource.platform=test # this actives data-test.sql above
spring.datasource.continueOnError=???? # depends on your needs
to active the application-test.properties during your testing, make sure "test" is one of the profiles that's active during your integration test. One way to do this is to annotate your test class with #ActiveProfiles({"test", ...}).
The simplest way seems to load the data with liquibase. You can do it with a normal Changeset (XML or JSON) or a Changeset in SQL-Format.
The most common way is to load CSV-Data or run an existing SQL-File.
I am developing a Spring Boot application that uses Spring Data JPA and will need to connect to many different databases e.g. PostreSQL, MySQL, MS-SQL, MongoDB.
I need to create all datasources in runtime i.e. user choose these data by GUI in started application:
-driver(one of the list),
-source,
-port,
-username,
-password.
And after all he writes native sql to choosen database and get results.
I read a lot of things about it in stack and spring forums(e.g. AbstractRoutingDataSource) but all of these tutorials show how to create datasources from xml configuration or static definition in java bean. It is possible to create many datsources in runtime? How to manage transactions and how to create many sessionFactories? It is possible to use #Transactional annotation? What is the best method to do this? Can someone explain me how to do this 'step by step'?
Hope it's not too late for an answer ;)
I developed a module which can be easily integrated in any spring project. It uses a meta-datasource to hold the tenant-datasource connection details.
For the tenant-datasource an AbstractRoutingDataSource is used.
Here you find my core implementation using the AbstractRoutingDataSource.
https://github.com/Dactabird/multitenancy
Here is an example to show how to integrate it. https://github.com/Dactabird/multitenancy-sample
In this example I'm using H2 embedded db. But of course you can use whatever you want.
Feel free to modify it for your purposes or to ask if questions are left!