Is there any way to Load the database schema from .sql or json or textfile to create the ORM mapping with JPA/Hibernate to database in spring-boot while starting up the server.
Spring Boot enables you to use database migration tools such as Liquibase and Flyway, you can read more about that on Spring's official documentation.
Edit: From the docs
85.5 Use a Higher-level Database Migration Tool
Spring Boot supports two higher-level migration tools: Flyway and Liquibase.
85.5.1 Execute Flyway Database Migrations on Startup
To automatically run Flyway database migrations on startup, add the org.flywaydb:flyway-core to your classpath.
The migrations are scripts in the form V__.sql (with an underscore-separated version, such as ‘1’ or ‘2_1’). By default, they are in a folder called classpath:db/migration, but you can modify that location by setting spring.flyway.locations. This is a comma-separated list of one or more classpath: or filesystem: locations. For example, the following configuration would search for scripts in both the default classpath location and the /opt/migration directory:
spring.flyway.locations=classpath:db/migration,filesystem:/opt/migration
You can also add a special {vendor} placeholder to use vendor-specific scripts. Assume the following:
spring.flyway.locations=classpath:db/migration/{vendor}
Rather than using db/migration, the preceding configuration sets the folder to use according to the type of the database (such as db/migration/mysql for MySQL). The list of supported databases is available in DatabaseDriver.
FlywayProperties provides most of Flyway’s settings and a small set of additional properties that can be used to disable the migrations or switch off the location checking. If you need more control over the configuration, consider registering a FlywayConfigurationCustomizer bean.
Spring Boot calls Flyway.migrate() to perform the database migration. If you would like more control, provide a #Bean that implements FlywayMigrationStrategy.
Flyway supports SQL and Java callbacks. To use SQL-based callbacks, place the callback scripts in the classpath:db/migration folder. To use Java-based callbacks, create one or more beans that implement Callback. Any such beans are automatically registered with Flyway. They can be ordered by using #Order or by implementing Ordered. Beans that implement the deprecated FlywayCallback interface can also be detected, however they cannot be used alongside Callback beans.
By default, Flyway autowires the (#Primary) DataSource in your context and uses that for migrations. If you like to use a different DataSource, you can create one and mark its #Bean as #FlywayDataSource. If you do so and want two data sources, remember to create another one and mark it as #Primary. Alternatively, you can use Flyway’s native DataSource by setting spring.flyway.[url,user,password] in external properties. Setting either spring.flyway.url or spring.flyway.user is sufficient to cause Flyway to use its own DataSource. If any of the three properties has not be set, the value of its equivalent spring.datasource property will be used.
There is a Flyway sample so that you can see how to set things up.
You can also use Flyway to provide data for specific scenarios. For example, you can place test-specific migrations in src/test/resources and they are run only when your application starts for testing. Also, you can use profile-specific configuration to customize spring.flyway.locations so that certain migrations run only when a particular profile is active. For example, in application-dev.properties, you might specify the following setting:
spring.flyway.locations=classpath:/db/migration,classpath:/dev/db/migration
With that setup, migrations in dev/db/migration run only when the dev profile is active.
85.5.2 Execute Liquibase Database Migrations on Startup
To automatically run Liquibase database migrations on startup, add the org.liquibase:liquibase-core to your classpath.
By default, the master change log is read from db/changelog/db.changelog-master.yaml, but you can change the location by setting spring.liquibase.change-log. In addition to YAML, Liquibase also supports JSON, XML, and SQL change log formats.
By default, Liquibase autowires the (#Primary) DataSource in your context and uses that for migrations. If you need to use a different DataSource, you can create one and mark its #Bean as #LiquibaseDataSource. If you do so and you want two data sources, remember to create another one and mark it as #Primary. Alternatively, you can use Liquibase’s native DataSource by setting spring.liquibase.[url,user,password] in external properties. Setting either spring.liquibase.url or spring.liquibase.user is sufficient to cause Liquibase to use its own DataSource. If any of the three properties has not be set, the value of its equivalent spring.datasource property will be used.
See LiquibaseProperties for details about available settings such as contexts, the default schema, and others.
There is a Liquibase sample so that you can see how to set things up.
Spring also supports a database initialization on its own, the official docs are here.
Spring Boot can automatically create the schema (DDL scripts) of your DataSource and initialize it (DML scripts). It loads SQL from the standard root classpath locations: schema.sql and data.sql, respectively.
Related
I have two environments like onprem and aws, for
onprem we have connected with Oracle database and required configuration
is done and but for aws we are using aurora postgres and whenever
i have tried to add postgres dependency then deploying into
aws environment it's giving error so is it possible if we have
multiple dependencies in same pom file and used for different environments.
Quarks version : 1.13.3.final
Sample configuration:
Oracle:
-------
quarkus.datasource.db-kind=oracle
quarkus.datasource.username=user-default
quarkus.datasource.password=password-default
quarkus.datasource.reactive.url=jdbc:oracle:thin://localhost:5432/default
Postgresql:
-----------
quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=user-default
quarkus.datasource.password=password-default
quarkus.datasource.reactive.url=postgresql://localhost:5432/default
Code or issue:
The issue i'm getting is as follows:
2021-11-11 07:18:45,173 WARN [org.hib.eng.jdb.env.int.JdbcEnvironmentInitiator] (main) HHH000342: Could not obtain connection to query metadata: java.sql.SQLException: Driver does not support the provided URL: jdbc:postgresql://
at io.agroal.pool.ConnectionFactory.connectionSetup(ConnectionFactory.java:220)
at io.agroal.pool.ConnectionFactory.createConnection(ConnectionFactory.java:204)
at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:490)
at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:472)
Dependencies:
quarkus-jdbc-oracle
quarkus-jdbc-postgresql
According to the official documentation, you can use Multiple Datasources.
However if you have more than 1, then you must provide a Name, otherwise how will Quarkus know which Drivers to use.
The format : quarkus.datasource.<datasourceCustomName>. If it is not your default DataSource which will not require a name.
Important to Notice: For now, multiple datasources are only supported for JDBC and the Agroal extension. So it is not currently possible to create multiple reactive datasources.
https://quarkus.io/guides/datasource#multiple-datasources
Multiple Datasources
Configuring Multiple Datasources
For now, multiple datasources are only supported for JDBC and the
Agroal extension. So it is not currently possible to create multiple
reactive datasources.
The Hibernate ORM extension supports defining several persistence
units using configuration properties. For each persistence unit, you
can point to the datasource of your choice.
Defining multiple datasources works exactly the same way as defining a
single datasource, with one important change: you define a name.
In the following example, you have 3 different datasources:
The default one,
A datasource named users,
A datasource named inventory,
each with its own configuration.
quarkus.datasource.db-kind=h2
quarkus.datasource.username=username-default
quarkus.datasource.jdbc.url=jdbc:h2:mem:default
quarkus.datasource.jdbc.max-size=13
quarkus.datasource.users.db-kind=h2
quarkus.datasource.users.username=username1
quarkus.datasource.users.jdbc.url=jdbc:h2:mem:users
quarkus.datasource.users.jdbc.max-size=11
quarkus.datasource.inventory.db-kind=h2
quarkus.datasource.inventory.username=username2
quarkus.datasource.inventory.jdbc.url=jdbc:h2:mem:inventory
quarkus.datasource.inventory.jdbc.max-size=12
quarkus.datasource.db-kind is fixed at build time. The property can not change at runtime.
You have to build the app for Oracle and seperate for Postgresql
https://quarkus.io/guides/all-config#quarkus-datasource_quarkus.datasource.db-kind
Is there a way to configure the H2 Compatibility Mode for the H2 Database that Spring Boot can auto configure to replace your regular database without just replacing it?
There are documented ways of disabling the autoconfiguration test database replacement:
https://stackoverflow.com/a/43557541/141042
I don't mind doing something like this, but most of the alternatives come with other complexities:
if you add a application.properties in your test classpath, this replaces your main application.properties during test runs, so then you're stuck maintaining two files (e.g. https://github.com/spring-projects/spring-boot/issues/10271)
if you set up a profile for test runs, then you have to make sure that any test needing the test database is marked with the profile
Is there a better way of doing this? I like the simplicity of the Spring Boot auto configured test database, but it seems like I have to force it into MySQL compatibility mode now to continue to work with my existing migrations.
So is there:
a way to configure the compatibility mode of h2 when spring autoconfigures the test database without disabling that mechanism?
a way of specifying the jdbc url for all tests without having to modify each test (e.g. to include a profile) or maintaining two application property files (e.g. a new application.properties in src/test/resources)
There isn't an option to set a custom URL for the embedded datasource that Spring Boot replaces in your tests. We offer a way to specify which connection type you want but that doesn't include the URL itself. I have no idea how easy we could add that but it's worth looking at least, I've created issue #19038
As for specifying the URL, you shouldn't add an application.properties in your test classpath for the reason you've mentioned. The SO thread you've referenced already has an answer that refers to application-test.properties.
How i could define some schema and data to be inserted into db for
sql database in spring boot
Also could i do this for embedded databases
For example i am using two databases and i want to populate some data or define some schema and apply to different databases before application starts.
A file named import.sql in the root of the classpath is executed on startup if Hibernate creates the schema from scratch (that is, if the ddl-auto property is set to create or create-drop). This can be useful for demos and for testing if you are careful but is probably not something you want to be on the classpath in production. It is a Hibernate feature (and has nothing to do with Spring).
You can take a look in spring docs
I want to generate DB structure from my Java classes
jpa.generate-ddl: true
jpa.ddl-auto: true
Also, I need to run SQL script before application will up because I have #PostConstruct methods where I use these data.
Can you show an example how to do it in Spring Boot?
A simple spring boot app with the required functionality can be found at.
https://github.com/salilotr89/Spring-boot-postgres-dbinit
Spring JDBC has a DataSource initializer feature. Spring Boot enables it by default and loads SQL from the standard locations schema.sql and data.sql (in the root of the classpath).
In addition Spring Boot will load the schema-${platform}.sql and data-${platform}.sql files (if present), where platform is the value of spring.datasource.platform, e.g. you might choose to set it to the vendor name of the database (hsqldb, h2, oracle, mysql, postgresql etc.).
Spring Boot enables the fail-fast feature of the Spring JDBC initializer by default, so if the scripts cause exceptions the application will fail to start. The script locations can be changed by setting spring.datasource.schema and spring.datasource.data, and neither location will be processed if spring.datasource.initialize=false.
To disable the fail-fast you can set spring.datasource.continue-on-error=true. This can be useful once an application has matured and been deployed a few times, since the scripts can act as ‘poor man’s migrations’ — inserts that fail mean that the data is already there, so there would be no need to prevent the application from running, for instance.
If you want to use the schema.sql initialization in a JPA app (with Hibernate) then ddl-auto=create-drop will lead to errors if Hibernate tries to create the same tables. To avoid those errors set ddl-auto explicitly to "" (preferable) or "none". Whether or not you use ddl-auto=create-drop you can always use data.sql to initialize new data.
http://docs.spring.io/spring-boot/docs/current/reference/html/howto-database-initialization.html
For Reference: Spring Boot - Loading Initial Data
I use liquibase to set up my database schema. I disable hibernate to create anything. Because of that, my import.sql is ignored. Is there any way to configure spring boot or liquibase or any other part to load test data after liquibase has created the tables?
If you need something crude (i.e, not for actual data migrations), you can use Spring's JDBC initializer in Spring Boot. In a nutshell, you'll need to:
create a data-test.sql to load your data, and place it in your src/main/resources directory. For different environments, just use the naming convention data-{platform}.sql
add applications-test.properties to src/main/resources with the following:
spring.datasource.platform=test # this actives data-test.sql above
spring.datasource.continueOnError=???? # depends on your needs
to active the application-test.properties during your testing, make sure "test" is one of the profiles that's active during your integration test. One way to do this is to annotate your test class with #ActiveProfiles({"test", ...}).
The simplest way seems to load the data with liquibase. You can do it with a normal Changeset (XML or JSON) or a Changeset in SQL-Format.
The most common way is to load CSV-Data or run an existing SQL-File.