I would like to initialize my postgres database with data.sql file. I have created queries like:
insert into network_hashrate (
rep_date, hashrate
)
select
date_from - (s.a || ' hour')::interval,
s.a::double precision
from generate_series(0, 9999, 1) AS s(a);
Is it even possible to populate database using postgres functions in Spring? If not, what are my other options. I need like 10k sample records.
According to Spring Boot doc:
Spring Boot can automatically create the schema (DDL scripts) of your DataSource and initialize it (DML scripts). It loads SQL from the standard root classpath locations: schema.sql and data.sql, respectively.
So if you need to populate data only - just create data.sql file with your sql-scripts, place it to resources folder, then check spring.jpa.hibernate.ddl-auto in the application.properties to be set to none.
If you need more flexible solution, you can use Flyway. To use it - add its dependency to your project
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-core</artifactId>
</dependency>
Turn the spring.jpa.hibernate.ddl-auto to validate.
Add spring.flyway.enabled=true to application.properties.
Place you 'migration' sql scripts to the 'default' location resources/db/migration folder. Call them like this, for example:
V1__schema_initialization.sql
V2__data_population.sql
When your spring boot app will be starting, Flyway check your database for missing schema and data then rolls these scripts sequentially.
More info about Flyway is here.
Seems you can run sql script after db scheme validate/created
Just name sql query file import.sql and spring should run it according this doc
You need something that will keep a track of what query ran and when ran. Also it should only run once not all the time when application startups.
liquibase is a option which can be used for that.
It will allow DDL as well as DML.
This link will give detail, How can you configure liquibase with spring
https://medium.com/#harittweets/evolving-your-database-using-spring-boot-and-liquibase-844fcd7931da
Related
In a Spring Boot app, I am using Hibernate and 2 tables is created properly. However, I also need to insert data one of these tables and for this purpose I thought I should use Flyway.
Then I just added insert clauses to the Flyway and use the following parameters for Hibernate and Flyway in application.properties:v
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.hibernate.ddl-auto= update # also tried none
spring.flyway.url=jdbc:mysql://localhost:3306
spring.flyway.schemas=demo-db
spring.flyway.user=root
spring.flyway.password=******
I have not used Flyway for initializing database and I am not sure if I can use Flyway with Hibernate as I mentioned above. Or, should I disable Hibernate table creation and create another migration script for table creation?
If you use flyway only for insert data don't do that. Try to use this:
With Hibernate:
In addition, a file named import.sql in the root of the classpath is executed on startup if Hibernate creates the schema from scratch (that is, if the ddl-auto property is set to create or create-drop).
With Basic SQL Scripts:
Spring Boot can automatically create the schema (DDL scripts) of your JDBC DataSource or R2DBC ConnectionFactory and initialize it (DML scripts). It loads SQL from the standard root classpath locations: schema.sql and data.sql
The issue here is that Hibernate does not automatically create tables. Additionally, if using Spring Boot, Flyway will run before the service using hibernate has started. As a result, your Flyway script are interacting with a table that does not exist.
The recommended way to do this is to use Flyway to manage both your database structure, your create tables etc, and static data. This will mean your database is versioned and provisioned ready for your service and hibernate can connect.
How i could define some schema and data to be inserted into db for
sql database in spring boot
Also could i do this for embedded databases
For example i am using two databases and i want to populate some data or define some schema and apply to different databases before application starts.
A file named import.sql in the root of the classpath is executed on startup if Hibernate creates the schema from scratch (that is, if the ddl-auto property is set to create or create-drop). This can be useful for demos and for testing if you are careful but is probably not something you want to be on the classpath in production. It is a Hibernate feature (and has nothing to do with Spring).
You can take a look in spring docs
Hi I'm using Spring Boot version 1.5.9.
When using Spring Boot to initialize schema.sql for mysql database, it works all fine and the database schema is getting created successfully. But on restart of the application this schema.sql script is executing again and the application fails to start because the tables already exist.
I tried spring.jpa.hibernate.ddl-auto=create-drop option in application.properties but it does not have any effect (probably because it only works for Hibernate entities which I'm not using)
Is there a way to have Spring Boot to re-create schema from schema.sql every time on restart if the database is not in-memory one?
GitHub:
https://github.com/itisha/spring-batch-demo/tree/database-input
According to the documentation you can simply ignore exceptions by setting spring.datasource.continue-on-error property to true
Spring Boot enables the fail-fast feature of the Spring JDBC
initializer by default, so if the scripts cause exceptions the
application will fail to start. You can tune that using
spring.datasource.continue-on-error.
or even turn it off with spring.datasource.initialize set to false
You can also disable initialization by setting spring.datasource.initialize to false.
A workaround could be, to change the create statements in your schema.sql
from
CREATE TABLE test .....
to
CREATE TABLE IF NOT EXISTS test ...
use the IF NOT EXISTS statements
turn off automatic schema creation to avoid conflicts: add this line in your application.properties
spring.jpa.hibernate.ddl-auto=none
I have an app which, depending on it's deployment config, can write the same data into one or more databases that have the same schema. Basically think of it like there's an A environment a B environment and in A we write to the A AND the B databases so that there's data from both environments there.
Further confusing the issue is that the DBs in the environments MAY (and often do) have different schemas. Though they are only different in that a change may have been made to A but not B yet.
Currently I am handling this through creating basic "entity" pojos that match the table structures and then writing specific JDBC queries for each side and then conditionally running them based on a config check.
I would love to be able to use like spring jpa to handle all the queries and allow me to do some DI for testing but the only way I can see to do this is to basically create a separate repository "AXyzRepository" and "BXyzRepository" for each Entity and perhaps even different entities for the different schemas.
Is there a way to tell a repository to like "hey, this time I want you to use /this/ ConfigurationProperty" or whatever? And to ignore any missing fields from an entity?
You can use Spring profiles to setup different db sources, schema.sql (that initialize db) and data.sql (to load initial data).
For example, you need two profiles: 'dev' and 'prod'. In application.properties you set parameter spring.profiles.active for example to prod value as default (and other options common for both profiles):
application.properties
spring.profiles.active=prod
spring.datasource.initialize=true
spring.jpa.hibernate.ddl-auto=none
Then you create two application-${profile}.properties files, for dev and prod profiles in your application resources folder:
application-dev.properties
spring.datasource.url=jdbc:h2:mem:mydb
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=
spring.datasource.platform=h2
application-prod.properties
spring.datasource.url=jdbc:mysql://myhost:3306/mydb
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.username=user
spring.datasource.password=password
spring.datasource.platform=mysql
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
Then you have to create two schema-${platform}.sql and two data-${platform}.sql files (for h2 and mysql platforms):
schema-h2.sql
CREATE TABLE...
...
schema-mysql.sql
DROP TABLE...
...
CREATE TABLE...
...
data-h2.sql
INSERT INTO TABLE...
...
data-mysql.sql
INSERT INTO TABLE...
...
In your dev environment you can set command line parameter -Dspring.profiles.active=dev (or setup it right in your IDE. For example in InelliJ IDE you can setup Spring profile in Run/Debug Configuration dialog) and work with H2 DB.
When you deploy your app to prod server it will use prod profile as the default.
More info is in the documentation.
I want to package prepopulated h2 db-files with a Spring boot application. The database is only going to be used to read values.
How am I able to load the H2 files that are available on the classpath? I am very uncertain how to setup the spring.datasource.url to get this to work.
I would suggest you export the schema and data into SQL scripts and use the in-memory H2 DB in combination with Spring Boot's built-in initialization feature: http://docs.spring.io/spring-boot/docs/current/reference/html/howto-database-initialization.html#howto-initialize-a-database-using-spring-jdbc
Granted, it may result in a slower startup if you have a relatively big dataset but if that were the case you probably wouldn't have put it in the JAR in the first place :)