Hi I'm using Spring Boot version 1.5.9.
When using Spring Boot to initialize schema.sql for mysql database, it works all fine and the database schema is getting created successfully. But on restart of the application this schema.sql script is executing again and the application fails to start because the tables already exist.
I tried spring.jpa.hibernate.ddl-auto=create-drop option in application.properties but it does not have any effect (probably because it only works for Hibernate entities which I'm not using)
Is there a way to have Spring Boot to re-create schema from schema.sql every time on restart if the database is not in-memory one?
GitHub:
https://github.com/itisha/spring-batch-demo/tree/database-input
According to the documentation you can simply ignore exceptions by setting spring.datasource.continue-on-error property to true
Spring Boot enables the fail-fast feature of the Spring JDBC
initializer by default, so if the scripts cause exceptions the
application will fail to start. You can tune that using
spring.datasource.continue-on-error.
or even turn it off with spring.datasource.initialize set to false
You can also disable initialization by setting spring.datasource.initialize to false.
A workaround could be, to change the create statements in your schema.sql
from
CREATE TABLE test .....
to
CREATE TABLE IF NOT EXISTS test ...
use the IF NOT EXISTS statements
turn off automatic schema creation to avoid conflicts: add this line in your application.properties
spring.jpa.hibernate.ddl-auto=none
Related
I have a Spring Batch application which will be triggered and started by a Kubernetes CronJob once a day and do some operations.
So it is not needed, that the application will run the whole time and idle until the scheduler starts the job.
To initialize the spring batch schema
spring:
batch:
jdbc:
initialize-schema: always
is used.
By doing so, every time the application is triggered to run, the schema will be initialzed again. This is why I get an error in the database logs which says
[380] ERROR: relation "batch_job_instance" already exists
because the schema will be initialzed always.
I searched for another option than just always or never, but seems nothing else.
I'm using Spring Batch Version 4.3.6 and PostgreSQL.
The spring.batch.jdbc.initialize-schema=always property tells Spring Batch Auto configuration to create the schema everytime the application restarts. This property also silently sets continueOnError= true in the auto-configuration, so even if the error comes it will not fail the application.
You will not observe this behavior locally when with embedded database, if you have any other local DB running on either docker or on system it will throw the error.
It is advisable that in containerize deployment you should set the property to never and do schema creation for batch metadata tables manually.
You can get these DDL script from Spring Batch org/springframework/batch/core/migration package. You can add this to your migration script if you are using Flyway or Liuquibase, or in docker initialization script for your DB service in docker-compose.
Note: Reference Spring Doc here
In a Spring Boot app, I am using Hibernate and 2 tables is created properly. However, I also need to insert data one of these tables and for this purpose I thought I should use Flyway.
Then I just added insert clauses to the Flyway and use the following parameters for Hibernate and Flyway in application.properties:v
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.hibernate.ddl-auto= update # also tried none
spring.flyway.url=jdbc:mysql://localhost:3306
spring.flyway.schemas=demo-db
spring.flyway.user=root
spring.flyway.password=******
I have not used Flyway for initializing database and I am not sure if I can use Flyway with Hibernate as I mentioned above. Or, should I disable Hibernate table creation and create another migration script for table creation?
If you use flyway only for insert data don't do that. Try to use this:
With Hibernate:
In addition, a file named import.sql in the root of the classpath is executed on startup if Hibernate creates the schema from scratch (that is, if the ddl-auto property is set to create or create-drop).
With Basic SQL Scripts:
Spring Boot can automatically create the schema (DDL scripts) of your JDBC DataSource or R2DBC ConnectionFactory and initialize it (DML scripts). It loads SQL from the standard root classpath locations: schema.sql and data.sql
The issue here is that Hibernate does not automatically create tables. Additionally, if using Spring Boot, Flyway will run before the service using hibernate has started. As a result, your Flyway script are interacting with a table that does not exist.
The recommended way to do this is to use Flyway to manage both your database structure, your create tables etc, and static data. This will mean your database is versioned and provisioned ready for your service and hibernate can connect.
I have quite large legacy multi module Java Spring boot project which connects to one MySql database and two Mongo databases (lets say codebook and report).
I'm trying to setup Mongock to be able to do Mongo database migrations. I need it on only one Mongodb database (report).
I added Mongock dependencies and configuration class (MongockApplicationRunner, MongoTransactionManager) to report database.
When I run the application it gives me exception: Scan package for changeLogs is not set: use appropriate setter (exception from RunnerBuilderBase class).
I debugged it and found that Mongock is trying to set configuration for codebook database.
I created another Mongock configuration, this time for codebook database. Run the application. Mongock loads my new configuration for codebook, but then tries to set another configuration for codebook database which failes because of config.getChangeLogsScanPackage().isEmpty().
It seems that Mongock is always setting some default configuration on codebook database.
When I debug it Mongock loads first codebook configuration from my mongockApplicationRunner (line .buildApplicationRunner();). The second codebook configuration comes from MongockContextBase.applicationRunner and that configuration has empty MongockSpringConfiguration.migrationScanPackage.
Why is this default configuration appearing and how to stop it from loading?
My Spring boot is 2.2.13.RELEASE and Mongock is 5.0.24.
After a lot of wasted time I found that besides Mongock configuration on report database I was also doing autoconfiguration (#EnableMongock).
I had annotation #EnableMongock on my MongockConfig class. Since codebook mongo template had #Primary annotation autocofiguration was loading that database and there were no properties to set changelog directory inside application.properties so ChangeLogsScanPackages were empty.
Simple solution, I just removed #EnableMongock annotation and it started working.
Don't mix autoconfiguration (#EnableMongock) with creating mongockApplicationRunner and transactionManager or you will end up with problem like me.
I would like to initialize my postgres database with data.sql file. I have created queries like:
insert into network_hashrate (
rep_date, hashrate
)
select
date_from - (s.a || ' hour')::interval,
s.a::double precision
from generate_series(0, 9999, 1) AS s(a);
Is it even possible to populate database using postgres functions in Spring? If not, what are my other options. I need like 10k sample records.
According to Spring Boot doc:
Spring Boot can automatically create the schema (DDL scripts) of your DataSource and initialize it (DML scripts). It loads SQL from the standard root classpath locations: schema.sql and data.sql, respectively.
So if you need to populate data only - just create data.sql file with your sql-scripts, place it to resources folder, then check spring.jpa.hibernate.ddl-auto in the application.properties to be set to none.
If you need more flexible solution, you can use Flyway. To use it - add its dependency to your project
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-core</artifactId>
</dependency>
Turn the spring.jpa.hibernate.ddl-auto to validate.
Add spring.flyway.enabled=true to application.properties.
Place you 'migration' sql scripts to the 'default' location resources/db/migration folder. Call them like this, for example:
V1__schema_initialization.sql
V2__data_population.sql
When your spring boot app will be starting, Flyway check your database for missing schema and data then rolls these scripts sequentially.
More info about Flyway is here.
Seems you can run sql script after db scheme validate/created
Just name sql query file import.sql and spring should run it according this doc
You need something that will keep a track of what query ran and when ran. Also it should only run once not all the time when application startups.
liquibase is a option which can be used for that.
It will allow DDL as well as DML.
This link will give detail, How can you configure liquibase with spring
https://medium.com/#harittweets/evolving-your-database-using-spring-boot-and-liquibase-844fcd7931da
I want to generate DB structure from my Java classes
jpa.generate-ddl: true
jpa.ddl-auto: true
Also, I need to run SQL script before application will up because I have #PostConstruct methods where I use these data.
Can you show an example how to do it in Spring Boot?
A simple spring boot app with the required functionality can be found at.
https://github.com/salilotr89/Spring-boot-postgres-dbinit
Spring JDBC has a DataSource initializer feature. Spring Boot enables it by default and loads SQL from the standard locations schema.sql and data.sql (in the root of the classpath).
In addition Spring Boot will load the schema-${platform}.sql and data-${platform}.sql files (if present), where platform is the value of spring.datasource.platform, e.g. you might choose to set it to the vendor name of the database (hsqldb, h2, oracle, mysql, postgresql etc.).
Spring Boot enables the fail-fast feature of the Spring JDBC initializer by default, so if the scripts cause exceptions the application will fail to start. The script locations can be changed by setting spring.datasource.schema and spring.datasource.data, and neither location will be processed if spring.datasource.initialize=false.
To disable the fail-fast you can set spring.datasource.continue-on-error=true. This can be useful once an application has matured and been deployed a few times, since the scripts can act as ‘poor man’s migrations’ — inserts that fail mean that the data is already there, so there would be no need to prevent the application from running, for instance.
If you want to use the schema.sql initialization in a JPA app (with Hibernate) then ddl-auto=create-drop will lead to errors if Hibernate tries to create the same tables. To avoid those errors set ddl-auto explicitly to "" (preferable) or "none". Whether or not you use ddl-auto=create-drop you can always use data.sql to initialize new data.
http://docs.spring.io/spring-boot/docs/current/reference/html/howto-database-initialization.html
For Reference: Spring Boot - Loading Initial Data