We have a Project with activiti and liquibase dependecies.
Activiti is automatically configured (exc. 'spring.activiti.async-executor-activate' and 'spring.activiti.database-schema-update' params in application.yml) and liquibase too (exc. 'spring.liquibase.change-log').
Now we need to rebuild indexes on activiti tables (ACT_*) with special liquibase changesets like <sql>alter index ... rebuild tablespace ...</sql>.
There is no problem on existing database, but it crashes on first app start with DB installation because of liquibase is trying to change a nonexistent ACT_* indexes.
How can I start liquibase after activiti DB installation considering Spring Boot autoconfiguration?
You could control the execution of these changesets by using preconditions. For example:
<changeSet id="1" author="bob">
<preConditions onFail="MARK_RAN">
<indexExists>your_act_index</indexExists>
</preConditions>
<sql>alter index ... rebuild tablespace ...</sql>
</changeSet>
This way indexes are only rebuilt if they exists, which is not the case on an empty database.
Related
I made a project from scratch just with the schema.sql and the data.sql just to try the schema.sql and the data.sql:
https://github.com/rmmcosta/TestSchema
Everything works fine. The table inside schema.sql is created in a MySql database (previously created and the grants were given to the user defined in application.properties) and the data.sql populates the data as it's supposed to do.
But, when I change schema.sql and data.sql to schema-mysql.sql and data-mysql.sql and I put in the application.properties the property spring.datasource.platform=mysql the schema-mysql.sql and the data-mysql.sql are not being executed.
No errors are being thrown, simple nothing happens on the database.
I tried with spring boot 2.2.4 and it works fine, but with spring boot 2.7.5 it isn't working.
Do you know if the spring.datasource.platform was deprecated? And if so, do you know how can I set the application.properties in order to run schema-mysql.sql?
Thank you in advance,
Ricardo
Note:
I tried without using spring.datasource.platform=mysql and with schema.sql and data.sql and everything works fine.
I tried with an old project, spring boot 2.2.4 and java 1.8, and works fine.
Do you know if the spring.datasource.platform was deprecated? And if so, do you know how can I set the application.properties in order to run schema-mysql.sql?
The property name changed to spring.sql.init.platform
https://github.com/spring-projects/spring-boot/blob/d870474fcd4899fac94d51311c4163832d6b109d/spring-boot-project/spring-boot-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json#L1148
Which occurred in Spring Boot 2.5: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.5.0-RC1-Configuration-Changelog
You need to use spring.sql.init.platform instead of spring.datasource.platform property in application.properties.
Please see the comment from latest documentation -
In addition, Spring Boot processes the schema-${platform}.sql and
data-${platform}.sql files (if present), where platform is the value
of spring.sql.init.platform
I am wondering if is it possible to insert in multiple databases in MongoDB. I have on my pom.xml project this connection structure:
<configuration.database.host>XXX.XXX.XXX.XXX</configuration.database.host>
<configuration.database.port>27017</configuration.database.port>
<configuration.database.username>useradmin</configuration.database.username>
<configuration.database.password>userpass</configuration.database.password>
<configuration.database>database1</configuration.database>
<configuration.database.collection.documentone>master</configuration.database.collection.documentone>
<configuration.database.collection.documentone.documenttwo>master_advice</configuration.database.collection.documentone.documenttwo>
Now, I need to add in the project another database to insert in antoher document some information, I don't know if I can add another connection and how can I do. for example, in the pom adding:
<configuration.database>database2</configuration.database>
<configuration.database.collection.documentone>cluster</configuration.database.collection.documentone>
of course, considering the configuration in the application.properties file.
I am using spring with liquibase to update my database. Since know I have not need to user rollback functonality, but the times come where I would like to make it work.
But I cant seems to fire it from my application.
I know that maven has plugin which helps with that, but until know I was not using it and when I add it I need to provide source and credentials to my database.
In this moment liquibase is configured in xml.
<bean id="liquibase" class="liquibase.integration.spring.SpringLiquibase">
<property name="dataSource" ref="p6spyDataSource"/>
<property name="changeLog" value="classpath:db.changelog-master.yaml"/>
.
.
</bean>
And in maven I have only dependency to liquibase-core.
And the place where I set liquibase.shouldRun is in application.properties
DataSource is taken from TomEE configuration server.xml file
So the question is if I can maybe somehow add maven plugin without adding credentials (should be taken from dataSource). Or is there other way to run rollback script from my changelog?
There are several related questions that have been posted previously about using Liquibase rollback with Spring Boot. This one seems the most similar to your post: Perform a liquibase:rollback from the command line when properties are in Spring-boot files (application.properties) and not liquibase.properties
Here is the answer as provided by Robert Kleinschmager:
The property names within springs application.properties and liquidate.properties are not compatible. You have three options
#1 just create a separate liquibase.properties file with the content you need - see liquibase doc as you only need to fix your current setup
#2 give the database parameters via command-line arguments
mvn liquibase:rollback -Dliquibase.rollbackCount=1 -Dliquibase.url=jdbc:postgresql://localhost:5432/comptesfrance -Dliquibase.username
see rollback goal for all arguments
#3 if you need a permanent solution, then you may add the liquibase properties into your application.properties and reuse them in the same file. i.e.
liquibase.url=jdbc:postgresql://localhost:5432/comptesfrance
spring.datasource.url=${liquibase.url}
I'm trying to set up the liquibase gradle plugin to generate my changelogs automatically. I have managed to run the plugin and obtain a changelog but the log being generated contains several problems.
The main problem that I'm having is that the generated changelog is asking the DB to delete the hibernate sequence table.
The relevant setup part of my build.gradle file is the following:
dependencies{
//liquibase
liquibaseRuntime 'org.liquibase:liquibase-core:3.8.1'
liquibaseRuntime 'com.vaadin:vaadin-spring-boot-starter'
liquibaseRuntime 'org.mariadb.jdbc:mariadb-java-client:'+mariaDbClient
liquibaseRuntime 'mysql:mysql-connector-java:8.0.19'
liquibaseRuntime 'org.liquibase.ext:liquibase-hibernate5:3.8'
liquibaseRuntime 'org.springframework.boot:spring-boot-starter'
liquibaseRuntime 'org.springframework.boot:spring-boot-starter-security'
liquibaseRuntime 'org.springframework.boot:spring-boot-starter-data-jpa'
liquibaseRuntime sourceSets.main.output
}
diff.dependsOn compileJava
diffChangeLog.dependsOn compileJava
generateChangelog.dependsOn compileJava
configurations {
liquibaseRuntime.extendsFrom runtime
}
liquibase {
activities {
diffMain {
changeLogFile 'src/main/resources/db/liquibase-changelog-gen.xml'
url 'jdbc:mysql://localhost:3306/ideasapps?useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC&hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy&hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy'
username 'xxx'
password 'xxx'
referenceUrl 'hibernate:spring:a.b.c?dialect=org.hibernate.dialect.MariaDBDialect&hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy&hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy'
}
}
}
I'm using a mariaDB database that is compatible with the MySQL driver I have setup. I had to use the MySQL connector because there is a bug with the MariaDB one and the hibernate liquibase plugin (CORE-3457). The MySQL one runs. Also, I had to specify the naming strategy because the default one wasn't matching the strategy spring uses.
When I run gradlew diff I get this weird part where it doesn't recognize liquibase own tables and the hibernate sequence table:
Unexpected Column(s):
ideasapps.databasechangelog.AUTHOR
ideasapps.databasechangelog.COMMENTS
ideasapps.databasechangelog.CONTEXTS
ideasapps.databasechangelog.DATEEXECUTED
ideasapps.databasechangelog.DEPLOYMENT_ID
ideasapps.databasechangelog.DESCRIPTION
ideasapps.databasechangelog.EXECTYPE
ideasapps.databasechangelog.FILENAME
ideasapps.databasechangelog.ID
ideasapps.databasechangeloglock.ID
ideasapps.databasechangelog.LABELS
ideasapps.databasechangelog.LIQUIBASE
ideasapps.databasechangeloglock.LOCKED
ideasapps.databasechangeloglock.LOCKEDBY
ideasapps.databasechangeloglock.LOCKGRANTED
ideasapps.databasechangelog.MD5SUM
ideasapps.databasechangelog.ORDEREXECUTED
ideasapps.databasechangelog.TAG
ideasapps.hibernate_sequence.next_val
It also says that the hibernate sequence is missing:
Missing Sequence(s):
hibernate_sequence
Finally, when I run gradlew diffChangelog I'm getting this changeset:
<changeSet author="Orion (generated)" id="1587596582656-67">
<dropTable tableName="hibernate_sequence"/>
</changeSet>
This breaks the application since it drops the table that hibernate uses to keep track of the current indexes. How to correctly setup this plugin to achieve correct changelog generation?
I didn't found out what was causing the issue but I managed to resolve it by changing the table to a sequence through liquibase:
<changeSet author="Lucas Carvalhaes" id="xxxxxxx">
<preConditions onFail="MARK_RAN">
<tableExists tableName="hibernate_sequence"/>
</preConditions>
<dropTable cascadeConstraints="true"
tableName="hibernate_sequence"/>
<createSequence
cycle="false"
ordered="true"
sequenceName="hibernate_sequence"
startValue="34494"/>
</changeSet>
I just used the original table value as the start value of the sequence to preserve my database (since it is a small value this isn't really a problem).
Is it possible for new Flyway migrations to be generated by JPA/Hibernate's automatic schema generation when a new model / field etc. are added via Java code.
It would be useful to capture the auto-generated SQL and save it directly to a new Flyway migration, for review / editing / committing to a project repository.
Thank you in advance for any assistance or enlightenment you can offer.
If your IDE of choice is IntelliJ IDEA, I'd recommend using the JPA Buddy plugin to do this. It can generate Flyway migrations by comparing your Java model to the target DB.
You can use it to keep your evolving model and your SQL scripts in sync.
Also, it can create the init script if your DB is empty.
Once you have it installed and have Flyway as your Maven/Gradle dependency, you can generate a migration like this:
Flyway doesn't have built-in support for diff, I use liquidbase within a maven spring boot project and changelogs can be created from JPA/hibernate changes by using:
mvn liquibase:diff
All of the options for liquibase diff are located here:
http://www.liquibase.org/documentation/maven/maven_diff.html
If you want to generate the update SQL automatically, you can ask Hibernate to do so; just add the lines below to your Spring Boot configuration:
spring.jpa.properties.javax.persistence.schema-generation.create-source=metadata
spring.jpa.properties.javax.persistence.schema-generation.scripts.action=update
spring.jpa.properties.javax.persistence.schema-generation.scripts.create-target=update.sql
When you execute the application, this will generate a file named update.sql on the root of your project. Now, you can just copy and paste them into your Flyway migration.
This was adapted from this other answer: https://stackoverflow.com/a/36966419/679240 ; it is basically the same logic, except that one wants to generate a database creation script, while I needed an update script, instead.
BTW, if you want to replace the names of the foreign keys on the script with more readable ones, you could use this regex: ^(alter table .*?)(\w+)(\s+add constraint )\w+( foreign key \()(.*?)(\).*) with this replacement: $1$2$3fk_$2__$5$4$5$6; this will change the names of the FKs in the script to fk_name_of_the_table__name_of_the_field.