We Keep Coding

Spring Batch initialize-schema always will throw error relation "batch_job_instance" already exists

I have a Spring Batch application which will be triggered and started by a Kubernetes CronJob once a day and do some operations.
So it is not needed, that the application will run the whole time and idle until the scheduler starts the job.
To initialize the spring batch schema
spring:
batch:
jdbc:
initialize-schema: always
is used.
By doing so, every time the application is triggered to run, the schema will be initialzed again. This is why I get an error in the database logs which says
[380] ERROR: relation "batch_job_instance" already exists
because the schema will be initialzed always.
I searched for another option than just always or never, but seems nothing else.
I'm using Spring Batch Version 4.3.6 and PostgreSQL.

The spring.batch.jdbc.initialize-schema=always property tells Spring Batch Auto configuration to create the schema everytime the application restarts. This property also silently sets continueOnError= true in the auto-configuration, so even if the error comes it will not fail the application.
You will not observe this behavior locally when with embedded database, if you have any other local DB running on either docker or on system it will throw the error.
It is advisable that in containerize deployment you should set the property to never and do schema creation for batch metadata tables manually.
You can get these DDL script from Spring Batch org/springframework/batch/core/migration package. You can add this to your migration script if you are using Flyway or Liuquibase, or in docker initialization script for your DB service in docker-compose.
Note: Reference Spring Doc here

Can you set the compatibility mode on the Autoconfigured Spring Boot H2 test DB?

Is there a way to configure the H2 Compatibility Mode for the H2 Database that Spring Boot can auto configure to replace your regular database without just replacing it?
There are documented ways of disabling the autoconfiguration test database replacement:
https://stackoverflow.com/a/43557541/141042
I don't mind doing something like this, but most of the alternatives come with other complexities:
if you add a application.properties in your test classpath, this replaces your main application.properties during test runs, so then you're stuck maintaining two files (e.g. https://github.com/spring-projects/spring-boot/issues/10271)
if you set up a profile for test runs, then you have to make sure that any test needing the test database is marked with the profile
Is there a better way of doing this? I like the simplicity of the Spring Boot auto configured test database, but it seems like I have to force it into MySQL compatibility mode now to continue to work with my existing migrations.
So is there:
a way to configure the compatibility mode of h2 when spring autoconfigures the test database without disabling that mechanism?
a way of specifying the jdbc url for all tests without having to modify each test (e.g. to include a profile) or maintaining two application property files (e.g. a new application.properties in src/test/resources)

There isn't an option to set a custom URL for the embedded datasource that Spring Boot replaces in your tests. We offer a way to specify which connection type you want but that doesn't include the URL itself. I have no idea how easy we could add that but it's worth looking at least, I've created issue #19038
As for specifying the URL, you shouldn't add an application.properties in your test classpath for the reason you've mentioned. The SO thread you've referenced already has an answer that refers to application-test.properties.

load schema from .sql to create orm mapping in springboot

Is there any way to Load the database schema from .sql or json or textfile to create the ORM mapping with JPA/Hibernate to database in spring-boot while starting up the server.

Spring Boot enables you to use database migration tools such as Liquibase and Flyway, you can read more about that on Spring's official documentation.
Edit: From the docs
85.5 Use a Higher-level Database Migration Tool
Spring Boot supports two higher-level migration tools: Flyway and Liquibase.
85.5.1 Execute Flyway Database Migrations on Startup
To automatically run Flyway database migrations on startup, add the org.flywaydb:flyway-core to your classpath.
The migrations are scripts in the form V__.sql (with an underscore-separated version, such as ‘1’ or ‘2_1’). By default, they are in a folder called classpath:db/migration, but you can modify that location by setting spring.flyway.locations. This is a comma-separated list of one or more classpath: or filesystem: locations. For example, the following configuration would search for scripts in both the default classpath location and the /opt/migration directory:
spring.flyway.locations=classpath:db/migration,filesystem:/opt/migration
You can also add a special {vendor} placeholder to use vendor-specific scripts. Assume the following:
spring.flyway.locations=classpath:db/migration/{vendor}
Rather than using db/migration, the preceding configuration sets the folder to use according to the type of the database (such as db/migration/mysql for MySQL). The list of supported databases is available in DatabaseDriver.
FlywayProperties provides most of Flyway’s settings and a small set of additional properties that can be used to disable the migrations or switch off the location checking. If you need more control over the configuration, consider registering a FlywayConfigurationCustomizer bean.
Spring Boot calls Flyway.migrate() to perform the database migration. If you would like more control, provide a #Bean that implements FlywayMigrationStrategy.
Flyway supports SQL and Java callbacks. To use SQL-based callbacks, place the callback scripts in the classpath:db/migration folder. To use Java-based callbacks, create one or more beans that implement Callback. Any such beans are automatically registered with Flyway. They can be ordered by using #Order or by implementing Ordered. Beans that implement the deprecated FlywayCallback interface can also be detected, however they cannot be used alongside Callback beans.
By default, Flyway autowires the (#Primary) DataSource in your context and uses that for migrations. If you like to use a different DataSource, you can create one and mark its #Bean as #FlywayDataSource. If you do so and want two data sources, remember to create another one and mark it as #Primary. Alternatively, you can use Flyway’s native DataSource by setting spring.flyway.[url,user,password] in external properties. Setting either spring.flyway.url or spring.flyway.user is sufficient to cause Flyway to use its own DataSource. If any of the three properties has not be set, the value of its equivalent spring.datasource property will be used.
There is a Flyway sample so that you can see how to set things up.
You can also use Flyway to provide data for specific scenarios. For example, you can place test-specific migrations in src/test/resources and they are run only when your application starts for testing. Also, you can use profile-specific configuration to customize spring.flyway.locations so that certain migrations run only when a particular profile is active. For example, in application-dev.properties, you might specify the following setting:
spring.flyway.locations=classpath:/db/migration,classpath:/dev/db/migration
With that setup, migrations in dev/db/migration run only when the dev profile is active.
85.5.2 Execute Liquibase Database Migrations on Startup
To automatically run Liquibase database migrations on startup, add the org.liquibase:liquibase-core to your classpath.
By default, the master change log is read from db/changelog/db.changelog-master.yaml, but you can change the location by setting spring.liquibase.change-log. In addition to YAML, Liquibase also supports JSON, XML, and SQL change log formats.
By default, Liquibase autowires the (#Primary) DataSource in your context and uses that for migrations. If you need to use a different DataSource, you can create one and mark its #Bean as #LiquibaseDataSource. If you do so and you want two data sources, remember to create another one and mark it as #Primary. Alternatively, you can use Liquibase’s native DataSource by setting spring.liquibase.[url,user,password] in external properties. Setting either spring.liquibase.url or spring.liquibase.user is sufficient to cause Liquibase to use its own DataSource. If any of the three properties has not be set, the value of its equivalent spring.datasource property will be used.
See LiquibaseProperties for details about available settings such as contexts, the default schema, and others.
There is a Liquibase sample so that you can see how to set things up.
Spring also supports a database initialization on its own, the official docs are here.
Spring Boot can automatically create the schema (DDL scripts) of your DataSource and initialize it (DML scripts). It loads SQL from the standard root classpath locations: schema.sql and data.sql, respectively.

Spring Boot schema.sql - drop db schema on restart

Hi I'm using Spring Boot version 1.5.9.
When using Spring Boot to initialize schema.sql for mysql database, it works all fine and the database schema is getting created successfully. But on restart of the application this schema.sql script is executing again and the application fails to start because the tables already exist.
I tried spring.jpa.hibernate.ddl-auto=create-drop option in application.properties but it does not have any effect (probably because it only works for Hibernate entities which I'm not using)
Is there a way to have Spring Boot to re-create schema from schema.sql every time on restart if the database is not in-memory one?
GitHub:
https://github.com/itisha/spring-batch-demo/tree/database-input

According to the documentation you can simply ignore exceptions by setting spring.datasource.continue-on-error property to true
Spring Boot enables the fail-fast feature of the Spring JDBC
initializer by default, so if the scripts cause exceptions the
application will fail to start. You can tune that using
spring.datasource.continue-on-error.
or even turn it off with spring.datasource.initialize set to false
You can also disable initialization by setting spring.datasource.initialize to false.

A workaround could be, to change the create statements in your schema.sql
from
CREATE TABLE test .....
to
CREATE TABLE IF NOT EXISTS test ...
use the IF NOT EXISTS statements

turn off automatic schema creation to avoid conflicts: add this line in your application.properties
spring.jpa.hibernate.ddl-auto=none

Spring JPA data removed after shutdown of application

i have an application thats build on Spring boot, using JPA repositories on HSQL database.
Problem is that while application is running, I create an entity,and it's persisted correctly to database(can be seen in database manager). But after application shutdown from eclipse, all data is removed;
Saving is performed like this
#Service
public class NotificationService {
#Autowired
private NotificationRepository notificationRepository;
public void notifyRefreshArticles(){
Notification notification = new Notification();
notification.setCreatedAt(LocalDateTime.now());
notification.setNotificationSeverity(NotificationSeverity.NORMAL);
notification.setNotificationType(NotificationType.REFRESH_ARTICLES);
notificationRepository.save(notification);
}
}
I pretty sure its configuration issue,but with spring boot basically only configuration that i have is this configuration file.
spring.datasource.driver-class-name=org.hsqldb.jdbc.JDBCDriver
spring.datasource.url=jdbc:hsqldb:hsql://localhost/rr_app_database
spring.datasource.username=XXXXXX
spring.datasource.password=XXXXXX
spring.datasource.show-sql=true

Do you have hbm2ddl text somewhere in your configuration properties. It should be set to update or none, apparently you might have create-drop.

Specify a local filename in application.properties data source URL:
spring.datasource.url=jdbc:hsqldb:file:/home/username/testedb
You can remove the spring.datasource.driver-class-name property as Spring Boot detects it by URL property.

Check the properties files if there exists a configuration line as below
spring.jpa.hibernate.ddl-auto=create
Just remove it or change to
spring.jpa.hibernate.ddl-auto=update

If you insert the data within a #Test, it's rolled back by default.
Your database might get dropped, if you restart the application, like LearningPhase suggested.
The insert is never really committed - because it's outside of an transaction or because an Exception is thrown and not handeled within the transaction.