Up until now I had been using in-memory H2 DB with Spring Batch. However, now I switched to connecting to external postgres DB. Here was my connection object (with some obfuscation):
#Bean
public DataSource postgresDatasource() {
DriverManagerDataSource datasource = new DriverManagerDataSource();
datasource.setDriverClassName("org.postgresql.Driver");
datasource.setUrl("jdbc:postgresql://x.x.x.x:xxxx/blah");
datasource.setUsername("Joe");
datasource.setPassword("password");
return datasource;
}
When I start my application, I get:
Caused by: org.springframework.jdbc.BadSqlGrammarException:
PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID,
JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? and JOB_KEY = ?];
nested exception is org.postgresql.util.PSQLException: ERROR: relation
"batch_job_instance" does not exist
I then read that Spring Batch uses the database to save metadata for its recover/retry functionality, and with embedded databases, these are tables Spring Batch sets up by default. Ok, so that would explain why I had never seen this error before.
However, it said I could set this property:
spring.batch.initialize-schema=never
So I put this in my application.properties file. However, I am still getting the error. I would be grateful for any ideas.
I was able to address this myself. Ultimately I needed the Spring Batch repository independent from my actual target relational database. So I found this reference:
https://github.com/spring-projects/spring-batch/blob/342d27bc1ed83312bdcd9c0cb30510f4c469e47d/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java#L84
I was able to take the DefaultBatchConfigurer class from that example and make a minor change to the data source by adding the #Qualifier for embedded/local data source:
#Autowired(required = false)
public void setDataSource(#Qualifier("dataSource") DataSource dataSource) {
this.dataSource = dataSource;
this.transactionManager = new DataSourceTransactionManager(dataSource);
}
Then, on my Spring Batch reader (in my other batch config class), I made a minor change to the data source by adding the #Qualifier for postgres data source:
#Bean
public ItemReader<StuffDto> itemReader(#Qualifier("postgresDataSource")DataSource dataSource) {
return new JdbcCursorItemReaderBuilder<StuffDto>()
.name("cursorItemReader")
.dataSource(dataSource)
.sql(GET_DATA)
.rowMapper(new BeanPropertyRowMapper<>(StuffDto.class))
.build();
}
Then lastly (or firstly really as I did these first), I explicitly named my data source beans so java could tell them apart to use as above:
#Configuration
public class PersistenceContext {
#Bean(name = "dataSource")
public DataSource dataSource() {
DriverManagerDataSource datasource = new DriverManagerDataSource();
datasource.setDriverClassName("org.h2.Driver");
datasource.setUrl("jdbc:h2:file:/tmp/test");
datasource.setUsername("sa");
datasource.setPassword("");
return datasource;
}
#Bean(name = "postgresDataSource")
public DataSource postgresDatasource() {
DriverManagerDataSource datasource = new DriverManagerDataSource();
datasource.setDriverClassName("org.postgresql.Driver");
datasource.setUrl("jdbc:postgresql://x.x.x.x:xxxx/blah");
datasource.setUsername("joe");
datasource.setPassword("password");
return datasource; }
}
Once I did all the above, the error disappeared and everything worked.
Related
So I am trying to understand how to build connection pools and to connect via java to an Oracle DB. i am trying to use dbcp2 to learn more about how all of this works. If I use the BasicDataSource I can connect and I see 5 connections via the number I set up when I created the pools like this.
private static BasicDataSource getDataSource() {
{
if (dataSource == null)
{
BasicDataSource ds = new BasicDataSource();
ds.setUrl("jdbc:oracle:thin:#pdb_tac");
ds.setUsername("hr");
ds.setPassword("my_password");
//ds.setDriverClassName("oracle.jdbc.driver.OracleDriver");
ds.setDriverClassName("oracle.jdbc.replay.OracleDataSourceImpl");
//ds.setDriverClass("oracle.jdbc.replay.OracleDataSourceImpl");
ds.setDefaultAutoCommit(false);
ds.setInitialSize(5);
ds.setMinIdle(5);
ds.setMaxIdle(10);
ds.setMaxOpenPreparedStatements(100);
dataSource = ds;
}
return dataSource;
}
}
Well, my issue is I need to be able to create a PoolingDataSource to deal with connecting to an Oracle RAC for high availability.
So I create my data source like this.
private static DataSource setupDataSource(){
String connectURI = "jdbc:oracle:thin:hr/my_password#pdb_tac";
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(connectURI,null);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(connectionFactory,null);
ObjectPool objectPool = new GenericObjectPool(poolableConnectionFactory);
poolableConnectionFactory.setPool(objectPool);
PoolingDataSource dataSource = new PoolingDataSource(objectPool);
return dataSource;
}
Well when I connect I only see 1 connection to the database not 5 like I do when I use BasicDataSource. Can someone help me understand what I am doing wrong? Thanks.
when there will be an issue with the external database for a live webapp and bean won't be able to get instantiated when retried again by the application then app goes down how can we fix this problem?
#Bean(destroyMethod = "close")
public DataSource dataSource(){
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setDriverClassName("com.mysql.jdbc.Driver");
hikariConfig.setJdbcUrl("jdbc:mysql://localhost:3306/spring-test");
hikariConfig.setUsername("root");
hikariConfig.setPassword("admin");
hikariConfig.setMaximumPoolSize(5);
hikariConfig.setConnectionTestQuery("SELECT 1");
hikariConfig.setPoolName("springHikariCP");
hikariConfig.addDataSourceProperty("dataSource.cachePrepStmts", "true");
hikariConfig.addDataSourceProperty("dataSource.prepStmtCacheSize", "250");
hikariConfig.addDataSourceProperty("dataSource.prepStmtCacheSqlLimit", "2048");
hikariConfig.addDataSourceProperty("dataSource.useServerPrepStmts", "true");
HikariDataSource dataSource = new HikariDataSource(hikariConfig);
return dataSource;
}
Your code will probably throw an exception if the datasource is down. Wrap it into try/catch like so:
try {
<your code goes here>
} catch (Exception e) {
return null;
}
But code calling for a datasource will have to accept the result may be null rather than throwing NullPointerExceptions.
I'm a C# developer and need to maintain an existing Java Service application developed using spring boot framework. The responsible developer left the company some time ago so I have no possibility to get some help...
So far I have no experience with Java and the used spring boot framework.
What I need to achive:
Check if used h2 Database is corrupted
If corrupted: Delete the database and create a new empty one
I guess I need to implement the check and recreation in the main entry point
public static void main(String[] args) {
SpringApplication.run(MessageServiceApplication.class, args);
}
As I know spring-boot and Hibernate creates the db automatically on startup if the database does not exists. So far so good. Now I need to check if the database is corrupted. I thought about executing a query on the database and if I get an exception I recreate the database.
The Database is a h2 file database.
Hopefully I can get some assistance.
Edit #1
I thought about implementing a utils class which gets called on startup:
public class H2DbUtils {
public boolean IsH2FileDatabaseCorrupted()
{
boolean isCorrupted = false;
// Implement Logic to determine if db is corrupted
return isCorrupted;
}
public boolean ReCreateH2DatabaseFile()
{
boolean reCreated = false;
// Implement Logic to recreate db
return reCreated;
}
}
Calling this class on startup
public static void main(String[] args) {
H2DbUtils h2DbUtils = new H2DbUtils();
if(h2DbUtils.IsH2FileDatabaseCorrupted()) {
h2DbUtils.ReCreateH2DatabaseFile();
}
SpringApplication.run(MessageServiceApplication.class, args);
}
Update 2018-03-20
Currently found the following solution to achive this:
#Configuration
#Component
public class DataSourceBean {
#Autowired
private Environment currentEnvironment;
private final Logger logInstance = LoggerFactory.getLogger(this.getClass());
#Bean
#Primary
public DataSource dataSource()
{
DataSource dataSource = null;
try
{
// We try to get the Meta Data out of the database.
// If this fails the database is corrupted or has an other problem
// All in all this means we need to delete the current database file
// to avoid further problems.
dataSource = this.getDataSource();
dataSource.getConnection().getMetaData();
return dataSource;
}
catch (Exception ex)
{
logInstance.error("The h2 database file '{}' seems to be corrupted! Error: {}",
currentEnvironment.getProperty("dataBaseFile"),
ex.getMessage());
// dataBaseFile=./db/mydatabase.db
String databaseFilePath = String.format("%s.%s", currentEnvironment.getProperty("dataBaseFile"), "h2.db");
databaseFilePath = databaseFilePath.replace("/", "\\");
File databaseFile = new File(databaseFilePath);
if (databaseFile.exists()) {
File parentDirectory = new File(databaseFile.getParent());
if (parentDirectory.isDirectory()) {
try {
FileUtils.deleteDirectory(parentDirectory);
} catch (Exception fex) {
logInstance.error("Error occurred deleting the folder {}. Error: {}",
parentDirectory.getAbsolutePath(),
fex.getMessage());
}
}
}
dataSource = this.getDataSource();
}
finally {
return dataSource;
}
}
#ConfigurationProperties(prefix = "spring.datasource")
private DataSource getDataSource() {
return DataSourceBuilder.create()
.url(currentEnvironment.getProperty("spring.datasource.url"))
.driverClassName(currentEnvironment.getProperty("spring.datasource.driverClassName"))
.username(currentEnvironment.getProperty("spring.datasource.username"))
.password(currentEnvironment.getProperty("spring.datasource.password"))
.build();
}
It is possible to overwrite the DataSource bean and check database files
#Bean
#Primary // this will override the datasource autoconfiguration and use your own everywhere
public DataSource dataSource() {
// Open Connection
// Check Database
// Close Connection
// IF File corrupted delete files
// create regular data source
}
I've tried to add several listeners to the spring boot application, for example:
SpringApplication springApplication = new SpringApplication(testApplication.class);
springApplication.addListeners(new FailedEvent(testApplication.class));
SpringApplication.run(testApplication.class, args);
But I never get to one of this listeners in the startup of the spring application. As SpringApplication.run seems to initzialise the whole spring context it is also not possible to inject or get the configuration environment to get the connection string as the application stops within SpringApplication.run as the db is corrupted.
I assume that spring tries to initzialise hibernate and so on and fails to create a database connection as the db is corrupted
org.h2.jdbc.JdbcSQLException: Allgemeiner Fehler: "java.lang.RuntimeException: rowcount remaining=2 SYS"
General error: "java.lang.RuntimeException: rowcount remaining=2 SYS" [50000-196]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.196.jar:1.4.196]
at org.h2.message.DbException.get(DbException.java:168) ~[h2-1.4.196.jar:1.4.196]
at org.h2.message.DbException.convert(DbException.java:295) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Database.openDatabase(Database.java:307) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Database.<init>(Database.java:270) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.openSession(Engine.java:64) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.openSession(Engine.java:176) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:154) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.createSession(Engine.java:137) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.createSession(Engine.java:27) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:354) ~[h2-1.4.196.jar:1.4.196]
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:116) ~[h2-1.4.196.jar:1.4.196]
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:100) ~[h2-1.4.196.jar:1.4.196]
at org.h2.Driver.connect(Driver.java:69) ~[h2-1.4.196.jar:1.4.196]
This happens within the SpringApplicatio.run context. Before I found no chance to check if the db is corrupted and if so delete the database.
I want my H2 database to be stored into a file, so that once I close the application and open it again, all the data that was previously written to the database is still there, but for some reason, at the moment whenever I start the application, the database is completely empty. Any suggestions?
#Bean
public DataSource dataSource() {
File f = new File(".");
JdbcDataSource ds = new JdbcDataSource();
ds.setURL("jdbc:h2:file:" + f.getAbsolutePath() + "/db/aurinko");
ds.setUser("");
ds.setPassword("");
return ds;
}
private Properties getHibernateProperties() {
Properties prop = new Properties();
prop.put("hibernate.format_sql", "true");
prop.put("hibernate.show_sql", "false");
prop.put("hibernate.dialect", "org.hibernate.dialect.H2Dialect");
prop.put("hibernate.hbm2ddl.auto", "update");
return prop;
}
#Bean
public SessionFactory sessionFactory() throws IOException {
LocalSessionFactoryBuilder builder = new LocalSessionFactoryBuilder(dataSource());
builder.scanPackages("io.aurinko.server.jpa").addProperties(getHibernateProperties());
SessionFactory result = builder.buildSessionFactory();
return result;
}
I was using spring-boot. Turns out that spring-boot generates its own H2 database. That means that I had two separate databases, one of which I was trying to use and the second one (only the in-memory one) that I was actually using.
May be try setting auto commit to true in the config/ property file. It may work
Can someone please tell me what I am missing I am trying to use JavaConfig in a Spring MVC project to setup the following database but I cant set the driverClass, user, password etc?
Can someone please tell me why
#Bean
public DataSource dataSource() {
DataSource ds = new DriverManagerDataSource();
try {
ds.setDriverClass("com.mysql.jdbc.Driver");
ds.setUser("jboss");
ds.setPassword("xoJ4u0Hs");
ds.setJdbcUrl("jdbc:mysql://6dhdbm01/jboss1");
} catch (Exception e) {
logger.error(e.getMessage());
}
return ds;
}
I FIXED IT: I HAD TO CHANGE THE CODE TO THIS:
#Bean
public DataSource dataSource() {
// com.mchange.v2.c3p0.ComboPooledDataSource ds = new com.mchange.v2.c3p0.ComboPooledDataSource();
BasicDataSource ds = new BasicDataSource();
try {
/*
This was old code for using C3P0 Database pooling
ds.setDriverClass("com.mysql.jdbc.Driver");
ds.setUser("jboss");
ds.setPassword("xoJ4u0Hs");
ds.setJdbcUrl("jdbc:mysql://6dhdbm01/jboss1");
*/
ds.setDriverClassName("com.mysql.jdbc.Driver");
ds.setUsername("jboss");
ds.setPassword("xoJ4u0Hs");
ds.setUrl("jdbc:mysql://6dhdbm01/jboss1");
} catch (Exception e) {
logger.error(e.getMessage());
}
return ds;
}
But I am getting this error.. Am I missing something in maven pom.xml
java.lang.ClassNotFoundException: org.apache.commons.pool.impl.GenericObjectPool
My guesses:
You're importing javax.activation.DataSource rather than javax.sql.DataSource
com.mysql.jdbc.Driver isn't on classpath
URL / Credentials are wrong
But really don't have much to go off of, first guess is a compile time issue, second two are deploy time issues.
EDIT:
Can't find what symbol? My guess is DriverManagerDataSource as it is not part of core, it's part of spring-jdbc. Also, wouldn't really recommend that DataSource implementation as it isn't a connection pool, it creates a new connection each time, look into commons-dbcp