I am facing an issue where my datasource bean is going down after a period of inactivity. My question is how could I re instantiate the datasource bean that gets hit on application startup.
Here is how we setup the bean on startup.
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
public DataSource dataSource(){
byte[] encryptedFile = fileRetriever.getFile(bucket, key);
String unencryptedJson = fileDecrypter.decryptFile(encryptedFile);
JSONParser parser = new JSONParser();
JSONObject jsonObject = null;
try{
jsonObject = (JSONObject) parser.parse(unencryptedJson);
}catch (Exception ex){
log.error(ex.getMessage());
}
String password = (String)jsonObject.get("password");
DataSource ds = DataSourceBuilder
.create()
.url(url)
.username(userName)
.password(password)
.build();
return ds;
}
This class also has a #Configuration annotation on it.
We have other applications that do not have this issue where the service needs to be bounced after inactivity, but they are also not setting up the data source in this manner and have all the details specified in the application.property file
I have added a custom health check that uses a repository that hits every 30 seconds so that should keep the data source bean alive but incase it does go down I would need a way to recreate it.
Thanks in advance
I assume that boot is configuring the DataSource for you. In this case, and since you are using MySQL, you can add the following to your application.properties up to 1.3
spring.datasource.test-on-borrow=true
spring.datasource.validationQuery=SELECT 1
Might considered a pooled datasource connector. Look at apache dbcb2.
Here is a sample i have that keeps a minimum of 10 idle and increases as needed from the pool.
private static DataSource createDataSource(String url, String userName, String passwrd) throws Exception {
Class.forName(DRIVER).newInstance();
Properties props = new Properties();
props.setProperty("user", userName);
props.setProperty("password", passwrd);
//Create a connection factory that the pool will use to create connections
ConnectionFactory cf = new DriverManagerConnectionFactory(url, props);
//Create the poolable connection factory
PoolableConnectionFactory pcf = new PoolableConnectionFactory(cf, null);
pcf.setValidationQuery("SELECT 1");
pcf.setDefaultAutoCommit(true);
GenericObjectPoolConfig poolConfig = new GenericObjectPoolConfig();
poolConfig.setMinIdle(10);
poolConfig.setMaxTotal(100);
AbandonedConfig abandonConfig = new AbandonedConfig();
abandonConfig.setRemoveAbandonedTimeout(60);
abandonConfig.setLogAbandoned(false);
abandonConfig.setRemoveAbandonedOnBorrow(true);
abandonConfig.setRemoveAbandonedOnMaintenance(true);
//Create the pool of connections
GenericObjectPool<PoolableConnection> connectionPool = new GenericObjectPool<>(pcf, poolConfig);
connectionPool.setTestOnBorrow(true);
connectionPool.setTestWhileIdle(true);
connectionPool.setTimeBetweenEvictionRunsMillis(10000);
connectionPool.setMinEvictableIdleTimeMillis(1000);
connectionPool.setAbandonedConfig(abandonConfig);
pcf.setPool(connectionPool);
//Pooling data source itself
PoolingDataSource<PoolableConnection> dataSource = new PoolingDataSource<>(connectionPool);
return dataSource;
}
Maven dependencies for apache dbcb2
<!-- Database connection pools -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-dbcp2</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
<version>2.4.2</version>
</dependency>
Related
So I am trying to understand how to build connection pools and to connect via java to an Oracle DB. i am trying to use dbcp2 to learn more about how all of this works. If I use the BasicDataSource I can connect and I see 5 connections via the number I set up when I created the pools like this.
private static BasicDataSource getDataSource() {
{
if (dataSource == null)
{
BasicDataSource ds = new BasicDataSource();
ds.setUrl("jdbc:oracle:thin:#pdb_tac");
ds.setUsername("hr");
ds.setPassword("my_password");
//ds.setDriverClassName("oracle.jdbc.driver.OracleDriver");
ds.setDriverClassName("oracle.jdbc.replay.OracleDataSourceImpl");
//ds.setDriverClass("oracle.jdbc.replay.OracleDataSourceImpl");
ds.setDefaultAutoCommit(false);
ds.setInitialSize(5);
ds.setMinIdle(5);
ds.setMaxIdle(10);
ds.setMaxOpenPreparedStatements(100);
dataSource = ds;
}
return dataSource;
}
}
Well, my issue is I need to be able to create a PoolingDataSource to deal with connecting to an Oracle RAC for high availability.
So I create my data source like this.
private static DataSource setupDataSource(){
String connectURI = "jdbc:oracle:thin:hr/my_password#pdb_tac";
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(connectURI,null);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(connectionFactory,null);
ObjectPool objectPool = new GenericObjectPool(poolableConnectionFactory);
poolableConnectionFactory.setPool(objectPool);
PoolingDataSource dataSource = new PoolingDataSource(objectPool);
return dataSource;
}
Well when I connect I only see 1 connection to the database not 5 like I do when I use BasicDataSource. Can someone help me understand what I am doing wrong? Thanks.
I can get connection using JDBC
OracleDataSource ds = (OracleDataSource) Class.forName("oracle.jdbc.pool.OracleConnectionPoolDataSource").newInstance();
//...
Connection connection = ds.getConnection("USER", "PASSWORD");
But I can't do it for Hibernate using hibernate.connection.datasource and jni:
I tried different variants, but it does not work.
String var1 = "oracle.jdbc.pool.OracleConnectionPoolDataSource";
String var2 = "java:/oracle.jdbc.pool.OracleConnectionPoolDataSource";
String var3 = "java:/oracle/jdbc/pool/OracleConnectionPoolDataSource";
String var4 = "java:/OracleConnectionPoolDataSource";
configuration.setProperty("hibernate.connection.datasource",var1);
All variants throw exceptions when it tries to build SessionFactory:
private static SessionFactory createSessionFactory(Configuration configuration) {
StandardServiceRegistryBuilder builder = new StandardServiceRegistryBuilder();
builder.applySettings(configuration.getProperties());
ServiceRegistry serviceRegistry = builder.build();
return configuration.buildSessionFactory(serviceRegistry);
}
Exceptions like this:
Exception in thread "main" org.hibernate.engine.jndi.JndiException: Error parsing JNDI name [oracle.jdbc.pool.OracleConnectionPoolDataSource]
maven dependecies for it:
<dependency>
<groupId>local.ora9iDriver</groupId>
<artifactId>ora9iDriver</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>C:/Users/micah/.m2/repository/local/ora9iDriver/oraDriver/1.0/ora9i.jar</systemPath>
</dependency>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc14</artifactId>
<version>9.0.2.0.0</version>
</dependency>
Can you check out the UCP with Hibernate blog and get some pointers?
I am going through spring boot application and mongoDb connection POC.
I have added following dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
Also I have gone through mongoB properties with properties: https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html
Can you please how do we define connection pooling mechanism here?
You cannot do this out of the box with application properties. You need to make use of MongoClientOptions to configure various aspects of connection pool.
Have a look at the documentation for various options available.
Here is a simple example.
#Bean(name="mongoTempl")
public MongoTemplate mongoTempl() throws Exception {
return new MongoTemplate(createMongoClient(new ServerAddress(host, port))
,dbName);
}
Mongo createMongoClient(ServerAddress serverAddress) {
final MongoClientOptions options = MongoClientOptions.builder()
.threadsAllowedToBlockForConnectionMultiplier(...)
.connectionsPerHost(...)
.connectTimeout(...)
.maxWaitTime(...)
.socketKeepAlive(...)
.socketTimeout(...)
.heartbeatConnectTimeout(...)
.minHeartbeatFrequency(...)
.build();
return new MongoClient(serverAddress, options);
}
You can use also MongoClientSettingsBuilderCustomizer like in this spring sample
#Bean
public MongoClientSettingsBuilderCustomizer customizer() {
return (builder) -> builder.applyToConnectionPoolSettings(
(connectionPool) -> {
connectionPool.maxSize(10);
connectionPool.minSize(2);
connectionPool.maxConnectionIdleTime(5, TimeUnit.MINUTES);
connectionPool.maxWaitTime(2, TimeUnit.MINUTES);
connectionPool.maxConnectionLifeTime(30, TimeUnit.MINUTES);
connectionPool.addConnectionPoolListener();
});
}
I want to change the default size of connection pool provided by java mongodb driver which is 100 according to mongo docs.
Below is the mongo client bean which I used to customize the connection pool size (refered this question). I set both min and max connectionPerHost attributes to 1 and ran 10 parallel worker threads which interact with the DB to make sure that my change is applied.
#Bean
public Mongo mongo() throws Exception {
MongoClientOptions.Builder clientOptions = new MongoClientOptions.Builder();
clientOptions.minConnectionsPerHost(1);
clientOptions.connectionsPerHost(1);
MongoClient mongoClient = new MongoClient(new MongoClientURI(env.getProperty("mongodbhost"), clientOptions));
return mongoClient;
}
Then I calculated the starting and ending time spots of each worker thread. So that I know for sure the threads are working parallely and my connection pool size haven't changed by these configuration.
Could anyone help me to get through this please? any help would be highly appreciated!
You can configure connection parameters by uri.
spring.data.mongodb.uri=mongodb://localhost:27017/?connectTimeoutMS=300000&minPoolSize=0&maxPoolSize=10&maxIdleTimeMS=900000
Please see the following documentation for other parameters.
https://docs.mongodb.com/manual/reference/connection-string/#connections-connection-options
With updated Spring boot(2.0.0 +) and Mongo DB java(3.9 +) driver versions following code can be used for creating configurable mongo template in spring boot.
Most of the configurations that were earlier part of MongoClientOptions are moved to MongoClientSettings.
import com.mongodb.*;
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import com.mongodb.connection.*;
import org.springframework.data.mongodb.core.MongoTemplate;
#Configuration
public class MongoConfig {
//-- variables
#Bean(name = "mongoTemplate")
public MongoTemplate getMongoTemplate(){
MongoTemplate mongoTemplate = new MongoTemplate(getMongoClient(), mongoDatabaseName);
return mongoTemplate;
}
private MongoClient getMongoClient(){
List<ServerAddress> serverAddressList = new ArrayList<>();
String[] hostPortList = mongoHostPortList.split(",");
for (String serverAddress : hostPortList) {
String[] hostPortArr = serverAddress.split(":");
serverAddressList.add(new ServerAddress(hostPortArr[0], Integer.parseInt(hostPortArr[1])));
}
MongoClientSettings mongoSettingsProperties = getMongoClientSettings();
MongoClient mongoClient = MongoClients.create(mongoSettingsProperties);
return mongoClient;
}
private MongoClientSettings getMongoClientSettings() {
return MongoClientSettings.builder()
.applicationName(appName)
.applyToSslSettings(sslBuilder ->
SslSettings.builder().
enabled(sslEnabled).
invalidHostNameAllowed(false).build())
.applyToConnectionPoolSettings(connPoolBuilder ->
ConnectionPoolSettings.builder().
maxWaitTime(maxWaitTime, MILLISECONDS).
maxSize(connectionPoolMinSize).
maxSize(connectionPoolMaxSize).build())
.applyToSocketSettings(socketBuilder ->
SocketSettings.builder().
connectTimeout(connectionTimeout,MILLISECONDS).build())
.readPreference(ReadPreference.secondaryPreferred())
.build();
}
}
Above code is verified with dependencies -
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
<version>2.3.1.RELEASE</version>
</dependency>
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
<version>3.11.2</version>
</dependency>
You can configure connection pool size via MongoDb uri parameters. Details - https://stackoverflow.com/a/50407284/6629515
I am setting up my DataSource in a Spring Boot / Spring Cloud Connectors project running on Cloud Foundry using Tomcat JDBC Connection Pool and MariaDB JDBC driver like so:
#Configuration
#Profile("cloud")
public class MyDataSourceConfiguration extends AbstractCloudConfig {
#Bean
public DataSource dataSource() {
Map<String, Object> dataSourceProperties = new HashMap<>();
dataSourceProperties.put("initialSize", "4"); // OK
dataSourceProperties.put("maxActive", "4"); // OK
dataSourceProperties.put("maxWait", "2000"); // OK
dataSourceProperties.put("connectionProperties",
"useUnicode=yes;characterEncoding=utf8;"); // ignored
DataSourceConfig conf = new DataSourceConfig(dataSourceProperties);
return connectionFactory().dataSource(conf);
}
}
For some reason only the properties referring to the pool size and maxWait but not the connectionProperties are getting picked up by the DataSource bean - see the log output:
maxActive=4; initialSize=4; maxWait=2000;
connectionProperties=null
Any hints ?
Note: Trying to set the connectionProperties via Spring's ConnectionConfig class didn't work either.
Try using the form of DataSourceConfig that takes separate PoolConfig and ConnectionConfig beans, like this:
#Bean
public DataSource dataSource() {
PoolConfig poolConfig = new PoolConfig(4, 4, 2000);
ConnectionConfig connectionConfig = new ConnectionConfig("useUnicode=yes;characterEncoding=utf8;");
DataSourceConfig dbConfig = new DataSourceConfig(poolConfig, connectionConfig);
return connectionFactory().dataSource(dbConfig);
}
Try the following:
Replace
connProperties.put("connectionProperties", "useUnicode=yes;characterEncoding=utf8;");
with
connProperties.put("connectionProperties", "useUnicode=yes;characterEncoding=UTF-8;");
Alternately, you can also specify the following property directly in application.properties
spring.datasource.connectionProperties=useUnicode=true;characterEncoding=utf-8;