Can someone please guide me, how I can refresh the datasource whenever the password is rotated for the database.
Basically I don't want any manual step to refresh the datasource(like calling any endpoint). Rather than I can poll a file to see if DB credentials are rotated using FileWatcher service.
I have already read few solutions over stackoverflow regarding the same ask. But I couldn't able to implement it successfully.
Since I am new to stackoverflow can't comment on others question to get solution clarified.
Below is simple class for creating the connection pool.
#Configuration
#EnableTransactionManagement
public class JpaConfig {
#Value("${db.username}")
private String username;
#Value("${db.password}")
private String password;
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setGenerateDdl(true);
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setJpaVendorAdapter(vendorAdapter);
factory.setPackagesToScan("com.example");
factory.setDataSource(dataSource());
Properties properties = new Properties();
properties.put("hibernate.format_sql", true);
properties.put("hibernate.show_sql", true);
properties.put("hibernate.hbm2ddl.auto", "none");
properties.put("hibernate.generate_statistics", false);
factory.setJpaProperties(properties);
return factory;
}
#Bean
public PlatformTransactionManager transactionManager() {
JpaTransactionManager txManager = new JpaTransactionManager();
txManager.setEntityManagerFactory(entityManagerFactory);
return txManager;
}
private DataSource dataSource() {
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setJdbcUrl("db_connection_url");
hikariConfig.setUsername(username);
hikariConfig.setPassword(password);
hikariConfig.setPoolName("test_pool");
HikariDataSource hikariDataSource = new HikariDataSource(hikariConfig);
return hikariDataSource;
}
}
You can use "Spring Cloud Bus". And whenever you change the datasource password configuration, your service can listen for this event so that it can create a new bean based on the new configuration.
Please look at this document
https://spring.io/projects/spring-cloud-bus
This repo has solution for this question : https://github.com/visa2learn/spring-cloud-vault-db-cred-rotation
I find it quite useful.
Just to make solution better read about SecretLeaseContainer :
Event-based container to request secrets from Vault and renew the associated Lease. Secrets can be rotated, depending on the requested RequestedSecret.getMode()
Related
I am trying to use multiple datasource in spring batch where first step connects to one database (sybase) and second step connects to differnt datasoruce (posgres) but I am getting below error when ever I try to run the same :
org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#713a4f8d] for key [org.springframework.jdbc.datasource.DriverManagerDataSource#25a51f53] bound to thread [http-nio-9093-exec-1]
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:448)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400)
..
..
Caused by: java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#713a4f8d] for key [org.springframework.jdbc.datasource.DriverManagerDataSource#25a51f53] bound to thread [http-nio-9093-exec-1]
at org.springframework.transaction.support.TransactionSynchronizationManager.bindResource(TransactionSynchronizationManager.java:193)
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:423)
#Configuration
#EnableJpaRepositories(
basePackages = "com.**.dao.posgres",
entityManagerFactoryRef = "positionEntityManager",
transactionManagerRef = "positionTransactionManager"
)
#EnableTransactionManagement
#EnableAutoConfiguration
public class PosgresdbConfig {
#Autowired
private Environment env;
#Bean
#Primary
#Qualifier("positionEntityManager")
public LocalContainerEntityManagerFactoryBean positionEntityManager() {
LocalContainerEntityManagerFactoryBean em
= new LocalContainerEntityManagerFactoryBean();
em.setDataSource(positionDataSource());
em.setPackagesToScan(
new String[] { "com.**.dao.posgres" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.hbm2ddl.auto",
env.getProperty("hibernate.hbm2ddl.auto"));
properties.put("hibernate.dialect",
env.getProperty("hibernate.dialect"));
em.setJpaPropertyMap(properties);
return em;
}
#Bean
#Primary
#Qualifier("positionDataSource")
public DataSource positionDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
env.getProperty("datasource.posgres.driver-class-name"));
dataSource.setUrl(env.getProperty("datasource.posgres.url"));
dataSource.setUsername(env.getProperty("datasource.posgres.username"));
dataSource.setPassword(env.getProperty("datasource.posgres.password"));
return dataSource;
}
#Bean
#Primary
#Qualifier("positionTransactionManager")
public PlatformTransactionManager positionTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
positionEntityManager().getObject());
return transactionManager;
}
}
#Configuration
#EnableJpaRepositories(
basePackages = "com.**.dao.Sybase",
entityManagerFactoryRef = "SybaseEntityManager",
transactionManagerRef = "SybaseTransactionManager"
)
public class SybasedbConfig {
#Autowired
private Environment env;
#Bean
public LocalContainerEntityManagerFactoryBean SybaseEntityManager() {
LocalContainerEntityManagerFactoryBean em
= new LocalContainerEntityManagerFactoryBean();
em.setDataSource(sybaseDataSource());
em.setPackagesToScan(
new String[] { "com.**.Sybase" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.hbm2ddl.auto",
env.getProperty("hibernate.hbm2ddl.auto"));
properties.put("hibernate.dialect",
env.getProperty("hibernate.dialect"));
em.setJpaPropertyMap(properties);
return em;
}
#Bean
public DataSource sybaseDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
env.getProperty("datasource.posgres.driver-class-name"));
dataSource.setUrl(env.getProperty("spring.datasource.url"));
dataSource.setUsername(env.getProperty("spring.datasource.username"));
dataSource.setPassword(env.getProperty("spring.datasource.password"));
return dataSource;
}
#Bean
public PlatformTransactionManager SybaseTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
SybaseEntityManager().getObject());
return transactionManager;
}
}
BATCH CONFIG
#Bean
public Step loadPositionData(final ItemReader<PositionDataResult> positionDataReader,
final ItemProcessor<Object, PositionDataResult> positionRecordProcessor,
final ItemWriter<Object> positionFileWriter) {
return stepBuilderFactory.get("position-load-account")
.<LoadPositionForRiskSP, LoadPositionForRiskSP>chunk(2000)
.reader(positionDataReader)
.processor(positionRecordProcessor)
.writer(positionFileWriter)
.listener(new NoDataFoundListener())
.transactionManager(positionTransactionManager)
.build();
}
Using first datasource works fine in first step reader:
#Transactional(value="SybaseTransactionManager", propagation= Propagation.NOT_SUPPORTED, isolation= Isolation.READ_UNCOMMITTED,readOnly = true)
public PositionDataResult loadDataFromSP() {}
Now using secondary datasource in second step gives error:
#Transactional(transactionManager="positionTransactionManager", readOnly = true, propagation = Propagation.REQUIRED, noRollbackFor = Exception.class)
public void write(List<? extends Object> list) throws Exception {}
The issue is related to making the ItemWriter#write method transactional here:
#Transactional(transactionManager="positionTransactionManager", readOnly = true, propagation = Propagation.REQUIRED, noRollbackFor = Exception.class)
public void write(List<? extends Object> list) throws Exception {}
The item writer will be called in a transaction driven by Spring Batch, so this annotation is not needed and should be removed. Same thing for the reader as well. The transaction manager and the transaction attributes of the step should defined using the methods on the StepBuilder.
If the step involves multiple transactional resources, then a JtaTransactionManager should be used to coordinate the transactions between those resources.
I'm trying to setup multiple databases in an existing spring boot application.
So, I've created a configuration bean like this for each database:
#Configuration
#ComponentScan("com.example.user.data")
#EnableJpaRepositories(basePackages = "com.example.user.data.dao",
entityManagerFactoryRef = "userEntityManagerFactory",
transactionManagerRef = "userTransactionManager")
public class UserDataConfig {
#Bean(name = "userDataSourceProps")
#ConfigurationProperties("app.user.datasource")
DataSourceProperties dataSourceProperties() {
return new DataSourceProperties();
}
#Bean(name = "userDataSource")
DataSource dataSource() {
return dataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class)
.build();
}
#Bean(name = "userEntityManagerFactory")
LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder entityManagerFactoryBuilder) {
return entityManagerFactoryBuilder
.dataSource(dataSource())
.packages("com.example.user.data.dao.entity")
.persistenceUnit("users")
.build();
}
#Bean(name = "userTransactionManager")
PlatformTransactionManager transactionManager(#Qualifier("userEntityManagerFactory") LocalContainerEntityManagerFactoryBean entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory.getObject());
}
}
And disabled these auto-configurations:
DataSourceAutoConfiguration
DataSourceTransactionManagerAutoConfiguration
Everything works fine, but the jpa/hibernate properties does not applied;
For example i have entities which follows naming convention : SpringPhysicalNamingStrategy (which should be used by default); After these changes, the naming convention does not work, even if i define the property:
spring.jpa.hibernate.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
I've debugged EntityManagerFactoryBuilder creation, its jpaProperties field is empty!
Have i missed something? or I should pass jpa-properties and/or hibernate-properties manually in creating LocalContainerEntityManagerFactoryBean? For example:
return entityManagerFactoryBuilder
.dataSource(dataSource())
.properties(jpaAndHibernateProperties())
.packages("com.example.user.data.dao.entity")
.persistenceUnit("users")
.build();
UPDATE:
After testing, I realized that only HibernateProperties (spring.jpa.hibernate path) does not applied. But JpaProperties (spring.jpa path) applied.
Based on org.springframework.boot.autoconfigure.orm.jpa.JpaBaseConfiguration#entityManagerFactory, we need to create vendor-properties and pass it to EntityManagerFactoryBuilder.properties:
return entityManagerFactoryBuilder
.dataSource(dataSource())
.properties(vendorProperties())
.packages("com.example.user.data.dao.entity")
.persistenceUnit(persistenceUnit)
.build();
protected Map<String, Object> vendorProperties() {
return new LinkedHashMap<>(this.hibernateProperties
.determineHibernateProperties(jpaProperties.getProperties(),
new HibernateSettings()
.ddlAuto(() -> "none")
.hibernatePropertiesCustomizers(new ArrayList<>()))
);
}
I am working on a Spring project that needs to connect to multiple databases.
The examples I have used are all connecting to one database using properties, set at the start of the application.
My current JPA configuration looks like this:
#Configuration
#EnableTransactionManagement
public class PersistenceJPAConfig{
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(dataSource());
em.setPackagesToScan(new String[] { "com.google.persistence.model" });
JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
em.setJpaProperties(additionalProperties());
return em;
}
#Bean
public DataSource dataSource(){
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.cj.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/spring_jpa");
dataSource.setUsername( "user" );
dataSource.setPassword( "password" );
return dataSource;
}
#Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory emf){
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
#Bean
public PersistenceExceptionTranslationPostProcessor exceptionTranslation(){
return new PersistenceExceptionTranslationPostProcessor();
}
Properties additionalProperties() {
Properties properties = new Properties();
properties.setProperty("hibernate.hbm2ddl.auto", "create-drop");
properties.setProperty("hibernate.dialect", "org.hibernate.dialect.MySQL5Dialect");
return properties;
}
}
In this case, the datasource properties (driver, url, username, password) are already set. But is it possible to modify this current bean or create a new method that would allow me to modify those properties during runtime?
For example, while the application is running, is there a way for me to manually disconnect from the current database, modify the datasource properties, and reconnect to the new database instead?
You can't change this properties in runtime but if you need to use multiple databases you can create multiple persistence units. Here you can find example:
http://www.baeldung.com/spring-data-jpa-multiple-databases
The feature you're looking for is called multi-tenancy.
See, for example, this guide.
As of Spring ORM v1.4 org.springframework.boot.orm.jpa.EntityScan was deprecated in favor of org.springframework.boot.autoconfigure.domain.EntityScan.
I was going to remove deprecated annotation in favor of new one, but such replacement cause IllegalStateException:
Caused by: java.lang.IllegalStateException: No persistence units parsed from {classpath*:META-INF/persistence.xml}
at org.springframework.orm.jpa.persistenceunit.DefaultPersistenceUnitManager.obtainDefaultPersistenceUnitInfo(DefaultPersistenceUnitManager.java:680) ~[spring-orm-4.3.5.RELEASE.jar:4.3.5.RELEASE]
With org.springframework.boot.orm.jpa.EntityScan annotation, application starts and works correctly.
Here is my config:
#Configuration
#EntityScan("com.app.domain")
#EnableJpaRepositories("com.app.persistence.jpa")
public class JpaInfrastructureConfig {
// ... config props
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(driverName);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
return dataSource;
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean entityManagerFactory = new LocalContainerEntityManagerFactoryBean();
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
entityManagerFactory.setJpaVendorAdapter(vendorAdapter);
entityManagerFactory.setDataSource(dataSource());
entityManagerFactory.setJpaProperties(new Properties() {{
put("hibernate.dialect", hibernateDialect);
put("hibernate.show_sql", hibernateShowSql);
put("hibernate.hbm2ddl.auto", hibernateHbm2ddl);
}});
return entityManagerFactory;
}
#Bean
public JpaTransactionManager transactionManager() {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(entityManagerFactory().getObject());
return transactionManager;
}
// ...
}
It seams that I've missed something, haven't I?
If you take a look at this issue on Spring Boot's issue tracker, you'll find the behaviour changes with the new annotation, as now documented in the release notes.
In your example, the simplest change would be to add a line to call LocalContainerEntityManagerFactoryBean.setPackagesToScan(…), as suggested in the release notes.
i have PC A that has a running wamp server with mysql database.
and PC B that wants to connect to that database, i am using Spring jpa, JDBC, trying to obtain direct access.
what i have done so far ?
the PC A connects to the database just fine as it is local.
PC B had no permissions to connect, "access denied" exception was thrown so i did the following :
GRANT ALL PRIVILEGES ON myDB To 'root'#'myip' IDENTIFIED BY 'root';
however, second try , another exception was thrown that says, user ''#'myip' has no privileges !
anyways just to make sure, i gave and empty user all privileges on that ip.
but still exception "select command denied" is thrown ,,(pretty sure it cant see the database)
these are my database.properties :
javax.persistence.jdbc.url=jdbc:mysql://myIp:3306/myDB?useUnicode=yes&characterEncoding=UTF-8&characterSetResults=UTF-8
javax.persistence.jdbc.user=root
javax.persistence.jdbc.password=root
hibernate.dialect=org.hibernate.dialect.MySQLDialect
spring config :
#Configuration
#EnableJpaRepositories("chechecn.elections.organizer.repository")
#EnableTransactionManagement
public class SpringConfig {
#Bean
public DataSource dataSource() {
EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
return builder.setType(EmbeddedDatabaseType.HSQL).build();
}
#Bean(name = "entityManagerFactory")
public EntityManagerFactory entityManagerFactory() {
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setGenerateDdl(true);
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setJpaVendorAdapter(vendorAdapter);
factory.setPackagesToScan("chechecn.elections.organizer");
factory.setJpaProperties(additionalJpaProperties());
// factory.setDataSource(dataSource());
factory.afterPropertiesSet();
return factory.getObject();
}
private Properties additionalJpaProperties() {
Properties properties = PropertiesReader.instance.getPropValues(PropertiesConstants.DATABASE_PROPERTIES);
return properties;
}
#Bean
public PlatformTransactionManager transactionManager() {
JpaTransactionManager txManager = new JpaTransactionManager();
txManager.setEntityManagerFactory(entityManagerFactory());
return txManager;
}
#Bean
public ServiceConnector serviceConnector() {
return new ServiceConnector();
}
}
Are you sure that you have root access for the other machine at that address?