I am trying to configure mongodb properties through application-{environment-name}.properties for connecting to mongodb.
Here's my code for making connection to mongo:
#Configuration
#EnableAutoConfiguration
public class SpringMongoConfig {
#Value("${db.connectionURL}")
private String databaseURL;
#Value("${db.name}")
private String databaseName;
#Bean
public MongoDbFactory mongoDbFactory() throws Exception {
System.out.println("database url: " + databaseURL + " db name: " + databaseName);
return new SimpleMongoDbFactory(new MongoClient(databaseURL), databaseName);
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory());
return mongoTemplate;
}
}
Here's my application-test.properties file:
db.connectionURL=localhost
db.name=rahab
I'm getting null values for databaseURL and databaseName. My idea is that the values are still not available during bean creation. But I don't have any idea to achieve this.
you can use PropertySource in the configuration to specify the location of the properties file which you have created. The example code is given below:
#Configuration
#EnableAutoConfiguration
#PropertySource("application-test.properties")
public class SpringMongoConfig {
#Value("${db.connectionURL}")
private String databaseURL;
#Value("${db.name}")
private String databaseName;
#Bean
public MongoDbFactory mongoDbFactory() throws Exception {
System.out.println("database url: " + databaseURL + " db name: " + databaseName);
return new SimpleMongoDbFactory(new MongoClient(databaseURL), databaseName);
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory());
return mongoTemplate;
}
}
Hope it helps to solve your issue.
Related
Hi I have this config file to connect to my remote mongodb service now I need to connect 1 more db simultaneously but I don't know how can anyone help?
#Configuration
public class MongoConfig{
#Bean
public MongoDbFactory mongoDbFactory(){
MongoClientURI uri = new MongoClientURI("mongodb://user:pass#localhost:27017/?authSource=admin");
return new SimpleMongoDbFactory(new MongoClient(uri), "DataBuild");
}
#Bean
public MongoOperations mongoOperations(){
return new MongoTemplate(mongoDbFactory());
}
#Bean
MongoTransactionManager transactionManager(MongoDbFactory dbFactory) {
return new MongoTransactionManager(dbFactory);
}
}
Take a look in this tutorial. Here is the #Configuration class they use:
#Configuration
public class MultipleMongoConfig {
#Primary
#Bean(name = "newdb1Properties")
#ConfigurationProperties(prefix = "spring.data.mongodb.newdb1")
public MongoProperties getNewDb1Props() throws Exception {
return new MongoProperties();
}
#Bean(name = "newdb2Properties")
#ConfigurationProperties(prefix = "spring.data.mongodb.newdb2")
public MongoProperties getNewDb2Props() throws Exception {
return new MongoProperties();
}
#Primary
#Bean(name = "newdb1MongoTemplate")
public MongoTemplate newdb1MongoTemplate() throws Exception {
return new MongoTemplate(newdb1MongoDatabaseFactory(getNewDb1Props()));
}
#Bean(name ="newdb2MongoTemplate")
public MongoTemplate newdb2MongoTemplate() throws Exception {
return new MongoTemplate(newdb2MongoDatabaseFactory(getNewDb2Props()));
}
#Primary
#Bean
public MongoDatabaseFactory newdb1MongoDatabaseFactory(MongoProperties mongo) throws Exception {
return new SimpleMongoClientDatabaseFactory(
mongo.getUri()
);
}
#Bean
public MongoDatabaseFactory newdb2MongoDatabaseFactory(MongoProperties mongo) throws Exception {
return new SimpleMongoClientDatabaseFactory(
mongo.getUri()
);
}
}
I am using springboot 2.6.6 with mysql and quite new to springbatch. I was getting an error "spring context is forming a cycle" when I kept the datasource in the same config file where readers and writers are kept in SampleJob.java. Suggested solution was to put the datasource in another class so I placed the datasource within the same class as main().
Now I am getting this issue:
Error
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.autoconfigure.batch.BatchDataSourceScriptDatabaseInitializer]: Factory method 'batchDataSourceInitializer' threw exception; nested exception is java.lang.IllegalStateException: Unable to detect database type
Caused by: java.lang.IllegalStateException: Unable to detect database type
SampleJob.java
#Configuration
public class SampleJob {
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Bean
public Job chunkJob() {
return jobBuilderFactory.get("Chunk Job")
.incrementer(new RunIdIncrementer())
.start(firstChunkStep())
.build();
}
private Step firstChunkStep() {
return stepBuilderFactory.get("First Chunk Step")
.<StudentJdbc, StudentJdbc>chunk(3)
.reader(jdbcItemReader())
.writer(flatFileItemWriter(null))
.build();
}
#Bean
#StepScope
public JdbcCursorItemReader<StudentJdbc> jdbcItemReader(){
JdbcCursorItemReader<StudentJdbc> jdbcReader = new JdbcCursorItemReader<StudentJdbc>();
jdbcReader.setSql("select id, first_name as firstName, last_name as lastName, email from students");
jdbcReader.setDataSource(universitydatasource());
jdbcReader.setRowMapper(new BeanPropertyRowMapper<StudentJdbc>() {
{
setMappedClass(StudentJdbc.class);
}
});
return jdbcReader;
}
#StepScope
#Bean
public FlatFileItemWriter<StudentJdbc> flatFileItemWriter(
#Value("#{jobParameters['outputFile']}") FileSystemResource fileSystemResource
){
FlatFileItemWriter<StudentJdbc> flatFileItemWriter = new FlatFileItemWriter<StudentJdbc>();
flatFileItemWriter.setResource(fileSystemResource);
flatFileItemWriter.setResource(fileSystemResource);
flatFileItemWriter.setHeaderCallback(new FlatFileHeaderCallback() {
#Override
public void writeHeader(Writer writer) throws IOException {
writer.write("Id, First Name, Last Name, Email");
}
});
flatFileItemWriter.setLineAggregator(new DelimitedLineAggregator(){
{
setFieldExtractor(new BeanWrapperFieldExtractor<StudentResponse>() {
{
setNames(new String[] {"id","firstName","lastName","email"});
}
});
}
});
flatFileItemWriter.setFooterCallback(new FlatFileFooterCallback() {
#Override
public void writeFooter(Writer writer) throws IOException {
writer.write("Created # "+new Date());
}
});
return flatFileItemWriter;
}
}
Main class file
#SpringBootApplication
#EnableBatchProcessing
public class ChunksApplication {
public static void main(String[] args) {
SpringApplication.run(ChunksApplication.class, args);
}
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource")
public DataSource datasource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="spring.universitydatasource")
public DataSource universitydatasource() {
return DataSourceBuilder.create().build();
}
}
application.properties
spring.datasource.url=jdbc:mysql://localhost:3306/udemy-springbatch-chunks
spring.datasource.username=____
spring.datasource.password=____
spring.datasource.platform=mysql
spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver
#alternate datasource for db input for reader
spring.universitydatasource.url=jdbc:mysql://localhost:3306/university?createDatabaseIfNotExist=true
spring.universitydatasource.username=____
spring.universitydatasource.password=____
spring.universitydatasource.platform=mysql
spring.universitydatasource.driverClassName=com.mysql.cj.jdbc.Driver
Found the solution. It seems that #ConfigurationProperties weren't picking up the url, username and password from application.properties so this worked in the main class file itself:
#Bean
#Primary
public DataSource datasource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url("jdbc:mysql://localhost:3306/udemy-springbatch-chunks");
dataSourceBuilder.username("____");
dataSourceBuilder.password("____");
return dataSourceBuilder.build();
}
#Bean("universityDatasource")
public DataSource universitydatasource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url("jdbc:mysql://localhost:3306/university?createDatabaseIfNotExist=true");
dataSourceBuilder.username("____");
dataSourceBuilder.password("____");
return dataSourceBuilder.build();
}
It works for me
spring.batch.schema=classpath:org/springframework/batch/core/schema-mysql.sql
My solution was this, you have to add the jdbcUrl
Properties file example:
spring.datasource.url=jdbc:mysql://localhost:3306/udemy-springbatch-chunks
spring.datasource.jdbcUrl=${spring.datasource.url}
spring.datasource.username=____
spring.datasource.password=____
spring.datasource.platform=mysql
spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver
#alternate datasource for db input for reader
spring.universitydatasource.url=jdbc:mysql://localhost:3306/university
spring.universitydatasource.jdbcUrl=${spring.universitydatasource.url}
spring.universitydatasource.username=____
spring.universitydatasource.password=____
spring.universitydatasource.platform=mysql
spring.universitydatasource.driverClassName=com.mysql.cj.jdbc.Driver
Regards.
I am trying to seed my multiple tenant (single database, multiple schemas) system with data but running into an issue that wasn't present when I was using the same code with a single database. I fully expect during my research that I have missed something obvious.
Each schema will contain the exact same table structure.
Here is my Tenant Context
public class TenantContext {
public static final String DEFAULT_TENANT_IDENTIFIER = "public";
private static final ThreadLocal<String> TENANT_IDENTIFIER = new ThreadLocal<>();
public static void setTenant(String tenantIdentifier) {
TENANT_IDENTIFIER.set(tenantIdentifier);
}
public static void reset(String tenantIdentifier) {
TENANT_IDENTIFIER.remove();
}
#Component
public static class TenantIdentifierResolver implements CurrentTenantIdentifierResolver {
#Override
public String resolveCurrentTenantIdentifier() {
String currentTenantId = TENANT_IDENTIFIER.get();
return currentTenantId != null ?
currentTenantId :
DEFAULT_TENANT_IDENTIFIER;
}
#Override
public boolean validateExistingCurrentSessions() {
return false;
}
}
}
And my HibernateConfig
#Configuration
public class HibernateConfig {
#Autowired
private JpaProperties jpaProperties;
#Bean
public JpaVendorAdapter jpaVendorAdapter() {
return new HibernateJpaVendorAdapter();
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(DataSource dataSource,
MultiTenantConnectionProvider multiTenantConnectionProvider, CurrentTenantIdentifierResolver currentTenantIdentifierResolver) {
Map<String, Object> jpaPropertiesMap = new HashMap<>();
jpaPropertiesMap.putAll(jpaProperties.getProperties());
jpaPropertiesMap.put(Environment.MULTI_TENANT, MultiTenancyStrategy.SCHEMA);
jpaPropertiesMap.put(Environment.MULTI_TENANT_CONNECTION_PROVIDER, multiTenantConnectionProvider);
jpaPropertiesMap.put(Environment.MULTI_TENANT_IDENTIFIER_RESOLVER, TenantContext.TenantIdentifierResolver.class);
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource);
entityManagerFactoryBean.setPackagesToScan(UppStudentAppBeApplication.class.getPackage().getName());
entityManagerFactoryBean.setJpaVendorAdapter(jpaVendorAdapter());
entityManagerFactoryBean.setJpaPropertyMap(jpaPropertiesMap);
return entityManagerFactoryBean;
}
}
And my TenantConenctionProvider
#Component
public class TenantConnectionProvider implements MultiTenantConnectionProvider {
private static Logger logger = LoggerFactory.getLogger(TenantConnectionProvider.class);
#Autowired
private DataSource dataSource;
public TenantConnectionProvider(DataSource dataSource) {
this.dataSource = dataSource;
}
#Override
public Connection getAnyConnection() throws SQLException {
return dataSource.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
connection.close();
}
#Override
public Connection getConnection(String tenantIdentifier) throws SQLException {
logger.info("Get connection for tenant " + String.join(":", tenantIdentifier ));
final Connection connection = getAnyConnection();
try {
//connection.createStatement().execute( String.format("SET SCHEMA \"%s\";", tenantIdentifier));
connection.setSchema(tenantIdentifier);
} catch ( SQLException e ) {
throw new HibernateException(
"Could not alter JDBC connection to specified schema [" +
tenantIdentifier + "]",
e
);
}
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
try {
//connection.createStatement().execute( String.format("SET SCHEMA \"%s\";", TenantContext.DEFAULT_TENANT_IDENTIFIER) );
connection.setSchema(TenantContext.DEFAULT_TENANT_IDENTIFIER);
} catch ( SQLException e ) {
throw new HibernateException(
"Could not alter JDBC connection to specified schema [" +
tenantIdentifier + "]",
e
);
}
releaseAnyConnection(connection);
}
#Override
public boolean supportsAggressiveRelease() {
return false;
}
#Override
public boolean isUnwrappableAs(Class unwrapType) {
return false;
}
#Override
public <T> T unwrap(Class<T> unwrapType) {
return null;
}
}
I call my seeding class that builds out my tenants and schemas using flyway migration.
I then try to loop through the saved tenants switching the TenantContext. Which when debugging appears to work. However when I try and do anything with the repo I get the following error.
o.h.engine.jdbc.spi.SqlExceptionHelper : ERROR: column campus0_.createdat does not exist
Hint: Perhaps you meant to reference the column "campus0_.created_at".
Position: 32
As I said earlier this worked fine previously when it was a single database and schema. I am not 100% sure on where I have gone wrong. Am I supposed to register the schemas some how? If so how can I onboard new tenants without redeploying? Should I use a custom query at this stage that uses the schema in the repo?
Thank you in advance for any help or advice.
EDIT 1
So I have now got past my initial hurdle by checking the hibernate properties so by changing the hibernate config as follows
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(DataSource dataSource,
MultiTenantConnectionProvider multiTenantConnectionProvider,
HibernateProperties hibernateProperties) {
Map<String, Object> jpaPropertiesMap = hibernateProperties.determineHibernateProperties(jpaProperties.getProperties(), new HibernateSettings());
//jpaPropertiesMap.putAll(jpaProperties.getProperties());
jpaPropertiesMap.put(Environment.MULTI_TENANT, MultiTenancyStrategy.SCHEMA);
jpaPropertiesMap.put(Environment.MULTI_TENANT_CONNECTION_PROVIDER, multiTenantConnectionProvider);
jpaPropertiesMap.put(Environment.MULTI_TENANT_IDENTIFIER_RESOLVER, TenantContext.TenantIdentifierResolver.class);
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource);
entityManagerFactoryBean.setPackagesToScan(UppStudentAppBeApplication.class.getPackage().getName());
entityManagerFactoryBean.setJpaVendorAdapter(jpaVendorAdapter());
entityManagerFactoryBean.setJpaPropertyMap(jpaPropertiesMap);
return entityManagerFactoryBean;
}
This has now removed the above naming error. However now it is saving to my default schema rather than the schema set in the TenantIdentifierResolver.
Have you implemented AsyncHandlerInterceptor - interceptor of Spring. Should be registered also in WebMvcConfigurer.
#Component
public class TenantRequestInterceptor implements AsyncHandlerInterceptor{
private SecurityDomain securityDomain;
public TenantRequestInterceptor(SecurityDomain securityDomain) {
this.securityDomain = securityDomain;
}
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
return Optional.ofNullable(request)
.map(req -> securityDomain.getTenantIdFromJwt(req))
.map(tenant -> setTenantContext(tenant))
.orElse(false);
}
#Override
public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) {
TenantContext.reset();
}
private boolean setTenantContext(String tenant) {
TenantContext.setCurrentTenant(tenant);
return true;
}
}
This is important, because here you fill TenantContext with tenant.
Have you debuged method getConnection(String tenantIdentifier) what value is as tenantIdentifier?
I am trying to configure ldap server with spring-boot for this i am using application.yml file to read ldap url,base,userdn,password etc. from code if i try to read the application.yml properties they are always returning null.
below are my files please help me in fixing the issue.
//application.yml
//---------------
spring:
profiles:
active: TEST
---
spring:
profiles: PROD
logging:
config: classpath:PROD/log4j2.yml
ldap:
url: ldap://pod-url
base: XXX
userDn: yyy
password: password
---
spring:
profiles: TEST
logging:
config: classpath:UAT/log4j2.yml
ldap:
url: ldap://ldap.forumsys.com:389
base: dc=example,dc=com
userDn: cn=read-only-admin,dc=example,dc=com
password: password
//LdapConfig.java
//----------------
#Configuration
#ComponentScan(basePackages = {"com.base.package.*"})
public class LdapConfig {
#Autowired
Environment env;
#Bean
public LdapContextSource contextSource() {
LdapContextSource contextSource = new LdapContextSource();
contextSource.setUrl(env.getProperty("ldap.url"));
contextSource.setBase(env.getProperty("ldap.base"));
contextSource.setUserDn(env.getProperty("ldap.userDn"));
contextSource.setPassword(env.getProperty("ldap.password"));
contextSource.afterPropertiesSet();
return contextSource;
}
#Bean
public LdapTemplate ldapTemplate() {
return new LdapTemplate(contextSource());
}
}
//from the below file i will be trying use ldapconfig properties
//AuthenticationServiceimpl.java
//------------------------------
public class AuthenticationServiceImpl implements AuthenticationService {
Logger logger = LogManager.getLogger(AuthenticationServiceImpl.class);
private LdapTemplate ldapTemplate;
private LdapContextSource ldapContextSource;
public boolean authenticateUser(String username, String password) {
ApplicationContext context = new AnnotationConfigApplicationContext(LdapConfig.class);
ldapTemplate = (LdapTemplate) context.getBean(LdapTemplate.class);
ldapContextSource = (LdapContextSource) context.getBean(LdapContextSource.class);
DirContext ctx = null;
try {
return ldapTemplate.authenticate(ldapContextSource.getBaseLdapPathAsString(), "(uid=" + username + ")",
password);
} catch (Exception e) {
logger.error("user " + username + " failed to authenticated " + e);
return false;
} finally {
if (ctx != null) {
org.springframework.security.ldap.LdapUtils.closeContext(ctx);
}
}
}
}
//and my main application is
#SpringBootApplication
public class Application {
public static void main(String[] args) {
Logger logger = LogManager.getLogger(Application.class);
SpringApplication.run(Application.class, args);
}
}
Use #Value :
#Configuration
#ComponentScan(basePackages = {"com.base.package.*"})
public class LdapConfig {
#Autowired
Environment env;
#Value("${ldap.url}")
private String ldapUrl;
#Bean
public LdapContextSource contextSource() {
LdapContextSource contextSource = new LdapContextSource();
contextSource.setUrl(ldapUrl);
// ...
}
#Bean
public LdapTemplate ldapTemplate() {
return new LdapTemplate(contextSource());
}
}
I have an API we wrote using Spring 4 with a Mongo database. When the application loads into my local WAS, I can see the app will go out and connect to the database. However when I go to execute a function that should open a query, I get socket closed error.
My Configuration:
#Bean
public MongoDbFactory mongoDbFactory() throws Exception {
logger.info("loading MongoDBFactory bean" );
String PROCESS_ID_MONGO_KEY = "PROCESS_ID_MONGO";
Credentials credentials = credentialsManager().getCredentialsFor(PROCESS_ID_MONGO_KEY);
MongoClient mongoClient = new MongoClient(
Arrays.asList(new ServerAddress(PropertiesManagerUtility.getKeyValue(CollectionType.CREDENTIAL, "mongo.url"), 27017)),
Arrays.asList(MongoCredential.createPlainCredential(credentials.getUserid(), "$external", credentials.getPassword().toCharArray())),
MongoClientOptions.builder()
.sslEnabled(true).connectTimeout(30)
.writeConcern(WriteConcern.MAJORITY)
.socketKeepAlive(true)
.build());
return new SimpleMongoDbFactory(mongoClient, PropertiesManagerUtility.getKeyValue(CollectionType.CREDENTIAL, "mongo.db"));
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
logger.info("loading MongoTemplate bean" );
// MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory());
return new MongoTemplate(mongoDbFactory());
}
My Dao
#Component("achResponseDMDao")
public class AchResponseDMDaoImpl implements IBasicDao<AchResponseDM>{
#Autowired
MongoTemplate mongoTemplate;
public AchResponseDMDaoImpl(MongoTemplate mongoTemplate){
this.mongoTemplate = mongoTemplate;
}
#Override
public AchResponseDM findByResponseCode( String responseCode){
Query query = new Query(Criteria.where("responseCode").is(responseCode));
return mongoTemplate.findOne(query, AchResponseDM.class);
}
...
}
Question is I thought Spring would give me a new connection using the MongoFactory but it appears that original connection gets closed and no more are created. What do I need to do? Thanks in advance.
Injecting the factory instead of the MongoTemplate instance created new connections as needed. The corresponding DAOImpl would have #Autowired MongoFactory mongoFactory; and the method would create an instance of new mongoTemplate(mongoFactory).find(...) or otherwise.
the resulting DAO looks like:
#Component("achResponseDMDao")
public class AchResponseDMDaoImpl implements IBasicDao<AchResponseDM>{
#Autowired
MongoFactory mongoFactory;
#Override
public AchResponseDM findByResponseCode( String responseCode){
Query query = new Query(Criteria.where("responseCode").is(responseCode));
List<AchResponseDM> listOfResponses = mongoTemplate(mongoFactory).find(query, AchResponseDM.class);
return (listOfResponses!=null && !listOfResponses.isEmpty())?listOfResponses.get(0):defaultNonNullResponse();
}
...
}