Hi I have this config file to connect to my remote mongodb service now I need to connect 1 more db simultaneously but I don't know how can anyone help?
#Configuration
public class MongoConfig{
#Bean
public MongoDbFactory mongoDbFactory(){
MongoClientURI uri = new MongoClientURI("mongodb://user:pass#localhost:27017/?authSource=admin");
return new SimpleMongoDbFactory(new MongoClient(uri), "DataBuild");
}
#Bean
public MongoOperations mongoOperations(){
return new MongoTemplate(mongoDbFactory());
}
#Bean
MongoTransactionManager transactionManager(MongoDbFactory dbFactory) {
return new MongoTransactionManager(dbFactory);
}
}
Take a look in this tutorial. Here is the #Configuration class they use:
#Configuration
public class MultipleMongoConfig {
#Primary
#Bean(name = "newdb1Properties")
#ConfigurationProperties(prefix = "spring.data.mongodb.newdb1")
public MongoProperties getNewDb1Props() throws Exception {
return new MongoProperties();
}
#Bean(name = "newdb2Properties")
#ConfigurationProperties(prefix = "spring.data.mongodb.newdb2")
public MongoProperties getNewDb2Props() throws Exception {
return new MongoProperties();
}
#Primary
#Bean(name = "newdb1MongoTemplate")
public MongoTemplate newdb1MongoTemplate() throws Exception {
return new MongoTemplate(newdb1MongoDatabaseFactory(getNewDb1Props()));
}
#Bean(name ="newdb2MongoTemplate")
public MongoTemplate newdb2MongoTemplate() throws Exception {
return new MongoTemplate(newdb2MongoDatabaseFactory(getNewDb2Props()));
}
#Primary
#Bean
public MongoDatabaseFactory newdb1MongoDatabaseFactory(MongoProperties mongo) throws Exception {
return new SimpleMongoClientDatabaseFactory(
mongo.getUri()
);
}
#Bean
public MongoDatabaseFactory newdb2MongoDatabaseFactory(MongoProperties mongo) throws Exception {
return new SimpleMongoClientDatabaseFactory(
mongo.getUri()
);
}
}
Related
I am using springboot 2.6.6 with mysql and quite new to springbatch. I was getting an error "spring context is forming a cycle" when I kept the datasource in the same config file where readers and writers are kept in SampleJob.java. Suggested solution was to put the datasource in another class so I placed the datasource within the same class as main().
Now I am getting this issue:
Error
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.autoconfigure.batch.BatchDataSourceScriptDatabaseInitializer]: Factory method 'batchDataSourceInitializer' threw exception; nested exception is java.lang.IllegalStateException: Unable to detect database type
Caused by: java.lang.IllegalStateException: Unable to detect database type
SampleJob.java
#Configuration
public class SampleJob {
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Bean
public Job chunkJob() {
return jobBuilderFactory.get("Chunk Job")
.incrementer(new RunIdIncrementer())
.start(firstChunkStep())
.build();
}
private Step firstChunkStep() {
return stepBuilderFactory.get("First Chunk Step")
.<StudentJdbc, StudentJdbc>chunk(3)
.reader(jdbcItemReader())
.writer(flatFileItemWriter(null))
.build();
}
#Bean
#StepScope
public JdbcCursorItemReader<StudentJdbc> jdbcItemReader(){
JdbcCursorItemReader<StudentJdbc> jdbcReader = new JdbcCursorItemReader<StudentJdbc>();
jdbcReader.setSql("select id, first_name as firstName, last_name as lastName, email from students");
jdbcReader.setDataSource(universitydatasource());
jdbcReader.setRowMapper(new BeanPropertyRowMapper<StudentJdbc>() {
{
setMappedClass(StudentJdbc.class);
}
});
return jdbcReader;
}
#StepScope
#Bean
public FlatFileItemWriter<StudentJdbc> flatFileItemWriter(
#Value("#{jobParameters['outputFile']}") FileSystemResource fileSystemResource
){
FlatFileItemWriter<StudentJdbc> flatFileItemWriter = new FlatFileItemWriter<StudentJdbc>();
flatFileItemWriter.setResource(fileSystemResource);
flatFileItemWriter.setResource(fileSystemResource);
flatFileItemWriter.setHeaderCallback(new FlatFileHeaderCallback() {
#Override
public void writeHeader(Writer writer) throws IOException {
writer.write("Id, First Name, Last Name, Email");
}
});
flatFileItemWriter.setLineAggregator(new DelimitedLineAggregator(){
{
setFieldExtractor(new BeanWrapperFieldExtractor<StudentResponse>() {
{
setNames(new String[] {"id","firstName","lastName","email"});
}
});
}
});
flatFileItemWriter.setFooterCallback(new FlatFileFooterCallback() {
#Override
public void writeFooter(Writer writer) throws IOException {
writer.write("Created # "+new Date());
}
});
return flatFileItemWriter;
}
}
Main class file
#SpringBootApplication
#EnableBatchProcessing
public class ChunksApplication {
public static void main(String[] args) {
SpringApplication.run(ChunksApplication.class, args);
}
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource")
public DataSource datasource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="spring.universitydatasource")
public DataSource universitydatasource() {
return DataSourceBuilder.create().build();
}
}
application.properties
spring.datasource.url=jdbc:mysql://localhost:3306/udemy-springbatch-chunks
spring.datasource.username=____
spring.datasource.password=____
spring.datasource.platform=mysql
spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver
#alternate datasource for db input for reader
spring.universitydatasource.url=jdbc:mysql://localhost:3306/university?createDatabaseIfNotExist=true
spring.universitydatasource.username=____
spring.universitydatasource.password=____
spring.universitydatasource.platform=mysql
spring.universitydatasource.driverClassName=com.mysql.cj.jdbc.Driver
Found the solution. It seems that #ConfigurationProperties weren't picking up the url, username and password from application.properties so this worked in the main class file itself:
#Bean
#Primary
public DataSource datasource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url("jdbc:mysql://localhost:3306/udemy-springbatch-chunks");
dataSourceBuilder.username("____");
dataSourceBuilder.password("____");
return dataSourceBuilder.build();
}
#Bean("universityDatasource")
public DataSource universitydatasource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url("jdbc:mysql://localhost:3306/university?createDatabaseIfNotExist=true");
dataSourceBuilder.username("____");
dataSourceBuilder.password("____");
return dataSourceBuilder.build();
}
It works for me
spring.batch.schema=classpath:org/springframework/batch/core/schema-mysql.sql
My solution was this, you have to add the jdbcUrl
Properties file example:
spring.datasource.url=jdbc:mysql://localhost:3306/udemy-springbatch-chunks
spring.datasource.jdbcUrl=${spring.datasource.url}
spring.datasource.username=____
spring.datasource.password=____
spring.datasource.platform=mysql
spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver
#alternate datasource for db input for reader
spring.universitydatasource.url=jdbc:mysql://localhost:3306/university
spring.universitydatasource.jdbcUrl=${spring.universitydatasource.url}
spring.universitydatasource.username=____
spring.universitydatasource.password=____
spring.universitydatasource.platform=mysql
spring.universitydatasource.driverClassName=com.mysql.cj.jdbc.Driver
Regards.
I'm using SpringBoot 2.1.8.RELEASE, I have two datasource, I manually control transactions.
transaction (marked"#Primary") works. But, transaction marked another datasource is not working properly.
Do you have any suggestions?
Maven configuration is as below
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
other omit...
One datasource configure as below:
#Configuration
#MapperScan(basePackages = {"com.XXX.XXXX.mapper","com.XXX.XXX.common.mapper"},sqlSessionFactoryRef = "masterSqlSessionFactory")
public class ProductDataSourceConfig {
static final String MAPPER_LOCATION = "classpath:mybatis/MYSQL/*.xml";
#Bean(name = "masterDataSource")
#ConfigurationProperties("spring.datasource.product")
public DataSource masterDataSource(){
return DataSourceBuilder.create().build();
}
#Bean(name = "masterTransactionManager")
public DataSourceTransactionManager masterTransactionManager() {
return new DataSourceTransactionManager(masterDataSource());
}
#Bean(name = "masterSqlSessionFactory")
public SqlSessionFactory masterSqlSessionFactory(#Qualifier("masterDataSource") DataSource masterDataSource)
throws Exception {
final SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
sessionFactory.setDataSource(masterDataSource);
sessionFactory.setMapperLocations(new PathMatchingResourcePatternResolver()
.getResources(ProductDataSourceConfig.MAPPER_LOCATION));
return sessionFactory.getObject();
}
}
another datasource configure as below:
#Configuration
#MapperScan(basePackages = "com.XXX.XXX.mapperTest",sqlSessionFactoryRef = "secondSqlSessionFactory")
public class TestDataSourceConfig {
static final String MAPPER_LOCATION = "classpath:mybatis/TESTMYSQL/*.xml";
#Primary
#Bean(name = "secondDataSource")
#ConfigurationProperties("spring.datasource.test")
public DataSource clusterDataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "secondTransactionManager")
public DataSourceTransactionManager clusterTransactionManager() {
return new DataSourceTransactionManager(clusterDataSource());
}
#Primary
#Bean(name = "secondSqlSessionFactory")
public SqlSessionFactory clusterSqlSessionFactory(#Qualifier("secondDataSource") DataSource clusterDataSource)
throws Exception {
final SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
sessionFactory.setDataSource(clusterDataSource);
sessionFactory.setMapperLocations(new PathMatchingResourcePatternResolver()
.getResources(TestDataSourceConfig.MAPPER_LOCATION));
return sessionFactory.getObject();
}
}
When configured as follows, submit as expected(in the "#Primary" db,record inserted ).
#Autowired
DataSourceTransactionManager dataSourceTransactionManager;
DefaultTransactionDefinition def = new DefaultTransactionDefinition();
def.setTimeout(500);
def.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
TransactionStatus transactionStatus = dataSourceTransactionManager.getTransaction(def);
insertTmemberEnterRecord(bean);
dataSourceTransactionManager.commit(transactionStatus);
But the following configuration as follows, submit is not work(No data inserted into table),also no error message.
#Qualifier("masterTransactionManager")
#Autowired
DataSourceTransactionManager dataSourceTransactionManager;
DefaultTransactionDefinition def = new DefaultTransactionDefinition();
def.setTimeout(500);
def.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
TransactionStatus transactionStatus = dataSourceTransactionManager.getTransaction(def);
insertTmemberEnterRecord(bean);
dataSourceTransactionManager.commit(transactionStatus);
Any suggestions are welcome.
I solved the problem.
The reason is that the corresponding mapper file should also be used.
I am trying to connect two db to Spring boot project.
When I try to query, I have empty result from DB.
The app is launching properly
When I try querying I receive a nullpointer exception
public class PGDataBaseConfig {
#Bean(name = "Pgconfig")
#Primary
#ConfigurationProperties("spring.pgdatasource")
public DataSourceProperties firstDataSourceProperties() {
return new DataSourceProperties();
}
#Bean(name = "PG")
#Primary
public DataSource firstDataSource() {
return firstDataSourceProperties().initializeDataSourceBuilder().type(HikariDataSource.class).build();
}
#Bean(name = "PGem")
#Primary
public LocalContainerEntityManagerFactoryBean pgEntity(EntityManagerFactoryBuilder builder,
#Qualifier("PG") DataSource firstDataSource) {
HashMap<String,String> map = new HashMap<>();
map.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQL9Dialect");
map.put("hibernate.show_sql", "true");
return builder.dataSource(firstDataSource).packages(".db.Postgress.Model").persistenceUnit("PGDB").properties(map)
.build();
}
#Primary
#Bean(name = "PGtrans")
public PlatformTransactionManager transactionManager(#Qualifier("PGem") EntityManagerFactory entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory);
}
}
Second DB config
#EnableTransactionManagement
#EnableJpaRepositories(entityManagerFactoryRef = "MSem",basePackages = "sla.db.ms", bootstrapMode = BootstrapMode.DEFAULT)
public class MSDataBaseConfig {
#Bean(name ="MSConfig" )
#ConfigurationProperties("spring.ms.datasource")
public DataSourceProperties thirdDataSorceProperties() {
return new DataSourceProperties();
}
#Bean(name = "MS")
public DataSource thirdDataSource() {
return thirdDataSorceProperties().initializeDataSourceBuilder().type(HikariDataSource.class).build();
}
#Bean(name = "MStrans")
public PlatformTransactionManager transactionManager(
#Qualifier("MSem") EntityManagerFactory
entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
i am using Spring boot + Spring Batch + JPA + Mysql. below are the follow i am following:
spring boot application
```
#SpringBootApplication(scanBasePackages = "com.xxx.xxxx.config, com.xxx.xxx.rest, com.xxx.xxx.domain, com.xxx.xxx.dataservice, com.xxx.xxx.batchconfiguration, com.xxx.xxx.steps")
#EnableAutoConfiguration
#EnableConfigurationProperties(Properties.class)
#EnableEncryptableProperties
public class SampleApplication
{
public static void main(String[] args) {
SpringApplication.run(SampleApplication.class, args);
}
}
```
Created #Entity classes based on table structure
created reposity interface like below
```
#Component
public interface ExampleRepository extends JpaRepository<tableClass, Long> {
Page<tableClass> findTop10ByStatus(tableClassStatus status,
Pageable pageable);
}
```
Batch configuration:
```
#Configuration
#EnableScheduling
public class BatchScheduler {
#Bean
public ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public JobExplorer jobExplorer() throws Exception {
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(
mapJobRepositoryFactory());
jobExplorerFactory.afterPropertiesSet();
return jobExplorerFactory.getObject();
}
#Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactory() throws Exception {
MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean();
factory.setTransactionManager(transactionManager());
return factory;
}
#Bean
public JobRepository jobRepository() throws Exception {
return mapJobRepositoryFactory().getObject();
}
#Bean
public JobLauncher jobLauncher() throws Exception {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(jobRepository());
return simpleJobLauncher;
}
}
```
batch configuration
```
#Configuration
#EnableBatchProcessing
#Import({ BatchScheduler.class })
public class BatchConfiguration2 {
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
private SimpleJobLauncher jobLauncher;
#Autowired
public ExampleRepository exampleRepository ;
#Scheduled(cron = "0/5 * * * * ?")
public void perform() throws Exception {
System.out.println("Job Started at :" + new Date());
JobParameters param = new JobParametersBuilder()
.addString("JobID", String.valueOf(System.currentTimeMillis())).toJobParameters();
JobExecution execution = jobLauncher.run(job(), param);
System.out.println("Job finished with status :" + execution.getStatus());
}
#Bean
public RepositoryMetadata repositoryMetadata() {
return new DefaultRepositoryMetadata(ExampleRepository.class);
}
#Bean
public RepositoryItemReader<tableClass> reader() {
RepositoryItemReader<tableClass> fullfillment = new RepositoryItemReader<tableClass>();
fullfillment.setRepository(ExampleRepository);
fullfillment.setMethodName("findTop10ByStatus");
List<tableClassStatus> list = new ArrayList<tableClassStatus>();
list.add(tableClassStatus.FULFILLMENT_READY);
fullfillment.setArguments(list);
HashMap<String, Sort.Direction> sorts = new HashMap<>();
sorts.put("id", Direction.DESC);
fullfillment.setSort(sorts);
return fullfillment;
}
/* #Bean
public RepositoryItemWriter<tableClass> writer() {
System.out.println("BatchConfiguration.writer()");
RepositoryItemWriter<tableClass> itemWriter = new RepositoryItemWriter<tableClass>();
itemWriter.setRepository(ExampleRepository);
itemWriter.setMethodName("save");
return itemWriter;
}*/
#Bean
public RepositoryItemWriter<tableClass> writer() {
System.out.println("BatchConfiguration.writer()");
DefaultCrudMethods defaultCrudMethods = new DefaultCrudMethods(repositoryMetadata());
RepositoryItemWriter<tableClass> itemWriter = new RepositoryItemWriter<tableClass>();
itemWriter.setRepository(ExampleRepository);
itemWriter.setMethodName(defaultCrudMethods.getSaveMethod().getName());
return itemWriter;
}
#Bean
public Step step1() throws Exception {
return this.stepBuilderFactory.get("step1")
.<tableClass, tableClass> chunk(1).reader(reader())
.processor(new QuoteOfferFullfillmentSubmitionProcessor()).writer(writer()).build();
}
#Bean
public Job job() throws Exception {
return this.jobBuilderFactory.get("job").incrementer(new RunIdIncrementer()).start(step1())
.listener(new JobCompletionListener()).build();
}
}
```
QuoteOfferFullfillmentSubmitionProcessor.java
```
public class QuoteOfferFullfillmentSubmitionProcessor implements ItemProcessor<QuoteOfferFulfillment, QuoteOfferFulfillment> {
#Override
public tableClass process(tableClass item) throws Exception {
System.out.println("Processor.process() ==> ID " + item.getId());
System.out.println("Processor.process() ==> " + item.getLenderName());
System.out.println(
"QuoteOfferFullfillmentSubmitionProcessor.process() ==> source" + item.getStatus());
item.setStatus(QuoteOfferFulfillmentStatus.PROCESSING);
return item;
}
}
```
JobCompletionListener.java
```
public class JobCompletionListener extends JobExecutionListenerSupport {
#Override
public void afterJob(JobExecution jobExecution) {
if (jobExecution.getStatus() == BatchStatus.COMPLETED) {
System.out.println("BATCH JOB COMPLETED SUCCESSFULLY");
}
}
}
```
after all configuration, running the application every 5 sec the job is running fine, getting values and printing in process class, after that coming to writer class and printing all the values with out Errors.
the problem is: its not updating the values in the Database in write method.
with out BatchScheduler, scheduler and job launcher (commit the code) restarting the spring boot its all updating the values in database.
Can you guys suggest me what wrong i am doing.
I have resolve the issue. in the batch configuration if i use
```
#Bean
public ResourcelessTransactionManager resourcelessTransactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public JobRepository jobRepository() throws Exception {
return new MapJobRepositoryFactoryBean(resourcelessTransactionManager()).getObject();
}
```
it started working.
Thanks,
Bala.
I am doing little project using Spring+Hibernate. When I deploy my war and get 404 status all the time. I have checked some answers about it, but I didnt find my mistake. I tried to switch between / and /*, it didnt help. Tomcat doesnt show any errors in logs. Thanks in advance.
My WebConfig class
#Configuration
#EnableWebMvc
#ComponentScan({"controller", "dao"})
#EnableTransactionManagement
public class WebConfig extends WebMvcConfigurerAdapter {
#Bean
public DataSource dataSource() {
SimpleDriverDataSource ds =
new SimpleDriverDataSource(org.h2.Driver.load(), "jdbc:h2:~/testdb", "sa", "sa");
return ds;
}
#Bean
#DependsOn("dataSource")
public LocalSessionFactoryBean sessionFactoryBean() {
LocalSessionFactoryBean sessionFactoryBean = new LocalSessionFactoryBean();
sessionFactoryBean.setDataSource(dataSource());
sessionFactoryBean.setPackagesToScan("entity");
sessionFactoryBean.setHibernateProperties(hibernateProperties());
return sessionFactoryBean;
}
private Properties hibernateProperties() {
Properties p = new Properties();
p.setProperty("hibernate.dialect", "org.hibernate.dialect.H2Dialect");
p.setProperty("hibernate.hbm2ddl.auto", "update");
p.setProperty("hibernate.hbm2ddl.import_files_sql_extractor", "org.hibernate.tool.hbm2ddl.MultipleLinesSqlCommandExtractor");
p.setProperty("hibernate.show_sql", "true");
p.setProperty("hibernate.format_sql", "true");
return p;
}
#Bean
#DependsOn("sessionFactoryBean")
public PlatformTransactionManager transactionManager() {
SessionFactory sessionFactory = sessionFactoryBean().getObject();
HibernateTransactionManager tm = new HibernateTransactionManager(sessionFactory);
return tm;
}
#Bean(name = "viewResolver")
public InternalResourceViewResolver getViewResolver() {
InternalResourceViewResolver viewResolver = new InternalResourceViewResolver();
viewResolver.setPrefix("/WEB-INF/pages/");
viewResolver.setSuffix(".jsp");
return viewResolver;
}
}
My WebAppInitializer class
public class SpringWebAppInitializer implements WebApplicationInitializer {
#Override
public void onStartup(ServletContext servletContext) throws ServletException {
AnnotationConfigWebApplicationContext appContext = new AnnotationConfigWebApplicationContext();
appContext.register(WebConfig.class);
ServletRegistration.Dynamic dispatcher = servletContext.addServlet(
"SpringDispatcher", new DispatcherServlet(appContext));
dispatcher.setLoadOnStartup(1);
dispatcher.addMapping("/*");
}
}
My Controller class
#Controller
public class MainController {
#Autowired
IDisksDao d;
#RequestMapping("/")
#Transactional
#ResponseBody
public String hello() {
return "hi";
}
#RequestMapping("disks/")
#Transactional
#ResponseBody
public ModelAndView getDisks() {
ModelAndView model = new ModelAndView("disks");
model.addObject("disks", d.getAllDisks() );
return model;
}
#RequestMapping("u/disks/{id}")
#Transactional
#ResponseBody
public List<Disk> getUserDisks(#PathVariable int id) {
return d.getAllUserDisks(id);
}
}
My project structure