My Application will not start. gives me an error creating a bean with the name of my file. I've searched and found similar posts to this question but they didnt seem to pertain to my error so I am hoping someone can find what may be causing this file to fail. Also, I am at a cross between two methods. Do I use the FlatFileReader or MomgoItemreader? I just need to retrieve two things from the DB firstname and lastname. I need to read the db and then write it to a file which I havent done yet. Any help would be greatly appreciated.
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured.
Reason: Failed to determine a suitable driver class
Here is my class
#Configuration
#EnableBatchProcessing
public class PaymentPortalJob {
private static final Logger LOG =
LoggerFactory.getLogger(PaymentPortalJob.class);
#SuppressWarnings("unused")
#Autowired
private JobBuilderFactory jobBuilderFactory;
#SuppressWarnings("unused")
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private PaymentAuditRepository paymentAuditRepository;
// #Bean
// public FlatFileItemReader<PaymentAudit> PaymentPortalReader() {
// LOG.info("Inside PaymentPortalReader Method {}", "");
// return new FlatFileItemReaderBuilder<PaymentAudit>
().name("PaymentPortalReader")
// .delimited().names(new String[] { "rxFname", "rxLname" })
// .fieldSetMapper(new BeanWrapperFieldSetMapper<PaymentAudit>
() {
// {
// setTargetType(PaymentAudit.class);
// }
// }).build();
//
// }
#Bean
public ItemReader<PaymentAudit> reader() {
LOG.info("inside of ItemReader");
MongoItemReader<PaymentAudit> reader = new MongoItemReader<PaymentAudit>();
try {
reader.setTemplate(mongoTemplate());
} catch (Exception e) {
LOG.error(e.toString());
}
reader.setCollection("local");
return reader();
}
#Bean
public ItemProcessor<PaymentAudit, PaymentAudit> processor() {
return new PaymentPortalNOSQLProcessor();
}
#Bean
public ItemWriter<PaymentAudit> writer() {
MongoItemWriter<PaymentAudit> writer = new MongoItemWriter<PaymentAudit>();
try {
writer.setTemplate(mongoTemplate());
} catch (Exception e) {
LOG.error(e.toString());
}
writer.setCollection("paymentPortal");
return writer;
}
#Bean
Job job(JobBuilderFactory jbf, StepBuilderFactory sbf, ItemReader<? extends PaymentAudit> ir,
ItemWriter<? super PaymentAudit> iw) {
Step s1 = sbf.get("file-db").<PaymentAudit, PaymentAudit>chunk(100).reader(ir).writer(iw).build();
return jbf.get("etl").incrementer(new RunIdIncrementer()).start(s1).build();
}
#Bean
public MongoDbFactory mongoDbFactory() throws Exception {
return new SimpleMongoDbFactory(new MongoClient(), "local");
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory());
return mongoTemplate;
}
}
appplication.yml
mongodb:
databasePort: xxxxx
databaseName: local
spring:
data:
mongodb:
host: localhost
port: xxxxx
profiles:
active: local
batch:
job:
enabled: true
Processor:
package com.spring_batchjob.job.item;
import org.springframework.batch.item.ItemProcessor;
import com.spring_batchjob.bean.PaymentAudit;
public class PaymentPortalNOSQLProcessor implements
ItemProcessor<PaymentAudit, PaymentAudit> {
#Override
public PaymentAudit process(PaymentAudit bean) throws Exception {
return bean;
}
}
repo:
package com.spring_batchjob.repository;
import java.time.LocalDateTime;
import java.util.List;
import org.springframework.data.mongodb.repository.MongoRepository;
import org.springframework.stereotype.Repository;
import com.spring_batchjob.bean.PaymentAudit;
#Repository
public interface PaymentAuditRepository extends
MongoRepository<PaymentAudit, String> {
// List<PaymentAudit> findByCreateDttmBetween(LocalDateTime createStart,
LocalDateTime createEnd);
List<PaymentAudit> findByRxFNameAndRxLName(String rxFName, String
rxLName);
}
Error after running app:
2018-10-26 13:26:34.256 WARN 16376 --- [ restartedMain]
ConfigServletWebServerApplicationContext : Exception encountered during
context initialization - cancelling refresh attempt:
org.springframework.beans.factory.UnsatisfiedDependencyException: Error
creating bean with name 'paymentPortalJob': Unsatisfied dependency expressed
through field 'jobBuilderFactory'; nested exception is
org.springframework.beans.factory.UnsatisfiedDependencyException: Error
creating bean with name
'org.springframework.batch.core.configuration.annotation.
SimpleBatchConfiguration': Unsatisfied dependency expressed through field
'dataSource'; nested exception is
org.springframework.beans.factory.BeanCreationException:
Error creating bean
with name 'dataSource' defined in class path resource
[org/springframework/boot/autoconfigure/jdbc/
DataSourceConfiguration$Hikari.class]: Bean instantiation via factory method
failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw
exception; nested exception is
org.springframework.boot.autoconfigure.jdbc.
DataSourceProperties$DataSourceBeanCreationException: Failed to determine a
suitable driver class
Related
I am writing junit test for below source code
#Slf4j
#RestController
#RequestMapping("/routeparameter")
public class RouteParameterController {
#Autowired
private RouteParameterProcessor routeParameterProcessor;
#GetMapping
#ApiOperation(value = "RouteParameter service", authorizations = {
#Authorization(value = "X-CSR-SECURITY_TOKEN") }, notes = "Initial implementation")
public void preProcess(#RequestParam("locationCd") String locationCd,
#ApiParam(value = "ManifestDt in yyyy-MM-dd format (EX: 2021-12-14)") #RequestParam("manifestDt") String manifestDt) throws Exception {
try {
log.info("Request to process route parameters for location: {} and date: {}", locationCd, manifestDt);
routeParameterProcessor.process(locationCd, manifestDt);
} catch (RouteParameterException e) {
log.warn("Exception {} occured while calling process()", e);
throw e;
} catch (Exception e) {
log.error("Exception {} occured while calling process()", e);
throw e;
}
}
I have written below test case
#WebMvcTest
public class RouteParameterControllerTest {
#Autowired
private MockMvc mockMvc;
#MockBean
private RouteParameterProcessor routeParameterProcessor;
#Test
public void preProcessSuccess() throws Exception {
String locationCD = "NQAA";
String manifestDt = "2021-12-14";
doNothing().when(routeParameterProcessor).process(locationCD, manifestDt);
ResultActions response = mockMvc.perform(get("/routeparameter"));
response.andExpect(status().isOk());
}
}
I am getting below exception. I tried to search solution but didn't find any. I am a beginner in junit testing. From exception trace I found that I missed mocking configuration related to security.
But i dont know what are things are missing here.
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'com.xxxx.xxx.xxx.xxx.ws.gateway.security.TokenAuthenticationFilter' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {#org.springframework.beans.factory.annotation.Autowired(required=true)}
at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:1716)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1272)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1226)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640)
I'm new to spring batch and I'm trying to learn about it.
I implemented datasource and batch configuration but it doesn't seem to work, because
Exception in thread "main" org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? and JOB_KEY = ?]; nested exception is java.sql.SQLSyntaxErrorException: Table '$tableName.batch_job_instance' doesn't exist
DataSourceConfiguration class :
#Configuration
public class DataSourceConfiguration {
#Bean
public DataSource mysqlDataSource() throws SQLException {
final SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
dataSource.setDriver(new com.mysql.cj.jdbc.Driver());
dataSource.setUrl("jdbc:mysql://localhost:3306/task4");
dataSource.setUsername("name");
dataSource.setPassword("password");
return dataSource;
}
}
And I have Batch configuration class, but I dont know #EnableBatchProcessing annotation works without springboot
#Configuration
#EnableBatchProcessing
#Import(DataSourceConfiguration.class)
public class BatchConfiguration {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Autowired
private DataSource dataSource;
#Bean
public ItemReader<User> itemReader() {
return new JdbcCursorItemReaderBuilder<User>()
.dataSource(this.dataSource)
.name("creditReader")
.sql("select userLogin, userEmail, userPassword, accountBalance from userInfo")
.rowMapper(new UserRowMapper())
.build();
}
#Bean
public ItemWriter<User> simpleItemWriter(){
return new SimpleUserWriter();
}
#Bean
public Job sampleJob() {
return this.jobs.get("sampleJob")
.start(step1())
.build();
}
#Bean
public Step step1() {
return this.steps.get("step1")
.<User,User>chunk(10)
.reader(itemReader())
.writer(simpleItemWriter())
.build();
}
}
Main class contains those lines of code :
public class Demo {
public static void main(String[] args) throws SQLException {
ApplicationContext context = new AnnotationConfigApplicationContext(BatchConfiguration.class);
context.getBean("jobRepository");
JobLauncher jobLauncher = context.getBean("jobLauncher",JobLauncher.class);
Job job = context.getBean("sampleJob", Job.class);
try {
JobExecution execution = jobLauncher.run(job, new JobParameters());
System.out.println("Job Exit Status : "+ execution.getStatus());
} catch (JobExecutionException e) {
System.out.println("Job ExamResult failed");
e.printStackTrace();
}
}
}
How can I solve this problem?Thanks for helping
The error seems to say that you are missing the tables needed by Spring batch.
Check https://docs.spring.io/spring-batch/docs/4.3.x/reference/html/schema-appendix.html
You can also find SQL files ready for the different DB engine under org/springframework/bacth/core/schema-<DB>.sql in the spring batch core jar file
I have deployed multitenant Spring boot application on AWS EC2. the code just works fine in the local system, but application is failing with below error after docker run in aws ec2.
] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'orderReportListener' defined in URL [jar:file:/app-service.jar!/BOOT-INF/classes!/com/aic/autofluence/appservice/scheduler/kafkaReportListener/OrderReportListener.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'reportFactoryImplData' defined in URL [jar:file:/app-service.jar!/BOOT-INF/classes!/com/aic/autofluence/appservice/scheduler/service/Impl/ReportFactoryImplData.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'orderReportRepository' defined in com.aic.autofluence.appservice.scheduler.repository.OrderReportRepository defined in #EnableJpaRepositories declared on TenantDatabaseConfig: Cannot create inner bean '(inner bean)#4293e066' of type [org.springframework.orm.jpa.SharedEntityManagerCreator] while setting bean property 'entityManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#4293e066': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'entityManagerFactory' available
2021-05-10 18:58:57.081 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'masterdb-persistence-unit'
2021-05-10 18:58:57.082 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : masterdb-connection-pool - Shutdown initiated...
2021-05-10 18:58:57.099 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : masterdb-connection-pool - Shutdown completed.
2021-05-10 18:58:57.103 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2021-05-10 18:58:57.124 INFO 1 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-05-10 18:58:57.147 ERROR 1 --- [ main] o.s.b.d.LoggingFailureAnalysisReporter :
***************************
APPLICATION FAILED TO START
***************************
Description:
Parameter 0 of constructor in com.aic.autofluence.appservice.scheduler.service.Impl.ReportFactoryImplData required a bean named 'entityManagerFactory' that could not be found.
Sample code I have as below:
tenantConfig: The EntityManagerFactory bean itself is not recognised. it is working good in local system, failing same in vm
#Configuration
#EnableTransactionManagement
#ComponentScan(basePackages = { "x.x.x.x.scheduler.repository", "x.x.x.x.scheduler.model" })
#EnableJpaRepositories(basePackages = {"x.x.x.x..scheduler.repository", "x.x.x.x..scheduler.service"},
entityManagerFactoryRef = "tenantEntityManagerFactory",
transactionManagerRef = "tenantTransactionManager")
public class TenantDatabaseConfig {
#Bean(name = "tenantJpaVendorAdapter")
public JpaVendorAdapter jpaVendorAdapter() {
return new HibernateJpaVendorAdapter ();
}
#Bean(name = "tenantTransactionManager")
public JpaTransactionManager transactionManager(#Qualifier("tenantEntityManagerFactory") EntityManagerFactory tenantEntityManager) {
JpaTransactionManager transactionManager = new JpaTransactionManager ();
transactionManager.setEntityManagerFactory(tenantEntityManager);
return transactionManager;
}
#Bean(name = "datasourceBasedMultitenantConnectionProvider")
#ConditionalOnBean(name = "masterEntityManagerFactory")
public MultiTenantConnectionProvider multiTenantConnectionProvider() {
return new DataSourceBasedMultiTenantConnectionProviderImpl();
}
#Bean(name = "currentTenantIdentifierResolver")
public CurrentTenantIdentifierResolver currentTenantIdentifierResolver() {
return new CurrentTenantIdentifierResolverImpl();
}
#Bean(name = "tenantEntityManagerFactory")
#ConditionalOnBean(name = "datasourceBasedMultitenantConnectionProvider")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
#Qualifier("datasourceBasedMultitenantConnectionProvider")
MultiTenantConnectionProvider connectionProvider,
#Qualifier("currentTenantIdentifierResolver")
CurrentTenantIdentifierResolver tenantResolver) {
LocalContainerEntityManagerFactoryBean emfBean = new LocalContainerEntityManagerFactoryBean ();
//All tenant related entities, repositories and service classes must be scanned
emfBean.setPackagesToScan("com.aic.autofluence.appservice");
emfBean.setJpaVendorAdapter(jpaVendorAdapter());
emfBean.setPersistenceUnitName("tenantdb-persistence-unit");
Map<String, Object> properties = new HashMap<>();
properties.put( Environment.MULTI_TENANT, MultiTenancyStrategy.DATABASE);
properties.put( Environment.MULTI_TENANT_CONNECTION_PROVIDER, connectionProvider);
properties.put( Environment.MULTI_TENANT_IDENTIFIER_RESOLVER, tenantResolver);
properties.put( Environment.DIALECT, "org.hibernate.dialect.MySQL5Dialect");
properties.put( Environment.SHOW_SQL, true);
properties.put( Environment.FORMAT_SQL, true);
properties.put( Environment.HBM2DDL_AUTO, "none");
emfBean.setJpaPropertyMap(properties);
return emfBean;
}
}
**MasterConfig: It is configured properly working fine**
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = {"x.x.x.x.mastertenant.model",
"x.x.x.x.mastertenant.repository"
},
entityManagerFactoryRef = "masterEntityManagerFactory",
transactionManagerRef = "masterTransactionManager")
public class MasterDatabaseConfig {
private static final Logger LOG = LoggerFactory.getLogger(MasterDatabaseConfig.class);
#Autowired
private MasterDatabaseConfigProperties masterDbProperties;
#Bean(name = "masterDataSource")
public DataSource masterDataSource() {
HikariDataSource hikariDataSource = new HikariDataSource ();
hikariDataSource.setUsername(masterDbProperties.getUsername());
hikariDataSource.setPassword(masterDbProperties.getPassword());
hikariDataSource.setJdbcUrl(masterDbProperties.getUrl());
hikariDataSource.setDriverClassName(masterDbProperties.getDriverClassName());
hikariDataSource.setPoolName(masterDbProperties.getPoolName());
// HikariCP settings
hikariDataSource.setMaximumPoolSize(masterDbProperties.getMaxPoolSize());
hikariDataSource.setMinimumIdle(masterDbProperties.getMinIdle());
hikariDataSource.setConnectionTimeout(masterDbProperties.getConnectionTimeout());
hikariDataSource.setIdleTimeout(masterDbProperties.getIdleTimeout());
LOG.info("Setup of masterDataSource succeeded.");
return hikariDataSource;
}
#Primary
#Bean(name = "masterEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean masterEntityManagerFactory() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean ();
em.setDataSource(masterDataSource());
em.setPackagesToScan(new String[]{x,x,x,x...});
em.setPersistenceUnitName("masterdb-persistence-unit");
JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter ();
em.setJpaVendorAdapter(vendorAdapter);
em.setJpaProperties(hibernateProperties());
LOG.info("Setup of masterEntityManagerFactory succeeded.");
return em;
}
#Bean(name = "masterTransactionManager")
public JpaTransactionManager masterTransactionManager(#Qualifier("masterEntityManagerFactory") EntityManagerFactory emf) {
JpaTransactionManager transactionManager = new JpaTransactionManager ();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
#Bean
public PersistenceExceptionTranslationPostProcessor exceptionTranslation() {
return new PersistenceExceptionTranslationPostProcessor ();
}
private Properties hibernateProperties() {
Properties properties = new Properties();
properties.put(org.hibernate.cfg.Environment.DIALECT, "org.hibernate.dialect.MySQL5Dialect");
properties.put(org.hibernate.cfg.Environment.SHOW_SQL, true);
properties.put(org.hibernate.cfg.Environment.FORMAT_SQL, true);
properties.put(org.hibernate.cfg.Environment.HBM2DDL_AUTO, "none");
return properties;
}
Any Idea what might be the issue?
Thanks
I have resolved this issue by removing the #ConditionalOnBean annotation. Thanks for the people who responded.
the code just works fine in the local system, but application is failing with below error after docker run in aws ec2.
Could you make sure that you have tried mvn clean package and deployed the lastest artifacts to EC2?
Judging from stacktrace, there is a missing bean of name: entityManagerFactory. You need to refer to this bean with the correct name.
Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'entityManagerFactory' available
Parameter 0 of constructor in com.aic.autofluence.appservice.scheduler.service.Impl.ReportFactoryImplData required a bean named 'entityManagerFactory' that could not be found.
In your code ,
#Primary
#Bean(name = "masterEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean masterEntityManagerFactory() {
// ...
#Bean(name = "tenantEntityManagerFactory")
#ConditionalOnBean(name = "datasourceBasedMultitenantConnectionProvider")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
// ...
Multiple beans are of the same type LocalContainerEntityManagerFactoryBean and of different names: masterEntityManagerFactory and tenantEntityManagerFactory.
If you wish to reference the bean by type instead of by name, you cannot do it because you have multiple beans of the same type LocalContainerEntityManagerFactoryBean. To explicitly tell Spring which bean to inject, specify the bean name per your defination.
Based on your stack trace, you referenced the bean by the name entityManagerFactory in a constructor. To fix this, you need to change the argument name from entityManagerFactory to either masterEntityManagerFactory or tenantEntityManagerFactory.
Another way to specify which bean name you wish to inject, is to use #Qualifier("beanName"). Useful Guide
there are few answers to the question already. But none of them works for me.
I can't figure it out for the life of me why the error is coming.
Following are the approaches I tried:
using AbstractMongoConfiguration
Manually registering the mongoTemplate bean with ApplicationContext
Whenever I try to run my test during maven build or while deploying on tomcat error below comes up
Here is the configuration.
package com.fordit.project.config;
#Configuration
#EnableMongoRepositories(basePackages = "com.company.project")
#ComponentScan(basePackages = "com.company.project")
public class ProjectConfiguration {
#Value("${project.db.driver_class}")
private String driverClassname;
#Value("${project.db.connection_string}")
private String connectionString;
#Bean
public DataSource dataSource() throws PropertyVetoException {
Properties mysqlProperties = new Properties();
mysqlProperties.setProperty("characterEncoding", "UTF-8");
mysqlProperties.setProperty("useUnicode", "true");
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setProperties(mysqlProperties);
cpds.setDriverClass(driverClassname);
cpds.setJdbcUrl(connectionString);
cpds.setAcquireIncrement(2);
return cpds;
}
#Bean
public static PropertyPlaceholderConfigurer getPropertyPlaceholderConfigurer() throws IOException {
PropertyPlaceholderConfigurer ppc = new PropertyPlaceholderConfigurer();
ResourceLoader resourceLoader = new PathMatchingResourcePatternResolver();
ppc.setLocations(
resourceLoader.getResource(System.getProperty("PROJECT_CONFIGURATION_FILE")));
return ppc;
}
#Bean
public static RoleHierarchy roleHierarchy() {
String roleHierarchyStringRepresentation
= Role.ROLE_ADMIN + " > " + Role.ROLE_FIRM + "\n"
+ Role.ROLE_FIRM + " = " + Role.ROLE_LAWYER+ "= "+Role.ROLE_USER;
//logger.info("Registered Role Hierarchy: \n{}", roleHierarchyStringRepresentation);
RoleHierarchyImpl roleHierarchy = new RoleHierarchyImpl();
roleHierarchy.setHierarchy(roleHierarchyStringRepresentation);
return roleHierarchy;
}
}
Mongo Configuration:
#Configuration
#ComponentScan(basePackages = "com.company.project")
#Profile("container")
public class MongoDBConfiguration extends AbstractMongoConfiguration {
#Value("${project.mongodb.hostname}")
private String host;
#Value("${project.mongodb.port}")
private Integer port;
#Value("${project.mongodb.name}")
private String db;
#Value("${project.mongodb.username}")
private String username;
#Value("${project.mongodb.password}")
private String password;
#Value("${project.mongodb.authenticationDB}")
private String authDB;
#Bean
public MongoTemplate mongoTemplate()
throws UnknownHostException, java.net.UnknownHostException {
return new MongoTemplate(
new SimpleMongoDbFactory(
mongoClient(),
getDatabaseName()
)
);
}
#Override
#Bean
public MongoClient mongoClient() {
MongoClient mongoClient = null;
try {
mongoClient = new MongoClient(
new ServerAddress(host, port),
Collections.singletonList(
MongoCredential.createMongoCRCredential(
username,
authDB,
password.toCharArray()
)
)
);
} catch (java.net.UnknownHostException ex) {
Logger.getLogger(MongoDBConfiguration.class.getName()).log(Level.SEVERE, null, ex);
}
return mongoClient;
}
#Override
protected String getDatabaseName() {
return db;
}
}
Error log:
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'forumDAL' defined in file [/home/ashay/projects/kuber/target/classes/com/fordit/kuber/forum/ForumDAL.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'forumRepository': Cannot resolve reference to bean 'mongoTemplate' while setting bean property 'mongoOperations'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'mongoTemplate' available
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:729)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:192)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1270)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1127)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:541)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:501)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:760)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:869)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:128)
at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:60)
at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:107)
at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:243)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117)
... 26 more
Get rid of the Profile("container") in MongoDBConfiguration.
Explanation: Because the #Profile is there, that means that that class will not be instantiated by Spring unless you are running Spring with that profile. My guess is that you are not setting spring.profiles.active property to "container" when you run your application via Tomcat or during your integration testing.
If you want to leave the #Profile("container") there then just make sure you set the profile to "container". There are multiple ways to do this. One quick easy way is to use Java system properties, e.g. -Dspring.profiles.active=container, when you execute your integration tests or run your application in Tomcat.
I have completed a "happy-path" (as below).
How I can advise a .transform call to have it invoke an error flow (via errorChannel) w/o interrupting the mainFlow?
Currently the mainFlow terminates on first failure occurrence in second .transform (when payload cannot be deserialized to type). My desired behavior is that I'd like to log and continue processing.
I've read about ExpressionEvaluatingRequestHandlerAdvice. Would I just add a second param to each .transform call like e -> e.advice(myAdviceBean) and declare such a bean with success and error channels? Assuming I'd need to break up my mainFlow to receive success from each transform.
On some commented direction I updated the original code sample. But I'm still having trouble taking this "all the way home".
2015-09-08 11:49:19,664 [pool-3-thread-1] org.springframework.integration.handler.ServiceActivatingHandler DEBUG handler 'ServiceActivator for [org.springframework.integration.dsl.support.BeanNameMessageProcessor#5f3839ad] (org.springframework.integration.handler.ServiceActivatingHandler#0)' produced no reply for request Message: ErrorMessage [payload=org.springframework.integration.handler.advice.ExpressionEvaluatingRequestHandlerAdvice$MessageHandlingExpressionEvaluatingAdviceException: Handler Failed; nested exception is org.springframework.integration.transformer.MessageTransformationException: failed to transform message; nested exception is com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "hasDoaCostPriceChanged" (class com.xxx.changehistory.jdbc.data.RatePlanLevelRestrictionLog), not marked as ignorable (18 known properties: "supplierUpdateDate", "fPLOSMaskArrival", "createDate", "endAllowed", "sellStateId", "ratePlanLevel", "ratePlanId", "startAllowed", "stayDate", "doaCostPriceChanged", "hotelId", "logActionTypeId" [truncated]])
at [Source: java.util.zip.GZIPInputStream#242017b8; line: 1, column: 32] (through reference chain: com.xxx.changehistory.jdbc.data.RatePlanLevelRestrictionLog["hasDoaCostPriceChanged"]), headers={id=c054d976-5750-827f-8894-51aba9655c77, timestamp=1441738159660}]
2015-09-08 11:49:19,664 [pool-3-thread-1] org.springframework.integration.channel.DirectChannel DEBUG postSend (sent=true) on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.integration.handler.advice.ExpressionEvaluatingRequestHandlerAdvice$MessageHandlingExpressionEvaluatingAdviceException: Handler Failed; nested exception is org.springframework.integration.transformer.MessageTransformationException: failed to transform message; nested exception is com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "hasDoaCostPriceChanged" (class com.xxx.changehistory.jdbc.data.RatePlanLevelRestrictionLog), not marked as ignorable (18 known properties: "supplierUpdateDate", "fPLOSMaskArrival", "createDate", "endAllowed", "sellStateId", "ratePlanLevel", "ratePlanId", "startAllowed", "stayDate", "doaCostPriceChanged", "hotelId", "logActionTypeId" [truncated]])
at [Source: java.util.zip.GZIPInputStream#242017b8; line: 1, column: 32] (through reference chain: com.xxx.changehistory.jdbc.data.RatePlanLevelRestrictionLog["hasDoaCostPriceChanged"]), headers={id=c054d976-5750-827f-8894-51aba9655c77, timestamp=1441738159660}]
2015-09-08 11:49:19,664 [pool-3-thread-1] org.springframework.integration.channel.DirectChannel DEBUG preSend on channel 'mainFlow.channel#3', message: GenericMessage [payload=java.util.zip.GZIPInputStream#242017b8, headers={id=b80106f9-7f4c-1b92-6aca-6e73d3bf8792, timestamp=1441738159664}]
2015-09-08 11:49:19,664 [pool-3-thread-1] org.springframework.integration.aggregator.AggregatingMessageHandler DEBUG org.springframework.integration.aggregator.AggregatingMessageHandler#0 received message: GenericMessage [payload=java.util.zip.GZIPInputStream#242017b8, headers={id=b80106f9-7f4c-1b92-6aca-6e73d3bf8792, timestamp=1441738159664}]
2015-09-08 11:49:19,665 [pool-3-thread-1] org.springframework.integration.channel.DirectChannel DEBUG preSend on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.aggregator.AggregatingMessageHandler#0]; nested exception is java.lang.IllegalStateException: Null correlation not allowed. Maybe the CorrelationStrategy is failing?, headers={id=24e3a1c7-af6b-032c-6a29-b55031fba0d7, timestamp=1441738159665}]
2015-09-08 11:49:19,665 [pool-3-thread-1] org.springframework.integration.handler.ServiceActivatingHandler DEBUG ServiceActivator for [org.springframework.integration.dsl.support.BeanNameMessageProcessor#5f3839ad] (org.springframework.integration.handler.ServiceActivatingHandler#0) received message: ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.aggregator.AggregatingMessageHandler#0]; nested exception is java.lang.IllegalStateException: Null correlation not allowed. Maybe the CorrelationStrategy is failing?, headers={id=24e3a1c7-af6b-032c-6a29-b55031fba0d7, timestamp=1441738159665}]
2015-09-08 11:49:19,665 [pool-3-thread-1] com.xxx.DataMigrationModule$ErrorService ERROR org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.aggregator.AggregatingMessageHandler#0]; nested exception is java.lang.IllegalStateException: Null correlation not allowed. Maybe the CorrelationStrategy is failing?
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:84)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:287)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:245)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:95)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:231)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:154)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:102)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:105)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101)
at org.springframework.integration.dispatcher.UnicastingDispatcher.access$000(UnicastingDispatcher.java:48)
at org.springframework.integration.dispatcher.UnicastingDispatcher$1.run(UnicastingDispatcher.java:92)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Null correlation not allowed. Maybe the CorrelationStrategy is failing?
at org.springframework.util.Assert.state(Assert.java:385)
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.handleMessageInternal(AbstractCorrelatingMessageHandler.java:369)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
... 22 more
UPDATED (09-08-2015)
code sample
#Bean
public IntegrationFlow mainFlow() {
// #formatter:off
return IntegrationFlows
.from(
amazonS3InboundSynchronizationMessageSource(),
e -> e.poller(p -> p.trigger(this::nextExecutionTime))
)
.transform(unzipTransformer())
.split(f -> new FileSplitter())
.channel(MessageChannels.executor(Executors.newCachedThreadPool()))
.transform(Transformers.fromJson(persistentType()), , e -> e.advice(handlingAdvice()))
// #see http://docs.spring.io/spring-integration/reference/html/messaging-routing-chapter.html#agg-and-group-to
.aggregate(a ->
a.releaseStrategy(g -> g.size() == persistenceBatchSize)
.expireGroupsUponCompletion(true)
.sendPartialResultOnExpiry(true)
.groupTimeoutExpression("size() ge 2 ? 10000 : -1")
, null
)
.handle(jdbcRepositoryHandler())
// TODO add advised PollableChannel to deal with possible persistence issue and retry with partial batch
.get();
// #formatter:on
}
#Bean
public ErrorService errorService() {
return new ErrorService();
}
#Bean
public MessageChannel customErrorChannel() {
return MessageChannels.direct().get();
}
#Bean
public IntegrationFlow customErrorFlow() {
// #formatter:off
return IntegrationFlows
.from(customErrorChannel())
.handle("errorService", "handleError")
.get();
// #formatter:on
}
#Bean
ExpressionEvaluatingRequestHandlerAdvice handlingAdvice() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setOnFailureExpression("payload");
advice.setFailureChannel(customErrorChannel());
advice.setReturnFailureExpressionResult(true);
advice.setTrapException(true);
return advice;
}
protected class ErrorService implements ErrorHandler {
private final Logger log = LoggerFactory.getLogger(getClass());
#Override
public void handleError(Throwable t) {
stopEndpoints(t);
}
private void stopEndpoints(Throwable t) {
log.error(ExceptionUtils.getStackTrace(t));
}
}
Turns out I had things wrong in a few places, like:
I had to autowire a Jackson2 ObjectMapper (that I get from Sprint Boot auto-config) and construct an instance of JsonObjectMapper to be added as second arg in Transformers.fromJson; made for more lenient unmarshalling to persistent type (stops UnrecognizedPropertyException); and thus waived need for ExpressionEvaluatingRequestHandlerAdvice
Choosing the proper variant of .split method in IntegrationFlowDefinition in order to employ the FileSplitter, otherwise you don't get this splitter rather a DefaultMessageSplitter which pre-maturely terminates flow after first record read from InputStream
Moved transform, aggregate, handle to a its own pubsub channel employing an async task executor
Still not 100% of what I need, but it's much further along.
See what I ended up w/ below...
#Configuration
#EnableIntegration
#IntegrationComponentScan
public class DataMigrationModule {
private final Logger log = LoggerFactory.getLogger(getClass());
#Value("${cloud.aws.credentials.accessKey}")
private String accessKey;
#Value("${cloud.aws.credentials.secretKey}")
private String secretKey;
#Value("${cloud.aws.s3.bucket}")
private String bucket;
#Value("${cloud.aws.s3.max-objects-per-batch:1024}")
private int maxObjectsPerBatch;
#Value("${cloud.aws.s3.accept-subfolders:false}")
private String acceptSubFolders;
#Value("${cloud.aws.s3.remote-directory}")
private String remoteDirectory;
#Value("${cloud.aws.s3.local-directory-ref:java.io.tmpdir}")
private String localDirectoryRef;
#Value("${cloud.aws.s3.local-subdirectory:target/s3-dump}")
private String localSubdirectory;
#Value("${cloud.aws.s3.filename-wildcard:}")
private String fileNameWildcard;
#Value("${app.persistent-type:}")
private String persistentType;
#Value("${app.repository-type:}")
private String repositoryType;
#Value("${app.persistence-batch-size:2500}")
private int persistenceBatchSize;
#Value("${app.persistence-batch-release-timeout-in-milliseconds:5000}")
private int persistenceBatchReleaseTimeoutMillis;
#Autowired
private ListableBeanFactory beanFactory;
#Autowired
private ObjectMapper objectMapper;
private final AtomicBoolean invoked = new AtomicBoolean();
private Class<?> repositoryType() {
try {
return Class.forName(repositoryType);
} catch (ClassNotFoundException cnfe) {
log.error("Unknown repository implementation!", cnfe);
System.exit(0);
}
return null;
}
private Class<?> persistentType() {
try {
return Class.forName(persistentType);
} catch (ClassNotFoundException cnfe) {
log.error("Unsupported type!", cnfe);
System.exit(0);
}
return null;
}
public Date nextExecutionTime(TriggerContext triggerContext) {
return this.invoked.getAndSet(true) ? null : new Date();
}
#Bean
public FileToInputStreamTransformer unzipTransformer() {
FileToInputStreamTransformer transformer = new FileToInputStreamTransformer();
transformer.setDeleteFiles(true);
return transformer;
}
#Bean
public MessageSource<?> amazonS3InboundSynchronizationMessageSource() {
AWSCredentials credentials = new BasicAWSCredentials(this.accessKey, this.secretKey);
AmazonS3InboundSynchronizationMessageSource messageSource = new AmazonS3InboundSynchronizationMessageSource();
messageSource.setCredentials(credentials);
messageSource.setBucket(bucket);
messageSource.setMaxObjectsPerBatch(maxObjectsPerBatch);
messageSource.setAcceptSubFolders(Boolean.valueOf(acceptSubFolders));
messageSource.setRemoteDirectory(remoteDirectory);
if (!fileNameWildcard.isEmpty()) {
messageSource.setFileNameWildcard(fileNameWildcard);
}
String directory = System.getProperty(localDirectoryRef);
if (!localSubdirectory.startsWith("/")) {
localSubdirectory = "/" + localSubdirectory;
}
if (!localSubdirectory.endsWith("/")) {
localSubdirectory = localSubdirectory + "/";
}
directory = directory + localSubdirectory;
FileUtils.mkdir(directory);
messageSource.setDirectory(new LiteralExpression(directory));
return messageSource;
}
#Bean
public IntegrationFlow mainFlow() {
// #formatter:off
return IntegrationFlows
.from(
amazonS3InboundSynchronizationMessageSource(),
e -> e.poller(p -> p.trigger(this::nextExecutionTime))
)
.transform(unzipTransformer())
.split(new FileSplitter(), null)
.publishSubscribeChannel(new SimpleAsyncTaskExecutor(), p -> p.subscribe(persistenceSubFlow()))
.get();
// #formatter:on
}
#Bean
public IntegrationFlow persistenceSubFlow() {
JsonObjectMapper<?, ?> jsonObjectMapper = new Jackson2JsonObjectMapper(objectMapper);
ReleaseStrategy releaseStrategy = new TimeoutCountSequenceSizeReleaseStrategy(persistenceBatchSize,
persistenceBatchReleaseTimeoutMillis);
// #formatter:off
return f -> f
.transform(Transformers.fromJson(persistentType(), jsonObjectMapper))
// #see http://docs.spring.io/spring-integration/reference/html/messaging-routing-chapter.html#agg-and-group-to
.aggregate(
a -> a
.releaseStrategy(releaseStrategy)
.correlationStrategy(m -> m.getHeaders().get("id"))
.expireGroupsUponCompletion(true)
.sendPartialResultOnExpiry(true)
, null
)
.handle(jdbcRepositoryHandler());
// #formatter:on
}
#Bean
public JdbcRepositoryHandler jdbcRepositoryHandler() {
return new JdbcRepositoryHandler(repositoryType(), beanFactory);
}
protected class JdbcRepositoryHandler extends AbstractMessageHandler {
#SuppressWarnings("rawtypes")
private Insertable repository;
public JdbcRepositoryHandler(Class<?> repositoryClass, ListableBeanFactory beanFactory) {
repository = (Insertable<?>) beanFactory.getBean(repositoryClass);
}
#Override
protected void handleMessageInternal(Message<?> message) {
repository.insert((List<?>) message.getPayload());
}
}
protected class FileToInputStreamTransformer extends AbstractFilePayloadTransformer<InputStream> {
#Override
protected InputStream transformFile(File payload) throws Exception {
return new GZIPInputStream(new FileInputStream(payload));
}
}
}
Yes, you are correct. To advice the handle() method of Transformer's MessageHandler you should use exactly that e.advice method of the second parameter of .transform() EIP-method. And yes: you should define ExpressionEvaluatingRequestHandlerAdvice bean for your purpose.
You can reuse that Advice bean for different goals to handle successes and failures the same manner.
UPDATE
Although it isn't clear to me how you'd like to continue the flow with the wrong message, but you you can use onFailureExpression and returnFailureExpressionResult=true of the ExpressionEvaluatingRequestHandlerAdvice to return something after the unzipErrorChannel().
BTW the failureChannel logic doesn't work without onFailureExpression:
if (this.onFailureExpression != null) {
Object evalResult = this.evaluateFailureExpression(message, actualException);
if (this.returnFailureExpressionResult) {
return evalResult;
}
}