I have an FTP Streaming Inbound Channel Adapter from Spring Integration which produces message with payloads of type InputStream, letting files be fetched without writing to the local file system.
#Bean
#InboundChannelAdapter(channel = Constants.CHANNEL_INBOUND_FTP_ADAPTER, poller = #Poller(fixedDelay = Constants.FIXED_POLLING_FROM_INBOUND_FTP_ADAPTER_DELAY))
public MessageSource<InputStream> ftpMessageSource() {
FtpStreamingMessageSource ftpStreamingMessageSource = new FtpStreamingMessageSource(ftpRemoteFileTemplate());
ftpStreamingMessageSource.setRemoteDirectory(ftpConnectionParameters.getRootDir());
ftpStreamingMessageSource.setFilter(chainFileListFilter());
ftpStreamingMessageSource.setMaxFetchSize(Constants.INBOUND_ADAPTER_MAX_FETCH_SIZE);
return ftpStreamingMessageSource;
}
After I transform with
#Bean
#org.springframework.integration.annotation.Transformer(inputChannel = Constants.CHANNEL_INBOUND_FTP_ADAPTER, outputChannel = Constants.CHANNEL_STREAMED_DATA)
public Transformer transformer() {
return new StreamTransformer(Charset.defaultCharset().name());
}
Then handle data to check it works and maybe for custom inteceptors for future:
#ServiceActivator(inputChannel = Constants.CHANNEL_STREAMED_DATA, outputChannel = "BATCH_ALARM_CHANNEL")
public Message<?> alarmHandler(Message<?> message) {
System.out.println(Constants.CHANNEL_ALARM);
System.out.println(message.getHeaders());
return message;
}
After this according to official Spring Batch Integration documentation I have one more Transformer which let us transform to JobLaunchRequest
#Transformer
public JobLaunchRequest toRequest(Message<File> message) {
JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();
jobParametersBuilder.addDate("dummy", new Date());
return new JobLaunchRequest(job, jobParametersBuilder.toJobParameters());
}
Here we have Message from last BATCH_ALARM_CHANNEL which needs in Spring Batch Jobs, but JobParametersBuilder doesn't allow to put complex object only primitive types. So how I can pass message payload for JobLaunching and do the rest of the work such as read, parse and save to DB?
Related
I want to create two beans of PubSubTemplate class to set different message converter.
I am having two subscriber among them one is receiving Json response and another one is receiving String response. To handle these two scenario I am creating two PubSubTemplate bean.
Below is my PubSubTemplateConfig.java :
#Configuration
public class PubSubTemplateConfig {
#Bean
public PubSubTemplate pubSubTemplateForUserCreation(PubSubPublisherTemplate pubSubPublisherTemplate,
PubSubSubscriberTemplate pubSubSubscriberTemplate) {
PubSubTemplate template = new PubSubTemplate(pubSubPublisherTemplate, pubSubSubscriberTemplate);
template.setMessageConverter(new JacksonPubSubMessageConverter(getObjectMapper()));
return template;
}
private ObjectMapper getObjectMapper() {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.registerModule(new JavaTimeModule());
return objectMapper;
}
#Bean
public PubSubTemplate pubSubTemplateForAuditTracker(PubSubPublisherTemplate pubSubPublisherTemplate,
PubSubSubscriberTemplate pubSubSubscriberTemplate) {
PubSubTemplate template = new PubSubTemplate(pubSubPublisherTemplate, pubSubSubscriberTemplate);
template.setMessageConverter(new SimplePubSubMessageConverter());
return template;
}
}
Below two are the subscriber configuration :
AuditsubscriptioncriptionConfiguration.java
#Configuration
public class AuditsubscriptioncriptionConfiguration {
#Value("${subscriptioncription.auditsubscriptioncription}")
private String subscription;
#Bean("pubsubAuditInputChannel")
public MessageChannel pubsubAuditInputChannel() {
return new DirectChannel();
}
#Bean
public PubSubInboundChannelAdapter auditMessageChannelAdapter(#Qualifier("pubsubAuditInputChannel") MessageChannel pubsubAuditInputChannel,
#Qualifier("pubSubTemplateForAuditTracker") PubSubTemplate pubSubTemplateForAuditTracker) {
PubSubInboundChannelAdapter adapter = new PubSubInboundChannelAdapter(pubSubTemplateForAuditTracker, subscription);
adapter.setOutputChannel(pubsubAuditInputChannel);
adapter.setPayloadType(String.class); //need changes
adapter.setAckMode(AckMode.MANUAL);
return adapter;
}
}
And UserSubscriptionConfiguration.java
#Configuration
public class UserSubscriptionConfiguration {
#Value("${subscription.userSubscriber}")
private String subscriber;
#Bean("pubsubInputChannel")
public MessageChannel pubsubInputChannel() {
return new DirectChannel();
}
#Bean
public PubSubInboundChannelAdapter userMessageChannelAdapter(#Qualifier("pubsubInputChannel") MessageChannel pubsubInputChannel,
#Qualifier("pubSubTemplateForUserCreation") PubSubTemplate pubSubTemplateForUserCreation) {
PubSubInboundChannelAdapter adapter = new PubSubInboundChannelAdapter(pubSubTemplateForUserCreation, subscriber);
adapter.setOutputChannel(pubsubInputChannel);
adapter.setPayloadType(UserChangeEvent.class);
adapter.setAckMode(AckMode.MANUAL);
return adapter;
}
}
Steps I observed during container start up :
Step 1. First pubSubTemplateForAuditTracker bean is getting created with SimplePubSubMessageConverter and then AuditMessageChannelAdapter bean is getting configured
Step 2. pubSubTemplateForUserCreation bean is getting created with JacksonPubSubMessageConverter and userMessageChannelAdapter is getting configured
Here, I should have two beans with two different message converter but while debugging I found that only one instance of PubSubTemplate is present and the attached message converter is JacksonPubSubMessageConverter. pubSubTemplateForAuditTracker bean is getting overridden with pubSubTemplateForUserCreation bean though I had defined them twice with #Bean annotation. This behavior is leading to an Error when auditMessageChannelAdapter is receiving a String message
My expectation is I want to have two separate PubSubTemplate bean with two different Message Converter.
Basically I want to create two beans of type PubSubTemplate with different behaviour.
Can someone please help me here.
I am exploring GCP pub/sub for the first time. Thank you
You can manually create multiple subscriptions like this
#PostConstruct
private void subscribeWithConcurrencyControl() {
// create subscription
TopicName topic = TopicName.ofProjectTopicName(projectId, this.eventTopic);
Subscription subscription = Subscription.newBuilder()
.setName("projects/XYZ/subscriptions/" + eventSubscription)
.setTopic(topic.toString())
.setPushConfig(PushConfig.getDefaultInstance())
.setAckDeadlineSeconds(100)
.build();
Subscription subscription2 = Subscription.newBuilder()
.setName("projects/XYZ/subscriptions/" + eventSubscription2)
.setTopic(topic.toString())
.setPushConfig(PushConfig.getDefaultInstance())
.setAckDeadlineSeconds(100)
.build();
try {
client.createSubscription(subscription);
} catch (AlreadyExistsException e) {
// nothing to do
}
try {
client.createSubscription(subscription2);
} catch (AlreadyExistsException e) {
// nothing to do
}
ProjectSubscriptionName subscriptionName1 = ProjectSubscriptionName.of(projectId, eventSubscription);
ProjectSubscriptionName subscriptionName2 = ProjectSubscriptionName.of(projectId, eventSubscription2);
// Instantiate an asynchronous message receiver.
MessageReceiver receiver = (PubsubMessage message, AckReplyConsumer consumer) -> {
// Handle incoming message, then ack the received message.
try {
process(message);
consumer.ack();
} catch (Exception e) {
LOG.error("Failed to process message", e);
consumer.nack();
}
};
// Provides an executor service for processing messages. The default `executorProvider` used
// by the subscriber has a default thread count of 5.
ExecutorProvider executorProvider =
InstantiatingExecutorProvider.newBuilder().setExecutorThreadCount(2).build();
FlowControlSettings flowControlSettings =
FlowControlSettings.newBuilder()
.setMaxOutstandingElementCount(100L)
.build();
// `setParallelPullCount` determines how many StreamingPull streams the subscriber will open
// to receive message. It defaults to 1. `setExecutorProvider` configures an executor for the
// subscriber to process messages. Here, the subscriber is configured to open 2 streams for
// receiving messages, each stream creates a new executor with 4 threads to help process the
// message callbacks. In total 2x4=8 threads are used for message processing.
subscriber1 = Subscriber.newBuilder(subscriptionName1, receiver)
.setParallelPullCount(20)
.setExecutorProvider(executorProvider)
.setCredentialsProvider(credentialsProvider)
.setFlowControlSettings(flowControlSettings)
.build();
subscriber2 = Subscriber.newBuilder(subscriptionName2, receiver)
.setParallelPullCount(20)
.setExecutorProvider(executorProvider)
.setCredentialsProvider(credentialsProvider)
.setFlowControlSettings(flowControlSettings)
.build();
// Start the subscriber.
subscriber1.startAsync().awaitRunning();
subscriber2.startAsync().awaitRunning();
}
In the process() method you can use the usual objectmapper and/or instanceof commands to determine the type of the message (or have different receivers for different subscriptions, or even transport the type of the message in the pubsub headers)
private void process(PubsubMessage message) {
try {
ModificationEvent modificationEvent = objectMapper.readValue(message.getData().toStringUtf8(), ModificationEvent.class);
} catch(...)
I have implemented the example as shown here Spring Dynamic Destination
In the rabbitmq, it is creating an exchange dynamically, but there is no option to provide binding or routing key. My requirement is to send a message to this dynamically created exchange with a routing key. How would i need to implement this to setup the routing key?
#Component
public class DDProducerBean {
#Autowired
private BinderAwareChannelResolver poChannelResolver = null;
public void publish(DDSocketVO ddSocketVO) throws Exception {
this.poChannelResolver.resolveDestination(ddSocketVO.getDestination()).send(MessageBuilder.withPayload(new ObjectMapper().
setVisibility(PropertyAccessor.FIELD, Visibility.ANY).
writeValueAsString(ddSocketVO)).build());
}
}
Here is the workaround as suggested Here
Basically create a MessageChannel with the dynamic destination using BinderAwareChannelResolver, then connect to RabbitMQ with RabbitAdmin API and bind the newly created exchange to another queue or exchange with routing key before sending messages.
#Autowired
private BinderAwareChannelResolver poChannelResolver;
public void publish(WebSocketVO webSocketVO) throws Exception {
MessageChannel channel = this.poChannelResolver.resolveDestination(webSocketVO.getDestination());
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setUsername(System.getProperty("spring.cloud.stream.binders.corerabbit.environment.spring.rabbitmq.username"));
connectionFactory.setPassword(System.getProperty("spring.cloud.stream.binders.corerabbit.environment.spring.rabbitmq.password"));
connectionFactory.setAddresses(System.getProperty("spring.cloud.stream.binders.corerabbit.environment.spring.rabbitmq.addresses"));
connectionFactory.setVirtualHost(System.getProperty("spring.cloud.stream.binders.corerabbit.environment.spring.rabbitmq.virtual-host"));
AmqpAdmin amqpAdmin = new RabbitAdmin(connectionFactory);
TopicExchange sourceExchange = new TopicExchange(webSocketVO.getDestination(), false, true);
TopicExchange destExchange = new TopicExchange("amq.topic");
amqpAdmin.declareBinding(BindingBuilder.bind(destExchange).to(sourceExchange).with(webSocketVO.getRoutingKeyExpression()));
channel.send(MessageBuilder.withPayload(new ObjectMapper().
setVisibility(PropertyAccessor.FIELD, Visibility.ANY).
writeValueAsString(webSocketVO)).build());
amqpAdmin.deleteExchange(webSocketVO.getDestination());
connectionFactory.destroy();
}
I was expecting #JmsListener will automatically convert json object into my object, but its payload is returning the data as string in json format rather than actual object.
#JmsListener(destination = "${default-queue-name-to-listen}")
public void receiveMessage(final Message<MyObject> message) throws JMSException {
logger.info("message received from the queue/topic : {}", message);
MyObject response = message.getPayload();
}
But we receive response in a String format like this: {"id":"1","name":"2222"}
And this is failing it at runtime.
My other piece of code is:
#Bean
public JmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory) throws URLSyntaxException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setSubscriptionDurable(true);
factory.setPubSubDomain(true);
return factory;
}
It depends on what the producer sent.
If a TextMessage was sent, you get a String. If an ObjectMessage sent, you can get an Object.
I am trying to implement internationalization in my application. I already went through many blogs & tutorials which explain how we can implement it using different libraries.
The one I am planning to use is I18N with spring.
My application's structure is something like this :-
My application's front end (based on Angular2) consumes Rest APIs that are exposed from the backend.
I am using Spring Rest for implementing the Rest APIs. For every API call I am preparing & sending appropriate messages to UI.
Now by default messages are in English but now I want to add internationalization support to it. How can I do it ?
Below is the example of one of the Rest API that I am exposing and the way I'm sending the messages :-
#CrossOrigin(methods = RequestMethod.POST)
#PostMapping(value = "/user/resetUserAccount", produces = MediaType.APPLICATION_JSON_VALUE)
public #ResponseBody ResponseEntity<String> resetUserAccount(#RequestBody InputObj inputObj) {
boolean isUserAccountReset = userService.resetUserAccount(inputObj);
if (isUserAccountReset) {
return new ResponseEntity<String>(successResponse("User Account Reset Successful").toString(), HttpStatus.OK);
}
return new ResponseEntity<String>(failureResponse("Failed to Reset User Account").toString(), HttpStatus.CONFLICT);
}
I have written 2 helper methods given below that prepare the response messages :-
private JSONObject successResponse(String apiMessage) {
JSONObject success = new JSONObject();
success.put("reponse", "success");
success.put("message", apiMessage);
return success;
}
private JSONObject failureResponse(String apiMessage) {
JSONObject failure= new JSONObject();
success.put("reponse", "failure");
success.put("message", apiMessage);
return failure;
}
Add the following to the configuration class
#Bean
public LocaleResolver localeResolver() {
SessionLocaleResolver slr = new SessionLocaleResolver();
slr.setDefaultLocale(Locale.US); // Set default Locale as US
return slr;
}
#Bean
public ResourceBundleMessageSource messageSource() {
ResourceBundleMessageSource source = new ResourceBundleMessageSource();
source.setBasenames("i18n/messages"); // name of the resource bundle
source.setUseCodeAsDefaultMessage(true);
return source;
}
Create a new directory named i18n inside resources directory and put your messages.properties and the other internationalized property files like messages_ru.properties, messages_fr.properties etc inside it. Create message key and values like below:
messages.properties
msg.success=User Account Reset Successful
msg.failure=Failed to Reset User Account
Now inject the MessageSource Bean where you want to internationalize the message, i.e. your controller and then accept the Locale from headers in controller method and get messages from properties files like below:
#Autowired
private MessageSource messageSource;
#CrossOrigin(methods = RequestMethod.POST)
#PostMapping(value = "/user/resetUserAccount", produces = MediaType.APPLICATION_JSON_VALUE)
public #ResponseBody ResponseEntity<String> resetUserAccount(#RequestHeader("Accept-Language") Locale locale, #RequestBody InputObj inputObj) {
boolean isUserAccountReset = userService.resetUserAccount(inputObj);
if (isUserAccountReset) {
return new ResponseEntity<String>(successResponse(messageSource.getMessage("msg.success",null,locale)).toString(), HttpStatus.OK);
}
return new ResponseEntity<String>(failureResponse(messageSource.getMessage("msg.failure",null,locale)).toString(), HttpStatus.CONFLICT);
}
Currently I have an application with Spring Integration DSL that has AMQP inbound gateway with different service activators, each service activator has kind of logic to decide, transform and call external web services (currently with CXF), but all this logic is in code without Spring Integration components.
These service activators are monitored, in the output channel that returns data from this application is an AMQP adapter that sends headers to a queue (after that, all headers are processed and saved in a database for future analysis). This works well, these service activators even have elapsed time in headers.
Now, the problem is, that I need to monitor external web service calls, like elapsed time in each operation, which service endpoint and operation was called, if an error occurred.
I've been thinking that logic code in each service activator should be converted into a Spring Integration flow, in each service activator, would call a new gateway with the name of the operation of the web service in a header, and monitoring every flow as currently I had been doing.
So, I'm not sure if this manual approach is the better approach, so I wonder if there is a way to get the name of the service operation with some kind of interceptor or something similar with CXF or Spring WS to avoid setting the name of the operation in headers in a manual way? What would be your recommendation?
To have more context here is the Spring Integration configuration:
#Bean
public IntegrationFlow inboundFlow() {
return IntegrationFlows.from(Amqp.inboundGateway(simpleMessageListenerContainer())
.mappedReplyHeaders(AMQPConstants.AMQP_CUSTOM_HEADER_FIELD_NAME_MATCH_PATTERN)
.mappedRequestHeaders(AMQPConstants.AMQP_CUSTOM_HEADER_FIELD_NAME_MATCH_PATTERN)
.errorChannel(gatewayErrorChannel())
.requestChannel(gatewayRequestChannel())
.replyChannel(gatewayResponseChannel())
)
.enrichHeaders(new Consumer<HeaderEnricherSpec>() {
#Override
public void accept(HeaderEnricherSpec t) {
t.headerExpression(AMQPConstants.START_TIMESTAMP, "T(java.lang.System).currentTimeMillis()");
}
})
.transform(getCustomFromJsonTransformer())
.route(new HeaderValueRouter(AMQPConstants.OPERATION_ROUTING_KEY))
.get();
}
#Bean
public MessageChannel gatewayRequestChannel() {
return MessageChannels.publishSubscribe().get();
}
#Bean
public MessageChannel gatewayResponseChannel() {
return MessageChannels.publishSubscribe().get();
}
private IntegrationFlow loggerOutboundFlowTemplate(MessageChannel fromMessageChannel) {
return IntegrationFlows.from(fromMessageChannel)
.handle(Amqp.outboundAdapter(new RabbitTemplate(getConnectionFactory()))
.exchangeName(LOGGER_EXCHANGE_NAME)
.routingKey(LOGGER_EXCHANGE_ROUTING_KEY)
.mappedRequestHeaders("*"))
.get();
}
And here is a typical service activator, as you can see, all this logic could be an integration flow:
#ServiceActivator(inputChannel="myServiceActivator", outputChannel = ConfigurationBase.MAP_RESPONSE_CHANNEL_NAME)
public Message<Map<String, Object>> myServiceActivator(Map<String, Object> input, #Header(AMQPConstants.SESSION) UserSession session) throws MyException {
Message<Map<String, Object>> result = null;
Map<String, Object> mapReturn = null;
ExternalService port = serviceConnection.getExternalService();
try {
if (input.containsKey(MappingConstants.TYPE)) {
Request request = transformer
.transformRequest(input, session);
Response response = port
.getSomething(request);
utils.processBackendCommonErrors(response.getCode(), response.getResponse());
mapReturn = transformer.convertToMap(response);
} else {
Request request = transformer
.transformRequest(input, session);
Response response = port
.getSomethingElse(request);
utils.processBackendCommonErrors(response.getCode(),
response.getResponse());
mapReturn = transformer.convertToMap(response);
}
} catch (RuntimeException e) {
String message = "unexcepted exception from the back-end";
logger.warn(message, e);
throw MyException.generateTechnicalException(message, null, e);
}
result = MessageBuilder.withPayload(mapReturn)
.build();
return result;
}
So far so good. Or I don't understand the problem, or you are not clear where it is.
Anyway you always can proxy any Spring Service with the AOP, since it looks like you are pointing to the code:
Response response = port
.getSomething(request);
When this (or similar) method is called, some MethodInterceptor can perform desired tracing logic and send result to some MessageChannel for further analysis or anything else to do:
public Object invoke(MethodInvocation invocation) throws Throwable {
// Extract required operation name and start_date from the MethodInvocation
Object result = invocation.proceed();
// Extract required data from the response
// Build message and send to the channel
return result;
}