I have created DirectChannel and have sent an object to my channel and want to receive it abck to store it in DB and send it in another service bus queue. Can you suggest how to receive the object from channel?
My Channel -
#Bean("tempChannel")
public MessageChannel tempChannel() {
return new DirectChannel();
}
Integration flow -
#Bean
public IntegrationFlow tempMessageFlow() {
return IntegrationFlows.from("tempChannel").handle().get();
}
For handle method I need to pass MessageHandler, how to I declare it and pass here?
I am sending message to channel using below piece of code, please do tell if this is alright-
tempChannel().send(messageObj);
The DirectChannel implements a SubscribableChannel. So, to get messages sent to this channel you need to subscribe(MessageHandler handler). What you have so far with that IntegrationFlow definition is OK: adding that handle() you subscribe to the tempChannel. Just handle message and forget you can do this:
.handle(m - > System.out.println("Processed message: " + m))
This is a lambda for that MessageHandler functional interface. There are many other handle() variants for other use-cases. For example process-n-reply is like this:
.handle((p, h) - > {
System.out.println("Processed message: " + m);
return "My new payload";
})
If you say that you need to do several operations on the same message, then look into a PublishSubscribeChannel. In Java DSL we have a publishSubscribeChannel(Consumer<PublishSubscribeSpec> publishSubscribeChannelConfigurer) to configure several subscribers as sub-flows.
To store into DB, you can use a JdbcMessageHandler: https://docs.spring.io/spring-integration/docs/current/reference/html/jdbc.html#jdbc-outbound-channel-adapter
Related
I have been trying to send messages to external systems(using rest template POST, PUT etc) from the service activators as below.
Below is my pubsub consumer class
public class MyConsumer{
#Autowired
ExternalService externalService;
#Bean
public PubSubInboundChannelAdapter messageChannelAdapter(final #Qualifier("myInputChannel") MessageChannel inputChannel,
PubSubTemplate pubSubTemplate)
{
PubSubInboundChannelAdapter adapter = new PubSubInboundChannelAdapter(pubSubTemplate, pubSubSubscriptionName);
adapter.setOutputChannel(inputChannel);
adapter.setAckMode(AckMode.AUTO_ACK);
adapter.setErrorChannelName("pubsubErrors");
return adapter;
}
#ServiceActivator(inputChannel = "pubsubErrors")
public void pubsubErrorHandler(Message<MessagingException> exceptionMessage) {
BasicAcknowledgeablePubsubMessage originalMessage = (BasicAcknowledgeablePubsubMessage) exceptionMessage
.getPayload().getFailedMessage().getHeaders().get(GcpPubSubHeaders.ORIGINAL_MESSAGE);
originalMessage.nack();
}
#Bean
public MessageChannel myInputChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "myInputChannel")
public MessageHandler messageReceiver_AddCustomer() {
return message -> {
externalService.postDataTOExternalSystems(new String((byte[]) message.getPayload());
};
}
#Bean
#ServiceActivator(inputChannel = "myInputChannel")
public MessageHandler messageReceiver_DeleteCustomer() {
return message -> {
externalService.deleteCustomer(new String((byte[]) message.getPayload());
BasicAcknowledgeablePubsubMessage originalMessage =
message.getHeaders().get(GcpPubSubHeaders.ORIGINAL_MESSAGE, BasicAcknowledgeablePubsubMessage.class);
originalMessage.ack();
};
}
}
ExternalService below is the service which sends data to the external systems.
public class ExternalService{
void postDataTOExternalSystems(Object obj){
// RequestEntity object formed with HttpEntity object using obj(in json) and headers
restTemplate.exchange("https://externalsystems/",HttpMethod.POST,requestEntity,Object.class);
}
void deleteDatafromExternalSystems(Object obj){
// RequestEntity object formed with HttpEntity object using obj(in json) and headers
restTemplate.exchange("https://externalsystems/",HttpMethod.Detele,requestEntity,Object.class);
}
}
Since both the methods messageReceiver_AddCustomer and messageReceiver_deleteCustomer are using same channel whats happening is when I try to just addcustomer, the deleteCustomer is also called by default.
I was thinking of creating a seperate channel for deleteCustomer, but creating in this way leads to creating channels for every usecase.
Hence would like to know three things here.
Is there is any other approach of sending through Spring integration through which I can send data to external systems using a single Channel or a different utilization of Channels.
If any error in the external service calls leads to unending of failure logs in the console
message_id: "6830962001745961"
publish_time {
seconds: 1675783352
nanos: 547000000
}
}, timestamp=1675783353720}]': error occurred in message handler
It's not clear what is your expectation for such a logic. You have two contradicting subscribers for the same input channel. It sounds more like you need a router first to determine where to proceed next with an input message from Pub/Sub: https://docs.spring.io/spring-integration/reference/html/message-routing.html#messaging-routing-chapter.
but creating in this way leads to creating channels for every usecase.
Sure! You can go without Spring Integration and just do everything with the plain if..else. So, what's a difference? Therefore I don't see a reasonable argument in your statement. Yo have different HTTP methods, and that is OK to map them via Spring Integration router to respective sub-flows. The MessageChannel is really first-class citizen in Spring Integration. It makes a solution as loosely-coupled as possible.
Several subscribers on your current DirectChannel will lead to a round-robin logic by default, so you'll be surprised that one message from Pub/Sub creates a customer, another deletes and so on in rounds. The PublishSubscribeChannel will make it bad as well: both of your subscribers are going to be called, so created first, then deleted immediately.
I am starting with Saga pattern using Spring cloud and rabbit mq. Following is the problem statement:
I call /service1 (producer) which publishes a message in rabbit mq and that message is consumed by the consumer service.
Now occurs tow cases:
Case 1: Consumer service does its part successfully.
Case 2: Consumer service fails to do its part, thus /service1 has to rollback its changes.
How does /service1 know if consumer is successful or not, so that it can send a success/failure response. Following is the project structure:
Producer:
#RestController
public class ProducerController {
private MessageChannel greet;
public ProducerController(HelloBinding binding) {
greet = binding.greeting();
}
#GetMapping("/greet/{name}")
public void publish(#PathVariable String name) {
String greeting = "Hello, "+name+"!";
Message<String> msg = MessageBuilder.withPayload(greeting)
.build();
this.greet.send(msg);
System.out.println("Message sent to the queue");
AMQP.Basic.Ack;
}
Consumer:
#EnableBinding(HelloBinding.class)
public class HelloListener {
#StreamListener(target=HelloBinding.GREETING)
public void processHelloChannelGreeting(String msg) {
System.out.println("Message received:- "+msg);
}
}
Now how do I tell the producer whether consumer's action is a success or a failure so that producer service sends appropriate response?
The producer can not know what happens after a message has been successfully published to a topic. If you want feedback from the consumer then you need to create a new "response" topic on which the consumer communicates success or failure of processing that message.
You can map the messages by keys.
Is it possible to send a return value of any method to a queue using an annotation, like
#SentTo("my.queue.name")
String send() {
return myString;
}
Do I definitely need a #RabbitListener to use #SendTo? Maybe another way out?
I'm trying to simplify my code.
#SendTo is only currently for replies from a #RabbitListener where the sender didn't set a replyTo header.
You could do what you want with a Spring Integration #Publisher annotation with its channel wired to a rabbitmq outbound channel adapter...
#Publisher(channel = "amqpOutboundChannel")
public String send() {
return myString;
}
#Bean
#ServiceActivator(inputChannel = "amqpOutboundChannel")
public AmqpOutboundEndpoint amqpOutbound(AmqpTemplate amqpTemplate) {
AmqpOutboundEndpoint outbound = new AmqpOutboundEndpoint(amqpTemplate);
outbound.setRoutingKey("my.queue.name"); // default exchange - route to queue 'my.queue.name'
return outbound;
}
The method has to be public and invoked from outside the bean itself.
I have the following integration configuration in my web app:
#Bean
IntegrationFlow giraFlow() {
return IntegrationFlows.from(
MessageChannels.direct("gira.input"))
.split()
.transform(transformer)
.handle(parserService)
.channel(routerChannel())
.get();
}
#Bean
MessageChannel routerChannel() {
return MessageChannels.queue("routerChannel", 10)
.get();
}
#Bean
IntegrationFlow routerChannelFlow() {
return IntegrationFlows.from(
routerChannel())
.route(p -> p.getKind().name(),
m -> m.suffix("Channel")
.channelMapping(TaskKind.CREATE.name(), "create")
.channelMapping(TaskKind.RELOAD.name(), "reload")
.get();
}
and a gateway:
#MessagingGateway
public interface GW {
#Gateway(requestChannel = "gira.input")
Task gira(Collection<Request> messages);
}
and a parserService
#Service
#Slf4j
public class ParserService {
public Task handle(IssueTrackerTask task) {
log.info("Parser service handling task {}", task);
return task;
}
}
I call the gateway method from Spring MVC controller and I want it to return me a Task object that the parserService returns in it's body method. The important thing is that I want controller to be blocked until it gets the value from the parserService. After it gets this value, I want my integration flow to proceed asynchronously with the routerChannelFlow, so that web controller method would return as fast as possible, and all heavy operations in the routerChannelFlow would be done without blocking controller.
Here's a part of the controller that has this gateway method call:
...
Task gira = gw.gira(messages);
log.info("Result {}", gira);
...
The problem is that log.info is never reached and gira() gateway is blocked forever.
How can I achieve my desired behavior?
P.S. The parserService is actually not needed in my app, this is just what I thought would help me to define a return value for my gateway, but it actually did not help:(
UPDATE
So here's what I got after Gary Russell's comment:
#Bean
public ThreadPoolTaskExecutor executor() {
ThreadPoolTaskExecutor pool = new ThreadPoolTaskExecutor();
pool.setCorePoolSize(10);
pool.setMaxPoolSize(10);
pool.setWaitForTasksToCompleteOnShutdown(true);
return pool;
}
#Bean
MessageChannel routerChannel() {
return MessageChannels
.publishSubscribe("routerChannel", executor())
.get();
}
#Bean
IntegrationFlow routerChannelFlow() {
return IntegrationFlows
.from(routerChannel())
.publishSubscribeChannel(s -> s
.subscribe(f -> f
.bridge(null))
.subscribe(process()))
.get();
}
#Bean
IntegrationFlow process() {
return f ->
f.<IssueTrackerTask, String>route(p -> p.getKind().name(),
m -> m.suffix("Channel")
.channelMapping(TaskKind.CREATE.name(), "create")
.channelMapping(TaskKind.RELOAD.name(), "reload")
}
And when I try to use this pipeline, I get the following error Dispatcher has no subscribers for channel 'application:development:9000.dummy'. It's definitely a misconfiguration issue, but I cannot figure out what am I doing wrong.
UPDATE
Changed channel("dummy") to bridge(null).
What is downstream of the create and reload channels?
What do you do with the Task result in the controller (aside from logging it)?
If you don't need a result, change the gateway return to void and add an executor channel downstream of the gateway.
If you want the Task object returned, you need the routerChannel to be a publish/subscribe channel with an executor and two subscribers - a bridge to nowhere (input channel and no output channel) which will return the Task to the gateway, and the router, which will route the Task on a separate thread; the downstream flows from the router must not return a result (the gateway will have long-before stopped waiting for a result).
Instead of adding an executor to routerChannel, you could make the two router channels executor channels instead.
Your last solution is almost there, you just don't need to use .channel("dummy"), but just .bridge(null) in that first subflow subscriber for the .publishSubscribeChannel().
What you need there is just to send message back to the gateway's replyChannel, which is TemporaryReplyChannel in message headers.
The BridgeHandler is the best way do "nothing" but just send message to the appropriate replyChannel from headers.
Also consider to add executor to the channel after .split() if your .transform() and .hanlde() logic is enough heavy.
I have a rest service that sends messages to my queue, and these are routed to file:
from("test-jms:queue:test.queue").to("file://test");
Also, I have an event-driven consumer on the endpoint. For now this only writes to the log if a message is consumed:
final Consumer consumer = endpoint.createConsumer(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String message = exchange.getIn().getBody(String.class);
LOG.info("Message processed: " + message);
}
});
This is all working fine. In the /test folder I am getting a new file for every message I recieve, and additionally the consumer creates a marker file appended with .camelLock. Using the readLock=none option prevents the consumer from making these marker files, as expected.
However, neither the message files nor the marker files are deleted after consumption. Am I perhaps missing something in my consumer implementation?
When you manually create a consumer like that with an inlined processor, you need to manually done the UoW of the Exchange when you are done to trigger work that would delete/move the file etc.
exchange.getUnitOfWork().done(exchange);
You can also try wrapping your processor with the UnitOfWorkProducer that should do the done of the UnitOfWork for you.
The key here, as Claus Ibsen pointed out, was to done the UnitOfWork (UoW). Now my event-driven consumer looks like this:
final Consumer consumer = endpoint.createConsumer(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String message = exchange.getIn().getBody(String.class);
LOG.info("Message processed: " + message);
ConsumerTemplate consumerTemplate = camelContext.createConsumerTemplate();
consumerTemplate.doneUoW(exchange);
}
});
Also, the delete=true option must be used when creating the endpoint:
Endpoint endpoint = camelContext.getEndpoint("file://test?delete=true");