I have a Spring Integration project where I want to process a message concurrently through multiple actions. So I have set up a publish-subscribe-channel with a task-executor. However I want to wait for all processing to complete before moving on. How would I do this?
<publish-subscribe-channel id="myPubSub" task-executor="my10ThreadPool"/>
<channel id="myOutputChannel"/>
<service-activator input-channel="myPubSub" output-channel="myOutputChannel"
ref="beanA" method="blah"/>
<service-activator input-channel="myPubSub" output-channel="myOutputChannel"
ref="beanB" method="blah"/>
<service-activator id="afterThreadingProcessor" input-channel="myOutputChannel" .../>
So in the above case, I want my afterThreadingProcessor to be invoked only once after both beanA and beanB have completed their work. However, in the above afterThreadingProcessor will be invoked twice.
Add apply-sequence="true" to the pub-sub channel (this adds default correlation data to the messages, including correlationId, sequenceSize, and sequenceNumber and allows default strategies to be used on downstream components).
Add an <aggregator/> before afterThreadingProcessor and route the output from the two <service-activator/>s to it.
Add a <splitter/> after the aggregator - the default splitter will split the collection made by the aggregator into two messages.
afterThreadingProcessor will be invoked once for each message on the second thread that completes its work.
You can make the configuration easier by using a chain...
<chain input-channel="myOutputChannel">
<aggregator />
<splitter />
<service-activator id="afterThreadingProcessor" input-channel="myOutputChannel" .../>
</chain>
To make a single call to the final service, just change your service to take a Collection<?> instead of adding the splitter.
EDIT:
In order to do what you want in comment #3 (run the final service on the original thread), something this should work...
<int:channel id="foo" />
<int:service-activator ref="twoServicesGateway" input-channel="foo"
output-channel="myOutputChannel" />
<int:gateway id="twoServicesGateway" default-request-channel="myPubSub"/>
<int:publish-subscribe-channel id="myPubSub" task-executor="my10ThreadPool"
apply-sequence="true"/>
<int:service-activator input-channel="myPubSub" output-channel="aggregatorChannel"
ref="beanA" method="blah"/>
<int:service-activator input-channel="myPubSub" output-channel="aggregatorChannel"
ref="beanB" method="blah"/>
<int:aggregator input-channel="aggregatorChannel" />
<int:service-activator id="afterThreadingProcessor" input-channel="myOutputChannel" .../>
In this case, the gateway encapsulates the two other services and the aggregator; the default service-interface is a simple RequestReplyExchanger. The calling thread will wait for the output. Since the aggregator has no output-channel the framework will send the reply to the gateway, and the waiting thread will receive it, return to the <service-activator/> and the result will then be sent to the final service.
You would probably want to put a reply-timeout on the gateway because, by default, it will wait indefinitely and, if one of the services returns a null, no agreggated response will ever be received.
Note that I indented the gateway flow just to show it runs from the gateway, they are NOT child elements of the gateway.
Same kind of behavior can now be achieved using a more cleaner approach introduced in Spring Integration 4.1.0 as an implementation of EIP Scatter-Gather pattern.
Checkout Scatter-Gather example gist:
https://gist.github.com/noorulhaq/b13a19b9054941985109
Related
I want to perform an event when a certain set of events have performed their task.
E.g
I have following pooling event
<int-jpa:inbound-channel-adapter id="eventForUpdateProject" channel="updateProjectRequest" entity-manager="entityManagerFactory"
jpa-query="SELECT p FROM Project p WHERE p.sysProject.id IS NOT NULL AND p.sysProject.sysupdate != p.updatedAt">
<int:poller fixed-rate="2000" >
<int:transactional propagation="REQUIRED" transaction-manager="transactionManager"/>
</int:poller>
</int-jpa:inbound-channel-adapter>
It activates three services.
<int:service-activator input-channel="updateProjectRequest" ref="S1"></int:service-activator>
<int:service-activator input-channel="updateProjectRequest" ref="S2"></int:service-activator>
<int:service-activator input-channel="updateProjectRequest" ref="S3"></int:service-activator>
Now i want to trigger a forth event once all these events have performed their tasks. Any idea How this is possible.
There are several tricks for such a task.
<aggregator>, when all those your services send their result for some correlation. The group release on the <aggregator> is performed only when release-strategy has met the requirements.
<publish-subscribe-channel> without any executor and with all your services as subscribers. Plus that your 4th one. Looks like it is your case, although you don't show the updateProjectRequest configuration. Without an executor all the subscribers get the same message one by one: the second subscriber performs his task only when the first has finished. Therefore your "forth event" will be triggered only after all previous.
<recipient-list-router> with similar semantics like previous approach but on several sequential channels.
I'd say that the first one is the best because you can use the same <publish-subscribe-channel> but already with an executor and all your services will be performed in parallel. Even if you don't have anything to return from those services you can emulate aggregation and correlation with a pair of service activators on the same sync <recipient-list-router> where the first one is one of your real service and the second is just for sending some artificial message for aggregation. All those <recipient-list-router>s should be subscribed on the async <publish-subscribe-channel>.
We have a requirement where the external client are calling one of our spring integration inbound webservice gateway and we need to acknowledge them right away with a generic OK status. Meanwhile, the payload will be submitted to a channel for asynchronous processing and we don't need the response(email will be sent based on some business validation).
We tried using asynchronous gateway and executor channel(with task executor), but can't figure out how to make the gateway respond immediately. Both the cases it is working as a one way webservice with no response.
Configuration:
<context:component-scan base-package="com.myapp.springintegration" />
<channel id="helloWorldRequestChannel">
<dispatcher task-executor="taskExecutor"/>
</channel>
<channel id="helloWorldReplyChannel"/>
<task:executor id="taskExecutor" pool-size="2"/>
<gateway id="helloServiceGateway"
service-interface="com.myapp.springintegration.HelloService"
default-request-channel="helloWorldRequestChannel" default-reply-channel="helloWorldReplyChannel"/>
<service-activator input-channel="helloWorldRequestChannel" ref="helloServiceImpl" method="hello" />
I don't see any <int-ws:inbound-gateway> but the requirements may be implemented like this:
<int-ws:inbound-gateway request-channel="requestChannel"/>
<int:recipient-list-router input-channel="requestChannel">
<int:recipient channel="response" />
<int:recipient channel="process" />
</int:recipient-list-router>
<int:transformer input-channel="response" expression="'OK'"/>
<int:channel id="process">
<int:dispatcher task-executor="testExecutor"/>
</int:channel>
You send request to the recipient-list-router, that distributes your message to two other channels. The first one (response) just returns simple reply to the WS immediately. The second (process) shifts the message to the separate Thread, so the subscriber to that channel will do its work not blocking the WS Thread.
Hope I am clear.
I'm looking through some code that is based on spring integration and I've noticed that all int-http:inbound-gateways use pollable request channels:
<int-http:inbound-gateway id="someId"
request-channel="queue-channel"
reply-channel="reply-channel"
request-payload-type="java.lang.String"
supported-methods="POST"
path="/rest/notifications"
auto-startup="true" />
<int:channel id="queue-channel" datatype="java.lang.String">
<int:queue capacity="100" />
</int:channel>
An explicit poller is specified in the configuration:
<int:poller id="mainSystemPoller" default="true" fixed-delay="500" max-messages-per-poll="1">
<int:transactional transaction-manager="transactionManager" propagation="REQUIRES_NEW" isolation="DEFAULT"/>
</int:poller>
So the very first channel up the stream is pollable. What are the benefits of using this approach? Does it just give us more flexibility over the business flow(transactional configurations, queue capacity, etc)?
Without the transaction configuration, there is little, if any, value in using a poller there because the http thread will wait in the gateway for the reply anyway.
However, in your case, it will start a transaction and cause everything downstream of the gateway to run in a transaction. But, as configured, this will single-thread your requests; you would need a task executor to handle multiple concurrent requests.
There are other ways to run concurrent web requests in transactions; you can use a transactional gateway instead and then the flow will run on on the web container thread, with the concurrency managed by your web container.
I am new in Spring integration and working on a SI project. I am doing a simple job of getting message from a channel (fromAdapter), calling a transformer and sending the output to another channel (toQueue). The below code is used in the SI configuration file ----
<int:channel id="fromAdapter"></int:channel>
<int:channel id="toQueue">
</int:channel>
<bean id="trans" class="src.MyTransformer"></bean>
<int:transformer input-channel="fromAdapter" output-channel="toQueue" ref="trans"></int:transformer>
However, now I have a slightly complex requirement. Instead of always sending the message to one transformer,based on some value of the message, I want to send the message to any one of 6 transformers. How can this be implemented?
The recipient list router will work, and may be appropriate if you want to send a message to multiple transformers, but if not, you'll have to be careful to make the selector expressions mutually exclusive. Maybe one of the simpler routers might be more appropriate. For example...
<header-value-router input-channel="routingChannel" header-name="foo">
<mapping value="1" channel="channel1" />
<mapping value="2" channel="channel2" />
</header-value-router>
or
<router id="spelRouter" input-channel="expressionRouter"
expression="payload.someProperty"
default-output-channel="defaultChannelForExpression"
resolution-required="false">
<mapping value="foo" channel="fooChannelForExpression"/>
<mapping value="bar" channel="barChannelForExpression"/>
</router>
You can declare those 6 transformers as subscribers to a single point-to-point channel and by default it will use a round robin dispatching strategy (it will only invoke a single transformer for each message, but it will always pick the next transformer in the list and then cycle).
In your case, you should simply declare all those transformers to use the exact same input and output channels and the above will automagically happen.
To pick the transformer based on some attribute of your message, you can use a recipient-list-router and define a selector-expression for each recipient in the list in order to match a particular kind of message. Also, for each recipient you should use a different channel name. Then each of those channels will be used as input by the desired transformer:
<recipient-list-router input-channel="fromAdapter" default-output-channel="toQueue">
<recipient channel="t1" selector-expression="payload.someFlag"/>
<recipient channel="t2" selector-expression="headers.someOtherFlag"/>
</recipient-list-router>
<transformer input-channel="t1" ref="transformer1" method="transform"/>
<transformer input-channel="t2" ref="transformer2" method="transform"/>
Keep in mind that with this approach, a message could match more than one selector expression so it's up to you to provide mutually exclusive expressions.
Or, if you are willing to write some infrastructure code, you can write your own implementation of LoadBalancingStrategy and provide that to your point-to-point channel. Your strategy will then be responsible for picking the right handler for each message.
I have this:
<si:poller max-messages-per-poll="10" id="defaultPoller" default="true">
<si:interval-trigger interval="5000"/>
</si:poller>
<si:channel id="emailIn"/>
<si:channel id="emailOut"/>
<si:service-activator input-channel="emailIn" output-channel="emailOut" ref="mailService" method="recieveMessage"/>
<si:gateway id="gateway" service-interface="com.blah.MailSender" default-request-channel="emailIn"/>
<si:outbound-channel-adapter channel="emailOut" ref="mailService" method="recieveMessage" />
And I thought what I was configuring was an async Queue. I want to be able to drop messages onto it, and have a nother thread pick them up and process then later. However, at the momment it seems to do it in a synchronous way.
Am i doing it wrong (obvioulsy yes), but wondering if there is something i'm missing in this config, or whether i just have the wrong approach?
Cheers
By default all channels in Spring Integration are synchronous. This is a conscious design decision that will help you keep transaction boundaries and security contexts for example. When you want to do asynchronous hand-off you should add a task executor to your dispatcher or a queue to your channel:
<channel>
<dispatcher task-executor="pool"/>
</channel>
<channel>
<queue capacity="10"/>
</channel>
Look at channel configurations in the reference guide for some details on dispatchers and queues. See also the section on DirectChannel and the section on ExecutorChannel below that one.