I want to use Spring Integration to expose a simple web service that pushes incoming message into ActiveMQ and responds immediately. My go-to solution was MarshallingWebServiceInboundGateway connected to Jms.outboundAdapter with IntegrationFlow. Below the Gateway and IntegrationFlow snippets. Problem with this is Adapter does not provide response (duh) which Gateway expects. The response I get back from the service is empty 202, with delay of about 1500ms. This is caused by a reply timeout I see in TRACE logs:
"2020-04-14 17:17:50.101 TRACE 26524 --- [nio-8080-exec-6] o.s.integration.core.MessagingTemplate : Failed to receive message from channel 'org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#518ffd27' within timeout: 1000"
No hard exceptions anywhere. The other problem is I cannot generate the response myself. I can't add anything to IntegrationFlow after the .handle with Adapter.
Any other way I can try to fulfill the scenario?
How, if at all possible, can I generate and return response in situation there is no better approach?
Most likely the proper way would be to use Gateways on both ends, but this is not possible. I cannot wait with response until message in the queue gets consumed and processed.
'''
#Bean
public MarshallingWebServiceInboundGateway greetingWebServiceInboundGateway() {
MarshallingWebServiceInboundGateway inboundGateway = new MarshallingWebServiceInboundGateway(
jaxb2Marshaller()
);
inboundGateway.setRequestChannelName("greetingAsync.input");
inboundGateway.setLoggingEnabled(true);
return inboundGateway;
}
#Bean
public IntegrationFlow greetingAsync() {
return f -> f
.log(LoggingHandler.Level.INFO)
.handle(Jms.outboundAdapter(this.jmsConnectionFactory)
.configureJmsTemplate(c -> {
c.jmsMessageConverter(new MarshallingMessageConverter(jaxb2Marshaller()));
})
.destination(JmsConfig.HELLO_WORLD_QUEUE));
}
'''
The logic and assumptions are fully correct: you can't return after one-way handle() and similar to that Jms.outboundAdapter().
But your problem that you fully miss one of the first-class citizens in Spring Integration - a MessageChannel. It is important to understand that even in the flow like yours there are channels between endpoints (DSL methods) - implicit (DirectChannel), like in your case, or explicit: when you use a channel() in between and can place there any possible implementation: https://docs.spring.io/spring-integration/docs/5.3.0.M4/reference/html/dsl.html#java-dsl-channels
One of the crucial channel implementation is a PublishSubscribeChannel (a topic in JMS specification) when you can send the same message to several subscribed endpoints: https://docs.spring.io/spring-integration/docs/5.3.0.M4/reference/html/core.html#channel-implementations-publishsubscribechannel
In your case the fists subscriber should be your existing, one-way Jms.outboundAdapter(). And another something what is going to generate response and reply it into a replyChannel header.
For this purpose Java DSL provides a nice hook via sub-flows configuration: https://docs.spring.io/spring-integration/docs/5.3.0.M4/reference/html/dsl.html#java-dsl-subflows
So, some sample of publish-subscriber could be like this:
.publishSubscribeChannel(c -> c
.subscribe(sf -> sf
.handle(Jms.outboundAdapter(this.jmsConnectionFactory))))
.handle([PRODUCE_RESPONSE])
Related
I have got two Spring Boot application. First one is REST application. The REST one is communicating with second application through RabbitMQ message queue. I'm sending a request to method with the #GetMapping("/") and this method producing a message to example-queue. A method with #RabbitListener(queues = {"example-queue"}) taking the message and create a object at database. Now, how can I send my response (saved object) to the method with #GetMapping("/")? I need a response from consumer to ResponseEntity.ok();. How can I do that? Thank you.
Just see if you can make the interaction with RabbitMQ consumer as a request-reply pattern.
The #RabbitListener can just return your object and be marked with a #SendTo. This way the framework will look into a replyTo property of the request message.
On the producer side you can just use an AmqpTemplate.convertSendAndReceive().
See more in docs: https://docs.spring.io/spring-amqp/docs/current/reference/html/#request-reply
Here I've three subflows and out of which one is HTTP outbound call. I want that HTTP call should try to get response till a mentioned time. If times out then the main flow should break and it should show a Error message in Json format as output.
Below is the code -
#Bean
public IntegrationFlow flow() {
return flow ->
flow.handle(validatorService, "validateRequest")
.split()
.channel(c -> c.executor(Executors.newCachedThreadPool()))
.scatterGather(
scatterer ->
scatterer
.applySequence(true)
.recipientFlow(flow1())
.recipientFlow(
f ->
f.gateway(
flow2(), gateway -> gateway.replyTimeout(3000L).errorChannel("errorChannel")))
.recipientFlow(flow3()),
gatherer ->
gatherer
.releaseLockBeforeSend(true)
.releaseStrategy(group -> group.size() == 2))
.aggregate(someMethod1())
.to(someMethod2());
}
private IntegrationFlow returnError() {
return IntegrationFlows.from("errorChannel").handle(System.out::println).get();
}
I've added the errorChannel but how do I send a customized message to the user?
See documentation for error handling in the messaging gateway: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#gateway-no-response.
Consider to add an errorChannel() along side with that replyTimeout() on the gateway definition to build an error reply you'd like. However you also may consider to add something like a request timeout for the RestTemplate you use for that HTTP call to prevent the long wait for HTTP response.
UPDATE
First of all you need to understand that request-reply is a bit of block-n-then-wait approach. We send a request and if the process consuming that message is blocking - performed immediately in a thread producing the message, then we don't reach "wait" part until that process lets go. In most cases (by default) a DirectChannel is used, so it is blocked because it is performed in the calling thread. This process might be your HTTP call which is also request-response pattern. So, we don't reach that "wait" part until this HTTP call returns, or timeout, or fail with error. Only after that a replyTimeout takes its effect to wait for the reply from the underlying process. This can be changes if an input channel of that process is not direct. See an ExecutorChannel or QueueChannel. This way a sending part exits immediately because there is nothing to block it and it goes to the "wait" part to observe a CountDownLatch.
So, you need to think again if that replyTimeout() option is appropriate for your or not. Perhaps the mentioned requestTimeout for the RestTemplate is better option for you, than rework your flow to the async solution to leverage that replyTimeout() feature. Again: see the documentation I've mentioned about that replyTimeout feature.
The error handling is described here: https://docs.spring.io/spring-integration/docs/current/reference/html/error-handling.html#error-handling.
It is really not recommended to rely on the global errorChannel bean. This is one which is used everywhere in async processes where there is no an explicit error channel configured.
You said in your question "send a customized message to the user", but your error handling flow is one-way - System.out::println. If you want to return anything from the error handling flow, the endpoint must be replying one, e.g.:
.handle((p, h) -> "The error during HTTP call: " + p)
Also see if you declare that returnError() correctly. It really cannot be just plain private method. The IntegrationFlow must be declared as a bean this or other way to initiate wiring process for endpoints and channels. Right now that one is just a plain, unused private method. The framework does not see that method to do anything. See basics of the Java DSL in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html#java-dsl
We have multiple applications consumer listening to the same kafka topic and a producer sets the message header when sending message to the topic so specific instance can evaluate the header and process the message. eg
#StreamListener(target=ITestSink.CHANNEL_NAME,condition="headers['franchiseName'] == 'sydney'")
public void fullfillOrder(#Payload TestObj message) {
log.info("sydney order request received message is {}",message.getName());
}
In Spring Cloud Stream 3.0.0 the #StreamListener is deprecated and I could not find the equivalent of the condition property in Function.
Any suggestion?
Though I was not able to find the equivalent for the functional approach either, I do have a suggestion.
The #StreamListener annotations condition does not stop the fact that the application must consume the message, read its header, and filter out specific records before passing it to the listener (fullfillOrder()). So it's safe to assume you're consuming every message that hits the topic regardless (by the event receiver that Spring Cloud has implemented for us under the hood), but the listener only gets executed when header == sydney.
If there was a way to configure the event receiver that Spring Cloud uses (to discard message before hitting listener), I would suggest looking into that. If not, would resort to filtering out any messages (non-sydney) before doing any processing. If you're familiar with Spring Cloud's functional approach, would look something like this:
#Bean
public Consumer<Message<TestObj>> fulfillOrder() {
return msg -> {
// to get header - msg.getHeaders().get(key, valueType);
// filter out bad messages
}
}
or
#Bean
public Consumer<ConsumerRecord<?, TestObj>> fulfillOrder() {
return msg -> {
// msg.headers().lastHeader("franchiseName").value() -> filter em out
}
}
Other:
^ my code assumes you're integrating the kafka-client API with Spring cloud stream via spring-cloud-stream-binder-kafka. based on tags listed, i will note Spring Cloud Stream has two versions of binders for Kafka - one for the kafka client library, and one for kafka streams library.
Without considering Spring Cloud / Frameworks, the high-lvl DSL in kafka streams doesn't give you access to headers, but the low-level Processor API does. From the example, it seems like you're leveraging the client binder and not spring-cloud-stream-binder-kafka-streams / kafka streams binder. I haven't seen an implementation of spring cloud stream + kafka streams binder using the low-level processor API, so i can't tell if that was the aim.
I had some legacy code that had SOAP services. Now I am building Rest API for some objects that may call one or more SOAP operation. I was looking into Spring Integration. From the docs
In addition to wiring together fine-grained components, Spring Integration provides a wide selection of channel adapters and gateways to communicate with external systems.
Above statement sounds enticing. I was writing rest microservice controller, Validation service, Rest request to SOAP request mapper and SOAP client. I some cases when there are multiple calls, there is even more code I had to write and I did write the code in many cases.
Spring Integration at high level looked like a framework oriented for Async messages. My problem is that the call need to be more or less a synchronous call and performance is critical. Had anyone used Spring integration for this problem and can you share your experiences.
To complement Artem's answer it's worth to note that if you're going to use one of Spring Integration DSLs (Java, Groovy or Scala) then (the synchronous) DirectChannel will be picked by default by Spring Integration to wire up the endpoints of your integration flow. This means that as long as your endpoints stay synchronous and you rely on default channels between them, the whole integration flow stay synchronous as well.
For instance (in Java DSL):
#Bean
public IntegrationFlow syncFlow() {
return IntegrationFlows
.from(/* get a REST message from microservice */)
// here the DirectChannel is used by default
.filter(/* validate (and filter out) incorrect messages */)
// here the DirectChannel is used by default too
.transform(/* map REST to SOAP */)
// guess what would be here?
.handle(/* send a message with SOAP client */)
.get();
}
This absolutely doesn't mean you tied up with synchronous flow forever. At any step you can go async or parallel. For example, if you decide to send SOAP messages in parallel all you need to do is to specify appropriate channel before SOAP client invocation:
#Bean
public IntegrationFlow syncFlow() {
// ... the same as above ...
.transform(/* map REST to SOAP */)
.channel(c -> c.executor(Executors.newCachedThreadPool())) // see (1)
.handle(/* send a message with SOAP client */)
.get();
}
(1) From this point on the downstream flow will be processed in parallel thanks to use of ExecutorChannel.
Note that message endpoints may also behave asynchronously depending on their logic.
I've used Spring Integration for building synchronous integration flows in my home and work projects and it's proven to be a very powerful yet flexible solution.
One of the first class citizens in Spring Integration is MessageChannel abstraction. The simplest, synchronous, and therefore direct method invocation is DirectChannel.
Not sure what makes you think that everything in Spring Integration is async. Actually it is always direct unless you tell to be async.
What I want to do is process AMQP messages in a very similar way the Http Requests are processed using spring-webmvc annotations such as #RequestMapping, #RequestParam etc. But, instead of the Http Request my source object will be an AMQP message. The AMQP message request will have two headers, for example -
method="POST"
url="/api/myobjects/{someParam}"
and the payload will contain data in json format.
If you have noticed, this is nothing but HTTP REST api mapped to AMQP message.
I want to be able to write a controller like handler, for example -
#Controller
public class MyObjectHandler {
#RequestMapping(value="/api/myobjects/{someParam}", method="POST")
public MyObject createMyObject(#Payload MyObject myObj, #PathParam String someParam) {
//... some processing
return myObj;
}
// ...more handlers
}
I have looked at spring-amqp/rabbitmq annotations and also spring integration annotations. They are close to what I want, but would not allow routing to handler methods based on header parameters, especially the REST url.
I don't expect that a readymade solution would be available for this. Just want to make sure I choose the best possible option. Some of the options I think are (in order of precedence)
If the spring-webmvc annotation processing mechanism is extensible, just extend it to use AMQP message as source instead of Http Request
Modify the spring-webmvc annotation processing mechanism to take the AMQP message as input instead of Http Request
Write your own solution with custom annotaions and their processors, which I think is a very involving task
Or any other possible approach than above?
Any guidance/direction is appreciated.
I think the starting point is likely AbstractMethodMessageHandler in spring-messaging.
There's currently a SimpAnnotationMethodMessageHandler implementation for websockets which invokes #Controllers.
You could use a #RabbisListener method that has a Message<?> parameter (Spring AMQP will convert the underlying Rabbit message to a spring-messaging message, including the headers). Then, invoke the message handler to route to the appropriate controller method.
If you come up with a robust implementation, please consider contributing it.