I would like to use an async Paho client in Spring Integration:
#Bean
public DefaultMqttPahoClientFactory clientFactory() {
return new DefaultMqttPahoClientFactory();
}
<int-mqtt:message-driven-channel-adapter id="mqttInbound"
client-id="${mqtt.default.client.id}.src"
url="${mqtt.url}"
topics="sometopic"
client-factory="clientFactory"
channel="output"/>
Do I have to write one on my own? I can only find examples for sync client factories.
Correct; at this time, only the sync client is supported.
Feel free to open an improvement JIRA issue here and, of course contributions are always welcome - guidelines here.
That said, it's not clear to me the benefit of using an async client in the inbound adapter as in your example; it is message-driven and arriving messages are delivered asynchronously on the client's thread. Obviously, the outbound adapter could gain some benefit from async operations.
The outbound adapter is pretty simple and shouldn't be too hard to write one that uses the async client - it would probably need to send the delivery tokens to some channel - but we should abstract away from the IMqttDeliveryToken object so we don't leak Paho into the upper layers.
I'm pretty sure that the DefaultMqttPahoClientFactory() provided by Spring Integration is using the "synchronous" mode of operation in Paho, so the likelihood is that you would need an alternative ClientFactory implementation.
Related
we have a larger multi service java spring app that declares about 100 exchanges and queues in RabbitMQ on startup. Some are declared explicitly via Beans, but most of them are declared implicitly via #RabbitListener Annotations.
#Component
#RabbitListener(
bindings = #QueueBinding(key = {"example.routingkey"},
exchange = #Exchange(value = "example.exchange", type = ExchangeTypes.TOPIC),
value = #Queue(name = "example_queue", autoDelete = "true", exclusive = "true")))
public class ExampleListener{
#RabbitHandler
public void handleRequest(final ExampleRequest request) {
System.out.println("got request!");
}
There are quite a lot of these listeners in the whole application.
The services of the application sometimes talk to each other via RabbitMq, so take a example Publisher that publishes a message to the Example Exchange that the above ExampleListener is bound to.
If that publish happens too early in the application lifecycle (but AFTER all the Spring Lifecycle Events are through, so after ApplicationReadyEvent, ContextStartedEvent), the binding of the Example Queue to the Example Exchange has not yet happend and the very first publish and reply chain will fail. In other words, the above Example Listener would not print "got request".
We "fixed" this problem by simply waiting 3 seconds before we start sending any RabbitMq messages to give it time to declare all queues,exchanges and bindings but this seems like a very suboptimal solution.
Does anyone else have some advice on how to fix this problem? It is quite hard to recreate as I would guess that it only occurs with a large amount of queues/exchanges/bindings that RabbitMq can not create fast enough. Forcing Spring to synchronize this creation process and wait for a confirmation by RabbitMq would probably fix this but as I see it, there is no built in way to do this.
Are you using multiple connection factories?
Or are you setting usePublisherConnection on the RabbitTemplate? (which is recommended, especially for a complex application like yours).
Normally, a single connection is used and all users of it will block until the admin has declared all the elements (it is run as a connection listener).
If the template is using a different connection factory, it will not block because a different connection is used.
If that is the case, and you are using the CachingConnectionFactory, you can call createConnection().close() on the consumer connection factory during initialization, before sending any messages. That call will block until all the declarations are done.
How to implement one-way operation in Web Services (using Java or Spring annotations)?
I have tried to add one way as given below
#WebService
public interface DanduServices {
#Oneway
public void saveDanduInformation(#WebParam(name = "serv") ServDTO Serv, #WebParam(name = "dandu") DanduDTO danduDto);
but it is still request-response not asynchronus or one way.
Could anyone suggest to make a operation one-way in service endpoint and let other operations behave as per request-response?
You need to think in terms of the protocol as well though. In HTTP when you send a request you wait for a response, if no response comes back after an amount of time then you will receive a time-out error. So when you talk about one-way (you should rather say async request maybe) you really need to specify exactly what you mean. Do you want to have confirmation that your message was received i.e. have the server respond back with an OK status code and go off and complete it's task but you not wait for the task to be completed? Then you would need to spawn another thread. Spring has AOP for this the same way it has for transactions with #Transactional. Instead you annotated your method with #Async and return a Future<Something>. You'll also need #EnableAsync in your config. Refer to this article for an example Hot To Do #Async
If you don't even care about if the server received your request you don't want to use TCP/HTTP but instead UDP which is used in VOIP (phone over internet) for instance and is quicker, but it will depend on your client.
I'm constructing a data pipeline using Akka streams and Akka HTTP. The use case is quite simple, receive a web request from a user which will do two things. First create a session by calling a 3rd party API, secondly committing this session to some persistent storage, when we have received the session it will then proxy the original user request but add the session data.
I have started working on the first branch of the data pipeline which is the session processing but I'm wondering if there is a more elegant way of unmarshalling the HTTP response from the 3rd party API to a POJO currently I'm using Jackson.unmarshaller.unmarshal which returns a CompletionStage<T> which I then have to unwrap into T. It's not very elegant and I'm guessing that Akka HTTP has more clever ways of doing this.
Here is the code I have right now
private final Source<Session, NotUsed> session =
Source.fromCompletionStage(
getHttp().singleRequest(getSessionRequest(), getMat())).
map(r -> Jackson.unmarshaller(Session.class).unmarshal(r.entity(), getMat())).
map(f -> f.toCompletableFuture().get()).
alsoTo(storeSession);
Akka Streams offers you mapAsync, a stage to handle asynchronous computation in your pipeline in a configurable, non-blocking way.
Your code should become something like
Source.fromCompletionStage(
getHttp().singleRequest(getSessionRequest(), getMat())).
mapAsync(4, r -> Jackson.unmarshaller(Session.class).unmarshal(r.entity(), getMat())).
alsoTo(storeSession);
Note that:
it is not just a matter of elegance in this case, as CompletableFuture.get is a blocking call. This can cause dreadful issues in your pipeline.
the Int parameter required by mapAsync (parallelism) allows for fine-tuning of how many parallel async operations can be run at the same time.
More info in mapAsync can be found in the docs.
I had some legacy code that had SOAP services. Now I am building Rest API for some objects that may call one or more SOAP operation. I was looking into Spring Integration. From the docs
In addition to wiring together fine-grained components, Spring Integration provides a wide selection of channel adapters and gateways to communicate with external systems.
Above statement sounds enticing. I was writing rest microservice controller, Validation service, Rest request to SOAP request mapper and SOAP client. I some cases when there are multiple calls, there is even more code I had to write and I did write the code in many cases.
Spring Integration at high level looked like a framework oriented for Async messages. My problem is that the call need to be more or less a synchronous call and performance is critical. Had anyone used Spring integration for this problem and can you share your experiences.
To complement Artem's answer it's worth to note that if you're going to use one of Spring Integration DSLs (Java, Groovy or Scala) then (the synchronous) DirectChannel will be picked by default by Spring Integration to wire up the endpoints of your integration flow. This means that as long as your endpoints stay synchronous and you rely on default channels between them, the whole integration flow stay synchronous as well.
For instance (in Java DSL):
#Bean
public IntegrationFlow syncFlow() {
return IntegrationFlows
.from(/* get a REST message from microservice */)
// here the DirectChannel is used by default
.filter(/* validate (and filter out) incorrect messages */)
// here the DirectChannel is used by default too
.transform(/* map REST to SOAP */)
// guess what would be here?
.handle(/* send a message with SOAP client */)
.get();
}
This absolutely doesn't mean you tied up with synchronous flow forever. At any step you can go async or parallel. For example, if you decide to send SOAP messages in parallel all you need to do is to specify appropriate channel before SOAP client invocation:
#Bean
public IntegrationFlow syncFlow() {
// ... the same as above ...
.transform(/* map REST to SOAP */)
.channel(c -> c.executor(Executors.newCachedThreadPool())) // see (1)
.handle(/* send a message with SOAP client */)
.get();
}
(1) From this point on the downstream flow will be processed in parallel thanks to use of ExecutorChannel.
Note that message endpoints may also behave asynchronously depending on their logic.
I've used Spring Integration for building synchronous integration flows in my home and work projects and it's proven to be a very powerful yet flexible solution.
One of the first class citizens in Spring Integration is MessageChannel abstraction. The simplest, synchronous, and therefore direct method invocation is DirectChannel.
Not sure what makes you think that everything in Spring Integration is async. Actually it is always direct unless you tell to be async.
I have a simple request reply test implemented using the following configuration:
<int:gateway id="myGateway"
service-interface="TestGateway"
default-request-channel="sendingChannel"
default-reply-channel="replyChannel"
default-reply-timeout="2000"
/>
<int:channel id="sendingChannel" />
<int:channel id="replyChannel" />
<int-jms:outbound-gateway id="myJmsGateway"
connection-factory="jmsConnectionFactory"
request-channel="sendingChannel"
request-destination-name="outQueue"
reply-channel="replyChannel"
reply-destination-name="outQueueReply"
receive-timeout="60000"
/>
and the Interface:
public interface TestGateway {
#Gateway
public String requestReply(#Header("myHeaderKey") String headerValue, String data);
}
While the above configuration does "work" I have the following reservations.
The configuration feels redundant. Extra gateway and two extra channels required. Both gateways implement a reply timeout (although the int:gateway timeout doesn't fire when connected to a int-jms:outbound-gateway).
The semantics of the gateway method change depending on what is implementing the request/reply. On Timeout the int-jms:outbound-gateway will throw an exception, which will propagate to the user of TestGateway. If the config is changed to replace int-jms:outbound-gateway the int:gateway will return null.
Given this the client code has to both handle null and the exception in the same way.
Are there any better ways to wire up the gateways? One option would be to change the int:channel's to PollableChannel's which solves problem 2 at the expense of an extra thread pool.
You don't need to configure reply channels; by default the jms gateway (with no reply channel) will return the message to the inbound gateway automatically.
When using direct channels, the messaging gateway's timeout only starts when the thread tries to receive any reply that was returned from the flow.
You can avoid the different semantics (null Vs exception) by adding an error-channel to the inbound gateway.
It's important to understand that myGateway isolates your client from the messaging system, you code to the interface only. Of course you could inject the JMS gateway directly but then you've added dependencies to your code. With a messaging gateway, you can change technologies without making any changes to your client code. You can also unit test your code by providing a test implementation of TestGateway. This is a powerful feature of Spring Integration.