I have two spring beans. A JmsHandler which receives a Message. This is processed and handed over to a MailHandler which sends a mail. Now I've seen that sometimes messages arrive at the JmsHandler in exact the same time. When entering the sendMail method one of the both isn't processed properly (no mail is sent). So I would expect that's an threading issue.
I've got two possiblities to handle this issue:
a) I set the sendMail method as synchronized. Which means the next execution will be stopped until the first is processed.
public class JmsHandler {
#Autowired
private MailHandler mailer;
public void handleMessage(Message msg) {
mailer.sendMail(msg.getPayload().toString());
}
}
#Component
public class MailHandlerImpl implements MailHandler {
#Override
public synchronized void sendMail(String message) {
// do some fancy stuff...
// do something to mail the message.
}
}
b) I define the MailHandler to the scope "prototype" and get it using an lookup from the ApplicationContext.
public class JmsHandler {
#Autowired
private ApplicationContext ctx;
public void handleMessage(Message msg) {
MailHandler mailer = ctx.getBean(MailHandler.class)
mailer.sendMail(msg.getPayload().toString());
}
}
#Component
#Scope("prototype")
public class MailHandlerImpl implements MailHandler {
#Override
public void sendMail(String message) {
// do some fancy stuff...
// do something to mail the message.
}
}
I think both ways are ok. But in terms of performance is there any issue? When using synchronized it will stop execution of the next process and wait until the first is finished. The second will create a new Instance of MailHandlerImpl when a message has to be send (and of course autowire other things not shown here).
Or is there a third alternative?
The only way to spot performance issue is to get a metrics about your application behaviour. You should see metrics and better use Timer so that you can see how many message has been handled and how fast it is processed. It's awesome.
If you are certain that mail handler is only capable of sending an email in certain moment, just make sure you only have one consumer. No need to use synchronized. If you prefer concurrent process/ multi consumer, which is generally preferred, create mail handler for each thread. Leverage reactor to ease your parallel processing (use thread pool executor in your event bus) and commons pool to hold your mail handler instances. Thread pool executor size and your mail handler pool size should be equal. Or, you can use email service such as mailgun to ease your concurrent email sending.
Related
I've recently started playing with Apache Camel, and one of the things I've been having issues with is properly performing shutdown logic on selective routes. Since the shutdown logic would vary between routes, Camel's RoutePolicy made the most sense. Here's an example of why I'm trying to do.
public class ProcessingRouteBuilder extends RouteBuilder {
private ProducerTemplate prodTemplate;
public class ProcessingRouteBuilder(ProducerTemplate aProdTemplate) {
prodTemplate = aProdTemplate;
}
#Override
public void configure() {
from("direct://processing")
.routePolicy(new RoutePolicySupport() {
#Override
public void onStop(Route route) {
super.onStop(route);
prodTemplate.sendBody("direct://shutdownRoute", "msg");
}
})
.process(ex -> // Do stuff)
from("direct://shutdownRoute")
.log("Running shutdown A route body - ${body}");
}
}
The shutdown is done like (http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html). The ProducerTemplate comes from the primary CamelContext (read that it is good practice to create one ProducerTemplate per context).
Running this gives me a DirectConsumerNotAvailableException, I've used seda and vm (i don't plan to interact with multiple contexts, but I gave this a shot anyways), both don't exception, but the shutdown routes are never hit. Some questions I have
I might be using the Producer Template wrong? It doesn't look like it's creating an exchange.
Can I even use the ProducerTemplate once the Shutdown hook has been initiated? I'm not sure how Camel performs the shutdown, but it makes sense that it wouldn't allow new messages to be sent, and if the shutdown route is even available at the time of sending.
One thing to note, that I'm not handling here, is ensuring that the shutdown route is performed after the processing route finishes processing all messages in its queue. I'm not entirely sure if the onStop() method is called after there are no more inflight messages and if not, how to enforce it?
I figure another approach is to use when/choice at the beginning of each route and send some sort of shutdown notifier or message, but this seems a little more clunkier.
Thanks guys!
To programmatic shut down a route you can also use the Control Bus EIP.
However the "stop" logic is not clear as you'd want to send a message to the shutdownroute when the processing route stops, but if the stop happen because you are shutting down the camel context it may be possible that the shutdownRoute has already been stopped.
I'm trying to create a Spring Cloud Stream Source Bean inside a Spring Boot Application that simply sends the results of a method to a stream (underlying Kafka topic is bound to the stream).
Most of the Stream samples I've seen use #InboundChannelAdapter annotation to send data to the stream using a poller. But I don't want to use a poller. I've tried setting the poller to an empty array but the other problem is that when using #InboundChannelAdapter you are unable to have any method parameters.
The overall concept of what I am trying to do is read from an inbound stream. Do some async processing, then post the result to an outbound stream. So using a processor doesn't seem to be an option either. I am using #StreamListener with a Sink channel to read the inbound stream and that works.
Here is some code i've been trying but this doesn't work at all. I was hoping it would be this simple because my Sink was but maybe it isn't. Looking for someone to point me to an example of a source that isn't a Processor (i.e. doesn't require listening on an inbound channel) and doesn't use #InboundChannelAdapter or to give me some design tips to accomplish what I need to do in a different way. Thanks!
#EnableBinding(Source.class)
public class JobForwarder {
#ServiceActivator(outputChannel = Source.OUTPUT)
#SendTo(Source.OUTPUT)
public String forwardJob(String message) {
log.info(String.format("Forwarding a job message [%s] to queue [%s]", message, Source.OUTPUT));
return message;
}
}
Your orginal requirement can be achieved through the below steps.
Create your custom Bound Interface (you can use the default #EnableBinding(Source.class) as well)
public interface CustomSource {
String OUTPUT = "customoutput";
#Output(CustomSource.OUTPUT)
MessageChannel output();
}
Inject your bound channel
#Component
#EnableBinding(CustomSource.class)
public class CustomOutputEventSource {
#Autowired
private CustomSource customSource;
public void sendMessage(String message) {
customSource.output().send(MessageBuilder.withPayload(message).build());
}
}
Test it
#RunWith(SpringRunner.class)
#SpringBootTest
public class CustomOutputEventSourceTest {
#Autowired
CustomOutputEventSource output;
#Test
public void sendMessage() {
output.sendMessage("Test message from JUnit test");
}
}
So if you don't want to use a Poller, what causes the forwardJob() method to be called?
You can't just call the method and expect the result to go to the output channel.
With your current configuration, you need an inputChannel on the service containing your inbound message (and something to send a message to that channel). It doesn't have to be bound to a transport; it can be a simple MessageChannel #Bean.
Or, you could use a #Publisher to publish the result of the method invocation (as well as being returned to the caller) - docs here.
#Publisher(channel = Source.OUTPUT)
Thanks for the input. It took me a while to get back to the problem. I did try reading the documentation for #Publisher. It looked to be exactly what I needed but I just couldn't get the proper beans initialized to get it wired properly.
To answer your question the forwardJob() method is called after some async processing of the input.
Eventually I just implemented using spring-kafka library directly and that was much more explicit and felt easier to get going. I think we are going to stick to kafka as the only channel binding so I think we'll stick with that library.
However, we did eventually get the spring-cloud-stream library working quite simply. Here was the code for a single source without a poller.
#Component
#EnableBinding(Source.class)
public class JobForwarder {
private Source source;
#Autowired
public ScheduledJobForwarder(Source source) {
this.source = source;
}
public void forwardScheduledJob(String message) {
log.info(String.format("Forwarding a job message [%s] to queue [%s]", message, Source.OUTPUT));
source.output().send(MessageBuilder.withPayload(message).build());
}
}
I am maintaining an existing application which receives requests from ActiveMQ and sends responses back to the sender via an ActiveMQ topic. At present there is a single message consumer class which receives messages via a simple DefaultMessageListenerContainer:
#Component
public class RequestConsumer {
#Autowired
CustomerService customerService;
#Autowired
JmsSenderService jmsSenderService;
public void handleMessage(Message message) {
if (message instanceof CustomerRequest) {
CustomerRequest customerRequest = (CustomerRequest) message;
Customer customer = customerService.getCustomerById(customerRequest.getId());
CustomerResponse customerResponse = new CustomerResponse();
customerResponse.addCustomer(customer);
jmsSenderService.sendCustomerResponse(customerResponse);
}
}
}
I need to extend the application to process a number of different requests (e.g. OrderRequest, InvoiceRequest, InventoryRequest, AddressRequest etc.) and send an appropriate response back to the sender. My first thought was to add the functionality to the existing class like so:
public void handleMessage(Message message) {
if (message instanceof CustomerRequest) {
// deal with CustomerRequest
} else if (message instanceof InvoiceRequest) {
// deal with InvoiceRequest
} else if (message instanceof InventoryRequest) {
// deal with InventoryRequest
}
}
However this will make the class quite large. I also thought about implementing one queue per request type (e.g. customer.request.queue, invoice.request.queue) and implementing multiple DefaultMessageListenerContainer, one per queue, but this doesn't seem like a great idea either because of the multiple boilerplated classes I'd need to create.
I feel like there must be a way to implement some kind of routing based on the type of incoming object and map it to an object-specific implementation to process the request, but I'm not sure whether this is something Spring/JMS already provides.
Has anyone done something like this before, and if so is there a "Springy" way to do it?
If you are willing to invest some time (as this solution comes with an initial time-investment) then there is an absolutely Spring-conform solution:
Spring Integration
This library (a good JMS example here) has prebuilt and well-tested solutions for all common messaging issues you might face, including the problem you just described above (message-type based routing).
I'm trying to read the unconsumed messages in a queue-channel. But couldn't find a way to do it. Is it possible? And if yes, kindly point to the right documents.
The purpose is to expose an API, such that the client can see the pending items on the UI.
Thanks,
You can simply reference the "queue-channel" in one of your ServiceActivator and do whatever you want with the messages:
#MessageEndpoint(value = "jobQueuer")
public class JobStartupQueuer {
#Resource
private Queue<Message> jobChannelQueue;
public boolean accept(Message<?> message) {
LOG.info("Channel size: {}", jobChannelQueue.size());
return true;
}
}
So the jobChannelQueue gets injected so in your ServiceActivator processing method (e.g. accept) we can reference the queue and its inner messages.
How can I send a message to an endpoint without waiting for that endpoint's route to be process (that is, my route should just dispatch the message and finish)?
Using wireTap or multicast is what you're after. A direct: endpoint will modify the Exchange for the next step no matter what ExchangePattern is specified. You can see by using this failing test:
public class StackOverflowTest extends CamelTestSupport {
private static final String DIRECT_INPUT = "direct:input";
private static final String DIRECT_NO_RETURN = "direct:no.return";
private static final String MOCK_OUTPUT = "mock:output";
private static final String FIRST_STRING = "FIRST";
private static final String SECOND_STRING = "SECOND";
#NotNull
#Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from(DIRECT_INPUT)
.to(ExchangePattern.InOnly, DIRECT_NO_RETURN)
.to(MOCK_OUTPUT)
.end();
from(DIRECT_NO_RETURN)
.bean(new CreateNewString())
.end();
}
};
}
#Test
public void testShouldNotModifyMessage() throws JsonProcessingException, InterruptedException {
final MockEndpoint myMockEndpoint = getMockEndpoint(MOCK_OUTPUT);
myMockEndpoint.expectedBodiesReceived(FIRST_STRING);
template.sendBody(DIRECT_INPUT, FIRST_STRING);
assertMockEndpointsSatisfied();
}
public static class CreateNewString {
#NotNull
public String handle(#NotNull Object anObject) {
return SECOND_STRING;
}
}
}
Now if you change the above to a wireTap:
from(DIRECT_INPUT)
.wireTap(DIRECT_NO_RETURN)
.to(MOCK_OUTPUT)
.end();
and you'll see it works as expected. You can also use multicast:
from(DIRECT_INPUT)
.multicast()
.to(DIRECT_NO_RETURN)
.to(MOCK_OUTPUT)
.end();
wireTap(endpoint) is the answer.
you can use a ProducerTemplate's asyncSend() method to send an InOnly message to an endpoint...
template.asyncSend("direct:myInOnlyEndpoint","myMessage");
see http://camel.apache.org/async.html for some more details
That might depend on what endpoints etc you are using, but one common method is to put a seda endpoint in between is one option.
from("foo:bar")
.bean(processingBean)
.to("seda:asyncProcess") // Async send
.bean(moreProcessingBean)
from("seda:asyncProcess")
.to("final:endpoint"); // could be some syncrhonous endpoint that takes time to send to. http://server/heavyProcessingService or what not.
The seda endpoint behaves like a queue, first in - first out. If you dispatch several events to a seda endpoint faster than the route can finish processing them, they will stack up and wait for processing, which is a nice behaviour.
You can use inOnly in your route to only send your message to an endpoint without waiting for a response. For more details see the request reply documentation or the event message documentation
from("direct:testInOnly").inOnly("mock:result");
https://people.apache.org/~dkulp/camel/async.html
Both for InOnly and InOut you can send sync or async. Seems strange that you can send InOnly but async, but at last here it explains that it waits for Camel processing and then fire and forget.
The Async Client API
Camel provides the Async Client API in the ProducerTemplate where we have added about 10 new methods to Camel 2.0. We have listed the most important in the table below:
Method
Returns
Description
setExecutorService
void
Is used to set the Java ExecutorService. Camel will by default provide a ScheduledExecutorService with 5 thread in the pool.
asyncSend
Future
Is used to send an async exchange to a Camel Endpoint. Camel will imeddiately return control to the caller thread after the task has been submitted to the executor service. This allows you to do other work while Camel processes the exchange in the other async thread.
asyncSendBody
Future
As above but for sending body only. This is a request only messaging style so no reply is expected. Uses the InOnly exchange pattern.
asyncRequestBody
Future
As above but for sending body only. This is a Request Reply messaging style so a reply is expected. Uses the InOut exchange pattern.
extractFutureBody
T
Is used to get the result from the asynchronous thread using the Java Concurrency Future handle.
The Async Client API with callbacks
In addition to the Client API from above Camel provides a variation that uses callbacks when the message Exchange is done.
Method
Returns
Description
asyncCallback
Future
In addition a callback is passed in as a parameter using the org.apache.camel.spi.Synchronization Callback. The callback is invoked when the message exchange is done.
asyncCallbackSendBody
Future
As above but for sending body only. This is a request only messaging style so no reply is expected. Uses the InOnly exchange pattern.
asyncCallbackRequestBody
Future
As above but for sending body only. This is a Request Reply messaging style so a reply is expected. Uses the InOut exchange pattern.
These methods also returns the Future handle in case you need them. The difference is that they invokes the callback as well when the Exchange is done being routed.
The Future API
The java.util.concurrent.Future API have among others the following methods:
Method
Returns
Description
isDone
boolean
Returns a boolean whether the task is done or not. Will even return true if the tasks failed due to an exception thrown.
get()
Object
Gets the response of the task. In case of an exception was thrown the java.util.concurrent.ExecutionException is thrown with the caused exception.