I'm using Camel in a Spring Application. What I need is to properly shut down my Application after camel has sent all his data. Basically camel has to read a file, split it in rows and send each row as kafka massage.
How can I shutdown my application after camel has finished to send all messages?
Here is my route:
public class Router extends RouteBuilder {
#Override
public void configure() throws Exception {
// Kafka Producer
from("file:{{file.dir}}?fileName={{file.name}}&noop=true")
.split(body().tokenize("\r\n|\n")).streaming()
.to("kafka:{{kafka.topic}}?brokers={{spring.kafka.producer.bootstrap-servers}}");
}
}
Basicaly you can call System.exit command
from("file:{{file.dir}}?fileName={{file.name}}&noop=true").routeId("fileconsumer")
.split(body().tokenize("\r\n|\n")).streaming()
.to("kafka:{{kafka.topic}}?brokers={{spring.kafka.producer.bootstrap-servers}}")
.process(exchange -> System.exit(0));
or you can stop camel route
CamelContext context = exchange.getContext();
context.stopRoute("fileconsumer");
Related
I've got a Spring Boot application. I've been designing microservices and these microservices communicate with each other via RabbitMQ. There is a channel service for directing my messages to related microservices. You can see the picture below.
My channel and microservices picture
Channel is directing messages to microservice 1, but microservice 1 should get my message to a specific method, for example, getById() method or createAThing() method.
How can I route my message to the related listener method?
Here is my code that consumes queue:
#RabbitListener(queues = "my-single-queue")
public void createObjectByMessage(Message message) {
// Create object with message
}
#RabbitListener(queues = "my-single-queue")
public void getByIdByMessage(Message message) {
// Get object with message ID
}
Or, how can I consume the message from the same queue but using a different routing key? I have no config class. I'm using #RabbitListener annotation.
I've recently started playing with Apache Camel, and one of the things I've been having issues with is properly performing shutdown logic on selective routes. Since the shutdown logic would vary between routes, Camel's RoutePolicy made the most sense. Here's an example of why I'm trying to do.
public class ProcessingRouteBuilder extends RouteBuilder {
private ProducerTemplate prodTemplate;
public class ProcessingRouteBuilder(ProducerTemplate aProdTemplate) {
prodTemplate = aProdTemplate;
}
#Override
public void configure() {
from("direct://processing")
.routePolicy(new RoutePolicySupport() {
#Override
public void onStop(Route route) {
super.onStop(route);
prodTemplate.sendBody("direct://shutdownRoute", "msg");
}
})
.process(ex -> // Do stuff)
from("direct://shutdownRoute")
.log("Running shutdown A route body - ${body}");
}
}
The shutdown is done like (http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html). The ProducerTemplate comes from the primary CamelContext (read that it is good practice to create one ProducerTemplate per context).
Running this gives me a DirectConsumerNotAvailableException, I've used seda and vm (i don't plan to interact with multiple contexts, but I gave this a shot anyways), both don't exception, but the shutdown routes are never hit. Some questions I have
I might be using the Producer Template wrong? It doesn't look like it's creating an exchange.
Can I even use the ProducerTemplate once the Shutdown hook has been initiated? I'm not sure how Camel performs the shutdown, but it makes sense that it wouldn't allow new messages to be sent, and if the shutdown route is even available at the time of sending.
One thing to note, that I'm not handling here, is ensuring that the shutdown route is performed after the processing route finishes processing all messages in its queue. I'm not entirely sure if the onStop() method is called after there are no more inflight messages and if not, how to enforce it?
I figure another approach is to use when/choice at the beginning of each route and send some sort of shutdown notifier or message, but this seems a little more clunkier.
Thanks guys!
To programmatic shut down a route you can also use the Control Bus EIP.
However the "stop" logic is not clear as you'd want to send a message to the shutdownroute when the processing route stops, but if the stop happen because you are shutting down the camel context it may be possible that the shutdownRoute has already been stopped.
Does Default Message Listener Container of Spring have any method like ErrorHandler where I can Capture MQ down Event.
I can get following logs from spring but need to report when MQ is down.
o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination
o.s.j.l.DefaultMessageListenerContainer : Successfully refreshed JMS Connection
How can I achieve this?
Finally I solved my issue by overrding refreshConnectionUntilSuccessful of DefaultMessageListenerContainer as below:
public class MessageListenerContainer extends DefaultMessageListenerContainer {
#Override protected void refreshConnectionUntilSuccessful() {
super.refreshConnectionUntilSuccessful();
// Your own implementation goes here like sending an email
logger.error(MessageListenerContainer.class, new Exception("MQ CONNECTION LOST"));
}}
I am configuring Kafka as a source in my RouteBuilder. My goal is to handle Kafka disconnection issues. My RouteBuilder is as follows:
new RouteBuilder() {
public void configure() {
onException(Exception.class).process(exchange -> {
final Exception exception = exchange.getException();
logger.error(exception.getMessage());
// will do more processing here
});
from(String.format("kafka:%s?brokers=%s:%s", topicName, host, port)).bean(getMyService(), "myMethod")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
// some more processing
}
});
}
};
I provided wrong host and port, and expected to see an exception. However, no exception is seen in the log, and the onException processing is not get called.
Any idea what I am doing wrong?
A similar problem can be reproduced by running https://github.com/apache/camel/blob/master/examples/camel-example-kafka/src/main/java/org/apache/camel/example/kafka/MessageConsumerClient.java locally without any Kafka server running. Doing so results in a constant flow of messages:
Connection to node -1 could not be established. Broker may not be available.
Is there a way to have an exception thrown?
Any help would be appreciated.
OnException in the RouteBuilder will be triggered when you have a message to route, but since you are unable to connect to Kafka cluster you don't have that. That's why you don't see exception handled.
It's just a good example how tricky Apache Camel is. I'm working on a project having Apache Camel Kafka and I see how badly this is designed. Every Kafka parameter has corresponding Camel URL query-parameter. What if Kafka introduces a new configuration parameter and Apache Camel is not updated to get a new query-parameter? Then there is no way to use this Kafka parameter at all! It's insane.
Example of such Kafka configuration parameter is client.dns.lookup (I need to set it to 'use_all_dns_ips') introduced in Kafka 2.1. No Apache Camel URL query-param to set this!
SOLUTION: Replace Apache Camel Kafka by Spring Kafka.
I have a spring boot application that uses the spring boot starter web. This creates a running Tomcat instance and sets up the http server running on a port. Within my camel route, I want to use this http server as the component for http requests, but I can't figure out how to utilize it. I see many many examples of configuring a jetty instance and consuming from it, but then wouldn't I in effect have two http servers running? I only want to have one. I assume the http server is already autowired up since I can consume from it with other spring code (such as a RestController) and I can see it started in my spring boot logs as well.
#Component
public class ExampleRoute extends RouteBuilder
{
#Override
public void configure() throws Exception
{
//#formatter:off
from( <want to take in an http request here> )
.log( LoggingLevel.INFO, log, "Hello World!" );
//#formatter:on
}
}
There is an example here: https://github.com/camelinaction/camelinaction2/tree/master/chapter7/springboot-camel
You can to register a ServletRegistrationBean that setup the Camel Servlet with Spring Boot.
#Bean
ServletRegistrationBean camelServlet() {
// use a #Bean to register the Camel servlet which we need to do
// because we want to use the camel-servlet component for the Camel REST service
ServletRegistrationBean mapping = new ServletRegistrationBean();
mapping.setName("CamelServlet");
mapping.setLoadOnStartup(1);
// CamelHttpTransportServlet is the name of the Camel servlet to use
mapping.setServlet(new CamelHttpTransportServlet());
mapping.addUrlMappings("/camel/*");
return mapping;
}
However for Camel 2.19 we plan on make this simpler and OOTB: https://issues.apache.org/jira/browse/CAMEL-10416
And then you can do
from("servlet:foo")
.to("bean:foo");
Where the HTTP url to call that Camel route will be http:localhost:8080/camel/foo