I have recently started with Apache Camel, and we are looking into creating custom components to abstract a lot of logic and simplify routes, but some of this logic involves http requests and other portions that have an existing camel component we want to utilize.
Is it possible to call other components (e.g. the http component) from within our custom component's producer?
I did see this question (Can a custom Camel component use routes and other components internally?) that mentions using the camel context, but how to you replicate the route call outside of a RouteBuilder?
You need to import CamelContext, Exchange, ProducerTemplate and ExchangeBuilder.
import org.apache.camel.CamelContext;
import org.apache.camel.Exchange;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.builder.ExchangeBuilder;
You then need to create instances of the producer template and camel context. I am using spring boot, so I can just inject the dependencies.
#Autowired
private ProducerTemplate producer;
#Autowired
private CamelContext camelContext;
In your method definition, you need to create an exchange request with the ExchangeBuilder. You can create a body and add headers to you exchange message here.
Exchange exchangeRequest = ExchangeBuilder.anExchange(camelContext)
.withBody("Hello World!")
.withHeader("username", "jdoe")
.withHeader("password", "pass")
.build();
You can then call the send method on the producer object to tap into your route and capture the response.
Exchange exchangeResponse = producer.send("direct:startRoute", exchangeRequest)
Related
I'm learning about Apache Camel routes in Spring Boot projects and I have a project that does extension from some endpoints. The endpoints are not in this project, only the extension is done here. The extension is done using #Consume from org.apache.camel in this way:
#Consume(uri = "direct:products.create.validate.interceptor")
public void executeCreate(RequestWrapper<Product> productWrapper) {
...
}
I try to understand how this direct:products.create.validate.interceptor is mapped to an endpoint from another service. Can somebody explain me how this #Consume annotation does the mapping?
Or another example is this:
#Consume(uri = "direct:listAccountsPostHook")
public void executeCreate(RequestWrapper<Account> accountWrapper) {
...
}
Where should I look to understand how they are mapped? In the controller of the other service? I can't find any example with #Consume. Thank you!
The #Consume annotation in Apache Camel is used to subscribe to a Camel endpoint and consume messages from it. The endpoint can be either a direct endpoint or any other type of endpoint such as a JMS queue or a REST endpoint, depending on your use case.
The endpoint URI, which is specified in the uri attribute of the #Consume annotation, determines where the messages are consumed from. In your example, direct:products.create.validate.interceptor and direct:listAccountsPostHook are both direct endpoints.
In Apache Camel, direct endpoints are in-memory endpoints that allow you to send messages directly to another endpoint in the same JVM. The mapping between the endpoint and the method that consumes the messages is done by Camel's routing engine.
More on Camel Direct endpoints you can read here.
To understand how the messages are being consumed, you should look at the Camel routes that are defined in your project. In a Spring Boot project, you can define your Camel routes in a RouteBuilder class. This is where you would specify the mapping between the direct endpoint and the method that will consume the messages.
For example, if you have a RouteBuilder class that looks like this:
public class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() {
from("direct:products.create.validate.interceptor")
.to("bean:myBean?method=executeCreate");
}
}
In this example, the direct endpoint direct:products.create.validate.interceptor is mapped to the executeCreate method in the myBean bean. The ?method=executeCreate part of the to URI tells Camel to call the executeCreate method on the myBean bean when a message is received at the endpoint.
So, in short, you should look for the Camel routes in your project that define the mapping between the endpoint and the method that consumes the messages.
I've two microservices interacting with each other via Kafka, that is the one publishes messages while the other consumes them. Both the publisher and the consumer run on Quarkus (1.12.0.Final) and use reactive messaging and Mutiny.
Producer:
package myproducer;
import myavro.MyAvro;
import io.smallrye.mutiny.Uni;
import org.eclipse.microprofile.reactive.messaging.Channel;
import org.eclipse.microprofile.reactive.messaging.Emitter;
import org.eclipse.microprofile.reactive.messaging.Message;
import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;
import java.util.concurrent.CompletableFuture;
#ApplicationScoped
public class Publisher {
#Channel("mytopic")
#Inject
public Emitter<MyAvro> myTopic;
#Override
public Uni<Void> publish(MyModel model) {
MyAvro avro = MyModelMapper.INSTANCE.modelToAvro(model);
return Uni.createFrom().emitter(e -> myTopic.send(Message.of(avro)
.addMetadata(toOutgoingKafkaRecordMetadata(avro))
.withAck(() -> {
e.complete(null);
return CompletableFuture.completedFuture(null);
})));
}
}
Consumer:
package myconsumer;
import myavro.MyAvro;
import io.smallrye.mutiny.Uni;
import io.smallrye.reactive.messaging.kafka.IncomingKafkaRecord;
import org.eclipse.microprofile.reactive.messaging.Incoming;
import javax.enterprise.context.ApplicationScoped;
#ApplicationScoped
public class Consumer {
#Incoming("mytopic")
public Uni<Void> consume(IncomingKafkaRecord<String, MyAvro> message) {
MyModel model = MyModelMapper.INSTANCE.avroToModel(message.getPayload());
return ...;
}
}
Dependencies:
include among others the artefacts
quarkus-smallrye-reactive-messaging-kafka
quarkus-resteasy-mutiny
quarkus-smallrye-opentracing
quarkus-mutiny
opentracing-kafka-client
Quarkus configuration (application.properties):
includes among others
quarkus.jaeger.service-name=myservice
quarkus.jaeger.sampler-type=const
quarkus.jaeger.sampler-param=1
quarkus.log.console.format=%d{yyyy-MM-dd HH:mm:ss} %-5p traceId=%X{traceId}, spanId=%X{spanId}, sampled=%X{sampled} [%c{2.}] (%t) %s%e%n
mp.messaging.incoming.mytopic.topic=abc
mp.messaging.incoming.mytopic.key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
mp.messaging.incoming.mytopic.value.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer
...
mp.messaging.incoming.mytopic.interceptor.classes=io.opentracing.contrib.kafka.TracingConsumerInterceptor
With this setup no traceId or spanId is logged at all (even though they should according to Quarkus' "Using OpenTracing" guide). Only after adding #org.eclipse.microprofile.opentracing.Traced a traceId and a spanId is set, but both are completely unrelated to each other on the producer and the consumer.
I checked my opentracing configuration against the beforementioned Quarkus' guide "Using OpenTracing" but found no hints for a misconfiguration on my side.
After reading discussions about issues in some Quarkus extensions relying on ThreadLocals when using with Mutiny I added the artefact quarkus-smallrye-context-propagation to my dependencies, but to no avail.
I suspect that the issue might be related to https://github.com/quarkusio/quarkus/issues/15182, though there it's about reactive routes instead of reactive messaging.
Any ideas?
This issue is not easy to solve, first I will try to explain what happens.
OpenTracing has the concepts of transactions and spans. A span is a block of execution (a method, a database call, a send to a Kafka topic), whereas a transaction is a distributed process that span multiple components (a group of spans).
The issue here is that, each time a span is created, it didn't find any OpenTracing transaction so it creates a new one. This is why none of your spans are correlated to each others.
In OpenTracing, when you create a span, you'll create it based on a span context. Each OpenTracing integration will creates a span context based on the extension technology (I didn't find a better term), for example, HTTP span context is based on HTTP headers and Kafka span context is based on Kafka Headers.
So, to correlate two spans, you need to have the span context created with some context from the underlying technology providing the right OpenTracing ID.
For example, to correlate two Kafka spans, you need to have a uber-trace-id header (this is the default name of the OpenTracing id in Jaeger) with the trace identifier (see tracespan-identity for the format of this header).
Knowing this, there is multiple things to do.
First, you need to add an uber-trace-id Kafka header inside your outgoing message in your #Traced method to correlate the span from the method with the span created inside the Kafka producer interceptor.
Tracer tracer = GlobalTracer.get(); // you can also inject it
JaegerSpanContext spanCtx = ((JaegerSpan)tracer.activeSpan()).context();
// uber-trace-id format: {trace-id}:{span-id}:{parent-span-id}:{flags}
//see https://www.jaegertracing.io/docs/1.21/client-libraries/#tracespan-identity
var uberTraceId = spanCtx.getTraceId() + ":" +
Long.toHexString(spanCtx.getSpanId()) + ":" +
Long.toHexString(spanCtx.getParentId()) + ":" +
Integer.toHexString(spanCtx.getFlags());
headers.add("uber-trace-id", openTracingId.getBytes());
Then, you need to correlate your #Traced method with the span from the incoming message, if any. For this, the easiest way is to add a CDI interceptor that will try to create a span context for all methods annotated with #Traced based on the method parameters (it will search for a Message parameter). For this to work, this interceptor needs to be executed before the OpenTracing interceptor, and sets the span context in the interceptor context.
This is our interceptor implementation, feel free to use it or adapt it for your needs.
public class KafkaRecordOpenTracingInterceptor {
#AroundInvoke
public Object propagateSpanCtx(InvocationContext ctx) throws Exception {
for (int i = 0 ; i < ctx.getParameters().length ; i++) {
Object parameter = ctx.getParameters()[i];
if (parameter instanceof Message) {
Message message = (Message) parameter;
Headers headers = message.getMetadata(IncomingKafkaRecordMetadata.class)
.map(IncomingKafkaRecordMetadata::getHeaders)
.get();
SpanContext spanContext = getSpanContext(headers);
ctx.getContextData().put(OpenTracingInterceptor.SPAN_CONTEXT, spanContext);
}
}
return ctx.proceed();
}
private SpanContext getSpanContext(Headers headers) {
return TracingKafkaUtils.extractSpanContext(headers, GlobalTracer.get());
}
}
This code uses both the Quarkus OpenTracing extension and the Kafka OpenTracing contrib library.
With both the correlation of outgoing message thanks to the addition of the OpenTracing Kafka Header created from the current span context, and the creation of a context from incoming message's header, the correlation should happens in any case.
I am learning spring integration reading/watching a different stuff but I can't understand what service activator is.
I understood that there are two types of integration:
chanel and gateways. chanel is unidirectional integration but gateways is request/reply model. Gateways can be inbound(our system gets request and sends response) and outbound (our system sends request and receives response)
When I read about gateways I often see termin "service activator"
Could you clarify what does it mean ?
The outbound gateway is essentially a particular case for a service activator abstraction for request/reply scenarios. Another case is an outbound channel adapter, which is a one-way, but still can be treated as a service activator because when we send a message to its inputChannel, we are going to call some code - which we can treat as a service. Therefore activating it.
A general component service activator exists there for all the use-cases which are not covered by particular implementation. Let's imaging you need to call some REST service. Right, you can use an HTTP Outbound Gateway with some specific options. Or you can write some custom code which uses a RestTemplate to call that service. you wrap your code into a service activator configuration and you end up with the same behavior for the whole integration solution.
A service activator is a call to a method in a bean.
<service-activator ref="myService" method="aMethod"/>
will call
#Service
public class MyService {
public A aMethod(#Header(value = "param1") String param){
//code
}
}
#Header annotation allows to use an existing value in the header. That is an example.
You can also use it like this:
<service-activator expression="#myService.aMethod('My param')"/>
I am using Apache Camel to implement Rest APIs. I've 2 RouteBuilder types defining all the Camel routes, my application needs. All REST endpoints reside in RestRouter, and it frames the execution using CustomRouter. For example, I've RestRouter to hold my REST routes
public class RestRouter extends RouteBuilder
{
#Override
public void configure() throws Exception
{
rest("/sample")
.post()
.route()
.routeId("postSample")
.to("direct:validate")
.to("direct:save")
.endRest();
}
}
And another RouteBuilder called CustomRouter to bundle non-REST routes.
public class CustomRouter extends RouteBuilder
{
#Override
public void configure() throws Exception
{
onException(ValidationException.class)
.handled(true)
.setBody(simple("${exchangeProperty[CamelExceptionCaught]}"))
.to("bean:exceptionHandler?method=constraintViolationHandler")
.setHeader(Exchange.CONTENT_TYPE, constant(ErrorResponse.class.getName()))
.setHeader(Exchange.HTTP_RESPONSE_CODE, constant(HttpStatus.SC_BAD_REQUEST));
validator()
.type(Sample.class)
.withBean("sampleValidator");
from("direct:validate")
.to("bean-validator://x"); // Runs javax/Hibernate annotated validations
from("direct:save")
.routeId("saveSample")
.inputTypeWithValidate(Sample.class)
.to("bean:sampleRepository?method=save");
}
}
Validation bean SampleValidator is a Camel org.apache.camel.spi.Validator which throws org.apache.camel.ValidationException for any violations.
Problem with setup is that Camel doesn't invoke my custom exception handler for ValidationException. Validation exception occurs for route saveSample. Here's my finding on how it goes further inside Camel processor types.
Control goes to RedeliveryErrorHandler's handleException() where it looks for the exception policy. Root of failing exchange (i.e. RestRouter -> postSample) is expected here to define the exception handler.
Later, Camel goes to failing UnitOfWork (i.e. to saveSample) to identify the exception handler.
That means, for below expression, routeId is from CustomRouter and exceptionPolicy is from the RestRouter. Combination never exists and Camel fails to find the exception processor.
processor = exceptionPolicy.getErrorHandler(routeId)
In above context, I've following questions
Is it a good practice to divide a functionality within multiple RouterBuilder types?
Shouldn't Camel use current UnitOfWork to resolve the exception policy?
Is there some way Camel can invoke my custom handler, provided different RouteBuilder types?
Edit
I can't move to have a single RouterBuilder.
One, because I've an Apache Avro object coming in payload for post from another orchestration service, and then transformation to my JPA entities is done via the bean methods, not using Camel's Transformer. This arrangement doesn't fit with how Camel invokes the Validator (seeContractAdvice). ContractAdvice is a CamelInternalProcessorAdvice which applies Transformer (if intype != outtype) and Validator.
Second, moving to single RouterBuilder will need me to move Avro-to-Entity logic to a Camel Transformer, and that approach would differ greatly with the way we're doing currently. But yes, single RouterBuilder + Transformer + Validator should work.
Have a look at this example from Camel In Action which demonstrates how to reuse the error-handling across route builders defined in Java DSL.
BaseRouteBuilder
and InboxRouteBuilder and OrderRouteBuilder
You can create a base class where you setup the context-scoped error configuration.Then your RouteBuilder classes are extending this base class and calling calling super.configure to get the common configuration.
See if it works when you have all the routes in a single RouteBuilder. "Global" exception handlers such as yours are not really global as they are applied to all routes built by that specific builder, so I wouldn't expect your onException to be applied to the REST route.
Alternatively move the onException in to the REST builder. The handler sets HTTP status codes, so on the surface looks like it would be better packaged with REST routes.
I am currently using a HTTP method for invoking some URL which will create a JIRA issue.
Now I want to use Apache Camel, how can I use that?
I need to invoke the following link through Camel:
http://localhost:8080/rest/api/2/project/" + key + /components
As I'm new to Camel, please suggest some solutions and examples too.
Thanks
See also this FAQ about using dynamic to endpoints in Camel
http://camel.apache.org/how-do-i-use-dynamic-uri-in-to.html
Essentially the EIP pattern for this is the recipient list.
So in your case it could also be simplified to as one EIP
<recipientList>
<simple>http://localhost:8080/rest/api/2/project/${header.myKey}/components</simple>
</recipientList>
Mind the http component in Camel is fully synchronous. If you want to do request/reply over HTTP and avoid having the caller block while waiting for the reply message, then you can use some of the other HTTP components from Camel such as:
camel-ahc
camel-http4
camel-jetty
You could easily use the CXFRS Component; if you need to do it using the HTTP Component for some reason you could easily use that as well:
<setHeader headerName="CamelHttpUri">
<simple>http://localhost:8080/rest/api/2/project/${header.myKey}/components</simple>
</setHeader>
<inOut uri="http://doesnt.matter.we/override/it/anyways" />
And of course you will need to enrich your message with the myKey header before getting to this part of the route.
I am using apache camel jetty
CamelContext context = new DefaultCamelContext();
public void configure(){
context.addRoutes(new RouteBuilder(){
from("jetty:localhost:9000/offers")
.to("direct:getOffers")
.end();
}
});
so here when the user will hit http://localhost:9000/offers then the endpoint direct:getOffers will get invoked
so now defining the getOffers end point
context.addRoutes(new RouteBuilder(){
public void configure(){
from("direct:getOffers")
.to("jetty:http://localhost:9008/api/v2.0/offers?
bridgeEndpoint=true")
.end();
}
});
Here another service is running at 9008 having a rest resource of
http://localhost:9008/api/v2.0/offers and this is the resource that i am trying to consume.
so when camel instance starts it registers both the routes then it does the processing as described above
Note Its important to add the option of ?bridgeEndpoint=true for this to work
You can consume REST service from camel using CXFRS Component.Apache camel has enough information about this.
http://camel.apache.org/cxfrs.html