I am working on a project and have decided to use Camel and ActiveMQ. I am attempting to create a route using Java and MQTT endpoints. Within this route I have also incorporated a Processor. This is what my route looks like:
from("mqtt:test?subscribeTopicName=zaq.avila.send")
//.process(new RestProcessor())
.to("mqtt:test?publishTopicName=zaq.avila.receive");
From my understanding, the route is consuming from zaq/avila/send, a processor is applied and then the message is published to zaq/avila/receive. The to() part does not appear to be happening, when I check the console, I see that the processor executes though no message is published to zaq/avila/receive. Also, within the web console I see that the messages in zaq/avila/send for enqueued and dequeued increment even when I only pusblished one message. In addition, if I shutdown ActiveMQ I get the following:
INFO | Waiting as there are still 1 inflight and pending exchanges to complete,
timeout in 7 seconds.
Also:
WARN | Error occurred while shutting down service: Endpoint[mqtt://test?publish
TopicName=zaq.avila.receive]. This exception will be ignored.
java.lang.NullPointerException
These exception make me wonder that the exchange is not completing and something is missing. I need help!
Have a look into Camel MQTT component documentation. There is note that this component can be only used for consuming messages if I understand it correctly.
Note: The component currently only supports polling (consuming) feeds.
It's quite weird. I'll investigate further.
This may not necessarily be the best answer, however, it works.
from("mqtt:test?subscribeTopicName=zaq.avila.send")
.process(new RestProcessor())
.to("jms:topic:zaq.avila.receive");
According to ActiveMQ Doc
MQTT messages are transformed into an JMS ByteMessage. Conversely, the body of any JMS Message is converted to a byte buffer to be the payload of an MQTT message.
I was able to publish an mqtt message to a topic, apply a processor and receive the modified message as an mqtt message even though the specified endpoint is JMS.
If anyone can think of any possible downfalls I would gladly appreciate hearing from you. In my opinion, this removes the need to publish messages as MQTT.
Related
Currently trying to make 2 Java implementations of Camel work to fire-and-forget Kafka messages for concurrency.
First implementation I got working by using a ProducerTemplate and sending an Exchange to a Direct Camel route that sends to Kafka using template.asyncSend - this allowed me to send messages concurrently because from my understanding, I’m not waiting to get a response because of asyncSend letting me fire and forget messages.
Now in the Rest implementation, I’ve configured the HTTP route to go to a Direct route using rest(), and then from there Direct sets the Exchange Pattern to InOnly and goes to a SEDA and the SEDA route handles sending the exchange message to Kafka. However when I run a concurrency test using the Rest DSL, I run into a RejectedExecutionException from DefaultAsyncProcessorAwaitManager saying that it's waiting on an asynchronous callback for the exchange id?
How can I make the Rest DSL just fire and forget?? In the ProducerTemplate version I don’t run into this error since asyncSend() returns a Future but I never invoke the method that gets the callback/result from the future. I tried seeing if using an async processor in the from(seda).process(async processor).to(kafka) part but I’m stuck on how else to achieve this.
TL;DR: Can I make the camel rest route not wait for the callback and just behave to fire and forget???
EDIT: In either implementation, can I receive acknowledgement that the message was successfully sent to Kafka but not waiting to see if the message was processed over on consumer-side and that it has consumed from the queue? I still haven't been able to wrap my head around this - I want to have concurrent users fire-and-forget messages/send asynchronously and receive acknowledgement that the message is in Kafka, but are these fundamentally contradicting each other? Would wanting to receive acks make this synchronous? But also, I don't understand why we are getting blocked for an asynchronous callback??
Thank you!
I first tried changing rest route from “from(direct).to(kafka)” to “from(direct).setPattern(InOnly).to(seda)” and “from(seda).to(kafka)” Then I tried using async processor instead of regular processor. Both times I expected maybe running the concurrency test would allow me to send a bunch of requests at once without running into this error where I'm waiting for an async callback.
I am using spring boot convertAndSend() to publish a message on a delayed exchange but the message is never being published on the queue and there isn't any exception being thrown
is never being published on the queue and there isn't any exception being thrown
When a system wants to communicate via a message broker the developer needs a clear understanding of the delivery semantics. At first one needs to know if and how often a message will be delivered to the broker (and potential consumers):
At most once – the message is delivered at most once but also not at all.
At least once – the message guaranteed to be delivered but can be delivered multiple times.
Exactly once – the message is guaranteed to be delivered exactly once.
The reasons why your messages are lost is because probably you are using at most once semantics.
You can configure at least once delivery semantics if you follow this guide
Does this solve your problem ? Tell me in the comments.
I have web service that handles requests and publishes messages to RabbitMQ (Written in spring boot).
The problem is when there is no connection I cannot detect it immediately and I am losing all my messages. How can I deal with this problem? I got AMQP Connection error after 30 seconds. In that interval I cannot handle this problem. I want to know when message delivered to RabbitMQ. If it is not delivered I need to store all messages and when rabbitmq is up resend all these messages. By the way, performance is important.
I have read documentations below. I think it could be solved with OperationsCallback but I dont know how..
https://docs.spring.io/spring-amqp/docs/current/reference/html/#scoped-operations
https://docs.spring.io/spring-amqp/docs/current/reference/html/#cf-pub-conf-ret
Thanks in Advance
its rather a rabbit MQ feature more than java or spring, which allows any message that is published to be acknowledged back to the publisher, see the below link for references on how:
https://www.rabbitmq.com/confirms.html
Good morning, I am currently managing a queue jms [wso2] message broker.
I have a java client who sends a message in the queue and a java ServletContextListener which activates every time a message is being delivered.
And everything works ok.
My bosses have asked me now if it was possible that once the message arrives in the queue, it automatically makes a call to a service.
I was reading something like [wso2] ESB Message Processor.
My questions are:
1) Is it possible to do such a thing without using [wso2] ESB? but only [wso2] message Broker.
If you have some ideas.
2) at this point [wso2] ESB would be a consumer and a call from them?
3) If someone can give me an example of how to handle the ESB once the message ne [wso2] Message Broker has arrived.
Thanks in advance
Not sure if I got it right, but here are some thoughts that come into my mind.
1.) I'm not familiar with wso2 mb (using activemq) but I assume that its not possible according to the documentation
2.) Yes, the esb acts as a message consumer (like your java client) and can do various things then. You can call other services, forward the message to another queue etc...Maybe the ESB can do the things you're servlet is doing
3.)You can create a simple proxy in esb that takes the jms message and forwards it to your servlet,service or whatever. There are plenty of cases, regarding jms messages the following link might help.
ESB JMS
One other thing I'm thinking of, if you send your jms message to a jms topic from your java client, you can then create several consumers for the same message. So your existing implementation will subscribe to that topic and you can create a simple second client (or use the esb) that connects to that topic as well. Both will receive the same message and can do whatever processing is needed.
Hope that helps.
I am implementing a client/workers system using ActiveMQ and I would like to implement a manual Message Acknowledgement and Message Rejection.
Why reject messages? If a worker has too many tasks coming at it, I want that worker to tell the broker to re-queue the original message.
I know there are ways to auto acknowledge or implement transactions, but I'd rather have something like this:
Messages need to be acknowledged within 5 seconds
If they are not acknowledged, the broker will send the message to a different worker
Works can manually reject a message at any time
How can I implement this (without just resending the message to the broker manually)
UPDATE:
To rephrase the question slightly:
How can I ensure unacknowledged messages are re-added back to the queue (and re-delivery can go back to the same consumer that previously did not acknowledge it even -- say that consumer went offline and then came back)
ActiveMQ web page about queues:
If a consumer receives a message and does not acknowledge it before
closing then the message will be redelivered to another consumer.
That's what you want, right? So you have to turn off the AUTO_ACKNOWLEDGEMENT mode and use another mode: CLIENT_ACKNOWLEDGE or probably more selective the INDIVIDUAL_ACKNOWLEDGE.
Rejecting a message is not (yet) possible, see ActiveMQ-Docu:
There is no JMS 'unacknowledge'.