I am using spring boot convertAndSend() to publish a message on a delayed exchange but the message is never being published on the queue and there isn't any exception being thrown
is never being published on the queue and there isn't any exception being thrown
When a system wants to communicate via a message broker the developer needs a clear understanding of the delivery semantics. At first one needs to know if and how often a message will be delivered to the broker (and potential consumers):
At most once – the message is delivered at most once but also not at all.
At least once – the message guaranteed to be delivered but can be delivered multiple times.
Exactly once – the message is guaranteed to be delivered exactly once.
The reasons why your messages are lost is because probably you are using at most once semantics.
You can configure at least once delivery semantics if you follow this guide
Does this solve your problem ? Tell me in the comments.
Related
Good morning, I am currently managing a queue jms [wso2] message broker.
I have a java client who sends a message in the queue and a java ServletContextListener which activates every time a message is being delivered.
And everything works ok.
My bosses have asked me now if it was possible that once the message arrives in the queue, it automatically makes a call to a service.
I was reading something like [wso2] ESB Message Processor.
My questions are:
1) Is it possible to do such a thing without using [wso2] ESB? but only [wso2] message Broker.
If you have some ideas.
2) at this point [wso2] ESB would be a consumer and a call from them?
3) If someone can give me an example of how to handle the ESB once the message ne [wso2] Message Broker has arrived.
Thanks in advance
Not sure if I got it right, but here are some thoughts that come into my mind.
1.) I'm not familiar with wso2 mb (using activemq) but I assume that its not possible according to the documentation
2.) Yes, the esb acts as a message consumer (like your java client) and can do various things then. You can call other services, forward the message to another queue etc...Maybe the ESB can do the things you're servlet is doing
3.)You can create a simple proxy in esb that takes the jms message and forwards it to your servlet,service or whatever. There are plenty of cases, regarding jms messages the following link might help.
ESB JMS
One other thing I'm thinking of, if you send your jms message to a jms topic from your java client, you can then create several consumers for the same message. So your existing implementation will subscribe to that topic and you can create a simple second client (or use the esb) that connects to that topic as well. Both will receive the same message and can do whatever processing is needed.
Hope that helps.
I am totally new to spring framework. I am trying to create a java maven project where I can have the connectivity to the rabbitMq and I even before publish the message, I want to check if the queues are alive or not. Is this possible to ping the queue to see if it a alive or not.. I am totally new to this rabbitMQ.
Thanks for the answers
Checking for the availability of a queue is a bit of an anti-pattern with messaging systems.
The message producer should not care if there is something on the other end to receive / process the message. The producer only cares that the RabbitMQ instance is available, with the correct exchange.
If the message must be delivered to a consumer, guaranteed, then the consumer needs to configure the queue with durability in mind and the producer should send the message with the persistence flag to ensure it is written to disk.
...
re-reading your question, i'm wondering if you mean "rabbitmq server" when you say "queue". are you wanting to check if the rabbitmq server is available?
if that is the case, the proper thing to do is use a heartbeat in your RabbitMQ connection. the spring framework should know how to do this, and should respond with some kind of event or other code that executes when the connection dies. i'm not really familiar with spring, though, so i don't know the details of doing that with this framework.
You might check this post or this RabbitMQ page on handling this.
I am working on a project and have decided to use Camel and ActiveMQ. I am attempting to create a route using Java and MQTT endpoints. Within this route I have also incorporated a Processor. This is what my route looks like:
from("mqtt:test?subscribeTopicName=zaq.avila.send")
//.process(new RestProcessor())
.to("mqtt:test?publishTopicName=zaq.avila.receive");
From my understanding, the route is consuming from zaq/avila/send, a processor is applied and then the message is published to zaq/avila/receive. The to() part does not appear to be happening, when I check the console, I see that the processor executes though no message is published to zaq/avila/receive. Also, within the web console I see that the messages in zaq/avila/send for enqueued and dequeued increment even when I only pusblished one message. In addition, if I shutdown ActiveMQ I get the following:
INFO | Waiting as there are still 1 inflight and pending exchanges to complete,
timeout in 7 seconds.
Also:
WARN | Error occurred while shutting down service: Endpoint[mqtt://test?publish
TopicName=zaq.avila.receive]. This exception will be ignored.
java.lang.NullPointerException
These exception make me wonder that the exchange is not completing and something is missing. I need help!
Have a look into Camel MQTT component documentation. There is note that this component can be only used for consuming messages if I understand it correctly.
Note: The component currently only supports polling (consuming) feeds.
It's quite weird. I'll investigate further.
This may not necessarily be the best answer, however, it works.
from("mqtt:test?subscribeTopicName=zaq.avila.send")
.process(new RestProcessor())
.to("jms:topic:zaq.avila.receive");
According to ActiveMQ Doc
MQTT messages are transformed into an JMS ByteMessage. Conversely, the body of any JMS Message is converted to a byte buffer to be the payload of an MQTT message.
I was able to publish an mqtt message to a topic, apply a processor and receive the modified message as an mqtt message even though the specified endpoint is JMS.
If anyone can think of any possible downfalls I would gladly appreciate hearing from you. In my opinion, this removes the need to publish messages as MQTT.
We are running a high throughput system that utilizes tibco-ems JMS to pass large numbers of messages to and from our main server to our client connections. We've done some statistics and have determined that JMS is the causing a lot of latency. How can we make tibco JMS more performant? Are there any resources that give a good discussion on this topic.
Using non-persistent messages is one option if you don't need persistence.
Note that even if you do need persistence, sometimes it's better to use non persistent messages, and in case of a crash perform a different recovery action (like resending all messages)
This is relevant if:
crashes are rare (as the recovery takes time)
you can easily detect a crash
you can handle duplicate messages (you may not know exactly which messages were delivered before the crash
EMS also provides some mechanisms that are persistent, but less bullet proof then classic guaranteed delivery
these include:
instead of "exactly once" message delivery you can use "at least once" or "up to once" delivery.
you may use the pre-fetch mechanism which causes the client to fetch messages to memory before your application request them.
EMS should not be the bottle neck. I've done testing and we have gotten a shitload of throughput on our server.
You need to try to determine where the bottle neck is. Is the problem in the producer of the message or the consumer. Are messages piling up on the queue.
What type of scenario are you doing.
Pub/sup or request reply?
are you having temporary queue pile up. Too many temporary queues can cause performance issues. (Mostly when they linger because you didn't close something properly)
Are you publishing to a topic with durable subscribers if so. Try bridging the topic to queue and reading from those. Durable subscribers can cause a little hiccup in performance too since it needs to track who has copies of all messages.
Ensure that your sending process has one session and multiple calls through that session. Don't open a complete session for each operation. Re-use where possible. Do the same for the consumer.
make sure you CLOSE when you are done. EMS doesn't clear things up. So if you make a connection and just close your app the connection still is there and sucking up resources.
review your tolerance for lost messages in the even of a crash. If you are doing Client ack and it doesn't matter if you crash processing the message then switch to auto. Also I believe if you are using (TEMS - Tibco EMS for WCF) there's a problem with the session acknowledge. So a message is only when its processed on the whole message, we switched from Client ACK to the one that had Dups ok and it worked better)
What is common and useful uses of JMS and Message Driven Beans?
Asynchronous communication: The caller returns quickly, and can continue its work (without creating a new thread) and the message can be processed later. Messages can be stored, and even when the server fails, they can continue to be processed, once the server starts up again. Messages can be distributed to multiple machines (optionally based on rules).
Callers and callees can be decoupled (the caller doesn't have to know, who will consume the message, and how many message consumers there are).
It can have enormous performance advantages compared to synchronous communication. Such a messaging middleware can be crucial for services that have to handle lots of messages per second (think of Twitter for example). But it's not restricted to human readable messages.
Another reason to choose JMS and MDBs is guaranteed delivery. A synchronous, point to point call fails if the receiver is unavailable, but a queue can be set up to guarantee delivery, handle retries or transactional failures, use error queues for messages that are "poison", etc.
There are two transmission models built into JMS: point-to-point using queues and publish/subscribe using topics. Each has its own advantages.
The downside of JMS and MDBs is speed of response. You might like the decoupling, but if you block and wait for a response it'll certainly be slower than a direct remote method invocation, because there are two network trips involved instead of just one.