I want to publish a message on a EventBus channel and receive a response from everyone who's listening to that channel. The thing is, how to I know everyone has already responded if I don't know how many responses to expect?
I assume I need to know how many consumers there are in order to know that I already got all responses or not.
Is there any way to know how many consumers are currently "listening" to a Vert.x EventBus address?
EventBus works on a "best effort" basis, so even if you could track the number of subscribers, there is a chance that some of them would never return your message.
If you're still inclined to try it, there are a couple of ways I can thin of. None are really recommended, but I'll try to highlight the pros and cons anywya.
One is to get a reference to handlerMap inside EventBusImpl through reflection.
https://github.com/eclipse-vertx/vert.x/blob/master/src/main/java/io/vertx/core/eventbus/impl/EventBusImpl.java#L48
If you get it once, when your application starts, that should have much impact on the performance. Of course that will break at runtime if Vert.x team decides to as much as change the name of the field.
Another option is to use vertx.sharedData(), for example getLocalMap(). All consumers will add themselves to the map, and producer will check the map to figure out how many consumers are listening. The problem with that implementation:
Lots of logic to implement this
Unregistering consumers is hard (there's no guarantee that consumer will be able to unregister itself)
Related
I have a situation in which Producer A writes on topics A,B and C however listener for topic C throws an exception. All writes are part of a transaction. I want to know if there is a way that all writes can be rolled back automatically, as if no there were no commits in the first place?
I don't think this can be achieved in Kafka out of the box. I would suggest to re-think the design since Kafka/ messaging system is not the best match for your requirement. Kafka consumers are meant to be independent business logic like a micro-service, even if one fails it should not affect the other. If its so critical you may consider a single topic/webservice with all required info in that topic/request and make the client transactional. Otherwise if non-critical(failure of a topic client is not affecting functionality of another topic client), then introduce some audit/alerting mechanism on top of clients to make sure that they are back online.
I have problem with counting responses from response queue. I mean, once per day we run a job which gather some data from db and send them to queue. When we receive all responses we should shutdown connection. The problem is how we can check if all responses arrived ? Keeping this in global variable is risky because of concurrence issue. Any idea ? I am quite new in JMS so maybe solution is obvious but I dont see it.
I don't know what your stack is or whatever tools you might be using to accomplish this but I've got this in mind and this might help you out (hopefully).
Generate a hash for each job you plan on queuing and store it in a concurrent list/map. (i.e: ConcurrentHashMap)
Send the job to the queue.
Once the job is done and sends back a response, reproduce the hash and store it a separate concurrent list/map that holds all the jobs that are done.
Now that you have two lists of all the jobs supposed to be executed and the jobs that you got a response from. There multiple ways to accomplish this. If you lookup Java Concurrency, you'd find plenty of tutorials and documentation. I like to use CyclicBarrierandCountDownLatch`. If plan on using any of these methods, take extra precautions to prevent your application from hanging or worse, a filthy memory leak.
OR, you could simply check on how many queuing requests and responses you've and if they are equal to each other, drop the connection.
Say I have the following components:
Producer produces numbers and sends messages to Consumer
Both Producer and Consumer send messages to Monitor
Monitor, say randomly, decides when the produce / consume process should stop and sends a message to Stopper
Stopper then stops both Producer and Consumer cleanly
I know this is easy to accomplish in a mutable language such as Java. I know also this can be resolved by allowing partial mutability with interfaces, such as described here.
However, it's not a good practice to have cyclic dependencies even if possible. So, let's assume all references are constructor-injected and final:
Producer has final Consumer and final Monitor
Consumer has final Monitor
Monitor has final Stopper
Stopper has final Producer and final Consumer
I found references such as this, but they don't seem to apply.
How would one go about un-cycling this case and cases such as this in general? In other words, I'm mostly interested in how to accomplish not forming the cycles from a design standpoint. Any hints?
You're right, this won't work if all dependencies are final and injected via the constructor.
But may I ask, why do they have to be injected via the constructor? There is nothing wrong at the end of the day to use setters to wire up beans.
In fact, in Spring, beans are usually instantiated first and injected afterwards. So you could look at that approach.
Other than that, you could look at a different way to model your problem (that does not have circular dependencies).
For example, since you are using queues already to send messages between the producer and consumer, why not also send messages on queues to the monitor? The stopper could also send messages to the producer and consumer.
Or, as Taylor suggests, an ESB.
There are probably many other ways to design it, have a read about (for example) Apache Camel Enterprise Integration Patterns for some ideas.
I understand that there are different ways (or permutations) to implementing a JMS Request-Reply mechanism, i.e. request queue and response queue, request topic and response topic, or a mix of either.
What I would like to know is, (1) what is the recommended (or most common) way and (2) how do the different permutations measure up?
Next, is it more correct to say
a. "Send a message to a queue" or b. "Send a message through a queue"?
Cheers!
Normally, use a queue. "Request" implies a recipient, not a notice to anyone who cares, so you probably want the behaviour of a queue.
Queues usually do better for one thing - or a limited number of peer things - receiving the message and processing it. They also tend to saner persistence models than topic, when it matters that the message actually get to someone who processes it. (eg: if dropping the message is a problem, you probably want a queue)
Topics are more broadcast oriented: say something, and anyone who cares will hear about it. Normally that goes hand-in-hand with "...and no direct response is expected" because the "zero or more listeners" model ... well, zero listeners is always a problem if you expect a response.
Topics can do persistence, but the rules are stranger, and seldom what you actually want.
Finally, I think most people say "to" a queue, because the queue and the thing(s) processing messages off it are distinct, but really, it doesn't matter much as log as you convey your meaning.
Also with a Queue you are able to have multiple subscribers process the messages so its kid of a built in loadbalancer. You cannot do this easily with a Topic.
is there any way two threads within the same process can communicate without knowing anything about each other's interface ?
basically, one thread is a STOMP server, the other is a client. they're supposed to communicate in a direct manner (not via a socket) and it should be independent of the implementation so i can't assume either of the threads is waiting for messages on some common message queue.
what i'm looking for is some kind of a built-in mechanism in java that allows threads within the same process to communicate.
is there such a mechanism ? and if not, is there any other way to approach this ?
You can use a concurrent message queue where threads can post and receive messages. Instead of knowing the other's thread interface, now each thread must be able to create own messages and understand the messages of other threads.
By using a distinct interface for these messages, this is rather easy. And as a bonus, there is a wide range of queues for concurrent access available, so you can pick the queue that fits most to your scenario.
Well, you can have a third thread to act as a message board. But then you'd have to hope that the two threads can agree on a protocol before hand. It would also be rather slow.
Can you provide more details/examples? What do you mean by "communicate" exactly?
There are a few ways I can think of for doing this, shared (global) state, PipedInputStream/PipedOutputStream etc. But the details will depend on what you're trying to do.