Could someone explain the Broker pattern to me in plain english? Possibly in terms of Java or a real life analogy.
Try to imagine that 10 people have messages they need to deliver. Another 10 people are expecting messages from the previous group. In an open environment, each person in the first group would have to deliver their message to the recipient manually, so each person has to visit at least one member of the second group. This is inefficient and chaotic.
In broker, there is a control class (in this case the postman) who receives all the messages from group one. The broker then organizes the messages based off destination and does any operations needed, before visiting each recipient once to deliver all messages for them. This is far more efficient.
In software design, this lets remote and heterogeneous classes communicate with each other easily. The control class has an interface which all incoming messages can interact with so a sorts of messages can be sent and interpreted correctly. Keep in mind this is not very scalable, so it loses effectiveness for larger systems.
Hope this helped!
Related
I am new to EAI and read that there are 2 ways to achieve EAI
1) Broker/ hub-spoke model
2) ESB
Is broker model a JMS?
I worked on Spring-integration which is lightweight ESB so have some Idea how ESB works.
But not sure about Broker model
Anyone who can ellaborate Broker model and how to implement it.
Thanks in advance
Regards
Ramandeep S.
A Broker or hub and spoke is a integration pattern based on a centralized middleware.
And yes, JMS is an implementation of this pattern.
See this:
Integration Hubs
... When translating the concept of hub and
spoke to the world of integration it is useful to have a closer look
at what a connection between two systems really entails, i.e. what
does the line between two boxes really represent? In some cases, the
line might be a message queue, in other cases it might be a
publish-subscribe topic or in yet other cases it might be the URI. So
depending on the system, having a lot of lines might now immediately a
problem. While it sure would be a pain to setup a lot of message
queues, publish-subscribe topics and URI's are largely logical
concepts and having a lot of them night mean a bit more maintenance
but is unlikely to be the end of the world.
But the Hub-and-Spoke architecture also provides another significant
benefit -- it decouples sender and receiver by inserting an active
mediator in the middle - the hub. For example, this hub can perform
the important function of routing incoming messages to the correct
destination. As such, it decouples the sender of the message from
having to know the location of the receiver. Having all messages
travel though a central component is also great for logging messages
or to control message flow. The Hub-and-Spoke style applied in this
manner is commonly referred to as Message Broker because the hub
brokers messages between the participants.
Data Format Considerations
A Message Broker should also include a protocol translation and data
transformation function. For example, a message may arrive via a
message queue, but has to be passed on via HTTP. Also, location
transparency is only an illusion unless data format translation is
also provided. Otherwise, a change in the destination (i.e. a request
in form of a message is now being serviced by another component) is
very likely to require a change in the message data format. Without a
Message Translator in between, the message originator would also have
to be changed. Therefore, the implementation of this type of
Hub-and-Spoke architecture typically includes data format translation
capabilities.
I have several similar systems which are authoritative for different parts of my data, but there's no way I can tell just from my "keys" which system owns which entities.
I'm working to build this system on top of AMQP (RabbitMQ), and it seems like the best way to handle this would be:
Create a Fanout exchange, called thingInfo, and have all of my other systems bind their own anonymous queues to that exchange.
Send a message out to the exchange: {"thingId": "123abc"}, and set a reply_to queue.
Wait for a single one of the remote hosts to reply to my message, or for some timeout to occur.
Is this the best way to go about solving this sort of problem? Or is there a better way to structure what I'm looking for? This feels mostly like the RPC example from the RabbitMQ docs, except I feel like using a broadcast exchange complicates things.
I think I'm basically trying to emulate the model described for MCollective's Message Flow, but, while I think MCollective generally expects more than one response, in this case, I would expect/require precisely one or, preferably, a clear "nope, don't have it, go fish" response from "everyone" (if it's really possible to even know that in this sort of architecture?).
Perhaps another model that mostly fits is "Scatter-Gather"? It seems there's support for this in Spring Integration.
It's a reasonable architecture (have the uninterested consumers simply ignore the message).
If there's some way to extract the pertinent data that the consumers use to decide interest into headers, then you can gain some efficiency by using a topic exchange instead of a fanout.
In either case, it gets tricky if more than one consumer might reply.
As you say, you can use a timeout if zero consumers reply, but if you think that might be frequent, you may be better off using arbitrary two-way messaging and doing the reply correlation in your code rather than using request/reply and tying up a thread waiting for a reply that will never come, and timing out.
This could also deal with the multi-reply case.
I'm currently in the process of making a pretty large Akka based Java application and I'm running into a couple issues that bug me to no end.
My current package layout looks kinda like this:
My Mobile class serving as the supervisor of the actors inside the actors package.
Since I don't want to create a new set of Actors for every HttpClient and Account, I pass those around in message objects, which are stored in the messages package, together with the endpoint ActorRef that receives the final result. This does however create a very cluttered messages package with a different message for each actor. Eg. MobileForActor1, Actor1ForMobile, MobileForActor2 etc. Now my question is, is there a Convention to use for this sort of stuff that deals with this problem and is my structure (Mobile->Actor1->Mobile->Actor2->etc.) the way Akka wants it to be or do I have to just sort of waterfall the messages (Mobile->Actor1->Actor2->etc.)?
Right now I'm sending a ConnectMessage to my Mobile actor which then sends it to Actor1, Actor1 processes it and sends a new message back to Mobile, Mobile sends that response then to Actor2 and the cycle continues with a new message being created based on the old message. Eg. new Message2(message1.foo, message1.bar, message1.baz, newComputatedResult, newComputatedResult2, etc);
Is this good practice or should I include the old instance (which may contain info that isn't useful anymore) and include the new stuff? Eg. new Message2(message1, newComputatedResult, newComputatedResult2, etc);
Or should I do something completely different?
I thought about using TypedActors but those require the use of a waterfall pattern and I don't know how I would pass on the ActorRef of the listener that wants to receive the final result.
I hope I made myself understandable enough because English is not my maiden languages and that the question is clear to everyone.
I'm a beginning Akka developer and love the idea but since the documentation doesn't cover this very well, I figured this would be the best place to ask. Thanks for reading!
I will venture a few comments in response to this because I've dealt with the same issues in my learning curve of Akka. I think you're asking for some rules of thumb so mine are contained herein.
First, creating actors is incredibly cheap; they are very lightweight. So, why not create one for each HttpClient and Account and give them suitable names derived from their identity? This also avoids you having to pass them around as much, probably, decluttering your code.
Second, keep your message names short, focused and starting with a verb. Each message should tell the actor to do something so you want the name to reflect that by using a verb.
Third, sets of messages go with the actor. I usually declare them in the actor class's companion object so that using them is like ActorClass.MessageName unless it is within ActorClass and then it is just MessageName.
Fourth, append a counter to the name of an actor. I often just combine a counter (use AtomicInteger) with the name of the type (Car-1, Car-2, etc.).
If the hierarchy is important to you, I would recommend only appending the parent actor to the name. Something like Phone-1-in-Car-7 meaning Phone-1 is contained within Car-7. You can then assemble the hierarchy both programmatically and manually by following the parent links.
I think "Message" in ConnectMessage is redundant. Just make the message name be "Connect" or even better "ConnectToThing" (whatever Thing is, if that's relevant).
I wouldn't compound your message names too much like you're suggesting with Message2. Use the minimal amount of information to be useful to whomever is going to read those names. I think the lack of response to this may have resulted from this part of your question. I found it confusing as a lot of detail is missing.
Hope this helps.
If I have many queues and each has an unique ID, would a Hashtable of Queues be the way to go? I know it sounds strange that I'm asking this, but just wondering if there might be a better way for optimization.
Sorry for the lack of information. I'm basically storing queues of messages which are identified by the client id.
The client will request to get messages from the server.
In the case when the ack does not reach the server, the message still remains in the queue until the client makes another attempt to get the oldest message.
The idea is to retain all the messages if the client fails to ack and to retrieve all messages in a FIFO manner.
The question doesn't provide any detail on what you want to do with this. And this is very important, because the usage pattern is critical in determining which data structure is going to be most efficient for your use case.
But I'd say that in the absence of other details, a HashTable of Queues sounds like a sensible choice, with the HashTable using the ID as a key and the corresponding Queue as a value.
Then the following operations will both be O(1) with very low overhead:
Add an item to the queue with a given ID
Pull the first item from the Queue with a given ID
Which is probably the usage pattern you are going to be needing in most cases.....
since java is statically typed, i would definitely use a hashtable... (that is, if we are talking about optimization)
I have an existing Protocol I'd like to write a java Client for. The Protocol consists of messages that have a header containing message type and message length, and then the announced number of bytes which is the payload.
I'm having some trouble modeling it, since creating a Class for each message type seems a bit excessive to me (that would turn out to be 20+ classes just to represent the messages that go over the wire) I was thinking about alternative models. But I can't come up with one that works.
I don't want anything fancy to work on the messages aside from notifying via publish subscribe when a message comes in and in some instances reply back.
Any pointers as to where to look?
A class for each message type is the natural OO way to model this. The fact that there are 20 classes should not put you off. (Depending on the relationship between the messages, you can probably implement common featues in superclasses.)
My advice is to not worry too much about efficiency to start with. Just focus on getting clean APIs that provide the required functionality. Once you've got things working, profile the code and see if the protocol classes are a significant bottleneck. If they are ... then think about how to make the code more efficient.