I have an existing Protocol I'd like to write a java Client for. The Protocol consists of messages that have a header containing message type and message length, and then the announced number of bytes which is the payload.
I'm having some trouble modeling it, since creating a Class for each message type seems a bit excessive to me (that would turn out to be 20+ classes just to represent the messages that go over the wire) I was thinking about alternative models. But I can't come up with one that works.
I don't want anything fancy to work on the messages aside from notifying via publish subscribe when a message comes in and in some instances reply back.
Any pointers as to where to look?
A class for each message type is the natural OO way to model this. The fact that there are 20 classes should not put you off. (Depending on the relationship between the messages, you can probably implement common featues in superclasses.)
My advice is to not worry too much about efficiency to start with. Just focus on getting clean APIs that provide the required functionality. Once you've got things working, profile the code and see if the protocol classes are a significant bottleneck. If they are ... then think about how to make the code more efficient.
Related
I have several similar systems which are authoritative for different parts of my data, but there's no way I can tell just from my "keys" which system owns which entities.
I'm working to build this system on top of AMQP (RabbitMQ), and it seems like the best way to handle this would be:
Create a Fanout exchange, called thingInfo, and have all of my other systems bind their own anonymous queues to that exchange.
Send a message out to the exchange: {"thingId": "123abc"}, and set a reply_to queue.
Wait for a single one of the remote hosts to reply to my message, or for some timeout to occur.
Is this the best way to go about solving this sort of problem? Or is there a better way to structure what I'm looking for? This feels mostly like the RPC example from the RabbitMQ docs, except I feel like using a broadcast exchange complicates things.
I think I'm basically trying to emulate the model described for MCollective's Message Flow, but, while I think MCollective generally expects more than one response, in this case, I would expect/require precisely one or, preferably, a clear "nope, don't have it, go fish" response from "everyone" (if it's really possible to even know that in this sort of architecture?).
Perhaps another model that mostly fits is "Scatter-Gather"? It seems there's support for this in Spring Integration.
It's a reasonable architecture (have the uninterested consumers simply ignore the message).
If there's some way to extract the pertinent data that the consumers use to decide interest into headers, then you can gain some efficiency by using a topic exchange instead of a fanout.
In either case, it gets tricky if more than one consumer might reply.
As you say, you can use a timeout if zero consumers reply, but if you think that might be frequent, you may be better off using arbitrary two-way messaging and doing the reply correlation in your code rather than using request/reply and tying up a thread waiting for a reply that will never come, and timing out.
This could also deal with the multi-reply case.
I am trying to design a simple FIX message encoder and decoder to encode (convert to FIX) and decode (convert from FIX) my business domain Order objects. I have designed something, but I am not able to achieve the beautiful design I want. Wanted to see if others who have experience building this kind of things have any better design ideas.
This is what I roughly have: a business Object Order, QuickFIX object Message.
I need to generate NewOrder/Cancel/Replace messages and the message could be different for different exchanges.
I can have ReplaceEncoder --> NewOrderEncoder --> AbstractEncoder, CancelEncoder --> AbstractEncoder.
But if I want another dimension to this, like having custom message generation for different exchanges, then it results in too many combinations of hierarchies.
Is my only bet is to mundanely write different code for different exchanges? How others achieve this? Thanks.
I think you will probably come across a similar problem that we have. That is that each FIX implementation is different. Some use 4.2 others 4.4, some use some tags others ignore them, some use many of their own tags others use very few. What we have done is created general FIX sessions with subclasses for FIX 4.2 and 4.4 and then subclasses for each specific sessions (ie individual brokers). That gives us a reasonable amount of reuse of code for sending and receiving FIX messages. With just the specifics changed for things like handling account names and passwords etc.
For message generation we have a factory method that returns and adapter. All the adapters have the same API which will convert our Business order object in to a FIX Message object. Of course each adapter is specific to the API of the broker. I guess we could probably reuse some code between the adapters but currently we don't.
Is my only bet is to mundanely write different code for different exchanges?
Certainly not. In a FIX message there are compulsory and non compulsory fields. You cannot negotiate on the required fields because then you could not guarantee the authenticity and completeness of the messages. Now I am not saying this is impossible, many counter parties have their own specific user level agreements with exchanges for their own specific messages.
With Quickfix, the XML data dictionary from where the engines confirms the completeness of the messages, is in your hand. Tweak it for your own requirements. You would certainly have multiple sessions. I am not sure if this is possible, haven't tried it myself, does different sessions allow different data dictionaries ? If yes, then use them for different counter parties. If that isn't possible, one way which crosses my mind is add extra code for processing your specific fields, not the whole message, in messages expected from certain counter parties.
One place where I had worked, we were using something on these lines. Receive whatever version you may, but once the message is received convert it into a specific version of FIX message, which only exists inside your system. So your engine basically reads only 1 FIX version of messages. But the added complexity is you have to code a converter. I am not sure how feasible is that for you.
FIX is an extraordinarily slippery protocol when it comes to message definitions.
In practice, every institution that offers a FIX interface has made modifications to the default message set. That means, for instance, a FIX4.4 NewOrderSingle message from counterparty A may have different fields than one from counterparty B.
In fact, counterparty A may have made up some fields whole-cloth and added them in. For any new counterparty, there's a chance you'll encounter fields that you've never seen before.
I've written a few adapters for a few different exchanges, and unfortunately, you're really forced to handle them individually. You may be able to capitalize on some commonalities, but you can't make any assumptions on that until you've reviewed their FIX interface's specs.
So, short answer to your question:
Is my only bet is to mundanely write different code for different exchanges?
Yep, pretty much.
What we ended up doing was writing a base fix layer that applies only the required fix tags. In the fix spec certain tags are flagged as required for each message type.
Once this message had been created we apply a filter to the message that is specific to a broker and instrument type.
ie if you trade options and equities with Goldman and JPMorgan you'd write the following filters:
Goldman-equity
Goldman-option
JPMorgan-Equity
JPMorgan-option
Each would apply vendor and instrument specific fields to the base message.
I'm developing a simple Multicast networked program, and was just curious on the best class structure and OOD patterns that best suit a client/server or client/client network. My dilemma is that I will be sending different kind of messages via datagram, and the receiver just sees a bunch of bytes coming at them. Now, I already implemented a sort of "ID" placeholder as the first byte of all of my byte arrays to distinguish between a message containing "hello world" and one containing the coordinates of a user for instance. The only option seems to be to just have a huge set of case statements in my "receive" method based on what the "ID" is but this seems like bad practice. Just looking for ideas to take advantage of Java's OOD patterns and all-around good coding practice.
On a side note (I think this is somewhat related...) would it be advantageous for me to ues object streams instead? It seemed to me that I would still be checking instancof on everythign coming in. Thanks!
There are several possibilities, but the Strategy Pattern seems to fit the bill best.
In your situation, I would create an interface with at least two methods one to determine if the class can handle the message (which will check your ID bits) and another to actually process the message. Then you create a separate message processing class for each type of message.
Your incoming message handler would then have a set or list of these message processing objects (typically each of a different class, but all of whom implement the interface). When a message is received, the message handler would iterate through the message processing objects until one can handle the message is found, giving an error if no message processor will handle the message.
There are a few options here, depending on circumstances.
You can certainly send serialized objects, one for each message type. Serialized Java objects typically only work between Java apps, but you can aslo try protocol buffers for interoperable Objects (http://code.google.com/p/protobuf/)
You can send documents in an interoperable format like JSON or XML
You can make up your own format, and parse the text as you like
me, i would probably go with Protocol Buffers
I'm trying to think in the best way on communication for the game I'm writing. The scenario is simple: tcp sockets and request for authentication, map updates, chat updates, etc. What I was thinking to use was set of classes, like User, Map, Creature, etc and have a Message class, which will have enum with message types and Object to store previously mentioned classes. After I will convert this with GSON to json and on other side I will decode it corresponding to the message type indicated by the element of enum. The problem is that I will pass sometimes too much unnecessary data and that's doesn't let me quiet plus the integration of new types of messages will not be very easy neither for me, nor for someone else who might use it. In the previous version I have used my own XML protocol which also doesn't let me very happy.
So what I'm asking is advice for me the better way for communication or maybe some improvement of my idea.
Thanks in advance,
Serhiy.
XML and JSOn are intended to make application integration simple, but still be human readable.
If you want a protocol tuned to your needs, I suggest you start by determining what information you want to send and how it would look. Document this before you even start implementing it. That way the data sent will suit your needs. (This is more work BTW which is why it is not done more often)
Here is a generic question. I'm not in search of the best answer, I'd just like you to express your favourite practices.
I want to implement a network protocol in Java (but this is a rather general question, I faced the same issues in C++), this is not the first time, as I have done this before. But I think I am missing a good way to implement it. In fact usually it's all about exchanging text messages and some byte buffers between hosts, storing the status and wait until the next message comes. The problem is that I usually end up with a bunch of switch and more or less complex if statements that react to different statuses / messages. The whole thing usually gets complicated and hard to mantain. Not to mention that sometimes what comes out has some "blind spot", I mean statuses of the protocol that have not been covered and that behave in a unpredictable way. I tried to write down some state machine classes, that take care of checking start and end statuses for each action in more or less smart ways. This makes programming the protocol very complicated as I have to write lines and lines of code to cover every possible situation.
What I'd like is something like a good pattern, or a best practice that is used in programming complex protocols, easy to mantain and to extend and very readable.
What are your suggestions?
Read up on the State design pattern to learn how to avoid lots of switch statements.
"sometimes what comes out has some "blind spot", I mean statuses of the protocol that have not been covered..."
State can help avoid gaps. It can't guarantee a good design, you still have to do that.
"...as I have to write lines and lines of code to cover every possible situation."
This should not be considered a burden or a problem: You must write lines of code to cover every possible situation.
State can help because you get to leverage inheritance. It can't guarantee a good design, you still have to do that.
Designing a protocol is usually all about the application space you are working within. For instance, http is all about handling web pages, graphics, and posts, while FTP is all about transferring files.
So in short, to start, you should decide what application space you are in, then define the actions that need to be taken. Then finally, before you start designing your actual protocol, you should seriously, seriously hunt for another protocol stack that does what you want to do and avoid implementing a protocol stack altoether. Only after you have determined that something else pre-built absolutely won't work for you should you start building your own protocol stack.
In C++ you can use Boost::Spirit library to parse your protocol message easily. The only "difficulty" is to define the grammar of your message protocol. Take a look at Gnutella source code to see how they solve this problem. Here http://www9.limewire.com/developer/gnutella_protocol_0.4.pdf is the Gnutella protocol specifications
Finite State Machine is what you want
FSM
So you define a whole bunch of states that you can be in as a receiver or sender (idle, connecting_phase1, connecting_phase2, packet expected,...)
Then define all the possible events (packet1 arrives, net closes, ...)
finally you have a table that says 'when in state x and event n happens do func y and transition to state q' - for every state and event (many will be null or dups)
Edit - how to make a FSM (rough sketch)
struct FSMNode
{
int m_nextState;
void (m_func*);
}
FSMNode states[NUMSTATES][NUMEVENTS]=
{ // state 0
{3, bang}, // event 0
{2,wiz},
{1, fertang}
}
{
{1, noop}, // event 0
{1, noop},
{3, ole}
}
.......
FSMNode node = states[mystate][event];
node.m_func(context);
mystate = node.m_nextState;
I am sure this is full of invalid syntax - but I hope you get the drift
Why not use XML as your protocol? You can encapsulate and categorize all your pieces of data inside XML nodes
Can't give you an example myself, but how about looking at how other (competent) people are doing it?
Like this one?
http://anonsvn.jboss.org/repos/netty/trunk/src/main/java/org/jboss/netty/handler/codec/http/
P.S. for that matter, I actually recommend using netty as your network framework and build your protocol on top of it. It should be very easy, and you'll probably get rid of bunch of headaches...
If you are using Java, consider looking at Apache MINA, it's documentation and samples should inspire you in the right way.
Right-click the network connection icon in the System Tray.
Click Troubleshoot problems.
The troubleshooter may find and fix the problem, in this case, you can get quickly started with your business.
If the troubleshooter can't fix the Winsocks problem, then you may get an error looking like:
"One or more network protocols are missing on this computer"