I'm trying to think in the best way on communication for the game I'm writing. The scenario is simple: tcp sockets and request for authentication, map updates, chat updates, etc. What I was thinking to use was set of classes, like User, Map, Creature, etc and have a Message class, which will have enum with message types and Object to store previously mentioned classes. After I will convert this with GSON to json and on other side I will decode it corresponding to the message type indicated by the element of enum. The problem is that I will pass sometimes too much unnecessary data and that's doesn't let me quiet plus the integration of new types of messages will not be very easy neither for me, nor for someone else who might use it. In the previous version I have used my own XML protocol which also doesn't let me very happy.
So what I'm asking is advice for me the better way for communication or maybe some improvement of my idea.
Thanks in advance,
Serhiy.
XML and JSOn are intended to make application integration simple, but still be human readable.
If you want a protocol tuned to your needs, I suggest you start by determining what information you want to send and how it would look. Document this before you even start implementing it. That way the data sent will suit your needs. (This is more work BTW which is why it is not done more often)
Related
dear stackoverflow community.
Currently, I am working on a project. This project shall have a server and clients connecting to it. Because of their simplicity, I'd like to use Java's integrated ServerSockets and Sockets.
In my project, data shall be sent from the client to the server and opposite.
My initial idea is to just send JSON and then parse it as receiver and get the data from that.
I'm a little unsure about that though, since JSON isn't something that's integrated into Java, but comes from Java script. Also, I'm currently using a Multithreaded-Socket-Server, so I have a ClientHandler Thread class. In that class, the messages were received, parsed and the "action" parameter was read out of the JSON and then I did a switch statement with multiple actions and their functions. I don't think that's a good way of doing that either.
So, my question is:
How can I do that better, and maybe do I have to use something else?
Thanks in advance.
It is true that JSON grew out of JavaScript, but it is a reasonable definition language on its own and I don't see any reason you shouldn't use it. There are libraries for parsing it so you don't have to.
Assuming your JSON structures are different for different purposes, and complex enough to need different classes to represent them, I like the idea of the JSON having a parameter that identifies the class to which it belongs, after which you can hand off parsing to a class that understands the designated output. Then a class can read the JSON, get the type, and some the specific parsing routine can go from there to an object created for the purpose.
I don't see anything wrong with an action string, either; it's served well enough for Swing and some other UIs, after all. Instead of branching out to a function, depending on complexity again, you could have action classes that all implemented an interface, and the action 'verb' could tell you which one (out of a map, say?) to get and execute the 'performAction()' method on or whatever you want to call it.
I don't know how clear this is from a quick description; would be willing to chat about it in an SO chat room if you care about it.
I am trying to design a simple FIX message encoder and decoder to encode (convert to FIX) and decode (convert from FIX) my business domain Order objects. I have designed something, but I am not able to achieve the beautiful design I want. Wanted to see if others who have experience building this kind of things have any better design ideas.
This is what I roughly have: a business Object Order, QuickFIX object Message.
I need to generate NewOrder/Cancel/Replace messages and the message could be different for different exchanges.
I can have ReplaceEncoder --> NewOrderEncoder --> AbstractEncoder, CancelEncoder --> AbstractEncoder.
But if I want another dimension to this, like having custom message generation for different exchanges, then it results in too many combinations of hierarchies.
Is my only bet is to mundanely write different code for different exchanges? How others achieve this? Thanks.
I think you will probably come across a similar problem that we have. That is that each FIX implementation is different. Some use 4.2 others 4.4, some use some tags others ignore them, some use many of their own tags others use very few. What we have done is created general FIX sessions with subclasses for FIX 4.2 and 4.4 and then subclasses for each specific sessions (ie individual brokers). That gives us a reasonable amount of reuse of code for sending and receiving FIX messages. With just the specifics changed for things like handling account names and passwords etc.
For message generation we have a factory method that returns and adapter. All the adapters have the same API which will convert our Business order object in to a FIX Message object. Of course each adapter is specific to the API of the broker. I guess we could probably reuse some code between the adapters but currently we don't.
Is my only bet is to mundanely write different code for different exchanges?
Certainly not. In a FIX message there are compulsory and non compulsory fields. You cannot negotiate on the required fields because then you could not guarantee the authenticity and completeness of the messages. Now I am not saying this is impossible, many counter parties have their own specific user level agreements with exchanges for their own specific messages.
With Quickfix, the XML data dictionary from where the engines confirms the completeness of the messages, is in your hand. Tweak it for your own requirements. You would certainly have multiple sessions. I am not sure if this is possible, haven't tried it myself, does different sessions allow different data dictionaries ? If yes, then use them for different counter parties. If that isn't possible, one way which crosses my mind is add extra code for processing your specific fields, not the whole message, in messages expected from certain counter parties.
One place where I had worked, we were using something on these lines. Receive whatever version you may, but once the message is received convert it into a specific version of FIX message, which only exists inside your system. So your engine basically reads only 1 FIX version of messages. But the added complexity is you have to code a converter. I am not sure how feasible is that for you.
FIX is an extraordinarily slippery protocol when it comes to message definitions.
In practice, every institution that offers a FIX interface has made modifications to the default message set. That means, for instance, a FIX4.4 NewOrderSingle message from counterparty A may have different fields than one from counterparty B.
In fact, counterparty A may have made up some fields whole-cloth and added them in. For any new counterparty, there's a chance you'll encounter fields that you've never seen before.
I've written a few adapters for a few different exchanges, and unfortunately, you're really forced to handle them individually. You may be able to capitalize on some commonalities, but you can't make any assumptions on that until you've reviewed their FIX interface's specs.
So, short answer to your question:
Is my only bet is to mundanely write different code for different exchanges?
Yep, pretty much.
What we ended up doing was writing a base fix layer that applies only the required fix tags. In the fix spec certain tags are flagged as required for each message type.
Once this message had been created we apply a filter to the message that is specific to a broker and instrument type.
ie if you trade options and equities with Goldman and JPMorgan you'd write the following filters:
Goldman-equity
Goldman-option
JPMorgan-Equity
JPMorgan-option
Each would apply vendor and instrument specific fields to the base message.
I'm developing a simple Multicast networked program, and was just curious on the best class structure and OOD patterns that best suit a client/server or client/client network. My dilemma is that I will be sending different kind of messages via datagram, and the receiver just sees a bunch of bytes coming at them. Now, I already implemented a sort of "ID" placeholder as the first byte of all of my byte arrays to distinguish between a message containing "hello world" and one containing the coordinates of a user for instance. The only option seems to be to just have a huge set of case statements in my "receive" method based on what the "ID" is but this seems like bad practice. Just looking for ideas to take advantage of Java's OOD patterns and all-around good coding practice.
On a side note (I think this is somewhat related...) would it be advantageous for me to ues object streams instead? It seemed to me that I would still be checking instancof on everythign coming in. Thanks!
There are several possibilities, but the Strategy Pattern seems to fit the bill best.
In your situation, I would create an interface with at least two methods one to determine if the class can handle the message (which will check your ID bits) and another to actually process the message. Then you create a separate message processing class for each type of message.
Your incoming message handler would then have a set or list of these message processing objects (typically each of a different class, but all of whom implement the interface). When a message is received, the message handler would iterate through the message processing objects until one can handle the message is found, giving an error if no message processor will handle the message.
There are a few options here, depending on circumstances.
You can certainly send serialized objects, one for each message type. Serialized Java objects typically only work between Java apps, but you can aslo try protocol buffers for interoperable Objects (http://code.google.com/p/protobuf/)
You can send documents in an interoperable format like JSON or XML
You can make up your own format, and parse the text as you like
me, i would probably go with Protocol Buffers
I have an existing Protocol I'd like to write a java Client for. The Protocol consists of messages that have a header containing message type and message length, and then the announced number of bytes which is the payload.
I'm having some trouble modeling it, since creating a Class for each message type seems a bit excessive to me (that would turn out to be 20+ classes just to represent the messages that go over the wire) I was thinking about alternative models. But I can't come up with one that works.
I don't want anything fancy to work on the messages aside from notifying via publish subscribe when a message comes in and in some instances reply back.
Any pointers as to where to look?
A class for each message type is the natural OO way to model this. The fact that there are 20 classes should not put you off. (Depending on the relationship between the messages, you can probably implement common featues in superclasses.)
My advice is to not worry too much about efficiency to start with. Just focus on getting clean APIs that provide the required functionality. Once you've got things working, profile the code and see if the protocol classes are a significant bottleneck. If they are ... then think about how to make the code more efficient.
Here is a generic question. I'm not in search of the best answer, I'd just like you to express your favourite practices.
I want to implement a network protocol in Java (but this is a rather general question, I faced the same issues in C++), this is not the first time, as I have done this before. But I think I am missing a good way to implement it. In fact usually it's all about exchanging text messages and some byte buffers between hosts, storing the status and wait until the next message comes. The problem is that I usually end up with a bunch of switch and more or less complex if statements that react to different statuses / messages. The whole thing usually gets complicated and hard to mantain. Not to mention that sometimes what comes out has some "blind spot", I mean statuses of the protocol that have not been covered and that behave in a unpredictable way. I tried to write down some state machine classes, that take care of checking start and end statuses for each action in more or less smart ways. This makes programming the protocol very complicated as I have to write lines and lines of code to cover every possible situation.
What I'd like is something like a good pattern, or a best practice that is used in programming complex protocols, easy to mantain and to extend and very readable.
What are your suggestions?
Read up on the State design pattern to learn how to avoid lots of switch statements.
"sometimes what comes out has some "blind spot", I mean statuses of the protocol that have not been covered..."
State can help avoid gaps. It can't guarantee a good design, you still have to do that.
"...as I have to write lines and lines of code to cover every possible situation."
This should not be considered a burden or a problem: You must write lines of code to cover every possible situation.
State can help because you get to leverage inheritance. It can't guarantee a good design, you still have to do that.
Designing a protocol is usually all about the application space you are working within. For instance, http is all about handling web pages, graphics, and posts, while FTP is all about transferring files.
So in short, to start, you should decide what application space you are in, then define the actions that need to be taken. Then finally, before you start designing your actual protocol, you should seriously, seriously hunt for another protocol stack that does what you want to do and avoid implementing a protocol stack altoether. Only after you have determined that something else pre-built absolutely won't work for you should you start building your own protocol stack.
In C++ you can use Boost::Spirit library to parse your protocol message easily. The only "difficulty" is to define the grammar of your message protocol. Take a look at Gnutella source code to see how they solve this problem. Here http://www9.limewire.com/developer/gnutella_protocol_0.4.pdf is the Gnutella protocol specifications
Finite State Machine is what you want
FSM
So you define a whole bunch of states that you can be in as a receiver or sender (idle, connecting_phase1, connecting_phase2, packet expected,...)
Then define all the possible events (packet1 arrives, net closes, ...)
finally you have a table that says 'when in state x and event n happens do func y and transition to state q' - for every state and event (many will be null or dups)
Edit - how to make a FSM (rough sketch)
struct FSMNode
{
int m_nextState;
void (m_func*);
}
FSMNode states[NUMSTATES][NUMEVENTS]=
{ // state 0
{3, bang}, // event 0
{2,wiz},
{1, fertang}
}
{
{1, noop}, // event 0
{1, noop},
{3, ole}
}
.......
FSMNode node = states[mystate][event];
node.m_func(context);
mystate = node.m_nextState;
I am sure this is full of invalid syntax - but I hope you get the drift
Why not use XML as your protocol? You can encapsulate and categorize all your pieces of data inside XML nodes
Can't give you an example myself, but how about looking at how other (competent) people are doing it?
Like this one?
http://anonsvn.jboss.org/repos/netty/trunk/src/main/java/org/jboss/netty/handler/codec/http/
P.S. for that matter, I actually recommend using netty as your network framework and build your protocol on top of it. It should be very easy, and you'll probably get rid of bunch of headaches...
If you are using Java, consider looking at Apache MINA, it's documentation and samples should inspire you in the right way.
Right-click the network connection icon in the System Tray.
Click Troubleshoot problems.
The troubleshooter may find and fix the problem, in this case, you can get quickly started with your business.
If the troubleshooter can't fix the Winsocks problem, then you may get an error looking like:
"One or more network protocols are missing on this computer"