I'm currently developing two Java networking applications for school projects. One over TCP and the other one over UDP. In both I have to implement simple custom protocol.
Even though I'm trying pretty hard, I can't find a way how to correctly test this kind of apps, or better develop with test first development.
If I have a client and I want real test without stubbing everything out, I have to implement server with simulated behaviour, which in case of simple apps like these is almost the whole project. I understand, that when something big, than writing few lines of Perl script to test it could really help.
Right now I'm developing server and client simultaneously, so that I can at least test by hand, but this doesn't seem like a clean way to develop. The only thing that is helping is tunneling the connection through logger, so that I can see all the data that goes through (using TunneliJ plugin for IDEA).
What is the best way to TDD a networking application with custom protocol? Should I just stub everything and be fine with it?
Separate the protocol from the network layer. Testing the logic of the protocol will become easier once you can feed it your own data, without the need to go through the network stack. Even though you are not using Python, I'd suggest to look at the structure of the Twisted framework. It's a nice example of how to unit-test networking applications.
We wound up with the same problem a while ago. We decided it was simpler to put two developers on the task: one to write the server and one to write the client. We started working in the same office so that we could code, test, modify, repeat a little bit more easily.
All in all, I think it was the best solution for us. It gave us the ability to actually test the program in conditions there were not ideal. For instance, our Internet went out a couple of times and our program crashed, so we fixed it. It worked rather well for us, but if you are a sole developer, it may not be the solution for you.
Whatever you do, when writing a custom protocol, I would check out Wireshark for monitoring your network traffic to make sure all of the packets are correct.
In my app I have code such as this
m_socket.receive(packet);
doSomething(packet);
I mock up the receive and hence can exercise everything that doSomething() needs to do.
Where does this break down for you? Here you are truly unit testing that your code behaves correctly, you can also mock the socket send, and se expectations for what you think should be sent according to your protocol.
We are of course not actually testing that the other end of the protocol is happy. That's integration testing. I always hanker after getting to IT as soon as possible. It's when you interact with the "other end" that you find the interesting stuff.
You are in the luck position of being in control of both ends, in that position I would probably spend some time instrument to create suitable, controllable test harnesses.
Related
What is the best way to simulate a network in Java?
I'm in the early stages of a networked peer to peer project, and to determine some of the required characteristics of the clients I'd like to be able to simulate 100+ instances concurrently on my PC.
Ideally I'd like to create a "simulation" version of the sockets, with their own inputs and output streams. Eventually, I'm going to use these streams for data transfer instead of just moving data around between java objects, so what I'm wanting to simulate is the kind of latency, data loss and other errors you might get in an actual network.
Ideally these simulation methods would be very close to the actual stream standards of java.net.*, so I wouldn't need to do much of a rewrite in order to move from simulation to the actual client.
Can anyone point me in the right direction?
You can use Akka to create millions of Actors on a single machine, then organize communication between them similar to 'real' network.
Here's an example project:
https://github.com/adelbertc/scalanet
Well you don't really need to use any tools but put your brains to design it better.
You need interfaces for the under lying communication framework.
All you need is to mock/substitute the real implementation with a dummy one once you have coded against the interfaces.This dummy implementation can introduce features like latency,dummy data etc.
You can go with spring container.You can write some dummy server sockets in the container to simulate conversations between multiple instances or better use a web container to take that headache away from you.
For simulation purposes you may want to check Omnet++, I'ts great for big scaled simulations with built in data analysis/statistics tools.
Writing is similar to c++, see the tutorials it's pretty straight-forward.
Example 6 hosts network (Taken from Omnet++ tutorial)
I am working on a Spring web application where I have a need to interact with a remotely based command-line java application to run a simple search query on this application and get back the results. I initially had integrated this into my Spring app but my app is, itself, needing a lot of memory (its an app that involves huge amounts of data) and I don't think they can coexist on one server anymore.
I am running everything on Amazon ec2 so the latency between the servers should be really low. I figure I could use a direct SSH connection but am not so sure if this is the best approach. I'd like to keep the command-line app I am interacting with as simple as possible (would rather not make it into a web-service if I don't have to). I'm still fairly new to Java so sorry if this sounds like a basic question.
You have several options other than a web service. Some of them are:
Protocol Buffers
JMS
Simple socket based client/server Java
Thrift
Assuming you have or can have spring on both ends -
Exposing service objects and consuming them from a different process becomes extremely trivial using Spring's remoting support. (RmiServiceExporter may be most appropriate and least trivial to setup in this case)
It really does away with all the boiler plate code and let's you focus on your business/service logic.
You can write a hello world service and consume it from another Java program in less than twenty minutes. Once you have this "infrastructure" setup, you are free to focus on your actual business logic.
(You absolutely don't have to know rmi to get this working though rmi working knowledge may help if you run into problems. But then, what is SO community for? ;))
I have a background in protocol stack - for 3g handsets. Now I need to communicate from an Android slate PC to a server, and I will code both sides of the interfacet.
Update: I ought to have said, from Android (multiple) slate to local server (multiple), then over satellite to a single central master server.
And now I think that I might not be implementing, just designing so looking for something easy for junior engineers to handle
SOAP looks ok, but are their any good IDEs to develop something for SOAP (or GSOAP) for Android (not sure yet which o/s the server will run; with luck I will get to choose).
Or should I just roll my own and use TCP/IP? (I have feeling, which I can't justify, SOAP might be quicker to develop and easier for others to maintain).
If I roll my own, I can just use C or C++ at both ends. IF SOAP, can I use C/C++ on Android (I know that I can if it's non-SOAP), or do I go with Java? And, if I do, should I then go with Java at the server, for maintainability?
Final note: I guess that SOAP will add an overhead and I will be doing this over a satellite link, where every byte costs.
Does any of that make sense, or do I need to explain it better?
Don't use SOAP if you control both ends. Think about JSON (or JSON-RPC) over HTTP or roll your own using prtocol buffers
SOAP is OK for B2B (remember that term, eh?) communications, but is very heavy and not pleasant to deal with. It might make sense to expose a SOAP interface to the outside world, especially in a Microsoft environment, but it is not ideal for internal use.
UPDATE:
If using SOAP both ends do not have to be the same language/library. There are some quirks, I'm sure, but generally interoperability while not great, exists. So you can use Java on the device and C++ on the server, or the other way around, doesn't matter. Regarding IDE -- for Java Eclipse has some relatively bad but usable plugins.
Now on to the satellite links... ARE YOU KIDDING ME? Seriously, depends on the size of your objects. If your requests and responses are going to be big, it might be tolerable... maybe. But if you are going to do a lot of small requests with short responses you are looking at 90% overhead. Check out these sample SOAP messages: http://www.w3schools.com/soap/soap_example.asp
I need a server. A simple one, to control a couple of computers. There are already a couple of programs in the lab, that perform some calculation and monitor tasks. They are executed on these computers. So I need a server to control them - to see the real time data from these computers, I want these programs to upload the calculation data to the server, upload also some files, that come together with this data. So the server needs to have a simple database. I also want to alter some of the calculation parameters in the realtime.
Because, you see, I'm a little tired of opening each computer with the terminal, looking at the process, get the files from each of the computers by ftp, put these files in the corresponding folder on the file storage, writing the schedule, when each program should continue it's work.
Maybe there is some middleware, that I can use for such needs? It should be simple and extensible. i thought of writing such server from scratch, it is not a big problem, but I have a severe time shortage and many other things to do.
And it would be cool, that this server would be developer-friendly. So I could just take it's API and write whatever I need.
I'm using Java, so it would be great, that this server would also "understand" Java. ;-) RMI is cool, but because of the network architecture, I'd prefer to use plain TCP/IP for these needs. Becacuse there is always problem with setting up RMI, when there computers are in differed subnetworks.
Thank you very much for your support in advance! Please help me, otherwise my girlfriend would break up with me, because I don't see her often spending most of my time at the lab... ;-(
I am almost finishing a software like that (actually 3 softwares) the server, the clients and the admin that logs into the server and command the clients.
My problem was specific so I had to go for a custom build from scratch (TCP/IP sockets). Its not hard, just write down the protocol.
If RMI doenst help you, then you must consider making your own proto, and you could exten and add new features later.
Maybe Google Protocol Buffers would help you to build your proto
http://code.google.com/p/protobuf/
Hmm, the two that spring to mind are Jetty and Glassfish. Depends a lot on what you need to do and how you want to go about it. Both are java based.
This seems like a problem for which Bundle-Bee was created for.
Has anyone used or is aware of a service browser to test AMF calls? I am looking for a tool similar to ZamfBrowser ( http://www.zamfbrowser.org ), but one that works for the Java environment. ZamfBrowser is geared towards AMFPHP.
The idea here is to provide a service browser, that allows developers to test Java services using the AMF protocol, without having to go through the Flex UI all the time. There has got to be something out there already for this, but I can't seem to locate anything..... It's kind of funny and strange that a service browser exists for AMFPHP but not for regular AMF calls in a Java environment.
I would imagine something exists under Blaze or LCDS? ... Trying to find it in the docs but can't seem to find anything ....
The best alternative I can think of at the moment is to use FlexMonkey to record stuff, and then to simulate it using that....which is okay I guess but still sucks because you have to go in and create the Flex UI first, whereas with something like ZamfBrowser, you simply point it at the service calls, it tells the server-side developers if their code works, etc. generates the required as3 classes for you... and makes the integration process much easier in a large team.
Any help or insight would be appreciated :)
Thanks!
There is quick and simple Pinta tool for testing AMF services (any programming language). It is not so powerful as SoapUI but it is 100 times smaller in size :) Installed both of them on my PC.
I think you are seeking BlazeMonster
Alright, so maybe all it took was a few more minutes of Googling. I found SoapUI http://www.soapui.org/tutorials/amf/amf-tutorial.html, which seems to be the tool for the job. The name obviously doesn't sound like it would have anything to do with testing AMF calls, but ye it seems to do the job, also supports JMS...
I'd still be interested in any other tools available, especially if they have the capability of doing more sophisticated scenarios for the AMF calls.
Thanks,
Tehsin Bhayani