We Keep Coding

Java Packet Handler Threading

I have a quick general question. I am currently using Netty to handle TCP and UDP packets coming from a client. I have the socket listeners each on a separate thread and it works great.
My concern now is, when traffic starts hitting it heavily, I don't think a single thread for each handler to manage messages will suffice. Is it correct to generate a new thread per message (I feel like it's not)? Or should I use something like a Threadpool for this? Any advice would be greatly appreciated.
Here is some example code for the message handler. I wrote some pseudo code to visualize this process. It might help you guys too.
#Override
public void messageReceived(ChannelHandlerContext ctx, DatagramPacket packet) {
//This will likely need made into a thread
//SocketDecoder.decode(packet.content().toString(CharsetUtil.US_ASCII));
//Handle decoded message, will return CharSequence
DatagramPacket response = Namespace.Decoder.createDatagram(packet, "hello");
ctx.writeAndFlush(response);
}

You need to make an educated guess.
By that I mean to say that you should consider the following programming standpoints for server sided design.
1) What does this server do? Does it just relay messages or is there processing to be done?
If there is processes that need to be done it is highly likely that handling incoming message will not be a concern as the processes may be the limiting factor.
2) How much "traffic" do you expect? What is the estimated server load at any time?
I know this is hard when you build your first server. But you need to know you target. If you think the whole world will use it you really need to prepare for it. Otherwise I would suggest just trying to test yourself.
Now as you know many game servers/messaging servers are multi-servers they have one main server and they delegate work to others which all can communicate with each other if needed. Perhaps that is what you need depending on the type of program you are after.
But really you should try to create a server to test. I would suggest a single threaded message listener/packet handler. And if you need to process something shoot off another processing thread.
Really server design principles like the only you ask only come into full force over thousands of users. And even then it heavily depends on the processes you need to run on the server.

Java Socket Reuse

member EJP commented on my comment here, saying that you could not reuse a Socket that has had a failed connection. I have a tremendous amount of respect for EJP, however, my response is that I find this amazing... If this is true, it would seem to put a severe restriction on the lifespan of any Java app using Sockets - eventually you'd run out, right?
Can anyone clarify the situation, or point to workarounds?

I have figured this out: EJP, you are absolutely correct about this
The issue is with Socket.close() the Java Socket object cannot be reused after this, and since closing either the Input or the OutputStream is going to call close (as per the Javadocs) this is the end point for this object.
However, it seems like it is absolutely possible to create a new Socket object and bind it to the same native socket. The native socket should hopefully have been released by the Java Socket, and be available for reuse, or?
K thanks all for consideration

Activity thread access

I have an app that needs to use a connection to a server over multiple sockets in different activities. I have set up a service which handles the connection throughout the program and holds all the input and output streams. The problem I am having is that i want to access the input streams from different activities using threads and im not sure how to do this without calling the streams each time from the service class in each activity which causes interference problems because multiple threads are retrieving the same data.
Does anyone have any ideas on how i should i should proceed, any help will be greatly appreciated.

well, if i got it correctly; your input stream object seems to be critical section. so you need a mechanism like semaphore that controls the usage of that stream object. otherwise, you can't know if there is an interference. Maybe i'm just wrong but it may lead you to a solution.

java networking, calls to write in average take 4x longer than calls to read, is it normal?

To be more specific, i have written a server with java NIO, and it works quiet well, after some testing i have found out that for some reason, in average a call to the SocketChannels write method takes 1ms, the read method on the other hand takes 0.22ms in average.
Now at first i was thinking that setting the sent/receive buffer values on Socket might help a bit, but after thinking about it, all the messages are very short(a few bytes) and i send a message about every 2 seconds on a single connection. Both sent and receive buffers are well over 1024 bytes in size so this can't really be the problem, i do have several thousand clients connected at once thou.
Now i am a bit out of ideas on this, is this normal and if it is, why ?

I would start by using Wireshark to eliminate variables.

#Nuoji i am using nonblocikng-io and yes i am using a Selector, as for when i write to a channel i do the following:
Since what i wrote in the second paragraph in my post is true, i assume that the channel is ready for writing in most cases, hence i do not at first set the interest set on the key to write, but rather try to write to the channel directly. In case however that, i can not write everything to the channel (or anything at all for that matter), i set the interest set on the key to write(that way the next time i try to write to the channel it is ready to write). Although in my testing where i got the results mentioned in the original post, this happens very rarely.
And yes i can give you samples of the code, although i didn't really want to bother anyone with it. What parts in particular would you like to see, the selector thread or the write thread ?

Java NIO Threading issue with SocketChannel.write()

Sometimes, while sending a large amount of data via SocketChannel.write(), the underlying TCP buffer gets filled up, and I have to continually re-try the write() until the data is all sent.
So, I might have something like this:
public void send(ByteBuffer bb, SocketChannel sc){
sc.write(bb);
while (bb.remaining()>0){
Thread.sleep(10);
sc.write(bb);
}
}
The problem is that the occasional issue with a large ByteBuffer and an overflowing underlying TCP buffer means that this call to send() will block for an unexpected amount of time. In my project, there are hundreds of clients connected simultaneously, and one delay caused by one socket connection can bring the whole system to a crawl until this one delay with one SocketChannel is resolved. When a delay occurs, it can cause a chain reaction of slowing down in other areas of the project, and having low latency is important.
I need a solution that will take care of this TCP buffer overflow issue transparently and without causing everything to block when multiple calls to SocketChannel.write() are needed. I have considered putting send() into a separate class extending Thread so it runs as its own thread and does not block the calling code. However, I am concerned about the overhead necessary in creating a thread for EACH socket connection I am maintaining, especially when 99% of the time, SocketChannel.write() succeeds on the first try, meaning there's no need for the thread to be there. (In other words, putting send() in a separate thread is really only needed if the while() loop is used -- only in cases where there is a buffer issue, perhaps 1% of the time) If there is a buffer issue only 1% of the time, I don't need the overhead of a thread for the other 99% of calls to send().
I hope that makes sense... I could really use some suggestions. Thanks!

Prior to Java NIO, you had to use one Thread per socket to get good performance. This is a problem for all socket based applications, not just Java. Support for non-blocking IO was added to all operating systems to overcome this. The Java NIO implementation is based on Selectors.
See The definitive Java NIO book and this On Java article to get started. Note however, that this is a complex topic and it still brings some multithreading issues into your code. Google "non blocking NIO" for more information.

The more I read about Java NIO, the more it gives me the willies. Anyway, I think this article answers your problem...
http://weblogs.java.net/blog/2006/05/30/tricks-and-tips-nio-part-i-why-you-must-handle-opwrite
It sounds like this guy has a more elegant solution than the sleep loop.
Also I'm fast coming to the conclusion that using Java NIO by itself is too dangerous. Where I can, I think I'll probably use Apache MINA which provides a nice abstraction above Java NIO and its little 'surprises'.

You don't need the sleep() as the write will either return immediately or block.
You could have an executor which you pass the write to if it doesn't write the first time.
Another option is to have a small pool of thread to perform the writes.
However, the best option for you may be to use a Selector (as has been suggested) so you know when a socket is ready to perform another write.

For hundreds of connections, you probably don't need to bother with NIO. Good old fashioned blocking sockets and threads will do you.
With NIO, you can register interest in OP_WRITE for the selection key, and you will get notified when there is room to write more data.

There are a few things you need to do, assuming you already have a loop using
Selector.select(); to determine which sockets are ready for I/O.
Set the socket channel to non-blocking after you've created it, sc.configureBlocking(false);
Write (possibly parts of) the buffer and check if there's anything left. The buffer itself takes care of current position and how much is left.
Something like
sc.write(bb);
if(sc.remaining() == 0)
//we're done with this buffer, remove it from the select set if there's nothing else to send.
else
//do other stuff/return to select loop
Get rid of your while loop that sleeps

I am facing some of the same issues right now:
- If you have a small amount of connections, but with large transfers, I would just create a threadpool, and let the writes block for the writer threads.
- If you have a lot of connections then you could use full Java NIO, and register OP_WRITE on your accept()ed sockets, and then wait for the selector to come in.
The Orielly Java NIO book has all this.
Also:
http://www.exampledepot.com/egs/java.nio/NbServer.html?l=rel
Some research online has led me to believe NIO is pretty overkill unless you have a lot of incoming connections. Otherwise, if its just a few large transfers - then just use a write thread. It will probably have quicker response. A number of people have issues with NIO not repsonding as quick as they want. Since your write thread is on its own blocking it wont hurt you.