My questions is more generic than the following scenario, though this covers everything needed.
It is for Java and the correct practices of socket programming.
Scenario:
One server with many clients. Usage of non-blocking I/O
The server is a client to another server. Usage of blocking I/O
Two cases for each: in one case all data fit inside the allocated bytebuffer, in the second case they do not fit (for one iteration only, not for the lifespan of the program).
All examples that I have found for a non-blocking I/O go something like this:
InetAddress host = InetAddress.getByName("localhost");
Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.configureBlocking(false);
serverSocketChannel.bind(new InetSocketAddress(host, 1234));
serverSocketChannel.register(selector, SelectionKey. OP_ACCEPT);
while (true) {
if (selector.select() <= 0)
continue;
Set<SelectionKey> selectedKeys = selector.selectedKeys();
Iterator<SelectionKey> iterator = selectedKeys.iterator();
while (iterator.hasNext()) {
key = (SelectionKey) iterator.next();
iterator.remove();
if (key.isAcceptable()) {
SocketChannel socketChannel = serverSocketChannel.accept();
socketChannel.configureBlocking(false);
socketChannel.register(selector, SelectionKey.OP_READ);
// Do something or do nothing
}
if (key.isReadable()) {
SocketChannel socketChannel = (SocketChannel) key.channel();
ByteBuffer buffer = ByteBuffer.allocate(BUFFER_SIZE);
socketChannel.read(buffer);
// Something something dark side
if (result.length() <= 0) {
sc.close();
// Something else
}
}
}
Does the read here reads all incoming data from that particular client and that particular request, if the buffer is large enough, or do I need to have it inside a while loop? If the buffer is not large enough?
In case of a write, do I also just do socketChannel.write(buffer) and I am good to go (at least from the programs point of view)?
The doc here does not specify the case when all incoming data fit in the buffer. It also makes it a bit confusing when I have a blocking I/O:
It is guaranteed, however, that if a channel is in blocking mode and there is at least one byte remaining in the buffer then this method will block until at least one byte is read.
Does this mean that here (blocking I/O) I need to read through a while loop either way (most examples that I found does this)? What about a write operation?
So, to sum it up, my question is, what is the proper way to read and write the data in my scenario, from the point of view of the middle server (client to the second server)?
If you had not called configureBlocking(false), then yes, you would use a loop to fill the buffer.
However… the point of a non-blocking socket is not to get hung up waiting on any one socket, since that would delay the reading from all the remaining sockets whose selected keys haven’t yet been processed by your Iterator. Effectively, if ten clients connect, and one of them happens to have a slow connection, some or all of the others might experience the same slowness.
(The exact order of the selected key set is not specified. It would be unwise to look at the source of the Selector implementation class, since the lack of any guarantee of order means future versions of Java SE are permitted to change the order.)
To avoid waiting for any one socket, you don’t try to fill up the buffer all in one go; rather, you read whatever the socket can give you without blocking, by reading only once per select() call.
Since each ByteBuffer might hold a partial data sequence, you’ll need to remember each ByteBuffer’s progress for each Socket. Luckily, SelectionKey has a convenient way to do that: the attachment.
You also want to remember how many bytes you’ve read from each socket. So, now you have two things you need to remember for each socket: the byte count, and the ByteBuffer.
class ReadState {
final ByteBuffer buffer = ByteBuffer.allocate(BUFFER_SIZE);
long count;
}
while (true) {
// ...
if (key.isAcceptable()) {
SocketChannel socketChannel = serverSocketChannel.accept();
socketChannel.configureBlocking(false);
// Attach the read state for this socket
// to its corresponding key.
socketChannel.register(selector, SelectionKey.OP_READ,
new ReadState());
}
if (key.isReadable()) {
SocketChannel socketChannel = (SocketChannel) key.channel();
ReadState state = (ReadState) key.attachment();
ByteBuffer buffer = state.buffer;
state.count += socketChannel.read(buffer);
if (state.count >= DATA_LENGTH) {
socketChannel.close();
}
buffer.flip();
// Caution: The speed of this connection will limit your ability
// to process the remaining selected keys!
anotherServerChannel.write(buffer);
}
For a blocking channel, you can just use one write(buffer) call, but as you can see, the use of a blocking channel may limit the advantages of the primary server’s use of non-blocking channels. It may be worth making the connection to the other server a non-blocking channel as well. That will make things more complicated, so I won’t address it here unless you want me to.
Related
I have a project that I'm working on to better understand Java NIO and network programming stuff. I'm trying to send a 400,000+ byte file through netcat to my server where it will be found and written to a file.
The Problem:
The program works perfectly when the file is below 10,000 bytes, or when I place a Thread.sleep(timeout) before the Select(). The file sends over but only reads 8192 bytes and then cancels out of the loop and goes back to the select() to capture the rest of the data. However the file captures what comes after. I need the complete data for further expansion to the project.
Things I've Tried:
I've tried to load the data onto another byte array which evidently works, but skips over the 8192 bytes (since the select() has been called again). Reads the rest of the 391000 bytes. When comparing the files the first 8192 bytes is missing.
I've tried various other things but I'm not adequate in NIO to understand what I'm messing up on.
My Code
This is just where I feel the code is messing bout (after debugging)
private void startServer() {
File temp = new File("Filepath");
Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.configureBlocking(false);
serverSocketChannel.socket().bind(listenAddress);
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
log.info("Server Socket Channel Started");
while(!stopRequested){
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
for(SelectionKey key : keys){
if(key.isAcceptable()){
try {
serverSocketChannel = (ServerSocketChannel) key.channel();
SocketChannel socket = serverSocketChannel.accept();
socket.configureBlocking(false);
socket.register(selector, SelectionKey.OP_READ);
}catch (IOException e) {
log.error("IOException caught: ", e);
}
}
if(key.isReadable(){ read(key); }
keys.remove(key);
}
}
} catch (IOException e) {
log.error("Error: ", e);
}
}
private void read(SelectionKey key) {
int count = 0;
File tmp = new File("Path");
try {
SocketChannel channel = (SocketChannel) key.channel();
byteBuffer.clear();
while((count = channel.read(byteBuffer)) > 0) {
byteBuffer.flip();
//in bytearrayoutputstream to append to data byte array
byteArrayOutputStream.write(byteBuffer.array(), byteBuffer.arrayOffset(), count);
byteBuffer.compact();
}
}
data = byteArrayOutputStream.toByteArray();
FileUtils.writeByteArrayToFile(tmp, data);
}
}
The above code is what I'm working with. I have more stuff in this class but I believe the main two functions having the problem are these two. I'm not too sure what steps I should take. The file I have to test my program contains many TCPs about 400,000 bytes. The select() collects the initial 8192 bytes and then runs read (which shouldn't happen until it captures all of the data in the stream), comes back and gathers the rest. I've allocated the byteBuffer to be 30720 bytes.
If not clear, I can post the rest of the code, let me know what your suggestions are.
Question
Why does this code only grab 8192 bytes when the allocated space is 30720? Why does it work in debug mode or with Thread.sleep()?
Previous person advised me to place my byteBuffer.clear() outside of loop, even after doing so, the problem persists.
The non-blocking API merely promises that the 'readable' state is raised if there are more than 0 bytes. It makes no guarantee that it'll wait until all the bytes you're interested in have arrived; there isn't even a way to say 'dont mark this channel as isReadable until at least X bytes are in'. There is no way to fix that directly; your code must instead be capable of dealing with a half filled buffer. For example, by either reading this data away so that the 'isReadable' state gets cleared until MORE bytes arrive.
Using the raw non-blocking APIs is rocket science (as in, it is very tricky to write your code correctly, it is easy to get a CPU core to spin to 100% because you're mismanaging your flags, and it is easy to have all threads frozen and the app reduced to being able to handle only a percent or two of what a normal threaded variant could have done due to accidental invocation of blocking methods.
I strongly suggest you first reconsider whether you need non-blocking at all (it always almost slower, and orders of magnitude harder to develop for. After all, you cannot make a single potentially blocking call anywhere in any handler code or your app will be dog slow under load, and java is not great at await/yield stuff – the only real benefit is that you get more finegrained control over buffer sizes, which is irrelevant unless you are very RAM constrained and can get away with tiny buffers for per-connection state). And if you then conclude that truly this is the only way, use a library that makes this API easier to use, such as netty.
Is it necessary to register interest to write to a NIO client socket channel to send data? Do I have to always call socketChannel.register(selector, SelectionKey.OP_WRITE), or some equivalent, before writing to the client SocketChannel to be able to write there?
Would not be enough simply to write data to client SocketChannel with channel.write(outputBuffer) and awake potentially blocked Selector, all in a client thread? The main selector loop would then look like this:
Selector selector = SelectorProvider.provider().openSelector();
serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.configureBlocking(false);
while (selector.select() > 0) {
Set<SelectionKey> readyKeys = selector.selectedKeys();
Iterator<SelectionKey> keyIterator = readyKeys.iterator();
while (keyIterator.hasNext()) {
SelectionKey key = (SelectionKey)keyIterator.next();
keyIterator.remove();
while (keyIterator.hasNext()) {
...
if (key.isAcceptable()) {
...
SocketChannel socketChannel = serverSocketChannel.accept();
socketChannel.configureBlocking(false);
socketChannel.register(selector, SelectionKey.OP_READ);
// client socket channel would be permanently in the read mode
...
} else if (key.isReadable()) {
...
} else if (key.isWritable()) {
// the key should know here that the underlying channel
// has something to be send to the wire, so it should get
// here if there are still data to be sent
socketChannel.write(outputBuffer)
}
It would only get to the if (key.isWritable()) branch when there is something remaining to be send, the remainder of data from the initial channel.write(outputBuffer) call; like when the message is too long and needs to be send out into chunks and I want no blocking. The loop would spin until outputBuffer.hasRemaining() is finally false.
I am even thinking, does have to be writing to Channel, that is sending data out, done through Selector at all? Leave only incoming traffic to be handled with Selector, as only incoming traffic need wait states?
UPDATE
From further reading of valuable user207421 posts, which I partly inspired me to post this question, NIO Javadoc, in the push to join the dots, I summarized this:
SocketChannel and SelectionKey are thread safe.
Reading and writing from/to channel socket outside the selector loop is indeed possible; it's independent of it. Reading outside does not make much sense, writing to socket definitively does and it's in fact how most of writing is done.
Writing of data is typically initiated outside the selector loop, in a different thread. write(buffer) will not necessary send all data - if the message is too huge for the socket buffer, remote client is very slow or even went unresponsive. In that case write() has to be called repeatedly. Buffer updates the current position with every call, depending how much was sent, which can be anything between 0 and buffer.remaining().
If write() returns 0 then, and user207421 cannot stress this more, it's the case, possibly the only one, when OP_WRITE needs to be registered and writing is then handled within selector loop.
When all data are written the OP_READ must be immediately restored
Having OP_WRITE set without any data to write would cause selector loop to endlessly spin; waste of CPU time.
It is unclear what to do if write() returns >0 but still not all remaining message data were sent, that is buffer.remaining() > 0. Should I also set OP_WRITE and pass it to selector loop, or loop write(buffer) inside writing thread until there is no remainder or 0 is returned?
Can actually such situation happen? Official SocketChannel Javadoc says initially Unless otherwise specified, a write operation will return only after writing all of the r requested bytes ..meaning, no, it cannot happen! But then they add: Some types of channels, depending upon their state, may write only some of the bytes ..so does it mean write() can return before sending all? Or does that mean that returns '0' if it sent, say, just half of the buffer?
In the write returned 0 scenario, when writing is passed to the selector loop, ie different thread; is the writing thread blocked to write a completely new message, ie new buffer, to the same socket channel? Or I have to ensure it does not happen by other means? The Javadoc says: This method may be invoked at any time. If another thread has already initiated a write operation upon this channel, however, then an invocation of this method will block until the first operation is complete. Does that cover my scenario?
Official Javadoc does not mention any special meaning for 0 as a return value for write() method. It just says: The number of bytes written, possibly zero. Hmm.
I'm also not sure how unresponsive or slow client is handled during the write. I'm guessing that returning 0 from write() and passing it to selector loop is exactly the reason it's done like that. Selector then can, and I'm only guessing here, manage writing in an optimal way: speeding it, slowing it, increasing intervals, sending smaller chunks, whatever; whatever the underlying TCP/IP stack reports back.
Classic NIO client:
Create a selector
Register channel to selector for READ WRITE
Loop:
select
iterate selectedKeys
do work for Readable channel and Writeable channel
I wrote a simple UDP NIO client for android like above, but found 30k+ HashMapNode memory allocations every 10 seconds. Since the channel was concerned both READ and WRITE, the select() call returned immediately because of it was writeable, during every select(), at least one SelectionKey was added to HashMap returned by SelectedKeys().
I changed the design to only register for READ at the beginning, and call select() with a small timeout(say 10ms), and if the buffer to write was not empty, register for WRITE, do the writes, and register for READ again, memory allocation problem fixed, but write operation will be delayed because you have to wait for the READ select timeout.
Any better approach?
Ok, I think I got it. Here it goes.
Main loop:
Open channel and configureBlocking(false)
Open selector
Register channel to selector, only concern about OP_READ
LOOP:
selector.select() // No timeout
if write buffer is empty or no channel been selected, continue
LOOP for selectionKey in selectedKeys:
if selectionKey is readable
do read operation
else if selectionKey is writeable
do write operation
register channel to OP_READ
remove selectionKey from selectedKeys
Write:
write data to write buffer
register channel to OP_WRITE
selector.wakeup()
If I obtain a SocketChannel that is set to the non-blocking mode, what happens when I write to the channel and the underlying socket buffer is full because the other side cannot keep up? Would the data be discarded or something to that effect?
The write() method returns zero and the data stays in the ByteBuffer. At that point you should register the channel for OP_WRITE, remember the output buffer, return to the select loop. When the channel becomes writable, retry the write, and this time as long as it completes, i.e. doesn't return zero or less than the remaining bytes in the buffer to be written, deregister OP_WRITE.
What is the efficient way of blocking on the socket for data after opening it.The method i used is to call read on input stream (this is a blocking call that waits till some data written to this socket).
//Socket creation
SocketForCommunication = new Socket();
InetSocketAddress sAddr = new InetSocketAddress("hostName", 8800);
SocketForCommunication.connect(sAddr,10000);
is = new DataInputStream(SocketForCommunication.getInputStream());
os = new DataOutputStream(SocketForCommunication.getOutputStream());
//Waiting on socket using read method for data
while(true)
{
int data = is.read();
if(data == HEADER_START)
{
processPackage(is);
}
}
Here problem is read can timeout.Is there a way to register a callback that gets called when data available to read on socket.?
The socket will timeout by default, but you can change this if you really want to. See the Socket.setSoTimeout() call (a timeout of zero means "indefinite").
N.B. Even if you specify a zero timeout, your O/S may or may not actually let you keep a socket open indefinitely. For example, idle sockets may get closed after a certain amount of time. In environments, e.g. shared web hosting environments, it's not uncommon for a housekeeping routine to also run (say) once a day and shut down idle sockets.
And of course, stuff happens on networks. Either way, you shouldn't rely on the socket staying open indefinitely...
You want to use the java.nio (non blocking IO) system. While its not callback driven, it allows you much more flexibility in handling IO.
http://rox-xmlrpc.sourceforge.net/niotut/index.html