I'd like to use the NIO to send/receive data to/from a distant machine. I can send or receive data at any time, when i need to send data i just send it without any queries from the distant machine, and the distant machine send me data at regular interval. I don't understand the NIO mechanism. What generates and read or write event on the Selector SelectionKey ? Is it possible to use only one ServerSocketChannel on my side, to read data from the distant machine et to write data to it ? That is what i understand but i don't see how the writing event can be triggered... Thank you for your explanation.
I already did some coding and i can read data coming in from the distant machine, but cannot write. I use Selector and i don't know how can i write data. The logged message "handle write" is never written, but in wireshark i can see my packet.
public class ServerSelector {
private static final Logger logger = Logger.getLogger(ServerSelector.class.getName());
private static final int TIMEOUT = 3000; // Wait timeout (milliseconds)
private static final int MAXTRIES = 3;
private final Selector selector;
public ServerSelector(Controller controller, int... servPorts) throws IOException {
if (servPorts.length <= 0) {
throw new IllegalArgumentException("Parameter(s) : <Port>...");
}
Handler consolehHandler = new ConsoleHandler();
consolehHandler.setLevel(Level.INFO);
logger.addHandler(consolehHandler);
// Create a selector to multiplex listening sockets and connections
selector = Selector.open();
// Create listening socket channel for each port and register selector
for (int servPort : servPorts) {
ServerSocketChannel listnChannel = ServerSocketChannel.open();
listnChannel.socket().bind(new InetSocketAddress(servPort));
listnChannel.configureBlocking(false); // must be nonblocking to register
// Register selector with channel. The returned key is ignored
listnChannel.register(selector, SelectionKey.OP_ACCEPT);
}
// Create a handler that will implement the protocol
IOProtocol protocol = new IOProtocol();
int tries = 0;
// Run forever, processing available I/O operations
while (tries < MAXTRIES) {
// Wait for some channel to be ready (or timeout)
if (selector.select(TIMEOUT) == 0) { // returns # of ready chans
System.out.println(".");
tries += 1;
continue;
}
// Get iterator on set of keys with I/O to process
Iterator<SelectionKey> keyIter = selector.selectedKeys().iterator();
while (keyIter.hasNext()) {
SelectionKey key = keyIter.next(); // Key is a bit mask
// Server socket channel has pending connection requests?
if (key.isAcceptable()) {
logger.log(Level.INFO, "handle accept");
protocol.handleAccept(key, controller);
}
// Client socket channel has pending data?
if (key.isReadable()) {
logger.log(Level.INFO, "handle read");
protocol.handleRead(key);
}
// Client socket channel is available for writing and
// key is valid (i.e., channel not closed) ?
if (key.isValid() && key.isWritable()) {
logger.log(Level.INFO, "handle write");
protocol.handleWrite(key);
}
keyIter.remove(); // remove from set of selected keys
tries = 0;
}
}
}
}
The protocol
public class IOProtocol implements Protocol {
private static final Logger logger = Logger.getLogger(IOProtocol.class.getName());
IOProtocol() {
Handler consolehHandler = new ConsoleHandler();
consolehHandler.setLevel(Level.INFO);
logger.addHandler(consolehHandler);
}
/**
*
* #param key
* #throws IOException
*/
#Override
public void handleAccept(SelectionKey key, Controller controller) throws IOException {
SocketChannel clntChan = ((ServerSocketChannel) key.channel()).accept();
clntChan.configureBlocking(false); // Must be nonblocking to register
controller.setCommChannel(clntChan);
// Register the selector with new channel for read and attach byte buffer
SelectionKey socketKey = clntChan.register(key.selector(), SelectionKey.OP_READ | SelectionKey.OP_WRITE, controller);
}
/**
* Client socket channel has pending data
*
* #param key
* #throws IOException
*/
#Override
public void handleRead(SelectionKey key) throws IOException {
Controller ctrller = (Controller)key.attachment();
try {
ctrller.readData();
} catch (CommandUnknownException ex) {
logger.log(Level.SEVERE, null, ex);
}
key.interestOps(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
}
/**
* Channel is available for writing, and key is valid (i.e., client channel
* not closed).
*
* #param key
* #throws IOException
*/
#Override
public void handleWrite(SelectionKey key) throws IOException {
Controller ctrl = (Controller)key.attachment();
ctrl.writePendingData();
if (!buf.hasRemaining()) { // Buffer completely written ?
// Nothing left, so no longer interested in writes
key.interestOps(SelectionKey.OP_READ);
}
buf.compact();
}
}
The controller
/**
* Fill buffer with data.
* #param msg The data to be sent
* #throws IOException
*/
private void writeData(AbstractMsg msg) throws IOException {
//
writeBuffer = ByteBuffer.allocate(msg.getSize() + 4);
writeBuffer.putInt(msg.getSize());
msg.writeHeader(writeBuffer);
msg.writeData(writeBuffer);
logger.log(Level.INFO, "Write data - message size : {0}", new Object[]{msg.getSize()});
logger.log(Level.INFO, "Write data - message : {0}", new Object[]{msg});
}
/**
* Write to the SocketChannel
* #throws IOException
*/
public void writePendingData() throws IOException {
commChannel.write(writeBuffer);
}
ServerSocketChannel is used to make a connection, but not send data. You need one ServerSocketChannel and one SocketChannel per each connection.
Examples of reading and writing using SocketChannel:
ByteBuffer buf = ByteBuffer.allocate(48);
int bytesRead = socketChannel.read(buf);
Your program will sleep at second line until data will come. You need to put this code in infinite loop and run it in background Thread. When data came you can process it from this thread, then wait for another data to come.
ByteBuffer buf = ByteBuffer.allocate(48);
buf.clear();
buf.put("Hello!".getBytes());
buf.flip();
while(buf.hasRemaining()) {
channel.write(buf);
}
There is no blocking methods, so if you sending small byte buffer you can call this from your main Thread.
Source
ADD:
Don't set OP_WRITE key on new connection. Only OP_READ. When you want to write some data you need to notify selector that you want to send something and send it in events loop. Good solution is to make a Queue of outcoming messages. Then follow this steps:
adding data to Queue
setting OP_WRITE to channel's key
in while (keyIter.hasNext()) loop you'll have writable key, write all data from queue and remove OP_WRITE key.
It's hard for me to understand your code, but I think you'll find out what's the problem. Also if you want to have only one connection there is no need to use Selector. And this is weird that you binding few ServerSocketChannels.
I would suggest you use blocking NIO (which is the default behaviour for SocketChannel) You don't need to use a Selector but you can use one thread for reading and another for writing.
Based on your example.
private final ByteBuffer writeBuffer = ByteBuffer.allocateDirect(1024*1024);
private void writeData(AbstractMsg msg) {
writeBuffer.clear();
writeBuffer.putInt(0); // set later
msg.writeHeader(writeBuffer);
msg.writeData(writeBuffer);
writeBuffer.putInt(0, writeBuffer.position());
writeBuffer.flip();
while(writeBuffer.hasRemaining())
commChannel.write(writeBuffer);
}
What generates and read or write event on the Selector SelectionKey?
OP_READ: presence of data or an EOS in the socket receive buffer.
OP_WRITE: room in the socket send buffer.
Related
I'm completely new to serial port communication and need some help grasping it.
I need to communicate with a control board. This board can sometimes send events that I need to react to, and I need to send events to the board and await a response.
We have established a protocol where each event is always 12 bytes and the first 2 bytes determine the event type.
I know that when I send a specific message, I need to await a message with specific signifying bytes. At the same time I want it to be possible to react to events that are sent from the board. For instance the board might say that it is overheating, and at the same time I'm asking it to perform some command and reply.
My question is, if I write to the port and block for a second while awaiting the expected response, how I do ensure I don't "steal" the data my listener expects? E.g. do a serial ports work like a stream, where once I've read I've advanced past the point where it can be re-read.
I've done some implementation of this using jSerialComm, hopefully this can shed some light on my question.
First a listener that is registered using the addDataListener method. I want this to trigger when an event is present on the port that starts with "T".
private static LockerSerialPort getLockerSerialPort(final DeviceClient client) {
return MySerialPort.create(COM_PORT)
.addListener(EventListener.newBuilder()
.addEventHandler(createLocalEventHandler())
.build());
}
private static EventHandler createLocalEventHandler() {
return new EventHandler() {
#Override
public void execute(final byte[] event) {
System.out.println(new String(event));
}
#Override
public byte[] getEventIdentifier() {
// I want this listener to be executed when events that start with T are sent to the port
return "T".getBytes();
}
#Override
public String getName() {
return "T handler";
}
};
}
Next, I want to be able to write to the port and immediately get the response because it is needed to know if the command was successful or not.
private byte[] waitForResponse(final byte[] bytes) throws LockerException {
write(bytes);
return blockingRead();
}
private void write(final byte[] bytes) throws LockerException {
try (var out = serialPort.getOutputStream()) {
out.write(bytes);
} catch (final IOException e) {
throw Exception.from(e, "Failed to write to serial port %s", getComPort());
}
}
public byte[] blockingRead() {
return blockingRead(DEFAULT_READ_TIMEOUT);
}
private byte[] blockingRead(final int readTimeout) {
serialPort.setComPortTimeouts(SerialPort.TIMEOUT_READ_SEMI_BLOCKING, readTimeout, 0);
try {
byte[] readBuffer = new byte[PACKET_SIZE];
final int bytesRead = serialPort.readBytes(readBuffer, readBuffer.length);
if (bytesRead != PACKET_SIZE) {
throw RuntimeException.from(null, "Expected %d bytes in packet, got %d", PACKET_SIZE, bytesRead);
}
return readBuffer;
} catch (final Exception e) {
throw RuntimeException.from(e, "Failed to read packet within specified time (%d ms)", readTimeout);
}
}
When I call waitForResponse("command"), how do I know my blocking read doesn't steal data from my listener?
Are these two patterns incompatible? How would one usually handle a scenario like this?
I am trying to use the following code which is an implementation of web sockets in Netty Nio. I have implment a JavaFx Gui and from the Gui I want to read the messages that are received from the Server or from other clients. The NettyClient code is like the following:
public static ChannelFuture callBack () throws Exception{
String host = "localhost";
int port = 8080;
try {
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.handler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(),
new ClientHandler(i -> {
synchronized (lock) {
connectedClients = i;
lock.notifyAll();
}
}));
}
});
ChannelFuture f = b.connect(host, port).sync();
//f.channel().closeFuture().sync();
return f;
}
finally {
//workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
ChannelFuture ret;
ClientHandler obj = new ClientHandler(i -> {
synchronized (lock) {
connectedClients = i;
lock.notifyAll();
}
});
ret = callBack();
int connected = connectedClients;
if (connected != 2) {
System.out.println("The number if the connected clients is not two before locking");
synchronized (lock) {
while (true) {
connected = connectedClients;
if (connected == 2)
break;
System.out.println("The number if the connected clients is not two");
lock.wait();
}
}
}
System.out.println("The number if the connected clients is two: " + connected );
ret.channel().read(); // can I use that from other parts of the code in order to read the incoming messages?
}
How can I use the returned channelFuture from the callBack from other parts of my code in order to read the incoming messages? Do I need to call again callBack, or how can I received the updated message of the channel? Could I possible use from my code (inside a button event) something like ret.channel().read() (so as to take the last message)?
By reading that code,the NettyClient is used to create connection(ClientHandler ),once connect done,ClientHandler.channelActive is called by Netty,if you want send data to server,you should put some code here. if this connection get message form server, ClientHandler.channelRead is called by Netty, put your code to handle message.
You also need to read doc to know how netty encoder/decoder works.
How can I use the returned channelFuture from the callBack from other parts of my code in order to read the incoming messages?
share those ClientHandler created by NettyClient(NettyClient.java line 29)
Do I need to call again callBack, or how can I received the updated message of the channel?
if server message come,ClientHandler.channelRead is called.
Could I possible use from my code (inside a button event) something like ret.channel().read() (so as to take the last message)?
yes you could,but not a netty way,to play with netty,you write callbacks(when message come,when message sent ...),wait netty call your code,that is : the driver is netty,not you.
last,do you really need such a heavy library to do network?if not ,try This code,it simple,easy to understanding
I'm writing a server to exchange messages among clients. One issue left to be solved is how to release a channel when a client happens to be closed. What I do is to start a monitor thread in which the all-Clients map is monitored, and I attempt to remove() a channel if an exception been detected when trying write() to it. However, after closing a client, the write() method in monitor thread don't throw an exception so the useless channel will never be released. Anybody know why?
public class ServerMonitor extends Thread{
private Map<String, SocketChannel> allClients;
private Set set;
private Iterator it;
private Entry entry;
private SocketChannel channel;
private ByteBuffer buf;
public ServerMonitor(Map<String, SocketChannel> allClients) {
this.allClients = allClients;
buf = ByteBuffer.allocateDirect(10);
byte b = 0;
buf.put(b);
buf.flip();
}
public void run(){
while(true) {
if(!allClients.isEmpty()) {
set = allClients.entrySet();
it = set.iterator();
while(it.hasNext()) {
entry = (Entry) it.next();
channel = (SocketChannel) entry.getValue();
try{
channel.write(buf);
} catch(Exception e) {
allClients.remove(entry.getKey());
//set.remove(entry);
}
}
}
try {
Thread.sleep(1000 * 5);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
Writing to a TCP socket is buffered locally and put on the wire asynchronously. So you can't rely on the first write after the peer closes to fail. You can rely on a subsequent write failing, but it could take a number of writes to get there.
I've run into this issue when writing applications that send data over TCP. You've discovered that the only real way to know if a client has closed the connection is by the IOException on a call to write(...). This is pretty much the way it works.
There is a cleaner solution. First of all, you must always handle the case that a client disconnects without you knowing, and properly remove them when you get the IOException on write(...). However, if the client sends a message telling the server it is disconnecting, you can use that to close the connection when you see it.
I'm trying to use server side code based on java NIO(non blocking) from 'The Rox Java NIO Tutorial'. There are lot of incoming socket connections and I would like to accept only 100. So if there are 100 active connections then new ones should be rejected/refused. But how to do that? There is only method ServerSocketChannel.accept() which returns SocketChannel object. Using that object I can call socketChannel.socket().close(), but connection is already open. Here is part of the code:
#Override
public void run() {
while (true) {
try {
// Wait for an event one of the registered channels
this.selector.select();
// Iterate over the set of keys for which events are available
Iterator selectedKeys = this.selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
if (!key.isValid()) {
continue;
}
// Check what event is available and deal with it
if (key.isAcceptable()) {
this.accept(key);
} else if (key.isReadable()) {
this.read(key);
} else if (key.isWritable()) {
this.write(key);
}
}
} catch (Exception e) {
logger.warn("Reading data", e);
}
}
}
and accept() mehod:
private void accept(SelectionKey key) throws IOException {
// For an accept to be pending the channel must be a server socket channel.
ServerSocketChannel serverSocketChannel = (ServerSocketChannel) key.channel();
// Accept the connection and make it non-blocking
if (noOfConnections < MAX_CONNECTIONS) {
SocketChannel socketChannel = serverSocketChannel.accept();
Socket socket = socketChannel.socket();
socket.setKeepAlive(true);
socketChannel.configureBlocking(false);
// Register the new SocketChannel with our Selector, indicating
// we'd like to be notified when there's data waiting to be read
socketChannel.register(this.selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);//listener for incoming data: READ from client, WRITE to client
noOfConnections++;
logger.info("Accepted: " + socket.getRemoteSocketAddress().toString());
} else {
// REJECT INCOMING CONNECTION, but how?
logger.warn("Server is full: " + noOfConnections + " / " + MAX_CONNECTIONS);
}
}
If connection is not accepted then accept() method is being called over and over.
Thanks for help!
There is no way to accomplish that, but I doubt that that's what you really want, or at least what you really should do.
If you want to stop accepting connections, change the interestOps in the server socket channel's selection key to zero, and change it back to OP_ACCEPT when you are ready to accept again. In the interim, isAcceptable() will never be true, so the problem you describe won't occur.
However that won't cause further connections to be refused: it will just leave them on the backlog queue where in my opinion and that of the designers of TCP they belong. There will be another failure behaviour if the backlog queue fills up: its effect in the client is system-dependent: connection refusals and/or timeouts.
I think any tuning of a backlog queue hardly ever would be a good solution. But probably, you can just stop listening.
Well, I managed this problem next way:
Pending-state connections on socket are in kind of "middle_state", that mean you cannot control/reject them.
Backlog socket parameter may be used/ignored/treated in different way by specific VM.
That mean you have to accept particular connection to receive associated object and operate it.
Use one thread to accept connection, pass accepted connection to second thread for processing.
Create some variable for number of active connections.
Now, while number of active connections is less than wished maximum, accept connection, rise the number by 1, and pass to second thread for processing.
Otherwise, accept connection and close that immediately.
Also, in connection process thread, than finished, decrease the number of active connections by 1 to point there is one more free channel available.
EDT: Just made the "stub" for server machanism for Java.Net NIO.
May be adapted for OP needs:
package servertest;
import java.io.IOException;
import java.net.InetAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Servertest extends Thread {
final int MAXIMUM_CONNECTIONS = 3;
int connectionnumber = 0;
/**
* #param args the command line arguments
* #throws java.io.IOException
*/
public static void main(String[] args){
new Servertest().start();
}
#Override
public void run() {
try {
ServerSocket sc = new ServerSocket(33000, 50, InetAddress.getLoopbackAddress());
while (sc.isBound()) {
Socket connection = sc.accept();
if(connectionnumber<=MAXIMUM_CONNECTIONS){
new ClientConnection(connection).start();
connectionnumber++;
} else {
//Optionally write some error response to client
connection.close();
}
}
} catch (IOException ex) {
Logger.getLogger(Servertest.class.getName()).log(Level.SEVERE, null, ex);
}
}
private class ClientConnection extends Thread{
private Socket connection;
public ClientConnection(Socket connection) {
this.connection=connection;
}
#Override
public void run() {
try {
//make user interaction
connection.close();
} catch (IOException ex) {
Logger.getLogger(Servertest.class.getName()).log(Level.SEVERE, null, ex);
}
connectionnumber--;
}
}
}
im using ZeroMQ to client / server application ,
now in my MT server i try to set timeout, i tried to set on the server :
socket.setReceiveTimeOut(2000);
socket.setSendTimeOut(2000);
with no luck , how do i set timeout on response .
this is my multi thread server code this is taken from zeromq examples for mt server :
/*
* Multithreaded Hello World server in Java
*
* #author Vadim Shalts
* #email vshalts#gmail.com
*
*/
import org.zeromq.ZMQ;
import org.zeromq.ZMQQueue;
class mtserver {
static void main(String[] args) {
final ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket clients = context.socket(ZMQ.ROUTER);
clients.bind ("tcp://*:5555");
ZMQ.Socket workers = context.socket(ZMQ.DEALER);
workers.bind ("inproc://workers");
for(int thread_nbr = 0; thread_nbr < 5; thread_nbr++) {
Thread worker_routine = new Thread() {
#Override
public void run() {
ZMQ.Socket socket = context.socket(ZMQ.REP);
socket.connect ("inproc://workers");
while (true) {
// Wait for next request from client (C string)
byte[] request = socket.recv (0);
System.out.println ("Received request: ["+new String(request,0,request.length-1)+"]");
// Do some 'work'
try {
Thread.sleep (1000);
} catch(InterruptedException e) {
e.printStackTrace();
}
// Send reply back to client (C string)
byte[] reply = "World ".getBytes();
reply[reply.length-1] = 0; //Sets the last byte of the reply to 0
socket.send(reply, 0);
}
}
};
worker_routine.start();
}
// Connect work threads to client threads via a queue
ZMQQueue zMQQueue = new ZMQQueue(context,clients, workers);
zMQQueue.run();
// We never get here but clean up anyhow
clients.close();
workers.close();
context.term();
}
}
Why do you need a timeout? The original examples don't use it.
Why do you need a timeout on the server?
A natural place to put a timeout in the client.
socket.setReceiveTimeOut(int milliseconds); works on a client side.