We Keep Coding

how to serve 1000s of concurrent connection using Java NIO

My sender is sending 10000 requests per second (or even more) but my ServerSocketChannel is only able to read and process (in thread) 8000 requests (~appx).
Dummy code is like this:
public class NioReceiver {
private int port = -1;
private static String message = null;
public void receive() throws IOException {
// Get the selector
Selector selector = Selector.open();
// Selector is open for making connection
// Get the server socket channel and register using selector
ServerSocketChannel SS = ServerSocketChannel.open();
InetSocketAddress hostAddress = new InetSocketAddress(this.port);
SS.bind(hostAddress);
SS.configureBlocking(false);
int ops = SS.validOps();
SelectionKey selectKy = SS.register(selector, ops, null);
for (;;) {
//Waiting for the select operation...
int noOfKeys = selector.select();
// The Number of selected keys are: noOfKeys
Set selectedKeys = selector.selectedKeys();
Iterator itr = selectedKeys.iterator();
while (itr.hasNext()) {
ByteBuffer buffer = ByteBuffer.allocate(1024 * 60);
SelectionKey ky = (SelectionKey) itr.next();
if (ky.isAcceptable()) {
// The new client connection is accepted
SocketChannel client = SS.accept();
client.configureBlocking(false);
// The new connection is added to a selector
client.register(selector, SelectionKey.OP_READ);
// The new connection is accepted from the client: client
} else if (ky.isReadable()) {
// Data is read from the client
SocketChannel client = (SocketChannel) ky.channel();
String output = null;
buffer.clear();
int charRead = -1;
try {
charRead = client.read(buffer);
} catch (IOException e) {
continue;
}
if (charRead <= 0) {
// client closed
client.close();
} else {
output = new String(buffer.array());
message = output;
try {
new Thread(() -> {
processAndStore(message);
}).start();
} catch (Exception e) {
System.err.println("Thread exception:::" + e.getMessage());
}
} // else if of client.isConnected()
} // else if of ky.isReadable()
itr.remove();
} // end of while loop
} // end of for loop
}
public void processAndStore(String output) {
String exchangeName = null;
String dataLine = null;
String Lines[] = output.split("\r\n");
for (int i = 0; i < Lines.length; i++) {
if (Lines[i].contains("Host: ")) {
exchangeName = Lines[i].substring(6);
}
if (Lines[i].isEmpty()) {
dataLine = Lines[i + 1];
}
}
StringBuffer updatedLastLine = null;
if (dataLine != null) {
if (dataLine.contains("POST")) {
updatedLastLine = new StringBuffer(dataLine.substring(0, dataLine.indexOf("POST")));
} else {
updatedLastLine = new StringBuffer(dataLine);
}
if (!dataLine.equals("")) {
try {
if (updatedLastLine.lastIndexOf("}") != -1) {
updatedLastLine.replace(updatedLastLine.lastIndexOf("}"), updatedLastLine.lastIndexOf("}") + 1, ",\"name\":\"" + exchangeName
+ "\"}");
} else {
return;
}
} catch (StringIndexOutOfBoundsException e) {
System.out.println(updatedLastLine + "::" + dataLine);
System.out.println(e);
}
store(updatedLastLine.toString());
}
}
}
public NioReceiver(int port) {
this.port = port;
}
}
When I am removing processing logic it is able to receive more requests but not all.
how can I improve my code to receive all 10000s incoming requests.

Use a thread pool / message queue instead of creating 1000's of threads for calling processAndStore().
Starting a thread is expensive.
Starting 10000 threads per second? Yikes!
As #EJP said in a comment:
The purpose of NIO is to reduce the number of required threads. You don't seem to have got the message.
In addition to that, profile your code to see where the bottleneck is, rather than guessing.
But, here are some guesses anyway:
Don't use StringBuffer, use StringBuilder.
Reason: See Difference between StringBuilder and StringBuffer.
Don't call lastIndexOf("}") three times.
Reason: lastIndexOf() is a sequential search, so relatively slow. The JVM may or may not optimize the multiple calls away, but if performance is critical, don't rely on it. Do it yourself by assigning result to variable. See also Does Java optimize method calls via an interface which has a single implementor marked as final?

My SerialPortEvent does not receive data using jSSC in a continous loop

I have been trying to use serial communication with my Arduino Uno and have used the library jSSC-2.6.0. I am using a SeriaPortEvent listener to receive bytes from the Serial Port (Arduino) and store them in a linked list.
public synchronized void serialEvent(SerialPortEvent serialPortEvent) {
if (serialPortEvent.isRXCHAR()) { // if we receive data
if (serialPortEvent.getEventValue() > 0) { // if there is some existent data
try {
byte[] bytes = this.serialPort.readBytes(); // reading the bytes received on serial port
if (bytes != null) {
for (byte b : bytes) {
this.serialInput.add(b); // adding the bytes to the linked list
// *** DEBUGGING *** //
System.out.print(String.format("%X ", b));
}
}
} catch (SerialPortException e) {
System.out.println(e);
e.printStackTrace();
}
}
}
}
Now if I send individual data in a loop and don't wait for any response the serialEvent usually prints bytes received, to the Console.
But If I try and wait till there is some data in the linked list the program just keeps on looping and the SerialEvent never adds bytes to the LinkedList, it even does not even register any bytes being received.
This works and the correct bytes are sent by Arduino received by SerialEvent and printed to the Console:
while(true) {
t.write((byte) 0x41);
}
But this method just stucks at this.available() which returns the size of the LinkedList,
as in no data is actually received from the Arduino or receieved by the serialEvent:
public boolean testComm() throws SerialPortException {
if (!this.serialPort.isOpened()) // if port is not open return false
return false;
this.write(SerialCOM.TEST); // SerialCOM.TEST = 0x41
while (this.available() < 1)
; // we wait for a response
if (this.read() == SerialCOM.SUCCESS)
return true;
return false;
}
I have debugged the program and sometimes debugging, the program does work but not always. Also the program only gets stuck when i try and check if there is some bytes in the linkedlist i.e while(available() < 1). Otherwise if I dont check I eventually receive the correct response of bytes from Arduino

Found the answer myself after wasting 4hours. I was better off using the readBytes() method with a byteCount of 1 and timeOut of 100ms just to be on the safe side. So now the read method looks like this.
private byte read() throws SerialPortException{
byte[] temp = null;
try {
temp = this.serialPort.readBytes(1, 100);
if (temp == null) {
throw new SerialPortException(this.serialPort.getPortName(),
"SerialCOM : read()", "Can't read from Serial Port");
} else {
return temp[0];
}
} catch (SerialPortTimeoutException e) {
System.out.println(e);
e.printStackTrace();
}
return (Byte) null;
}

Delays while sending data with Java NIO

I need your advice on a Java NIO package. I have an issue with delays while sending packets over network. The original code is actually my port of the SFML book source code to Java, but here I'll show you only a minimal working example, where the problem is reproduced. Though this code does contain some pieces from SFML library (actually creating a window and an event loop), I believe this has no impact on the issue.
Here I'll show only parts of the code, full version is available here.
So, the program has two entities: Server and Client. If you start an application in a server mode, then a Server is created, starts to listen for new connections, and a new Client is automatically created and tries to connect to the Server. In client mode only a Client is created and connects to the Server.
The application also creates a new basic GUI window and starts an event loop, where everything happens.
The Client sends packets to the Server. It handles them by just logging the fact of accepting. There are two types of packets the Client can send: periodical packet (with an incremental ID) and an event packet (application reacts to pressing SPACE or M buttons).
Client sends packets:
public void update(Time dt) throws IOException {
if (!isConnected) return;
if (tickClock.getElapsedTime().compareTo(Time.getSeconds(1.f / 20.f)) > 0) {
Packet intervalUpdatePacket = new Packet();
intervalUpdatePacket.append(PacketType.INTERVAL_UPDATE);
intervalUpdatePacket.append(intervalCounter++);
PacketReaderWriter.send(socketChannel, intervalUpdatePacket);
tickClock.restart();
}
}
public void handleEvent(Event event) throws IOException {
if (isConnected && (event.type == Event.Type.KEY_PRESSED)) {
KeyEvent keyEvent = event.asKeyEvent();
if (keyEvent.key == Keyboard.Key.SPACE) {
LOGGER.info("press SPACE");
Packet spacePacket = new Packet();
spacePacket.append(PacketType.SPACE_BUTTON);
PacketReaderWriter.send(socketChannel, spacePacket);
}
if (keyEvent.key == Keyboard.Key.M) {
LOGGER.info("press M");
Packet mPacket = new Packet();
mPacket.append(PacketType.M_BUTTON);
PacketReaderWriter.send(socketChannel, mPacket);
}
}
}
Server accepts packets:
private void handleIncomingPackets() throws IOException {
readSelector.selectNow();
Set<SelectionKey> readKeys = readSelector.selectedKeys();
Iterator<SelectionKey> it = readKeys.iterator();
while (it.hasNext()) {
SelectionKey key = it.next();
it.remove();
SocketChannel channel = (SocketChannel) key.channel();
Packet packet = null;
try {
packet = PacketReaderWriter.receive(channel);
} catch (NothingToReadException e) {
e.printStackTrace();
}
if (packet != null) {
// Interpret packet and react to it
handleIncomingPacket(packet, channel);
}
}
}
private void handleIncomingPacket(Packet packet, SocketChannel channel) {
PacketType packetType = (PacketType) packet.get();
switch (packetType) {
case INTERVAL_UPDATE:
int intervalId = (int) packet.get();
break;
case SPACE_BUTTON:
LOGGER.info("handling SPACE button");
break;
case M_BUTTON:
LOGGER.info("handling M button");
break;
}
}
Here is a PacketReaderWriter object:
package server;
import java.io.*;
import java.nio.ByteBuffer;
import java.nio.channels.SocketChannel;
public class PacketReaderWriter {
private static final int PACKET_SIZE_LENGTH = 4;
private static final ByteBuffer packetSizeReadBuffer = ByteBuffer.allocate(PACKET_SIZE_LENGTH);
private static ByteBuffer clientReadBuffer;
private static byte[] encode(Packet packet) throws IOException {
try (
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos)
) {
oos.writeObject(packet);
return baos.toByteArray();
}
}
private static Packet decode(byte[] encodedPacket) throws IOException, ClassNotFoundException {
try (ObjectInputStream oi = new ObjectInputStream(new ByteArrayInputStream(encodedPacket))) {
return (Packet) oi.readObject();
}
}
public static void send(SocketChannel channel, Packet packet) throws IOException {
byte[] encodedPacket = encode(packet);
ByteBuffer packetSizeBuffer = ByteBuffer.allocate(PACKET_SIZE_LENGTH).putInt(encodedPacket.length);
packetSizeBuffer.flip();
// Send packet size
channel.write(packetSizeBuffer);
// Send packet content
ByteBuffer packetBuffer = ByteBuffer.wrap(encodedPacket);
channel.write(packetBuffer);
}
public static Packet receive(SocketChannel channel) throws IOException, NothingToReadException {
int bytesRead;
// Read packet size
packetSizeReadBuffer.clear();
bytesRead = channel.read(packetSizeReadBuffer);
if (bytesRead == -1) {
channel.close();
throw new NothingToReadException();
}
if (bytesRead == 0) return null;
packetSizeReadBuffer.flip();
int packetSize = packetSizeReadBuffer.getInt();
// Read packet
clientReadBuffer = ByteBuffer.allocate(packetSize);
bytesRead = channel.read(clientReadBuffer);
if (bytesRead == -1) {
channel.close();
throw new NothingToReadException();
}
if (bytesRead == 0) return null;
clientReadBuffer.flip();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
baos.write(clientReadBuffer.array(), 0, bytesRead);
clientReadBuffer.clear();
try {
return decode(baos.toByteArray());
} catch (ClassNotFoundException e) {
e.printStackTrace();
return null;
}
}
}
And here is the problem: I have quite big delays between pressing a button (and sending a corresponding packet from the Client) and accepting this packet on the Server. If I start a new instance of the application in a client mode (just add a new Client in short), the delays become even bigger.
I don’t see any reason why these periodical packets create so much network load that other packets just cannot get through, but maybe I'm just missing something. Here I have to say that I’m not a Java expert, so don’t blame me too much for not seeing something obvious :)
Does anyone have any ideas?
Thanks!

I decided to take a look at the Github repo.
Your Server.run() looks like this.
public void run() {
while (isRunning) {
try {
handleIncomingConnections();
handleIncomingPackets();
} catch (IOException e) {
e.printStackTrace();
}
try {
// Sleep to prevent server from consuming 100% CPU
sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
The sleep(100) will result in approximately 10 calls to handleIncomingPackets() per second. handleIncomingPackets() in turn will select a Client channel and call handleIncomingPacket() on a single received Packet. In total the server will be able to handle 10 Packets/second per Client if I understand it correctly.
The Client on the other hand tries to send 20 packets per second of the type PacketType.INTERVAL_UPDATE. Either the Client must send fewer packets per second or the Server needs to be able to handle more packets per second.
The current sleep(100) means that there will always be a latency of up to around 100ms before the server can respond to a single packet, even in a non-overloaded situation. This might be fine though if you make sure you really read all packets available on the channel instead of just a single one each time.
In summary: the smallest change you'd have to do to improve response times is to decrease the sleep() time. 10 ms would be fine. But I'd also suggest trying to check if there's more than one packet available in each iteration.
Update:
In the c++ file you linked my hunch is that it's reading more than one packet per iteration.
<snip>
while (peer->socket.receive(packet) == sf::Socket::Done)
{
// Interpret packet and react to it
handleIncomingPacket(packet, *peer, detectedTimeout);
</snip>
The while loop will read all available packets. Compared to your Java version where you read a single packet per client per server iteration.
if (packet != null) {
// Interpret packet and react to it
handleIncomingPacket(packet, channel);
}
You need to make sure that you read all available packets the Java version also.
If you just want to convince yourself that the client code sends more packets than the server code can handle it's quickly done by setting the sleep() to 10 ms temporarily.

Client to close the connection

I got a socket listener which keep listening for data. The problem now is that the client which send data will finally close the connection by itself. Based on my codes below I am wondering do I still need to perform this part of the codes where it does writeBuffer.close();?
Should I remove the final part and just put the socket closing the catch?
public void run()
{
BufferedWriter writeBuffer = null;
BufferedReader readBuffer = null;
String message="";
try {
writeBuffer = new BufferedWriter(new OutputStreamWriter(receivedSocketConn1.getOutputStream()));
readBuffer = new BufferedReader(new InputStreamReader(receivedSocketConn1.getInputStream()));
int m = 0, count=0;
int nextChar=0;
while ((nextChar=readBuffer.read()) != -1)
{
message += (char) nextChar;
if (nextChar == '#')
{
System.out.println("\n\nSending PA : "+message);
writeBuffer.write("$PA\r\n");
writeBuffer.flush();
message="";
}
}
}
catch (Exception ex)
{
System.out.println("MyError:Exception has been caught in in the main first try");
ex.printStackTrace(System.out);
}
/*finally
{
try
{
if ( writeBuffer != null )
{
writeBuffer.close();
}
else
{
System.out.println("MyError:writeBuffer is null in finally close");
}
}
catch(IOException ex)
{
ex.printStackTrace(System.out);
}
}*/
}

It's always a good idea to explicitly close the connections you're using. Think about it, it might be possible that the client never closes the connection (of course, then you'd have to implement some kind of timeout mechanism that closes the connection on the server side after a certain amount of time, but that's a different matter).
My point is - it never hurts to be careful, and manage your resources in a conservative fashion.

How do I handle multiple streams in Java?

I'm trying to run a process and do stuff with its input, output and error streams. The obvious way to do this is to use something like select(), but the only thing I can find in Java that does that is Selector.select(), which takes a Channel. It doesn't appear to be possible to get a Channel from an InputStream or OutputStream (FileStream has a getChannel() method but that doesn't help here)
So, instead I wrote some code to poll all the streams:
while( !out_eof || !err_eof )
{
while( out_str.available() )
{
if( (bytes = out_str.read(buf)) != -1 )
{
// Do something with output stream
}
else
out_eof = true;
}
while( err_str.available() )
{
if( (bytes = err_str.read(buf)) != -1 )
{
// Do something with error stream
}
else
err_eof = true;
}
sleep(100);
}
which works, except that it never terminates. When one of the streams reaches end of file, available() returns zero so read() isn't called and we never get the -1 return that would indicate EOF.
One solution would be a non-blocking way to detect EOF. I can't see one in the docs anywhere. Alternatively is there a better way of doing what I want to do?
I see this question here:
link text
and although it doesn't exactly do what I want, I can probably use that idea, of spawning separate threads for each stream, for the particular problem I have now. But surely that isn't the only way to do it? Surely there must be a way to read from multiple streams without using a thread for each?

As you said, the solution outlined in this Answer is the traditional way of reading both stdout and stderr from a Process. A thread-per-stream is the way to go, even though it is slightly annoying.

You will indeed have to go the route of spawning a Thread for each stream you want to monitor. If your use case allows for combining both stdout and stderr of the process in question you need only one thread, otherwise two are needed.
It took me quite some time to get it right in one of our projects where I have to launch an external process, take its output and do something with it while at the same time looking for errors and process termination and also being able to terminate it when the java app's user cancels the operation.
I created a rather simple class to encapsulate the watching part whose run() method looks something like this:
public void run() {
BufferedReader tStreamReader = null;
try {
while (externalCommand == null && !shouldHalt) {
logger.warning("ExtProcMonitor("
+ (watchStdErr ? "err" : "out")
+ ") Sleeping until external command is found");
Thread.sleep(500);
}
if (externalCommand == null) {
return;
}
tStreamReader =
new BufferedReader(new InputStreamReader(watchStdErr ? externalCommand.getErrorStream()
: externalCommand.getInputStream()));
String tLine;
while ((tLine = tStreamReader.readLine()) != null) {
logger.severe(tLine);
if (filter != null) {
if (filter.matches(tLine)) {
informFilterListeners(tLine);
return;
}
}
}
} catch (IOException e) {
logger.logExceptionMessage(e, "IOException stderr");
} catch (InterruptedException e) {
logger.logExceptionMessage(e, "InterruptedException waiting for external process");
} finally {
if (tStreamReader != null) {
try {
tStreamReader.close();
} catch (IOException e) {
// ignore
}
}
}
}
On the calling side it looks like this:
Thread tExtMonitorThread = new Thread(new Runnable() {
public void run() {
try {
while (externalCommand == null) {
getLogger().warning("Monitor: Sleeping until external command is found");
Thread.sleep(500);
if (isStopRequested()) {
getLogger()
.warning("Terminating external process on user request");
if (externalCommand != null) {
externalCommand.destroy();
}
return;
}
}
int tReturnCode = externalCommand.waitFor();
getLogger().warning("External command exited with code " + tReturnCode);
} catch (InterruptedException e) {
getLogger().logExceptionMessage(e, "Interrupted while waiting for external command to exit");
}
}
}, "ExtCommandWaiter");
ExternalProcessOutputHandlerThread tExtErrThread =
new ExternalProcessOutputHandlerThread("ExtCommandStdErr", getLogger(), true);
ExternalProcessOutputHandlerThread tExtOutThread =
new ExternalProcessOutputHandlerThread("ExtCommandStdOut", getLogger(), true);
tExtMonitorThread.start();
tExtOutThread.start();
tExtErrThread.start();
tExtErrThread.setFilter(new FilterFunctor() {
public boolean matches(Object o) {
String tLine = (String)o;
return tLine.indexOf("Error") > -1;
}
});
FilterListener tListener = new FilterListener() {
private boolean abortFlag = false;
public boolean shouldAbort() {
return abortFlag;
}
public void matched(String aLine) {
abortFlag = abortFlag || (aLine.indexOf("Error") > -1);
}
};
tExtErrThread.addFilterListener(tListener);
externalCommand = new ProcessBuilder(aCommand).start();
tExtErrThread.setProcess(externalCommand);
try {
tExtMonitorThread.join();
tExtErrThread.join();
tExtOutThread.join();
} catch (InterruptedException e) {
// when this happens try to bring the external process down
getLogger().severe("Aborted because auf InterruptedException.");
getLogger().severe("Killing external command...");
externalCommand.destroy();
getLogger().severe("External command killed.");
externalCommand = null;
return -42;
}
int tRetVal = tListener.shouldAbort() ? -44 : externalCommand.exitValue();
externalCommand = null;
try {
getLogger().warning("command exit code: " + tRetVal);
} catch (IllegalThreadStateException ex) {
getLogger().warning("command exit code: unknown");
}
return tRetVal;
Unfortunately I don't have to for a self-contained runnable example, but maybe this helps.
If I had to do it again I would have another look at using the Thread.interrupt() method instead of a self-made stop flag (mind to declare it volatile!), but I leave that for another time. :)