We Keep Coding

Best way to handle many clients (with threads?)

So my question goes here. Now if my Server has over 20 clients, it also has 20 threads and my desktop with an ryzen CPU goes to 100% at usage at 30 Threads. Now I'd like to handle a mass-amount of clients by one server, but the CPU is just getting over-used. My wise is very simple how I do it, but there must be a better way; because I saw many good java servers so far yet. I don't know what I do wrong though. In the following I share my code, how I do it in principle.
while(this.isRunning()) {
ServerSocket server = new ServerSocket(8081);
Socket s = server.accept();
new Thread(new WorkerRunnable(s)).start();
//now here if e.g. over 25 users connect there are 25 threads. CPU is at 100%. Is there a better way to handle this?
The worker runnable is identifing the clients. After that they will get into a chat-room. Its like a group chat for e.g.
Edit: Relevant parts of my very unfinished code which is still very WIP
private boolean state;
private ServerSocket socket;
#Override
public void run() {
while(this.isRunning()==true) {
try {
if(this.socket==null) this.socket = new ServerSocket(this.getPort());
Socket connection = this.socket.accept();
IntroductionSession session = new IntroductionSession(this, connection);
new Thread(session).start();
//register timeout task for 3 secs and handle it async
System.out.println(ManagementFactory.getThreadMXBean().getThreadCount());
//this.handleIncomingConnection(connection);
} catch(Exception e) {
e.printStackTrace();
//System.exit(1);
}
}
}
private class IntroductionSession implements Runnable {
private boolean alive = true;
private BaseServer server;
private Socket socket;
private boolean introduced = false;
public IntroductionSession(BaseServer server, Socket socket) {
this.server = server;
this.socket = socket;
}
private void interrupt() {
System.out.println("Not mroe alive");
this.alive = false;
}
private void killConnection() {
this.killConnection("no_reason");
}
private void killConnection(String reason) {
try {
if(this.from_client!=null) this.from_client.close();
if(this.to_client!=null) this.to_client.close();
this.socket.close();
switch(reason) {
case "didnt_introduce":
System.out.println("Kicked connection, cause it didn't introduce itself");
break;
case "unknown_type":
System.out.println("Kicked unknown connection-type.");
break;
case "no_reason":
default:
//ignore
break;
}
} catch (IOException e) {
switch(reason) {
case "didnt_introduce":
System.out.println("Error at kicking connection, which didn't introduce itself");
break;
case "unknown_type":
System.out.println("Error at kicking unknown connection-type.");
break;
case "no_reason":
default:
System.out.println("Error occured at kicking connection");
break;
}
e.printStackTrace();
}
}
private ObjectInputStream from_client;
private ObjectOutputStream to_client;
#Override
public void run() {
while(this.alive==true) {
try {
if(this.to_client==null) {
this.to_client = new ObjectOutputStream(this.socket.getOutputStream());
//this.to_client.flush();
}
if(this.from_client==null) this.from_client = new ObjectInputStream(this.socket.getInputStream());
//Time runs now, if socket is inactive its getting kicked
new Timer().schedule(new java.util.TimerTask() {
#Override
public void run() {
if(IntroductionSession.this.introduced==false) {
IntroductionSession.this.killConnection("didnt_introduce");
Thread.currentThread().interrupt();
IntroductionSession.this.interrupt();
}
}
}, 5000
);
Object obj = this.from_client.readObject();
while(obj!=null) {
if(obj instanceof IntroductionPacket) {
IntroductionPacket pk = (IntroductionPacket) obj;
introduced = true;
if(isCompatible(pk)==false) {
try {
this.to_client.writeObject(new DifferentVersionKickPacket(BaseServer.version));
this.to_client.close();
this.from_client.close();
IntroductionSession.this.socket.close();
System.out.println("Kicked socket, which uses another version.");
} catch(Exception e) {
Thread.currentThread().interrupt();
//ignore
System.out.println("Error at kicking incompatible socket.");
e.printStackTrace();
}
} else {
this.server.handleIncomingConnection(this.socket, this.from_client, this.to_client);
}
Thread.currentThread().interrupt();
}
}
} catch(StreamCorruptedException e) {
//unknown client-type = kick
this.killConnection("unknown_type");
} catch (IOException|ClassNotFoundException e) {
e.printStackTrace();
this.killConnection("no_reason");
}/* catch(SocketException e) {
}*/
}
Thread.currentThread().interrupt();
}
}
Extending class, which is an actual server:
#Override
public void handleIncomingConnection(Socket connection, ObjectInputStream from_client, ObjectOutputStream to_client) {
new AuthenticationSession(connection, from_client, to_client).run();
}
private class AuthenticationSession implements Runnable {
private Socket socket;
private ObjectInputStream from_client;
private ObjectOutputStream to_client;
public AuthenticationSession(Socket socket, ObjectInputStream from_client, ObjectOutputStream to_client) {
this.socket = socket;
this.to_client = to_client;
this.from_client = from_client;
}
//TODO: Implement app id for access tokens
#Override
public void run() {
try {
while(this.socket.isConnected()==true) {
/*ObjectOutputStream to_client = new ObjectOutputStream(socket.getOutputStream()); //maybe cause problems, do it later if it does
ObjectInputStream from_client = new ObjectInputStream(socket.getInputStream());*/
Object object = from_client.readObject();
while(object!=null) {
if(object instanceof RegisterPacket) {
RegisterPacket regPacket = (RegisterPacket) object;
System.out.println("Username:" + regPacket + ", password: " + regPacket.password + ", APP-ID: " + regPacket.appId);
} else {
System.out.println("IP " + this.socket.getInetAddress().getHostAddress() + ":" + this.socket.getPort() + " tried to send an unknown packet.");
this.socket.close();
}
}
}
}/* catch(EOFException eofe) {
//unexpected disconnect
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}*/
catch(Exception e) {
e.printStackTrace();
System.exit(1);
}
/*catch(Exception e) {
//e.printStackTrace();
Thread.currentThread().interrupt();
}*/
}
}
Please dont look at its very bad formatting and stuff I did in hope to fix it, the tasks dont die whyever though.

Generally, in production grade server code, we don't work with direct creation of socket and handling of requests. It's a nightmare to work with low level sockets, close connections and prevent leaks. Rather, we rely on production grade frameworks such as Java Spring Framework or Play Framework.
My question is, why aren't you using any server-side frameworks such as the ones I listed above?
If you're wondering how these frameworks handle thousands of concurrent requests, look into design patterns such as Thread Pool. These frameworks abstract away the complexities and handle the thread pool for you.
If the clients aren't expected to receive an immediate response, you could also look into introducing messaging queue such as Kafka. The server will pick the messages one by one from the queue and process them. However, bear in mind that this is asynchronous and may not meet your requirements.
If you're not just restricted to one server, you could look into deploying your server code to Azure or AWS VMSS (Virtual machine scale set). Based on CPU load rules you configure, the system will autoscale and dynamically manage resources for you.
I would suggest reading upon system design principles related to servers to reinforce your understanding.
Don't reinvent the wheel.

Since you are doing a Chat Application you need to think of doing a Single Threaded Event Loop.
You can Keep a Map of String (Client id) and Socket (Client socket).
Map<String, Socket> clientSockets;
You Server thread will accept new Client Sockets and will just put it in the above map. Then there will be another Thread which will do the Event Loop and whenever there is data in any of the Client Socket in InputStream it should send that data to all other Client Sockets (Group Chat). This should happen infinitely with a Sleep interval.

How to interrupt reading on System.in?

If I start reading from System.in, it will block the thread until it gets data. There is no way to stop it. Here are all the ways that I've tried:
Interrupting the thread
Stopping the thread
Closing System.in
Calling System.exit(0) does indeed stop the thread, but it also kills my application so not ideal.
Entering a char into the console makes the method return, but I can't rely on user input.
Sample code that does not work:
public static void main(String[] args) throws InterruptedException {
Thread th = new Thread(() -> {
try {
System.in.read();
} catch (IOException e) {
e.printStackTrace();
}
});
th.start();
Thread.sleep(1000);
System.in.close();
Thread.sleep(1000);
th.interrupt();
Thread.sleep(1000);
th.stop();
Thread.sleep(1000);
System.out.println(th.isAlive()); // Outputs true
}
When I run this code, it will output true and run forever.
How do I read from System.in in an interruptible way?

You should design the run method so that it can determine for itself when to terminate. Calling stop() or similar methods upon the thread would be inherently unsafe.
However, there still remains the question of how to avoid blocking inside System.in.read? To do that you could poll System.in.available until it returns > 0 prior to reading.
Example code:
Thread th = new Thread(() -> {
try {
while(System.in.available() < 1) {
Thread.sleep(200);
}
System.in.read();
} catch (InterruptedException e) {
// sleep interrupted
} catch (IOException e) {
e.printStackTrace();
}
});
Of course, it is generally considered favorable to use a blocking IO method rather than polling. But polling does have its uses; in your situation, it allows this thread to exit cleanly.
A Better Approach:
A better approach that avoids polling would be to restructure the code so that any Thread you intend to kill is not allowed direct access to System.in. This is because System.in is an InputStream that should not be closed. Instead the main thread or another dedicated thread will read from System.in (blocking) then write any contents into a buffer. That buffer, in turn, would be monitored by the Thread you intend to kill.
Example code:
public static void main(String[] args) throws InterruptedException, IOException {
PipedOutputStream stagingPipe = new PipedOutputStream();
PipedInputStream releasingPipe = new PipedInputStream(stagingPipe);
Thread stagingThread = new Thread(() -> {
try {
while(true) {
stagingPipe.write(System.in.read());
}
} catch (IOException e) {
e.printStackTrace();
}
});
stagingThread.setDaemon(true);
stagingThread.start();
Thread th = new Thread(() -> {
try {
releasingPipe.read();
} catch (InterruptedIOException e) {
// read interrupted
} catch (IOException e) {
e.printStackTrace();
}
});
th.start();
Thread.sleep(1000);
Thread.sleep(1000);
th.interrupt();
Thread.sleep(1000);
Thread.sleep(1000);
System.out.println(th.isAlive()); // Outputs false
}
But Wait! (Another Java API Fail)
Unfortunately, as pointed out by user Motowski, there exists a "Won't Fix" bug in the Java API implementation of PipedInputSteam. So if you use the unmodified library version of PipedInputSteam as shown above, it will sometimes trigger a long sleep via wait(1000). To work around the bug, Developers must make their own FastPipedInputStream subclass as described here.

I've written a wrapper InputStream class that allows to be interrupted:
package de.piegames.voicepi.stt;
import java.io.IOException;
import java.io.InputStream;
public class InterruptibleInputStream extends InputStream {
protected final InputStream in;
public InterruptibleInputStream(InputStream in) {
this.in = in;
}
/**
* This will read one byte, blocking if needed. If the thread is interrupted while reading, it will stop and throw
* an {#link IOException}.
*/
#Override
public int read() throws IOException {
while (!Thread.interrupted())
if (in.available() > 0)
return in.read();
else
Thread.yield();
throw new IOException("Thread interrupted while reading");
}
/**
* This will read multiple bytes into a buffer. While reading the first byte it will block and wait in an
* interruptable way until one is available. For the remaining bytes, it will stop reading when none are available
* anymore. If the thread is interrupted, it will return -1.
*/
#Override
public int read(byte b[], int off, int len) throws IOException {
if (b == null) {
throw new NullPointerException();
} else if (off < 0 || len < 0 || len > b.length - off) {
throw new IndexOutOfBoundsException();
} else if (len == 0) {
return 0;
}
int c = -1;
while (!Thread.interrupted())
if (in.available() > 0) {
c = in.read();
break;
} else
Thread.yield();
if (c == -1) {
return -1;
}
b[off] = (byte) c;
int i = 1;
try {
for (; i < len; i++) {
c = -1;
if (in.available() > 0)
c = in.read();
if (c == -1) {
break;
}
b[off + i] = (byte) c;
}
} catch (IOException ee) {
}
return i;
}
#Override
public int available() throws IOException {
return in.available();
}
#Override
public void close() throws IOException {
in.close();
}
#Override
public synchronized void mark(int readlimit) {
in.mark(readlimit);
}
#Override
public synchronized void reset() throws IOException {
in.reset();
}
#Override
public boolean markSupported() {
return in.markSupported();
}
}
Adjust the Thread.yield() to sleep as long as the maximum latency you can accept and prepare for some exceptions when interrupting, but apart from that it should work fine.

How to detect if kafka broker is not available from consumer in java?

I have a simple Kafka consumer in Java. I'm trying to catch an exception if Kafka broker is not available. I need it to interrupt the thread.
I have code like this:
KafkaConsumer<String, String> kafkaConsumer = new KafkaConsumer<>(kafkaConsumerProperties());
kafkaConsumer.subscribe(Arrays.asList(topic));
try {
ConsumerRecords<String, String> records = kafkaConsumer.poll(500);
// records handling
} catch(Exception e) {
System.out.println(e.getMessage());
}finally {
kafkaConsumer.close();
}
If Kafka server is down I don't catch any exceptions but following messages are displayed in log:
18/03/28 13:33:39 WARN clients.NetworkClient: [Consumer clientId=consumer-3, groupId=JAVA] Connection to node -1 could not be established. Broker may not be available.
18/03/28 13:33:40 WARN clients.NetworkClient: [Consumer clientId=consumer-1, groupId=JAVA] Connection to node -1 could not be established. Broker may not be available.
Is there a way to handle it in my thread?

Something like this can work :
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
System.out.println("Starting exit...");
consumer.wakeup(); 1
try {
mainThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
You do consumer.wakeup() to interrupt current consumer's operation.
The mainThread.join() is put in there to ensure that main thread actually finishes and is not shut down in middle of processing after wakeup. Please remember that shutdownHook is responsible for handling interrupts as well, not only ordinary program shutdown.

I've solved the problem somewhat clumsy, but it works for me.
public boolean testSocket(String serversList) {
String[] sockets = serversList.split(",");
int unactive = 0;
for (int i = 0; i < sockets.length; i++) {
try {
String[] socket = sockets[i].split(":");
(new Socket(socket[0], Integer.valueOf(socket[1]))).close();
} catch (IOException e) {
unactive++;
}
}
if (unactive < sockets.length) return true;
return false;
}

Thread.join() not behaving as expected

I'm pretty new to Multithreading in java but am totally stumped about why this isn't behaving as I want it to.
I have a Producer-Consumer wherein I have
private void produceConsume() {
try {
Thread producer = new Thread(new Runnable() {
public void run() {
try {
produce();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
completedProduce = true;
}
}
private void produce() throws InterruptedException {
synchronized (this) {
while (queue.size() == capacity) {
wait();
}
try(InputStream is = new FileInputStream(file)) {
queue.add("hello");
} catch (IOException e) {
LOG.error("Error doing stream stuff: " + e.getMessage(), e);
}
notify();
}
}
});
producer.start();
List<Thread> consumers = new ArrayList<>();
for (int i = 0; i < noOfThreads; i++) {
Thread consumer = new Thread(new Runnable() {
#Override
public void run() {
try {
consume();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private void consume() throws InterruptedException {
while (queue.size() > 0 || !completedProduce) {
synchronized (this) {
while (queue.size() == 0 && !completedProduce) {
wait();
}
String s = queue.poll();
System.out.println(s);
}
notify();
}
}
}
});
consumer.start();
consumers.add(consumer);
}
for (Thread t : consumers) {
t.join();
}
producer.join();
} catch (Exception e) {
LOG.error("InterruptedException e: " + e.getMessage(), e);
} finally {
LOG.info("We are done with this file!");
}
}
Now, I've noticed that all functionality changes based on where I put my producer.join() statement. For example, if I put producer.join() right after producer.start() then everything works - but the number of threads has no impact on runtime. This makes sense as I'm slowed down drastically by how long it takes to produce and so the longest task wins out.
However, if I put producer.join() where it is in the example provided (I do the join when I do the join for the consumers) then everything just stops running before the producer actually finishes. As in, the program stalls after the first thing is consumed, waiting for something, but the thread never dies.
How do I make it so that things run correctly and nothing stalls waiting for another process to finish?
Thanks in advance,

CompletionService - submit() does not block to ensure all threads are created

I am submitting Callables to an ExecutorCompletionService and it seems like the submit() method does not block code while submitting Callables. Here is the code that I have:
ExecutorService executor = Executors.newFixedThreadPool(30);
BlockingQueue<Future<Data>> completionQueue = new LinkedBlockingQueue();
ExecutorCompletionService<Data> completionService = new ExecutorCompletionService<Data>(executor, completionQueue);
while(receivingPackets) {
Callable<Data> splitPacketCallable = new SplitPacket(packetString);
completionService.submit(splitPacketCallable);
try {
// Allow submit to finish
TimeUnit.MILLISECONDS.sleep(50);
} catch (InterruptedException ex) {
System.out.println("Something went wrong with sleeping");
}
try {
Future<Data> dataFuture = completionService.poll();
if (dataFuture != null) {
Data data = dataFuture.get();
fileWriter.writeLine(data.toString());
}
} catch (InterruptedException ex) {
System.out.println("Error from poll: " + ex.toString());
} catch (ExecutionException ex) {
System.out.println("Error from get: " + ex.toString());
}
}
// Finish any remaining threads
while (!completionQueue.isEmpty()) {
try {
Future<Data> dataFuture = completionService.take();
Data data = dataFuture.get();
fileWriter.writeLine(data.toString());
} catch (InterruptedException ex) {
System.out.println("Error from take: " + ex.toString());
} catch (ExecutionException ex) {
System.out.println("Error from get: " + ex.toString());
}
}
fileWriter.close();
executor.shutdown();
A few things to note:
Data is a class that stores data in a special format. SplitPacket is a class that implements Callable that takes in a String that has arrived and splits it into chunks to be saved in Data. fileWriter and its method writeLine is a Runnable Class that will asynchronously write to a single file from multiple threads.
If I play with the sleep in the for loop, I start getting inconstant results in my output file. If I sleep for 50 ms every time I submit a Callable, everything works perfectly. However, if I submit with a low value (say 0-5 ms), I start getting dropped threads in the output. To me, this implies that the submit() method of ExecutorCompletionService does not block. However, because blocking a submitted callable seems vital, I also assume I am just implementing this wrong.
In my case, I don't know how many packets will be coming in so I need to be able to continuously add Callables to the Executor. I have tried this with a for loop instead of a while loop so that I can send a given number of packets and see if they get printed on the other end, and I can only get them to go through if I have a delay after submit.
Is there a way to fix this without adding a hack-y delay?

If you look at the source of ExecutorCompletionService you will see that the Futures are being added to completionQueue after the task is marked as done.
private class QueueingFuture extends FutureTask<Void> {
QueueingFuture(RunnableFuture<V> task) {
super(task, null);
this.task = task;
}
protected void done() { completionQueue.add(task); }
private final Future<V> task;
}
You may have an empty queue but still running tasks.
The simplest thing you can do is just count the tasks.
int count = 0;
while(receivingPackets) {
...
completionService.submit(splitPacketCallable);
++count;
...
try {
Future<Data> dataFuture = completionService.poll();
if (dataFuture != null) {
--count;
...
}
...
}
// Finish any remaining threads
while (count-- > 0) {
...
}