I have this 'Worker' class, which uses a resource 'Client'.
There may be any number of threads, running the 'Worker' at any given time.
The 'Client' is not thread-safe, thus I'm using 'ThreadLocal' for it.
The 'Client' connects to some server and executes a HTTP 'Request' that the worker feeds the 'Client'.
public class Worker {
// Client is NOT thread-safe !!!
private static ThreadLocal<Client> client = new ThreadLocal<Client>();
#Override
protected void onGet(Request req) {
handleRequest(req);
}
private void handleRequest(Request req) {
someRunnableExecutor(new Runnable() {
#Override
public void run() {
get_client().send_req(req);
}
});
}
private Client get_client() {
Client c = client.get();
if (c == null) {
c = new Client();
client.set(c);
}
return c;
}
At the current implementation (above), stripped down for clarity, there are as many "active" 'Clients' as there are running 'Workers'.
This is a problem because the server is being exhausted.
What I can do is only fix the 'Worker'. Have no access to the 'Client', server or the executor that runs the workers.
What I want to do is to have a Queue of 'Client'(s) and a piece of a synchronized code, in the 'Worker', that takes a 'Client' off the Queue, if the Queue is empty the 'Worker' should wait till there is one in the Queue for him to take. Then put the 'Client' back into the Queue - synchronized as well.
I really want to keep it as simple as possible, with the possible minimum changes made to the code.
No new classes, no factories, just some data structure to hold the 'Client'(s) and synchronization.
I am a bit puzzled with how to achieve that generally, as well as by the fact that the 'Client' is not thread-safe and that I have to 'ThreadLocal'(ize) it. Is this how do I put that in a Queue?
private static Queue<ThreadLocal<CLient>> queue =
new LinkedList<ThreadLocal<CLient>>();
Also, how/where do I initialize that Queue, once, with say 5 clients?
Please share your thoughts.
You don't need ThreadLocal here, as you want to have less Clients than Workers. All you need in BlockingQueue.
Notice! I supposed that Client's send_req is synchronous, if it's not - the code needs some changes in run() method
public class Worker {
private static final int CLIENTS_NUMBER = 5;
private static final BlockingQueue<Client> queue = new LinkedBlockingQueue<>(CLIENTS_NUMBER);
static {
for (int i = 0; i < CLIENTS_NUMBER; i++)
queue.put(new Client());
}
#Override
protected void onGet(Request req) {
handleRequest(req);
}
private void handleRequest(Request req) {
someRunnableExecutor(new Runnable() {
#Override
public void run() {
try {
Client client = takeClient();
client.send_req(req);
putClient(client);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
});
}
private Client takeClient() throws InterruptedException {
return queue.take();
}
private void putClient(Client client) throws InterruptedException {
queue.put(client);
}
}
Related
I am working on a streaming java application that is using several long running worker threads. The application receives data, processes it, and then sends it along toward a third party using their SDK. There is an Engine class that recieves data and submits it to Workers. The worker threads will live for as long as the application runs, which could be months if not years.
I have included sample code that represents this key part of this question.
import java.util.Map;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.TimeUnit;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class BarEngine implements Engine
{
static Logger log = LoggerFactory.getLogger(BarEngine.class);
private static final int WORKER_COUNT = 5;
private BlockingQueue<Map<String, Object>> queue;
private FooWorker[] workers = new FooWorker[WORKER_COUNT];
public BarEngine()
{
for (int i = 0; i < WORKER_COUNT; i++)
{
workers[i] = new FooWorker(i, queue);
workers[i].start();
}
}
// From Engine Interface
#Override
public void sendEvent(final Map<String, Object> data)
{
try
{
queue.put(data);
}
catch (InterruptedException e)
{
log.error("Unexpected Exception", e);
}
}
// From Engine Interface
#Override
public void shutDown()
{
// Shuts down engine
}
public static class FooWorker extends Thread
{
static Logger log = LoggerFactory.getLogger(FooWorker.class);
private volatile boolean run = true;
private int id;
private BlockingQueue<Map<String, Object>> queue;
private Client client;
public FooWorker(int id, BlockingQueue<Map<String, Object>> queue)
{
this.id = id;
this.queue = queue;
client = Client.build(id);
}
#Override
public void run()
{
setName("FooWorker-" + id);
while (run)
{
try
{
Map<String, Object> data = queue.poll(5, TimeUnit.SECONDS);
if (null != data)
{
sendEvent(data);
}
}
catch (Throwable e)
{
log.error("Unexpected Exception", e);
}
}
}
private void sendEvent(Map<String, Object> data)
{
try
{
client.submit(data);
}
catch (Throwable e)
{
log.error("Unexpected Exception", e);
}
}
// dummy client classs
public static class Client
{
public void submit(Map<String, Object> data)
{
// submits data
}
public static Client build(int id)
{
// Builds client
return new Client();
}
}
}
}
I have been doing a bit of research, and I have not found a satisfactory answer.
Java Concurrency in Practice : Does not provide much guidance on long running threads.
When should we use Java's Thread over Executor? : Heavily suggest I should ALWAYS use an Executor. Does not cover the application long-life threads per se.
Java Executor and Long-lived Threads : Discusses managing long lived threads with Executor but does not answer if one SHOULD manage long live threads with Executor
My question is: Should I keep these long running Threads bare like this? If not, what should I replace it with (Like ExecutorService or something else)?
Answering your question, if you have threads which has the same lifetime of the application, in my opinion it doesn't matter if you are using a Thread or Executer service (which is again using Threads underneath) as long as you manage the thread's life cycle properly.
From design point of view your application falls in to software category what we called a "middleware". Generally a middleware application should be efficient as well as scalable, both which are essential qualities of such server yet you have ignored both. Your application's threads run busy-wait loops per thread keeping the CPU busy at all time. Even when the incoming load is very low this keeps happening. Which is a not good quality to have for such application.
Alternatively, I'm proposing you to use a ThreadPool implementation such as ThreadPoolExecutor which already have solved what you are trying to accomplish here. ThreadPoolExecutor leverages the functionality of a BlockingQueue if all initially fired up threads are busy at the moment. also it can stop threads if the load is low and fire up again if wanted. I have coded the structure of the design I'm proposing. Take a look at the following code. I assumed that Client is not thread-safe so I'm constructing a Client per thread. If your real client implementation is thread-safe you can use one client across all threads.
import java.util.Map;
import java.util.concurrent.*;
public class BarEngine implements Engine {
private static final int WORKER_COUNT = 5;
private ExecutorService threadPool;
public BarEngine() {
this.threadPool = new ThreadPoolExecutor(1, WORKER_COUNT, 10, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(100));
}
// From Engine Interface
#Override
public void sendEvent(final Map<String, Object> data) {
threadPool.submit(new FooWorker(data));
}
// From Engine Interface
#Override
public void shutDown() {
this.threadPool.shutdown();
// Shuts down engine
}
public static class FooWorker implements Runnable {
private final Client client;
private final Map<String, Object> data;
public FooWorker(Map<String, Object> data) {
client = Client.build(Thread.currentThread().getId());
this.data = data;
}
#Override
public void run() {
try {
if (null != data) {
sendEvent(data);
}
} catch (Throwable e) {
//todo log
}
}
private void sendEvent(Map<String, Object> data) {
try {
client.submit(data);
} catch (Throwable e) {
//todo log
}
}
// dummy client classs
public static class Client {
public void submit(Map<String, Object> data) {
// submits data
}
public static Client build(long id) {
// Builds client
return new Client();
}
}
}
}
Yes, what you posted is exactly something for ExecutorService. Some advices:
There are two main interfaces Executor, ExecutorService.
Executors should be ended if you dont need them via shutdown/shutdownNow methods on ExecutorService interface. If not, you will face memleaks.
Create your ExecutorService via Executors, for example:
Executors.newFixedThreadPool(5);
If you use Executor service, you can push data directly to ExecutorService via Runnable because ExecutorService doing queuing it self and split tasks into its workers...
I have got a server and client architecture that exchange information. I want to return from the server the number of connected channels. I want to return the message of the server to the clients using promise. My code is:
public static void callBack () throws Exception{
String host = "localhost";
int port = 8080;
try {
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.handler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(), new ClientHandler(promise));
}
});
ChannelFuture f = b.connect(host, port).sync();
//f.channel().closeFuture().sync();
}
finally {
//workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
callBack();
while (true) {
Object msg = promise.get();
System.out.println("The number if the connected clients is not two");
int ret = Integer.parseInt(msg.toString());
if (ret == 2){
break;
}
}
System.out.println("The number if the connected clients is two");
}
When I run one client it is always receiving the message The number if the connected clients is not two and the returning number is always one. When I run a second client it is receiving always as a returning value two, however, the first client still is receiving one. I cannot find which is the correct way to update the promise for the case of the first client.
EDIT:
Client Server:
public class ClientHandler extends ChannelInboundHandlerAdapter {
public final Promise<Object> promise;
public ClientHandler(Promise<Object> promise) {
this.promise = promise;
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
RequestData msg = new RequestData();
msg.setIntValue(123);
msg.setStringValue("all work and no play makes jack a dull boy");
ctx.writeAndFlush(msg);
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println(msg);
promise.trySuccess(msg);
}
}
The code from the client handler storing the message received from server to the promise.
With the Netty framework, a Promise and a Future are write-once objects, this principle makes them easier to use in a multithreaded environment.
Since a Promise doesn't do what you want, we need to see if other technologies are fit for your conditions, your conditions basically boil down to:
Read from multiple threads
Write from a single thread only (as inside a Netty channel the read method can only be executed by 1 thread at the same time, unless the channel is marked shareable)
For these requirements, the best fitting match is a volatile variable, as this is thread-safe for reading, and can safely be updated by 1 thread without worrying about the write order.
To update your code for usage with a volatile variable, it requires some modifications, as we cannot easily pass the reference link to the variable inside your function, but we must pass a function that updates the backend variable.
private static volatile int connectedClients = 0;
public static void callBack () throws Exception{
//....
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(),
new ClientHandler(i -> {connectedClients = i;});
//....
}
public static void main(String[] args) throws Exception {
callBack();
while (true) {
System.out.println("The number if the connected clients is not two");
int ret = connectedClients;
if (ret == 2){
break;
}
}
System.out.println("The number if the connected clients is two");
}
public class ClientHandler extends ChannelInboundHandlerAdapter {
public final IntConsumer update;
public ClientHandler(IntConsumer update) {
this.update = update;
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
RequestData msg = new RequestData();
msg.setIntValue(123);
msg.setStringValue("all work and no play makes jack a dull boy");
ctx.writeAndFlush(msg);
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println(msg);
update.accept(Integer.parseInt(msg));
}
}
While the approach above should work, we quickly see that the while loop inside the main class uses a large share of CPU time, and this may affect other parts of your local client system, luckily, this problem is also solvable if we add other parts to the system, namely synchronization. By leaving the initial read of the connectedClients outside the synchronization block, we can still profit from the quick reads in the case of the "true" case, and in case of the "false' case, we can safe important CPU cycles that can be used in other parts of your system.
To tackle this problem, we use the following steps when reading:
Store the value of connectedClients in a separate variable
Compare this variable with the target value
If it's true, then break early out of the loop
If false, go inside a synchronized block
start a while true loop
Read out the variable again, since the value might be changed now
Check the condition, and break if condition is correct now
If not, wait for a change in the value
And the following when writing:
synchronize
Update the value
Wake up all other threads waiting for this value
This can be implemented in code as the following:
private static volatile int connectedClients = 0;
private static final Object lock = new Object();
public static void callBack () throws Exception{
//....
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(),
new ClientHandler(i -> {
synchronized (lock) {
connectedClients = i;
lock.notifyAll();
}
});
//....
}
public static void main(String[] args) throws Exception {
callBack();
int connected = connectedClients;
if (connected != 2) {
System.out.println("The number if the connected clients is not two before locking");
synchronized (lock) {
while (true) {
connected = connectedClients;
if (connected == 2)
break;
System.out.println("The number if the connected clients is not two");
lock.wait();
}
}
}
System.out.println("The number if the connected clients is two: " + connected );
}
Server side changes
However, not all of your problems are related to the client side.
SInce you posted a link to your github repository, you never send a request from the server back to the old clients when a new person has joined. Because this is not done, the client is never notified about the change, make sure to do this as well.
As we know, ThreadPoolExecutor uses some BlockingQueue as a queue of incoming tasks. What I want is to have ThreadPoolExecutor that has a second queue for the task results which are ready. I want to use this queue as a source for input/output services which send or store these results.
Why I want to create a separate queue? Because I want to decouple action of sending results from action of obtaining results. Also, I suppose any Exceptions and Delays that accompany input/output operations should not affect my ThreadPoolExecutor which is calculating the result.
I have created some naive implementation of this. I would like to get some criticism on this. May be, it can be implemented with out-of-the-box Java classes better? I use Java 7.
public class ThreadPoolWithResultQueue {
interface Callback<T> {
void complete(T t);
}
public abstract static class CallbackTask<T> implements Runnable {
private final Callback callback;
CallbackTask(Callback callback) {
this.callback = callback;
}
public abstract T execute();
final public void run() {
T t = execute();
callback.complete(t);
}
}
public static class CallBackTaskString extends CallbackTask<String> {
public CallBackTaskString(Callback callback) {
super(callback);
}
#Override
public String execute() {
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
}
return hashCode() + "-" + System.currentTimeMillis();
}
}
public static void main(String[] args) throws InterruptedException {
BlockingQueue<Runnable> workQueue = new LinkedBlockingQueue<Runnable>();
final BlockingQueue<String> resultQueue = new LinkedBlockingQueue<String>();
Callback<String> addToQueueCallback = new Callback<String>() {
#Override
public void complete(String s) {
System.out.println("Adding Result To Queue " + s);
resultQueue.add(s); //adding to outgoing queue. some other executor (or same one?) will process it
}
};
ThreadPoolExecutor executor = new ThreadPoolExecutor(3, 5, 1000l, TimeUnit.DAYS, workQueue);
for (int i = 0; i <= 5; i++) {
executor.submit(new CallBackTaskString(addToQueueCallback));
};
System.out.println("All submitted.");
executor.shutdown();
executor.awaitTermination(10l, TimeUnit.SECONDS);
System.out.println("Result queue size " + resultQueue.size());
}
}
For the sake of makinf a library component, you would have to wrap things up...
You could extend The thread pool executor which has a number of methods to intercept the submitted tasks, so you would queue thing out to a queue passed in the constructor.
That's basically ExecutorCompletionService, but you would allow the user to plug a queue instead of appearing as one.
Otherwise, this is typical proxying of the task. Fair job.
Folks. I'm newbie in network programming and come across the following issue. I need to write the server which can maintain a connection with more than one client simultaneously. What I've written is the following:
Main class:
public class Main {
public static void main(String args[]) throws Exception{
ConnectionUtils.waitForClients();
}
}
ConnectionUtils class:
public class ConnectionUtils {
private static ServerSocket server;
static{
try {
server = new ServerSocket(54321);
} catch (Exception e) {
}
}
private static Runnable acceptor = new Runnable() {
#Override
public void run() {
try {
Client c = new Client(server.accept());
new Thread(acceptor).start();
c.sendLine("Hello client \n");
} catch (Exception e) {
}
}
};
public static void waitForClients(){
Thread clientAcceptor = new Thread(acceptor);
clientAcceptor.start();
}
}
and it works, more-or-less. But what is the downside of that approach? I suspect there're too much disadvantage, but I can't catch their.
The problem is that you creating an infinite number of threads where threads are expensive resources. You should be using a ThreadPool to limit the number of threads created in your program.
Consider using Executors instead of using this low-level code, In Oracle documentation about Executors, there is an example similar to what you doing. Check it out!
Heh interesting. I wouldn't expect it to be wrong but it sure isn't how I'd write it.
I'd probably have 1 thread in an infinite (semi-infinite with stop condition) loop that accepts and spawn threads, rather than something that looks like a recursive method but isn't. However as far as I can see it's not wrong.
Having said that, if you don't use your main thread for anything, why not do something like (and keep in mind i'm not a network programmer either)
public class ConnectionUtils {
protected boolean stop = false;
public static void waitForClients() {
while (!stop) {
Client c = new Client(server.accept());
new Thread(new ClientDelegate(c)).start();
}
}
}
public static class ClientDelegate implements Runnable {
private Client client;
public ClientDelegate(Client c) { this.client = c; }
public static void run() {
c.sendLine("Hello client\n");
}
}
My program is a client connected to multiple servers. I save connection objects to all servers in a static map object:
server1 -> connection1
server2 -> connection2
serverN -> connectionN
public class CacheConnection {
private final static Map cacheConnection = new HashMap();
public static void add(String serverName, Socket sock) {
synchronized (cacheConnection) {
cacheConnection.put(serverName, sock);
}
}
public static Socket get(String serverName) {
return (Socket) cacheConnection.get(serverName);
}
..
}
I have many threads getting connections from this map to communicate with the servers. How can I ensure a connection can only be used by one thread at a time?
For example, I want to be sure thread 1 and thread 2 cannot use connection 1 at the same time.
I am not completely sure, what you want. I assume that you want to guarantee that only one thread at a time accesses one particular server.
If your connection is something like a socket, then you can use it as a lock in a synchronization statement:
private void send(Connection c, Data d) {
synchronized (c) {
// for each connection object, only one thread may be inside this block.
// all other threads wait until the thread currently in this block exits it.
c.send(d);
}
}
// somewhere else ...
Data data = determineDataToSend()
Connection connection = map.get(key);
send(connection, data)
You can put the logic also into a decorator for the connection. This is especially useful if your connection has more than one method that send or receive (e.g., because you use a higher abstraction level like RMI):
public interface PowerfulConnection {
public void doA();
public int doB(ParameterForB param);
}
public class ConnectionImpl implements PowerfulConnection {
// handles the actual connection
}
/**
* This method is a decorator for PowerfulConnection that synchronizes all method accesses.
*/
public class SynchronizedConnection implements PowerfulConnection {
private PowerfulConnection target;
public SynchronizedConnection(PowerfulConnection target) {
if (target == null) throw new NullPointerException();
this.target = target;
}
public synchronized void doA() {
target.doA();
}
public synchronized int doB(ParameterForB param) {
return target.doB(param);
}
}
If you are using the decorator approach, then the only thing you need to change is the instance creation. Instead of:
private void connect(key, connectionParams) {
map.put(key, new ConnectionImpl(connectionParams));
}
use
private void connect(key, connectionParams) {
map.put(key, new SynchronizedConnection(new ConnectionImpl(connectionParams)));
}
Or, in your initial example, the get method could just remove the connection from the map. Of course, that means the client would have to be sure (probably in a finally block, to call add again, when done)
Then have wait and notify loops for when a client comes in to ask for a connection, and it's not there.