I have got a server and client architecture that exchange information. I want to return from the server the number of connected channels. I want to return the message of the server to the clients using promise. My code is:
public static void callBack () throws Exception{
String host = "localhost";
int port = 8080;
try {
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.handler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(), new ClientHandler(promise));
}
});
ChannelFuture f = b.connect(host, port).sync();
//f.channel().closeFuture().sync();
}
finally {
//workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
callBack();
while (true) {
Object msg = promise.get();
System.out.println("The number if the connected clients is not two");
int ret = Integer.parseInt(msg.toString());
if (ret == 2){
break;
}
}
System.out.println("The number if the connected clients is two");
}
When I run one client it is always receiving the message The number if the connected clients is not two and the returning number is always one. When I run a second client it is receiving always as a returning value two, however, the first client still is receiving one. I cannot find which is the correct way to update the promise for the case of the first client.
EDIT:
Client Server:
public class ClientHandler extends ChannelInboundHandlerAdapter {
public final Promise<Object> promise;
public ClientHandler(Promise<Object> promise) {
this.promise = promise;
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
RequestData msg = new RequestData();
msg.setIntValue(123);
msg.setStringValue("all work and no play makes jack a dull boy");
ctx.writeAndFlush(msg);
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println(msg);
promise.trySuccess(msg);
}
}
The code from the client handler storing the message received from server to the promise.
With the Netty framework, a Promise and a Future are write-once objects, this principle makes them easier to use in a multithreaded environment.
Since a Promise doesn't do what you want, we need to see if other technologies are fit for your conditions, your conditions basically boil down to:
Read from multiple threads
Write from a single thread only (as inside a Netty channel the read method can only be executed by 1 thread at the same time, unless the channel is marked shareable)
For these requirements, the best fitting match is a volatile variable, as this is thread-safe for reading, and can safely be updated by 1 thread without worrying about the write order.
To update your code for usage with a volatile variable, it requires some modifications, as we cannot easily pass the reference link to the variable inside your function, but we must pass a function that updates the backend variable.
private static volatile int connectedClients = 0;
public static void callBack () throws Exception{
//....
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(),
new ClientHandler(i -> {connectedClients = i;});
//....
}
public static void main(String[] args) throws Exception {
callBack();
while (true) {
System.out.println("The number if the connected clients is not two");
int ret = connectedClients;
if (ret == 2){
break;
}
}
System.out.println("The number if the connected clients is two");
}
public class ClientHandler extends ChannelInboundHandlerAdapter {
public final IntConsumer update;
public ClientHandler(IntConsumer update) {
this.update = update;
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
RequestData msg = new RequestData();
msg.setIntValue(123);
msg.setStringValue("all work and no play makes jack a dull boy");
ctx.writeAndFlush(msg);
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println(msg);
update.accept(Integer.parseInt(msg));
}
}
While the approach above should work, we quickly see that the while loop inside the main class uses a large share of CPU time, and this may affect other parts of your local client system, luckily, this problem is also solvable if we add other parts to the system, namely synchronization. By leaving the initial read of the connectedClients outside the synchronization block, we can still profit from the quick reads in the case of the "true" case, and in case of the "false' case, we can safe important CPU cycles that can be used in other parts of your system.
To tackle this problem, we use the following steps when reading:
Store the value of connectedClients in a separate variable
Compare this variable with the target value
If it's true, then break early out of the loop
If false, go inside a synchronized block
start a while true loop
Read out the variable again, since the value might be changed now
Check the condition, and break if condition is correct now
If not, wait for a change in the value
And the following when writing:
synchronize
Update the value
Wake up all other threads waiting for this value
This can be implemented in code as the following:
private static volatile int connectedClients = 0;
private static final Object lock = new Object();
public static void callBack () throws Exception{
//....
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(),
new ClientHandler(i -> {
synchronized (lock) {
connectedClients = i;
lock.notifyAll();
}
});
//....
}
public static void main(String[] args) throws Exception {
callBack();
int connected = connectedClients;
if (connected != 2) {
System.out.println("The number if the connected clients is not two before locking");
synchronized (lock) {
while (true) {
connected = connectedClients;
if (connected == 2)
break;
System.out.println("The number if the connected clients is not two");
lock.wait();
}
}
}
System.out.println("The number if the connected clients is two: " + connected );
}
Server side changes
However, not all of your problems are related to the client side.
SInce you posted a link to your github repository, you never send a request from the server back to the old clients when a new person has joined. Because this is not done, the client is never notified about the change, make sure to do this as well.
Related
I have the following code:
public static Client client = ClientBuilder.newClient();
public static void main(String[] args) throws InterruptedException {
for (int i = 0; i < 100; i++) {
System.out.println(Thread.activeCount());
client.target("https://en.wikipedia.org/wiki/Special:Random").request().buildGet().submit(new InvocationCallback<Response>() {
#Override
public void completed(Response response) {
response.close();
}
#Override
public void failed(Throwable throwable) {
}
});
}
Thread.sleep(10000L);
System.out.println(Thread.activeCount());
}
Each time I call .submit() the client creates a new thread for the new get request that just permanently stays open. The problem with this is that I eventually reach the max # of threads and the program malfunctions because it cannot create a new thread. How do I avoid this problem?
Output:
1
2
3
4
...
101
The ideal solution would be to make this program single threaded and hang the program until each get-request receives a response. I'm open to using an alternative API if this is not possible with javax.ws.rs.
I am using below class to send data to our messaging queue by using socket either in a synchronous way or asynchronous way as shown below.
sendAsync - It sends data asynchronously without any timeout. After sending (on LINE A) it adds to retryHolder bucket so that if acknowledgement is not received then it will retry again from the background thread which is started in a constructor.
send - It internally calls sendAsync method and then sleep for a particular timeout period and if acknowledgement is not received then it removes from retryHolder bucket so that we don't retry again.
So the only difference between those two above methods is - For async I need to retry at all cost but for sync I don't need to retry but looks like it might be getting retried since we share the same retry bucket cache and retry thread runs every 1 second.
ResponsePoller is a class which receives the acknowledgement for the data that was sent to our messaging queue and then calls removeFromretryHolder method below to remove the address so that we don't retry after receiving the acknowledgement.
public class SendToQueue {
private final ExecutorService cleanupExecutor = Executors.newFixedThreadPool(5);
private final ScheduledExecutorService executorService = Executors.newScheduledThreadPool(3);
private final Cache<Long, byte[]> retryHolder =
CacheBuilder
.newBuilder()
.maximumSize(1000000)
.concurrencyLevel(100)
.removalListener(
RemovalListeners.asynchronous(new LoggingRemovalListener(), cleanupExecutor)).build();
private static class Holder {
private static final SendToQueue INSTANCE = new SendToQueue();
}
public static SendToQueue getInstance() {
return Holder.INSTANCE;
}
private SendToQueue() {
executorService.submit(new ResponsePoller()); // another thread which receives acknowledgement and then delete entry from the `retryHolder` cache accordingly.
executorService.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
// retry again
for (Entry<Long, byte[]> entry : retryHolder.asMap().entrySet()) {
sendAsync(entry.getKey(), entry.getValue());
}
}
}, 0, 1, TimeUnit.SECONDS);
}
public boolean sendAsync(final long address, final byte[] encodedRecords, final Socket socket) {
ZMsg msg = new ZMsg();
msg.add(encodedRecords);
// send data on a socket LINE A
boolean sent = msg.send(socket);
msg.destroy();
retryHolder.put(address, encodedRecords);
return sent;
}
public boolean send(final long address, final byte[] encodedRecords, final Socket socket) {
boolean sent = sendAsync(address, encodedRecords, socket);
// if the record was sent successfully, then only sleep for timeout period
if (sent) {
try {
TimeUnit.MILLISECONDS.sleep(500);
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
// if key is not present, then acknowledgement was received successfully
sent = !retryHolder.asMap().containsKey(address);
// and key is still present in the cache, then it means acknowledgment was not received after
// waiting for timeout period, so we will remove it from cache.
if (!sent)
removeFromretryHolder(address);
return sent;
}
public void removeFromretryHolder(final long address) {
retryHolder.invalidate(address);
}
}
What is the best way by which we dont retry if anyone is calling send method but we still need to know whether acknowledgement was received or not. Only thing is I dont need to retry at all.
Do we need separate bucket for all the sync calls just for acknowledgement and we dont retry from that bucket?
The code has a number of potential issues:
An answer may be received before the call to retryHolder#put.
Possibly there is a race condition when messages are retried too.
If two messages are sent to the same address the second overwrites the first?
Send always wastes time with a sleep, use a wait+notify instead.
I would store a class with more state instead. It could contain a flag (retryIfNoAnswer yes/no) that the retry handler could check. It could provide waitForAnswer/markAnswerReceived methods using wait/notify so that send doesn't have to sleep for a fixed time. The waitForAnswer method can return true if an answer was obtained and false on timeout. Put the object in the retry handler before sending and use a timestamp so that only messages older than a certain age are retried. That fixes the first race condition.
EDIT: updated example code below, compiles with your code, not tested:
public class SendToQueue {
private final ExecutorService cleanupExecutor = Executors.newFixedThreadPool(5);
private final ScheduledExecutorService executorService = Executors.newScheduledThreadPool(3);
// Not sure why you are using a cache rather than a standard ConcurrentHashMap?
private final Cache<Long, PendingMessage> cache = CacheBuilder.newBuilder().maximumSize(1000000)
.concurrencyLevel(100)
.removalListener(RemovalListeners.asynchronous(new LoggingRemovalListener(), cleanupExecutor)).build();
private static class PendingMessage {
private final long _address;
private final byte[] _encodedRecords;
private final Socket _socket;
private final boolean _retryEnabled;
private final Object _monitor = new Object();
private long _sendTimeMillis;
private volatile boolean _acknowledged;
public PendingMessage(long address, byte[] encodedRecords, Socket socket, boolean retryEnabled) {
_address = address;
_sendTimeMillis = System.currentTimeMillis();
_encodedRecords = encodedRecords;
_socket = socket;
_retryEnabled = retryEnabled;
}
public synchronized boolean hasExpired() {
return System.currentTimeMillis() - _sendTimeMillis > 500L;
}
public synchronized void markResent() {
_sendTimeMillis = System.currentTimeMillis();
}
public boolean shouldRetry() {
return _retryEnabled && !_acknowledged;
}
public boolean waitForAck() {
try {
synchronized(_monitor) {
_monitor.wait(500L);
}
return _acknowledged;
}
catch (InterruptedException e) {
return false;
}
}
public void ackReceived() {
_acknowledged = true;
synchronized(_monitor) {
_monitor.notifyAll();
}
}
public long getAddress() {
return _address;
}
public byte[] getEncodedRecords() {
return _encodedRecords;
}
public Socket getSocket() {
return _socket;
}
}
private static class Holder {
private static final SendToQueue INSTANCE = new SendToQueue();
}
public static SendToQueue getInstance() {
return Holder.INSTANCE;
}
private void handleRetries() {
List<PendingMessage> messages = new ArrayList<>(cache.asMap().values());
for (PendingMessage m : messages) {
if (m.hasExpired()) {
if (m.shouldRetry()) {
m.markResent();
doSendAsync(m, m.getSocket());
}
else {
// Or leave the message and let send remove it
cache.invalidate(m.getAddress());
}
}
}
}
private SendToQueue() {
executorService.submit(new ResponsePoller()); // another thread which receives acknowledgement and then delete entry from the cache accordingly.
executorService.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
handleRetries();
}
}, 0, 1, TimeUnit.SECONDS);
}
public boolean sendAsync(final long address, final byte[] encodedRecords, final Socket socket) {
PendingMessage m = new PendingMessage(address, encodedRecords, socket, true);
cache.put(address, m);
return doSendAsync(m, socket);
}
private boolean doSendAsync(final PendingMessage pendingMessage, final Socket socket) {
ZMsg msg = new ZMsg();
msg.add(pendingMessage.getEncodedRecords());
try {
// send data on a socket LINE A
return msg.send(socket);
}
finally {
msg.destroy();
}
}
public boolean send(final long address, final byte[] encodedRecords, final Socket socket) {
PendingMessage m = new PendingMessage(address, encodedRecords, socket, false);
cache.put(address, m);
try {
if (doSendAsync(m, socket)) {
return m.waitForAck();
}
return false;
}
finally {
// Alternatively (checks that address points to m):
// cache.asMap().remove(address, m);
cache.invalidate(address);
}
}
public void handleAckReceived(final long address) {
PendingMessage m = cache.getIfPresent(address);
if (m != null) {
m.ackReceived();
cache.invalidate(address);
}
}
}
And called from ResponsePoller:
SendToQueue.getInstance().handleAckReceived(addressFrom);
Design-wise: I feel like you are trying to write a thread-safe and somewhat efficient NIO message sender/receiver but (both) code I see here aren't OK and won't be without significant changes. The best thing to do is either:
make full use of the 0MQ framework. I see things and expectations here that are actually available out-of-the-box in ZMQ and java.util.concurrent API.
or have a look at Netty (https://netty.io/index.html) preferably if it applies to your project. "Netty is an asynchronous event-driven network application framework
for rapid development of maintainable high performance protocol servers & clients." This will save you time if your project gets complex, otherwise it might be overkill to start with (but then expect issues ...).
However if you think you are almost at it with your code or #john's code then I will just give advices to complete:
don't use wait() and notify(). Don't sleep() either.
use a single thread for your "flow tracker" (i.e. ~the pending message Cache).
You don't actually need 3 threads to process pending messages except if this processing itself is slow (or does heavy stuff) which is not the case here as you basically make an async call (as far as it is really async.. is it?).
The same for the reverse path: use an executor service (multiple threads) for your received packets processing only if the actual processing is slow/blocking or heavy.
I'm not an expert in 0MQ at all but as far as socket.send(...) is thread-safe and non-blocking (which I'm not sure personally - tell me) the above advices shall be correct and make things simpler.
That said, to strictly answer your question:
Do we need separate bucket for all the sync calls just for acknowledgement and we dont retry from that bucket?
I'd say no, hence what do you think of the following? Based on your code and independently of my own feelings this seems acceptable:
public class SendToQueue {
// ...
private final Map<Long, Boolean> transactions = new ConcurrentHashMap<>();
// ...
private void startTransaction(long address) {
this.transactions.put(address, Boolean.FALSE);
}
public void updateTransaction(long address) {
Boolean state = this.transactions.get(address);
if (state != null) {
this.transactions.put(address, Boolean.TRUE);
}
}
private void clearTransaction(long address) {
this.transactions.remove(address);
}
public boolean send(final long address, final byte[] encodedRecords, final Socket socket) {
boolean success = false;
// If address is enough randomized or atomically counted (then ok for parallel send())
startTransaction(address);
try {
boolean sent = sendAsync(address, encodedRecords, socket);
// if the record was sent successfully, then only sleep for timeout period
if (sent) {
// wait for acknowledgement
success = waitDoneUntil(new DoneCondition() {
#Override
public boolean isDone() {
return SendToQueue.this.transactions.get(address); // no NPE
}
}, 500, TimeUnit.MILLISECONDS);
if (success) {
// Message acknowledged!
}
}
} finally {
clearTransaction(address);
}
return success;
}
public static interface DoneCondition {
public boolean isDone();
}
/**
* WaitDoneUntil(Future f, int duration, TimeUnit unit). Note: includes a
* sleep(50).
*
* #param f Will block for this future done until maxWaitMillis
* #param waitTime Duration expressed in (time) unit.
* #param unit Time unit.
* #return DoneCondition finally met or not
*/
public static boolean waitDoneUntil(DoneCondition f, int waitTime, TimeUnit unit) {
long curMillis = 0;
long maxWaitMillis = unit.toMillis(waitTime);
while (!f.isDone() && curMillis < maxWaitMillis) {
try {
Thread.sleep(50); // define your step here accordingly or set as parameter
} catch (InterruptedException ex1) {
//logger.debug("waitDoneUntil() interrupted.");
break;
}
curMillis += 50L;
}
return f.isDone();
}
//...
}
public class ResponsePoller {
//...
public void onReceive(long address) { // sample prototype
// ...
SendToQueue.getInstance().updateTransaction(address);
// The interested sender will know that its transaction is complete.
// While subsequent (late) calls will have no effect.
}
}
I have this 'Worker' class, which uses a resource 'Client'.
There may be any number of threads, running the 'Worker' at any given time.
The 'Client' is not thread-safe, thus I'm using 'ThreadLocal' for it.
The 'Client' connects to some server and executes a HTTP 'Request' that the worker feeds the 'Client'.
public class Worker {
// Client is NOT thread-safe !!!
private static ThreadLocal<Client> client = new ThreadLocal<Client>();
#Override
protected void onGet(Request req) {
handleRequest(req);
}
private void handleRequest(Request req) {
someRunnableExecutor(new Runnable() {
#Override
public void run() {
get_client().send_req(req);
}
});
}
private Client get_client() {
Client c = client.get();
if (c == null) {
c = new Client();
client.set(c);
}
return c;
}
At the current implementation (above), stripped down for clarity, there are as many "active" 'Clients' as there are running 'Workers'.
This is a problem because the server is being exhausted.
What I can do is only fix the 'Worker'. Have no access to the 'Client', server or the executor that runs the workers.
What I want to do is to have a Queue of 'Client'(s) and a piece of a synchronized code, in the 'Worker', that takes a 'Client' off the Queue, if the Queue is empty the 'Worker' should wait till there is one in the Queue for him to take. Then put the 'Client' back into the Queue - synchronized as well.
I really want to keep it as simple as possible, with the possible minimum changes made to the code.
No new classes, no factories, just some data structure to hold the 'Client'(s) and synchronization.
I am a bit puzzled with how to achieve that generally, as well as by the fact that the 'Client' is not thread-safe and that I have to 'ThreadLocal'(ize) it. Is this how do I put that in a Queue?
private static Queue<ThreadLocal<CLient>> queue =
new LinkedList<ThreadLocal<CLient>>();
Also, how/where do I initialize that Queue, once, with say 5 clients?
Please share your thoughts.
You don't need ThreadLocal here, as you want to have less Clients than Workers. All you need in BlockingQueue.
Notice! I supposed that Client's send_req is synchronous, if it's not - the code needs some changes in run() method
public class Worker {
private static final int CLIENTS_NUMBER = 5;
private static final BlockingQueue<Client> queue = new LinkedBlockingQueue<>(CLIENTS_NUMBER);
static {
for (int i = 0; i < CLIENTS_NUMBER; i++)
queue.put(new Client());
}
#Override
protected void onGet(Request req) {
handleRequest(req);
}
private void handleRequest(Request req) {
someRunnableExecutor(new Runnable() {
#Override
public void run() {
try {
Client client = takeClient();
client.send_req(req);
putClient(client);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
});
}
private Client takeClient() throws InterruptedException {
return queue.take();
}
private void putClient(Client client) throws InterruptedException {
queue.put(client);
}
}
How do I make it retry the send attempt if user data is null. Max 2 retries, 1 retry after 10 seconds?
public class UserHandler {
private List users = new ArrayList();
public void addUser(username) {} //adds user
public Userdata findUser(username) {} //finds user
public void sendTo(String username, String message) {
Userdata user = findUser(username);
if(user != null) {
Out out = new Out(user.getClientSocket());
out.println(message);
}
}
}
Do I really have to manually insert a thread and sleep it inside sendTo()?
EDIT: the server uses java 1.4.2
You're got more of an architectural problem to solve first. In a single-threaded program the sequence is normally:
Do stuff;
Call sendTo();
Do more stuff.
You have to work out if what you want is:
Do stuff;
Call sendTo();
If (2) fails, wait 10 seconds and sendTo() again;
If (3) fails, throw an error;
Do more stuff.
The point being that this is still synchronous. If so you'll need a thread. You should use the Java 5 Executors.
public void sendTo(final String username, final String message) {
if (!internalSendTo(username, message)) {
// attempt resend
ExecutorService exec = Executors.newSingleThreadExecutor();
final AtomicBoolean result = new AtomicBoolean(false);
exec.submit(new Runnable() {
boolean b = internalSendto(username, message);
result.set(b);
});
try {
exec.awaitTermination(10, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// still didn't work
} finally {
exec.shutdownNow();
}
}
}
private boolean internalSendTo(String username, String message) {
Userdata user = findUser(username);
boolean success = false;
if (user != null) {
Out out = new Out(user.getClientSocket());
// do the communication here
success = true;
}
return success;
}
Now that's just a rough sketch of how it might work. It should give you some appreciation for the issues however.
Do you want this or do you want:
Do stuff;
Call sendTo();
If (2) fails, queue the send and keep going;
Do more stuff.
Basically this is the asynchronous approach. If you go this way you then have to answer questions like:
What happens if after 10+ seconds (or some arbitrary interval) it still hasn't worked?
What processes attempt the sendTo() calls?
What if they block/die?
Do I need multiple senders?
etc
Basically it gets much more complicated.
I would recommend to use AOP and Java annotations. Try a read-made mechanism from jcabi-aspects:
#RetryOnFailure(attempts = 3)
public void sendTo(String username, String message) {
// try to do it
}
My program is a client connected to multiple servers. I save connection objects to all servers in a static map object:
server1 -> connection1
server2 -> connection2
serverN -> connectionN
public class CacheConnection {
private final static Map cacheConnection = new HashMap();
public static void add(String serverName, Socket sock) {
synchronized (cacheConnection) {
cacheConnection.put(serverName, sock);
}
}
public static Socket get(String serverName) {
return (Socket) cacheConnection.get(serverName);
}
..
}
I have many threads getting connections from this map to communicate with the servers. How can I ensure a connection can only be used by one thread at a time?
For example, I want to be sure thread 1 and thread 2 cannot use connection 1 at the same time.
I am not completely sure, what you want. I assume that you want to guarantee that only one thread at a time accesses one particular server.
If your connection is something like a socket, then you can use it as a lock in a synchronization statement:
private void send(Connection c, Data d) {
synchronized (c) {
// for each connection object, only one thread may be inside this block.
// all other threads wait until the thread currently in this block exits it.
c.send(d);
}
}
// somewhere else ...
Data data = determineDataToSend()
Connection connection = map.get(key);
send(connection, data)
You can put the logic also into a decorator for the connection. This is especially useful if your connection has more than one method that send or receive (e.g., because you use a higher abstraction level like RMI):
public interface PowerfulConnection {
public void doA();
public int doB(ParameterForB param);
}
public class ConnectionImpl implements PowerfulConnection {
// handles the actual connection
}
/**
* This method is a decorator for PowerfulConnection that synchronizes all method accesses.
*/
public class SynchronizedConnection implements PowerfulConnection {
private PowerfulConnection target;
public SynchronizedConnection(PowerfulConnection target) {
if (target == null) throw new NullPointerException();
this.target = target;
}
public synchronized void doA() {
target.doA();
}
public synchronized int doB(ParameterForB param) {
return target.doB(param);
}
}
If you are using the decorator approach, then the only thing you need to change is the instance creation. Instead of:
private void connect(key, connectionParams) {
map.put(key, new ConnectionImpl(connectionParams));
}
use
private void connect(key, connectionParams) {
map.put(key, new SynchronizedConnection(new ConnectionImpl(connectionParams)));
}
Or, in your initial example, the get method could just remove the connection from the map. Of course, that means the client would have to be sure (probably in a finally block, to call add again, when done)
Then have wait and notify loops for when a client comes in to ask for a connection, and it's not there.