We Keep Coding

Is ZeroMQ server listening infinitely?

I am trying to implement client server using ZeroMQ.
I am running a server in an infinite loop, bound to a socket and polling the the socket infinitely.
When a client sends a request, the server receives only for the first time. The subsequent requests are not received by the server, below is my code snippet
Server :
ZMQ.Socket socket = context.socket(ZMQ.REP);
socket.bind ("tcp://*:5555");
System.out.println("Server is in receive mode");
while (!Thread.currentThread ().isInterrupted ()) {
Poller poller = new Poller(1);
poller.register(socket, Poller.POLLIN);
poller.poll();
if (poller.pollin(0)) {
ZMsg zmqMessage = ZMsg.recvMsg(socket);
if (zmqMessage!=null) {
zmqMessage.getFirst().getData();
}
}
Client :
ZMQ.Socket socket = context.socket(ZMQ.REQ);
socket.connect ("tcp://localhost:5555");
ZMsg readyFrame = new ZMsg();
readyFrame.add(new ZFrame("READY"));
readyFrame.send(socket);
I tried poll out in client side like below but it did not work.
Poller poller = new Poller(1);
poller.register(socket, Poller.POLLOUT);
poller.pollout(0);

ZeroMQ is a wonderfull piece of art from Pieter HINTJENS' and Martin SUSTRIK's team. Yes, there are some low-level, technology-related issues, that still require some design efforts, nevertheless the ZeroMQ is stable and very mature.
System thinking - the toys are working as distributed automata
Yes, normal work-flow of SEQ programming languages, that work serially ( or "just"-concurrent ) suddenly gets new dimension - a distributed automata dimension.
So the local workflow is dependent on externally operated parties.
This is the case for each of the ZeroMQ Formal Communication Patterns' primitive archetypes bear human-familiar names:
one REQ-ests, second REP-lies
one PUB-lishes, anyone SUB-scribes to just listen
one PUSH-es, the other PULL-s to receive
each party, bound together in PAIR may both speak and listen, whenever, as needed
etc for BROKER, DEALER, XPUB, XSUB, et al
This is the by-design reason, why your server-side REQ-archetype behaviour will not receive any next message from any other party ( yes, there might be more clients connected to the same REQ-transport-class node ), until it indeed REP-lies ( be it an empty message or not ) to the REP-side of the distributed automata.
The best next step
Well, the best next step one may ever do in going professional in this direction is IMHO to get a bit more global view, which may sound complicated for the first few things one tries to code with ZeroMQ, but if you at least jump to the page 265 of the [Code Connected, Volume 1] [available asPdf >>> http://hintjens.wdfiles.com/local--files/main%3Afiles/cc1pe.pdf ], if it were not the case of reading step-by-step thereto.
The fastest-ever learning-curve would be to have first an un-exposed view on the Fig.60 Republishing Updates and Fig.62 HA Clone Server pair for a possible High-availability approach and then go back to the roots, elements and details.
Overheads:
As a minor note, it would be fair and resource-wise to lower the processing overheads once the Poller would be created "outside" the while(){}-loop, as there is no visible reason for reinstating such element and re-register it's services for each loop again and again:
Poller poller = new Poller(1); // new INSTANCE
poller.register( socket, Poller.POLLIN ); // pay COSTS of SETUP
// JUST ONCE, HERE
while ( !Thread.currentThread ().isInterrupted () ) {// inf LOOP
poller.poll(); // POLL !BLOCKING
if ( poller.pollin( 0 ) ) { // if ITEMS{ ... proc 'em }
ZMsg zmqMessage = ZMsg.recvMsg( socket );
if ( zmqMessage != null )
zmqMessage.getFirst().getData();
}
}
Anyway: Enjoy the worlds of distributed computing!

A REP socket must send a reply before it can receive again.
If you're just wanting a 1 way communication you might be better using a PUB & SUB.

Spring websockets Broken pipe & client not receiving messages

I have a few problems with using websockets:
java.io.IOException: Broken Pipe
Client doesn't receive messages
TL;DR
Main things I want to know:
Please list all possible scenarios why the client side closes the connection (apart from refreshing or closing the tab).
Can a Broken Pipe Exception occur, apart from the server sending a message to the client over a broken connection? If yes, then how?
What are the possible scenarios why a server doesn't send a message, although the server does send heartbeats? (When this happens, I need to restart the application for it to work again. This is a terrible solution, because it already is in production.)
I have a SpringMVC project that uses websockets; SockJS client side and org.springframework.web.socket.handler.TextWebSocketHandler server side.
A JSON is generated server side and send to the client. Sometimes, I get a java.io.IOException: Broken Pipe. I googled/StackOverflowed a lot and found too many things I don't understand, but the reason is probably the connection is closed client side and the server still sends a message (for example, a heartbeat). Does this sound okay? What are other causes for this exception to arise? What are the reasons for the client side to close the connection (apart from refreshing or closing the tab)?
Also, sometimes the client side doesn't get any messages from the server, although the server should send them. I log before and after sending the message, and both log statements are printed. Does anyone has an idea why this can occur? I have no errors in the console log of Chrome. Refreshing the page doesn't work, I need to restart the spring project...
If you need more info, please leave a comment.
Client side
function connect() {
var socket = new SockJS('/ws/foo');
socket.onopen = function () {
socket.send(fooId); // ask server for Foo with id fooId.
};
socket.onmessage = function (e) {
var foo = JSON.parse(e.data);
// Do something with foo.
};
}
Server side
Service
#Service
public class FooService implements InitializingBean {
public void updateFoo(...) {
// Update some fields of Foo.
...
// Send foo to clients.
FooUpdatesHandler.sendFooToSubscribers(foo);
}
}
WebSocketHandler
public class FooUpdatesHandler extends ConcurrentTextWebSocketHandler {
// ConcurrentTextWebSocketHandler taken from https://github.com/RWTH-i5-IDSG/BikeMan (Apache License version 2.0)
private static final Logger logger = LoggerFactory.getLogger(FooUpdatesHandler.class);
private static final ConcurrentHashMap<String, ConcurrentHashMap<String, WebSocketSession>> fooSubscriptions =
new ConcurrentHashMap<>();
public static void sendFooToSubscribers(Foo foo) {
Map<String, WebSocketSession> sessionMap = fooSubscriptions.get(foo.getId());
if (sessionMap != null) {
String fooJson = null;
try {
fooJson = new ObjectMapper().writeValueAsString(foo);
} catch (JsonProcessingException ignored) {
return;
}
for (WebSocketSession subscription : sessionMap.values()) {
try {
logger.info("[fooId={} sessionId={}] Sending foo...", foo.getId(), subscription.getId());
subscription.sendMessage(new TextMessage(fooJson));
logger.info("[fooId={} sessionId={}] Foo send.", foo.getId(), subscription.getId());
} catch (IOException e) {
logger.error("Socket sendFooToSubscribers [fooId={}], exception: ", foo.getId(), e);
}
}
}
}
}

Just an educated guess: Check your networking gear. Maybe there is a misconfigured firewall terminating these connections; or even worse, broken networking gear causing the connections to terminate. If your server has multiple NICs (which is likely the case), it's also possible that there is some misconfiguration using these NICs, or in connecting to the server via different NICs.

If this problem occurs accidently than it is possible that you have some problem with any cache - please check if spring or SocksJS has own caches for socket interaction.
Is this happens on your devices (or on devices that you control)?
Additionally I can suggest you to use some network packet analyzer like wireshark. With such tool you'll see current network activity 'online'
Some external reasons that can desctroy connection without correct stopping it (and you cannot manage it without connection checkups):
device suspend/poweroff
network failure
browser closing on some error
I think that is a small part of full list of possible reasons to destroy connection.

php socket programming issue: Connection reset by peer

I have an php application, which connects to a java application using sockets. In java application we have timeout set for 10 seconds.
I have a huge amount of data which needs to be processed in the middle of the application. Due to which when i try to use the socket after processing I receive the socket error of connection reset by peer.
Now when I try to shutdown(socket_shutdown) the connection and re open the connection, am receiving an error Transport endpoint is not connected on socket_send,socket_receive. Would appreciate any guidance on this issue.
NOTE : am trying to use singleton method for socket connections.

I am using in my PHP application massively socket connections [10 different PHP Apache threads per second to my own self written C++ Server].
This is the code which I use in PHP and it works:
$sIP = gethostbyname('MyDomain');
# create the socket
$iSocket = socket_create( AF_INET, SOCK_STREAM, SOL_TCP );
if ($iSocket === FALSE )
{
writeLogFile(sprintf("socket_create() failed: %s",
socket_strerror(socket_last_error())));
exit;
}
# load some default values from the app table
$iTimeOut = 180
$iPort = 11211;
socket_set_option($iSocket, SOL_SOCKET, SO_RCVTIMEO, array("sec" => $iTimeOut, "usec" => 0));
$iResult = socket_connect($iSocket, $sIP, $iPort);
if ( $iResult === FALSE )
{
$sError = sprintf("<br>socket_connect(%d) failed with code %d: %s",
$iSocket, $iResult, socket_strerror(socket_last_error($iSocket)));
writeLogFile($sError);
die($sError);
exit;
}
# $sHttpParameter has the value which has to be sent over the socket
socket_write($iSocket, $sHttpParameter, strlen($sHttpParameter));
# for my purpose the answer is just an OK/FAILED
$sAnswer = socket_read($iSocket, 1000);
socket_close($iSocket);
Are you sure the problem is not the JAVA application?

RabbitMQ Stomp over websocket : Unable to retrieve queued messages

I am using Durable subscription of RabbitMQ Stomp (documentation here). As per the documentation, when a client reconnects (subscribes) with the same id, he should get all the queued up messages. However, I am not able to get anything back, even though the messages are queued up on the server side. Below is the code that I am using:
RabbitMQ Version : 3.6.0
Client code:
var sock;
var stomp;
var messageCount = 0;
var stompConnect = function() {
sock = new SockJS(options.url);
stomp = Stomp.over(sock);
stomp.connect({}, function(frame) {
debug('Connected: ', frame);
console.log(frame);
var id = stomp.subscribe('<url>' + options.source + "." + options.type + "." + options.id, function(d) {
console.log(messageCount);
messageCount = messageCount + 1;
}, {'auto-delete' : false, 'persistent' : true , 'id' : 'unique_id', 'ack' : 'client'});
}, function(err) {
console.log(err);
debug('error', err, err.stack);
setTimeout(stompConnect, 10);
});
};
Server Code:
public class WebSocketConfig extends AbstractWebSocketMessageBrokerConfigurer {
#Override
public void configureMessageBroker(final MessageBrokerRegistry config) {
config.enableStompBrokerRelay("<endpoint>", "<endpoint>").setRelayHost(host)
.setSystemLogin(username).setSystemPasscode(password).setClientLogin(username)
.setClientPasscode(password);
}
#Override
public void registerStompEndpoints(final StompEndpointRegistry registry) {
registry.addEndpoint("<endpoint>").setAllowedOrigins("*").withSockJS();
}
}
Steps I am executing:
Run the script at client side, it sends subscribe request.
A queue gets created on server side (with name stomp-subscription-*), all the messages are pushed in the queue and client is able to stream those.
Kill the script, this results in disconnection. Server logs show that client is disconnected and messages start getting queued up.
Run the script again with the same id. It somehow manages to connect to server, however, no message is returned from the server. Message count on that queue remains the same (also, RabbitMQ Admin console doesn't show any consumer for that queue).
After 10 seconds, the connection gets dropped and following gets printed on the client logs:
Whoops! Lost connection to < url >
Server also shows the same messages (i.e. client disconnected). As shown in the client code, it tries to establish the connection after 10 seconds and then, same cycle gets repeated again.
I have tried the following things:
Removed 'ack' : 'client' header. This results in all the messages getting drained out of queue, however, none reaches to client. I added this header after going through this SO answer.
Added d.ack(); in the function, before incrementing messageCount. This results in error at server side as it tries to ack the message after session is closed (due to disconnection).
Also, in some cases, when I reconnect with number of queued up messages is less than 100, I am able to get all the messages. However, once it crosses 100, nothing happens(not sure whether this has anything to do with the problem).
I don't know whether the problem exists at server or client end. Any inputs?

Finally, I was able to find (and fix) the issue. We are using nginx as proxy and it had proxy_buffering set to on (default value), have a look at the documentation here.
This is what it says:
When buffering is enabled, nginx receives a response from the proxied
server as soon as possible, saving it into the buffers set by the
proxy_buffer_size and proxy_buffers directives.
Due to this, the messages were getting buffered (delayed), causing disconnection. We tried bypassing nginx and it worked fine, we then disabled proxy buffering and it seems to be working fine now, even with nginx proxy.

Client-Server communication where Server initiates

I would like to have this setup:
Server hosting TCP socket server
Multiple clients connected over TCP (keeping connection open)
Then I would like to initiate a message from the Server to the client. I can't figure out how to do this, and have multiple client sessions at the same time. Techniques I've read involve the Server listening on a port, and when it receives communicate from a client, it launches a new thread to handle and process that, and then it goes back to listening on the port for the next request of another client.
So, then how would I tap into that and send a message to a client running on one of those threads?
My actual usage scenario if you are interested is below. Final goal is like a remote control for your file system to upload files to the server.
- Each client has a java background application running in the system tray that connects to the server
- Server hosts connections, and also hosts a RESTFul webservice to initiate communication
- Mobile device connects to Server over RESTFul webservices to request informatino about the client's filesystem. So it can drill down and find a file, then click and have the file uploaded to the server.
The idea here is mobile users needing to upload files from their desktop to the server while away from their office on a mobile device. (and this is for custom product, so can't use a third-party app_
PS: I've been looking at the simple Client-Server chat program here: http://way2java.com/networking/chat-program-two-way-communication/

You want to have a server listening at all times on a specified port. Once the server notices an incoming connection on that port you should create a new Thread to handle the communication between that client and the server, while the main thread keeps on listening for other incoming connections. This way you can have multiple clients connected to one server. Like so:
private void listen() throws IOException {
serverSocket = new ServerSocket(port)
while (GlobalFlags.listening) {
new ServerThread(serverSocket.accept();
if (GlobalFlags.exit) {
serverSocket.close();
break;
}
}
}
Where the GlobalFlags are variables to control the listening process and are not really necessary. You could do a while True and just keep listening for ever and ever.
In my project I have a main server controller which had listeners running in Threads. The controller controlled the GlobalFlags. I'm sure instead of using global flags there is a better way to do inter thread communication but for me this was the simplest at the time.
The ServerThread should be looping all the time switching between sending output to the client and receiving input from the client. Like so:
ServerThread(Socket socket) {
super("GameServerThread");
this.socket = socket;
try {
this.socket.setTcpNoDelay(true);
} catch (SocketException e) {
// Error handling
}
this.terminate = false;
}
#Override
public void run() {
try {
out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(
new InputStreamReader(
socket.getInputStream()));
String inputLine, outputLine;
while ((inputLine = in.readLine()) != null) {
outputLine = processInput(inputLine);
out.println(outputLine);
if (terminate) {
break;
}
}
}
out.close();
in.close();
socket.close();
} catch (Exception e) {
// Error handling, should not use Exception but handle all exceptions by themselves.
}
On the client side you have a thread running through a similar loop, receiving input from the server and then sending output to the server.
In this example processInput is the function used to process the client's input. If you want the server to initiate contact you can make the server send something to the outputstream before listening for input and make the client listen first.
I have extracted this example from one of my own projects and the this.socket.setTcpNoDelay(true) is supposed to make the process faster. Reference here: http://www.rgagnon.com/javadetails/java-0294.html
"java.net.Socket.setTcpNoDelay() is used to enable/disable TCP_NODELAY which disable/enable Nagle's algorithm.
Nagle's algorithm try to conserve bandwidth by minimizing the number of segments that are sent. When applications wish to decrease network latency and increase performance, they can disable Nagle's algorithm (that is enable TCP_NODELAY). Data will be sent earlier, at the cost of an increase in bandwidth consumption. The Nagle's algorithm is described in RFC 896.
You get the current "TCP_NODELAY" setting with java.net.Socket.getTcpNoDelay()"
So to send a message to a specific client you could put all the threads upon creation in an ArrayList so you can keep track of all the currently connected clients. You can have the processInput method halt and polling a queue/variable until another class puts the message to be send in the queue/variable. So how to gain a handle on the class depends on your implementation of processInput. You could give every thread an ID (which is what I did in my project) and maybe have the processInput method poll an ArrayList at index=ID. Then to send output to the client you would have to set the variable at index=ID.
This method seems kind of clunky to me personally but I'm not really sure how else I would do it. You would probably use Queues and have processInput write the input to its Queue and then wait for another class to read it and put its response in the Queue. But I have personally never worked with Queues in java so you should read up on that yourself.

In my knowledge
1) Server hosting TCP socket server -- Possible
2) Multiple clients connected over TCP -- Possible
3) Then I would like to initiate a message from the Server to the client -- Not Possible. The Client has to initiate a connection creation, then the server might be able to send data packets to You. Example: You need to open Facebook website on your browser, Facebook server cannot decide to send its page to your PC on its own because your PC will not have a static IP address, and also if Facebook hypothetically writes code to initiate connection to Your PC, then it is as good as Your PC is the server and Facebook website/server acts as client.