We Keep Coding

ZeroMQ PUB/SUB topology on the same machine

Is it possible for a publisher to publish to multiple clients on the same machine using ZeroMQ? I'd like a set of clients, each of which can make standard Request/Response calls using SocketType.REQ and SocketType.REP, but which can also receive notifications using SocketType.SUB and SocketType.PUB.
I've tried to implement this topology, taken from here, although my version only has one publisher.
Here is my publisher:
public class ZMQServerSmall
{
public static void main(String[] args)
{
try (ZContext context = new ZContext())
{
ZMQ.Socket rep = context.createSocket(SocketType.REP);
rep.bind("tcp://*:5555");
ZMQ.Socket pub = context.createSocket(SocketType.PUB);
pub.bind("tcp://*:7777");
while (!Thread.currentThread().isInterrupted())
{
String req = rep.recvStr(0);
rep.send(req + " response");
pub.sendMore("Message header");
pub.send("Message body");;
}
}
}
}
Here is my proxy (I included a Listener to try to see what's going on):
public class ZMQForwarderSmall
{
public static void main(String[] args)
{
try
(
ZContext context = new ZContext();
)
{
ZMQ.Socket frontend = context.createSocket(SocketType.XSUB);
frontend.connect("tcp://*:7777");
ZMQ.Socket backend = context.createSocket(SocketType.XPUB);
backend.bind("tcp://*:6666");
IAttachedRunnable runnable = new Listener();
Socket listener = ZThread.fork(context, runnable);
ZMQ.proxy(frontend, backend, listener);
}
catch (Exception e)
{
System.err.println(e.getMessage());
}
}
private static class Listener implements IAttachedRunnable
{
#Override
public void run(Object[] args, ZContext ctx, Socket pipe)
{
while (true)
{
ZFrame frame = ZFrame.recvFrame(pipe);
if (frame == null)
break; // Interrupted
System.out.println(frame.toString());
frame.destroy();
}
}
}
}
Here is my Subscriber:
public class ZMQClientSmall
{
public static void main(String[] args) throws IOException
{
String input;
try
(
ZContext context = new ZContext();
BufferedReader stdIn = new BufferedReader(new InputStreamReader(System.in))
)
{
ZMQ.Socket reqSocket = context.createSocket(SocketType.REQ);
reqSocket.connect("tcp://localhost:5555");
ZMQ.Socket subSocket = context.createSocket(SocketType.SUB);
subSocket.connect("tcp://localhost:6666");
subSocket.subscribe("".getBytes(ZMQ.CHARSET));
while ((input = stdIn.readLine()) != null)
{
reqSocket.send(input.getBytes(ZMQ.CHARSET), 0);
String response = reqSocket.recvStr(0);
String address = subSocket.recvStr(ZMQ.DONTWAIT);
String contents = subSocket.recvStr(ZMQ.DONTWAIT);
System.out.println("Notification received: " + address + " : " + contents);
}
}
}
}
Here is the test. I open four terminals; 1 publisher, 1 proxy, and 2 clients. When I make a request in either of the two client terminals, I expect to see a notification in both, but instead I only see the notification in the terminal that made the request. I know that both clients are using the same address (localhost:6666), but I'd hoped that the proxy would solve that problem.
Can anyone see anything obviously wrong here?

Q : Is it possible for a publisher to publish to multiple clients on the same machine using ZeroMQ?
Oh sure, it is. No doubts about that.
Check the code. The responsibility of the order-of-execution is there. In distributed-systems this always so.
Once the [Client]-No1 instance gets a plausible .readLine()-ed input it will jump-in:
while ((input = stdIn.readLine()) != null)
{
reqSocket.send(input.getBytes(ZMQ.CHARSET), 0);
String response = reqSocket.recvStr(0);
String address = subSocket.recvStr(ZMQ.DONTWAIT);
String contents = subSocket.recvStr(ZMQ.DONTWAIT);
System.out.println( "Notification received: "
+ address + " : "
+ contents
);
}
Next it .send()-s over REQ and blocks (awaiting REP response)
Given the [Client]-No2 instance also gets a plausible manual .readLine()-ed input it will jump-in the same while(){...}, yet it will not proceed any farther than into again blocking wait for REP-response. That will not get .recv()-ed any time but after the -No1 got served from the REP-side, so while the -No1 might have gotten out of the blocking-.recv(), not so the -No2 ( which will still hang inside the its blocking-.recv() for any next REP-side response ( which may come but need not ), while the No1 has already proceeded to the PUB/SUB-.recv(), which it will receive ( but never the No2 ), next rushing into the next blocking-input-feed from .readLine() Et Cetera, Et Cetera, Et Cetera, ..., Ad Infinitum
So, these SEQ-of-In-Loop (REQ)-parts followed by (SUB)-parts in whatever number N > 1 of [Client]-instances, have effectively generated an EXCLUSIVE Tick-Tock-Tick-Tock clock-machine, mutually blocking an exclusive delivery of the PUB-ed in an N-interleaved order ( not speaking about the manual, .readLine()-driven, blocking step )
The ZMQServerSmall is not aware about anything wrong, as it .send()-s in-order to any .recvStr()-ed counterparty over REQ/REP and PUB-s to all counterparties ( that do not read autonomously, but only after 've been manually .readLine() unblocked and only then ( after REQ/REP episodic ( potentially infinitely blocked ) steps ) may .recv() its next ( so far not read message-part ( yet, there I do not see any code that works with explicitly handling the presence / absence of the multipart-flags on the SUB-side .recv() operations )
while (!Thread.currentThread().isInterrupted())
{
String req = rep.recvStr(0);
rep.send(req + " response");
pub.sendMore("Message header");
pub.send("Message body");;
}
The ZMQServerSmall in the meantime sends ( N - 1 )-times more messages down the PUB-broadcast lane, so the Tick-Tock-Tick-Tock MUTEX REQ/SUB-loop-blocking "pendulum" is not 2-State, but N-State on the receiving sides ( all receive the same flow of PUB-ed messages,yet interleaved by the N-steps of the REQ/REP MUTEX-stepping )

How would I receive input from console without blocking?

Note: I understand that the console is for debugging and games should use GUI. This is for testing/experience
I'm writing a game that runs at 60fps. Every update, I check to see if the user has entered a String command. If so, it gets passed through, but if not, null is paas through, and the null is ignored.
Scanner is out of the question, since hasNext(), the method used to see if there is data to read, can potentially block and causes problems.
I've tried using BufferedReader.ready(). Not only did I have problems (never returned true), but I've read that it's not recommended for a few reasons.
BufferedReader.available() always returned 0, which in the documentation, it state's that InputStream.available() will always return 0 unless overriden. Here is my attempt:
class Game {
public static void main(String[] args) {
InputReader reader = new InputReader(System.in);
int timePerLoop = 1000/30;
Game game = new Game();
while(true) {
long start = System.nanoTime();
game.update(reader.next());
long end = System.nanoTime();
long sleepTime = timePerLoop + ((start - end) / 10000000);
if(sleepTime > 0)
try {
Thread.sleep(sleepTime);
}catch(InterruptedException e) {
e.printStackTrace();
}
else
Thread.yield();
}
}
public void update(String command) {
if(commands != null) {
//handle command
}
//update game
}
}
InputReader.java
public class InputReader {
private InputStream in;
public InputReader(InputStream stream) {
in = stream;
}
public String next() {
String input = null;
try {
while(in.available > 0) {
if(input == null)
input = "";
input += (char) in.read();
}
}catch(IOException e) {
e.printStackTrace();
}
return input;
}
}
InputStream by itself has the same problem as above. I'm not completely sure what type the object stored in System.in, but using available() yields the same results.
I've tried using the reader() from System.console(), but console() returns null. I've read into the subject, and I am not confused why. This is not the way to do it.
The goal is to check the stream to see if it contains data to read, so I can read the data knowing it won't block.
I do not want to use a separate Thread to handle user input, so please don't recommend or ask why.
The input has to be from the console. No new sockets are to be created in the process. I have read a few topics about it, but none of them clearly states a solution. Is this possible?

As you have said yourself, a custom GUI or an additional thread is the correct way to do this. However in absence of that, have you tried using readLine() for example: String inputR = System.console().readLine();
Some alterations to main():
Replace: InputReader reader = new InputReader(System.in); with:
Console c = System.console();
Replace: game.update(reader.next());
with: game.update(c.readLine());
Edit: This thread could also be helpful: Java: How to get input from System.console()

JeroMQ does not work across platforms

I'm trying to get a device running a native C build of ZMQ v 3.2.0 to work with a Java application built with JeroMQ (pure Java impl) using pub/sub ZMQ sockets. However, it seems that JeroMQ is using different flag configurations preceding the message payload vs. the C implementation. IIRC, JeroMQ is intended to be compatible with v 3.2.2, so I'm not sure if this is a bug in the JeroMQ port
My Java test code is similar the psenvpub example:
public class psenvpub {
public static void main (String[] args) throws Exception {
// Prepare our context and publisher
Context context = ZMQ.context(1);
Socket publisher = context.socket(ZMQ.PUB);
publisher.bind("tcp://*:15052");
byte seq = 1;
while( !Thread.currentThread().isInterrupted() ){
byte[] message = new byte[8];
message[0] = 0;
message[1] = 0;
message[2] = 0;
message[3] = 0;
message[4] = seq++;
message[5] = 0;
message[6] = 0;
message[7] = 0;
publisher.send(message);
try{
Thread.sleep(1000);
}
catch(Exception e ){
break;
}
}
}
}
I'm using a perl script for the Native C endpoint:
use strict;
use warnings;
use Vocollect::ZMQ::Context;
use ZMQ::Constants qw(ZMQ_SUB);
my $ctx = Vocollect::ZMQ::Context->new();
my $sock = $ctx->socket(ZMQ_SUB);
$sock->connect('tcp://localhost:15052');
$sock->subscribe('');
while (1) {
my $msg = $sock->recv(10000);
print "Received msg\n" if defined($msg);
}
When the subscriber receives its first message, it crashes due to an assert failure in the libzmq source code:
Assertion failed: options.recv_identity (..\..\..\src\socket_base.cpp:990)
Which is:
void zmq::socket_base_t::extract_flags (msg_t *msg_)
{
// Test whether IDENTITY flag is valid for this socket type.
if (unlikely (msg_->flags () & msg_t::identity))
zmq_assert (options.recv_identity);
// Remove MORE flag.
rcvmore = msg_->flags () & msg_t::more ? true : false;
}
An wireshark trace of the packets sent shows that the handshaking sequence as well as the flags are different between a JeroMQ pub/sub and a native C pub/sub. I have not seen any issues when using endpoints that either both JeroMQ or both native C libzmq.

A usable solution for console output capturing - java

I am making a program that will have the ability to run the java compiler and jvm right from within it (Don't ask me why I am reinventing the wheel, if your reply does not help, save it, I am already quite frustrated spending hours on solutions that do not work!). So far I have managed it to track whenever I input something in my textField that starts with java so that it will actually wrap up the text and give it a run like so:
if(String.valueOf(object).startsWith("java")){
try{
Runtime runtime = Runtime.getRuntime();
Process process = runtime.exec(String.valueOf(object));
}
catch(Exception e){gsc.mainWindow.printf("error");}
Consider gsc.mainWindow.printf(...); my output to a JTextArea within a JFrame.
What I have managed now is to run the commands, but anything fails I shall be able to print it directly to my output. I know this has been answered a ton of times before, read about 10 ways to do this, but none of them worked or was understandable to the point that I could run it. I need the code to be simple enough as this will have to be outputting what the proccess will be writing in the default system's Console (cmd,terminal) and then stop (I thought that this can be a method call alltogether). I am quite bad with this kind of stuff, even a multithread solution could fit my needs, nothing too professional really, I just need it to work. Any information you need, ask away!
Thanks in advance! :)

I don't know you if you want to read this, but you know, in the Java world, you should always look for a solution before implementing your own. And the solution for common problems, most of the time, comes from Apache Commons or other Apache projects. Saying that everything but your solution doesn't work or is too complicated to you will only cost you time and money (and your job, eventually).
Apache Commons Exec is what you need to solve your problem faster and easier.
---- Edit ----
Here is some code of how to capture the output of the child process. There's a class just for it, the PumpStreamHandler:
DefaultExecutor exec = new DefaultExecutor();
PumpStreamHandler streamHandler = new PumpStreamHandler();
exec.setStreamHandler(streamHandler);
CommandLine commandline = CommandLine.parse(command); //where command is your command line
exec.execute(commandline);
---- Edit 2 ----
Here is the copy-paste solution you want to capture the message using an OutputStream:
public abstract class LogOutputStream extends OutputStream {
protected static final String LINE_SEPERATOR = System.getProperty("line.separator");
public static final int DEFAULT_BUFFER_LENGTH = 2048;
protected boolean hasBeenClosed = false;
protected byte[] buf;
protected int count;
private int bufLength;
public LogOutputStream() {
bufLength = DEFAULT_BUFFER_LENGTH;
buf = new byte[DEFAULT_BUFFER_LENGTH];
count = 0;
}
public void close() {
flush();
hasBeenClosed = true;
}
public void write(final int b) throws IOException {
if (hasBeenClosed) {
throw new IOException("The stream has been closed.");
}
if (b == 0) {
return;
}
if (count == bufLength) {
final int newBufLength = bufLength + DEFAULT_BUFFER_LENGTH;
final byte[] newBuf = new byte[newBufLength];
System.arraycopy(buf, 0, newBuf, 0, bufLength);
buf = newBuf;
bufLength = newBufLength;
}
buf[count] = (byte) b;
count++;
}
public void flush() {
if (count == 0) {
return;
}
if (count == LINE_SEPERATOR.length()) {
if (((char) buf[0]) == LINE_SEPERATOR.charAt(0)
&& ((count == 1) ||
((count == 2) && ((char) buf[1]) == LINE_SEPERATOR.charAt(1)))) {
reset();
return;
}
}
final byte[] theBytes = new byte[count];
System.arraycopy(buf, 0, theBytes, 0, count);
log(new String(theBytes));
reset();
}
private void reset() {
count = 0;
}
public abstract void log(String message);
}
Then just create a subclass of it, implement the public void log(String message) with the code that updates the UI, and it's done.

Java: How keep running process in thread and read it's values?

I don't know if this is the best approach so that's why I'm asking your help guys.
This is my problem, I'm developing an app where you have the 'read' button, when the user hit this button then the program start to read some values and save this values in my database, okay ?
So I think when the user hit 'read' I start a thread, that's because the user can do another thing if he wants without the app been freeze.
But I cannot access the values read by this thread.
There's another way to do that ?
EDIT:
private void jtb_readerItemStateChanged(java.awt.event.ItemEvent evt) {
// getting some values provided by the user
if (buttonReaderState()){
if (supervisory == null)
supervisory = new Supervisory(ip, broadcast, deviceID);
supervisory.start();
}
}
// Supervisory class
public void start(){
Scan scan = new Scan();
Thread t = new Thread(scan);
t.start();
threadState = true;
}
class Scan extends Thread{
public void run(){
// inside the tread I have to initiate another 'supervisory' object, is that right ?
Supervisory s = new Supervisory(ip, broadcast, deviceID);
while (threadState){
try {
s.active();
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
public void active() throws IOException, Exception {
// getting this values from my hardware like temperature and anothers things.
for (int i = 0; i < ois.size(); i++) {
ObjectIdentifier oi = ois.get(i);
//analog input
if (i == 1)
aI = Float.parseFloat(getPresentValue(localDevice, oi));
//analog output
if (i == 2)
aO = Float.parseFloat(getPresentValue(localDevice, oi));
//binary input
if (i == 3)
bI = getBinaryValue(getPresentValue(localDevice, oi));
//binary output
if (i == 4)
bO = getBinaryValue(getPresentValue(localDevice, oi));
}
}
After reading this values I would like to show this values in the interface that I'm building, but it seems that I cannot have access to these values (aI,aO,bI,bO).

Pass a reference to the interface you have. E.g. you can add a JFrame owner field to Supervisory class and pass your values there.