is probably a stupid question but I can not find answer on the internet. I need to have two separate networks geographically and I would like to communicate with the nodes of a network with nodes of the other network through a broker MQTT. This protocol allows you to do this or are constrained to be on the same network?
thanks
First line on Wikipedia:
MQTT[1] (formerly Message Queue Telemetry Transport) is a publish-subscribe based "light weight" messaging protocol for use on top of the TCP/IP protocol. It is designed for connections with remote locations where a "small code footprint" is required and/or network bandwidth is limited.
With MQTT you can have remote brokers and clients connecting. It has one-to-may message distribution over TCP/IP with a small transport overhead, one advantage of MQTT is that it supports TLS and ACL.
A good introduction post is here, http://jpmens.net/2013/02/25/lots-of-messages-mqtt-pub-sub-and-the-mosquitto-broker/
Related
Recently I started using ActiveMQ to act as a Message Middleware in my new project, this is the first time I try use ActiveMQ, the projects I had participated before used our previous company's inner message framework like Swallow. So before I begin implementing the system, I need to clear some design points.
Cases in our system will use ActiveMQ include sending mail, sending tasks to queue and doing tasks from queue, asynchronous request/response, so what kind of protocol and network connection is the right choice for our cases? I list some protocols and network connection options here:
ActiveMQ protocols:
MQTT
WS
Openwire
Stomp
Stomp
ActiveMQ Network connections:
VM
TCP
UDP
HTTP
Failover
Discovery
I will also consider the aspects of HA and cluster for my system, so can anybody gives me some ideas to decide how to choose the protocol and network connection?
Thanks a lot.
Openwire has historically been the default protocol the NIO transport can give performance improvements over TCP so if you use ActiveMQ as your only broker use one of these. However using AMQP means in the future you could possibly use RabbitMQ, another popular Message Broker. There are others, STOMP or MQTT are lightweight, VM is designed to be used when the application resides on the same machine as the broker so gets very high throughput.
As ActiveMQ can enable all protocols by default do some quick tests to gain an idea of throughput on the specific application you are building. Then consider the above points in making a decison.
Regarding UDP, TCP, HTTP I would choose TCP. UDP is unreliable and TCP is more than adequate in sending 1000's per second. HTTP could be useful if your company has awkward firewall rules.
I would wrap this in a failover transport. I have never used discovery but would argue this is more advanced and not required initially as it requires a discovery agent. Its only purpose is too discover the ActiveMQ broker dynamically (although you still have to know where the discover agent is).
Well, the title is quite self expleneationry.. I'm tryin to improve mq client connection time since it takes to much time in my app.
(look here for code)
MQ previously had the UDP protocol available on AIX only. It was removed a few releases ago. TCP/IP is the transport of choice now. If you are having long connection times, you should get your network guys involved.
IBM MQ clients connect to queue manager over TCP sockets as TCP is connection oriented and reliable. Messaging applications require this kind of QoS from underlying transport. UDP is not connection oriented and reliable when compared to TCP.
You say your application takes lot of time to connect, have you looked at if there are any issues in your network? Are you connecting over slow network? Is your application the only one that takes time to connect or other applications have similar problem?
I have a system S of IP addresses that generate and receive data via JMS queues and topics. At the time being this system S is configured to connect to a broker with IP x.x.x.x.
I would like to use the same system S against two different brokers: in other terms I would like to use a sort of "network splitter" that receive the traffic from S and forward it to x.x.x.x AND also to another address y.y.y.y (a second JMS broker). The system S must be completely unaware of the splitting operation. The requirement is not to use networks of jms brokers, jms broker proxies or to play with brokers topology in general. The reason for that is to be general enough to manage protocols other than JMS as possible next steps. JMS connections uses TCP+TLS with mutual authentication.
The system S is made of both Linux and Windows servers.
I would like to do this at software level in order to cope with virtualization of both networks and servers.
What is the best strategy to achieve this? Is there specific software/library that you would recommend, preferably in java?
Thanks!
I'm writting a java application, and I need to quickly discover any other running clients on any wired or wireless local networks in order to establish a TCP connection.
what's the best way of doing this? Are there libraries or code snippets that would do this?
Multicast UDP is a good way of doing this. It's used in a couple of technologies that support automatic discovery of networked devices over local IP networks (UPnP and ZeroConf).
Multicast UDP is not TCP, but it is still based on IP and, so, uses the same addressing mechanism i.e. IP addresses. Quite often it is compared to radio broadcasting i.e. a multicast sender only needs to send 1 message (i.e. it is like a broadcast) but only clients that are "tuned-in" to the multicast channel will receive it.
You can do a quick search on google or wikipedia for these as a starter, but the basic idea is as follows:
when a client starts, it sends out a multicast UDP "hello" message to some pre-specified multicast address and port (e.g. UPnP uses 239.255.255.250:1900)
existing clients are listening for incoming multicast "hello" messages on the specified address and port - when a client receives one, it sends a response to the sender
the client sending the "hello" message receives a response from each existing client on the network and is now aware of the IP address of each client
If you are looking for libraries to use, UPnP libraries can tend to be a bit heavyweight and a lot of folk generally don't like them, so ZeroConf might be a little more suitable. I don't know of any java implementations of such things but I'm sure you can find some with a little digging around.
A network scan can be very long, even longer on wireless networks. If you need them quickly thru Java you may implement a "meeting point" server on your network. This server listen to a predefined port, clients register on the server on startup and the server can distribute information about the clients on request.
HTH.
I guess you need to do a scan on your application's port on all IPs in your subnet.
Just what are the available IPs - or what is your subnet for that matter?
I'm afraid determining that could turn out to be impossible as the network is designed to be transparent to your application.
So, i'd use brute force: pick your IP and change the last byte. Might be too much, might be not enough though.
Or you send a broadcast (which usually would be targeted at x.x.x.255) and see who answers.
See Datagram Broadcasting and Multicasts. But i think that's not TCP/IP anymore.
There is a JGroups toolkit for reliable multicast communications. It allows automatic discovery of additional clients using Multicast techniques as described in other answers.
It also provides communication APIs on top of multicast sockets.
It is used in a number of projects such as JBoss, Tomcat and more to provide an infrastructure for distributed cache. See more here.
I have a serial hardware device that I'd like to share with multiple applications, that may reside on different machines within or spanning multiple networks. A key requirement is that the system must support bi-directional communication, such that clients/serial device can exist behind firewalls and/or on different networks and still talk to each other (send and receive) through a central server. Another requirement of the system is that the clients must be able to determine if the gateway/serial device is offline/online.
This serial device is capable of receiving and sending packets to a wireless network. The software that communicates with the serial device is written in Java, and I'd like to keep it a 100% Java solution, if possible.
I am currently looking at XMPP, using Jive software's Openfire server and Smack API. With this solution, packets coming off the serial device are delivered to clients via XMPP. Similarly, any client application may send packets to the serial device, via the Smack API. Packets are just byte arrays, and limited is size to around 100 bytes, so they can be converted to hex strings and sent as text in the body of a message. The system should be tolerant of the clients/serial device going offline, meaning they will automatically reconnect when they are available again, but packets will be discarded if the client is offline. The packets must be sent and received in near real-time, so offline delivery is not desired. Security should be provided by messaging system and provided client API.
I'd like to hear of other possible solutions. I thought of using JMS but it seems a bit too heavyweight and I'm not sure it will support the requirement of knowing if clients and/or the gateway/serial device is offline.
Jini might fit the job. It works really well in distributed environments where multicast is available but it also works in unicast, and is quite fast. Not only it provides remote services, but also remote events, and distributed transactions if you need them. A downside is that it only works with Java.
Where I work, Jini is used in a infrastructure with more that 1000 machines, with each machine providing remote services used to access the devices connect to the machine serial ports.
You really need to provide a bit more detail... do the clients need guaranteed delivery? What about offline delivery? Is this part of a larger system? Do you need encryption? Security?
If you want the smallest footprint possible, then should transmit data using SocketServer, Sockets, and serialization. But then you lose all of the advantages of the 3rd party solutions you mentioned, which typically include reliability, delivery guarantees, security, management, etc.
I would personally use JMS, but that's because I'm familiar with it. There are a number of stand-alone servers that can be deployed out-of-the-box with virtually no configuration. They all provide for guaranteed delivery, some security, encryption, and a number of other easy-to-use features. Coding a JMS publisher or subscriber is pretty easy.
Update:
If you want the most ease in coding, then I would look at the third-party solutions. Looking at Smack/XMPP, the API seems to be a little easier than a JMS for the functionality you asked for. You still have to setup/configure a server, etc.
The Smack API also has a lot of extra baggage that you don't need either, but its "concepts" are a little more intuitive since its all chat/IM concepts.
I would still look at OpenJMS or ActiveMQ. I think knowing JMS will be more valuable in the future as compared to knowing XMPP. Take a look at their Getting Started documentation or the Sun Tutorial to see how much coding is involved. In JMS parlance, you will want an administered "Topic" and a "Queue" where the Serial Port App will receive and send messages respectively. All of your clients will open a subscription to the Topic and send their outbound messages to the Queue. When you send messages, their delivery mode should be non-persistent.
I ended up using XMPP via the Smack API. What led me to this decision was its native support for presence (is the client online/offline) and robust connection handling (it automatically reconnects if a the underlying connection breaks). Another benefit of XMPP is that it's compatible with Google Talk, so I don't need to setup a server. Thanks for the suggestions. In case anyone is interested, I have released the code on Google Code http://code.google.com/p/xbee-xmpp/