We Keep Coding

UDP socket multicast

I will start by saying that I'm not absolutely a network expert.
I'm trying to create a UDP multicast socket system with four RaspBerry Pi model B.
Into each Rpi is running a listening script (I will call it "listen.py") socket client.
Case 1 - I'm perfectly able to send datagram if I run the socket server (I will call it "server.py") directly inside of the Rpi (by SSH, with my laptop as well).
I have:
-listen.py on boot of the Rpi (here I create a socket client);
-I run the server.py (the server.py is obviously into the Rpi)
In this case I receive answer from all 4 Rpi that joined the multicast group (224.1.1.1)
Case 2 - I created a socket server java and a socket client python in my laptop and everything works perfectly (same code, same multicast group, same multicast port).I tried also without multicast using just 'localhost' and everything works as well. With netstat I can see multicast group and port.
netstat -lu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
udp 0 0 224.1.1.1:21581 *:* //MulticastGroupIp:Port
udp 0 0 192.168.1.103:ntp *:* //Static ip of one Rpi
udp 0 0 *:ntp *:*
My problem is that if I run the server.py from my laptop (no SSH) I have no answer from the listen.py, socket client, which is running into the Rpi. I absolutely need to send datagram from my laptop (I want a java class) to the 4 Rpi.
I have excluded the possibilities of bad code, wrong multicast ip group, wrong multicast port group. Firewall is off in my laptop and in the router as well.
The parameter net.ipv4.ip_forward on the Rpi is 1.
I know that UDP is unreliable but here I'm speaking about 100% of the datagram lost.
IGMP is enabled on my D-Link router.
I have the same problem with Ethernet and WiFi configuration.
I really need other ideas.....So I tried to write here.
Sorry, this is my first post and I'm newbie a bit of everything.
I will appreciate any suggestion.
Probably it's something stupid that I can't understand.
Maybe some problems with the routing table???? This is mine:
route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default D-Link.Home 0.0.0.0 UG 0 0 0 wlan0
192.168.1.0 * 255.255.255.0 U 0 0 0 wlan0
Thanks in advance
For me the code works well and the problem is not there but if can be useful to understand better the situation.....
Code of server.py (useless to say that I have python installed in my machine with Windows 7):
import socket
import sys
# import time
print 'message:'
n = sys.stdin.readline()
n = n.strip('\n')
MCAST_GRP = '224.1.1.1'
MCAST_PORT = 21581
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 2)
sock.sendto(n, (MCAST_GRP, MCAST_PORT))
or with java (It's exactly what I want...a server socket java but at the moment it's not the priority):
public class PythonScriptScan {
private static int portMulticasting = 21581;
// private boolean broadcast = true;
private DatagramSocket socket;
private String groupMulticast = "224.1.1.1"; // group address
private int delay = 3000;
public PythonScriptScan() {
try {
socket = new DatagramSocket();
} catch (SocketException e) {
e.printStackTrace();
System.exit(1);
}
}
public void start(String agentName) {
try {
InetAddress group = InetAddress.getByName(groupMulticast);
#SuppressWarnings("resource")
MulticastSocket s = new MulticastSocket(portMulticasting);
s.joinGroup(group);
// while (broadcast) {
byte[] buf = new byte[10240];
buf = agentName.getBytes();
DatagramPacket packet = new DatagramPacket(buf, buf.length, group, portMulticasting);
System.out.println(packet);
socket.send(packet);
// OK, I'm done talking - leave the group...
s.leaveGroup(group);
try {
Thread.sleep(delay);
} catch (InterruptedException e) {
System.exit(0);
}
// }
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
public static void main(String[] args) throws IOException {
System.out.println("Insert Message");
#SuppressWarnings("resource")
Scanner sc = new Scanner(System.in);
String agentName = sc.nextLine();
PythonScriptScan agent = new PythonScriptScan();
agent.start(agentName);
}
and listen.py (It's running on boot in each Rpis, the configuration that I want is with Ethernet, each Rpis has a static ip and they are connected properly with my D-link router):
#!/usr/bin/python
import socket
import struct
import fcntl
import subprocess
import sys
import errno
import time
import os
# Create the socket
MCAST_GRP = "224.1.1.1"
MCAST_PORT = 21581
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
except socket.error, msg:
print 'Could not create socket. Error code: ' + str(msg[0]) + ' , Error message : ' + msg[1]
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# Bind to the server address
# sock.bind(('', MCAST_PORT))
sock.bind((MCAST_GRP, MCAST_PORT))
# Tell the operating system to add the socket to the multicast group on all interfaces
mreq = struct.pack('4sl', socket.inet_aton(MCAST_GRP), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
# Receive loop
try:
while True:
print >>sys.stderr, '\nWAITING TO RECEIVE MESSAGE'
d = sock.recvfrom(10240)
data = d[0]
addr = d[1]
print data.strip(), addr
finally:
print >>sys.stderr, 'closing socket'
sock.close()

UDP hole punching can't connect externally

I have previously made a simple peer to peer chat program using UDP hole punching that worked and now I am trying to do something similar but within a game made using libGDX. The game itself runs fine and the connections work on a LAN but I have difficult trying with external connections. I understand how UDP hole punching works as follows:
If both person A and B know each other's IP addresses and ports then:
A sends a UDP packet to B which punches a hole in A's NAT but is dropped by B's firewall
A waits for a reply
B sends a UDP packet to A which punches a hole in its own NAT and goes through A's firewall
B waits for a reply
A receives B's initial message and sends a second message to B
B receives A's message
My networking code is in one class:
private boolean connected;
private DatagramSocket socket;
private DatagramPacket packet;
private InetAddress peerIP;
public NetworkManager(InetAddress peerIP){
this.peerIP = peerIP;
log("Created with peer ip: " + peerIP.getHostAddress());
connected = false;
}
#Override
public void run(){
try{
log("Setting up socket");
socket = new DatagramSocket(Constants.CLIENT_PORT);
log("Socket successfully setup");
//Punch hole
log("Punching UDP Hole");
sendBytes("one");
//Receive
log("Waiting for peer reply");
receiveBytes(3);
//Send second message
if(Arrays.equals(packet.getData(), "one".getBytes()) ){
sendBytes("two");
}
}catch(Exception e){
log("Error connecting to peer");
return;
}
log("Successfully connected");
connected = true;
}
private synchronized void receiveBytes(int length) throws Exception {
packet = new DatagramPacket(new byte[length], length);
log("Receiving " + length + " bytes...");
socket.receive(packet);
log("Received bytes " + packet.getData()+ " from " + packet.getAddress());
}
private synchronized void sendBytes(String s) throws Exception {
byte[] sendBytes = s.getBytes();
packet = new DatagramPacket(sendBytes, sendBytes.length, peerIP, Constants.CLIENT_PORT);
log("Sending " + sendBytes.length + " bytes...");
socket.send(packet);
log("Bytes sent");
}

If both A and B are in the same network, i.e if they are in the same LAN, A can connect to B and vice versa. You don't need UDP Hole punching here.
But if they are in different networks, or behind different NATs, you can try to achieve a direct connection using UDP hole punching. Here you need a mediator server. And A and B need to ping mediator server, which will pass the IP and port number to the other corresponding clients.
Most important point to note here is that not all NATs supports hole punching. It is upto the NAT implementation, which is mostly not available in public domain. So even if you use UDP Hole Punching, you need a relay server as a fallback. In case it doesn't get hole punched, the relay server can pass the messages to each other.
http://www.brynosaurus.com/pub/net/p2pnat/
This link explains UDP Hole Punching architecture.

Losing message when using multicast in local machine

I wrote a java program that is able to send/receive messages via a multicast address.
My code for multicast sender:
public boolean multicastSender(DataTransferObject message) {
try {
InetAddress group = InetAddress.getByName(ip);
MulticastSocket s = new MulticastSocket(port);
s.joinGroup(group);
String msg = jsh.dtoToJsonString(message);
DatagramPacket data = new DatagramPacket(
msg.getBytes(), msg.length(), group, port);
s.send(data);
s.leaveGroup(group);
return true;
} catch (IOException e) {
System.out.println(e.toString());
e.printStackTrace();
return false;
}
}
My code for multicast receiver:
public DataTransferObject multicastReceiver() {
try {
InetAddress group = InetAddress.getByName(ip);
MulticastSocket s = new MulticastSocket(port);
s.joinGroup(group);
byte[] buffer = new byte[1000 * 1024];
DatagramPacket data = new DatagramPacket(buffer, buffer.length);
String jsonData;
DataTransferObject dto;
while (true) {
s.receive(data);
jsonData = new String(buffer, 0, data.getLength());
dto = jsh.dtoFromJson(jsonData);
if (dto == null) {
continue;
} else {
return dto;
}
}
} catch (IOException e) {
System.out.println(e.toString());
e.printStackTrace();
return null;
}
}
My program has 4 parallel threads that listen and receive messages from four different multicast addresses (224.0.0.[1->4]) and ports [66601->66604]. In the test, I run two programs at the same time to test the communication between them. Yet I discovered that sometimes, program A sends a message to a multicast address, in a correct case, A will also receive it and so will B. But sometimes, I see that A announced that it sent the message but after that, neither A or B receive it again. I run the test on my local machine (Mac OS 10.9 with core 2 duo) . Should this problem because of my computer or something else?
Thank you very much.

I've seen this happen in a Windows environment. I understand that it's because the network layer is separate from the JVM.
What this means is one JVM may read and process/discard this message, yet other JVM's won't read the same message as the network layer has already delivered the UDP message to the first consumer, no matter how many JVM's have subscribed to the same group. This may or may not be a bug in the operating system - it's very subjective. Personally I think the OS should track registrants in order to replicate the delivery, as that's the intention of clients - it's a bit misleading to think otherwise given that an application is joining a group to receive multicasts. The operating system should take this into consideration.
I have come to this conclusion through empirical evidence, and may not be true for other platforms.

UDP, which I guess underlies your multicast implementation, does not guarantee delivery. The receiver or network will drop packets when overloaded. You'll need a higher-level protocol to ensure delivery (google reliable multicast).

Java: Receiving an UDP datgram packet with multiple DatagramSockets

I am trying to implement a method that sends an UDP packet to multiple receivers. I thought that this should be doable setting setReuseAddress(true) on the receiving DatagramSocket instances.
My problem is that in certain conditions I need to limit the communication to the local computer - hence the localhost interface (useLocalhost=true in the demo code below). In such a case suddenly only the first receiver socket gets the incoming packet, the two other don't see anything.
I tested this on Windows (oracle 64bit) and Linux (OpenJDK 64bit), therefore I only see three possibilities:
This is an intended and known behavior (and I don't understand the whole mechanism - aka "bug in my brain")
There is a bug in the Java JRE
There is a bug in my code.
Does somebody have any experience on that topic and can me help to identify where the problem is located?
See below a minimal working example that demonstrates this. Note that I am using the broadcast address for simulating network packets that come from a real external host.
If everything goes right you should see three lines at the end (in this or a different order):
Thread-0 - packet received
Thread-1 - packet received
Thread-2 - packet received
 
public static void main(String[] args) throws Exception {
boolean useLocalhost = true;
InetSocketAddress addr;
String sendPacketTo = "192.168.1.255"; // we use broadcast so that packet comes from an real external address
if (useLocalhost)
sendPacketTo = "localhost"; // does not work (only listener 1 received packet)
addr = new InetSocketAddress(15002);
new MyThread(addr).start(); // Datagram socket listener 1
new MyThread(addr).start(); // Datagram socket listener 2
new MyThread(addr).start(); // Datagram socket listener 3
DatagramSocket so = new DatagramSocket();
so.setBroadcast(true); // does not change anything
so.connect(new InetSocketAddress(sendPacketTo, 15002));
so.send(new DatagramPacket("test".getBytes(), 4));
Thread.sleep(1000);
System.exit(0);
}
public static class MyThread extends Thread {
DatagramSocket socket;
public MyThread(InetSocketAddress addr) throws SocketException {
super();
setDaemon(true);
socket = new DatagramSocket(null);
socket.setReuseAddress(true);
socket.setBroadcast(true); // does not change anything
socket.bind(addr);
System.out.println("Listener started: " + socket.getLocalAddress());
}
public void run() {
byte[] buf = new byte[10];
DatagramPacket p = new DatagramPacket(buf, buf.length);
try {
socket.receive(p);
System.out.println(Thread.currentThread().getName() + " - packet received");
} catch (IOException e) {
e.printStackTrace();
}
}
}

192.168.1.255 is a broadcast address, so the datagram is broadcast, under the rules for UDP broadcast. 127.0.0.1 is a unicast address, so the packet is unicast. So you get different behaviour.
As #DavidSchwartz commented, your code is a mixture. Connecting to a broadcast address for example doesn't have a lot of meaning, and neither does binding to it. I think what you are looking for is multicast.

You can use multicast on localhost
However, there are several things you need to be careful of to make it work.
example:
lo0 (127.0.0.1)
en0 (192.168.0.111)
en1 (10.1.0.111)
for each interface 2 separate sockets, one for receiving, one for
sending. In the above example this means creating a total of 6
sockets.
Never bind() a socket that will send multicast UDP packets.
Always bind() a socket that will receive multicast UDP packets.
Never try to setsockopt() or reconfigure multicast sockets after you call bind()
Instead, when machine's interfaces change due to cables being unplugged/plugged,
destroy all send/receive multicast sockets and recreate them.
sample code:
iMulticastSocketInterfaceIPAddress would be one of the three interfaces
/* use setsockopt() to request that the kernel join a multicast group */
struct ip_mreq mreq;
mreq.imr_multiaddr.s_addr=inet_addr( "239.192.0.133" );
myAddress.sin_addr.s_addr = mreq.imr_multiaddr.s_addr;
mreq.imr_interface.s_addr=( htonl(iMulticastSocketInterfaceIPAddress) );
theErr = setsockopt( CFSocketGetNative( mSocketBroadcast ) ,IPPROTO_IP,IP_ADD_MEMBERSHIP,&mreq,sizeof(mreq));

Listening for TCP and UDP requests on the same port

I am writing a Client/Server set of programs
Depending on the operation requested by the client, I use make TCP or UDP request.
Implementing the client side is straight-forward, since I can easily open connection with any protocol and send the request to the server-side.
On the servers-side, on the other hand, I would like to listen both for UDP and TCP connections on the same port. Moreover, I like the server to open new thread for each connection request.
I have adopted the approach explained in: link text
I have extended this code sample by creating new threads for each TCP/UDP request.
This works correctly if I use TCP only, but it fails when I attempt to make UDP bindings.
Please give me any suggestion how can I correct this.
tnx
Here is the Server Code:
public class Server {
public static void main(String args[]) {
try {
int port = 4444;
if (args.length > 0)
port = Integer.parseInt(args[0]);
SocketAddress localport = new InetSocketAddress(port);
// Create and bind a tcp channel to listen for connections on.
ServerSocketChannel tcpserver = ServerSocketChannel.open();
tcpserver.socket().bind(localport);
// Also create and bind a DatagramChannel to listen on.
DatagramChannel udpserver = DatagramChannel.open();
udpserver.socket().bind(localport);
// Specify non-blocking mode for both channels, since our
// Selector object will be doing the blocking for us.
tcpserver.configureBlocking(false);
udpserver.configureBlocking(false);
// The Selector object is what allows us to block while waiting
// for activity on either of the two channels.
Selector selector = Selector.open();
tcpserver.register(selector, SelectionKey.OP_ACCEPT);
udpserver.register(selector, SelectionKey.OP_READ);
System.out.println("Server Sterted on port: " + port + "!");
//Load Map
Utils.LoadMap("mapa");
System.out.println("Server map ... LOADED!");
// Now loop forever, processing client connections
while(true) {
try {
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
// Iterate through the Set of keys.
for (Iterator<SelectionKey> i = keys.iterator(); i.hasNext();) {
SelectionKey key = i.next();
i.remove();
Channel c = key.channel();
if (key.isAcceptable() && c == tcpserver) {
new TCPThread(tcpserver.accept().socket()).start();
} else if (key.isReadable() && c == udpserver) {
new UDPThread(udpserver.socket()).start();
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
} catch (Exception e) {
e.printStackTrace();
System.err.println(e);
System.exit(1);
}
}
}
The UDPThread code:
public class UDPThread extends Thread {
private DatagramSocket socket = null;
public UDPThread(DatagramSocket socket) {
super("UDPThread");
this.socket = socket;
}
#Override
public void run() {
byte[] buffer = new byte[2048];
try {
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
socket.receive(packet);
String inputLine = new String(buffer);
String outputLine = Utils.processCommand(inputLine.trim());
DatagramPacket reply = new DatagramPacket(outputLine.getBytes(), outputLine.getBytes().length,
packet.getAddress(), packet.getPort());
socket.send(reply);
} catch (IOException e) {
e.printStackTrace();
}
socket.close();
}
}
I receive:
Exception in thread "UDPThread" java.nio.channels.IllegalBlockingModeException
at sun.nio.ch.DatagramSocketAdaptor.receive(Unknown Source)
at server.UDPThread.run(UDPThread.java:25)
10x

It should work. One of the problems with this code, it seems, is that the ByteBuffer size is set to 0, meaning that the datagram is discarded (as it mentions in the comments). If you need to receive any information over UDP and you are on a reliable network, you can set the size quite big and receive big datagrams made up of multiple packets. Otherwise, on an unreliable network, set this to the MTU size. Make sure you flip() the ByteBuffer after receiving anything in it.
Also, creating new threads for each request is a bad idea, create a 'session' thread for each different IP you receive in a HashMap or something, and then do a guarded block on the session object. Wake up the thread sleeping on that object when you receive a message after passing in new information. The selector code you have is designed to avoid the creation of threads in this way.
Edit: based on the above code, you're create a datagram channel and then using the socket to receive datagrams directly? That's doesn't make sense. Use the channel methods only after binding the channel. Also, don't do this in a separate thread. Your code isn't thread-safe and will bust itself up. Hand the received information off to the separate 'session' thread as mentioned earlier. The selector is designed to tell you what channels can be read from without blocking (although blocking is disabled anyway, so it will tell you what channels have data to be read from).

AFAIK, you should be able to listen for both TCP connections and UDP messages on the same port. It would help if you posted your UDP code, and the exception + stacktrace that you are seeing.

You can't use DatagramSocket.receive() in non-blocking mode. You have to use the read() or receive() methods of your DatagramChannel directly.
In fact as you are using non-blocking mode and a Selector, it is quite impossible to see why you're also using a UDPThread at all. Just call udpserver.receive() instead of starting the thread.