my task is to receive UDP packets on an Google Compute Engine. Given is the port, which is 300 and I already have a simple Java program which can handle these UDP packets.
My problem now. Get the UDP packets to my Java program.
For that I don't know how to do this.
What I have tried so far ist to let the Java program direct listen to the port, which didn't work. I suspect the traffic from outside the GCE must be routed to the inside?
First make sure you've added a firewall rule on your GCE VM instance's network which allows incoming traffic for UDP protocol on port 300.
You can do this by going to the Developers Console, select your project, then Compute-> Compute Engine-> Networks, click on the right network and verify firewall rules (and tags as well if you used them). If the rule is not added, then add a rule for the traffic.
"gcloud" is a command-line tool which you can use it to list and verify your firewall rules as well [1]:
$ gcloud compute firewall-rules list
If the firewall rules are good, then use a simple troubleshooting tool like netcat to test if traffic is being forwarded to your VM instance.
1. Listing Google Compute Engine firewall rules
I recommend not using Java to process UDP. I experienced inexplicable short periods where all UDP traffic arriving was being lost. This was caused by garbage collection.
The architecture that works for me is to use a GCE VM with a C++ app receiving UDP data, then using libCURL, convert the data to HTTP and dispatch it to Google App Engine for processing. To make the UDP receiver scalable and tolerant, use network balancing and multiple VMs listening for UDP.
Related
I have implemented a simple P2P multicast network where each peer connected to the network sends data to every other peer. In order to make it possible, I made use of Java MulticastSocket Class, so every peer joins a multicast group defined by an available for multicast IP address, and a port number.
I got some issues while trying to run the program using EC2 services, where each generated instance represents a peer of the network, because the peers are not able to communicate in so far as exchanged messages do not reach the receivers.
Hence, basically my question is: is there a way to make EC2 instances communicate to each other using Java MulticastSocket? Can anyone help?
Natively, you can't.
Q. Does Amazon VPC support multicast or broadcast?
No.
https://aws.amazon.com/vpc/faqs/
Same thing for EC2 Classic (though there is hardly any reason you'd ever want to use that, if your account even allows it).
VPC looks like Ethernet, but it isn't. Put a packet sniffer on and try a ping. Watch the ARP traffic on both sides and you'll see something enlightening -- the source machine arps for the target and gets a response, but you'll see neither of these packets on the target machine. The ARP response comes from the network infrastructure itself.
There's a workhackaround, if you're feeling crafty: you can build an overlay mesh network that transports multicast over unicast.
See https://aws.amazon.com/articles/6234671078671125
We are working on a Android project with the below requirements.
The application should be able to send data to all the devices which are running our application which exists in the WiFi LAN.
Some payloads are expected to be of size >= 5MB.
The data shouldn't be lost and if lost the client should know the failure.
All the devices should be able to communicate with all other. There will be no message targeted to a specific device instead all the messages should be reached all the devices in the N/W.
No internet hence no remote server.
Study we have done:-
UDP Broadcasting - UDP doesn't guarantee the message delivery but this is a prime requirement in our case. Hence not an option.
TCP - TCP guarantees the message delivery but requires the receiver IP address to be known before hand and in our case we need to send the message to all the devices inside the LAN. Hence not a straight option.
Solutions we are looking into:-
A Hybrid approach - Name one of the devices in the N/W as Server. Post all the messages to a local Server. The Server keeps a open socket to all the devices(which have our application) & when there is a message from a device then it routes the message to all the devices. The disadvantages of this approach are,
Server having multiple sockets open each per device. But in our case we are expecting devices <=5 in LAN.
Server discovery using continuous UDP broadcast.
We want to have all the data in all the devices. So if we newly introduce any device into the LAN then that device needs to get all the data from the server.
So my question, have you any time worked on these kind of hybrid approaches? Or can you suggest any other approaches?
Your hybrid approach is the way to go.
Cleanly split your problem into parts and solve them independently:
Discovery: Devices need to be able to discover the server, if there is any.
Select server: Decide which of your devices assumes the server role.
Server implementation: The server distributes all data to all devices and sends notifications as necessary. Push or pull with notifications does not matter.
Client implementation: Clients only talk to the server. The device which contains the server should also contain a normal client, potentially passing data to the server directly, but using the same abstract protocol.
You could use mDNS (aka Bonjour or zeroconf) for the discovery, but I would not even recommend that. It often createsmore problems than it solves, and it does not solve your 'I need one server' problem. I would suggest you handcraft a simple UDP broadcast protocol for the discovery, which already tells you who the server is, if there is any.
Select server: One approach is to use network meta data which you have anyway, for example 'use the device with the highest IP address'. This often works better than fancy arbitration algorithms. Once you established a server new devices would use this, rather than switching the server role.
Use UDP broadcast for the discovery, with manual heuristic repeats. No fancy logic, just make your protocol resilient against repeated packets and repeat your packets. (Your WLAN router may repeat your packets without your knowledge anyway.)
You may want to use two TCP connections per client, potentially to two different server ports, but that does not matter much: One control connection (always very responsive, no big amounts of data, just a few hundred bytes per message) and one data connection (for the bulk of the data, your > 5 MB chunks). This is so that everything stays responsive.
As I am programming a network chat (java, but should not make a difference for the question), and wanted to use UDP, I ran into the problem of it not working over the internet. After a little research I found out that you have to have port forwarding for the specific port activated. So now it comes to my question:
Does UDP work over the Internet in a not configurable way?
For example, if I would program a whole Network Game would it make sense to use UDP? Or would I require the Player to activate Portforwarding and open the Port etc?
When would it make sense to use UDP then? And why?
I'm actually not understanding the whole point of UDP then.
For my programming point of view I would like to have a way to use it intuitive.
Like creating the DatagramSocket and the DatagramPacket, configure the Packet with the Data and the Destination and send it away over the internet.
As for my Users I don't want them to have to configure any specific things like opening the exact port they want to use etc. I just want them to use the program (server and client) and it should work.
The problem you've run into is not one of UDP vs TCP (although using the unreliable, unordered UDP as the basis of a chat application seems like an odd choice to me).
The problem is that of NAT traversal. In a nutshell, home routers perform a network function called NAT - Network Address Translation. They do it in order to use a single public IP address for all machines inside the NAT (which are given private addresses - usually 10.0.0.0 or 192.168.0.0). The router then switches the source IP address in all packets sent from inside the LAN from the private address to the public one. It uses port numbers to "remember" which machine sent what to what address, in order to perform the backwards translation when the response arrives.
The problem arises when someone wants to initiate a connection to a machine behind a NAT. Without seeing an outgoing connection first, the NAT doesn't know to which internal computer and port it should forward the packet. This is what happens to you.
There are various fixes for this issue, with the simplest one being manual port forwarding (as you've discovered), but it's a well known problem faced by any peer-to-peer application. If you need to contact a machine behind NAT (i.e. contact most home users) and you want your application to work out-of-the box (without your users fiddling with their routers) you need to research NAT traversal techniques, implement them in your application, and hope that the user's home routers support them. It's a huge pain in the neck.
EDITED: with Joachim Pileborg's correct suggestions!
UDP is often a better choice for action-based games, where it's vitally important to have updates to the client or server with the latest data about a player, player input, or the game world.
TCP begins with a 3-way handshake to establish a connection (which takes time). If your game communication protocol is via TCP, all packets in a message have to arrive before the message becomes available. Even a small amount of Internet congestion could cause your game to lag.
TCP is good for communications that MUST arrive in full.
With UDP, the client or server can send the latest player/game state in individual packets that do not depend on arriving in order. If a packet is late, or arrives out of order... it should be ignored.
UDP is good for communications that need to be fast, but losing individual packets is OK.
Both should be available in your Java platform.
Here's some further reading:
http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/
I have a application which listens for commands over IP.
The program works fine locally but when I try to send the application commands with a remote address it won't connect.
Is there anyway to get around the router blocking the inbound network traffic?
I'm using JAVA
Thanks.
If this is a pair of hosts you control, you can open the incoming port. You might also
succeed using a tunneling program such as Hamachi to effectively set up a VPN linking
the hosts.
If you're talking about a pair of unrelated computers (for example trying to set up
a connection for a game) there's no general solution that doesn't involve installing
and trusting additional software. The usual solution is to use a public server and
relay the traffic between the end points.
given a flash application that opens a socket connection to a webserver, is it possible to reads packets exchanged with a java application, without redirect all the flash traffic ( that is, without programming a socket proxy)?
What you are trying to do requires lower level network analysis than sockets. Namely libpcap and its Java bindings , jNetPcap. This will let you capture packets much in the same way wireshark does, but from Java. The other options are analysing Wireshark logs after an experiment and that can get clunky quite quickly. You may also consider writing a custom wireshark dissector.
The only way I can think is to modify the hosts file to list your Java server address as if it was the destination address.
BTW if your are only interested in examine the network traffic for that app, you could also use fiddler