My professor gave me the task to realize a file transfer via UDP, which implements the protection mechanisms for reliability like as TCP (CRC check, correct packet order, ACK/NACK). I got some default classes from him (Socket & Channel) to simulate packet loss and delay on local machine. A packet loss when sending the packets is also simulated in its classes. However, this means that if an ACK is not sent correctly i can't notice it either, since an ACK from server to client is not confirmed with an ACK for an ACK from client to server.
I thought that packet loss can only happen when receiving packets.
Is it possible in real cases that an packet can be lost while sending without getting a code exception?
Greetings
Is it possible in real cases that an packet can be lost while sending without getting a code exception?
Easy. Simply send the packets from your application faster than the network interface can forward these. Of course any intermediate systems on the way (i.e. switches, routers) might also get overloaded and loose packets.
But at the end it does not actually matter how the packet is lost, i.e. if one the local system while sending, on the remote system while receiving or in between while forwarding. One simply cannot assume that a successful send will be matched by a successful recv.
IP packets contain the transport protocol (TCP, UDP, etc.) in their payload, and you lose IP packets all the time due to things like oversubscription at some point in the path. QoS, controls can also use something like RED that purposely discards packets in order to keep queues from filling and doing tail-drop. TCP can realize that TCP segments (not packets) are lost and resend them, but UDP has no mechanism to realize that its datagrams (not packets) are lost. UDP is a fire-and-forget protocol.
Related
Sorry for such the long post. I have done lots and lots of research on this topic and keep second guessing myself on which path I should take. Thank you ahead of time for your thoughts and input.
Scenario:
Create a program in Java that sends packets of data every .2-.5 seconds that contain simple "at that moment" or live data to approximately 5-7 clients. (Approximately only 10 bytes of data max per packet). The server can be connected via wifi or ethernet. The clients however are restricted to only using Wifi. The client(s) will not be sending any packets to the server as it will only display the data retrieved from the server.
My Thoughts:
I originally started out creating the server program using TCP as the transport layer. This would use the ServerSocket class within Java. I also made it multithreaded to accept multiple clients.
TCP sounded like a great idea for various reasons.
With TCP, the message will always get sent unless the connection fails.
TCP rearranges the order of packets sent.
TCP has flow control and requires more time to set up when started.
So, perfect! The flow control is crucial for this scenario (since the client wants the most concurrent information), the setup time isn't that big of a deal, and I am pretty much guaranteed that the message will be received. So, its official, I have made my decision to go with TCP....
Except, what happens when TCP gets behind due to packet loss? "The biggest problem with TCP in this scenario is its congestion control algorithm, which treats packet loss as a sign of bandwidth limitations and automatically throttles the sending of packets. On 3G or Wi-Fi networks, this can cause a significant latency."
After seeing the prominent phrase "significant latency", I realized that it would not be good for this scenario since the client needs to see the live data at that moment, not continue receiving data from .8 seconds ago. So, I went back to the drawing board.
After doing more research, I discovered another procedure that involved UDP with Multicasting. This uses a DatagramSocket on the server side and a MulticastSocket on the client side.
UDP with Multicasting also has its advantages:
UDP is connectionless, meaning that the packets are broadcasted to anyone who is listening.
UDP is fast and is great for audio/video (although mine is simple data like a string).
UDP doesn't guarantee delivery - this can be good or bad. It will not get behind and create latency, but there is no guarantee that all client(s) might receive the data at that moment.
Sends one packet that gets passed along.
Since the rate of sending the packets will be rather quick (.2-.5 sec), I am not too concerned about losing packets (unless it is consistently dropping packets and looks to be unresponsive). The main concern I have with UDP above anything is that it doesn't know the order of which the packets were sent. So, for example, lets say I wanted to send live data of the current time. The server sends packets which contain "11:59:03", "11:59:06", "11:59:08", etc. I would not want the data to be presented to the client as "11:59:08", "11:59:03", "11:59:06", etc.
After being presented with all of the information above, here are my questions:
Does TCP "catch up" with itself when having latency issues or does it always stay behind once the latency occurs when receiving packets? If so, is it rather quick to retrieving "live" data again?
How often do the packets of data get out of order with UDP?
And above all:
In your opinion, which one do you think would work best with this scenario?
Thanks for all of the help!
Does TCP "catch up" with itself when having latency issues or does it always stay behind once the latency occurs when receiving packets?
TCP backs its transmission speed off when it encounters congestion, and attempts to recover by increasing it.
If so, is it rather quick to retrieving "live" data again?
I don't know what that sentence means, but normally the full speed is eventually restored.
How often do the packets of data get out of order with UDP?
It depends entirely on the intervening network. There is no single answer to that.
NB That's a pretty ordinary link you're citing.
UDP stands for 'User Datagram Protocol', not 'Universal Datagram Protocol'.
UDP packets do have an inherent send order, but it isn't guaranteed on receive.
'UDP is faster because there is no error-checking for packets' is meaningless. 'Error-checking for packets' implies retransmission, but that only comes into play when there has been packet loss. Comparing lossy UDP speed to lossless TCP speed is meaningless.
'UDP does error checking' is inconsistent with 'UDP is faster because there is no error-checking for packets'.
'TCP requires three packets to set up a socket connection, before any user data can be sent' and 'Once the connection is established data transfer can begin' are incorrect. The client can transmit data along with the third handshake message.
The list of TCP header fields is incomplete and the order is incorrect.
For TCP, 'message is transmitted to segment boundaries' is meaningless, as there are no messages.
'The connection is terminated by closing of all established virtual circuits' is incorrect. There are no virtual circuits. It's a packet-switching network.
Under 'handshake', and several other places, he fails to mention the TCP four-way close protocol.
The sentences 'Unlike TCP, UDP is compatible with packet broadcasts (sending to all on local network) and multicasting (send to all subscribers)' and 'UDP is compatible with packet broadcast' are meaningless (and incidentally lifted from an earlier version of a Wikipedia article). The correct term here is not 'compatible' but supports.
I'm not fond of this practice of citing arbitrary Internet rubbish here and then asking questions based on it.
UDP packets will may get out of order if there are a lot of hops between the server and the client, but more likely than getting out of order is that some will get dropped (again, the more hops the more chance of that). If your main concern with UDP is the order, and if you have control over the client, then you can simply have the client discard any messages with an earlier timestamp than the last one received.
Let's say you want to make a real-time game, perhaps a 2D topdown game.
Things you would do in the game to keep it simple are:
Connect to the server
Move the player with the keys
Possibly press space bar to attack
Send a chat message
But what would happen if a datagram from any of these situations get lost?
What would you do?
1) Connecting to the server
If you send a UDP datagram to the server, the server would take your IP and port and then create a player
based on an ID it gives you, this it how it identifies you every time it receives a datagram from you.
But what if upon "connection" (not actually a connection, just a udp datagram that says 'make me a part of your server'), this initial datagram gets lost. Is it right to say that you would just resend it if after a certain period of time you did not receive a reply back from the server?
2) Moving the player/attacking
If at any time we move the player/press a movement key/attack key, we would send a keystroke
to the server telling it what key we pressed/released.
The server would then relay this to all the other clients.
But what if this keystroke datagram gets lost? Either upon client->server or server->clients.
Because keystrokes are happening all the time, is it efficient to just resend it all the time?
Or would we just ignore the fact that it got lost along the way because there's a chance that if you press a key again it will most likely make it the next time?
3) Sending a chat message
This would be something that MUST reach the server.
If it got lost along the way, after a certain period of time if we did not receive a reply from the other side/receiving end, do we re-send it?
TL;DR
Is it okay to just keep resending datagrams if we know after a certain period of time it did not reach?
And also, what about confirming if a datagram got successfully sent?
If client sends a chat message to the server, and the server receives it, should the server send a reply back to the client to confirm that it received it? What would happen if that reply got lost, what then?
Typically games using UDP have an application level protocol on-top so they can send some messages reliably and others not. If they wanted to send everything reliably they would be better off using TCP. However fast-paced games can not afford the delay TCP introduces and if the packet does get lost it's often too late to re-send it!
One way to implement reliable messages is, like you suppose, to send an ACKnowledgment reply that this particular packet was received - but what if that gets dropped you ask? The sender of the reliable packet typically re-sends a certain number of times (e.g. 3 times) and if no ACK is still received the sender then assumes the peer has dropped.
An excellent place for articles and game networking including how to minimise data sent is Shawn Hargreaves blog. (This is focused at C# XNA but the same logic applies irrespective of language/framework/platform).
Lastly:
Game network programming for fast paces games is hard. If your game is slow and/or turn based consider TCP and a lock-step approach which makes things considerably easier.
Don't re-invent the wheel, there are libraries that do things such as implementing reliability on top of UDP for when you need it, e.g. FalconUDP. Though this is a .NET one - I would like to see a Java port :)
If a datagram gets lost there is nothing the receiver can do, because he doesn't know about it. It's up to the sender to re-send, and it's up to you to design an ACK-based or NACK-based protocol so that the receiver knows when to do that.
Whether it is okay to just keep resending UDP datagrams depends on how the server treats the data in the datagrams. For example, if you resend a chat message, and the server receives both the original message and the resent message, does the server display the message twice? Note that you might end up resending even if the original message was received, because the reply to the original message might have gotten lost or delayed.
The best bet is to use TCP instead of UDP, since TCP has already solved all these problems for you. World of Warcraft, for example, started with TCP for chat and UDP for gameplay, and eventually converted to TCP for everything.
I have an assignment to measure the number of UDP messages dropped between client(s) and a server. The client and server are wrote in Java. The assignment is to measure how many packets are lost using varying sizes and numbers.
The assignment says the 'server' should count how many messages have been dropped. I don't see how that is possible because the messages are UDP-- therefore, how could the server even know a message was dropped?
a complex way : you can create application level reliability.
receiver should reply with ack. if sender doesn't get ack in a timeout value, it is considered as dropped.
(added the term : complex)
If a datagram from the client, saying how many datagrams it sent so far, would reach the server, the server could calculate it, couldn't it?
I sometimes receive already received packets (I used sniffer and system ACKs them). Now I read all data (until socket timeout) and then send new request, but this is ugly. I was thinking about using sequence numbers but i didn't find it in Socket interface. Any clues?
No you don't. If the receiving TCP stack misses a packet, it will re-request it, but it can't have delivered the original one to you, because it missed it. And if it gets a packet it has already received, it will drop it.
TCP will deliver all the bytes that are sent, in the order they are sent. Nothing else (well, except some edge cases around disconnects).
Something else is going on.
EDIT:
To be clear, I'm talking about the bytes that are delivered to your application through the socket's InputStream. What happens on the wire is largely irrelevant unless you have some horrific network retransmission problem that you're trying to investigate. And if the receiving stack does get a duplicate packet, it will ACK it, because if it didn't then the sender would re-send it... again.
It sounds like you're trying to account for things that TCP already takes care of. It has sequence numbers built in and will take care of any lost data for you, and from the receiving point you should be waiting until you receive all your expected data, rather than reissuing a request. If you don't want to wait for a response to complete before issuing a new request, consider pipe-lining requests with multiple connections.
I am using MulticastSocket to receive UDP Multicast packets. How can I determine to which address such a packet was sent? With the methods provided, I am only able to determine the sender address.
Of course, I am the one who sets the To-Address when creating the listening socket, but can I really be sure about this? What about broadcast packets? What about packets that somehow end up here?
I really want to distinguish if the packet was REALLY multicast.
Thank you!
Update: Currently it seems like unicast packets just sent to that port also end up in the multicast sockets receive() :( Binding to devices also gives me no better results
I'm a bit fuzzy on the details but a multicast packet will have been sent to the ip/port combo you subscribed to (and this info will be in the packet, somewhere), assuming you managed to have a clear path of intermediary routers that understand multicast. If you want to make sure the multicastsocket is receiving from the right network interface, there's a bunch of functions to bind it to a specific interface.
I don't think you have any way of knowing if the packet was "really" multicast, i.e. someone could always forge one, since there's no real security built in.