I am attempting to send one character at a time to a server using a Java client.
But packets are being grouped even though I disabling Nagle's algorithm.
s.setTcpNoDelay(true);
Any solutions?
Tcp connections guarantee you that the data will arrive in the order that you sent it. It does not guarantee that it will all arrive in one packet. It depends on the network and the size of the Tcp packet. So your data can arrive in fragments. The best approach is to use parse the data and let the receiver determine if the packet is complete.
TCP is not designed to give you control over how much data is in a packet, and as Gusman said, even if you manage to craft your TCP packets like that, the recipient's kernel is allowed to merge your packets together or split them up. TCP is designed to give you a continuous stream of characters (as opposed to UDP, which lets you specify which data should go into one packet).
However, I doubt that you want to use UDP. Judging from your question, what you really want is a way to delimit messages that are sent over TCP. If every message you send is one line, you could e.g. try ending messages with a newline, then on the other side, read until you get a newline. If you're sending binary data, the usual approach is to send the length of the data before the actual data so that the recipient knows how much he has to send.
If you really need to craft TCP packets like that, e.g. for an attack, you should write your code in C or so and use low-level interfaces of your operating system that let you craft your own IP packets with arbitrary headers and implement TCP yourself.
Related
I'm trying to write a routine that will poll incoming UDP multicast messages sent to multiple ports on a single multicast group, across all network interfaces.
I can do this using DatagramSocket, but I can't find a way to check if data is available, or to make it non-blocking. All I can do is set a timeout, call receive and wait for an exception if there's nothing there.
Usually, there is at most one port and one network interface with data, so with 4 ports and 4 network interfaces and a timeout of 50ms, it takes ~800ms to read.
If I look at equivalent C# code, there is a Socket.Available property which returns the amount of data ready to be read. If it's zero, I can skip the socket (/port/network interface) and reading is much faster.
Is there a way to do something similar in Java?
I'm new to the whole UDP thing ('cause everyone loves TCP) and need to ask a few questions about Java's implementation.
I need somebody to tell me whether:
The DatagramPackets sent by Java are fragmented automatically due to
network configurations and data size.
The DatagramPackets are rearranged to be in the correct frag sequential order by Java after being fragmented automatically due to network configurations and data size... before the receive() call returns the result.
If fragmented DatagramPackets that're incomplete are dropped or generate Exceptions when dropped. (Some fragments received, others lost)
I'm concerned that Java drops it silently, or data is not arranged correctly... which would mean that I have to implement a pseudo TCP kind of thing to have both the benefits of UDP, as well as the checking of TCP.
UDP is largely implemented in the OS and Java has very little say in the matter.
packets over 576 bytes long can be fragmented;
packets can be lost;
packets can arrive out of order.
There is no way for Java, or you to tell whether these have happened.
What you can do is implement a protocol to detect this. e.g. adding a sequence number, length and checksum to the start of each packet.
which would mean that I have to implement a pseudo TCP kind of thing to have both the benefits of UDP, as well as the checking of TCP.
And now you are starting to understand why "everyone loves TCP" or most people do. UDP has its uses but for most applications TCP is simplest.
The DatagramPackets sent by Java are fragmented automatically due to network configurations and data size.
Yes, but Java has nothing to do with it.
The DatagramPackets are rearranged to be in the correct frag sequential order by Java after being fragmented automatically due to network configurations and data size... before the receive() call returns the result.
Yes, but not by Java, and only if all the fragments arrive. This happens at the IP layer.
If fragmented DatagramPackets that're incomplete are dropped or generate Exceptions when dropped. (Some fragments received, others lost)
They are dropped silently. No exception. Again Java has nothing to do with it. It all happens at the IP layer. If and only if all the fragments arrive, the datagram is reassembled and passed up to the UDP layer. Java is not involved in any way.
I'm concerned that Java drops it silently
Java does nothing. IP drops it silently.
or data is not arranged correctly
A datagram is received either intact and entire or not at all. Again Java has nothing to do with it.
which would mean that I have to implement a pseudo TCP kind of thing to have both the benefits of UDP, as well as the checking of TCP.
Correct. You do.
Sorry for such the long post. I have done lots and lots of research on this topic and keep second guessing myself on which path I should take. Thank you ahead of time for your thoughts and input.
Scenario:
Create a program in Java that sends packets of data every .2-.5 seconds that contain simple "at that moment" or live data to approximately 5-7 clients. (Approximately only 10 bytes of data max per packet). The server can be connected via wifi or ethernet. The clients however are restricted to only using Wifi. The client(s) will not be sending any packets to the server as it will only display the data retrieved from the server.
My Thoughts:
I originally started out creating the server program using TCP as the transport layer. This would use the ServerSocket class within Java. I also made it multithreaded to accept multiple clients.
TCP sounded like a great idea for various reasons.
With TCP, the message will always get sent unless the connection fails.
TCP rearranges the order of packets sent.
TCP has flow control and requires more time to set up when started.
So, perfect! The flow control is crucial for this scenario (since the client wants the most concurrent information), the setup time isn't that big of a deal, and I am pretty much guaranteed that the message will be received. So, its official, I have made my decision to go with TCP....
Except, what happens when TCP gets behind due to packet loss? "The biggest problem with TCP in this scenario is its congestion control algorithm, which treats packet loss as a sign of bandwidth limitations and automatically throttles the sending of packets. On 3G or Wi-Fi networks, this can cause a significant latency."
After seeing the prominent phrase "significant latency", I realized that it would not be good for this scenario since the client needs to see the live data at that moment, not continue receiving data from .8 seconds ago. So, I went back to the drawing board.
After doing more research, I discovered another procedure that involved UDP with Multicasting. This uses a DatagramSocket on the server side and a MulticastSocket on the client side.
UDP with Multicasting also has its advantages:
UDP is connectionless, meaning that the packets are broadcasted to anyone who is listening.
UDP is fast and is great for audio/video (although mine is simple data like a string).
UDP doesn't guarantee delivery - this can be good or bad. It will not get behind and create latency, but there is no guarantee that all client(s) might receive the data at that moment.
Sends one packet that gets passed along.
Since the rate of sending the packets will be rather quick (.2-.5 sec), I am not too concerned about losing packets (unless it is consistently dropping packets and looks to be unresponsive). The main concern I have with UDP above anything is that it doesn't know the order of which the packets were sent. So, for example, lets say I wanted to send live data of the current time. The server sends packets which contain "11:59:03", "11:59:06", "11:59:08", etc. I would not want the data to be presented to the client as "11:59:08", "11:59:03", "11:59:06", etc.
After being presented with all of the information above, here are my questions:
Does TCP "catch up" with itself when having latency issues or does it always stay behind once the latency occurs when receiving packets? If so, is it rather quick to retrieving "live" data again?
How often do the packets of data get out of order with UDP?
And above all:
In your opinion, which one do you think would work best with this scenario?
Thanks for all of the help!
Does TCP "catch up" with itself when having latency issues or does it always stay behind once the latency occurs when receiving packets?
TCP backs its transmission speed off when it encounters congestion, and attempts to recover by increasing it.
If so, is it rather quick to retrieving "live" data again?
I don't know what that sentence means, but normally the full speed is eventually restored.
How often do the packets of data get out of order with UDP?
It depends entirely on the intervening network. There is no single answer to that.
NB That's a pretty ordinary link you're citing.
UDP stands for 'User Datagram Protocol', not 'Universal Datagram Protocol'.
UDP packets do have an inherent send order, but it isn't guaranteed on receive.
'UDP is faster because there is no error-checking for packets' is meaningless. 'Error-checking for packets' implies retransmission, but that only comes into play when there has been packet loss. Comparing lossy UDP speed to lossless TCP speed is meaningless.
'UDP does error checking' is inconsistent with 'UDP is faster because there is no error-checking for packets'.
'TCP requires three packets to set up a socket connection, before any user data can be sent' and 'Once the connection is established data transfer can begin' are incorrect. The client can transmit data along with the third handshake message.
The list of TCP header fields is incomplete and the order is incorrect.
For TCP, 'message is transmitted to segment boundaries' is meaningless, as there are no messages.
'The connection is terminated by closing of all established virtual circuits' is incorrect. There are no virtual circuits. It's a packet-switching network.
Under 'handshake', and several other places, he fails to mention the TCP four-way close protocol.
The sentences 'Unlike TCP, UDP is compatible with packet broadcasts (sending to all on local network) and multicasting (send to all subscribers)' and 'UDP is compatible with packet broadcast' are meaningless (and incidentally lifted from an earlier version of a Wikipedia article). The correct term here is not 'compatible' but supports.
I'm not fond of this practice of citing arbitrary Internet rubbish here and then asking questions based on it.
UDP packets will may get out of order if there are a lot of hops between the server and the client, but more likely than getting out of order is that some will get dropped (again, the more hops the more chance of that). If your main concern with UDP is the order, and if you have control over the client, then you can simply have the client discard any messages with an earlier timestamp than the last one received.
I am using Java Socket API for communication. But sometime I am getting, packet attached in single packet. How can I avoid the same. Is there any method to resolve same in Java NIO or java NIO 2. I am sure that packets are coming separately. But both stored in single buffer.
Please note that here Packet is nothing but logical separation of data. The data is send by third party system. They send one by one. But I am receiving two packet at same time.
This is the way it's supposed to work. TCP uses packets to transfer data, but it's not visible from the high-level socket API : you open a output stream and send as much data as you want. This data is split into packets by the TCP/IP protocol stack. And at the receiving side, you open an input stream and receive the data, without knowing it has been split into packets.
If you want two application-level packets, then design a transfer protocol using separators between your packets, or fixed-size chunks of data, or anything else allwoing to distinguish what is part of a logical packet and what is part of the next one.
I sometimes receive already received packets (I used sniffer and system ACKs them). Now I read all data (until socket timeout) and then send new request, but this is ugly. I was thinking about using sequence numbers but i didn't find it in Socket interface. Any clues?
No you don't. If the receiving TCP stack misses a packet, it will re-request it, but it can't have delivered the original one to you, because it missed it. And if it gets a packet it has already received, it will drop it.
TCP will deliver all the bytes that are sent, in the order they are sent. Nothing else (well, except some edge cases around disconnects).
Something else is going on.
EDIT:
To be clear, I'm talking about the bytes that are delivered to your application through the socket's InputStream. What happens on the wire is largely irrelevant unless you have some horrific network retransmission problem that you're trying to investigate. And if the receiving stack does get a duplicate packet, it will ACK it, because if it didn't then the sender would re-send it... again.
It sounds like you're trying to account for things that TCP already takes care of. It has sequence numbers built in and will take care of any lost data for you, and from the receiving point you should be waiting until you receive all your expected data, rather than reissuing a request. If you don't want to wait for a response to complete before issuing a new request, consider pipe-lining requests with multiple connections.