We Keep Coding

Do I have to handle message fragmentation for Sockets?

I am currently using java.net.Socket to send messages from the client and reading messages from the server. All my messages are fairly short so far, and I have never had any problems.
One of my friends noticed that I was not handling message fragmentation, where the data could come in pieces, and has advised that I should create a buffer to handle this. I insisted that TCP handles this for me, but I'm not 100% sure.
Who is right?
Also, I plan on creating a client in C as well in the future. Do Berkeley sockets handle message fragmentation?
Details: Currently, in Java, the server creates a socket and reads the first byte from the message with InputStream#read(). That first byte determines the length of the entire message, and creates a byte array of the appropriate length, and calls InputStream#read(byte[]) once and assumes that the entire message has been read.

If you are talking about WebSockets,you may be mixing different concepts.
One thing is TCP/IP message fragmentation.
Other thing is how buffering works. You read buffers of data, and you need a framing protocol that tells you when you have a complete "message" (or frame). Basically you:
Read buffer.
Has complete header? No-> Goto 1, Yes-> continue
Read until having all the bytes that the head indicates as message
length.
Has complete message? No-> Goto 3, Yes -> continue
Yield message.
Goto 1.
Other different thing is WebSocket message fragmentation. WebSocket has already a framing protocol and messages can be split in different data frames, and control frames can be interleaved with data frames: https://developer.mozilla.org/en-US/docs/WebSockets/Writing_WebSocket_servers#Message_Fragmentation
If you are writing a WebSocket client or server you have to be ready for this situation.

Expanding on what nos said, TCP will break up large messages into smaller chunks, if the message is large enough. Often, it isn't. Often, the data you write is already split into parts (by you), into meaningful chunks like discrete messages.
The stuff about the reads/writes taking different amounts of calls comes from how the data is written, how it travels over the wire, and how you read it.
If you write 2 bytes 100 times, and then 20 seconds later go to read, it will say there is 200 bytes to be read, which you can read all at once if you want. If you pass a massive 2mb buffer to be written (I dont even know if thats possible), it would take longer to write out, giving more of a chance to the reading program to get different read calls.

Details: Currently, in Java, the server creates a socket and reads the first byte from the message with InputStream#read(). That first byte determines the length of the entire message, and creates a byte array of the appropriate length, and calls InputStream#read(byte[]) once and assumes that the entire message has been read.
That won't work. Have a look at the contract for InputStream.read(byte[]). It isn't obliged to transfer more than one byte. The correct technique is to read the length byte and then use DataInputStream.readFully(), which has the obligation to fill the buffer.

Datagramsocket: how receive(...) handles fragmentation of a packet

I came to know from my Professor that, a datagram packet sent using UDP socket gets fragmented in the lower layers and may arrive as multiple packets at the receiver end. For e.g, if I send a 1000 bytes data in a datagram packet, at the receiving end it might arrive as, say 2 bytes, 500 bytes, 12 bytes, and so on. Therefore, he suggested to do multiple receive(...) to receive the entire 1000 byte packet sent by the sender.
Later when I went through the Java documentation for datagram socket receive(...) and there is a line that reads as follows: "This method blocks until a datagram is received. ..." Does it mean that entire datagram packet is received and don't need to do multiple receive (even though it's the case in theory) when we use Java?
Pls. clarify. If multiple receive(...) for each packet is the only option to get around this problem, pls. give suggestions on how to do this.

Any call to receive() will give you an entire packet - the fragment handling happens in two layers below the socket. The fragmentation and defragmentation happens in the Network/Internet layer (IP), so the socket will never see the fragments but only receive entire and full UDP/TCP packets (only full packets gets sent to the listening port).
So, no, you do not need multiple receive() to get a single packet, but you should be aware that UDP is not reliable so if one fragment gets lost in the Network layer (and in some cases if it arrives out of order), you won't be able to get the packet.
You might also want to check the methods getReceiveBufferSize() and setReceiveBufferSize() if you're having trouble receiving packets - if the buffer size is smaller than the packet size, it's not guaranteed that you can receive the packet.

NETTY ISSUE Client 1024 byte message

i wanted to report this error directly, but did not find any possibility yet from main page netty.io
I have noticed an error while sending data to channel. It happens not always, in 10-20% of cases, but it happens.
Following,
as example, if I get first connection with message of 1024 byte data, everything is fine till now, than I create socket to forwarded address doing it with HexDumpProxyInboundHandler
and here is fine everything, except one thing, I have created an listener on forwarded address with traffic logging, where i get the messages which were sent by Netty. I would expect the data of 1024 bytes on it, but it happens not always, not also in 100% of cases.
sometimes...
exactly sometimes the nightmare begins here,
if i get on same channel next message after 1024 bytes message, the data gets written in following possible forms:
3.1 either the first message and second message are merged and the data that i get on my port listener is correct, 1024 + 72(as example) and in correct byte order too (but in merged form, what is already not correct for me)
3.2 or the first message and second message are merged too, but with one little difference, with different order, 72(as example) + 1024 bytes, even if data was correctly received by server socket, and in correct order.. the sending order was incorrect also.
3.3 or finally the first message of 1024 gets send as is following by second message which gets send as is too, so everything is fine here too, and that is the correct and expected behavior..
Also, the error happens not always, but it happens, and always, if it happens, it happens only if by first connection the first message was 1024 byte long and the second message was sent immediately after first message without the received data before.
Now the question to community, is that possible to switch off such strange buffering behavior in Netty? So that all messages received on server socket are sent exactly in same way to the client socket channel without merging the data.
Thank you in advance!

This "strange" behavior has nothing todo with netty. Its up to the network layer how much bytes get transfered at once, so its really expected to see this. if you need to have all 1024 bytes you will need t buffer them until you receive enough.

Ok, after long night I have finally solved my problem. It seems, the Netty project is still buggy in this way and will accept incoming messages for sending in the wrong order.
So, what I do, I fill the buffer with incoming messages until the remote connection by client gets opened, so I send than the full correct buffer instead to let things do by Netty.

Java Socket issue, buffer receives two packets at same time.

I am using Java Socket API for communication. But sometime I am getting, packet attached in single packet. How can I avoid the same. Is there any method to resolve same in Java NIO or java NIO 2. I am sure that packets are coming separately. But both stored in single buffer.
Please note that here Packet is nothing but logical separation of data. The data is send by third party system. They send one by one. But I am receiving two packet at same time.

This is the way it's supposed to work. TCP uses packets to transfer data, but it's not visible from the high-level socket API : you open a output stream and send as much data as you want. This data is split into packets by the TCP/IP protocol stack. And at the receiving side, you open an input stream and receive the data, without knowing it has been split into packets.
If you want two application-level packets, then design a transfer protocol using separators between your packets, or fixed-size chunks of data, or anything else allwoing to distinguish what is part of a logical packet and what is part of the next one.

What is a TCP window update?

I'm making my own custom server software for a game in Java (the game and original server software were written with Java). There isn't any protocol documentation available, so I am having to read the packets with Wireshark.
While a client is connecting the server sends it the level file in Gzip format. At about 94 packets into sending the level, my server crashes the client with an ArrayIndexOutOfBoundsException. According to the capture file from the original server, it sends a TCP Window Update at about that point. What is a TCP Window Update, and how would I send one using a SocketChannel?

TCP windows are used for flow control between the peers on a connection. With each ACK packet, a host will send a "window size" field. This field says how many bytes of data that host can receive before it's full. The sender is not supposed to send more than that amount of data.
The window might get full if the client isn't receiving data fast enough. In other words, the TCP buffers can fill up while the application is off doing something other than reading from its socket. When that happens, the client would send an ACK packet with the "window full" bit set. At that point, the server is supposed to stop sending data. Any packets sent to a machine with a full window will not be acknowledged. (This will cause a badly behaved sender to retransmit. A well-behaved sender will just buffer the outgoing data. If the buffer on the sending side fills up too, then the sending app will block when it tries to write more data to the socket!)
This is a TCP stall. It can happen for a lot of reasons, but ultimately it just means the sender is transmitting faster than the receiver is reading.
Once the app on the receiving end gets back around to reading from the socket, it will drain some of the buffered data, which frees up some space. The receiver will then send a "window update" packet to tell the sender how much data it can transmit. The sender starts transmitting its buffered data and traffic should flow normally.
Of course, you can get repeated stalls if the receiver is consistently slow.
I've worded this as if the sender and receiver are different, but in reality, both peers are exchanging window updates with every ACK packet, and either side can have its window fill up.
The overall message is that you don't need to send window update packets directly. It would actually be a bad idea to spoof one up.
Regarding the exception you're seeing... it's not likely to be either caused or prevented by the window update packet. However, if the client is not reading fast enough, you might be losing data. In your server, you should check the return value from your Socket.write() calls. It could be less than the number of bytes you're trying to write. This happens if the sender's transmit buffer gets full, which can happen during a TCP stall. You might be losing bytes.
For example, if you're trying to write 8192 bytes with each call to write, but one of the calls returns 5691, then you need to send the remaining 2501 bytes on the next call. Otherwise, the client won't see the remainder of that 8K block and your file will be shorter on the client side than on the server side.

This happens really deep in the TCP/IP stack; in your application (server and client) you don't have to worry about TCP windows. The error must be something else.

TCP WindowUpdate - This indicates that the segment was a pure WindowUpdate segment. A WindowUpdate occurs when the application on the receiving side has consumed already received data from the RX buffer causing the TCP layer to send a WindowUpdate to the other side to indicate that there is now more space available in the buffer. Typically seen after a TCP ZeroWindow condition has occurred. Once the application on the receiver retrieves data from the TCP buffer, thereby freeing up space, the receiver should notify the sender that the TCP ZeroWindow condition no longer exists by sending a TCP WindowUpdate that advertises the current window size.
https://wiki.wireshark.org/TCP_Analyze_Sequence_Numbers

A TCP Window Update has to do with communicating the available buffer size between the sender and the receiver. An ArrayIndexOutOfBoundsException is not the likely cause of this. Most likely is that the code is expecting some kind of data that it is not getting (quite possibly well before this point that it is only now referencing). Without seeing the code and the stack trace, it is really hard to say anything more.

You can dive into this web site http://www.tcpipguide.com/free/index.htm for lots of information on TCP/IP.

Do you get any details with the exception?
It is not likely related to the TCP Window Update packet
(have you seen it repeat exactly for multiple instances?)
More likely related to your processing code that works on the received data.

This is normally just a trigger, not the cause of your problem.
For example, if you use NIO selector, a window update may trigger the wake up of a writing channel. That in turn triggers the faulty logic in your code.
Get a stacktrace and it will show you the root cause.