I am writing an application which grabs an XML file from the server and then works with the data inside. My question is, because TCP ensures that all packets arrive and is beyond my control to control how it breaks that data apart, does it make sense to cap the buffer size? If so, I can send the data over in chunks and reassemble them on the client side. Obviously I cannot make an infinite buffer. The XML can get fairly large, up to 256kb and I am bit worried about reserving a buffer of that size. The data is pulled by an Android device but we can assume the device have 1gb of RAM.
The TCP receive buffer size has nothing to do with the size of the data being transferred. Obviously, you can transport gigabytes of data over TCP streams and that doesn't require the buffer to be of the same size. The buffer size generally has to do with performance (both network and processor on the endpoints) and can be small - you probably don't have to change the default settings in most cases.
You don't need to reassemble it at the client side yourself. Just attach an XML parser directly to the socket InputStream.
The default buffers in the network stack are generally tuned to be good, on average. Unless your application is particularly unusual (which it does not sound like), you would be better off not changing the buffer size. The fact that the endpoints are different will also result in tension that prevents easy selection of anything more optimal for both simultaneously.
As suggested, if you use a streaming parser on the receiving side, the buffer size does not really matter. Send the messages as you have them ready to reduce latency caused by batching the entire document.
Related
When you perform onNext() on a stream response in GRPC it queues it up for transmission, this allocates on direct buffer memory rather than heap as such java.lang.OutOfMemoryError: Direct buffer memory will not generate a useful heap dump. This can be simulated by creating a
message Chunk {
bytes data = 1
}
And sending multiple small chunks into a stream where the receiving end may not be as quick will cause this to trigger. The proper fix would be to make sure the server does not do anything stupid like send many small chunks, but this still can be a vector of DoS attack that could shut down a service.
My question is in GRPC, is there a setting on the server side to limit the amount and block further onNext until the queue is diminished, with a timeout to cancel the operation when the transfer takes too long? That way it won't shutdown the service but just the GRPC call.
I am thinking the answer would somewhere be in this Github issue though it seems a lot of code for something so fundamental.
The local send buffer size on the server is hard-coded to 32 KB. You can use the ServerCallStreamObserver.isReady() or the onReadyHandler to block to achieve flow control (and also timeout for how long you wait).
Currently I am using this code on both Server and Client Side. Client is an android device.
BufferedOutputStream os = new BufferedOutputStream(socket.getOutputStream(),10000000);
BufferedInputStream sin = new BufferedInputStream(socket.getInputStream(),10000000);
os.write("10000000\n".getBytes());
os.flush();
for (int i =0;i<10000000;i++){
os.write((sampleRead[i]+" ").getBytes());
}
os.flush();
The problem is that this code takes about 80 secs to transfer data from android client to server while it takes only 8 seconds to transfer the data back from server to client. The code is same on both sides and buffer is also same. I also tried with different buffer sizes but the problem is with this segment
for (int i =0;i<10000000;i++){
os.write((sampleRead[i]+" ").getBytes());
}
The buffering takes most of the time while the actual transfer takes only about 6-7 seconds on a 150mbps hotspot connection. What could be the problem and how to solve it?
First of all, as a commenter has already noted, using a monstrously large buffer is likely to be counter productive. Once your stream buffer is bigger than the size of a network packet, app-side buffering loses its effectiveness. (The data in your "big" buffer needs to be split packet-sized chunks by the TCP/IP stack before it goes onto the network.) Indeed, if the app-side buffer is really large, you may find that your data gets stuck in the buffer for a long time waiting for the buffer to fill ... while the network is effectively idle.
(The Buffered... readers, writers and streams are primarily designed to avoid lots of syscalls that transfer tiny amounts of data. Above 10K or so, the buffering doesn't performance help much.)
The other thing to now is that in a lot of OS environments, the network throughput is actually limited by virtualization and default network stack tuning parameters. To get a better throughput, you may need to tune at the OS level.
Finally, if your network path is going over a network path that is congested, has a high end-to-end latency or links with constrained data rate, then you are unlikely to get fast data transfers no matter how you tune things.
(Compression might help ... if you can afford the CPU overhead at both ends ... but some data links already do compression transparently.)
You could compress the data transfer, it will save a lot of memory and well to transfer a compress stream of data is cheaper... For that you need to implement compress logic in client side and decompress logic in server side, see GZIPInputStream... And try reducing the buffer size is huge for a mobile device...
I am converting the details that has to be sent from my C++ function to Java as strings and as a char* which will be sent through socket.
My buffer size is 10 MB. Can I send the 10MB in one shot or should I split and send as chunks of smaller memory?
What is the difference between those two approaches? If I should send as smaller memory what should be the chunk size?
Can I send the 10MB in one shot
Yes.
or should I split and send as chunks of smaller memory?
No.
What is the difference between those two approaches?
The difference is that in case 1 you are letting TCP make all the decisions it is good at, with all the extra knowledge it has that you don't have, about the path MTU, the RTT, the receive window at the peer, ... whereas in case 2 you're trying to do TCP's job for it. Keeping a dog and barking yourself.
If I should send as smaller memory what should be the chunk size?
As big as possible.
When you call the write() function, you provide a buffer and number of bytes you want to write. However it is not guaranteed that the OS will send/write all the bytes that you are willing to write in a single shot. (In case of blocking sockets, the write() call would block until it copies the entire chunk to the TCP buffer. However in case of non-blocking ones, the write() would return and would not block and would write the just the bytes it is able to).
The TCP/IP stack runs in the OS and each OS will have its own implemenation of the stack. This stack would determine the buffer sizes and moreover the TCP/IP would itself take care of all the low level statistics such as MSS, the available receiver window size, which would let TCP run the flow control, congestion control related algorithms.
Therefore it is best that let TCP decide how would it want to send your data. Instead of you breaking the data into chunks, let the TCP stack do it for you.
Just be careful with the thing that always check the number of bytes actually sent which is returned by the write() call.
At application level, say using java, how much do I have to worry about the actual TCP packet size? So, for example, I am trying to write an application that should send data over TCP socket's outputstream, do I have to always keep into account the size of the data written to the stream? Since java sockets are streaming sockets, I havent actually considered the size of data units, but the TSO (TCP Segmentation offload) is "turned on" for the OS/NIC, then I can write a 64KB data slice or MSS to the outputstream and thus try to save the precious CPU time of slicing the data to less than 1500 bytes (< MTU). How effective could my programming be, in terms of being able to take care of this dynamically? I know we can get NetworkInterface.getMTU() to determine OS/NIC MTU size, but not sure how it can help that.
So, I can say that overall, I am a bit confused on how to maximize my throughput of byte writing to the outputstream.
how much do I have to worry about the actual TCP packet size?
Almost never. You can setNoTcpDelay(true); but this rarely makes a difference.
So, for example, I am trying to write an application that should send data over TCP socket's outputstream, do I have to always keep into account the size of the data written to the stream?
I doubt it. If you have a 1 Gb connection or slower, you will have trouble writing a program so inefficient it can't use this bandwidth.
Since java sockets are streaming sockets, I havent actually considered the size of data units, but the TSO (TCP Segmentation offload) is "turned on" for the OS/NIC, then I can write a 64KB data slice or MSS to the outputstream and thus try to save the precious CPU time of slicing the data to less than 1500 bytes (< MTU).
I don't see how give most decent network adapter support TCP offloading.
How effective could my programming be, in terms of being able to take care of this dynamically?
Java doesn't support it in any case.
I know we can get NetworkInterface.getMTU() to determine OS/NIC MTU size, but not sure how it can help that.
Me neither.
So, I can say that overall, I am a bit confused on how to maximize my throughput of byte writing to the outputstream.
The most significant change you can make in Java is to use NIO. I suggest blocking NIO as this is the simplest change from NIO. If you use direct ByteBuffers this can save redundant memory copies from Java to native memory.
Do you know you have a problem using the maximum bandwidth of your network? If you haven't measured this is the cause of your problem, it's just a guess.
TCP buffers, paces, decides segment sizes etc behind the scenes for you. There is nothing you can do to help except write as much as possible as fast as possible, and use a large socket send buffer at the sender and a large socket receive buffer at the receiver.
Following this thread.
Streaming large files in a java servlet.
Is it possible to find the total internet bandwidth available in current machine thru java?
what i am trying to do is while streaming large files thru servlet, based on the number of parallel request and the total band width i am trying to reduce the BUFFER_SIZE of the stream for each request. make sense?
Is there any pure java way? (without JNI)
Maybe you can time how long the app need to send one package (the buffer). And if that is larger than x milliseconds, then make your buffer smaller. You can use other values for the original bufferSize and if (stop - start > 700).
This is based on the thread you noticed:
ServletOutputStream out = response.getOutputStream();
InputStream in = [ code to get source input stream ];
String mimeType = [ code to get mimetype of data to be served ];
int bufferSize = 1024 * 4;
byte[] bytes = new byte[bufferSize];
int bytesRead;
response.setContentType(mimeType);
while ((bytesRead = in.read(bytes)) != -1) {
long start = System.currentTimeMillis();
out.write(bytes, 0, bytesRead);
long stop = System.currentTimeMillis();
if (stop - start > 700)
{
bufferSize /= 2;
bytes = new byte[bufferSize];
}
}
// do the following in a finally block:
in.close();
out.close();
The only way to find available bandwidth is to monitor / measure it. On windows you have access to Net.exe and can get the throughput on each NIC.
If you're serving the content through a servlet, then you could calculate how fast each servlet output stream is going. Collect that data for all streams for a user/session, and you could determine at least what the current bandwidth usage is.
A possible way to calculate the rate could be instead of writing the large files through the servlet output stream, write to a new FilterOutputStream that would keep track of your download rates.
The concept of "total internet bandwidth available in current machine" is really hard to define. However, tweaking the local buffer size will not affect how much data you can push through to an individual client.
The rate at which a given client can take data from your server will vary with the client, and with time. For any given connection, you might be limited by your local upstream connection to the Internet (e.g., server on DSL) or you might be limited somewhere in the core (unlikely) or the remote end (e.g., server in a data center, client on a dialup line). When you have many connections, each individual connection may have a different bottleneck. Measuring this available bandwidth is a hard problem; see for example this list of research and tools on the subject.
In general, TCP will handle using all the available bandwidth fairly for any given connection (though sometimes it may react to changes in available bandwidth slower than you like). If the client can't handle more data, the write call will block.
You should only need to tweak the buffersize in the linked question if you find that you are seeing low bandwidth and the cause of that is insufficient data buffered to write to the network. Another reason you might tweak the buffer size is if you have so many active connections that you are running low on memory.
In any case, the real answer may be to not buffer at all but instead put your static files on a separate server and use something like thttpd to serve them (using a system call like sendfile) instead of a servlet. This helps ensure that the bottleneck is not on your server, but somewhere out in the Internet, beyond your control.
EDIT: Re-reading this, it's a little muddled because it's late here. Basically, you shouldn't have to do this from scratch; use one of the existing highly scalable java servers, since they'll do it better and easier.
You're not going to like this, but it actually doesn't make sense, and here's why:
Total bandwidth is independent of the number of connections (though there is some small overhead), so messing with buffer sizes won't help much
Your chunks of data are being broken into variable-sized packets anyway. Your network card and protocol will deal with this better than your servlet can
Resizing buffers regularly is expensive -- far better to re-use constant buffers from a fixed-size pool and have all connections queue up for I/O rights
There are a billion and a half libraries that assist with this sort of server
Were this me, I would start looking at multiplexed I/O using NIO. You can almost certainly find a library to do this for you. The IBM article here may be a useful starting point.
I think the smart money gives you one network I/O thread, and one disk I/O thread, with multiplexing. Each connection requests a buffer from a pool, fills it with data (from a shared network or disk Stream or Channel), processes it, then returns the buffer to the pool for re-use. No re-sizing of buffers, just a bit of a wait for each chunk of data. If you want latency to stay short, then limit how many transfers can be active at a time, and queue up the others.