When you perform onNext() on a stream response in GRPC it queues it up for transmission, this allocates on direct buffer memory rather than heap as such java.lang.OutOfMemoryError: Direct buffer memory will not generate a useful heap dump. This can be simulated by creating a
message Chunk {
bytes data = 1
}
And sending multiple small chunks into a stream where the receiving end may not be as quick will cause this to trigger. The proper fix would be to make sure the server does not do anything stupid like send many small chunks, but this still can be a vector of DoS attack that could shut down a service.
My question is in GRPC, is there a setting on the server side to limit the amount and block further onNext until the queue is diminished, with a timeout to cancel the operation when the transfer takes too long? That way it won't shutdown the service but just the GRPC call.
I am thinking the answer would somewhere be in this Github issue though it seems a lot of code for something so fundamental.
The local send buffer size on the server is hard-coded to 32 KB. You can use the ServerCallStreamObserver.isReady() or the onReadyHandler to block to achieve flow control (and also timeout for how long you wait).
Related
I'm running a vertx java web server to handle large body requests. In order to avoid memory overflow, I'm using the vertx backpressure mechanism with a Pump and implementation of the WriteStream interface which works properly to pause/resume the socket.
Changing the TCP receive buffer size (HttpServerOptions) is a good way to slow down the increase of the virtual memory when the server receives a large request but cannot limit it.
So I need the writeQueueFull method to return true when the vertx input buffer size reaches a given threshold. The problem is that I haven't found a way to monitor this memory amount that vertx uses at runtime.
I could simply look at the JVM memory usage (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) to take the pause/resume decision but it's not very precise. Is there another way?
Thanks
I'm using IgniteDataStreamer with allowOverwrite to load continious data.
Question 1.
From javadoc:
Note that streamer will stream data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the streamer.
Reordering is not acceptable in my case. Will perNodeParallelOperations set to 1 guarantee keeping order of addData calls? There is a number of caches being simultaneously loaded with IgniteDataStreamer, so Ignite server node threads will all be utilized anyway.
Question 2.
My streaming application could hang for a couple of seconds due to GC pause. I want to avoid cache loading pause at that moments and keep high average cache writing speed. Is iy possible to configure IgniteDataStreamer to keep (bounded) queue of incoming batches on server node, that would be consumed while streaming (client) app hangs? See question 1, queue should be consumed sequentially. It's OK to utilize some heap for it.
Question 3.
perNodeBufferSize javadoc:
This setting controls the size of internal per-node buffer before buffered data is sent to remote node
According to javadoc, data transfer is triggered by tryFlush / flush / autoFlush, so how does it correlate with perNodeBufferSize limitation? Would flush be ignored if there is less than perNodeBufferSize messages (I hope no)?
I don't recommend trying to avoid reordering in DataStreamer, but if you absolutely need to do that, you will also need to set data streamer pool size to 1 on server nodes. If it's larger then data is split into stripes and not sent sequentially.
DataStreamer is designed for throughput, not latency. So there's not much you can do here. Increasing perThreadBufferSize, perhaps?
Data transfer is automatically started when perThreadBufferSize is reached for any stripe.
I am converting the details that has to be sent from my C++ function to Java as strings and as a char* which will be sent through socket.
My buffer size is 10 MB. Can I send the 10MB in one shot or should I split and send as chunks of smaller memory?
What is the difference between those two approaches? If I should send as smaller memory what should be the chunk size?
Can I send the 10MB in one shot
Yes.
or should I split and send as chunks of smaller memory?
No.
What is the difference between those two approaches?
The difference is that in case 1 you are letting TCP make all the decisions it is good at, with all the extra knowledge it has that you don't have, about the path MTU, the RTT, the receive window at the peer, ... whereas in case 2 you're trying to do TCP's job for it. Keeping a dog and barking yourself.
If I should send as smaller memory what should be the chunk size?
As big as possible.
When you call the write() function, you provide a buffer and number of bytes you want to write. However it is not guaranteed that the OS will send/write all the bytes that you are willing to write in a single shot. (In case of blocking sockets, the write() call would block until it copies the entire chunk to the TCP buffer. However in case of non-blocking ones, the write() would return and would not block and would write the just the bytes it is able to).
The TCP/IP stack runs in the OS and each OS will have its own implemenation of the stack. This stack would determine the buffer sizes and moreover the TCP/IP would itself take care of all the low level statistics such as MSS, the available receiver window size, which would let TCP run the flow control, congestion control related algorithms.
Therefore it is best that let TCP decide how would it want to send your data. Instead of you breaking the data into chunks, let the TCP stack do it for you.
Just be careful with the thing that always check the number of bytes actually sent which is returned by the write() call.
I need to read the most recent incoming UDP packet, regardless of dropped packets in between reads. Incoming packets are coming in 3x faster than the maximum application processing speed. In an attempt to achieve this, I used setReceiveBufferSize(int size) of Java's DatagramSocket class to set the SO_RCVBUF to be the same size as my expected packet in bytes.
However, there is still a three packet delay before I get the most recent packet (and if the incoming rate is 10x the receive rate, there is a 10 packet delay). This suggests that SO_RCVBUF contains more than just the newest packet.
First, are the units of setReceiveBufferSize(int size) in bytes? It is not explicitly stated in the javadocs. Second, is there a way to disable SO_RCVBUF so that I only receive the most recent incoming packet? For example, zero is an illegal argument to the function, but I could theoretically set the receive buffer size to one.
this looks like an unusual problem ;)
i would recommend to split your application into separate threads:
reciever (minimal work, no parsing/etc)
handles the incoming packets and puts the last read object into an asyncronous variable
processing (from what you wrote, looks like this takes a long time)
reads the object from the asyncronous space, and processes it (don't forget to ignore the previous)
if you need to hack things like SO_RCVBUF, i think you should step a bit closer to the io processing subsystem with C/C++
You've done exactly the wrong thing. Set the receive buffer as large as possible. 512k for example. Setting it low only increases the probability of dropped packets. And either speed up the receiving code or slow down the sending code. There's no point in sending packets that can't be received.
I am writing an application which grabs an XML file from the server and then works with the data inside. My question is, because TCP ensures that all packets arrive and is beyond my control to control how it breaks that data apart, does it make sense to cap the buffer size? If so, I can send the data over in chunks and reassemble them on the client side. Obviously I cannot make an infinite buffer. The XML can get fairly large, up to 256kb and I am bit worried about reserving a buffer of that size. The data is pulled by an Android device but we can assume the device have 1gb of RAM.
The TCP receive buffer size has nothing to do with the size of the data being transferred. Obviously, you can transport gigabytes of data over TCP streams and that doesn't require the buffer to be of the same size. The buffer size generally has to do with performance (both network and processor on the endpoints) and can be small - you probably don't have to change the default settings in most cases.
You don't need to reassemble it at the client side yourself. Just attach an XML parser directly to the socket InputStream.
The default buffers in the network stack are generally tuned to be good, on average. Unless your application is particularly unusual (which it does not sound like), you would be better off not changing the buffer size. The fact that the endpoints are different will also result in tension that prevents easy selection of anything more optimal for both simultaneously.
As suggested, if you use a streaming parser on the receiving side, the buffer size does not really matter. Send the messages as you have them ready to reduce latency caused by batching the entire document.