I'm using IgniteDataStreamer with allowOverwrite to load continious data.
Question 1.
From javadoc:
Note that streamer will stream data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the streamer.
Reordering is not acceptable in my case. Will perNodeParallelOperations set to 1 guarantee keeping order of addData calls? There is a number of caches being simultaneously loaded with IgniteDataStreamer, so Ignite server node threads will all be utilized anyway.
Question 2.
My streaming application could hang for a couple of seconds due to GC pause. I want to avoid cache loading pause at that moments and keep high average cache writing speed. Is iy possible to configure IgniteDataStreamer to keep (bounded) queue of incoming batches on server node, that would be consumed while streaming (client) app hangs? See question 1, queue should be consumed sequentially. It's OK to utilize some heap for it.
Question 3.
perNodeBufferSize javadoc:
This setting controls the size of internal per-node buffer before buffered data is sent to remote node
According to javadoc, data transfer is triggered by tryFlush / flush / autoFlush, so how does it correlate with perNodeBufferSize limitation? Would flush be ignored if there is less than perNodeBufferSize messages (I hope no)?
I don't recommend trying to avoid reordering in DataStreamer, but if you absolutely need to do that, you will also need to set data streamer pool size to 1 on server nodes. If it's larger then data is split into stripes and not sent sequentially.
DataStreamer is designed for throughput, not latency. So there's not much you can do here. Increasing perThreadBufferSize, perhaps?
Data transfer is automatically started when perThreadBufferSize is reached for any stripe.
Related
I'm running a vertx java web server to handle large body requests. In order to avoid memory overflow, I'm using the vertx backpressure mechanism with a Pump and implementation of the WriteStream interface which works properly to pause/resume the socket.
Changing the TCP receive buffer size (HttpServerOptions) is a good way to slow down the increase of the virtual memory when the server receives a large request but cannot limit it.
So I need the writeQueueFull method to return true when the vertx input buffer size reaches a given threshold. The problem is that I haven't found a way to monitor this memory amount that vertx uses at runtime.
I could simply look at the JVM memory usage (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) to take the pause/resume decision but it's not very precise. Is there another way?
Thanks
When you perform onNext() on a stream response in GRPC it queues it up for transmission, this allocates on direct buffer memory rather than heap as such java.lang.OutOfMemoryError: Direct buffer memory will not generate a useful heap dump. This can be simulated by creating a
message Chunk {
bytes data = 1
}
And sending multiple small chunks into a stream where the receiving end may not be as quick will cause this to trigger. The proper fix would be to make sure the server does not do anything stupid like send many small chunks, but this still can be a vector of DoS attack that could shut down a service.
My question is in GRPC, is there a setting on the server side to limit the amount and block further onNext until the queue is diminished, with a timeout to cancel the operation when the transfer takes too long? That way it won't shutdown the service but just the GRPC call.
I am thinking the answer would somewhere be in this Github issue though it seems a lot of code for something so fundamental.
The local send buffer size on the server is hard-coded to 32 KB. You can use the ServerCallStreamObserver.isReady() or the onReadyHandler to block to achieve flow control (and also timeout for how long you wait).
I have two different types of server and clients working at the moment and i am trying to decide which one would be better for an MMO server or at least a small MMO like server with at least 100 players at a time.
my first server is using a thread per connection model and sending objects over the socket using ObjectOutputStream.
my second server is using java nio to use only one thread for all the connections and using select to loop through them. this server is also using ObjectOutputStream to send data
for my question, what would be a better approach to an MMO server and if the single thread model is better how would sending an object over the socket channel be affected, would it not be read all the way and not get the full object?
for each object being sent over it just contains for example an int and 2 floats for sending position and player id.
I will relate this question to why MMO use UDP over TCP. The reason being that UDP promises fast delivery whereas TCP promises guaranteed delivery.
A similar analogy can be applied to a single-threaded vs a multi-threaded model.Regardless of which you choose, your overall CPU cycles remain the same i.e. the server can process only so much information per second.
Lets see what happens in each of these scenarios
1.Single-Threaded Model :
In this, your own implementation or the underlying library will end up creating a pipeline where the requests start queuing. If you are at min load, the queue will remain virtually empty and execution will be real-time, however a lot of CPU may be wasted. At max load, there will be a long queue-up and execution will have latency with increasing load, however delivery will be guaranteed and CPU utilization will be optimum. Typically a slow client will slow everybody else down.
Multi-Threaded Model :
In this, depending on how your own implementation or the underlying library implements mutli-threading, parallel execution of requests will start happening. The catch to MT is that it's easy to get fooled. For example, java.util.concurrent.ThreadPoolExecutor doesnt actually do any parallel processing unless you set the queue size to a low value. Once parallel processing starts happening, at min load, your execution will be superfast and CPU utilization will be optimum and game performance will be great. However, at max load your RAM usage will be high and CPU utilization will still be optimum. Typically you'll need to put thread interrupts to avoid a slow client hogging all the threads, which will mean glitchy performance for the slow client. Additionally as you start exhausting your thread pool and resources, threads will either get queued or just get dropped leading to glitchy performance.
In gaming, performance matters more over stability, hence there is no question that you should use MT wherever you can, however tuning your thread parameters to compliment your server resources will decide whether its a boon or a complete bane
I am using the Java library Jedis ontop of a Redis queue which I am using as a producer/consumer queue. It was easy to set up and is working nicely.
Consumer code below
List<String> messages = jedis.blpop(0, redisQueueName);
String message = messages.get(1);
//do some stuff
I'm looking to see if I can speed up performance as I have a large amount of items sitting in the Redis queue waiting to be picked up. I've timed my custom processing code and it does not take too long (20000 nano seconds).
Would best practice be to pull multiple items from Redis at once and process them in a batch? Or am I better looking at tuning the Redis server for better performance?
Yes pulling in batch is indeed the best practice. You will be avoiding network round trip.
One more thing is to trim the queue if it goes beyond certain range as the queue grows rapidly and you want to have the control over the queue size (memory size). Sometimes you may not need to perform each and every entry in the queue instead you may skip few as the queue size grows big.
If you want to retain first entered elements, ie.,
For retaining first 100 elements alone
Ltrim queue 0 100
To retain last 100 elements you can do
Ltrim queue -1 100
Hope this helps
I am looking to get some ideas on how I can solve my failover problem in my Java service.
At a high level, my service receives 3 separate object streams of data from another service, performs some amalgamating logic and then writes to a datastore.
Each object in the stream will have a unique key. Data from the 3 streams can arrive simultaneously, there is no guaranteed ordering.
After the data arrives, it will be stored in some java.util.concurrent collection, such as a BlockingQueue or a ConcurrentHashMap.
The problem is that this service must support failover, and I am not sure how to resolve this if failover were to take place when data is stored in an in-memory object.
One simple idea I have is the following:
Write to a file/elsewhere when receiving an object, and prior to adding to queue
When an object is finally procesed and stored in the datastore
When failover occurs, ensure that same file is copied across and we know which objects we need to receive data for
Performance is a big factor in my service, and as IO is expensive this seems like a crude approach and is quite simplistic.
Therefore, I am wondering if there are any libraries etc out there that can solve this problem easily?
I would use Java Chronicle partly because I wrote it but mostly because ...
it can write and read millions of entries per second to disk in a text or a binary format.
can be shared between processes e.g. active-active clustering with sub-microsecond latency.
doesn't require a system call or flush to push out the data.
the producer is not slowed by the consumer which can be GBs ahead (more than the total memory of the machine)
it can be used in a low heap GC-less and lockless manner.