Can any one please suggest, how to implement/use redis pipeline like behavior in aerospike java client.
Redis is a single-threaded database with a simple request/response protocol. Since every command must be processed one by one, and each request has to have a response back, this can add up to a lot of latency if you have lots of operations to do. Pipelining is a way to send multiple commands at once, have the server process all of them, then get all the results back in a batch.
Aerospike is multi-threaded with its own custom wire protocol that can run multiple commands in parallel over the same connection without any special support. The official drivers handle sending commands as efficiently as possible.
Aerospike does have something called Multiple Operations which means you can send multiple commands that act on the same key as one combined command. The java (and other language) drivers also support async operations which should further increase concurrent performance in your code.
Related
We are evaluating redis clients between jedis and lettuce. One of the feature we are looking is pipelining commands.
Behaviour in Jedis:
We simply call sync on pipeline object to send all the commands to redis. The commands are batched together by client and a single request is made to redis.
How to we achieve the same in lettuce
Do we disable autoflush and call flush() similar to sync in jedis.
In autoflush is pipelining implicit. If so when does lettuce decide to do the flush of commands. Is there any configuration to tune this behaviour
Any help or references regarding is much appreciated.
You can read my answer to another question which has a bit more detail here.
but tl;dr:
The sync interface does not pipeline, but both the async and reactive interfaces do.
Auto-flushing will pipeline, but it will write commands individually to the socket. You will generally perform better if you flush manually because multiple commands are written to the socket at once.
In both cases (auto vs manual flushing) you can send multiple requests before awaiting the results
First off, has anyone done a performance comparison for throughput/latency between a GRPC client-server implementation v/s a websocket+protobuf client-server implementation? Or at least something similary.
In order to reach this goal, I am trying out the example JAVA helloworld grpc client-server and trying to compare the latency of the response with a similar websocket client-server. Currently i am trying this out with both client and server on my local machine.
The websocket client-server has a simple while loop on the server side. For the grpc server i notice that it uses an asynchronous execution model. I suspect it creates a new thread for each client request, resulting in additional processing overheads. For instance, the websocket response latency i am measuring is in the order of 6-7 ms and the grpc example is showing a latency of about 600-700ms, accounting for protobuf overhead.
In order to do a similar comparison for grpc, is there a way to run the grpc server synchronously? I want to be able to eliminate the overhead of the thread creation/dispatch and other such internal overhead introduced by the asynchronous handling.
I do understand that there is a protobuf overhead involved in grpc that is not there in my websocket client-server example. However i can account for that by measuring the overhead introduced by protobuf processing.
Also, if i cannot run the grpc server synchronously, can i at least measure the thread dispatch/asynchronous processing overhead?
I am relatively new to JAVA, so pardon my ignorance.
Benchmarking in Java is easy to get wrong. You need to do many seconds worth of warm-up for multiple levels of JIT to kick in. You also need time for the heap size to level-off. In a simplistic one-shot benchmark, it's easy to see the code that runs last is fastest (independent of what that code is), due to class loading. 600 ms is an insanely large number for gRPC latency; we see around 300 µs median latency on Google Compute Engine between two machines, with TLS. I expect you have no warm-ups, so you are counting the time it takes for Java to load gRPC and are measuring Java using its interpreter with gRPC.
There is not a synchronous version of the gRPC server, and even if there was it would still run with a separate thread by default. grpc-java uses a cached thread pool, so after an initial request gRPC should be able to re-use a thread for calling the service.
The cost of jumping between threads is generally low, although it can add tail latency. In some in-process NOOP benchmarks we see RPC completion in 8 µs using the extra threads and 4 µs without. If you really want though, you can use serverBuilder.directExecutor() to avoid the thread hop. Note that most services would get slower with that option and have really poor tail latency, as service processing can delay I/O.
In order to do a similar comparison for grpc, is there a way to run the grpc server synchronously? I want to be able to eliminate the overhead of the thread creation/dispatch and other such internal overhead introduced by the asynchronous handling.
You can create a synchronous client. Generally the asynchronous is way faster. (Tested in Scala) You can simply use all resources you got in an non-blocking way. I would create a test on how many request from how many clients the server can handle per second. You can than limit the incoming request per client to make sure that your service will not crash. Asynchronous is also better for HTTP 2. HTTP 2 provides Multiplexing.
For a Benchmark I can recommend Metrics. You can expose the metrics via log or http endpoint.
I need to pull data from a lot of clients connecting to a java server through a web socket.
There are a lot of web socket implementations, and I picked vert.x.
I made a simple demo where I listen to text frames of json, parse them with jackson and send response back. Json parser doesn't influence significantly on the throughput.
I am getting overall speed 2.5k per second with 2 or 10 clients.
Then I tried to use buffering and clients don't wait for every single response but send batch of messages (30k - 90k) after a confirmation from a server - speed increased up to 8k per second.
I see that java process has a CPU bottleneck - 1 core is used by 100%.
Mean while nodejs client cpu consumption is only 5%.
Even 1 client causes server to eat almost a whole core.
Do you think it's worth to try other websocket implementations like jetty?
Is there way to scale vert.x with multiple cores?
After I changed the log level from debug to info I have 70k. Debug level causes vert.x print messages for every frame.
It's possible specify number of verticle (thread) instances by e.g. configuring DeploymentOptions http://vertx.io/docs/vertx-core/java/#_specifying_number_of_verticle_instances
You was able to create more than 60k connections on a single machine, so I assume the average time of a connection was less than a second. Is it the case you expect on production? To compare other solutions you can try to run https://github.com/smallnest/C1000K-Servers
Something doesn't sound right. That's very low performance. Sounds like vert.x is not configured properly for parallelization. Are you using only one verticle (thread)?
The Kaazing Gateway is one of the first WS implementations. By default, it uses multiple cores and is further configurable with threads, etc. We have users using it for massive IoT deployments so your issue is not the JVM.
In case you're interested, here's the github repo: https://github.com/kaazing/gateway
Full disclosure: I work for Kaazing
I have a server that needs to stream data to multiple clients as quickly as possible in the Observer pattern.
At least 200 messages need to be sent to each client per second until the client disconnects from the sever, and each message consists of 8 values of several primitive types. Because each message needs to be sent as soon as it is created, messages cannot be combined into one large message. Both the server and the clients reside on the same LAN.
Which technology is more suitable to implement streaming under this situation, RMI or socket?
The overhead of RMI is significant so it would not be suitable. It would be better to create a simple protocol and use sockets to send the data.
Depending on the latency that is acceptable you should configure socket buffer sizes and turn off Nagle algorithm.
I would not use RMI for this, RMI is just there for Remote Method Invocation, i.e. when the client wants to execute a method (i.e. some business logic) on the server side.
Sockets are OK for this, but you might want to consider JMS (Java Messaging Service) for this specific scenario. JMS supports something called a Topic, which is essentially a broadcast to all listeners interested in that topic. It is also generally optimised to be very fast.
You can use something like Apache ActiveMQ to achieve what you want. You also have lots of options such as persistence (in case the queue goes down messages remain in queue), message expiry (in case you want the messages to become outdated if a client does not pick them up) etc.
You can obviously implement all this using normal Sockets and take care of everything yourself, but JMS provides all this for you. You can send text or binary data, or even serialized object (I don't personally recommend the latter).
RMI is a request/response protocol, not a streaming protocol. Use TCP.
A) If I have a java application which starts up and waits for socket connections on a specified port. Is there a way i could maintain a session connection in php after a user validates, i.e a persistent connection?
B) I mean, I'm trying to understand how its done with mysql and php. How does mysql or php know what the last opened connection was so you don't have to do mysql_connect after ?
C) Is there any benefit to opening and closing a connection on each page load or is it better to maintain a persistent connection?
D) If the latter is true in C, can you describe or give an example of how it would be achieved in a php --> Java connection
A) No, there isn't.
B) mysql_pconnect() works because of how the web server and php cooperates. The web server will typically launch a number of child processes which handles request. Each child process can only handle a single request at a time, and concurrency is achieved by sending concurrent requests to different processes.
Each such process has its own instance of PHP, which is reused for each new request. This allows PHP modules to maintain some state between requests. You cannot do this from regular PHP code, it has to be an extension written in C. There's no guarantees about this, though. A process can be killed and relaunched at any time.
Sidenote: Of course, not all web servers uses processes like this. It's rather common to use a threaded approach instead. This doesn't work on PHP, though, since not all extensions are thread safe. Therefore PHP always has to run on a web server that creates child processes to handle requests. This mode (MPM) is called prefork on Apache.
C) As I said, you don't have the choice. On a fast network, the overhead for opening a new connection is quite small, though.