I am using grpc-streaming in java. I have a long-lasting open stream where the client and server communicate simultaneously. When I call onNext to send a message, grpc buffers the message internally and will send it on the wire async'ly. Now, if the stream is lost in the middle of sending data, onError is called. I wonder what are the right practices:
to find out which messages were sent successfully
how to retry unsent messages
Currently, I am thinking of implementing an "ack" mechanism in the application layer where for every x items received, the receiver sends back an ack message. Then in order to implement retries, I need to buffer items on the sender side and only remove them from the buffer when the ack is received. Also, on the receiver side, I need to implement a mechanism to ignore duplicate items received.
Example:
Suppose we send an ack for every 100 items sent. We receive ack on batch 3 (200-300) and then we receive an error while sending items 300-400. we try again to send items 300-400 but the client has successfully received 300-330 and it is going to receive them again. so, the client needs to ignore the first 30 items.
It is possible to implement this in the application layer. However, I am wondering if there are better practices/frameworks out there that solve this problem.
The term often used is guaranteed delivery to describe delivery data from one place to another without loss.
Your use case is similar to trying to provide guaranteed delivery over best effort delivery transport layers like UDP. The usual approach is to acknowledge every packet, although you could devise a scheme to check at a higher level as you suggest.
You also usually want to use some form of sliding window which means you don't have to wait for the previous ack before sending the next packet - this helps avoid delays.
There is a very good overview of this approach on UDP in this answer: https://stackoverflow.com/a/15630015/334402
For your case, you will receive a response for your RPC calls which will effectively be the ack - using a sliding window would allow you make th next call before you have received the ack from the previous one.
Your duplicate delivery example is also quite common - one common way to avoid double counting or getting confused is to have packet numbers and simply discard any duplicated packets.
Related
CometD 3.1.4 java implementation uses a buffer to keep track of the incoming message and this buffer has a default size of 2MB. When restarting from a disconnection, spikes can occur and you can exceed the limit.
What behaviour does the library have? Bytes are lost and if subsequent notifications from the server side are received they can be processed ?
The client-side buffering limit can be configured as explained here.
However, I recommend that you review your logic, as sending MiBs of messages to the client is typically not recommended, as the client may take a long while to process them all.
Also, if the messages are few but with large data, you may want to setup things so that the client gets a CometD message with a download URI to be downloaded on the side, rather than sending the data in the CometD messages.
Having said that, you can write a server-side extension to, for example, discard old messages so that you don't send MiBs of data in case of reconnections.
The message acknowledgment extension guarantees server-to-client delivery, so -- provided you don't exceed the client-side receive buffer -- you can guarantee that queued messages are delivered to clients.
You may need a combination of things that is specific to your application.
You may need a server-side listener to control the size of the message queue, the acknowledgment extension to guarantee delivery, and maybe a larger client-side buffer.
This is not done by default by CometD because everybody wants a different solution: some want to fail the session, some want to discard all messages, some want to keep only the last N, etc.
CometD provides you with the hooks you need to implement your logic.
I want 100 messages to be delivered together to a consumer through activemq, but at the same time producer will be producing messages one at a time.
Reason I want this is because I don't want to handle the overhead of processing each message individually on delivery, instead we want to do bulk processing on delivery.
Is it possible to achieve this through ActiveMQ or should i write my own modifications to achieve this.
ActiveMQ is a JMS 1.1 client / broker implementation therefore there is no API to deliver messages in bulk, the async listener dispatches them one at a time. The client does prefetch more than one message though so the overhead of processing them using async listeners is quite low.
You could achieve your goal by placing every message into a buffer and only doing your processing when the buffer contains N messages. To make it work, you'd want to use an acknowledgement mode such as CLIENT_ACKNOWLEDGE that allows you to not acknowledge the messages that are sitting in the buffer until they are processed; that way if your client crashed with some messages in its memory, they would be re-delivered when the client comes back up.
i'm working on an Java client-server application.
The client send a message sequence (the messages can be different types, i,ve got header), and listens for the replies. I've got 2 thread, one for the transmission and one for the receipts.
So i need to handle the replies, in case of errors or in case of the replies doesn't arrive, for example i can try to send the message another time.
My question is.. is there any java patterns that can helps me?
i would like to handle the send and the relative repliy like a single transaction, but note that i don't need to have a synchronous communication. I send all the message in the sequence in the TX thread and wait for the replies on the RX thread.
I've thought to the mediator Pattern, but i don't know if it is the right way.
Thanks
If the question is purely about transmission protocol I would take a look at NAK. http://en.wikipedia.org/wiki/NAK_(protocol_message)
I have implemented a protocol I made up called JCast that sends files over multicast. The files are broken down into small fragments that are numbered. The receiving clients then respond back with any missing fragments that it did not get (these are the NAK's). The server then would resend only the NAK'd fragments.
EDIT: The benefit of NAK over ACK is that the server can send all the packets it needs to without having to wait for ACK's. Since networks are very much improved nowadays, most of the packets would arrive. The few that do not arrive would then just be resent.
I sometimes receive already received packets (I used sniffer and system ACKs them). Now I read all data (until socket timeout) and then send new request, but this is ugly. I was thinking about using sequence numbers but i didn't find it in Socket interface. Any clues?
No you don't. If the receiving TCP stack misses a packet, it will re-request it, but it can't have delivered the original one to you, because it missed it. And if it gets a packet it has already received, it will drop it.
TCP will deliver all the bytes that are sent, in the order they are sent. Nothing else (well, except some edge cases around disconnects).
Something else is going on.
EDIT:
To be clear, I'm talking about the bytes that are delivered to your application through the socket's InputStream. What happens on the wire is largely irrelevant unless you have some horrific network retransmission problem that you're trying to investigate. And if the receiving stack does get a duplicate packet, it will ACK it, because if it didn't then the sender would re-send it... again.
It sounds like you're trying to account for things that TCP already takes care of. It has sequence numbers built in and will take care of any lost data for you, and from the receiving point you should be waiting until you receive all your expected data, rather than reissuing a request. If you don't want to wait for a response to complete before issuing a new request, consider pipe-lining requests with multiple connections.
We have a custom messaging system written in Java, and I want to implement a basic batching/compression feature that basically under heavy load it will aggregate a bunch of push responses into a single push response.
Essentially:
if we detect 3 messages were sent in the past second then start batching responses and schedule a timer to fire in 5 seconds
The timer will aggregate all the message responses received in the next 5 seconds into a single message
I'm sure this has been implemented before I'm just looking for the best example of it in Java. I'm not looking for a full blown messaging layer, just the basic detect messages per second and schedule some task (obviously I can easily write this myself I just want to compare it with any existing algorithms to make sure I'm not missing any edge cases or that I've simplified the problem as much as possible).
Are there any good open source examples of building a basic QoS batching/throttling/compression implementations?
we are using a very similar mechanism for high load.
it will work as you described it
* Aggregate messages over a given interval
* Send a List instead of a single message after that.
* Start aggregating again.
You should watch out for the following pitfalls:
* If you are using a transacted messaging system like JMS you can get into trouble because your implementation will not be able to send inside the JMS transaction so it will keep aggregating. Depending on the size of your data structure to hold the messages this can run out of space. If you are have very long transactions sending many messages this can pose a problem.
* Sending a message in such a way will happen asynchronous because a different thread will be sending the message and the thread calling the send() method will only put it in the data structure.
* Sticking to the JMS example you should keep in mind that they way messages are consumed is also changed by this approach. Because you will receive the list of messages from JMS as a single message. So once you commit this single JMS message you commited the entire list of messages. You should check if this i a problem to your requirements.