We Keep Coding

Jetty Websockets - Correctly sending async messages when handling unreliable connections

I'm using Jetty 9.3.5 and I would like to know what is the proper way to handle unreliable connections when sending websocket messages, specifically: I noticed cases when a websocket connection does not close normally so, even though the client side is down, it takes a lot of time until onClose() is triggered on the server (for ex. a user closes the laptop lid and puts it in standby - it can take 1-2 hours until the close event is received on the server side).
Thus, because the client is still registered, the server keeps sending messages that begin to build up. This becomes an issue when sending a large number of messages.
I've tested sending byte messages with:
Session.getRemote().sendBytes(ByteBuffer, WriteCallback)
Session.getRemote().sendBytesByFuture(ByteBuffer);
To simulate the connection down on one side (ie. user puts laptop in standby), on Linux, I assigned an IP address to eth0 interface, started sending the messages and then brought it down:
ifconfig eth0 192.168.1.1
ifconfig eth0 up
--- start sending messages (simple incremented numbers) and connect using Chrome browser and print them ---
ifconfig eth0 down
This way: the messages were still being sent by Jetty, the Chrome client did not receive them, the onCllose or onError was not triggered on server-side
My questions regarding Jetty are:
Is there a way to clear queued messages that were not delivered?
I've tried, but with no luck:
Session.getRemote().flush();
Can a max number of queued messages be set?
I've tried:
WebSocketServletFactory.getPolicy().setMaxBinaryMessageBufferSize(1)
Can I detect if the client does not receive the message? (or if the connection is in abnormal state let's say)
I've tried:
session.getRemote().sendBytes(bb, new WriteCallback() {
#Override
public void writeSuccess() {
//print success }
#Override
public void writeFailed(Throwable arg0) {
//print fail
}
});
But this prints success even though the messages are not received.
I also tried to use, but couldn't find a solution:
factory.getPolicy().setIdleTimeout(...);
factory.getPolicy().setAsyncWriteTimeout(3000);
sendPing()
Thanks in advance!

Unfortunately, the WebSocket protocol, being a message passing protocol isn't really designed for this level of nuance between messages.
The first message MUST complete before you can even think of sending the next message. So if you have a message in process, then there is no way to safely cancel that message.
At best, an API could exist to truncate that message with a CONTINUATION / empty payload / fin=true.
But even then the remote endpoint wouldn't know that you canceled the message, it would just see a partial message.
Detecting connectivity issues is best handled with either OS level events (like Android's Connectivity intents), or via periodic websocket PING (which inserts itself in front of the line for outgoing websocket frames.
However, even with PING, if your outgoing websocket frame is in-progress, even the PING cannot be sent until that websocket frame is done sending.
RemoteEndpoint.flush() will attempt to flush any pending messages (and frames), not clear out pending messages (or frames).
As for detecting if client got the message, you'll need to implement some sort of message ACK into your own layer to verify that, the protocol has no such concept. (Some libs/apis built on top of websocket have implemented message ACK in that layer. The cometd message ack extension comes to mind as a real world example)
What sort of situation are you attempting to solve for?
Perhaps using the RemoteEndpoint.sendPartialString(String, boolean) or RemoteEndpoint.sendPartialBytes(ByteBuffer, boolean) to send smaller frames of the whole message could be useful to you. However, the other side might not have an API that can read those partial frames (eg: Javascript in a browser).

Incoming messages are treated as outgoing in Atmosphere

I have a problem concerning receiving messages (I use #ManagedService). I use the same connection to send and receive messages between browser and my Java program. I can see that all the messages pass through ManagedAtmosphereHandler.message(AtmosphereResource resource, Object o) method.
If it is an incoming message, Atmosphere iterates through all methods marked #Message. Then it tries to find a decoder and eventually invokes correct method.
For outgoing messages, Atmosphere retrieves invoked method. It does so by getting localAttribute named "ManagedAtmosphereHandler" (name of current class). It is present only for outgoing messages. Then the message is encoded and send to browser.
The problem is, sometimes invokedMethod is set for incoming messages. It results in treating my incoming messages as outgoing. Does anybody know why it happens? My outgoing messages are scheduled and I suppose that's the reason why it happens, but I'm not sure. When are these localAttributes set and what are they for?

I updated Atmosphere 2.3.0-RC6 to 2.3.0 and it works like a charm now.

Parse byteBuffer in Websphere MQ Exit

I'm trying to build a custom mq exit to archive messages that hit a queue. I have the following code.
class MyMqExits implements WMQSendExit, WMQReceiveExit{
#Override
public ByteBuffer channelReceiveExit(MQCXP arg0, MQCD arg1, ByteBuffer arg2) {
// TODO Auto-generated method stub
if ( arg2){
def _bytes = arg2.array()
def results = new String(_bytes)
println results;
}
return arg2;
}
...
The content of the message (header/body) is in the byte buffer, along with some unreadable binary information. How can I parse the message (including the body and the queue name) from arg2? We've gone through IBM's documentation, but haven't found an object or anything that makes this easy.

Assuming the following two points:
1) Your sender application has not hard coded the queue name where it puts messages. So you can change the application configuration to send messages to a different object.
2) MessageId of the archived message is not important, only message body is important.
Then one alternative I can think of is to create an Alias queue that resolves to a Topic and use two subscribers to receive messages.
1) Subscriber 1: An administratively defined durable subscriber with a queue provided to receive messages. Provide the same queue name from which your existing consumer application is receiving messages.
2) Subscriber 2: Another administratively defined durable subscriber with queue provided. You can write a simple java application to get messages from this queue and archive.
3) Both subscribers subscribe to the same topic.
Here are steps:
// Create a topic
define topic(ANY.TOPIC) TOPICSTR('/ANY_TOPIC')
// Create an alias queue that points to above created topic
define qalias(QA.APP) target(ANY.TOPIC) targtype(TOPIC)
// Create a queue for your application that does business logic. If one is available already then no need to create.
define ql(Q.BUSLOGIC)
// Create a durable subscription with destination queue as created in previous step.
define sub(SB.BUSLOGIC) topicstr('/ANY_TOPIC') dest(Q.BUSLOGIC)
// Create a queue for application that archives messages.
define ql(Q.ARCHIVE)
// Create another subscription with destination queue as created in previous step.
define sub(SB.ARCHIVE) topicstr('/ANY_TOPIC') dest(Q.ARCHIVE)
Write a simple MQ Java/JMS application to get messages from Q.ARCHIVE and archive messages.

A receive exit is not going to give you the whole message. Send and receive exits operate on the transmission buffers sent/received by channels. These will contain various protocol flows which are not documented because the protocol is not public, and part of those protocol flows will be chunks of the messages broken down to fit into 32Kb chunks.
You don't give enough information in your question for me to know what type of channel you are using, but I'm guessing it's on the client side since you are writing it in Java and that is the only environment where that is applicable.
Writing the exit at the client side, you'll need to be careful you deal with the cases where the message is not successfully put to the target queue, and you'll need to manage syncpoints etc.
If you were using QMgr-QMgr channels, you should use a message exit to capture the MQXR_MSG invocations where the whole message is given to you. If you put any further messages in a channel message exit, the messages you put are included in the channel's Syncpoint and so committed if the original messages were committed.
Since you are using client-QMgr channels, you could look at an API Exit on the QMgr end (currently client side API Exits are only supported for C clients) and catch all the MQPUT calls. This exit would also give you the MQPUT return codes so you could code your exit to look out for, and deal with failed puts.
Of course, writing an exit is a complicated task, so it may be worth finding out if there are any pre-written tools that could do this for you instead of starting from scratch.

I fully agree with Morag & Shashi, wrong approach. There is an open source project called Message Multiplexer (MMX) that will get a message from a queue and output it to one or more queues. Context information is maintained across the message put(s). For more info on MMX go to: http://www.capitalware.com/mmx_overview.html
If you cannot change the source or target queues to insert MMX into the mix then an API Exit may do the trick. Here is a blog posting about message replication via an API Exit: http://www.capitalware.com/rl_blog/?p=3304

This is quite an old question but it's worth replying with an update that's relevant to MQ 9.2.3 or later. There is a new feature called Streaming Queues (see https://www.ibm.com/docs/en/ibm-mq/9.2?topic=scenarios-streaming-queues) and one of the use-cases it is designed to support is putting a copy of every message sent to a given queue, to an alternative queue. Another application can then consume the duplicate messages and archive them separately to the application that is processing the original messages.

How to make a JMS Synchronous request

I have an webapp that is expected to fetch and display data from an External App which is accessible only via messaging (JMS).
So, if a user submits a request on a browser, the same HTTP request thread will have to interact with the Messaging system (MQ Series) such that the same request thread can display the data received from the Messaging System.
Is there a pattern I can make use of here? I saw some vague references on the net that use "Correlation ID" in this way:
Msg m = new TextMsg("findDataXYZ");
String cr_id = m.setCorrelationID(id);
sendQueue.send(m).
// now start listening to the Queue for a msg that bears that specific cr_id
Response r = receiverQueue.receive(cr_id);
Is there something better out there? The other patterns I found expect the response to be received asynchronously.. which is not an option for me, since I have to send the response back on the same HTTP request.

The request/reply messaging pattern is useful for your requirement. You typically use a CorrelationId to relate request & reply messages.
While sending request message you set JMSReplyTo destination on the message. Typically a temporary queue is used as JMSReplyTo destination. When creating a consumer to receive response use a selector with JMSCorrelationId, something like
cons = session.createConsumer(tempDestination,"JMSCorrelationId="+requestMsg.JMSMessageId);
At the other end, the application that is processing the request message must use the JMSReplyTo destination to send response. It must also use the MessageId of the request message and set it as CorrelationId of the response message.

First, open the response queue. Then pass that object to the set reply-to method on the message. That way the service responding to your request knows where to send the reply. Typically the service will copy the message ID to the correlation ID field so when you send the message, take the message ID you get back and use that to listen on the reply queue. Of course if you use a dynamic reply-to queue even that isn't neessary - just listen for the next message on the queue.
There's sample code that shows all of this. If you installed to the default location, the sample code lives at "C:\Program Files (x86)\IBM\WebSphere MQ\tools\jms\samples\simple\SimpleRequestor.java" on a Windows box or /var/mqm/toolsjms/samples/simple/SimpleRequestor.java on a *nix box.
And on the off chance you are wondering "install what, exactly?" the WMQ client install is downloadable for free as SupportPac MQC71.

Work around for MessageNotReadableException in Java

I am building a small api around the JMS API for a project of mine. Essentially, we are building code that will handle the connection logic, and will simplify publishing messages by providing a method like Client.send(String message).
One of the ideas being discussed right now is that we provide a means for the users to attach interceptors to this client. We will apply the interceptors after preparing the JMS message and before publishing it.
For example, if we want to timestamp a message and wrote an interceptor for that, then this is how we would apply that
...some code ...
Message message = session.createMessage()
..do all the current processing on the message and set the body
for(interceptor:listOfInterceptors){
interceptor.apply(message)
}
One of the intrerceptors we though of was to compress the message body. But when we try to read the body of the message in the interceptor, we are getting a MessageNotReadableException. In the past, I normally compressed the content before setting it as the body of the message - so never had to worry about this exception.
Is there any way of getting around this exception?

It looks like your JMS client attempts to read a write-only message. Your interceptor cannot work this way, please elaborate how you were compressing message earlier.