I'm using ActiveMQ Artemis version 2.19.1, and I'm facing an issue in a 6-node (3 masters) cluster where redistribution is failing for large messages with the below warning logs:
23:35:05,551 WARN [org.apache.activemq.artemis.core.server] AMQ222303: Redistribution by Redistributor[TEST_QUEUE/2244] of messageID = 196,950,715 failed: java.lang.UnsupportedOperationException: Method not supported with Large Messages
at org.apache.activemq.artemis.protocol.amqp.broker.AMQPLargeMessage.getData(AMQPLargeMessage.java:311) [artemis-amqp-protocol-2.19.1.jar:2.19.1]
at org.apache.activemq.artemis.protocol.amqp.broker.AMQPMessage.anyMessageAnnotations(AMQPMessage.java:1374) [artemis-amqp-protocol-2.19.1.jar:2.19.1]
at org.apache.activemq.artemis.protocol.amqp.broker.AMQPMessage.hasScheduledDeliveryTime(AMQPMessage.java:1352) [artemis-amqp-protocol-2.19.1.jar:2.19.1]
at org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl.processRoute(PostOfficeImpl.java:1499) [artemis-server-2.19.1.jar:2.19.1]
at org.apache.activemq.artemis.core.server.cluster.impl.Redistributor$1.run(Redistributor.java:169) [artemis-server-2.19.1.jar:2.19.1]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.19.1.jar:2.19.1]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.19.1.jar:2.19.1]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.19.1.jar:2.19.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_322]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_322]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.19.1.jar:2.19.1]
Later I see broker is removing consumers with below warning since properties=null:
00:10:28,280 WARN [org.apache.activemq.artemis.core.server] AMQ222151: removing consumer which did not handle a message, consumer=ServerConsumerImpl [id=57, filter=null, binding=LocalQueueBinding [address=TEST_QUEUE, queue=QueueImpl[name=TEST_QUEUE, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=localhost], temp=false]#9c000b7, filter=null, name=TEST_QUEUE, clusterName=TEST_QUEUE1e359f55-c92b-11ec-b908-005056a3af3f]], message=Reference[157995028]:RELIABLE:AMQPLargeMessage( [durable=true, messageID=157995028, address=TEST_QUEUE, size=0, scanningStatus=SCANNED, applicationProperties={VER=05, trackingId=62701757c80c3004d037ded6}, messageAnnotations={}, properties=null, extraProperties = TypedProperties[_AMQ_AD=TEST_QUEUE]]: java.lang.IllegalArgumentException: Array must not be empty or null
at org.apache.qpid.proton.codec.CompositeReadableBuffer.append(CompositeReadableBuffer.java:688) [proton-j-0.33.10.jar:]
at org.apache.qpid.proton.engine.impl.DeliveryImpl.send(DeliveryImpl.java:345) [proton-j-0.33.10.jar:]
at org.apache.qpid.proton.engine.impl.SenderImpl.send(SenderImpl.java:74) [proton-j-0.33.10.jar:]
at org.apache.activemq.artemis.protocol.amqp.proton.ProtonServerSenderContext$LargeMessageDeliveryContext.deliverInitialPacket(ProtonServerSenderContext.java:686) [artemis-amqp-protocol-2.19.1.jar:2.19.1]
at org.apache.activemq.artemis.protocol.amqp.proton.ProtonServerSenderContext$LargeMessageDeliveryContext.deliver(ProtonServerSenderContext.java:587) [artemis-amqp-protocol-2.19.1.jar:2.19.1]
I have 6 consumers for this queue if one message out of many (let's say 1,000) is large - it should process other messages but, the processing is stopped completely with 0 consumers on the queue.
When you send large messages, within the first block sent the large message is parsed for properties. Most likely you are using large properties in a way the server is not being able to parse the first few bytes of the message. You should avoid using large properties and let the large portion only for the client.
We tried following up with many tests on this JIRA, and the only plausible scenarios would be through large properties, or some incomplete message your client is generating in a way the server can't parse it.
https://issues.apache.org/jira/browse/ARTEMIS-3837
if you provide a reproducer showing how the property is failing to be parsed we will follow up with a possible fix.
Please move any discussion regarding this to the JIRA. It could be a bug caused by your anti-pattern.
My use case warrants the use of a publish-subscribe model, so I'm using topics that are defined in my Spring Boot client. I've used the tool MQTT.fx and two Spring Boot clients to test things. The two Spring Boot clients are able to communicate with each other, but the MQTT.fx client only connects to the broker and does not produce or consume messages. The two Spring Boot clients both connect to port 61616, and the MQTT.fx client connects to port 1883.
For my project, I need to get the MQTT.fx client working as it is highly representative of the microcontroller that is intended to work with the server. Failure on MQTT.fx is failure on the microcontroller.
The acceptor configuration in my broker.xml file is as show below.
<acceptors>
<!-- Acceptor for every supported protocol -->
<acceptor name="artemis">tcp://0.0.0.0:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=false;suppressInternalManagementObjects=false</acceptor>
<!-- AMQP Acceptor. Listens on default AMQP port for AMQP traffic.-->
<acceptor name="amqp">tcp://0.0.0.0:5672?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=AMQP;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpMinLargeMessageSize=102400;amqpDuplicateDetection=true</acceptor>
<!-- STOMP Acceptor. -->
<acceptor name="stomp">tcp://0.0.0.0:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true</acceptor>
<!-- HornetQ Compatibility Acceptor. Enables HornetQ Core and STOMP for legacy HornetQ clients. -->
<acceptor name="hornetq">tcp://0.0.0.0:5445?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.;protocols=HORNETQ,STOMP;useEpoll=true</acceptor>
<!-- MQTT Acceptor -->
<acceptor name="mqtt">tcp://0.0.0.0:1883?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=MQTT;useEpoll=true</acceptor>
</acceptors>
The logs when the Spring Boot clients communicate are as below. First, a message is sent to the topic server.weatherForecast, and consequently, after some processing, a response is sent to AMEBAA000105.device.weatherForecast.
2021-10-04 14:14:04,860 [AUDIT](Thread-8 (activemq-netty-threads))
AMQ601715: User admin(admins)#127.0.0.1:1125 successfully authenticated
2021-10-04 14:14:04,860 [AUDIT](Thread-8 (activemq-netty-threads)) AMQ601267: User admin(admins)#127.0.0.1:1125 is creating a core session on target resource ActiveMQServerImpl::name=0.0.0.0 [with parameters: [3b141c41-24ef-11ec-aa52-00155d831300, null, ****, 102400, RemotingConnectionImpl [ID=6280c69f, clientID=null, nodeID=b719c384-1d0a-11ec-8b7c-00155d831300, transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection#417b6fef[ID=6280c69f, local= /127.0.0.1:61616, remote=/127.0.0.1:1125]], true, true, false, false, null, org.apache.activemq.artemis.core.protocol.core.impl.CoreSessionCallback#103d417f, true, OperationContextImpl [61828916] [minimalStore=9223372036854775807, storeLineUp=0, stored=0, minimalReplicated=9223372036854775807, replicationLineUp=0, replicated=0, paged=0, minimalPage=9223372036854775807, pageLineUp=0, errorCode=-1, errorMessage=null, executorsPending=0, executor=OrderedExecutor(tasks=[])], {}]]
2021-10-04 14:14:04,864 [AUDIT](Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#b62d79)) AMQ601500: User admin(admins)#127.0.0.1:1125 is sending a message CoreMessage[messageID=528267,durable=true,userID=3b146a62-24ef-11ec-aa52-00155d831300,priority=4, timestamp=Mon Oct 04 14:14:04 IST 2021,expiration=0, durable=true, address=jms.topic.server.weatherForecast,size=314,properties=TypedProperties[__AMQ_CID=3b13ce1f-24ef-11ec-aa52-00155d831300]]#1994364957, with Context: RoutingContextImpl(Address=null, routingType=null, PreviousAddress=null previousRoute:null, reusable=null, version=0)
..................................................
2021-10-04 14:14:04,865 [AUDIT](Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#b62d79)) AMQ601501: User admin(admins)#127.0.0.1:1110 is consuming a message from 41207748-6ed3-42d2-b75e-044805212686: Reference[528267]:RELIABLE:CoreMessage[messageID=528267,durable=true,userID=3b146a62-24ef-11ec-aa52-00155d831300,priority=4, timestamp=Mon Oct 04 14:14:04 IST 2021,expiration=0, durable=true, address=jms.topic.server.weatherForecast,size=314,properties=TypedProperties[__AMQ_CID=3b13ce1f-24ef-11ec-aa52-00155d831300]]#1994364957
2021-10-04 14:14:04,868 [AUDIT](Thread-4 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#b62d79)) AMQ601502: User admin(admins)#127.0.0.1:1110 is acknowledging a message from 41207748-6ed3-42d2-b75e-044805212686: CoreMessage[messageID=528267,durable=true,userID=3b146a62-24ef-11ec-aa52-00155d831300,priority=4, timestamp=Mon Oct 04 14:14:04 IST 2021,expiration=0, durable=true, address=jms.topic.server.weatherForecast,size=314,properties=TypedProperties[__AMQ_CID=3b13ce1f-24ef-11ec-aa52-00155d831300]]#1994364957
2021-10-04 14:14:08,059 [AUDIT](Thread-8 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#b62d79)) AMQ601500: User admin(admins)#127.0.0.1:1110 is sending a message CoreMessage[messageID=528349,durable=true,userID=3cfc1623-24ef-11ec-aa52-00155d831300,priority=4, timestamp=Mon Oct 04 14:14:08 IST 2021,expiration=0, durable=true, address=null,size=190,properties=TypedProperties[__AMQ_CID=d04d0e87-24ee-11ec-aa52-00155d831300]]#600964226, with Context: RoutingContextImpl(Address=jms.topic.AMEBAA000105.device.weatherForecast, routingType=null, PreviousAddress=jms.topic.AMEBAA000105.device.weatherForecast previousRoute:null, reusable=null, version=0)
..................................................
The logs when using MQTT.fx are as below. Initially AMEBAA000105.device.weatherForecast is subscribed to, and then a message is sent to server.weatherForecast. The Spring Boot client meant to receive these messages does not receive anything, and consequently does not send anything to AMEBAA000105.device.weatherForecast. When the publication of a message to AMEBAA000105.device.weatherForecast from the Spring Boot client is triggered by another Spring Boot client, MQTT.fx does not capture the message.
2021-10-04 14:24:35,443 [AUDIT](Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#b62d79)) AMQ601265: User admin(admins)#192.168.0.107:18640 is creating a core consumer on target resource ServerSessionImpl() [with parameters: [534062, MQTT_FX_Client.AMEBAA000105.device.weatherForecast, null, 0, false, false, -1]]
2021-10-04 14:24:43,719 [AUDIT](Thread-10 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#b62d79)) AMQ601500: User admin(admins)#192.168.0.107:18640 is sending a message CoreMessage[messageID=534380,durable=false,userID=null,priority=0, timestamp=0,expiration=0, durable=false, address=server.weatherForecast,size=200,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]#1214406991, with Context: RoutingContextImpl(Address=server.weatherForecast, routingType=null, PreviousAddress=null previousRoute:null, reusable=null, version=0)
The above logs are the audit logs. In the artemis.log file, MQTT.fx keeps pinging the broker.
2021-10-04 14:30:43,727 TRACE [org.apache.activemq.artemis.core.protocol.mqtt] MQTT(MQTT_FX_Client): OUT >> PINGRESP
2021-10-04 14:31:43,728 TRACE [org.apache.activemq.artemis.core.protocol.mqtt] MQTT(MQTT_FX_Client): OUT >> PINGRESP
2021-10-04 14:32:43,729 TRACE [org.apache.activemq.artemis.core.protocol.mqtt] MQTT(MQTT_FX_Client): OUT >> PINGRESP
2021-10-04 14:33:43,731 TRACE [org.apache.activemq.artemis.core.protocol.mqtt] MQTT(MQTT_FX_Client): OUT >> PINGRESP
2021-10-04 14:34:43,732 TRACE [org.apache.activemq.artemis.core.protocol.mqtt] MQTT(MQTT_FX_Client): OUT >> PINGRESP
2021-10-04 14:35:43,733 TRACE [org.apache.activemq.artemis.core.protocol.mqtt] MQTT(MQTT_FX_Client): OUT >> PINGRESP
Logs when subscribing or publishing from MQTT.fx in the artemis.log file are as below
2021-10-04 14:22:27,702 TRACE [org.apache.activemq.artemis.core.protocol.mqtt] MQTT(MQTT_FX_Client): IN << SUBSCRIBE(3)
AMEBAA000105.device.weatherForecast : AT_MOST_ONCE
2021-10-04 14:24:43,718 TRACE [org.apache.activemq.artemis.core.protocol.mqtt] MQTT(MQTT_FX_Client): IN << PUBLISH(-1) topic=server.weatherForecast, qos=AT_MOST_ONCE, retain=false, dup=false, payload={"serialNumber" : "AMEBAA000105"}
Any help is much appreciated.
You say that when you're testing the Spring Boot clients you send a message to server.weatherForecast and the after some processing a response is sent to AMEBAA000105.device.weatherForecast. However, that is not what the logs show. The logs indicate that you send a message to jms.topic.server.weatherForecast and jms.topic.AMEBAA000105.device.weatherForecast respectively. Notice the jms.topic. prefix.
I believe this is the root of your problem because according to the logs the MQTT actually will use server.weatherForecast and AMEBAA000105.device.weatherForecast. Since the two clients aren't using matching names then they can never work together with your current configuration.
My guess is that you're using a legacy Artemis 1.x client from your Spring Boot applications and that is why the jms.topic. prefix is being used. You should either move to a newer client (e.g. one that matches the broker version you're using) or configure the anycastPrefix and multicastPrefix to support the legacy 1.x client. The default broker.xml has a comment which explains how to do this:
<!-- Note: If an acceptor needs to be compatible with HornetQ and/or Artemis 1.x clients add
"anycastPrefix=jms.queue.;multicastPrefix=jms.topic." to the acceptor url.
See https://issues.apache.org/jira/browse/ARTEMIS-1644 for more information. -->
This comment appears in the acceptors block by default, but it looks like you removed it since it's not in the XML you pasted.
If you want to confirm which library the client is using the simplest way to to find out is simply to inspect the client's environment. However, if that doesn't work and you have access to the broker logs then you can enable TRACE logging for org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl and then look at the version passed in the CreateSessionMessage. If it's < 131 then it's not a 2.18.0 client.
Changing the signature of consumeWeatherForecastRequest method to private void consumeWeatherForecastRequest(String incomingMessage) should fix your issue.
I'm trying to use Reactor Netty TcpClient in reactive way to interact with hosts, that may be unreachable. Here is an example of a channel initialization logic:
ConnectionProvider connectionProvider = ConnectionProvider.fixed("fixed", 50);
TcpClient.create(connectionProvider)
.host(host).port(port)
.wiretap(true)
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 50)
.doOnConnect(x -> log.trace("Connect to {}:{}", host, port))
.doOnConnected(conn -> log.trace("Connected {}", conn.channel()))
.connect()
.subscribe(this::utilizeConnection);
the output, that i receiving :
2019-09-04 08:23:13.612 TRACE 71988 --- [ioEventLoop-4-3] c.c.pcb.poc.network.tcp.NettyTcpSender : Connect to 192.168.88.210:2000
2019-09-04 08:23:13.684 WARN 71988 --- [actor-tcp-nio-4] io.netty.util.concurrent.DefaultPromise : An exception was thrown by reactor.netty.resources.PooledConnectionProvider$DisposableAcquire.operationComplete()
reactor.core.Exceptions$ErrorCallbackNotImplemented: io.netty.channel.ConnectTimeoutException: connection timed out: /192.168.88.210:2000
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /192.168.88.210:2000
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267) ~[netty-transport-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) ~[netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) ~[netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:405) [netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [netty-transport-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) [netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.36.Final.jar:4.1.36.Final]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Assembly trace from producer [reactor.core.publisher.MonoCreate] :
reactor.core.publisher.Mono.create(Mono.java:183)
reactor.netty.resources.PooledConnectionProvider.acquire(PooledConnectionProvider.java:130)
Error has been observed by the following operator(s):
|_ Mono.create ⇢ reactor.netty.resources.PooledConnectionProvider.acquire(PooledConnectionProvider.java:130)
|_ Mono.doOnSubscribe ⇢ reactor.netty.tcp.TcpClientDoOn.connect(TcpClientDoOn.java:58)
The 'inbound' and 'outbound' are having a dedicated method to handle their errors, but they works on top of a Connection instance that won't be created if you got the 'connection timeout'.
I tried:
The exception, that i receiving is wrapped in 'ErrorCallbackNotImplemented'. But I wasn't able to find any way to implement any 'ErrorCallback'
The log contains a warning message from 'io.netty.util.concurrent.DefaultPromise' . but I wasn't able to find a way how to make own Promise to handle it in a right way.
No any configurations i've found that may somehow intercept connection timeouts.
workaround. The blocked approach to create a connection ( .block() instead of .subscribe()) will allow me to catch any Connection creating exceptions within plain try-catch block, but i'll lose the benefits of reactive approach with such workaround.
Do somebody may suggest me at least something to help me to find a right way to handle a 'io.netty.channel.ConnectTimeoutException'?
Do not forget to implement your error callback
Usually reactor.core.Exceptions$ErrorCallbackNotImplemented happens when there is subscription over labmda based .subscribe method (same for Mono and Flux).
If you are going to look at the sources here and here, you will find the place where reactor.core.Exceptions$ErrorCallbackNotImplemented is thrown!
Action Points
In order to handle the original io.netty.channel.ConnectTimeoutException I would recommend looking at Handling Errors section of the original Project Reactor documentation
Jms outbound-channel-adapter works perfectly fine but I see intermittently this error in the log, however, MQ message still gets delivered.
2019-06-07 10:16:22 [JMSCCThreadPoolWorker-5] INFO o.s.j.c.CachingConnectionFactory - Encountered a JMSException - resetting the underlying JMS Connection
com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ1107: A problem with this connection has occurred.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:578)
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:214)
at com.ibm.msg.client.wmq.internal.WMQConnection.consumer(WMQConnection.java:794)
at com.ibm.mq.jmqi.remote.api.RemoteHconn.callEventHandler(RemoteHconn.java:2903)
at com.ibm.mq.jmqi.remote.api.RemoteHconn.driveEventsEH(RemoteHconn.java:628)
at com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.processHconn(RemoteDispatchThread.java:691)
at com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.run(RemoteDispatchThread.java:233)
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:263)
at com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:99)
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:284)
at com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:312)
at com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1214)
Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
...This is errorChannel configuration:
<int:header-enricher id="errorMsg.HeaderEnricher"
input-channel="errorChannel"
output-channel="omniAlertsJmsErrorChannel">
...and jms outbound-channel-adapter configured:
<int-jms:outbound-channel-adapter
id="jmsOutToNE" channel="umpAlertNotificationJMSChannel"
destination="senderTopic"
jms-template="jmsQueueTemplate"
>
I am expecting omniAlertsJmsErrorChannel to receive the MessageHandlingException which is not happening from jmsOutToNE adapter. All other channel/flow errors are being routed to omniAlertsJmsErrorChannel.
Also, wondering if there jms outbound-channel-adapter is retrying internally when com.ibm.mq.MQException happens and it gets successful in the subsequent try?
The errorChannel is out of use in any MessageHandler. Their exception are just thrown to the caller. To handle those exception you need to consider adding a request-handler-advice-chain with the ExpressionEvaluatingRequestHandlerAdvice. Or if we talk about a retry, you also can add to that chain a RequestHandlerRetryAdvice.
See more info in the Reference Manual.
Not sure though why your message is still delivered to the MQ. There is no any retry out-of-the-box in the <int-jms:outbound-channel-adapter. It might a behavior of the MQ Connection Factory adapter in the IBM library.
ActiveMQ 5.14.1 is working well but is flooding the log with that exception.
Connecting to:
<transportConnector name="wss" uri="wss://0.0.0.0:443?maximumConnections=5000&wireFormat.maxFrameSize=104857600"/>
Exception:
2016-12-07 11:28:56,106 | WARN | / | org.eclipse.jetty.servlet.ServletHandler | qtp1782247761-29
java.lang.NullPointerException
at org.apache.activemq.transport.ws.jetty9.WSServlet.doGet(WSServlet.java:88)[activemq-http-5.14.1.jar:5.14.1]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)[tomcat-servlet-api-8.0.24.jar:]
at org.eclipse.jetty.websocket.servlet.WebSocketServlet.service(WebSocketServlet.java:167)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)[tomcat-servlet-api-8.0.24.jar:]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.server.Server.handle(Server.java:499)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)[jetty-all-9.2.13.v20150730.jar:9.2.13.v20150730]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_80]
Is that a real bug?
This appears to be the same as this issue (AMQ-6491) which happens when you try and send a GET request to the broker for whatever reason. Normally you wouldn't do that but some people seem to have used it to hack around limitations in Chrome or the like. This is fixed in the ActiveMQ 5.14.2 release.