We use IBM MQ for some JMS integration between some of our micro services. Because IBM MQ docker image does not run on Mac book laptops with Apple M1 processor we configured our applications requiring JMS integration to be able to run against ActiveMQ broker as well.
We use the latest version of ActiveMQ classic which is 5.17.3 and we run it as a docker container.
For most of us who tried to run against ActiveMQ everything works OK but we have a colleague that is unable to start the application because it fails to connect to the ActiveMQ broker. All we can see in the application logs is this:
Caused by: javax.jms.JMSException: Cannot send, channel has already failed: tcp://127.0.0.1:61616
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:80)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1423)
at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1488)
at org.apache.activemq.ActiveMQXAConnection.createSession(ActiveMQXAConnection.java:74)
at org.apache.activemq.ActiveMQXAConnection.createXASession(ActiveMQXAConnection.java:61)
at com.atomikos.datasource.xa.jms.JmsTransactionalResource.refreshXAConnection(JmsTransactionalResource.java:74)
... 10 common frames omitted
Caused by: org.apache.activemq.transport.InactivityIOException: Cannot send, channel has already failed: tcp://127.0.0.1:61616
at org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:329)
at org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:318)
at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:94)
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:116)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)
at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1394)
... 14 common frames omitted
Googling for org.apache.activemq.transport.InactivityIOException: Cannot send, channel has already failed brought us some old internet posts but none of them brought any light.
Part of the ActiveMQ docker logs we can see below log entry after broker is started which makes us think the broker is started OK. Also he can access the ActiveMQ admin console when hitting http://localhost:8161/admin URL
INFO | Listening for connections at : tcp://somelaptopid:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600
Related
I have installed rabbit MQ 3.8.3 on windows 10 and I can see it is running as windows services.
When I try to access http://localhost:15672/ it is unreachable.
I have enabled the rabbit MQ management plugin in sbin directory
rabbitmq-plugins enable rabbitmq_management
But still, http://localhost:15672/ is unreachable.
Getting the following error in java service :
org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused: connect
I have also run the command to see if anything is running on port 5672:
Command : netstat -ano | find "5672"
Response : TCP 0.0.0.0:25672 0.0.0.0:0 LISTENING 2900
How do I fix this?
The management UI can be accessed using a Web browser at http://{node-hostname}:15672/
The management plugin is included in the RabbitMQ distribution. Like any other plugin, it must be enabled before it can be used.
please refer below URL:
https://www.rabbitmq.com/management.html
Stop rabbitmq
Create the RABBITMQ_BASE environment variable set its value to full
path of rabbit mq server. (eg. C:\Program Files\RabbitMQ Server)
Restart the rabbit mq and make sure the rabbit mq plugin is enabled.
I've got problem forwarding jmx from kubernetes pod to localhost.
Everything works fine when port is forwarded to the same port:
kubectl port-forward ... 9010:9010 OR
kubectl port-forward ... 9010
However, when I try to forward to a different local port,
kubectl port-forward ... 9011:9010
neither jconsole or jms can connect.
It is blocking me from profiling multiple applications at once.
JMS Error:
com.oracle.jmc.rjmx.ConnectionException caused by java.rmi.NoSuchObjectException: no such object in table
at com.oracle.jmc.rjmx.internal.RJMXConnection.connect(RJMXConnection.java:406)
at com.oracle.jmc.rjmx.internal.ServerHandle.doConnect(ServerHandle.java:88)
at com.oracle.jmc.rjmx.internal.ServerHandle.connect(ServerHandle.java:78)
at com.oracle.jmc.console.ui.editor.internal.ConsoleEditor$ConnectJob.run(ConsoleEditor.java:73)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Caused by: java.rmi.NoSuchObjectException: no such object in table
at java.rmi/sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283)
at java.rmi/sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260)
at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
at java.management.rmi/javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2105)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:321)
at com.oracle.jmc.rjmx.internal.RJMXConnection.connectJmxConnector(RJMXConnection.java:451)
at com.oracle.jmc.rjmx.internal.RJMXConnection.establishConnection(RJMXConnection.java:427)
at com.oracle.jmc.rjmx.internal.RJMXConnection.connect(RJMXConnection.java:399)
... 4 more
jconsole error:
Connection to localhost:9011 did no succeed.
This is an RMI issue. The problem is that the RMI stub delivered to the JMX client is created for 9010 and so it fails when attempting to connect at 9011. There's a couple of decent solutions outlined here. Another option is to switch to JMXMP which is a pure socket JMX protocol so port forwarding works without any additional workarounds.
I am using IBM MQ v7.5 with JBOSS EAP 6.4 with JCA resource adapter and MDB.
MQ server is running in HP NonStop Integrity Server v5.3.1.12
The application is working correctly. But I am seeing the following exception in my log
[com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016027: Local XARecoveryModule.xaRecovery got XA exception XAException.XAER_RMFAIL: javax.transaction.xa.XAException
at com.ibm.mq.connector.RecoveryXAResource.checkExceptions(RecoveryXAResource.java:147)
at com.ibm.mq.connector.RecoveryXAResource.recover(RecoveryXAResource.java:514)
at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.recover(XAResourceWrapperImpl.java:185)
at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.xaRecoveryFirstPass(XARecoveryModule.java:541) [jbossjts-jacorb-4.17.29.Final-redhat-1.jar:4.17.29.Final-redhat-1]
at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkFirstPass(XARecoveryModule.java:181) [jbossjts-jacorb-4.17.29.Final-redhat-1.jar:4.17.29.Final-redhat-1]
at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:747) [jbossjts-jacorb-4.17.29.Final-redhat-1.jar:4.17.29.Final-redhat-1]
at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:375) [jbossjts-jacorb-4.17.29.Final-redhat-1.jar:4.17.29.Final-redhat-1] Caused by: com.ibm.msg.client.wmq.v6.jms.internal.ConfigEnvironment$1: MQJMS1068: failed to obtain XAResource.
at com.ibm.msg.client.wmq.v6.jms.internal.ConfigEnvironment.newException(ConfigEnvironment.java:379)
at com.ibm.msg.client.wmq.v6.jms.internal.MQXAConnection.createXASession(MQXAConnection.java:155)
at com.ibm.msg.client.jms.internal.JmsXAConnectionImpl.createXASession(JmsXAConnectionImpl.java:125)
at com.ibm.mq.jms.MQXAConnection.createXASession(MQXAConnection.java:88)
at com.ibm.mq.connector.RecoveryXAResource.recover(RecoveryXAResource.java:490)
... 5 more Caused by: javax.transaction.xa.XAException: The method 'xa_open' has failed with errorCode '-3'.
at com.ibm.mq.jmqi.JmqiXAResource.<init>(JmqiXAResource.java:274)
at com.ibm.mq.jmqi.JmqiXAResource.getInstance(JmqiXAResource.java:122)
at com.ibm.mq.jmqi.JmqiEnvironment.newJmqiXAResource(JmqiEnvironment.java:1598)
at com.ibm.msg.client.wmq.v6.base.internal.MQXAQueueManager.getXAResource(MQXAQueueManager.java:175)
at com.ibm.msg.client.wmq.v6.jms.internal.MQXAConnection.createXASession(MQXAConnection.java:134)
... 8 more
Can anyone help my understanding why this is happening?
Also, even if I stop my server, the connection to the MQ server was not killed. It remains as an orphan. Is it somehow related to this exception?
I am using NoTransaction in resource adapter configuration in standalone.xml
The XA error code -3 is XAER_RMERR - which means that a problem occurred with the resource manager, in this case MQ.
Have you looked at the queue manager's error logs?
I missed an important point in this question (due to lack of knowledge in IBM MQ).That is, my MQ server is running in HP NonStop Integrity Server. These servers doesn't support XA connection. Hence this error.
Now, the question is how to stop it.
I have stopped the periodic recovery in JBOSS with the following properties:
name="RecoveryEnvironmentBean.periodicRecoveryPeriod" value="0"
name="RecoveryEnvironmentBean.recoveryBackoffPeriod" value="0"
This is a known issue. IBM has issued fixpacks to address this issue. Please see:
http://www-01.ibm.com/support/docview.wss?uid=swg1IC97579
IBM MQ versions 7.5.0.4 and later have this fix.
And, yes, your orphaned connection is a result of this issue.
Getting below error with MQ(Message Queue), how to resolve this?
Message : com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ1107: A problem with
this connection has occurred. An error has occurred with the WebSphere MQ JMS
connection. Use the linked exception to determine the cause of this error.
Caused by [1] -->
Message : com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with
compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
Caused by [3] -->
Message : com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009;AMQ9208: Error on receive
from host '/xx.xx.xx.xx:xxxx
(emb701t7.dci.bt.com)'. [1=-1,2=ffffffff,3=/xx.xx.xx.xx:xxxx
(emb701t7.dci.bt.com),4=TCP]
Please take me forward.
I faced the same error and its resolved now!
As Magic Wand mentioned in the comment, below error comes up if you have a lower version of MQ jar i.e if your MQ jar version is 7.0.1.5 or below
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009;AMQ9208: Error on receive from host
This bug has been fixed in IBM MQ version 7.0.1.6 or higher as mentioned here: http://www-01.ibm.com/support/docview.wss?uid=swg1IV00348
Add the newer version of below ibm.mq jars and make sure you have the additional jars too mentioned below:
com.ibm.mq.commonservices.jar
com.ibm.mq.jar
com.ibm.mq.jmqi.jar
com.ibm.mqbind.jar
com.ibm.mqjms.jar
fscontext.jar
jms-1.1-20020430.jar
providerutil.jar
The TCP connection between the MQ client running in weblogic and the MQ Queue Manager channel process is being broken unexpectedly (that is what the return code 2009 means). This could be down to a number of issues such as the queue manager being killed, the channel process exiting for some reason, a firewall killing the connection etc. Have a look on the MQ queue manager error logs for issues with the channel process at the same time. Take a note of how often the error message occurs i.e. is the first error 2 hours after the weblogic server starts.
You could take a network trace to see how the TCP connection is being closed and by who.
I am currently trying to create a JMS Queue to a Web Sphere server, Now i have followed all the tutorials on-line and have reached a wall. I have set up the server and my test application and can not seem to get it to connect. This is the output i get when i try to connect, was hopping someone on here has the knowledge to help me fix this problem as its driving me crazy
This is the output i get when i try to connect:
Exception in thread "P=848735:O=0:CT" java.lang.NoClassDefFoundError: com.ibm.ffdc.Manager
at com.ibm.ws.naming.util.RasUtil.logException(RasUtil.java:164)
at com.ibm.ws.naming.util.RasUtil.logException(RasUtil.java:72)
at com.ibm.ws.naming.util.Helpers.getOrb(Helpers.java:398)
at com.ibm.ws.naming.util.WsnInitCtxFactory.getInitialContextInternal(WsnInitCtxFactory.java:462)
at com.ibm.ws.naming.util.WsnInitCtx.getContext(WsnInitCtx.java:128)
at com.ibm.ws.naming.util.WsnInitCtx.getContextIfNull(WsnInitCtx.java:765)
at com.ibm.ws.naming.util.WsnInitCtx.lookup(WsnInitCtx.java:164)
at com.ibm.ws.naming.util.WsnInitCtx.lookup(WsnInitCtx.java:179)
at javax.naming.InitialContext.lookup(InitialContext.java:436)
at message.JMSMessageSender.connect(JMSMessageSender.java:68)
at client.JMSTester.main(JMSTester.java:36)
Caused by: java.lang.ClassNotFoundException: com.ibm.ffdc.Manager
at java.net.URLClassLoader.findClass(URLClassLoader.java:434)
at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:688)
at java.lang.ClassLoader.loadClass(ClassLoader.java:667)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:358)
at java.lang.ClassLoader.loadClass(ClassLoader.java:650)
... 11 more
A quote from a possibly related IBM technote
Add the WebSphere Application Server Administration Thin Client JAR jar
com.ibm.ws.admin.client_X.0.0.jar
to the application Java Build Path located in the runtimes folder of the WebSphere Application Server installation directory.
Note that the technote is not special for JMS. But possibly in your case the JMS libraries introduce a dependency on FFDC as well.