We Keep Coding

Cannot connect to ActiveMQ "Classic"

We use IBM MQ for some JMS integration between some of our micro services. Because IBM MQ docker image does not run on Mac book laptops with Apple M1 processor we configured our applications requiring JMS integration to be able to run against ActiveMQ broker as well.
We use the latest version of ActiveMQ classic which is 5.17.3 and we run it as a docker container.
For most of us who tried to run against ActiveMQ everything works OK but we have a colleague that is unable to start the application because it fails to connect to the ActiveMQ broker. All we can see in the application logs is this:
Caused by: javax.jms.JMSException: Cannot send, channel has already failed: tcp://127.0.0.1:61616
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:80)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1423)
at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1488)
at org.apache.activemq.ActiveMQXAConnection.createSession(ActiveMQXAConnection.java:74)
at org.apache.activemq.ActiveMQXAConnection.createXASession(ActiveMQXAConnection.java:61)
at com.atomikos.datasource.xa.jms.JmsTransactionalResource.refreshXAConnection(JmsTransactionalResource.java:74)
... 10 common frames omitted
Caused by: org.apache.activemq.transport.InactivityIOException: Cannot send, channel has already failed: tcp://127.0.0.1:61616
at org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:329)
at org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:318)
at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:94)
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:116)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)
at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1394)
... 14 common frames omitted
Googling for org.apache.activemq.transport.InactivityIOException: Cannot send, channel has already failed brought us some old internet posts but none of them brought any light.
Part of the ActiveMQ docker logs we can see below log entry after broker is started which makes us think the broker is started OK. Also he can access the ActiveMQ admin console when hitting http://localhost:8161/admin URL
INFO | Listening for connections at : tcp://somelaptopid:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600

IBM MQ v7.5 with JBOSS EAP 6.4 with JCA resource adapter and MDB throws 'xa_open' has failed with errorCode '-3'

I am using IBM MQ v7.5 with JBOSS EAP 6.4 with JCA resource adapter and MDB.
MQ server is running in HP NonStop Integrity Server v5.3.1.12
The application is working correctly. But I am seeing the following exception in my log
[com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016027: Local XARecoveryModule.xaRecovery got XA exception XAException.XAER_RMFAIL: javax.transaction.xa.XAException
at com.ibm.mq.connector.RecoveryXAResource.checkExceptions(RecoveryXAResource.java:147)
at com.ibm.mq.connector.RecoveryXAResource.recover(RecoveryXAResource.java:514)
at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.recover(XAResourceWrapperImpl.java:185)
at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.xaRecoveryFirstPass(XARecoveryModule.java:541) [jbossjts-jacorb-4.17.29.Final-redhat-1.jar:4.17.29.Final-redhat-1]
at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkFirstPass(XARecoveryModule.java:181) [jbossjts-jacorb-4.17.29.Final-redhat-1.jar:4.17.29.Final-redhat-1]
at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:747) [jbossjts-jacorb-4.17.29.Final-redhat-1.jar:4.17.29.Final-redhat-1]
at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:375) [jbossjts-jacorb-4.17.29.Final-redhat-1.jar:4.17.29.Final-redhat-1] Caused by: com.ibm.msg.client.wmq.v6.jms.internal.ConfigEnvironment$1: MQJMS1068: failed to obtain XAResource.
at com.ibm.msg.client.wmq.v6.jms.internal.ConfigEnvironment.newException(ConfigEnvironment.java:379)
at com.ibm.msg.client.wmq.v6.jms.internal.MQXAConnection.createXASession(MQXAConnection.java:155)
at com.ibm.msg.client.jms.internal.JmsXAConnectionImpl.createXASession(JmsXAConnectionImpl.java:125)
at com.ibm.mq.jms.MQXAConnection.createXASession(MQXAConnection.java:88)
at com.ibm.mq.connector.RecoveryXAResource.recover(RecoveryXAResource.java:490)
... 5 more Caused by: javax.transaction.xa.XAException: The method 'xa_open' has failed with errorCode '-3'.
at com.ibm.mq.jmqi.JmqiXAResource.<init>(JmqiXAResource.java:274)
at com.ibm.mq.jmqi.JmqiXAResource.getInstance(JmqiXAResource.java:122)
at com.ibm.mq.jmqi.JmqiEnvironment.newJmqiXAResource(JmqiEnvironment.java:1598)
at com.ibm.msg.client.wmq.v6.base.internal.MQXAQueueManager.getXAResource(MQXAQueueManager.java:175)
at com.ibm.msg.client.wmq.v6.jms.internal.MQXAConnection.createXASession(MQXAConnection.java:134)
... 8 more
Can anyone help my understanding why this is happening?
Also, even if I stop my server, the connection to the MQ server was not killed. It remains as an orphan. Is it somehow related to this exception?
I am using NoTransaction in resource adapter configuration in standalone.xml

The XA error code -3 is XAER_RMERR - which means that a problem occurred with the resource manager, in this case MQ.
Have you looked at the queue manager's error logs?

I missed an important point in this question (due to lack of knowledge in IBM MQ).That is, my MQ server is running in HP NonStop Integrity Server. These servers doesn't support XA connection. Hence this error.
Now, the question is how to stop it.
I have stopped the periodic recovery in JBOSS with the following properties:
name="RecoveryEnvironmentBean.periodicRecoveryPeriod" value="0"
name="RecoveryEnvironmentBean.recoveryBackoffPeriod" value="0"

This is a known issue. IBM has issued fixpacks to address this issue. Please see:
http://www-01.ibm.com/support/docview.wss?uid=swg1IC97579
IBM MQ versions 7.5.0.4 and later have this fix.
And, yes, your orphaned connection is a result of this issue.

java.lang.ClassNotFoundException when creating a mq connection factory through wsadmin

I get a java.lang.ClassNotFoundException when trying to create a websphere mq connection factory through wsadmin.
WASX7015E: Exception running command: "AdminTask.createWMQConnectionFactory(...)"; exception information:
com.ibm.websphere.management.exception.ConnectorException
org.apache.soap.SOAPException: [SOAPException: faultCode=SOAP-ENV:Client; msg=com.ibm.ws.messaging.admin.command.AdminCommandException
Server stack trace
JMXTransformException java.lang.ClassNotFoundException: com.ibm.ws.messaging.admin.command.AdminCommandException
at java.lang.Class.forNameImpl(Native Method)
at java.lang.Class.forName(Class.java:185)
at com.ibm.ws.util.WsObjectInputStream.loadClass(WsObjectInputStream.java:228)
at com.ibm.ws.util.WsObjectInputStream.access$000(WsObjectInputStream.java:54)
...
I'm using a websphere application server wsadmin thin client, based on this article http://www.ibm.com/developerworks/websphere/library/techarticles/1207_vansickel/1207_vansickel.html.
I probably have to include one or more jars from websphere for this to succeed. Can somebody tell me which ones?
Currently I have these jars on the classpath:
/usr/share/wsadmin/lib/jython/jython.jar
/usr/share/wsadmin/lib/com.ibm.ws.admin.client_7.0.0.jar
/usr/share/wsadmin/lib/com.ibm.ws.security.crypto.jar
/usr/share/wsadmin/lib/batch.jar
I'm using Websphere 7.0.0.

It's odd, but I found a reference to this CNFE occuring when you reference a queue that doesn't exist or whose case differs from the one in your wsadmin invocation. IOW it's some kind of follow-on failure only.
Maybe triple-check any parameters, especially with respect to case.
http://www-01.ibm.com/support/docview.wss?uid=swg21631640

Nexus 2.10 broken pipe IOException

I am using nexus 2.10 having recently upgraded from nexus 2.8. The reason I upgraded was because I was getting the following exception in my logs and thus the jar was not downloaded and Nexus 2.10 gives a few more details in the log regarding the broken pipe error.
So in my logs I get the following error (replaced i[p address with string host name).
2014-10-21 15:18:22,556+0100 INFO [qtp-855494215-66] anonymous org.sonatype.nexus.proxy.storage.remote.httpclient.HttpClientRemoteStorage - Initializing remote transport for proxy repository "Central" [id=central]...
2014-10-21 15:23:59,440+0100 WARN [qtp-855494215-69] anonymous org.sonatype.nexus.content.internal.ContentServlet - org.eclipse.jetty.io.EofException, caused by: java.io.IOException: Broken pipe [client=host.name,ua=m2e/3.9.1.v20130814-1242/1.4.0.20130601-0317/1.6.0_45,req=GET http://xx.xx:8081/nexus/content/groups/public/org/springframework/spring-jdbc/4.1.1.RELEASE/spring-jdbc-4.1.1.RELEASE.jar]
2014-10-21 15:24:43,329+0100 WARN [qtp-855494215-67] anonymous org.sonatype.nexus.content.internal.ContentServlet - org.eclipse.jetty.io.EofException, caused by: java.io.IOException: Broken pipe [client=host.name,ua=m2e/3.9.1.v20130814-1242/1.4.0.20130601-0317/1.6.0_45,req=GET http://xx.xx5:8081/nexus/content/groups/public/org/springframework/spring-tx/4.1.1.RELEASE/spring-tx-4.1.1.RELEASE.jar]
2014-10-21 15:24:58,333+0100 WARN [qtp-855494215-66] anonymous org.sonatype.nexus.content.internal.ContentServlet - org.eclipse.jetty.io.EofException, caused by: java.io.IOException: Broken pipe [client=host.name,ua=m2e/3.9.1.v20130814-1242/1.4.0.20130601-0317/1.6.0_45,req=GET http://xx.xx:8081/nexus/content/groups/public/org/springframework/spring-context/4.1.1.RELEASE/spring-context-4.1.1.RELEASE-sources.jar]
Can anyone advise how to get past this error because currently some jars are not being downloaded. This does not apply to all jars for eg i just downloaded the nekohtml-0.9.5.jar no issues. Could this be possibly something to do with size? In my maven repository I just get a spring-test-4.1.1.RELEASE.jar.lastUpdated jar which is only 1kb in size.
Also I am running behind a firewall so have a proxy server configured in the nexus settings.
UPDATE:
this seems to be an issue only when done through eclipse. I was able to download sources and jars etc fine when I did it from the command line. I am using Eclipse keplar and the m2e plugin. Can anyone comment on this?
All advice appreciated
Thanks

The broken pipe exception occurs when nexus has received an http request from a client, prepares a response, but when it goes to write the response it finds the socket has already been closed.
Likely causes of this are:
An end user initiating a request and then canceling it (or
navigating away from a web page before it completes)
A client which does not properly close socket connections
A client's read timeout setting being reached
A reverse proxy in between nexus and the client (such as apache or
nginx) terminating the connection
Therefore you'll need to begin your investigation on the client side. If it turns out the client or reverse proxy's read timeout setting is triggering this then you'll need to investigate why nexus is responding slowly.

Your Nexus server is probably not able to reach some external repositories if it is running inside a network behind a proxy. You have to make sure you have the http proxy configured in Nexus and that any URL's of remote proxy repositories you want to access are allowed in the proxy.
You can use the "Browse Remote" feature to check if the access works. More details are in the Nexus book http://books.sonatype.com/nexus-book/reference/configxn-sect-customizing-server.html

GlassFish Server does not connect to SQL database (error w.r.t. SQL driver)

I am using a brand new developing pc and need to test a personal application that runs on a local GlassFish server 3.1.2 and should connect with a local SQL database called 'funkOneDB' (my IDE is NetBeans 7.2.1). But I can't get the GlassFish server to connect with the database, and the problem seems to be related to the (place of the) SQL driver in the GlassFish Server's directories (more problem specifics in a few lines).
I am fairly certain I correctly set up the related JDBC Resource and Connection Pool on the GlassFish Server (as I mimic a set-up already existing and working properly on another developing pc).
The Resource specifics are:
jndi name: jdbc/FunkResource
pool name: FunkPool
The (most important) Pool specifics are:
pool name: FunkPool
resource type: javax.sql.Datasource
datasource classname: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
additional properties correspond to the specifics in the XML GlassFish-resources of the application (username, password, url, etc.; no problems there)
I first placed the necessary SQL driver in the GlassFish Server's directories, i.e. the file mysql-connector-java-5.1.18-bin.jar at ..\GlassFish3\GlassFish\domains\domain1\lib\ext.
Yet, when I perform a ping test from the JDBC Pool 'FunkPool' at the GlassFish server, I get the following error:
Ping Connection Pool failed for FunkPool. WEB9031: WebappClassLoader unable to load resource [com.mysql.jdbc.SQLError], because it has not yet been started, or was already stopped Please check the server.log for more details.
In the server.log I only find the following extra logging exception and failure info:
(i) Exception while creating an unpooled [test] connection for pool [ FunkPool ], WEB9031: WebappClassLoader unable to load resource [com.mysql.jdbc.SQLError], because it has not yet been started, or was already stopped
(ii) RestResponse.getResponse() gives FAILURE. endpoint = 'http://localhost:4848/management/domain/resources/ping-connection-pool.json'; attrs = '{id=FunkPool}'
Note however, that when I ping the database funkOneDB from my IDE NetBeans via jdbc:mysql://localhost:33066/funkOneDB, it's succesful. As already mentioned, the credentials and other data I use for this IDE-based ping are the same data I use in the JDBC Connection Pool.
I searched for the problem also on stackoverflow for some. And I did find some people talking about it, like
Glassfisch MySQL ping ERROR (no answer by anybody), or
Struggling to create MySQL Connection Pool on Glassfish (tried that solution, i.e. putting the SQL driver one level up in ..\GlassFish3\GlassFish\domains\domain1\lib\, but this creates other errors, even after restarting the Glassfish server), or
GlassFish not loading connector
(even tried this solution, no succes).
Can somebody help me solve this problem? Many thanks in advance!
With kind regards,
Heinz

Place the mysql driver in the lib folder of your project. Then do a clean-and-build. It's also helpful to have netbeans communicate directly with your database. This will allow you to view the database structure and the contents of your database right from your IDE. For help integrating MySQL with netbeans, look here: netbeans.org/kb/docs/ide/mysql.html

My friend, i had this same exception:
RestResponse.getResponse() gives FAILURE. endpoint = 'http://localhost:4848/management/domain/resources/ping-connection-pool.json'; attrs = '{id=FunkPool}'
The cause of my error was that, i put wrong credentials. Check your credentials in your client DB App (SQL Developer, for example).

I had the same problem with SQL server and Netbeans. To resolve it, i put the sqljdbc.jar in the java direcory "Java\jdk1.8.0_121\lib directory" and it works :)
I've just spebnt 10 hours on this bug.