I have the following problem:
I try to connect to an ActiveMQ broker (which is now down) using the following piece of code
connectionFactory = new ActiveMQConnectionFactory(this.url + "?timeout=2000");
connection = connectionFactory.createConnection();
connection.start();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
LOGGER.info("Connected to " + this.url);
The problem is that the timeout does not have any effect
connection.start()
is blocked forever.
I inspected ActiveMQ log and found the following info:
2013-12-20 01:49:03,149 DEBUG [ActiveMQ Task-1] (FailoverTransport.java:786) - urlList connectionList:[tcp://localhost:61616?timeout=2000], from: [tcp://localhost:61616?timeout=2000]
2013-12-20 01:49:03,149 DEBUG [ActiveMQ Task-1] (FailoverTransport.java:1040) - Connect fail to: tcp://localhost:61616?timeout=2000, reason: java.lang.IllegalArgumentException: Invalid connect parameters: {timeout=2000}
The timeout parameter is specified here http://activemq.apache.org/cms/configuring.html
Has anybody any idea how to pass timeout argument to ActiveMQConnectionFactory?
Or how to set a timeout for connection.start() ?
Thank you!
Update: I found this on Stackoverflow: ActiveMQ - CreateSession failover timeout after a connection is resumed . I tried it but the following exception is thrown:
javax.jms.JMSException: Could not create Transport. Reason: java.lang.IllegalArgumentException: Invalid connect parameters: {transport.timeout=5000}
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:35)
I use ActiveMQ 5.8.0 from maven repo
It appears that your url is invalid still in both cases when attempting to set the timeout property.
If you're trying to have a failover URL, which it looks like you are since it is getting in to the Failover code then you're probably looking for initialReconnectDelay (and possibly maxReconnectAttempts which would throw an exception if the server is still down after the number of attempts is reached).
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("failover://(tcp://localhost:61616)?initialReconnectDelay=2000&maxReconnectAttempts=2");
Related
I am writing a Java client to connect to my JBoss EAP 7.3 server running ActiveMQ, and I am getting various connection responses as I alter the parameters. Please help me correct the parameters/code. I get:
09:46:57.227 [main] INFO org.xnio.nio - XNIO NIO Implementation Version 3.4.6.Final
09:46:57.606 [Remoting "config-based-naming-client-endpoint" I/O-1] DEBUG org.xnio.nio - Started channel thread 'Remoting "config-based-naming-client-endpoint" I/O-1', selector sun.nio.ch.WindowsSelectorImpl#17ab1d7e ...
jboss.naming.client.connect.options. has the following options {}
09:46:57.763 [main] DEBUG org.jboss.naming.remote.client.HaRemoteNamingStore - Failed to connect to server http-remoting://127.0.0.1:8080
java.lang.RuntimeException: java.io.IOException: For now upgrade responses must have a content length of zero.
at org.jboss.naming.remote.protocol.IoFutureHelper.get(IoFutureHelper.java:95)
at org.jboss.naming.remote.client.HaRemoteNamingStore.failOverSequence(HaRemoteNamingStore.java:198)
... at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:146)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at com.goprecise.ams.demo.SendJmsToProcess.main(SendJmsToProcess.java:46)
Caused by: java.io.IOException: For now upgrade responses must have a content length of zero.
... at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:571)
... at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:335)
at org.jboss.naming.remote.client.EndpointCache$EndpointWrapper.connect(EndpointCache.java:122)
at org.jboss.naming.remote.client.HaRemoteNamingStore.failOverSequence(HaRemoteNamingStore.java:197)
... 8 common frames omitted
javax.naming.CommunicationException: Failed to connect to any server. Servers tried: [http-remoting://127.0.0.1:8080 (java.io.IOException: For now upgrade responses must have a content length of zero.)]
at org.jboss.naming.remote.client.HaRemoteNamingStore.failOverSequence(HaRemoteNamingStore.java:244)
at org.jboss.naming.remote.client.HaRemoteNamingStore.namingStore(HaRemoteNamingStore.java:149)
This is the Java client code in a try catch block attempting to connect:
Properties env = new Properties();
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
env.put(Context.PROVIDER_URL, "http-remoting://127.0.0.1:8080");
env.put(Context.SECURITY_PRINCIPAL,adminUser);
env.put(Context.SECURITY_CREDENTIALS, adminPassword);
Context namingContext = new InitialContext(env);
String CONNECTION_FACTORY = "java:jboss/exported/jms/RemoteConnectionFactory";
ConnectionFactory connectionFactory = (ConnectionFactory) namingContext.lookup(CONNECTION_FACTORY);
System.out.println("Got ConnectionFactory");
Destination destination = (Destination) namingContext.lookup(QUEUE); // Sure QUEUE is correct
System.out.println("Got JMS Endpoint " + QUEUE);
JMSContext context = connectionFactory.createContext(adminUser, adminPassword);
context.createProducer().send(destination, xmlContent);
System.out.println("Got JMS destination");
And these are my JNDI tree values in the EAP management console for java:jboss/exported >> JMS >>
URI java:jboss/exported/jms/RemoteConnectionFactory
Class Name org.apache.activemq.artemis.jms.client.ActiveMQJMSConnectionFactory
Value ActiveMQConnectionFactory [serverLocator=ServerLocatorImpl
[initialConnectors=[TransportConfiguration(name=http-connector,
factory=org-apache-activemq-artemis-core-remoting-impl-netty-
NettyConnectorFactory) ?httpUpgradeEndpoint=http-
acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=
8080&host=kubernetes-docker-internal], discoveryGroupConfiguration=null],
clientID=null, consumerWindowSize = 1048576, dupsOKBatchSize=1048576,
transactionBatchSize=1048576, readOnly=falseEnableSharedClientID=true]
It looks to me like you're using the wrong InitialContextFactory implementation. Try using org.wildfly.naming.client.WildFlyInitialContextFactory instead of org.jboss.naming.remote.client.InitialContextFactory.
You can find a full JMS client example for JBoss EAP 7.3 here.
Jms outbound-channel-adapter works perfectly fine but I see intermittently this error in the log, however, MQ message still gets delivered.
2019-06-07 10:16:22 [JMSCCThreadPoolWorker-5] INFO o.s.j.c.CachingConnectionFactory - Encountered a JMSException - resetting the underlying JMS Connection
com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ1107: A problem with this connection has occurred.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:578)
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:214)
at com.ibm.msg.client.wmq.internal.WMQConnection.consumer(WMQConnection.java:794)
at com.ibm.mq.jmqi.remote.api.RemoteHconn.callEventHandler(RemoteHconn.java:2903)
at com.ibm.mq.jmqi.remote.api.RemoteHconn.driveEventsEH(RemoteHconn.java:628)
at com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.processHconn(RemoteDispatchThread.java:691)
at com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.run(RemoteDispatchThread.java:233)
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:263)
at com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:99)
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:284)
at com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:312)
at com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1214)
Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
...This is errorChannel configuration:
<int:header-enricher id="errorMsg.HeaderEnricher"
input-channel="errorChannel"
output-channel="omniAlertsJmsErrorChannel">
...and jms outbound-channel-adapter configured:
<int-jms:outbound-channel-adapter
id="jmsOutToNE" channel="umpAlertNotificationJMSChannel"
destination="senderTopic"
jms-template="jmsQueueTemplate"
>
I am expecting omniAlertsJmsErrorChannel to receive the MessageHandlingException which is not happening from jmsOutToNE adapter. All other channel/flow errors are being routed to omniAlertsJmsErrorChannel.
Also, wondering if there jms outbound-channel-adapter is retrying internally when com.ibm.mq.MQException happens and it gets successful in the subsequent try?
The errorChannel is out of use in any MessageHandler. Their exception are just thrown to the caller. To handle those exception you need to consider adding a request-handler-advice-chain with the ExpressionEvaluatingRequestHandlerAdvice. Or if we talk about a retry, you also can add to that chain a RequestHandlerRetryAdvice.
See more info in the Reference Manual.
Not sure though why your message is still delivered to the MQ. There is no any retry out-of-the-box in the <int-jms:outbound-channel-adapter. It might a behavior of the MQ Connection Factory adapter in the IBM library.
I am getting sporadic errors from a java service that is listening to OracleAQ.
It seems to be happening each night, and I can't be sure what is going on. Could it really be a database connection problem ?
Or does the "Dequeue failed" suggest that it was connected and something else happened ?
Here is the exception below :
[2013-11-04 18:16:16,508] WARN org.springframework.jms.listener.DefaultMessageListenerContainer - Setup of JMS message listener invoker failed for destination 'MYCOMPANY_INFO_QUEUE' - trying to recover. Cause: JMS-120: Dequeue failed; nested exception is java.sql.SQLException: Io exception: Socket read timed out
oracle.jms.AQjmsException: JMS-120: Dequeue failed
at oracle.jms.AQjmsError.throwEx(AQjmsError.java:311)
at oracle.jms.AQjmsConsumer.dequeue(AQjmsConsumer.java:2234)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:1028)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:951)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:929)
at oracle.jms.AQjmsConsumer.receive(AQjmsConsumer.java:781)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:430)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1096)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1088)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:985)
at java.lang.Thread.run(Thread.java:662)
[Linked-exception]
java.sql.SQLException: Io exception: Socket read timed out
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255)
at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:976)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1168)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3285)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3329)
at oracle.jms.AQjmsConsumer.dequeue(AQjmsConsumer.java:1732)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:1028)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:951)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:929)
at oracle.jms.AQjmsConsumer.receive(AQjmsConsumer.java:781)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:430)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1096)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1088)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:985)
at java.lang.Thread.run(Thread.java:662)
[2013-11-04 18:16:16,569] INFO org.springframework.jms.listener.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
The jms receive timeout should be configured in seconds (while the db timeout is in milliseconds). So make sure your jms value is less. For example, here is my working spring config:
<bean id="xxxJmsTemplate" class="org.springframework.jms.core.JmsTemplate102">
<property name="connectionFactory" ref="xxxJmsConnectionFactory"/>
<property name="defaultDestinationName" value="some_queue"/>
<property name="receiveTimeout" value="10"/><!-- seconds -->
</bean>
PS: The special Spring constant RECEIVE_TIMEOUT_NO_WAIT value of -1 does not seem to work for this setting. But setting a reasonably short time in seconds should do the trick.
I suggest looking at your dequeue options for wait time.
import oracle.AQ.AQDequeueOption;
...
AQDequeueOption options = null;
options = new AQDequeueOption();
options.setWaitTime(AQDequeue.WAIT_NONE);
//WAIT_NONE = do not wait if messages are not available
//WAIT_FOREVER = waits "forever"; default value
...
The WAIT_FOREVER setting is default and will wait until a message is available on the queue; however this holds the database connection open.
I believe this is the reason why you experience the errors "sporadically"; most of the time messages are being enqueued and running smoothly; and then when messages are not enqueued (each night) your database connection is timing out.
I'm trying to connect to remote HDFS cluster. I've read some documentation and getting started's but didn't find a best solution how to do that.
Situation: I have HDFS on xxx-something.com. I can connect to it via SSH and everything works.
But what I'm trying to do, get the files from it to my local machine.
What I've done:
I've created core-site.xml in my conf folder (I'm creating Play! application). There I've changed fs.default.name config to hdfs://xxx-something.com:8020 (not sure about the port).
Then I'm trying to launch a simple test:
val conf = new Configuration()
conf.addResource(new Path("conf/core-site.xml"))
val fs = FileSystem.get(conf)
val status = fs.listStatus(new Path("/data/"))
And I'm getting errors:
13:56:09.012 [specs2.DefaultExecutionStrategy1] WARN org.apache.hadoop.conf.Configuration - conf/core-site.xml:a attempt to override final parameter: fs.trash.interval; Ignoring.
13:56:09.012 [specs2.DefaultExecutionStrategy1] WARN org.apache.hadoop.conf.Configuration - conf/core-site.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
13:56:09.013 [specs2.DefaultExecutionStrategy1] WARN org.apache.hadoop.conf.Configuration - conf/core-site.xml:a attempt to override final parameter: fs.checkpoint.dir; Ignoring.
13:56:09.022 [specs2.DefaultExecutionStrategy1] DEBUG org.apache.hadoop.fs.FileSystem - Creating filesystem for hdfs://xxx-something.com:8021
13:56:09.059 [specs2.DefaultExecutionStrategy1] DEBUG org.apache.hadoop.conf.Configuration - java.io.IOException: config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:226)
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:213)
at org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:53)
at org.apache.hadoop.net.NetUtils.<clinit>(NetUtils.java:62)
Thanks in advance!
Update:
probably the port was wrong. Now I set it to 22, I'm still getting same errors, but after 3 times it does say:
14:01:01.877 [specs2.DefaultExecutionStrategy1] DEBUG org.apache.hadoop.ipc.Client - Connecting to xxx-something.com/someIp:22
14:01:02.187 [specs2.DefaultExecutionStrategy1] DEBUG org.apache.hadoop.ipc.Client - IPC Client (47) connection to xxx-something.com/someIp:22 from britva sending #0
14:01:02.188 [IPC Client (47) connection to xxx-something.com/someIp:22 from britva] DEBUG org.apache.hadoop.ipc.Client - IPC Client (47) connection to xxx-something.com/someIp:22 from britva: starting, having connections 1
14:01:02.422 [IPC Client (47) connection to xxx-something.com/someIp:22 from britva] DEBUG org.apache.hadoop.ipc.Client - IPC Client (47) connection to xxx-something.com/someIp:22 from britva got value #1397966893
And afterwards:
Call to xxx-something.com/someIp:22 failed on local exception: java.io.EOFException
java.io.IOException: Call to xxx-something.com/someIp:22 failed on local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
at org.apache.hadoop.ipc.Client.call(Client.java:1071)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at com.sun.proxy.$Proxy1.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:118)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:222)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:187)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1328)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:65)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1346)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:244)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:122)
at HdfsSpec$$anonfun$1$$anonfun$apply$3.apply(HdfsSpec.scala:33)
at HdfsSpec$$anonfun$1$$anonfun$apply$3.apply(HdfsSpec.scala:17)
at testingSupport.specs2.MyNotifierRunner$$anon$2$$anon$1.executeBody(MyNotifierRunner.scala:16)
at testingSupport.specs2.MyNotifierRunner$$anon$2$$anon$1.execute(MyNotifierRunner.scala:16)
Caused by: java.io.EOFException
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:807)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
What does it mean?
You'll need to find the fs.default.name property in the $HADOOP_HOME/conf/core-site.xml on the server running the Name Node (HDFS master) to get the correct port. It might be 8020, or it could be something else. That's what you should use. Make sure there's no firewall between you and the server that disallows connections on the port.
I am trying to catch exception for an ActiveMQ connection which could not be established because of the broker being down.
With following code:
String url = ActiveMQConnection.DEFAULT_BROKER_URL;
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
connection.start();
The attempt to connect to the broker goes into infinite loop, if the broker is down. If I change url to
String url = "failover:(tcp://127.0.0.1:61616/)?startupMaxReconnectAttempts=2";
It makes 2 attempts and then throws an exception.(which is what I want.)
Now if I initialize the connection object using Spring Bean with the following:
<bean id="jmsFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<!--<value>tcp://0.0.0.0:61616</value>-->
<value>failover:(tcp://127.0.0.1:61616/)?startupMaxReconnectAttempts=2</value>
</property>
</bean>
I get an error message for failure to connect in 2 attempts but then, it still tries to connect again after every 5 seconds giving the same error message again and going on in infinite loop.
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
these messages repeat!!
I want to know how to stop this infinite polling and catch an exception (may be using PostInit) in case of failure.
You could implement ExceptionListener in your listener class and override onException message. There you could handle your notification logic. While it try to reconnect automatically.
public class QueueListener implements MessageListener,ExceptionListener{
public void onMessage(Message message) {
}
public void onException(JMSException jsme) {
// Send my notifcation here.
}
}