I am trying to catch exception for an ActiveMQ connection which could not be established because of the broker being down.
With following code:
String url = ActiveMQConnection.DEFAULT_BROKER_URL;
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
connection.start();
The attempt to connect to the broker goes into infinite loop, if the broker is down. If I change url to
String url = "failover:(tcp://127.0.0.1:61616/)?startupMaxReconnectAttempts=2";
It makes 2 attempts and then throws an exception.(which is what I want.)
Now if I initialize the connection object using Spring Bean with the following:
<bean id="jmsFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<!--<value>tcp://0.0.0.0:61616</value>-->
<value>failover:(tcp://127.0.0.1:61616/)?startupMaxReconnectAttempts=2</value>
</property>
</bean>
I get an error message for failure to connect in 2 attempts but then, it still tries to connect again after every 5 seconds giving the same error message again and going on in infinite loop.
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
these messages repeat!!
I want to know how to stop this infinite polling and catch an exception (may be using PostInit) in case of failure.
You could implement ExceptionListener in your listener class and override onException message. There you could handle your notification logic. While it try to reconnect automatically.
public class QueueListener implements MessageListener,ExceptionListener{
public void onMessage(Message message) {
}
public void onException(JMSException jsme) {
// Send my notifcation here.
}
}
Related
I'm using the HiveMQ library in my Java Spring application to connect to a Mosquitto instance as I find it more user-friendly compared to the Paho client. But something is going wrong with the automatic reconnection. From time to time the connection is lost and the application doesn't succeed in reconnecting (see logs 1). This can also be triggered by restarting the Mosquitto broker itself (see logs 2).
This is my client builder code with additional logging in the disconnect to check if the credentials are still correct:
client = MqttClient.builder()
.useMqttVersion5()
.identifier(identifier)
.serverHost(host)
.serverPort(port)
.sslWithDefaultConfig()
// https://www.hivemq.com/blog/hivemq-mqtt-client-features/reconnect-handling/
.automaticReconnectWithDefaultConfig()
.addDisconnectedListener(context -> logger.error("MQTT user {} with identifier {} on {}:{} has disconnected reason: {}",
username, identifier, host, port, context.getCause().getMessage()))
.buildAsync();
client.connectWith()
.simpleAuth()
.username(username)
.password(password.getBytes())
.applySimpleAuth()
.cleanStart(false)
.keepAlive(60)
.send();
1/ This is shown in my logs after the connection has been lost by the application itself:
2022-03-16 02:10:33.502 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: Timeout while waiting for PINGRESP
2022-03-16 02:11:25.090 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: CONNECT failed as CONNACK contained an Error Code: NOT_AUTHORIZED.
2022-03-16 02:12:27.200 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: Timeout while waiting for CONNACK
2/ This is the logs after the broker has been restarted, some expected time-outs, but also in the end "not authorized":
2022-03-16 10:17:37.178 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: Server closed connection without DISCONNECT.
2022-03-16 10:17:48.441 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: io.netty.channel.ConnectTimeoutException: connection timed out: ***/***:8883
2022-03-16 10:18:00.747 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: io.netty.channel.ConnectTimeoutException: connection timed out: ***/***:8883
2022-03-16 10:18:10.625 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: No route to host: ***/***:8883
2022-03-16 10:18:26.845 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: io.netty.channel.ConnectTimeoutException: connection timed out: ***/***:8883
2022-03-16 10:18:42.584 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: CONNECT failed as CONNACK contained an Error Code: NOT_AUTHORIZED.
In both cases, the connection is back to normal with an application restart.
any ideas?
It appears that your question is answered in this issue:
If you set the username and password on the connect call, they will not be stored and reused when the client reconnects (for security reasons).
The following code (from the issue linked above) demonstrates the approach:
Mqtt3Client.builder()
.identifier("ePCR mobile-" + currentTimeMillis())
.serverHost(config.getHost())
.serverPort(config.getPort())
.automaticReconnectWithDefaultConfig()
.simpleAuth()
.username(config.getUsername())
.password(config.getPassword())
.applySimpleAuth()
.buildRx();
full code
public class SolrToMongodb {
private static final Logger LOGGER = LoggerFactory.getLogger(SolrToMongodb.class);
public static void main(String[] args) throws IOException, SolrServerException {
SolrToMongodb main = new SolrToMongodb();
main.run();
}
public void run() throws IOException, SolrServerException {
SparkConfig config = new SparkConfig();
JavaSparkContext jsc = new JavaSparkContext(config.sparkConf("admiralty-stream"));
SolrClient client = new HttpSolrClient(Constant.SOLR_STREAMING);
SolrQuery q = new SolrQuery();
q.set("q","*:*");
q.set("indent","on");
q.set("wt", "json");
client.query(q);
try {
CloudSolrClient cloudSolrClient = new CloudSolrClient(Constant.ZOOKEEPER_SOLR);
SolrJavaRDD solrRDD = SolrJavaRDD.get(cloudSolrClient.getZkHost(), "admiraltyStream", jsc.sc());
JavaRDD<SolrDocument> resultsRDD = solrRDD.queryShards(q);
JavaRDD<Object> objectJavaRDD = resultsRDD.map(new Function<SolrDocument, Object>() {
#Override
public Object call(SolrDocument v1) throws Exception {
System.out.println(v1.getFieldValueMap());
return v1.getFieldValueMap();
}
});
}
catch (Exception e){
System.out.println("Exception here : "+e.getMessage());
}
}}
ERROR LOG :
2017-08-02 10:02:58,709 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 0
2017-08-02 10:02:59,688 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 1
2017-08-02 10:03:01,630 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 2
2017-08-02 10:03:02,579 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 3
2017-08-02 10:03:03,540 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 4
2017-08-02 10:03:04,484 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 5
Exception :
Exception here : No live SolrServers available to handle this request:[http://xxx.xxx.ph:8983/solr/admiraltyStream, http://xxx.xxx.ph:8983/solr/admiraltyStream, http://xxx.xxx.ph:8983/solr/admiraltyStream]
Using the CloudSolrClient instead of HttpSolrClient allows solrj to do a round-robin load balancing between the available solr servers, and of course is recommended in a SolrCloud context. The "No live SolrServers” message indicates a problem with the collection admiraltyStream.
Specifically, behind the scenes, SolrJ is using LBHttpSolrClient (which is using a set of HttpSolrClient instances) for round-robin requests between shards. I think that your problem is actually this: some shard is not available (i.e. the leader and the replicas).
I would review the currently online replicas (http://solr.server:8983/#/~cloud): there you should see if all the replicas for your collection online.
There must be at least one replica per shard; and I think that in your case:
the node you’re trying to connect directly using HttpSolrClient is up and running
when using CloudSolrClient (i.e. Zookeeper -> Solr), there’s something wrong in your cluster state: Solr believes that there are no replicas for at least one shard, so the list of the available HttpSolrClient instances within a given instance of LBHttpSolrClient is empty
I have the following problem:
I try to connect to an ActiveMQ broker (which is now down) using the following piece of code
connectionFactory = new ActiveMQConnectionFactory(this.url + "?timeout=2000");
connection = connectionFactory.createConnection();
connection.start();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
LOGGER.info("Connected to " + this.url);
The problem is that the timeout does not have any effect
connection.start()
is blocked forever.
I inspected ActiveMQ log and found the following info:
2013-12-20 01:49:03,149 DEBUG [ActiveMQ Task-1] (FailoverTransport.java:786) - urlList connectionList:[tcp://localhost:61616?timeout=2000], from: [tcp://localhost:61616?timeout=2000]
2013-12-20 01:49:03,149 DEBUG [ActiveMQ Task-1] (FailoverTransport.java:1040) - Connect fail to: tcp://localhost:61616?timeout=2000, reason: java.lang.IllegalArgumentException: Invalid connect parameters: {timeout=2000}
The timeout parameter is specified here http://activemq.apache.org/cms/configuring.html
Has anybody any idea how to pass timeout argument to ActiveMQConnectionFactory?
Or how to set a timeout for connection.start() ?
Thank you!
Update: I found this on Stackoverflow: ActiveMQ - CreateSession failover timeout after a connection is resumed . I tried it but the following exception is thrown:
javax.jms.JMSException: Could not create Transport. Reason: java.lang.IllegalArgumentException: Invalid connect parameters: {transport.timeout=5000}
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:35)
I use ActiveMQ 5.8.0 from maven repo
It appears that your url is invalid still in both cases when attempting to set the timeout property.
If you're trying to have a failover URL, which it looks like you are since it is getting in to the Failover code then you're probably looking for initialReconnectDelay (and possibly maxReconnectAttempts which would throw an exception if the server is still down after the number of attempts is reached).
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("failover://(tcp://localhost:61616)?initialReconnectDelay=2000&maxReconnectAttempts=2");
I am getting sporadic errors from a java service that is listening to OracleAQ.
It seems to be happening each night, and I can't be sure what is going on. Could it really be a database connection problem ?
Or does the "Dequeue failed" suggest that it was connected and something else happened ?
Here is the exception below :
[2013-11-04 18:16:16,508] WARN org.springframework.jms.listener.DefaultMessageListenerContainer - Setup of JMS message listener invoker failed for destination 'MYCOMPANY_INFO_QUEUE' - trying to recover. Cause: JMS-120: Dequeue failed; nested exception is java.sql.SQLException: Io exception: Socket read timed out
oracle.jms.AQjmsException: JMS-120: Dequeue failed
at oracle.jms.AQjmsError.throwEx(AQjmsError.java:311)
at oracle.jms.AQjmsConsumer.dequeue(AQjmsConsumer.java:2234)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:1028)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:951)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:929)
at oracle.jms.AQjmsConsumer.receive(AQjmsConsumer.java:781)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:430)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1096)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1088)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:985)
at java.lang.Thread.run(Thread.java:662)
[Linked-exception]
java.sql.SQLException: Io exception: Socket read timed out
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255)
at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:976)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1168)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3285)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3329)
at oracle.jms.AQjmsConsumer.dequeue(AQjmsConsumer.java:1732)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:1028)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:951)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:929)
at oracle.jms.AQjmsConsumer.receive(AQjmsConsumer.java:781)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:430)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1096)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1088)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:985)
at java.lang.Thread.run(Thread.java:662)
[2013-11-04 18:16:16,569] INFO org.springframework.jms.listener.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
The jms receive timeout should be configured in seconds (while the db timeout is in milliseconds). So make sure your jms value is less. For example, here is my working spring config:
<bean id="xxxJmsTemplate" class="org.springframework.jms.core.JmsTemplate102">
<property name="connectionFactory" ref="xxxJmsConnectionFactory"/>
<property name="defaultDestinationName" value="some_queue"/>
<property name="receiveTimeout" value="10"/><!-- seconds -->
</bean>
PS: The special Spring constant RECEIVE_TIMEOUT_NO_WAIT value of -1 does not seem to work for this setting. But setting a reasonably short time in seconds should do the trick.
I suggest looking at your dequeue options for wait time.
import oracle.AQ.AQDequeueOption;
...
AQDequeueOption options = null;
options = new AQDequeueOption();
options.setWaitTime(AQDequeue.WAIT_NONE);
//WAIT_NONE = do not wait if messages are not available
//WAIT_FOREVER = waits "forever"; default value
...
The WAIT_FOREVER setting is default and will wait until a message is available on the queue; however this holds the database connection open.
I believe this is the reason why you experience the errors "sporadically"; most of the time messages are being enqueued and running smoothly; and then when messages are not enqueued (each night) your database connection is timing out.
Hi dear community of java addicts.
I was getting these exceptions in a CentOs VM, probably running with low RAM and then I noted that the time was not correctly synchronized between the other VM needed to communicate with my nice component....
I was wondering to know, When ? Why ? How ? A SocketException: end of file is produced in a LINUX server...
These are my logs:
2012-05-16 13:22:41,863 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][initDatabaseProperties] - Initializing database custom properties.
2012-05-16 13:22:41,864 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][initDatabaseProperties] - Setting NLS_DATE_FORMAT to : DD/MM/YYYY HH24:MI:SS
2012-05-16 13:22:47,096 [Timer-2] ERROR [org.jboss.remoting.transport.socket.SocketClientInvoker][handleException] - Got marshalling exception, exiting
java.net.SocketException: end of file
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:685)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.ConnectionValidator.doCheckConnectionWithoutLease(ConnectionValidator.java:828)
at org.jboss.remoting.ConnectionValidator.run(ConnectionValidator.java:345)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
2012-05-16 13:22:47,288 [Thread-2624] WARN [org.jboss.remoting.Client][removeListener] - unable to remove remote callback handler: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
2012-05-16 13:22:47,329 [Thread-2625] WARN [org.jboss.remoting.Client][removeListener] - unable to remove remote callback handler: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
2012-05-16 13:22:51,146 [Timer-4] WARN [org.jboss.remoting.transport.bisocket.BisocketServerInvoker][run] - org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask#7a7385ac: detected failure on control connection Thread[control: Socket[addr=ams-dev-bo.swissbytes.ch/192.168.0.190,port=11641,localport=57623],5,] (5c4o020-jlorp4-h29d35xs-1-h2aawvkq-l2t: requesting new control connection
2012-05-16 13:22:51,159 [controlConnectionRecreate:control: Socket[addr=ams-dev-bo.swissbytes.ch/192.168.0.190,port=11641,localport=57623]] ERROR [org.jboss.remoting.transport.bisocket.BisocketServerInvoker][createControlConnection] - unable to get secondary locator
org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:613)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.getSecondaryLocator(BisocketClientInvoker.java:640)
at org.jboss.remoting.transport.bisocket.BisocketServerInvoker.createControlConnection(BisocketServerInvoker.java:230)
at org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask$1.run(BisocketServerInvoker.java:1048)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:192)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:465)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:913)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:602)
... 5 more
2012-05-16 13:22:51,161 [controlConnectionRecreate:control: Socket[addr=ams-dev-bo.swissbytes.ch/192.168.0.190,port=11641,localport=57623]] ERROR [org.jboss.remoting.transport.bisocket.BisocketServerInvoker][run] - Unable to recreate control connection: InvokerLocator [null://ams-dev-bo.swissbytes.ch:11641/null]
java.io.IOException: unable to get secondary locator: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
at org.jboss.remoting.transport.bisocket.BisocketServerInvoker.createControlConnection(BisocketServerInvoker.java:235)
at org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask$1.run(BisocketServerInvoker.java:1048)
2012-05-16 13:22:56,870 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][runFilterRec] - Query for the level[1] was executed.
2012-05-16 13:22:56,871 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][doFilterResultsCotainCipAdress] - doFilterResultsCotainCipAdress() - Searching Cip Address: 1213194
2012-05-16 13:22:56,871 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][runFilterRec] - cipAddress: 1213194 was NOT FOUND in Filter Results
2012-05-16 13:22:56,871 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][runFilterRec] - Current result size is not on filter's range : filter1 => [ 1 , 10 ] vs 0
2012-05-16 13:22:56,871 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.jms.request.SimilarDebtorsProcessor][delegateFixedResult] - Fixed result GET_SIMILAR_DEBTORS totalSize: 1 -> fixedSize: 1
2012-05-16 13:22:56,872 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.util.ListFragmenter][fragmentList] - Preparing [0] fragments in chunks of size [500]
2012-05-16 13:22:56,872 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.util.ListFragmenter][fragmentList] - List fragment range[0-1]
2012-05-16 13:22:56,872 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.jms.request.SimilarDebtorsProcessor][processFragment] - Sending 'SimilarDebtors' message fragment[0-1]
2012-05-16 13:22:56,980 [jmsContainer-1] ERROR [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][onMessage] - JMSException caused by Message[ID:JBM-5a9ac639-f2a4-436c-8170-37378d8b606b], somenthing is wrong with the communication.
2012-05-16 13:22:56,982 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][onMessage] - Queue listener will be stopped.
2012-05-16 13:22:56,983 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][stopListener] - Listener successfully stopped
2012-05-16 13:22:56,984 [jmsContainer-1] DEBUG [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][errorTemplateSend] - Trying to send error message to AMS-WA error queue.
2012-05-16 13:22:56,985 [jmsContainer-1] DEBUG [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][errorTemplateSend] - Adapter id: 1
2012-05-16 13:22:57,014 [jmsContainer-1] ERROR [org.jboss.jms.client.container.ClosedInterceptor][invoke] - ClosedInterceptor.ClientSessionDelegate[ioy8-vyl6fa2h-1-hbl1g92h-qrmrca-a50o4c5]: method getTransacted() did not go through, the interceptor is CLOSED
2012-05-16 13:22:57,016 [jmsContainer-1] ERROR [org.springframework.jms.listener.DefaultMessageListenerContainer][rollbackOnExceptionIfNecessary] - Application exception overridden by rollback exception
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is org.jboss.jms.exception.MessagingNetworkFailureException; nested exception is org.jboss.remoting.CannotConnectException: Error setting up client lease upon performing connect.
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:292)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:474)
at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:548)
at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
at ch.swissbytes.cipadapter.jms.request.CipAdapterBean.errorTemplateSend(CipAdapterBean.java:226)
at ch.swissbytes.cipadapter.jms.request.CipAdapterBean.onMessage(CipAdapterBean.java:160)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:506)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:463)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:240)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.jboss.jms.exception.MessagingNetworkFailureException
at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:240)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:198)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.container.StateCreationAspect.handleCreateConnectionDelegate(StateCreationAspect.java:80)
at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect0.invoke(StateCreationAspect0.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.createConnectionDelegate(ClientConnectionFactoryDelegate.java)
at org.jboss.jms.client.JBossConnectionFactory.createConnectionInternal(JBossConnectionFactory.java:205)
at org.jboss.jms.client.JBossConnectionFactory.createConnection(JBossConnectionFactory.java:87)
at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
at $Proxy0.createConnection(Unknown Source)
at org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.doCreateConnection(UserCredentialsConnectionFactoryAdapter.java:174)
at org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.createConnection(UserCredentialsConnectionFactoryAdapter.java:149)
at org.springframework.jms.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:316)
at org.springframework.jms.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:270)
at org.springframework.jms.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:215)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:461)
... 12 more
Caused by: org.jboss.remoting.CannotConnectException: Error setting up client lease upon performing connect.
at org.jboss.remoting.Client.connect(Client.java:1804)
at org.jboss.remoting.Client.connect(Client.java:652)
at org.jboss.jms.client.remoting.JMSRemotingConnection$1.run(JMSRemotingConnection.java:374)
at java.security.AccessController.doPrivileged(Native Method)
at org.jboss.jms.client.remoting.JMSRemotingConnection.start(JMSRemotingConnection.java:368)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:165)
... 32 more
Caused by: java.lang.Exception: Error setting up client lease
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:508)
at org.jboss.remoting.Client.setupClientLease(Client.java:1912)
at org.jboss.remoting.Client.connect(Client.java:1800)
... 37 more
Caused by: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:613)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:474)
... 39 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:192)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:465)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:913)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:602)
... 42 more
2012-05-16 13:22:57,031 [jmsContainer-1] WARN [org.springframework.jms.listener.DefaultMessageListenerContainer][handleListenerException] - Execution of JMS message listener failed
javax.jms.IllegalStateException: The object is closed
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:157)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
at org.jboss.jms.client.delegate.ClientSessionDelegate$getTransacted_N1613179584734032131.invokeNext(ClientSessionDelegate$getTransacted_N1613179584734032131.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate.getTransacted(ClientSessionDelegate.java)
at org.jboss.jms.client.JBossSession.getTransacted(JBossSession.java:154)
at org.springframework.jms.listener.AbstractMessageListenerContainer.rollbackOnExceptionIfNecessary(AbstractMessageListenerContainer.java:574)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:442)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:240)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:662)
2012-05-16 13:22:57,072 [jmsContainer-1] WARN [org.jboss.remoting.Client][removeListener] - unable to remove remote callback handler: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
2012-05-16 13:23:35,981 [org.springframework.scheduling.timer.TimerFactoryBean#0] DEBUG [ch.swissbytes.cipadapter.services.tasks.CheckQueueListenerStatus][run] - Checking queueListener status
And error was originated in this method:
private void initDatabaseProperties(final Session session) {
logger.info("Initializing database custom properties.");
properties.getProperty(CommonConstants.CIP_DATE_FORMAT_PROP);
final String dateFormat = DateUtil.VIEW_DATE_FORMAT;
logger.info("Setting NLS_DATE_FORMAT to : " + dateFormat);
final String queryString = "ALTER SESSION SET NLS_DATE_FORMAT = '" + StringUtils.trim(dateFormat) + "'";
session.createSQLQuery(queryString).executeUpdate();
}
I can't answer for what exceptions JBoss throws, but in general EOS on a socket is caused by exactly one thing: receiving a FIN from the peer as the result of a close or shutdown output by the peer.