I'm using the HiveMQ library in my Java Spring application to connect to a Mosquitto instance as I find it more user-friendly compared to the Paho client. But something is going wrong with the automatic reconnection. From time to time the connection is lost and the application doesn't succeed in reconnecting (see logs 1). This can also be triggered by restarting the Mosquitto broker itself (see logs 2).
This is my client builder code with additional logging in the disconnect to check if the credentials are still correct:
client = MqttClient.builder()
.useMqttVersion5()
.identifier(identifier)
.serverHost(host)
.serverPort(port)
.sslWithDefaultConfig()
// https://www.hivemq.com/blog/hivemq-mqtt-client-features/reconnect-handling/
.automaticReconnectWithDefaultConfig()
.addDisconnectedListener(context -> logger.error("MQTT user {} with identifier {} on {}:{} has disconnected reason: {}",
username, identifier, host, port, context.getCause().getMessage()))
.buildAsync();
client.connectWith()
.simpleAuth()
.username(username)
.password(password.getBytes())
.applySimpleAuth()
.cleanStart(false)
.keepAlive(60)
.send();
1/ This is shown in my logs after the connection has been lost by the application itself:
2022-03-16 02:10:33.502 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: Timeout while waiting for PINGRESP
2022-03-16 02:11:25.090 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: CONNECT failed as CONNACK contained an Error Code: NOT_AUTHORIZED.
2022-03-16 02:12:27.200 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: Timeout while waiting for CONNACK
2/ This is the logs after the broker has been restarted, some expected time-outs, but also in the end "not authorized":
2022-03-16 10:17:37.178 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: Server closed connection without DISCONNECT.
2022-03-16 10:17:48.441 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: io.netty.channel.ConnectTimeoutException: connection timed out: ***/***:8883
2022-03-16 10:18:00.747 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: io.netty.channel.ConnectTimeoutException: connection timed out: ***/***:8883
2022-03-16 10:18:10.625 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: No route to host: ***/***:8883
2022-03-16 10:18:26.845 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: io.netty.channel.ConnectTimeoutException: connection timed out: ***/***:8883
2022-03-16 10:18:42.584 ERROR 1 --- [client.mqtt-1-2] MqttConfig : MQTT user *** with identifier SERVICE on ***:8883 has disconnected reason: CONNECT failed as CONNACK contained an Error Code: NOT_AUTHORIZED.
In both cases, the connection is back to normal with an application restart.
any ideas?
It appears that your question is answered in this issue:
If you set the username and password on the connect call, they will not be stored and reused when the client reconnects (for security reasons).
The following code (from the issue linked above) demonstrates the approach:
Mqtt3Client.builder()
.identifier("ePCR mobile-" + currentTimeMillis())
.serverHost(config.getHost())
.serverPort(config.getPort())
.automaticReconnectWithDefaultConfig()
.simpleAuth()
.username(config.getUsername())
.password(config.getPassword())
.applySimpleAuth()
.buildRx();
Related
I'm running my application and Neo4j 4 in Docker compose environment. After starting my application I'm getting some weird logs that connection pool is closing connection to DB (Closing connection pool towards graphdb(172.21.0.4):7687) and after this neo4jClient is unable to query DB (logs below). What is the reason of this behaviour?
BTW. I created Spring Health Check (driver.verifyConnectivity()), but it always return OK (no error is thrown).
Any ideas?
#Configuration
class Neo4jConfiguration {
private val logger = LoggerFactory.getLogger(Neo4jConfiguration::class.java)
#Bean
fun neo4jDriver(
#Value("\${spring.data.neo4j.host}") host: String?,
#Value("\${spring.data.neo4j.port}") port: Int?): Driver {
val connectionUrl = "neo4j://$host:$port"
logger.info("Connecting to Neo4j on `$connectionUrl`")
return GraphDatabase.driver(connectionUrl/*, AuthTokens.basic("neo4j", "secret")*/)
}
#Bean
fun neo4jClient(): ReactiveNeo4jClient = ReactiveNeo4jClient.create(neo4jDriver(null, null))
#Bean
fun neo4jTransactionManager() = ReactiveNeo4jTransactionManager(neo4jDriver(null, null))
}
Docker compose:
version: '3.7'
services:
graphdb:
image: neo4j:4.0.0
ports:
- 7474:7474
- 7687:7687
environment:
NEO4J_AUTH: none
NEO4J_dbms_connectors_default__listen__address: 0.0.0.0
volumes:
- ./docker/neo4j/data:/data
networks:
- things
networks:
things:
name: things
Full logs:
2020-02-24 20:57:32.922 INFO 1 --- [ restartedMain] c.t.r.repo.neo4j.Neo4jConfiguration : Connecting to Neo4j on `neo4j://graphdb:7687`
2020-02-24 20:57:33.321 INFO 1 --- [ restartedMain] Driver : Routing driver instance 656417291 created for server address graphdb:7687
2020-02-24 20:57:43.329 INFO 1 --- [o4jDriverIO-2-3] LoadBalancer : Routing table for database 'system' is stale. Ttl 1582577863326, currentTime 1582577863328, routers AddressSet=[], writers AddressSet=[], readers AddressSet=[], database 'system'
2020-02-24 20:57:43.437 INFO 1 --- [o4jDriverIO-2-2] ConnectionPool : Closing connection pool towards graphdb(172.21.0.4):7687, it has no active connections and is not in the routing table registry.
2020-02-24 20:57:43.440 INFO 1 --- [o4jDriverIO-2-2] LoadBalancer : Updated routing table for database 'system'. Ttl 1582578163422, currentTime 1582577863439, routers AddressSet=[0.0.0.0:7687], writers AddressSet=[0.0.0.0:7687], readers AddressSet=[0.0.0.0:7687], database 'system'
2020-02-24 20:58:02.694 INFO 1 --- [ault-executor-1] LoadBalancer : Routing table for database '<default database>' is stale. Ttl 1582577882693, currentTime 1582577882694, routers AddressSet=[], writers AddressSet=[], readers AddressSet=[], database '<default database>'
2020-02-24 20:58:02.777 INFO 1 --- [o4jDriverIO-2-2] ConnectionPool : Closing connection pool towards graphdb(172.21.0.4):7687, it has no active connections and is not in the routing table registry.
2020-02-24 20:58:02.777 INFO 1 --- [o4jDriverIO-2-2] LoadBalancer : Updated routing table for database '<default database>'. Ttl 1582578182776, currentTime 1582577882777, routers AddressSet=[0.0.0.0:7687], writers AddressSet=[0.0.0.0:7687], readers AddressSet=[0.0.0.0:7687], database '<default database>'
2020-02-24 20:58:02.803 WARN 1 --- [o4jDriverIO-2-2] LoadBalancer : Failed to obtain a connection towards address 0.0.0.0:7687
org.neo4j.driver.exceptions.SessionExpiredException: Server at 0.0.0.0:7687 is no longer available
at org.neo4j.driver.internal.cluster.loadbalancing.LoadBalancer.lambda$acquire$9(LoadBalancer.java:204) ~[neo4j-java-driver-4.0.0.jar:4.0.0-d03d93ede8ad65657eeb90ed890757203ecfaa7a]
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(Unknown Source) ~[na:na]
[... removed ...]
Caused by: org.neo4j.driver.exceptions.ServiceUnavailableException: Unable to connect to 0.0.0.0:7687, ensure the database is running and that there is a working network connection to it.
at org.neo4j.driver.internal.async.connection.ChannelConnectedListener.databaseUnavailableError(ChannelConnectedListener.java:76) ~[neo4j-java-driver-4.0.0.jar:4.0.0-d03d93ede8ad65657eeb90ed890757203ecfaa7a]
at org.neo4j.driver.internal.async.connection.ChannelConnectedListener.operationComplete(ChannelConnectedListener.java:70) ~[neo4j-java-driver-4.0.0.jar:4.0.0-d03d93ede8ad65657eeb90ed890757203ecfaa7a]
at org.neo4j.driver.internal.async.connection.ChannelConnectedListener.operationComplete(ChannelConnectedListener.java:37) ~[neo4j-java-driver-4.0.0.jar:4.0.0-d03d93ede8ad65657eeb90ed890757203ecfaa7a]
[... removed ...]
... 7 common frames omitted
Caused by: org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /0.0.0.0:7687
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:na]
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) ~[na:na]
[... removed ...]
I have just come across the same problem, after a little experimenting turns out that the ip should be "bolt://$host:$port" and not "neo4j://$host:$port"
Seems like some of the spring.io tutorials are out of date.
I have a local two instance Solr Cloud setup with a single zookeeper instance. I am trying to connect via SolrJ to execute a query however my code hangs for 2mins or so when executing the query and then fails. I have followed the basic example on the Solr wiki. The logs/code is below
2016-07-24 13:29:01.932 INFO 83666 --- [qtp699221219-28] org.apache.zookeeper.ZooKeeper : Initiating client connection, connectString=localhost:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$3#496eab9
2016-07-24 13:29:01.948 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ConnectionManager : Waiting for client to connect to ZooKeeper
2016-07-24 13:29:01.953 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-07-24 13:29:01.955 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Socket connection established to localhost/127.0.0.1:2181, initiating session
2016-07-24 13:29:01.967 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1561cdd875e0004, negotiated timeout = 10000
2016-07-24 13:29:01.972 INFO 83666 --- [back-3-thread-1] o.a.solr.common.cloud.ConnectionManager : Watcher org.apache.solr.common.cloud.ConnectionManager#4bb95d56 name:ZooKeeperConnection Watcher:localhost:2181 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
2016-07-24 13:29:01.972 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ConnectionManager : Client is connected to ZooKeeper
2016-07-24 13:29:01.973 INFO 83666 --- [qtp699221219-28] o.apache.solr.common.cloud.SolrZkClient : Using default ZkACLProvider
2016-07-24 13:29:01.974 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Updating cluster state from ZooKeeper...
2016-07-24 13:29:01.990 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Loaded empty cluster properties
2016-07-24 13:29:01.995 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Updated live nodes from ZooKeeper... (0) -> (2)
2016-07-24 13:31:24.653 ERROR 83666 --- [qtp699221219-28] o.a.s.client.solrj.impl.CloudSolrClient : Request to collection foo failed due to (0) java.net.ConnectException: Operation timed out, retry? 0
and my code is:
String zkHostString = "localhost:2181";
CloudSolrClient solr = new CloudSolrClient.Builder().withZkHost(zkHostString).build();
solr.setDefaultCollection("foo");
SolrQuery query = new SolrQuery();
query.set("q", "*:*");
QueryResponse response = null;
try {
response = solr.query(query);
} catch (SolrServerException e) {
return null;
}
//Do Something with the results...
Urgh, I'm an idiot, the zookeeper instance and solr instances are inside docker, the code posted above is not. So Zookeeper reported back the solr urls using the docker containers ip...The host needs to connect via localhost and not the docker container ip.
Eg: Zookeeper responds [http://172.17.0.5:8983/solr/foo_shard1_replica2, http://172.17.0.6:8984/solr/foo_shard1_replica1]
but my code needs to call [http://localhost:8983/solr/foo_shard1_replica2, http://localhost:8984/solr/foo_shard1_replica1]
I have the following problem:
I try to connect to an ActiveMQ broker (which is now down) using the following piece of code
connectionFactory = new ActiveMQConnectionFactory(this.url + "?timeout=2000");
connection = connectionFactory.createConnection();
connection.start();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
LOGGER.info("Connected to " + this.url);
The problem is that the timeout does not have any effect
connection.start()
is blocked forever.
I inspected ActiveMQ log and found the following info:
2013-12-20 01:49:03,149 DEBUG [ActiveMQ Task-1] (FailoverTransport.java:786) - urlList connectionList:[tcp://localhost:61616?timeout=2000], from: [tcp://localhost:61616?timeout=2000]
2013-12-20 01:49:03,149 DEBUG [ActiveMQ Task-1] (FailoverTransport.java:1040) - Connect fail to: tcp://localhost:61616?timeout=2000, reason: java.lang.IllegalArgumentException: Invalid connect parameters: {timeout=2000}
The timeout parameter is specified here http://activemq.apache.org/cms/configuring.html
Has anybody any idea how to pass timeout argument to ActiveMQConnectionFactory?
Or how to set a timeout for connection.start() ?
Thank you!
Update: I found this on Stackoverflow: ActiveMQ - CreateSession failover timeout after a connection is resumed . I tried it but the following exception is thrown:
javax.jms.JMSException: Could not create Transport. Reason: java.lang.IllegalArgumentException: Invalid connect parameters: {transport.timeout=5000}
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:35)
I use ActiveMQ 5.8.0 from maven repo
It appears that your url is invalid still in both cases when attempting to set the timeout property.
If you're trying to have a failover URL, which it looks like you are since it is getting in to the Failover code then you're probably looking for initialReconnectDelay (and possibly maxReconnectAttempts which would throw an exception if the server is still down after the number of attempts is reached).
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("failover://(tcp://localhost:61616)?initialReconnectDelay=2000&maxReconnectAttempts=2");
I am trying to catch exception for an ActiveMQ connection which could not be established because of the broker being down.
With following code:
String url = ActiveMQConnection.DEFAULT_BROKER_URL;
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
connection.start();
The attempt to connect to the broker goes into infinite loop, if the broker is down. If I change url to
String url = "failover:(tcp://127.0.0.1:61616/)?startupMaxReconnectAttempts=2";
It makes 2 attempts and then throws an exception.(which is what I want.)
Now if I initialize the connection object using Spring Bean with the following:
<bean id="jmsFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<!--<value>tcp://0.0.0.0:61616</value>-->
<value>failover:(tcp://127.0.0.1:61616/)?startupMaxReconnectAttempts=2</value>
</property>
</bean>
I get an error message for failure to connect in 2 attempts but then, it still tries to connect again after every 5 seconds giving the same error message again and going on in infinite loop.
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
ERROR transport.failover.FailoverTransport - Failed to connect to [tcp://127.0.0.1:61616/] after: 2 attempt(s)
WARN jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'destinationQueue' - retrying in 5000 ms. Cause: Connection refused
these messages repeat!!
I want to know how to stop this infinite polling and catch an exception (may be using PostInit) in case of failure.
You could implement ExceptionListener in your listener class and override onException message. There you could handle your notification logic. While it try to reconnect automatically.
public class QueueListener implements MessageListener,ExceptionListener{
public void onMessage(Message message) {
}
public void onException(JMSException jsme) {
// Send my notifcation here.
}
}
Hi dear community of java addicts.
I was getting these exceptions in a CentOs VM, probably running with low RAM and then I noted that the time was not correctly synchronized between the other VM needed to communicate with my nice component....
I was wondering to know, When ? Why ? How ? A SocketException: end of file is produced in a LINUX server...
These are my logs:
2012-05-16 13:22:41,863 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][initDatabaseProperties] - Initializing database custom properties.
2012-05-16 13:22:41,864 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][initDatabaseProperties] - Setting NLS_DATE_FORMAT to : DD/MM/YYYY HH24:MI:SS
2012-05-16 13:22:47,096 [Timer-2] ERROR [org.jboss.remoting.transport.socket.SocketClientInvoker][handleException] - Got marshalling exception, exiting
java.net.SocketException: end of file
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:685)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.ConnectionValidator.doCheckConnectionWithoutLease(ConnectionValidator.java:828)
at org.jboss.remoting.ConnectionValidator.run(ConnectionValidator.java:345)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
2012-05-16 13:22:47,288 [Thread-2624] WARN [org.jboss.remoting.Client][removeListener] - unable to remove remote callback handler: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
2012-05-16 13:22:47,329 [Thread-2625] WARN [org.jboss.remoting.Client][removeListener] - unable to remove remote callback handler: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
2012-05-16 13:22:51,146 [Timer-4] WARN [org.jboss.remoting.transport.bisocket.BisocketServerInvoker][run] - org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask#7a7385ac: detected failure on control connection Thread[control: Socket[addr=ams-dev-bo.swissbytes.ch/192.168.0.190,port=11641,localport=57623],5,] (5c4o020-jlorp4-h29d35xs-1-h2aawvkq-l2t: requesting new control connection
2012-05-16 13:22:51,159 [controlConnectionRecreate:control: Socket[addr=ams-dev-bo.swissbytes.ch/192.168.0.190,port=11641,localport=57623]] ERROR [org.jboss.remoting.transport.bisocket.BisocketServerInvoker][createControlConnection] - unable to get secondary locator
org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:613)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.getSecondaryLocator(BisocketClientInvoker.java:640)
at org.jboss.remoting.transport.bisocket.BisocketServerInvoker.createControlConnection(BisocketServerInvoker.java:230)
at org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask$1.run(BisocketServerInvoker.java:1048)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:192)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:465)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:913)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:602)
... 5 more
2012-05-16 13:22:51,161 [controlConnectionRecreate:control: Socket[addr=ams-dev-bo.swissbytes.ch/192.168.0.190,port=11641,localport=57623]] ERROR [org.jboss.remoting.transport.bisocket.BisocketServerInvoker][run] - Unable to recreate control connection: InvokerLocator [null://ams-dev-bo.swissbytes.ch:11641/null]
java.io.IOException: unable to get secondary locator: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
at org.jboss.remoting.transport.bisocket.BisocketServerInvoker.createControlConnection(BisocketServerInvoker.java:235)
at org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask$1.run(BisocketServerInvoker.java:1048)
2012-05-16 13:22:56,870 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][runFilterRec] - Query for the level[1] was executed.
2012-05-16 13:22:56,871 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][doFilterResultsCotainCipAdress] - doFilterResultsCotainCipAdress() - Searching Cip Address: 1213194
2012-05-16 13:22:56,871 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][runFilterRec] - cipAddress: 1213194 was NOT FOUND in Filter Results
2012-05-16 13:22:56,871 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.services.RuleSetExecutorImpl][runFilterRec] - Current result size is not on filter's range : filter1 => [ 1 , 10 ] vs 0
2012-05-16 13:22:56,871 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.jms.request.SimilarDebtorsProcessor][delegateFixedResult] - Fixed result GET_SIMILAR_DEBTORS totalSize: 1 -> fixedSize: 1
2012-05-16 13:22:56,872 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.util.ListFragmenter][fragmentList] - Preparing [0] fragments in chunks of size [500]
2012-05-16 13:22:56,872 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.util.ListFragmenter][fragmentList] - List fragment range[0-1]
2012-05-16 13:22:56,872 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.jms.request.SimilarDebtorsProcessor][processFragment] - Sending 'SimilarDebtors' message fragment[0-1]
2012-05-16 13:22:56,980 [jmsContainer-1] ERROR [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][onMessage] - JMSException caused by Message[ID:JBM-5a9ac639-f2a4-436c-8170-37378d8b606b], somenthing is wrong with the communication.
2012-05-16 13:22:56,982 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][onMessage] - Queue listener will be stopped.
2012-05-16 13:22:56,983 [jmsContainer-1] INFO [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][stopListener] - Listener successfully stopped
2012-05-16 13:22:56,984 [jmsContainer-1] DEBUG [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][errorTemplateSend] - Trying to send error message to AMS-WA error queue.
2012-05-16 13:22:56,985 [jmsContainer-1] DEBUG [ch.swissbytes.cipadapter.jms.request.CipAdapterBean][errorTemplateSend] - Adapter id: 1
2012-05-16 13:22:57,014 [jmsContainer-1] ERROR [org.jboss.jms.client.container.ClosedInterceptor][invoke] - ClosedInterceptor.ClientSessionDelegate[ioy8-vyl6fa2h-1-hbl1g92h-qrmrca-a50o4c5]: method getTransacted() did not go through, the interceptor is CLOSED
2012-05-16 13:22:57,016 [jmsContainer-1] ERROR [org.springframework.jms.listener.DefaultMessageListenerContainer][rollbackOnExceptionIfNecessary] - Application exception overridden by rollback exception
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is org.jboss.jms.exception.MessagingNetworkFailureException; nested exception is org.jboss.remoting.CannotConnectException: Error setting up client lease upon performing connect.
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:292)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:474)
at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:548)
at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
at ch.swissbytes.cipadapter.jms.request.CipAdapterBean.errorTemplateSend(CipAdapterBean.java:226)
at ch.swissbytes.cipadapter.jms.request.CipAdapterBean.onMessage(CipAdapterBean.java:160)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:506)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:463)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:240)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.jboss.jms.exception.MessagingNetworkFailureException
at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:240)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:198)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.container.StateCreationAspect.handleCreateConnectionDelegate(StateCreationAspect.java:80)
at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect0.invoke(StateCreationAspect0.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.createConnectionDelegate(ClientConnectionFactoryDelegate.java)
at org.jboss.jms.client.JBossConnectionFactory.createConnectionInternal(JBossConnectionFactory.java:205)
at org.jboss.jms.client.JBossConnectionFactory.createConnection(JBossConnectionFactory.java:87)
at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
at $Proxy0.createConnection(Unknown Source)
at org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.doCreateConnection(UserCredentialsConnectionFactoryAdapter.java:174)
at org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.createConnection(UserCredentialsConnectionFactoryAdapter.java:149)
at org.springframework.jms.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:316)
at org.springframework.jms.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:270)
at org.springframework.jms.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:215)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:461)
... 12 more
Caused by: org.jboss.remoting.CannotConnectException: Error setting up client lease upon performing connect.
at org.jboss.remoting.Client.connect(Client.java:1804)
at org.jboss.remoting.Client.connect(Client.java:652)
at org.jboss.jms.client.remoting.JMSRemotingConnection$1.run(JMSRemotingConnection.java:374)
at java.security.AccessController.doPrivileged(Native Method)
at org.jboss.jms.client.remoting.JMSRemotingConnection.start(JMSRemotingConnection.java:368)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:165)
... 32 more
Caused by: java.lang.Exception: Error setting up client lease
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:508)
at org.jboss.remoting.Client.setupClientLease(Client.java:1912)
at org.jboss.remoting.Client.connect(Client.java:1800)
... 37 more
Caused by: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:613)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:474)
... 39 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:192)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:465)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:913)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:602)
... 42 more
2012-05-16 13:22:57,031 [jmsContainer-1] WARN [org.springframework.jms.listener.DefaultMessageListenerContainer][handleListenerException] - Execution of JMS message listener failed
javax.jms.IllegalStateException: The object is closed
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:157)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
at org.jboss.jms.client.delegate.ClientSessionDelegate$getTransacted_N1613179584734032131.invokeNext(ClientSessionDelegate$getTransacted_N1613179584734032131.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate.getTransacted(ClientSessionDelegate.java)
at org.jboss.jms.client.JBossSession.getTransacted(JBossSession.java:154)
at org.springframework.jms.listener.AbstractMessageListenerContainer.rollbackOnExceptionIfNecessary(AbstractMessageListenerContainer.java:574)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:442)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:240)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:662)
2012-05-16 13:22:57,072 [jmsContainer-1] WARN [org.jboss.remoting.Client][removeListener] - unable to remove remote callback handler: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://ams-dev-bo.swissbytes.ch:4457//?JBM_clientMaxPoolSize=200&clientLeasePeriod=10000&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&failureDisconnectTimeout=0&marshaller=org.jboss.jms.wireformat.JMSWireFormat&socket.check_connection=false&stopLeaseOnFailure=true&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat&useClientConnectionIdentity=true&validatorPingPeriod=10000&validatorPingTimeout=5000]
2012-05-16 13:23:35,981 [org.springframework.scheduling.timer.TimerFactoryBean#0] DEBUG [ch.swissbytes.cipadapter.services.tasks.CheckQueueListenerStatus][run] - Checking queueListener status
And error was originated in this method:
private void initDatabaseProperties(final Session session) {
logger.info("Initializing database custom properties.");
properties.getProperty(CommonConstants.CIP_DATE_FORMAT_PROP);
final String dateFormat = DateUtil.VIEW_DATE_FORMAT;
logger.info("Setting NLS_DATE_FORMAT to : " + dateFormat);
final String queryString = "ALTER SESSION SET NLS_DATE_FORMAT = '" + StringUtils.trim(dateFormat) + "'";
session.createSQLQuery(queryString).executeUpdate();
}
I can't answer for what exceptions JBoss throws, but in general EOS on a socket is caused by exactly one thing: receiving a FIN from the peer as the result of a close or shutdown output by the peer.