Reactor rabbitmq AlreadyClosedException - java

In my project I am using Springboot version 2.1.2.RELEASE, reactor-rabbitmq version 1.0.0.RELEASE. I am creating rabbit receiver, subscribe it and process messagess with manual ack. But after some time it can be after an hour or after 1-2 days of work, I am getting heartbeat missing error, than channel is closing and I get "com.rabbitmq.client.AlreadyClosedException: connection is already closed due to connection error; cause: java.io.IOException: Connection reset by peer" and my receiver no longer receive messages. It works only after restart.
Rabbit client has
connectionFactory.setAutomaticRecoveryEnabled(true);
and
connectionFactory.setTopologyRecoveryEnabled(true);
so it should auto-recovery by default, but it doesn't work.
public void startReceiver(int parallelism) {
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.useNio();
connectionFactory.setPort(5672);
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
connectionFactory.setVirtualHost("/");
connectionFactory.setRequestedHeartbeat(10);
Address[] addresses = {new Address("localhost")};
ReceiverOptions receiverOptions = new ReceiverOptions()
.connectionFactory(connectionFactory)
.connectionSupplier(cf -> cf.newConnection(addresses, "receiver"))
.connectionSubscriptionScheduler(Schedulers.elastic());
Receiver receiver = RabbitFlux.createReceiver(receiverOptions);
receiver.consumeManualAck("test-data", new ConsumeOptions().qos(200))
.doOnSubscribe(s -> System.out.println("Receiver started."))
.retry()
.parallel(parallelism)
.runOn(Schedulers.newParallel("parallel-receiver", parallelism))
.doOnNext(d -> processMessage(d))
.subscribe();
}
private void processMessage(AcknowledgableDelivery message) {
try {
//some processing
} catch (Exception e) {
e.printStackTrace();
}
message.ack();
}
I getting errors
com.rabbitmq.client.MissedHeartbeatException: Heartbeat missing with heartbeat = 10 seconds,
at com.rabbitmq.client.impl.AMQConnection.handleHeartbeatFailure(AMQConnection.java:686) ~[amqp-client-5.5.1.jar!/:5.5.1],
at com.rabbitmq.client.impl.nio.NioLoop.lambda$handleHeartbeatFailure$0(NioLoop.java:273) [amqp-client-5.5.1.jar!/:5.5.1],
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181],
reactor.rabbitmq.Receiver : Cancelling consumer amq.ctag--x02OWhVo3_DPutsPQ0qDw consuming from test-data,
reactor.core.Exceptions$ErrorCallbackNotImplemented: com.rabbitmq.client.AlreadyClosedException: connection is already closed due to connection error; cause: java.io.IOException: Connection reset by peer,
Caused by: com.rabbitmq.client.AlreadyClosedException: connection is already closed due to connection error; cause: java.io.IOException: Connection reset by peer,
at com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:257) ~[amqp-client-5.5.1.jar!/:5.5.1],
at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:426) ~[amqp-client-5.5.1.jar!/:5.5.1],
at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:420) ~[amqp-client-5.5.1.jar!/:5.5.1],
at com.rabbitmq.client.impl.recovery.RecoveryAwareChannelN.basicAck(RecoveryAwareChannelN.java:93) ~[amqp-client-5.5.1.jar!/:5.5.1],
at com.rabbitmq.client.impl.recovery.AutorecoveringChannel.basicAck(AutorecoveringChannel.java:428) ~[amqp-client-5.5.1.jar!/:5.5.1],
at reactor.rabbitmq.AcknowledgableDelivery.ack(AcknowledgableDelivery.java:56) ~[reactor-rabbitmq-1.0.0.RELEASE.jar!/:1.0.0.RELEASE],
at reactor.rabbitmq.AcknowledgableDelivery.ack(AcknowledgableDelivery.java:73) ~[reactor-rabbitmq-1.0.0.RELEASE.jar!/:1.0.0.RELEASE],

Related

ShutdownSignalException: connection error RabbitMQ on STS4

I am attempting to connect to RabbitMQ locally on port 15672 but am getting connection error. I am unsure what could be causing this as I am attempting to learn RabbitMQ... This is tutorial 1 I can't even get running. (https://www.rabbitmq.com/tutorials/tutorial-one-java.html)
Below is the code and error. The only changes from the tutorial I have made are specifying the port and username/password. Any ideas?
package send.java;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;
public class Send {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception{
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
factory.setPort(15672);
factory.setUsername("guest");
factory.setPassword("guest");
try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel()){
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = "Hello World!";
channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
System.out.println(" [x] Sent '" + message + "'");
}
}
}
Error
Exception in thread "main" java.io.IOException
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:129)
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:125)
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:375)
at com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection(RecoveryAwareAMQConnectionFactory.java:64)
at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init(AutorecoveringConnection.java:156)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1106)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1063)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1021)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1182)
at send.java.Send.main(Send.java:15)
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error
at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)
at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)
at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:502)
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:317)
... 7 more
Caused by: java.io.EOFException
at java.base/java.io.DataInputStream.readUnsignedByte(DataInputStream.java:294)
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:184)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:598)
at java.base/java.lang.Thread.run(Thread.java:832)
Port 15672 is the (default) HTTP port for the admin (management plugin) UI.
The default AMQP port is 5672.

Why is AdminClient not failing when creating a topic with Kafka cluster not running?

Why don't I get an error message when I run the following producer code and Kafka is not even running?
I would expect the createTopics method to throw an exception, but it doesn't happen. Why?
final Properties properties = new Properties();
properties.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
properties.put(AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, "1000");
AdminClient adminClient = AdminClient.create(properties);
NewTopic newTopic = new NewTopic("events", 1, (short) 1);
adminClient.createTopics(Arrays.asList(newTopic));
adminClient.close();
The method createTopics returns a CreateTopicsResult with KafkaFutures as values. As you are currently not blocking your code for this action to be finished (using get) and not catching any Exception, your code will just run fine without any notification that the broker is not available.
The following code will throw an ExecutionException when your broker is not available:
final Properties properties = new Properties();
properties.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
properties.put(AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, "1000");
properties.put(AdminClientConfig.DEFAULT_API_TIMEOUT_MS_CONFIG, "5000");
AdminClient adminClient = AdminClient.create(properties);
NewTopic newTopic = new NewTopic("events-test", 1, (short) 1);
CreateTopicsResult topicResult = adminClient.createTopics(Arrays.asList(newTopic));
KafkaFuture<Void> resultFuture = topicResult.all();
try {
resultFuture.get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
adminClient.close();
I tested it using Kafka client 2.5.0 and here is the Excpetion:
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=createTopics, deadlineMs=1601136227182) timed out at 1601136227183 after 1 attempt(s)
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at org.michael.big.data.kafka.java.BasicAdminClient.main(BasicAdminClient.java:27)
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=createTopics, deadlineMs=1601136227182) timed out at 1601136227183 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
Be aware that the class Admin (the super class of AdminClient) is annotated as Evolving and might be changed in future releases.

How do I fix this issue mqtt Paho timed out but did not report as connection lost

I have a problem with my service using paho mqtt. It times out but no report as connection lost. Did anyone have a solution to this problem?
public void connect(String client_id) {
String tmpDir = System.getProperty("java.io.tmpdir");
MqttDefaultFilePersistence dataStore = new MqttDefaultFilePersistence( tmpDir );
MqttConnectOptions options = new MqttConnectOptions();
options.setKeepAliveInterval(120);
try {
_client = new MqttClient( _url, MqttClient.generateClientId(), dataStore );
_client.connect( options );
_client.setCallback( this );
_client.subscribe( TOPIC );
LOGGER.info("Subscribed to " + Arrays.toString(TOPIC));
LOGGER.info("MAX CORES " + MAX_CORES);
} catch ( MqttException e ) {
LOGGER.log( Level.INFO, "Mosquitto Error: " + e.fillInStackTrace() );
connect(null);
}
}
#Override
public void messageArrived(final String topic, final MqttMessage message) throws Exception {
threadPool.execute(new Runnable(){// code});
}
Using mqtt v 1.0.0
Nov 6, 2015 2:36:52 AM org.eclipse.paho.client.mqttv3.internal.ClientState checkForActivity
SEVERE: paho-595916529752865: Timed out as no activity, keepAlive=120,000 lastOutboundActivity=1,446,777,172,455 lastInboundActivity=1,446,777,117,734 time=1,446,777,412,455 lastPing=1,446,777,172,454
As a quick fix
just set:
setConnectionTimeout(0) use zero to disable timeout.
setConnectionTimeout public void setConnectionTimeout(int
connectionTimeout) Sets the connection timeout value. This value,
measured in seconds, defines the maximum time interval the client will
wait for the network connection to the MQTT server to be established.
The default timeout is 30 seconds. A value of 0 disables timeout
processing meaning the client will wait until the network connection
is made successfully or fails. Parameters: connectionTimeout - the
timeout value, measured in seconds. It must be >0;
Hope it'll help out.

Eclipse paho Mqtt:Getting java.io.EOF Exception

I am getting "java.io.eof" exception,when i am trying to subscribe mqtt client. I am using eclipse paho library and using mosquitto broker. I am not getting any answer of this,so please help me why this happens ?
Mqtt connection and subscribe
I am using this code for connecting and subscribing to mosquitto
private void buildClient(String clientId){
log.debug("Connecting... "+clientId);
try {
mqttClient = new MqttClient(envConfiguration.getBrokerUrl(), clientId,new MemoryPersistence());
System.out.println(mqttClient.isConnected());
} catch (MqttException e) {
log.debug("build client stopped due to "+e.getCause());
}
chatCallback = new ChatCallback(this.userService,this);
mqttClient.setCallback(chatCallback);
mqttConnectOptions = new MqttConnectOptions();
mqttConnectOptions.setCleanSession(true);
}
#Override
public void connect(String clientId,String topic) {
try{
if(mqttClient == null || !mqttClient.getClientId().equals(clientId)){
buildClient(clientId);
mqttClient.connect(mqttConnectOptions);
subscribe(clientId,topic);
}
}catch (Exception e) {
log.debug("connection attempt failed "+ e.getCause() + " trying...");
}
}
#Override
public void subscribe(String clientId,String topic) throws MqttException {
if(mqttClient != null && mqttClient.isConnected()){
mqttClient.subscribe(topic,0);
/*try {
log.debug("Subscribing... with client id :: " + clientId + "topic");
mqttClient.subscribe(topic,2);
} catch (MqttException e) {
log.debug("subscribing error.."+e.getLocalizedMessage());
}*/
}
}
}
And mqtt call back
#Override
public void connectionLost(Throwable arg0) {
log.debug("Connection lost... attampting retrying due to "
+ arg0);
arg0.printStackTrace();
// chatServiceimpl.connect();
}
#Override
public void deliveryComplete(IMqttDeliveryToken arg0) {
log.debug("delivered message" + arg0);
// TODO Auto-generated method stub
}
#Override
public void messageArrived(String arg0, MqttMessage arg1) throws Exception {
log.debug("Message recived..." + arg1.toString());
userService.saveChat(arg1.toString());
}
I am facing this error when i am subscribing to mosquitto
Error logs
2015-11-30/18:19:00.877 [MQTT Call: 25287] DEBUG c.s.s.ChatCallback: Message recived...{ "id":"37153topic25287T1448886285.79573", "from":"37153", "to":"25287", "chatBody":[{"type": "text", "message":"The fact "}]}
2015-11-30/18:19:00.878 [MQTT Call: 25287] DEBUG c.s.s.u.UserService: Saving chat...
2015-11-30/18:19:00.883 [MQTT Call: 25287] DEBUG c.s.s.u.UserService: Get user by id::37153
2015-11-30/18:19:00.885 [MQTT Call: 25287] DEBUG c.s.s.u.UserService: Get user by id::25287
2015-11-30/18:19:00.886 [MQTT Rec: 25287] DEBUG c.s.s.ChatCallback: Connection lost... attampting retrying due to Connection lost (32109) - java.io.EOFException
Connection lost (32109) - java.io.EOFException
at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:138)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:267)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttInputStream.readMqttWireMessage(MqttInputStream.java:56)
at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:100)
... 1 more
Mosquitto Logs
1448889230: Client 25287 disconnected.
1448889230: New client connected from 192.168.2.63 as 25287 (c0, k60).
1448889231: New connection from 192.168.2.242 on port 1883.
1448889231: Client 25287 already connected, closing old connection.
1448889231: Client 25287 disconnected.
1448889231: New client connected from 192.168.2.242 as 25287 (c1, k60).
1448889231: New connection from 192.168.2.63 on port 1883.
1448889231: Client 25287 already connected, closing old connection.
1448889231: Client 25287 disconnected.
1448889231: New client connected from 192.168.2.63 as 25287 (c0, k60).
1448889269: New connection from 192.168.2.242 on port 1883.
You have multiple clients connecting to the broker with the same clientid, this is not allowed and as one connects the broker will disconnect the currently connected client.
If both clients have automatic reconnection logic then they will just continue to kick each other off.
Change the client id on one of the clients.
As hardillb mentioned above, you have multiple clients connecting. Server (mosquitto) will disconnect the old connection when it receives a connect request from the same client again.
use the isConnected() method on MqttClient object to know if its connected. for eg.
if (! m_client.isConnected()) {
// reconnect
}

How do I kill "ActiveMQ Transport" threads after interrupting JMS MessageConsumer.receive()

I have a class that acquires an ActiveMQ connection using ActiveMQConnectionFactory.createConnection().
It then creates and owns the session, the destination and the consumer (queue) listening on that destination.
I call receive() or receive(millis) on the consumer to get my messages off the queue. In certain scenarios I have to kill (interrupt) the thread in which the receive method is being called. I try to close the session and the connection right after that. Unfortunately I am constantly getting an exception when calling close and the associated "ActiveMQ Transport" threads (and related connections to the broker) are staying alive. The exception I get is
org.myorg.awsiface.communication.MessagingException: Failed to close JMS connection
at
org.myorg.aws.communication.transport.JMSMessageTransport.cleanUp(JMSMessageTransport.java:253)
at
org.myorg.aws.communication.protocol.ContextFinishedProtocol.cleanUp(ContextFinishedProtocol.java:94)
at org.myorg.myservice.job.Job.run(Job.java:206)
Caused by: javax.jms.JMSException: java.io.InterruptedIOExceptio)
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1227)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1219)
at org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1799)
at org.apache.activemq.ActiveMQMessageConsumer.doClose(ActiveMQMessageConsumer.java:636)
at org.apache.activemq.ActiveMQMessageConsumer.close(ActiveMQMessageConsumer.java:627)
at org.myorg.aws.communication.transport.JMSMessageTransport.cleanUp(JMSMessageTransport.java:232)
... 2 more
Caused by: java.io.InterruptedIOException
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:102)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1225)
... 7 more
I've tried the following connection URLs
failover://tcp://broker.ip.address:61616
tcp://broker.ip.address:61616
failover://tcp://broker.ip.address:61616?trace=true&closeAsync=false
Also I've tried using the MessageConsumer::receiveNoWait() call and just executing my own Thread.sleep, but I always end up with the exception above whenever I call the following to close the connection
try {
// I added the following two lines to see if the taskRunner was handling the threads - still no luck
TaskRunnerFactory taskRunner = ((ActiveMQConnection)connection).getSessionTaskRunner();
taskRunner.shutdown();
if (producer != null) {
producer.close();
}
if (consumer != null) {
consumer.close();
}
session.close();
if (connection instanceof ActiveMQConnection) {
ActiveMQConnection amqConnection = (ActiveMQConnection) connection;
amqConnection.stop();
}
connection.stop();
connection.close();
}
catch (ConnectionClosedException e) {
// NOOP - this is ok
}
catch (Exception e) {
throw new MessagingException("Failed to close JMS connection", e, log);
}
I'm not sure why the close is not working for you. But you can accomplish what you want by using an asynchronous message listener.
Connection connection = connectionFactory.createConnection();
connection.start();
Session session =
connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Destination queue = session.createQueue("queueName");
Consumer consumer = session.createConsumer( queue );
consumer.setMessageListener( new MessageListener()
{
public void onMessage( Message message )
{
// whatever was after the receive() method call
}
} );
Then there's no need to be interrupted. When you are done, close things out:
consumer.close();
session.close();
queue.close();
connection.close();
One solution could be to set daemon flag on TCP ActiveMQ transport thread,
e.g. tcp://URI:PORT?daemon=true
Check out ActiveMQ documentation: http://activemq.apache.org/tcp-transport-reference.html

Categories