We Keep Coding

Read thread is stopped for socket

The below log messages indicate that the connection between my client and the server is being closed by the server since i am getting "Read thread is stopped for socket" from the Wireshark the message indicates that the server is closing the connection after it has sent the final "ACK" message indeed , the server is sending a FIN,ACK after the client sends a Capabilities-Exchange message. This suggests that the server is properly receiving the client's message, but for some reason it is not responding with a message of its own and instead is closing the connection. the localhost is my laptop , below is the logs
2023-01-09 13:11:43,946 INFO DictionaryImpl - Mobicents Diameter Dictionary loaded in 188ms -- Vendors[10] Commands[52] Types[20] AVPs[510]
2023-01-09 13:11:44,044 INFO StackImpl - (-)(-)(-)(-)(-) Started Mobicents DIAMETER Stack v1.5.10.0-build639 (-)(-)(-)(-)(-)
2023-01-09 13:11:44,544 INFO StackCreator - Diameter CLIENT :: Adding Listener for [AppId [Vendor-Id:0; Auth-Application-Id:4; Acct-Application-Id:0]].
2023-01-09 13:11:44,547 INFO StackCreator - Diameter CLIENT :: Supporting 1 applications.
2023-01-09 13:11:45,152 INFO NetworkGuard - Open server socket ServerSocket[addr=/127.0.0.1,localport=8080]
2023-01-09 13:12:00,267 INFO TCPTransportClient - Read thread is stopped for socket [Socket[addr=10.10.10.113/10.10.203.113,port=3868,localport=60884]]
2023-01-09 13:12:00,267 INFO TCPTransportClient - Read thread is stopped for socket [Socket[addr=10.10.10.113/10.105.10.113,port=3868,localport=60884]]
2023-01-09 13:12:00,267 INFO TCPTransportClient - Read thread is stopped for socket [Socket[addr=10.10.10.113/10.105.10.113,port=3868,localport=60884]]
do you have any idea please
I am expecting to connect to the server and send sendInitial request

IBM MQ - Java api - PCFMessageAgent - connection fails

I changed the MQIVP sample in MQ with a server connection channel of my own local.server.con and it is working fine. But I tried connecting to the same channel with PCFMessageAgent and the connection is failing with errors in MQ log. What is the relation between my channel and SYSTEM.DEFAULT.MODEL.QUEUE which gives the error.
C:\Program Files\IBM\WebSphere MQ\Tools\wmqjava\samples>java -Djava.library.path="C:\Program Files\IBM\WebSphere MQ\java\lib" MQIVPMod
Websphere MQ for Java Installation Verification Program
5724-B4 (C) Copyright IBM Corp. 2002, 2014. All Rights Reserved.
================================================================
Please enter the IP address of the MQ server :10.40.1.16
Please enter the port to connect to : (1414)1415
Please enter the server connection channel name :local.server.con
Please enter the user name (or RETURN for none) :test
Please enter the password for the user :test123
Please enter the queue manager name :local
Success: Connected to queue manager.
Success: Opened SYSTEM.DEFAULT.LOCAL.QUEUE
Success: Put a message to SYSTEM.DEFAULT.LOCAL.QUEUE
Success: Got a message from SYSTEM.DEFAULT.LOCAL.QUEUE
Success: Closed SYSTEM.DEFAULT.LOCAL.QUEUE
Success: Disconnected from queue manager
Tests complete -
SUCCESS: This MQ Transport is functioning correctly.
Press Enter to continue ...
My PCFMessageAgent code and error:
new PCFMessageAgent(host, Integer.parseInt(port), channelName); // connect
com.ibm.mq.MQException: MQJE001: Completion Code '2', Reason '2035'.
at com.ibm.mq.MQDestination.open(MQDestination.java:323)
at com.ibm.mq.MQQueue.<init>(MQQueue.java:236)
at com.ibm.mq.MQQueueManager.accessQueue(MQQueueManager.java:2674)
at com.ibm.mq.pcf.PCFAgent.open(PCFAgent.java:448)
at com.ibm.mq.pcf.PCFAgent.open(PCFAgent.java:394)
at com.ibm.mq.pcf.PCFAgent.connect(PCFAgent.java:287)
at com.ibm.mq.pcf.PCFAgent.<init>(PCFAgent.java:190)
at com.ibm.mq.pcf.PCFMessageAgent.<init>(PCFMessageAgent.java:157)
at test.wmq.PCFTest.main(PCFTest.java:49)
And the MQ log :
5/2/2017 14:01:31 - Process(6048.60) User(MUSR_MQADMIN) Program(amqzlaa0.exe)
Host(BLR_SWG_N09505) Installation(Installation1)
VRMF(8.0.0.4) QMgr(local)
AMQ8077: Entity 'test#blr_swg_n09505' has insufficient authority to access
object 'SYSTEM.DEFAULT.MODEL.QUEUE'.
EXPLANATION:
The specified entity is not authorized to access the required object. The
following requested permissions are unauthorized: get
ACTION:
Ensure that the correct level of authority has been set for this entity against
the required object, or ensure that the entity is a member of a privileged
group.
----- amqzfubn.c : 518 --------------------------------------------------------
5/2/2017 14:01:32 - Process(8004.41) User(MUSR_MQADMIN) Program(amqrmppa.exe)
Host(BLR_SWG_N09505) Installation(Installation1)
VRMF(8.0.0.4) QMgr(local)
AMQ9208: Error on receive from host BLR_SWG_N09505 (10.40.1.16).
EXPLANATION:
An error occurred receiving data from BLR_SWG_N09505 (10.40.1.16) over TCP/IP.
This may be due to a communications failure.
ACTION:
The return code from the TCP/IP recv() call was 10054 (X'2746'). Record these
values and tell the systems administrator.
----- amqccita.c : 4076 -------------------------------------------------------
5/2/2017 14:01:32 - Process(8004.41) User(MUSR_MQADMIN) Program(amqrmppa.exe)
Host(BLR_SWG_N09505) Installation(Installation1)
VRMF(8.0.0.4) QMgr(local)
AMQ9999: Channel 'local.server.con' to host '10.40.1.16' ended abnormally.
EXPLANATION:
The channel program running under process ID 8004(7988) for channel
'local.server.con' ended abnormally. The host name is '10.40.1.16'; in some
cases the host name cannot be determined and so is shown as '????'.
ACTION:
Look at previous error messages for the channel program in the error logs to
determine the cause of the failure. Note that this message can be excluded
completely or suppressed by tuning the "ExcludeMessage" or "SuppressMessage"
attributes under the "QMErrorLog" stanza in qm.ini. Further information can be
found in the System Administration Guide.
----- amqrmrsa.c : 930 --------------------------------------------------------

You need to go and read up on MQ permissions (i.e. authorizations). It is best to do permissions on the group rather than principle (UserId).
setmqaut -m {QM_NAME} -n SYSTEM.ADMIN.COMMAND.QUEUE -t queue -g {GROUP} +put +inq +dsp
setmqaut -m {QM_NAME} -n SYSTEM.DEFAULT.MODEL.QUEUE -t queue -g {GROUP} +get +inq +dsp

There is no relation between channels and model queues.
But I think, that the PCFMessageAgent is trying to create a dynamic queue to use as a ReplyToQ to receive responses, and it seems it tries to create the dynamic queue by opening the SYSTEM.DEFAULT.MODEL.QUEUE.

AMQP Channel shutdown but Consumer not always restart

I have a frequent Channel shutdown: connection error issues (under 24.133.241:5671 thread, name is truncated) in RabbitMQ Java client (my producer and consumer are far apart). Most of the time consumer is automatically restarted as I have enabled heartbeat (15 seconds). However, there were some instances only Channel shutdown: connection error but no Consumer raised exception and no Restarting Consumer (under cTaskExecutor-4 thread).
My current workaround is to restart my application. Anyone can shed some light on this matter?
2017-03-20 12:42:38.856 ERROR 24245 --- [24.133.241:5671] o.s.a.r.c.CachingConnectionFactory
: Channel shutdown: connection error
2017-03-20 12:42:39.642 WARN 24245 --- [cTaskExecutor-4] o.s.a.r.l.SimpleMessageListenerCont
ainer : Consumer raised exception, processing can restart if the connection factory supports
it
...
2017-03-20 12:42:39.642 INFO 24245 --- [cTaskExecutor-4] o.s.a.r.l.SimpleMessageListenerCont
ainer : Restarting Consumer: tags=[{amq.ctag-4CqrRsUP8plDpLQdNcOjDw=21-05060179}], channel=Ca
ched Rabbit Channel: AMQChannel(amqp://21-05060179#10.24.133.241:5671/,1), conn: Proxy#7ec317
54 Shared Rabbit Connection: SimpleConnection#44bac9ec [delegate=amqp://21-05060179#10.24.133
.241:5671/], acknowledgeMode=NONE local queue size=0

Generally, this is due to the consumer thread being "stuck" in user code somewhere, so it can't react to the broken connection.
If you have network issues, perhaps it's stuck reading or writing to a socket; make sure you have timeouts set for any I/O operations.
Next time it happens take a thread dump to see what the consumer threads are doing.

Spring AMQP – Is a #RabbitListener Polling under the Hood?

Summary
I want to asynchronously handle messages from an AMQP/RabbitMQ queue. I have implemented a #RabbitListener method (from spring-rabbit) for this but it seems that this listener is actually polling my queue under the hood. Is that to be expected? I would have expected the listener to somehow be notified by RabbitMQ instead of having to poll.
If it’s to be expected, can I somehow also consume messages asynchronously with Spring AMQP without polling?
What I Have Observed
When I send a message, it is correctly picked up by the listener. I still see a continuous stream of log messages which indicate that the listener continues to poll the empty queue:
…
15:41:10.543 [pool-1-thread-3] DEBUG o.s.a.r.l.BlockingQueueConsumer - ConsumeOK : Consumer: tags=[{amq.ctag-bUsK4KQN6_QHzf8DoDC_ww=myQueue}], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.1.1:5672/,1), acknowledgeMode=MANUAL local queue size=0
15:41:10.544 [main] DEBUG o.s.a.r.c.CachingConnectionFactory - Creating cached Rabbit Channel from AMQChannel(amqp://guest#127.0.1.1:5672/,2)
15:41:10.545 [main] DEBUG o.s.amqp.rabbit.core.RabbitTemplate - Executing callback on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.1.1:5672/,2)
15:41:10.545 [main] DEBUG o.s.amqp.rabbit.core.RabbitTemplate - Publishing message on exchange [], routingKey = [myQueue]
Sent: Hello World
15:41:10.559 [pool-1-thread-4] DEBUG o.s.a.r.l.BlockingQueueConsumer - Storing delivery for Consumer: tags=[{amq.ctag-bUsK4KQN6_QHzf8DoDC_ww=myQueue}], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.1.1:5672/,1), acknowledgeMode=MANUAL local queue size=0
15:41:10.560 [SimpleAsyncTaskExecutor-1] DEBUG o.s.a.r.l.BlockingQueueConsumer - Received message: (Body:'Hello World'MessageProperties [headers={}, timestamp=null, messageId=null, userId=null, appId=null, clusterId=null, type=null, correlationId=null, replyTo=null, contentType=text/plain, contentEncoding=UTF-8, contentLength=0, deliveryMode=PERSISTENT, expiration=null, priority=0, redelivered=false, receivedExchange=, receivedRoutingKey=myQueue, deliveryTag=1, messageCount=0])
15:41:10.571 [SimpleAsyncTaskExecutor-1] DEBUG o.s.a.r.l.a.MessagingMessageListenerAdapter - Processing [GenericMessage [payload=Hello World, headers={timestamp=1435844470571, id=018f39f6-ebca-aabf-7fe3-a095e959f65d, amqp_receivedRoutingKey=myQueue, amqp_deliveryMode=PERSISTENT, amqp_consumerQueue=myQueue, amqp_consumerTag=amq.ctag-bUsK4KQN6_QHzf8DoDC_ww, amqp_contentEncoding=UTF-8, contentType=text/plain, amqp_deliveryTag=1, amqp_redelivered=false}]]
Received: Hello World
15:41:10.579 [SimpleAsyncTaskExecutor-1] DEBUG o.s.a.r.l.BlockingQueueConsumer - Retrieving delivery for Consumer: tags=[{amq.ctag-bUsK4KQN6_QHzf8DoDC_ww=myQueue}], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.1.1:5672/,1), acknowledgeMode=MANUAL local queue size=0
15:41:11.579 [SimpleAsyncTaskExecutor-1] DEBUG o.s.a.r.l.BlockingQueueConsumer - Retrieving delivery for Consumer: tags=[{amq.ctag-bUsK4KQN6_QHzf8DoDC_ww=myQueue}], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.1.1:5672/,1), acknowledgeMode=MANUAL local queue size=0
15:41:12.583 [SimpleAsyncTaskExecutor-1] DEBUG o.s.a.r.l.BlockingQueueConsumer - Retrieving delivery for Consumer: tags=[{amq.ctag-bUsK4KQN6_QHzf8DoDC_ww=myQueue}], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.1.1:5672/,1), acknowledgeMode=MANUAL local queue size=0
…
The last log message basically repeats infinitely every second.
My Test Code
The first two methods are probably the most interesting part; the rest is mainly Spring configuration:
#Configuration
#EnableRabbit
public class MyTest {
public static void main(String[] args) throws InterruptedException {
try (ConfigurableApplicationContext appCtxt =
new AnnotationConfigApplicationContext(MyTest.class)) {
// send a test message
RabbitTemplate template = appCtxt.getBean(RabbitTemplate.class);
Queue queue = appCtxt.getBean(Queue.class);
template.convertAndSend(queue.getName(), "Hello World");
System.out.println("Sent: Hello World");
// Now that the application with its message listeners is running,
// block this thread forever; make sure, though, that the
// application context can sanely be closed.
appCtxt.registerShutdownHook();
Object blockingObj = new Object();
synchronized (blockingObj) {
blockingObj.wait();
}
}
}
#RabbitListener(queues = "#{ #myQueue }")
private void processHello(#Payload String msg,
#Header(AmqpHeaders.DELIVERY_TAG) long deliveryTag, Channel channel)
throws IOException {
System.out.println("Received: " + msg);
channel.basicAck(deliveryTag, false);
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(rabbitConnFactory());
}
#Bean
public ConnectionFactory rabbitConnFactory() {
return new CachingConnectionFactory();
}
#Bean
public SimpleRabbitListenerContainerFactory
rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory result =
new SimpleRabbitListenerContainerFactory();
result.setConnectionFactory(rabbitConnFactory());
result.setAcknowledgeMode(AcknowledgeMode.MANUAL);
return result;
}
#Bean
public Queue myQueue() {
return new Queue("myQueue", false);
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(rabbitConnFactory());
}
}

It's not polling rabbitmq; when a message arrives asynchronously from rabbit, it is placed in an internal queue in the consumer; handing over the message to the listener thread which is blocked, waiting for the arrival.
The DEBUG message you are seeing is after the listener thread times out waiting for a new message to arrive from rabbitmq.
You can increase the receiveTimeout to reduce the logs, or simply disable DEBUG logging for the BlockingQueueConsumer.
Increasing the timeout will make the container less responsive to container stop() requests.
EDIT:
In response to your comment below...
Yes, we could interrupt the thread but it's a bit more involved than that. The receive timeout is also used to ack messages when txSize is > 1.
Let's say you only want to ack every 20 messages (instead of every message). People do that to improve performance in high volume environments. The timeout is also used to ack (the txSize is actually every n messages or timeouts).
Now, let's say 19 messages arrive then none for 60 seconds and your timeout is 30 seconds.
This will mean that the 19 messages are un-acked for a long time. With the default configuration, the ack will be sent 1 second after the 19th message arrives.
There really is little overhead in this timeout (we simply loop back and wait again), so it is unusual for it to be increased.
Also, while the container is stopped when the context is closed, people stop and start containers all the time.

Browsing Service Bus queue hangs in hasMoreElements()

we have on-premise Service Bus installation. I can publish and subscribe/read messages with QPID AMQP 1.0 0.24 client. However queue browsing doens§t work, the call to hasMoreElements() hangs indefinitely when there are no more messages in the queue. The stack trace is:
Thread [main] (Suspended)
waiting for: ConnectionEndpoint (id=19)
Object.wait(long) line: not available [native method]
ConnectionEndpoint(Object).wait() line: 503
Receiver.drainWait() line: 533
QueueBrowserImpl$MessageEnumeration.hasMoreElements() line: 154
Qpid.testBrowseTopic(Connection, Context) line: 209
Qpid.runTest(Qpid$Options) line: 93
Qpid.main(String[]) line: 63
The code:
ConnectionFactory connectionFactory = (ConnectionFactory) context.lookup("MS_SERVICE_BUS");
connection = connectionFactory.createConnection();
session = connection.createSession(false/*transacted*/, Session.AUTO_ACKNOWLEDGE);
Queue queue = (Queue) context.lookup("MY_QUEUE");
browser = session.createBrowser(queue);
Enumeration<Message> msgs = browser.getEnumeration();
while (msgs.hasMoreElements()) {// hangs when there are no more messages
Message message = msgs.nextElement();
//printMessage(message);
}
The same behaviour for QPID 0.22. Is this bug in QPID client or Service Bus?
Thanks, Jan

There are couple things going on here:
1) Service Bus doesn't support message browse over AMQP at this time. Because the session was created with AUTO_ACKNOWLEDGE, every message you get from the Enumeration is immediately removed from the queue.
2) I have reproduced the hang in hasMoreElements() with QPid 0.25. It appears that hasMoreElements() is waiting for more messages to arrive in the queue, at least at first. If I send more messages, the loop will continue and some of the newly-arrived messages will be returned, but it stops fairly quickly. I am still investigating to determine what's going on there.