ExecutorService not shutting down after Tomcat shutdown - java

Thanks for this opportunity to ask questions here.
Summary
I have a spring boot application which runs on Tomcat. I'm subscribing ActiveMQ topic with StompClient. After connected i get messages via StompHandler's handleFrame method.
In that method i create MessageUtils which implements Runnable interface. In MessageUtils run method, creating new thread with ExecutorService and do the task generateMessage which send messages to kafka topic.
Problem
When I shutdown the tomcat, threads are still alive. In catalina.out;
A web application appears to have started a thread named [foo] but has failed to stop it. This is very likely to create a memory leak
So the tomcat cannot shutdown properly.
Code Samples
#Service
StompService.class
#EventListener(ApplicationReadyEvent.class)
public void start() {
logI("Service run Client Methods");
List<String> topics = Arrays.asList(topicListString.split(","));
for (String topic : topics) {
StompClient client = new StompClient(topic, username, password, url, topic, bootstrapAddress);
try {
client.run();
runMap.put(topic, client);
boolean connected = client.getSession().isConnected();
logI("Topic: " + topic + " is connected: " + connected);
} catch (InterruptedException e) {
logE("InterruptedException during start of stomp client: ", e);
} catch (TimeoutException e) {
logE("TimeoutException during start of stomp client: ", e);
} catch (ExecutionException e) {
logE("ExecutionException during start of stomp client: ", e);
} catch (Exception e) {
logE("Unexpected exception during start of stomp client: ", e);
}
}
}
StompClient.class
public void run() throws ExecutionException, TimeoutException, InterruptedException {
WebSocketClient client = new StandardWebSocketClient();
WebSocketStompClient stompClient = new WebSocketStompClient(client);
ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();
taskScheduler.afterPropertiesSet();
StompHeaders connectHeaders = new StompHeaders();
connectHeaders.add("login", this.userName);
connectHeaders.add("passcode", this.password);
stompClient.getDefaultHeartbeat();
stompClient.setTaskScheduler(taskScheduler);
stompClient.setMessageConverter(new StringMessageConverter());
stompClient.setAutoStartup(true);
StompSessionHandler sessionHandler = new StompHandler(this.topic, this.bootstrapAddress);
StompSession stompSession = null;
try {
stompSession = stompClient.connect(url, new WebSocketHttpHeaders(), connectHeaders, sessionHandler)
.get(5, TimeUnit.SECONDS);
} catch (Exception e) {
logE("Cannot connect with stomp client." , e);
}
this.setSession(stompSession);
}
StompHandler.class which extends StompSessionHandlerAdapter
#Override
public void handleFrame(StompHeaders headers, Object payload) {
String msg = (String) payload;
MessageUtils message = new MessageUtils();
message.setHeaders(headers);
message.setTopic(topic);
message.setMsg(msg);
message.setBootstrapAddress(bootstrapAddress);
message.run();
}
MessageUtils.class
#Override
public void run() {
ExecutorService executorService = Executors.newSingleThreadExecutor();
executorService.execute(new Runnable() {
public void run() {
generateMessage(getMsg().toString());
}
});
executorService.shutdown();
try {
executorService.awaitTermination(200, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
logI("InterruptedException during await termination", e);
}
}

Related

WebSocketClient disconnected RemoteEndpoint unavailable

I am trying to implement a WebSocketClient in Thread where Client is continuously sending message to the server, But I see Exception:
org.eclipse.jetty.websocket.api.WebSocketException: RemoteEndpoint
unavailable, current state [CLOSED], expecting [OPEN or CONNECTED]
here is the code:
public class ProcessAnalysisThread extends Thread {
static String dest = "ws://localhost:8086/Echo/";
WebSocketClient client;
EAMClientSocket socket;
public void run() {
initializeWSClient();
while (true) {
System.out.println("Thread running");
try {
Thread.sleep(500);
socket.sendMessage("echo");
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
void initializeWSClient() {
client = new WebSocketClient();
try {
socket = new EAMClientSocket(this);
client.start();
URI echoUri = new URI(dest);
System.out.println("URI Resolved");
ClientUpgradeRequest request = new ClientUpgradeRequest();
client.connect(socket, echoUri, request);
System.out.println("Connected and Waiting .....");
socket.getLatch().await();
System.out.println("Latched");
System.out.println("RequestURI: " + request.getRequestURI().toString());
socket.sendMessage("echo");
System.out.println("Sent Msg: echo");
socket.sendMessage("test");
System.out.println("Sent Msg: test");
// Thread.sleep(10000l);
} catch (Throwable t) {
System.out.println("Exception");
t.printStackTrace();
} finally {
try {
client.stop();
} catch (Exception e) {
System.out.println("Exception1");
e.printStackTrace();
}
}
}
}
Here is the output:
Connected and Waiting .....
Connected to server
Latched
RequestURI: ws://localhost:8086/Echo/
Sent Msg: echo
Sent Msg: test
Message received from server:echo
Message received from server:test
Thread running
Exception in thread "Thread-0" org.eclipse.jetty.websocket.api.WebSocketException: RemoteEndpoint unavailable, current state [CLOSED], expecting [OPEN or CONNECTED]
at org.eclipse.jetty.websocket.common.WebSocketSession.getRemote(WebSocketSession.java:252)
Looks like my thread not holding socket reference here

JUnit testing hangs at clientSocket.accept()?

I need to test some client/server application using JUnit and I'm having trouble testing the connection of a client to the server. It seems that the test hangs at the "Socket clientSocket = serverSocket.accept()" line in the server class, and it's not letting me connect a new client.
Here's the JUnit class:
#Test
public void testNetServerComm_1() throws Exception {
server = new NetServerComm();
assertNotNull(server);
}
#Test
public void testAddMessageToQueue_1() throws Exception {
NetServerComm fixture = new NetServerComm();
NetServerSideMessage msg = new NetServerSideMessage(birdsong.comm.ServerSideMessage.Type.BIRDSONG_MESSAGE, "",
"");
fixture.addMessageToQueue(msg);
assertEquals(1, fixture.getMsgQueueSize());
}
#Test
public void testClientIsConnected_1() throws Exception {
NetServerComm fixture = new NetServerComm();
fixture.start();
//test doesn't go further that this..
String nickname = "john";
new NetClientComm().connect("127.0.0.1", nickname);
boolean result = fixture.isClientConnected(nickname);
assertTrue(result);
}
Client Class:
//connects to the server
#Override
public void connect(String host, String nickname) {
try {
clientNickname = nickname;
System.out.println("Address= " + host + " and NickName = " + nickname);
socket = new Socket(host, NetServerComm.PORTO); //8080
System.out.println("Socket = " + socket);
serverMsg = new NetServerSideMessage(ServerSideMessage.Type.CLIENT_CONNECTED, null, clientNickname);
out = new ObjectOutputStream(socket.getOutputStream());
out.writeObject(serverMsg);
isConnected = true;
messageHandler = new Thread(new MessageHandler(socket));
messageHandler.start();
} catch (IOException e) {
System.out.println("Failed connecting to the server.");
e.printStackTrace();
}
}
Server Class:
//starts the server
#Override
public void start() {
try {
serverSocket = new ServerSocket(PORTO); //8080
System.out.println("Servidor is online: " + serverSocket);
try {
while (true) {
Socket clientSocket = serverSocket.accept();
//test hangs at the method above^^^
checkNewClient(clientSocket);
if (nameCheck == true)
new Thread(new ClientHandler(clientSocket)).start();
}
} finally {
System.out.println("Closing the server..");
serverSocket.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
It looks like you start() method is getting executed by the same Thread as the test, so it blocks there (at the accept() part). Could you try to have this method executed by a new Thread instead ? – Berger
That seems to have done the job :) – JDev

ActiveMQ + MQTT + subscribe to "ActiveMQ.Advisory.Connection"

This is the context:
A Java application subscribes to the Topic "ActiveMQ.Advisory.Connection" from an ActiveMQ 5.9.1 via MQTT (Paho 0.4.0):
public class SupervisorMqttClient implements MqttCallback {
private MqttClient client = null;
private MemoryPersistence persistence = null;
private MqttConnectOptions connOpts = null;
private final int STATUS_OK = 0;
private final int STATUS_ERROR = 1;
private String mqttServer = null;
private String clientId = null;
private int status = STATUS_OK;
public SupervisorMqttClient() {
try {
this.init();
} catch (MqttException e) {
Logger.error(e.getLocalizedMessage());
Logger.debug(e);
}
}
private void init() throws MqttException {
Properties props = PropertiesManager.getInstance("supervisor");
mqttServer = props.getProperty("supervisor.mqtt.server");
String supervisorID = props.getProperty("supervisor.mqtt.client.number");
clientId = Supervisor.APP_NAME+"-"+supervisorID;
connOpts = new MqttConnectOptions();
connOpts.setKeepAliveInterval(30);
connOpts.setCleanSession(true); // important non-durable
persistence = new MemoryPersistence();
client = new MqttClient(mqttServer, clientId, persistence);
connectAndSubscribe();
}
private void connectAndSubscribe() throws MqttSecurityException, MqttException {
try {
client.connect(connOpts);
client.setCallback(this);
client.subscribe("ActiveMQ/Advisory/Connection");
} catch (MqttSecurityException e) {
Logger.error(e.getLocalizedMessage());
Logger.debug(e);
} catch (MqttException e) {
Logger.error(e.getLocalizedMessage());
Logger.debug(e);
processError(e);
}
}
public void publish(String orderType, JSONObject jsonExtraData) {
if (status == STATUS_ERROR) {
connectAndSubscribe();
}
if (status == STATUS_OK) {
// some code here
}
}
#Override
public void connectionLost(Throwable err) {
Logger.info("Connection lost");
}
#Override
public void deliveryComplete(IMqttDeliveryToken arg0) {
Logger.info("deliveryComplete");
}
#Override
public void messageArrived(String topic, MqttMessage msg) throws Exception {
System.out.println("MQTT Mesage Arrived[" + topic + "] Msg[" + msg.toString() + "]");
}
private void processError(MqttException e) {
status = STATUS_ERROR;
try {
if (client.isConnected()) {
Logger.error("disconnecting");
client.disconnect();
}
} catch (MqttException ex) {
Logger.error(ex.getLocalizedMessage());
Logger.debug(ex);
}
}
}
The connection with ActiveMQ is established fine. This topic offers information about the connections (open/close) in the ActiveMQ, but my problem is that messages I catch are empty:
MQTT Mesage Arrived[ActiveMQ/Advisory/Connection] Msg[]
Is there any way to catch them using MQTT? or I should use JMS for that?
Thanks,
Jon Ander.
The question would be what do you want the MQTT client to receive on the Advisory topic as the message body. The advisories generally include much of the information as message properties however those cannot be mapped to MQTT as MQTT messages don't have properties. The body of the Connection advisory is a copy of the ConnectionInfo object that was used to create the connection. On the MQTT side there is not much you could do with that as all you would receive would be the serialized bytes of that object which you wouldn't be able to do anything with.

How to implement java production grade RabbitMQ consumer

I having a problem with the RabbitMQ Work Queue implementation. im current running it in Tomcat, and i have the following class constantly listerning to new task in the queue. But after a day or two, sudden it behaving strangely, where by the object DeliveryOK return by channel.queueDeclare(taskQueueName, isDurable, false, false, null); is always zero. (i print out this in the log below mentioning "Current poolSize").
But in Rabbit admin (./rabbitmqadmin list queues or the RabbitMq Admin portal) it always return a number greater than zero (say 1267 messages in the queue). And it will not reduce to zero until i restart the tomcat, the class below only able to detect that there are actually some messages in the queue.
Initially i thought that this class was terminated somehow, but it is able to consume those messages that newly arrive. It will not consume those 1267 messages that is left hanging inside the queue. For example messages 1267 in the queue, will not be consume until i restart tomcat.
From the code below, is it because buggy implementation or is there a better way to implement a queue consumer specifically for RabbitMQ? i have read a related stack post(Producer/Consumer threads using a Queue), but im not sure if it helps.
Also, is it true that this consumer implementation below will not survive a RunTimeException?
MqConsumer Class:
#Service
public class MqConsumer implements Runnable{
private static final Logger logger = LoggerFactory.getLogger(MqConsumer.class);
private final int MAX_ALERT_THRESHOLD = 10000;
#Autowired
private AsynchSystemConnections asynchSystemConnections;
public MqConsumer(){
}
#PostConstruct
private void init() {
(new Thread(new MqConsumer(asynchSystemConnections))).start();
}
public MqConsumer(AsynchSystemConnections asynchSystemConnections){
this.asynchSystemConnections = asynchSystemConnections;
}
#Override
public void run() {
logger.info("Execute Consumer instance...");
while (true) { // infinite loop until it die due server restart
boolean toSleep = consume(asynchSystemConnections);
if (toSleep){
logger.error("Sleeping for 1 second...");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
logger.error("", e);
}
}
}
}
private boolean consume(AsynchSystemConnections asynchSystemConnections) {
com.rabbitmq.client.Connection mqConnection = null;
Channel mqChannel = null;
DatasiftMq dMq = null;
try {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(asynchSystemConnections.getMqServerHost());
mqConnection = factory.newConnection();
mqChannel = mqConnection.createChannel();
//consumePushInteractionJob method will forward to AsynchTwService.consume(connection, channel, AsynchTwService.PUSH_INTERACTION_QUEUE )
dMq = asynchSystemConnections.getAsynchService().consumePushInteractionJob(mqConnection, mqChannel);
int poolSize = asynchSystemConnections.getAsynchService().getPushInteractionQueueSize();
logger.info("Current poolSize: " + poolSize);
} catch(NullPointerException e) {
logger.error("", e);
if (dMq != null) {
try {
logger.error("Removing JSON with" + dMq.getLogHeader(dMq));
asynchSystemConnections.getAsynchService().ack(mqChannel, dMq.getDelivery());
logger.error("Removed JSON with" + dMq.getLogHeader(dMq));
} catch (IOException e1) {
logger.error("Remove JSON Failed: ", e);
}
}
return true;
} catch (IOException e) {
logger.error("Unable to create new MQ Connection from factory.", e);
return true;
} catch (InterruptedException e) {
logger.error("", e);
return true;
} catch (ClassNotFoundException e) {
logger.error("", e);
return true;
} catch (Exception e) {
logger.error("Big problem, better solve this fast!!", e);
asynchSystemConnections.getNotificationService().notifySystemException(null, e);
return true;
} finally {
try {
asynchSystemConnections.getAsynchService().ack(mqChannel, dMq.getDelivery());
asynchSystemConnections.getAsynchService().disconnect(mqConnection, mqChannel);
} catch (IOException e) {
logger.error("", e);
}
}
return false;
}
AsynchTwService Class:
#Service("asynchTwService")
public class AsynchTwService implements AsynchService {
static final String FAVOURITE_COUNT_QUEUE = "favourite_count_queue";
static final String FRIENDS_FOLLOWERS_QUEUE = "friends_followers_queue";
static final String DIRECT_MESSAGE_RECEIVE_QUEUE = "direct_message_receive_queue";
static final String PUSH_INTERACTION_QUEUE = "push_interaction_queue";
private static String mqServerHost;
private static final Logger logger = LoggerFactory.getLogger(AsynchTwService.class);
private static final boolean isDurable = true;
private boolean autoAck = false;
private ConcurrentHashMap<String, Integer> currentQueueSize = new ConcurrentHashMap<String, Integer>();
#Override
public Connection getConnection() throws IOException{
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(mqServerHost);
return factory.newConnection();
}
#Override
public void produce(Connection connection, Channel channel, Object object, String taskQueueName) throws IOException {
sendToQueue(connection, channel, object, taskQueueName);
}
#Override
public QueueItem consume(Connection connection, Channel channel, String taskQueueName) throws IOException, InterruptedException, ClassNotFoundException{
Serializer serializer = new Serializer();
try {
Delivery delivery = listenFromQueue(connection, channel, taskQueueName);
Object messageObj = serializer.toObject(delivery.getBody());
QueueItem queueItem = (QueueItem)messageObj;
queueItem.setDelivery(delivery);
return queueItem;
} catch (InterruptedException e) {
throw e;
} catch (ClassNotFoundException e) {
logger.error("Unable to serialize the message to QueueItem object", e);
throw e;
}
}
#Override
public int getQueueSize(String taskQueueName){
return this.currentQueueSize.get(taskQueueName);
}
private Delivery listenFromQueue(Connection connection, Channel channel, String taskQueueName) throws IOException, InterruptedException, ClassNotFoundException{
try {
DeclareOk ok = channel.queueDeclare(taskQueueName, isDurable, false, false, null);
currentQueueSize.put(taskQueueName, ok.getMessageCount());
logger.info("Queue ("+ taskQueueName + ") has items: " +ok.getMessageCount());
} catch (IOException e) {
// TODO Auto-generated catch block
}
logger.info(" [*] Consuming "+taskQueueName+" message...");
QueueingConsumer consumer = new QueueingConsumer(channel);
try {
channel.basicConsume(taskQueueName, autoAck, consumer);
} catch (IOException e) {
logger.error("", e);
}
try {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
return delivery;
} catch (ShutdownSignalException e) {
logger.error("Unable to retrieve message from Queue", e);
throw e;
} catch (ConsumerCancelledException e) {
logger.error("Unable to retrieve message from Queue", e);
throw e;
} catch (InterruptedException e) {
logger.error("Unable to retrieve message from Queue", e);
throw e;
}
}
private void sendToQueue(Connection connection, Channel channel, Object object, String taskQueueName) throws IOException{
//Initialization, create Message Queue broker connection
try{
channel.queueDeclare(taskQueueName, isDurable, false, false, null);
}catch(IOException e) {
logger.error(e.getMessage());
logger.error("Error create Message Queue connection for queue name:" + taskQueueName, e);
throw e;
}
//send message to broker
try {
long start = System.currentTimeMillis();
Serializer serializer = new Serializer();
logger.info("Sending Twitter QueueItem to Message Queue...");
channel.basicPublish("", taskQueueName, MessageProperties.PERSISTENT_TEXT_PLAIN,
serializer.toBytes(object));
logger.info("Queue successfully sent, process took: " + (System.currentTimeMillis()-start)+ "ms");
} catch (IOException e) {
logger.error("Error while sending object to queue : " + taskQueueName, e);
throw e;
}
}
public static String getMqServerHost() {
return mqServerHost;
}
public static void setMqServerHost(String mqServerHost) {
AsynchTwService.mqServerHost = mqServerHost;
}
#Override
public void disconnect(Connection connection, Channel channel) throws IOException{
try {
if (channel != null){
if (channel.isOpen()){
channel.close();
}
}
if (connection != null){
if (connection.isOpen()){
connection.close();
}
}
logger.debug("MQ Channel Disconnected");
} catch (IOException e) {
throw e;
}
}
#Override
public void ack(Channel channel, QueueingConsumer.Delivery delivery) throws IOException {
// this is made as another method call is to avoid Ack too fast un intentionally
try {
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
logger.info("[x] acked" );
} catch (IOException e) {
logger.error("Unable Acknowledge Queue Message", e);
throw e;
}
}
#Override
public DatasiftMq consumeDatasiftInteraction(Connection connection, Channel channel,
String taskQueueName) throws IOException, InterruptedException, ClassNotFoundException {
Serializer serializer = new Serializer();
try {
Delivery delivery = listenFromQueue(connection, channel, taskQueueName);
Object messageObj = serializer.toObject(delivery.getBody());
DatasiftMq dto = (DatasiftMq)messageObj;
dto.setDelivery(delivery);
return dto;
} catch (InterruptedException e) {
throw e;
} catch (ClassNotFoundException e) {
logger.error("Unable to serialize the message to DatasiftDTO object", e);
throw e;
}
}
#Override
public void reQueue(Channel channel, Delivery delivery) throws IOException {
try {
channel.basicNack(delivery.getEnvelope().getDeliveryTag(), false, true);
logger.info("[x] Nacked" );
} catch (IOException e) {
logger.error("Unable Acknowledge Queue Message", e);
throw e;
}
}
}
Seems like you are missing some basics here.
Taken from here and some code of mine.
Setting up the connection outside of the consumer thread:
//executed once
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("someHost");
factory.setUsername("user");
factory.setPassword("pass");
Connection connection = factory.newConnection();
What you have to do inside your thread:
//Consumer - executed in a Thread
QueueingConsumer consumer = new QueueingConsumer(connection.createChannel());
boolean autoAck = false;
channel.basicConsume("hello", autoAck, consumer);
while (!Thread.current().isInterrupted())) {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
//...
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
}
In general I do still recommand you check out the spring-amqp library it integrates perfectly.

MQTT client re-using for publish / subscribe

I have an web server which is getting a lot of messages to the same topic and is returning response messages to another topic.
I am currently re-using the same MQTT client instance both for the callback and for sending the response messages by keeping the MQTT client connected all the time.
However, after one cycle of receiving a message and sending a response, I am able to receive another message but cannot send the response - I have to restart the application server.
Is it a good approach to have a single MQTTclient instance? Is it OK to keep it connected all the time? What's the best approach for this kind of requirement?
Here is my code:
public static void registerCallBack(String topicName, String userName,
String password, String clientId, MqttCallback callback,
MqttClient client) {
MqttConnectOptions options = new MqttConnectOptions();
options.setCleanSession(true);
options.setKeepAliveInterval(30);
options.setUserName(userName);
options.setPassword(password.toCharArray());
// Connect to Broker
try {
options.setSocketFactory(SslUtil.getSocketFactory(
ManagerProps.MQTT_BROKER_CA_FILE.getValue(), ""));
client.setCallback(callback);
client.connect(options);
client.subscribe(topicName, 0);
log.info("successfuly registered callback to topic " + topicName);
} catch (MqttException me) {
log.error("MqttException, " + me);
} catch (Exception e) {
log.error("Exception, " + e);
}
}
public static String publishMessage(MqttClient client, String message,
String topic, String userName, String password) {
MqttConnectOptions options = new MqttConnectOptions();
options.setCleanSession(true);
options.setKeepAliveInterval(30);
options.setUserName(userName);
options.setPassword(password.toCharArray());
try {
MqttMessage msg = new MqttMessage();
msg.setPayload(message.getBytes());
client.publish(topic, msg);
} catch (MqttException e) {
log.error("MqttException, " + e);
} catch (Exception e) {
log.error("Exception, " + e);
}
return message;
}
I was seeing something similar, and got this to work:
final CallbackConnection connection = mqtt.callbackConnection();
connection.listener(new org.fusesource.mqtt.client.Listener() {
public void onConnected() {
}
public void onDisconnected() {
}
public void onFailure(Throwable value) {
value.printStackTrace();
System.exit(-2);
}
public void onPublish(UTF8Buffer topic, Buffer msg, Runnable ack) {
String body = msg.utf8().toString();
if( body.startsWith("REPLY: ")) {
// Don't reply to your own reply
System.out.println("Replied");
System.out.println("");
} else {
try{
byte[] reply = "REPLY: Hello Back".getBytes();
connection.publish(destination, reply, QoS.AT_MOST_ONCE, true, null) ;
msg.clear();
}catch (Exception e){
e.printStackTrace();
}
}
}
});

Categories