Thanks for this opportunity to ask questions here.
Summary
I have a spring boot application which runs on Tomcat. I'm subscribing ActiveMQ topic with StompClient. After connected i get messages via StompHandler's handleFrame method.
In that method i create MessageUtils which implements Runnable interface. In MessageUtils run method, creating new thread with ExecutorService and do the task generateMessage which send messages to kafka topic.
Problem
When I shutdown the tomcat, threads are still alive. In catalina.out;
A web application appears to have started a thread named [foo] but has failed to stop it. This is very likely to create a memory leak
So the tomcat cannot shutdown properly.
Code Samples
#Service
StompService.class
#EventListener(ApplicationReadyEvent.class)
public void start() {
logI("Service run Client Methods");
List<String> topics = Arrays.asList(topicListString.split(","));
for (String topic : topics) {
StompClient client = new StompClient(topic, username, password, url, topic, bootstrapAddress);
try {
client.run();
runMap.put(topic, client);
boolean connected = client.getSession().isConnected();
logI("Topic: " + topic + " is connected: " + connected);
} catch (InterruptedException e) {
logE("InterruptedException during start of stomp client: ", e);
} catch (TimeoutException e) {
logE("TimeoutException during start of stomp client: ", e);
} catch (ExecutionException e) {
logE("ExecutionException during start of stomp client: ", e);
} catch (Exception e) {
logE("Unexpected exception during start of stomp client: ", e);
}
}
}
StompClient.class
public void run() throws ExecutionException, TimeoutException, InterruptedException {
WebSocketClient client = new StandardWebSocketClient();
WebSocketStompClient stompClient = new WebSocketStompClient(client);
ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();
taskScheduler.afterPropertiesSet();
StompHeaders connectHeaders = new StompHeaders();
connectHeaders.add("login", this.userName);
connectHeaders.add("passcode", this.password);
stompClient.getDefaultHeartbeat();
stompClient.setTaskScheduler(taskScheduler);
stompClient.setMessageConverter(new StringMessageConverter());
stompClient.setAutoStartup(true);
StompSessionHandler sessionHandler = new StompHandler(this.topic, this.bootstrapAddress);
StompSession stompSession = null;
try {
stompSession = stompClient.connect(url, new WebSocketHttpHeaders(), connectHeaders, sessionHandler)
.get(5, TimeUnit.SECONDS);
} catch (Exception e) {
logE("Cannot connect with stomp client." , e);
}
this.setSession(stompSession);
}
StompHandler.class which extends StompSessionHandlerAdapter
#Override
public void handleFrame(StompHeaders headers, Object payload) {
String msg = (String) payload;
MessageUtils message = new MessageUtils();
message.setHeaders(headers);
message.setTopic(topic);
message.setMsg(msg);
message.setBootstrapAddress(bootstrapAddress);
message.run();
}
MessageUtils.class
#Override
public void run() {
ExecutorService executorService = Executors.newSingleThreadExecutor();
executorService.execute(new Runnable() {
public void run() {
generateMessage(getMsg().toString());
}
});
executorService.shutdown();
try {
executorService.awaitTermination(200, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
logI("InterruptedException during await termination", e);
}
}
I can't figure out why nothing happens.
I'm trying to write a java program that sends a json object to a jetty server.
The server is already written(by someone else,its an project) and only excepts certain json objects. But he doesn't get anything from my program.
public class client {
final static String HOST = "localhost";
final static int PORT = 3000;
public static void main(String[] args)
{
String URIString = "ws://" + HOST + ":" + PORT + "/servlets";
URI uri = URI.create(URIString);
WebSocketClient client = new WebSocketClient();
JSONObject js = new JSONObject();
js.put("toke","hallo");
Clientsocket clientsocket = new Clientsocket();
try {
client.start();
Future<Session> fut = client.connect(clientsocket, uri);
clientsocket.getSession().getRemote().sendString(js.toJSONString());;
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
public class Clientsocket extends WebSocketAdapter {
//private static final Logger LOG = Log.getLogger(Clientsocket.class);
#Override
public void onWebSocketClose(int statusCode,String reason)
{
super.onWebSocketClose(statusCode, reason);
//LOG.info("Websocket Close:{} - {} ", statusCode,reason);
}
#Override
public void onWebSocketConnect(Session session)
{
super.onWebSocketConnect(session);
//LOG.info("Websocket Connect: {}", session);
}
}
error message when started:
2018-06-22 14:26:43.789:INFO::main: Logging initialized #257ms to org.eclipse.jetty.util.log.StdErrLog
java.lang.NullPointerException
[both classes: clientsocket, client][1]
I think you didn't wait for the connect to complete.
Future<Session> fut = client.connect(clientsocket, uri);
Session session = fut.get(); // wait for connect to complete (or throw exception)
session.getRemote().sendString(js.toJSONString());
We are creating a java listener to read multiple device data those are configured on particular server ip and port.Device following below rule.
device send a login packet.
server will return ack packet in response.
after receive ack device will send information packet.
server reads that data.
on last step we stuck, we are sending the ack but cant get the information packet back from device(though we check the generated ack through opensource sofware).For ref we are attaching code.(if we remove while(true) than get login packet but after that socket connection will close and again device will send login packet but if we keep it then we dont get any packet)
//--------------Main class------------------------------------------
public class Main {
public static void main(String[] args) {
Server server = new Server(listen_port, pool_size, pm);
new Thread(server).start();
logger.info("Server Started .....");
}
}
//--------------------------------------------------------------
public class Server implements Runnable {
private ServerSocket serverSocket = null;
public void run()
{
this.m_stop = false;
while (!this.m_stop)
try {
this.m_pool.execute(new Handler(this.serverSocket.accept()));
} catch (IOException e) {
LOGGER.debug("Unable to accept connection ", e);
}
}
}
//--------------------------------------------------------------
public class Handler implements Runnable {
private Socket m_clientSocket;
private String imei;
public Handler(Socket socket) {
this.m_clientSocket = socket;
}
public void run() {
DataOutputStream clientDataOS = null;
DataInputStream clientDataIS = null;
try {
logger.info("data is coming");
m_clientSocket.setSoTimeout(300000);
clientDataIS = new DataInputStream(this.m_clientSocket.getInputStream());
clientDataOS = new DataOutputStream(this.m_clientSocket.getOutputStream());
while (true) {
String pkt = "";
logger.info("Waiting for input strem");
byte[] byte_pkt = IOUtils.toByteArray(clientDataIS);
logger.info("Got input stream");
for (byte b : byte_pkt) {
pkt += String.format("%02X ", b);
}
logger.info(pkt);
if (byte_pkt.length > 0) {
logger.info("");
if (Byte.valueOf(byte_pkt[3]) == 1) {
imei = "xyz";
logger.info("login packet");
byte[] rep_pkt = Utils.getReceptionPacket(byte_pkt);//send back to device
clientDataOS.write(rep_pkt);
clientDataOS.flush();
} else if (Byte.valueOf(byte_pkt[3]) == 34) {
logger.info("information packet");
Utils.processPackets(byte_pkt);
} else {
logger.info("Unkown packet format");
}
logger.info(imei);
} else {
logger.info("InputStream is empty.");
}
}
} catch (SocketException se) {
logger.error("Failure on reading data", se);
} catch (IOException e) {
logger.error("Failure on reading data", e);
} catch (Exception e) {
logger.error("Error while processing data", e);
} finally {
try {
IOUtils.closeQuietly(clientDataOS);
IOUtils.closeQuietly(clientDataIS);
this.m_clientSocket.close();
} catch (IOException e) {
logger.debug("Error when sending out response ::", e);
}
}
}
}
I having a problem with the RabbitMQ Work Queue implementation. im current running it in Tomcat, and i have the following class constantly listerning to new task in the queue. But after a day or two, sudden it behaving strangely, where by the object DeliveryOK return by channel.queueDeclare(taskQueueName, isDurable, false, false, null); is always zero. (i print out this in the log below mentioning "Current poolSize").
But in Rabbit admin (./rabbitmqadmin list queues or the RabbitMq Admin portal) it always return a number greater than zero (say 1267 messages in the queue). And it will not reduce to zero until i restart the tomcat, the class below only able to detect that there are actually some messages in the queue.
Initially i thought that this class was terminated somehow, but it is able to consume those messages that newly arrive. It will not consume those 1267 messages that is left hanging inside the queue. For example messages 1267 in the queue, will not be consume until i restart tomcat.
From the code below, is it because buggy implementation or is there a better way to implement a queue consumer specifically for RabbitMQ? i have read a related stack post(Producer/Consumer threads using a Queue), but im not sure if it helps.
Also, is it true that this consumer implementation below will not survive a RunTimeException?
MqConsumer Class:
#Service
public class MqConsumer implements Runnable{
private static final Logger logger = LoggerFactory.getLogger(MqConsumer.class);
private final int MAX_ALERT_THRESHOLD = 10000;
#Autowired
private AsynchSystemConnections asynchSystemConnections;
public MqConsumer(){
}
#PostConstruct
private void init() {
(new Thread(new MqConsumer(asynchSystemConnections))).start();
}
public MqConsumer(AsynchSystemConnections asynchSystemConnections){
this.asynchSystemConnections = asynchSystemConnections;
}
#Override
public void run() {
logger.info("Execute Consumer instance...");
while (true) { // infinite loop until it die due server restart
boolean toSleep = consume(asynchSystemConnections);
if (toSleep){
logger.error("Sleeping for 1 second...");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
logger.error("", e);
}
}
}
}
private boolean consume(AsynchSystemConnections asynchSystemConnections) {
com.rabbitmq.client.Connection mqConnection = null;
Channel mqChannel = null;
DatasiftMq dMq = null;
try {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(asynchSystemConnections.getMqServerHost());
mqConnection = factory.newConnection();
mqChannel = mqConnection.createChannel();
//consumePushInteractionJob method will forward to AsynchTwService.consume(connection, channel, AsynchTwService.PUSH_INTERACTION_QUEUE )
dMq = asynchSystemConnections.getAsynchService().consumePushInteractionJob(mqConnection, mqChannel);
int poolSize = asynchSystemConnections.getAsynchService().getPushInteractionQueueSize();
logger.info("Current poolSize: " + poolSize);
} catch(NullPointerException e) {
logger.error("", e);
if (dMq != null) {
try {
logger.error("Removing JSON with" + dMq.getLogHeader(dMq));
asynchSystemConnections.getAsynchService().ack(mqChannel, dMq.getDelivery());
logger.error("Removed JSON with" + dMq.getLogHeader(dMq));
} catch (IOException e1) {
logger.error("Remove JSON Failed: ", e);
}
}
return true;
} catch (IOException e) {
logger.error("Unable to create new MQ Connection from factory.", e);
return true;
} catch (InterruptedException e) {
logger.error("", e);
return true;
} catch (ClassNotFoundException e) {
logger.error("", e);
return true;
} catch (Exception e) {
logger.error("Big problem, better solve this fast!!", e);
asynchSystemConnections.getNotificationService().notifySystemException(null, e);
return true;
} finally {
try {
asynchSystemConnections.getAsynchService().ack(mqChannel, dMq.getDelivery());
asynchSystemConnections.getAsynchService().disconnect(mqConnection, mqChannel);
} catch (IOException e) {
logger.error("", e);
}
}
return false;
}
AsynchTwService Class:
#Service("asynchTwService")
public class AsynchTwService implements AsynchService {
static final String FAVOURITE_COUNT_QUEUE = "favourite_count_queue";
static final String FRIENDS_FOLLOWERS_QUEUE = "friends_followers_queue";
static final String DIRECT_MESSAGE_RECEIVE_QUEUE = "direct_message_receive_queue";
static final String PUSH_INTERACTION_QUEUE = "push_interaction_queue";
private static String mqServerHost;
private static final Logger logger = LoggerFactory.getLogger(AsynchTwService.class);
private static final boolean isDurable = true;
private boolean autoAck = false;
private ConcurrentHashMap<String, Integer> currentQueueSize = new ConcurrentHashMap<String, Integer>();
#Override
public Connection getConnection() throws IOException{
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(mqServerHost);
return factory.newConnection();
}
#Override
public void produce(Connection connection, Channel channel, Object object, String taskQueueName) throws IOException {
sendToQueue(connection, channel, object, taskQueueName);
}
#Override
public QueueItem consume(Connection connection, Channel channel, String taskQueueName) throws IOException, InterruptedException, ClassNotFoundException{
Serializer serializer = new Serializer();
try {
Delivery delivery = listenFromQueue(connection, channel, taskQueueName);
Object messageObj = serializer.toObject(delivery.getBody());
QueueItem queueItem = (QueueItem)messageObj;
queueItem.setDelivery(delivery);
return queueItem;
} catch (InterruptedException e) {
throw e;
} catch (ClassNotFoundException e) {
logger.error("Unable to serialize the message to QueueItem object", e);
throw e;
}
}
#Override
public int getQueueSize(String taskQueueName){
return this.currentQueueSize.get(taskQueueName);
}
private Delivery listenFromQueue(Connection connection, Channel channel, String taskQueueName) throws IOException, InterruptedException, ClassNotFoundException{
try {
DeclareOk ok = channel.queueDeclare(taskQueueName, isDurable, false, false, null);
currentQueueSize.put(taskQueueName, ok.getMessageCount());
logger.info("Queue ("+ taskQueueName + ") has items: " +ok.getMessageCount());
} catch (IOException e) {
// TODO Auto-generated catch block
}
logger.info(" [*] Consuming "+taskQueueName+" message...");
QueueingConsumer consumer = new QueueingConsumer(channel);
try {
channel.basicConsume(taskQueueName, autoAck, consumer);
} catch (IOException e) {
logger.error("", e);
}
try {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
return delivery;
} catch (ShutdownSignalException e) {
logger.error("Unable to retrieve message from Queue", e);
throw e;
} catch (ConsumerCancelledException e) {
logger.error("Unable to retrieve message from Queue", e);
throw e;
} catch (InterruptedException e) {
logger.error("Unable to retrieve message from Queue", e);
throw e;
}
}
private void sendToQueue(Connection connection, Channel channel, Object object, String taskQueueName) throws IOException{
//Initialization, create Message Queue broker connection
try{
channel.queueDeclare(taskQueueName, isDurable, false, false, null);
}catch(IOException e) {
logger.error(e.getMessage());
logger.error("Error create Message Queue connection for queue name:" + taskQueueName, e);
throw e;
}
//send message to broker
try {
long start = System.currentTimeMillis();
Serializer serializer = new Serializer();
logger.info("Sending Twitter QueueItem to Message Queue...");
channel.basicPublish("", taskQueueName, MessageProperties.PERSISTENT_TEXT_PLAIN,
serializer.toBytes(object));
logger.info("Queue successfully sent, process took: " + (System.currentTimeMillis()-start)+ "ms");
} catch (IOException e) {
logger.error("Error while sending object to queue : " + taskQueueName, e);
throw e;
}
}
public static String getMqServerHost() {
return mqServerHost;
}
public static void setMqServerHost(String mqServerHost) {
AsynchTwService.mqServerHost = mqServerHost;
}
#Override
public void disconnect(Connection connection, Channel channel) throws IOException{
try {
if (channel != null){
if (channel.isOpen()){
channel.close();
}
}
if (connection != null){
if (connection.isOpen()){
connection.close();
}
}
logger.debug("MQ Channel Disconnected");
} catch (IOException e) {
throw e;
}
}
#Override
public void ack(Channel channel, QueueingConsumer.Delivery delivery) throws IOException {
// this is made as another method call is to avoid Ack too fast un intentionally
try {
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
logger.info("[x] acked" );
} catch (IOException e) {
logger.error("Unable Acknowledge Queue Message", e);
throw e;
}
}
#Override
public DatasiftMq consumeDatasiftInteraction(Connection connection, Channel channel,
String taskQueueName) throws IOException, InterruptedException, ClassNotFoundException {
Serializer serializer = new Serializer();
try {
Delivery delivery = listenFromQueue(connection, channel, taskQueueName);
Object messageObj = serializer.toObject(delivery.getBody());
DatasiftMq dto = (DatasiftMq)messageObj;
dto.setDelivery(delivery);
return dto;
} catch (InterruptedException e) {
throw e;
} catch (ClassNotFoundException e) {
logger.error("Unable to serialize the message to DatasiftDTO object", e);
throw e;
}
}
#Override
public void reQueue(Channel channel, Delivery delivery) throws IOException {
try {
channel.basicNack(delivery.getEnvelope().getDeliveryTag(), false, true);
logger.info("[x] Nacked" );
} catch (IOException e) {
logger.error("Unable Acknowledge Queue Message", e);
throw e;
}
}
}
Seems like you are missing some basics here.
Taken from here and some code of mine.
Setting up the connection outside of the consumer thread:
//executed once
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("someHost");
factory.setUsername("user");
factory.setPassword("pass");
Connection connection = factory.newConnection();
What you have to do inside your thread:
//Consumer - executed in a Thread
QueueingConsumer consumer = new QueueingConsumer(connection.createChannel());
boolean autoAck = false;
channel.basicConsume("hello", autoAck, consumer);
while (!Thread.current().isInterrupted())) {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
//...
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
}
In general I do still recommand you check out the spring-amqp library it integrates perfectly.
I have an web server which is getting a lot of messages to the same topic and is returning response messages to another topic.
I am currently re-using the same MQTT client instance both for the callback and for sending the response messages by keeping the MQTT client connected all the time.
However, after one cycle of receiving a message and sending a response, I am able to receive another message but cannot send the response - I have to restart the application server.
Is it a good approach to have a single MQTTclient instance? Is it OK to keep it connected all the time? What's the best approach for this kind of requirement?
Here is my code:
public static void registerCallBack(String topicName, String userName,
String password, String clientId, MqttCallback callback,
MqttClient client) {
MqttConnectOptions options = new MqttConnectOptions();
options.setCleanSession(true);
options.setKeepAliveInterval(30);
options.setUserName(userName);
options.setPassword(password.toCharArray());
// Connect to Broker
try {
options.setSocketFactory(SslUtil.getSocketFactory(
ManagerProps.MQTT_BROKER_CA_FILE.getValue(), ""));
client.setCallback(callback);
client.connect(options);
client.subscribe(topicName, 0);
log.info("successfuly registered callback to topic " + topicName);
} catch (MqttException me) {
log.error("MqttException, " + me);
} catch (Exception e) {
log.error("Exception, " + e);
}
}
public static String publishMessage(MqttClient client, String message,
String topic, String userName, String password) {
MqttConnectOptions options = new MqttConnectOptions();
options.setCleanSession(true);
options.setKeepAliveInterval(30);
options.setUserName(userName);
options.setPassword(password.toCharArray());
try {
MqttMessage msg = new MqttMessage();
msg.setPayload(message.getBytes());
client.publish(topic, msg);
} catch (MqttException e) {
log.error("MqttException, " + e);
} catch (Exception e) {
log.error("Exception, " + e);
}
return message;
}
I was seeing something similar, and got this to work:
final CallbackConnection connection = mqtt.callbackConnection();
connection.listener(new org.fusesource.mqtt.client.Listener() {
public void onConnected() {
}
public void onDisconnected() {
}
public void onFailure(Throwable value) {
value.printStackTrace();
System.exit(-2);
}
public void onPublish(UTF8Buffer topic, Buffer msg, Runnable ack) {
String body = msg.utf8().toString();
if( body.startsWith("REPLY: ")) {
// Don't reply to your own reply
System.out.println("Replied");
System.out.println("");
} else {
try{
byte[] reply = "REPLY: Hello Back".getBytes();
connection.publish(destination, reply, QoS.AT_MOST_ONCE, true, null) ;
msg.clear();
}catch (Exception e){
e.printStackTrace();
}
}
}
});