We are building an integration project using Apache Camel (Camel 2.10.3, Java DSL based).
We have a route that extracts data from a database (lets call it IN_DB), does some logic and inserts into another database (OUT_DB) once a day, and another route that subscribes to a JMS topic for XML data, does some logic and inserts it into the same database (OUT_DB) throughout the day.
The requirement is that when the JMS topic connection goes down for whatever reason, we keep trying to reconnect indefinitely, and once reconnection is successful we need to go back to the database (IN_DB) and do another load to fill in the gap where the topic was down.
My question is how can we do this logic ('I was connected then I got disconnected and now I am connected again') in Camel? What happens to the route that starts with a topic consumer when the topic goes down, will the route just stop? Or will it issue an error message to some error queue? Do I have to write my own handler to monitor the topic connection, or will Camel automatically reconnect when the topic comes back up and set some message header, or set some context variable to indicate that the 'I was connected then I got disconnected and now I am connected again' scenario has happened? I am happy building the route logic around calling the database load I just can't figure out the best way to 'detect' in Camel that this scenario has happened.
Any suggestions much appreciated.
In terms of the reconnecting to your queue, it depends on what JMS broker you are using. If you are using ActiveMQ then you can configure how it reconnects through the URI so it can try and reconnect to another broker, reconnect to the same broker after a timeout etc. The docs for it are here.
To detect when the connection has failed, the easiest from the point of the view of the program is to just use persistent queues instead of topics. However, assuming that isn't feasible then I think you've got two options.
Define a JMS exception listener. This should let you know when the underlying connection has disappeared.
To detect when its back up again I think you're stuck with posting a message to a particular topic and watching for messages from this topic in another route. When you read a message on this topic, you know the broker has come back up so you can reload your data from the DB.
or 2. You could post regular hearbeat messages onto your JMS topic. If you stop receiving them you know the broker has gone down. Once you start to get them again, you know it is back up and you need to reload data from the DB.
Sorry to answer some years later ! I was looking for a solution for this problem since a few days, in order to supervising an AMQP connection in an Apache Camel application.
I finally find a way, I do not know if in the last Camel versions (i am using 2.25.4), it is possible do listen more "natively" the JMS connection changes :
The JMS consumer route must use the option "messageListenerContainerFactory" in the URL : "messageListenerContainerFactory=#customJmsListernerContainerFactory"
The factory would be instancied before of course and added to a registry associated to the camel context :
MessageListenerContainerFactory customJmsListenerContainerFactory = new CustomJmsListenerContainerFactory();
this.registry.put("customJmsListernerContainerFactory", customJmsListenerContainerFactory);
not working on producer, for this kind, you must add an exception clause :
onException(javax.jms.JMSException.class)...
create the class CustomJmsListenerContainerFactory used above, implementing MessageListenerContainerFactory :
public class CustomJmsListenerContainerFactory implements MessageListenerContainerFactory {
#Override
public AbstractMessageListenerContainer createMessageListenerContainer(JmsEndpoint endpoint) {
CustomJmsListenerContainer customJmsListener = new CustomJmsListenerContainer();
return customJmsListener;
}
create a class CustomJmsListenerContainer extending DefaultMessageListenerContainer AND implementing JmsConnectionListener. The trick is to ovveride createConnection, in order to temporary get the created connection in order to manually add a listener (the container itself):
public class CustomJmsListenerContainer extends DefaultMessageListenerContainer implements JmsConnectionListener {
protected Connection createConnection() throws JMSException {
JmsConnection myConnection = (JmsConnection)super.createConnection();
this.myConnection.addConnectionListener(this);
return myConnection;
}
#Override
public void onConnectionEstablished(URI remoteURI) {
LOGGER.info("JMS connection detected");
#Override
public void onConnectionFailure(Throwable error) {
LOGGER.info("JMS connection failure");
}
#Override
public void onConnectionInterrupted(URI remoteURI) {
LOGGER.info("JMS connection interrupted");
#Override
public void onConnectionRestored(URI remoteURI) {
LOGGER.info("JMS connection restored");
#Override
public void onInboundMessage(JmsInboundMessageDispatch envelope) {
// TODO Auto-generated method stub
}
#Override
public void onSessionClosed(Session session, Throwable cause) {
// TODO Auto-generated method stub
}
#Override
public void onConsumerClosed(MessageConsumer consumer, Throwable cause) {
// TODO Auto-generated method stub
}
#Override
public void onProducerClosed(MessageProducer producer, Throwable cause) {
// TODO Auto-generated method stub
}
}
Related
I am setting up Genysys PSDK to connect to tServer and listen to events using handler. I have created
MessageHandler
to receive messages but looks like its not working somehow. Genesys Document states that we need to implement AsyncInvoker in order to receive events.
Can someone help me how can I setup AsyncInvoker when working with spring boot application?
Important:
You need to know that your event-handling logic will be executed by using the protocol invoker. Please set the invoker appropriate for your application needs.
I was opening tserver connection before setting handler.
Setting
tserver.setMessageHandler(new MessageHandler() {
#Override
public void onMessage(Message arg0) {
// TODO Auto-generated method stub
System.out.println("Message Handler" + arg0.toString());
}
});
before opening connection worked for me.
Could anyone tell me if the server-side implementation is using stomp WebSocket, is the client also expected to implement stomp?
I am trying to implement a spring boot application and I am confused if I should go with or without stomp implementation. From my research, I understand, if you want to scale the application, it is better to use stomp and embedded broker( RabbitMQ for eg.) as it will handle the sessions, heartbeat etc. instead of an in-memory broker.
The examples available online just shows implementations with and without stomp.
I am basically trying to get different datasets from the table upon client request and write to a WebSocket continuously.
Could anyone please confirm if my understanding so far is correct?
What are the essential things I will have to take care of if I go with stomp + websocket?
Updating the usecase below:
The mobile client would be displaying charts upon user login. There would be links in the left panel for eg. Sales, Discounts etc. which upon clicking, the request will reach server through websocket channel. Server will check the datatype in the request, generate the model using data from DB and write the data to the websocket.
Updating code - v1
MyWebSocketHandler:
#Component
public class MyWebSocketHandler extends TextWebSocketHandler {
Logger logger = LoggerFactory.getLogger(getClass());
#Autowired
DashboardUtil dashboardutil;
#Resource(name = "socketSessionsMap")
private Map<String, WebSocketSession> socketSessionsMap;
#Override
public void handleTextMessage(WebSocketSession session, TextMessage message)
throws InterruptedException, IOException {
try {
//Gets the socket session from map and writes a json to that socket - did for testing purpose.
socketSessionsMap.put("session", session);
//String payload = message.getPayload();
String jsonString = dashboardutil.getDataInJSON(); // gets hardcoded json model
session.sendMessage(new TextMessage(jsonString));
} catch (Exception e) {
e.printStackTrace();
}
}
#Override
public void afterConnectionEstablished(WebSocketSession session) throws Exception {
}
#Override
public void afterConnectionClosed(WebSocketSession session, CloseStatus status) throws Exception {
}
}
WebSecurityConfig:
#Configuration
#EnableWebSocket
public class WebSocketConfig implements WebSocketConfigurer {
#Autowired
private MyWebSocketHandler myWebSocketHandler;
#Override
public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
registry.addHandler(myWebSocketHandler, "/socketHandler").setAllowedOrigins("*").withSockJS();
}
}
Could anyone tell me if the server-side implementation is using stomp
WebSocket, is the client also expected to implement stomp?
You can register multiple handlers in your web socket configuration. So in theory you can provide a handler for STOMP and another one for plain web socket. If you only provide a STOMP handler then the handshake from a standard web socket client will fail.
From my research, I understand, if you want to scale the application,
it is better to use stomp and embedded broker( RabbitMQ for eg.) as it
will handle the sessions, heartbeat etc. instead of an in-memory
broker.
That's correct. STOMP also offers a few more nice features especially the subscription to certain endpoints.
I am basically trying to get different datasets from the table upon
client request and write to a WebSocket continuously.
That's a really simple description ...
You should think about if you need to share sessions across multiple instances or if you need to send messages between web socket sessions.
From your description it sounds like you just accept a web socket connection and continuously push data to the client. If you want to scale this application you can just put a load balancer in front of your instances and you are good to go.
I am developing an web application which supports TCP connection by Spring Integration.
It has two functions.
send messages to Server and receive reply from Server at a time
just receive messages from Server
(The application on server is not developed by Spring Integration.)
In this case, TCP Adapters should be used and I need provide to collaborate TCP Outbound and Inbound Channel Adapters.
But, my application has a few restrictions.
this application has no database.
messages' payload format is already specified. (It means I can not add any correlation data such as a transaction id to payload .
So, I think that collaborating TCP Outbound and Inbound Channel Adapters is difficult. Then, I plan to extend TCP Outbound gateway to add just receiving messages function.
How should I extend it ? (or do you have other ideas?)
Thanks in advance.
If you can determine the message type, something like this should work...
public class ExtendedTcpOutpboundGateway extends TcpOutboundGateway {
private final MessageChannel unsolicitedMessageChannel;
public ExtendedTcpOutpboundGateway(MessageChannel unsolicitedMessageChannel) {
this.unsolicitedMessageChannel = unsolicitedMessageChannel;
}
#Override
public boolean onMessage(Message<?> message) {
if (isUnsolicitedMessage(message)) {
this.messagingTemplate.send(this.unsolicitedMessageChannel, message);
return false;
}
else {
return super.onMessage(message);
}
}
private boolean isUnsolicitedMessage(Message<?> message) {
// TODO Add logic here to determine message type
return false;
}
}
If you can't tell the difference, it's not clear how you could implement your requirements.
I have just started experimenting with Spring and rabbitMQ.
I would like to create a microsevice infrastructure with rabbit and spring,
I have been following Spring boot tutorial
But it is very simplistic. As well I am looking at the documentation (springs, Rabbit) for how to create an RPC, i understand the Rabbits approach, but i would like to leverage Spring template to save me the boilerplate.
I just cant seem to understand where to register the reciveAndReplay callback at.
I tried doing this:
sending
System.out.println("Sending message...");
Object convertSendAndReceive = rabbitTemplate.convertSendAndReceive("spring-boot", "send and recive: sent");
System.out.println("GOT " + convertSendAndReceive); //is null
receiving
#Component
public class Receiver {
#Autowired
RabbitTemplate rabbitTemplate;
public void receiveMessage(String message) {
this.rabbitTemplate.receiveAndReply("spring-boot", (Message)->{
return "return this statement";
});
}
}
But its not a big surprise this doesn't work the message is received but nothing comes back. I assume that this needs to be registered somewhere in the factory/template at the bean creation level but i don't seem to understand where and sadly the documentation is unclear.
First, please use the Spring AMQP Documentation.
You would generally use a SimpleMessageListenerContainer wired with a POJO listener for RPC.
The template receiveAndReply method is intended for "scheduled" server-side RPC - i.e. only receive (and reply) when you want to, rather than whenever a message arrives in the queue. It does not block waiting for a message.
If you want to use receiveAndReply(), there's a test case that illustrates it.
EDIT:
This code...
this.template.convertAndSend(ROUTE, "test");
sends a message to the queue.
This code...
this.template.setQueue(ROUTE);
boolean received = this.template.receiveAndReply(new ReceiveAndReplyMessageCallback() {
#Override
public Message handle(Message message) {
message.getMessageProperties().setHeader("foo", "bar");
return message;
}
});
Receives a message and from that queue; adds a header and returns the same messsage to the reply queue. received will be false if there was no message to receive (and reply to).
This code:
Message receive = this.template.receive();
receives the reply.
This test is a bit contrived because the reply is sent to the same queue as the request. We can't use sendAndReceive() on the client side in this test because the thread would block waiting for the reply (and we need to execute the receiveAndReply()).
Another test in that class has a more realistic example where it does the sendAndReceive()s on different threads and the receiveAndReply()s on the main thread.
Note that that test uses a listener container on the client side for replies; that is generally no longer needed since the rabbit broker now supports direct reply-to.
receiveAndReply() was added for symmetry - in most cases, people use a listener container and listener adapter for server-side RPC.
I have got a Spring MDP implemented using Spring DefaultMessageListenderContainer listening to an input queue on WebSphere MQ v7.1. If there is a bad message coming in (that causes RuntimeException), what currently happens is that, the transaction is rolled back, and the message is put back into the queue. However the MDP goes into an infinite loop.
Question 1: For my requirements I would like to be able to shut down the processing the moment it sees a bad message. No retries needed. Is it possible to shutdown the message listener gracefully in case it sees a bad message (as opposed to crude System.exit() or methods of that sort)? I definitely don't like it to go into an infinite loop.
Edit:
Question 2: Is there a way to stop or suspend the listener container to stop further processing of messages?
The usual way to process this is to have an error queue and when you see a bad message to put it into the error queue.
Some systems handle this for you such as IBM MQ Series. You just need to configure the error queue and how many retries you want ant it will put it there.
An administrator will then look through these queues and take proper action on the messages that are in the queue (i.e. fix and resubmit them)
Actually, System.exit() is too brutal and... won't work. Retrying of failed messages is handled on the broker (WMQ) side so the message will be redelivered once you restart your application.
The problem you are describing is called poison-message and should be handled on the broker side. It seems to be described in Handling poison messages in WMQ manual and in How WebSphere Application Server handles poison messages.
I solved the problem in the following manner, not sure if this is the best way, however it works.
MDP Implements ApplicationContextAware; I also maintain a listener state (enum with OPEN, CLOSE, ERROR values) MDP Code fragment below:
//context
private ConfigurableApplicationContext applicationContext;
//listener state
private ListenerState listenerState = ListenerState.OPEN;
#Override
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
this.applicationContext = (ConfigurableApplicationContext) applicationContext;
}
//onMessage method
public void processMessages(....) {
try {
process(...);
} catch (Throwable t) {
listenerState = ListenerState.ERROR;
throw new RuntimeException(...);
}
}
#Override
public void stopContext() {
applicationContext.stop();
}
In the java main that loads the spring context i do this:
//check for errors for exit
Listener listener = (Listener)context.getBean("listener");
listenerContainer listenerContainer =
(ListenerContainer)context.getBean("listenerContainer");
try {
while(true) {
Thread.sleep(1000); //sleep for 1 sec
if(!listener.getListenerState().equals(ListenerState.OPEN)) {
listener.stopContext();
listenerContainer.stop();
System.exit(1);
}> }
} catch (InterruptedException e) {
throw new RuntimeException(e);
}