I am adding messages to rabbitmq queue using spring amqp template in my spring batch item writer.
public class AmqpAsynchRpcItemWriter<T> implements ItemWriter<T> {
protected String exchange;
protected String routingKey;
protected String queue;
protected String replyQueue;
protected RabbitTemplate template;
BlockingQueue<Object> blockingQueue;
public void onMessage(Object msgContent) {
try {
blockingQueue.put(msgContent);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
#Override
public void write(List<? extends T> items) throws Exception {
for (T item : items) {
Message message = MessageBuilder
.withBody(item.toString().getBytes())
.setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN)
.setReplyTo(this.replyQueue)
.setCorrelationId(item.toString().getBytes()).build();
template.send(this.exchange, this.routingKey, message);
}
for (T item : items) {
Object msg = blockingQueue.poll(60, TimeUnit.SECONDS);
if (msg instanceof Exception) {
throw (Exception) msg;
} else if (msg == null) {
System.out.println("reply timeout...");
break;
}
}
}
}
Messages are going to be processed on different remote servers. I am trying to handle the use case where if my message processing is failed (due to some exception) the step execution will be stopped.
I want to purge all the remaining messages in that queue so that remaining messages in queue should not be consumed and processed as they will also be failed.
If the step is failed, my item writer will again queue all the messages, so I need to purge all remaining message on any exception.
How can I purge the queue using spring amqp ?
I could do it using
admin.purgeQueue(this.queue, true);
I would use RabbitAdmin instead
http://docs.spring.io/autorepo/docs/spring-amqp-dist/1.3.4.RELEASE/api/org/springframework/amqp/rabbit/core/RabbitAdmin.html#purgeQueue%28java.lang.String,%20boolean%29
#Autowired private RabbitAdmin admin;
...
admin.purgeQueue("queueName", false);
you can use
AMQP.Queue.PurgeOk queuePurge(java.lang.String queue)
"See queuePurge:
http://www.rabbitmq.com/amqp-0-9-1-quickref.html#queue.purge"
Related
I am NOT able to stop an JMS consumer dynamically using a Spring Boot REST endpoint.
The number of consumers stays as is. No exceptions either.
IBM MQ Version: 9.2.0.5
pom.xml
<dependency>
<groupId>com.ibm.mq</groupId>
<artifactId>mq-jms-spring-boot-starter</artifactId>
<version>2.0.8</version>
</dependency>
JmsConfig.java
#Configuration
#EnableJms
#Log4j2
public class JmsConfig {
#Bean
public MQQueueConnectionFactory mqQueueConnectionFactory() {
MQQueueConnectionFactory mqQueueConnectionFactory = new MQQueueConnectionFactory();
mqQueueConnectionFactory.setHostName("my-ibm-mq-host.com");
try {
mqQueueConnectionFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
mqQueueConnectionFactory.setCCSID(1208);
mqQueueConnectionFactory.setChannel("my-channel");
mqQueueConnectionFactory.setPort(1234);
mqQueueConnectionFactory.setQueueManager("my-QM");
} catch (Exception e) {
log.error("Exception while creating JMS connecion...", e.getMessage());
}
return mqQueueConnectionFactory;
}
}
JmsListenerConfig.java
#Configuration
#Log4j2
public class JmsListenerConfig implements JmsListenerConfigurer {
#Autowired
private JmsConfig jmsConfig;
private Map<String, String> queueMap = new HashMap<>();
#Bean
public DefaultJmsListenerContainerFactory mqJmsListenerContainerFactory() throws JMSException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(jmsConfig.mqQueueConnectionFactory());
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setSessionTransacted(true);
factory.setConcurrency("5");
return factory;
}
#Override
public void configureJmsListeners(JmsListenerEndpointRegistrar registrar) {
queueMap.put("my-queue-101", "101");
log.info("queueMap: " + queueMap);
queueMap.entrySet().forEach(e -> {
SimpleJmsListenerEndpoint endpoint = new SimpleJmsListenerEndpoint();
endpoint.setDestination(e.getKey());
endpoint.setId(e.getValue());
try {
log.info("Reading message....");
endpoint.setMessageListener(message -> {
try {
log.info("Receieved ID: {} Destination {}", message.getJMSMessageID(), message.getJMSDestination());
} catch (JMSException ex) {
log.error("Exception while reading message - " + ex.getMessage());
}
});
registrar.setContainerFactory(mqJmsListenerContainerFactory());
} catch (JMSException ex) {
log.error("Exception while reading message - " + ex.getMessage());
}
registrar.registerEndpoint(endpoint);
});
}
}
JmsController.java
#RestController
#RequestMapping("/jms")
#Log4j2
public class JmsController {
#Autowired
ApplicationContext context;
#RequestMapping(value = "/stop", method = RequestMethod.GET)
public #ResponseBody
String haltJmsListener() {
JmsListenerEndpointRegistry listenerEndpointRegistry = context.getBean(JmsListenerEndpointRegistry.class);
Set<String> containerIds = listenerEndpointRegistry.getListenerContainerIds();
log.info("containerIds: " + containerIds);
//stops all consumers
listenerEndpointRegistry.stop(); //DOESN'T WORK :(
//stops a consumer by id, used when there are multiple consumers and want to stop them individually
//listenerEndpointRegistry.getListenerContainer("101").stop(); //DOESN'T WORK EITHER :(
return "Jms Listener stopped";
}
}
Here is the result that I noticed.
Initial # of consumers: 0 (as expected)
After server startup and queue connection, total # of consumers: 1 (as expected)
After hitting http://localhost:8080/jms/stop endpoint, total # of consumers: 1 (NOT as expected, should go back to 0)
Am I missing any configuration ?
You need to also call shutDown on the container; see my comment on this answer DefaultMessageListenerContainer's "isActive" vs "isRunning"
start()/stop() set/reset running; initialize()/shutDown() set/reset active. It depends on what your requirements are. stop() just stops the consumers from getting new messages, but the consumers still exist. shutDown() closes the consumers. Most people call stop + shutdown and then initialize + start to restart. But if you just want to stop consuming for a short time, stop/start is all you need.
You will need to iterate over the containers and cast them to call shutDown().
The configuration class(part):
public static RabbitQueueConfig clubProNotAvailableConfig =
new RabbitQueueConfig("club-pro-not-available", "club-pro-not-available", "club-pro-not-available-status", "3-3");
#Bean
public SimpleMessageListenerContainer listenerContainer5(ClubProNotAvailableListener listener, ConnectionFactory connectionFactory) {
return initListenerContainer(listener, clubProNotAvailableConfig, connectionFactory);
}
private SimpleMessageListenerContainer initListenerContainer(
ChannelAwareMessageListener listener,
RabbitQueueConfig config,
ConnectionFactory connectionFactory
) {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory);
listenerContainer.setQueueNames(config.getQueue());
listenerContainer.setMessageListener(listener);
listenerContainer.setAcknowledgeMode(AcknowledgeMode.MANUAL);
listenerContainer.setConcurrency(config.getThreadPoolSize());
listenerContainer.setPrefetchCount(1);
return listenerContainer;
}
Method of sending a message:
try {
success = clientRepository.updateAnketa(privatePersonProfile.getProfileId(), clubProAnketa, null);
} catch (ClubProNotAvailableException e) {
ClubProNotAvailableRabbit clubProNotAvailableRabbit = new ClubProNotAvailableRabbit();
clubProNotAvailableRabbit.setRequestContextRabbit(RequestContextRabbit.createContext(requestContextService.getContext()));
clubProNotAvailableRabbit.setCountRetry(0L);
clubProNotAvailableRabbit.setProfileId(privatePersonProfile.getProfileId());
clubProNotAvailableRabbit.setNameMethod(ChangeMethod.CHANGE_ANKETA);
clubProNotAvailableRabbit.setChangeAnketaData(anketa);
rabbitTemplate.convertAndSend(config.getExchange(), config.getRoutingKey(), clubProNotAvailableRabbit, new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setHeader("x-delay", 10000);
return message;
}
});
throw new ClubProNotAvailableException();
}
Configuration in the broker:
Queue configuration:
configuration of the exchanger:
I've read the documentation, tried a couple of options, but I can't apply it to my code.
What am I doing wrong? I will be very grateful for your help.
It looks like you don't have the delayed exchange plugin; you have also declared the exchange as a simple fanout; this is what the exchange should look like this:
Also, to set the delay when sending, you should use:
template.convertAndSend(exchangeName, queue.getName(), "foo", message -> {
message.getMessageProperties().setDelay(1000);
return message;
});
I had to customized Sftp Inbound default handler LoggingHandler and using my own CustomizedErrorHandler which extends ErrorHandler. But I can't return any message to my controller after handling exceptions.
I was researching couple of days and I found nothing to show my customized message to my UI using Controller. Below are some code snippet from my CustomizedErrorHandler, SftpInboundConfiguration.
SftpInboundConfiguration
public IntegrationFlow fileFlow() {
SftpInboundChannelAdapterSpec spec = Sftp
.inboundAdapter(getSftpSessionFactory())
.preserveTimestamp(true)
.remoteDirectory(getSourceLocation())
.autoCreateLocalDirectory(true)
.deleteRemoteFiles(false)
.localDirectory(new File(getDestinationLocation()));
return IntegrationFlows
.from(spec, e -> e.id(BEAN_ID)
.autoStartup(false)
.poller(sftpPoller())
)
.channel(sftpReceiverChannel())
.handle(sftpInboundMessageHandler())
.get();
}
... ... ...
public PollerMetadata sftpPoller() {
PollerMetadata pollerMetadata = new PollerMetadata();
List<Advice> adviceChain = new ArrayList<>();
pollerMetadata.setErrorHandler(customErrorMessageHandler());
pollerMetadata.setTrigger(new PeriodicTrigger(5000));
return pollerMetadata;
}
... ... ...
private CustomErrorMessageHandler customErrorMessageHandler() {
return new CustomErrorMessageHandler(
controlChannel(),
BEAN_ID
);
}
CustomErrorMessageHandler
public class CustomErrorMessageHandler implements ErrorHandler {
private final MessageChannel CONTROL_CHANNEL;
private final String BEAN_ID;
public CustomErrorMessageHandler(
MessageChannel controlChannel,
String beanID
) {
this.CONTROL_CHANNEL = controlChannel;
this.BEAN_ID = beanID;
}
public void handleError(#NotNull Throwable throwable) {
final Throwable rootCause = ExceptionUtils.getRootCause(throwable);
if (rootCause instanceof MessagingException) {
log.error("MessagingException : {} ", rootCause.getMessage());
} else if (rootCause instanceof SftpException) {
log.error("SftpException : {}", rootCause.getMessage());
} ... ... ...
else {
log.error("Unknown : Cause : {} , Error : {}",
rootCause, rootCause.getMessage());
}
log.info("Stopping SFTP Inbound");
boolean is_stopped = CONTROL_CHANNEL.send(
new GenericMessage<>("#" + BEAN_ID + ".stop()"));
if (is_stopped) {
log.info("SFTP Inbound Stopped.");
} else {
log.info("SFTP Inbound Stop Failed.");
}
}
}
Now I want to save some customized message from if-else statements and need to show it in UI. Is there any way to save the message and show it using Route or Controller ?
Don't customize the error handler, use poller.errorChannel("myErrorChannel") instead.
Then add an error channel flow
#Bean
IntegrationFlow errors() {
return IntegrationFLows.from("myErrorChannel")
.handle(...)
...
.get();
The message sent to the handler is an ErrorMessage with a MessagingException payload, with cause and failedMessage which was the message at the point of the failure and originalMessage which is the original message emitted by the adapter.
After handling the exception, you can simply call a method on your controller to tell it the state.
I use MQ in my project via SpringJMS, as a broker I use ActiveMQ.
I need to set expiration message-based, so I tried to used message.setJMSExpiration but without success. All messages coming to ActiveMQ have expiration=0.
Does anyone has success with setting Expiration per message using Spring?
For configuring JmsTemplate I used default value explicitQosEnabled = false; so I expected to keep expiration from my Message props. But as I see in ActiveMQSession.class this message properties will be override:
long expiration = 0L;
if (!producer.getDisableMessageTimestamp()) {
long timeStamp = System.currentTimeMillis();
message.setJMSTimestamp(timeStamp);
if (timeToLive > 0) {
expiration = timeToLive + timeStamp;
}
}
message.setJMSExpiration(expiration);
//me: timeToLive coming from default values of Producer/JmsTemplate...
What I am doing wrong ? or it is just impossible with this tools.
I don't know why Spring decided to exclude this, but you can extend JmsTemplate and overload some methods, passing a timeToLive argument.
public class MyJmsTemplate extends JmsTemplate {
public void send(final Destination destination,
final MessageCreator messageCreator, final long timeToLive)
throws JmsException {
execute(new SessionCallback<Object>() {
public Object doInJms(Session session) throws JMSException {
doSend(session, destination, messageCreator, timeToLive);
return null;
}
}, false);
}
protected void doSend(Session session, Destination destination,
MessageCreator messageCreator, long timeToLive) throws JMSException {
Assert.notNull(messageCreator, "MessageCreator must not be null");
MessageProducer producer = createProducer(session, destination);
try {
Message message = messageCreator.createMessage(session);
if (logger.isDebugEnabled()) {
logger.debug("Sending created message: " + message);
}
doSend(producer, message, timeToLive);
// Check commit - avoid commit call within a JTA transaction.
if (session.getTransacted() && isSessionLocallyTransacted(session)) {
// Transacted session created by this template -> commit.
JmsUtils.commitIfNecessary(session);
}
} finally {
JmsUtils.closeMessageProducer(producer);
}
}
protected void doSend(MessageProducer producer, Message message,
long timeToLive) throws JMSException {
if (isExplicitQosEnabled() && timeToLive > 0) {
producer.send(message, getDeliveryMode(), getPriority(), timeToLive);
} else {
producer.send(message);
}
}
}
JMSExpiration is not the way to set an expiration. See the javadocs for Message...
JMS providers set this field when a message is sent. This method can be used to change the value for a message that has been received.
In other words, it's ignored on a send - the time to live is set on the producer.send() method.
To expire a message set explicitQosEnabled to true and setTimeToLive(...).
akka novice here. Currently building a system to call some web services and update database... but akka actors are not working exactly as expected...My code sample...
Application Runner
public class Application
{
public static void main(String[] args) throws InterruptedException
{
ActorSystem system = ActorSystem.create("system");
ActorRef master = system.actorOf(Props.create(MasterActor.class));
String url = "http://some-web-service-url";
master.tell(url, ActorRef.noSender());
system.shutdown();
}
}
MasterActor
public class MasterActor extends UntypedActor
{
private final LoggingAdapter log = Logging.getLogger(getContext().system(), getSelf());
private final ActorRef childActor = getContext().actorOf(Props.create(ChildActor.class));
#Override
public void onReceive(Object message) throws Exception
{
if(message instanceof String)
{
childActor.tell(message, getSelf());
}else if(message instanceof Boolean){
log.info("all done");
}else {
unhandled(message);
}
}
}
ChildActor
public class ChildActor extends UntypedActor
{
private final LoggingAdapter log = Logging.getLogger(getContext().system(), getSelf());
#Override
public void onReceive(Object message) throws Exception
{
if (message instanceof String) {
String url = (String) message;
Integer result = getWebServiceResult(url);
log.info("result: {}", result);
getSender().tell(true, getSelf());
}else {
unhandled(message);
}
}
private Integer getWebServiceResult(final String url) throws Exception
{
ExecutionContextExecutor executor = getContext().dispatcher();
Future<Integer> future = Futures.future(new Callable<Integer>()
{
#Override
public Integer call() throws Exception
{
return new HttpClient().fetchData(url); //some web service call
}
}, executor);
return (Integer) Await.result(future, Duration.create(7000, TimeUnit.SECONDS));
}
}
but child actor is unable to send the message to its sender, master...getting this error stack...
[INFO] [03/28/2015 01:02:45.521] [system-akka.actor.default-dispatcher-3] [akka://system/user/$a/$a] result: TheWebservice Result
[INFO] [03/28/2015 01:02:45.528] [system-akka.actor.default-dispatcher-4] [akka://system/user/$a] Message [java.lang.Boolean] from Actor[akka://system/user/$a/$a#1601481298] to Actor[akka://system/user/$a#1257171720] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
I am unable to find what's wrong (spent 3 days)... In my sense this code should work...can you tell me what's I am doing wrong.
Thanks in advance...
You have a race condition. You're shutting down your ActorSystem
system.shutdown();
before the child actor has a chance to send its reply. Remember that more or less everything in akka is asynchronous.
Add, for example, a
Thread.sleep(someTime);
before the shutdown to see the message sent and received.
Just to deal with the exit part :
You could register a shutdown hook like below and shutdown the akka system in it
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
system.shutdown();
}
});
Then enter into a never ending loop as shown below while waiting for the processing to complete so that when it is done processing you press Ctrl + C on your terminal or raise a System.exit(0) within your code.
while(false == shutdownFlag){
Thread.sleep(sometime);
}