This is my JMS configuration:
#EnableJms
#Configuration
public class VmsJmsConfig implements JmsListenerConfigurer {
#Value("${spring.activemq.broker-url}")
String brokerUrl;
#Value("${spring.activemq.ssl.trustStorePath}")
String trustStorePath;
#Value("${spring.activemq.ssl.trustStorePass}")
String trustStorePass;
#Bean
public DefaultJmsListenerContainerFactory defaultJmsListenerContainerFactory(ConnectionFactory conFactory) {
DefaultJmsListenerContainerFactory defaultJmsListenerContainerFactory = new DefaultJmsListenerContainerFactory();
defaultJmsListenerContainerFactory.setConnectionFactory(conFactory);
defaultJmsListenerContainerFactory.setConcurrency("10-20");
return defaultJmsListenerContainerFactory;
}
#Bean("conFactory")
public ConnectionFactory activeMQSslConnectionFactory() throws Exception {
ActiveMQSslConnectionFactory activeMQSslConnectionFactory = new ActiveMQSslConnectionFactory(brokerUrl);
activeMQSslConnectionFactory.setTrustStore(trustStorePath);
activeMQSslConnectionFactory.setTrustStorePassword(trustStorePass);
return activeMQSslConnectionFactory;
}
#Bean
public DefaultMessageHandlerMethodFactory handlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(messageConverter());
return factory;
}
#Bean
public MessageConverter messageConverter() {
return new MappingJackson2MessageConverter();
}
#Override
public void configureJmsListeners(JmsListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(handlerMethodFactory());
}
}
My app is listening to an ActiveMQ queue. It consumes a message, transforms it, sends a downstream request, waits for the response, and sends that response to another queue.
I want it to consume multiple messages at the same time and process them in parallel, but no matter how many consumer I set in setConcurrency() it only consumes 1 message at a time, and there are more than 1000 messages pending in queue.
I tried changing the concurency, but no luck. But when I comment downstream call, it consumes 10-20 messages at a time, I couldn't find reason for that.
Related
Using JMSConfig i am creating MQ Connection factory and i have InboundGatewayConfig and OutboundGatewayConfig , in Inbound Config i am reading the Message payload from one queue and in Outbound config sending the message to another queue and my goad is toset reply channel to acknowledge the sender queue once queue receive the message.
InboundGatewayConfig:
#Configuration
#EnableIntegration
public class InboundGatewayConfig {
#Autowired
JmsConfig jmsConfig;
#Value("${fcb.inbound.receiver.queue.name}")
private String orderRequestDestination;
#Bean
public MessageChannel inboundOrderRequestChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel inboundOrderResponseChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "inboundOrderRequestChannel")
public OrderService orderService() {
return new OrderService();
}
#Bean
public JmsInboundGateway jmsInboundGateway() {
JmsInboundGateway gateway = new JmsInboundGateway(
defaultMessageListenerContainer(),
channelPublishingJmsMessageListener());
gateway.setRequestChannel(inboundOrderRequestChannel());
return gateway;
}
#Bean
public DefaultMessageListenerContainer defaultMessageListenerContainer() {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(jmsConfig.fcbCachingConnectionFactory());
container.setDestinationName(orderRequestDestination);
container.setRecoveryInterval(5000);
container.setErrorHandler(t -> System.out.println("Error in JMS Configurations \t" + t.getCause()));
return container;
}
#Bean
public ChannelPublishingJmsMessageListener channelPublishingJmsMessageListener() {
ChannelPublishingJmsMessageListener channelPublishingJmsMessageListener = new ChannelPublishingJmsMessageListener();
channelPublishingJmsMessageListener.setExpectReply(true);
return channelPublishingJmsMessageListener;
}
}
OutboundGatewayConfig :
#Configuration
public class OutboundGatewayConfig {
#Value("${fcb.inbound.receiver.queue.name}")
private String orderRequestDestination;
#Value("${fcb.inbound.response.queue.name}")
private String orderResponseDestination;
#Autowired
JmsConfig jmsConfig;
#Bean
public MessageChannel outboundOrderRequestChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel outboundOrderResponseChannel() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel = "outboundOrderRequestChannel")
public JmsOutboundGateway jmsOutboundGateway() {
JmsOutboundGateway gateway = new JmsOutboundGateway();
gateway.setConnectionFactory(jmsConfig.fcbCachingConnectionFactory());
gateway.setRequestDestinationName(orderRequestDestination);
gateway.setReplyDestinationName(orderResponseDestination);
gateway.setReplyChannel(outboundOrderResponseChannel());
return gateway;
}
}
and this my service class
public class OrderService {
private static final Logger LOGGER =
LoggerFactory.getLogger(OrderService.class);
public Message<?> order(Message<?> order) {
LOGGER.info("received order='{}'", order);
Message<?> status = MessageBuilder.withPayload("Accepted")
.setHeader("jms_correlationId",
order.getHeaders().get("jms_messageId"))
.setReplyChannelName("inboundOrderResponseChannel").build();
LOGGER.info("sending status='{}'", status);
return status;
}
}
While running the application InboundGateway able to receive the message payload from request queue but while reply , giving bellow warning message.
2022-04-07 16:06:24.303 WARN 19408 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'xxx.yyy.queue' - trying to recover. Cause: Cannot determine reply destination: Request message does not contain reply-to destination, and no default reply destination set.
Hey guys I have an application with spring boot that receives a message from a TIBCO EMS, I perform some operations and finally, I need to send that message to an MQ
I have a class Consumer
In the line #JmsLister I have a destination queue from TIBCO EMS because is who sends the message and when I called a method sendMessage I pass a destination queue from the parameter that is to MQ
#Component
public class ClassConsumer {
#JmsListener(destination = "${jms.destination.queue}",
containerFactory = Constants.GENERIC_JMS_EMS_LISTENER_BEAN_NAME)
public void consumerProcess(String request, #Header(JmsHeaders.ID) String id) {
ValoresDtoIn in = operationService.convertXml(request);
DatosDto datosDto = operationService.fillObject(in);
this.operacionService.sendMessage(datosDto, mqConstants.getMqDestiQueue(),id);
}
}
and when it reaches the sendMessage method it marks the error
This is my class service
#Service
public class Operations implements OperationsService {
#Autowired
#Qualifier(Constants.GENERIC_JMS_TEMPLATE_BEAN_NAME)
private JmsTemplate jmsTemplate;
#Override
public void sendMessage(String message, String destination, String id) {
ConnectionFactory connectionFactory = this.jmsTemplate.getConnectionFactory();
if (!Objects.isNull(connectionFactory)) {
try (Connection connection = connectionFactory.createConnection()) {
Queue queue = connection.createSession().createQueue(destination);
this.processorService.setId(id);
this.jmsTemplate.convertAndSend(queue, message, this.processorService);
} catch (JMSException ex) {
LOGGER.error(ex.getMessage(), ex);
}
}
}
}
and I had two classes from configuration, one for TibEms and the other from MQ
to TIBCO
#Configuration
public class TibConnectionConfig {
#Bean(name = Constants.GENERIC_CONNECTION_FACTORY_BEAN_NAME)
public TibjmsConnectionFactory tibjmsConnectionFactory() throws JMSException {
TibjmsConnectionFactory tibjmsConnectionFactory = new TibjmsConnectionFactory();
tibjmsConnectionFactory.setServerUrl(constants.getGenericHost());
tibjmsConnectionFactory.setUserName(constants.getGenericUsername());
tibjmsConnectionFactory.setUserPassword(constants.getGenericPassword());
tibjmsConnectionFactory.setReconnAttemptTimeout(constants.getGenericReconnAttemptTimeout());
tibjmsConnectionFactory.setSSLPassword(constants.getCertificatePassword());
tibjmsConnectionFactory.setSSLTrustedCertificate(constants.getCertificatePath());
tibjmsConnectionFactory.setSSLEnableVerifyHostName(false);
return tibjmsConnectionFactory;
}
#Bean(name = Constants.GENERIC_CACHING_CONNECTION_FACTORY_BEAN_NAME)
public CachingConnectionFactory cachingConnectionFactory(
#Qualifier(Constants.GENERIC_CONNECTION_FACTORY_BEAN_NAME) TibjmsConnectionFactory tibjmsConnectionFactory)
throws JMSException {
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(tibjmsConnectionFactory());
cachingConnectionFactory.setSessionCacheSize(constants.getGenericCacheSize());
return cachingConnectionFactory;
}
#Bean(name = Constants.GENERIC_JMS_TEMPLATE_BEAN_NAME)
public JmsTemplate jmsTemplate(
#Qualifier(Constants.GENERIC_CACHING_CONNECTION_FACTORY_BEAN_NAME) CachingConnectionFactory connection)
throws JMSException {
JmsTemplate jmsTemplate = new JmsTemplate(connection);
jmsTemplate.setDefaultDestinationName(constants.getGenericDestinationQueue());
jmsTemplate.setReceiveTimeout(constants.getGenericReceiveTimeout());
jmsTemplate.setDeliveryMode(constants.getGenericDeliverymode());
jmsTemplate.setSessionAcknowledgeModeName(constants.getSessionMode());
jmsTemplate.setExplicitQosEnabled(true);
jmsTemplate.setSessionTransacted(false);
return jmsTemplate;
}
#Bean(name = Constants.GENERIC_JMS_EMS_LISTENER_BEAN_NAME)
public DefaultJmsListenerContainerFactory jmsEmsListenerContainerFactory(
#Qualifier(Constants.GENERIC_CACHING_CONNECTION_FACTORY_BEAN_NAME) CachingConnectionFactory connection,
GenericErrorHandler genericErrorHandler) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connection);
factory.setErrorHandler(genericErrorHandler);
factory.setConcurrency(Constants.CONCURRENT_CONSTANT);
return factory;
}
}
To MQ
#Configuration
public class MqConnectionConfig {
#Bean(name = Constants.CONNECTION_FACTORY_BEAN_NAME)
public MQConnectionFactory mqConnectionFactory() throws JMSException {
MQConnectionFactory mqConnectionFactory = new MQConnectionFactory();
mqConnectionFactory.setHostName(constants.getHost());
mqConnectionFactory.setPort(constants.getPort());
mqConnectionFactory.setQueueManager(constants.getManager());
mqConnectionFactory.setChannel(constants.getChannel());
mqConnectionFactory.setTransportType(constants.getTranspType());
return mqConnectionFactory;
}
#Bean(name = Constants.CACHING_CONNECTION_FACTORY_BEAN_NAME)
public CachingConnectionFactory cachingConnectionFactory(
#Qualifier(Constants.CONNECTION_FACTORY_BEAN_NAME) MQConnectionFactory mqConnectionFactory)
throws JMSException {
CachingConnectionFactory cachingConnectionFactory =
new CachingConnectionFactory(mqConnectionFactory());
cachingConnectionFactory.setSessionCacheSize(10);
return cachingConnectionFactory;
}
#Bean(name = Constants.TEMPLATE_BEAN_NAME)
public JmsTemplate jmsTemplate(
#Qualifier(Constants.CONNECTION_FACTORY_BEAN_NAME) CachingConnectionFactory cachingConnectionFactory)
throws JMSException {
JmsTemplate jmsTemplate = new JmsTemplate(cachingConnectionFactory);
jmsTemplate.setDefaultDestinationName(constants.getMqDestiQueue);
jmsTemplate.setReceiveTimeout(constants.getTimeout());
jmsTemplate.setDeliveryMode(constants.getDeliveryMode());
jmsTemplate.setSessionAcknowledgeModeName(constants.getSessionMode());
jmsTemplate.setExplicitQosEnabled(true);
jmsTemplate.setSessionTransacted(false);
return jmsTemplate;
}
#Bean(name = Constants.EMS_LISTENER_BEAN_NAME)
public DefaultJmsListenerContainerFactory mqJmsListenerContainerFactory(
#Qualifier(Constants.CACHING_CONNECTION_FACTORY_BEAN_NAME) CachingConnectionFactory cachingConnectionFactory,
GenericEmsErrorHandler genericIbmErrorHandler) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(cachingConnectionFactory);
factory.setErrorHandler(genericIbmErrorHandler);
factory.setConcurrency(Constants.CONCURRENT_CONSTANT);
return factory;
}
}
and when I running the app sends me this error
ErrorHandler, unmanaged exception description: Listener method Consumer.consumerProcess(java.lang.String,java.lang.String)'
threw exception; nested exception is org.springframework.jms.InvalidDestinationException: Not allowed to create destination;
nested exception is javax.jms.InvalidDestinationException: Not allowed to create destination.
Can anyone help me? What is wrong?
I have no idea how to fix the error
I need to create application which should receive data from a data producer and process it. I chose RabbitMQ as a message-broker. My tests show me not the best results:
Sent - 100 msg;
Produce - 100 msg/s;
Consume - 6 msg/s;
To solve it I set listenerContainer.setAcknowledgeMode(AcknowledgeMode.NONE);
But I need acknowledge for some queues. And I can't do it in parallel using workers, because the order of messages is important for the data processing.
Is it possible to increase the speed of receiving, having acknowledges?
Producer:
#Bean
Queue queue() {
return new Queue(queueName, false);
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("ip");
connectionFactory.setUsername("name");
connectionFactory.setPassword("pswd");
return connectionFactory;
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory());
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(connectionFactory());
}
#Bean
public FanoutExchange exchange() {
return new FanoutExchange("exchange-1");
}
#Bean
public Binding binding(){
return BindingBuilder.bind(queue()).to(exchange());
}
...
rabbitTemplate.setExchange("exchange-1");
rabbitTemplate.convertAndSend(data);
Consumer:
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("ip");
connectionFactory.setUsername("name");
connectionFactory.setPassword("pswd");
return connectionFactory;
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(connectionFactory());
}
#Bean
Queue queue() {
return new Queue("queue-1", false);
}
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(connectionFactory());
listenerContainer.setQueues(queue());
listenerContainer.setMessageListener(new Receiver());
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
return listenerContainer;
}
...
#Override
public void onMessage(Message message) {
System.out.println("Received message: " + fromBytes(message.getBody()) + " \n Time = " + System.currentTimeMillis());
}
Tested on instance with 2 vCPU and 4 Gb memory.
You can increase the container's prefetchCount, which will greatly improve performance. However, if you reject and requeue a message, ordering will be lost (the requeued message will be behind the prefetched messages).
I am reading mails using spring mail inbound channel adapter once message is read i am performing some db operations in service activator of corresponding channel. My requirement is if any db operation fails adapter should read same message again.
Mail configuration :
#Bean
public DirectChannel inputChannel() {
return new DirectChannel();
}
#Bean
public IntegrationFlow pop3MailFlow() {
String url = "[url]";
return IntegrationFlows
.from(Mail.pop3InboundAdapter(url)
.javaMailProperties(p -> p.put("mail.pop3.socketFactory.class", "javax.net.ssl.SSLSocketFactory")),e -> e.autoStartup(true)
.poller(Pollers.fixedDelay(2000).transactionSynchronizationFactory(transactionSynchronizationFactory())))
.channel(inputChannel())
.handle(inboundEmailProcessor(),"messageProcess")
.get();
}
#Bean
public InboundEmailProcessor inboundEmailProcessor() {
return new InboundEmailProcessor();
}
#Bean
public TransactionSynchronizationFactory transactionSynchronizationFactory() {
TransactionSynchronizationFactory synchronizationFactory = new DefaultTransactionSynchronizationFactory(expressionEvaluatingTransactionSynchronizationProcessor());
return synchronizationFactory;
}
#Bean
public ExpressionEvaluatingTransactionSynchronizationProcessor expressionEvaluatingTransactionSynchronizationProcessor() {
ExpressionEvaluatingTransactionSynchronizationProcessor processor = new ExpressionEvaluatingTransactionSynchronizationProcessor();
ExpressionParser parser = new SpelExpressionParser();
processor.setAfterRollbackExpression(parser.parseExpression("new com.muraai.ex.config.Exp().process(payload)"));
return processor;
}
public class InboundEmailProcessor {
#Autowired
AttachmentsRepository attachmentsRepository;
#Transactional(rollbackFor = Exception.class)
public void messageProcess() {
// some db operations
// if it fails the same message should be read again
}
}
I thought this would work but its not working. Is there any way to achieve my requirement
public class Exp {
public void process(MimeMessage message) throws MessagingException {
message.setFlag(Flags.Flag.SEEN, false);
}
}
You need IMAP for that; with POP3, the server always marks them read.
You can add a spring-retry interceptor advice to the poller's advice chain and/or send the failed message to an error channel.
The retry advice can be configured for number of retries, back off policy etc.
I am experimenting with the Spring Reactor 3 components and Spring Integration to create a reactive stream (Flux) from a JMS queue.
I am attempting to create a reactive stream (Spring Reactor 3 Flux) from a JMS queue (ActiveMQ using Spring Integration) for clients to get the JMS messages asynchronously. I believe that I have everything hooked up correctly but the client does not receive any of the JMS messages until the server is stopped. Then all of the messages get "pushed" to the client a once.
Any help would be appreciated.
Here is the configuration file that I am using to configure the JMS, Integration components and the reactive publisher:
#Configuration
#EnableJms
#EnableIntegration
public class JmsConfiguration {
#Value("${spring.activemq.broker-url:tcp://localhost:61616}")
private String defaultBrokerUrl;
#Value("${queues.patient:patient}")
private String patientQueue;
#Autowired
MessageListenerAdapter messageListenerAdapter;
#Bean
public DefaultJmsListenerContainerFactory myFactory(
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
configurer.configure(factory, jmsConnectionFactory());
return factory;
}
#Bean
public Queue patientQueue() {
return new ActiveMQQueue(patientQueue);
}
#Bean
public ActiveMQConnectionFactory jmsConnectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(defaultBrokerUrl);
connectionFactory.setTrustedPackages(Arrays.asList("com.sapinero"));
return connectionFactory;
}
// Set the jackson message converter
#Bean
public JmsTemplate jmsTemplate() {
JmsTemplate template = new JmsTemplate();
template.setConnectionFactory(jmsConnectionFactory());
template.setDefaultDestinationName(patientQueue);
template.setMessageConverter(jacksonJmsMessageConverter());
return template;
}
#Bean
public MessageListenerAdapter messageListenerAdapter() {
MessageListenerAdapter messageListenerAdapter = new MessageListenerAdapter();
messageListenerAdapter.setMessageConverter(jacksonJmsMessageConverter());
return messageListenerAdapter;
}
#Bean
public AbstractMessageListenerContainer messageListenerContainer() {
DefaultMessageListenerContainer defaultMessageListenerContainer = new DefaultMessageListenerContainer();
defaultMessageListenerContainer.setMessageConverter(jacksonJmsMessageConverter());
defaultMessageListenerContainer.setConnectionFactory(jmsConnectionFactory());
defaultMessageListenerContainer.setDestinationName(patientQueue);
defaultMessageListenerContainer.setMessageListener(messageListenerAdapter());
defaultMessageListenerContainer.setCacheLevel(100);
defaultMessageListenerContainer.setErrorHandler(new ErrorHandler() {
#Override
public void handleError(Throwable t) {
t.printStackTrace();
}
});
return defaultMessageListenerContainer;
}
#Bean // Serialize message content to json using TextMessage
public MessageConverter jacksonJmsMessageConverter() {
MappingJackson2MessageConverter converter = new MappingJackson2MessageConverter();
converter.setTargetType(MessageType.TEXT);
converter.setTypeIdPropertyName("_type");
return converter;
}
#Bean
public MessageChannel jmsOutboundInboundReplyChannel() {
return MessageChannels.queue().get();
}
#Bean
public Publisher<Message<String>> pollableReactiveFlow() {
return IntegrationFlows
.from(Jms.messageDrivenChannelAdapter(messageListenerContainer()).get())
.channel(MessageChannels.queue())
.log(LoggingHandler.Level.DEBUG)
.log()
.toReactivePublisher();
}
#Bean
public MessageChannel jmsChannel() {
return new DirectChannel();
}
The controller that creates the Flux is:
#RestController
#RequestMapping("patients")
public class PatientChangePushController {
private LocalDateTime lastTimePatientDataRetrieved = LocalDateTime.now();
private int durationInSeconds = 30;
private Patient patient;
AtomicReference<SignalType> checkFinally = new AtomicReference<>();
#Autowired
PatientService patientService;
#Autowired
#Qualifier("pollableReactiveFlow")
private
Publisher<Message<String>> pollableReactiveFlow;
#Autowired
private JmsTemplate jmsTemplate;
#Autowired
private Queue patientQueue;
/**
* Subscribe to a Flux of a patient that has been updated.
*
* #param id
* #return
*/
#GetMapping(value = "/{id}/alerts", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Message<String>> getPatientAlerts(#PathVariable Long id) {
Flux<Message<String>> messageFlux = Flux.from(pollableReactiveFlow);
return messageFlux;
}
#GetMapping(value = "/generate")
public void generateJmsMessage() {
for (long i = 0L; i < 100; i++) {
Patient patient = new Patient();
patient.setId(i);
send(patient);
System.out.println("Message was sent to the Queue");
}
}
void send(Patient patient) {
this.jmsTemplate.convertAndSend(this.patientQueue, patient);
}
}
If anyone can tell me why the messages do not get sent to the client until after the server is killed, I would appreciate it.
Works well for me:
#SpringBootApplication
#RestController
public class SpringIntegrationSseDemoApplication {
public static void main(String[] args) {
SpringApplication.run(SpringIntegrationSseDemoApplication.class, args);
}
#Autowired
private ConnectionFactory connectionFactory;
#Autowired
private JmsTemplate jmsTemplate;
#Bean
public Publisher<Message<String>> jmsReactiveSource() {
return IntegrationFlows
.from(Jms.messageDrivenChannelAdapter(this.connectionFactory)
.destination("testQueue"))
.channel(MessageChannels.queue())
.log(LoggingHandler.Level.DEBUG)
.log()
.toReactivePublisher();
}
#GetMapping(value = "/events", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> getPatientAlerts() {
return Flux.from(jmsReactiveSource())
.map(Message::getPayload);
}
#GetMapping(value = "/generate")
public void generateJmsMessage() {
for (int i = 0; i < 100; i++) {
this.jmsTemplate.convertAndSend("testQueue", "testMessage #" + (i + 1));
}
}
}
In one terminal I have curl http://localhost:8080/events which waits for SSEs from that Flux.
In other terminal I perform curl http://localhost:8080/generate and see in the first one:
data:testMessage #1
data:testMessage #2
data:testMessage #3
data:testMessage #4
I use Spring Boot 2.0.0.BUILD-SNAPSHOT.
Also see here: https://spring.io/blog/2017/03/08/spring-tips-server-sent-events-sse