I have implement jms with spring boot, I am using #JmsListener to listen the topic
#Component
public class AMQListner {
BlockingQueue<MessageBO> queue = new ArrayBlockingQueue<>(1024);
#JmsListener(destination = "${spring.activemq.topic}")
public void Consume(TextMessage message) {
try {
String json = message.getText();
MessageBO bo = ObjectMapperConfig.getInstance().readValue(json, MessageBO.class);
queue.add(bo);
} catch (JMSException e) {
e.printStackTrace();
} catch (JsonParseException e) {
e.printStackTrace();
} catch (JsonMappingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Now I want a listener that listen that blocking-queue, if it has value , then process. can we achieve this using annotation in spring boot ?
First of all, the proper way is to create a handler bean instead of having a member with the message queue in the receiver class.
public interface MessageHandler extends Consumer<MessageBO> {
public default void handle(MessageBO msg) { accept(msg); }
}
#Component
public class AMQListener {
#Resource("multiplexer")
MessageHandler handler;
#JmsListener(destination = "${spring.activemq.topic}")
public void Consume(TextMessage message) {
try {
String json = message.getText();
MessageBO bo = ObjectMapperConfig.getInstance().readValue(json, MessageBO.class);
handler.handle(bo);
} catch (Exception e) {
e.printStackTrace();
}
}
}
Then, you would have the queue in the handler bean
#Component("multiplexer")
public class MessageMultiplexer implements MessageHandler {
#Autowired
MessageHandler actualConsumer;
ExecutorService executor = Executors.newFixedThreadPool(4);
public void accept(MessageBO msg) {
executor.submit(msg -> actualConsumer.handle(msg));
}
}
The Executor is pretty much the queue in this case.
Caveat: you do not have your 1024 limit in this way. You can do that by using the ThreadPoolExecutor constructor and pass it a limited queue.
Related
There are 2 sqs listener in my project. I want one of them to have the same setting and one of them different setting. The only value I want to change is maxNumberOfMessages.
What is the most practical way to do this ? ı want set different maxNumberOfMessages value for one of listener.
this is my config ;
#Bean
public AWSCredentialsProvider awsCredentialsProvider(#Value("${cloud.aws.profile}") String profile,
#Value("${cloud.aws.region.static}") String region,
#Value("${cloud.aws.roleArn}") String role,
#Value("${cloud.aws.user}") String user) {
...
return new AWSStaticCredentialsProvider(sessionCredentials);
}
#Bean
#Primary
#Qualifier("amazonSQSAsync")
public AmazonSQSAsync amazonSQSAsync(#Value("${cloud.aws.region.static}") String region, AWSCredentialsProvider awsCredentialsProvider) {
return AmazonSQSAsyncClientBuilder.standard()
.withCredentials(awsCredentialsProvider)
.withRegion(region)
.build();
}
#Bean
#Primary
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSqs) {
SimpleMessageListenerContainerFactory factory = new SimpleMessageListenerContainerFactory();
factory.setAmazonSqs(amazonSqs);
factory.setMaxNumberOfMessages(1);
factory.setWaitTimeOut(10);
factory.setQueueMessageHandler(new SqsQueueMessageHandler());
return factory;
}
This is listener;
#SqsListener(value = "${messaging.queue.blabla.source}", deletionPolicy = SqsMessageDeletionPolicy.NEVER)
public void listen(Message message, Acknowledgment acknowledgment, #Header("MessageId") String messageId) {
log.info("Message Received");
try {
....
acknowledgment.acknowledge().get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
} catch (Exception ex) {
throw new RuntimeException(ex.getMessage());
}
}
Following hack worked for me (if each listener listens to different queue)
#Bean
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSqs) {
return new SimpleMessageListenerContainerFactory() {
#Override
public SimpleMessageListenerContainer createSimpleMessageListenerContainer() {
SimpleMessageListenerContainer simpleMessageListenerContainer = new SimpleMessageListenerContainer() {
#Override
protected void startQueue(String queueName, QueueAttributes queueAttributes) {
// A place to configure queue based maxNumberOfMessages
try {
if (queueName.endsWith(".fifo")) {
FieldUtils.writeField(queueAttributes, "maxNumberOfMessages", 1, true);
}
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
super.startQueue(queueName, queueAttributes);
}
};
simpleMessageListenerContainer.setAmazonSqs(amazonSqs);
return simpleMessageListenerContainer;
}
};
}
ı found the solution and share on example repo on github.
github link
if ı add #EnableAsync annotation on listener class and #Async annotation to handler method my problem is solving :)
Unfortunately, the solution from Sushant didn't compile for me in Kotlin(because QueueAttributes is static protected class), but I used it to write following:
#Bean
fun simpleMessageListenerContainerFactory(sqs: AmazonSQSAsync): SimpleMessageListenerContainerFactory =
object : SimpleMessageListenerContainerFactory() {
override fun createSimpleMessageListenerContainer(): SimpleMessageListenerContainer {
val container = object : SimpleMessageListenerContainer() {
override fun afterPropertiesSet() {
super.afterPropertiesSet()
registeredQueues.forEach { (queue, attributes) ->
if (queue.contains(QUEUE_NAME)) {
FieldUtils.writeField(
attributes,
"maxNumberOfMessages",
NEW_MAX_NUMBER_OF_MESSAGES,
true
)
}
}
}
}
container.setWaitTimeOut(waitTimeOut)
container.setMaxNumberOfMessages(maxNumberOfMessages)
container.setAmazonSqs(sqs)
return container
}
}
I have the following working socket server configuration, and would like to add a handler if any exception occurs, eg inside the Deserializer during read of the message.
Therefore I added a #ServiceActivator(inputChannel = "errorChannel"). But the method is never invoked. Why?
#MessageEndpoint
public class SocketEndpoint {
#ServiceActivator(inputChannel = "mainChannel")
public String handleMessage(String message) {
return "normal response";
}
#ServiceActivator(inputChannel = "errorChannel")
public String handleError(MessagingException message) {
//TODO this is never invoked!
return "some error";
}
}
#Bean
public TcpInboundGateway mainGateway(
#Qualifier("tcpFactory") TcpConnectionFactoryFactoryBean factory,
#Qualifier("mainChannel") MessageChannel mainChannel,
#Qualifier("errorChannel") MessageChannel errorChannel) throws Exception {
TcpInboundGateway g = new TcpInboundGateway();
g.setConnectionFactory(factory.getObject());
g.setRequestChannel(mainChannel);
g.setErrorChannel(errorChannel);
return g;
}
#Bean
public TcpConnectionFactoryFactoryBean fact() {
TcpConnectionFactoryFactoryBean f = new TcpConnectionFactoryFactoryBean();
f.setType("server");
//....
f.setDeserializer(new MyDeserializer());
return f;
}
class MyDeserializer implements Deserializer<String> {
#Override
public String deserialize(InputStream inputStream)
throw new RuntimeException("catch me in error-channel");
}
}
throw new RuntimeException("catch me in error-channel");
It can't go to the error channel since there's no message yet (messages sent to error channels are messages that fail downstream processing).
The standard deserializers (that extend AbstractByteArraySerializer) publish a TcpDeserializationExceptionEvent when deserialization fails. See the ByteArrayCrLfSerializer for an example:
https://github.com/spring-projects/spring-integration/blob/master/spring-integration-ip/src/main/java/org/springframework/integration/ip/tcp/serializer/ByteArrayCrLfSerializer.java#L78
public int fillToCrLf(InputStream inputStream, byte[] buffer) throws IOException {
int n = 0;
int bite;
if (logger.isDebugEnabled()) {
logger.debug("Available to read: " + inputStream.available());
}
try {
...
}
catch (SoftEndOfStreamException e) {
throw e;
}
catch (IOException e) {
publishEvent(e, buffer, n);
throw e;
}
catch (RuntimeException e) {
publishEvent(e, buffer, n);
throw e;
}
}
See the documentation. The Deserializer needs to be a bean so that it gets an event publisher.
You can then listen for the event(s) with an ApplicationListener< TcpDeserializationExceptionEvent> or an #EventListener method.
I am working on sftp outbound adapter with Spring Integration 4.3 release.
I am able to successfully send the file to sftp location but i want to update the database record to complete status.
I am looking ExpressionEvaluatingRequestHandlerAdvice for an option but not able to figure it out how to call a method from setOnSuccessExpressionString.
Tried with below option.
#Bean
public ExpressionEvaluatingRequestHandlerAdvice afterPut() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
StandardEvaluationContext context = new StandardEvaluationContext();
///context.setBeanResolver((BeanResolver) new SftpPutBean());
ExpressionParser parser = new SpelExpressionParser();
try {
context.registerFunction("mymethod", SftpPutBean.class.getDeclaredMethod("mymethod", new Class[] { String.class }));
} catch (NoSuchMethodException e) {
e.printStackTrace();
} catch (SecurityException e) {
e.printStackTrace();
}
//String abc
String helloWorldReversed = parser.parseExpression("#mymethod(headers['transmissionId'])").getValue(advice, String.class);
advice.setOnSuccessExpressionString("helloWorldReversed");
advice.setPropagateEvaluationFailures(true);
return advice;
}
public void mymethod(String id) {
try {
TransmissionQueue abc = transmissionQueueDataService.findById(Integer.parseInt(id));
abc.setStatus("COMPLETED");
transmissionQueueDataService.saveTransmissionQueue(abc);
} catch (Exception e) {
e.printStackTrace();
}
}
You need to register your function with the Spring Integration evaluation context factory. Instructions here.
To provide a SpEL Function via Java Configuration, you should declare a SpelFunctionFactoryBean bean for each function. The sample above can be configured as follows:
#Bean
public SpelFunctionFactoryBean xpath() {
return new SpelFunctionFactoryBean(XPathUtils.class, "evaluate");
}
I need to listen to multiple queues (exists in same Queue Manager). I have the working spring boot application code for listening to single queue. But is there any way I can connect to multiple queues from the single spring boot application?
Also is there anyway if I can switch listeners from one queue to another queue at runtime?
I have the code to read from single queue which is as follows:
public class ConnectionConfiguration {
private static final Logger logger = LogManager.getLogger(ConnectionConfiguration.class);
#Value("${LDR_LOCAL_MQ_READ_FACTORYNAME}")
String connectionFactory;
#Value("${LDR_LOCAL_MQ_QUEUENAME}")
String localQueue;
#Value("${jmsConcurrency}")
String concurrency;
#Value("${servers.mq.host}")
private String host;
#Value("${servers.mq.port}")
private Integer port;
#Value("${servers.mq.queue-manager}")
private String queueManager;
#Value("${servers.mq.channel}")
private String channel;
#Value("${servers.mq.queue}")
private String queue;
#Value("${servers.mq.timeout}")
private long timeout;
#Bean
public ConnectionFactory connectionFactory() {
JndiObjectFactoryBean jndiObjectFactoryBean = new JndiObjectFactoryBean();
jndiObjectFactoryBean.setResourceRef(true);
jndiObjectFactoryBean.setJndiName(connectionFactory);
try {
jndiObjectFactoryBean.afterPropertiesSet();
} catch (IllegalArgumentException e) {
logger.error(e.getMessage(), e);
e.printStackTrace();
} catch (NamingException e) {
logger.error(e.getMessage(), e);
e.printStackTrace();
}
return (ConnectionFactory) jndiObjectFactoryBean.getObject();
}
#Bean
public MQQueueConnectionFactory mqQueueConnectionFactory() {
MQQueueConnectionFactory mqQueueConnectionFactory = new MQQueueConnectionFactory();
try {
mqQueueConnectionFactory.setHostName(host);
mqQueueConnectionFactory.setQueueManager(queueManager);
mqQueueConnectionFactory.setPort(port);
mqQueueConnectionFactory.setChannel(channel);
mqQueueConnectionFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
mqQueueConnectionFactory.setCCSID(1208);
} catch (Exception e) {
e.printStackTrace();
}
return mqQueueConnectionFactory;
}
#Bean
public SimpleMessageListenerContainer queueContainer(MQQueueConnectionFactory mqQueueConnectionFactory) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(mqQueueConnectionFactory);
container.setDestinationName(queue);
container.setMessageListener(getListenerWrapper());
container.start();
return container;
}
#Bean
public MQListener getListenerWrapper() {
return new MQListener();
}
#Bean
public JmsTemplate getJmsTemplate() {
try {
return new JmsTemplate(mqQueueConnectionFactory());
} catch (Exception exp) {
logger.error(exp.getMessage(), exp);
}
return null;
}
}
Just add a listener container bean for each queue.
To change the queue, call stop(), then shutdown(), change the destination, then initialize(), then start().
I am creating a basic POST JSON api endoint. I would like to unit test it, and want to make sure I am doing it appropriately in the Play framework. So far I am using Guice for dependency injection and JUnit for my unit testing library.
Here is my controller code:
public class NotificationController extends Controller {
private RabbitQueueService _rabbitQueueService;
#Inject
public NotificationController(RabbitQueueService service) {
_rabbitQueueService = service;
}
#BodyParser.Of(BodyParser.Json.class)
public Result post() {
ObjectMapper mapper = new ObjectMapper();
Notification notification;
try {
JsonNode notificationJsonNode = Controller.request().body().asJson();
notification = mapper.readValue(notificationJsonNode.toString(),
Notification.class);
_rabbitQueueService.push(notification);
return Results.created(notificationJsonNode, "UTF-8");
} catch (JsonParseException e) {
e.printStackTrace();
} catch (JsonMappingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return Results.badRequest();
}
}
My RabbitQueueService code:
public class RabbitQueueService {
private Channel _channel;
private Connection _connection;
public RabbitQueueService() {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(Config.RABBITMQ_HOST);
try {
_connection = factory.newConnection();
_channel = _connection.createChannel();
_channel.queueDeclare(Config.RABBITMQ_QUEUE, false, false, false, null);
_channel.exchangeDeclare(Config.RABBITMQ_EXCHANGE, "fanout");
_channel.queueBind(Config.RABBITMQ_QUEUE, Config.RABBITMQ_EXCHANGE, "");
} catch (IOException e) {
e.printStackTrace();
}
}
public void push(Notification notification) {
try {
_channel.basicPublish(Config.RABBITMQ_EXCHANGE, "", null, notification.getBytes());
_channel.close();
_connection.close();
} catch (IOException e) {
e.printStackTrace();
}
}
public void pop() {
}
}
My MockQueueService code:
public class MockQueueService extends RabbitQueueService {
#Override
public void push(Notification notification) {
/* Do nothing because you know... thats what I do */
}
#Override
public void pop() {
/* Do nothing because you know... thats what I do */
}
}
and finally my current unit test code:
public class ApplicationTest {
#Test
public void addMessageToQueue() {
running(fakeApplication(), new Runnable() {
public void run() {
FakeRequest request = new FakeRequest("/POST", "/api/v1/notifications");
Notification notification = new Notification(UUID.randomUUID(),
new NotificationType(UUID.randomUUID(),
"Critical"),
"Test notification message");
try {
ObjectMapper mapper = new ObjectMapper();
String json = mapper.writeValueAsString(notification);
JsonNode node;
node = mapper.readTree(json);
request.withJsonBody(node);
} catch (IOException e) {
e.printStackTrace();
}
route(request);
}
});
}
}
This all works fine when making a curl request to test my endpoint through play run. My main question is: how do I use the MockQueueService in my unit test? I don't see anyway to do it with fakeApplication() helper. I could instantiate it directly like
NotificationController nc = new NotificationController(new MockQueueService());
nc.post();
but the problem is I need to override the body of the play request with an appropriate request body and I think I need a FakeRequest for that.
Any help, samples, or advice would be helpful.
UPDATE
I have posted a gist example with the necessary example files. The things specifically that I did to get it working:
Setup a new GlobalUnitTest file that I passed into the fakeApplication helper
Changed NotificationController to be a singleton. This allowed me to pull in the NotificationController instance so I could check the QueueService count as part of the assertion.
FakeApplication takes a bunch of arguments that you could use to inject your new service. You could use a combination of any of these:
additionalPlugins
additionalConfiguration
withGlobal
They each let you specify some additional configuration you could use only during testing. Another thing you could do is have a separate Global object just for testing, that is used to create your controllers. The Global object is used to return your controller instance when you use # in your route definition. Then, you can create a separate application.test.conf that refers to GlobalTest that is loaded when you run play test.