We Keep Coding

Mockito doAnswer()

Can I somehow use doAnswer() when an exception is thrown?
I'm using this in my integration test to get method invocations and the test in configured the #RabbitListenerTest...
#RunWith(SpringRunner.class)
#SpringBootTest
public class MyIT {
#Autowired
private RabbitTemplate rabbitTemplate;
#Autowired
private MyRabbitListener myRabbitListener;
#Autowired
private RabbitListenerTestHarness harness;
#Test
public void testListener() throws InterruptedException {
MyRabbitListener myRabbitListener = this.harness.getSpy("event");
assertNotNull(myRabbitListener);
final String message = "Test Message";
LatchCountDownAndCallRealMethodAnswer answer = new LatchCountDownAndCallRealMethodAnswer(1);
doAnswer(answer).when(myRabbitListener).event(message);
rabbitTemplate.convertAndSend("exchange", "key", message);
assertTrue(answer.getLatch().await(20, TimeUnit.SECONDS));
verify(myRabbitListener).messageReceiver(message);
}
#Configuration
#RabbitListenerTest
public static class Config {
#Bean
public MyRabbitListener myRabbitListener(){
return new MyRabbitListener();
}
}
}
It works ok but when I introduce an Exception being thrown, It doesn't i.e
This works
#RabbitListener(id = "event", queues = "queue-name")
public void event(String message) {
log.info("received message > " + message);
}
This doesn't
#RabbitListener(id = "event", queues = "queue-name")
public void event(String message) {
log.info("received message > " + message);
throw new ImmediateAcknowledgeAmqpException("Invalid message, " + message);
}
Any help appreciated

The LatchCountDownAndCallRealMethodAnswer is very basic
#Override
public Void answer(InvocationOnMock invocation) throws Throwable {
invocation.callRealMethod();
this.latch.countDown();
return null;
}
You can copy it to a new class and change it to something like
private volatile Exception exeption;
#Override
public Void answer(InvocationOnMock invocation) throws Throwable {
try {
invocation.callRealMethod();
}
catch (RuntimeException e) {
this.exception = e;
throw e;
}
finally {
this.latch.countDown();
}
return null;
}
public Exception getException() {
return this.exception;
}
then
assertTrue(answer.getLatch().await(20, TimeUnit.SECONDS));
assertThat(answer.getException(), isInstanceOf(ImmediateAcknowledgeAmqpException.class));
Please open a github issue; the framework should support this out-of-the-box.

Guaranteed message delivery asynchronously

I wrote a code that guarantees the delivery of messages and their processing. But it works in one thread.
How to refactor code so that it works in parallel threads or asynchronously? In this case, messages must be guaranteed to be delivered even if the application crashes. They will be delivered after a new start of the application or with the help of other running instances of this application.
Producer:
#Async("threadPoolTaskExecutor")
#EventListener(condition = "#event.queue")
public void start(GenericSpringEvent<RenderQueueObject> event) {
RenderQueueObject renderQueueObject = event.getWhat();
send(RENDER_NAME, renderQueueObject);
}
private void send(String routingKey, Object queue) {
try {
log.info("SEND message");
rabbitTemplate.convertAndSend(routingKey, objectMapper.writeValueAsString(queue));
} catch (JsonProcessingException e) {
log.warn("Can't send event!", e);
}
}
Consumer
#Slf4j
#RequiredArgsConstructor
#Service
public class RenderRabbitEventListener extends RabbitEventListener {
private final ApplicationEventPublisher eventPublisher;
#RabbitListener(bindings = #QueueBinding(value = #Queue(Queues.RENDER_NAME),
exchange = #Exchange(value = Exchanges.EXC_RENDER_NAME, type = "topic"),
key = "render.#")
)
public void onMessage(Message message, Channel channel) {
String routingKey = parseRoutingKey(message);
log.debug(String.format("Event %s", routingKey));
RenderQueueObject queueObject = parseRender(message, RenderQueueObject.class);
handleMessage(queueObject);
}
public void handleMessage(RenderQueueObject render) {
GenericSpringEvent<RenderQueueObject> springEvent = new GenericSpringEvent<>(render);
springEvent.setRender(true);
eventPublisher.publishEvent(springEvent);
}
}
public class Exchanges {
public static final String EXC_RENDER_NAME = "render.exchange.topic";
public static final TopicExchange EXC_RENDER = new TopicExchange(EXC_RENDER_NAME, true, false);
}
public class Queues {
public static final String RENDER_NAME = "render.queue.topic";
public static final Queue RENDER = new Queue(RENDER_NAME);
}
And so my message is processed. If I add #Async, then there will be parallel processing, but if the application crashes, then at a new start, messages will not be sent again.
#EventListener(condition = "#event.render")
public void startRender(GenericSpringEvent<RenderQueueObject> event) {
RenderQueueObject render = event.getWhat();
storageService.updateDocument(
render.getGuid(),
new Document("$set", new Document("dateStartRendering", new Date()).append("status", State.rendering.toString()))
);
Future<RenderWorkObject> submit = taskExecutor.submit(new RenderExecutor(render));
try {
completeResult(submit);
} catch (IOException | ExecutionException | InterruptedException e) {
log.info("Error when complete results after invoke executors");
}
}
private void completeResult(Future<RenderWorkObject> renderFuture) throws IOException, ExecutionException, InterruptedException {
RenderWorkObject renderWorkObject = renderFuture.get();
State currentState = renderWorkObject.getState();
if (Stream.of(result, error, cancel).anyMatch(isEqual(currentState))) {
storageService.updateDocument(renderWorkObject.getGuidJob(), new Document("$set", toUpdate));
}
}
I tried to customize the configuration to fit my needs. But it didn’t work:
#Bean
Queue queue() {
return Queues.RENDER;
}
#Bean
TopicExchange exchange() {
return Exchanges.EXC_RENDER;
}
#Bean
Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(Queues.RENDER_NAME);
}
#Bean
public RabbitTemplate rabbitTemplate(#Qualifier("defaultConnectionFactory") ConnectionFactory connectionFactory) {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
return template;
}
#Bean
public SimpleMessageListenerContainer container(#Qualifier("defaultConnectionFactory") ConnectionFactory connectionFactory, RabbitEventListener listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory);
container.setQueueNames(Queues.RENDER_NAME);
container.setQueues(Queues.RENDER);
container.setExposeListenerChannel(true);
container.setMaxConcurrentConsumers(20);
container.setConcurrentConsumers(10);
container.setPrefetchCount(1000);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
return container;
}
#Bean
public ConnectionFactory defaultConnectionFactory() {
CachingConnectionFactory cf = new CachingConnectionFactory();
cf.setAddresses("127.0.0.1:5672");
cf.setUsername("guest");
cf.setPassword("guest");
cf.setVirtualHost("/");
cf.setPublisherConfirms(true);
cf.setPublisherReturns(true);
cf.setChannelCacheSize(25);
ExecutorService es = Executors.newFixedThreadPool(20);
cf.setExecutor(es);
return cf;
}
I would be grateful for any idea

I think I found a solution. I changed the RenderRabbitEventListener so that it again sent the message to the queue if the message was received from Rabbit in case of crash. Thanks to this, my consumer will always work in parallel. This will work in parallel in the event of a failure of all nodes, as well as in the event of a failure of one node.
Here are the changes I made:
#RabbitListener(bindings = #QueueBinding(value = #Queue(Queues.RENDER_NAME),
exchange = #Exchange(value = Exchanges.EXC_RENDER_NAME, type = "topic"),
key = "render.#")
)
public void onMessage(Message message, Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) long tag
) {
RenderQueueObject queueObject = parseRender(message, RenderQueueObject.class);
if (message.getMessageProperties().isRedelivered()) {
log.info("Message Redelivered, try also");
try {
channel.basicAck(tag, false);
MessageConverter messageConverter = rabbitTemplate.getMessageConverter();
String valueAsString = parseBody(message);
Message copyMessage = messageConverter.toMessage(valueAsString, new MessageProperties());
rabbitTemplate.convertAndSend(
message.getMessageProperties().getReceivedRoutingKey(),
copyMessage);
return;
} catch (IOException e) {
log.info("basicAck exception");
}
}
log.info("message not redelievered");
String routingKey = parseRoutingKey(message);
log.debug(String.format("Event %s", routingKey));
handleMessage(queueObject);
}

spring cloud aws multiple sqs listener

There are 2 sqs listener in my project. I want one of them to have the same setting and one of them different setting. The only value I want to change is maxNumberOfMessages.
What is the most practical way to do this ? ı want set different maxNumberOfMessages value for one of listener.
this is my config ;
#Bean
public AWSCredentialsProvider awsCredentialsProvider(#Value("${cloud.aws.profile}") String profile,
#Value("${cloud.aws.region.static}") String region,
#Value("${cloud.aws.roleArn}") String role,
#Value("${cloud.aws.user}") String user) {
...
return new AWSStaticCredentialsProvider(sessionCredentials);
}
#Bean
#Primary
#Qualifier("amazonSQSAsync")
public AmazonSQSAsync amazonSQSAsync(#Value("${cloud.aws.region.static}") String region, AWSCredentialsProvider awsCredentialsProvider) {
return AmazonSQSAsyncClientBuilder.standard()
.withCredentials(awsCredentialsProvider)
.withRegion(region)
.build();
}
#Bean
#Primary
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSqs) {
SimpleMessageListenerContainerFactory factory = new SimpleMessageListenerContainerFactory();
factory.setAmazonSqs(amazonSqs);
factory.setMaxNumberOfMessages(1);
factory.setWaitTimeOut(10);
factory.setQueueMessageHandler(new SqsQueueMessageHandler());
return factory;
}
This is listener;
#SqsListener(value = "${messaging.queue.blabla.source}", deletionPolicy = SqsMessageDeletionPolicy.NEVER)
public void listen(Message message, Acknowledgment acknowledgment, #Header("MessageId") String messageId) {
log.info("Message Received");
try {
....
acknowledgment.acknowledge().get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
} catch (Exception ex) {
throw new RuntimeException(ex.getMessage());
}
}

Following hack worked for me (if each listener listens to different queue)
#Bean
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSqs) {
return new SimpleMessageListenerContainerFactory() {
#Override
public SimpleMessageListenerContainer createSimpleMessageListenerContainer() {
SimpleMessageListenerContainer simpleMessageListenerContainer = new SimpleMessageListenerContainer() {
#Override
protected void startQueue(String queueName, QueueAttributes queueAttributes) {
// A place to configure queue based maxNumberOfMessages
try {
if (queueName.endsWith(".fifo")) {
FieldUtils.writeField(queueAttributes, "maxNumberOfMessages", 1, true);
}
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
super.startQueue(queueName, queueAttributes);
}
};
simpleMessageListenerContainer.setAmazonSqs(amazonSqs);
return simpleMessageListenerContainer;
}
};
}

ı found the solution and share on example repo on github.
github link
if ı add #EnableAsync annotation on listener class and #Async annotation to handler method my problem is solving :)

Unfortunately, the solution from Sushant didn't compile for me in Kotlin(because QueueAttributes is static protected class), but I used it to write following:
#Bean
fun simpleMessageListenerContainerFactory(sqs: AmazonSQSAsync): SimpleMessageListenerContainerFactory =
object : SimpleMessageListenerContainerFactory() {
override fun createSimpleMessageListenerContainer(): SimpleMessageListenerContainer {
val container = object : SimpleMessageListenerContainer() {
override fun afterPropertiesSet() {
super.afterPropertiesSet()
registeredQueues.forEach { (queue, attributes) ->
if (queue.contains(QUEUE_NAME)) {
FieldUtils.writeField(
attributes,
"maxNumberOfMessages",
NEW_MAX_NUMBER_OF_MESSAGES,
true
)
}
}
}
}
container.setWaitTimeOut(waitTimeOut)
container.setMaxNumberOfMessages(maxNumberOfMessages)
container.setAmazonSqs(sqs)
return container
}
}

spring boot and BlockingQueue listener

I have implement jms with spring boot, I am using #JmsListener to listen the topic
#Component
public class AMQListner {
BlockingQueue<MessageBO> queue = new ArrayBlockingQueue<>(1024);
#JmsListener(destination = "${spring.activemq.topic}")
public void Consume(TextMessage message) {
try {
String json = message.getText();
MessageBO bo = ObjectMapperConfig.getInstance().readValue(json, MessageBO.class);
queue.add(bo);
} catch (JMSException e) {
e.printStackTrace();
} catch (JsonParseException e) {
e.printStackTrace();
} catch (JsonMappingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Now I want a listener that listen that blocking-queue, if it has value , then process. can we achieve this using annotation in spring boot ?

First of all, the proper way is to create a handler bean instead of having a member with the message queue in the receiver class.
public interface MessageHandler extends Consumer<MessageBO> {
public default void handle(MessageBO msg) { accept(msg); }
}
#Component
public class AMQListener {
#Resource("multiplexer")
MessageHandler handler;
#JmsListener(destination = "${spring.activemq.topic}")
public void Consume(TextMessage message) {
try {
String json = message.getText();
MessageBO bo = ObjectMapperConfig.getInstance().readValue(json, MessageBO.class);
handler.handle(bo);
} catch (Exception e) {
e.printStackTrace();
}
}
}
Then, you would have the queue in the handler bean
#Component("multiplexer")
public class MessageMultiplexer implements MessageHandler {
#Autowired
MessageHandler actualConsumer;
ExecutorService executor = Executors.newFixedThreadPool(4);
public void accept(MessageBO msg) {
executor.submit(msg -> actualConsumer.handle(msg));
}
}
The Executor is pretty much the queue in this case.
Caveat: you do not have your 1024 limit in this way. You can do that by using the ThreadPoolExecutor constructor and pass it a limited queue.

How to simulate throwing an exception only once in retry with JUnit/Mockito test?

I put a simple retry because the operation can rarely fail. The simplified code is below. The method putObject can accidentally throw an exception, in this case the retry should allow to invoke this method again. Is it possible to write a JUnit test for this?
I know that with Mockito library we can force to throw an Exception invoking a method but how to force this exception to be thrown only once?
public class RetryExample {
Bucket bucket = new Bucket();
static int INTERNAL_EXCEPTION_CODE = 100;
class AException extends RuntimeException {
private static final long serialVersionUID = 1L;
int statusCode;
public int getStatusCode() {
return statusCode;
}
}
class Bucket {
public void putObject(String fileName, byte[] data) throws AException {
System.out.println("PutObject=" + fileName + " data=" + data);
}
}
public void process(String fileName, byte[] data) throws AException {
try {
retryOperation((f, d) -> bucket.putObject(f, d), fileName, data);
} catch (Exception ex) {
throw new AException("Failed to write data", ex);
}
}
private <T, U> void retryOperation(BiConsumer<T, U> biConsumer, T t, U u) {
int retries = 0;
boolean retry = false;
AException lastServiceException = null;
do {
try {
biConsumer.accept(t, u);
} catch (AException e) {
lastServiceException = e;
int statusCode = e.getStatusCode();
if (statusCode == INTERNAL_EXCEPTION_CODE) {
throw e;
} else {
break;
}
}
retries++;
if (retries >= 3) {
retry = false;
}
} while (retry);
if (lastServiceException != null) {
throw lastServiceException;
}
}
Test Class:
public class RetryExampleTest {
...
#Test
public void test() {
RetryExample retryExample = new RetryExample();
String fileName = "TestFile";
byte[] data = simulatedPayload(10000);
try {
retryExample.process(fileName, data);
} catch (Exception e) {
fail("Exception thrown=" + e);
}
}

According to the Mockito documentation you can set different behavior for consecutive method calls.
when(mock.someMethod("some arg"))
.thenThrow(new RuntimeException())
.thenReturn("foo");
In case of a void method you can do something similar (Mockito documentation)
doThrow(new RuntimeException())
.doNothing()
.when(mock).doSomething();

I think you can use a global data object to store the times of throw Exceptions, so in the Mockito library invoke the Exception method just taken the global data object to record the times. It would be simple. Just all by your control.