I have simple servicebus consumer:
val consumeFunc = (context: ServiceBusReceivedMessageContext) => consumeMessageOrder(context, onConsumed)
val queueName = messageType match {
case MessageBusService.Order => config.ordersQueueName
}
new ServiceBusClientBuilder()
.connectionString(config.connectionString)
.processor()
.queueName(queueName)
.processMessage { context =>
consumeFunc(context)
}
.processError(context => processErrorOrder(context, countdownLatch))
.buildProcessorClient()
and error handling method:
def processErrorOrder(context: ServiceBusErrorContext, countdownLatch: CountDownLatch): Unit = {
import ServiceBusFailureReason._
context.getException match {
case e: ServiceBusException
if List(MESSAGING_ENTITY_DISABLED, MESSAGING_ENTITY_NOT_FOUND, UNAUTHORIZED).contains(
e.getReason
) =>
log.error(s"An unrecoverable error occurred. Stopping processing with reason ${e.getReason} ${e.getMessage}")
countdownLatch.countDown()
case e: ServiceBusException =>
log.error(s"Error source ${context.getErrorSource}, reason ${e.getReason}, message: ${e}")
}
}
its in scala, but code is based on official tutorial AzureDocs
I can not find a way, to create some retry policy here, how to handle retries here and if on failed X retried land failing message to deadletter queue ?
thanks
Related
I have a Spring Boot app, where I receive one single message from a Azure Service Bus queue session.
The code is:
#Autowired
ServiceBusSessionReceiverAsyncClient apiMessageQueueIntegrator;
.
.
.
Mono<ServiceBusReceiverAsyncClient> receiverMono = apiMessageQueueIntegrator.acceptSession(sessionid);
Disposable subscription = Flux.usingWhen(receiverMono,
receiver -> receiver.receiveMessages(),
receiver -> Mono.fromRunnable(() -> receiver.close()))
.subscribe(message -> {
// Process message.
logger.info(String.format("Message received from quque. Session id: %s. Contents: %s%n", message.getSessionId(),
message.getBody()));
receivedMessage.setReceivedMessage(message);
timeoutCheck.countDown();
}, error -> {
logger.info("Queue error occurred: " + error);
});
As I am receiving only one message from the session, I use a CountDownLatch(1) to dispose of the subscription when I have received the message.
The documentation of the library says that it is possible to use Mono.usingWhen instead of Flux.usingWhen if I only expect one message, but I cannot find any examples of this anywhere, and I have not been able to figure out how to rewrite this code to do this.
How would the pasted code look if I were to use Mono.usingWhen instead?
Thank you conniey. Posting your suggestion as an answer to help other community members.
By default receiveMessages() is a Flux because we imagine the messages from a session to be "infinitely long". In your case, you only want the first message in the stream, so we use the next() operator.
The usage of the countdown latch is probably not the best approach. In the sample, we had one there so that the program didn't end before the messages were received. .subscribe is not a blocking call, it sets up the handlers and moves onto the next line of code.
Mono<ServiceBusReceiverAsyncClient> receiverMono = sessionReceiver.acceptSession("greetings-id");
Mono<ServiceBusReceivedMessage> singleMessageMono = Mono.usingWhen(receiverMono,
receiver -> {
// Anything you wish to do with the receiver.
// In this case we only want to take the first message, so we use the "next" operator. This returns a
// Mono.
return receiver.receiveMessages().next();
},
receiver -> Mono.fromRunnable(() -> receiver.close()));
try {
// Turns this into a blocking call. .block() waits indefinitely, so we have a timeout.
ServiceBusReceivedMessage message = singleMessageMono.block(Duration.ofSeconds(30));
if (message != null) {
// Process message.
}
} catch (Exception error) {
System.err.println("Error occurred: " + error);
}
You can refer to GitHub issue:ServiceBusSessionReceiverAsyncClient - Mono instead of Flux
I have following flow:
return flow -> flow.channel(inputChannel())
...
.gateway(childFlow, addMyInterceptor(str)); // by name
}
Consumer<GatewayEndpointSpec> addMyInterceptor(String objectIdHeader) {
return endpointSpec -> endpointSpec.advice(addMyInterceptorInternal(objectIdHeader))
.errorChannel(errorChannel());
}
default IdempotentReceiverInterceptor addMyInterceptorInternal(String header) {
MessageProcessor<String> headerSelector = message -> headerExpression(header).apply(message);
var interceptor = new IdempotentReceiverInterceptor(new MetadataStoreSelector(headerSelector, idempotencyStore()));
interceptor.setDiscardChannel(idempotentDiscardChannel());
return interceptor;
}
When IdempotentReceiverInterceptor encounters that message is duplicated - I see that application hangs on after 4-th duplicated message. I understand that it is because gateway expected response(like here: PubSubInboundChannelAdapter stops to receive messages after 4th message) but I don't have any ideas how to return result from interceptor.
Could you please explain it for me?
As long as all channels are direct (default) - i.e. no async handoffs in the flow using queue or executor channels, set the gateway's replyTimeout to 0 when the flow might not return a reply
How I get information inside a Throwable Scala object?
The code is an example about throwable.getMessage.
JsResultException(errors:List((,List(JsonValidationError(List('eoh' is undefined on object: {"store":"8767565","sku":"1983278364782364782"}),WrappedArray())))))
I need to extract JsResultException, JsonValidationError as string, message 'eoh' is undefined on object message and JSON before object:.
Is this for make graceful log.
Consider converting JsResultException.errors which is
Seq[(JsPath, Seq[JsonValidationError])]
where JsonValidationError.errors is yet another sequence Seq[String], into a simpler tuple
Seq[(JsPath, String)]
like so
case JsResultException(errors) =>
errors.map { case (path, validationErrors) => path -> validationErrors.map(_.messages.mkString(",")).mkString(",") }
This would produce a more managable structure similar to
List((/id,error.path.missing), (/name,error.path.missing))
instead of
List((/id,List(JsonValidationError(List(error.path.missing),WrappedArray()))), (/name,List(JsonValidationError(List(error.path.missing),WrappedArray())))))]
Here is a working example
case class User(name: String, id: Int)
object User {
implicit val formats = Json.format[User]
}
val raw = """{ "nam": "mario", "i": 5 }"""
try {
Json.parse(raw).as[User]
} catch {
case JsResultException(errors) =>
errors.map { case (path, validationErrors) => path -> validationErrors.map(_.messages.mkString(",")).mkString(",") }
}
Also consider using validation to avoid throwing exceptions like so
Json.parse(raw).validate[User] match {
case s: JsSuccess[User] => s
case JsError(errors) =>
errors.map { case (path, validationErrors) => path -> validationErrors.map(_.messages.mkString(",")).mkString(",") }
}
You can always use the scala.util.Try and pattern match the Failure.
import scala.util._
def someExceptionThrowingMethod(): T = ???
Try(someExceptionThrowingMethod()) match {
case Success(t: T) => ???
case Failure(exception: Throwable) => exception match {
case JsResultException((_, JsonValidationError(headMessage :: _) :: _, _) :: _) =>
//here headMessage is the 'eoh' is undefined on object: {"store":"8767565","sku":"1983278364782364782"} message you wrote above
case otherException: Throwable => ???
}
}
Im using https://github.com/AsyncHttpClient/async-http-client this library in my scala project, and im performing some http calls with it, but now on some http calls I need to retry a call if I dont get my expected result for 3 times.
How should I implement something like this?
thaknks
This is an example of retry function based on Future.recoverWith
If you run it you can see that it prints "run process" until the Future is successful but not more than 'times' times
object X extends App{
type Request = String
type Response = String
import scala.concurrent.ExecutionContext.Implicits.global
def retry(request: Request, process: Request => Future[Response], times: Int): Future[Response] ={
val responseF = process(request)
if(times > 0)
responseF.recoverWith{
case ex => println("fail")
retry(request, process, times - 1)
}
else
responseF
}
def process(s: Request): Future[Response] = {
println("run process")
if(Random.nextBoolean()) Future.successful("good") else Future.failed(new Exception)
}
val result = retry("", process, 3)
import scala.concurrent.duration._
println(Await.result(result, 1.second))
}
I got EOFException when I use elasticsearch_river as output and try tu connect to rabbitmq.
Here is my config file :
input {
file {
path => "/tmp/logstash/logstash-1.4.2/log.txt"
}
}
filter {
}
output {
elasticsearch_river {
rabbitmq_host => "hostname"
rabbitmq_port => 15671
es_host => "hostname"
user => "user"
password => "password"
}
}
This is the stack trace :
Exception in thread ">output" java.io.IOException
at com.rabbitmq.client.impl.AMQChannel.wrap(com/rabbitmq/client/impl/AMQChannel.java:106)
at com.rabbitmq.client.impl.AMQChannel.wrap(com/rabbitmq/client/impl/AMQChannel.java:102)
at com.rabbitmq.client.impl.AMQConnection.start(com/rabbitmq/client/impl/AMQConnection.java:378)
at com.rabbitmq.client.ConnectionFactory.newConnection(com/rabbitmq/client/ConnectionFactory.java:516)
at com.rabbitmq.client.ConnectionFactory.newConnection(com/rabbitmq/client/ConnectionFactory.java:533)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
at RUBY.new_connection_impl(/tmp/logstash/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/march_hare-2.1.2-java/lib/march_hare/session.rb:387)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:271)
at RUBY.converting_rjc_exceptions_to_ruby(/tmp/logstash/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/march_hare-2.1.2-java/lib/march_hare/session.rb:357)
at RUBY.new_connection_impl(/tmp/logstash/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/march_hare-2.1.2-java/lib/march_hare/session.rb:382)
at RUBY.initialize(/tmp/logstash/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/march_hare-2.1.2-java/lib/march_hare/session.rb:82)
at RUBY.connect(/tmp/logstash/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/march_hare-2.1.2-java/lib/march_hare/session.rb:69)
at RUBY.connect(/tmp/logstash/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/march_hare-2.1.2-java/lib/march_hare.rb:20)
at RUBY.connect(/tmp/logstash/logstash-1.4.2/lib/logstash/outputs/rabbitmq/march_hare.rb:111)
at RUBY.register(/tmp/logstash/logstash-1.4.2/lib/logstash/outputs/rabbitmq/march_hare.rb:18)
at RUBY.prepare_river(/tmp/logstash/logstash-1.4.2/lib/logstash/outputs/elasticsearch_river.rb:111)
at RUBY.register(/tmp/logstash/logstash-1.4.2/lib/logstash/outputs/elasticsearch_river.rb:89)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at RUBY.outputworker(/tmp/logstash/logstash-1.4.2/lib/logstash/pipeline.rb:220)
at RUBY.start_outputs(/tmp/logstash/logstash-1.4.2/lib/logstash/pipeline.rb:152)
at java.lang.Thread.run(java/lang/Thread.java:745)
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException
....
use 5672,
"15672 is the port for the HTTP API / web UI.
AMQP uses port 5672."
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2013-October/030989.html
also don't forget to install the plugin for ES as described in the link below and restart ES.
https://github.com/elastic/elasticsearch-river-rabbitmq/blob/master/README.md