We Keep Coding

CompletableFuture.runAsync not executing the number of times it was invoked

I am new to asynchronus coding in Java.
This is my code in spring boot application:
public void func(String someString){
if(someString != null){
doAsync(someString);
publish(topic, someString);
}
}
public void doAsync(String someString) {
log.info("Inside do Async");
CompletableFuture.runAsync(() -> {
try {
execute(someString);
} catch (Throwable throwable) {
log.error("Error");
}
}, executorService);
}
private void execute(String someString) {
try {
log.info("Inside Execute");
DBcall(someString);
} catch (Throwable e) {
log.info("Error");
}
}
The func() is being called around 200k times through an event in a Queue, In logs the log Inside do Async appeared for 200k times but the log Inside Execute appeared just for 195k times.
I see no errors/exception occurred in this flow.
Why is it not running consistently for all 200k events? Am I missing something in the implementation?
The publish() function publishes the same message to another subscriber which is in the same service, there are some 10k-11k errors in that subscriber flow(null pointer error). Is error in this flow is main reason behind not executing all async call?

RabbitMq: works through times, not as I expect

And so, step by step, what do I want to do:
I receive data if an error occurs when sending to another system, then I want to send data to rabbitMQ:
#Override
public void updateAnketaIfThrowThenSendMessageInRabbit(ProfileId profileId, ChangeClientAnketaRequest anketa, String profileVersion) {
try {
anketaService.updateAnketa(profileId, anketa, profileVersion);
} catch (ClubProNotAvailableException e) {
rabbitTemplate.convertAndSend(config.getExchange(), config.getRoutingKey(), clubProNotAvailableRabbit);
Anketa a = conversionService.convert(conversionService.convert(anketa, UgAnketa.class), Anketa.class);
profileService.updateProfileAnketa(profileId, a, null);
}
}
}
Next, I want to accept these data and queues and try sending them again at a certain time interval.
For this i:
I accept messages
I'm trying to resend it:
a) If everything was successful, I delete it from the queue
b) If an error occurred, I call the stop method for the container. After a certain time I use the scheduler to call the start method for the container
#Override
public void onMessage(Message message, Channel channel) throws IOException {
ClubProNotAvailableRabbit data = null;
try {
data = OBJECT_MAPPER.readValue(message.getBody(), ClubProNotAvailableRabbit.class);
MDC.put(data.getRequestContextRabbit().getRequestId(), UUID.randomUUID().toString());
requestContextService.put(createRequestContext(data.getRequestContextRabbit(), data.getRequestContextRabbit().getFront()));
methodCall(data);
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
} catch (ClubProNotAvailableException e) {
listenerContainer.stop();
throw new ClubProNotAvailableException();
}
}
public void startContainer() {
listenerContainer.start();
}
I have encountered such problems:
The message is not delivered to the queue every time. Sometimes I have to call the convert And Send method several times.
When I got messages from the queue and an error occurred, I turn off the container, then when it turns on, the queue is empty, and when I turn off, I see this message:
2020-07-20 21:36:59.878 [INFO ] o.s.a.r.l.SimpleMessageListenerContainer - Workers not finished.
2020-07-20 21:36:59.878 [WARN ] o.s.a.r.l.SimpleMessageListenerContainer - Closing channel for unresponsive consumer: Consumer#77416991: tags=[[amq.ctag-E62UisbYdAAOQIM2bWr08w]], channel=Cached Rabbit Channel: AMQChannel(amqp://usergate_tst#10.64.177.12:5672/,35), conn: Proxy#6c60c170 Shared Rabbit Connection: SimpleConnection#5fb65b3a [delegate=amqp://usergate_tst#10.64.177.12:5672/, localPort= 59801], acknowledgeMode=MANUAL local queue size=0
How can I fix this situation?
CONTINUED QUESTION.
I corrected the code like this:
#Override
public void onMessage(Message message, Channel channel) throws IOException, InterruptedException {
ClubProNotAvailableRabbit data = null;
try {
data = OBJECT_MAPPER.readValue(message.getBody(), ClubProNotAvailableRabbit.class);
MDC.put(data.getRequestContextRabbit().getRequestId(), UUID.randomUUID().toString());
requestContextService.put(createRequestContext(data.getRequestContextRabbit(), data.getRequestContextRabbit().getFront()));
methodCall(data);
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
} catch (ClubProNotAvailableException e) {
channel.basicReject(message.getMessageProperties().getDeliveryTag(), true);
Thread.sleep(20000);
}
}
Thread.sleep here for experiment.
I expect that when I grab a message from the queue in the rabbitmq admin console, I will see it go to Unacked status, this is how it happens.
Then, when an error occurs, I call the basicReject method, and I want the status to become ready, immediately after the basicReject call line, but it becomes ready as soon as the method completes completely.
Unacked status:
Although the baseReject method has already worked.
Why is this happening? how is it supposed to work and what mechanism? why doesn't the message become immediately ready (status in console rabbit) after calling the baseReject method?

Closing channel for unresponsive consumer:
This means the listener is "stuck" in your code - you can't call stop() from the listener itself - the container.stop() waits for the listener to exit. You should use stop(() -> log.info("stopped container")) instead.
You need to basicReject in the catch case - the container won't handle it for you with MANUAL acks.
You MUST use MANUAL acks if you ack/nack the message yourself.
It's generally better to let the container take care of acking your messages.

How to identify the MQTT topic that received the message?

The client is subscribed to a x / # topic. There is the possibility of receiving message in the topics x / start and x / stop, and depending on the topic, it performs an action. I wonder how I can identify if it's coming up in the start or stop topic.
In the current code, I send an "action" key in the JSON: "start" or "stop". I want to delete this key and use the format that said above, identifying the topic.
Any further information they deem necessary, please request that I edit the post!
JDK 8
The code:
private MqttCallback callback = new MqttCallback() {
public void connectionLost(Throwable throwable) {
try {
connect();
} catch (MqttException e) {
e.printStackTrace();
}
}
public void messageArrived(String s, MqttMessage mqttMessage) throws Exception {
String messageReceived = new String(mqttMessage.getPayload());
actionPerformed(messageReceived);
}
public void deliveryComplete(IMqttDeliveryToken iMqttDeliveryToken) {
}
};
private void actionPerformed(String message) throws IOException {
ClientDTO clientDTO = new ObjectMapper().readValue(message, ClientDTO.class);
if (clientDTO.getAction().equalsIgnoreCase("start")) {
startView(clientDTO);
} else if (clientDTO.getAction().equalsIgnoreCase("stop")) {
stopView();
}
}
public void connect() throws MqttException {
MqttConnectOptions options = new MqttConnectOptions();
options.setUserName("a_nice_username");
options.setPassword("a_cool_password".toCharArray());
options.setAutomaticReconnect(true);
MqttClient client = new MqttClient("someaddress", MqttClient.generateClientId());
client.setCallback(callback);
try {
client.connect(options);
client.subscribe(topic);
TaskbarIcon.alteraIconeOnline();
} catch (Exception e) {
TaskbarIcon.alteraIconeOffline();
}
}
public void tipoConexao(int tipoConex) throws IOException {
switch (tipoConex) {
case 0:
topic += "/operador/" + getIdReceived() + "/#";
System.out.println(topic);
break;
//etc
}

The s in this method is the topic: public void messageArrived(String s, MqttMessage mqttMessage)
As is very well documented here:
messageArrived
void messageArrived(java.lang.String topic, MqttMessage message) throws java.lang.Exception
This method is called when a message arrives from the server.
This method is invoked synchronously by the MQTT client. An acknowledgment is not sent back to the server until this method
returns cleanly.
If an implementation of this method throws an Exception, then the client will be shut down. When the client is next re-connected, any
QoS 1 or 2 messages will be redelivered by the server.
Any additional messages which arrive while an implementation of this method is running, will build up in memory, and will then back up
on the network.
If an application needs to persist data, then it should ensure the data is persisted prior to returning from this method, as after
returning from this method, the message is considered to have been
delivered, and will not be reproducible.
It is possible to send a new message within an implementation of this callback (for example, a response to this message), but the
implementation must not disconnect the client, as it will be
impossible to send an acknowledgment for the message being processed,
and a deadlock will occur.
Parameters:
topic - name of the topic on the message was published to
message - the actual message.
Throws:
java.lang.Exception - if a terminal error has occurred, and the client should be shut down.

Netty 4 - The pool returns a channel which is not yet ready to send the the actual message

I have created an inbound handler of type SimpleChannelInboundHandler and added to pipeline. My intention is every time a connection is established, I wanted to send an application message called session open message and make the connection ready to send the actual message. To achieve this, the above inbound handler
over rides channelActive() where session open message is sent, In response to that I would get a session open confirmation message. Only after that I should be able to send any number of actual business message. I am using FixedChannelPool and initialised as follows. This works well some time on startup. But if the remote host closes the connection, after that if a message is sent calling the below sendMessage(), the message is sent even before the session open message through channelActive() and its response is obtained. So the server ignores the message as the session is not open yet when the business message was sent.
What I am looking for is, the pool should return only those channel that has called channelActive() event which has already sent the session open message and it has got its session open confirmation message from the server. How to deal with this situation?
public class SessionHandler extends SimpleChannelInboundHandler<byte[]> {
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
super.channelActive(ctx);
if (ctx.channel().isWritable()) {
ctx.channel().writeAndFlush("open session message".getBytes()).;
}
}
}
// At the time of loading the applicaiton
public void init() {
final Bootstrap bootStrap = new Bootstrap();
bootStrap.group(group).channel(NioSocketChannel.class).remoteAddress(hostname, port);
fixedPool = new FixedChannelPool(bootStrap, getChannelHandler(), 5);
// This is done to intialise connection and the channelActive() from above handler is invoked to keep the session open on startup
for (int i = 0; i < config.getMaxConnections(); i++) {
fixedPool.acquire().addListener(new FutureListener<Channel>() {
#Override
public void operationComplete(Future<Channel> future) throws Exception {
if (future.isSuccess()) {
} else {
LOGGER.error(" Channel initialzation failed...>>", future.cause());
}
}
});
}
}
//To actually send the message following method is invoked by the application.
public void sendMessage(final String businessMessage) {
fixedPool.acquire().addListener(new FutureListener<Channel>() {
#Override
public void operationComplete(Future<Channel> future) throws Exception {
if (future.isSuccess()) {
Channel channel = future.get();
if (channel.isOpen() && channel.isActive() && channel.isWritable()) {
channel.writeAndFlush(businessMessage).addListener(new GenericFutureListener<ChannelFuture>() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
if (future.isSuccess()) {
// success msg
} else {
// failure msg
}
}
});
fixedPool.release(channel);
}
} else {
// Failure
}
}
});
}

If there is no specific reason that you need to use a FixedChannelPool then you can use another data structure (List/Map) to store the Channels. You can add a channel to the data structure after sending open session message and remove it in the channelInactive method.
If you need to perform bulk operations on channels you can use a ChannelGroup for the purpose.
If you still want you use the FixedChannelPool you may set an attribute in the channel on whether open message was sent:
ctx.channel().attr(OPEN_MESSAGE_SENT).set(true);
you can get the attribute as follows in your sendMessage function:
boolean sent = ctx.channel().attr(OPEN_MESSAGE_SENT).get();
and in the channelInactive you may set the same to false or remove it.
Note OPEN_MESSAGE_SENT is an AttributeKey:
public static final AttributeKey<Boolean> OPEN_MESSAGE_SENT = AttributeKey.valueOf("OPEN_MESSAGE_SENT");

I know this is a rather old question, but I stumbled across the similar issue, not quite the same, but my issue was the ChannelInitializer in the Bootstrap.handler was never called.
The solution was to add the pipeline handlers to the pool handler's channelCreated method.
Here is my pool definition code that works now:
pool = new FixedChannelPool(httpBootstrap, new ChannelPoolHandler() {
#Override
public void channelCreated(Channel ch) throws Exception {
ChannelPipeline pipeline = ch.pipeline();
pipeline.addLast(HTTP_CODEC, new HttpClientCodec());
pipeline.addLast(HTTP_HANDLER, new NettyHttpClientHandler());
}
#Override
public void channelAcquired(Channel ch) {
// NOOP
}
#Override
public void channelReleased(Channel ch) {
// NOOP
}
}, 10);
So in the getChannelHandler() method I assume you're creating a ChannelPoolHandler in its channelCreated method you could send your session message (ch.writeAndFlush("open session message".getBytes());) assuming you only need to send the session message once when a connection is created, else you if you need to send the session message every time you could add it to the channelAcquired method.

Stopping a Camel route from outside the route

We have a data processing application that runs on Karaf 2.4.3 with Camel 2.15.3.
In this application, we have a bunch of routes that import data. We have a management view that lists these routes and where each route can be started. Those routes do not directly import data, but call other routes (some of them in other bundles, called via direct-vm), sometimes directly and sometimes in a splitter.
Is there a way to also completely stop a route/therefore stopping the entire exchange from being further processed?
When simply using the stopRoute function like this:
route.getRouteContext().getCamelContext().stopRoute(route.getId());
I eventually get a success message with Graceful shutdown of 1 routes completed in 10 seconds - the exchange is still being processed though...
So I tried to mimic the behaviour of the StopProcessor by setting the stop property, but that also didn't help:
public void stopRoute(Route route) {
try {
Collection<InflightExchange> browse = route.getRouteContext().getCamelContext().getInflightRepository()
.browse();
for (InflightExchange inflightExchange : browse) {
String exchangeRouteId = inflightExchange.getRouteId();
if ((exchangeRouteId != null) && exchangeRouteId.equals(route.getId())) {
this.stopExchange(inflightExchange.getExchange());
}
}
} catch (Exception e) {
Notification.show("Error while trying to stop route", Type.ERROR_MESSAGE);
LOGGER.error(e, e);
}
}
public void stopExchange(Exchange exchange) throws Exception {
AsyncProcessorHelper.process(new AsyncProcessor() {
#Override
public void process(Exchange exchange) throws Exception {
AsyncProcessorHelper.process(this, exchange);
}
#Override
public boolean process(Exchange exchange, AsyncCallback callback) {
exchange.setProperty(Exchange.ROUTE_STOP, Boolean.TRUE);
callback.done(true);
return true;
}
}, exchange);
}
Is there any way to completely stop an exchange from being processed from outside the route?

Can you get an exchange?
I use exchange.setProperty(Exchange.ROUTE_STOP, true);
Route stop flow and doesn't go to next route.