Recently, I find some BlockingOperationException in my netty4 project.
Some people said that when using the sync() method of start netty's ServerBootstrap can cause dead lock, because sync() will invoke await() method, and there is a method called 'checkDeadLock' in await().
But I don't think so. ServerBootstrap use the EventLoopGroup called boosGroup, and Channel use the workerGroup to operation IO, I don't think they will influence each other, they have different EventExecutor.
And in my practice, Deadlock exception doesn't appear in the Netty startup process, most of which occurs after the Channel of the await writeAndFlush.
Analysis source code, checkDeadLock, BlockingOperationException exception thrown is when the current thread and executor thread of execution is the same.
My project code is blow:
private void channelWrite(T message) {
boolean success = true;
boolean sent = true;
int timeout = 60;
try {
ChannelFuture cf = cxt.write(message);
cxt.flush();
if (sent) {
success = cf.await(timeout);
}
if (cf.isSuccess()) {
logger.debug("send success.");
}
Throwable cause = cf.cause();
if (cause != null) {
this.fireError(new PushException(cause));
}
} catch (LostConnectException e) {
this.fireError(new PushException(e));
} catch (Exception e) {
this.fireError(new PushException(e));
} catch (Throwable e) {
this.fireError(new PushException("Failed to send message“, e));
}
if (!success) {
this.fireError(new PushException("Failed to send message"));
}
}
I know Netty officials advise not to use sync() or await() method, but I want to know what situation will causes deadlocks in process and the current thread and executor thread of execution is the same.
I change my project code.
private void pushMessage0(T message) {
try {
ChannelFuture cf = cxt.writeAndFlush(message);
cf.addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws PushException {
if (future.isSuccess()) {
logger.debug("send success.");
} else {
throw new PushException("Failed to send message.");
}
Throwable cause = future.cause();
if (cause != null) {
throw new PushException(cause);
}
}
});
} catch (LostConnectException e) {
this.fireError(new PushException(e));
} catch (Exception e) {
this.fireError(new PushException(e));
} catch (Throwable e) {
this.fireError(new PushException(e));
}
}
But I face a new problem, I can't get the pushException from the ChannelHandlerListener.
BlockingOperationException will be throw by netty if you call sync*or await* on a Future in the same thread that the EventExecutor is using and to which the Future is tied to. This is usually the EventLoop that is used by the Channel itself.
Can not call await in IO thread is understandable. However, there are 2 points.
1. If you call below code in channel handler, no exception will be reported, because the the most of the time the check of isDone in await returns true, since you are in IO thread, and IO thread is writing data synchronously. the data has been written when await is called.
ChannelPromise p = ctx.writeAndFlush(msg);
p.await()
If add a handler in different EventExecutorGroup, this check is not necessary, since that executor is newly created and is not the same one with the channel's IO executor.
Related
I need to re-throw the runtime exception from the CompletableFuture Exception block to the parent method, but looks like the runtimeException is getting logged at the exception block itself and not propagating back to the parent method. Can anyone please advise/suggest me what I am doing wrong here or is this the right way to throw?
I have a function call like below :
void method1(String msg)
{
try
{
method2(msg);
}
catch(RuntimeException e)
{
throw new CustomException("");
}
}
void method2(String msg)
{
CompletableFuture<Void> future = asyncTask.process(msg);
future.whenComplete((res,ex) ->
if(ex != null)
{
log.error("");
throw RuntimeException("abc");
}
else
{ postProcessTasks(msg);
}
);
}
The calling code doesn't wait for the Future to complete or get the status. Here's the core of what needs to be added.
CompletableFuture<Void> future = asyncTask.process(msg)
.thenAccept(this::postProcessTasks);
try {
future.get();
} catch (ExecutionException e) {
Throwable asyncException = e.getCause();
// .. do something here
}
If you intend to propagate an asynchronous Exception back to the caller then whenComplete is probably not the call to use; It is meant to handle exceptions within the asynch call.
In the below code snipped when dispose() is called, then the emitter thread is interrupted (InterruptedException is thrown out of sleep method).
Observable<Integer> obs = Observable.create(emitter -> {
for (int i = 0; i < 10; i++) {
if (emitter.isDisposed()) {
System.out.println("> exiting.");
emitter.onComplete();
return;
}
emitter.onNext(i);
System.out.println("> calculation = " + i);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
emitter.onComplete();
});
Disposable disposable = obs
.subscribeOn(Schedulers.computation())
.subscribe(System.out::println);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
disposable.dispose();
From debugging session I see that the interrupt origins from FutureTask which is cancelled during disposal. In there, the thread that is calling dispose() is checked against runner thread, and if it does not match, the emitter is interrupted. The thread is different since I used computation Scheduler.
Is there any way to make dispose not interrupt such emitter or is it how this actually should always be handled? An issue I see with this approach is when I would have an interruptible operation (simulated here by sleep) that I would want to complete normally before calling onComplete().
Please refer to What's different in 2.0 - Error handling.
One important design requirement for 2.x is that no Throwable errors should be swallowed. This means errors that can't be emitted because the downstream's lifecycle already reached its terminal state or the downstream cancelled a sequence which was about to emit an error.
So you can either wrap everything inside a try/catch and properly emit the error:
Observable<Integer> obs = Observable.create(emitter -> {
try {
// ...
} catch (InterruptedException ex) {
// check if the interrupt is due to cancellation
// if so, no need to signal the InterruptedException
if (!disposable.isDisposed()) {
observer.onError(ex);
}
}
});
or setup a global error consumer to ignore it:
RxJavaPlugins.setErrorHandler(e -> {
// ..
if (e instanceof InterruptedException) {
// fine, some blocking code was interrupted by a dispose call
return;
}
// ...
Log.warning("Undeliverable exception received, not sure what to do", e);
});
I have a multithreaded code in java.
I debug with intellij. I know how to switch debug contexts between threads.
Nevertheless the console shows only exception thrown for the main thread.
How can i see the exceptions thrown from any thread?
basically when I run integration test with an Executor, some exception is thrown from secondary thread.
But nothing is printed to the debug-console.
Actually the the code flies and never reach to the try catch i have.
It's like I cannot step-next after the code that throws an exception
try {
while (true) {
logger.debug("New thread. polling on DB, polling on deployment service");
ConfigAction configAction = configActionFactory.get(ConfigActionsEnum.PushToDeployment);
while (configAction != null) {
configAction = configAction.execute() ? configAction.nextActivity() : null;
}
}
} catch (Exception e) {
e.printStackTrace();
logger.error("DeploymentContextListener run failed", e);
}
whereas when I run with the main thread, the same code I see the exception:
public void run() {
try {
while (true) {
logger.debug("New thread. polling on DB, polling on deployment service");
ConfigAction configAction = configActionFactory.get(ConfigActionsEnum.PushToDeployment);
while (configAction != null) {
configAction = configAction.execute() ? configAction.nextActivity() : null;
}
}
} catch (Exception e) {
e.printStackTrace();
logger.error("DeploymentContextListener run failed", e);
}
it is written in one of the try-catch i have
} catch (Exception ex) {
logger.error("failed to get from DB: getLatestConfigVersionWaitingForDeploy", ex);
}
My problem was that i used:
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(() -> { }
from what I have read:
invokeAll - call “future.get()”. Wait for all threads to finish.
Submit -> attach the errors to Future objects, not the the std-error
Execute -> the errors are thrown to the std-error
We're using Spring Cloud AWS to interact with SQS. We use the #SqsListener annotation to pull messages off our queues. We have deletionPolicy = NEVER, which means we manually acknowledge all messages we pick off.
Our problem is that the SimpleMessageListenerContainer (which handles the processing of messages from a queue) waits for all worker threads to finish before picking further messages off the queue.
In other words, what we're seeing is this:
Pull 10 messages off a queue.
Start 10 threads to do work.
One of the threads doing work gets blocked on a slow IO call.
The application is now blocked from fetching more messages off the queue, and is as such blocked from doing more work at all, until that slow call finishes.
We can see the code in SimpleMessageListenerContainer.AsynchronousMessageListener which is responsible for this
#Override
public void run() {
while (isQueueRunning()) {
try {
ReceiveMessageResult receiveMessageResult = getAmazonSqs().receiveMessage(this.queueAttributes.getReceiveMessageRequest());
CountDownLatch messageBatchLatch = new CountDownLatch(receiveMessageResult.getMessages().size());
for (Message message : receiveMessageResult.getMessages()) {
if (isQueueRunning()) {
MessageExecutor messageExecutor = new MessageExecutor(this.logicalQueueName, message, this.queueAttributes);
getTaskExecutor().execute(new SignalExecutingRunnable(messageBatchLatch, messageExecutor));
} else {
messageBatchLatch.countDown();
}
}
try {
messageBatchLatch.await();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
} catch (Exception e) {
getLogger().warn("An Exception occurred while polling queue '{}'. The failing operation will be " +
"retried in {} milliseconds", this.logicalQueueName, getBackOffTime(), e);
try {
//noinspection BusyWait
Thread.sleep(getBackOffTime());
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
}
}
}
}
Ideally, we'd like for the message listener to continually pick messages off the queue for processing.
We can't seem to implement our own MessageListenerContainer since the AbstractMessageListenerContainer is package local.
Is there any way around this behavior?
What is holding the message polling thread is the messageBatchLatch.await() statement. It seems that just removing the latch would do it. Something like:
#Override
public void run() {
while (isQueueRunning()) {
try {
ReceiveMessageResult receiveMessageResult = getAmazonSqs().receiveMessage(this.queueAttributes.getReceiveMessageRequest());
for (Message message : receiveMessageResult.getMessages()) {
if (isQueueRunning()) {
MessageExecutor messageExecutor = new MessageExecutor(this.logicalQueueName, message, this.queueAttributes);
getTaskExecutor().execute(new SignalExecutingRunnable(messageExecutor));
}
}
} catch (Exception e) {
getLogger().warn("An Exception occurred while polling queue '{}'. The failing operation will be " +
"retried in {} milliseconds", this.logicalQueueName, getBackOffTime(), e);
try {
//noinspection BusyWait
Thread.sleep(getBackOffTime());
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
}
}
}
}
This will work if your TaskExecutor implementation:
- Has a fixed-size thread pool
- Blocks when execute function gets called and no threads are available.
This is how most of implementations work but it is worth checking yours.
I tried to use multiple threads, sadly no luck:
public synchronized boolean pingServer(final String ip, final short port) {
final boolean[] returnbol = new boolean[1];
Thread tt = new Thread(new Runnable() {
#Override
public void run() {
try {
Socket s = new Socket(ip, port);
s.close();
returnbol[0] = true;
} catch (IOException e) {
returnbol[0] = false;
}
}
});
tt.start();
try {
tt.join();
} catch (InterruptedException e) {
tt.stop();
}
tt.stop();
return returnbol[0];
}
The main thread still Freezes for some reason.
Is there a "lagless" way to ping a server?
What exactly did you want to got in
try {
tt.join();
} catch (InterruptedException e) {
tt.stop();
}
block?
Here you joined to parallel thread and waits till this thread will ends (got ping result).
You have next options:
Wait till ping ends
Don't wait... and don't got result
Use some concurrency classes like Future<> to got result (but you will block thread at moment you ask result if it not retrieved yet)
Or you can use some 'callback' function/interface to threw result from inner 'ping' thread.
You will need to remove the following lines from your code.
The tt.join() will force the main thread to wait for tt to finish.
try {
tt.join();
} catch (InterruptedException e) {
tt.stop();
}
tt.stop();
Use a Future instead to get the result for later use