I am using Server-Sent events on one browser, and a spring boot application on the back end. When I shot down the client, I get the next exception:
14:35:09.458 [http-nio-8084-exec-25] ERROR o.a.c.c.C.[Tomcat].[localhost] - Exception Processing ErrorPage[errorCode=0, location=/error]
org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
I understand this is the expected behavior; on the other hand, my application works fine, but I have awful logs full of those exceptions. I guess this is caused by Tomcat. Is there a way to catch these exceptions, or at least to prevent Tomcat from writing this exception stack trace to the log? I mean, without modifying Tomcat's code.
To prevent this exception in logs you can try some changes in your code that performs push on client. Here is my example. I listen to api called and then it called I push socket to the client. I think you could understand the code:
#GetMapping("/api/status/{groupId}/{groupstatusId}")
#ResponseStatus(HttpStatus.NO_CONTENT)
#ExceptionHandler(IOException.class)
public void listenToNewStatus(#PathVariable Long groupId, #PathVariable String groupstatusId, IOException e) {
Group group = groupDTOService.findById(groupId);
if (group != null) {
if (StringUtils.containsIgnoreCase(ExceptionUtils.getRootCauseMessage(e), "Broken pipe")) {
logger.info("broken pipe");
} else {
template.convertAndSend("/topic/callstatus/" + group.getUser().getId(), groupstatusId);
}
}
In this code to prevent broken pipe I add annotation #ExceptionHandler(IOException.class) and check if exception contains broken pipe then nothing else send message to client.
I see that this question is quite old, but in case someone is still looking for an answer, there are a few blog posts on how to mute ClientAbortException so it doesn't flood your logs.
https://tutorial-academy.com/jersey-workaround-clientabortexception-ioexception/
Related
I'm running an HL Fabric private network and submitting transactions to the ledger from a Java Application using Fabric-Java-Sdk.
Occasionally, like 1/10000 of the times, the Java application throws an exception when I'm submitting the transaction to the ledger, like the message below:
ERROR 196664 --- [ Thread-4] org.hyperledger.fabric.sdk.Channel
: Future completed exceptionally: sendTransaction
java.lang.IllegalArgumentException: The proposal responses have 2
inconsistent groups with 0 that are invalid. Expected all to be
consistent and none to be invalid. at
org.hyperledger.fabric.sdk.Channel.doSendTransaction(Channel.java:5574)
~[fabric-sdk-java-2.1.1.jar:na] at
org.hyperledger.fabric.sdk.Channel.sendTransaction(Channel.java:5533)
~[fabric-sdk-java-2.1.1.jar:na] at
org.hyperledger.fabric.gateway.impl.TransactionImpl.commitTransaction(TransactionImpl.java:138)
~[fabric-gateway-java-2.1.1.jar:na] at
org.hyperledger.fabric.gateway.impl.TransactionImpl.submit(TransactionImpl.java:96)
~[fabric-gateway-java-2.1.1.jar:na] at
org.hyperledger.fabric.gateway.impl.ContractImpl.submitTransaction(ContractImpl.java:50)
~[fabric-gateway-java-2.1.1.jar:na] at
com.apidemoblockchain.RepositoryDao.BaseFunctions.Implementations.PairTrustBaseFunction.sendTrustTransactionMessage(PairTrustBaseFunction.java:165)
~[classes/:na] at
com.apidemoblockchain.RepositoryDao.Implementations.PairTrustDataAccessRepository.run(PairTrustDataAccessRepository.java:79)
~[classes/:na] at java.base/java.lang.Thread.run(Thread.java:834)
~[na:na]
While my submitting method goes like this:
public void sendTrustTransactionMessage(Gateway gateway, Contract trustContract, String payload) throws TimeoutException, InterruptedException, InvalidArgumentException, TransactionException, ContractException {
// Prepare
checkIfChannelIsReady(gateway);
// Execute
trustContract.submitTransaction(getCreateTrustMethod(), payload);
}
I'm using a 4 org network with 2 peers each and I am using 3 channels, one for each chaincode DataType, in order to keep the things clean.
I think that the error coming from the Channel doesn't make sense because I am using the Contract to submit it...
Like I'm opening the gateway and then I keep it open for continuously submit the txs.
try (Gateway gateway = getBuilder(getTrustPeer()).connect()) {
Contract trustContract = gateway.getNetwork(getTrustChaincodeChannelName()).getContract(getTrustChaincodeId(), getTrustChaincodeName());
while (!terminateLoop) {
if (message) {
String payload = preparePayload();
sendTrustTransactionMessage(gateway, trustContract, payload);
}
...
wait();
}
...
}
EDIT:
After reading #bestbeforetoday advice, I've managed to catch the ContractException and analyze the logs. Still, I don't fully understand where might be the bug and, therefore, how to fix it.
I'll add 3 prints that I've taken to the ProposalResponses received in the exception and a comment after it.
ProposalResponses-1
ProposalResponses-2
ProposalResponses-3
So, in the first picture, I can see that 3 proposal responses were received at the exception and the exception cause message says:
"The proposal responses have 2 inconsistent groups with 0 that are invalid. Expected all to be consistent and none to be invalid."
In pictures, 2/3 is represented the content of those responses and I notice that there are 2 fields saving null value, namely "ProposalRespondePayload" and "timestamp_", however, I don't know if those are the "two groups" referred at the message cause of the exception.
Thanks in advance...
It seems that, while the endorsing peers all successfully endorsed your transaction proposal, those peer responses were not all byte-for-byte identical.
There are several things that might differ, including read/write sets or the value returned from the transaction function invocation. There are several reasons why differences might occur, including non-deterministic transaction function implementation, different transaction function behaviour between peers, or different ledger state at different peers.
To figure out what caused this specific failure you probably need to look at the peer responses to identify how they differ. You should be getting a ContractException thrown back from your transaction submit call, and this should allow you to access the proposal responses by calling e.getProposalResponses():
https://hyperledger.github.io/fabric-gateway-java/release-2.2/org/hyperledger/fabric/gateway/ContractException.html#getProposalResponses()
Hey i try to using Ktor with the Kotlin Hands on. But I have a Problem when access the Web Page with "/".
embeddedServer(Netty, port) {
install(FreeMarker) {
templateLoader = ClassTemplateLoader(this::class.java.classLoader, "templates")
outputFormat = HTMLOutputFormat.INSTANCE
}
routing {
static("/static") {
resources("files")
}
get("/") {
call.respond(FreeMarkerContent("index.ftl", mapOf("entries" to blogEntries), ""))
}
}
}.start(wait = false)
There is my code i use. I cant find an error there is only this message in the console:
[eventLoopGroupProxy-4-1] INFO ktor.application - Unhandled: GET - /, io failed
Is there anyone who has an idea how can I fix that error?
The exact line where that message is generated is in io.ktor.server.engine.logFailure: https://github.com/ktorio/ktor/blob/2e1b0046e4235865da7cbbfbf2e887f410775278/ktor-server/ktor-server-host-common/jvm/src/io/ktor/server/engine/DefaultEnginePipeline.kt#L107
private fun ApplicationEnvironment.logFailure(call: ApplicationCall, cause: Throwable) {
...
is IOException -> log.info("$status: $logString, io failed: ${cause.message ?: "unknown error"}")
...
}
Add a breakpoint on this line. Then when your app fails, cause will contain the real exception.
In my case it was an error in the FreeMarker template. Which is weird, because usually those exceptions print normally. This time it only printed Unhandled: <route>, io failed.
It seems that this is a bug in Ktor: these exceptions are thrown in normal behavior, and frequently enough that you don't want a full stacktrace. But sometimes (like in this case) the exception isn't normal and you do want a stack trace.
I suggest using the CallLogging feature and routes tracing to troubleshoot your problem.
I have a service that is expected to execute requests at ~5 or more requests/min. This service depends on Apache AsyncHttpClient. After every few minutes, the clients hits some condition which causes java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED. All requests to the client start failing with same exception message. After service is restarted, this cycle repeats.
It is really hard to debug this problem as the request execution failure surprisingly does not cause a callback to the failed() method of the AsyncResponse.
From what I could gather, there has been a fix HTTPCORE-370 in HttpCore NIO which solved a similar problem in 4.3.2. I am using the following version -
commons-httpclient-3.1.jar
httpasyncclient-4.1.1.jar
httpcore-4.4.4.jar
httpcore-nio-4.4.4.jar
Yet seeing this problem.
I've been dealing with this same exception in my application, and I finally found a helpful suggestion from this post - http://httpcomponents.10934.n7.nabble.com/I-O-reactor-status-STOPPED-td29059.html
You can use #getAuditLog() method of the I/O reactor to find out exactly
what exception caused it to terminate.
If you keep a reference to your ConnectionManager's IOReactor, you can call this method to get insight into the actual problem:
http://hc.apache.org/httpcomponents-core-4.4.x/httpcore-nio/apidocs/org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor.html#getAuditLog()
Turns out I was doing something incredibly stupid in my own code. But I couldn't figure it out until I read the audit log.
If you see OutOfMemoryError before this, try this
-XX:MaxDirectMemorySize=512M
See https://issues.apache.org/jira/browse/HTTPASYNC-104
In my case, using Elasticsearch high level client, this exception is due to esclient.indexAsync(indexRequest,RequestOptions.DEFAULT,null)
I fixed it by add an action listeners in all async requests like this
esclient.indexAsync(indexRequest,RequestOptions.DEFAULT,
new ActionListener<IndexResponse>() {
#Override
public void onResponse(IndexResponse response) {
}
#Override
public void onFailure(Exception e) {
});
We had encountered the same issue and after lots of digging we found that a proper IOReactorExceptionHandler needs to provided to the HttpAsyncClient to avoid this. It's unfortunate that it's not well covered in the documentation.
Below is a snippet of our code where a more robust client builder tries to add the exception handler. Note that IOExceptions would still stop the I/O reactor as they might imply underlying network communication failures. You may adjust according to your unique use cases.
public RobustCloseableHttpAsyncClientBuilder withDefaultExceptionHandler() {
return withExceptionHandler(
new IOReactorExceptionHandler() {
#Override
public boolean handle(RuntimeException ex) {
logger.error(
"RuntimeException occurs in callback, handled by default exception handler and the I/O reactor will be resumed.",
ex);
return true;
}
#Override
public boolean handle(IOException ex) {
logger.error(
"IOException occurs in callback, handled by default exception handler and the I/O reactor will be stopped.",
ex);
return false;
}
});
}
Read this issue report in elasticsearch on Github for more exposure.
The code directly below attempts to send a message through a Java websocket; however, it occasionally produces the error (shown in the error stack below). Could anyone suggest what the problem'solution?
Code:
if(currSession != null && currSession.isOpen()) {
try {
currSession.sendMessage(new BinaryMessage(flowTable.getBytes()));
} catch (IOException e) {
e.printStackTrace();
}
Error Stack:
java.lang.IllegalStateException: The remote endpoint was in state [BINARY_PARTIAL_WRITING] which is an invalid state for called method
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$StateMachine.checkState(WsRemoteEndpointImplBase.java:1015)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$StateMachine.binaryPartialStart(WsRemoteEndpointImplBase.java:963)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendPartialBytes(WsRemoteEndpointImplBase.java:140)
at org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendBinary(WsRemoteEndpointBasic.java:56)
at org.springframework.web.socket.adapter.standard.StandardWebSocketSession.sendBinaryMessage(StandardWebSocketSession.java:202)
at org.springframework.web.socket.adapter.AbstractWebSocketSession.sendMessage(AbstractWebSocketSession.java:107)
at com.hp.fucms.impl.TopoRestController.setFlowTable(TopoRestController.java:147)
Wrap your code in a synchronized method and funnel all calls through this new method. It appears the tomcat web socket cannot handle multiple messages being placed on the same websocket session at the same time. I have code which has been running flawlessly under Glassfish and fell apart instantly when I moved to Tomcat. I then altered my code as explained above and all my problems went away....and there was much rejoicing.
I've been playing with the new Servlet 3.0 async features with Tomcat 7.0.4. I found this Chat Application, that lets clients hang on GET request to get message updates. This is working just fine when it comes to receiving the messages.
The problem arises when the client is disconnected i.e. the user closes the browser. It seems that the server does not raise IOException, even though the client has disconnected. The message thread (see the source code from link above) is happily writing to all stored AsyncContext's output streams.
Is this a Tomcat bug? or am I missing something here? If this is not a bug, then how I'm supposed to detect whether the client has closed the connection?
The code there at line 44 - 47 is taking care of it,
} catch(IOException ex) {
System.out.println(ex);
queue.remove(ac);
}
And here too at 75 - 83, using timeout thingie,
req.addAsyncListener(new AsyncListener() {
public void onComplete(AsyncEvent event) throws IOException {
queue.remove(ac);
}
public void onTimeout(AsyncEvent event) throws IOException {
queue.remove(ac);
}
});
EDIT: After getting a little more insight.
Tomcat 7.0.4 is still in beta. So, you can expect such behaviour
I tried hard but can't find the method setAsyncTimeout() in the doc, neither here, nor here. So, I think they dropped it completely in the final version due to some unknown valid reason
The example states, "why should I use the framework instead of waiting for Servlet 3.0 Async API". Which infers that its written before the final thingie
So, what I can say, after combining all these fact, that you are trying to work with the thing that is broken in a sense. That also, may be, the reason for different and weird results.