Hope you are doing well. Basically I am calling Java API from .NET .
Most of the time it is working absolutely fine but the thing it sometimes once or twice in a day it gives me Socket exception. I am putting down the calling code and the exception stack trace. Any suggestion to fix this once and for all will be helpful.
Thanks in advance.!!
Code :
var client = new HttpClient{Timeout = TimeSpan.FromMinutes(30), BaseAddress = new Uri(AppSettingsConfiguration.GetConfig<string>("appSettings:Generic"))};
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.Add("key", AppSettingsConfiguration.GetConfig<string>("appSettings:JavaAPIKey"));
var response = client.PostAsync(AppSettingsConfiguration.GetConfig<string>("appSettings:Generic"), new StringContent(JsonConvert.SerializeObject(order), Encoding.UTF8, "application/json")).Result;
Stack Trace :
Error reported : One or more errors occurred. Inner Exception
System.Net.Http.HttpRequestException: An error occurred while sending the request. --->
System.Net.WebException: Unable to connect to the remote server --->
System.Net.Sockets.SocketException: A connection attempt failed because the connected party did
not properly respond after a period of time, or established connection failed because connected host has failed to respond {IP} at
System.Net.Sockets.Socket.InternalEndConnect(IAsyncResult asyncResult) at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6,
Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception) --- End of inner exception stack trace --- at
System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context)
at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) --- End of inner
exception stack trace --- Stack trace :
Related
let's say my Java chaincode (running on Fabric 1.4.4) wants to throw an exception to show that the new asset to be created already exists. I am throwing a RunTimeException with the problem or error (In this case, "Contract LL00001 already registered") which is logged in the Peer node executing the transaction:
2019-11-29 20:15:37.807 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.1.4] func2 -> INFO 16a8ec Contract LL00001 already registered
2019-11-29 20:15:37.807 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.1.4] func2 -> INFO 16a8ed java.lang.RuntimeException: Contract LL00001 already registered
But then, after the stack trace I see that the peer node is returning it as 500 error without includeing my error description or any reference to the error Exception in java (this makes sense as that error is language agnostic):
2019-11-29 20:15:37.807 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.1.4] func2 -> INFO 16a8ff 20:15:37:804 SEVERE org.hyperledger.fabric.shim.impl.ChaincodeInnvocationTask call [1f56a053] Invoke failed with error code 500. Sending ERROR
Which is logged in my client java application (which uses fabrik-java-sdk):
org.hyperledger.fabric.sdk.exception.InvalidArgumentException: Proposal response is invalid.
at org.hyperledger.fabric.sdk.ProposalResponse.getChaincodeActionResponsePayload(ProposalResponse.java:272)
at ...
So I just know that there was a problem from the chaincode, but I can't know what the problem is. How can I get the error type and description so I can show the problem to the user? Now I need to go to the peer node an check logs there to see what problem is.
Note: I am extending the new org.hyperledger.fabric.contract.ContractInterface in my chaincode class.
Update: peer node logs the error exception (org.hyperledger.fabric.shim.ChaincodeException) and seems to return correctly the error message ("The document was not found") in the 500 response as shown in log, but this message does not get to Java SDK
2019-12-23 22:11:09.178 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.9.7] func2 -> INFO 5aa7 22:11:09:176 SEVERE org.hyperledger.fabric.shim.impl.ChaincodeInnvocationTask call [12cc4ad0] Invoke failed with error code 500. Sending ERROR
2019-12-23 22:11:09.179 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.9.7] func2 -> INFO 5aa8 22:11:09:177 FINE org.hyperledger.fabric.shim.impl.ChaincodeSupportClient$2 accept > sendToPeer 12cc4ad09a1feb7fc1246ac04bf69509204ca74368be2c7e4bbf0a503e90417f
2019-12-23 22:11:09.181 UTC [endorser] callChaincode -> INFO 5aa9 [mrrc][12cc4ad0] Exit chaincode: name:"mrrc" (36ms)
2019-12-23 22:11:09.181 UTC [endorser] SimulateProposal -> ERRO 5aaa [mrrc][12cc4ad0] failed to invoke chaincode name:"mrrc" , error: transaction returned with failure: The document was not found
Edit: It seems to be a error in Java SDK. I have created a JIRA issue in Fabric's JIRA:
https://jira.hyperledger.org/browse/FABJ-508
To throw this error back to your fabric java sdk client, one way would be to make your chaincode class extend the ChaincodeBase class (which can be imported in your java program by importing org.hyperledger.fabric.shim ) and then you can use its newErrorResponse method in each of your chaincode methods to throw your custom errors wherein you can provide the error string as it's first (or only) parameter. You can possibly have a look at this example from the fabric samples repo:
https://github.com/hyperledger/fabric-samples/blob/master/chaincode/abstore/java/src/main/java/org/hyperledger/fabric-samples/ABstore.java#L28
To have a look at the overloaded implementations of the newErrorResponse method, so that you can see what other possible params you can pass to it, please follow:
https://jar-download.com/artifacts/org.hyperledger.fabric-chaincode-java/fabric-chaincode-shim/1.4.0/source-code/org/hyperledger/fabric/shim/ChaincodeBase.java
UPDATE : If you are using the newer chaincodeInterface (as suggested by icordoba) for your chaincode implementations instead, then you should throw an instance of ChaincodeException class which can be imported by importing org.hyperledger.fabric.shim.ChaincodeException, to achieve the same, you can have a look at a sample chaincode here
I'm trying to use Reactor Netty TcpClient in reactive way to interact with hosts, that may be unreachable. Here is an example of a channel initialization logic:
ConnectionProvider connectionProvider = ConnectionProvider.fixed("fixed", 50);
TcpClient.create(connectionProvider)
.host(host).port(port)
.wiretap(true)
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 50)
.doOnConnect(x -> log.trace("Connect to {}:{}", host, port))
.doOnConnected(conn -> log.trace("Connected {}", conn.channel()))
.connect()
.subscribe(this::utilizeConnection);
the output, that i receiving :
2019-09-04 08:23:13.612 TRACE 71988 --- [ioEventLoop-4-3] c.c.pcb.poc.network.tcp.NettyTcpSender : Connect to 192.168.88.210:2000
2019-09-04 08:23:13.684 WARN 71988 --- [actor-tcp-nio-4] io.netty.util.concurrent.DefaultPromise : An exception was thrown by reactor.netty.resources.PooledConnectionProvider$DisposableAcquire.operationComplete()
reactor.core.Exceptions$ErrorCallbackNotImplemented: io.netty.channel.ConnectTimeoutException: connection timed out: /192.168.88.210:2000
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /192.168.88.210:2000
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267) ~[netty-transport-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) ~[netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) ~[netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:405) [netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [netty-transport-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) [netty-common-4.1.36.Final.jar:4.1.36.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.36.Final.jar:4.1.36.Final]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Assembly trace from producer [reactor.core.publisher.MonoCreate] :
reactor.core.publisher.Mono.create(Mono.java:183)
reactor.netty.resources.PooledConnectionProvider.acquire(PooledConnectionProvider.java:130)
Error has been observed by the following operator(s):
|_ Mono.create ⇢ reactor.netty.resources.PooledConnectionProvider.acquire(PooledConnectionProvider.java:130)
|_ Mono.doOnSubscribe ⇢ reactor.netty.tcp.TcpClientDoOn.connect(TcpClientDoOn.java:58)
The 'inbound' and 'outbound' are having a dedicated method to handle their errors, but they works on top of a Connection instance that won't be created if you got the 'connection timeout'.
I tried:
The exception, that i receiving is wrapped in 'ErrorCallbackNotImplemented'. But I wasn't able to find any way to implement any 'ErrorCallback'
The log contains a warning message from 'io.netty.util.concurrent.DefaultPromise' . but I wasn't able to find a way how to make own Promise to handle it in a right way.
No any configurations i've found that may somehow intercept connection timeouts.
workaround. The blocked approach to create a connection ( .block() instead of .subscribe()) will allow me to catch any Connection creating exceptions within plain try-catch block, but i'll lose the benefits of reactive approach with such workaround.
Do somebody may suggest me at least something to help me to find a right way to handle a 'io.netty.channel.ConnectTimeoutException'?
Do not forget to implement your error callback
Usually reactor.core.Exceptions$ErrorCallbackNotImplemented happens when there is subscription over labmda based .subscribe method (same for Mono and Flux).
If you are going to look at the sources here and here, you will find the place where reactor.core.Exceptions$ErrorCallbackNotImplemented is thrown!
Action Points
In order to handle the original io.netty.channel.ConnectTimeoutException I would recommend looking at Handling Errors section of the original Project Reactor documentation
I have an odd issue with my Tomcat + Spring websocket application. When a user disconnects without sending a "closing" signal, due to power loss or a plug pull, the thread will block about 10 seconds later.
The thread blocks on this function :
org.springframework.web.socket.WebSocketSession.sendMessage(WebSocketMessage<?> wsm) throws IOException;
I have tried putting a line in my AppConfig to try and set a timeout of 3 seconds but it does not seem to work properly as the block seems to go on for upwards of 15 minutes before throwing an exception.
#Bean(name="servletServerContainerFactoryBean")
public int maxSessionIdleTimeout() {
return 3000;
}
Here is the stack trace after an eventual SocketTimeoutException
Step: 2304
SendB -> test isOpen -> sendMes -> Done -> Finished Send.
SendB -> test2 isOpen -> sendMes -> User closed connection during packet sending: s01
Propogating exception up!
java.io.IOException: java.net.SocketTimeoutException
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:315)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:250)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendPartialString(WsRemoteEndpointImplBase.java:223)
at org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendText(WsRemoteEndpointBasic.java:49)
at org.springframework.web.socket.adapter.standard.StandardWebSocketSession.sendTextMessage(StandardWebSocketSession.java:197)
at org.springframework.web.socket.adapter.AbstractWebSocketSession.sendMessage(AbstractWebSocketSession.java:102)
at org.infpls.noxio.game.module.game.session.NoxioSession.sendPacket(NoxioSession.java:40)
at org.infpls.noxio.game.module.game.dao.lobby.GameLobby.step(GameLobby.java:117)
at org.infpls.noxio.game.module.game.dao.lobby.GameLobby$GameLoop.run(GameLobby.java:274)
Caused by: java.net.SocketTimeoutException
at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.doWrite(WsRemoteEndpointImplServer.java:81)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:494)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:309)
... 8 more
## CRITICAL ## Ejecting player: test2 :: Exception thrown on packet send...
java.io.IOException: java.net.SocketTimeoutException
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:315)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:250)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendPartialString(WsRemoteEndpointImplBase.java:223)
at org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendText(WsRemoteEndpointBasic.java:49)
at org.springframework.web.socket.adapter.standard.StandardWebSocketSession.sendTextMessage(StandardWebSocketSession.java:197)
at org.springframework.web.socket.adapter.AbstractWebSocketSession.sendMessage(AbstractWebSocketSession.java:102)
at org.infpls.noxio.game.module.game.session.NoxioSession.sendPacket(NoxioSession.java:40)
at org.infpls.noxio.game.module.game.dao.lobby.GameLobby.step(GameLobby.java:117)
at org.infpls.noxio.game.module.game.dao.lobby.GameLobby$GameLoop.run(GameLobby.java:274)
Caused by: java.net.SocketTimeoutException
at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.doWrite(WsRemoteEndpointImplServer.java:81)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:494)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:309)
... 8 more
Finished Send. Step Finished.
Having threads be blocked for 15 minutes at a time is a major problem. Any info on why this happens and how to fix it would be greatly appreciated. Thank you!
Found an answer finally. It's actually a system setting.
Source
How many times to retry before deciding that something is wrong and
it is necessary to report this suspicion to network layer. Minimal RFC
value is 3, it is default, which corresponds to 3sec-8min depending on
RTO.
/proc/sys/net/ipv4/tcp_retries2
I'm having trouble in mule service. Daily, the service crashes, about 4 times a day, making it necessary to redeploy my service.
Here's my stack trace.
ERROR 2016-02-04 10:30:38,063 [[app_service].NoSessionConnector.receiver.03] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : Timeout for connection (java.net.SocketException). Message payload is of type: HttpResponse
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. Timeout for connection (java.net.SocketException)
java.net.SocketOutputStream:-2 (null)
2. Timeout for connection (java.net.SocketException). Message payload is of type: HttpResponse (org.mule.execution.ResponseDispatchExcepti
on)
org.mule.transport.http.HttpMessageProcessTemplate:141 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/execution/ResponseDispatc
hException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.net.SocketException: Tempo esgotado para conexão
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
I've tried changing the number of threads in maxThreadsActive and maxBufferSize, but is still occurring the same error
Seems like the tcp connection has not been usable for a long time.
Use the the Socket.setKeepAlive() or apply the heart beat mechanism.
I got strange behavior when my application tries to send an email.
20:59:08,926 ERROR [release.com.mycompany.mail.GenericMail] (EJB default - 5) [MY_EJB INBOUND] Sending message failed!: javax.mail.MessagingException: Can't send command to SMTP host;
nested exception is:
java.net.SocketException: Broken pipe
at com.sun.mail.smtp.SMTPTransport.sendCommand(SMTPTransport.java:2106) [mail-1.4.4-redhat-2.jar:1.4.4-redhat-2]
at com.sun.mail.smtp.SMTPTransport.sendCommand(SMTPTransport.java:2093) [mail-1.4.4-redhat-2.jar:1.4.4-redhat-2]
at com.sun.mail.smtp.SMTPTransport.close(SMTPTransport.java:1184) [mail-1.4.4-redhat-2.jar:1.4.4-redhat-2]
at javax.mail.Transport.send0(Transport.java:197) [mail-1.4.4-redhat-2.jar:1.4.4-redhat-2]
at javax.mail.Transport.send(Transport.java:124) [mail-1.4.4-redhat-2.jar:1.4.4-redhat-2]
at com.mycompany.MailUtils.sendMail(MailUtils.java:258) [classes:]
Before this exception some timeout exception has been thrown:
20:57:50,291 ERROR [org.jboss.as.ejb3] (EJB default - 9) [ ] JBAS014122: Error during
retrying timeout for timer: [id=6be904b5-c1ef-4f0e-a277-d4c9f93e21b3 timedObjectId=SOME_EJB
auto-timer?:false persistent?:false timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl#99fdab1 initialExpiration=/* date */ 00:00:00 UTC 2015 intervalDuration(in milli sec)=0 nextExpiration=/* other date */ 21:00:00 UTC 2015
timerState=RETRY_TIMEOUT: javax.ejb.EJBTransactionRolledbackException: JBAS014373:
EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation#66668127 - could not obtain lock within 5000MILLISECONDS
To be honest i don't know whats happened. I got these kind of exception for 5 hours.
I want to know whats happened and be able to avoid exceptions in future.
UPDATE 1
SOME_EJB is an ejb with works with timerService. Runs every 3 minutes, when the conditions are met sends an email.
My only idea is that there was some network/database issue and it caused that single execution of task took more than 3 minutes.
MailUtils is a #Stateless ejb
Is the mail sent or not?
It looks like JavaMail is closing the connection to the mail server when the exception occurs. If a previous error caused the server to drop the connection immediately, JavaMail may be getting this exception while trying to send the SMTP BYE command before closing the connection.
Turn on JavaMail Session debugging to see what error the server might have reported before this exception occurs.