I have an apache proxy between browser and my custom web server. So after starting the webserver, I can access my web application for like 4-5 hours. After that I get this error
Proxy Error The proxy server received an invalid response from an
upstream server. The proxy server could not handle the request GET /.
Reason: Error reading from remote server
To access the web application again, I have to restart the web server. I tried to check the log I got from my web server and only error in that log , I am finding is this
com.mysql.jdbc.CommunicationsException: Communications link failure
due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException MESSAGE: Connection timed out
STACKTRACE:
java.net.SocketException: Connection timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:113)
at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:160)
at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:188)
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1910)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2304)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2803)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1573)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3170)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3099)
at com.mysql.jdbc.Statement.execute(Statement.java:695)
at hra.database.Pool.getConnection(Pool.java:62)
** END NESTED EXCEPTION **
Last packet sent to the server was 925624 ms ago.
And I tried to fix this database error, but I still get the above mentioned proxy error, so definitely the log information is not useful.
This is what I am getting from Apache Proxy log
(70007)The timeout specified has expired: proxy: error reading status
line from remote server
Does anyone know what can be the issue or any helpful pointers to fix this issue?
apache proxy fails because customwebserver does not answer. your
customwebserver logs a crash waiting for mysql to answer. rebooting
customwebserver fixes it, so mysql is not the problem. it looks like the problem the way your customwebserver handles its connections to mysql.
can we see your jdbc config? do you have autoReconnect=true ?
What is likely happening is that your up-stream server is shutting itself down because of inactivity. Find the setting on the server that controls that and turn it up.
I can see that you see two connection timeout errors
Apache Proxy Web server timing out while connecting to custom web server
Custom web server timing out while connecting to database
If the timeout set on proxy server is less than timeout set for database connection, before the database connection times out, proxy server would time out as well.
As a first thing resolve all DB connectivity issues and test your application by directly connecting to custom web server (not via proxy).
Observe and identify normal response time and max response time.
Set proxy time out ProxyTimeout to be a value that is more than (max response time) + (network delay)
If possible you may share proxy server configurations to help you further.
I think like stated by the previous answer that you encounter a timeout raised by one of the members of your infrastructure:
server
network device (routers are clever beasts but firewalls can be clever too)
Did you setup a real connection pool ? Using heart beat mechanism to keep alive connections ?
This induces some network traffic but avoids such problems ?
Try to use a network capture tool to have many logs to analyze
HTH
jerome
Related
In my batch application when I am sending requests across a network using a Web Service and Java, after running about 30000 requests and receiving the responses, the program throws a java.net.connectexception connection timed out exception.
I am using WildFly, along with some Java code in the middle to configure the requests (Strings) before sending it across the network.
After research the possible reasons I found for this is that there is either a Firewall blocking my access, which can't be true since it ran 90% of the requests already.
I've also seen somewhere that says that I could have overloaded the server, although I'm not sure what that means exactly.
You have filled up the server's listen backlog queue. This is caused by creating new connections faster than the server can accept them. You should look into connection pooling at the client, and handling multiple requests per connection at the server.
Disclaimer: there is a lot of information on similar topics. In our case it works as expected without AWS ELB (Elastic Load Balancer), i.e. when the client drops, ServletOutputStream.flush() throws IOException.
Setup: we have an instance running Tomcat 7 (Java 1.7) behind ELB (HTTPS:443 -> HTTP:8080). The servlet streams data to the client through HTTP long lived connection.
Problem: when the client disconnects, the server keeps streaming data, i.e. ServletOutputStream.flush() or .write() does not throw IOException. The ELB kind of "buffers" the connection (we can see it with IpTraf monitor), so from the Tomcat side it appears as the client is still there. Without the ELB, IOException is thrown properly, so the servlet can stop streaming. We have disabled connection draining and reduced connection timeout to 1 sec, we also reduced all timeouts on Tomcat's HTTP Connector including KeepAlive to just few seconds. Nothing helps.
Question: is there anything we can do with the ELB configuration / Tomcat / Java side to allow disconnection detection in this setup?
We had the same kind of problem (but in .NET with IIS).
We seem to have solved it by switching from the classic ELB to the Application ELB. Now writing to the output stream of a closed connection gives an exception, where first (on classic ELB) it didn't.
Hope this helps anyone running into the same problem
I am facing a problem with CXF framework trying to connect to a https backend service. Since the service is out side of my network I am using a proxy to connect. I am getting a SocketTimedOutException when I set the readTimeout Value to like 60000 which is the default. When I set the timeout to 0(infinite) it gives a connectionResetException after a while. I wrote a program on my own which used HttpsUrlConnection to connect to the same service with proxy and I am able to work with it.
I was initially under the feeling that the connection itself was not happening but later I came to know that that would have thrown a ConnectException rather than a socketException. It gave me some relief , but I want to know how I can deal with the SocketException, both the timeout and the Connection Reset.
Please explain to me whoever knows about this.
Thanks,
Sachin
You should blame on your network or your proxy, but not your codes. SocketTimedOutException and ConnectionTimedOutException are thrown when network is blocked or weak. So, the solution to both exceptions is smoothening your network.
I'm new to webMethods, invoking a weblogic Web Services from webMethods (client), I got this error:
com.wm.app.b2b.server.ServiceException: org.apache.axis2.AxisFault: Connection timed out
at pub.clientimpl.wssClient(clientimpl.java:2945)
at pub.clientimpl.soapClient(clientimpl.java:1299)
at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.wm.app.b2b.server.JavaService.baseInvoke(JavaService.java:439)
at com.wm.app.b2b.server.invoke.InvokeManager.process(InvokeManager.java:643)
at com.wm.app.b2b.server.util.tspace.ReservationProcessor.process(ReservationProcessor.java:46)
at com.wm.app.b2b.server.invoke.StatisticsProcessor.process(StatisticsProcessor.java:44)
at com.wm.app.b2b.server.invoke.ServiceCompletionImpl.process(ServiceCompletionImpl.java:243)
at com.wm.app.b2b.server.invoke.ValidateProcessor.process(ValidateProcessor.java:51)
at com.wm.app.b2b.server.invoke.PipelineProcessor.process(PipelineProcessor.java:171)
at com.wm.app.b2b.server.ACLManager.process(ACLManager.java:276)
at com.wm.app.b2b.server.invoke.DispatchProcessor.process(DispatchProcessor.java:30)
at com.wm.app.b2b.server.AuditLogManager.process(AuditLogManager.java:363)
at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:547)
at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:386)
at com.wm.app.b2b.server.ServiceManager.invoke(ServiceManager.java:234)
at com.wm.app.b2b.server.comm.DefaultServerRequestHandler.handleMessage(DefaultServerRequestHandler.java:119)
at com.wm.app.b2b.server.HTTPMessageHandler.process(HTTPMessageHandler.java:160)
at com.wm.app.b2b.server.HTTPDispatch.handleRequest(HTTPDispatch.java:181)
at com.wm.app.b2b.server.Dispatch.run(Dispatch.java:354)
at com.wm.util.pool.PooledThread.run(PooledThread.java:131)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.axis2.AxisFault: Connection timed out
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
Any idea? Thank you in advance for your help
Possibilities:-
- Connectivity is not there
- Provider is not up
- both are there but processing time by server is more than defined timeout
you are not providing enough information about what you are trying to do but you can look into 2 main things:
1- check the service ACL profile. Maybe this service can not be invoked through an anonymous call.
2- check the port. maybe the port is blocked (assuming that all service cannot be invoked in this Integration Server [IS])
best of luck,
M
There could be a number of reasons for this.
Can the webMethods server reach the weblogic server at all? Try to ping the weblogic server from the webMethods server. Is there a firewall blocking? Are you using the correct port number?
Do you have access to the weblogic server so you can see if the request actually reaches it? It could be that it takes a very long time to execute and then webMethods would time out before it is finished. If it is supposed to take a long time then increasing the timeout can work. Another solution is to implement it as an asynchronous request.
As I understand you have created a consuming webservice in webmethods. I would try creating an independent SOAP UI project based on the WSDL provided to you for the development of the IS hosted consumer. This will allow you to prove your external connectivity and to the provide webservice. You can work back and confirm your consuming service is configured appropriately.
I have a RED5 application, which works under Red5 0.8.0 (although it drops the connection on RTMPT after couple seconds). I've checked out the latest revision (which is 4441) and made some changes to the app. Now I cannot connect to the application. I get everytime - NetConnection.Connect.Rejected.
In the logs I see:
[INFO] [http-0.0.0.0-80-exec-3] org.red5.server.net.rtmp.codec.RTMPProtocolDecoder - Action connect
[WARN] [Red5_Scheduler_Worker-3] org.red5.server.net.rtmp.RTMPConnection - Closing RTMPMinaConnection from 0:0:0:0:0:0:0:1%0 : 52966 to localhost (in: 3458 out: 3262), with id 1 due to long handshake
The handshake time is set to 15 seconds, so it must be enough. In the logs I am not even getting any attempts to try to connect. I have a log with the method name in start, connect, disconnect methods, however these won't be shown.
Can anyone help?
due to long handshake can also mean you have still some browser window open that tries to connect to the application after you did re-start red5.
You always have to close all browser windows when you restart red5.
Also the first thing you need to look at is the red5/error.log.
Nothing else counts :)
I guess you have some startup issue, Red5 0.8 are not 100% compatible to latest versions.
Sebastian
A possible case of connection rejection from the server is user authentication failure.If you implement that in the connect method of your application check that the user gets authenticated. RTMP connection closure just tells you that the server pinged the client after every X ms and after certain number of such trials server got no response from the client(which is called as handshake timeout).In that case server closes the connection with the 'idle' client.