This question already has answers here:
FtpClient storeFile always return False
(5 answers)
Closed 1 year ago.
I have a Java program that uploads new/changed files to my Web site via FTP. It currently uses the Apache Commons Net Library, version 3.8.0.
After moving to a new city, the program, which I’ve been using for almost 20 years, began failing. It still connects to the FTP server and signs in successfully. But when it tries to upload a file, it pauses for 20-30 seconds, then fails. It always fails on the first file, 100% of the time.
The failing call is to org.apache.commons.net.ftp.FTPClient.storeFile(). The documentation says storeFile() turns True if successfully completed, false if not. Curiously, the method is also documented to throw various forms of IOException. The documentation doesn’t say when or why the method decides to return a boolean versus throwing an exception.
My problem is that storeFile() is returning a false (indicating failure), and never throws an exception. So, I have no error message to tell me what caused the failure. File name & path look OK. The Web hosting company tried to determine why the failure was occurring, but was unsuccessful.
This problem has been going on for weeks now. Anyone have any ideas on how to debug this?
If the cause of your problem is moving to a new city, and you can still open the control connection, the most likely culprit is a change to your underlying ISP and network that is blocking the data transfer stream from opening.
FTP port 21 is used for opening connections and is normally allowed by all networks but then a new, random, unprivileged port is negotiated over the control connection and then used for the actual DATA transfers. I bet your "storeFile()" is trying to open a data connection and hitting a block which is probably causing a timeout. You may be interpretting this as "never throws an exception" but in reality it might throw a Timeout Exception after you sit around and wait long enough.
One thing I would recommend is find a way to have your FTP client use PASSIVE mode for the FTP data transfer. This is designed into the protocol to avoid these types of problems. You can read about it in detail on the wiki https://en.wikipedia.org/wiki/File_Transfer_Protocol under "Communications and Data Transfer"
Related
I'm creating a small utility which receives a lot of HTTP requests. It is written in java and uses embedded-jetty to handle requests via https.
I have a load-testing tool for it, but when it is being run for some time it starts to throw exceptions:
java.net.BindException: Address already in use: connect
(note, this is on sender's side, not in my project)
As I understand this means no more free sockets were found in system when another connect was called. Throughput is about 1000 requests per second, and failures start to appear somewhere after 20000 to 50000 requests.
However when I use the same load testing tool with another program (a kind of simple consumer, written in scala using netty by some colleague - it simply receives all requests and returns empty ok response) - there is no so problem with sockets (though typical speed is 1.5-2 times slower).
I wonder if this could be fixed by telling Jetty somehow to close connections immediately after response was sent. Anyway each new request is sent via new connection. I tried to play with Connector#setIdleTimeout - it seems to be 30000 by default but have not succeeded.
What can I do to fix this - or at least to research the matter deeper to find its cause (if I am wrong in my suggestions)?
UPD Thanks for suggestions, I think I am not allowed to post the source, but I get the idea that I should study client's code (this will make me busy for some time since it is written in scala).
I found that really there was a problem with client - it sends requests with Connection: Keep-Alive in header, though creates new HttpURLConnection for each request and calls disconnect() method after it.
To solve this trouble on the server-side it was sufficient to send Connection: close in response header, since I have no allowance to change testing utility.
I know this sort of question has been asked for a few times, I read some of them but did not get any smarter.
My Java application is connecting to a database server via JDBC through a SSH Tunnel. The tunnel is opened once at beginning. When starting I opened the database connection everytime it is used. Due to changes in the app I needed the connection opened on startup and decided to keep it open until my application is closed. When I close the app I sometimes, not always, get following error:
- Could not retrieve transation read-only status server
java.sql.SQLException: Could not retrieve transation read-only status server
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 369.901 milliseconds ago. The last packet sent successfully to the server was 8 milliseconds ago.
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
Could this be because of the always open database connection? I test the application only for short times. It will run on 4 computers all day long. Can I expect this error more often then? The connection is used every few minutes, so it should be more performant to keep it open, but maybe during break it is not used for half an hour.
What would you recommend me to do? Always reopen the connection or keep it like it is and find a workaround when I get this error? Do you maybe have another idea why this error appears?
Just ask if you need more error log, database code or whatever you need.
Thanks!
I'm going to say yes, your issue is probably due to the fact that you have a persistent connection open.
I took over a website a while ago and the guy before me had the same idea: Open a single connection, send the queries through it when needed, and never close it. A month after I took over the site, the database wouldn't return any more query results just like this.
As a general rule of thumb and for good programming practice, always clean up after yourself. If you're not using a variable, set it to null and delete it. Not using a connection? Terminate it. This is way less prominent in Java than it is in C++ as Java does all the cleaning up for you for the most part.
we've been having this problem for a long time and still cannot find out where is the problem. Our application uses RTMP for videostreaming and if the webclient cannot connect it skips to RTMPT (RTMP over HTTP). This causes the video to freeze after couple seconds of playback.
I have already found some forums where people seems to be havoing the same issue, but none of the proposed solutions worked. One suggestion was to turn of the video recording, but it didn't work. I have also read, that it seems to be a thread problem in the red5, but before hacking into the RED5 I would like to know, if maybe somebody has a patch or anything which repairs this.
One thing more, we've been testing this on Macs if that should count. Thank you very much in advance.
the very first thing you should look at is really the red5/error log.
Also Red5 occassionally produces output that might be not in the log but just to plain std.out
There is a red5-debug.sh or red5-highpref.sh that does output/log everything to a file called std.out.
You should use those logs to start your analysis. Eventually you will already see something into it. For example exception like:
broken pipe
connection closed due to too long xxx
handshake error
encoding issue in packet xyz
unexpected connection closed
call xyz cannot be handled
too many connections
heap space error
too many open files
Some of them are operating system specific, like for example the number of open files. Some are not.
Also it is very important that you are using the latest revision of Red5 and not an old version. You did not tell us what version you are using.
However, just from symptoms like video freezes *occassional disconnects* or similar you won't be able to start a real analysis of the problem.
Sebastian
Were you connected to the server when the video freezed? Or after that? I am not sure but I think connection closed which caused the stream to freeze.Just check in the access logs of Red5 if there are any logs for 'idle' packets(possibly after a 'send' packet(s) and more than one in number).
Another thing you could have a look at is your web server log files because RTMPT is over HTTP. I once had a problem with my anti DDOS program on the server. RTMPT will make many connections after each other and these TCP connections remain alive for about 4 minutes by default. You can easily get hundreds connections at the same time being seen as a DDOS-attack and as a result the IP-adres of the client will be banned.
My java application does use DB Connection pooling. One of the functionality started failing today with this error:
[BEA][SQLServer JDBC Driver]No more data available to read
This doesn't occur daily. Once I restart my application server things look fine for some days and this error comes back again.
Anyone encountered this error? Reasons might vary, but I would like to know those various reasons to mitigate my issue.
Is it possible that the database or network connection has briefly had an outage? You might expect any currently open result sets then to become invalid with resulting errors.
I've never seen this particular error, but then I don't work with BEA or SQL Server, but a quick google does show other folks suggesting such a cause.
When you're using a connection pool, if you do get such a glitch, then all connections in teh pool become "stale" or invalid. My application server (WebSphere) has the option to discard the entire connection pool after particular errors are detected. The result then is that one unlucky request sees the error, but then subsequent requests get a new connection and recover. If you don't discard the whole pool then you get a failure as each stale connection is used and discarded.
I suggest you investigate to see a). whether your app server has such a capability b). how you application responds if the database is bounced, if this replicates the error then maybe you've found the cause.
I've been caught catching SocketExceptions belonging to subspecies like for example Broken pipe or Connection reset. The question is what to do with the slippery bastards once they're caught.
Which ones may I happily ignore and which need further attention? I'm looking for a list of different SocketExceptions and their causes.
In terms of Java web development, a Broken pipe or a Connection reset basically means that the other side has closed the connection. This can under each be caused by the client pressing Esc while the request is still running or navigating away by link/bookmark/addressbar while the request is still running. You see this particular error often in long running requests such as large file downloads and unnecessarily large/slow business tasks (which is not good for the impatient user, about 3 secs is really the max). In rare cases it can also be caused by a hardware/network problem, such as a network outage at either server or client side.
This exception can be thrown when a flush() or close() on the outputstream of the response is invoked. You as server side cannot do anything against it. You cannot recover from it as you cannot (re)connect the client due to security restrictions in HTTP. In most cases you also shouldn't even try to, because this is often client's own decision. Just ignore it or log it for pure statistics.
One of the other causes is usually the TCP/IP stack settings on the Operating System. Haven't tried it on Linux yet but one platform i've worked on is Sun's Solaris 9/10 Operating System. The basic idea is that Solaris has a tunable TCP/IP stack which you can tune while running your web applications.
So there are two parameters that you should be aware of
tcp_conn_req_max_q0 - queue of incomplete handshakes
tcp_conn_req_max_q1 - queue of complete handshakes
tcp_keepalive_interval - keepalive
tcp_time_wait_interval - time of a TCP segment that's considered alive
in the internet
All the above parameters affect how much load can the system take (from a TCP/IP perspective) and on the flipside affects the occurrence of certain types of SocketExceptions - such as the ones BalusC pointed above.
This is obviously quite convoluted but the point i'm trying to make is that the OS you're hosting your apps on more often than not, offers you mitigation strategies.