I have a C# Winforms application that iteratively FTPs files from an FTP server, parses the file for information, then returns the information from each file in a loop. After so many FTP pulls into memory (I'm loading the text files into an array) Mcafee sees my pulls as a port scanning virus and disables to connection.
I thought delaying the thread (Thread.Sleep(int)) might trick my virus scanner from getting this error but the tradeoff is efficiency. Does anyone know the specs on how fast I can run and not get this port scan error? I'm not going outside the company firewall (both my laptop and the FTP server and within the firewall).
The reason for the warning is that for each transfer of a file a new connection between server and client is opened and in general the port is increased by one each time this happens. From the outside this can look like a port-scan to so called personal firewalls leading to this effect.
There are a couple of possible solutions:
It seems that you use the so called active mode for the data transfer, i.e. the FTP server is asked to open the data connection to your system. Switch to passive mode where the connections are established by your client, so the sympthom of an incoming port-scan shouldn't exist anymore keeping your personal firewall quiet.
Whitelist your application or the peers server in your personal firewall preventing it from blocking things.
Change the setting of your FTP-client (a java library you use in your program or why is there a Java-tag with this question?) to use the same port for the data transfer with active mode. Because it's always the same port, this should keep your personal firewall quiet as well.
Related
I'm using the Apache-Commons-Net library.
I want to know if the FTP server is active-mode or passive-mode.
Is there a method like that in the Apache-Commons-Net library?
(GetDataConnectionMode() is a method that tells you the connected mode after connecting in Java. I want to check Maud before)
private FTPClient ftp;
// the code I want
if(active){
ftp.enterLocalActiveMode();
}else{
ftp.enterLocalPassiveMode();
}
// Apply active mode or passive mode to the environment of the server for oneself
As #Erwin already commented, servers are not in active or passive mode. Servers usually support both modes at the same time.
A mode that you need to use, typically depends on a configuration of the network between you and the server, not on some "switch" on the server. There's hardly any way to detect that.
All you can do is to try one mode and fallback to the other if the first mode fails. In general, you should start with the passive mode, as that tends to work more often.
Read also my article on FTP connection modes and the network configuration needed for them, to understand, what this is about.
I have a application which listens for commands over IP.
The program works fine locally but when I try to send the application commands with a remote address it won't connect.
Is there anyway to get around the router blocking the inbound network traffic?
I'm using JAVA
Thanks.
If this is a pair of hosts you control, you can open the incoming port. You might also
succeed using a tunneling program such as Hamachi to effectively set up a VPN linking
the hosts.
If you're talking about a pair of unrelated computers (for example trying to set up
a connection for a game) there's no general solution that doesn't involve installing
and trusting additional software. The usual solution is to use a public server and
relay the traffic between the end points.
I'm writing a Java application like AIM where I want a lookup server to help two clients connect to each other with the ServerSocket class. Unfortunately a serversocket needs an open port or it will be blocked by your firewall. Without having the clients manually change their firewall settings is there a way I can either find an open port, open a port (it's okay if it asks for permission) I just don't want every client to manually have to change firewall settings. Thanks!
Creating a ServerSocket is what opens the port on the server end (assuming the port is NOT also blocked on the firewall). On many systems, port numbers <1024 are not blocked by the firewall, and therefore often don't require custom configuration. However, since many ISPs have been more and more stringent with what ports they block within their network, many pieces of software have moved to operating over port 80. Why port 80? It's the port used for HTTP, and therefore pretty much open (at firewalls) 100% of the time for any service that operates on the web.
To answer the question of how to find an open port, port scanners perform this function. Basically all they do is try to establish a connection on a range of ports (say port 1-1024). When a connection is successful (in Java you would see this as a good connection over a Socket), you can consider that port "open". You don't need to exchange any data in order to make the connection, you just need to establish the connection, and then close it.
Also, if there is no server software of any kind running on the server, it won't appear "open", even if it isn't blocked. Without a successful Socket (TCP) connection, you won't know what ports are not blocked by user's firewall.
To reiterate, I'm switching back and forth between two related, but separate concepts. Firewalls can block/open ports, or have port forwarding. That's not the same as a port being "open" for connections on a given machine. In order for a machine to accept connections on a given port, there must be a piece of software listening for a connection on that port, otherwise no connection can be made.
I would suggest to look at a couple of alternative solutions that are less cumbersome - scanning for open ports can take quite some time, can cause panicky reactions from firewalls that feel attacked, and so on. There are a couple of techniques that are in active use and have been developed exactly to solve the problem of servers behind firewalls.
UDP Hole Punching, zero user side configuration needed. Simple explanation on how Skype uses this technique can be found here
UPnP / IGD could be used as an alternative, though less devices support it out of the box nowadays due to security problems.
STUN with a Java implementation of client and serverside called JSTUN
Whatever solution you choose, test it thoroughly with different internet service providers, there's a plethora of limitations you can expect wrt blocked ports.
I am using Jersey 1.4, the ApacheHttpClient, and the Apache MultiThreadedHttpConnectionManager class to manage connections. For the HttpConnectionManager, I set staleCheckingEnabled to true, maxConnectionsPerHost to 1000 and maxTotalConnections to 1000. Everything else is default. We are running in Tomcat and making connections out to multiple external hosts using the Jersey client.
I have noticed that after after a short period of time I will begin to see sockets in a CLOSE_WAIT state that are associated with the Tomcat process. Some monitoring with tcpdump shows that the external hosts appear to be closing the connection after some time but it's not getting closed on our end. Usually there is some data in the socket read queue, often 24 bytes. The connections are using https and the data seems to be encrypted so I'm not sure what it is.
I have checked to be sure that the ClientRequest objects that get created are closed. The sockets in CLOSE_WAIT do seem to get recycled and we're not running out of any resources, at least at this time. I'm not sure what's happening on the external servers.
My question is, is this normal and should I be concerned?
Thanks,
John
This is likely to be a device such as the firewall or the remote server timing out the TCP session. You can analyze packet captures of HTTPS using Wireshark as described on their SSL page:
http://wiki.wireshark.org/SSL
The staleCheckingEnabled flag only issues the check when you go to actually use the connection so you aren't using network resources (TCP sessions) when they aren't needed.
EDIT: Learned that Webmethods actually uses NLST, not LIST, if that matters
Our business uses the WebMethods integration server to handle most of our outbound communications, and its FTP functionality leaves something to be desired. We are having a problem that may be specific to WebMethods, but if anyone can point me in a direction of what kinds of things might cause this I'd appreciate it.
When polling two of our partners' FTP servers, we connect without issue but, when doing a NLST on a directory that is empty (no files and no subdirectories) it's timing out. The actual error is:
com.wm.net.ftpCException: [ISC.0064.9010] java.net.SocketTimeoutException: Accept timed out
It's being thrown during the invocation of the pub.client.ftp:ls service. I've logged in with a number of FTP clients without a problem to the same sites. I've used whatever the default FTP client is in windows, FileZilla and lftp. All without issue. The servers themselves aren't the same FTP server software from what I can tell. One is Microsoft FTP, the other I'm uncertain on but is definitely not Microsoft.
Any idea what could cause an FTP client to timeout when waiting for a NLST response on an empty directory? The visible responses from the FTP server appear to be the same, but is there a difference in how NLST responds for an empty directory that I'm unaware of?
This problem is consistent on these two servers. Everything functions fine on directories with files or subdirectories within it, but not when empty.
Any thoughts or directions would be appreciated.
Thanks!
Eric Sipple
I tried this in WebMethods IS Version 6.5 Updates WmPRT_6-5-1_SP1, IS_6-5_SP3.
It worked perfectly first time.
I turned on debugging on the FTP server (Debian's default ftpd). WebMethods' NLST honours the active/passive parameter passed to it.
There's nothing special about the NLST command, nor its correct behaviour with an empty directory -- if LIST works, then so should RETR, STOR and NLST. If NLST works with a non-empty directory, it should work with an empty one.
So my guess is that either:
Your version of WM has a bug mine doesn't
Your FTP server has a bug mine doesn't
There's a wacky protocol-aware firewall in your system that doesn't like FTP data sockets with no data in them.
Firewall vendors are a bit wayward when it comes to FTP...
When testing with other clients, make sure it's from the same machine on which WebMethods Integration Server is running.
Just for the record, here's what should happen for an active NLST
client opens a listening socket, and sends a PORT command with that socket's details
client sends NLST command
server connects to client's listening socket (this is the data socket)
server transmits listing over data socket (in this case, zero bytes)
server closes data socket
... and in passive mode
client sends PASV command
server opens a listening socket, and replies with PASV response containing its details
client connects to listening socket (this is the data socket)
client sends NLST command
server transmits listing over data socket (zero bytes again)
server closes data socket
I am not sure if it was the same problem but I had similar symptoms a while ago using another FTP client in Java (commons.net). The problem turned out to be caused by the active/passive mode of the connection.
I am sorry I can't give you more details, that's all I can remember... hope that help.
Guillermo Vasconcelos was correct in his answer. There are two FTP modes, Active and Passive. The default FTP mode is active. Active requires the server to connect back to the client on some TCP/IP port. This does not work with firewalls because chances are that this port would be blocked or if you are behind a Router with NAT, not mapped.
If you use Passive (PASV) mode instead, you should not get the hang.
I'm going to run some new tests with the settings to passive tomorrow when maintenance is done here, but I'm not sure that's the issue. We are able to get a directory listing if there are files or subdirectories in that directory. It only fails when the directory we're NLST-ing on is empty.
Would the active/passive difference only manifest for an empty directory, or is there another possibility?
FTP requires that both the specified port and the one above it be opened through the firewall. When I had problems with webMethods timing out, it was because the firewall did not have the return port open.
Howard