This question already has answers here:
FtpClient storeFile always return False
(5 answers)
Closed 1 year ago.
I have a Java program that uploads new/changed files to my Web site via FTP. It currently uses the Apache Commons Net Library, version 3.8.0.
After moving to a new city, the program, which I’ve been using for almost 20 years, began failing. It still connects to the FTP server and signs in successfully. But when it tries to upload a file, it pauses for 20-30 seconds, then fails. It always fails on the first file, 100% of the time.
The failing call is to org.apache.commons.net.ftp.FTPClient.storeFile(). The documentation says storeFile() turns True if successfully completed, false if not. Curiously, the method is also documented to throw various forms of IOException. The documentation doesn’t say when or why the method decides to return a boolean versus throwing an exception.
My problem is that storeFile() is returning a false (indicating failure), and never throws an exception. So, I have no error message to tell me what caused the failure. File name & path look OK. The Web hosting company tried to determine why the failure was occurring, but was unsuccessful.
This problem has been going on for weeks now. Anyone have any ideas on how to debug this?
If the cause of your problem is moving to a new city, and you can still open the control connection, the most likely culprit is a change to your underlying ISP and network that is blocking the data transfer stream from opening.
FTP port 21 is used for opening connections and is normally allowed by all networks but then a new, random, unprivileged port is negotiated over the control connection and then used for the actual DATA transfers. I bet your "storeFile()" is trying to open a data connection and hitting a block which is probably causing a timeout. You may be interpretting this as "never throws an exception" but in reality it might throw a Timeout Exception after you sit around and wait long enough.
One thing I would recommend is find a way to have your FTP client use PASSIVE mode for the FTP data transfer. This is designed into the protocol to avoid these types of problems. You can read about it in detail on the wiki https://en.wikipedia.org/wiki/File_Transfer_Protocol under "Communications and Data Transfer"
I try to create a spooledfile with the library jt400.
I have done the program which list the spool in a outqueue without problem but when i want to create the SpooledFile object, my as400 return me an error
com.ibm.as400.access.ServerStartupException: Not able to pass
connection to server job. Server job timed out.
I don't understand why. Is there something to do on os400 about the security.
Thanks to help me.
It would be helpful to see the stack trace. JT400 may require PRINT host service for spooled file operations. On IBM i, check if QNPSERVD job is present (wrkjob QNPSERVD) if not use STRHOSTSVR *netprt to start. Also check that the print server port (default is 8474 for plain connection and 9474 for encrypted) is accessible from your Java server.
I have a very unusual error condition that I can't seem to find the solution to. I'm hoping someone out here in StackOverflowland can help. Before you just look at the title and say "your CLASSPATH is wrong," read on!
I work with a server-based Java application that utilizes a number of third-party libraries. The particular library that I'm getting the error with is a file transfer library. Part of the functionality of the application allows the user to connect to remote servers (FTP, SFTP and FTPS) and send/receive files.
One of our customers is using the application and occasionally gets a "Class Not Found Exception" (or more specifically, "No Class Definition Found") when attempting to connect to a remote host. What's weird is that they don't get it all the time. They have sent me logs that show a successful connection and then another one which shows this error. There was no restart of the server between the two and both connections are to the same host using the same connection settings.
What would cause a "Class Not Found Exception" to occur occasionally within a Java application on a class that has already been successfully used? I've contacted the third-party vendor and they are just as puzzled.
I wasn't able to determine anything from the stack trace, but since it was requested, here it is:
java.lang.NoClassDefFoundError: com.enterprisedt.net.j2ssh.configuration.ConfigurationLoader (initialization failure)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:139)
at com.enterprisedt.net.j2ssh.SshThread.a(SshThread.java:93)
at com.enterprisedt.net.j2ssh.SshThread.<init>(SshThread.java:73)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.startTransportProtocol(TransportProtocolCommon.java:515)
at com.enterprisedt.net.j2ssh.SshClient.connect(SshClient.java:593)
at com.enterprisedt.net.ftp.ssh.SCPClient.connectSSH(SCPClient.java:1137)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.connect(SSHFTPClient.java:920)
at com.enterprisedt.net.ftp.async.internal.ConnectTask.connect(ConnectTask.java:154)
at com.enterprisedt.net.ftp.async.internal.ConnectTask.run(ConnectTask.java:216)
at com.enterprisedt.net.ftp.async.internal.FTPTaskProcessor$b.run(FTPTaskProcessor.java:590)
This is my first question on Stackoverflow after being developing in Java for a few years, generally I find other Q&A's very helpfull, but for this problem I haven't found a solution yet, though I found a similar question here :
In my case, I'm trying to connect to a Windows NT server with LS file listing that runs a WRQ Reflection ftp server (which they say does not run in sftp mode, but accepts regular ftp-connections).
But it fails: UNKNOWN XFB/Gateway
Caused by: org.apache.commons.net.ftp.parser.ParserInitializationException: Unknown parser type: UNKNOWN XFB/Gateway
at org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory.createFileEntryParser(DefaultFTPFileEntryParserFactory.java:132)
at org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:2263)
at org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2046)
at org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2093)
at org.apache.commons.vfs.provider.ftp.FTPClientWrapper.listFilesInDirectory(FTPClientWrapper.java:137)
at org.apache.commons.vfs.provider.ftp.FTPClientWrapper.listFiles(FTPClientWrapper.java:113)
at org.apache.commons.vfs.provider.ftp.FtpFileObject.doGetChildren(FtpFileObject.java:138)
at org.apache.commons.vfs.provider.ftp.FtpFileObject.getChildFile(FtpFileObject.java:111)
at org.apache.commons.vfs.provider.ftp.FtpFileObject.getInfo(FtpFileObject.java:196)
at org.apache.commons.vfs.provider.ftp.FtpFileObject.doGetType(FtpFileObject.java:319)
at org.apache.commons.vfs.provider.AbstractFileObject.getType(AbstractFileObject.java:441)
... 66 more
I am not sure wether which one of the available parsers would work (The UNIX one because of the LS file listing?) and if neither would suffice, where to begin to implement my own parser.
The problem is that I can't test the connection myself. Our Java-application runs in a client's domain that tries to connect to a 3rd party that only accepts incoming connections from the clients domain and at the moment they are not really helpfull (at least not in allowing me to connect to their ftp-server from outside the client's domain).
And just committing a change + building a new release (& let our client test it) based on a hunch is something I rather not do just now.
Our code always worked fine with other ftp-servers our client used our program with to export files, but this is a new connection they set up.
So in short:
1a. Which of the standard apache-commons EntryParsers is likely to be able to work with the 3rd party's ftp-server
1b. If neither would suffice, what information do I need to implement our own EntryParser (looks rather complex for a FTP-n00b like me)?
Can anyone give me a pointer on how to set up a suitable free test-ftp server locally which uses the XFB Gateway protocol? My company doesn't like me installing evaluation-editions of potential expensive enterprise solutions, like WRQ Reflection
We have some applications that sometimes get into a bad state, but only in production (of course!). While taking a heap dump can help to gather state information, it's often easier to use a remote debugger. Setting this up is easy -- one need only add this to his command line:
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=PORT
There seems to be no available security mechanism, so turning on debugging in production would effectively allow arbitrary code execution (via hotswap).
We have a mix of 1.4.2 and 1.5 Sun JVMs running on Solaris 9 and Linux (Redhat Enterprise 4). How can we enable secure debugging? Any other ways to achieve our goal of production server inspection?
Update: For JDK 1.5+ JVMs, one can specify an interface and port to which the debugger should bind. So, KarlP's suggestion of binding to loopback and just using a SSH tunnel to a local developer box should work given SSH is set up properly on the servers.
However, it seems that JDK1.4x does not allow an interface to be specified for the debug port. So, we can either block access to the debug port somewhere in the network or do some system-specific blocking in the OS itself (IPChains as Jared suggested, etc.)?
Update #2: This is a hack that will let us limit our risk, even on 1.4.2 JVMs:
Command line params:
-Xdebug
-Xrunjdwp:
transport=dt_socket,
server=y,
suspend=n,
address=9001,
onthrow=com.whatever.TurnOnDebuggerException,
launch=nothing
Java Code to turn on debugger:
try {
throw new TurnOnDebuggerException();
} catch (TurnOnDebugger td) {
//Nothing
}
TurnOnDebuggerException can be any exception guaranteed not to be thrown anywhere else.
I tested this on a Windows box to prove that (1) the debugger port does not receive connections initially, and (2) throwing the TurnOnDebugger exception as shown above causes the debugger to come alive. The launch parameter was required (at least on JDK1.4.2), but a garbage value was handled gracefully by the JVM.
We're planning on making a small servlet that, behind appropriate security, can allow us to turn on the debugger. Of course, one can't turn it off afterward, and the debugger still listens promiscuously once its on. But, these are limitations we're willing to accept as debugging of a production system will always result in a restart afterward.
Update #3: I ended up writing three classes: (1) TurnOnDebuggerException, a plain 'ol Java exception, (2) DebuggerPoller, a background thread the checks for the existence of a specified file on the filesystem, and (3) DebuggerMainWrapper, a class that kicks off the polling thread and then reflectively calls the main method of another specified class.
This is how its used:
Replace your "main" class with DebuggerMainWrapper in your start-up scripts
Add two system (-D) params, one specifying the real main class, and the other specifying a file on the filesystem.
Configure the debugger on the command line with the onthrow=com.whatever.TurnOnDebuggerException part added
Add a jar with the three classes mentioned above to the classpath.
Now, when you start up your JVM everything is the same except that a background poller thread is started. Presuming that the file (ours is called TurnOnDebugger) doesn't initially exist, the poller checks for it every N seconds. When the poller first notices it, it throws and immediately catches the TurnOnDebuggerException. Then, the agent is kicked off.
You can't turn it back off, and the machine is not terribly secure when its on. On the upside, I don't think the debugger allows for multiple simultaneous connections, so maintaining a debugging connection is your best defense. We chose the file notification method because it allowed us to piggyback off of our existing Unix authen/author by specifying the trigger file in a directory where only the proper uses have rights. You could easily build a little war file that achieved the same purpose via a socket connection. Of course, since we can't turn off the debugger, we'll only use it to gather data before killing off a sick application. If anyone wants this code, please let me know. However, it will only take you a few minutes to throw it together yourself.
If you use SSH you can allow tunneling and tunnel a port to your local host. No development required, all done using sshd, ssh and/or putty.
The debug socket on your java server can be set up on the local interface 127.0.0.1.
You're absolutely right: the Java Debugging API is inherently insecure. You can, however, limit it to UNIX domain sockets, and write a proxy with SSL/SSH to let you have authenticated and encrypted external connections that are then proxied into the UNIX domain socket. That at least reduces your exposure to someone who can get a process into the server, or someone who can crack your SSL.
Export information/services into JMX and then use RMI+SSL to access it remotely. Your situation is what JMX is designed for (the M stands for Management).
Good question.
I'm not aware of any built-in ability to encrypt connections to the debugging port.
There may be a much better/easier solution, but I would do the following:
Put the production machine behind a firewall that blocks access to the debugging port(s).
Run a proxy process on the host itself that connects to the port, and encrypts the input and output from the socket.
Run a proxy client on the debugging workstation that also encrypts/decrypts the input. Have this connect to the server proxy. Communication between them would be encrypted.
Connect your debugger to the proxy client.