I've created Java function that downloads files from FTP server. It works fine from my local machine. But I need to run it under linux server (means another host and port). And the function gives an error
The collection, array, map, iterator, or enumeration portion of a for statement cannot be null
Caused in a line with the code:
for(String f : ftpNames) {
ftpclient.retrieveFile(f, os); // os is OutputStream
}
So it doesn't see the files...
I added
ftpclient.enterRemotePassiveMode();
And ftpclient.getPassiveHost() returns 227 Entering Passive Mode (x,x,x,x,204,15)
Tried to list and download them via shell - it works.
How should I modify my code to solve the problem? Thanks.
UPD. I got log from FTP server I'm trying to get files from, and there is such string:
425 Cannot open data connection
Full code:
static boolean ftpFilesDownload(String ip, int port, String login, String passwd, String ftpdir, String localdir) throws IOException {
Boolean result = false;
FTPClient client = new FTPClient();
String separator = File.separator;
try {
client.connect(ip, port);
System.out.println(client.getReplyString());
client.login(login, passwd);
System.out.println(client.getReplyString());
client.setControlKeepAliveTimeout(1000*60*5);
client.setControlKeepAliveReplyTimeout(1000*60*5);
client.setFileType(FTP.BINARY_FILE_TYPE);
System.out.println("client setFileType success");
client.changeWorkingDirectory(ftpdir);
System.out.println(client.getReplyString());
client.printWorkingDirectory();
System.out.println("directory changed");
FTPFile[] ftpFiles = client.listFiles();
System.out.println(ftpFiles);
String[] ftpNames = client.listNames();
System.out.println("the files are " + Arrays.toString(ftpNames)); // so null here...
for(String f : ftpNames) {
String localfile = localdir + f;
OutputStream os = new FileOutputStream(localfile);
try {
result = client.retrieveFile(f, os);
System.out.println("DOWNLOADING STARTED);
System.out.println(client.getReplyString());
client.noop();
}
catch(Exception e) {
System.out.println(e);
result = false;
}
finally {
if(os != null)
os.close();
}
}
client.logout();
System.out.println(client.getReplyString());
}
catch(Exception e)
{
System.out.println(e);
result = false;
}
finally
{
try
{
client.disconnect();
}
catch(Exception e)
{
System.out.println(e);
result = false;
}
}
return result;
}
As the error message explains, you're trying to iterate over a null object. You should check for this (or make sure an empty Iterable is used perhaps)
If this is an execptional (error) state, I'd check for this explicitly and throw some kind of runtime exception, e.g.:
if (ftpNames == null) {
throw new IllegalArgumentException("Cannot use a null set of FTP servers");
}
for (String f : ftpNames) {
ftpclient.retrieveFile(f, os); // os is OutputStream
}
Alternatively you could try to continue with no FTP servers, but seems a bit pointless.
Try to use ftpclient.enterLocalActiveMode();
Related
Iam currently struggling a little bit with the FTPSClient from Apache Commons. See code down below. I try to write a file to FTP Server (vsftpd) when using FTPClient things are working perfectly fine. When using my code snippet I will always get a 451 Error, when debugging and waiting after Util.copyStream() returned everything works fine or settings a Thread.sleep(100). This also does not happen when I do not set the ftpsClient.execProt("P). Does anyone know by what this could be caused.
final FTPSClient client;
client = new FTPSClient("TLS", false);
client.setUseClientMode(true);
client.setDefaultPort(21);
// connect
try {
client.connect("serverAddress", 21);
} catch (SSLException e) {
throw e;
}
// setup any after connected
client.setSoTimeout(300);
client.setListHiddenFiles(true);
client.enterLocalPassiveMode();
FTPClientConfig ftpConfig;
try {
ftpConfig = new FTPClientConfig(client.getSystemType());
} catch (FTPConnectionClosedException e) {
throw e;
} catch (IOException e) {
ftpConfig = new FTPClientConfig();
}
client.configure(ftpConfig);
final FTPSClient ftpsClient = client;
// remove data buffer limit
ftpsClient.execPBSZ(0);
// set data channel encrypted
ftpsClient.execPROT("P");
client.login("user", "password");
if (!FTPReply.isPositiveCompletion(client.getReplyCode())) {
throw new IOException("Authentication failed: " + client.getReplyString().trim());
}
// postconfigure connection
if (!client.setFileTransferMode(FTP.STREAM_TRANSFER_MODE) || !client.setFileType(FTP.BINARY_FILE_TYPE)) {
throw new IOException("Failed to correctly configure client: " + client.getReplyString().trim());
}
InputStream input;
OutputStream output;
input = new FileInputStream(pathToLocalFile);
output = client.storeFileStream("foobar.txt");
final var number = Util.copyStream(input, output);
System.out.println(number);
input.close();
// Thread.sleep(100);
output.close();
// Must call completePendingCommand() to finish command.
if (!client.completePendingCommand()) {
client.logout();
client.disconnect();
System.err.println("File transfer failed.");
}
This library has been around for a long time, and things change a bit under the hood. Try:
input = new FileInputStream(pathToLocalFile);
boolean result = client.storeFile("foobar.txt", input);
if (result) {
System.out.println("\tFile Transfer Completed Successfully");
}
I have noticed that every once in a while when transferring files to a mainframe, it won't complete. I think it has something to do with the file length, but I've never been able to track it down. I also don't use the stream_transfer_mode.
I am trying to ftp the file from Linux VM to an AS400 server. I was able to login to server in passive mode but when trying to use the STOR command to upload the file getting below error:
STOR XX.YY600040.XXXZZZXXX
**550 Dataset not found, DSN=FTPID.XX.YY600040.XXXZZZXXX**
Not sure why the ftpid that i am using is getting prefixed to the filename. Is there any way to avoid it?
Below is the sample code that i am using:
private static String sendFTPFile(String fileName) throws Exception {
StringBuffer ftpMessage = new StringBuffer();
if (SHOW_DEBUG) ftpMessage.append("<ul>");
FTPClient ftp = null;
try {
String server = "****";
String username = "****";
String password = "XXXXX";
String hostDir = "";
String localFileName = fileName;
String localFilePath = "***/**/*";
boolean binaryTransfer = false, error = false;
FTPClient ftp = new FTPClient();
ftp.addProtocolCommandListener(new PrintCommandListener(
new PrintWriter(System.out)));
int reply;
ftp.connect(server)
// After connection attempt, you should check the reply code to verify
// success.
reply = ftp.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply))
{
ftp.disconnect();
error = true;
}
if(!error) {
if (!ftp.login(username, password))
{
ftp.logout();
error = true;
}
if(!error) {
if (binaryTransfer)
ftp.setFileType(FTP.BINARY_FILE_TYPE);
// Use passive mode as default
ftp.enterLocalPassiveMode();
InputStream input;
input = new FileInputStream(localFilePath+localFileName);
boolean ftpSuccess = ftp.storeFile(hostDir+localFileName, input);
input.close();
if (!ftpSuccess) {
throw new Exception("File ftp error");
}else {
if (SHOW_DEBUG) ftpMessage.append("<li>isFtpSuccess()...success").append("</li>");
}
ftp.logout();
if (SHOW_DEBUG) ftpMessage.append("<li>ftp.logout()...success").append("</li>");
}
}
}
catch (Exception ex) {
ex.printStackTrace();
System.out.println("Exception occur while transfering file using ftp"+ ex.toString());
throw new Exception(ftpMessage.toString(), ex);
}
finally {
if (ftp!=null && ftp.isConnected())
{
try
{
ftp.disconnect();
}
catch (Exception e)
{
e.printStackTrace();
System.out.println("Exception occur while transfering file using ftp"+ e.toString());
throw new Exception(ftpMessage.toString(), e);
}
}
if (SHOW_DEBUG) ftpMessage.append("</ul>");
}
return ftpMessage.toString();
}
I did some more debugging on doing ftp from linux to as400. And when doing pwd after logging to ftp server its giving message as:
ftp> pwd
257 "'FTPID.'" is current prefix
And thinking how to remove that prefix so i ran the below command got output as no prefix defined:
ftp> cd ..
200 "" no prefix defined
And after that when i uploaded the file using put command it was uploaded successfully. So i did some research on how to go back one directory using Apache commons.net api's that i am using and found the CDUP method.
When i ran the FTPClient.cdup() method before uploading the file. I was able to successfully FTP the file from java code as well.
ftp.cdup();
input = new FileInputStream(localFilePath+localFileName);
boolean ftpSuccess = ftp.storeFile(hostDir+localFileName, input);
This question already has answers here:
FtpClient storeFile always return False
(5 answers)
Closed 8 years ago.
I want to upload a file in a specific path in an ftp server the code is quite simple:
public static void main(String[] args) {
String server = "xx.xx.xx.xx";
String user = "xxx";
String pass = "xxx";
FTPClient ftpClient = new FTPClient();
try {
ftpClient.connect(server);
System.out.println("Connected to " + server + ".");
System.out.print(ftpClient.getReplyString());
ftpClient.login(user, pass);
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
// uploads first file using an InputStream
File firstLocalFile = new File("/tmp/PAR.TXT");
String firstRemoteFile = "/DATA/OUTFILES/PAR.TXT";
InputStream inputStream = new FileInputStream(firstLocalFile);
System.out.println("Start uploading first file");
boolean done = ftpClient.storeFile(firstRemoteFile, inputStream);
System.out.println("done:"+done);
inputStream.close();
if (done) {
System.out.println("The file is uploaded successfully.");
}
} catch (IOException ex) {
System.out.println("Error: " + ex.getMessage());
ex.printStackTrace();
} finally {
try {
if (ftpClient.isConnected()) {
ftpClient.logout();
ftpClient.disconnect();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
I always get done = false.
Here's the result:
Connected to xx.xx.xx.xx.
220 "Welcome (logging activated)"
Start uploading file
done:false
I printed the FtpClient#getReplyCode(). and i get this:
500 Illegal PORT command.
You can only access files relative to the root folder of the ftp server. You need to configure your ftp server to add a virtual folder pointing to the path you want.
I passed to PassiveMode and it works now
I'm creating a class to use in Coldfusion to replace the cfftp tag so I can do FTP over SSL. I have a custom tag that interacts with the class, using either FTPClient or FTPSClient. Connecting, logging in, putting a file, changing directories, listing files all work, but I cannot get a file for the life of me. I have tried both the retrieveFile() and retrieveFileStream methods and neither works. Below is my getFile method implemented with retrieveFileStream() and the inputstream is always null, no matter what. The file is there and the permissions are good. I just have no idea at this point. I can connect and get the file through WSFTP without a problem, so I think it's something in my implementation. Any help is appreciated!
public void getFile(String localFileName, String remoteFileName, String transferMode) {
try {
int transferFileType = 0;
existsFile(remoteFileName);
if (getReturnValue() != "YES" || replyCode == 550) {
throw new IOException("File " + remoteFileName + " does not exist");
}
else {
Boolean transferComplete = false;
File downloadFile = new File(localFileName);
OutputStream output = new BufferedOutputStream(new FileOutputStream(downloadFile));
InputStream input;
byte[] bytesArray = new byte[4096];
int bytesRead = -1;
if (!downloadFile.canWrite()) {
setSucceeded(false);
output.close();
throw new IOException("Cannot write to file " + localFileName);
}
if (!isConnected()) {
setSucceeded(false);
output.close();
throw new IOException("Connection closed by server.");
}
if (getSecure()) {
if (transferMode.toUpperCase() == "BINARY") {
ftps.setFileType(ftps.BINARY_FILE_TYPE);
}
else {
ftps.setFileType(ftps.ASCII_FILE_TYPE);
}
ftps.enterLocalPassiveMode();
ftps.setRemoteVerificationEnabled(false);
try {
input = ftps.retrieveFileStream(remoteFileName);
setReplyCode(true);
if (input == null || replyCode == 550) {
setSucceeded(false);
output.close();
throw new IOException("Cannot read file " + remoteFileName);
}
else {
while ((bytesRead = input.read(bytesArray)) != -1) {
output.write(bytesArray, 0, bytesRead);
output.flush();
}
input.close();
output.close();
transferComplete = ftps.completePendingCommand();
setReplyCode(true);
}
}
catch (IOException e) {
processError(e);
}
}
else {
if (transferMode.toUpperCase() == "BINARY") {
ftp.setFileType(ftps.BINARY_FILE_TYPE);
}
else {
ftp.setFileType(ftps.ASCII_FILE_TYPE);
}
ftp.enterLocalPassiveMode();
ftp.setRemoteVerificationEnabled(false);
try {
input = ftp.retrieveFileStream(remoteFileName);
setReplyCode(true);
if (input == null || replyCode == 550) {
setSucceeded(false);
output.close();
throw new IOException("Cannot read file " + remoteFileName);
}
else {
while ((bytesRead = input.read(bytesArray)) != -1) {
output.write(bytesArray, 0, bytesRead);
output.flush();
}
input.close();
output.close();
transferComplete = ftp.completePendingCommand();
setReplyCode(true);
}
}
catch (IOException e) {
processError(e);
}
}
//setReturnValue("Bytes Read: " + Integer.toString(bytesRead));
setSucceeded(transferComplete);
setReplyCode(true);
}
}
catch (IOException e) {
processError(e);
}
}
Here's my open connection method:
public void open (String server_in, int port_in, int timeout_in, String username_in, String password_in, Boolean implicit_in, Boolean secure_in) {
try {
FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_UNIX);
if (secure_in) {
setSecure(true);
ftps = new FTPSClient("SSL", implicit_in); // Create the client object
ftps.configure(conf); // Set the system type
String[] protocolVersions = {"SSLv3"};
ftps.setEnabledProtocols(protocolVersions); // Enable SSLv3 protocol
ftps.setAutodetectUTF8(true); // Enable auto detect
ftps.connect(server_in, port_in); // Connect
setReplyCode(true); // Get server response
ftps.setConnectTimeout(timeout_in);
if (!FTPReply.isPositiveCompletion(replyCode))
{
ftps.disconnect();
throw new Exception("FTP server refused connection.");
}
ftps.login(username_in, password_in);
setReplyCode(true); // Get server response
ftps.execPBSZ(0); // Set protection buffer to 0
ftps.execPROT("P"); // Private protocol
ftps.enterLocalPassiveMode();
}
else {
setSecure(false);
ftp = new FTPClient(); // Create the client object
ftp.configure(conf); // Set the system type
ftp.connect(server_in, port_in);
setReplyCode(true); // Get server response
ftp.setAutodetectUTF8(true); // Enable auto detect
ftp.setConnectTimeout(timeout_in);
if (!FTPReply.isPositiveCompletion(replyCode))
{
ftp.disconnect();
throw new Exception("FTP server refused connection.");
}
ftp.login(username_in, password_in);
setReplyCode(true); // Get server response
ftp.enterLocalPassiveMode();
}
setSucceeded(true);
}
catch (Exception e) {
processError(e);
}
}
Ok, figured it out. If you look at my getFile() method above you'll see that I first check to make sure the file exists with existsFile(remoteFileName). Well, the method I was using to see if a file exists was to to open an InputStream with retrieveFileStream() to it within a try block and throw an error if it failed. I never called completePendingCommand() after the retrieveFileStream() in existsFile(), so trying to open another stream to the file always failed. Phew!
I'm using the following code to upload an xml file to the directory /home/domainname/public_html/guest in the server. However, the file is uploaded only to the location /home/domainname. It is not uploading to the child directories. Please advise.
FTPClient client = new FTPClient();
FileInputStream fis = null;
try {
client.connect(Util.getProductsXMLFTPServer());
client.login(Util.getProductsXMLFTPUser(), Util.getProductsXMLFTPPassword());
//
// Create an InputStream of the file to be uploaded
//
fis = new FileInputStream(new File(Util.getProductsXMLFTPInputFilePath(), Util.getProductsXMLFTPOutputFileName()));
client.changeWorkingDirectory(Util.getProductsXMLFTPUploadPath());
client.storeFile(Util.getProductsXMLFTPOutputFileName(), fis);
client.logout();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fis != null) {
fis.close();
}
client.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
I checked your code, it works. I've only changed file type declaration to binary, which may be not needed for XML files.
Here's my complete code for reference:
package apachenet.ftp;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
public class App {
public static void main( String[] args ) {
FTPClient client = new FTPClient();
FileInputStream fis = null;
try {
client.connect(/*Util.getProductsXMLFTPServer()*/"127.0.0.1");
client.login(/*Util.getProductsXMLFTPUser()*/"pwyrwinski",
/*Util.getProductsXMLFTPPassword()*/"secret");
client.setFileType(FTP.BINARY_FILE_TYPE); // optional
fis = new FileInputStream(
new File(/* Util.getProductsXMLFTPInputFilePath() */"/home/pwyrwinski",
/* Util.getProductsXMLFTPOutputFileName() */"img.png"));
client.changeWorkingDirectory(/*Util.getProductsXMLFTPUploadPath()*/ "someDir");
client.storeFile(/*Util.getProductsXMLFTPOutputFileName()*/"img_bis.png", fis);
client.logout();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fis != null) {
fis.close();
}
client.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
As you can see it's roughly the same as yours.
Util class calls are replaced with raw data.
When I ran it, file /home/pwyrwinski/img.png was uploaded to {FTP_USER_ROOT}/someDir directory on ftp server with name changed to img_bis.png. I assume this is exactly what you wanted to achieve.
Let's go back to your problem.
Try to check what is returned from
Util.getProductsXMLFTPUploadPath() call. My guess is it's not what
you're expecting - so debug it in your IDE or print it to the console.
Check if path returned from Util.getProductsXMLFTPUploadPath()
call starts with slash, it shouldn't.
UPDATE 1.
Does direcory /home/domainname/public_html/guest exist on server?
Add following method to your class:
private static void showServerReply(FTPClient ftpClient) {
String[] replies = ftpClient.getReplyStrings();
if (replies != null && replies.length > 0) {
for (String aReply : replies) {
System.out.println("SERVER: " + aReply);
}
}
}
and call it after every ftp-client's method call. This will give you codes and descriptions of every command result. I suspect client.changeWorkingDirectory(...) ends with error, probably: 550 Permission Denied (or No such file or folder).
Next modification will be:
client.login(Util.getProductsXMLFTPUser(), Util.getProductsXMLFTPPassword());
System.out.println(client.printWorkingDirectory()); // added this line!
this will tell us what is current working directory after login in.
Please post your results.
FTPClient ftpClient = new FTPClient();
try {
System.out.println("before server connection");
ftpClient.connect(server, port);
System.out.println("before user name and passwod");
ftpClient.login(user, pass);
ftpClient.enterLocalActiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
System.out.println("connection sucess");
// windows working fine
File secondLocalFile = new File("/home/aims/archived_reports/tes_S_000000123/test.pdf");
// String secondRemoteFile = "/archived_reports/PermanentRecord.pdf";
//linux
// File secondLocalFile = new File("/archived_reports/tes_S_000009123/test.pdf");
String secondRemoteFile = "remotefilename.pdf";
InputStream inputStream = new FileInputStream(secondLocalFile);
System.out.println("Start uploading second file");
ftpClient.changeWorkingDirectory("/reports");// home/ftp.test/reports folder
System.out.println("Prasent Working Directory :"+ftpClient.printWorkingDirectory());
OutputStream outputStream = ftpClient.storeFileStream(secondRemoteFile);
int returnCode = ftpClient.getReplyCode();
System.out.println(returnCode);
byte[] bytesIn = new byte[4096];
int read = 1;
while ((read = inputStream.read(bytesIn)) != -1) {
outputStream.write(bytesIn, 0, read);
}
System.out.println();
inputStream.close();
outputStream.close();
boolean completed = ftpClient.completePendingCommand();
if (completed) {
System.out.println("The second file is uploaded successfully.");
}