Java Apache FTPClient most downloaded files are empty or missing - java

This is my code that is supposed to download entire FTP directory to a local folder. It does it well, but most of the files are 0KB in size. Only JSON files seem to contain all their data.
Things I tried:
Changing FTP file type with client.setFileType("FTP.BINARY_FILE_TYPE");
Using OutputStream instead of FileOutputStream
Code:
public static void copyFolder(File destination, FTPFile sourceFile, FTPClient ftpClient) throws IOException{
if (!sourceFile.isDirectory()) {
//copy file
File downloadFile = new File(destination + "/"+ sourceFile.getName());
String remoteFile = sourceFile.getName();
FileOutputStream outputStream = new FileOutputStream(downloadFile);
System.out.println(remoteFile);
System.out.println(downloadFile.getPath());
boolean success = ftpClient.retrieveFile(remoteFile, outputStream);
if(success) {
System.out.println("Retrieved " + remoteFile);
}
outputStream.close();
}else{
//loop through a subdirectory
ftpClient.changeWorkingDirectory(ftpClient.printWorkingDirectory() + "/" + sourceFile.getName());
System.out.println(ftpClient.printWorkingDirectory());
FTPFile[] contents = ftpClient.listFiles(ftpClient.printWorkingDirectory());
File newDest = new File(destination + "/" + sourceFile.getName());
if(!newDest.exists()){
newDest.mkdir();
}
for(FTPFile file : contents){
copyFolder(newDest, file, ftpClient);
}
return;
}
}
How to get the transfer correctly?
log from ftp
tree of the ftp directory
tree of downloading directory
Trying to download it on the same computer ended with losing connection a few times - between and during file downloads. Also it seems that few files are downloaded. I will change the title of question to be more specific.
Only two files are being copied for some reason – https://pastebin.com/XNWqRMDj They are not empty.

The problem is your changeWorkingDirectory call. It's failing most of the time.
ftpClient.changeWorkingDirectory(ftpClient.printWorkingDirectory() + "/" + sourceFile.getName());
It should be:
ftpClient.changeWorkingDirectory(destination + "/" + sourceFile.getName());
For a complete working code for downloading FTP folders in Java, see:
Download all folders recursively from FTP server in Java

Related

How to differentiate between an empty file that has no extension and a directory in java?

Let's suppose I have a zip file containing two elements: elem1 (created by linux command touch elem1) and elem2 (created by linux command mkdir elem2)
Now, in java, I use the following code to extract the content of the zip
// ...
// Suppose we have a valid inputStream on a zip file
// ...
ZipInputStream zipInputStream = new ZipInputStream(inputStream);
ZipEntry entry = zipInputStream.getNextEntry();
while (entry != null) {
int entrySize = (int) entry.getSize();
File file = Paths.get(extractPath).resolve(entry.getName()).toFile();
if (/*Condition to detect a directory*/) {
System.out.println("This is a directory");
FileUtils.forceMkdir(file);
} else if (/*Condition to detect an empty file*/) {
System.out.println("This is an empty file");
} else {
System.out.println("This is something else");
}
entry = zipInputStream.getNextEntry();
}
I would like to specify the right conditions to detect whether entry is a directory, or an empty file without extension. Knowing that these entries are still in memory and do not exist on the filesystem, the command file.isDirectory() always returns false; so I cannot not use it to check for directory.
Any ideas please ?
I created both an empty folder and an empty file without extension and evaluated them with the code below:
public static void main(String[] args) {
String path = System.getProperty("user.home") + File.separator + "Desktop" + File.separator;
File file = new File(path + "EmptyFile");
File folder = new File (path + "EmptyFolder");
System.out.println("Is 'folder' a folder? " + (Files.isDirectory(folder.toPath())? "Yes" : "No" ));
System.out.println("Is 'file' a folder? " + (Files.isDirectory(file.toPath())? "Yes" : "No" ));
}
The (not surprising) result:
Is 'folder' a folder? Yes
Is 'file' a folder? No
The reason why this works is because the function Files.isDirectory(...) looks in the file attributes set by the Operating System to determine whether the item being examined is a "File folder" or simply a "file". My assumption is that Zip programs do not contain such metadata (not even Windows zip). Therefore, "isDirectory" test cannot be performed using the Files.isDirectory(...) function. My quick research discovered that, the way to do this (and I am kind of shocked) is by examining the file name and check to see if the name ends with the file separator. In fact, this is how ZipEntry.isDirectory() works.
Attempting to zip an empty folder is not allowed for Windows zip (maybe allowed with other software?). However, I was able to include empty directories with 7-zip. That wasn't the only difference. The isDirectory() test failed when the zip was created with Windows zip because the file was skipped altogether. So, in order for this to work, create the zip file with zip software other than the one that comes with Windows. Then,
public static void main(String[] args) throws IOException {
String path = System.getProperty("user.home") + File.separator + "Desktop" + File.separator;
FileInputStream inputStream = new FileInputStream(path + "Desktop.zip");
ZipInputStream zipInputStream = new ZipInputStream(inputStream);
ZipEntry entry = zipInputStream.getNextEntry();
while (entry != null) {
File file = Paths.get(entry.getName()).toFile();
if (entry.isDirectory()) {
System.out.println(entry.getName() + " is a directory");
} else if (file.getName().endsWith(".lnk")) {
System.out.println(file.getName() + " is a shortcut");
} else {
System.out.println(entry.getName() + " is a file");
}
entry = zipInputStream.getNextEntry();
}
zipInputStream.close();
}
Outputs for me:
EmptyFile is a file
EmptyFolder/ is a directory
How We Test Wireless Routers _ PCMag_files/ is a directory
How We Test Wireless Routers _ PCMag_files/00hSyhn9j5PNrcOot1tMzz9.1578945749.fit_lim.size_100x100.png is a file
...
etc
One last note... obviously, if a ZipEntry is not a directory, it is a file. Therefore, no else if is needed. That is, unless you would like to make a distinction between file types. In the example above, I wanted to check if a particular file was a shortcut. Just keep in mind that this is not necessary. The logic should only test entries for isDirectory and if the test fails, it is simply a file.

Android : Copying a file to the internal storage. File not found

I'm trying to backup my database file on the internal storage :
File fromFile = new File(context.getDatabasePath("database.db").getPath());
FileChannel fromFileChannel = new FileInputStream(fromFile).getChannel();
File toFile = new File(context.getFilesDir() + "/database.db");
if (toFile.getParentFile() != null)
toFile.getParentFile().mkdirs();
FileChannel toFileChannel = new FileOutputStream(toFile).getChannel();
Log.i("LOG",fromFileChannel.transferTo(0, fromFileChannel.size(), toFileChannel)+"");
fromFileChannel.close();
toFileChannel.close();
The log returns "69632" without any error, so it looks like it worked.
The problem is that I can't find the file. And the /Android/data folder doesn't contain a folder with my app package name. What am I doing wrong?
Use
File toFile = new File(Environment.getExternalStorageDirectory()
+ File.separator + "Android/data" + File.separator + context.getPackageName() + File.separator + "/database.db");
context.getFilesDir() will give you the directory private to your app, so the files in that will not be available to other apps.

Apache-commons-fileupload: how to read and display the data in a temporary file uploaded into server directory by spring mvc

I am working on a service project, who's UI allows its users to upload a file. I need to write a service that can upload this file to server and read and display the contents of this file. Can anyone tell me how to do this ?
//Controller definition begins
#RequestMapping(value = "/uploadFile", method = RequestMethod.POST)
public #ResponseBody
String uploadFileHandler(#RequestParam("name") String name,
#RequestParam("file") MultipartFile file) {
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload();
if (!file.isEmpty()) {
try {
byte[] bytes = file.getBytes();
// Creating the directory to store file
String rootPath = System.getProperty("catalina.home");
File dir = new File(rootPath + File.separator + "tmpFiles");
if (!dir.exists())
dir.mkdirs();
// Create the file on server
File serverFile = new File(dir.getAbsolutePath()
+ File.separator + name);
BufferedOutputStream stream = new BufferedOutputStream(
new FileOutputStream(serverFile));
stream.write(bytes);
stream.close();
logger.info("Server File Location="
+ serverFile.getAbsolutePath());
return "You successfully uploaded file=" + name;
} catch (Exception e) {
return "You failed to upload " + name + " => " + e.getMessage();
}
} else {
return "You failed to upload " + name
+ " because the file was empty.";
}
}
Now i want to know how i can display the contents of the uploaded file. It is being converted to FILE format and is stored in the tempfiles directory of the tomcat server and the data is non-human readable.I need to convert this back to xlsx(file being uploaded is xlsx) or be able to read the data from it directly to update the Db.Also i am using apache commons-io and file upload in Spring MVC as you can see from above code.
You need a library capable of reading xlsx file types, such as Apache POI
In case you choose this library, there is a very good example section in the example secion of it's website

uploaded file not saved to the webcontent directory

I am developing and application using eclipse IDE. My application has a file upload functionality.
I am able to achieve how to upload the file and also to save it. But the problem is that the file uploaded didn't get store to my dynamic web project directory.
The file uploaded get store to my server directory with .metadata folder having path
file:///E:/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/
I want to store my uploaded folder to my Webcontent folder having upload folder having images folder like WebContent/upload/images.
No doubt I am able to view the image file but, the path i want is like above only.
below code I am using to store the uploaded file
#RequestMapping(value = "/company/UploadFile.action", method = RequestMethod.POST)
public #ResponseBody String uploadFile(FileUploadBean uploadItem, BindingResult result,HttpServletRequest request, HttpServletResponse response) {
System.out.println("FILE UPLOAD ITEM SI SSLSL ::"+uploadItem);
ExtJSFormResult extjsFormResult = new ExtJSFormResult();
if (result.hasErrors()){
for(ObjectError error : result.getAllErrors()){
System.err.println("Error: " + error.getCode() + " - " + error.getDefaultMessage());
}
//set extjs return - error
extjsFormResult.setSuccess(false);
return extjsFormResult.toString();
}
// Some type of file processing...
System.err.println("-------------------------------------------");
System.err.println("Test upload: " + uploadItem.getFile().getOriginalFilename());
System.err.println("-------------------------------------------");
try{
MultipartFile file = uploadItem.getFile();
String fileName = null;
InputStream inputStream = null;
OutputStream outputStream = null;
if (file.getSize() > 0) {
inputStream = file.getInputStream();
/*if (file.getSize() > 10000) {
System.out.println("File Size:::" + file.getSize());
extjsFormResult.setSuccess(false);
return extjsFormResult.toString();
}*/
System.out.println("also path ::"+request.getRealPath("") + "/upload/images/");
System.out.println("PATHI SIS SIS"+this.getClass().getProtectionDomain().getCodeSource().getLocation().getPath());
System.out.println("size::" + file.getSize());
InetAddress addr = InetAddress.getLocalHost();
byte[] ipAddr = addr.getAddress();
System.out.println("HOST NAME"+request.getRealPath("ResourceMgt"));
System.out.println("HOST ADDR"+addr.getHostAddress());
System.out.println("HOST "+request.getRequestURI());
System.out.println("HOST "+request.getRequestURL());
fileName = request.getRealPath("") + "/upload/images/"
+ file.getOriginalFilename();
outputStream = new FileOutputStream(fileName);
System.out.println("FILEN ANEM AND PATH IS ::"+fileName);
System.out.println("fileName:" + file.getOriginalFilename());
int readBytes = 0;
byte[] buffer = new byte[40000];
while ((readBytes = inputStream.read(buffer, 0, 40000)) != -1) {
outputStream.write(buffer, 0, readBytes);
}
companyservice.saveImages(file.getOriginalFilename(),fileName);
outputStream.close();
inputStream.close();
}
}catch (Exception e) {
// TODO: handle exception
e.printStackTrace();
}
//set extjs return - sucsess
extjsFormResult.setSuccess(true);
return extjsFormResult.toString();
}
please suggest me how can I store the file uploaded to my WebContent having upload folder with images folder. My above code is working perfectly Just there is some issue with specifying the path.
Have you tried to change the destination of the outputStream?
fileName = request.getRealPath("") + "/upload/images/"
+ file.getOriginalFilename();
Instead of request.getRealPath("") put an absolute destination or play with ClassPath. For example:
fileName = "/opt/tomcat/webapps/upload/images/"
+ file.getOriginalFilename();
forum member
now I am able to upload the file successfully, but the file get stored to the deployed directory on the server.
As soon as I remove the project and redeployed the project to my Tomcat server 6.0 all the files I had uploaded gets deleted from that.
I am using JAVA as my server side technology with Tomcat server 6.0.
I am able to upload the file successfully, but the file get stored to the deployed directory on the server.
As soon as I remove the project and redeployed the project to my Tomcat server 7.0 all the files I had uploaded gets deleted from that.
I am using JAVA and JSF as my server side technology with Tomcat server 7.0 in Eclipse IDE

FTP client - listfiles

I am unable to get the exact file list using FTPClient. Sample code as below :
FTPClient client = new FTPClient();
client.connect("x.x.x.x");
client.login("abcd", "abcd");
FTPFile[] ftpFiles = client.listFiles();
for (FTPFile ftpFile : ftpFiles) {
System.out.println("FTPFile: " + ftpFile.getName());
}
I tried to set to PASV mode using enterLocalPassiveMode()/enterRemotePassiveMode()/pasv(). But, it doesnt work.
Please also check Apache Commons FTPClient.listFiles ..
Thank you
I don't know what files is, but you're getting the results of client.listFiles in ftpFiles, and not in files. Then in your for loop you go over files.
Try this.
String[] fileFtp = client.listNames();//if it is directory. then list of file names
//download file
for (int i =0;i<fileFtp.length;i++) {
String fileName = fileFtp[i];
OutputStream out = new FileOutputStream(new File("local temp file name"));
if (!client.retrieveFile(fileName, out)) {
sysout("Could not download the file. "+ fileName);
} else {
sysout("Downloaded file # : "+localFileName);
}
}
This should work.
Thanks.

Categories