We Keep Coding

Java Apache FTPClient most downloaded files are empty or missing

This is my code that is supposed to download entire FTP directory to a local folder. It does it well, but most of the files are 0KB in size. Only JSON files seem to contain all their data.
Things I tried:
Changing FTP file type with client.setFileType("FTP.BINARY_FILE_TYPE");
Using OutputStream instead of FileOutputStream
Code:
public static void copyFolder(File destination, FTPFile sourceFile, FTPClient ftpClient) throws IOException{
if (!sourceFile.isDirectory()) {
//copy file
File downloadFile = new File(destination + "/"+ sourceFile.getName());
String remoteFile = sourceFile.getName();
FileOutputStream outputStream = new FileOutputStream(downloadFile);
System.out.println(remoteFile);
System.out.println(downloadFile.getPath());
boolean success = ftpClient.retrieveFile(remoteFile, outputStream);
if(success) {
System.out.println("Retrieved " + remoteFile);
}
outputStream.close();
}else{
//loop through a subdirectory
ftpClient.changeWorkingDirectory(ftpClient.printWorkingDirectory() + "/" + sourceFile.getName());
System.out.println(ftpClient.printWorkingDirectory());
FTPFile[] contents = ftpClient.listFiles(ftpClient.printWorkingDirectory());
File newDest = new File(destination + "/" + sourceFile.getName());
if(!newDest.exists()){
newDest.mkdir();
}
for(FTPFile file : contents){
copyFolder(newDest, file, ftpClient);
}
return;
}
}
How to get the transfer correctly?
log from ftp
tree of the ftp directory
tree of downloading directory
Trying to download it on the same computer ended with losing connection a few times - between and during file downloads. Also it seems that few files are downloaded. I will change the title of question to be more specific.
Only two files are being copied for some reason – https://pastebin.com/XNWqRMDj They are not empty.

The problem is your changeWorkingDirectory call. It's failing most of the time.
ftpClient.changeWorkingDirectory(ftpClient.printWorkingDirectory() + "/" + sourceFile.getName());
It should be:
ftpClient.changeWorkingDirectory(destination + "/" + sourceFile.getName());
For a complete working code for downloading FTP folders in Java, see:
Download all folders recursively from FTP server in Java

getOutputStream() has already been called Spring MVC

Our requirement is to download files from FTP server which we used a download files in sequential manner one after another which is taking huge time.
Sequential code:
for(String mediaValue: generatedMediaTypes) {
// Logic here is getting the user selected files from Ui and keep it in string array str
try {
ddownloadMultipleNavDbs(str[i]),uniqueID);
//now we download all the files here with downloadfile logic with help of unique id we are a folder by that name and keep all this files in that folder
} catch (Exception ex) {
logger.error(ex.getMessage());
}
}
After downloading all files into uniqueID folder now I zip and send to client by downloadZip(request,uniqueID):
public void downloadZipFile (HttpServletResponse response, String suuid) throws IOException
{
try
{
ZipOutputStream zipOutputStream = new ZipOutputStream(response.getOutputStream());
response.setContentType("application/octet-stream");
File loadableFolder = new File(uniqueID);
String timeStamp = new java.sql.Timestamp(Calendar.getInstance().getTime().getTime()).toString().replaceAll(
"-", "_").replaceAll(" ", "_").replaceAll(":", "_");
response.setHeader(
"Content-Disposition",
"attachment; filename=" + "File" + timeStamp.substring(
0, timeStamp.indexOf(".")) + ".ZIP");
File[] loadableMedias = //here we get the files from that uinqueID folder
ZipUtil.addFilesToZipStream(Medias, Folder,
zipOutputStream);
zipOutputStream.close();
response.flushBuffer();
response.getOutputStream().flush();
response.getOutputStream().close();
}
It works perfectly fine in above code.
When I used executorservice for downloading files in parallel
executorService.submit(new myThread(str[i]),uniqueID);
am facing error:
getOutputStream() has already been closed
Can anyone please explain me why am facing this error?
Why we face this error and how to resolve it?

Apache-commons-fileupload: how to read and display the data in a temporary file uploaded into server directory by spring mvc

I am working on a service project, who's UI allows its users to upload a file. I need to write a service that can upload this file to server and read and display the contents of this file. Can anyone tell me how to do this ?
//Controller definition begins
#RequestMapping(value = "/uploadFile", method = RequestMethod.POST)
public #ResponseBody
String uploadFileHandler(#RequestParam("name") String name,
#RequestParam("file") MultipartFile file) {
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload();
if (!file.isEmpty()) {
try {
byte[] bytes = file.getBytes();
// Creating the directory to store file
String rootPath = System.getProperty("catalina.home");
File dir = new File(rootPath + File.separator + "tmpFiles");
if (!dir.exists())
dir.mkdirs();
// Create the file on server
File serverFile = new File(dir.getAbsolutePath()
+ File.separator + name);
BufferedOutputStream stream = new BufferedOutputStream(
new FileOutputStream(serverFile));
stream.write(bytes);
stream.close();
logger.info("Server File Location="
+ serverFile.getAbsolutePath());
return "You successfully uploaded file=" + name;
} catch (Exception e) {
return "You failed to upload " + name + " => " + e.getMessage();
}
} else {
return "You failed to upload " + name
+ " because the file was empty.";
}
}
Now i want to know how i can display the contents of the uploaded file. It is being converted to FILE format and is stored in the tempfiles directory of the tomcat server and the data is non-human readable.I need to convert this back to xlsx(file being uploaded is xlsx) or be able to read the data from it directly to update the Db.Also i am using apache commons-io and file upload in Spring MVC as you can see from above code.

You need a library capable of reading xlsx file types, such as Apache POI
In case you choose this library, there is a very good example section in the example secion of it's website

FileUploader - Save data in the project

I am uploading a file with the PF 3.5 File Uploader
My Upload Method looks like that:
public void handleFileUpload(FileUploadEvent event) {
log.info("Method handleFileUpload invoked");
FacesMessage msg = new FacesMessage("Succesful", event.getFile().getFileName() + " is uploaded.");
FacesContext.getCurrentInstance().addMessage(null, msg);
InputStream inputStream = null;
OutputStream out = null;
try {
File targetFolder = new File("\\resources\\uploads");
if(!targetFolder.exists()) {
targetFolder.mkdirs();
}
inputStream = event.getFile().getInputstream();
File outFile = new File(targetFolder, event.getFile().getFileName());
log.info("copy file stream to " + outFile.getAbsolutePath());
out = new FileOutputStream(outFile);
int read = 0;
byte[] bytes = new byte[size];
log.info("read file stream");
while ((read = inputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
} catch (IOException e) {
log.error(e);
} finally {
...
}
at the moment my files get uploaded to \\resources\\uploads". Thats the path to a folder on theC:`.
However, I want to upload my uploads to a path in my eclipse project. How to change the path? I really appreciate your answer!!!

However, I want to upload my uploads to a path in my eclipse project.
That's absolutely not recommended for the reasons mentioned in this answer: Uploaded image only available after refreshing the page. The point is: the IDE's workspace and server's deploy folder is absolutely not intented as a permanent file storage. The uploaded files would be unreachable and/or disappear like by magic.
Just keep them in a path external to the IDE's workspace and server's deploy folder. You're doing it fine. I'd only make the path configurable by a system property, environment variable or properties file setting so that you don't need to edit, recompile, rebuild, etc the code everytime when you change the upload location.
If your concrete problem is more the serving of the uploaded file, then just add the upload folder as another context in server's configuration, or create a simple servlet for the serving job, or as you're using PrimeFaces, just use <p:fileDownload> or <p:graphicImage> with StreamedContent pointing to the desired FileInputStream.
See also:
How to save uploaded file in JSF

Property file path Eclipse vs Deployed Java Web App

I'm maintaining properties files containing database credentials in protected folders on our internal server for each app that I deploy. I'm not allowed to store the credentials within the WAR file.
When testing on my PC the path is a windows mount, but when I depoloy to the server it is a unix path
I have been handling the retreival as such
//siteDbCedentialFolder obtained from web.xml
Properties prop = new Properties();
InputStream in = null;
try {
//assume running on server first
in = new FileInputStream("/abc/data/" + siteDbCedentialFolder + "/props.txt");
} catch (java.io.IOException ex) {
// Probabaly Running locally
in = new FileInputStream("W:/internal/abc/data/" + siteDbCedentialFolder + "/props.txt");
} catch (Exception xx) {
xx.printStackTrace();
}
prop.load(in);
in.close();
is my approach to use a catch to get the local path OK or is there a better way to code this?

Better way is to put the txt file in user home directory and get it by
System.getProperty("user.home");
and to get the full path
String pathToFile = System.getProperty("user.home") + File.separator + "data" + File.separator + "props.txt";
It would work across all the platform, provided that you need to put the file at proper place (user home)