We Keep Coding

Java application crashed due to lots of files being copied

I have an application which copies a number of files from a directory to a certain destination. The problem is that when you select a large folder it gets more intense for the app itself and then crashes. Is there any way to make it not crash? Maybe split it up into smaller parts?
This is my code:
public void startProcess(File orgDir, File destDir) {
Screen1Controller sf = new Screen1Controller();
String selectedExtension = sf.selectedExtension; // Gets selected extension from Screen1
String extensionType = sf.typeOfExtension; // Gets selected extension type from Screen1
int y = 1; // This is for searching for duplicates.. See below.
try {
File[] files = orgDir.listFiles();
for (File file : files) { // Goes through the files in the given directory
if (!file.isDirectory() && file.getName().endsWith(selectedExtension)){
File destinationPath = new File(destDir.getCanonicalPath() + "\\");
destDir = new File(destinationPath + "\\" + extensionType); // Sets the destination path
destDir.mkdir();
System.out.println("file:" + file.getCanonicalPath()); // Prints the file path
try{
String fileNameWithOutExt = file.getName().replaceFirst("[.][^.]+$", ""); // Gets the current file without the extension
File destFile = new File(destDir.getPath() + "\\" + file.getName()); // If a file of the same name exists in the dest folder
if (Files.exists(Paths.get(destFile.getPath()))) // Checks if there is a file with the same name in the folder
{
System.out.println("There is a duplicate.");
File[] destFiles = destDir.listFiles();
for (File destinationFile : destFiles) // Searches through the destination folder
{
if(destinationFile.getName().startsWith(fileNameWithOutExt)){ // Checks if the selected file has the same name as the file that's going to be moved.
y++; // Increments y by 1 to keep track of how many there are of the same/similar name
}
}
File newFile = new File(orgDir.getPath() + "\\" + fileNameWithOutExt + "." + y + selectedExtension); // Creates a new file with new name.
file.renameTo(newFile); // Renames to a unique name and moves the file to the destination folder
File destPath = new File(destDir.getPath() + "\\" + newFile.getName()); // Gets the destination path for the file
System.out.println(newFile.getCanonicalPath());
Files.copy(Paths.get(newFile.getCanonicalPath()), Paths.get(destPath.getPath())); // Renames the original file back to its original name
newFile.renameTo(new File(orgDir.getPath() + "\\" + fileNameWithOutExt + selectedExtension));
} else {
Files.copy(Paths.get(file.getPath()), Paths.get(destFile.getPath())); // Moves the file to the destination folder
}
}catch(Exception e){
e.printStackTrace();
}
} else{
startProcess(file, destDir);
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
}

I am assuming your program is correct and this is purely memory issue. Increase the memory settings by running program with options -Xms 1024m -Xmx 1g, increase the values if necessary, be aware of your total available memory.
Will be nice to see a stack trace of exception to know for sure what caused the crash.

Writing uploaded files with non-ASCII file names to disk from Java servlet

I have a servlet that writes uploaded files to disk, using Apache Commons fileupload. That's all working fine in general.
It runs on both Windows and Linux servers using Tomcat. On Windows it handles files with non-ASCII file names correctly and the files are saved properly.
On Linux (CentOS 6) however the file names are not saved correctly when containing non-ASCII characters.
If have tried three different versions of writing the file. In Windows all work, in Linux none do but they produce different results.
Version 1:
String fileName = URLDecoder.decode(encFilename, "UTF-8");
String filePath = uploadFolder + File.separator + fileName;
File uploadedFile = new File(filePath);
item.write(uploadedFile);
Version 2:
String fileName = URLDecoder.decode(encFilename, "UTF-8");
String filePath = uploadFolder + File.separator + fileName;
File uploadedFile = new File(filePath);
InputStream input = item.getInputStream();
try {
Files.copy(input, uploadedFile.toPath());
} catch (Exception e) {
log.error("Error writing file to disk: " + e.getMessage());
} finally {
input.close();
}
Uploading a file called: Это тестовый файл.txt I get the following results on Linux:
Version 1: A file named: ??? ???????? ????.txt
Version 2: Error writing file to disk: Malformed input or input contains unmappable characters: /tmp/Это тестовый файл.txt
While on a Windows machine with Tomcat 7 and Java 7 the file name is written correctly as Это тестовый файл.txt
A third version uses the approach from this post and doesn't use FileUpload. The result is the same as what's produced by version 2.
Version 3:
Part filePart = request.getPart("file");
String fileName = "";
for (String cd : filePart.getHeader("content-disposition").split(";")) {
if (cd.trim().startsWith("filename")) {
fileName = cd.substring(cd.indexOf('=') + 1).trim().replace("\"", "");
fileName = fileName.substring(fileName.lastIndexOf('/') + 1).substring(fileName.lastIndexOf('\\') + 1); // MSIE fix.
}
}
String filePath = uploadFolder + File.separator + fileName;
File uploadedFile = new File(filePath);
InputStream input = filePart.getInputStream();
try {
Files.copy(input, uploadedFile.toPath());
} catch (Exception e) {
log.error("Error writing file to disk: " + e.getMessage());
} finally {
input.close();
}
Tomcat is running with -Dfile.encoding=UTF-8 and locale shows LANG=en_US.UTF-8
touch "Это тестовый файл.txt" produces a file with that name.
The file contents are always written correctly. (except of course where no file is written at all).
What am I missing or doing wrong?

I solved the problem by converting all use of java.io.File to java.nio.Files and java.nio.Path. So it seems the java.io.File api is buggy. Using this it works fine on both Windows and Linux.
// The filename is passed as a URLencoded string
String fileName = URLDecoder.decode(request.getParameter("fileName"), "UTF-8");
Path filePath = Paths.get(uploadFolder, fileName);
Part filePart = request.getPart("file");
InputStream input = filePart.getInputStream();
try {
Files.copy(input, filePath);
} catch (Exception e) {
log.error("Error writing file to disk: " + e.getMessage());
} finally {
input.close();
}
I ran into the same problem in several other parts of the app that worked with the uploaded files and in all cases getting rid of java.io.File and using java.nio instead solved the problem.

Can't delete file in java on windows ....?

I am trying to use file.delete() but it does not delete file ? I tested it on linux it delete files but on windows it does not delete file why..?
code :
private File getFiletobeUpload(File foto) {
boolean errorRename = true;
File uploadFile = null;
File testFile = foto;
String subdirname = this.checkDir(foto);
if (testFile.canWrite()) {
uploadFile = new File(AppConstants.PHOTOPATH + "/" + subdirname + "/" + testFile.getName());
try {
FileInputStream origStream = new FileInputStream(testFile);
FileOutputStream outStream = new FileOutputStream(uploadFile);
origStream.getChannel().transferTo(0, testFile.length(), outStream.getChannel());
origStream.close();
origStream = null;
outStream.close();
outStream = null;
} catch (Exception e) {
this.errorString += "error while writing to orig dir";
logger.error(e);
}
errorRename = !testFile.delete();
if (errorRename) {
this.errorString += "error while deleting the file";
}
}
testFile = null;
return uploadFile;
}

your code seems a bit odd, you are checking "testFile" for write access but then actually reading from it (FileInputStream). Is the first try/catch block running without exception? Maybe you should check if the file even exists:
System.out.println("File exists: "+testFile.exists());
errorRename = !testFile.delete();
Also if you just want to rename a file, use:
file.renameTo(File dest)

A general suggestion: consider using the classes in the java.nio package to your file IO if you can (not sure which version of Java you're running with) because the error handling is improved and you should be able to find a more specific reason for the failure based on the type of exception thrown and the exception message.

As per API, the disk-drive specifier - "/" for the UNIX root directory, and "\\" for a Microsoft Windows UNC pathname.
so for windows:
uploadFile = new File(AppConstants.PHOTOPATH + "\\" + subdirname + "\\" + testFile.getName());

Can't delete file after being renamed (file is opened)

I am using icefaces to upload files to relative path in my web app (mywebapp/audio), then after the file is getting uploaded I rename it to save its extension as follows:
public static File changeExtToWav(FileInfo fileInfo,
StringBuffer originalFileName) {
log.debug("changeExtToWav");
int mid = fileInfo.getFile().getName().lastIndexOf(".");
String fileName = fileInfo.getFile().getName().substring(0, mid);
originalFileName.append(fileName);
log.debug("originalFileName: " + originalFileName);
String newFileName = fileName + "_" + new Date().getTime() + "."
+ "wav";
File newFile = new File(fileInfo.getFile().getParent() + "/"
+ newFileName);
log.debug("newFileName: " + newFile.getName());
fileInfo.getFile().renameTo(newFile);
return newFile;
}
after the file is getting uploaded, sometimes I want to delete it from UI button as follows:
try {
File fileToDelete = new File(filePath); // correct file path
log.debug("file exists: " + fileToDelete.exists()); // true
fileToDelete.delete();
} catch (Exception e) {
e.printStackTrace();
}
the file path is correct, and I get no exceptions, but the file is not deleted (I am using java 6 btw).
please advise how to fix this issue.
UPDATE: using the following useful method, I can get that the file is opened, any ideas how to close it ?
public String getReasonForFileDeletionFailureInPlainEnglish(File file) {
try {
if (!file.exists())
return "It doesn't exist in the first place.";
else if (file.isDirectory() && file.list().length > 0)
return "It's a directory and it's not empty.";
else
return "Somebody else has it open, we don't have write permissions, or somebody stole my disk.";
} catch (SecurityException e) {
return "We're sandboxed and don't have filesystem access.";
}
}

Well if the file is open, then there is two solutions :
You have a stream in your program open on this file. Note that afaik it's a problem on Windows, with Unix I can delete a File even if a stream is opened on it.
You have an other process using this file. So in this case you can't do anything from Java.
In the log it tells also that it can be a permission problem, are you sure you have enough privileges?
You can also use Files#delete(Path path) (jdk7) to have more details about the issue.

Why am I having nondeterministic file deletion errors when using DiskFileItem?

My upload servlet keeps throwing me an exception saying that the file that I'm trying to replace (near the end of my code) could not be deleted at (seemingly) random. I don't know what's causing this since I'm not using any streams and the file isn't open in my browser. Does anyone know what could be causing this? I'm completely clueless on this one as the code seems correct to me. This is the first time I've used DiskFileItem so I'm not sure if there are any nuances to handle there.
Keep in mind that it sometimes works, sometimes doesn't. I'm lost on that.
Problem Area:
File destination = new File(wellnessDir + File.separator + fileName + ".pdf");
System.out.println("destination file exists: " + destination.exists());
System.out.println("file to be moved exists: " + uploadedFile.exists());
if(destination.exists()){
boolean deleted = destination.delete();
if(!deleted)
throw new Exception("Could not delete file at " + destination);
}
My System outs always say that both file and destination exist. I'm trying to get the upload to overwrite the existing file.
Full code: (& pastebin)
private void uploadRequestHandler(ServletFileUpload upload, HttpServletRequest request)
{
// Handle the request
String fileName = "blank";
try{
List items = upload.parseRequest(request);
//Process the uploaded items
Iterator iter = items.iterator();
File uploadedFile = new File(getHome() + File.separator + "temp");
if(uploadedFile.exists()){
boolean tempDeleted = uploadedFile.delete();
if(!tempDeleted)
throw new Exception("Existing temp file could not be deleted.");
}
//write the file
while (iter.hasNext()) {
DiskFileItem item = (DiskFileItem) iter.next();
if(item.isFormField()){
String fieldName = item.getFieldName();
String fieldValue = item.getString();
if(fieldName.equals("fileName"))
fileName = fieldValue;
//other form values would need to be handled here, right now only need for fileName
}else{
item.write(uploadedFile);
}
}
if(fileName.equals("blank"))
throw new Exception("File name could not be parsed.");
//move file
File wellnessDir = new File(getHome() + File.separator + "medcottage" + File.separator + "wellness");
File destination = new File(wellnessDir + File.separator + fileName + ".pdf");
System.out.println("destination file exists: " + destination.exists());
System.out.println("file to be moved exists: " + uploadedFile.exists());
if(destination.exists()){
boolean deleted = destination.delete();
if(!deleted)
throw new Exception("Could not delete file at " + destination);
}
FileUtil.move(uploadedFile, new File(wellnessDir + File.separator + fileName + ".pdf"));
writeResponse();
} catch (Exception e) {
System.out.println("Error handling upload request.");
e.printStackTrace();
}
}
edit: to add, getHome() and "home" aren't really in the code, that's just to protect my home path

After much testing and aggravation, finally tried it on a different machine, same code, worked great. Has something to do with me transferring domains on my work machine and it messing with permissions.