I'll put my code first:
#Post
public Representation post(InputStream zip) throws Throwable {
createFile(zip, "C:/temp\abc.zip");
return new StringRepresentation("File uploaded");
}
public void createFile(InputStream zipStream, uploadedFileLocation) throws Exception {
try {
writeToFile(zipStream, uploadedFileLocation);
FileUtils.forceDelete(new File(uploadedFileLocation));
} catch (Exception e) {
throw e;
}
}
private void writeToFile(InputStream uploadedInputStream, String uploadedFileLocation) {
try {
OutputStream out = new FileOutputStream(new File(uploadedFileLocation));
int read = 0;
byte[] bytes = new byte[1024];
out = new FileOutputStream(new File(uploadedFileLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
uploadedInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
I am trying make a server that allows a user to upload a zip file. The server then write the zip file to disk, unzip it, then delete the zip while keeping the unzipped portion on the server. However, when I send the zip file to my server, it cannot get deleted. When using FileUtils.forceDelete(), it says that it cannot delete the file. It is ideal for the zip to be deleted after it is unzipped.
EDIT: I can only delete the file after post(InputStream zip) returns. If I call delete from within the post method, it won't delete because post hasn't returned. Is there a way around this?
Hmm. It appears you're trying to delete a directory?
FileUtils.forceDelete(new File(uploadedFileLocation));
But uploadedFileLocation is "C:\temp" from your post method. I'm not sure if this would cause the issue, or if this is intended behavior, but that code does not delete the zip file, and is attempting to delete the actual C:\temp directory.
Also, potentially worth noting: in your writeToFile method, you're initializing OutputStream out twice. I'm not 100% on this, but it could be that the first initialization is holding the file pointer open until the entire object is deleted from the stack (i.e., when post returns). Take out the second initialization and see if any changes occur?
I guess your problem might be caused by you using "C:/temp" instead of "C:/temp/fileName", so you end up trying to delete a folder, that you might not have permissions to.
You don't need to use file location, why not use the actual file?
Pass the file as an argument and do file.delete(); when you are done.
If the problem is not that you are trying to delete a directory, then it is most likely that the file you are trying to delete is still open. On Windows, that means that will stop you from deleting the file.
By the looks of it, your writeToFile method opens the file twice ... but only closes it once. I suspect that that means that it will leak a file descriptor, and the OS will think that the application is still using the file.
It turns out I should have called file.delete(); instead of using FileUtils.
I don't think so. Sure, you won't get an exception due to the file not being deleted. But instead File.delete() will return false and your application will leave the file in the file system. In the long term, that could be worse, unless you've got something else in place to clean out the orphaned files.
Related
I'm trying to write an InputStream that is an mp4 that I get from calling an external SOAP service, when I do so, it always generates this tmp files for my chosen temporary directory(java.io.tmpdir) that aren't removable and stay after the writing is done.
Writing images that I also get from the SOAP service works normal without the permanent tmp on the directory. I'm using java 1.8 SpringBoot
tmp files
This is what I'm doing:
File targetFile = new File("D:/archive/video.mp4");
targetFile.getParentFile().mkdirs();
targetFile.setWritable(true);
InputStream inputStream = filesToWrite.getInputStream();
OutputStream outputStream = new FileOutputStream(targetFile);
try {
int byteRead;
while ((byteRead = inputStream.read()) != -1) {
outputStream.write(byteRead);
}
} catch (IOException e) {
logger.fatal("Error# SaveFilesThread for guid: " + guid, e);
}finally {
try {
inputStream.close();
outputStream.flush();
outputStream.close();
}catch (Exception e){
e.printStackTrace();
}
also tried:
byte data[] = IOUtils.toByteArray(inputStream);
Path file = Paths.get("video.mp4");
Files.write(file, data);
And from apache commons IO:
FileUtils.copyInputStreamToFile(initialStream, targetFile);
When your code starts, the damage is already done. Your code is not the source of the temporary files (It's.. a ton of work for something that could be done so much simpler, though, see below), it's the framework that ends up handing you that filesToWrite variable.
It is somewhat likely that you can hook in at an earlier point and get the raw inputstream representing the socket or HTTP connection, and start saving the files straight from there. Alternatively, Perhaps filesToWrite has a way to get at the files themselves, in which case you can just move them into place instead of copying them over.
But, your code to do this is a mess, it has bad exception handling, and leaks memory, and is way too much code for a simple job, and is possibly 2000x to 10000x slower than needed depending on your harddisk (I'm not exaggerating, calling single-byte read() on unbuffered streams is thousands of times slower!)
// add `throws IOException` to your method signature.
// it saves files, it's supposed to throw IOException,
// 'doing I/O' is in the very definition of your method!
try (InputStream in = filesToWrite.getInputStream();
OutputStream out = new FileOutputStream(targetFile)) {
in.transferTo(out);
}
That's it. That solves all the problems - no leaks, no speed loss, tiny amount of code, fixes the deplorable error handling (which, here, is 'log something to the log, then print something to standard out, then potentially leak a bunch of resources, then don't tell the calling code anything went wrong and return exactly as if the copy operation succeeded).
I've recently inherited a Java API and am having trouble with file uploads. Unfortunately, Java isn't a language I have much experience in so I'm a bit stumped by this.
The MultiPartFile is being received ok, and I can find the file in the temp directory, but when I try to use File.transferTo() to create the final file I just get the below error;
java.nio.file.NoSuchFileException: C:\Users\myUser\AppData\Local\Temp\undertow3706399294849267898upload -> S:\Dev\PolicyData\Temp.xlsx
As I mentioned the temp undertow file exists, and the directory on the S drive also exist, (but there's no Temp.xlsx as my understanding is this should be created by transferTo()). Any solutions I've found to this problem so far are resolved using absolute file paths.
This is a simplified version of the code but the error remains the same.
SpringBoot framework is "1.5.3.RELEASE", running Java 1.8.0_131
ResponseEntity handleFileUpload(#RequestPart(name = "file") MultipartFile file, #PathVariable Long stageFileTypeId) {
if (!file.isEmpty()) {
try {
String filePath = "S:\\Dev\\PolicyData\\Temp.xlsx";
log.info("Upload Path = {}", filePath);
File dest = new File(filePath);
file.transferTo(dest);
return ResponseUtil.wrapOrNotFound(Optional.ofNullable(filePath));
}
catch (Exception ex) {
log.error("An error has occurred uploading the file", ex);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();
}
}
else {
log.error("An error has occurred, no file was received");
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();
}
}
If you need any more information please let me know.
Thanks,
Neil
The API for MultipartFile is a bit tricky. The transferTo(File) method javadoc states that (bold are mine):
This may either move the file in the filesystem, copy the file in the
filesystem, or save memory-held contents to the destination file. If
the destination file already exists, it will be deleted first.
If the target file has been moved in the filesystem, this operation
cannot be invoked again afterwards. Therefore, call this method just
once in order to work with any storage mechanism.
It seems that the Undertow implementantion already called it to move the in-memory uploaded file to "C:\Users\myUser\AppData\Loca\Temp\undertow3706399294849267898upload" so another transferTo is failing.
I came across the same problem using javax.servlet.http.Part in a Wildfly containter with Undertow.
If you are using Spring framework >= 5.1, you could try the Multipart.transferTo(Path) method, using dest.toPath()
Or you can copy from the inputStream, with something like this:
try (InputStream is = multipartFile.getInputStream()) {
Files.copy(is, dest.toPath());
}
I am using Kryo to save binary files of user data. The user can open one of their files in my application. I'm not sure if I have a clean approach to detecting whether they tried to open a file of some other type.
Right now, I'm writing a simple FileHeader object to the file before the user's data. The file header has info about what version of the app saved the file.
public void write (UserProject project, File file) throws FileNotFoundException {
OutputStream outputStream = new DeflaterOutputStream(new FileOutputStream(file));
Output output = new Output(outputStream);
kryo.writeObject(output, new FileHeader());
kryo.writeObject(output, project);
output.close();
}
So when I load a file, I can try to deserialize the file header and the user project and catch any Exception that might occur. But doing a catch-all block could hide certain issues I could perhaps react to in a more elegant way that simply showing the user an error no matter the exception. Here's what I'm doing now:
public Project read (File file) throws FileNotFoundException, FileVersionException, UnreadableException {
InputStream inputStream = new InflaterInputStream(new FileInputStream(file));
Input input = new Input(inputStream);
try {
FileHeader fileHeader = kryo.readObject(input, FileHeader.class);
if (fileHeader.fileVersion > CURRENT_FILE_VERSION)
throw new FileVersionException(/* */);
Project project = kryo.readObject(input, Project.class);
return project;
} catch (Exception e){
if (DEBUG) e.printStackTrace();
throw new UnreadableException(e); //caller will show user error msg
} finally {
input.close();
}
}
I suppose there's also a very tiny (infinitesimal?) chance that some file actually loads without throwing an exception, in which case a very unexpected error could happen elsewhere in my application. Not sure if I should worry about this...a user should not expect to open an incorrect file type and have it work correctly.
You could use magic numbers, a set of bytes that describes the type of file. Like .jpg, .pdf, .wav, etc. all have a few bytes at the beginning of each file, so even if these types are saved with different extensions you can check to see if the file's magic number is OK.
Magic Number Description
However, if you're serializing and deserializing you may have to tack on some additional data to the file after serializing and remove it before deserializing.
I am trying to delete a file and then recreate it. First I check to see if the file already exists, then, if it does, I delete it. Then I try to create a new file in the same place with the same name. When I do this I get this error:
java.nio.file.AccessDeniedException: inputLog.txt
However, if the file did not exist before running these three operations, then the file is created without issue.
Here is my code:
final Path INPUTLOGPATH = FileSystems.getDefault().getPath("inputLog.txt");
try {
reader = Files.newBufferedReader(INPUTLOGPATH, charset);
} catch (IOException e) {
reader = null;
}
if (reader != null) {
try {
Files.delete(INPUTLOGPATH);
} catch (IOException e) {
e.printStackTrace();
}
}
try {
Files.createFile(INPUTLOGPATH);
} catch (IOException e) {
e.printStackTrace();
}
First I check to see if the file already exists, then, if it does, I delete it.
Why? Opening the file for output will already do all that. You're just repeating work that the operating system already has to do. Remove all this. You're doing it wrong by not closing the file reader, but it's irrelevant. Don't write unnecessary code.
Then I try to create a new file in the same place with the same name
That is also unnecessary as shown. Just open the file for output when you need it.
As you have it now:
you're opening the file, which is a search, among many other things
you're deleting the file, which is another search
you're creating the file, which is another search
then presumably you're opening the file for output, which requires another search, another deletion, and another creation, internally to the operating system.
Don't do this. Just remove all this code. It accomplishes exactly nothing.
You're also introducing all kinds of timing-window problems by this approach, and you still have to deal with eventual failure at the point where you actually open the file for output.
I'm serving a file from the file system dynamically with a jsp
Here's my code:
<%# page import="java.io.*,java.util.*"
InputStream in = null;
OutputStream responseOut = null;
File file = new File(request.getAttribute("fileToServe"));
try{
in = new FileInputStream(file);
responseOut = response.getOutputStream();
// Transfer bytes from in to out
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0) {
responseOut.write(buf, 0, len);
}
}finally{
if( responseOut != null ) try {
responseOut.close();
} catch( IOException ioe ){}
if( in != null ) try {
in.close();
} catch( IOException ioe ){}
}
file.delete();
%>
The problem I'm facing is, the file is delete only the first time the code is run, which is after the server restart. Subsequent calls doesn't delete the file.
I used ProcessExplorer to track this and and yeap, the Java VM is holding that file, I don't really know why is this happening.
We will run on Windows OS, always, is there a work around for this?
I've found a number of resources on the internet about this, but I can't figure out from them how to solve this problem.
What creates the file? I only see reading then deleting it in that code.
Things to watch out for:
Reading a file requires read permissions from the file but deleting the file requires write permission from the directory; and
Make sure you close() any files you create. If you don't you might lose data or it may take time to flush an implicit close.
Lastly, using an attribute like fileToServe that comes from the user is really dangerous. I'm hoping you're sanitizing that elsewhere. You should ensure that only allowed files are served this way.
Ok, I've checked, after cletus comment, the place where this file is being created. The close method on the stream that wrote the file was missing.
Mystery solved
cletus, please add your answer so I can accept it