I'm serving a file from the file system dynamically with a jsp
Here's my code:
<%# page import="java.io.*,java.util.*"
InputStream in = null;
OutputStream responseOut = null;
File file = new File(request.getAttribute("fileToServe"));
try{
in = new FileInputStream(file);
responseOut = response.getOutputStream();
// Transfer bytes from in to out
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0) {
responseOut.write(buf, 0, len);
}
}finally{
if( responseOut != null ) try {
responseOut.close();
} catch( IOException ioe ){}
if( in != null ) try {
in.close();
} catch( IOException ioe ){}
}
file.delete();
%>
The problem I'm facing is, the file is delete only the first time the code is run, which is after the server restart. Subsequent calls doesn't delete the file.
I used ProcessExplorer to track this and and yeap, the Java VM is holding that file, I don't really know why is this happening.
We will run on Windows OS, always, is there a work around for this?
I've found a number of resources on the internet about this, but I can't figure out from them how to solve this problem.
What creates the file? I only see reading then deleting it in that code.
Things to watch out for:
Reading a file requires read permissions from the file but deleting the file requires write permission from the directory; and
Make sure you close() any files you create. If you don't you might lose data or it may take time to flush an implicit close.
Lastly, using an attribute like fileToServe that comes from the user is really dangerous. I'm hoping you're sanitizing that elsewhere. You should ensure that only allowed files are served this way.
Ok, I've checked, after cletus comment, the place where this file is being created. The close method on the stream that wrote the file was missing.
Mystery solved
cletus, please add your answer so I can accept it
Related
I'm trying to write an InputStream that is an mp4 that I get from calling an external SOAP service, when I do so, it always generates this tmp files for my chosen temporary directory(java.io.tmpdir) that aren't removable and stay after the writing is done.
Writing images that I also get from the SOAP service works normal without the permanent tmp on the directory. I'm using java 1.8 SpringBoot
tmp files
This is what I'm doing:
File targetFile = new File("D:/archive/video.mp4");
targetFile.getParentFile().mkdirs();
targetFile.setWritable(true);
InputStream inputStream = filesToWrite.getInputStream();
OutputStream outputStream = new FileOutputStream(targetFile);
try {
int byteRead;
while ((byteRead = inputStream.read()) != -1) {
outputStream.write(byteRead);
}
} catch (IOException e) {
logger.fatal("Error# SaveFilesThread for guid: " + guid, e);
}finally {
try {
inputStream.close();
outputStream.flush();
outputStream.close();
}catch (Exception e){
e.printStackTrace();
}
also tried:
byte data[] = IOUtils.toByteArray(inputStream);
Path file = Paths.get("video.mp4");
Files.write(file, data);
And from apache commons IO:
FileUtils.copyInputStreamToFile(initialStream, targetFile);
When your code starts, the damage is already done. Your code is not the source of the temporary files (It's.. a ton of work for something that could be done so much simpler, though, see below), it's the framework that ends up handing you that filesToWrite variable.
It is somewhat likely that you can hook in at an earlier point and get the raw inputstream representing the socket or HTTP connection, and start saving the files straight from there. Alternatively, Perhaps filesToWrite has a way to get at the files themselves, in which case you can just move them into place instead of copying them over.
But, your code to do this is a mess, it has bad exception handling, and leaks memory, and is way too much code for a simple job, and is possibly 2000x to 10000x slower than needed depending on your harddisk (I'm not exaggerating, calling single-byte read() on unbuffered streams is thousands of times slower!)
// add `throws IOException` to your method signature.
// it saves files, it's supposed to throw IOException,
// 'doing I/O' is in the very definition of your method!
try (InputStream in = filesToWrite.getInputStream();
OutputStream out = new FileOutputStream(targetFile)) {
in.transferTo(out);
}
That's it. That solves all the problems - no leaks, no speed loss, tiny amount of code, fixes the deplorable error handling (which, here, is 'log something to the log, then print something to standard out, then potentially leak a bunch of resources, then don't tell the calling code anything went wrong and return exactly as if the copy operation succeeded).
I have a problem generating ZIP files via zip4j.
I am able to produce ZIP archive using following code (I omitted some parts, that are not related to the issue), which is basically taken from zip4j tutorial:
File zipFile = new File(zipName);
ZipParameters params = new ZipParameters();
params.setCompressionMethod(Zip4jConstants.COMP_DEFLATE);
params.setCompressionLevel(Zip4jConstants.DEFLATE_LEVEL_NORMAL);
byte[] buffer = new byte[8192];
try (ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipFile))) {
for (/* loop through list of input data */) {
String outputName = /* method to determine file name */;
try (InputStream in = /* method to get IS */ ) {
params.setFileNameInZip(outputName);
File tmpEntry = new File(outputName);
tmpEntry.createNewFile();
out.putNextEntry(tmpEntry, params);
int len;
while ((len = in.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
tmpEntry.delete();
out.closeEntry();
in.close();
}
}
}
The problem is that although all files are correctly included in ZIP archive, their declared file size is 0. I can unzip them using "dumb" ZIP readers (like build-in TotalCommander's one), because all data are actually here, but more "clever" programs (like 7zip) produce CRC error and refuse to open them as corrupted.
I would say I need to declare the file size somehow (and I am definitely not doing this in my code snippet), but I was unable to find the (probably obvious) solution. I have googled that native java.util.zip.ZipEntry has .setSize() method, but I don't see anything like this in zip4j...
Anyone knows the correct approach to this?
So the solution is quite simple. One just needs to add following setting into ZipParameters:
params.setSourceExternalStream(true);
How I found that?
I digged deeper into the code and find following in closeEntry() method of CipherOutputStream.java class of zip4j:
if (zipParameters.isSourceExternalStream()) {
fileHeader.setUncompressedSize(totalBytesRead);
if (localFileHeader.getUncompressedSize() != totalBytesRead) {
localFileHeader.setUncompressedSize(totalBytesRead);
}
}
This is the only place where setUncompressedSize() - which sounds like having something to do with declared file size - method is called. So I got suspicious and tried to set the parameter in my code. And it does the job.
The lesson learned is that this example of using zip4j is incorrect because it is missing that vital line of code.
Up till early this year the US Treasury web site posted monthly US Receipts and Outlays data in txt format. It was easy to write a program to read and store the info. All I use were:
URL url = new URL("https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts1214.txt")
URLConnection connection.openConnection();
InputStream is = connection.getInputStream();
Then I just read the InputStream into a local file.
Now when I try same code, for May, I get an InputStream with nothing in it.
Just clicking on "https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts0415.xlsx" opens an excel worksheet (the download path has since changed).
Which is great if you don't mind clicking on each link separately ... saving the file somewhere ... opening it manually to enable editing ... then saving it again as a real .xlsx file (because they really hand you an .xls file.)
But when I create a URL from that link, and use it to get an InputStream, the is empty. I also tried url.openStream() directly. No different.
Can anyone see a way I can resume using a program to read the new format?
In case its of interest I now use this code to write the stream to the file bit by bit... but there are no bits, so I don't know if it works.
static void copyInputStreamToFile( InputStream in, File file ) {
try {
OutputStream out = new FileOutputStream(file);
byte[] buf = new byte[1024];
System.out.println("reading: " + in.read(buf));
//This is what tells me it is empty, i.e. the loop below is ignored.
int len;
while((len=in.read(buf))>0){
out.write(buf,0,len);
}
out.close();
in.close();
} catch (Exception e) {
e.printStackTrace();
}
}
Any help is appreciated.
I have a situation where I need to scan the runtime classpath for a resource file (say, res/config/meta.cfg), and then create a File handle for it. The best I've been able to come up with is:
// This file is located inside a JAR that is on the runtime classpath.
String fileName = "res/config/meta.cfg";
try {
InputStream inStream = ClassLoader.getSystemResourceAsStream(fileName);
File file = new File(String.format("${java.io.tmpdir}/%s", fileName));
FileOutputStream foutStream = null;
foutStream = new FileOutputStream(file);
int read = 0;
byte[] bytes = new byte[1024];
while((read = inStream.read(bytes)) != -1)
foutStream.write(bytes, 0, read);
foutStream.close();
return file;
} catch (Exception exc) {
throw new RuntimeException(exc);
}
So essentially, read in the resource as an InputStream, and then write the stream to a temp file (under {$java.io.tmpdir}) so that we can obtain a valid File handle for it.
This seems like going 3 sides around the barn. Is there a better/easier/more elegant way of doing this? Thanks in advance!
No.
Of course you can (and probably should) use a library to copy the InputStream's content to a file but that obviously is not the point of your question.
The classpath does not consist of directories only; resources can be inside archives (typically JARs) or on servers, and may not exist as something that can be accessed via a java.io.File object.
Typically the core problem is to use java.io.File objects where an InputStream would be sufficient. Sometimes you can't do anything against it when using a third-party library but it is a hint that the library designers didn't work very carefully. If you need the file handle in your own code you should have another look why it can't be an InputStream. Most of the time it can.
I'll put my code first:
#Post
public Representation post(InputStream zip) throws Throwable {
createFile(zip, "C:/temp\abc.zip");
return new StringRepresentation("File uploaded");
}
public void createFile(InputStream zipStream, uploadedFileLocation) throws Exception {
try {
writeToFile(zipStream, uploadedFileLocation);
FileUtils.forceDelete(new File(uploadedFileLocation));
} catch (Exception e) {
throw e;
}
}
private void writeToFile(InputStream uploadedInputStream, String uploadedFileLocation) {
try {
OutputStream out = new FileOutputStream(new File(uploadedFileLocation));
int read = 0;
byte[] bytes = new byte[1024];
out = new FileOutputStream(new File(uploadedFileLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
uploadedInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
I am trying make a server that allows a user to upload a zip file. The server then write the zip file to disk, unzip it, then delete the zip while keeping the unzipped portion on the server. However, when I send the zip file to my server, it cannot get deleted. When using FileUtils.forceDelete(), it says that it cannot delete the file. It is ideal for the zip to be deleted after it is unzipped.
EDIT: I can only delete the file after post(InputStream zip) returns. If I call delete from within the post method, it won't delete because post hasn't returned. Is there a way around this?
Hmm. It appears you're trying to delete a directory?
FileUtils.forceDelete(new File(uploadedFileLocation));
But uploadedFileLocation is "C:\temp" from your post method. I'm not sure if this would cause the issue, or if this is intended behavior, but that code does not delete the zip file, and is attempting to delete the actual C:\temp directory.
Also, potentially worth noting: in your writeToFile method, you're initializing OutputStream out twice. I'm not 100% on this, but it could be that the first initialization is holding the file pointer open until the entire object is deleted from the stack (i.e., when post returns). Take out the second initialization and see if any changes occur?
I guess your problem might be caused by you using "C:/temp" instead of "C:/temp/fileName", so you end up trying to delete a folder, that you might not have permissions to.
You don't need to use file location, why not use the actual file?
Pass the file as an argument and do file.delete(); when you are done.
If the problem is not that you are trying to delete a directory, then it is most likely that the file you are trying to delete is still open. On Windows, that means that will stop you from deleting the file.
By the looks of it, your writeToFile method opens the file twice ... but only closes it once. I suspect that that means that it will leak a file descriptor, and the OS will think that the application is still using the file.
It turns out I should have called file.delete(); instead of using FileUtils.
I don't think so. Sure, you won't get an exception due to the file not being deleted. But instead File.delete() will return false and your application will leave the file in the file system. In the long term, that could be worse, unless you've got something else in place to clean out the orphaned files.