Creation of single zip containing multiple zips fails for ServletOutputStream - java

I am using commons compress to zip multiple files and send it the client from a Servlet.
The files could be a combination of any type of files(text, video, audio, archives, images etc). I take the inputStream of file and write to ServletOutputStream using IOUtils.copy(is, os).
The code usually works fine for any document combination but when there is a request to download files that contain more than 1 zip, I get java.io.IOException: Closed
As a result, the zip file created is corrupted even though the size of zip is summation of individual filesizes(I am not using compression).
I tried to locally create zip and use FileOutputStream instead of response.getOutputStream() in the constructor of ZipArchiveOutputStream and it succeeds.
So, it looks like the problem exists for ServletOutputStream.
Can anyone suggest any workaround.
Here is my code :
`try (ZipArchiveOutputStream zos = new ZipArchiveOutputStream( response.getOutputStream())) {
//get fileList
for(File file : files) {
addFileToZip(zos, file.getName(), new BufferedInputStream(new FileInputStream(file)));
}
zos.close()
}
`
public static void addFileToZip(ZipArchiveOutputStream zipOutputStream, String filename, InputStream inputStream) throws FileNotFoundException {
if(zipOutputStream != null && inputStream != null) {
try {
zipOutputStream.putArchiveEntry(new ZipArchiveEntry(filename));
IOUtils.copy(inputStream, zipOutputStream);
logger.debug("fileAddedToZip :" + filename);
} catch (IOException e) {
logger.error("Error in adding file :" + filename, e);
} finally {
try {
inputStream.close();
zipOutputStream.closeArchiveEntry(); //**Starts to fail here after 1st zip is added**
} catch (IOException e) {
logger.error("Error in closing zip entry :" + filename, e);
}
}
}
`
Here is the exception trace :
`
java.io.IOException: Closed
at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:627)
at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:577)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.writeOut(ZipArchiveOutputStream.java:1287)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.writeOut(ZipArchiveOutputStream.java:1272)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.writeDataDescriptor(ZipArchiveOutputStream.java:997)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.closeArchiveEntry(ZipArchiveOutputStream.java:461)
at xxx.yyy.zzz.util.ZipUtils.addFileToZip(ZipUtils.java:110)
line 110 is zipOutputStream.closeArchiveEntry(); //**Starts to fail here after 1st zip is added**
Thanks in advance.

The problem is that you use try-with-resources which automatically closes the stream you create in it, and yet you also close it manually, and therefore when the JVM tries to auto-close it is when you get java.io.IOException: Closed exception because it is already closed.
If you use try-with-resources, you don't need to close the streams you create in it. Remove your manual zos.close() statement:
try (ZipArchiveOutputStream zos =
new ZipArchiveOutputStream(response.getOutputStream())) {
//get fileList
for(File file : files) {
addFileToZip(zos, attachment.getFileName(), is);
}
} // Here zos will be closed automatically!
Also note that once zos is closed, it will also close the servlet's underlying OutputStream so you will not be able to add further entries. You have to add all before it is closed.

Related

File has been moved, can not be read again (Spring mvc)

I am using spring MVC where through API I am uploading zip file using MultipartFile. In backend I have to convert uploaded zip file into InputStream for further processing. But my code is giving error intermittently " File has been moved, can not be read again ".
here is the code snippet :
File temp = null;
InputStream stream = null;
try {
InputStream initialStream = inputFile.getInputStream();
byte[] buffer = new byte[initialStream.available()];
initialStream.read(buffer);
temp = File.createTempFile("upload", null);
try (OutputStream outStream = new FileOutputStream(temp)) {
outStream.write(buffer);
}
ZipFile zipFile = new ZipFile(temp);
stream = zipFile.getInputStream(zipFile.getEntries().nextElement());
} catch (Exception e) {
log.error("Exception occurred while processing zip file " + e.getMessage());
throw e;
} finally {
if (temp != null)
temp.delete();
}
return stream;
Here inputFile is MultipartFile.
Could you please suggest what is wrong here?
Your code is returning an input stream from a file that you have deleted - last line is temp.delete().
ZipInputStream has a small internal buffer for decoding, so that may explain why some read calls work after the delete, but it will not be possible to continue reading from a file that you deleted, hence the exception.
Also, the call initialStream.available() is unlikely to be the correct way to determine the size of the input stream file part. Try printing the size / check how to read the actual length of the file in the multipart stream - such as part.getSize(), or transfer the bytes into a new ByteArrayOutputStream() before assigning to buffer.
I would not recommend doing any work with files or multipart streams using direct transfer to byte[] as you risk OutOfMemoryException. However in your case where you are happy to have byte[] for the ZIP and you read the first entry of the ZIP file (and are ignoring other entries) then you could try extracting the first entry as InputStream without writing to a file as follows:
// Read a zip input stream from a zip stored in byte[]:
ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream(buffer));
// Select first entry from ZIP
ZipEntry entry = zis.getNextEntry();
// You should be able to read the entry from zis directly,
// if this is text file you could test with:
// zis.transferTo(System.out);
return zis;
You should ensure that you close the stream after use.
Potential issues I can see in your code:
temp file is used as zip file, yet you delete the temp file prior to
returning. How can you use the zip file as file stream if you have
deleted it?
Do you support concurrent uploads? If yes, then you have concurrent
resource access problem. Multiple calls to create temp file:
"upload" and process it. Why don't you create a different
filename e.g. with datetime suffix + random number suffix.

Invalid zip file getting generated from a byte array

I am trying to compress set of files and storing it to memory .
Below is the code I am using
try (ByteArrayOutputStream zipBaos = new ByteArrayOutputStream();
ZipOutputStream zs = new ZipOutputStream(zipBaos)) {
Path pp = Paths.get(sourceDirPath);
Files.walk(pp)
.filter(path -> !Files.isDirectory(path) && pp.relativize(path).toString().contains(instituteId)
&& pp.relativize(path).toString().contains("dumps-" + hostCount))
.forEach(LambdaExceptionUtil.rethrowConsumer(path -> {
ZipEntry zipEntry = new ZipEntry(pp.relativize(path).toString());
try {
downloadedfilename.add(zipEntry.getName().substring(
zipEntry.getName().lastIndexOf(File.separator) + 1, zipEntry.getName().length()));
zs.putNextEntry(zipEntry);
Files.copy(path, zs);
zs.closeEntry();
} catch (IOException e) {
LOGGER.error("Exception in Zipping downloaded files {}", e);
throw e;
}
}));
return zipBaos.toByteArray();
}
}
Now later when I am trying store this byte array content again in the file system as a form of zip file
FileUtils.writeByteArrayToFile(new File(location + File.separator + name), content);
Zip file is getting created and it is showing proper size as well .
But when I am trying to open the file windows complaining it to be invalid.
Note: I can open it with 7Zip but not with the windows explorer.
Thanks.
Adding ZipOutputStream finish() and flush() resolved the issue
zs.finish();
zs.flush();
return zipBaos.toByteArray();

Java selectively copy files from a zip such that file timestamp should be preserved

I have followed following approach to decompress a zip using apache commons compress:
But since I am using OutputStream & IOUtils.copy(ais, os); (code below) to unzip and copy file, the timestamp is not preserved. Is there another way to directly copy the file from the zip such that file timestamp can be preserved.
try (ArchiveInputStream ais =
asFactory.createArchiveInputStream(
new BufferedInputStream(
new FileInputStream(archive)))) {
System.out.println("Extracting!");
ArchiveEntry ae;
while ((ae = ais.getNextEntry()) != null) {
// check if file needs to be extracted {}
if(!extract())
continue;
if (ae.isDirectory()) {
File dir = new File(archive.getParentFile(), ae.getName());
dir.mkdirs();
continue;
}
File f = new File(archive.getParentFile(), ae.getName());
File parent = f.getParentFile();
parent.mkdirs();
try (OutputStream os = new FileOutputStream(f)) {
IOUtils.copy(ais, os);
os.close();
} catch (IOException innerIoe) {
...
}
}
ais.close();
if (!archive.delete()) {
System.out.printf("Could not remove archive %s%n",
archive.getName());
archive.deleteOnExit();
}
} catch (IOException ioe) {
...
}
EDIT: With the help of jbx's answer below, following change will make it work.
IOUtils.copy(ais, os);
os.close();
outFile.setLastModified(entry.getLastModifiedTime().toMillis()); // this line
You could set the lastModifiedTime file attribute using NIO. Do it to the file exactly after you write it (after you close it). The operating system would have marked its last modified time to the current time at that point.
https://docs.oracle.com/javase/tutorial/essential/io/fileAttr.html
You will need to get the last modified time from the zip file, so maybe using NIO's
Zip Filesystem Provider` to browse and extract files from the archive would be better than your current approach (unless the APIs you are using provide you the same information).
https://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/zipfilesystemprovider.html

getOutputStream() has already been called Spring MVC

Our requirement is to download files from FTP server which we used a download files in sequential manner one after another which is taking huge time.
Sequential code:
for(String mediaValue: generatedMediaTypes) {
// Logic here is getting the user selected files from Ui and keep it in string array str
try {
ddownloadMultipleNavDbs(str[i]),uniqueID);
//now we download all the files here with downloadfile logic with help of unique id we are a folder by that name and keep all this files in that folder
} catch (Exception ex) {
logger.error(ex.getMessage());
}
}
After downloading all files into uniqueID folder now I zip and send to client by downloadZip(request,uniqueID):
public void downloadZipFile (HttpServletResponse response, String suuid) throws IOException
{
try
{
ZipOutputStream zipOutputStream = new ZipOutputStream(response.getOutputStream());
response.setContentType("application/octet-stream");
File loadableFolder = new File(uniqueID);
String timeStamp = new java.sql.Timestamp(Calendar.getInstance().getTime().getTime()).toString().replaceAll(
"-", "_").replaceAll(" ", "_").replaceAll(":", "_");
response.setHeader(
"Content-Disposition",
"attachment; filename=" + "File" + timeStamp.substring(
0, timeStamp.indexOf(".")) + ".ZIP");
File[] loadableMedias = //here we get the files from that uinqueID folder
ZipUtil.addFilesToZipStream(Medias, Folder,
zipOutputStream);
zipOutputStream.close();
response.flushBuffer();
response.getOutputStream().flush();
response.getOutputStream().close();
}
It works perfectly fine in above code.
When I used executorservice for downloading files in parallel
executorService.submit(new myThread(str[i]),uniqueID);
am facing error:
getOutputStream() has already been closed
Can anyone please explain me why am facing this error?
Why we face this error and how to resolve it?

java Extracting Zip file

I'm looking for a way to extract Zip file. So far I have tried java.util.zip and org.apache.commons.compress, but both gave a corrupted output.
Basically, the input is a ZIP file contain one single .doc file.
java.util.zip: Output corrupted.
org.apache.commons.compress: Output blank file, but with 2 mb size.
So far only the commercial software like Winrar work perfectly. Is there a java library that make use of this?
This is my method using java.util library:
public void extractZipNative(File fileZip)
{
ZipInputStream zis;
StringBuilder sb;
try {
zis = new ZipInputStream(new FileInputStream(fileZip));
ZipEntry ze = zis.getNextEntry();
byte[] buffer = new byte[(int) ze.getSize()];
FileOutputStream fos = new FileOutputStream(this.tempFolderPath+ze.getName());
int len;
while ((len=zis.read(buffer))>0)
{
fos.write(buffer);
}
fos.flush();
fos.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally
{
if (zis!=null)
{
try { zis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Many thanks,
Mike
I think your input may be compressed by some "incompatible" zip program like 7zip.
Try investigating first if it can be unpacked with a classical WinZip or such.
Javas zip handling is very well able to deal with zipped archives that come from a "compatible" zip compressor.
It is an error in my code. I need to specify the offset and len of bytes write.
it works for me
ZipFile Vanilla = new ZipFile(new File("Vanilla.zip")); //zipfile defined and needs to be in directory
Enumeration<? extends ZipEntry> entries = Vanilla.entries();// all (files)entries of zip file
while(entries.hasMoreElements()){//runs while there is files in zip
ZipEntry entry = entries.nextElement();//gets name of file in zip
File folderw =new File("tkwgter5834");//creates new directory
InputStream stream = Vanilla.getInputStream(entry);//gets input
FileInputStream inpure= new FileInputStream("Vanilla.zip");//file input stream for zip file to read bytes of file
FileOutputStream outter = new FileOutputStream(new File(folderw +"//"+ entry.toString())); //fileoutput stream creates file inside defined directory(folderw variable) by file's name
outter.write(inpure.readAllBytes());// write into files which were created
outter.close();//closes fileoutput stream
}
Have you tried jUnrar? Perhaps it might work:
https://github.com/edmund-wagner/junrar
If that doesn't work either, I guess your archive is corrupted in some way.
If you know the environment that you're going to be running this code in, I think you're much better off just making a call to the system to unzip it for you. It will be way faster than anything that you implement in java.
I wrote the code to extract a zip file with nested directories and it ran slowly and took a lot of CPU. I wound up replacing it with this:
Runtime.getRuntime().exec(String.format("unzip %s -d %s", archive.getAbsolutePath(), basePath));
That works a lot better.

Categories