I am trying to compress set of files and storing it to memory .
Below is the code I am using
try (ByteArrayOutputStream zipBaos = new ByteArrayOutputStream();
ZipOutputStream zs = new ZipOutputStream(zipBaos)) {
Path pp = Paths.get(sourceDirPath);
Files.walk(pp)
.filter(path -> !Files.isDirectory(path) && pp.relativize(path).toString().contains(instituteId)
&& pp.relativize(path).toString().contains("dumps-" + hostCount))
.forEach(LambdaExceptionUtil.rethrowConsumer(path -> {
ZipEntry zipEntry = new ZipEntry(pp.relativize(path).toString());
try {
downloadedfilename.add(zipEntry.getName().substring(
zipEntry.getName().lastIndexOf(File.separator) + 1, zipEntry.getName().length()));
zs.putNextEntry(zipEntry);
Files.copy(path, zs);
zs.closeEntry();
} catch (IOException e) {
LOGGER.error("Exception in Zipping downloaded files {}", e);
throw e;
}
}));
return zipBaos.toByteArray();
}
}
Now later when I am trying store this byte array content again in the file system as a form of zip file
FileUtils.writeByteArrayToFile(new File(location + File.separator + name), content);
Zip file is getting created and it is showing proper size as well .
But when I am trying to open the file windows complaining it to be invalid.
Note: I can open it with 7Zip but not with the windows explorer.
Thanks.
Adding ZipOutputStream finish() and flush() resolved the issue
zs.finish();
zs.flush();
return zipBaos.toByteArray();
Related
I have followed following approach to decompress a zip using apache commons compress:
But since I am using OutputStream & IOUtils.copy(ais, os); (code below) to unzip and copy file, the timestamp is not preserved. Is there another way to directly copy the file from the zip such that file timestamp can be preserved.
try (ArchiveInputStream ais =
asFactory.createArchiveInputStream(
new BufferedInputStream(
new FileInputStream(archive)))) {
System.out.println("Extracting!");
ArchiveEntry ae;
while ((ae = ais.getNextEntry()) != null) {
// check if file needs to be extracted {}
if(!extract())
continue;
if (ae.isDirectory()) {
File dir = new File(archive.getParentFile(), ae.getName());
dir.mkdirs();
continue;
}
File f = new File(archive.getParentFile(), ae.getName());
File parent = f.getParentFile();
parent.mkdirs();
try (OutputStream os = new FileOutputStream(f)) {
IOUtils.copy(ais, os);
os.close();
} catch (IOException innerIoe) {
...
}
}
ais.close();
if (!archive.delete()) {
System.out.printf("Could not remove archive %s%n",
archive.getName());
archive.deleteOnExit();
}
} catch (IOException ioe) {
...
}
EDIT: With the help of jbx's answer below, following change will make it work.
IOUtils.copy(ais, os);
os.close();
outFile.setLastModified(entry.getLastModifiedTime().toMillis()); // this line
You could set the lastModifiedTime file attribute using NIO. Do it to the file exactly after you write it (after you close it). The operating system would have marked its last modified time to the current time at that point.
https://docs.oracle.com/javase/tutorial/essential/io/fileAttr.html
You will need to get the last modified time from the zip file, so maybe using NIO's
Zip Filesystem Provider` to browse and extract files from the archive would be better than your current approach (unless the APIs you are using provide you the same information).
https://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/zipfilesystemprovider.html
i'm using the following code from the web
File dir = new File(dest.getAbsolutePath(), archiveName);
// create output directory if it doesn't exist
if (!dir.exists()) {
dir.mkdirs();
}
System.err.println(pArchivePath);
ZipFile zipFile = null;
try {
zipFile = new ZipFile(pArchivePath);
Enumeration<?> enu = zipFile.entries();
while (enu.hasMoreElements()) {
ZipEntry zipEntry = (ZipEntry) enu.nextElement();
String name = zipEntry.getName();
long size = zipEntry.getSize();
long compressedSize = zipEntry.getCompressedSize();
System.out.printf("name: %-20s | size: %6d | compressed size: %6d\n",
name, size, compressedSize);
File file = new File(name);
if (name.endsWith("/")) {
System.err.println("make dir " + name);
file.mkdirs();
continue;
}
File parent = file.getParentFile();
if (parent != null) {
parent.mkdirs();
}
InputStream is = zipFile.getInputStream(zipEntry);
FileOutputStream fos = new FileOutputStream(file);
byte[] bytes = new byte[1024];
int length;
while ((length = is.read(bytes)) >= 0) {
fos.write(bytes, 0, length);
}
is.close();
fos.close();
}
zipFile.close();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (zipFile != null) {
try {
zipFile.close();
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
to unzip an archive. It's packed with 7zip or Winrar as an .zip archive but renamed to .qtz (but i don't thing this causes the problem..)
So if i run the code to unzip my archive everything works fine: i get the output on sysout/err listing all files and also no exception occurs, but if i look in the destination directory ... it's empty - just the root folder exists.
I also used
Runtime.getRuntime().exec(String.format("unzip %s -d %s", pArchivePath, dest.getPath()));
but I can't use this anymore 'cause a new process is started and I'm continuing working on the archive right after the unzip process in the java code.
Well the question is.. why doesn't this peace of code work? There a lot of similar examples but none of them worked for me.
br, Philipp
EDIT: The following solved my Problem
File file = new File(dir.getParent(), name);
So i didn't set the right parent path for this file.
The below fragment in your code:
File parent = file.getParentFile();
if (parent != null) {
parent.mkdirs();
}
where does this create the parent directories? Because I tried your code and it's not creating in the destination directory but rather in my Eclipse's project directory. Looking at your code, the destination directory is nowhere used, right?
The code actually extracts the contents of the zip file but not where I expected it.
I am thinking because I don't see you doing something like this:
ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipPath+ "Zip.zip"));
out.putNextEntry(new ZipEntry("Results.csv"));
I am still working on it but I think that the problem because that makes the file inside of the zip
Also you should probably use the ZipOutputStream to write; like
out.write(bytes, 0, length);
I am using commons compress to zip multiple files and send it the client from a Servlet.
The files could be a combination of any type of files(text, video, audio, archives, images etc). I take the inputStream of file and write to ServletOutputStream using IOUtils.copy(is, os).
The code usually works fine for any document combination but when there is a request to download files that contain more than 1 zip, I get java.io.IOException: Closed
As a result, the zip file created is corrupted even though the size of zip is summation of individual filesizes(I am not using compression).
I tried to locally create zip and use FileOutputStream instead of response.getOutputStream() in the constructor of ZipArchiveOutputStream and it succeeds.
So, it looks like the problem exists for ServletOutputStream.
Can anyone suggest any workaround.
Here is my code :
`try (ZipArchiveOutputStream zos = new ZipArchiveOutputStream( response.getOutputStream())) {
//get fileList
for(File file : files) {
addFileToZip(zos, file.getName(), new BufferedInputStream(new FileInputStream(file)));
}
zos.close()
}
`
public static void addFileToZip(ZipArchiveOutputStream zipOutputStream, String filename, InputStream inputStream) throws FileNotFoundException {
if(zipOutputStream != null && inputStream != null) {
try {
zipOutputStream.putArchiveEntry(new ZipArchiveEntry(filename));
IOUtils.copy(inputStream, zipOutputStream);
logger.debug("fileAddedToZip :" + filename);
} catch (IOException e) {
logger.error("Error in adding file :" + filename, e);
} finally {
try {
inputStream.close();
zipOutputStream.closeArchiveEntry(); //**Starts to fail here after 1st zip is added**
} catch (IOException e) {
logger.error("Error in closing zip entry :" + filename, e);
}
}
}
`
Here is the exception trace :
`
java.io.IOException: Closed
at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:627)
at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:577)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.writeOut(ZipArchiveOutputStream.java:1287)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.writeOut(ZipArchiveOutputStream.java:1272)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.writeDataDescriptor(ZipArchiveOutputStream.java:997)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.closeArchiveEntry(ZipArchiveOutputStream.java:461)
at xxx.yyy.zzz.util.ZipUtils.addFileToZip(ZipUtils.java:110)
line 110 is zipOutputStream.closeArchiveEntry(); //**Starts to fail here after 1st zip is added**
Thanks in advance.
The problem is that you use try-with-resources which automatically closes the stream you create in it, and yet you also close it manually, and therefore when the JVM tries to auto-close it is when you get java.io.IOException: Closed exception because it is already closed.
If you use try-with-resources, you don't need to close the streams you create in it. Remove your manual zos.close() statement:
try (ZipArchiveOutputStream zos =
new ZipArchiveOutputStream(response.getOutputStream())) {
//get fileList
for(File file : files) {
addFileToZip(zos, attachment.getFileName(), is);
}
} // Here zos will be closed automatically!
Also note that once zos is closed, it will also close the servlet's underlying OutputStream so you will not be able to add further entries. You have to add all before it is closed.
I have a directory where I programmatically (in Java) do recursive unzipping (which seems to work), but in the end I'm left with a directory that has a lot of subdirectories and files. Every time I run this method I want to start with a clean slate, so I always delete the folder and its left-over files and subdirectories present in the temp directory.
root = new File(System.getProperty("java.io.tmpdir")+ File.separator + "ProductionTXOnlineCompletionDataPreProcessorRoot");
if(root.exists()){
try {
FileUtils.deleteDirectory(root);
} catch (IOException e) {
e.printStackTrace();
}
}
if(root.mkdir()){
rawFile = createRawDataFile();
}
I'm getting a really strange error from FileUtils.deleteDirectory though.
14:55:27,214 ERROR [stderr] (Thread-3 (HornetQ-client-global-threads-2098205981)) java.io.IOException: Unable to delete directory C:\Users\Admin\AppData\Local\Temp\ProductionTXOnlineCompletionDataPreProcessorRoot\ProductionTXOnlineCompletionDataPreProcessor8718674704286818303.
It seems to think that I have a period at the end of my directory (It doesn't, so it's no surprise that it can't delete it). Sometimes, this error appears on folders in the subdirectory. Has anyone seen this before?
I'm using the Commons IO 2.4 jar.
EDIT I've confirmed that the directories do not have periods, so unless they're invisible, I don't know why the method would think that there are periods. And the File's path that I give the method is set right before feeding it as an argument, and as anyone can see - it doesn't have a period at the end.
I'm running the program on Windows 7.
EDIT This is the code I used for recursively unzipping:
private void extractFolder(String zipFile) throws IOException
{
int BUFFER = 2048;
File file = new File(zipFile);
ZipFile zip = null;
String newPath = zipFile.substring(0, zipFile.length() - 4);
BufferedOutputStream dest = null;
BufferedInputStream is = null;
try{
zip = new ZipFile(zipFile);
Enumeration<? extends ZipEntry> zipFileEntries = zip.entries();
while (zipFileEntries.hasMoreElements())
{
ZipEntry entry = (ZipEntry) zipFileEntries.nextElement();
String currentEntry = entry.getName();
File destFile = new File(newPath, currentEntry);
File destinationParent = destFile.getParentFile();
destinationParent.mkdirs();
if (!entry.isDirectory())
{
is = new BufferedInputStream(zip
.getInputStream(entry));
int currentByte;
byte data[] = new byte[BUFFER];
FileOutputStream fos = new FileOutputStream(destFile);
dest = new BufferedOutputStream(fos, BUFFER);
// read and write until last byte is encountered
while ((currentByte = is.read(data, 0, BUFFER)) != -1) {
dest.write(data, 0, currentByte);
}
dest.flush();
}
if (currentEntry.endsWith(".zip")){
// found a zip file, try to open
extractFolder(destFile.getAbsolutePath());
}
}
}catch(Exception e){
e.printStackTrace();
}finally{
if(dest!=null) {dest.close();}
if(is!=null) {is.close();}
zip.close();
}
}
I put the original zip into the root directory, and recursively unzip from there.
This is the relevant code showing that:
downloadInputStreamToFileInRootDir(in, rawFile);
try {
extractFolder(rawFile.getCanonicalPath());
} catch (IOException e) {
e.printStackTrace();
}catch (Exception e){
}
I just noticed that I use rawFile.getCanonicalPath() (rawFile is set in the first code excerpt) as the argument for extractFolder initially and then switch to destFile.getAbsolutePath() ... Maybe that has something to do with it. The problem with testing this is that the issue isn't deterministic. It sometimes happens and sometimes not.
The period is part of the error message. It's not trying to delete a file path with a period at the end. See the FileUtils source:
if (!directory.delete()) {
final String message =
"Unable to delete directory " + directory + ".";
throw new IOException(message);
}
This question already has answers here:
How to create Uncompressed Zip archive in Java
(4 answers)
Closed 6 years ago.
I'm trying to compress directory content into zip archive using java.
Everything is fine, but I just want to clarify some facts.
Here is the code which I use to compress files:
public void pack(#Nonnull String archiveName, #Nonnull File outputDir, #Nonnull File targetDir) {
File zipFile = new File(outputDir, "out.zip");
ZipOutputStream zipOutputStream = null;
OutputStream outputStream;
try {
// create stream for writing zip archive
outputStream = new FileOutputStream(zipFile);
zipOutputStream = new ZipOutputStream(outputStream);
// write files recursively
writeFiles(zipOutputStream, targetDir.listFiles(), "");
} catch (IOException e) {
LOGGER.error("IO exception while packing files to archive", e);
} finally {
// close output streams
if (zipOutputStream != null) {
try {
zipOutputStream.close();
} catch (IOException e) {
LOGGER.error("Unable to close zip output stream", e);
}
}
}
}
/**
* Writes specified files and their children (in case of directories) to archive
*
* #param zipOutputStream archive output stream
* #param files which should be added to archive
* #param path path relative of root of archive where files should be placed
*/
private void writeFiles(#Nonnull ZipOutputStream zipOutputStream, #Nullable File[] files, #Nonnull String path) throws IOException {
if (files == null || files.length == 0) {
return;
}
for (File file : files) {
if (file.isDirectory()) {
// recursively add files in this directory
String fullDirectoryName = path + file.getName() + "/";
File[] childFiles = file.listFiles();
if (childFiles != null && childFiles.length > 0) {
// write child files to archive. current directory will be created automatically
writeFiles(zipOutputStream, childFiles, fullDirectoryName);
} else {
// empty directory. write directory itself to archive
ZipEntry entry = new ZipEntry(fullDirectoryName);
zipOutputStream.putNextEntry(entry);
zipOutputStream.closeEntry();
}
} else {
// put file in archive
BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(file));
zipOutputStream.putNextEntry(new ZipEntry(path + file.getName()));
ByteStreams.copy(bufferedInputStream, zipOutputStream);
zipOutputStream.closeEntry();
bufferedInputStream.close();
}
}
}
Now there are the questions:
Is it correct that by default (and in my case too) I will get already compressed archive (using Deflate method)?
How to get uncompressed archive:
If I set method zipOutputStream.setMethod(ZipOutputStream.STORED) I have to provide size, compressed size (is it will be equal to size?) and crc, otherwise I will get exceptions
If I don't want to calculate size and crc by myself I can use DEFLATE method with zero level:
zipOutputStream.setMethod(ZipOutputStream.DEFLATED);
zipOutputStream.setLevel(ZipOutputStream.STORED);So, is it correct that in this case I get not compressed archive at all?
Is there more convenient-obvious method to creating not-compressed archives?
Rather than re-invent the wheel I'd seriously consider using an existing library for this, such as Apache Ant. The basic idiom for creating a zip file is:
Project p = new Project();
p.init();
Zip zip = new Zip();
zip.setProject(p);
zip.setDestFile(new File(outputDir, "out.zip"));
FileSet fs = new FileSet();
fs.setProject(p);
fs.setDirectory(targetDir);
zip.addFileset(fs);
zip.perform();
By default you will get a compressed archive. For an uncompressed zip all you need to add is
zip.setCompress(false);
after the setDestFile (in fact anywhere before the perform).