I need to package a few files (total upto 4 GB in size) which will be available online. An android app needs to download this 'on the fly' without saving the archive on to the device. So basically the device won't save the archive and then unpack, as it would require double the space. Which package format should I choose that will support it (eg. zip, tar.gz etc.)?
Use .zip! You can use ZipInputStream and ZipOutputStream to read and write from .zip files on the fly. No need to extract the files from the archive.
Link to documentation
And here is a quick example:
InputStream is =...
ZipInputStream zis = new ZipInputStream(new BufferedInputStream(is));
try {
// ZipEntry contains data about files and folders in the archive.
ZipEntry ze;
// This loops through the whole content of the archive
while ((ze = zis.getNextEntry()) != null) {
// Here we read the whole data of one ZipEntry
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int count;
while ((count = zis.read(buffer)) != -1) {
baos.write(buffer, 0, count);
}
// The ZipEntry contains data about the file, like its filename
String filename = ze.getName();
// And that's the file itself as byte array
byte[] bytes = baos.toByteArray();
// Do something with the file
}
} finally {
zis.close();
}
Related
I converted a zip file of 5 files into a byte array. I want to output a zip file on my disk from that bytearray. My process was first reading the byte[] into a ByteArrayInputStream then into a ZipInputStream.
InputStream plainTextStream = new ByteArrayInputStream(plainText);
ZipInputStream zipInStream = new ZipInputStream(plainTextStream);
I want this to be outputted into a zip file on my disk so here I thought I will need a file and a ZipOutPutStream passing that zip file.
ZipOutputStream zipOutStream = new ZipOutputStream(new FileOutputStream(file));
With a zip entry I traversed the ZipInPutStream writing to a FileOutputStream each entry, using a buffer. At the end of each main loop I put an entry into the ZipOutPutStream.
ZipEntry entry = null;
while((entry = zipInStream.getNextEntry()) != null){
FileOutputStream fileOutStream = new FileOutputStream(entry.getName());
byte[] byteBuff = new byte[1024];
int bytesRead = 0;
while ((bytesRead = zipInStream.read(byteBuff)) != -1)
{
fileOutStream.write(byteBuff, 0, bytesRead);
}
fileOutStream.close();
zipOutStream.putNextEntry(entry);
zipInStream.closeEntry();
}
I add the first file from the zip (there are 5 files), but when trying to add the 2nd file, I get an error on
zipOutStream.putNextEntry(entry)
java.util.zip.ZipException: invalid entry size (expected 18401 but got 0 bytes)
Through debugging I can't figure out where it goes wrong. I assume it may have something to do with the buffer when putting in the first outputstream(entry.getName())? The bytesRead while loop could be an issue.
This is all assuming the logic makes sense. I hope I can approach a solution to this error.
You never write the content of the zipped files to the zip output stream.
You don't need to write the output to a file stream, just write it directly to the zip output stream.
You should be using try-with-resources.
try (ZipInputStream zipInStream = new ZipInputStream(new ByteArrayInputStream(plainText));
ZipOutputStream zipOutStream = new ZipOutputStream(new FileOutputStream(file));
) {
byte[] byteBuff = new byte[1024];
for (ZipEntry entry; (entry = zipInStream.getNextEntry()) != null; ) {
zipOutStream.putNextEntry(entry);
for (int bytesRead; (bytesRead = zipInStream.read(byteBuff)) != -1; ) {
zipOutStream.write(byteBuff, 0, bytesRead);
}
}
}
There is no need to call closeEntry().
To resolve (expected 18401 but got 0 bytes)
Create a new blank excel file.
Copy data from the file you copied from zip to the new created file in step 1.
Use the new file, it should work as its worked for me.
Thanks.
Is there a way to use ZipOutputStream (from java.util.zip) with a password?
I'm working on copying files from a (unencrypted) zip file into another encrypted zipfile. Using ZipInpuStream/ZipOutputStream it is possible to read from the original zip file into the target zip file but I haven't seen a recipe that talks about how to use password on the ZipOutputStream.
ZipInputStream zis = new ZipInputStream(new FileInputStream(inputFile));
ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(outputFile));
byte[] buff = new byte[1024];
while (true) {
int bytesRead = zis.read(buff, 0, 1024);
if (bytesRead < 0) {
// finished
break;
}
zos.write(buff, 0, bytesRead);
}
zos.flush();
zos.close();
zis.close();
I'm afraid it's difficult to set a password for Java.util.zip, Try to use zip4j https://github.com/srikanth-lingala/zip4j
Follow up of Question: Java: how to compress a byte[] using ZipOutputStream without intermediate file
I can zip data without an intermediate file (or memory file). I now need to zip chunks of data and add them to a single zip file.
I am using a single ZipOutputStream as suggested in the previous question.
String infile = "test.txt";
FileInputStream in = new FileInputStream(infile);
String outfile = "test.txt.zip";
FileOutputStream out = new FileOutputStream(outfile);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(baos);
ZipEntry entry = new ZipEntry("test_unzip.txt");
entry.setSize(2048);
zos.putNextEntry(entry);
int len = 0;
while (len > -1) {
byte[] buf = new byte[10];
len = in.read(buf);
zos.write(buf);
out.write(baos.toByteArray());
baos.reset();
}
zos.closeEntry();
zos.finish();
zos.close();
in.close();
out.close();
I have tried different sizes for buf, reordering zos.finish and zos.closeEntry, and also tried with and without baos.reset.
I have also tried reading the entire contents of infile into a single buf but still not working.
I expected a valid .zip file that will unzip into test_unzip.txt.
However, when i try unzip test.txt.zip on my command line i get the following error:
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of test.txt.zip or
test.txt.zip.zip, and cannot find test.txt.zip.ZIP, period.
I'm trying to read .srt files that are located in zip file itself located in a zip file. I succeed to read .srt files that were in a simple zip with the extract of code below :
for (Enumeration enume = fis.entries(); enume.hasMoreElements();) {
ZipEntry entry = (ZipEntry) enume.nextElement();
fileName = entry.toString().substring(0,entry.toString().length()-4);
try {
InputStream in = fis.getInputStream(entry);
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
String ext = entry.toString().substring(entry.toString().length()-4, entry.toString().length());
But now i don't know how i could get to the zip file inside the zip file.
I tried using ZipFile fis = new ZipFile(filePath) with filePath being the path of the zip file + the name of zip file inside. It didn't recognize the path so i don't know if i am clear.
Thanks.
ZipFile only works with real files, because it's intended for use as a random access mechanism which needs to be able to seek directly to specific locations in the file to read entries by name. But as VGR suggests in the comments, while you can't get random access to the zip-inside-a-zip you can use ZipInputStream, which provides strictly sequential access to the entries and works with any InputStream of zip-format data.
However, ZipInputStream has a slightly odd usage pattern compared to other streams - calling getNextEntry reads the entry metadata and positions the stream to read that entry's data, you read from the ZipInputStream until it reports EOF, then you (optionally) call closeEntry() before moving on to the next entry in the stream.
The critical point is that you must not close() the ZipInputStream until you have finished reading the final entry, so depending what you want to do with the entry data you might need to use something like the commons-io CloseShieldInputStream to guard against the stream getting closed prematurely.
try(ZipInputStream outerZip = new ZipInputStream(fis)) {
ZipEntry outerEntry = null;
while((outerEntry = outerZip.getNextEntry()) != null) {
if(outerEntry.getName().endsWith(".zip")) {
try(ZipInputStream innerZip = new ZipInputStream(
new CloseShieldInputStream(outerZip))) {
ZipEntry innerEntry = null;
while((innerEntry = innerZip.getNextEntry()) != null) {
if(innerEntry.getName().endsWith(".srt")) {
// read the data from the innerZip stream
}
}
}
}
}
}
Find the code to extract .zip files recursively:
public void extractFolder(String zipFile) throws ZipException, IOException {
System.out.println(zipFile);
int BUFFER = 2048;
File file = new File(zipFile);
ZipFile zip = new ZipFile(file);
String newPath = zipFile.substring(0, zipFile.length() - 4);
new File(newPath).mkdir();
Enumeration zipFileEntries = zip.entries();
// Process each entry
while (zipFileEntries.hasMoreElements())
{
// grab a zip file entry
ZipEntry entry = (ZipEntry) zipFileEntries.nextElement();
String currentEntry = entry.getName();
File destFile = new File(newPath, currentEntry);
//destFile = new File(newPath, destFile.getName());
File destinationParent = destFile.getParentFile();
// create the parent directory structure if needed
destinationParent.mkdirs();
if (!entry.isDirectory())
{
BufferedInputStream is = new BufferedInputStream(zip
.getInputStream(entry));
int currentByte;
// establish buffer for writing file
byte data[] = new byte[BUFFER];
// write the current file to disk
FileOutputStream fos = new FileOutputStream(destFile);
BufferedOutputStream dest = new BufferedOutputStream(fos,
BUFFER);
// read and write until last byte is encountered
while ((currentByte = is.read(data, 0, BUFFER)) != -1) {
dest.write(data, 0, currentByte);
}
dest.flush();
dest.close();
is.close();
}
if (currentEntry.endsWith(".zip"))
{
// found a zip file, try to open
extractFolder(destFile.getAbsolutePath());
}
}
}
I am seeking for most efficient way (in terms of speed) to retrieve some file out of the middle of a ZIP file.
e.g. I have ZIP file, which includes 700 folders (tagged 1 to 700). Each folder equals picture and mp3 file. There is special folder called Info, which contains XML file. Problem is, I need to iterate through this ZIP file to find XML file and then I am displaying images from desired folders. I am using ZipFile approach (thus I am iterating through whole ZIP file, even if I want folder 666, I need to go through 665 items in ZIP file) -> selecting from ZIP file is extremely slow.
I would like to ask you, If you have faced similar issue, how have you solved this? Is there any approach in Java, which turns my ZIP file into virtual folder to browse it much more quicker? Is there any external library, which is the most efficient in terms of time?
Source Code snippet:
try {
FileInputStream fin = new FileInputStream(
"sdcard/external_sd/mtp_data/poi_data/data.zip");
ZipInputStream zin = new ZipInputStream(fin);
ZipEntry ze = null;
while ((ze = zin.getNextEntry()) != null) {
// Log.d("ZE", ze.getName());
if (ze.getName().startsWith("body/665/")) {
// Log.d("FILE F", "soubor: "+ze.getName());
if (ze.getName().endsWith(".jpg")
|| ze.getName().endsWith(".JPG")) {
Log.d("OBR", "picture: " + ze.getName());
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int count;
while ((count = zin.read(buffer)) != -1) {
baos.write(buffer, 0, count);
}
byte[] bytes = baos.toByteArray();
bmp = BitmapFactory.decodeByteArray(bytes, 0,
bytes.length);
photoField.add(bmp);
i++;
}
}
}
}
The ZipFile.getEntry() and ZipFile.getInputStream() methods can be used to access a specific file in a ZIP archive. For example:
ZipFile file = ...
ZipEntry entry = file.getEntry("folder1/picture.jpg");
InputStream in = file.getInputStream(entry);