We Keep Coding

Create a directory set of files to multiple zips?

I want to convert my directory(i.e. too large) which i want to convert to multiple zip files . Right now i can able to convert all files in the dirctory to one zip file using the below code .
package bulkimport;
import java.io.*;
import java.util.zip.*;
public class Tzip {
static final int BUFFER = 2048;
public static void main (String argv[]) {
try {
BufferedInputStream origin = null;
FileOutputStream dest = new
FileOutputStream("D:/abc/file.zip");
ZipOutputStream out = new ZipOutputStream(new
BufferedOutputStream(dest));
//out.setMethod(ZipOutputStream.DEFLATED);
byte data[] = new byte[BUFFER];
// get a list of files from current directory
File f = new File("D://current//");
String files[] = f.list();
for (int i=0; i<files.length; i++) {
System.out.println("Adding: "+files[i]);
FileInputStream fi = new
FileInputStream("D://current//"+files[i]);
origin = new
BufferedInputStream(fi, BUFFER);
ZipEntry entry = new ZipEntry(files[i]);
out.putNextEntry(entry);
int count;
while((count = origin.read(data, 0,
BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
}
out.close();
} catch(Exception e) {
e.printStackTrace();
}
}
}
What i need to do to above code in such a way that it will create multiple zip files based on our condition(i.e.By setting threshold as 50MB it will create multiple zip files of each file having 50MB of compressed data )
Any help is appreciated
Thanks

How to get .png files alone from one folder

Actually, i'm trying to zip all the files from one folder & .png files from another folder. I can able to get all the files from one folder. But i can't able to get the .png files from another folder in java. Is there any way ?
Code:
public class Zip {
public static void zip(String filepath,String reportFileName){
try {
File inFolder=new File(filepath);
File inFolder1=new File("../Agent_Portal_Auto_Testing/ReportCharts");
File outFolder=new File(reportFileName);
ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(new FileOutputStream(outFolder)));
BufferedInputStream in = null;
byte[] data = new byte[1000];
String files[] = inFolder.list();
String files1[]=inFolder1.list();
for (int i=0; i<files.length; i++) {
in = new BufferedInputStream(new FileInputStream
(inFolder.getPath() + "/" + files[i]), 1000);
out.putNextEntry(new ZipEntry(files[i]));
int count;
while((count = in.read(data,0,1000)) != -1) {
out.write(data, 0, count);
}
}
for (int i=0; i<files1.length; i++) {
in = new BufferedInputStream(new FileInputStream
(inFolder1.getPath() + "/" + files1[i]), 1000);
out.putNextEntry(new ZipEntry(files1[i]));
int count;
while((count = in.read(data,0,1000)) != -1) {
out.write(data, 0, count);
}
}
out.closeEntry();
out.flush();
out.close();
}
catch(Exception e) {
e.printStackTrace();
}
}
}
In the above code, i'm getting all the files from ReportCharts folder. But i need to get only the .png files.

See http://docs.oracle.com/javase/7/docs/api/java/io/File.html#listFiles(java.io.FileFilter)
You can use the file filter to filter out only the PNG files
http://docs.oracle.com/javase/7/docs/api/java/io/FileFilter.html
File [] pngFiles = directory.listFiles(new FileFilter() {
public boolean accept(File file) {
return file.isFile() && file.getName().toLowerCase().endsWith(".png");
}
});

you can add verify if you file is a .png one with :
if (files1[i].contains(".png"))
in your for loop.

A better way to convert a directory of files into bytes

I have been messing with this for some time and it's getting better and better, but it's still a little slow for me. Can anyone help speed this up / make the design better, please?
Also, the files must only be numbers and the file must end with the file extension ".dat"
I never added the checks because I didn't feel is was necessary.
public void preloadModels() {
try {
File directory = new File(signlink.findcachedir() + "raw", File.separator);
File[] modelFiles = directory.listFiles();
for (int modelIndex = modelFiles.length - 1;; modelIndex--) {
String modelFileName = modelFiles[modelIndex].getName();
byte[] buffer = getBytesFromInputStream(new FileInputStream(new File(directory, modelFileName)));
Model.method460(buffer, Integer.parseInt(modelFileName.replace(".dat", "")));
}
} catch (Throwable e) {
return;
}
}
public static final byte[] getBytesFromInputStream(InputStream inputStream) throws IOException {
byte[] buffer = new byte[32 * 1024];
int bufferSize = 0;
for (;;) {
int read = inputStream.read(buffer, bufferSize, buffer.length - bufferSize);
if (read == -1) {
return Arrays.copyOf(buffer, bufferSize);
}
bufferSize += read;
if (bufferSize == buffer.length) {
buffer = Arrays.copyOf(buffer, bufferSize * 2);
}
}
}

I would do the following.
public void preloadModels() throws IOException {
File directory = new File(signlink.findcachedir() + "raw");
for (File file : directory.listFiles()) {
if (!file.getName().endsWith(".dat")) continue;
byte[] buffer = getBytesFromFile(file);
Model.method460(buffer, Integer.parseInt(file.getName().replace(".dat", "")));
}
}
public static byte[] getBytesFromFile(File file) throws IOException {
byte[] buffer = new byte[(int) file.length()];
try (DataInputStream dis = new DataInputStream(new FileInputStream(file))) {
dis.readFully(buffer);
return buffer;
}
}
If this is still too slow, most likely the limitation is the speed of hard drive.

How about using Apache Commons IOUtils class.
IOUtils.toByteArray(InputStream input)

I think the easiest way is to add all directories content to archive. Have a look at java.util.zip. It has some bugs with file names before 7th version. There is also Apache Commons implementation.

Modifying a text file in a ZIP archive in Java

My use case requires me to open a txt file, say abc.txt which is inside a zip archive which contains key-value pairs in the form
key1=value1
key2=value2
.. and so on where each key-value pair is in a new line.
I have to change one value corresponding to a certain key and put the text file back in a new copy of the archive. How do I do this in java?
My attempt so far:
ZipFile zipFile = new ZipFile("test.zip");
final ZipOutputStream zos = new ZipOutputStream(new FileOutputStream("out.zip"));
for(Enumeration e = zipFile.entries(); e.hasMoreElements(); ) {
ZipEntry entryIn = (ZipEntry) e.nextElement();
if(!entryIn.getName().equalsIgnoreCase("abc.txt")){
zos.putNextEntry(entryIn);
InputStream is = zipFile.getInputStream(entryIn);
byte [] buf = new byte[1024];
int len;
while((len = (is.read(buf))) > 0) {
zos.write(buf, 0, len);
}
}
else{
// I'm not sure what to do here
// Tried a few things and the file gets corrupt
}
zos.closeEntry();
}
zos.close();

Java 7 introduced a much simpler way for doing zip archive manipulations - FileSystems API, which allows to access contents of a file as a file system.
Besides much more straightforward API, it is doing the modification in-place and doesn't require to rewrite other (irrelevant) files in a zip archive (as done in the accepted answer).
Here's sample code that solves OP's use case:
import java.io.*;
import java.nio.file.*;
public static void main(String[] args) throws IOException {
modifyTextFileInZip("test.zip");
}
static void modifyTextFileInZip(String zipPath) throws IOException {
Path zipFilePath = Paths.get(zipPath);
try (FileSystem fs = FileSystems.newFileSystem(zipFilePath, null)) {
Path source = fs.getPath("/abc.txt");
Path temp = fs.getPath("/___abc___.txt");
if (Files.exists(temp)) {
throw new IOException("temp file exists, generate another name");
}
Files.move(source, temp);
streamCopy(temp, source);
Files.delete(temp);
}
}
static void streamCopy(Path src, Path dst) throws IOException {
try (BufferedReader br = new BufferedReader(
new InputStreamReader(Files.newInputStream(src)));
BufferedWriter bw = new BufferedWriter(
new OutputStreamWriter(Files.newOutputStream(dst)))) {
String line;
while ((line = br.readLine()) != null) {
line = line.replace("key1=value1", "key1=value2");
bw.write(line);
bw.newLine();
}
}
}
For more zip archive manipulation examples, see demo/nio/zipfs/Demo.java sample which you can download here (look for JDK 8 Demos and Samples).

You had almost got it right. One possible reason, the file was shown as corrupted is that you might have used
zos.putNextEntry(entryIn)
in the else part as well. This creates a new entry in the zip file containing information from the existing zip file. Existing information contains entry name(file name) and its CRC among other things.
And then, when u try to update the text file and close the zip file, it will throw an error as the CRC defined in the entry and the CRC of the object you are trying to write differ.
Also u might get an error if the length of the text that you are trying to replace is different than the one existing i.e. you are trying to replace
key1=value1
with
key1=val1
This boils down to the problem that the buffer you are trying to write to has length different than the one specified.
ZipFile zipFile = new ZipFile("test.zip");
final ZipOutputStream zos = new ZipOutputStream(new FileOutputStream("out.zip"));
for(Enumeration e = zipFile.entries(); e.hasMoreElements(); ) {
ZipEntry entryIn = (ZipEntry) e.nextElement();
if (!entryIn.getName().equalsIgnoreCase("abc.txt")) {
zos.putNextEntry(entryIn);
InputStream is = zipFile.getInputStream(entryIn);
byte[] buf = new byte[1024];
int len;
while((len = is.read(buf)) > 0) {
zos.write(buf, 0, len);
}
}
else{
zos.putNextEntry(new ZipEntry("abc.txt"));
InputStream is = zipFile.getInputStream(entryIn);
byte[] buf = new byte[1024];
int len;
while ((len = (is.read(buf))) > 0) {
String s = new String(buf);
if (s.contains("key1=value1")) {
buf = s.replaceAll("key1=value1", "key1=val2").getBytes();
}
zos.write(buf, 0, (len < buf.length) ? len : buf.length);
}
}
zos.closeEntry();
}
zos.close();
The following code ensures that even if data that is replaced is of less length than the original length, no IndexOutOfBoundsExceptions occur.
(len < buf.length) ? len : buf.length

Only a little improvement to:
else{
zos.putNextEntry(new ZipEntry("abc.txt"));
InputStream is = zipFile.getInputStream(entryIn);
byte[] buf = new byte[1024];
int len;
while ((len = (is.read(buf))) > 0) {
String s = new String(buf);
if (s.contains("key1=value1")) {
buf = s.replaceAll("key1=value1", "key1=val2").getBytes();
}
zos.write(buf, 0, (len < buf.length) ? len : buf.length);
}
}
That should be:
else{
zos.putNextEntry(new ZipEntry("abc.txt"));
InputStream is = zipFile.getInputStream(entryIn);
long size = entry.getSize();
if (size > Integer.MAX_VALUE) {
throw new IllegalStateException("...");
}
byte[] bytes = new byte[(int)size];
is.read(bytes);
zos.write(new String(bytes).replaceAll("key1=value1", "key1=val2").getBytes());
}
In order to capture all the occurrences
The reason is that, with the first, you could have "key1" in one read and "=value1" in the next, not being able to capture the occurrence you want to change

Issues with zipping files in Java

I am currently trying to zip all files within a directory.
The zip file is being created and the files are being processed - but for some reason the files are not appearing within the zip file.
The code being used to complete this task is as follows:
public class FileZipper {
public void zipDir( String dir, String zipFileName ) {
try{
File dirObj = new File(dir);
ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipFileName));
Logger.info("Creating : " + zipFileName);
addDir(dirObj, out);
out.close();
}
catch (Exception e){
Logger.error( e, "Error zipping directory" );
}
}
private void addDir(File dirObj, ZipOutputStream out) throws IOException {
File[] files;
if( !dirObj.isDirectory() ){
files = new File[] { dirObj };
}
else{
files = dirObj.listFiles();
}
byte[] tmpBuf = new byte[1024];
for (int i = 0; i < files.length; i++) {
if (files[i].isDirectory()) {
addDir(files[i], out);
continue;
}
FileInputStream in = new FileInputStream(files[i].getAbsolutePath());
Logger.info(" Adding: " + files[i].getAbsolutePath());
out.putNextEntry(new ZipEntry(files[i].getAbsolutePath()));
int len;
while ((len = in.read(tmpBuf)) > 0) {
out.write(tmpBuf, 0, len);
}
out.closeEntry();
in.close();
}
}
}
When reviewing the logging information, the files within the directories are being found and processed, but the created zip file contains no data.
Any help with this issues will be greatly appreciated.
Thanks

Apart from the fact that adding the file by its absolute path is probably not what you want, this code works just fine for me.

Hy,
Give a set of files name to this function, and a zip name.
It should work.
private void zipFiles (ArrayList<String> listWithFiles, String zipName) {
try {
byte[] buffer = new byte[1024];
// create object of FileOutputStream
FileOutputStream fout = new FileOutputStream(zipName);
// create object of ZipOutputStream from FileOutputStream
ZipOutputStream zout = new ZipOutputStream(fout);
for (String currentFile : listWithFiles) {
// create object of FileInputStream for source file
FileInputStream fin = new FileInputStream(currentFile);
// add files to ZIP
zout.putNextEntry(new ZipEntry(currentFile ));
// write file content
int length;
while ((length = fin.read(buffer)) > 0) {
zout.write(buffer, 0, length);
}
zout.closeEntry();
// close the InputStream
fin.close();
}
// close the ZipOutputStream
zout.close();
} catch (IOException ioe) {
System.out.println("IOException :" + ioe);
}
}
All good to you,
dAN