We Keep Coding

Java Code to Zip/FTP Directory in a UNIX Server

I want to write a couple of methods in a Java package, which would be deployed in a UNIX Server.
As of now, my code was for Windows Server, for which I used the following code to zip Directory.
public static final void zipDirectory(File fBatchDirectory, String batchName, String ondemandDocExtension) throws IOException
{
//Set zip file name
File zip = new File(fBatchDirectory + "\\" + StringUtils.replace(batchName,".ind", "") + ".zip");
//filter file
FileFilter filter = new FileFilter(ondemandDocExtension);
File[] files = fBatchDirectory.listFiles(filter);
if(files.length > 0)
{
ZipOutputStream zos = new ZipOutputStream( new FileOutputStream(zip) );
zip(files, fBatchDirectory, zos , ondemandDocExtension);
zos.close();
}
}
private static final void zip(File[] files, File base,ZipOutputStream zos , String docExtension) throws IOException
{
byte[] buffer = new byte[8192];
int read = 0;
for (int i = 0, n = files.length; i < n; i++)
{
//Add to zip only if its file
if (files[i].isFile())
{
FileInputStream in = new FileInputStream(files[i]);
ZipEntry entry = new ZipEntry(files[i].getPath().substring(base.getPath().length() + 1));
zos.putNextEntry(entry);
while (-1 != (read = in.read(buffer)))
{
zos.write(buffer, 0, read);
}
in.close();
}
}
}
I am confused as to how to replicate the same functionality to zip Directory in Java, for UNIX?
And then I want to FTP the files from a UNIX Serve to another UNIX Server.
Any pointers would be greatly appreciated.

At a first glance, the only problem I see is at this line:
File zip = new File(fBatchDirectory + "\\" + StringUtils.replace(batchName,".ind", "") + ".zip");
Because you are explicitly using the double backslash (\\) in your filename. If you change that for File.separator your code should work for both operating systems:
File zip = new File(fBatchDirectory + File.separator + StringUtils.replace(batchName,".ind", "") + ".zip");
For the FTP part of it, you can get down and dirty and use an FTP client or use a more high level library like Apache Commons VFS which, by the way, inspired the new IO FileSystem API in Java 7, but I don't now about any library implementing the FTP protocol with the new API at the moment.

Zipping Files using util.zip No directory

I have the following situation:
I am able to zip my files with the following method:
public boolean generateZip(){
byte[] application = new byte[100000];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
// These are the files to include in the ZIP file
String[] filenames = new String[]{"/subdirectory/index.html", "/subdirectory/webindex.html"};
// Create a buffer for reading the files
try {
// Create the ZIP file
ZipOutputStream out = new ZipOutputStream(baos);
// Compress the files
for (int i=0; i<filenames.length; i++) {
byte[] filedata = VirtualFile.fromRelativePath(filenames[i]).content();
ByteArrayInputStream in = new ByteArrayInputStream(filedata);
// Add ZIP entry to output stream.
out.putNextEntry(new ZipEntry(filenames[i]));
// Transfer bytes from the file to the ZIP file
int len;
while ((len = in.read(application)) > 0) {
out.write(application, 0, len);
}
// Complete the entry
out.closeEntry();
in.close();
}
// Complete the ZIP file
out.close();
} catch (IOException e) {
System.out.println("There was an error generating ZIP.");
e.printStackTrace();
}
downloadzip(baos.toByteArray());
}
This works perfectly and I can download the xy.zip which contains the following directory and file structure:
subdirectory/
----index.html
----webindex.html
My aim is to completely leave out the subdirectory and the zip should only contain the two files. Is there any way to achieve this?
(I am using Java on Google App Engine).
Thanks in advance

If you are sure the files contained in the filenames array are unique if you leave out the directory, change your line for constructing ZipEntrys:
String zipEntryName = new File(filenames[i]).getName();
out.putNextEntry(new ZipEntry(zipEntryName));
This uses java.io.File#getName()

You can use Apache Commons io to list all your files, then read them to an InputStream
Replace the line below
String[] filenames = new String[]{"/subdirectory/index.html", "/subdirectory/webindex.html"}
with the following
Collection<File> files = FileUtils.listFiles(new File("/subdirectory"), new String[]{"html"}, true);
for (File file : files)
{
FileInputStream fileStream = new FileInputStream(file);
byte[] filedata = IOUtils.toByteArray(fileStream);
//From here you can proceed with your zipping.
}
Let me know if you have issues.

You could use the isDirectory() method on VirtualFile

Whats wrong with this zip method?

I have a method which zips up 5 files. It produces a zip file without error, but I cannot open it to examine the contents. I tried emailing it and gmail said it cannot send corrupt files. Trying to open with WinRAR in Windows results in an error stating:
The archive is either in unknown format or damaged
This is the method:
private void zipTestFiles() throws FileNotFoundException, IOException
{
File[] filenames = fileDir.listFiles(fileNameFilter(Constants.PAGE_MON_FILENAME_FILTER));
byte[] buf = new byte[1024];
String outFilename = Environment.getExternalStorageDirectory().getAbsolutePath() + File.separatorChar + Constants.PAGEMONITOR_ZIP;
DeflaterOutputStream out = new DeflaterOutputStream(new BufferedOutputStream(new FileOutputStream(outFilename)));
for (int i=0; i<filenames.length; i++)
{
FileInputStream in = new FileInputStream(filenames[i]);
int len;
while ((len = in.read(buf)) > 0)
{
out.write(buf, 0, len);
}
in.close();
}
out.close();
}

You should use ZipOutputStream instead of DeflaterOutputStream. And do not forget to create entries. Read javadoc of ZipOutputStream before writing the implementation.

Try with ZipOutputStream which already exists in Java. DeflaterOutputStream only uses DEFLATE method to compress but doesn't put ZIP headers automatically.

java gzip can't keep original file's extension name

I'm using GZIPOutputStream to gzip one xml file to gz file, but after zipping I find the extension name of the xml file (.xml) is missing in the gz file hierarchy. I need to keep the extension name because the zipped gz file will be used by third party system which expects getting a .xml file after unzipping gz file. Are there any solutions for this? My test code is:
public static void main(String[] args) {
compress("D://test.xml", "D://test.gz");
}
private static boolean compress(String inputFileName, String targetFileName){
boolean compressResult=true;
int BUFFER = 1024*4;
byte[] B_ARRAY = new byte[BUFFER];
FileInputStream fins=null;
FileOutputStream fout=null;
GZIPOutputStream zout=null;
try{
File srcFile=new File(inputFileName);
fins=new FileInputStream (srcFile);
File tatgetFile=new File(targetFileName);
fout = new FileOutputStream(tatgetFile);
zout = new GZIPOutputStream(fout);
int number = 0;
while((number = fins.read(B_ARRAY, 0, BUFFER)) != -1){
zout.write(B_ARRAY, 0, number);
}
}catch(Exception e){
e.printStackTrace();
compressResult=false;
}finally{
try {
zout.close();
fout.close();
fins.close();
} catch (IOException e) {
e.printStackTrace();
compressResult=false;
}
}
return compressResult;
}

Maybe I'm missing something, but when I've gzipped files in the past, say test.xml, the output I get would be test.xml.gz. Perhaps if you changed the output filename to test.xml.tz you would still preserve your original file extension.

Not sure what the problem is here, you are calling your own compress function
private static boolean compress(String inputFileName, String targetFileName)
with the following arguments
compress("D://test.xml", "D://test.gz");
Quite obviously you are going to lose the .xml portion of the filename, you never pass it into your method.

Your code is perfectly fine. give the output file names as "D://test.xml.gz" you missed the file extension(.xml).
Ex: compress("D://test.xml", "D://test.xml.gz");

You can also use an ArchiveOutput stream (like Tar) before GZipping it.

Use the ZipOutputStream with ZipEntry instead of GZipOutputStream. so that it will keep the original file extension.
Sample code as below..
ZipOutputStream zipOutStream = new ZipOutputStream(new FileOutputStream(zipFile));
FileInputStream inStream = new FileInputStream(file); // Stream to read file
ZipEntry entry = new ZipEntry(file.getPath()); // Make a ZipEntry
zipOutStream.putNextEntry(entry); // Store entry

I created a copy of GZIPOutputStream and changed the code to allow for a different filename "in the gzip":
private final byte[] header = {
(byte) GZIP_MAGIC, // Magic number (short)
(byte)(GZIP_MAGIC >> 8), // Magic number (short)
Deflater.DEFLATED, // Compression method (CM)
8, // Flags (FLG)
0, // Modification time MTIME (int)
0, // Modification time MTIME (int)
0, // Modification time MTIME (int)
0, // Modification time MTIME (int)
0, // Extra flags (XFLG)
0 // Operating system (OS)
};
private void writeHeader() throws IOException {
out.write(header);
out.write("myinternalfilename".getBytes());
out.write(new byte[] {0});
}
Info about gzip format: http://www.gzip.org/zlib/rfc-gzip.html#specification

I also had the same issue, I found that (apache) commons-compress has a similar class - GzipCompressorOutputStream that can be configured with parameters.
final File compressedFile = new File("test-outer.xml.gz");
final GzipParameters gzipParameters = new GzipParameters();
gzipParameters.setFilename("test-inner.xml");
final GzipCompressorOutputStream gzipOutputStream = new GzipCompressorOutputStream(new FileOutputStream(compressedFile), gzipParameters);
Dependency:
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.8</version>
</dependency>

How to use java.util.zip to archive/deflate string in java for use in Google Earth?

Use Case
I need to package up our kml which is in a String into a kmz response for a network link in Google Earth. I would like to also wrap up icons and such while I'm at it.
Problem
Using the implementation below I receive errors from both WinZip and Google Earth that the archive is corrupted or that the file cannot be opened respectively. The part that deviates from other examples I'd built this from are the lines where the string is added:
ZipEntry kmlZipEntry = new ZipEntry("doc.kml");
out.putNextEntry(kmlZipEntry);
out.write(kml.getBytes("UTF-8"));
Please point me in the right direction to correctly write the string so that it is in doc.xml in the resulting kmz file. I know how to write the string to a temporary file, but I would very much like to keep the operation in memory for understandability and efficiency.
private static final int BUFFER = 2048;
private static void kmz(OutputStream os, String kml)
{
try{
BufferedInputStream origin = null;
ZipOutputStream out = new ZipOutputStream(os);
out.setMethod(ZipOutputStream.DEFLATED);
byte data[] = new byte[BUFFER];
File f = new File("./icons"); //folder containing icons and such
String files[] = f.list();
if(files != null)
{
for (String file: files) {
LOGGER.info("Adding to KMZ: "+ file);
FileInputStream fi = new FileInputStream(file);
origin = new BufferedInputStream(fi, BUFFER);
ZipEntry entry = new ZipEntry(file);
out.putNextEntry(entry);
int count;
while((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
}
}
ZipEntry kmlZipEntry = new ZipEntry("doc.kml");
out.putNextEntry(kmlZipEntry);
out.write(kml.getBytes("UTF-8"));
}
catch(Exception e)
{
LOGGER.error("Problem creating kmz file", e);
}
}
Bonus points for showing me how to put the supplementary files from the icons folder into a similar folder within the archive as opposed to at the same layer as the doc.kml.
Update Even when saving the string to a temp file the errors occur. Ugh.
Use Case Note The use case is for use in a web app, but the code to get the list of files won't work there. For details see how-to-access-local-files-on-server-in-jboss-application

You forgot to call close() on ZipOutputStream. Best place to call it is the finally block of the try block where it's been created.
Update: To create a folder, just prepend its name in the entry name.
ZipEntry entry = new ZipEntry("icons/" + file);