I have a problem generating ZIP files via zip4j.
I am able to produce ZIP archive using following code (I omitted some parts, that are not related to the issue), which is basically taken from zip4j tutorial:
File zipFile = new File(zipName);
ZipParameters params = new ZipParameters();
params.setCompressionMethod(Zip4jConstants.COMP_DEFLATE);
params.setCompressionLevel(Zip4jConstants.DEFLATE_LEVEL_NORMAL);
byte[] buffer = new byte[8192];
try (ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipFile))) {
for (/* loop through list of input data */) {
String outputName = /* method to determine file name */;
try (InputStream in = /* method to get IS */ ) {
params.setFileNameInZip(outputName);
File tmpEntry = new File(outputName);
tmpEntry.createNewFile();
out.putNextEntry(tmpEntry, params);
int len;
while ((len = in.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
tmpEntry.delete();
out.closeEntry();
in.close();
}
}
}
The problem is that although all files are correctly included in ZIP archive, their declared file size is 0. I can unzip them using "dumb" ZIP readers (like build-in TotalCommander's one), because all data are actually here, but more "clever" programs (like 7zip) produce CRC error and refuse to open them as corrupted.
I would say I need to declare the file size somehow (and I am definitely not doing this in my code snippet), but I was unable to find the (probably obvious) solution. I have googled that native java.util.zip.ZipEntry has .setSize() method, but I don't see anything like this in zip4j...
Anyone knows the correct approach to this?
So the solution is quite simple. One just needs to add following setting into ZipParameters:
params.setSourceExternalStream(true);
How I found that?
I digged deeper into the code and find following in closeEntry() method of CipherOutputStream.java class of zip4j:
if (zipParameters.isSourceExternalStream()) {
fileHeader.setUncompressedSize(totalBytesRead);
if (localFileHeader.getUncompressedSize() != totalBytesRead) {
localFileHeader.setUncompressedSize(totalBytesRead);
}
}
This is the only place where setUncompressedSize() - which sounds like having something to do with declared file size - method is called. So I got suspicious and tried to set the parameter in my code. And it does the job.
The lesson learned is that this example of using zip4j is incorrect because it is missing that vital line of code.
Related
How can I download .class file and and load it into jvm using class loader , I have write a simple code simulates downloading a .class file the I tried to load it into JVM
public class face {
public static void main(String[] args) throws IOException,
ClassNotFoundException {
File f = new File("Task.class");
int count;
byte[] buffer = new byte[1024];
DataInputStream dis = new DataInputStream(new FileInputStream(f));
StringBuilder all = new StringBuilder();
while ((count = dis.read(buffer)) > 0) {
// System.out.write(buffer, 0, count);
all.append(buffer);
// System.out.flush();
}
File b = new File("Task.class");
FileOutputStream fos = new FileOutputStream(b);
DataOutputStream dos = new DataOutputStream(fos);
dos.write(all.toString().getBytes());
ClassLoader lod = face.class.getClassLoader();
lod.loadClass(b.getAbsolutePath());
}
}
Use Class.forName(<package_qualified_class_name>)
First, I would like to applogies for the long list of suggestion here, but you have managed to cram an impressive number of mistakes into a small piece of code.
I suggest don't do any of these things
don't use DataInputStream or DataOutputStream when it doesn't add anything. You don't use any method which requires it.
don't write binary data to a StringBuilder. A StringBuilder is for text.
don't copy an entire buffer if you only read part of it. i.e. you need to record the length actual read and copy only the amount used.
don't append a byte[] to a StringBuilder. It won't do what you expect.
don't use a String to store binary data.
don't convert a String to byte[] using the default encoding unless you know you have ASCII data (which you don't)
don't write to a file you just read. As this doesn't make sense. You should have tested this works without the file copy and you would have found this didn't work, before you attempted something more complicated.
you can't write to a file which you still have open in windows. I suggest you close() a file when you are finished with it.
don't attempt to load a class using the file name. You load it by package.class name.
I suggest you try a one liner to load a class first and show this works. The class should appear in your class path, and when you write to the file, you should write it to a directory appropriate for the package.
Instead of doing all this, you could add a http://yourserver/basepath to your class path and it will load the classes from a web service. i.e. you might be able to do this without writing any code at all.
I have a situation where I need to scan the runtime classpath for a resource file (say, res/config/meta.cfg), and then create a File handle for it. The best I've been able to come up with is:
// This file is located inside a JAR that is on the runtime classpath.
String fileName = "res/config/meta.cfg";
try {
InputStream inStream = ClassLoader.getSystemResourceAsStream(fileName);
File file = new File(String.format("${java.io.tmpdir}/%s", fileName));
FileOutputStream foutStream = null;
foutStream = new FileOutputStream(file);
int read = 0;
byte[] bytes = new byte[1024];
while((read = inStream.read(bytes)) != -1)
foutStream.write(bytes, 0, read);
foutStream.close();
return file;
} catch (Exception exc) {
throw new RuntimeException(exc);
}
So essentially, read in the resource as an InputStream, and then write the stream to a temp file (under {$java.io.tmpdir}) so that we can obtain a valid File handle for it.
This seems like going 3 sides around the barn. Is there a better/easier/more elegant way of doing this? Thanks in advance!
No.
Of course you can (and probably should) use a library to copy the InputStream's content to a file but that obviously is not the point of your question.
The classpath does not consist of directories only; resources can be inside archives (typically JARs) or on servers, and may not exist as something that can be accessed via a java.io.File object.
Typically the core problem is to use java.io.File objects where an InputStream would be sufficient. Sometimes you can't do anything against it when using a third-party library but it is a hint that the library designers didn't work very carefully. If you need the file handle in your own code you should have another look why it can't be an InputStream. Most of the time it can.
This should be simple but it has cost me hours. Everything I find on this site indicates I am doing it right but the file still cannot be found.
Inside a jar file I have two files 'CDAkeystore.jks' and 'CDAtruststore.jks' at top level.
Yet when I call
securityProps.setProperty("javax.net.ssl.keyStore","CDAkeystore.jks");
I get a system cannot find the file requested error.
The class file calling this method is inside the same jar in the usual package arrangement.
The jar file is as follows:
com ..... (a lot more class files)
org ..... (lots of class files)
META-INF
CDAtruststore.jks
CDAkeystore.jks
How can this be SOOO difficult?!!
---------- Added INfo ------n
Since the object using the path is open source I found the routine they are using to load the file. It is:
InputStream keystoreInputStream = preBufferInputStream(new FileInputStream(keyStoreName));
which according to the documentation of FileInputStream(String name) is
Creates a FileInputStream by opening a connection to an actual file, the file named by the path name 'name' in the file system. So how should this path be expressed?
Use YourClass.class.getResourceAsStream() or this.getClass().getResourceAsStream(). You can also use class loader if you are in multiple class loaders environment.
The answer is, in short, that you can't. At least in this situation. I am stuck with passing a path to a file to a library implementation that I have no control over. So the library method accesses the file on the assumption that the file exists in unzipped form in the OS's file system. It is getting the path from a Property.setProperty(stringKey, stringPath)
So the only solution I found was an ugly hack. I need to take the resource in my jar and copy it to a file on the system. Then I would pass the path to that file in the above setProperty() method. The ugly hack is implemented as follows (if anyone else can come up with a nicer solution I would be happy). It does solve the problem. The library routine is able to find my newly created file.
/* This evil reads a file as a resource inside a jar and dumps the file where ever
* the loader of this jar/application defines as the current directory. This pain is done
* so the SecurityDomian class can load the file; it cannot access the file from
* the jar. This means the 'name' passed in contains the file name inside the jar
* prefixed with "/" so it is not read with an assumed package extension.
*/
private boolean createFileFromResource(String name)
{
// Dont bother if the file already exists
if(new File(name.replace("/", "")).exists())
{
return true;
}
byte[] keystoreFile = new byte[2048];
ByteArrayOutputStream byteArrayOut = new ByteArrayOutputStream(2048);
// Get the resource
InputStream inputStream = this.getClass().getResourceAsStream(name);
try
{
int bytesRead = 0;
while(true)
{
// Read the resource into the buffer keystoreFile in 2048 byte chunks
bytesRead = inputStream.read(keystoreFile);
if(bytesRead < 0)
{
break;
}
// Copy and append the chunks to the ByteArrayOutputStream (this class
// does the auto-extending of the output array as more chunks are
// added so you don't have to do it.
byteArrayOut.write(keystoreFile, 0, bytesRead);
}
inputStream.close();
// Now create a file at the root of where ever the loader happens to think
// the root is. So remove the "/" in front of the file name
FileOutputStream outputStream = new FileOutputStream(name.replace("/", ""));
// Write out the file. Note you will be left with garbage at that location.
byteArrayOut.writeTo(outputStream);
outputStream.flush();
outputStream.close();
}
catch (IOException e)
{
e.printStackTrace();
return false;
}
return true;
}
I'm working on a game, and I need to load multiple image files (png, gif, etc.) that I'll eventually want to convert into BufferedImage objects. In my setup, I'd like to load all of these images from a single zip file, "Resources.zip". That resource file will contain images, map files, and audio files - all contained in various neatly ordered sub-directories. I want to do this because it will (hopefully) make resource loading easy in both applet and application versions of my program. I'm also hoping that for the applet version, this method will make it easy for me to show the loading progress of the game resources zip file (which could eventually amount to 10MB depending on how elaborate this game gets, though I'm hoping to keep it under that size so that it's browser-friendly).
I've included my zip handling class below. The idea is, I have a separate resource handling class, and it creates a ZipFileHandler object that it uses to pull specific resources out of the Resources.zip file.
import java.io.BufferedInputStream;
import java.io.File;
import java.io.IOException;
import java.util.Enumeration;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
public class ZipFileHandler
{
private ZipFile zipFile;
public ZipFileHandler(String zipFileLocation)
{
try
{
zipFile = new ZipFile(zipFileLocation);
}
catch (IOException e) {System.err.println("Unable to load zip file at location: " + zipFileLocation);}
}
public byte[] getEntry(String filePath)
{
ZipEntry entry = zipFile.getEntry(filePath);
int entrySize = (int)entry.getSize();
try
{
BufferedInputStream bis = new BufferedInputStream(zipFile.getInputStream(entry));
byte[] finalByteArray = new byte[entrySize];
int bufferSize = 2048;
byte[] buffer = new byte[2048];
int chunkSize = 0;
int bytesRead = 0;
while(true)
{
//Read chunk to buffer
chunkSize = bis.read(buffer, 0, bufferSize); //read() returns the number of bytes read
if(chunkSize == -1)
{
//read() returns -1 if the end of the stream has been reached
break;
}
//Write that chunk to the finalByteArray
//System.arraycopy(src, srcPos, dest, destPos, length)
System.arraycopy(buffer, 0, finalByteArray, bytesRead, chunkSize);
bytesRead += chunkSize;
}
bis.close(); //close BufferedInputStream
System.err.println("Entry size: " + finalByteArray.length);
return finalByteArray;
}
catch (IOException e)
{
System.err.println("No zip entry found at: " + filePath);
return null;
}
}
}
And I use the ZipFileHandler class like this:
ZipFileHandler zfh = new ZipFileHandler(the_resourceRootPath + "Resources.zip");
InputStream in = new ByteArrayInputStream(zfh.getEntry("Resources/images/bg_tiles.png"));
try
{
BufferedImage bgTileSprite = ImageIO.read(in);
}
catch (IOException e)
{
System.err.println("Could not convert zipped image bytearray to a BufferedImage.");
}
And the good news is, it works!
But I feel like there might be a better way to do what I'm doing (and I'm fairly new to working with BufferedInputStreams).
In the end, my question is this:
Is this even a good idea?
Is there a better way to load a whole bunch of game resource files in a single download/stream, in an applet- AND application-friendly way?
I welcome all thoughts and suggestions!
Thanks!
Taking multiple resources and putting in them in one compressed file is how several web applications work (i.e. GWT) It is less expensive to load one large file than multiple small ones. This assumes that you are going to use all those resources in your app. If not Lazy loading is also a viable alternative.
That being said, it is usually best to get the app working and then to profile to find where the bottlenecks are. If not you will end up with a lot of complicated code and it will take you a lot longer to get your app working. 10%-20% of the code takes 80-90% of the time to execute. You just don;t know which 10-20% that is until the project is mostly complete.
If your goal is to learn the technologies and tinker, then good going - looks good.
If you are using a Java program, it is usually considered good practice to bundle it as a jar file anyway. So why do not put your classes simply inside this jar file (in directories, of course). Then you can simply use
`InputStream stream = MayClass.class.getResourceAsStream(imagePath);`
to load the data for each image, instead of having to handle all the zip by yourself (and it also works for jars not actually on the file system, such as http url in applets).
I also assume the jar will be cached, but you should measure and compare the performance to your solution with an external zip file.
I'm serving a file from the file system dynamically with a jsp
Here's my code:
<%# page import="java.io.*,java.util.*"
InputStream in = null;
OutputStream responseOut = null;
File file = new File(request.getAttribute("fileToServe"));
try{
in = new FileInputStream(file);
responseOut = response.getOutputStream();
// Transfer bytes from in to out
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0) {
responseOut.write(buf, 0, len);
}
}finally{
if( responseOut != null ) try {
responseOut.close();
} catch( IOException ioe ){}
if( in != null ) try {
in.close();
} catch( IOException ioe ){}
}
file.delete();
%>
The problem I'm facing is, the file is delete only the first time the code is run, which is after the server restart. Subsequent calls doesn't delete the file.
I used ProcessExplorer to track this and and yeap, the Java VM is holding that file, I don't really know why is this happening.
We will run on Windows OS, always, is there a work around for this?
I've found a number of resources on the internet about this, but I can't figure out from them how to solve this problem.
What creates the file? I only see reading then deleting it in that code.
Things to watch out for:
Reading a file requires read permissions from the file but deleting the file requires write permission from the directory; and
Make sure you close() any files you create. If you don't you might lose data or it may take time to flush an implicit close.
Lastly, using an attribute like fileToServe that comes from the user is really dangerous. I'm hoping you're sanitizing that elsewhere. You should ensure that only allowed files are served this way.
Ok, I've checked, after cletus comment, the place where this file is being created. The close method on the stream that wrote the file was missing.
Mystery solved
cletus, please add your answer so I can accept it