We Keep Coding

Java in Mac OS X Catalina - Zip file not found in Resources, while the file is found when run from Eclipse

I have a compressed file in the Resources folder which is decompressed in the folder where the program is run for the first time. If the program starts from Eclipse, the file is found and decompressed without problems. When I export the program in a jar file, and run the program with:
java -jar JRS2020-31.jar
the output is:
java.lang.RuntimeException: Error unzipping file initialData/compressed.zip
at InterpreteSQL.Main.unzip(Main.java:111)
at InterpreteSQL.Main.CreateInitialDirectoryIfNotFound(Main.java:77)
at InterpreteSQL.Main.main(Main.java:60)
Caused by: java.io.FileNotFoundException: file:/Users/XXX/JRS2020-31.jar!/initialData/compressed.zip (No such file or directory)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:219)
at java.util.zip.ZipFile.<init>(ZipFile.java:149)
at java.util.zip.ZipFile.<init>(ZipFile.java:120)
at InterpreteSQL.Main.unzip(Main.java:88)
... 2 more
Note that other files in resources (help html files) are opened regularly in the program.
This is the code that opens the file:
public static void unzip(String zipFilePath, String unzipDir) throws Exception {
try{
ZipFile zipFile = new ZipFile(Main.class.getClassLoader().getResource(zipFilePath).getFile());
Enumeration<? extends ZipEntry> entries = zipFile.entries();
while(entries.hasMoreElements()){
...
and it is called with:
unzip("initialData/compressed.zip", "JRS_directory");
Note that the program is run in the home folder, and that it can create directories and files.
Any idea about the problem? Thank you very much.

You'reusing getResources wrong, in two ways.
You're asking the resource to turn itself into a file for no reason. In your eclipse run, the resource IS a file. However, in a jar run, it's not (it is an entry in a jar, which is not a file). Don't call .getFile() on these resources; the whole point of the abstraction is that it might not be a file at all. Could be an entry in a jar. Could be imported via the network. All you know is: Load this data from where-ever you're finding the class files.
Don't use ZipFile; use ZipInputStream which has very similar API. Then use getResourceAsStream.
The proper form is Main.class.getResource(). Avoid the classloader intermediate. It is both a pointless method call, and in rare cases will break (in certain contexts, a class's classloader is null, causing a nullpointerexception). Note that this form means the string you pass in is relative to the class location. If you don't want that, add a leading slash.
While you're at it, these are resources that must be closed, so let's use the try-with-resources construct to ensure this is done properly even in the face of exceptions.
Putting this together:
try (InputStream raw = Main.class.getResourceAsStream("/" + zipFilePath);
ZipInputStream zip = new ZipInputStream(raw)) {
for (ZipEntry entry; (entry = zip.getNextEntry()) != null; ) {
// do something with entry here
}
}

handling about 450.000 files in a zip

My question is simple. Would Java handle a .zip file with about 450,000 files in there? The code that I wrote would not load all of the files, just one specific file would be searched in the zip, and be read line by line. The file size is about 500kb.
Would this work or will I get an OutOfMemory Exception?
Oh sry, uncompressed there about 0,5MB. Zipped are they whole files about 250mb.
Ok, the name of the Files are IDs + Date(unique) in that zip file. If i have to check a log, ill call Java and give the ID + Date and Java is reading just that one file, never more.
Edit: It works, it works very well. About 400.000 files in a zip, if u have the Memory to Zip the Files works without any problem.
Edit2: It works on Linux Filesystems witout a problem, on NTFS sometimes it crashed. NTFS has a problem with that musch files in 1 Zip.

Using the zip filesystem in Java 7, you can actually access one individual file pretty easily and open a BufferedReader on it.
First you have to create the FileSystem:
public static FileSystem getZipFileSystem(final String zipPath)
{
final Path path = Paths.get(zipPath).toAbsolutePath();
final Map<String, Object> env = new HashMap<>();
final URI uri = URI.create("jar:file:" + path.toString());
return FileSystems.newFileSystem(uri, env, null);
}
Once you have done that, you can create a BufferedReader from an entry in the zip itself:
try (
final FileSystem fs = getZipFileSystem("/path/to/the.zip");
final BufferedReader reader = Files.newBufferedReader(fs.getPath("path/to/entry"),
StandardCharsets.UTF_8);
) {
// operate on the reader
}
You could also read all lines in the entry at once using Files.readAllLines().
If you wish to copy a zip entry to a file on the filesystem, you can also do that:
Files.copy(zipfs.getPath("path/to/entry"), Paths.get("file/on/local/fs"));
Or you can directly copy the result to an OutputStream, or directly create an entry from an OutputStream...
Or even walk the entire zip using Files.walkFileTree().
Or get all the entries in a "directory" in a zip using Files.newDirectoryStream(). Note that as its name says, this is a stream; unlike File.listFiles() (which only works on files on disk anyway), this returns a iterator over the entries.
Or... Or... Or...
Note that a FileSystem needs to be .close()d.

I'm not sure that I understand what you're trying to do.
If it's 0.5 MB/file and 450,000 files, you'll need 225GB. You won't have enough memory to do all this in a single zip in memory even if you get 90% compression.
I'd recommend breaking it into manageable chunks. You'll be able to parallelize that way too, so it's not a bad idea.

Do I need to delete tmp files created by my java application?

I output several temporary files in my application to tmp directories but was wondering if it is best practise that I delete them on close or should I expect the host OS to handle this for me?
I am pretty new to Java, I can handle the delete but want to keep the application as multi-OS and Linux friendly as possible. I have tried to minimise file deletion if I don't need to do it.
This is the method I am using to output the tmp file:
try {
java.io.InputStream iss = getClass().getResourceAsStream("/nullpdf.pdf");
byte[] data = IOUtils.toByteArray(iss);
iss.read(data);
iss.close();
String tempFile = "file";
File temp = File.createTempFile(tempFile, ".pdf");
FileOutputStream fos = new FileOutputStream(temp);
fos.write(data);
fos.flush();
fos.close();
nopathbrain = temp.getAbsolutePath();
System.out.println(tempFile);
System.out.println(nopathbrain);
} catch (IOException ex) {
ex.printStackTrace();
System.out.println("TEMP FILE NOT CREATED - ERROR ");
}

createTempFile() only creates a new file with a unique name, but does not mark it for deletion. Use deleteOnExit() on the created file to achieve that. Then, if the JVM shuts down properly, the temporary files should be deleted.
edit:
Sample for creating a 'true' temporary file in java:
File temp = File.createTempFile("temporary-", ".pdf");
temp.deleteOnExit();
This will create a file in the default temporary folder with a unique random name (temporary-{randomness}.pdf) and delete it when the JVM exits.
This should be sufficient for programs with a short to medium run time (e.g. scripts, simple GUI applications) that do sth. and then exit. If the program runs longer or indefinitely (server application, a monitoring client, ...) and the JVM won't exit, this method may clog the temporary folder with files. In such a situation the temporary files should be deleted by the application, as soon as they are not needed anymore (see delete() or Files helper class in JDK7).
As Java already abstracts away OS specific file system details, both approaches are as portable as Java. To ensure interoperability have a look at the new Path abstraction for file names in Java7.

Copy file exception with TrueZIP

Here is my code to copy a war file to another using TrueZIP.
TFile srcFile = new TFile(sourceFilePath);
TFile destFile = new TFile(destFilePath);
if (!destFile.getParentFile().exists()) {
destFile.getParentFile().mkdirs();
}
try {
srcFile.cp_rp(destFile);
TVFS.umount();
} catch (IOException e) {
e.printStackTrace();
}
For example, source file location:
I:\Code-Base\test.7.2.0\projects\test\main\branches\test.7.2.0_for_test\ui\portlets\dist\portlets.war\WEB-INF\server-config.wsdd
and destination location:
D:\deploy\work\237798_237980\web\deploy\prtlets.war\WEB-INF\server-config.wsdd
I've already checked that both paths exists, but I get an InputException error::
de.schlichtherle.truezip.io.InputException: de.schlichtherle.truezip.fs.FsReadOnlyArchiveFileSystemException: This is a read-only archive file system!
at de.schlichtherle.truezip.socket.IOSocket.copy(IOSocket.java:102)
at de.schlichtherle.truezip.file.TBIO.cp0(TBIO.java:221)
at de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:179)
at de.schlichtherle.truezip.file.TBIO.cp_r(TBIO.java:138)
at de.schlichtherle.truezip.file.TFile.cp_rp(TFile.java:3210)
at com.accela.work.WorkThread.run(WorkThread.java:110)
at com.accela.work.Worker.getUpgradePackageByVersion(Worker.java:162)
at com.accela.work.Main.generateUpgradePackage(Main.java:114)
at com.accela.work.Main.getUpgradePackageByVersion(Main.java:107)
at com.accela.work.Main.main(Main.java:75)
Caused by: de.schlichtherle.truezip.fs.FsReadOnlyArchiveFileSystemException: This is a read-only archive file system!
at de.schlichtherle.truezip.fs.FsReadOnlyArchiveFileSystem.mknod(FsReadOnlyArchiveFileSystem.java:54)
at de.schlichtherle.truezip.fs.FsBasicArchiveController$1Output.mknod(FsBasicArchiveController.java:273)
at de.schlichtherle.truezip.fs.FsBasicArchiveController$1Output.getLocalTarget(FsBasicArchiveController.java:220)
at de.schlichtherle.truezip.fs.FsBasicArchiveController$1Output.getLocalTarget(FsBasicArchiveController.java:217)
at de.schlichtherle.truezip.fs.FsContextController$Output.getLocalTarget(FsContextController.java:296)
at de.schlichtherle.truezip.fs.FsContextController$Output.getLocalTarget(FsContextController.java:280)
at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:47)
at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:21)
at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:47)
at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:21)
at de.schlichtherle.truezip.fs.FsSyncController$Output.getLocalTarget(FsSyncController.java:421)
at de.schlichtherle.truezip.fs.FsSyncController$Output.getLocalTarget(FsSyncController.java:408)
at de.schlichtherle.truezip.fs.FsLockController$Output$1GetLocalTarget.call(FsLockController.java:498)
at de.schlichtherle.truezip.fs.FsLockController$Output$1GetLocalTarget.call(FsLockController.java:495)
at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:316)
at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
at de.schlichtherle.truezip.fs.FsLockController$Output.getLocalTarget(FsLockController.java:501)
at de.schlichtherle.truezip.fs.FsLockController$Output.getLocalTarget(FsLockController.java:484)
at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:47)
at de.schlichtherle.truezip.socket.DelegatingOutputSocket.getLocalTarget(DelegatingOutputSocket.java:21)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Output$GetLocalTarget.call(FsFalsePositiveArchiveController.java:374)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Output$GetLocalTarget.call(FsFalsePositiveArchiveController.java:367)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Output.getLocalTarget(FsFalsePositiveArchiveController.java:364)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Output.getLocalTarget(FsFalsePositiveArchiveController.java:348)
at de.schlichtherle.truezip.socket.InputSocket.getPeerTarget(InputSocket.java:50)
at de.schlichtherle.truezip.fs.FsBasicArchiveController$1Input.getDelegate(FsBasicArchiveController.java:199)
at de.schlichtherle.truezip.socket.DelegatingInputSocket.getBoundSocket(DelegatingInputSocket.java:43)
at de.schlichtherle.truezip.socket.DelegatingInputSocket.newInputStream(DelegatingInputSocket.java:63)
at de.schlichtherle.truezip.fs.FsContextController$Input.newInputStream(FsContextController.java:273)
at de.schlichtherle.truezip.fs.FsResourceController$Input.newInputStream(FsResourceController.java:242)
at de.schlichtherle.truezip.socket.DelegatingInputSocket.newInputStream(DelegatingInputSocket.java:63)
at de.schlichtherle.truezip.fs.FsSyncController$Input.newInputStream(FsSyncController.java:378)
at de.schlichtherle.truezip.fs.FsLockController$Input$1NewInputStream.call(FsLockController.java:455)
at de.schlichtherle.truezip.fs.FsLockController$Input$1NewInputStream.call(FsLockController.java:452)
at de.schlichtherle.truezip.fs.FsLockController.locked(FsLockController.java:328)
at de.schlichtherle.truezip.fs.FsLockController.writeLocked(FsLockController.java:268)
at de.schlichtherle.truezip.fs.FsLockController$Input.newInputStream(FsLockController.java:459)
at de.schlichtherle.truezip.fs.FsFinalizeController$Input.newInputStream(FsFinalizeController.java:177)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input$NewInputStream.call(FsFalsePositiveArchiveController.java:333)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input$NewInputStream.call(FsFalsePositiveArchiveController.java:326)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$TryChild.call(FsFalsePositiveArchiveController.java:507)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController.call(FsFalsePositiveArchiveController.java:104)
at de.schlichtherle.truezip.fs.FsFalsePositiveArchiveController$1Input.newInputStream(FsFalsePositiveArchiveController.java:323)
at de.schlichtherle.truezip.socket.IOSocket.copy(IOSocket.java:100)
... 9 more

TrueZIP does a simple test to check if the archive file is writable. If this test fails, the archive file system is set read-only as indicated by the exception.
In most cases, this is simply an issue with the access permissions. But Windows is particularly bitchy. For example, if there is another tool concurrently accessing the archive file (many Explorer plug-ins do this) then the file is effectively read-only, too.
So please stay away from the archive file (and best, its directory) while the operation is running.

You cannot do a replace on a read-only file, because you would have to delete it, i.e. write to it.
Make sure your destFile is writeable.

get File from JAR

I'm using Spring's Resource abstraction to work with resources (files) in the filesystem. One of the resources is a file inside a JAR file. According to the following code, it appears the reference is valid
ResourcePatternResolver resourceResolver = new PathMatchingResourcePatternResolver();
// The path to the resource from the root of the JAR file
Resource fileInJar = resourcePatternResolver.getResources("/META-INF/foo/file.txt");
templateResource.exists(); // returns true
templateResource.isReadable(); // returns true
At this point, all is well, but then when I try to convert the Resource to a File
templateResource.getFile();
I get the exception
java.io.FileNotFoundException: class path resource [META-INF/foo/file.txt] cannot be resolved to absolute file path because it does not reside in the file system: jar:file:/D:/m2repo/uic-3.2.6-0.jar!/META-INF/foo/file.txt
at org.springframework.util.ResourceUtils.getFile(ResourceUtils.java:198)
at org.springframework.core.io.ClassPathResource.getFile(ClassPathResource.java:174)
What is the correct way to get a File reference to a Resource that exists inside a JAR file?

What is the correct way to get a File
reference to a Resource that exists
inside a JAR file?
The correct way is not doing that at all because it's impossible. A File represents an actual file on a file system, which a JAR entry is not, unless you have a special file system for that.
If you just need the data, use getInputStream(). If you have to satisfy an API that demands a File object, then I'm afraid the only thing you can do is to create a temp file and copy the data from the input stream to it.

If you want to read it, just call resource.getInputStream()
The exception message is pretty clear - the file does not reside on the file-system, so you can't have a File instance. Besides - what will do do with that File, apart from reading its content?

A quick look at the link you provided for Resource documentation, says the following:
Throws: IOException if the resource cannot be resolved as absolute file path,
i.e. if the resource is not available in a file system
Maybe the text file is inside a jar? In that case you will have to use getInputStream() to read its contents.

Just adding an example to the answers here. If you need a File (and not just the contents of it) from within your JAR, you need to create a temporary file from the resource first. (The below is written in Groovy):
InputStream inputStream = resourceLoader.getResource('/META-INF/foo/file.txt').inputStream
File tempFile = new File('file.txt')
OutputStream outputStream = new FileOutputStream(tempFile)
try {
IOUtils.copy(inputStream, outputStream)
} catch (IOException e) {
// Handle exception
} finally {
outputStream.close()
}