Summary:
I have a program I want to ship as a single jar file.
It depends on three big resource files (700MB each) in a binary format. The file content can easily be accessed via indexing, my parser therefore reads these files as RandomAccessFile-objects.
So my goal is to access resource files from a jar via File objects.
My problem:
When accessing the resource files from my file system, there is no issue, but I aim to pack them into the jar file of the program, so the user does not need to handle these files themselves.
The only way I found so far to access a file packed in a jar is via InputStream (generated by class.getResourceAsStream()), which is totally useless for my application as it would be much too slow reading these files from start to end instead of using RandomAccessFile.
Copying the file content into a file, reading it and deleting it in runtime is no option eigher for the same reason.
Can someone confirm that there is no way to achieve my goal or provide a solution (or a hint so I can work it out myself)?
What I found so far:
I found this answer and if I understand the answer it says that there is no way to solve my problem:
Resources in a .jar file are not files in the sense that the OS can access them directly via normal file access APIs.
And since java.io.File represents exactly that kind of file (i.e. a thing that looks like a file to the OS), it can't be used to refer to anything in a .jar file.
A possible workaround is to extract the resource to a temporary file and refer to that with a File.
I think I can follow the reasoning behind it, but it is over eight years old now and while I am not very educated when it comes to file systems and archives, I know that the Java language has evolved quite much since then, so maybe there is hope? :)
Probably useless background information:
The files are genomes in the 2bit format and I use the TwoBitParser from biojava via the wrapper class TwoBitFacade?. The Javadocs can be found here and here.
Resources are not files, and they live in a JAR file, which is not a random access medium.
Related
I need to store data into files inside .jar file and read it again.
I know that I can use Class.getResourceAsStream() method but it returns an InputStream that I can read from. But I look for a way to write.
I need to store data into files inside .jar file and read it again
No you don't.
Instead store the 'default' file inside the Jar. If it is changed, store the altered file in another place. One common place is a sub-directory of user.home. When checking for the file, first check the existence of an altered file on the file system, and if it does not exist, load the default file.
Note that it is generally better to describe the goal, rather than the strategy. 'Store changed file in Jar' is a strategy, whereas 'Save preferences between runs' might be the goal.
Related: What is the XY problem?
This is not really supported. In principle, you could operate on the jar file, but there's no guarantee the new contents would be correctly loaded. Let your build tools manage the jar file -- and choose something else for persistent storage managed by your program itself. Like a file.
It is unlikely that you can change a loaded jar safely. Changing it while it is being used is not a good idea.
If you want to do this, use a plain file system directory and add/remove files from it. Even this may not work as you expect.
You can manipulate any jar file using the package java.util.jar (or indeed just java.util.zip). As files inside a jar will be compressed, this isn't the most time efficient way for you to store data.
You should probably use a directory somewhere else (e.g. System.getProperty("user.home") + "/.myProgram") or see java.util.prefs.
Class.getResource() returns a URL. Theoretically, you can use this URL to create your InputStream and OutputStream. But in most cases, the generated JAR is a read-only file (or archive). So your application might trip when trying to use it.
I need to store data into files inside .jar file and read it again.
I know that I can use Class.getResourceAsStream() method but it returns an InputStream that I can read from. But I look for a way to write.
I need to store data into files inside .jar file and read it again
No you don't.
Instead store the 'default' file inside the Jar. If it is changed, store the altered file in another place. One common place is a sub-directory of user.home. When checking for the file, first check the existence of an altered file on the file system, and if it does not exist, load the default file.
Note that it is generally better to describe the goal, rather than the strategy. 'Store changed file in Jar' is a strategy, whereas 'Save preferences between runs' might be the goal.
Related: What is the XY problem?
This is not really supported. In principle, you could operate on the jar file, but there's no guarantee the new contents would be correctly loaded. Let your build tools manage the jar file -- and choose something else for persistent storage managed by your program itself. Like a file.
It is unlikely that you can change a loaded jar safely. Changing it while it is being used is not a good idea.
If you want to do this, use a plain file system directory and add/remove files from it. Even this may not work as you expect.
You can manipulate any jar file using the package java.util.jar (or indeed just java.util.zip). As files inside a jar will be compressed, this isn't the most time efficient way for you to store data.
You should probably use a directory somewhere else (e.g. System.getProperty("user.home") + "/.myProgram") or see java.util.prefs.
Class.getResource() returns a URL. Theoretically, you can use this URL to create your InputStream and OutputStream. But in most cases, the generated JAR is a read-only file (or archive). So your application might trip when trying to use it.
I need to store data into files inside .jar file and read it again.
I know that I can use Class.getResourceAsStream() method but it returns an InputStream that I can read from. But I look for a way to write.
I need to store data into files inside .jar file and read it again
No you don't.
Instead store the 'default' file inside the Jar. If it is changed, store the altered file in another place. One common place is a sub-directory of user.home. When checking for the file, first check the existence of an altered file on the file system, and if it does not exist, load the default file.
Note that it is generally better to describe the goal, rather than the strategy. 'Store changed file in Jar' is a strategy, whereas 'Save preferences between runs' might be the goal.
Related: What is the XY problem?
This is not really supported. In principle, you could operate on the jar file, but there's no guarantee the new contents would be correctly loaded. Let your build tools manage the jar file -- and choose something else for persistent storage managed by your program itself. Like a file.
It is unlikely that you can change a loaded jar safely. Changing it while it is being used is not a good idea.
If you want to do this, use a plain file system directory and add/remove files from it. Even this may not work as you expect.
You can manipulate any jar file using the package java.util.jar (or indeed just java.util.zip). As files inside a jar will be compressed, this isn't the most time efficient way for you to store data.
You should probably use a directory somewhere else (e.g. System.getProperty("user.home") + "/.myProgram") or see java.util.prefs.
Class.getResource() returns a URL. Theoretically, you can use this URL to create your InputStream and OutputStream. But in most cases, the generated JAR is a read-only file (or archive). So your application might trip when trying to use it.
I'm trying to load classes from a jar file from a URL into memory then run one of the classes loaded. I don't want the class files or the jar to ever be accessible by the user. I've seen a couple similar questions, but they have gone unanswered.
I know it will probably use URLClassLoader and ByteArrayInputStream/ByteArrayOutputStream.
Thanks in advance for any help.
Similar unanswered questions:
How to load a jar from an URL without downloading it?
Load jar from URL
You are on the right track (I think). The steps would be:
Use URLConnection to open a stream to the JAR file.
Read the stream and write it to ByteArrayOutputStream, and extract the byte array.
Open a ByteArrayInputStream on the byte array, then wrap that in a JarInputStream.
Iterate through the members of the entries of the JAR file, saving the entry information and buffering the entry file content in memory.
Write a custom class loader that uses the in-memory cache of the JAR file entries and content.
I can't point you at example code because I couldn't find any.
(You pretty much have to cache stuff in memory since the JarFile API requires a RandomAccessFile and that implies that the data is in the file-system. On some systems you could create a temporary file, open a RAF on it, unlink it, and then fetch and write the URL into the RAF. But that isn't portable ...)
I should point out that if you are doing this as part of some licensing or "intellectual property management" scheme, you are probably wasting your time. Any practical scheme that you care to implement in client-side code (i.e. code that runs on the user's machine) can be broken ... unless the machine is totally locked down.
I am doing a project in java and in that i need to add and modify my
text file at runtime,which is grouped in the jar.
I am using class.getResourceAsStream(filename) this method we
can read that file from class path.
i want to write into the same textfile.
What is the possible solution for this.
If i can't update the text file in jar what other solution is there?
Appreciate any help.
The easiest solution here is to not put the file in the jar. It sounds like you are putting files in your jar so that your user only needs to worry about one file that contains everything related to that program. This is an artificial constraint and just add headaches.
There is a simple solution that still allows you to distribute just the jar file. At start up, attempt to read the file from the file system. If you don't find it, use default values that are encoded in you program. Then when changes are made, you can write it to the file system.
In general, you can't update a file that you located using getResourceAsStream. It might be a file in a JAR/ZIP file ... and writing it would entail rewriting the entire JAR file. It might be a remote file served up by a Url classloader.
For your sanity (and good practice), you should not attempt to update files that you access via the classpath. If you need to, read the file out of the JAR file (or whatever), copy it into the regular file system, and then update the copy.
I'm not saying that it is impossible to do this in all cases. Indeed, in most normal cases you can do it with some effort. However, this is not supported, and there are no standard APIs for doing this.
Furthermore, attempts to update resources are liable to cause anomalies in the classloader. For example, I'd expect resources in JAR files to not update (from the perspective of the application) until the application restarted. But resources in exploded JAR files probably would update ... though new resources might not show up.
Finally, there are cases where updating a resource is impossible:
When the user doesn't have write access to the application's installation directory. This is typical for a properly administered UNIX / Linux machine.
When the JAR file is fetched from a remote server, you are likely not to be able to write the updates back.
When you are using an arbitrary custom classloader, you've got no way of knowing where the actual bytes of an updated resource should be stored, and no way of storing them.
All JAR rewriting techniques in Java look similar. Open the Jar file, read all of it's contents, and write a new Jar file containing the unmodified contents (and the modifications you whished to make). Such techniques are not advisable for a Jar file on the class path, much less a Jar file you're running from.
If you decide you must do it this way, Java World has a few articles:
Modifying Archives, Part 1
Modifying Archives, Part 2
A good solution that avoids the need to put your items into a Jar file is to read (if present) a properties file out of a hidden subdirectory in the user's home directory. The logic looks a bit like this:
if (the hidden directory named after my application doesn't exist) {
makeTheHiddenDirectory();
writeTheDefaultPropertiesFile();
}
Properties appProps = new Properties();
appProps.load(new FileInputStream(fileInHiddenDir));
...
... After the appProps have changed ...
...
appProps.store(new FileOutputStream(fileInHiddenDir), "Do not modify this file");
Look to java.util.Properties, and keep in mind that they have two different load and store formats (key = value based and XML based). Pick the one that suits you best.
If i can't update the text file in jar what other solution is there?
Store the information in any of:
Cookies
The server
Deploy the applet using 1.6.0_10+, launch it using JWS and use the PersistenceService to store the information. Here is my demo. of the PersistenceService.
Also, if your users will agree to a trusted applet (which seems overkill for this), you might write the information to a sub-directory of user.home.