I want to embed a pre-trained model in JAR file, and later use it in prediction using Tensorflow's Java API. My Tensorflow version is 1.12.0.
I have tensorflow model exported in Python using tf.saved_model.simple_save. This saves the PB file and variables to an export directory. I can successfully load this model in Java using SavedModelBundle.load and run predictions as long as the export directory is locally available.
Now I want to use my predictor in a restricted/remote environment as a custom function for Aster Analytics database. This database takes a jar file, and runs a user defined function during SQL calls.
My problem is that SavedModelBundle loads from a local directory and does not utilize file streams or similar methods to read resources in a JAR file. As this database has access restrictions, I can not create a permanent local directory and move the exported files there. Similarly, I can not call Tensorflow Serving or any other RPC/REST calls to an external server.
I don't want to create a temp directory and copy exported directory there at each function call (as it may leave left-over directories, and there could be simultaneous calls creating race conditions etc.)
Are there any efficient way to deliver the model in the jar file, and then read it? I'm hoping there is a way to use SavedModelBundle to read JAR resources. Alternatively, I'm hoping that I can read the model's graph using a file stream, and then initialize variables using files included in the JAR. However I do not know how to do either.
I appreciate any suggestions and/or directions.
Thx in advance.
Related
I have a cloud function that triggers a jar, it start well until I try to write files in /src/main/resources, where I can read, in the moment that I try to write the program fails and throw an exception.
So can I write directly to /src/main/resources or I must to implement the Cloud Storage library?
References:
https://cloud.google.com/storage/docs/reference/libraries#client-libraries-install-java
https://cloud.google.com/appengine/docs/standard/java11/using-cloud-storage
How to read a file from maven resources folder in GCP Cloud Function?
Thanks to all.
No java code (regardless of platform) can write to that 'directory', in the sense that the point of that dir is that these files end up in the same place that your jar files end up, and that you read them using MyClass.class.getResource. If you ever write new File("src/...") you messed up ('src'? In production there is no 'src', obviously).
That abstraction (MyClass.class.getResource and friends) offer, intentionally, no way to write anything, because jar files are not meant to be writable.
The right place to store mutable data on disk is nowhere near your jar files; for end user systems, generally write in System.getProperty("user.home") and then from there a subdir, such as .yourapp, or ask the user which data directory is appropriate; different OSes have different 'flavours' (linux wants /var or ~, apple wants ~/Library/Application Support or /Library/Application Support, windows is all over the place but probably wants HOME/My Documents, etcetera), and the only one that is common amongst all of them is 'some subdir off of the user's home directory'.
For cloud in particular, you must implement the Cloud Storage Library; the actual underlying OS system your google cloud app runs on is epheremeral (it can just disappear on you and its resources are tossed into the void, if your app is written such that this is a bad thing, your app is broken).
I know that .jar files are basically archives as well as being applications. What I'm asking is how can I store data(actual files not just strings) packed inside my program? I want to do this within my Java code.
The reason for this if your wondering is that I'm producing a server mod of a game. The server starts and creates all the level data and I want to store all these file inside my .jar app.
Yes you can do this.
Non-code resources in a JAR file on the classpath can be accessed using Class.getResourceAsStream(String). Applications routinely do this, for example, to embed internationalized messages as resource bundles.
To get your file into the JAR file (at project build time!), just copy it into the appropriate place in the input directory tree before you run the jar command. Build tools such as Maven, Gradle, etc can automate that for you.
Is there a way to add files to the archive within the app?
In theory, your application could store files inside its own JAR file, under certain circumstances:
The JAR has to be a file in the local file system; i.e. not a JAR that was fetched from a remote server.
The application has to have write access to the JAR file and its parent directory.
The application must not need to read back the file it wrote to the JAR in the current classloader; i.e. without exiting and restarting.
The JAR must not need to be be signed.
The procedure would be:
Locate the JAR file and open as a ZIP archive reader.
Create a ZIP archive writer to write a new version of JAR file.
Write the application's files to the writer.
Write all resources from the ZIP reader to the writer, excluding old versions of the applications files.
Close the reader and writer.
Rename the new version of the JAR to replace the old one.
The last step might not work if the initial JAR is locked by the JVM / OS. In that case, you need do the renaming in a wrapper script.
However, I think that most people would agree that this is a BAD IDEA. It is simpler and more robust to just write regular files.
The other answers have provided some good strategies, but I am going to suggest going in a somewhat different direction.
This game supposedly has graphics and is a desktop application. It is most easy to distribute desktop applications from a web server.
If both those things are true of your game, then look into using Java Web Start to deploy it.
JWS offers APIs not available to other apps. & one of particular interest to this problem is the PersistenceService. The PersistenceService allows for small amounts of data to be stored and restored by an app. (even when it is in a sand-box). I have made a small demo. of the PersistenceService.
The idea would be to check the PersistenceService for the application data, and if not found, use the data in the Jars. If the user/application alters the data, write the altered data to the PersistenceService.
JWS also offers other nice features like splash screens, desktop integration, automatic updates..
This is not possible. You however can look into embedded databases for your usecase. Java 6 comes with JavaDB. If you doesn't want to use it then you can find more here http://java-source.net/open-source/database-engines
I would recommend that you consider having two JARs: one to store your application's class files and another JAR to store the user data. If you do not have two separate JARs, then you will have difficulties obtaining a write lock from the Operating System (since you would be trying to overwrite the JAR containing your program while java is reading it).
To create a JAR, use the java.util.jar.JarFile class. There is also another question on stackoverflow which describes how to create/write a JAR file.
Don't do this. A jar file is a source of application classes and resources, not a file system. You wouldn't try to save files into a exe, would you?
By creating a file in the Source Packages (ex: /src/resource/file.txt) its contents can be read using Class.getResourceAsStream(String)
This is a working implementation of the following answer
InputStream is = Class.class.getResourceAsStream("/resource/file.txt");
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
StringBuilder sb = new StringBuilder();
String line;
while((line = br.readLine()) != null) {
sb.append(line).append("\n");
}
System.out.println(sb.toString());
I am making the setup of java swing application by using Inno Setup as an exe i am selecting the jar file of my project, I am also adding other necessary resources as folder.
When I am installing the setup on the client side. it is putting the jar and other
resources in program files folder but there client can extract the my java classes
and other resources from jar. I want that client can only use the resources by
application program but he could not extract the resources. How is it possible?
There is literally nothing you can do to entirely prevent someone from extracting the resources.
The best you can do is to make the process a bit difficult; e.g. by storing the resources in the JAR file in encrypted form. The problem is that your program would need to decrypt the resources in order to use them. Someone with sufficient skills and patience can reverse engineer your decryption code and capture the unencrypted resources.
By the way, this is not a Java-specific problem. Any application that you provide to a user as an executable can be reverse engineered ... assuming that the user has the wherewithal to run it in the first place.
The bottom line is that if you are not prepared for the possibility that someone might extract the resources, you should not distribute the executable.
I am doing a project in java and in that i need to add and modify my
text file at runtime,which is grouped in the jar.
I am using class.getResourceAsStream(filename) this method we
can read that file from class path.
i want to write into the same textfile.
What is the possible solution for this.
If i can't update the text file in jar what other solution is there?
Appreciate any help.
The easiest solution here is to not put the file in the jar. It sounds like you are putting files in your jar so that your user only needs to worry about one file that contains everything related to that program. This is an artificial constraint and just add headaches.
There is a simple solution that still allows you to distribute just the jar file. At start up, attempt to read the file from the file system. If you don't find it, use default values that are encoded in you program. Then when changes are made, you can write it to the file system.
In general, you can't update a file that you located using getResourceAsStream. It might be a file in a JAR/ZIP file ... and writing it would entail rewriting the entire JAR file. It might be a remote file served up by a Url classloader.
For your sanity (and good practice), you should not attempt to update files that you access via the classpath. If you need to, read the file out of the JAR file (or whatever), copy it into the regular file system, and then update the copy.
I'm not saying that it is impossible to do this in all cases. Indeed, in most normal cases you can do it with some effort. However, this is not supported, and there are no standard APIs for doing this.
Furthermore, attempts to update resources are liable to cause anomalies in the classloader. For example, I'd expect resources in JAR files to not update (from the perspective of the application) until the application restarted. But resources in exploded JAR files probably would update ... though new resources might not show up.
Finally, there are cases where updating a resource is impossible:
When the user doesn't have write access to the application's installation directory. This is typical for a properly administered UNIX / Linux machine.
When the JAR file is fetched from a remote server, you are likely not to be able to write the updates back.
When you are using an arbitrary custom classloader, you've got no way of knowing where the actual bytes of an updated resource should be stored, and no way of storing them.
All JAR rewriting techniques in Java look similar. Open the Jar file, read all of it's contents, and write a new Jar file containing the unmodified contents (and the modifications you whished to make). Such techniques are not advisable for a Jar file on the class path, much less a Jar file you're running from.
If you decide you must do it this way, Java World has a few articles:
Modifying Archives, Part 1
Modifying Archives, Part 2
A good solution that avoids the need to put your items into a Jar file is to read (if present) a properties file out of a hidden subdirectory in the user's home directory. The logic looks a bit like this:
if (the hidden directory named after my application doesn't exist) {
makeTheHiddenDirectory();
writeTheDefaultPropertiesFile();
}
Properties appProps = new Properties();
appProps.load(new FileInputStream(fileInHiddenDir));
...
... After the appProps have changed ...
...
appProps.store(new FileOutputStream(fileInHiddenDir), "Do not modify this file");
Look to java.util.Properties, and keep in mind that they have two different load and store formats (key = value based and XML based). Pick the one that suits you best.
If i can't update the text file in jar what other solution is there?
Store the information in any of:
Cookies
The server
Deploy the applet using 1.6.0_10+, launch it using JWS and use the PersistenceService to store the information. Here is my demo. of the PersistenceService.
Also, if your users will agree to a trusted applet (which seems overkill for this), you might write the information to a sub-directory of user.home.
I am attempting to store the change made to my application's properties. The .properties file is located in resources package, which is different from the package that contains my UI and model.
I opened the package using:
this.getClass().getClassLoader().getResourceAsStream("resources/settings.properties")
Is there a functional equivalent of this that permits me to persist changes to the Properties Class in the same .Properties file?
In general, you cannot put stuff back into a resource you got from the classloader:
Class loader resources are often read-only; i.e. held in read-only files / read-only directories.
If you got the resource from a JAR file, JAR files are not simply updateable. (To "update" you need to extract the old JAR's contents and create a new JAR with the updated contents. It is all to do with the structure of ZIP files ...)
In some cases, the class loader resource will have been downloaded on-the-fly, and there is no way to push changes back to the place where you downloaded from.
Even if you can update a resource you got from the classloader, it is a bad idea / bad practice.
Doing this "pollutes" the clean application installation with a user's preferences. Among other things, this means that the installation cannot be shared with other users (unless you handle preferences for multiple users ...).
There are security issues with having applications installed as writeable so that embedded preferences can be updated. Think viruses! Think one user who might be inclined to trash another user's preferences!
There are administration issues with having user-specific copies of applications. And if the user has to install his own copy of an app, there are potential security issues with that as well.
There may be technical issues with file locking or caching on some platforms that either get in the way of (safe) updates or make it difficult for an application to load the updated resource without a restart.
Finally, this is NOT the way that system administrators (and educated users) expect software to behave. Java applications should deal with user preferences in the expected way:
You can use the Java Preferences API.
You can write a Properties file containing the preferences to an OS-appropriate user-writable directory.
On Windows, you could use a Windows-specific API to store the preferences in the Windows registry, except that this makes your application Windows dependent. (I can't see any real advantage in doing this, but I am not a Window expert.)
When you wrap your app up as a JAR file, your properties file will be one (possibly compressed) file within that JAR, and it would be a bad idea to try to write to your own JAR.
getResourceAsStream() is meant to open resources for reading, and these can be anywhere on the classpath. You can't write to URLs or inside JARs, you can only write to files, so it doesn't make sense to give you the same API for output.
Find yourself a directory you're allowed to write into, and write your properties there.
It may be a good idea to copy your properties from your installation classpath (possibly inside a JAR) directly out to a file if it doesn't yet exist, as a first operation upon application startup. This will give you a properties file you can write to, yet the master copy of this properties file will come from your project deliverable.
It sounds like you want to store user preferences. Consider using the Java Preferences API for that.
In addition to Carl's answer, if you're going to read and write to this file frequently, and expect that your application will expand in scope, consider whether to go one step (or several steps) further and use a file-based database like SQLite. There are a few JDBC wrappers for SQLite that would allow you to go beyond the basic string key-value lookup that the Java Properties interface provides.
even though writing the file into resources is not good practical, we still need to do it when our application only run in IDEA locally without deployment, then we can do it as below:
URL resource = Thread.currentThread().getContextClassLoader().getResource("settings.properties");
String path= resource.getPath();
OutputStream outputStream = new FileOutputStream(path);
//outputStream write