Example: I have two shared objects (same should apply to .dlls). The first shared object is from a third-party library, we'll call it libA.so. I have wrapped some of this with JNI and created my own library, libB.so. Now libB depends on libA.
When webstarting, both libraries are places in some webstart working area. My java code attempts to load libB. At this point the system loader will attempt to load libA which is not in the system library path (java.library.path won't help this). The end result is that libB has an unsatisfied link and cannot be used.
I have tried loading libA before libB, but that still does not work. Seems the OS wants to do that loading for me. Is there any way I can make this work other than statically compiling?
I'm not sure if this would be handled exactly the same way for webstart, but we ran into this situation in a desktop application when dealing with a set of native libraries (dlls in our case).
Loading libA before libB should work, unless one of those libraries has a dependency that is unaccounted for and not in the path. My understanding is that once it gets to a system loadLibrary call (i.e. Java has found the library in its java.library.path and is now telling the OS to load it) - it is completely dependent on the operating system to find any dependent libraries, because at that point it is the operating system that is loading the library for the process, and the OS only knows how to look in the system path. That seems hard to set in the case of a Webstart app, but there is a way around this that does not involve static compiling. You may be able to shuffle where your libraries are - I am unsure
If you use a custom classloader, you can override loadLibrary and findLibrary so that it can locate your libraries from within a jar in your classpath, and if you also make it aware of your native library dependencies (i.e. libB depends on libA depends on libX, then when loading libB you can catch yourself and ensure you load libA first, and in checking that notice and load libX first. Then the OS doesn't try to find a library that isn't in your path. It's klunky and a bit painful, but ensuring Java finds them and loads them all in the correct order can work.
Static compilation proved to be the only way to webstart multiple dependent native libraries.
Are both native libraries packaged into a signed jar which is listed as
<nativelib ...>
In the JNLP file?
Related
I have the following distribution
.__bin
| |__start.sh
|
|__lib
|__Main.jar
|__Utility.jar
|__lib.so
Main.jar contains main method, in which I would like to access native functions provided by the lib.so bundled in the distribution. start.sh simply sets the classpath and bootstraps java.
What is the best approach to load the lib.so library inside the application? I am aware of two methods:
System.load(absolute path)
System.loadLibrary(name)
Former requires the absolute path of the library in the file system, latter requires setting the application specific java.library.path JVM argument. Which is better solution in this case? Would setting the library path in the start.sh script possibly break something? How could I get the absolute file system path of the library for the former solution?
Assuming I am not missing anything, isn't setting System.load the best approach? Your lib.so is not going to change since you have it in your structure. If you would want to give control to the end user to load the native/shared library from elsewhere have an optional parameter in start.sh. Else use the System.load().Just make sure to check the type of OS, since you will be having the different formats for file paths.
The load() method requires the complete path name of the library as an argument. For example, on a Solaris system you might write:
System.load("/home/me/libs/libmylib.so");
to load the libmylib.so library in the /home/me/libs directory.
Using the load() method is system-dependent because it uses a pathname to load the library and pathnames are usually system-dependent. Thus, loadLibrary() is sometimes a better choice. However, dynamically loadable libraries are system-dependent in nature so the use of load() may not compromise system-independence any more than the act of loading the library itself.
The loadLibrary() method requires just the name of a to load:
System.loadLibrary("mylib");
The loadLibrary() method searches for the library. The search performed by loadLibrary() depends on the system you are running on, but typically, it searches the directories listed in one of your environment variables set up to that purpose.
You can explicitly specify your absolute path.
Apart from system independence, there are no real differences security wise.
References :
1) https://security.stackexchange.com/questions/35724/security-implications-of-javas-system-load-vs-system-loadlibrary
2) https://www.chilkatsoft.com/java-loadLibrary-Linux.asp
We have an app that sometimes is installed with an associated app. Both may be installed separately, at different times, and neither is usually in the OS Path environment setting.
IF both apps are installed, the one I'm working on needs to use a JNI library from the other app. This library uses a dozen or so other native libs. While I can FIND the JNI lib, I can't seem to find a way to use it without requiring the user to change their system setup.
I've found the (hacky) technique to add the JNI lib to the java.library.path, I've been unable to find any way of updating the native Path so the JNI lib can find it's associated libs.
The only things that have worked so far are to:
Add the folder that the JNI and associated files are in to the OS path before launching our app.
Launch our app so the Current Working Directory is the JNI lib folder.
Neither of which makes for a hassle-free user experience.
So, is there any way for a Java app to modify it's own environment Path so the JNI lib can find it's associated native libs? (currently testing on Win7, will also need to support OS/X)
This can not be done with an unknown location at run time. According to jni documentation
"To load your shared native library module in Java, simply use Java's System.loadLibrary method in a Java class:"
as well as
"Another common reason for the native library not loading is because it is not in your path. On Windows make sure the path environment variable contains the path to the native library. On Unix make sure that your LD_LIBRARY_PATH contains the path to the native library. Adding paths to LD_LIBRARY_PATH can slow down other programs on your system so you may want to consider alternative approaches. For example you could recompile your native library with extra path information using -rpath if you're using GNU, see the GNU linker documentation (ld man page). You could use a command such as ldconfig (Linux) or crle (Solaris) to add additional search paths to the default system configuration (this requires root access and you will need to read the man pages)."
So thus you need to know the location and must be in a path to be able to load the file. If the location is not known it can not be done due to how jni works and the jvm works.
One way to load a JNI lib with dependent libraries is to load each of the dependents then load the JNI library.
For example, if bar.so is dependent on foo.so and both libraries exist in /some/dir, do the following:
System.load("/some/dir/foo.so");
System.load("/some/dir/bar.so");
Use System.load() instead of System.loadLibrary() so you can specify the absolute path to the library.
You'll have to load all dependents of all loaded libraries unless they can be found in the java.library.path path.
Hope this helps.
I am attempting to launch a Java application using JNLP; the app includes a 3rd party library (Lab Streaming Layer) that relies on native libraries loaded with JNA.
As far as I understand, I have followed all of the appropriate steps to make this work:
All of the native libraries are in JAR files, separated by OS and architecture.
All of the native library JAR files are on the server, accessible from the codebase URL.
The JNLP file contains a <resources> element with os and arch specified with a <nativelib> element inside it with the correct href attribute.
The JNLP file is not malformed and launches normally.
When I go to run my application, I get a java.lang.UnsatisifiedLinkError that reports "Unable to load library 'liblsl32.dylib': Can't obtain InputStream for darwin/liblsl32.dylib". This happens after the program has started up, the first time it tries to make use of the library with the native lib dependency. I have corrected this exact error with this library with a separate, non-web-launch application by putting the dylib in the correct place. I have also tried renaming all the dylibs to be jnilibs, per this information: http://permalink.gmane.org/gmane.comp.java.jna.user/3328, but everything happens identically.
It looks like I should be able to specify jna.library.path in the JNLP file, but I have no idea what this would be, if it would change from launch-to-launch, or how I would find out what it should be.
Native.getWebStartLibraryPath(name) should give you what you need to help JNA find the native library.
You can either construct an absolute pathname or just pass the path to NativeLibrary.addSearchPath(libName, path).
EDITED
You'll also need to name your bundled OS X shared library with a .jnilib suffix in order for web start to recognize it as a native library. This is a naming issue peculiar to OS X.
Edit the advice about using libo to get a single library to support multiple architectures (32 & 64bit) simultaneously, but the underlying issue is 1.6 related while I develop & use 1.8.
The root issue is really that OSX uses the extension .jnilib for libraries that it's going load, rather than .dylib. This means that if you're making a native jar, you should make sure that the extension for the library that you want to load is .jnilib rather than .dylib. Once you make a resource jar containing that .jnilib then it will be found by the JNLP classloader; otherwise JNA uses some heuristics to find it.
So, when creating your jar containing the library, you should make sure that it has a .jnilib extension, and is at the root of the .jar, so for example if we're dealing with liblsl32.dylib, we would do something akin to:
ln -fs liblsl32.dylib liblsl32.jnilib
jar cf lsl32.jar liblsl32.jnilib
This lsl32.jar is the resource that we're going to reference in our .jnlp file. We need to sign it; so I do this using the keystore keystore with the password password using the myself identity.
jarsigner -storepass password -keystore keystore lsl32.jar myself
For the resources I had to put the reference into the resource stanza of the jnlp file for Mac:
<resources os="Mac">
<nativelib href="darwin.jar"/>
<nativelib href="lsl32.jar"/>
</resources>
The references to darwin.jar is needed to load the core JNA library, and lsl32.jar is needed to load the liblsl32.jnilib library.
Instance referencing the library using:
Native.loadLibrary("lsl32", MyClass.class);
should successfully load it.
When signing the .jar files, I made sure that all of them have the relevant Codebase/Permissions attributes on-board, in a file called file.manifest. This file contains:
Permissions: all-permissions
Codebase: http://localhost/demo
... making sure that Codebase matches your own codebase, and you update the jar using:
jar cfm lsl32.jar file.manifest
For debug purposes, if you use lsof of the java process that the jnlp is running within, you can determine if it's using the JNLP loaded library or the JNA loaded library by the presence of the file in the list of loaded files, if it's using JNLPs mechanism, then there will be references to liblsl32.jnilib in the list of items, if it's using JNAs mechanism, then there will be no mention of the file as it's extracted to a temporary file.
In the 1.6 VM case, if you're not architecture compatible with the VM, and you're loading via JNA's load-mechanism, you'll get the error:
no matching architecture in universal wrapper
in amongst the noise.
If you're running across multiple modes (32bit/64bit) you could merge both the 32bit and 64bit libraries into one library, and reference that, or you could just install the 64 bit library.
To put the two together, you would do something like:
lipo -create liblsl.jnilib liblsl32.dylib liblsl64.dylib
then reference liblsl, not liblsl32.
I tend to stick with the newer VM, even with all the security issues. gatekeeper prevents me from loading self-signed apps on the 1.6 VM without turning it off, which makes for an unstable testing environment.
The only webstart issues I've ever had with JNA have been caused by the enhanced security of newer JRE versions.
I can't really call this an answer, but I did finally get this working via brute force. I modified the library I am using to request .jnilib (instead of its original .dylib), then created a native library called darwin.jar that contains the following structure:
/META-INF/MANIFEST.MF
/META-INF/INTERAC.DSA
/META-INF/INTERAC.SF
/liblsl32.jnilib
/liblsl32.dylib
/liblsl64.jnilib
/liblsl64.dylib
/darwin/liblsl32.jnilib
/darwin/liblsl32.dylib
/darwin/liblsl64.jnilib
/darwin/liblsl64.dylib
Unclear what about that works, but apparently this combination contains some permutation of things I had not otherwise tried. I am out of time or energy to troubleshoot further and find an exact answer. I suspect the .dylib versions of the files are not needed, since I explicitly changed the code to look for .jnilib, but they are in there for completeness sake.
If adding user-defined library in Eclipse, one has an ability to set "Native library location". This field allows to enter some directory path.
When does this path plays a part?
Eclipse uses this information to build the java.library.path when it launches a Java program.
Background: Some Java frameworks depend on native code. This code usually comes in the form of a native shared library (*.so, *.dll). In Java, you can see methods with the attribute native. The code will load the shared library using System.loadLibrary().
In order to make the code independent of absolute paths, you just pass the name of the shared library to System.loadLibrary(). The System property java.library.path is then used to determine in which directories the VM should look to locate the file.
Together with Eclipse's feature to define user libraries, you can easily add Java libraries that depend on native code to your projects.
Are you referring to the Java Build Path configuration?
You may need this location if your project uses JNI or JNA. This directory is the location of native code (e.g. a Windows DLL written in C.)
I don't think this information is actually required until you try to run the code. You could provide this information via the Run Configuration for example.
I have an app(for CentOs) that uses two custom made shared libraries "libMyInit.so" (linked with libMyInit.so.1 and libMyInit.so.1.0.1) and libMyUtil.so (linked with libMyUtil.so.1 and libMyUtil.so libMyUtil.so.1.0.1). Thes are present inside the app packaging like MyApp/bin/libMyInit.so & MyApp/bin/util/libMyUtil.so along with their respective linked libraries.
When I have to run the app I have to explictly add the above mentioned paths to LD_LIBRARY_PATH without which it give the error like:
ERROR:Unable to load native MyUtil library. Reason: no MyUtil in java.library.path.
Is there any way I can avoid the manual adding of libraries to LD_LIBRARY_PATH ?
Something that I can do at compile time of shared libraries or later so that the custom made .so libraries are found in the java.library.path ?
IMHO the best way of doing this, is to use a custom System.loadLibrary in the static part of your classes. Have a look at my Java portaudio bindings here#github, especially at the LoadLibrary and JPA classes.
The LoadLibrary class selects a native library based on OS type and architecture, unpacks the lib from its JAR into the temp folder and loads it from there.
JPA now just has to call LoadLibrary.load and everything works automagically on all supported OSs and you only need one single JAR file.
One of the ways is to copy those libraries to one of /usr/lib, /lib, /usr/local/lib directories.