Packaging of JCUDA 0.6.0 on Mac OSX 10.9.3 - java

I was able to compile the native 0.6.0 libraries of JCuda on a Mac OSX 10.9.3 64-bit system (http://www.jcuda.org/downloads/downloads.html). Unfortunately Im not smart enough to package everything into a jar. Can someone help me to do so? I created a Netbeans project and copied the contents of all provided Java src folders into it. Then I packaged everything into a jar. Using Jarsplice 0.4.0 I then tried to build a Jar file containing all the native library files listed below (except the first .a file). Unfortunately when I linked the resulting jar into my project and run it, it wasnt working since a libJCudaRuntime-apple-x86_64.jnilib was missing.
Thank you in advance
Radek
My lib folder contains:
libCommonJNI.a
libJCublas-apple-x86_64.dylib
libJCublas2-apple-x86_64.dylib
libJCudaDriver-apple-x86_64.dylib
libJCudaRuntime-apple-x86_64.dylib
libJCufft-apple-x86_64.dylib
libJCurand-apple-x86_64.dylib
libJCusparse-apple-x86_64.dylib

Just to have this answered here as well: The native libraries can not be packed directly into a JAR (or rather: They can currently not be loaded if they are only contained in the JAR).
Although the infrastructure of the JCuda LibUtils class basically offers the possibility to do this, it is not "officially" supported. One reason for that is that this would only make sense when all native libraries for all operating systems were available (in 32 and 64 bit). Then it would be possible to put all these libraries into a JAR, and unpack and load them transparently at runtime.
It usually takes a while until all native libraries are available. The problem may be alleviated with things like https://github.com/MysterionRise/mavenized-jcuda , which manages the dependencies to the native libraries.

Related

MCP 1.8.9 Exception in thread "main" java.lang.UnsatisfiedLinkError: no lwjgl64 in java.library.path [duplicate]

I'm working on a Java project that uses the JNI. The JNI calls a custom library that I've written myself, let's say mylib.dll, and that depends on a 3rd party library, libsndfile-1.dll.
When I run my program it crashes with
java.lang.UnsatisfiedLinkError: C:\...path...\mylib.dll: Can't find dependent libraries.
I've searched this site (and others) and I've tried a number of fixes:
I ran dependency walker. DW gave a couple of warnings -- that two libraries required by libsndfile, MPR.DLL and SHLWAPI.DLL, had "unresolved imports" -- but the DW FAQ said that these warnings could be safely ignored.
I fixed the method names in mylib.dll, as suggested here. The method names had somehow gotten mangled by the compiler, but I added linker flags and the dll method names now match those in my jni header file exactly.
I put all of these DLLs in the same directory -- the same directory as the .jar that calls them -- to ensure that they're on the right PATH.
No dice.
Does anyone have any idea what's going on?
I'm doing my development in Visual Studio 2010 on a MacBook pro (via Parallels). I'm doing my testing in Windows XP on a toshiba laptop.
I'm pretty sure the classpath and the shared library search path have little to do with each other. According to The JNI Book (which admittedly is old), on Windows if you do not use the java.library.path system property, the DLL needs to be in the current working directory or in a directory listed in the Windows PATH environment variable.
Update:
Looks like Oracle has removed the PDF from its website. I've updated the link above to point to an instance of the PDF living at University of Texas - Arlington.
Also, you can also read Oracle's HTML version of the JNI Specification. That lives in the Java 8 section of the Java website and so hopefully will be around for a while.
Update 2:
At least in Java 8 (I haven't checked earlier versions) you can do:
java -XshowSettings:properties -version
to find the shared library search path. Look for the value of the java.library.path property in that output.
I want to inform this interesting case, after tried all the above method, the error is still there. The weird thing is it works on a Windows 7 computer, but on Windows XP it is not. Then I use dependency walker and found on the Windows XP there is no VC++ Runtime as my dll requirement. After installing VC++ Runtime package here it works like a charm. The thing that disturbed me is it keeps telling Can't find dependent libraries, while intuitively the JNI dependent dll is there, however it finally turns out the JNI dependent dll requires another dependent dl. I hope this helps.
You need to load your JNI library.
System.loadLibrary loads the DLL from the JVM path (JDK bin path).
If you want to load an explicit file with a path, use System.load()
See also: Difference between System.load() and System.loadLibrary in Java
If you load a 32 bit version of your dll with a 64 bit JRE you could have this issue. This was my case.
Please verify your library path is right or not. Of course, you can use following code to check your library path path:
System.out.println(System.getProperty("java.library.path"));
You can appoint the java.library.path when launching a Java application:
java -Djava.library.path=path ...
Did have identical problem with on XP machine when installing javacv and opencv in combination with Eclipse. It turned out that I was missing the following files:
msvcp100.dll
msvcr100.dll
Once these were installed, the project compiled and ran OK.
When calling System.loadLibrary(), the JVM will look on the java.library.path for your native library. However, if that native library declares any dependencies on other native libraries, then the operating system will be tasked with finding those native library dependencies.
Since the operating system has no concept of the java.library.path, it will not see any directories you place on the java.library.path. Instead, it will only search the directories on PATH environment variable of the operating system. This is totally fine if the native library dependency is an operating system native library because it will be found on the PATH. However, if the native library dependency is a native library that you or someone else created, then it will not be found on the PATH unless you place it there. This behavior is strange, unexpected, and not well documented, but it is documented in the OpenJDK issue tracker here. You can also find another StackOverflow answer reinforcing this explanation, here.
So, you have a couple of options. You could either load each native library in the correct dependency order using System.loadLibrary(), or you could modify the PATH to include the directories where your native libraries are stored.
Short answer: for "can't find dependent library" error, check your $PATH (corresponds to bullet point #3 below)
Long answer:
Pure java world: jvm uses "Classpath" to find class files
JNI world (java/native boundary): jvm uses "java.library.path" (which defaults to $PATH) to find dlls
pure native world: native code uses $PATH to load other dlls
I found a great article by some friends at keepsafe that went through the same thing I did. It worked for me, so hopefully it helps you out as well! Have a read if you're interested (The Perils of Loading Native Libraries on Android) or just use
compile 'com.getkeepsafe.relinker:relinker:1.2.3'
and replace
System.loadLibrary("myLibrary");
with
ReLinker.loadLibrary(context, "mylibrary");
installing Microsoft Visual C++ 2010 SP1 Redistributable Fixed it
I used to have exactly the same problem, and finally it was solved.
I put all the dependent DLLs into the same folder where mylib.dll was stored and make sure the JAVA Compiler could find it (if there is no mylib.dll in the compilation path, there would be an error reporting this during compiling). The important thing you need to notice is you must make sure all the dependent libs are of the same version with mylib.dll, for example if your mylib.dll is release version then you should also put the release version of all its dependent libs there.
Hope this could help others who have encountered the same problem.
I had the same issue, and I tried everything what is posted here to fix it but none worked for me.
In my case I'm using Cygwin to compile the dll. It seems that JVM tries to find the JRE DLLs in the virtual Cygwin path.
I added the the Cygwin's virtual directory path to JRE's DLLs and it works now.
I did something like:
SET PATH="/cygdrive/c/Program Files/Java/jdk1.8.0_45";%PATH%
In my situation, I was trying to run a java web service in Tomcat 7 via a connector in Eclipse. The app ran well when I deployed the war file to an instance of Tomcat 7 on my laptop. The app requires a jdbc type 2 driver for "IBM DB2 9.5". For some odd reason the connector in Eclispe could not see or use the paths in the IBM DB2 environment variables, to reach the dll files installed on my laptop as the jcc client. The error message either stated that it failed to find the db2jcct2 dll file or it failed to find the dependent libraries for that dll file. Ultimately, I deleted the connector and rebuilt it. Then it worked properly. I'm adding this solution here as documentation, because I failed to find this specific solution anywhere else.
Creating static library worked for me, compiling using g++ -static. It bundles the dependent libraries along with the build.
place the required dlls in folder and set the folder path in PATH environment variable.
make sure updated environment PATH variable is reflected.
I was facing same issue with ffmpeg library after merging two Android projects as one project.
Actually issue was arriving due to two different versions of ffmpeg library but they were loaded with same names in memory. One library was placed in JNiLibs while other was inside another library used as module. I was not able to modify the code of module as it was readonly so I renamed the one used in my own code to ffmpegCamera and loaded it in memory with same name.
System.loadLibrary("ffmpegCamera");
This resolved the issue and now both versions of libraries are loading well as separate name and process id in memory.
I faced the same problem after migrating my CI into a new machine.
I was still facing it even after applying all the above solutions.
The problem was in my new machine, there was Microsoft Visual C++ 2010 SP1 Redistributable x86 installed in it. But my new machine was having 64-bit CPU and operating system. So the fix was that i just updated and installed the 64 bit version from here .
Go to http://tess4j.sourceforge.net/usage.html and click on Visual C++ Redistributable for VS2012
Download it and run VSU_4\vcredist_x64.exe or VSU_4\vcredist_x84.exe depending upon your system configuration
Put your dll files inside the lib folder, along with your other libraries (eg \lib\win32-x86\your dll files).

Struts2 and PI4J [duplicate]

I'm working on a Java project that uses the JNI. The JNI calls a custom library that I've written myself, let's say mylib.dll, and that depends on a 3rd party library, libsndfile-1.dll.
When I run my program it crashes with
java.lang.UnsatisfiedLinkError: C:\...path...\mylib.dll: Can't find dependent libraries.
I've searched this site (and others) and I've tried a number of fixes:
I ran dependency walker. DW gave a couple of warnings -- that two libraries required by libsndfile, MPR.DLL and SHLWAPI.DLL, had "unresolved imports" -- but the DW FAQ said that these warnings could be safely ignored.
I fixed the method names in mylib.dll, as suggested here. The method names had somehow gotten mangled by the compiler, but I added linker flags and the dll method names now match those in my jni header file exactly.
I put all of these DLLs in the same directory -- the same directory as the .jar that calls them -- to ensure that they're on the right PATH.
No dice.
Does anyone have any idea what's going on?
I'm doing my development in Visual Studio 2010 on a MacBook pro (via Parallels). I'm doing my testing in Windows XP on a toshiba laptop.
I'm pretty sure the classpath and the shared library search path have little to do with each other. According to The JNI Book (which admittedly is old), on Windows if you do not use the java.library.path system property, the DLL needs to be in the current working directory or in a directory listed in the Windows PATH environment variable.
Update:
Looks like Oracle has removed the PDF from its website. I've updated the link above to point to an instance of the PDF living at University of Texas - Arlington.
Also, you can also read Oracle's HTML version of the JNI Specification. That lives in the Java 8 section of the Java website and so hopefully will be around for a while.
Update 2:
At least in Java 8 (I haven't checked earlier versions) you can do:
java -XshowSettings:properties -version
to find the shared library search path. Look for the value of the java.library.path property in that output.
I want to inform this interesting case, after tried all the above method, the error is still there. The weird thing is it works on a Windows 7 computer, but on Windows XP it is not. Then I use dependency walker and found on the Windows XP there is no VC++ Runtime as my dll requirement. After installing VC++ Runtime package here it works like a charm. The thing that disturbed me is it keeps telling Can't find dependent libraries, while intuitively the JNI dependent dll is there, however it finally turns out the JNI dependent dll requires another dependent dl. I hope this helps.
You need to load your JNI library.
System.loadLibrary loads the DLL from the JVM path (JDK bin path).
If you want to load an explicit file with a path, use System.load()
See also: Difference between System.load() and System.loadLibrary in Java
If you load a 32 bit version of your dll with a 64 bit JRE you could have this issue. This was my case.
Please verify your library path is right or not. Of course, you can use following code to check your library path path:
System.out.println(System.getProperty("java.library.path"));
You can appoint the java.library.path when launching a Java application:
java -Djava.library.path=path ...
Did have identical problem with on XP machine when installing javacv and opencv in combination with Eclipse. It turned out that I was missing the following files:
msvcp100.dll
msvcr100.dll
Once these were installed, the project compiled and ran OK.
When calling System.loadLibrary(), the JVM will look on the java.library.path for your native library. However, if that native library declares any dependencies on other native libraries, then the operating system will be tasked with finding those native library dependencies.
Since the operating system has no concept of the java.library.path, it will not see any directories you place on the java.library.path. Instead, it will only search the directories on PATH environment variable of the operating system. This is totally fine if the native library dependency is an operating system native library because it will be found on the PATH. However, if the native library dependency is a native library that you or someone else created, then it will not be found on the PATH unless you place it there. This behavior is strange, unexpected, and not well documented, but it is documented in the OpenJDK issue tracker here. You can also find another StackOverflow answer reinforcing this explanation, here.
So, you have a couple of options. You could either load each native library in the correct dependency order using System.loadLibrary(), or you could modify the PATH to include the directories where your native libraries are stored.
Short answer: for "can't find dependent library" error, check your $PATH (corresponds to bullet point #3 below)
Long answer:
Pure java world: jvm uses "Classpath" to find class files
JNI world (java/native boundary): jvm uses "java.library.path" (which defaults to $PATH) to find dlls
pure native world: native code uses $PATH to load other dlls
I found a great article by some friends at keepsafe that went through the same thing I did. It worked for me, so hopefully it helps you out as well! Have a read if you're interested (The Perils of Loading Native Libraries on Android) or just use
compile 'com.getkeepsafe.relinker:relinker:1.2.3'
and replace
System.loadLibrary("myLibrary");
with
ReLinker.loadLibrary(context, "mylibrary");
installing Microsoft Visual C++ 2010 SP1 Redistributable Fixed it
I used to have exactly the same problem, and finally it was solved.
I put all the dependent DLLs into the same folder where mylib.dll was stored and make sure the JAVA Compiler could find it (if there is no mylib.dll in the compilation path, there would be an error reporting this during compiling). The important thing you need to notice is you must make sure all the dependent libs are of the same version with mylib.dll, for example if your mylib.dll is release version then you should also put the release version of all its dependent libs there.
Hope this could help others who have encountered the same problem.
I had the same issue, and I tried everything what is posted here to fix it but none worked for me.
In my case I'm using Cygwin to compile the dll. It seems that JVM tries to find the JRE DLLs in the virtual Cygwin path.
I added the the Cygwin's virtual directory path to JRE's DLLs and it works now.
I did something like:
SET PATH="/cygdrive/c/Program Files/Java/jdk1.8.0_45";%PATH%
In my situation, I was trying to run a java web service in Tomcat 7 via a connector in Eclipse. The app ran well when I deployed the war file to an instance of Tomcat 7 on my laptop. The app requires a jdbc type 2 driver for "IBM DB2 9.5". For some odd reason the connector in Eclispe could not see or use the paths in the IBM DB2 environment variables, to reach the dll files installed on my laptop as the jcc client. The error message either stated that it failed to find the db2jcct2 dll file or it failed to find the dependent libraries for that dll file. Ultimately, I deleted the connector and rebuilt it. Then it worked properly. I'm adding this solution here as documentation, because I failed to find this specific solution anywhere else.
Creating static library worked for me, compiling using g++ -static. It bundles the dependent libraries along with the build.
place the required dlls in folder and set the folder path in PATH environment variable.
make sure updated environment PATH variable is reflected.
I was facing same issue with ffmpeg library after merging two Android projects as one project.
Actually issue was arriving due to two different versions of ffmpeg library but they were loaded with same names in memory. One library was placed in JNiLibs while other was inside another library used as module. I was not able to modify the code of module as it was readonly so I renamed the one used in my own code to ffmpegCamera and loaded it in memory with same name.
System.loadLibrary("ffmpegCamera");
This resolved the issue and now both versions of libraries are loading well as separate name and process id in memory.
I faced the same problem after migrating my CI into a new machine.
I was still facing it even after applying all the above solutions.
The problem was in my new machine, there was Microsoft Visual C++ 2010 SP1 Redistributable x86 installed in it. But my new machine was having 64-bit CPU and operating system. So the fix was that i just updated and installed the 64 bit version from here .
Go to http://tess4j.sourceforge.net/usage.html and click on Visual C++ Redistributable for VS2012
Download it and run VSU_4\vcredist_x64.exe or VSU_4\vcredist_x84.exe depending upon your system configuration
Put your dll files inside the lib folder, along with your other libraries (eg \lib\win32-x86\your dll files).

Making my Java program easily distributable

I have installed the Java 3D API on PC via the exe installer, which simply created a new directory with j3dcore.jar, vecmath.jar, j3dutils.jar in a lib sub-directory and j3dcore-ogl.dll in a bin sub-directory.
Netbeans had no issues and my code compiled and executed smoothly, however once I built my project and tried to run it from the command prompt I got an UnsatisfiedLinkError saying that no j3dcore-ogl in java.library.path.
Google came to the rescue and gave me 3 viable solutions:
by copying the dll file into my JRE's bin directory
by adding the path of the dll file to the library path (java -Djava.library.path=dllpath)
load the dll in the program with System.load() (I couldn't get this one to work, actually)
My question is: Is there an elegant solution to this problem, that I missed?
It seems tedious that for each different PC someone would like to use this program on, he'd have to either copy the dll or add it to the library path before it can run. (Side question: How come Netbeans didn't have a problem with the dll?)
Making my Java program easily distributable
If you mean 'easy for the end user' look to Java Web Start.
A passer-by asks:
Can you package the dll dependencies with Web Start?
Yes, but much, much better. You can package the natives for each platform in separate Jars, and supply them only to the platform that uses that native, even so far as partitioning the download between 32 & 64 bit versions of the natives.
JWS puts the natives on the run-time class-path of the application, ready for loading in code.
This all happens automatically for the end user, they click a link, approve the trust dialog(s) when asked, and the application installs - possibly with desktop integration, and appears on screen like magic.
JWS apps. that use natives need to be distributed as all-permissions security level, because the JVM cannot guarantee the actions of anything that 'goes native'.
Edit - After re-reading your question, your issue sounds different. However I'm able to get my running like so, by just dropping all dll files in the same directory as the .bat file starting the java process:
java -classpath ./YourJar.jar;./lib/j3dcore.jar;./lib/vecmath.jar;./lib/j3dutils.jar package.MainClass
And that works on multiple user's PCs, so I know simply dropping it in the working directory works.
I believe it depends on the version of Java being used - 64 bit or 32 bit. The correct dll file (of the same name) needs to be in the working directory.
I think I was getting a similar problem when the wrong dll was being used, and it's not OS-dependent (if your 64 bit OS has 32-bit Java installed, you'd need the 32 bit j3dcore-ogl.dll file).
So the question is, which version of Java are you using (when running outside of your IDE), and which version of the dll are you putting (if any) in the working directory? I don't need any dll files in my path settings to get this working on other's PCs, and did not use System.load(), and did NOT copy files into my user's JRE/bin directory - so I know this is possible without the 3 options you mention.
If you put the dlls in the same directory than you Jar, does it work?
If yes, you could consider distributing it like this.
I guess DLL are searched in all folders in %PATH% on windows. (LD_LIBRARY_PATH for UNIX flavors)
Could you try by adding the path to dll to %path% variable?
It appears that you are trying package a product with many jars as dependencies. You may benefit from One-Jar. It claims to have native dll support.

Is it impossible to embed Java3D in a way that I don't need to install it?

I'm running a big application and a small part of it includes Java 3D, the problem is many users need to use the code, but it isn't practical for everyone to install Java 3D just to run the application if they aren't even going to use that section of the application.
Is it possible through compiling an extra jar, or changing some paths, to include Java 3D in a project without installing it on a system? Or perhaps to manually include any dlls?
The demos at java3d prove that it is possible.
You only need to include the required jar files to your projects distribution.
Yes, it is possible. You will need to pull the required jars and native libraries from the java3d site. I pulled them from the java3d demos, but that requires digging through the xml launch descriptor files.
The place you'll run into the most trouble is when linking to the .so / .dll files. This is typically specified on the classpath before your app starts, but since you don't know what platform you're using until your app starts, it's a catch22.
There are two possible solutions:
1. Bootstrap your program with a simple class that detects the platform and sets up a new jvm (with the proper libraries specified) for the real application.
2. Dynamically load the libraries (I've never actually used this method for native implementations, but I see no reason why it wouldn't be possible).
Unfortunately, neither method is terribly straight forward.
I've done this with no problems (and many users tested that it works, Windows OS) by simply adding the jars to the classpath and dropping the appropriate dlls into the "working" directory where the .bat file running Java started. One catch is I had to have two bat files - one 32 bit and one 64 bit. Each bat file copy/renames the appropriate dll to j3dcore-ogl.dll, since depending on which version of Java (32/64 bit) you're running, a different jar is needed.
Example .bat file:
del j3dcore-ogl.dll
copy j3dcore-ogl-64.dll j3dcore-ogl.dll
java -Xmx2048m -classpath ./YourJar.jar;./lib/tinylaf.jar;./lib/j3dcore.jar;./lib/j3dutils.jar;./lib/vecmath.jar your.package.MainClass
#pause

How can I include platform-specific native libraries in the .JAR file using Eclipse?

I am just starting to learn JNI. I have been following a simple example, and I have created a Java app that calls a Hello World method in a native library. I'd like to target Win32 and Linux x86.
My library resides in a DLL, and I can call it just fine using LoadLibrary when the DLL is added to the root of my Eclipse project.
However, I can't figure out how to get Eclipse to export a runnable JAR that includes the DLL and the .SO file for Linux.
So my question is basically; how would you go about creating a project in Eclipse and include several versions of the same native library?
Thank you,
Martin
For runnable JARs, what you need to do is extract to the temporary directory (maybe in a static { } block) and then load the DLL from that directory (using System.loadLibrary() in the same block). To do this, you need to subclass ClassLoader and override the findLibrary() method to allow libraries to be found in that directory. You can do whatever logic you need to here to load the particular platform libraries. To be honest, the naming for the libraries on the different platforms should be similar anyway -- I believe that you omit the 'lib' part when loading, and the extension. That's the gist of it. Probably easier to use One-JAR as the other poster mentioned :)
You might want to check out the One-JAR project. It lets you package your application and its dependencies (including native libraries) to a single jar file.

Categories