On a recent question, I got comments asking whether I was using "native code" in my application. Now, I know that there is some way to call code in traditional binary libraries (DLLs, SOs) from inside a Java application using a thing called "JNI". I have read that Wikipedia entry but I never used this.
I am using a number of libraries, some of which may or may not use native code. How do I find out if they do? I did not have to install any SOs (running on Linux), but I guess that doesn't mean the libraries are not using any? Do I have to browse through all the documentation (which varies greatly in quality between libraries) or can I do some analysis on the JARs?
Usually using libraries that require JNI requires some additional setup (like putting .dll or .so files in the right places or setting the java.library.path System property).
If you did nothing of that, then chances are that you're not using JNI anywhere. This is also somewhat likely, as only quite specialized libraries require JNI.
However there's also JNA, which is a wrapper around JNI which simplifies its usage and which sometimes makes it unnecessary to do any explicit setup. If that's used by one of your libraries, then it's harder to detect.
If you get a crash dump, then checking that for any non-standard libraries can give a hint if a user-loaded native library is to blame.
To nitpick, every single Java application uses JNI indirectly at least. For example, System class contains several native methods, which map to the native JRE (as can be seen from its source code).
Whether your program/libraries use (directly or indirectly) some other native functions than those contained in the standard JRE, is indeed hard to detect. The .dlls / .sos may be packed into the .jar, to be extracted only when needed, so not having to install native libraries doesn't guarantee that it doesn't use any. It should usually be stated in the library documentation, though, because the vendor probably won't provide binaries for every imaginable system that Java runs on. But to be sure, I think the only way is to scan through the source code for native methods.
If a java library (jar) uses a native library (dll on Windows or so on Linux) the chances are that it is a system wide and well known library (such as glibc on Linux) or a custom one. In the last case it is common to pack it within the jar, so you can just open it up with a ZIP decompressor (i.e. 7zip on Windows will do fine) and browse the files. You should see dll files if it is targeted to Windows, so files if targeted to a Unix platform, or even both. The native library files usually are left at the root level of the jar.
If the jar uses custom libraries but it is packed along with an application it is common to leave the native libraries in an external folder with other application files (in this case there is no consensus). In this case you should look for the application launcher (a bat / sh file) or the configuration file if the lanucher is binary (an ini / conf file) and find out the JVM configuration (where java.library.path points to).
Probably the easiest way of doing this is using jmap or pmap to show which .so (shared object) files are mapped into your Java process. If there's anything other than stuff that's in /lib, /usr/lib or the Java lib directory it's a JNI suspect. You can also look at the /proc entry for the Java process under /proc/<pid>/maps. See the following manpages:
http://linux.die.net/man/1/jmap-java-1.6.0-openjdk
http://linux.die.net/man/1/pmap
http://linux.die.net/man/5/proc
Related
I found some reference that says I can add #!/usr/bin/java --source 12 at the beginning of the file and run the file directly from the terminal.
I am able to run using this on my local machine, however, when I tried the same on Github action I'm getting the error error: invalid value for --source option: 12
I'm not really an expert in shell scripting or java, can someone help me understand what this --source means is it the java version, I tried setting up the same version (jdk18) on Github action but still did not work.
java (the runtime executable) can only run class files. Until, that is, java12, where a normal JDK distribution (and not the bizarro JREs that some packagers like Azul publish ^1) has a java.exe that can run java files straight up. It's simple sugar - the compiler is still involved, of course. It's just that java will execute it for you.
You don't need --source 12; just java MySourceFile.java is fine. Edit that comment at the top of your source file, it should just be #!/usr/bin/java. The thing you do need is that java on your command line PATH is a java v12 or higher, and it is not. There isn't anything your java source file can do about this, you're going to have to impose on your users to have at least java12 installed or this simply does not and can not be made to work. It looks like you're on some linux distro or other; apt or yum or snap or whatever package manager you have will tell you how to fix this: Install java17, and uninstall the rest or use java-alternatives or whatever mechanism your package manager has to set the default executable (the one /usr/bin/java links to). Read the documentation of the package supplying java17, possibly following some links to the generalized java infrastructure package, it should tell you.
Mostly this is a red herring, this just isn't how java files are distributed. It's virtually pointless because:
Java is not a language that tends to be used for quick shell script-esque things. Such things tend to be self-fulfilling prophecies: Because nobody does this, library authors aren't thinking about it when they develop their APIs and the users of their libraries won't file enhancement requests for this either. Because common libraries aren't convenient when used for quick off-the-cuff shell scripts, java isn't used for it, thus perpetuating the cycle.
Any serious java app would definitely involve packages, dependencies, and more - and such apps cannot be run like this.
Class files are as platform agnostic as source files are. There is no sane reason to distribute java-written shell-script-esque tooling as a source file instead of a jar, except for off-the-cuff editing off them, which gets you right back to point #1 and #4.
The java core APIs work on a model of lowest common denominator: If there is a major OS that cannot or doesn't work in a certain way, then java simply does not expose this at all. For example, on all posix systems (i.e. pretty much every major OS except windows), you have your usual TERM, KILL, HUP, etc signals. Java core libs don't let you interact with them (unless you dip into hidden sun.misc.* API which doesn't reliably work in the first place). This makes java extra unsuitable for quick command line scripting where you want a different model: If at least one OS can do it, the language should have a library for it, and that library should simply fast-crash if you attempt to use it on an OS that doesn't support it. One easy way around this is a third party library that adds support for OS-specific stuff, but your model of distribution (stick #!/usr/bin/java at the top and distribute a source file) cannot include dependencies.
Java as a runtime model is mostly focused on running things eventually very quickly, at the cost of starting off slowly. This is fantastic for web servers which need to run efficiently but will be running for quite some time. It's utterly unsuitable for shell scripting, though.
CONCLUSION: You don't want to stick #!/usr/bin/java at the top even if you could make it work.
[1] A JRE is a java distribution without compilers and other development tools like jstack. These cannot run java SomeSourceFile.java, obviously; they do not have a compiler. However, JREs died - there are no JREs anymore; JDK8 is the last one that shipped with an official JRE. The JRE serves as a distribution model: The end user installs a JRE, and you ship your jars to them. This model is obsolete (you are now responsible to get something that can run your class files on the deployment machine), and therefore JREs died. However, some packagers of OpenJDK builds, such as Azul, still publish them, confusing matters. Hence, 'bizarro'. Azul and co have relatively good reasons for doing it, but, you shouldn't be using these unless you really know what you are doing.
I have some Java-app and a customer with some UWP-app implemented in C#, distributed through the Windows Store etc., who wants to use some pieces of my app. Those pieces are pretty OS-independent, only parsing of some special binary file formats, applying some business logic configured using YAML files and stuff. No network, GUI, only some accesses to files etc.
We currently use IKVM to make the code of interest available to C# but ran into different problems already. Some were supporting .NET Core, some had to do with the native toolchain in Release etc. While right now things seem to work after applying some workarounds, I'm looking for alternatives to IKVM already a bit.
The only thing I currently use of IKVM is simply creating a DLL of my code using ikvmc, which can then be referenced in the UWP-project. The compiler is summarized like the following:
The ikvmc tool converts Java bytecode to .NET dll's and exe's.
That's where the support to create native Windows images of GraalVM came into my mind. Others seem to already build native binaries for Windows and according to the docs, GraalVM is able to create shared libs using "--shared". From my understanding, IKVM implements a JVM in .NET and maps things as needed and possible. That sounds pretty much like what "Substrate VM" does in case of a native image, doesn't it?
This executable includes the application, the libraries, the JDK and
does not run on the Java VM, but includes necessary components like
memory management and thread scheduling from a different virtual
machine, called “Substrate VM”. Substrate VM is the name for the
runtime components (like the deoptimizer, garbage collector, thread
scheduling etc.).
https://www.graalvm.org/docs/reference-manual/native-image/
So, is there any chance that a native image in form of a DLL can replace the DLL created by ikvmc currently? Did anyone try that already and has any experiences? Did anyone try already to create a native DLL and consume that in some other native Windows app? From my understanding UWP "only" applies additional restrictions which one might be able to work around again. Or is this approach totally impossible for some reasons?
Thanks all for your input!
I'm not very familiar with the IKVM project, so this answer is mostly about the generic question:
Can you create a native DLL/shared library and consume that in some other native Windows app?
It should be possible. You can compile Java code into a shared library. The entry points are marked with the #CEntrypoint annotation.
You can then use the generated shared library and the header files to consume your library from a native application.
This way for example GraalVM distributions use the GraalVM JIT compiler by default:
The GraalVM JIT is written in Java
Compiled as a shared library with the native-image
Used in Hotspot.
Here's a page describing how to consume those from Java through the JNI: https://www.graalvm.org/reference-manual/native-image/ImplementingNativeMethodsInJavaWithSVM/
which could be very similar to how would you use a shared library from a C# application.
GraalVM native images are not very flexible, unlike IKVM.NET images. Unless you like writing wrappers and playing with P/Invoke, you should stick to IKVM.NET.
NOTE: I am behind an IKVM.NET fork
I am attempting to port an application that was written with a combination of c++ for the back end, and java for the front end. This application relies on the library opencv 2.4.13, which is outdated, as well as multiple other libraries. The concern i have is that i do not want the end user to need to install these dependant programs, as they have been proving challenging to install on any but a select few linux distributions. I believe the term i am looking for is statically linking, but i'm a bit unfamiliar with c++ compilation at the moment, so i am unsure the steps i need to take to make these files portable. The java application requires these files to be libraries, and while i have managed to get them to compile on one machine, the problem seems to be getting them to run on a different one after compilation.
Don't bother - this might also give you licensing problems, depending on what libraries you need.
Instead, just figure out what platforms your application is supposed to run on and package the libraries for each platform with your jar - or download them at startup, or provide them as a separate package. The exact mechanics you choose depend on your use case, the point is you don't need to rely on system wide installs.
My first guess was that a *.so (unix) or *.dll (win) is a shared library. At least that is my usage writing code in C.
Now the context is developing for Android, which is done in Java->Dalvik style.
I have troubles with a perceived ambiguity of the the term shared library there. In java it seems that *.jar files or *.dex files contain the code that is somewhat both, shared and a library.
Is there a reference to tell what is meant by shared library?
A very specific problem is that Android developers aapt packaging thing is having the option --shared-lib
--shared-lib
Make a shared library resource package that can be loaded by an application at runtime to access the libraries resources. Implies
--non-constant-id.
which I have no clue what now is meant by shared library. I am aware that there is this JNI interface and some Android NDK stuff might also have *.so files which might be meant, but I am really not sure.
On Android developers code in Java, the classes he writes are compiled into Java bytecode (bundled in jar along with other classes automatically created during the building process). once created, this bytecode is translated to Dalvik bytecode (.dex) which is included in the application bundle (apk).
In addition to the Java code, application can be programmed using "native libraries" written in C code and compiled with the android-ndk for the device specific architecture, generating .so files (dynamically linked) or .a libraries (linked statically). Normally this is done for performance issues or reusing libraries written in C and avoid porting them to Java. As you said in the question, this binding is done via the JNI library.
These libraries must be included in the application bundle (apk) and loaded at runtime by the application with System.loadLibrary().
I know it's possible to load a DLL (Windows library) in java.
And I know that a compiled java program runs anywhere..
Can a java program that loads a DLL run in a Unix environment ? (if the DLL file is present there)
No. A DLL runs native Windows instructions that are not compatible with a UNIX operating system. However shared libraries (.so) can be accessed using JNI.
To ensure a portable "Write once run anywhere" model, calls to native libraries should be avoided in favor of a pure Java implementation.
I would be very surprised. The DLL (provided it's native) will be built with OS and platform-specifics.
No. You can't take a Windows PE/COFF DLL and stick it in a Unix environment, it just won't work. (Unless you're talking about Cygwin/MSYS)
You will have to compile the library for the target system and use that.
It is possible to load the DLL in unix indeed. However many dlls use the windows API which is not available on UNIX. You must also check for copyright issues that you may encounter doing that. Native library access is possible using JNI. But this (loading DLL in unix) is not a supported and reliable configuration.
To give you a brief answer and a short advice: the answer is no and the advice is to not waste your time doing this.