Has anyone developed JNI bindings to call OpenSSL's hash functions from Java?
I am aware that Java has hash functions built in. The problem is that these run at only a third the speed of the OpenSSL native implementations. There is an OpenSSL-Java project on sourceforge, but it has no files to download.
My goal here is actually not to call OpenSSL, it is to overcome Java's suboptimal performance regarding hashing.
Try https://www.nayuki.io/page/native-hash-functions-for-java they implement hash functions in c, x86 assembly and x86-64 assembly
Boost up to 1.68× for SHA-256, but I see no difference with small files.
So If you tend to use with small file, I suggest stick with java.
I strongly believe you have to write a C routine that not only calculated the hashes, but also does the file IO. Don't forget to use memory mapped IO for speed, else the limiting part is you copying buffers.
Apparently you can but it's because of JNI you don't get the performance boost.
Related
I'm researching methods for computing expensive vector operations in Java, e.g. dot-products or multiplications between large matrices. There are a few good threads on here on this topic, like this and this. It appears that there is no reliable way of having the JIT compile code to use CPU vector instructions (SSE2, AVX, MMX...). Moreover, high-performance linear algebra libraries (ND4J, jblas, ...) do in fact make JNI calls to BLAS/LAPACK libraries for the core routines. And I understand BLAS/LAPACK packages to be the de facto standard choices for native linear algebra computations.
On the other hand others (JAMA, ...) implement algorithms in pure Java without native calls.
My questions are:
What are the best practices here?
Is making native calls to BLAS/LAPACK actually a recommended choice? Are there other libraries worth considering?
Is the overhead of JNI calls negligible compared to the performance gain? Does anyone have experience as to where the threshold lies (e.g. how small an input should be to make JNI calls more expensive than a pure Java routine?)
How big is the portability tradeoff?
I hope this question could be of help both for those who develop their own computation routines, and for those who just want to make an educated choice between different implementations.
Insights are appreciated!
There are no clear best practices for every case. Whether you could/should use a pure Java solution (not using SIMD instructions) or (optimized with SIMD) native code through JNI depends on your particular application and specifically the size of your arrays and possible restrictions on the target system.
There could be a requirement that you are not allowed to install specific native libraries in the target system and BLAS is not already installed. In that case you simply have to use a Java library.
Pure Java libraries tend to perform better for arrays with length much smaller than 100 and at some point after that you get better performance using native libraries through JNI. As always, your mileage may vary.
Pertinent benchmarks have been performed (in random order):
http://ojalgo.org/performance_ejml.html
http://lessthanoptimal.github.io/Java-Matrix-Benchmark/
Performance of Java matrix math libraries?
These benchmarks can be confusing as they are informative. One library may be faster for some operation and slower for some other. Also keep in mind that there may be more than one implementation of BLAS available for your system. I currently have 3 installed on my system blas, atlas and openblas. Apart from choosing a Java library wrapping a BLAS implementation you also have to choose the underlying BLAS implementation.
This answer has a fairly up to date list except it doesn't mention nd4j that is rather new. Keep in mind that jeigen depends on eigen so not on BLAS.
Openssl is known to be a standard in crypto world. Openssl is written in pure C, so the problem comes that how to use Opnessl in different languages? such Java. Though Java provides JCE and BouncyCastle, it's hard to know whether they have implementation deviations, i.e., for same input(legal or invalid), JCE and Openssl output different results.
What I come up with so far is to invoke openssl command directly. In real products, how to solve this problem?
Any help is really appreciated.
OpenSSL is using standardized algorithms, this means that no matter which library you use, the decryption result for the same key and algorithm will be the same.
For example:
If you take a BlowFish encryption algorithm with the key "I am a blowfish", and encrypt it with OpenSSL, you can use any library you want to decrypt it and get the same result.
Having said that, it IS possible to use OpenSSL libraries (usually libcrypto and libssl) in any language or OS. It does require some piping though.
In your example, you can use JNI to access OpenSSL library functions directly.
This approach requires that you make sure you have the right libraries for your target architecture and OS.
Personally? I'd see check why there are deviations as that is your main problem.
how to use Opnessl in different languages? such Java.
In Java use JNI. It can be very fast. Pay attention to JNI notion of “pinning” - the native method receives a direct pointer to the elements - without copying - but this has some implications. Pinning is optional. To use JNI one should usually create wrapper functions to invoke target functions (OpenSSL in our case). So you should create shared library (DLL or so) according to JNI rules (argument types jint, etc) and call OpenSSL function.
It depends on language - in general in managed languages you should wrap C code (in COM or Managed C++ for example to use in .NET). In C++ - directly. C# also has mechanism of invoking C functions.
I'm building an encryption application on Android.
Using library encrypt in Java.
But it's slowly during encrypt processing.
I think about using dll encrypt wrote by C++.
But does it actually make encrypt processing faster ?
If you have experience about that, please help me.
The only way to tell this is by profiling. C++ should be faster, but JNI overhead could slow it down. So could bad programming. Also note that faster isn't always better for encryption- there are attacks on encryption implementations that are based on detecting that the programmer saw certain optimizations (such as not performing a calculation in 1 branch of an if statement) that lets it guess the values of certain keybits. Take a well written, open source, debugged library over one that's faster. And never write your own encryption unless you're an expert- even if you write the algorithm perfectly you'll have side vector attacks like the one above in it.
Does anybody know any libraries of CL-procedures (it will be better if there is a good documentation)?
And Im also interested in D-languages binding.
Has somebody seen benchmarks that compares performance of native code applications with OpenCL and/or OpenGL and performance of Java Binding? I know that DLL calls cause of performance decline. Does an application written on C/C++ will be anyway faster than the same on Java?
As Jakob already said, my D wrapper is # https://github.com/Trass3r/cl4d
With inlining, -version=NO_CL_EXCEPTIONS and proper dead code elimination the code should be nearly equivalent to a manually coded app using the C API directly.
So the wrapper introduces almost no overhead, performance depends on your kernels and clever memory transport.
How about JavaCL which works for me ?
As far as I have seen the cost of binding is fairly small compared to other overheads such as compiling the CL code and exchanging data with the GPU.
Which is the fastest way of calling a native library from Java?
The ones I know about are
NativeCall - what we're currently using
JNA - haven't used it, but looks reasonable
JNI - looks horrendous to write, but we'll do it if we get the speed
Swig makes JNI easier too.
In terms of speed, I suspect there will be subtle variations - I strongly suggest you pick a call that you know you'll be making a lot, and benchmark all of the solutions offered.
JNI is the fastest. JNA is very slow compared to JNI (the call overhead is probably one order of magnitude), but it is a fantastic library because it makes native access so easy. JNA is great if you need to make an occasional call to some native API. If you care about performance, I wouldn't use it in any "tight loops."
I'm not sure where NativeCall fits in the spectrum.
Quite a few parameters influence the performances of interfaces between programing languages: what device the JVM runs on, who developed it (in case it's not the usual Sun JVM), whether you will need to call back Java code from native code, the threading model of the JVM on your operating system and how asynchronous will the native code be...
You may not find a reliable benchmark that measures exactly what you need, I'm afraid.
This blog entry claims that due to the introspection mechanisms used by JNA, it'll be significantly slower than JNI. I suspect that NativeCall will use similar mechanisms and thus perform in a similar fashion.
However you should probably benchmark based on the particular objects you're referencing and/or marshalling between Java and C.
I would second the recommendation of SWIG. That makes life particularly easy (easier) for the Java/C interfacing.