AES GCM with hardware accelerated support before Java 9 - java

My use case involves encrypting large amounts of sensitive data on the order of petabytes using AES with GCM in Java and I will need high throughput to make that task complete in reasonable time.
As was previously asked and answered on here, AES/GCM is still severely crippled in the latest JDK 8 (~17MB/s on my hardware). I'm happy to know that JEP 246 for adding significant hardware acceleration for GCM made it onto the roadmap for Java 9, however General Availability for the next major Java release has been pushed out to over a year away as of me writing this.
So far the fastest implementation I have found is the Bouncy Castle Provider, with no hardware acceleration and throughput on the order of 170MB/s. I know that my hardware supports acceleration by running benchmarks of the native libraries GnuTLS (gnutls-cli --benchmark-ciphers) and OpenSSL (openssl speed -evp aes-128-gcm), both clocking in at over 2GB/s for AES/GCM. Unfortunately I can't find a current Java wrapper around either of these. The closest I've come across is Apache JuiCE, built on OpenSSL, but it was abandoned over 8 years ago.
Has anyone come across a JCE Provider that supports hardware acceleration for AES/GCM that I could use in the near term, either commercial or open source?

Netty-tcnative provides an alternative SSL engine using JNI to interface with native OpenSSL. You should be able to access the crypto functions directly in a similar way. That said, JEP 246: Leverage CPU Instructions for GHASH and RSA is now part of Java 9, so you should be able to get similar performance without resorting to native code.

Related

Encryption & Decryption for Mobile Webservice

I'm working on a mobile App which is to be build in Android (Native) , iOS (Native) & PhoneGap. For security I'm already using SSL, but as per client requirement another encryption is to be implemented in all webservices( Mobile end and Server) . But I'm unable to implement encryption which works well in Java, Objective-C and JavaScript.
I could manage to get AES-256 working in all the platforms , but it works very slow in Android. Library used for the same was RNCryptor.
Can you please suggest me any Encryption/Decryption library which is compatible along at least Java & Objective-C.
AES-256 a correct choice and should not be a performance problem. Most cpu chips include special instructions to allow faster implementations, Apple ARM chips do as do may Intel chips. If you are going to claim that encryption is slow for an implementation you need to supply test times for all platforms, generate them and post them.
Obtaining the same results from encryption, AES-256 in this case, is simply supplying the exact same inputs with the exact required lengths and exact same options--that is all.
Providing secure encryption is more than just a key, data and an AES-256 library. There needs to be an iv, if the key is week it needs to be extended, passwords are generally extended with PBKDF2 or it's like. These require more information to be added to the encrypted data that is passed. There is also data padding such as PKCS#7. RNCryptor handles all this but for interoperability requires the other-end to use the same scheme. Then there is the issue of securing the encryption key and exchanging it with the other-side.

Is it possible to create a Hybrid GPU accelerated application in Java that utilize CUDA & OpenCL?

In my experience applications written in CUDA run faster than written in OpenCL when run on the same NVidia hardware.
How can this capability be utilized without losing the cross-platform capabilities of OpenCL?
I suspect it may be possible to create a "failback" system where, if there are no NVidia devices available and/or no CUDA version of the requested kernel, then the system would failback to utilizing the OpenCL version. Alternatively, large tasks could be load balanced across NVidia and non-NVidia hardware. Ideally such an application would need to be cross platform and also function on machines that don't have NVidia hardware available.
As far as I can tell, this boils down to being able to utilize CUDA support as dynamic libraries (dll/.so). I am already using JOCL to access OpenCL but I don't see how I would be able to bind to kernels generated with CUDA as all examples I'm able to find are stand-alone applications.
Are there any open-source examples of such systems?
Are there any technical limitations that make developing such a hybrid application impossible?
Answering the question:
The development is possible, and you can do it without any problem
with the tools commented in the comments. (examples, JCUDA and JOCL) A
quick google search will bring you many free wrappers to put CUDA and OpenCL to Java.
As for the failsafe, CUDA_ERROR_INVALID_DEVICE will be returned at
init CUDA in a non-CUDA system in JCUDA. JOCL will give similar error at initializing stage. Then you can simply select the one that didn't fail or the best one for you. (or in the last case, CPU code only in Java)
However, I cannot understand the background of your question. Since I couldn't find any situation where OpenCL was slower to CUDA. At least, not in the last version of the standards. And my personal usage has shown that even in some cases OpenCL is faster (+-5%). Of course you need to implement both properly, otherwise, one of them will be deeply penalized by a wrong deployment.
You would better take the way of using just one of the both options, either CUDA (if you find it easyer and gives you good performance without any headache) or OpenCL (for flexibility). Using both, maintaining both, and selecting properly the useful one for each case as well as having to deal with the fail safe code, will make your project terribly difficult.
Maybe also have a look at OpenCL which, in theory, should be a bit more cross-platform and also allows to transparently run on different processors (read: GPU and/or CPU as available).

fastest bidirectional java prolog implementation

I am trying to use a a java-prolog layer on top of my current agent-based simulation. I am running Mac OS X Snow Leopard and Netbeans 6.9 and tried to use JPL from swi-prolog however I haven't managed to run a test successfully due to problems with the libraries and their paths. I need a very fast implementation and I was wondering if anyone has any benchmarks or personal experience with bidirectional java-prolog interfaces.
In 2007 I ran the classic Prolog benchmarks (programs such as queens, tak, qsort, nrev and so on) against four Prolog implementations written in Java: tuProlog, JIProlog, JLog and JavaLog. The overall fastest implementation was JLog. All details can be found in an article accepted at SAC 2008.
I'd suggest you have a look also at jTrolog, which was officially born after I measured the performance of the other Java-based Prolog engines. Perhaps PrologCafe may be of some interest, too.
Note, however, that I am not informed about the maintenance state of any of them. Choose carefully if you need some kind of support by their respective authors.
Are you looking for a way to exchange messages efficiently between a java process and a prolog process?
I might recommend Google's Protocol Buffers, which provides a language-neutral and efficient means for exchanging messages between different systems...
http://code.google.com/p/protobuf/
http://www.swi-prolog.org/pldoc/package/protobufs.html

Running Java bytecode on the Android - Sun JVM on top of DalvikVM

Since both the java implementation (OpenJDK) and Android's virtual machine DalvikVM are opensource it must be possible to implement Sun's JavaVM on top Google's DalvikVM. This would make it possible to run JVM based apps and languages (Clojure, Jython) out-of-the-box on the android.
Is there an ongoing effort to produce such an implementation of the Sun JVM?
The OpenJDK makes use of native code so it would be a non-trivial port... there is at least one VM (JikesRVM) that is written in Java, unfortunately it is not a completely working implementation of Java.
Since DalvikVM runs classes that were converted from .class files it should be possible to convert the classes over. Then the "only" issue is when languages generate bytecode on the fly - for that it would require the extra step of converting the generated bytecode over to the DalvikVM format while the program is running on the DalvikVM.
Hmmm.... sort of a JITT (Just In Time Translator) that covertes class files to a DalvikVM files at runtime on the phone. I wonder how slow that would be.
Porting OpenJDK to Android platform is possible. There are effort like : Shark, Zero and caciocavallo that vastly ease the port process (= no ASM, simple AWT peer). Plus Android is nothing but a linux kernel behind. The only question is when will it be done by anybody ?
By the way, both iphones and android phones got Jazelle compatible processor, somebody with very strong processor hacking skills would be very welcome to add Jazelle support to OpenJDK.
Doing so, we could choose between : very light resource acceleration (Jazelle) and JIT ;-)
About iPhone, it is the same thing : a port is possible. Only Apple has put a section in the the iPhone license that clearly forbid VM usage. As per European law, to me, this license section is unlegal. Two reasons : You can not force/link buy of two of your product. Here I tune and Iphones are linked. You can not refuse to sell something that you can sell. Here as soon as a VM would be build for iPhone, if it is refused to be put on the iTune store, then this point will apply. Is there anybody that want to chalenge Apple licence legality on earth ? I don't think so, unhappy people will be flying to Android or any other platform.
In the meantime I have found a possible solution (only JavaME):
"MicroEmulator is a pure Java implementation of Java ME in Java SE."
Running Java ME applications on Android
technically it should be possible to interpret hotspot byte-code on dalvik vm or dalvik's byte-code on hotspot, but it will never be efficient (not mentioning elegant) design for a long run. the cost of doing that might become very high especially with respect to maintenance in the future evolution of such a split ecosystem.
i think, from the very beginning, dalvik vm is a matter of power/market control rather than innovative effort to improve java virtual machine ecosystem. dalvik vm and hotspot vm are lake two highway bridges across the same river build in parallel five meters from each other. it divides and brings confusion to java technological ecosystem.
google is, in my opinion, definitely the technological leader with innovative contributions over last decade that dwarf far more conservative oracle, but in this very subject they have shaken the java ecosystem in nearly destructive way. we (programmers) should strive for a single uniform solution to this problem. that is the main idea behind the concept of "virtual machine" anyway - it should not belong to neither google nor oracle.
try GNURoot app (proot ptrace container) with debian Wheezy;
apt-get update
apt-get install openjdk-7-jre
/usr/lib/jvm/java-7-openjdk-armel/jre/bin/java ...
(including long path)
I don't think the problem is technical. In theory you can migrate OpenJDK to the iPhone for example, but there are legal restriction in place.
See http://davy.preuveneers.be/phoneme/ for MIDP, Foundation and Personal Profile jvms for Android ARM apks. (Thank you Davy!!!)
I have repackaged the Foundation jvm for command line use and host it at vkfox.com/android/bin/foundation-jvm.bin for your pleasure. One could use the eclipse compiler in the JavaIDEdroid free app, or the old kopi or kopisusu compilers with this jvm, togeather with the stubbed cdc + foundation jar file from a J2ME jdk for an onboard development system. Using the features of TerminalIDE -- console and sun compiler -- with indicated ref jars would also get you there. This is essentially equivalent to a headless jdk1.4 with the possibility of MIDP or awt graphics.

Is .NET/Mono or Java the better choice for cross-platform development? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
How much less libraries are there for Mono than for Java?
I lack the overview over both alternatives but I have pretty much freedom of choice for my next project. I'm looking for hard technical facts in the areas of
performance (for example, I'm told Java is good for threading, and I hear the runtime code optimization has become very good recently for .NET)
real world portability (it's both meant to be portable, what's Catch-22 for each?)
tool availability (CI, build automation, debugging, IDE)
I am especially looking for what you actually experienced in your own work rather than the things I could google. My application would be a back-end service processing large amounts of data from time series.
My main target platform would be Linux.
Edit:
To phrase my question more adequately, I am interested in the whole package (3rd party libraries etc.), not just the language. For libraries, that probably boils down to the question "how much less libraries are there for Mono than for Java"?
FYI, I have since chosen Java for this project, because it seemed just more battle-worn on the portability side and it's been around for a while on older systems, too. I'm a tiny little bit sad about it, because I'm very curious about C# and I'd love to have done some large project in it, but maybe next time. Thanks for all the advice.
Mono does a better job at targeting the platforms I want to support. Other than that, it is all subjective.
I share C# code across the following platforms:
- iOS (iPhone/iPad)
- Android
- The Web (HTML5)
- Mac (OS X)
- Linux
- Windows
I could share it even more places:
- Windows Phone 7
- Wii
- XBox
- PS3
- etc.
The biggie is iOS since MonoTouch works fantastically. I do not know of any good way to target iOS with Java. You cannot target Windows Phone 7 with Java, so I would say that the days of Java being better for mobile are behind us.
The biggest factor for me though is personal productivity (and happiness). C# as a language is years ahead of Java IMHO and the .NET framework is a joy to use. Most of what is being added in Java 7 and Java 8 has been in C# for years. JVM languages like Scala and Clojure (both available on the CLR) are pretty nice though.
I see Mono as a platform in it's own right (a great one) and treat .NET as the Microsoft implementation of Mono on Windows. This means that I develop and test on Mono first. This works wonderfully.
If both Java and .NET (Mono let's say) were Open Source projects without any corporate backing, I would choose Mono over Java every time. I believe it is just a better platform.
Both .NET/Mono and the JVM are great choices, although I would personally use some other language than Java on the JVM.
My take on some of the other comments:
Issue: Performance.
**Answer: Both the JVM and the CLR perform better than detractors say they do. I would say that the JVM performs better. Mono is generally slower than .NET (though not always).
I personally would take ASP.NET MVC over J2EE any day both as a developer and an end-user. Support for Google Native Client is pretty cool too. Also, I know that poor GUI performance for desktop Java apps is supposed to be a thing of the past but I keep finding slow ones. Then again, I could say the same for WPF. GTK# is plenty fast though so there is no reason they have to be slow.
Issue: Java has a larger ecosystem of libraries available.
Answer: Probably true, but it is a non-issue in practice.
Practically every Java library (including the JDK) runs just dandy on .NET/Mono thanks to IKVM.NET. This piece of technology is a true marvel. The integration is amazing; you can use a Java library just like it was native. I have only had to use Java libraries in one .NET app though. The .NET/Mono ecosystem generally offers more than I need.
Issue: Java has better (broader) tools support
Answer: Not on Windows. Otherwise I agree. MonoDevelop is nice though.
I want to give a shout-out to MonoDevelop; it is a jewel. MonoDevelop integrates most of the tools I want use including code completion (intellisense), Git/Subversion integration, support for unit tests, SQL integration, debugging, easy refactoring, and assembly browsing with on-the-fly decompilation. It is wonderful to use the same environment for everything from server-side web to mobile apps.
Issue: Compatibility across platforms.
Answer: Mono is a single code-base across all platforms, including Windows.
Develop for Mono first and deploy to .NET on Windows if you like. If you compare .NET from MS to Java though then Java has the edge in terms of consistency across platforms. See next answer...
Issue: Mono lags .NET.
Answer: No it does not. IMHO, this is an often stated but incorrect statement.
The Mono distribution from Xamarin ships with C#, VB.NET, F#, IronPython, IronRuby, and I think maybe Boo out of the box. The Mono C# compiler is completely up to date with MS. The Mono VB.NET compiler does lag the MS version. The other compilers are the same on both platforms (as are other .NET languages like Nemerle, Boo, and Phalanger (PHP) ).
Mono ships with a lot of the actual Microsoft written code including the Dynamic Language Runtime (DLR), Managed Extensibility Framework (MEF), F#, and ASP.NET MVC. Because Razor is not Open Source, Mono currently ships with MVC2 but MVC3 works on Mono just fine.
The core Mono platform has kept pace with .NET or many years and the compatibility is impressive. You can use the full C# 4.0 language and even some C# 5.0 features today. In fact, Mono often leads .NET in many ways.
Mono implements parts of the CLR spec that even Microsoft does not support (like 64 bit arrays). One of the most exciting new pieces of technology in the .NET world is Rosylyn. Mono has offered the C# compiler as a service for many years. Some of what Rosylyn offers is available via NRefractory as well. An example of were Mono is still ahead would be the SIMD instructions to accelerate gaming performance.
Microsoft does offer a number of products on top of .NET that are not available in Mono which is were the misconception about Mono lagging comes from. Windows Presentation Foundation (WPF), Entity Framework (EF), WCF (Windows Communication Foundation) are examples of products which do not work, or are poorly supported, on Mono. The obvious solution is to use cross-platform alternatives like GTK#, NHibernate, and ServiceStack instead.
Issue: Microsoft is evil.
Answer: True. So what.
Many people offer the following reasons to avoid using Mono:
1) You should not use Mono because Microsoft tech should be avoided
2) Mono sucks because it does not let you use every technology that Microsoft offers
To me, it is clear that these statements are incompatible. I reject the first statement but will skip that argument here. The second statement is true of all .NET alternatives.
The JVM is a great platform and the explosion of JVM languages is awesome. Use what makes you happy. For now, that is often .NET/Mono for me.
Well....Java is actually more portable. Mono isn't implemented everywhere, and it lags behind the Microsoft implementation significantly. The Java SDK seems to stay in better sync across platforms (and it works on more platforms).
I'd also say Java has more tool availability across all those platforms, although there are plenty of tools available for .NET on Windows platforms.
Update for 2014
I still hold this opinion in 2014. However, I'll qualify this by saying I'm just now starting to pay some attention to Mono after a long while of not really caring, so there may be improvements in the Mono runtime (or ecosystem) that I haven't been made aware of. AFAIK, there is still no support for WPF, WCF, WF, of WIF. Mono can run on iOS, but to my knowledge, the Java runtime still runs on far more platforms than Mono. Also, Mono is starting to see some much improved tooling (Xamarin), and Microsoft seems to have a much more cross-platform kind of attitude and willingness to work with partners to make them complimentary, rather than competitive (for example, Mono will be a pretty important part of the upcoming OWIN/Helios ASP.NET landscape). I suspect that in the coming years the differences in portability will lessen rapidly, especially after .NET being open-sourced.
Update for 2018
My view on this is starting to go the other way. I think .NET, broadly, particularly with .NET Core, has started to achieve "portability parity" with Java. There are efforts underway to bring WPF to .NET Core for some platforms, and .NET Core itself runs on a great many platforms now. Mono (owned by Xamarin, which is now owned by Microsoft) is a more mature and polished product than ever, and writing applications that work on multiple platforms is no longer the domain of deep gnosis of .NET hackery, but is a relatively straightforward endeavor. There are, of course, libraries and services and applications that are Windows-only or can only target specific platforms - but the same can be said of Java (broadly).
If I were in the OP's shoes at this point, I can think of no reason inherent in the languages or tech stacks themselves that would prevent me from choosing .NET for any application going forward from this point.
I actually develop in .NET, run all my tests first on Mono, and then on Windows. That way I know my applications are cross platform. I have done this very successfully on both ASP.NET and Winforms applications.
I am not really sure where some people get the impression Mono is so horrible from, but it certainly has done it's job in my cases and opinions.It is true you will have a bit of lag for the latest and greatest inventions in the .NET world, but so far, .NET 2.0 on Windows and Linux is very solid for me.
Keep in mind there are obviously many quirks to this, but most of them come from making sure you are writing portable code. While the frameworks do a great job of abstracting away what OS you are running on, little things like Linux's case sensitivity in paths and file names takes a bit of getting used to, as do things like permissions.
.NET is definitely very cross platform due to Mono based on my experiences so far.
Java actually is as cross-platform as everyone says it is. There's a JVM implementation for just about any mainstream OS out there (even Mac OS X, finally), and they all work really well. And there's tons of open source tools out there that are just as cross platform.
The only catch is that there are certain native operations you can't do in Java without writing some DLLs or SOs. It's very rare that these come up in practice. In all those cases, though, I've been able to get around it by spawning native processes and screen-scraping the results.
I think the question is phrased incorrectly. C# vs. Java is much less interesting in terms of cross-platform usage than is (a) which platforms you need to support, and (b) considering the core libraries and available third party libraries. The language is almost the least important part of the decision-making process.
Java is a better choice for Cross-Platform development.
Performance. Java and .Net have similar performance level due to the virtual machine, but JVM normally has better performance because of years and years optimization.
Library. Although this depends on your task, Java has much more open source or third party libraries available there. For server App, J2EE, Spring, Struts, etc. For GUI, although .Net provides Win32 layer API but this causes compatibility issues. Java has Swing, SWT, AWT, etc. It works in most cases.
Compatibility. This is the key issues that need to be considered when develop the cross-platform program. Two issue: first, platform compatibility. Java still wins since JDK is well maintained by single and original company Sun. Mono is not maintained by MS, so you have no guarantee yet for update compatibility. 2. Backward compatibility. Sun maintains a good reputation on their backward compatibility, although sometimes this seems too rigid and slows the pace.
Tools. Java has good cross-platform IDEs. Netbeans, Eclipse, etc. Most of them are free. VS Studio is good but only on Windows, and not cost a bit. Both of them provides good unit tests, debugs, profiles, etc.
Hence I'd suggest that Java is a better choice. As a show case, there are some famous desktop cross-platforms apps developed by Java: Vuze, Limewire, BlogBridge, CrossFTP, not to mention those IDEs. As to .Net, I have limited knowledge on such success apps.
I've been asking the same question off-late and IMHO, .NET/Mono seems to be a better option simply because Mono has a great track record for cross-platform desktop applications (as opposed to Java) and of course, Mono is improving by leaps and bounds these days.
I'm going to say Java as well. If you look at it in terms of maturity, a lot more time and effort has been expended by Sun (and others) in getting the JVM to work on non-Windows platforms.
In contrast, Mono is definitely a second class citizen in the .NET ecosystem.
Depending on who your target customers are, you may also find there is real pushback against using Mono - does Novell offer the same kind of vendor support for Mono that you would get for Java or .NET on Windows?
If you were primarily targeting hosting your service on Windows, it would make sense to be considering this choice, but since you're targeting Linux primarily, it seems like kind of a no-brainer to me.
Java was designed to be cross-platform; C#/.Net wasn't. When in doubt, use the tool that was designed for your purpose.
EDIT: in fairness, .NET was designed to work on embedded/PC/Server environments, so that's SORT of cross-platform. But it wasn't designed for Linux.
I think the answer is "it depends." Java runs on just about anything, but .NET/Mono are (IMHO) a better framework for the desktop. So I guess the answer really depends on what platforms you plan on targeting.
To add a bit more to the conversation, Java is more portable if you remain about one version behind - Java 5 still has many excellent features so you can wait for Java 6 and still have a lot of range in terms of language and libraries to develop with. The Mac is the primary platform that can take some time to catch up to the latest Java version.
Java also has an excellent standards body that intelligently grows the platform based on input from many different companies. This is an oft overlooked feature but it keeps even new features working well across multiple platforms and provides a lot of range in library support for some esoteric things (as optional extensions).
I would vote for Java being more portable than C#. Java definitely also has a very rich set of standard libraries. There is also a broad set of open source 3rd party libraries out there such as those provided by the Jakarta project (http://jakarta.apache.org/).
All the usual suspects exist for CI, Unit testing, etc too. Cross platform IDE support is also very good with the likes of Eclipse, Netbeans, IntelliJ IDEA etc.
There are other language choices too. I've become quite fond of Python, which works well on Windows, Linux, and Mac, and has a rich set of libraries.
While Mono has its share of problems I think it has a better cross-platform compatibility story especially IF you have reliance on native platform invocation.
There are not enough words on Stack Overflow to stress how much smoother it is to get something native called and executed in .NET/Mono on (at least in my experience 3...) multiple platforms vs. the equivalent Java effort.
Gatorhall do you have some data to back that up?
Performance. Java and .Net have similar performance level due to the virtual machine, but JVM normally has better performance because of years and years optimization.
Background: I'm a Windows guy since Windows 3.1 and currently a Linux user (still running Windows 7, great OS, on a VM for Visual Studio 2010 and other tools).
The point: me and a lot of users (windows, linux, etc) I know, may disagree from you. Java tends to perform slower even on a linux desktop application, ASP.NET perform's faster that java server pages many of the times. Some may agree that even non-compiled PHP performs better i several scenarios.
Java is more cross-platform? I have no doubts about this (the history back this on), but faster (not saying .NET is) not so certain and I would like to see some real benchmarks.

Categories