I'm required to write a JCE provider. I have learned that I need to use the bouncy castle clean room implementation. Could you please tell me what "clean room implementation" means in in this context?
Clean room implementation means in this context that the code was written only from the description and specification, but not with access to any potentially troublesome code.
This is important as the Sun source code previously was under a license which was incompatible with especially GPL distributions. By ensuring that the code was not written without any access to such code, it is guaranteed that the Sun license could not later be found to apply to the clean room implementation, and then again be incompatible with the GPL license or any other restrictions.
In other words, it is a guarantee that the license shown will hold.
Despite the name, most re-implementations are done in rooms with mostly empty beer cans and pizza boxes. This is speaking from past experience, you understand :-)
However, in the context of chip development, a clean room is one where small imperfections (like dust) can cause serious problems at the nano scale. So they're kept incredibly clean, with air scrubbers, protective suits (protecting the environment, notvthe wearer) etc.
By analogy, software clean room development is done without any access to imperfections that could cause problems. In the case of open-source software, this means not having access to an implementation that might be "tainted" by an incompatible license. (It's not just GPL, but this is commonly cited due to its viral nature). However, this also applies to closed source reverse engineering projects, like the creation of the IBM compatible BIOS that opened the market to IBM clones back in the 80's.
Related
Can we completely reverse-engineer the source code from java bytecode ? Why this feature is allowed in Java and How successful are java decompilers against obfuscators.?
I know this question is old but I kept looking for a reliable answer until I found nothing.
So in this post I summarize some of my effort to obfuscate a J2EE JAR.
It seems , that by year 2014 (time of writing) there are not many options out there.
If you read this review later then things may have changed or fixed.
When I think why , I start to sense that the whole obfuscation effort gives a false sense of security. Don't get me wrong. It does add a level of security, but not as much as I would hope.
I will try to give a preview of what I found to explain myself. My recommendation are personal , others may disagree with it.
So to begin with: obfuscation in Java is the process of taking bytecode and making it less readable (using a decompiler of course) while maintaining its original functionality.What can we do, Java ,working as an interperter, must keep its bytecode exposed. You run the obfuscator as a measure of security in case the class file falls into the wrong hands. The result of the obfuscation is a reverse-mapping files and a JAR with the obfuscated classes. The reverse mapping file is used of-course to perform stack trace reading (a.k.a re-trace) or to revert the bytecode to its original shape. The runtime performance hit of an obfuscated class should not pass the 10% (but this really depends on what you do in your code).
But there is a big “but” . Obfuscation will scramble your code but it won’t make it hacker-proof. Bare in mind you only buy time and a determined hacker will find a way to reverse engineer your bytecode into its pure algorithm.
IMHO: the best way to hide a sensitive piece of code is to drown it in some huge pile of meaningless code.
Some of the hackers will try to modify your bytecode (by code injection) to help them achieve their goals. Some obfuscators offer additional level of JAR hardening , making it harder to modify.
De-obfuscators and de-compilers: my favourite Java decompiler is JD-GUI . However, when it comes to de-obfuscators I found the market pretty empty. Most of the tools ask you for a hint (what obfuscation tool was used to encrypt the source JAR) , yet none of them really deliver results (some of them even crash when trying to de-cipher the JAR). They are open source projects with low maintenance. I couldn’t even find a paid application to do a decent de-obfuscation. so enlighten me if you know something.
Free solutions
There are open source , free obfuscators which usually simply rename the classes/methods names, making it one letter method (i.e. from printUsage(String params) to a(String p) ).
They might ,as hinted here , even strip debugging information to make it a bit more difficult. (debugging information is kept at the end of every Java method bytecode and contains: line numbers, variables names ,etc.).
Its a nice effort , but an experience Java developer with a debugger can very easily deduce the purpose of each parameter while doing few live runs.
One of the nice open source obfuscators is ProGuard but there are several more tools.
Nevertheless , if you truly security fanatic you will probably want something stronger. Stronger demands more features (and more money) which leads us to the next bullet:
Paid solutions
While free products may only change classes method names , paid product will usually offer more features:
code/flow obfuscation: this will change the method code and inject empty loops/dead code/confusing switch tables and alike. Some of them may even scramble the exception table content. the obfuscation strength usually determine the output size.
Note: regarding code obfuscation: I deliberately avoided the details in my review. Some of the bytecode I saw and analyzed expose their obfuscation methods, and I wish to protect their IP. I do have an opinion about who uses better algorithms. contact me if you wish to know.
classes/method renaming : well this is the obvious , we discussed it in the free obfuscation. Some of the product will rename the class name and then recursively search for reflection usage of that class and fix those too. Paid products may even rename Spring /Wink configuration files for the same purpose (renaming in reflection).
String encryption: for every string “like this” in the code, it will encrypt it to some level and keep the key somewhere (in the class constant table/static blocks/a new method or any other mean).
debug information : stripping parts or scrambling.many of them will remove the line numbers info.
class
hardening: all kinds of methods like injecting some signing scheme into the beginning of the class/method, making sure an outsider won’t be able to easily modify the JAR and run it. Less important for Android or applets as most of them are digitally signed anyhow. some will combine hardening with water-marking to track pirated copies. But we all know anti-pirating methods by software are doomed to be hacked. Game industry suffered from it for decades until network based subscriptions arrived.
Since most products here deal with Java , some of them provides Android integration. It means it will not only obfuscate the Java (dalvik) code , but also manipulates the Android's manifest file and resources. Some offer anti debugging: remove the debug flag in android apps.
Nice GUI app to configure the various options and maybe do a re-trance on a given log file. The UI is usually used to generate a config file. with such file you can later re-play the obfuscation many times, even from command line.
Incremental build support - this is useful for large groups who release product updates/fixes frequently. You can tell the obfuscator to preserve old “obfuscation” result and randomly obfuscate only “new” code flows. this way you can be sure minimal impact on your methods signature. Without this flag , each obfuscation cycle on a JAR would yield a different output as most good tools use some level of randomness in their algorithms.
CLI and distributed builds. When you work alone then running an obfuscator is not a big issue. you need to configure the obfuscator to your relevant options and run it.However, in enterprise , when integrating obfuscator into the the build script things are a bit different. There is another level of complexity: build engine tasks (like ant/maven) and license management. The good news that all obfuscator I tested have command line API. In distributed build environment there are cluster/pool of build machines to support concurrent demand of builds. The cluster is dynamic and virtual, machines are going up or down, depending on various conditions. Some obfuscation products are based on cpuID license file or hostname. This can create quite a challenge for the build teams to integrate. Some prefer a local floating license server. Some may require public license server (but then: not all build farms have access to the public internet). Some offer multi-site license (which in my opinion is the best).
Some offer code optimizations - algebric equivalence and dropping of dead code. Its nice, but I believe that today's JDK do good job in optimizing bytecode. Its true that dead code makes you downloadable bigger, but with today's bandwidth its less than a problem. I also want to believe that in software today 20:80 thumb rule still applies. in any application 20% is probably a dead code anyway.
So who are the players I tried ?
KlassMaster by Zelix.com - one of the oldest in the industry. Yet they deliver a solid product with 3-4 releases per year. This been going for decades (since 1997). Zelix provides good email support and answered all my emails in a timely manner. They have a nice GUI client to either obfuscate a JAR or create a config file for future obfuscation. It simple and slick. nothing special here. They provided simple to read on-line documentation for all their flags. they support both “exclude” and “include” regular expressions for what the engine should obfuscate. The thing I liked about their process most is that it also adds “noise” to the exception table. It makes it a bit more confusing regarding the method exception handling. Their flow obfuscator strength is quite good and can be configured between 3 possible levels (light,medium and aggressive). Another feature I liked is the fine tuning they provide for debug info stripping (online line numbers, or online local variables or both). Klass Master doesn’t provide any
dedicated Android flags or anti-tamper methods. Their licensing model is quite simple: a text file to be placed near the KlassMaster main JAR. They also support incremental obfuscation.
JFuscator from secureTeam.net : While secureTeam also has a .Net tool , I focus on their Java tool capabilities. Their (Swing based) GUI tool seems nice but it crash when trying the simplest obfuscation task. the error was always the same: Error reading '/opt/sun-jdk1.7.0_55/jre\lib\rt.jar'. Reason: ''/opt/sun-jdk1.7.0_55/jre\lib\rt.jar': no such file or directory' . Now of course I have my Java installed in /opt/sun-jdk1.7.0_55/jre. You can image that they simply didn’t expect linux back slash structure. I contacted secureTeam.net support by email with the minor “path” problem. They asked if I am a linux user and after I replied I am , they never answered my email. I also tried their web site on-line chat : no response. So there I stopped testing. Without further results, I couldn’t examine the obfuscated bytecode quality. From their web site it seems they have anti-tamper method , String manipulation, method renaming and few other features.
GuartIt4J (by Arxan.com) : Arxan is fairly solid player in the mobile environment and as such they offer Android obfuscator which of course works well for Java. They have one of the most flexible engines.They provide code obfuscation,string encryption and alike You can define the complexity of code obfuscation. it is simply an integer. the higher - the longer your method turns out. ofcourse, you must be carefull not to exceed the JVM 64KB limit per class… As I said before one of the best strategies to hide a sensitive code is not to encrypt it , but to inject it into huge pile of garbage. This is exactly what GuardIt does. It can also explode in the same way the methods exception table. I managed to create a method with 100 exceptions in its exception table (pre-obfuscator it was 5). what they miss: their re-trace program is not part of the supplied main JAR. Nevertheless, they were kind enough to send me a sample Java program that performs re-trace given the reverse mapping file and the log. They don’t support incremental obfuscation and no flexibility regarding debug information. Debug information stripping is either all or nothing. watching the output JAR you will tons of conditions and jumps that were injected. Bare in mind , exploding the class size has its performance hit. In some methods I measured almost 50% performance hit when applying long obfuscation (no I/O in those methods). so extrapolating the code comes with a price.(from a 400 opcodes - I went up to 2200 opcodes after obfuscation). JD-GUI , my de-compiler failed to open such classes and crashed (IndexOutOfBoundException). They also supply complete class encryption . Meaning the class is encrypted with some symetrical key which demands a special (or custom written) class loader to open it in memory. This is an anti-tamper mechanism as well as hiding code. Just remember that a JVM can’t run that class without the class loader help. Its a nice feature, but the secret key and the bootstrap loader JAR are probably there. If he got the encrypted JAR the hacker will eventually get his hands and decrypt the classes. Yet this another level of obstacle the common hacker will need to pass. What I didn’t like here is the license file policy: is bounded to CPUid or need to install a floating license server.
SecureIt (by Allatori.com) : SecureIt offers all the general code obfuscation, string encryption ,renaming and such. On top of the standard obfuscation methods they also offer some kind of water-marking which is an anti-tamper/pirating method. They support Android and JavaME (who uses ME these days?!). They support incremental obfuscation. The one thing to note about configuring SecureIt: it is all command line. No GUI tool this time. Personally , I don’t mind command line tools as long as they come with good documentation. Luckily they have a very good documentation and a rich API with many flags to tune if you wish. you can re-trace with they tool (also a command line ) . They can’t obfuscate the exception table. I didn’t check their licensing mechanism.
DashO (by Preemptive.com) : DashO obfuscator will be remembered probably as the best UI tool you can get (to create your configuration). Like SecureIt they lake the exception table obfuscation but they have all the rest of the required features (as well as CLI, Spring framework and gradle/ant integration, and even an eclipse plugin) . Well, they do document a try-catch obfuscator (which is same as exception table obfuscator) , but it is only a recommendation to the engine. When I tried it , it had nil effect on the exception table. As I said , the GUI tool is superb and has a re-trace embedded into it. they also offer some kind of application signing and water-marking as an anti-tamper/pirating mechanism. DashO provides superb Android integration and also combine in their product a door for analytics uploads. You can actually track your application. Injecting crash log uploaders and reporting code to your JAR. Nevertheless that’s not the scope of obfuscation - that’s a whole different code injection product. They have a very good support. both online and by phone. Their licensing scheme is based on monthly subscription or one time purchase payment. A bit different than others. They are using a floating license server to support large environments.
I hope this helps a bit..
Can we completely reverse-engineer the source code from java bytecode ?
Not completely, because some aspects of source code, such as whitespace, local variable names, and comments, are not preserved in bytecode. Otherwise, yes -- while you can't get the exact same source code out, you can almost always get something that can at least be compiled back to the same bytecode.
Why this feature is allowed in Java
It's not so much "allowed" as it is "not prevented". And it's not prevented because doing so is impossible -- the code must be runnable to be useful; if the code is runnable, then it is analyzable; if it is analyzable, then with sufficient analysis it can be converted back to source.
How successful are java decompilers against obfuscators?
Not very. Most obfuscators I've seen (esp. ProGuard) are primarily effective in removing meaningful function and class names; obfuscating the logic itself is not typically attempted.
you can get source code from binary these days. Although the source code obtained by Java's bytecode is more readable, obfuscating will make it slightly unreadable. Its not that only Java can be reverse engineered to code. Even C/C++ these days (with Hexrays plugin for IDA Pro) can be decompiled to source. Obfuscaters will make it hard to read but not impossible. There is nothing that can save your program from an intelligent and capable reverse engineer. :).
Good luck.
Can we completely reverse-engineer the source code from java bytecode
?
The java class file is based on a spec so anyone can read into it. A tool like JD-GUI will tear into your source code easily. It is not a 'feature' per se. While 100% reverse-engineering is not possible, most of your code can be reverse engineered.
How successful are java decompilers against obfuscators?
Depends. The point of the obfuscator is to remove any meaningful names and try to introduce confusion in the code without impacting performance. Most developers are great at obfuscating code themselves :) Pro-guard is pretty good at obfuscation.
Can I use Java on FreeRTOS?
If so, are there any good URLs to help me get started?
What problems can I expect?
You need a Java VM to run Java code. I doubt very much that there is a Java VM implementation specifically designed to run on FreeRTOS, rather you would probably have to hook the VM into the underlying OS in the porting layer for your target. Some VMs implement scheduling themselves and do not need an OS to support threading.
Consider:
JamaicaVm (Commercial & Free Personal Edition)
MicroJVM (Comercial)
Aonix Perc (Commercial)
PreonVm (Commercial)
AvianVM (Open source)
Open Mika (Open source)
Squawk (Open source)
SimplRJT (Open source needs no RTOS for threading support)
Kaffe (Open source).
You might also benefit from reading this article by Michael Barr and Jason Steinhorn
My contribution is to suggest one looks to: http://www.ajile.com
It seems that they deliver 'java on a chip'; I'm software so I have no idea if it works. I can say that 10 years working with applications, middle ware and operating systems 'stuff' where we fortunately had some-times had ways to access people on the chip-design teams. (Imho) most of the time there was a Chinese wall between the 'hard ware' and 'software' cliques. I am deliberately including "virtual machine" people in 'hardware' group.
I haven't run things on aJile silicon. I'm just promoting a seemingly more natural option that could evolve into a Java Machine that works -- Somewhat similar to the BBC (and now ARM silicon), Forth, Smalltalk, or M2 that began life on an "Ideal Machines".
If I may; I have one quite significant caveat based on my hard-won lessons. No "virtual" anything is going to make sense. There will always be a conflict between the "virtual folk" and any "programmer mob". That's as may be. It will not often help Ms/Mr End-user of XYZ, Vermont, USA ... (Or, will it?)
My personally most satisfying tasks/projects were when we engaged with any available (or accessible) the hardware (or ideal-machine / virtual-machine) in a peer-to-peer dialogue. That doesn't seem to happen that much these days; may be we just need DSL-s instead of hardware or virtual machines??? (irony :flag).
~ aloha
We are currently using MICO to establish the communication between a server and a client, where the client is a simulator written in C++ and the server is a java program displaying an animation of what happens in the simulation. It seems that the developemt of MICO has slowed down to an almost halt and bugs that only allow us to hack around them (as we don't have the time to first figure out which parts of MICO are responsible for codegeneration and so on) keep us from making real progress.
So, does any one of you know of a good drop-in replacement? We would like to have the following:
The compiler can generate both C++ and Java-Code from the idl.
The project should still show considerable support.
The implementation should be open-sourced (GPL or BSD, or something alike), as we use our programs to teach students as well.
The migration from MICO should be as easy as possible. (This is not a hard requirement, but would be a good thing)
I found some other CORBA implementations, e.g. TAO, but I didn't find any of which I could
say that they are still supported. Correct me if I am wrong here.
The Free CORBA® Downloads page might be of interest to you.
Just naming:
orbit2 1, also pyorbit etc.
omniORB
TAO (has already been mentioned)
1 On my Ubuntu box, apt-rdepends -r liborbit2 returns 5530 lines...
I don't know where CORBA or MICO in particular has gone in the last 5 years, but back then a drop in replacement was not really possible, since differences between vendors where still there.
Not necessarily API differences (POA, etc.) but
in implementation behavior,
in custom extensions which were required to make it work in a real-world environment (threading, load balancing, security, etc.),
in how the development tools worked
and also about the whole deployment or runtime story.
We had Orbix ASP/2000/Whatever and ORBacus which were interchangeable having a small compatibility layer, some Makefile framework to hide differences in tools (e.g. the IDL compiler) and some scripts for wrapping ORB specific processes.
Unfortunately, ORBacus has long been bought by (then) IONA, which already make Orbix. IONA itself has been bought by someone else (I forgot). The original authors of ORBacus, plus some devs from IONA Orbix, changed their ways somewhat and produced Ice, which is not CORBA but somewhat alike - of course without the glitches ;-)
Concerning TOA, I think it would be the "best" choice concerning still being developed as it is driven by research on the Washington University. But last time I looked, they didn't have Java implementation but people seemed to use JacORB.
Maybe all this helps you little, unless it brings even more confusion :-)
TAO as C++ ORB is still actively supported and developed (see http://www.cs.wustl.edu/~schmidt/commercial-support.html). For Java I would propose JacORB.
On http://www.orbzone.org there is an overview of available corba implementations.
Out of curiosity, I wrote an own simple implementation of Set for a special case (where the set of all possible entries is fixed, but it's no enum). It was actually quite easy, but obviously, my implementation is unusable without a lot of tests (and maybe even then, but that's another topic; it was mainly an exercise).
So is there any testsuite available I could use for unit-testing it?
Use the Guava SetTestSuiteBuilder.
https://github.com/google/guava/blob/master/guava-testlib/src/com/google/common/collect/testing/SetTestSuiteBuilder.java
Examples:
https://github.com/google/guava/blob/master/guava-testlib/src/com/google/common/collect/testing/TestsForSetsInJavaUtil.java
It is released as part of "guava-testlib" in maven central.
Oracle refuses to open source the test suite for java, this is the heart of a debate between the OSS community and Oracle.
See here for more info:
http://arstechnica.com/open-source/news/2010/12/apache-resigns-from-jcp-in-protest-of-oracle-governance-failures.ars
So I doubt there is anything you can do to verify you conform to the standard.
Here's an excerpt:
The heart of the issue is that Apache can't certify that its open source Java implementation—called Harmony—conforms with the Java language standards because Oracle refuses to supply the necessary test suites under a suitably open license. Oracle's position on the issue falls afoul of JCP policies, which stipulate that standards and other relevant materials must be freely redistributable and made available under terms that are conducive to enabling third-party open source implementations.
This conflict has been a subject of considerable friction for quite some time, because Sun similarly denied the ASF appropriate access to the test suite prior to Oracle's acquisition of the company. Oracle had initially sided with Apache in calling for the test suites to be published under open terms, but reversed its position after acquiring Sun. Oracle's approach to this issue suggested very strongly that the company would likely not be willing to fix any of the long-standing JCP governance problems that had arisen under Sun.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I'm a programmer who is starting to build small commercial apps (for mobile phones (java,android), web pages and c++ apps built in visual studio).
Now the problem is that i don't understand how licencing works. From what I've seen I am not allowed to use any library, not even java's basic one because they are either under GPL or something else.
For example, am I obliged to pay Sun to use their java libraries to build my app ? or how can I write a mobile app when it is necessary to release my code. How do big companies get around that ?
Another thing I don't understand, what libraries and apis am I allowed to use for free that are windows specific (like MFC, .NET, COM). How do this licences work? if I just bought the visual studio suit will it be enough to allow me to use this api's ? I'm in a situation that I learned to use this libraries/apis/frameworks and now when it's time to actually use them for something I must pay for them.
If that's true than that means it's impossible to build commercial apps without spending thousands of dollars on other things ?
Here are some cases and how to deal with them:
copying source code into your own code
This is where you should be very aware where you're copying from and under which license(s) the code was released. Most of the cases you will NOT be allowed to copy it. In almost all other cases you will need to put the original copyright notice in the code. In some cases (GPL) you're obliged to release the source code along with any public distribution of your product. In some cases you can purchase a license for redistribution rights in binary form. In all cases it's good practice to put a comment, marking the origin of the code.
Using libraries but not shipping
If you compile an application for Android or for Windows, you'll inevitably use their API's and libraries. As long as these belong to the platform, you do not need to ship the libraries with your app. So you only need to care if you're allowed to use them (link to them) from within your app. For OS and Language libs, this is usually the case.
Shipping 3rd party libraries with your app
Here again, you will need to know exactly what you're doing because there are big differences. This field is quite complex... Some development environments and frameworks include "redistributable" packages, which are meant for inclusion in your app. But they come with license terms, too, and it's worthwhile to read and understand the implications.
Rule of thumb: BSD style licenses are fine if you give proper credit in your documentation. GPL is almost always a no-go for commercial applications unless you're willing to release your entire source code with your product.
Disclaimer: IANAL
Your question is extremely broad, covering a wide range of libraries and associated licenses. IANAL but the short answer is that most of the library code that you mention can be used for commercial purposes without paying anything; particularly the Java and .Net code. The respective platform holders WANT people to use them. It's only when you get into more specialised situations that they start extracting revenue, usually through support contracts or special VM versions (like the old JRockit VM).
There is a big difference between copying code and use it as your own, versus linking to it.
The semantics can get complicated very fast, as there are many types of licences, so as a warm recommendation, hire the services of a good lawyer that is familiar with the field. That can save you a lot of trouble in the future!!
Big companies are careful in examining the license for every tool/sdk/library that they incorporate in their solutions.
In what's concerning the obligation of releasing your source code, you don't have to if you are using either Microsoft sdks or the Java base libraries.
But in order to be 100% sure, you always have to read the licenses. A good start to understanding how licensing works would be to read the following Wikipedia articles:
CopyLeft - if a license is copyleft then all the restrictions of that license will also apply to your code, but you will not be able to impose further restrictions to the code or product. That means that if you use an open source sdk that has a License which is Copyleft you will have to distribute your source code as open source.
CopyFree - if a license is copyfree, you may impose further restrictions on it like adding a fee for the end product and refuse to distribute the code.
GNU public license - the copyleft standard applies if you included or linked to copyleft based license
Lesser GNU public license - the copyleft standard applies only if you directly included copyleft licensed code into your source base. But it does not apply if you merely linked to a copyleft type licensed product
Try to spend some time and go trough a few articles on the licensing subject and then you will start to understand licenses a bit better.
HTH,
JP