I have an issue here..and its making me really nervous.
I was working on this game, and it was going great, so I took a copy of it on my laptop to work do some work while away from my computer.
long story short, hard-drive failure + poor back ups led to me losing a very important class.
Is there a way to decompile the apk to retrieve the bit of code that was lost? It isn't overly complicated or sophisticated, its just that its impossible to re-write it without reading every. single. line. of. code. in the entire application since it initializes a LOT of classes and loads a bunch of stuff in a specific way.
With a quick google search I was able to find apktool, which decompiles it into a bunch of .smali files, which I don't think were designed for human reading.
All I need to recover is one very big method in the class. I found the smali file that contains it and I think I found the line where it starts. something like
.method public declared-synchronized load(Lcom/X/X/game/X;)I
Anyone help would be appreciated since I would have to scrap the entire game without this method.
A quick google resulted in a way to decompile apks (decompile apk to java source). However, even though it results in java code, you probably won't have any variable names (just default ones like param1) as those are unrecoverably removed when you compile the source code to byte code, also, depending on the decompiler, for/foreach loops will be while loops instead, if/else blocks might not represent your original control flow due to compiler optimization.
As a general advise: Use some sort of source control. On your own server, paid account with github doesn't matter what, but use source control. Even if you are just one person developing on a project. It helps with this situation, it helps with reverting to a previous version, it helps finding a bug you introduced. When the tools are available, use them.
Related
From what I understand, obfuscating a java web application will just make it a little harder to read your application, but reverse engineering is still possible.
My goal is just to make it very difficult to read, and not be able to decompile and run (not sure if that's possible, I guess it will still run just with ugly variable names??)
So variable names like:
String username = "asdfsadf";
will become
String aw34Asdf234jkasdjl_asdf2343 = "asdfsdaf";
Is this correct:
public classes and variables will remain unchanged
ONLY private strings/classes/methods can be renamed
string encrytion can be used for some sensitive string data like encryption keys etc.
Really my goal is so that someone can't just decompile and release the code.
Web applications run server side. Clients will not see the code unless you mess things up.
There are plenty of good Java obfuscators which will do what you say, and much more. Here are some from google:
ProGuard
yGuard
JODE
Although these will make it much more difficult to read the decompiled code (and some decompilers will refuse to even try), keep in mind that it is always possible for someone to reverse-engineer the code if they have the binary, and are knowledgeable and patient enough.
The problem here is that the code needs to be in proper java syntax when you compile it. So no matter what obfustication you applied, if I have access to even just the bytecode I can figure out a way to reconstruct the source.
(http://www.program-transformation.org/Transform/JavaDecompilers#Java_Bytecode_Decompilers)
What you would need to do is keep the proprietary part of the software in such a place that your pirates would not be able to see it. As far as I am aware, that is the ONLY way to avoid hijacking your software.
You cannot prevent java code from being decompiled and run. Even if it is obfuscated, there may be people out there that are still able to figure out what your code is doing, despite the obfuscation. Everything you publish can be reverse engineered.
There exist even much stronger efforts in other languages to prevent decompiling and debugging, disk copy protection solutions for example, and even they get reverse engineered and hacked frequently.
If you don't want people to reverse engineer your code, let it run server side only, don't publish it and try to harden the server as much as possible.
http://www.excelsior-usa.com/protect-java-web-applications.html
Disclaimer: I work for Excelsior.
http://www.arxan.com/products/server/guardit-for-java/
Disclaimer: I don't work for Arxan.
No amount of obfuscation can protect you against "decompile & compile again" (without trying to understand what the code does). Decompilers don't care for unreadable variable names, nor do compilers.
Incidentally, if someone has access to your code, they don't need to decompile it to use it.
So the question is really: What do you want to achieve? When you know that then you can go to the next question: How much does it cost and how much money can I earn?
Usually, that equation is: You can't save/earn any money from obfuscation but doing it costs you time and money (good obfuscators aren't free). So it's a negative ROI.
Instead, try this approach: Create a great product (so people will feel it's justified to pay for it), fix bugs quickly (-> the thieves have to steal your work again and again just to keep up), add new features. That way, honest consumers have reason to buy from you.
If you plan to get money from thieves and criminals, well, forget it. They don't want to pay you, no matter what. You can make their lives a little bit harder but at a cost.
I recently lost my android game code. When I tried to retrieve it from APK in my test device, I was able to recover all assets, xml and few java files - less complex ones (using dex2java converter). However original code of the more complex java files (like renderer) couldnt be recovered properly. Lots of modifications had been made to the code (like additional while, break, continue, labels).
Please suggest how the actual JAVA code(unaltered) can be recovered from the APK.
Thanks
The simple answer is that it cannot.
Source code is not included into the compiled version. The compiled classes only contain JVM instructions. From these instructions, you can often reconstruct the logic, but rarely if ever the original source.
I'm fairly new to programming and have been taking courses on Lynda to learn the fundamentals. I have some knowledge of Java and HTML, but I wanted to refresh my memory so I can start learning Objective-C. The Lynda course has us working in JavaScript because of its pretty core syntax. So, in order to get a point of reference, I tried downloading some .js files integrated into HTML pages from various sources. However, this proved to be unhelpful and I am at a loss for understanding because of the way the files are formatted. It seems as if most files put one line of code after the other. I realize that because of flexible whitespace restrictions with JavaScript that this does not hinder the way the code runs, but why did the developers choose to put it all on one line like that? They obviously didn't write the code that way, as that would be extremely tedious and hard to work with, so why did it come out that way when I try to view it? Is it just something that happens when you try to download the resources of a page? Any clarification would be appreciated.
Below is a photo of a JavaScript file I tried viewing. As you can see, all the code is restricted to one single line.
Also, if anyone could offer some insight about where to go after I've finished my course if I'm looking to develop for iOS, that would be greatly appreciated. Lynda also offers an Objective-C Essentials course, as well as an iOS Development course, but I feel like it's a pretty linear path that could be expanded on greatly with some literature or other online documentation.
what you are seeing is a minified version of the javascript file. The main advantage of minification is that it reduces the amount of data that needs to be transferred (bandwidth usage).
If you wish to view the code in human readable format, you can use online tools like this
Yes as karthikr said it's minified. Which means its all there but without the line breaks. So to see it all you have to scroll right.
Or you can use http://jsbeautifier.org/ to bring back the break lines.
There are several reasons for minifying javascript. One is that it makes the code less readable (yeah, some devs don't want you to "steal" functions and see what it does easily). Another, and a big part of why, is that it reduces bandwidth. A file with long variable names and whitespaces everywhere can be multiple times bigger than a minified version - so it improves performance!
Bandwidth costs money, especially for users and especially if they're on mobile devices with a bandwidth limit.
So to solve this problem developers will minimize the file size of what ever they can.
The JavaScripts you are seeing have been minified by libraries such as Uglify or YUI compressor (list not exhaustive).
Doing this will take out unnecessary whitespace and reduce the lengths of variable and function names that are not globally exported.
Developers may also gzip the files too which will reduce the filesize even further.
Edit: grammar
Can we completely reverse-engineer the source code from java bytecode ? Why this feature is allowed in Java and How successful are java decompilers against obfuscators.?
I know this question is old but I kept looking for a reliable answer until I found nothing.
So in this post I summarize some of my effort to obfuscate a J2EE JAR.
It seems , that by year 2014 (time of writing) there are not many options out there.
If you read this review later then things may have changed or fixed.
When I think why , I start to sense that the whole obfuscation effort gives a false sense of security. Don't get me wrong. It does add a level of security, but not as much as I would hope.
I will try to give a preview of what I found to explain myself. My recommendation are personal , others may disagree with it.
So to begin with: obfuscation in Java is the process of taking bytecode and making it less readable (using a decompiler of course) while maintaining its original functionality.What can we do, Java ,working as an interperter, must keep its bytecode exposed. You run the obfuscator as a measure of security in case the class file falls into the wrong hands. The result of the obfuscation is a reverse-mapping files and a JAR with the obfuscated classes. The reverse mapping file is used of-course to perform stack trace reading (a.k.a re-trace) or to revert the bytecode to its original shape. The runtime performance hit of an obfuscated class should not pass the 10% (but this really depends on what you do in your code).
But there is a big “but” . Obfuscation will scramble your code but it won’t make it hacker-proof. Bare in mind you only buy time and a determined hacker will find a way to reverse engineer your bytecode into its pure algorithm.
IMHO: the best way to hide a sensitive piece of code is to drown it in some huge pile of meaningless code.
Some of the hackers will try to modify your bytecode (by code injection) to help them achieve their goals. Some obfuscators offer additional level of JAR hardening , making it harder to modify.
De-obfuscators and de-compilers: my favourite Java decompiler is JD-GUI . However, when it comes to de-obfuscators I found the market pretty empty. Most of the tools ask you for a hint (what obfuscation tool was used to encrypt the source JAR) , yet none of them really deliver results (some of them even crash when trying to de-cipher the JAR). They are open source projects with low maintenance. I couldn’t even find a paid application to do a decent de-obfuscation. so enlighten me if you know something.
Free solutions
There are open source , free obfuscators which usually simply rename the classes/methods names, making it one letter method (i.e. from printUsage(String params) to a(String p) ).
They might ,as hinted here , even strip debugging information to make it a bit more difficult. (debugging information is kept at the end of every Java method bytecode and contains: line numbers, variables names ,etc.).
Its a nice effort , but an experience Java developer with a debugger can very easily deduce the purpose of each parameter while doing few live runs.
One of the nice open source obfuscators is ProGuard but there are several more tools.
Nevertheless , if you truly security fanatic you will probably want something stronger. Stronger demands more features (and more money) which leads us to the next bullet:
Paid solutions
While free products may only change classes method names , paid product will usually offer more features:
code/flow obfuscation: this will change the method code and inject empty loops/dead code/confusing switch tables and alike. Some of them may even scramble the exception table content. the obfuscation strength usually determine the output size.
Note: regarding code obfuscation: I deliberately avoided the details in my review. Some of the bytecode I saw and analyzed expose their obfuscation methods, and I wish to protect their IP. I do have an opinion about who uses better algorithms. contact me if you wish to know.
classes/method renaming : well this is the obvious , we discussed it in the free obfuscation. Some of the product will rename the class name and then recursively search for reflection usage of that class and fix those too. Paid products may even rename Spring /Wink configuration files for the same purpose (renaming in reflection).
String encryption: for every string “like this” in the code, it will encrypt it to some level and keep the key somewhere (in the class constant table/static blocks/a new method or any other mean).
debug information : stripping parts or scrambling.many of them will remove the line numbers info.
class
hardening: all kinds of methods like injecting some signing scheme into the beginning of the class/method, making sure an outsider won’t be able to easily modify the JAR and run it. Less important for Android or applets as most of them are digitally signed anyhow. some will combine hardening with water-marking to track pirated copies. But we all know anti-pirating methods by software are doomed to be hacked. Game industry suffered from it for decades until network based subscriptions arrived.
Since most products here deal with Java , some of them provides Android integration. It means it will not only obfuscate the Java (dalvik) code , but also manipulates the Android's manifest file and resources. Some offer anti debugging: remove the debug flag in android apps.
Nice GUI app to configure the various options and maybe do a re-trance on a given log file. The UI is usually used to generate a config file. with such file you can later re-play the obfuscation many times, even from command line.
Incremental build support - this is useful for large groups who release product updates/fixes frequently. You can tell the obfuscator to preserve old “obfuscation” result and randomly obfuscate only “new” code flows. this way you can be sure minimal impact on your methods signature. Without this flag , each obfuscation cycle on a JAR would yield a different output as most good tools use some level of randomness in their algorithms.
CLI and distributed builds. When you work alone then running an obfuscator is not a big issue. you need to configure the obfuscator to your relevant options and run it.However, in enterprise , when integrating obfuscator into the the build script things are a bit different. There is another level of complexity: build engine tasks (like ant/maven) and license management. The good news that all obfuscator I tested have command line API. In distributed build environment there are cluster/pool of build machines to support concurrent demand of builds. The cluster is dynamic and virtual, machines are going up or down, depending on various conditions. Some obfuscation products are based on cpuID license file or hostname. This can create quite a challenge for the build teams to integrate. Some prefer a local floating license server. Some may require public license server (but then: not all build farms have access to the public internet). Some offer multi-site license (which in my opinion is the best).
Some offer code optimizations - algebric equivalence and dropping of dead code. Its nice, but I believe that today's JDK do good job in optimizing bytecode. Its true that dead code makes you downloadable bigger, but with today's bandwidth its less than a problem. I also want to believe that in software today 20:80 thumb rule still applies. in any application 20% is probably a dead code anyway.
So who are the players I tried ?
KlassMaster by Zelix.com - one of the oldest in the industry. Yet they deliver a solid product with 3-4 releases per year. This been going for decades (since 1997). Zelix provides good email support and answered all my emails in a timely manner. They have a nice GUI client to either obfuscate a JAR or create a config file for future obfuscation. It simple and slick. nothing special here. They provided simple to read on-line documentation for all their flags. they support both “exclude” and “include” regular expressions for what the engine should obfuscate. The thing I liked about their process most is that it also adds “noise” to the exception table. It makes it a bit more confusing regarding the method exception handling. Their flow obfuscator strength is quite good and can be configured between 3 possible levels (light,medium and aggressive). Another feature I liked is the fine tuning they provide for debug info stripping (online line numbers, or online local variables or both). Klass Master doesn’t provide any
dedicated Android flags or anti-tamper methods. Their licensing model is quite simple: a text file to be placed near the KlassMaster main JAR. They also support incremental obfuscation.
JFuscator from secureTeam.net : While secureTeam also has a .Net tool , I focus on their Java tool capabilities. Their (Swing based) GUI tool seems nice but it crash when trying the simplest obfuscation task. the error was always the same: Error reading '/opt/sun-jdk1.7.0_55/jre\lib\rt.jar'. Reason: ''/opt/sun-jdk1.7.0_55/jre\lib\rt.jar': no such file or directory' . Now of course I have my Java installed in /opt/sun-jdk1.7.0_55/jre. You can image that they simply didn’t expect linux back slash structure. I contacted secureTeam.net support by email with the minor “path” problem. They asked if I am a linux user and after I replied I am , they never answered my email. I also tried their web site on-line chat : no response. So there I stopped testing. Without further results, I couldn’t examine the obfuscated bytecode quality. From their web site it seems they have anti-tamper method , String manipulation, method renaming and few other features.
GuartIt4J (by Arxan.com) : Arxan is fairly solid player in the mobile environment and as such they offer Android obfuscator which of course works well for Java. They have one of the most flexible engines.They provide code obfuscation,string encryption and alike You can define the complexity of code obfuscation. it is simply an integer. the higher - the longer your method turns out. ofcourse, you must be carefull not to exceed the JVM 64KB limit per class… As I said before one of the best strategies to hide a sensitive code is not to encrypt it , but to inject it into huge pile of garbage. This is exactly what GuardIt does. It can also explode in the same way the methods exception table. I managed to create a method with 100 exceptions in its exception table (pre-obfuscator it was 5). what they miss: their re-trace program is not part of the supplied main JAR. Nevertheless, they were kind enough to send me a sample Java program that performs re-trace given the reverse mapping file and the log. They don’t support incremental obfuscation and no flexibility regarding debug information. Debug information stripping is either all or nothing. watching the output JAR you will tons of conditions and jumps that were injected. Bare in mind , exploding the class size has its performance hit. In some methods I measured almost 50% performance hit when applying long obfuscation (no I/O in those methods). so extrapolating the code comes with a price.(from a 400 opcodes - I went up to 2200 opcodes after obfuscation). JD-GUI , my de-compiler failed to open such classes and crashed (IndexOutOfBoundException). They also supply complete class encryption . Meaning the class is encrypted with some symetrical key which demands a special (or custom written) class loader to open it in memory. This is an anti-tamper mechanism as well as hiding code. Just remember that a JVM can’t run that class without the class loader help. Its a nice feature, but the secret key and the bootstrap loader JAR are probably there. If he got the encrypted JAR the hacker will eventually get his hands and decrypt the classes. Yet this another level of obstacle the common hacker will need to pass. What I didn’t like here is the license file policy: is bounded to CPUid or need to install a floating license server.
SecureIt (by Allatori.com) : SecureIt offers all the general code obfuscation, string encryption ,renaming and such. On top of the standard obfuscation methods they also offer some kind of water-marking which is an anti-tamper/pirating method. They support Android and JavaME (who uses ME these days?!). They support incremental obfuscation. The one thing to note about configuring SecureIt: it is all command line. No GUI tool this time. Personally , I don’t mind command line tools as long as they come with good documentation. Luckily they have a very good documentation and a rich API with many flags to tune if you wish. you can re-trace with they tool (also a command line ) . They can’t obfuscate the exception table. I didn’t check their licensing mechanism.
DashO (by Preemptive.com) : DashO obfuscator will be remembered probably as the best UI tool you can get (to create your configuration). Like SecureIt they lake the exception table obfuscation but they have all the rest of the required features (as well as CLI, Spring framework and gradle/ant integration, and even an eclipse plugin) . Well, they do document a try-catch obfuscator (which is same as exception table obfuscator) , but it is only a recommendation to the engine. When I tried it , it had nil effect on the exception table. As I said , the GUI tool is superb and has a re-trace embedded into it. they also offer some kind of application signing and water-marking as an anti-tamper/pirating mechanism. DashO provides superb Android integration and also combine in their product a door for analytics uploads. You can actually track your application. Injecting crash log uploaders and reporting code to your JAR. Nevertheless that’s not the scope of obfuscation - that’s a whole different code injection product. They have a very good support. both online and by phone. Their licensing scheme is based on monthly subscription or one time purchase payment. A bit different than others. They are using a floating license server to support large environments.
I hope this helps a bit..
Can we completely reverse-engineer the source code from java bytecode ?
Not completely, because some aspects of source code, such as whitespace, local variable names, and comments, are not preserved in bytecode. Otherwise, yes -- while you can't get the exact same source code out, you can almost always get something that can at least be compiled back to the same bytecode.
Why this feature is allowed in Java
It's not so much "allowed" as it is "not prevented". And it's not prevented because doing so is impossible -- the code must be runnable to be useful; if the code is runnable, then it is analyzable; if it is analyzable, then with sufficient analysis it can be converted back to source.
How successful are java decompilers against obfuscators?
Not very. Most obfuscators I've seen (esp. ProGuard) are primarily effective in removing meaningful function and class names; obfuscating the logic itself is not typically attempted.
you can get source code from binary these days. Although the source code obtained by Java's bytecode is more readable, obfuscating will make it slightly unreadable. Its not that only Java can be reverse engineered to code. Even C/C++ these days (with Hexrays plugin for IDA Pro) can be decompiled to source. Obfuscaters will make it hard to read but not impossible. There is nothing that can save your program from an intelligent and capable reverse engineer. :).
Good luck.
Can we completely reverse-engineer the source code from java bytecode
?
The java class file is based on a spec so anyone can read into it. A tool like JD-GUI will tear into your source code easily. It is not a 'feature' per se. While 100% reverse-engineering is not possible, most of your code can be reverse engineered.
How successful are java decompilers against obfuscators?
Depends. The point of the obfuscator is to remove any meaningful names and try to introduce confusion in the code without impacting performance. Most developers are great at obfuscating code themselves :) Pro-guard is pretty good at obfuscation.
I was just messing around. I downloaded the dex2jar http://code.google.com/p/dex2jar/ and the Java Decompiler JD-GUI http://java.decompiler.free.fr/?q=jdgui
I got my own apk file (signed, sealed and on Google Play), used dex2jar to make it into a jar repository.
command line (Windows users use .bat, everyone else .sh):
d2j-dex2jar.bat -f MyAwesomeApp.apk
I dragged and dropped the output into a JD-GUI, and all the class files, the original code reappeared.
I was taken aback a bit. Is my java/Android code this exposed? How is ProGuard protecting my apk if it can be decompiled and regenerated so easily? It doesn't seem obfuscated at all...
Thanks in advance.
Obfuscators usually simply change classes, methods and fields names to names that have no meaning. So, if you have "ScoreCalculator.computeScore(Player p, Match m)" you end up with "A.zk(F f, R r)". This is similar to what Uglify or Closure compiler do for javascript, except that in javascript it is to reduce source length.
It is possible to understand what the method does anyway, it is only harder.
Aslo, Java uses late binding (as DLLs or SO files). So, calls that go outside your code (like to java.util, java.lang etc.. packages) cannot be obfuscated. Also, if your code needs to receive calls from outside (a typical example, register a listener on a button), that code cannot be obfuscated. Same happens for a DLL, where you can see clearly the name of method that need to be called form outside the DLL and calls to other DLLs.
However, the mapping between a certain source code and the compiled code is not necessarily one to one. Older C compilers used to produce the same op code for a given source directive, so decompilers were very effective. Then C compilers added many optimizations to resulting op code, and these optimizations made decompiler mostly ineffective [1]
Java never implemented (a lot of) optimizations at compile time, because to run on different platforms (there including different android devices), Java decided to apply serious optimizations later, at run time, based on the architecture and hardware properties of the running device (this is what "HotSpot" is mostly about [2]).
Good obfuscators usually also reorder bytecode instructions, or insert some useless ones, or apply some optimizations upfront to make decompilers unable (or less able) to derive source code so easily.
This technique is useless when it comes to people who can read bytecode, as any possible C obfuscation is useless if a person can read assembler code.
As many cracking softwares demonstrate, reverse engineering is always possible, even with C or other laguages, even on firmware (think about iPhone firmwares), cause the client your code is running on is always untrusted, and can always be tampered with.
If you have very mission critical code, something worth a lot of money that someone else may steal, I'd suggest to run it server side, or validate it server side somehow.
I might also add, that there is modern alternative to this APKTool->dex2jar->JD-GUI route!
Just try open-source APK and DEX decompiler called Jadx: https://sourceforge.net/projects/jadx/files/
It has also online version here: http://www.javadecompilers.com/apk