Libraries such as ASM, BCEL, Javaassist and AspectJ are all capable of runtime bytecode manipulation but how do they achieve this?
I have done some basic bytecode manipulation using ASM before but i don't understand how it works. Is the Java Agent executed in the JVM before the remainder of the program, allowing for ASM to load the compiled classes and edit them before they are executed by the JVM?
If so, is it possible to perform java bytecode manipulation without using an external library like ASM and loading the compiled class files with an BufferedReader and writing a custom parser etc. for example?
These libraries settle on standard Java APIs which, of course, you can also use yourself without these libraries.
First of all, Java class files are just sequences of bytes in a well defined format, as specified in JVMS §4, The class File Format. The primary task of the mentioned libraries is to provide tools for processing byte sequences in this format. The second is about getting the definitions of existing or exporting modified or newly created classes.
There are two different way of dealing with the second task. One is to read compiled classes from persistent storage like file systems or jar files, etc. and writing them back to these storage while the particular code is not running, like build and deployment tools do. This should be trivial, as it just boils down to reading and writing bytes.
The other is to manipulate classes at runtime, which can be done by Java Agents via the Instrumentation API. It offers mechanisms for intercepting classes at loading/definition time before their first use, but also redefinition of classes. The latter can’t change them freely, currently, it has to retain all field and method declarations, so it can be mainly used to change the executable code of the methods.
If you want examples for class file processing without additional 3rd party libraries, there are some answers on Stackoverflow
Extract the class name from a class file
Find all class dependencies
Parse the constant pool
Iterate over all instructions of a method
Of course, these examples are only single-purpose code or sketches. If you expand them to something more general or useful, you will soon end up at basically re-implementing these libraries.
Class files are just a sequence of bytes, the format of which is specified in The Java Virtual Machine Specification. BufferedReader is for text files so you'd want BufferedInputStream, but the format is quite complicated.
You can load manipulated class files as if they were generated by javac. You can also load them dynamically with java.net.URLClassLoader.newInstance or similar. Java Agent allows modification of class files as they are loaded, either through a Java or a native interface (the latter being necessary if you want to modify classes the are loaded before classes that load classes).
Recently java development group at Oracle announced a new JEP( Java enhancement proposal) for class file manipulation. Now we will not need any additional libraries.
https://openjdk.org/jeps/8280389
Related
I'm trying to read a .qm translation files with Java.
.qm files are binary files. I don't have access to the .ts files.
And I don't find much info on these .qm files.
How are they structured ?
Regards,
There's no documentation that I know of, but if you look at QTranslator::load you should be able to follow the format of the QM file.
You will probably need to reimplement QTranslator in Java, as you need not only the ability to load the files, but also to extract and apply translations in Qt fashion.
As per request of OP:
You could use those files by using the Qt libraries and JNI. By using the translator in a c++ dll you can translate strings easily. However, you cannot extract the files or list the contained translations. But if all you need is the actual translation, this solution should work.
I cannot give a real example, because I only now how it works in theory, I haven't tried it, because it's not trivial. But if you are eager to try it out, the general idea would be:
Create a C++ dll and build it against QtCore. The easiest way is to download Qt from their website qt.io. You can for example create a default library project with QtCreator. Note: Besides Qt5Core.dll, Qt requires other libraries to correctly run. They are all included in the installation, but once you deploy your application, those of course have to be includes as well.
Include JNI to the C++ project and link against it. if you're new to this, here is a nice tutorial: Java Programming Tutorial
Create your wrapper methods. Methods in cpp you can call from java that take java strings, convert them to QString, translate them with QTranslator and convert them back.
Load the library in Java and execute those methods
Important:
First, I don't know how java handles dll dependencies. If you encounter errors while loading the dll, it's probably because dependencies of your dll are not present. Second, Qt typically requires a QCoreApplication running in the main thread for most of it's operations. I tested the translator without such an app, and it worked. So apparently for translations only the app is not required. However, depending on what you do in your dll, I think this is important to know.
If you need more details, feel free to ask.
When programming in C++ we have header files and source files. When our company sells our product (which is just a library of useful functions essentially) we provide the client only our header files and compiled source code (not the actual source code). This way clients buying the code see the function definitions in the header files and can make calls to these functions using the compiled code.
How would a company who is developing in Java go about sharing their library functionality without giving out their code? Since there are no header files in Java is this just done through documentation; i.e, provide the client with an API document showing all the function signatures and also provide a jar file with the compiled classes?
In Java you don't need source code to compile against - the bytecode contains all the metadata you need.
So for example, if I create an interface with some methods in, compile it into a class file, then package that up into a jar file and give you that jar file, you can use that interface and see all it's members etc without having the source code.
Typically you'd also provide documentation to explain the meaning of the methods etc, but they're not required in order to compile against the jar file.
Take for example YWorks YFiles, they offer their proprietary lib but they do some obfuscation on their distribution(using their own obfuscator).
As for guidance how to use their lib, they ship their lib with javadocs, some howto documents and some sample code.
Whenever I build my app all classes (logically) are visible in the .jar that comes out of it.
Aswell as a class that holds information to my MYSQL server (for the app to connect to). But I dont want this information to be publicly visible!
How can I "hide" this code or "hide" the class?
Thanks!!
I think you mean you dont want someone to do reverse engineering with your .class inside your jar file. There are many decompilers that can do that.
So you would need to Obfuscate your code with an obfuscator utility.
The process of obfuscation will convert bytecode into a logical
equivalent version that is extremely difficult for decompilers to pick
apart. Keep in mind that the decompilation process is extremely
complicated and cannot be easily 'tweaked' to bypassed obfuscated
code. Essentially the process is as follows:
Compile Java source code using a regular compiler (ie. JDK)
Run the obfuscator, passing in the compiled class file as a
parameter. The result will be a different output file (perhaps with a
different extension).
This file, when renamed as a .class file, will be functionally
equivalent to the original bytecode. It will not affect performance
because a virtual machine will still be able to interpret it.
Here is an article describing this process in more detail and
introducing an early obfuscator, Crema:
http://www.javaworld.com/javaworld/javatips/jw-javatip22.html
I'm writing a C++ dll (file extension in Visual Studio is .cpp but the code is C) which uses JNI for loading and calling instance methods of some Java classes. As a result my project is made up by C++ source and header files plus some Java class files (the class files for the Java objects used through JNI). I would like to create a single dll library including also the Java class files "in a single bundle". Is it possible to put both the C and Java files into the dll?
This is perfectly possible, though not necessarily intuitive. I haven't tested this but I would imagine it would work easily if you knew what you were doing.
First off, you're going to need a way to package the class file's binary into the native binary you own, and then of course be able to seamlessly read that chunk at any point in time. How you achieve this is up to you, there is no real 'right way' to do it.
That being said, if you can get a handle to the memory where the .class file's data resides (after loading the DLL) and pass it to Java as a byte[] or ByteBuffer, you'd then want to use any one of ClassLoader's defineClass overloads along with a call to findLoadedClass and then finally loadClass.
This would allow you to load in a class from virtually any data source. As I mentioned before, how you store/retrieve the native data for the class is up to you.
In Android applications, resources are specified in xml documents, which automatically are built into the R class, readily accessible within the source code as strongly typed.
Is there any way I could use a similar approach for a regular Java desktop application?
What I'd like to accomplish, is both the removal of strings from the code (as a separation of "layers", more or less) and to make it easy to add support for localization, by simply telling the program to choose the xml file corresponding to the desired language.
I've googled around a bit, but the things I'm looking for seem to be drowning in results about parsing or outputting xml, rather than tools utilizing xml to generate code.
Eclipse's message bundle implementation (used by plugins for example) integrates with the Externalize Strings feature and generates both a static class and a resource properties file for your strings:
http://www.eclipse.org/eclipse/platform-core/documents/3.1/message_bundles.html
For this integration to work Eclipse needs to see org.eclipse.osgi.util.NLS on the class path. From memory, the dependencies of the libraries it was available in were a little tricky for the project I used this approach in, so I just got the source and have it as a stand-alone class in my core module (see the comments for more on that).
It provides the type safety you're looking for and the IDE features save a lot of time. I've found no downsides to the approach so far.
Edit: this is actually what ghostbust555 mentioned in the comments, but not clear in that article that this isn't limited to Eclipse plugins and you refer to your resources via static members of a messages class.
I haven't seen any mention of others using this approach with their own applications, but to me it makes complete sense given the IDE integration and type safety.
I'm not sure if this is what you mean but check out internationalization- http://netbeans.org/kb/docs/java/gui-automatic-i18n.html
Are you looking for something that parses XML files and generates Java instances of similar "struct-like" objects, like JAXP, and JAXB?
I came across ResGen which, given resource bundle XML files generates Java files that can be used to access the resources in a type-safe way.
http://eigenbase.sourceforge.net/resgen/