Situation
Hi, I have 2 problems.
The situation is that I'm writing a Java API for Windows that also provides tools for injecting code into a process and then manipulate the target. I have already implemented the injection-part, for example injecting a jar into another jar. At this point my jar gets called (while the target already is at runtime) and starts in a complete static context.
Goals & problems
From here I have two goals:
I'd like to interact with the targets objects, thus I need references. For many objects this is already possible because they provide static access to their instances. For example awt.Frames#getFrames() provides access to all created Frame objects. But it would be awesome if there is a possibility to get access to arbitrary objects on the heap. Something like 'Heap#getAllObjectInstances()'.
Given an object instance, I'd like to hook up onto arbitrary functions of this object. For example whenever BufferStrategy#show() gets called, I want it to call another method first.
So I summarize the problems as follows:
How to get arbitrary object references from a static context?
How to hook up onto arbitrary functions?
Remarks
What I've done so far, remarks and ideas:
The JDI (Java Debugger Interface) provides such a method via VirtualMachine#allClasses() -> ReferenceType#instances(0). But the JDI needs the target JVM to be started with additional debug parameter which is no option for me. One could go down to low-level and analyze the heap with memory tools, but I hope someone knows a more high-level approach. Using the Windows API would be an option for me as I'm familiar with JNA/JNI, but I don't know such a tool.
The last resort would be to use IAT hooking with C-Code, a very low-level approach, I'd like to avoid this. As I can assume having a object reference at this point, maybe does the Reflection API provide a method to change an objects method? Or at least simply provide a hooking mechanism?
Be aware that changing the targeted code certainly is no option for me. And that it is already at runtime, thus ByteCode-Manipulation could also be an option.
Scenario
A scenario where this would come in handy:
The target is a game, deployed as jar. It renders with a Double-Buffer-Strategy, using the BufferStrategy class. It displays the image with BufferStrategy#show(). We inject our jar inside the game and like to draw an overlay with additional information. For this we get an reference to the used BufferStrategy and hook up onto its show-method. So that it calls our drawOverlay-method everytime it gets called, then we pass back to the original show-method.
What you need is JVMTI agent - a native library that makes use of JVM Tool Interface.
Agents can be attached dynamically to a running VM using the Attach API.
See VirtualMachine.loadAgentPath.
To get all instances of a given class use JVMTI IterateOverInstancesOfClass function.
See the related question for details.
To intercept a method of a foreign class you'll need JVMTI RetransformClasses API. The same can be also achieved by using Java-level instrumentation API, see Instrumentation.retransformClasses.
For the example of JVMTI-level method interception refer to demo/jvmti/mtrace from Oracle JDK demos and samples package.
Java-level instrumentation will be easier with bytecode manipulation libraries like Byte Buddy.
Related
I am looking towards some approach where by using Java agent or instrumenting classes (preferably something at lower level than user classes) to intercept all object creation in JVM (new or any alternative ways to create Object), There is a similar question which doesn't focus on Java agent or something lower than instrumenting user classes
Java Objects can be created in several different ways.
From Java code, when a Java method, either interpreted or compiled, executes one of the following bytecode instructions: new, newarray, anewarray, multianewarray.
From native code, when native methods, including those in standard class library, call one of JNI functions: NewObject, NewObjectArray, NewStringUTF, NewDirectByteBuffer, etc.
Directly from VM runtime, when a new object is created internally by JVM, for example, in response to Object.clone(), Throwable.getStackTrace(), Class.getInterfaces(), etc.
Unfortunately, there is no single point where you can collect objects from all these sources. However, there are means for intercepting all of them.
Objects instantiated from Java can be caught by an Instrumentation agent. The agent needs to define a ClassFileTransformer that will scan the bytecode of all loaded classes for object-creating instructions and modify it.
Note: there is no need to intercept all new instructions, you can instrument Object() constructor instead. But you still need to intercept array allocation instructions.
JNI functions can be intercepted by JVMTI agent. You need to define your own native hooks for NewObjectArray, NewStringUTF etc. and then replace JNI function table. See JVMTI Reference for the details.
Objects created by the VM can be caught by JVMTI Event Callback mechanism. The desired event is VMObjectAlloc.
Note: JVM will not post VMObjectAlloc event for objects allocated from Java or by JNI functions.
All other ways of object instantiation (cloning, reflection, deserialization) fall into one of the above categories.
Get JDK 8 Demos and Samples from Oracle Java SE Downloads website.
There is a sample JVMTI agent for exactly this question.
Look under
jvmti/heapTracker
jvmti/hprof
You can take a look at this opensource java agent created by devexperts team
https://github.com/Devexperts/aprof
It provides nice reports to detect where memory is allocated. But, as i know, it doesn't intercept new objects created via JNI or sun.misc.Unsafe.allocateInstance in current version
It is pure java agent which manipulates bytecode with ASM. Before each object allocation aprof inserts method call which traks allocation size and location stack (where this allocation occurs)
I'm looking for a lightweight way of embedding some GUI-based object inspection facilities in a Java application.
Ideally it would be something like the variable inspector in the Eclipse debugger, which lets you see all the fields of a given object instance, and drill down to inspect fields contained within these. Doesn't have to be pretty, just needs to have a way to inspect the value of all fields
I don't just want to use a debugger: this idea is that the facility can be used on the application at runtime, allowing for quick diagnostics without restarting the application.
It needs to be pretty lightweight: since this is meant to be embedded in a deployed application, I don't want to pull in a whole load of extra dependencies. So Swing would be OK, but pulling in a whole native widget library would be out of the question (sorry, no SWT....)
It needs to be able to inspect an arbitrary Java/JVM object (presumably via reflection)
Does anyone know a tool like this?
I do not know "ready" framework but can recommend you to either use JDPA directly or use one of libraries that are using byte code engineering done at runtime. For example take a look on java-interceptor that can help you if you can control the code you want to interspect.
There is ReflectionUI.
By default it will only present you the public getter/setter properties but you could customize it to access the private/protected fields for your debugging purposes.
I am writing an application in Qt that I want to extend with plugins.
My application also has a library that the plugins will use. So, I need a 2 way communication. Basically, the plugins can call the library, and my application which loads the plugins will call them.
Right now, I have my library written in C++, so it has some classes. The plugins can include the header files, link to it and use it. I also have a header file with my interface, which is abstract base class that the plugins must have implemented. They should also export a function that will return a pointer to that class, and uses C linkage.
Up to this point I believe that everything is clear, a standard plugin interface. However, there are 3 main problems, or subtasks:
How to use the library from other languages?
I tried this with Python only. I used SIP to generate a Python component that I successfully imported in a test.py file, and called functions from a class in the library. I haven't tried with any other language.
How to generate the appropriate declaration, or stub, for my abstract class in other languages? Since the plugins must implement this class, I should be able to somehow generate an equivalent to a header in the other languages, like .py files for Python, .class files for Java, etc.
I didn't try this yet, but I suppose there are generators for other languages.
How am I going to make instances of the objects in the plugins? If I got to this point the class would be implemented in the plugins. Now I will need to call the function that returns the instance of the implemented abstract class, and get a pointer to it.
Based on my research, in order to make this work I will have to get a handle to the Python interpreter, JVM, etc., and then communicate with the plugin from there.
It doesn't look too complex, but when I started my research even for the simplest case it took a good amount of work. And I successfully got only to the 1st point, and only in Python. That made me wonder if I am taking the right approach? What are your thoughts on this.. maybe I should not have used Qt in my library and the abstract base class, but only pure C++. It could probably make the things a bit easier. Or maybe I should have used only C in my library, and make the plugins return a C struct instead of a class. That I believe would make the things much easier, since calling the library would be a trivial thing. And I believe the implementation of a C struct would be much easier that implementing C++ class, and even easier that implementing a C++ class that uses Qt objects.
Please point me to the right direction, and share your expertise on this. Also, if you know of any book on the subject, I'd be more than happy to purchase it. Or some links that deal with this would do.
C++ mangles its symbols, and has special magic to define classes, which is sort of hacked on top of standard (C) object files. You don't want your files from other languages to understand that magic. So I would certainly follow your own suggestion, to do everything in pure C.
However, that doesn't mean you can't use C++. Only the interface has to be C, not the implementation. Or more strictly speaking, the object file that is produced must not use special features that other languages don't use.
While it is possible for a plugin to link to your program and thus use functions from it, I personally find it more readable (and thus maintainable) to call a plugin function after loading it, passing an array of function pointers which can be used by the plugin.
Every language has support for opening shared object (SO or DLL) files. Use that.
Your interface will consist of functions which have several arguments and return types, which probably have special needs in how they are passed in or retrieved. There probably are automated systems for this, but personally I would just write the interface file by hand. The most important is that you properly document the interface, so people can use any language they want, as long as they know how to load object files from their language.
Different languages have very different ways of storing objects. I would recommend to make the creator of the data also the owner of the memory. So if your program has a class with a constructor (which is wrapped in C functions for the plugin interface), the class is the one creating the data, and your program, not the plugin, should own it. This means that the plugin will need to notify your program when it's done with it and at that point your program can destroy it (unless it is still needed, of course). In languages which support it, such as Python and C++, this can be done automatically when their interface object is destroyed. (I'm assuming here that the plugin will create an object for the purpose of communicating with the actual object; this object behaves like the real object, but in the target language instead of C.)
Keep any libraries (such as Qt) out of the interface. You can allow functions like "Put resource #x at this position on the screen", but not "Put this Qt object at this position on the screen". The reason is that when you require the plugin to pass Qt objects around, they will need to understand Qt, which makes it a lot harder to write a plugin.
If plugins are completely trusted, you can allow them to pass (opaque) pointers to those objects, but for the interface that isn't any different from using other number types. Just don't require them to do things with the objects, other than calling functions in your program.
I am trying to figure out a way to find a class instance inside a working JVM. The JVM is embedded into another process and an object is created by this process. The same process executes my Java code. I do not have a direct access to the Java object reference in my Java code, but I know the class of that object. I want to find out if there are objects of this class already instantiated in JVM, and if they are, I want to find them (get say an Object array with references to all the instances of this class). This mechanism can be either a Java API or an JNI API.
I know it is possible since debuggers let me inspect all instances. I just cannot seem to find the way.
Thanks
Nikita
In case anybody is curious, It is possible and I am doing it now using JVMTI. Not straightforward, but very doable. You can look at my other post about jvmti and you will find the answer.
Perplexed by jvmti object allocation callback behavior
Thanks
Use Java Instrumentation APIs.
This should be possible from another process using the debug interface
http://docs.oracle.com/javase/6/docs/jdk/api/jpda/jdi/
However, if I understand your requirement correctly, you would need your process to debug itself which is probably not going to work.
We have a big Java application under Android ("big" just means it's too much work to translate the application). We must access to an engine written in .Net (this engine is also too "big" ...). This engine is only calculation.
We therefore seek a solution with monodroid. Our main problem is interop betwen monodroid and Java. At this time, we get :
call a Java function in a .jar library from a Mono application
But we can not call and start a Java activity. Is it possible ?
The second problem is that we do not know how to communicate from Java to Mono. Is it also possible?
There are several ways to call integrate Java and managed code, depending on what exactly you want to do.
Java to Managed
If you need to call some managed method, you may be able to use Android Callable Wrappers, which are generated for every Java.Lang.Object subclass. However, there are a number of limitations, so that may not be ideal.
If you need to create an Activity, you can use Context.startActivity(), the same as you would in Java. You can view the generated obj\Debug\android\AndroidManifest.xml to determine the appropriate class name to use, or you can use e.g. ActivityAttribute.Name to manually control the Java-side name. (Using ActivityAttribute.Name is not recommended, as it slows down type loading.)
The same is true for Services: use Context.startContext() and continue on your merry way.
If you need to share data, the easiest way would be to use a ContentProvider. ContentProviders are usually intended for cross-process data sharing, but they should be usable intra-process as well, when you need to share data between Java & managed code and you hit the limitations of Android Callable Wrappers.
Managed to Java
By and large, calling Java code from C# is the mirror of Java code calling C#: you can use e.g. Context.StartActivity() to start a Java activity, use a Java-side ContentProvider through the the Context.ContentResolver property, etc.
An example of starting a Java activity from managed code is the GoogleMaps sample, in which Context.StartActivity() is used to launch the included Java activity.
You can also use Java Native Interface (JNI) support to create Java instances from managed code and invoke methods on those instances. This is painful and brittle, but it works and allows invoking APIs that aren't otherwise exposed.
You can easily call Java activity from native code like this:
var intent = new Intent().SetClassName(this,"com.myapp.java.JavaActivity");
StartActivity(intent);
As I understood from this article you can invoke native code from Java via ACW, but I think that it's too difficult