We Keep Coding

Starting Instrumentation Agent after VM Startup

I was hoping for someone to explain this item since I might be getting this wrong:
I was reading about Java Agent Instrumentation which says that the agent can start after VM startup. So if I want to dynamically replace some class (without brining down the app) is this what I am going to go for using agent-main? Or do I need to do something more here?
I know people might ask "Are you talking about JRebel" - not really because I want to do something simple and JRebel is an overkill.
instrument docs - Java docs for Instrumentation
I understand all the instrumentation overrides, but I am slightly confused how I can hook this agent with -agent argument after the app has started.

First your agent class needs to specify an agentmain method like:
public class MyAgent {
public static void agentmain(final String args, final Instrumentation inst) {
try {
System.out.println("Agent loaded.");
} catch (Exception e) {
// Catch and handle every exception as they would
// otherwise be ignored in an agentmain method
e.printStackTrace();
}
}
}
Compile it and pack it inside a jar-file for example. If you choose the jar-variant then it must specify the Agent-Class key in its manifest-file (MANIFEST.MF). It points to the class implementing the agentmain method. It could look like:
Manifest-Version: 1.0
Agent-Class: package1.package2.MyAgent
If it is located inside those packages, as an example.
After that you can load the agent via the VirtualMachine#loadAgent method (documentation). Note that the mechanism used by those classes are part of the Attach library of Java. They decided, as most users don't need it, to not directly add it to the systems path but you can just add it. It is located at
pathToYourJDKInstallation\jre\bin\attach.dll
And it needs to be somewhere where the system property java.library.path is pointing at. You could for example just copy it to your .../Windows/System32 folder or adjust the property or stuff like that.
As an example, if you want to inject an agent-jar inside another currently running jar, you could use a method like this:
public void injectJarIntoJar(final String processIdOfTargetJar,
final String pathToAgentJar, final String[] argumentsToPass) {
try {
final VirtualMachine vm = VirtualMachine.attach(processIdOfTargetJar);
vm.loadAgent(pathToAgentJar, argumentsToPass.toString());
vm.detach();
} catch (AttachNotSupportedException | AgentLoadException
| AgentInitializationException | IOException e) {
System.err.println("Unable to inject jar into target jar.");
}
}
With the same technique you can inject dll-libraries (if they implement the corresponding agent-methods via the native agent interface) into jars.
Actually, if that helps you, I have written some small library for that kind of stuff some time ago. See Mem-Eater-Bug, the corresponding class is Injector.java and the whole project has a small Wiki.
It has an example showing how to use that technique to manipulate a SpaceInvaders game written as Java application.

So apparently you want to reload classes at runtime. Such that your project can react to changes of the code without restarting.
To achieve this you need to prepare your project and write a very clean architecture, it involves using interfaces, factory-patterns, proxy-patterns and a routine that checks for updates and then destroys and rebuilds all current objects.
Unfortunately this might not be an easy task, but it is doable, depending on the size of your project and the amount of code that should react dynamically to changes.
I found this article helpful, let me explain how it works. You can easily load a class with ClassLoader.loadClass(...) and you can also use that to reload a class, very easy. However at the time you have compiled your code classes are some kind of hardwired already. So your old code will continue to create instances of the old classes although you have reloaded the class.
This is the reason why we need some kind of architecture that allows exchanging the old class with the new class. Also it is pretty obvious that current instances of the old class can not automatically be transferred to the new version as everything could have changed. So you will also need a custom method that collects and rebuilds those instances.
The approach described in the article uses an Interface instead of an actual class in the first place. This allows to easily exchange the class behind that interface without breaking the code that uses the interface.
Then you need a factory where you ask for instances of that Interface. The factory can now check if the underlying class-file has changed, if so it reloads it and obtains a reference to the new class version. It can now always create an instance of the interface which uses the up-to-date class.
The factory, by that, is also able to collect all created instances in order to exchange them later, if the code base has changed. But the factory should reference them using WeakReference (documentation), else you have a big memory leak because the Garbage Collector would not be able to delete instances because the factory holds references to them.
Okay, now we are able to always obtain up-to-date implementations of an Interface. But how can we easily exchange existing instances. The answer is by using a proxy-pattern (explanation).
It is simple, you have a proxy class which is the actual object you are working with. It has all the methods of the Interface and upon calling methods it simply forwards to the real class.
Your factory, as it has a list of all current instances using WeakReference, can now iterate the list of proxies and exchange their real class with a new up-to-date version of the object.
Existing proxies that are used all around your project will now automatically use the new real version as the proxy itself has not changed, only its internal reference to the real target has changed.
Now some sample code to give you a rough idea.
The interface for the objects you want to monitor:
public interface IExample {
void example();
}
The real class, which you want to rebuild:
public class RealExample implements IExample {
#Override
public void example() {
System.out.println("Hi there.");
}
}
The proxy class that you will actually use:
public class ProxyExample implements IExample {
private IExample mTarget;
public ProxyExample(final IExample target) {
this.mTarget = target;
}
#Override
public void example() {
// Forward to the real implementation
this.mRealExample.example();
}
public void exchangeTarget(final IExample target) {
this.mTarget = target;
}
}
The factory you will mainly use:
public class ExampleFactory {
private static final String CLASS_NAME_TO_MONITOR = "somePackage.RealExample";
private final List<WeakReference<ProxyExample>> mInstances;
private final URLClassLoader mClassLoader;
public ExampleFactory() {
mInstances = new LinkedList<>();
// Classloader that will always load the up-to-date version of the class to monitor
mClassLoader = new URLClassLoader(new URL[] {getClassPath()}) {
public Class loadClass(final String name) {
if (CLASS_NAME_TO_MONITOR.equals(name)) {
return findClass(name);
}
return super.loadClass(name);
}
};
}
private IExample createRealInstance() {
return (IExample) this.mClassLoader.loadClass(CLASS_NAME_TO_MONITOR).newInstance();
}
public IExample createInstance() {
// Create an up-to-date instance
final IExample instance = createRealInstance();
// Create a proxy around it
final ProxyExample proxy = new ProxyExample(instance);
// Add the proxy to the monitor
this.mInstances.add(proxy);
return proxy;
}
public void updateAllInstances() {
// Iterate the proxies and update their references
// Use a ListIterator to easily remove instances that have been cleared
final ListIterator<WeakReference<ProxyExample>> instanceIter =
this.mInstances.listIterator();
while (instanceIter.hasNext()) {
final WeakReference<ProxyExample> reference = instanceIter.next();
final ProxyExample proxy = reference.get();
// Remove the instance if it was already cleared,
// for example by the garbage collector
if (proxy == null) {
instanceIter.remove();
continue;
}
// Create an up-to-date instance for exchange
final IExample instance = createRealInstance();
// Update the target of the proxy instance
proxy.exchangeTarget(instance);
}
}
}
And finally how to use it:
public static void main(final String[] args) {
final ExampleFactory factory = new ExampleFactory();
// Get some instances using the factory
final IExample example1 = factory.createInstance();
final IExample example2 = factory.createInstance();
// Prints "Hi there."
example1.example();
// Update all instances
factory.updateAllInstances();
// Prints whatever the class now contains
example1.example();
}

Attaching an agent at runtime requires use of the attach API which is contained in the tools.jar until Java 8 and is contained in its own module starting from Java 9. The location of the tools.jar and the name of its classes is system-dependent (OS, version, vendor) and as of Java 9 it does not exist at all but must be resolved via its module.
If you are looking for an easy way to access this functionality, try out Byte Buddy which has a subproject byte-buddy-agent for this. Create a Java agent as you are used to it but add anAgent-Main entry where you would put your Pre-Main in the manifest. Also, name the entry method agentmain, not premain.
Using byte-buddy-agent, you can write a program:
class AgentLoader {
public static void main(String[] args) {
String processId = ...
File agentJar = ...
ByteBuddyAgent.attach(processId, agentJar);
}
}
and you are done.

How to execute scala code using java

I am new to scala. I have a requirement to execute the scala class using java.
My exact requirement is: I need to pass the entire scala class (file) as an argument to the java jar. That jar should read the scala code and execute it. I have searched many sites but did not find the appropriate answer. Is there any way to do the same?
Thank you in Advance.

Besides of your motivation to do that, it is for sure possible (I did it using my IDE - sbt project)
I just made scala class as below:
import com.google.common.base.Objects
class Car(_color: String, _valid: Boolean) {
val color: String = _color
val valid: Boolean = _valid
override def toString = Objects.toStringHelper(this).add("color",color).add("valid", valid).toString
}
After that I made class with main method to test it.
public class Test {
public static void main(String[] args) {
Car test = new Car("test", true);
System.out.println("test = " + test);
}
}
It compiled without any problems and the result was like below:
test = Car{color=test, valid=true}

Scala has its own compiler scalac whereas java uses javac. Since scalac compiles to class file that java can read and assuming that you are only using java libraries in the class then you can load the class in java. So what you need is to call scalac to compile the scala file and then load the generate class file using ClassLoader

Elegant way to repace the (Class-)Type of an Instance in Java bytecode

I've been working for some time on a solution for the following problem. Let's say I have following Java code:
public class A {
public void start() {
List l = new ArrayList();
l.add("abc");
System.out.println( l.get(0) );
}
}
The result of compiling this code is a class file A.class, containing Java byte code. I've written a ClassLoader, which loads A.class and executes start(). My intention is replacing in the loaded A.class (byte code) new ArrayList() with MyArrayList():
//Before
List l = new ArrayList();
//After
List l = new MyArrayList();
The class MyArrayList is defined similarly to: class MyArrayList extends ArrayList

You can use a tool like ASM to parse the file and to replace the occurrence of any ArrayList with MyArrayList. This would however require that MyArrayList defines constructors with an identical signature to the constructors defined in ArraysList.
The next question is when you define the changes. You have the following possibilities:
Change the implementation at build time: You could write a build tool plugin that applies your transformations.
Change the implementation at run time using a Java agent: The Attach API allows you to change a loaded class even at run time.
Change the implementation before a class is loaded for a first time: You could use a custom class loader for loading your altered class. This needs to happen before a class is loaded regularly!

Java 6 annotation processor with Filer.createClassFile() implementation

I am trying to write an annotation processor in Java 6. I wrote a sample implementation, which creates a new source file in the process method and it works fine.
#SupportedAnnotationTypes(value = {"*"})
#SupportedSourceVersion(SourceVersion.RELEASE_6)
public class BrownfieldAnnotationProcessor extends AbstractProcessor{
public boolean process(Set<? extends TypeElement> annotations,
RoundEnvironment roundEnv) {
try {
JavaFileObject f = processingEnv.getFiler().
createSourceFile("in.test.ExtraClass");
processingEnv.getMessager().printMessage(Diagnostic.Kind.NOTE,
"Creating " + f.toUri());
Writer w = f.openWriter();
try {
PrintWriter pw = new PrintWriter(w);
pw.println("package in.test;");
pw.println("public class ExtraClass implements TestInterface{");
pw.println(" public void print() {");
pw.println(" System.out.println(\"Hello boss!\");");
pw.println(" }");
pw.println("}");
pw.flush();
} finally {
w.close();
}
} catch (IOException x) {
processingEnv.getMessager().printMessage(Diagnostic.Kind.ERROR,
x.toString());
}
return true;
}
}
But in my case, i do not want another java file to be created and instead i want to generate a class file directly. How to create a class file? Should i use a dynamic compiler to compile this source in string to create the class? In that scenario, i can directly store the class files in the file system. What is the usage of
processingEnv.getFiler().createClassFile()
method?
I tried googling around, but could never find an example using this method.

I see only few reasons for creating class files instead of source code files. Actually this are the same reasons why you would ever need to create bytecode by hand in any situation.
You absolutly need control over the bytecode (maybe for 'optimization' or testing purposes)
You need to use features that are available in bytcode but not in the Java language
I think you rarely need to do any of the above. If this is really what you want to do, you should know a lot about the format of class files and bytecode of course. You would basically be creating some kind of compiler at this point.
You could argue that by shipping precompiled classes, or by generating bytcode directly, you can skip a step in the built process and the user's program will compiler faster. That may be true true but I'm sure the speed will be neglectable. It would be easier to just work with source code and pass it to the filer for compilation.
Should i use a dynamic compiler to compile this source in string to create the class?
No, I see no advantage here. If you create a source code file with the filer it will be compiled automatically. You only need to do this if your source code is not Java but some other language that you want to compile for the JVM.

instantiating a Scala class using reflection Java's `newInstance`

For some special use-case I have a small utility to load Java classes from jars using a dynamic class loader DynamicClassLoader. This works fine for Java classes contained in jars. Loading Scala classes from a jar also works without problems. However, instantiating the loaded Scala class leads to the following exception. It looks like the Scala class has private default constructor? Note the compiled Scala class name ending with $
java.lang.IllegalAccessException: Class XXX can not access a member of class ScalaClassYYY$ with modifiers "private"
The snippet below illustrates the idea of what I'm trying to achieve and gives a bit more context. The exception happens at the annotated line:
// deploy and register the new code
byte[] jarBytes = (byte[]) ((Object) message.getAttachment("jar"));
String registerClassName = message.getAttachment("register");
logger.debug("the register is '" + registerClassName + "'");
DynamicClassLoader loader = new DynamicClassLoader(jarBytes);
Class<?> registerClass = loader.lookUp(registerClassName);
// ===> this is where the java.lang.IllegalAccessException happens
IRegisterExecutor registerExecutor = (IRegisterExecutor) registerClass.newInstance();
registerExecutor.register();
Any ideas how to fix?

Obviously, you need to make the default constructor public (it won't work for Java classes without a public default constructor either). E.g.
class ScalaClassYYY() {
...
}
or if you want primary constructor to take some arguments,
class ScalaClassYYY(arg1: Int) {
def this() = this(0)
}
But from
Note the compiled Scala class name ending with $
it seems like you are actually trying to instantiate a Scala object:
object ScalaClassYYY { ... }
In this case, you shouldn't create a new instance and instead use the existing one:
(IRegisterExecutor) registerClass.getField("MODULE$").get(null);
EDIT:
I don't see in your answer how you add a default public constructor to a Scala class that does NOT require any parameters.
A class (not an object) that doesn't require any parameters has a default public constructor already (my first example).
Actually in Java all classes by default offer a public default constructor
No. Only those classes which have no constructors which take arguments.
remove the "(it won't work for Java classes without a public default constructor either)" because it is wrong
The documentation for Class.newInstance() says
IllegalAccessException - if the class or its nullary constructor is not accessible.
So I am pretty sure it's right. If it does work for Java classes without a public default constructor, this seems to be a major bug in the class loader you use. You can test it with a Java class which looks like this:
public class TestClass implements IRegisterExecutor {
public TestClass(int dummy) {}
// some implementation for IRegisterExecutor methods to get it to compile
}