We Keep Coding

How to inspect java method references (double colon) operator usages in classes during build time

Is there a way to detect usages of java method reference (double colon) operator inside the code?
I need to discover all instance/static method references used in a given class in order to be able to detect some errors (must verify that the target method has a particular annotation - #Good in the below example) during build time. As by convention a method reference should be used only to some of the methods when it is passed to a constructor of some helper class (Info in the below example).
class X {
Info init() {
return new Info(X::beta); // good code: target method has #Good annotation
return new Info(X::alpha); // bad code: target method has no #Good annotation
}
void alpha() {
}
#Good
void beta() {
}
}
The intention is to be able to click on the method reference as this makes it easy to follow as otherwise if just passing Method instance or just method name it would lack this ability.
(The example is not very good but I'm now allowed to share more details, sorry about that!)
I can see IntelliJ IDEA "knows" about them - when you ctrl+click on them it navigates to the target method so there should be some form of a static analysis used there.
I'm already using ObjectWeb ASM to detect invocations to certain methods but it seems it lacks the ability to detect method references (::)
EDIT:
Just a note that you can also pass new Info(x -> x.alpha()) as #Thomas below mentioned in the comments but this would not pass our review process but I guess the additional ability to detect it would not hurt.
EDIT2: What exactly are you trying to achieve with these checks? What makes beta worthy of receiving the annotation?
Answer:
When the init() method is called we obtain the Info instance and from it obtain the lambda which must be a method reference. Then we use javassist ProxyFactory and create a sub-class of class X then instantiate it and intercept all its methods via setting a method handler. So now it is safe to execute the lambda without allowing it to make any side effects - the method body is skipped and the only thing we do is to capture which is the X method that the lambda actually is calling - in the example this will lead to a java.lang.Method instance pointing to X.beta or X.alpha method. Then we can check if it has the #Good annotation and proceed accordingly - which is to call the lambda without any proxying, but that call might happen later, like a millisecond later or an hour later. If there is no #Good annotation we cannot proceed - it is a bug.
So the problem is that this will happen at runtime later and there might be a bug not caught early enough and that is the reason I would like to inspect the X class at build time and catch all the bugs :)

This is a bit of a shot in the dark, as I'm neither very proficient with ASM nor sure if this approach addresses your problem. Having said that, I found that, in a similar setting, asm.MethodVisitor calls MethodVisitor.visitInvokeDynamicInsn(...) for (some? all?) method references.
E.g., if I compile this variant of your class X along with an Info:
class Info {
public Info(Runnable alpha) {}
}
class X {
Info init() { return new Info(this::alpha); }
void alpha() {}
}
... and I then feed the resulting X.class into a mini ClassVisitor + printing MethodVisitor (Groovy for brevity):
class MyMethodVisitor extends MethodVisitor {
MyMethodVisitor(MethodVisitor parent) { super(Opcodes.ASM8, parent) }
#Override
void visitInvokeDynamicInsn(String name, String descriptor, Handle bootstrapMethodHandle, Object... bootstrapMethodArguments) {
println "visitInvokeDynamicInsn($name, $descriptor, $bootstrapMethodHandle, $bootstrapMethodArguments)"
super.visitInvokeDynamicInsn(name, descriptor, bootstrapMethodHandle, bootstrapMethodArguments)
}
}
class MyClassVisitor extends ClassVisitor {
MyClassVisitor() { super(Opcodes.ASM8) }
#Override
MethodVisitor visitMethod(int access, String name, String descriptor, String signature, String[] exceptions) {
println "Starting method '$name'"
new MyMethodVisitor(super.visitMethod(access, name, descriptor, signature, exceptions))
}
}
def clr = new ClassReader(new File("./X.class").bytes)
clr.accept(new MyClassVisitor(), ClassReader.SKIP_FRAMES)
Then the method visitor prints, amongst other details, a call to visitInvokeDynamicInsn from within the method visitation of X::init with the desired X::alpha among the arguments (the xyz being my local package):
Visiting method '<init>'
Visiting method 'init'
visitInvokeDynamicInsn(run, (xyz/X;)Ljava/lang/Runnable;,
java/lang/invoke/LambdaMetafactory.metafactory(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite; (6),
[()V, xyz/X.alpha()V (5), ()V])
Visiting method 'alpha'
So it would seem possible to peel the method out of those arguments. I am not sure if this reliable (e.g., whether this bytecode is guaranteed by specification, or whether it can depend on compilation/optimization details).

Mocking the arguments passed to callbacks (lambdas)

How would I mock methods that accept a lambda using Mockito so that I am able to control which arguments are passed into the callback? I am specifically trying to mock the JDBI method useExtension which is used like this:
jdbi.useExtension(OrgUnitDao.class, dao -> {
// Skip if already loaded
// Skip if already loaded
if (dao.orgUnitsAreLoaded()) {
I would like to substitute the dao object passed back into the callback so that I could control the branching using the return value of dao.orgUnitsAreLoaded().
The signature looks like this
public <E,X extends Exception> void useExtension(Class<E> extensionType,
ExtensionConsumer<E,X> callback)
throws NoSuchExtensionException,
X extends Exception

This is the full answer to my question. It's simplified down to the very basics of how to do the stubbing and so doesn't reflect the production code I am to test, but it shows exactly the mechanics needed to do it.
final Jdbi jdbi = mock(Jdbi.class);
doAnswer(invocation -> {
System.out.println("this is the doAnswer lambda - just setting up the answer and the mocks");
final Class<OrgUnitDao> daoClass = invocation.getArgument(0);
final ExtensionConsumer callback = invocation.getArgument(1);
final OrgUnitDao mock1 = mock(daoClass);
when(mock1.orgUnitsAreLoaded()).thenReturn(false);
// call the actual callback method
callback.useExtension(mock1);
return null;
}).when(jdbi).useExtension(eq(OrgUnitDao.class), any());
// This is the method call I am to test
// Regard this as hidden away in some outer method in
// the System-Under-Test, but that I have been able
// to inject all its dependencies
jdbi.useExtension(OrgUnitDao.class, new Foo());
/// Further down, outside of the method
// Only replaced the lambda with this to get toString() for debugging ...
class Foo implements ExtensionConsumer<OrgUnitDao, RuntimeException> {
#Override
public void useExtension(OrgUnitDao orgUnitDao) throws RuntimeException {
System.out.println("A real method call, now using the passed in mocked dao:" + orgUnitDao.orgUnitsAreLoaded());
}
#Override
public String toString() {
return "OrgUnitDao class";
}
}

To parallel the conversation on the question "Calling callbacks with Mockito", your lambda might be called synchronously during the execution of your method-under-test, or it might be called later based on some external factor or interaction. Like Dawood's answer there, your answer here using a Mockito Answer will work, and is the only way to go if you are looking for the synchronous style (where mockJdbi calls your lambda immediately before methodUnderTest returns). If your lambdas are going to be called asynchronously, or if your system tolerates you calling the lambda asynchronously, you might want to test the state after your method-under-test returns but before you interact with the lambda.
// MockitoJUnitRunner, MockitoRule, or MockitoAnnotations.initMocks populate these.
// Especially useful for the ArgumentCaptor's generic arguments.
#Mock Jdbi mockJdbi;
#Mock OrgUnitDao mockOrgUnitDao;
#Captor ArgumentCaptor<ExtensionConsumer<OrgUnitDao, RuntimeException>>
extensionConsumerCaptor;
#Test public void yourTest() throws Exception {
// No stubbing needed! Just create the system under test.
YourSystemUnderTest systemUnderTest = new YourSystemUnderTest(mockJdbi);
// Call the method under test, which presumably calls useExtension(...).
systemUnderTest.methodUnderTest();
// Assert anything that should be true before the lambda is called.
assertFalse(systemUnderTest.getSomeState());
// Confirm that useExtension was called, and simultaneously acquire the lambda.
// ArgumentCaptor.capture() is a matcher, so every argument requires a matcher like eq.
verify(mockJdbi).useExtension(eq(OrgUnitDao.class), extensionConsumerCaptor.capture());
// Prepare the mock DAO and call the lambda.
when(mockDao.getFoo()).thenReturn("bar");
extensionConsumerCaptor.getValue().useExtension(mockDao);
// Assert anything that should be true after the lambda is called.
assertTrue(systemUnderTest.getSomeState());
}
Though lambdas reduce the boilerplate previously associated with anonymous inner classes, you may also prefer using the Captor style because it saves you from creating lengthy Answer implementations and hiding your test assertions or Mockito verifications in them. This is especially tempting if your project prefers BDD-style mocks with clear "given-when-then" structure (though my example more-closely resembles "given-when-then-when-then").

Can we mock a spy object using mockito?

I have a class called RadiationControl and I created a spy for it in the following way.
RadiationControl radCtrl = new RadiationControl();
RadiationControl spyRadCtrl = Mockito.spy(radCtrl);
I have a chained method call inside a different class called StationMonitor which is being called by using the RadiationControl object. When I am trying to use the above created spy and trying to access that which has method parameters and they vary from time to time.
StationMonitorObject stationMonitorObject = radCtrl.getStationMonitorLoader().retrieveCVStationMonitorObject(Long.parseLong(syngId), status);
Thus with the above syntax when I try to stub the spy for that method call it's complaining to stub properly.
StationMonitorLoader stationMonitorLoader = StationMonitorLoader.getLoader(domain);
Mockito.doReturn(stationMonitorLoader).when(spyRadCtrl).getStationMonitorLoader();
Mockito.doReturn(stationMonitorObject).when(stationMonitorLoader).retrieveCVStationMonitorObject(any(Long.class), null);
Is there any better approach to deal such scenario ?

Is there any better approach to deal such scenario ?
Yes.
The problem here is:
radCtrl.getStationMonitorLoader()
.retrieveCVStationMonitorObject(Long.parseLong(syngId), status);
This is a violation of the law of demeter (aka don't talk to strangers!).
The method retrieveCVStationMonitorObject() should be available in class RadiationControl and delegate the call to its dependency (which looks like being a StationMonitorLoader...)

Create Java8 function reference programmatically

Just a theoretic question, I do not have practical use-case currently.
Assuming some my API accepts function reference as an argument and I would like to both feed it directly from code via '::' syntax or collect matching functions via reflection, store in some Map and invoke conditionally.
It is possible to programmatically convert method into Consumer<String>?
Map<String, Consumer<String>> consumers = new HashMap<>();
consumers.put("println", System.out::println);
Method method = PrintStream.class.getMethod("println", String.class);
consumers.put("println", makeFunctionReference(method));
...
myapi.feedInto(consumers.get(someInput.getConsumerId()));
Update:
Though not satisfied by solutions in currently provided answers, but after getting the hint about LambdaMetaFactory I tried to compile this code
public class TestImpl {
public static void FnForString(String arg) {}
}
public class Test {
void test() {
List<String> strings = new ArrayList<>();
Consumer<String> stringConsumer = TestImpl::FnForString;
strings.stream().forEach(stringConsumer);
strings.stream().forEach(TestImpl::FnForString);
stringConsumer.accept("test");
}
}
and after feeding only Test class into CFR decompiler I'm getting following back:
public class Test {
void test() {
ArrayList strings = new ArrayList();
Consumer<String> stringConsumer =
(Consumer<String>)LambdaMetafactory.metafactory(
null, null, null,
(Ljava/lang/Object;)V,
FnForString(java.lang.String),
(Ljava/lang/String;)V)();
strings.stream().forEach(stringConsumer);
strings.stream().forEach(
(Consumer<String>)LambdaMetafactory.metafactory(
null, null, null,
(Ljava/lang/Object;)V,
FnForString(java.lang.String ),
(Ljava/lang/String;)V)());
stringConsumer.accept("test");
}
}
Out of that I see that:
This is somehow possible to do in '1-liner' manner
No exception handling is required
I have no idea what is (Ljava/lang/Object;)V (and others) in decompiler's output. It should match to MethodType in metafactory() arguments. Additionally - either decompiler 'eats/hides' something, but there seems to be now invocations of methods during getting of function reference.
(offtop) Obtaining function reference even in compiled code is at least one function call - in general this may be not unnoticeably cheap operation in performance critical code.
And ... with both Test and TestImpl classes provided, CFR reconstructs absolutely same code that I've compiled.

You could do this with reflection like this:
consumers.put("println", s -> {
try {
method.invoke(System.out, s);
} catch (InvocationTargetException | IllegalAccessException e) {
throw new RuntimeException(e);
}
});
But it you want your code to compile to the same code using a method-reference (i.e. using invokedynamic instructions), you can use a MethodHandle. This does not have the overhead of reflection and so it will perform a lot better.
MethodHandles.Lookup lookup = MethodHandles.lookup();
MethodType methodType = MethodType.methodType(void.class, String.class);
MethodHandle handle = lookup.findVirtual(PrintStream.class, "println", methodType);
consumers.put("println", s -> {
try {
handle.invokeExact(System.out, s);
} catch (Throwable e) {
throw new RuntimeException(e);
}
});
consumers.get("println").accept("foo");
In this code, first a MethodHandles.Lookup object is retrieved. This class is reponsible for creating MethodHandle objects. Then a MethodType object, which represents the arguments and return type accepted and returned by a method handle is created: in this case, it is a method that returns void (hence void.class) and takes a String (hence String.class). Finally, the handle is obtained by finding the println method on the PrintStream class.
You can refer to this question (and this one) for more information about what MethodHandles are.

The simplest, albeit not necessarily most performant, approach would be just wrapping the Method into a Consumer.
final Method m = ...
final T target = ...
Consumer<String> c = (arg1) => m.invoke(t, arg1);
Using the LambdaMetaFactory may yield more optimal code, but considering that you're dispatching through a Map it's probably not worth it.
This is somehow possible to do in '1-liner' manner
If you really want to emulate what the bytecode does that's only true for for sufficiently tortured definitions of one-liner. Your decompiler lies to you to some extent.
No exception handling is required
That is because the concept of checked exceptions does not exist on the bytecode level. This can be emulated with static helper methods that do a sneaky rethrow for you.
I have no idea what is (Ljava/lang/Object;)V (and others) in decompiler's output. It should match to MethodType in metafactory() arguments. Additionally - either decompiler 'eats/hides' something, but there seems to be now invocations of methods during getting of function reference.
It looks like pseudocode for invokedynamic calls. What the JVM really does is more complicated and can't be expressed concisely in java since it involves lazy initialization. It's best to read the java.lang.invoke package description to get an idea what really happens.
The java-level equivalent to the linking stage would be putting the CalleSite's dynamicInvoker MH into a static final MethodHandle field and calling its invokeExact method.
(offtop) Obtaining function reference even in compiled code is at least one function call - in general this may be not unnoticeably cheap operation in performance critical code.
as mentioned above, the linking stage is equivalent to putting the methodhandle in a static field once and then calling that in the future instead of attempting to resolve the method a second time.

The decompiler failed badly on your code, however, there is no correct decompiling anyway, besides recreating the original Java 8 method reference, which is not what you were interested in.
The lambda expressions and method references are compiled using an invokedynamic byte code instruction which has no equivalent in the Java programming language. The equivalent code would be something like:
public static void main(String... arg) {
Consumer<String> consumer=getConsumer();
consumer.accept("hello world");
}
static Consumer<String> getConsumer() {
try {
MethodHandles.Lookup lookup=MethodHandles.lookup();
MethodType consumeString = MethodType.methodType(void.class, String.class);
return (Consumer<String>)LambdaMetafactory.metafactory(lookup, "accept",
MethodType.methodType(Consumer.class, PrintStream.class),
consumeString.changeParameterType(0, Object.class),
lookup.findVirtual(PrintStream.class, "println", consumeString), consumeString)
.getTarget().invokeExact(System.out);
}
catch(RuntimeException | Error e) { throw e; }
catch(Throwable t) { throw new BootstrapMethodError(t); }
}
except, that everything that is done within getConsumer() is originally handled by a single invokedynamic instruction which will treat all involved MethodHandles and MethodType instances like constants and whose result of the first-time evaluation gets an intrinsic caching facility. You can’t model that using ordinary Java source code.
Still, the Consumer<String> returned by the getConsumer() method above is the exact equivalent of the expression System.out::println (when assigned to a Consumer<String>) bearing the same behavior and performance characteristics.
You may study “Translation of Lambda Expressions” by Brian Goetz for getting a deeper understanding of how it works. Also, the API documentation of LambdaMetafactory is quite exhaustive.

Design Patterns, override a method without need to re compile / relink

We are building a product that needs to run on production environments. We need to modify some of the functionality of a existing library. The existing library has class's and methods, we need to override 1 or more methods so that the caller uses our overriden methods instead of the original library.
OriginalLibrary
package com.original.library ;
public class OriginalLibrary {
public int getValue() {
return 1 ;
}
public int getAnotherValue() {
return 1 ;
}
}
Original Client
public class MyClient {
private OriginalLibraryClass originalLibraryObject ;
public MyClient () {
originalLibraryObject = new OriginalLibraryClass() ;
System.out.println(originalLibraryObject.getValue()) ;
System.out.println(originalLibraryObject.getAnotherValue()) ;
}
}
Output
1
2
Now, I need to change getValue() to return 3, instead of 1
Needed Output
3
2
package com.original.library.improved ;
public class OriginalLibrary extends com.original.library.OriginalLibrary {
public int getValue() {
return 3 ;
}
public int getAnotherValue() {
return super.getAnotherValue() ;
}
}
If I do the above, I need to tell my Original Client to reorder and use my new com.original.library.improved jar file before com.original.library.
I am almost convinced that this is the most non intrusive way to launch my improved services over and above the OriginalLibrary. I would have preferred a solution where I need to tell the customer to just add my jar file, no need to recompile, relink your client code.
Similar (not same) questions on a google search
here
here

java assist is excellent library for bytecode manipulation. I have modified code below as per your sample code given, You have to explore javaassist more for your actual requirenment
CtClass etype = ClassPool.getDefault().get("com.original.library.OriginalLibrary");
// get method from class
CtMethod cm = etype.getDeclaredMethod("getValue");
// change the method bosy
cm.setBody("return 3;");
etype.rebuildClassFile();
// give the path where classes is placed, In my eclipse it is bin
etype.writeFile("bin");
OriginalLibrary originalLibraryObject;
originalLibraryObject = new OriginalLibrary();
System.out.println(originalLibraryObject.getValue());
System.out.println(originalLibraryObject.getAnotherValue());
Now output of getValue is 3 because I changed body of that method.

A couple of questions -
How is the client getting an instance of your library's class?
If they are using new OriginalLibrary(), then you're pretty much stuck with creating a new subclass of OriginalLibrary and then asking your client to use your new OriginalLibraryImproved class. This is a common problem encountered in projects and is one reason why a library should not allow its clients to instantiate its classes directly using the new operator.
If instead, your client is instantiating OriginalLibrary using a factory method provided by the library (say, OriginalLibrary.getInstance()), you may want to check if there are any hooks into the factory that allow you to change the object being returned.
Do you have full control of the source code of the original library?
If yes, then you definitely should (and I cannot emphasize this strongly enough) provide factory methods for any class in the library that is instantiable. Doing this allows you to change the actual object being returned without modifying the client (as long as the returned object's class is a subclass of the return value from the factory method).
If not, then I suggest you do the following.
Create a subclass of OriginalLibrary (say, OriginalLibraryImproved).
Create a Factory class named OriginalLibraryFactory that has a static method named getInstance(). Write code to return an instance of OriginalLibraryImproved from this method.
Ask your client to replace all occurrences of new OriginalLibrary() with OriginalLibraryFactory.getInstance(). Note that this approach will only involve adding an extra import for the factory class. The client will still refer to the returned instance using the same OriginalLibrary reference as before.
The advantage of this approach is that it gives you complete flexibility to change the implementation details of OriginalLibraryImproved without affecting the client in anyway. You could also swap OriginalLibararyImproved with a newer version like OriginalLibraryImprovedVer2 and the client will be oblivious to the fact that it is using a new class. You'll just have to make sure that OriginalLibraryImprovedVer2 subclasses OriginalLibrary.
An even more flexible approach is to use the Wrapper or Decorator pattern to avoid the pitfalls of inheritance. You can understand more about the Decorator pattern here.
In a nutshell, try to avoid forcing your clients to use new and try to avoid inheritance unless you have very compelling reasons.