Just a theoretic question, I do not have practical use-case currently.
Assuming some my API accepts function reference as an argument and I would like to both feed it directly from code via '::' syntax or collect matching functions via reflection, store in some Map and invoke conditionally.
It is possible to programmatically convert method into Consumer<String>?
Map<String, Consumer<String>> consumers = new HashMap<>();
consumers.put("println", System.out::println);
Method method = PrintStream.class.getMethod("println", String.class);
consumers.put("println", makeFunctionReference(method));
...
myapi.feedInto(consumers.get(someInput.getConsumerId()));
Update:
Though not satisfied by solutions in currently provided answers, but after getting the hint about LambdaMetaFactory I tried to compile this code
public class TestImpl {
public static void FnForString(String arg) {}
}
public class Test {
void test() {
List<String> strings = new ArrayList<>();
Consumer<String> stringConsumer = TestImpl::FnForString;
strings.stream().forEach(stringConsumer);
strings.stream().forEach(TestImpl::FnForString);
stringConsumer.accept("test");
}
}
and after feeding only Test class into CFR decompiler I'm getting following back:
public class Test {
void test() {
ArrayList strings = new ArrayList();
Consumer<String> stringConsumer =
(Consumer<String>)LambdaMetafactory.metafactory(
null, null, null,
(Ljava/lang/Object;)V,
FnForString(java.lang.String),
(Ljava/lang/String;)V)();
strings.stream().forEach(stringConsumer);
strings.stream().forEach(
(Consumer<String>)LambdaMetafactory.metafactory(
null, null, null,
(Ljava/lang/Object;)V,
FnForString(java.lang.String ),
(Ljava/lang/String;)V)());
stringConsumer.accept("test");
}
}
Out of that I see that:
This is somehow possible to do in '1-liner' manner
No exception handling is required
I have no idea what is (Ljava/lang/Object;)V (and others) in decompiler's output. It should match to MethodType in metafactory() arguments. Additionally - either decompiler 'eats/hides' something, but there seems to be now invocations of methods during getting of function reference.
(offtop) Obtaining function reference even in compiled code is at least one function call - in general this may be not unnoticeably cheap operation in performance critical code.
And ... with both Test and TestImpl classes provided, CFR reconstructs absolutely same code that I've compiled.
You could do this with reflection like this:
consumers.put("println", s -> {
try {
method.invoke(System.out, s);
} catch (InvocationTargetException | IllegalAccessException e) {
throw new RuntimeException(e);
}
});
But it you want your code to compile to the same code using a method-reference (i.e. using invokedynamic instructions), you can use a MethodHandle. This does not have the overhead of reflection and so it will perform a lot better.
MethodHandles.Lookup lookup = MethodHandles.lookup();
MethodType methodType = MethodType.methodType(void.class, String.class);
MethodHandle handle = lookup.findVirtual(PrintStream.class, "println", methodType);
consumers.put("println", s -> {
try {
handle.invokeExact(System.out, s);
} catch (Throwable e) {
throw new RuntimeException(e);
}
});
consumers.get("println").accept("foo");
In this code, first a MethodHandles.Lookup object is retrieved. This class is reponsible for creating MethodHandle objects. Then a MethodType object, which represents the arguments and return type accepted and returned by a method handle is created: in this case, it is a method that returns void (hence void.class) and takes a String (hence String.class). Finally, the handle is obtained by finding the println method on the PrintStream class.
You can refer to this question (and this one) for more information about what MethodHandles are.
The simplest, albeit not necessarily most performant, approach would be just wrapping the Method into a Consumer.
final Method m = ...
final T target = ...
Consumer<String> c = (arg1) => m.invoke(t, arg1);
Using the LambdaMetaFactory may yield more optimal code, but considering that you're dispatching through a Map it's probably not worth it.
This is somehow possible to do in '1-liner' manner
If you really want to emulate what the bytecode does that's only true for for sufficiently tortured definitions of one-liner. Your decompiler lies to you to some extent.
No exception handling is required
That is because the concept of checked exceptions does not exist on the bytecode level. This can be emulated with static helper methods that do a sneaky rethrow for you.
I have no idea what is (Ljava/lang/Object;)V (and others) in decompiler's output. It should match to MethodType in metafactory() arguments. Additionally - either decompiler 'eats/hides' something, but there seems to be now invocations of methods during getting of function reference.
It looks like pseudocode for invokedynamic calls. What the JVM really does is more complicated and can't be expressed concisely in java since it involves lazy initialization. It's best to read the java.lang.invoke package description to get an idea what really happens.
The java-level equivalent to the linking stage would be putting the CalleSite's dynamicInvoker MH into a static final MethodHandle field and calling its invokeExact method.
(offtop) Obtaining function reference even in compiled code is at least one function call - in general this may be not unnoticeably cheap operation in performance critical code.
as mentioned above, the linking stage is equivalent to putting the methodhandle in a static field once and then calling that in the future instead of attempting to resolve the method a second time.
The decompiler failed badly on your code, however, there is no correct decompiling anyway, besides recreating the original Java 8 method reference, which is not what you were interested in.
The lambda expressions and method references are compiled using an invokedynamic byte code instruction which has no equivalent in the Java programming language. The equivalent code would be something like:
public static void main(String... arg) {
Consumer<String> consumer=getConsumer();
consumer.accept("hello world");
}
static Consumer<String> getConsumer() {
try {
MethodHandles.Lookup lookup=MethodHandles.lookup();
MethodType consumeString = MethodType.methodType(void.class, String.class);
return (Consumer<String>)LambdaMetafactory.metafactory(lookup, "accept",
MethodType.methodType(Consumer.class, PrintStream.class),
consumeString.changeParameterType(0, Object.class),
lookup.findVirtual(PrintStream.class, "println", consumeString), consumeString)
.getTarget().invokeExact(System.out);
}
catch(RuntimeException | Error e) { throw e; }
catch(Throwable t) { throw new BootstrapMethodError(t); }
}
except, that everything that is done within getConsumer() is originally handled by a single invokedynamic instruction which will treat all involved MethodHandles and MethodType instances like constants and whose result of the first-time evaluation gets an intrinsic caching facility. You can’t model that using ordinary Java source code.
Still, the Consumer<String> returned by the getConsumer() method above is the exact equivalent of the expression System.out::println (when assigned to a Consumer<String>) bearing the same behavior and performance characteristics.
You may study “Translation of Lambda Expressions” by Brian Goetz for getting a deeper understanding of how it works. Also, the API documentation of LambdaMetafactory is quite exhaustive.
Related
I would like to generate follwing method with bytebuddy stackmanipulation.
public void test() {
this.process(() -> {
System.out.println("Im lambda call");
}
}
if such code would be compiled with javac it would produce:
private method in the "this" type called lambda$test$0 without any arguments
within the method test there would be invoke dynamic instruction
invoke dynamic instruction in the test method to invoke the lambda
If i had just ASM i could do something like this
methodVisitor.visitVarInsn(ALOAD, 0);
methodVisitor.visitInvokeDynamicInsn("run", "(LmyTest/Test;)Ljava/lang/Runnable;", new Handle(Opcodes.H_INVOKESTATIC, "java/lang/invoke/LambdaMetafactory", "metafactory", "(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;", false), new Object[]{Type.getType("()V"), new Handle(Opcodes.H_INVOKESPECIAL, "myTest/Test", "lambda$test$0", "()V", false), Type.getType("()V")});
methodVisitor.visitMethodInsn(INVOKEVIRTUAL, process ....)
How would i do this with just bytebuddy? Im using bytebuddy as its abstraction over opcodes is just much more pleasant to write than raw asm.
Let`s assume that the content of Runnable (method call of lambda$test$0) has already been generated
MethodDescription.InDefinedShape targetCall = thisType.getDeclaredMethods().filter(named("lambda$test$0")).getOnly();
InvokeDynamic methodRef = InvokeDynamic.lambda(targetCall, new TypeDescription.ForLoadedType(Runnable.class)).withoutArguments();
StackManipulation.Size size = new StackManipulation.Compound(
stackManipulations...
methodRef... ??
).apply(mv, ctx);
This is where i dont know what to do. How can i get StackManipulation from InvokeDynamic?
To create a stack manipulation for a dynamic method invocation, instead of InvokeDynamic (a more high level Implementation), use
MethodInvocation.invoke(targetCall).dynamic("run",
Collections.singletonList(thisType),
TypeDescription.ForLoadedType.of(Runnable.class),
Collections.emptyList());
You can chain this StackManipulation with the rest of your code.
In this case, ASM might however not be the worst option, it is a rather good byte code emitter and Byte Buddy does therefore not attempt to reimplement it. If it is the least amount of code, you can wrap it in a stack manipulation and implement the method using it, too, if this is an option you consider.
Is there a way to detect usages of java method reference (double colon) operator inside the code?
I need to discover all instance/static method references used in a given class in order to be able to detect some errors (must verify that the target method has a particular annotation - #Good in the below example) during build time. As by convention a method reference should be used only to some of the methods when it is passed to a constructor of some helper class (Info in the below example).
class X {
Info init() {
return new Info(X::beta); // good code: target method has #Good annotation
return new Info(X::alpha); // bad code: target method has no #Good annotation
}
void alpha() {
}
#Good
void beta() {
}
}
The intention is to be able to click on the method reference as this makes it easy to follow as otherwise if just passing Method instance or just method name it would lack this ability.
(The example is not very good but I'm now allowed to share more details, sorry about that!)
I can see IntelliJ IDEA "knows" about them - when you ctrl+click on them it navigates to the target method so there should be some form of a static analysis used there.
I'm already using ObjectWeb ASM to detect invocations to certain methods but it seems it lacks the ability to detect method references (::)
EDIT:
Just a note that you can also pass new Info(x -> x.alpha()) as #Thomas below mentioned in the comments but this would not pass our review process but I guess the additional ability to detect it would not hurt.
EDIT2: What exactly are you trying to achieve with these checks? What makes beta worthy of receiving the annotation?
Answer:
When the init() method is called we obtain the Info instance and from it obtain the lambda which must be a method reference. Then we use javassist ProxyFactory and create a sub-class of class X then instantiate it and intercept all its methods via setting a method handler. So now it is safe to execute the lambda without allowing it to make any side effects - the method body is skipped and the only thing we do is to capture which is the X method that the lambda actually is calling - in the example this will lead to a java.lang.Method instance pointing to X.beta or X.alpha method. Then we can check if it has the #Good annotation and proceed accordingly - which is to call the lambda without any proxying, but that call might happen later, like a millisecond later or an hour later. If there is no #Good annotation we cannot proceed - it is a bug.
So the problem is that this will happen at runtime later and there might be a bug not caught early enough and that is the reason I would like to inspect the X class at build time and catch all the bugs :)
This is a bit of a shot in the dark, as I'm neither very proficient with ASM nor sure if this approach addresses your problem. Having said that, I found that, in a similar setting, asm.MethodVisitor calls MethodVisitor.visitInvokeDynamicInsn(...) for (some? all?) method references.
E.g., if I compile this variant of your class X along with an Info:
class Info {
public Info(Runnable alpha) {}
}
class X {
Info init() { return new Info(this::alpha); }
void alpha() {}
}
... and I then feed the resulting X.class into a mini ClassVisitor + printing MethodVisitor (Groovy for brevity):
class MyMethodVisitor extends MethodVisitor {
MyMethodVisitor(MethodVisitor parent) { super(Opcodes.ASM8, parent) }
#Override
void visitInvokeDynamicInsn(String name, String descriptor, Handle bootstrapMethodHandle, Object... bootstrapMethodArguments) {
println "visitInvokeDynamicInsn($name, $descriptor, $bootstrapMethodHandle, $bootstrapMethodArguments)"
super.visitInvokeDynamicInsn(name, descriptor, bootstrapMethodHandle, bootstrapMethodArguments)
}
}
class MyClassVisitor extends ClassVisitor {
MyClassVisitor() { super(Opcodes.ASM8) }
#Override
MethodVisitor visitMethod(int access, String name, String descriptor, String signature, String[] exceptions) {
println "Starting method '$name'"
new MyMethodVisitor(super.visitMethod(access, name, descriptor, signature, exceptions))
}
}
def clr = new ClassReader(new File("./X.class").bytes)
clr.accept(new MyClassVisitor(), ClassReader.SKIP_FRAMES)
Then the method visitor prints, amongst other details, a call to visitInvokeDynamicInsn from within the method visitation of X::init with the desired X::alpha among the arguments (the xyz being my local package):
Visiting method '<init>'
Visiting method 'init'
visitInvokeDynamicInsn(run, (xyz/X;)Ljava/lang/Runnable;,
java/lang/invoke/LambdaMetafactory.metafactory(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite; (6),
[()V, xyz/X.alpha()V (5), ()V])
Visiting method 'alpha'
So it would seem possible to peel the method out of those arguments. I am not sure if this reliable (e.g., whether this bytecode is guaranteed by specification, or whether it can depend on compilation/optimization details).
How would I mock methods that accept a lambda using Mockito so that I am able to control which arguments are passed into the callback? I am specifically trying to mock the JDBI method useExtension which is used like this:
jdbi.useExtension(OrgUnitDao.class, dao -> {
// Skip if already loaded
// Skip if already loaded
if (dao.orgUnitsAreLoaded()) {
I would like to substitute the dao object passed back into the callback so that I could control the branching using the return value of dao.orgUnitsAreLoaded().
The signature looks like this
public <E,X extends Exception> void useExtension(Class<E> extensionType,
ExtensionConsumer<E,X> callback)
throws NoSuchExtensionException,
X extends Exception
This is the full answer to my question. It's simplified down to the very basics of how to do the stubbing and so doesn't reflect the production code I am to test, but it shows exactly the mechanics needed to do it.
final Jdbi jdbi = mock(Jdbi.class);
doAnswer(invocation -> {
System.out.println("this is the doAnswer lambda - just setting up the answer and the mocks");
final Class<OrgUnitDao> daoClass = invocation.getArgument(0);
final ExtensionConsumer callback = invocation.getArgument(1);
final OrgUnitDao mock1 = mock(daoClass);
when(mock1.orgUnitsAreLoaded()).thenReturn(false);
// call the actual callback method
callback.useExtension(mock1);
return null;
}).when(jdbi).useExtension(eq(OrgUnitDao.class), any());
// This is the method call I am to test
// Regard this as hidden away in some outer method in
// the System-Under-Test, but that I have been able
// to inject all its dependencies
jdbi.useExtension(OrgUnitDao.class, new Foo());
/// Further down, outside of the method
// Only replaced the lambda with this to get toString() for debugging ...
class Foo implements ExtensionConsumer<OrgUnitDao, RuntimeException> {
#Override
public void useExtension(OrgUnitDao orgUnitDao) throws RuntimeException {
System.out.println("A real method call, now using the passed in mocked dao:" + orgUnitDao.orgUnitsAreLoaded());
}
#Override
public String toString() {
return "OrgUnitDao class";
}
}
To parallel the conversation on the question "Calling callbacks with Mockito", your lambda might be called synchronously during the execution of your method-under-test, or it might be called later based on some external factor or interaction. Like Dawood's answer there, your answer here using a Mockito Answer will work, and is the only way to go if you are looking for the synchronous style (where mockJdbi calls your lambda immediately before methodUnderTest returns). If your lambdas are going to be called asynchronously, or if your system tolerates you calling the lambda asynchronously, you might want to test the state after your method-under-test returns but before you interact with the lambda.
// MockitoJUnitRunner, MockitoRule, or MockitoAnnotations.initMocks populate these.
// Especially useful for the ArgumentCaptor's generic arguments.
#Mock Jdbi mockJdbi;
#Mock OrgUnitDao mockOrgUnitDao;
#Captor ArgumentCaptor<ExtensionConsumer<OrgUnitDao, RuntimeException>>
extensionConsumerCaptor;
#Test public void yourTest() throws Exception {
// No stubbing needed! Just create the system under test.
YourSystemUnderTest systemUnderTest = new YourSystemUnderTest(mockJdbi);
// Call the method under test, which presumably calls useExtension(...).
systemUnderTest.methodUnderTest();
// Assert anything that should be true before the lambda is called.
assertFalse(systemUnderTest.getSomeState());
// Confirm that useExtension was called, and simultaneously acquire the lambda.
// ArgumentCaptor.capture() is a matcher, so every argument requires a matcher like eq.
verify(mockJdbi).useExtension(eq(OrgUnitDao.class), extensionConsumerCaptor.capture());
// Prepare the mock DAO and call the lambda.
when(mockDao.getFoo()).thenReturn("bar");
extensionConsumerCaptor.getValue().useExtension(mockDao);
// Assert anything that should be true after the lambda is called.
assertTrue(systemUnderTest.getSomeState());
}
Though lambdas reduce the boilerplate previously associated with anonymous inner classes, you may also prefer using the Captor style because it saves you from creating lengthy Answer implementations and hiding your test assertions or Mockito verifications in them. This is especially tempting if your project prefers BDD-style mocks with clear "given-when-then" structure (though my example more-closely resembles "given-when-then-when-then").
I am attempting to verify that a method with the following signature was called:
public void process(Map<String, Set<String>> data) {
...
}
The nested parameterized Set is causing me difficulties. I can get it to verify correctly with the any() matcher like so:
verify(dataProcessor).process(Matchers.<Map<String, Set<String>>> any());
As described in Mockito: Verifying with generic parameters although annoyingly it doesn't work if I do a direct static import of Matchers.any and call it as just:
verify(dataProcessor).process(<Map<String, Set<String>>> any())
But anyMapOf(clazz, clazz) seems the more appropriate matcher in this case. Since you can't do Set.class I'm not sure how you would do this. The following doesn't work because of the lack of generic:
verify(dataProcessor).process(anyMapOf(String.class, Set.class));
Is it possible to verify this situation with anyMapOf or should I stick with Matchers.<>any()?
There's no way to use anyMapOf to do this. It's designed to help with the simple case of mapping simple classes to simple classes in Java 7, and yours is more complex than that.
Java 8 parameter inference improved, so in Java 8, you can just use any().
verify(dataProcessor).process(Matchers.any());
Barring that, the best way to make this look is either like you wrote above:
verify(dataProcessor).process(Matchers.<Map<String, Set<String>>>any());
Or by extracting the matcher to a static function, which gives Java just enough information it needs to infer the type on its own:
#Test public void yourTest() {
// ...
verify(dataProcessor).process(anyStringSetMap());
}
private static Map<String, Set<String>> anyStringSetMap() {
return any();
}
(Caveat: Note that the return value of anyStringSetMap() is null; it's the side-effect of calling any that you're looking for. The extracted method is just to inform the Java compiler of the expected return type; beware that doing anything fancier will probably break in really-quite-interesting ways.)
I'm having problems with an MBean that takes a Map<String, Object> as a parameter. If I try to execute it via JMX using a proxy object, I get an Exception:
Caused by: javax.management.ReflectionException
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:231)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:668)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
Caused by: java.lang.IllegalArgumentException: Unable to find operation updateProperties(java.util.HashMap)
It appears that it attempts to use the actual implementation class rather than the interface, and doesn't check if this is a child of the required interface. The same thing happens for extended classes (for example declare HashMap, pass in LinkedHashMap). Does this mean it's impossible to use an interface for such methods? At the moment I'm getting around it by changing the method signature to accept a HashMap, but it seems odd that I wouldn't be able to use interfaces (or extended classes) in my MBeans.
Edit: The proxy object is being created by an in-house utility class called JmxInvocationHandler. The (hopefully) relevant parts of it are as follows:
public class JmxInvocationHandler implements InvocationHandler
{
...
public static <T> T createMBean(final Class<T> iface, SFSTestProperties properties, String mbean, int shHostID)
{
T newProxyInstance = (T) Proxy.newProxyInstance(iface.getClassLoader(), new Class[] { iface }, (InvocationHandler) new JmxInvocationHandler(properties, mbean, shHostID));
return newProxyInstance;
}
...
private JmxInvocationHandler(SFSTestProperties properties, String mbean, int shHostID)
{
this.mbeanName = mbean + MBEAN_SUFFIX + shHostID;
msConfig = new MsConfiguration(properties.getHost(0), properties.getMSAdminPort(), properties.getMSUser(), properties.getMSPassword());
}
...
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable
{
if (management == null)
{
management = ManagementClientStore.getInstance().getManagementClient(msConfig.getHost(),
msConfig.getAdminPort(), msConfig.getUser(), msConfig.getPassword(), false);
}
final Object result = management.methodCall(mbeanName, method.getName(), args == null? new Object[] {} : args);
return result;
}
}
Got it. JMX invocations sometimes make cannon-fodder of the best intended utility classes .... :)
This guy, I suspect, is a problem:
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable
{
if (management == null)
{
management = ManagementClientStore.getInstance().getManagementClient(msConfig.getHost(),
msConfig.getAdminPort(), msConfig.getUser(), msConfig.getPassword(), false);
}
final Object result = management.methodCall(mbeanName, method.getName(), args == null? new Object[] {} : args);
return result;
}
because the MBean's operation signature (which cares not a whit about inheritance) is determined from the classes of the passed arguments. Since you cannot pass an actual concrete object for which getClass() will return java.util.Map, you will never make a match using the direct types of the arguments themselves. (Similar problems occur with primitives for the same reason).
See this blog post starting with the paragraph opening with "One of the tricky parts of making MetaMBean", as it explains this problem (or the problem I think you're having) in a bit more detail, but the invoke method of the MBeanServer[Connection] is:
invoke(ObjectName name, String operationName, Object[] params, String[] signature)
The first 2 and the last arguments are navigational in that they specify exactly which operation amongst all the ops published in the server should be invoked. The best way to sidestep this issue is to avoid having to "guess" the signature and only rely on the ObjectName and the operation name, which in turn can be done by interrogating (and possibly caching) the MBeanInfo and MBeanOperationInfos of the target MBean. The MBeanOperationInfos will provide you the signature so you don't have to guess.
If this is indeed your issue, there's a couple of ways you can address it:
If the MBean's operation names are unique (i.e. there's no overloading) then you can just use the op name to retrieve the MBeanInfo.
If the MBean's operation is overloaded (i.e. there are multiple operations with the same name but different parameters)... but they all have different parameter counts, then you can easilly determine the correct signature by iterating all the matching op names in the MBeanOperationInfos and matching by param count.
If #1 and #2 do not apply.... then it's tricky and I would re-evaluate the method signatures of your MBean's code.
If #1 and #2 do not apply and #3 will not comply, take a look at this class in Gmx called MetaMBean. In the latest revision, it uses Groovy to create a compiled runtime interface using the MBean's MBeanInfo to make inheritance (and autoboxing) work in method invocation. The same method could be implemented in JavaScript (which has the virtue of being built into Java 6+) or several other JVM scripting languages. Alternatively, look at the previous version which attempted to pattern match against known operation signatures (and worked pretty well actually, but since I was working with Groovy anyways......)
I hope this is helpful. If this turns out not to be the root cause, then forget I said anything....