I'm trying to write a plugin to perform some custom Java checks.
The checks are pretty simple and all look similar to this:
#ActivatedByDefault
#SqaleConstantRemediation("1h")
#SqaleSubCharacteristic(SECURITY_FEATURES)
#Rule(name = "Methods annotated with #RequestMapping should also have #PreAuthorize annotation", priority = CRITICAL)
public class RequestMappingHasPreAuthorizeAnnotationCheck extends MethodVisitor {
#Override
public void visitMethod(MethodTree methodTree) {
if (hasRequestMappingButNotPreAuthorize(methodTree.modifiers())) {
reportIssue(methodTree,
"Methods annotated with #RequestMapping should also be annotated with #PreAuthorize");
}
}
private static boolean hasRequestMappingButNotPreAuthorize(ModifiersTree modifiers) {
return isAnnotatedWith(modifiers, REQUEST_MAPPING) && !isAnnotatedWith(modifiers, PRE_AUTHORIZE);
}
Relevant part of pom.xml:
<packaging>sonar-plugin</packaging>
<dependencies>
<dependency>
<groupId>org.sonarsource.java</groupId>
<artifactId>java-checks</artifactId>
<version>3.9</version>
</dependency>
<dependency>
<groupId>org.sonarsource.sonarqube</groupId>
<artifactId>sonar-plugin-api</artifactId>
<version>5.3</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.sonarsource.sonar-packaging-maven-plugin</groupId>
<artifactId>sonar-packaging-maven-plugin</artifactId>
<version>1.15</version>
<extensions>true</extensions>
<configuration>
<pluginClass>com.company.sonar.security.RestSecurityPlugin</pluginClass>
<pluginDescription>Verifies security annotations</pluginDescription>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.sonar</groupId>
<artifactId>sonar-dev-maven-plugin</artifactId>
<version>1.8</version>
</plugin>
</plugins>
</build>
When I build with
mvn clean package
it generates my snapshot jar in target, but displays this (along w/ similar messages for other libraries):
[WARNING] com.google.guava:guava:jar:10.0.1:compile is provided by SonarQube plugin API and will not be packaged in your plugin
When I try to deploy it to my local test sonar server using
mvn sonar-dev:upload -DsonarHome=~/sonarqube-5.3
I get the folowing (snipped) stack trace:
Java::JavaLang::NoClassDefFoundError (com/google/common/collect/Iterables):
org.sonar.squidbridge.annotations.AnnotationBasedRulesDefinition.addRuleClasses(AnnotationBasedRulesDefinition.java:90)
org.sonar.squidbridge.annotations.AnnotationBasedRulesDefinition.addRuleClasses(AnnotationBasedRulesDefinition.java:86)
org.sonar.squidbridge.annotations.AnnotationBasedRulesDefinition.load(AnnotationBasedRulesDefinition.java:75)
com.company.sonar.security.RestSecurityRulesDefinition.define(RestSecurityRulesDefinition.java:25)
org.sonar.server.rule.RuleDefinitionsLoader.load(RuleDefinitionsLoader.java:54)
org.sonar.server.rule.RegisterRules.start(RegisterRules.java:100)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:497)
org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110)
org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89)
org.sonar.core.platform.ComponentContainer$1.start(ComponentContainer.java:291)
org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84)
org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169)
org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132)
org.picocontainer.behaviors.Stored.start(Stored.java:110)
org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016)
org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009)
org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767)
org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:131)
org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:84)
org.sonar.server.platform.platformlevel.PlatformLevelStartup.access$001(PlatformLevelStartup.java:45)
org.sonar.server.platform.platformlevel.PlatformLevelStartup$1.doPrivileged(PlatformLevelStartup.java:82)
org.sonar.server.user.DoPrivileged.execute(DoPrivileged.java:45)
org.sonar.server.platform.platformlevel.PlatformLevelStartup.start(PlatformLevelStartup.java:78)
org.sonar.server.platform.Platform.executeStartupTasks(Platform.java:197)
org.sonar.server.platform.Platform.restart(Platform.java:141)
org.sonar.server.platform.Platform.restart(Platform.java:125)
org.sonar.server.platform.ws.RestartAction.handle(RestartAction.java:63)
org.sonar.server.ws.WebServiceEngine.execute(WebServiceEngine.java:85)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
(snip)
The RestSecurityRulesDefinition class looks like this:
public class RestSecurityRulesDefinition implements RulesDefinition {
#Override
public void define(Context context) {
NewRepository repository = context.createRepository("rest-security", "java").setName("rest-security");
#SuppressWarnings("rawtypes")
List<Class> ruleClasses = new ArrayList<>();
ruleClasses.add(PreAuthorizeInfoCheck.class);
ruleClasses.add(RequestMappingHasPartnerSecuredAnnotationCheck.class);
ruleClasses.add(RequestMappingHasPreAuthorizeAnnotationCheck.class);
ruleClasses.add(RequestMappingHasTimedAnnotationCheck.class);
ruleClasses.add(RestControllerWithExceptionHandlerAnnotationCheck.class);
// exception gets thrown on the following line
AnnotationBasedRulesDefinition.load(repository, "java", ruleClasses);
repository.done();
}
Server is freshly downloaded 5.3 with sonar.web.dev=true.
Any help on what I might be missing?
Related
spring boot version : 2.4.1
spring cloud version : 2020.0.0
My code
#Configuration
public class BaseConfig {
#Bean
public Module sortJacksonModule() {
return new SortJacksonModule();
}
}
my pom.xml dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-commons</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-openfeign-core</artifactId>
</dependency>
my pom.xml plugin
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
When run with IntelliJ IDEA, it work well.
But when run with jar(by mvn clean package), it show
Caused by: java.lang.NoClassDefFoundError: feign/codec/EncodeException
at org.springframework.cloud.openfeign.support.SortJacksonModule.setupModule(SortJacksonModule.java:47) ~[spring-cloud-openfeign-core-3.0.0.jar!/:3.0.0]
at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:819) ~[jackson-databind-2.11.3.jar!/:2.11.3]
at com.fasterxml.jackson.databind.ObjectMapper.registerModules(ObjectMapper.java:1021) ~[jackson-databind-2.11.3.jar!/:2.11.3]
at org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.configure(Jackson2ObjectMapperBuilder.java:712) ~[spring-web-5.3.2.jar!/:5.3.2]
at org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.build(Jackson2ObjectMapperBuilder.java:680) ~[spring-web-5.3.2.jar!/:5.3.2]
at org.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration$JacksonObjectMapperConfiguration.jacksonObjectMapper(JacksonAutoConfiguration.java:101) ~[spring-boot-autoconfigure-2.4.1.jar!/:2.4.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_232]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_232]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.3.2.jar!/:5.3.2]
... 113 common frames omitted
Caused by: java.lang.ClassNotFoundException: feign.codec.EncodeException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[na:1.8.0_232]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[na:1.8.0_232]
at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:151) ~[demo-spring-core-11-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[na:1.8.0_232]
... 124 common frames omitted
After study the error log, I found that feign.codec.EncodeException is optional dependency in spring-cloud-openfeign-core, so ClassNotFoundException is right behavior(optional dependency not include in final jar).
So my question is: Why IntelliJ IDEA can run without any error? I try both IntelliJ IDEA run and mvn spring-boot:run, both work fine.
update: add example
After more study, I found out this only happen when the class not called.
try {
System.out.println("not important code");
} catch (Exception e) {
throw new EncodeException("not exist class");
}
In this example, the try catch never throw an exception. And the EncodeException class is in an optional dependency.
This code run well in IntelliJ IDEA, but fail when run as java -jar xxx.jar
========== update again with minimal demo
I create a minimal demo to reproduce this issue.
a standalone demo-module
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-core</artifactId>
<version>10.10.1</version>
<optional>true</optional>
</dependency>
import feign.codec.EncodeException;
/**
* Hello world!
*/
public class App {
public void testOptional() {
try {
System.out.println("test");
} catch (Exception e) {
throw new EncodeException("never throw this");
}
}
}
demo spring project(create by spring initializr and add a dependency)
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>com.example</groupId>
<artifactId>demo-module</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
#Component
public class MyMain implements ApplicationRunner {
#Override
public void run(ApplicationArguments args) throws Exception {
new App().testOptional();
}
}
Inspect your project classpath in Idea ( CTRL-Alt-Shift-S ) - I daresay optional jar is somewhere on module compile classpath and it is enough to run your class in IDE - but not in standalone jar. Optional means in maven context that it is present on classpath while compiling, but not packed into resulting artifact.
There is an option called Enable launch optimization in IntelliJ IDEA run config, uncheck it and everything work as expected.
I am working on a spark program which uses Drools internally. The code works fine. Now, I am writing test cases to test the drools code. Because of this bug https://issues.redhat.com/browse/DROOLS-329, I had to add the dependency
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
<version>2.5.16</version>
</dependency>
then the JVM argument for jacoco testing
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables>
<jacoco-agent.destfile>${project.build.directory}/jacoco.exec</jacoco-agent.destfile>
</systemPropertyVariables>
<argLine>-Ddrools.dialect.java.compiler=JANINO</argLine>
</configuration>
</plugin>
After adding that Spark test cases started failing as they have dependency on
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
<version>3.0.8</version>
</dependency>
So, I commented out the Drools Tests and 2.5.16 janino dependency in POM. Now the Spark code is working fine. How can we make sure that both Janino 2.5.16 and Janino 3.0.8 coexist together during Maven test build that way both Drools tests as well as Spark tests pass?
Here is the Drools code that I am using
public static void applyRules(ResultObj obj) {
try {
KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();
kbuilder.add( ResourceFactory.newClassPathResource( "rules/rules.drl", DroolsPreprocessor.class),
ResourceType.DRL );
if ( kbuilder.hasErrors() ) {
System.err.println( kbuilder.getErrors().toString() );
}
StatelessKieSession session = kbuilder.newKieBase().newStatelessKieSession();
session.execute(obj);
System.out.println("Reached the end ");
} catch (Exception e) {
e.printStackTrace();
}
}
Here is the Drools dependency
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-decisiontables</artifactId>
<version>7.7.0.Final</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
<version>7.7.0.Final</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-compiler</artifactId>
<version>7.7.0.Final</version>
</dependency>
With Janino 2.5.16 Spark is not happy it throws this exception
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/janino/InternalCompilerException
at org.apache.spark.sql.catalyst.expressions.codegen.JavaCode$.variable(javaCode.scala:63)
at org.apache.spark.sql.catalyst.expressions.codegen.JavaCode$.isNullVariable(javaCode.scala:76)
at org.apache.spark.sql.catalyst.expressions.Expression.$anonfun$genCode$3(Expression.scala:145)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:141)
at org.apache.spark.sql.catalyst.expressions.codegen.CodegenContext.$anonfun$generateExpressions$1(CodeGenerator.scala:1170)
at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.map(TraversableLike.scala:238)
at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
at scala.collection.AbstractTraversable.map(Traversable.scala:108)
at org.apache.spark.sql.catalyst.expressions.codegen.CodegenContext.generateExpressions(CodeGenerator.scala:1170)
at org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection$.createCode(GenerateUnsafeProjection.scala:290)
at org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection$.create(GenerateUnsafeProjection.scala:338)
at org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection$.generate(GenerateUnsafeProjection.scala:327)
at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.createCodeGeneratedObject(Projection.scala:123)
at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.createCodeGeneratedObject(Projection.scala:119)
at org.apache.spark.sql.catalyst.expressions.CodeGeneratorWithInterpretedFallback.createObject(CodeGeneratorWithInterpretedFallback.scala:52)
at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:150)
at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:143)
at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:135)
at org.apache.spark.sql.kafka010.KafkaRecordToRowConverter.<init>(KafkaRecordToRowConverter.scala:36)
at org.apache.spark.sql.kafka010.KafkaRelation.<init>(KafkaRelation.scala:52)
at org.apache.spark.sql.kafka010.KafkaSourceProvider.createRelation(KafkaSourceProvider.scala:150)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:339)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:279)
at org.apache.spark.sql.DataFrameReader.$anonfun$load$2(DataFrameReader.scala:268)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:268)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:203)
If I don't add Janino 2.5.16 dependency I get the following errors for testing
java.lang.RuntimeException: wrong class format
at org.drools.compiler.commons.jci.compilers.EclipseJavaCompiler$2.findType(EclipseJavaCompiler.java:302)
at org.drools.compiler.commons.jci.compilers.EclipseJavaCompiler$2.findType(EclipseJavaCompiler.java:255)
at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(LookupEnvironment.java:174)
at org.eclipse.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage(PackageBinding.java:214)
at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.findImport(CompilationUnitScope.java:466)
at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.findSingleImport(CompilationUnitScope.java:520)
at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInImports(CompilationUnitScope.java:397)
at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInTypes(CompilationUnitScope.java:445)
at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:860)
at org.eclipse.jdt.internal.compiler.Compiler.processCompiledUnits(Compiler.java:550)
at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:462)
at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:417)
at org.drools.compiler.commons.jci.compilers.EclipseJavaCompiler.compile(EclipseJavaCompiler.java:436)
at org.drools.compiler.commons.jci.compilers.AbstractJavaCompiler.compile(AbstractJavaCompiler.java:49)
at org.drools.compiler.rule.builder.dialect.java.JavaDialect.compileAll(JavaDialect.java:420)
at org.drools.compiler.compiler.DialectCompiletimeRegistry.compileAll(DialectCompiletimeRegistry.java:61)
at org.drools.compiler.compiler.PackageRegistry.compileAll(PackageRegistry.java:109)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileAll(KnowledgeBuilderImpl.java:1471)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.wireAllRules(KnowledgeBuilderImpl.java:950)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.addPackage(KnowledgeBuilderImpl.java:938)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.addPackageFromDrl(KnowledgeBuilderImpl.java:543)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.addKnowledgeResource(KnowledgeBuilderImpl.java:743)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.add(KnowledgeBuilderImpl.java:2288)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.add(KnowledgeBuilderImpl.java:2277)
I am trying to test out some code that works with Java Doc, it is used under the maven-javadoc-plugin. I am trying to get it to work under jdk11. I am after an implementation of RootDoc which I can use when running tests.
Currently the tests use EasyDoclet which gives me a RootDoc like so:
EasyDoclet easyDoclet = new EasyDoclet(new File("dir"), "com.foo.bar");
RootDoc rootDoc = easyDoclet.getRootDoc()
However I could not get this to work under jdk11.
The first issue I had was tools.jar is missing so I changed my pom.xml to have:
<dependency>
<groupId>org.seamless</groupId>
<artifactId>seamless-javadoc</artifactId>
<version>1.1.1</version>
<exclusions>
<exclusion>
<groupId>com.sun</groupId>
<artifactId>tools</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- maybe this will get what ever was in tools.jar -->
<dependency>
<groupId>com.github.olivergondza</groupId>
<artifactId>maven-jdk-tools-wrapper</artifactId>
<version>0.1</version>
</dependency>
This lead to many instances of:
java.lang.NoClassDefFoundError: com/sun/tools/javadoc/PublicMessager
The PublicMessager class seems to exist to make public some constructors, I am not sure why it exists under the com.sun.tools package. I tried to make a copy of this class:
public static class PublicMessager extends
com.sun.tools.javadoc.main.Messager {
public PublicMessager(Context context, String s) {
super(context, s);
}
public PublicMessager(Context context, String s, PrintWriter printWriter, PrintWriter printWriter1, PrintWriter printWriter2) {
super(context, s, printWriter, printWriter1, printWriter2);
}
}
And the error message changes to:
java.lang.IllegalAccessError: superclass access check failed: class com.fun.javadoc.FooBar$PublicMessager (in unnamed module #0x4abdb505) cannot access class com.sun.tools.javadoc.main.Messager (in module jdk.javadoc) because module jdk.javadoc does not export com.sun.tools.javadoc.main to unnamed module #0x4abdb50
I exposed jdk.javadoc to the unnamed module using:
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Dfile.encoding=UTF-8</argLine>
<argLine>--add-opens=jdk.javadoc/com.sun.tools.javadoc.main=ALL-UNNAMED</argLine>
</configuration>
</plugin>
</plugins>
</build>
This meant that my custom version of PublicMessager would no longer have the errors shown however the version from seamless under com.sun.tools could not be found. I made my own version of EasyDoclet which used my PublicMessager however it turned out that the following two classes are missing:
import com.sun.tools.javadoc.JavadocTool;
import com.sun.tools.javadoc.ModifierFilter;
At this point I am not sure what to do. halp!
Perhaps an alternative would be to instead find the jdk11 equivalent of RootDoc which I think is DocletEnvironment and then some how get an implementation of that, I have no idea how to get an implementation of DocletEnvironment.
I would like to introduce Byte Buddy to my company and I have prepared a demo for my colleagues. Since we use Spring a lot, I thought the best example would be instrumentation of SpringBoot application. I have decided to add logs to RestController methods.
Instrumented application is a simple SpringBoot Hello World example:
#RestController
public class HelloController {
private static final String template = "Hello, %s!";
#RequestMapping("/hello")
public String greeting(
#RequestParam(value = "name", defaultValue = "World") String name) {
return String.format(template, name);
}
#RequestMapping("/browser")
public String showUserAgent(HttpServletRequest request) {
return request.getHeader("user-agent");
}
}
And here is my Byte Buddy agent:
public class LoggingAgent {
public static void premain(String agentArguments,
Instrumentation instrumentation) {
install(instrumentation);
}
public static void agentmain(String agentArguments,
Instrumentation instrumentation) {
install(instrumentation);
}
private static void install(Instrumentation instrumentation) {
createAgent(RestController.class, "greeting")
.installOn(instrumentation);
}
private static AgentBuilder createAgent(
Class<? extends Annotation> annotationType, String methodName) {
return new AgentBuilder.Default().type(
ElementMatchers.isAnnotatedWith(annotationType)).transform(
new AgentBuilder.Transformer() {
#Override
public DynamicType.Builder<?> transform(
DynamicType.Builder<?> builder,
TypeDescription typeDescription,
ClassLoader classLoader) {
return builder
.method(ElementMatchers.named(methodName))
.intercept(
MethodDelegation
.to(LoggingInterceptor.class)
.andThen(
SuperMethodCall.INSTANCE));
}
});
}
}
Interceptor logs method execution:
public static void intercept(#AllArguments Object[] allArguments,
#Origin Method method) {
Logger logger = LoggerFactory.getLogger(method.getDeclaringClass());
logger.info("Method {} of class {} called", method.getName(), method
.getDeclaringClass().getSimpleName());
for (Object argument : allArguments) {
logger.info("Method {}, parameter type {}, value={}",
method.getName(), argument.getClass().getSimpleName(),
argument.toString());
}
}
When executed with -javaagent parameter this example works well. When however I try to load the agent on the running JVM with Attach API:
VirtualMachine vm = VirtualMachine.attach(args[0]);
vm.loadAgent(args[1]);
vm.detach();
I've got the following exception on the first logging attempt:
Exception in thread "ContainerBackgroundProcessor[StandardEngine[Tomcat]]" java.lang.IncompatibleClassChangeError: Class ch.qos.logback.classic.spi.ThrowableProxy does not implement the requested interface ch.qos.logback.classic.spi.IThrowableProxy
at ch.qos.logback.classic.pattern.ThrowableProxyConverter.subjoinExceptionMessage(ThrowableProxyConverter.java:180)
at ch.qos.logback.classic.pattern.ThrowableProxyConverter.subjoinFirstLine(ThrowableProxyConverter.java:176)
at ch.qos.logback.classic.pattern.ThrowableProxyConverter.recursiveAppend(ThrowableProxyConverter.java:159)
at ch.qos.logback.classic.pattern.ThrowableProxyConverter.throwableProxyToString(ThrowableProxyConverter.java:151)
at org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter.throwableProxyToString(ExtendedWhitespaceThrowableProxyConverter.java:35)
at ch.qos.logback.classic.pattern.ThrowableProxyConverter.convert(ThrowableProxyConverter.java:145)
at ch.qos.logback.classic.pattern.ThrowableProxyConverter.convert(ThrowableProxyConverter.java:1)
at ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:36)
at ch.qos.logback.core.pattern.PatternLayoutBase.writeLoopOnConverters(PatternLayoutBase.java:114)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:141)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:1)
at ch.qos.logback.core.encoder.LayoutWrappingEncoder.doEncode(LayoutWrappingEncoder.java:130)
at ch.qos.logback.core.OutputStreamAppender.writeOut(OutputStreamAppender.java:187)
at ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:212)
at ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:100)
at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:84)
at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)
at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:270)
at ch.qos.logback.classic.Logger.callAppenders(Logger.java:257)
at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421)
at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383)
at ch.qos.logback.classic.Logger.log(Logger.java:765)
at org.slf4j.bridge.SLF4JBridgeHandler.callLocationAwareLogger(SLF4JBridgeHandler.java:221)
at org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:303)
at java.util.logging.Logger.log(Unknown Source)
at java.util.logging.Logger.doLog(Unknown Source)
at java.util.logging.Logger.logp(Unknown Source)
at org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:181)
at org.apache.juli.logging.DirectJDKLog.error(DirectJDKLog.java:147)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1352)
at java.lang.Thread.run(Unknown Source)
I run the example on 64-bit HotSpot with Java8:
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)
Byte Buddy version is 1.4.32. Here is agent maven configuration:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>pl.halun.demo.bytebuddy</groupId>
<artifactId>byte-buddy-agent-demo</artifactId>
<version>1.0</version>
<properties>
<jdk.version>1.8</jdk.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.4.1.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>net.bytebuddy</groupId>
<artifactId>byte-buddy</artifactId>
<version>1.4.32</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<finalName>${project.artifactId}-${project.version}-full</finalName>
<appendAssemblyId>false</appendAssemblyId>
<archive>
<manifestEntries>
<Premain-Class>pl.halun.demo.bytebuddy.logging.LoggingAgent</Premain-Class>
<Agent-Class>pl.halun.demo.bytebuddy.logging.LoggingAgent</Agent-Class>
<Can-Redefine-Classes>true</Can-Redefine-Classes>
<Can-Retransform-Classes>true</Can-Retransform-Classes>
<Can-Set-Native-Method-Prefix>true</Can-Set-Native-Method-Prefix>
</manifestEntries>
</archive>
</configuration>
<executions>
<execution>
<id>assemble-all</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
And here is pom file for the instrumented application:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>pl.halun.demo.bytebuddy.instrumented.app</groupId>
<artifactId>byte-buddy-agent-demo-instrumented-app</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.4.1.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
<properties>
<java.version>1.8</java.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</pluginRepository>
</pluginRepositories>
From my point of view it is very valuable option to add logs on the running server and I hate to loose this part of demo. I tried to experiment with different redefinition strategies, but until now nothing seems to work.
What you observe is a classical version conflict, I think. Spring Boot most likely comes with a version of ThrowableProxy that is not compatible to the version that is added with the Java agent. When loading a Java agent at runtime, Spring's version is already loaded whereas the startup attachment prepends the agent-bundled version on the class path where the version of your agent is loaded.
Java agents are typically added to the class path. This is also where your Spring boot application is living. You need to make sure that a Java agent does not contain dependencies that are incompatible with your application's dependencies or you need to shade all dependencies to avoid such conflicts.
There is however another problem: When writing a Java agent that is attached at runtime, you meet additional constraints on most JVMs where on HotSpot, you are not allowed to change the class file format of any class that is already loaded. There is also a chance that your class is already loaded where currently, no effect would be visible as you do not enable retransformation.
A runtime-capable agent would need to use the Advice component which inlines code into target code rather then using the classical delegation model:
class MyAdvice {
#Advice.OnMethodEnter
static void intercept(#Advice.BoxedArguments Object[] allArguments,
#Advice.Origin Method method) {
Logger logger = LoggerFactory.getLogger(method.getDeclaringClass());
logger.info("Method {} of class {} called", method.getName(), method
.getDeclaringClass().getSimpleName());
for (Object argument : allArguments) {
logger.info("Method {}, parameter type {}, value={}",
method.getName(), argument.getClass().getSimpleName(),
argument.toString());
}
}
}
You can use the above advice class by registering it as a visitor. Such visitors only apply to declared methods, i.e. not to inherited methods and inline their code into existing methods. This way, the logging will not be visible on the call stack and it also becomes legal to retransform already loaded classes:
new AgentBuilder.Default()
.disableClassFormatChanges()
.with(AgentBuilder.RedefinitionStrategy.RETRANSFORMATION)
.type(isAnnotatedWith(annotationType))
.transform(new AgentBuilder.Transformer() {
#Override
public DynamicType.Builder<?> transform(
DynamicType.Builder<?> builder,
TypeDescription typeDescription,
ClassLoader classLoader) {
return builder.visit(Advice.to(MyAdvice.class).on(named(methodName)));
}
});
As for the attachment, look into the byte-buddy-agent project which allows you to call:
ByteBuddyAgent.attach(agentJar, processId);
The above helper supports other VMs where the attachment API often lives in a different namespace.
Update: Here is the problem with Spring Boot. Spring Boot creates custom class loaders that have the system class loader (class path) as their parent. These class loaders consider classes from the system class loader first. When you add the agent, the entire Spring boot app is both on the class loader and in these child class loaders. A class like IThrowableProxy now exists twice in two class loaders but is not considered to be equal by the JVM. Depending on the state of the VM, some classes might already be linked to the original IThrowableProxy whereas other classes are loaded after the agent was attached and get linked to the new IThrowableProxy from the agent. Both classes are not equal and the error that you see is thrown where the VM complains that the class does not implement the correct IThrowableProxy (but the previous one). If the agent is attached at start up, this problem does not exist as the class path's IThrowableProxy is always loaded.
This is not an easy error to fix, in the end, Byte Buddy cannot help you with such class path issues and Spring Boot is quite free in its interpretation of the class loader contract. The easiest way would be to not use Spring Boot types in your agent. You can still match the annotation with for example
isAnnotatedWith(named("org.springframework.web.bind.annotation.RestController"))
The question is how you can communicate with Spring Boot. One work-arround would be to add all shared classes to the class path on start up. Typically, I do avoid the usage of shared classes altogether but only use them in the Advice classes where the code is inlined in the class loader of the target application. Simply set the Spring Boot dependency in provided scope, the advice code itself is never executed.
I have found a strange interaction between cobertura-maven-plugin 2.6 and jmockit 1.8. A particular pattern in our production code has a class with a lot of static methods that effectively wraps a different class that acts like a singleton. Writing unit tests for these classes went fine until I tried to run coverage reports with cobertura, when this error cropped up:
java.lang.ExceptionInInitializerError
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.NullPointerException
at com.example.foo.MySingleton.<clinit>(MySingleton.java:7)
... 13 more
This then leads to a NoClassDefFoundError and being unable to initialize the singleton class. Here's a full SSCCE (the shortest I can get it down to) that replicates the error; line 7 of MySingleton is Logger.getLogger().
Here's the "singleton"...
package com.example.foo;
import org.apache.log4j.Logger;
public class MySingleton {
private static final Logger LOG = Logger.getLogger(MySingleton.class);
private boolean inited = false;
private Double d;
MySingleton() {
}
public boolean isInited() {
return inited;
}
public void start() {
inited = true;
}
public double getD() {
return d;
}
}
And the static class...
package com.example.foo;
import org.apache.log4j.Logger;
public class MyStatic {
private static final Logger LOGGER = Logger.getLogger(MyStatic.class);
private static MySingleton u = new MySingleton();
public static double getD() {
if (u.isInited()) {
return u.getD();
}
return 0.0;
}
}
And the test that breaks everything...
package com.example.foo;
import mockit.Expectations;
import mockit.Mocked;
import mockit.Tested;
import org.junit.Test;
public class MyStaticTest {
#Tested MyStatic myStatic;
#Mocked MySingleton single;
#Test
public void testThatBombs() {
new Expectations() {{
single.isInited(); result = true;
single.getD(); /*result = 1.2;*/
}};
// Deencapsulation.invoke(MyStatic.class, "getD");
MyStatic.getD();
}
}
And the maven pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example.foo</groupId>
<artifactId>test</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Test</name>
<dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
<dependency>
<groupId>org.jmockit</groupId>
<artifactId>jmockit</artifactId>
<version>1.8</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.6</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
To summarize the summary: When running an ordinary unit test (mvn clean test), the above tests just fine; when run with cobertura (mvn clean cobertura:cobertura) it throws the nasty set of exceptions shown at the top. Obviously a bug somewhere, but whose?
The cause for this problem isn't so much a bug, but a lack of robustness in JMockit when mocking a class that contains a static initializer. The next version of JMockit (1.9) will be improved on this point (I already have a working solution).
Also, the problem would not have occurred if Cobertura marked its generated methods (four of them with names starting with "__cobertura_", added to every instrumented class) as "synthetic", so that JMockit would have ignored them when mocking a Cobertura-instrumented class. Anyway, fortunately this won't be necessary.
For now, there are two easy work-arounds which avoid the problem:
Make sure any class to be mocked was already initialized by the JVM by the time the test starts. This can be done by instantiating it or invoking a static method on it.
Declare the mock field or mock parameter as #Mocked(stubOutClassInitialization = true).
Both work-arounds prevent the NPE that would otherwise get thrown from inside the static class initializer, which is modified by Cobertura (to see these bytecode modifications, you can use the javap tool of the JDK, on classes under the target/generated-classes directory).