Jacoco, as all other profiling capabilities in java I know of, depends on java agents/instrumentation.
Java instrumentation can only impact bytecode when a class is loaded.
I wonder if it is somehow possible to detect unused classes with Jacoco? Obviously, there might be classes that are "dead" and are in turn, never loaded by the ClassLoader.
When coverage output (HTML report, etc.) is generated after the test run has executed, JaCoCo scans additional classes in the runtime classpath that haven't been loaded during test execution, so they can also be instrumented and included in the output.
I implemented this same mechanism in my own code coverage tool (JMockit Coverage), which also relies on java.lang.instrument. It's the only way to have all relevant classes instrumented for coverage.
Yes, though it might be easier to specify -verbose:class if you are not already using Jacoco.
I just ran a test: I created a file called Delete.java that is never called by any other method. The generated Jacoco report shows the Delete class with 0% coverage.
Related
I have a Gradle project with the JaCoCo plugin applied to it. When I run my tests and create a jacocoTestReport I get this classes not matching error
[ant:jacocoReport] Classes in bundle 'e-services' do no match with execution data. For report generation the same class files must be used as at runtime.
[ant:jacocoReport] Execution data for class eservices/model/persistence/Event does not match.
The classes should match as I'm doing a clean -> build -> test locally. I suspect the mismatch comes from the fact that I'm using jackson.map.ObjectMapper to create an object from a JSON and somehow this causes the classId stored in jacoco's .exec file not match the compiled class id.
My test uses the Event class extensively and still I get 0% coverage due to class mismatch:
import eservices.model.persistence.Event;
event = mapper.readValue(json, Event.class);
event.setTenId(TenIds.getInternalId());
Is there a way to get coverage from this scenario?
This is a JaCoCo Known limitation as JaCoCo relies on the checksum of the runtime bytecodes matching the checksum of the bytecodes it uses for report generation. Typically it happens when you have two libraries instrumenting bytecodes, like PowerMock and JaCoCo, or JPA and JaCoCo.
This is referenced in the following JaCoCo issue Coverage is missing a class that was in fact tested #193 and it is labeled as 'wontfix' 'known limitation'
I'm trying to fiddle with Foursquare's HeapAudit, and am attempting to set it up using IntelliJ IDEA. I have managed to get it to build just fine, using the dependencies from the pom.xml.
However, when I actually try to run the JUnit tests, basically all of them fail. I'm guessing this is because using HeapAudit requires the JVM to be started with it as a -javaagent, according to the github:
$ java -javaagent:heapaudit.jar MyTest
Presumably the tests would pass if I put this line in, and referenced the heapaudit.jar i downloaded/built earlier. However, it seems to me that if I make changes the the source, I'm gonna need to re-package this silly .jar file in order to see if it works.
Is there any way of running the tests with a -javaagent without going through the whole rigmarole of compile -> package-into-jar every testing cycle? Perhaps getting IntelliJ to attached the newly-compiled .class files as a -javaagent before running the tests?
1) Have a jar just with a META-INF/MANIFEST.MF
The manifest must be properly configured with Premain-Class and other attributes. The jar doesn't need any other files. Use this jar with the -javaagent. Provided that the agent classes are in the classpath, the agent will start normally.
This might fail when using maven-surefire-plugin with forkMode=never because by default the application classes are loaded in a child ClassLoader.
Works fine with Eclipse and Intellij.
If doing this, double check the manifest syntax (once I spent a long time to figure out that a package name was wrong).
2) Use ea-agent-loader
It will allow you to load the agent (any agent) in runtime (it uses VM.attach()). However the VM.attach() sometimes disrupts debugging and breakpoints might fail to trigger.
It will have the same issues with the surefire in forkMode=never
3) Load the agent in runtime.
Write your on code to load the agent in runtime. And call it from your #BeforeClass You will still need a jar (which you can generate in runtime if you want).
Just you need to call this (only once):
AgentLoader.loadAgentClass(YourAgentClass.class.getName());
As I read from Art of unit test, knowing that .NET can hide seam methods for testing in production runtime. (p.78~p.80).
Such as,
public class LogAnalyzer
{
...
internal LogAnalyzer (IExtensionManager extentionMgr)
{
manager = extentionMgr;
}
}
run like this.
using System.Runtime.CompilerServices;
[assembly:
InternalsVisibleTo("AOUT.CH3.Logan.Tests")]
So LogAnalyzer() can only be called for test classes, without worries of adding extra cost on production code on purpose of testability.
After brief survey, seems Java does not have equivalent feature.
But does Java have alternatives?
Thanks.
What about implementing your own custom ClassLoader? You can define your own annotation like #HideFromProductionCode and have your custom ClassLoader throw an exception if it loads a class that has the #HideFromProductionCode annotation. See How to set my custom class loader to be the default?
Alternately, just add a script to your build process that goes through all your compiled production code and looks for the #HideFromProductionCode annotation.
One fairly straightforward approach would be to use a Maven-like directory structure, with separate directories for production code and test code (typically under directories called src/main and src/test). When unit tests are run, the classpath includes both the main directory and the test directory. But when you build the JAR that gets deployed in production, only classes defined in the main directory are included; this way, production code that references test classes will result in a compile error.
We have a jar that we lost the source code to. I decompiled the jar and created new source from it. I want to then verify that the source code and the old jar have the same behavior. I am writing unit tests to do the verification the problem is that they both have same namespace / class name so I do not know how to disambiguate the old jar and the new source code. What can I do or is it impossible?
You need to only have one version on the class path at once to guarantee that you are running that version of the code. Develop your unit test separate from the code so you can drop in either version.
Give the new source a temporary namespace for testing purposes. Then instead of import, you can refer your new classes as:
com.yourfirm.test.packagename.TheClassName
the old ones can be simply imported and refered to as TheClassName. This way you can tell by looking at your test cases which is which.
Or simply run the tests with -cp oldpackage.jar and then -cp newpackage.jar.
It's possible, but you have to mess around with class loading. Instead of putting either of the jars on the classpath, you'll need to load them at runtime. Check out JCL for a library to allow you to do this. (Disclaimer: I have never used JCL.)
Basically, each test would have to load the class from the old JAR, grab the results of the method you're testing, then unload that JAR, load up the new one, run the same method against the new version, and compare the results.
I'd change which classes are being tested at runtime with the classpath. This approach would be less error-prone in terms of ensuring that you're running the same test code against both binaries. Otherwise you introduce more complexity around whether the tests are correct.
It sounds like you are trying to execute the tests against both jars at the same time. I don't know of a way to disambiguate the old/new jars if they are both in the classpath.
If your unit tests output results to stdout/stderr, you could run the tests against the original jar and save the results. Then run the tests against the new jar and save the results in a separate file. Then diff the files.
Another approach would be to refactor the new source code so that it has a unique namespace. You could then test against both jars at the same time, but it could be a lot of work to make existing programs use the new jar.
If you run your tests via ant (Junit-task), you can control the ant classpath seperately for both runs (once via jar, once via fileset of classes).
I am looking to obfuscate our Java web app code within our existing Ant build script, but am running into problems around unit testing. I am obfuscating the code right after it has been compiled, before it is jar-ed and before the unit tests are ran.
However, if I obfuscate my production code and not my test code, all my tests fail because they are trying to call methods that no longer exist because they have been renamed by the obfuscator. I can mark certain methods to not obfuscate so they can be used by external systems such as our test suite, but since we are shooting for high unit test coverage we will need to mark all of our methods as un-obfuscatable.
If I obfuscate the test classes as well, I run into two problems:
1: The production classes and the test classes get merged into the same output directory and I am unable to exclude the test classes from the production .jar files
2: I cannot run my normal Ant batchtest call:
<batchtest todir="${basedir}/reports">
<fileset dir="${basedir}/components/common/build-zkm">
<include name="**/*Test.class"/>
</fileset>
</batchtest>
because the obfuscator has changed the names of the tests.
I could just run the obfuscator on the resulting .war/.ear files, but I want to have our unit tests run against the modified code to drive out any bugs caused by the obfuscator.
I am currently working with Zelix KlassMaster, but I am still in the evaluation phase so I would be open to other options if they would work better.
I use yguard (it is free, which is why I mention it).
You should be able to tell the obfuscator not to obfuscate certain things (looking here it seems you can).
Some as others have said, don't obfuscate the tests, but do obfuscate the rest.
However, I would suggest that you do the following:
compile
jar the un-obfuscated files (if desired)
test the un-obfuscated files
if they pass the tests then obfuscate jar the obfuscated files
test the obfuscated files
It will be slower, but if the tests fail in step 3 it'll be easier to fix (potentially) and if the tests fail at 5 then you know there is an issue with the obfuscation not your source code.
Can you tell it to run the obfuscator such that it effectively refactors the code including the references from the tests (i.e. when a production name changes, the test code changes its reference) but not to obfuscate the tests themselves (i.e. don't change the names of the test classes or their methods)? Given previous experience with obfuscators I'd expect that to work.
So for example, suppose we had unobfuscated source of:
public class ProductionCode
{
public void productionMethod() {}
}
public class ProductionCodeTest
{
public void testProductionMethod()
{
new ProductionCode().productionMethod();
}
}
You want to set the options of the obfuscator to make it effectively:
public class Xyzzy
{
public void ababa() {}
}
public class ProductionCodeTest
{
public void testProductionMethod()
{
new Xyzzy(). ababa();
}
}
That way your "run the tests" Ant tasks should be able to stay the same, because the API of the tests hasn't changed - merely the implementation of the methods.
The obfuscator should not change your public calls. It seems that yo should run the other tests before obfuscation because they check internal functionality that should not change after obfuscation.
So if that is the case, why not just run the tests that call public functionality? All you need to do is have a separate class with those calls and re-build it using the obfuscated code and then run that dll.