clover: how does it work? - java

I am evaluating clover currently and wonder how to use it best. First I'd like to understand how it works conceptually.
1) What does instrumentation mean? Are the test-calls attached to implementation's statements?
2) How is this done? Are the tests actually executed with some fancy execution context (similar to JRebel e.g.) for this? Or is it more like static analysis ?
3) After a "clover-run", some DB is saved to disk, and based on this, reports are generated right? Is the DB-Format accessible? I mean Can I launch my own analysis on it, e.g. using my own reporting tools ? What information does the DB contain exactly? Can I see the mapping between test and implementation there ?
4) Are there other tools that find the mapping between test and implementation? Not just the numbers, but which test, actually covers a line of code ...
Thanks, Bastl.

How is this done? Are the tests actually executed with some fancy execution context (similar to JRebel e.g.) for this? Or is it more like static analysis?
During code instrumentation by Clover it detects which methods are test methods (by default it recognizes JUnit3/4 and TestNG). Such methods gets additional instrumentation instructions. Shortly speaking, entering a test method will usually instantiate a dedicated coverage recorder which measures coverage exclusively for this test. More information about per test recording strategies available in Clover:
https://confluence.atlassian.com/display/CLOVER/Clover+Performance+Results
https://confluence.atlassian.com/display/CLOVER/About+Distributed+Per-Test+Coverage
After a "clover-run", some DB is saved to disk, and based on this, reports are generated right?
A Clover database (clover.db) contains information about code structure (packages, files, classes, methods, statements, branches), it has also information about test methods. There are also separate coverage recording files (produced at runtime) containing information about number of "hits" of given code element. Clover supports both global coverage (i.e. for the whole run) as well as per-test coverage (i.e. coverage from a single test).
More information is here:
https://confluence.atlassian.com/display/CLOVER/Managing+the+Coverage+Database
Is the DB-Format accessible?
The API is still in development (https://jira.atlassian.com/browse/CLOV-1375), but there is a possibility to get basic information. See:
https://confluence.atlassian.com/display/CLOVER/Database+Structure
for more details about DB model and code samples.
But the question is: do you really need to manually read this DB? You wrote that:
Can I see the mapping between test and implementation there ?
Such mapping is already provided by Clover - in the HTML report for example if you click on a source line it will pop up a list of test methods hitting this line.
PS: I'm a Clover developer at Atlassian, feel free to contact me if you have any questions.

What does instrumentation mean?
Additional code is woven in with your code.
Are the test-calls attached to implementation's statements?
I am not sure what you mean but it could be instructions or call to methods. Trivial methods will be inlined by the JIT at runtime.
How is this done?
There are many ways to do it, but often the Instrumentation class is to used to capture when a class is being loaded and a library like Objectweb's ASM is used to manipulate the code.
Are the tests actually executed with some fancy execution context
The context counts which lines have been executed.
Or is it more like static analysis ?
No, it is based on what is called.
After a "clover-run", some DB is saved to disk, and based on this, reports are generated right? Is the DB-Format accessible?
You had best ask the producers of clover as to the content of their files.
Are there other tools that find the mapping between test and implementation? Not just the numbers, but which test, actually covers a line of code ...
There are many code coverage tools available including EMMA, JaCoCo, Cobertura, IDEA has one builtin.

Related

Is it possible to debug runtime generated groovy code?

I'm working on a project where we need to compile groovy classes at runtime and then instantiate objects from these classes and execute methods on it. The source for these classes exists as a string only in live environments.
These classes can contain pretty complex code, so there is a good chance of bugs hiding in there.
The problem with our approach is, that when we notice missbehavior in these classes, we can't use a comfortable method of debugging.
We can of course write and execute tests against these classes, but often times you just want to know what's going on step-by-step.
So my question is: Is there is a way to debug runtime generated groovy/Java classes?
The steps we currently take to track down bugs:
1) Write tests to reproduce the behavior
2) Read through the code. (obviously has a super bad success rate in complex classes)
3) Do instrumentation. We call a static "_break" method which we have written in a utils class (so no runtime generated stuff). In that _break method we can add a breakpoint. So this is almost as if we were debugging the runtime generated classes directly. The Problem with that approach is, that you have to recompile and add a new version of the groovy classes to the testsystem everytime you want to add/remove a _break call.
If you're wondering how we can even write tests for these classes, here is how:
For unit tests we copy the code from the strings into regular groovy classes. These are used for development and unit testing, because it gives us code completion and a fast way to at least execute the classes against simple tests.
We can debug the code on unit test level. The problem here is, that the data setup is too complex to reproduce certain combinations in unit tests.
For integration tests, we do the whole compilation, adding, executing process, just like it would happen in the live system.
We use Intellij 2017 as IDE and I currently have no idea if or how we could "connect" the bytecode to either the strings which is was generated from, or the copied groovy classes we use for unit testing.
Any other tool that would allow us to debug would be fine as well.

How can I capture or enforce REST API coverage from Java unit tests?

Having embraced code coverage as a concept and driven our code in general to have 70%+ line coverage, management are now interested in REST API coverage.
We're currently experimenting with defining our JAX-RS REST resources using RAML and using code generation to build the JAX-RS models and interfaces. Clearly RAML can define the set of resources that should be available by which methods, indicates which MIME types can be accepted and produced, and which status codes are acceptable responses to any given invocation.
Are there any tools or test frameworks that I could use to track a set of unit tests and ensure that all applicable resources have been exercised, all status codes have been generated etc.?
Not sure whether it's better to boil this down to a number and track against a threshold / over time; or whether to just log unit test failure when the coverage is incomplete. Happy to consider either type of solution for now!
I don't know of a specific tool that would do that. If none is available, I suggest just measuring code coverage of the generated code. It would probably be easy to find untested resources by looking for resource-implementing classes with lower than average coverage (even if there is uncovered code in a well-tested generated class).
I doubt that you could achieve "status code coverage" for every resource by this approach, since any reasonable API implementation will implement some error statuses in a generic way in one place, but then again maybe you don't really need that if you're covering all of the generated code.

How to check for dead java methods at runtime

I am trying to create an index of unused Java methods in the form of a json file.
There are a couple different ways in which the methods can be referenced. I have already checked for all the other ways and have a relatively small list of possibly unused java methods.
The final way in which a method can be used is in other java files. They would be called with a basic class.method(args,args2,etc...) syntax somewhere in the java source code.
My question is, is there an easy way to just check my list of possible unused methods to see if any of them are not used in the java code. It would be ideal if this could be done at runtime, but it would also work if I could create a file that I could then read in at runtime.
I have tried using pre-built software like UCDetector, but the source code is huge, and running UCDetector takes hours and often doesn't even finish. It also checks all methods to see if they are used which is a waste of time since I have narrowed it down to a small number of possible methods to check.
You should use your IDE (eclipse, intelliJ), or some static code analysis tool such as findbugs, pmd, checkstyle.
It seems like you are trying to reinvent the wheel.
One option might be to use "coverage analysis" tools to see what is not used (https://en.m.wikipedia.org/wiki/Java_Code_Coverage_Tools). If you have good branch coverage with your unit tests, simply running the tests with coverage will yield the result you're looking for. If you don't have good tests coverage, you might run the application itself with code instrumented for coverage calculation, but as with unit tests - the quality of the result will depend on the amount of code executed with your unit tetst or manual test.
Some examples of the coverage tools you might use are : JaCoCo (http://www.eclemma.org/jacoco/) and Cobertura (http://cobertura.github.io/cobertura).
Alternatively you might instrument your code yourself in order to log methods usage, as it might be more lightweight than calculating full line coverage. This is however indeed reinventing the wheel.
This SO question has similar solutions: How to find unused/dead code in java projects

How to find a list of methods used only within tests [duplicate]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
What tools do you use to find unused/dead code in large java projects? Our product has been in development for some years, and it is getting very hard to manually detect code that is no longer in use. We do however try to delete as much unused code as possible.
Suggestions for general strategies/techniques (other than specific tools) are also appreciated.
Edit: Note that we already use code coverage tools (Clover, IntelliJ), but these are of little help. Dead code still has unit tests, and shows up as covered. I guess an ideal tool would identify clusters of code which have very little other code depending on it, allowing for docues manual inspection.
An Eclipse plugin that works reasonably well is Unused Code Detector.
It processes an entire project, or a specific file and shows various unused/dead code methods, as well as suggesting visibility changes (i.e. a public method that could be protected or private).
CodePro was recently released by Google with the Eclipse project. It is free and highly effective. The plugin has a 'Find Dead Code' feature with one/many entry point(s). Works pretty well.
I would instrument the running system to keep logs of code usage, and then start inspecting code that is not used for months or years.
For example if you are interested in unused classes, all classes could be instrumented to log when instances are created. And then a small script could compare these logs against the complete list of classes to find unused classes.
Of course, if you go at the method level you should keep performance in mind. For example, the methods could only log their first use. I dont know how this is best done in Java. We have done this in Smalltalk, which is a dynamic language and thus allows for code modification at runtime. We instrument all methods with a logging call and uninstall the logging code after a method has been logged for the first time, thus after some time no more performance penalties occur. Maybe a similar thing can be done in Java with static boolean flags...
I'm suprised ProGuard hasn't been mentioned here. It's one of the most mature products around.
ProGuard is a free Java class file shrinker, optimizer, obfuscator,
and preverifier. It detects and removes unused classes, fields,
methods, and attributes. It optimizes bytecode and removes unused
instructions. It renames the remaining classes, fields, and methods
using short meaningless names. Finally, it preverifies the processed
code for Java 6 or for Java Micro Edition.
Some uses of ProGuard are:
Creating more compact code, for smaller code archives, faster transfer across networks, faster loading, and smaller memory
footprints.
Making programs and libraries harder to reverse-engineer.
Listing dead code, so it can be removed from the source code.
Retargeting and preverifying existing class files for Java 6 or higher, to take full advantage of their faster class loading.
Here example for list dead code: https://www.guardsquare.com/en/products/proguard/manual/examples#deadcode
One thing I've been known to do in Eclipse, on a single class, is change all of its methods to private and then see what complaints I get. For methods that are used, this will provoke errors, and I return them to the lowest access level I can. For methods that are unused, this will provoke warnings about unused methods, and those can then be deleted. And as a bonus, you often find some public methods that can and should be made private.
But it's very manual.
Use a test coverage tool to instrument your codebase, then run the application itself, not the tests.
Emma and Eclemma will give you nice reports of what percentage of what classes are run for any given run of the code.
We've started to use Find Bugs to help identify some of the funk in our codebase's target-rich environment for refactorings. I would also consider Structure 101 to identify spots in your codebase's architecture that are too complicated, so you know where the real swamps are.
In theory, you can't deterministically find unused code. Theres a mathematical proof of this (well, this is a special case of a more general theorem). If you're curious, look up the Halting Problem.
This can manifest itself in Java code in many ways:
Loading classes based on user input, config files, database entries, etc;
Loading external code;
Passing object trees to third party libraries;
etc.
That being said, I use IDEA IntelliJ as my IDE of choice and it has extensive analysis tools for findign dependencies between modules, unused methods, unused members, unused classes, etc. Its quite intelligent too like a private method that isn't called is tagged unused but a public method requires more extensive analysis.
In Eclipse Goto Windows > Preferences > Java > Compiler > Errors/Warnings
and change all of them to errors. Fix all the errors. This is the simplest way. The beauty is that this will allow you to clean up the code as you write.
Screenshot Eclipse Code :
IntelliJ has code analysis tools for detecting code which is unused. You should try making as many fields/methods/classes as non-public as possible and that will show up more unused methods/fields/classes
I would also try to locate duplicate code as a way of reducing code volume.
My last suggestion is try to find open source code which if used would make your code simpler.
The Structure101 slice perspective will give a list (and dependency graph) of any "orphans" or "orphan groups" of classes or packages that have no dependencies to or from the "main" cluster.
DCD is not a plugin for some IDE but can be run from ant or standalone. It looks like a static tool and it can do what PMD and FindBugs can't. I will try it.
P.S. As mentioned in a comment below, the Project lives now in GitHub.
There are tools which profile code and provide code coverage data. This lets you see (as code is run) how much of it is being called. You can get any of these tools to find out how much orphan code you have.
FindBugs is excellent for this sort of thing.
PMD (Project Mess Detector) is another tool that can be used.
However, neither can find public static methods that are unused in a workspace. If anyone knows of such a tool then please let me know.
User coverage tools, such as EMMA. But it's not static tool (i.e. it requires to actually run the application through regression testing, and through all possible error cases, which is, well, impossible :) )
Still, EMMA is very useful.
Code coverage tools, such as Emma, Cobertura, and Clover, will instrument your code and record which parts of it gets invoked by running a suite of tests. This is very useful, and should be an integral part of your development process. It will help you identify how well your test suite covers your code.
However, this is not the same as identifying real dead code. It only identifies code that is covered (or not covered) by tests. This can give you false positives (if your tests do not cover all scenarios) as well as false negatives (if your tests access code that is actually never used in a real world scenario).
I imagine the best way to really identify dead code would be to instrument your code with a coverage tool in a live running environment and to analyse code coverage over an extended period of time.
If you are runnning in a load balanced redundant environment (and if not, why not?) then I suppose it would make sense to only instrument one instance of your application and to configure your load balancer such that a random, but small, portion of your users run on your instrumented instance. If you do this over an extended period of time (to make sure that you have covered all real world usage scenarios - such seasonal variations), you should be able to see exactly which areas of your code are accessed under real world usage and which parts are really never accessed and hence dead code.
I have never personally seen this done, and do not know how the aforementioned tools can be used to instrument and analyse code that is not being invoked through a test suite - but I am sure they can be.
There is a Java project - Dead Code Detector (DCD). For source code it doesn't seem to work well, but for .jar file - it's really good. Plus you can filter by class and by method.
Netbeans here is a plugin for Netbeans dead code detector.
It would be better if it could link to and highlight the unused code. You can vote and comment here: Bug 181458 - Find unused public classes, methods, fields
Eclipse can show/highlight code that can't be reached. JUnit can show you code coverage, but you'd need some tests and have to decide if the relevant test is missing or the code is really unused.
I found Clover coverage tool which instruments code and highlights the code that is used and that is unused. Unlike Google CodePro Analytics, it also works for WebApplications (as per my experience and I may be incorrect about Google CodePro).
The only drawback that I noticed is that it does not takes Java interfaces into account.
I use Doxygen to develop a method call map to locate methods that are never called. On the graph you will find islands of method clusters without callers. This doesn't work for libraries since you need always start from some main entry point.

Java call hierarchy analysis

Are there any tools out there that I could use to help me in analyzing call hierarchies of Java code? I am essentially looking for a way to select a method, generate the hierarchy, and then serialize to disk the highest level pieces of the hierarchy paths (all the roots) so that I can get some stats on it and get a better idea of risk/testing effort if this code were to be refactored.
My end goal is for any given method in a Java web app (no matter what layer it's in), figure out all the entry points into the application (Struts actions, web services) that a given method is utilized by. Eclipse has a nice tool that I use all the time for call hierarchies, but the method I'm analyzing is so heavily utilized that it won't be practical for me to go through manually expanding the trees and typing everything by hand.
Any help would be greatly appreciated.
Take a look at Cobertura. It's a tool to monitor test coverage but there's no reason it can't be used for what you need. The reports (sample report) are very comprehensive, allowing you to drill down to see which lines were executed and now many times.
A similar tool is Emma, which has been updated more recently than Cobertura. I've never used Emma so I can't comment on how well it works. The sample report looks similar to Cobertura's.

Categories