I'm trying to write rules for detecting some errors in annotated multi-threaded java programs. As a toy example, I'd like to detect if any method annotated with #ThreadSafe calls a method without such an annotation, without synchronization. I'm looking for a tool that would allow me to write such a test.
I've looked at source analyzers, like CheckStyle and PMD, and they don't really have cross-class analysis capabilities. Bytecode analysers, like FindBugs and JLint seem rather difficult to extend.
I'd settle for a solution to something even simpler, but posing the same difficulty: writing a custom rule that checks whether each overriden method is annotated with #Override.
Have you tried FindBugs? It actually supports a set of annotations for thread safety (the same as those used in Java Concurrency in Practice). Also, you can write your own custom rules. I'm not sure whether you can do cross-class analysis, but I believe so.
Peter Ventjeer has a concurrency checking tool (that uses ASM) to detect stuff like this. I'm not sure if he's released it publicly but he might able to help you.
And I believe Coverity's static/dynamic analysis tools for thread safety do checking like this.
You can do cross-class analysis in PMD (though I've never used it for this specific purpose). I think it's possible using this visitor pattern that they document, though I'll leave the specifics to you.
A simple tool to checkup on annotations is apt (http://java.sun.com/j2se/1.5.0/docs/guide/apt/ also part of Java 6 api in javax.annotation.processing) however this only has type information (ie I couldn't find a quick way to get at the inheritance hierarchy using the javax.lang.model api, however if you can load the class you can get that information using reflection).
Try javap + regexes (eg. Perl)
Related
I want to modify / make the rule target only public interfaces (not public classes etc). Is this possible ? Im using this rule in Java code but its too strict for my project and I would love to know if there is a way to change it a little bit.
Link for rule: https://rules.sonarsource.com/java/RSPEC-1213
For an existing ruleset on SonarQube, talk to your sonar administrator to change the rules that are enforced on the code and remove that particular one from global enforcement.
There have been a few times I've gone to the admins of the tool for the install that I use and said "this rule isn't one that I care about or will enforce and only makes it confusing" and had them remove that rule from the globally run ruleset.
Is it possible to write your own rule?
Yes, it is possible. From SonarQube's docs: Adding coding rules you have some options. Either you can write a plugin for SonarQube and add that to your instance (docs), or you can write an external application that analyzes the code which SonarQube consumes.
If you don't have your own instance of sonarqube or aren't up to writing the associated plugin or external tooling... you might want to instead lookout PMD (site).
For PMD, writing a custom rule can be much simpler (docs). One of the ways that PMD works is by 'compiling' the Java code into an XML representation of the abstract syntax tree for Java and then running xpath queries against that XML (tutorial).
The xpath rule can then be included in a project's configuration.
What about turning it off for the code that I'm working on?
If a specific rule is one that you don't want to invoke, you could suppress it with #SuppressWarnings("java:S106") (that particular spares warnings is for System.out.println use, but the same structure can be used for other warnings) or by adding // NOSONAR too strict on the line. There are spots where I have such comments where following the rule for a particular set of code is problematic and suppress it for that line, method, or class - with the comment about why that is done.
That particular rule... I'm gonna agree with the Java (and now Oracle) guidelines and follow it. The reason is that if anyone else works on the code, they'll expect it to follow that convention. Having a consistent understanding of what things should be where in code so that another developer doesn't need to go dig through an entire file to find the constructor when it is expected to be at the top (under the field definition) is a good thing. What's more, it limits the future cases where a developer goes through to make things consistent with conventions and results in a lot of style: updating code to follow style guide commits later.
We are trying to keep our code 'consistent' over time on our projects and follow this pattern
https://blog.twitter.com/engineering/en_us/topics/insights/2019/onepattern.html
For any classes annotated with #Entity or #Data, we consider these data objects and would like to enforce that
The new keyword cannot be used on any objects NOT annotated with #Data (ie. all business logic is created by guice) - This is the hard one here!
Only get/set/is member methods are in the data objects(perhaps along with equals) - This could be done by scanning and reflection
All objects with no #Data are annotated with #Singleton(we program in stateless) OR with #ExceptionNotSingleton for exceptions (our exceptions are very rare as we build stateless systems - Could again be done with scanning and reflection
NO static methods allowed on business objects even utilities -> scan and reflection
The 4th point is for a specific project webpieces where since we use that platform, we can fix bugs in any code by swapping out the class. swapping static methods doesn't work very well.
What tool can do something like this such that I can feed it a simple processor? A gradle plugin would be nice where I can feed it some small snippets of code, but I am not sure that exists.
I could add a unit test to every project as well I guess and then just need a class scanner to inspect the classes.
I am looking for a plugin perhaps with a link to some examples I could try out.
What you're talking about is similar to static code analysis.
The basic (and simplified) idea is to parse the source code into a tree and then analyze that tree for the specific patterns you're looking for.
In you case, for example, whenever you see a constructor invocation in the tree, you check whether the target class is annotated with #Data or not. Or, whenever you're analyzing a class, you visit all its method leaves and look for statics. If you spot violations – you log them and probably break the build.
The trick here is to define your rules in the term of formal patterns in AST tree.
There are a plenty of code analyzers for Java, but IMHO the most popular one is Checkstyle. All of them probably work nice with Gradle (Checkstyle and PMD do), so it's not really a question about build tool, but about extending the analyzer.
Usually, static code analyzers allows users to extend them with custom checks / rules / plugins / etc. So does Checkstyle: read more about writing new checks. This article describes the AST and navigation in details. Checkstyle also provides a GUI tool to analyze Java classes to help find the patterns and write checks. Go ahead an load some of you clases there and look if you can track your issues via AST tree.
When you're comfortable with AST, its navigation, visitors and so on – go and write your own checks and integrate them in you build.
This idea is applicable for other static code analyzers as well, like PMD or SpotBugs (although SpotBugs's docs are poor).
I have a legacy code base that I want to refactor to reduce the visibility of methods to the minimum possible (private, protected, default) such that the code still works. Many of the methods in the codebase are unnecessarily public, and I'd like to change that to reduce the interface burden and simplify documentation as the code evolves in the future. Is there a tool that will analyze the codebase and generate a list of suggested methods whose visibility can be reduced? I can specify all the entry points into the code (just the main methods), and the codebase doesn't use reflection.
An Eclipse plugin will be even better.
Take a look at UCDetector (Unnecessary Code Detector - pronounced "You See Detector")
http://www.ucdetector.org/
It is an eclipse plugin and it helps to reduce visibility. I have used it for long time and it works very well.
Although there are tools that you can use to report the occurrence of methods which visibility can be reduced, I am not aware of something that allows you to transform the code to solve those issues.
However, you may find interesting taking a look to JTransformer and Ekeko.
Both allows you to query and accomplish custom code transformations based on logic programming techniques. JTransformer may be a bit more mature, but Ekeko also looks quite interesting. To the best of my knowledge they both are open source and include an Eclipse plugin.
I am wondering if there is a tool that can identify cohesive blocks of code within JAVA source code.
For example if I had a long method that I would like to extract another method from - is there any tool that automatically can tell me large chunks of code that would be worth extracting?
There are plug-ins like PMD (for eclipse) & FindBugs etc., to do static code review which flags code based on rules your configured.
Google CodePro Analytics has an Eclipse plug-in that can provide a bunch of statistics like lines of code and cyclomatic complexity that can be good indicators that a method should be refactored.
I don't think you will find a tool that can automatically refactor 'cohesive' blocks of code into methods. There is too much subjectivity in that.
I looked for a similar tool with another question: https://stackoverflow.com/questions/12016289/tool-for-visualizing-dependencies-inside-a-java-class just on a slightly higher level: a single class.
I think the same answer applies: There isn't anything like that. There are tools though that provide information from which you might extract the information you are looking for.
I'd look into DependencyFinder. It provides access to all the bits and pieces of the code, so you could find clusters of code elements that access a common set of variables. Unfortunately I found the API a little confusing and not well documented, so you'll need some try and error or get into contact with the author. It also probably won't give you access to whitespace I think. But I don't think this is a valid approach anyway.
Another Tool you might want to look into is JaMoPP It should even have information about whitespace. Although it is a Java Plugin you can use the underlying library independent of eclipse (I think).
Check out Sonar It has very good support for finding duplicate code blocks.
Sonar uses PMD and FindBugs underlying. It also generates some custom metrics like class complexity, method complexity which points to classes / methods that are too large and which are candidate for breaking down.
Control blocks (i.e. conditionals and loops) are "cohesive" in that you cannot readily extract blocks of code that cross control block boundaries. Find blocks that can be replaced by a method call, that makes the original method easier to understand. You will have the best impact on complexity by extracting out the regions of deepest control flow nesting, so this is a good place to start. You don't need a tool as such - the code itself has the info you need.
In software development we are all using the libraries by software providers. Consider in class A there are four functions viz., x,y,z. I just want my development team to avoid using the function x. So instead of telling them not to use, I found an idea. Inherit the class and override all the functions and for the function x an unsupportedmethod exception is thrown and for the rest I'm calling the super methods. There also I found a problem, developers can use the base class A directly, how to avoid the class A being used directly. I found a similar functionality in OSGi, the lib bundles can be brought in and then not exported and so on. Is there are any way to achieve this is java?
I suppose code reviews exist for these reasons. Consider situation where you can not edit the source of a third party, what would you do ? Like Siddharth says, sub class it and throw a meaningful exception and document it with a clear reasons. If someone is using base class even after that, mostly it may not out of ignorance,but it may out of curiosity. That kind of thing can be appreciated personally and for learning, but for the project sake developer has to follow the guidelines.
I think simply telling your developers what to do is preferred over a complex software solution. Sometimes the simple thing is better.
But, if you insist on going down this path, you can enforce your architecture standards using aspects if you're a Spring user. Weave the offending methods with an aspect that throws an exception if they're called.
You can edit library class file in hex editor and modify its access modifier from public to package private. Also you can rename it and then use inheritance to wrap this class. Here you can find class file specification. Once I've tried this technique to substitute jdbc driver class with wraper class that provide some additional logging and other useful tricks.
There is a variety of tools that check source code for adherence to certain rules, such as formatting, dead code, naming conventions for variables etc. Popular ones for Java include the Maven Enforcer plugin, checkstyle and PMD.
These might allow you to write a rule that forbids certain method calls. Then you could check automatically at compile time. As far as I can tell, unfortunately none of the tools above support "illegal method calls" out-of-the-box; however, at least for PMD writing new checks is fairly simple.