Checkstyle vs. PMD - java
We are introducing static analysis tools into the build system for our Java product. We are using Maven2 so Checkstyle and PMD integration come for free. However it looks like there is a large overlap in functionality between these two tools, in terms of enforcing basic style rules.
Is there a benefit from utilizing both of these? I don't want to maintain 2 tools if one will work. If we choose one, which one should we use and why?
We are also planning on using FindBugs. Are there other static analysis tools we should look at?
Update: Consensus seems to be that PMD is preferred over CheckStyle. I don't see a solid reason to use both, and I don't want to maintain 2 sets of rule files, so we will probably aim for PMD exclusively. We'll also be bringing in FindBugs, and perhaps, eventually, Macker to enforce architectural rules.
You should definitely use FindBugs. In my experience, the false-positive rate is very low, and even the least-critical warnings it reports are worth addressing to some extent.
As for Checkstyle vs. PMD, I would not use Checkstyle since it is pretty much only concerned with style. In my experience, Checkstyle will report on a ton of things that are completely irrelevant. PMD on the other hand is also able to point out questionable coding practices and its output is generally more relevant and useful.
Both softwares are useful. Checkstyle will help you during your programming by checking your coding style i.e braces, naming etc. Simple things but very numerous!
PMD will help you by checking more complicate rules like during the design of your classes, or for more special problems like implementing correctly the clone function. Simply, PMD will check your programming style
However, both softwares suffers from similar rules sometimes bad explained. With a bad configuration, you may check things twice or two opposite things i.e "Remove useless constructors" and "Always one constructor".
If we choose one, which one should we use and why?
These tools are not competing but are complementary and should be used simultaneously.
The convention type (Checkstyle) is the glue that enables people to work together and to free up their creativity instead of spending time and energy at understanding inconsistent code.
Checkstyle examples:
Is there javadoc on public methods ?
Is the project following Sun naming conventions ?
Is the code written with a consistent format ?
while PMD reminds you bad practices:
Catching an exception without doing anything
Having dead code
Too many complex methods
Direct use of implementations instead of interfaces
Implementing the hashcode() method without the not equals(Object object) method
source:
http://www.sonarsource.org/what-makes-checkstyle-pmd-findbugs-and-macker-complementary/
We use both:
Checkstyle to make sure that everyone in the team write code in a similar maner
PMD to find problematic code areas and next refactoring targets
If your primary place of use is while developing in eclipse, then CodePro from Instantiations will be best. Earlier it was a commercial tool, but now Google bought Instantiations so CodePro analytix is free now.
Check out http://code.google.com/javadevtools/download-codepro.html
If you reviewed Checkstyle, PMD and Findbugs rule lists, you have seen that all three provide valuable output and all three overlap to a degree and also bring their own, unique rules to the table. This is why tools like Sonar use all three.
That said, Findbugs has the most specific or niche rules (e.g. "Dubious catching of IllegalMonitorStateException" - how often are you likely to run into that?) so it is usable with little or no configuration and its warnings should be taken seriously. With Checkstyle and PMD the rules are more general and style-related so they should only be used with custom configuration files to spare the team from an avalanche of irrelevant feedback ("Tab char on line 5", "Tab char on line 6", "Tab char on line 7"... you get the picture). They also provide powerful tools to write your own advanced rules, e.g. the Checkstyle DescendentToken rule.
When using all three (especially with a tool like Sonar), all of them should be configured separately (takes at least a few days to cover all the rules) while paying attention to prevent duplication (all three tools detect that hashCode() has been overridden and equals() not, for example).
In summary, if you consider static code analysis valuable, rejecting the value any of the three provides makes no sense, but to use all three, you have to invest time to configure them to give you usable feedback.
Sonar (http://www.sonarsource.org/) is a very useful open platform to manage code quality, and includes Checkstyle, PMD, Findbugs and much more.
This also indicates that all 3 tools have their right to exist...
Both tools are configurable and can do just about the same things. That said, if we're talking about out-of-the-box stuff, there is a great deal of overlap, but there are distinct rules/checks as well. For example, Checkstyle has stronger support for checking Javadoc and finding magic numbers, to name a couple. Additionally, Checkstyle has an "import control" feature that looks similar to the functionality of Macker (I've not used Macker).
If there are things that are important to you that Checkstyle does out-of-the-box that PMD doesn't, you might consider a minimal Checkstyle configuration with only those checks. Then institute a policy that the Checkstyle configuration cannot grow, simply remove checks as you implement similar functionality with, say, custom PMD rules.
Also consider that if you decide that the Checkstyle "import control" feature covers what you wanted from Macker, then you could implement PMD/Checkstyle instead of PMD/Macker. Either way it's two tools, but with Checkstyle, you'd get the stuff that PMD doesn't do out-of-the-box "for free."
Checkstyle and PMD both are good at checking coding standards and are easy to extend.
But PMD has additional rules to check for cyclomatic complexity,Npath complexity,etc which allows you write healthy code.
Another advantage of using PMD is CPD (Copy/Paste Detector).It finds out code duplication across projects and is not constrained to JAVA.It works for JSP too.
Neal Ford has a good presentation on Metrics Driven Agile Development, which talks about many tools that are helpful for Java/Java EE Development
I find Checkstyle and PMD are best for enforcing style issues and simple obvious coding bugs. Although I've found that I like using Eclipse and all the warnings it provides better for that purpose. We enforce stuff by using shared preferences and marking them as actual errors. That way, they never get checked in in the first place.
What I would strongly and enthusiastically recommend is using FindBugs. Because it works at the bytecode level it can check things that are impossible at the source level. While it spits out its fair share of junks, it has found many actual and important bugs in our code.
And 10 years later ...
In 2018 I use all of them Checkstyle, PMD and FindBugs.
Start with FindBugs. Maybe add PMD and Checkstyle later.
Never blindly enforce the default rules !
Steps:
run one tool with default rules on a project which has a lot of code
adapt the rules to this project, comment out useless rules with some notes
focus on the low hanging fruits rules (NPE, logger checks, unclosed resource checks, ...)
perform some fixes for rules you find worthwhile (one at a time !)
do this for each tool but not all at once !
repeat this process
Ideally each project can have separate rules.
I like running the rules via the build (via maven plugins) and fail on rule errors once I know a project passes all the rules I defined.
This forces developers to take action, because reporting is not enough.
From that point on your project is pretty much bullet proof and you could even add more rules later on and/or write custom rules.
One point I have not seen so far is that there are plugins for IDEs that will enforce CheckStyle rulesets on your code, whereas PMD plugins will only report on violations. For example, in a multi-site project over several programming teams, it's important to actively enforce standards, rather than just to report on them.
Both tools have plugins available for IntelliJ, NetBeans, and Eclipse (in my view this covers most usage). I'm not as familiar with NetBeans, so can only comment on IntelliJ and Eclipse.
Anyway, the PMD plugins for IntelliJ, and Eclipse, will generate reports on demand on PMD violations within the project codebase.
The CheckStyle plugins, on the other hand, will highlight violations on the fly, and can (at least for IntelliJ, I have less experience with Eclipse) be configured to automatically convert some issues (e.g. for 'OneStatementPerLine', will place CR-LF between statements, for 'NeedBraces', will add braces where they are missing, etc.). Obviously, only the simpler violations can be automatically fixed, but it's still a help on legacy projects, or projects located over several locations.
'On demand' for PMD means that the developer must consciously decide to run the report. Whereas Checkstyle violations are automatically reported to them as they develop. While PMD does contain a more extensive ruleset, in my mind the automatic enforecement/reporting of violations in IDEs is worth the hassle of maintaining 2 sets of rules.
So for any projects I work on, we use both tools, Checkstyle enforced in the IDE, PMD reported in the IDE, and both reported and measured in builds (through Jenkins).
Take a look at qulice-maven-plugin that combines together Checkstyle, PMD, FindBugs and a few other static analyzers, and pre-configures them. The beauty of this combination is that you don't need to configure them individually in every project:
<plugin>
<groupId>com.qulice</groupId>
<artifactId>qulice-maven-plugin</artifactId>
<version>0.15</version>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
I would echo the comment that PMD is the more current product for Java style/convention checking. With respect to FindBugs, many commercial development groups are using Coverity.
PMD is what I find more people referring to. Checkstyle was what people were referring to 4 years ago but I believe PMD is maintained more continuously and what other IDEs/plugins choose to work with.
I have just started to use Checkstyle and PMD. To me, PMD is more easier to create customized rules for things such that whether there exists System.gc(), Runtime.gc(), as long as you can write the XPath Query which is also not difficult at all. However, PMD has not shown me that it has the feature to show column number. So for things like check column limits. You might would like to use Checkstyle.
PMD is the finest tool when compare with checkstyles. Checkstyles may not have the capability to analyse the code while PMD offering many features to do so! Offcourse PMD has not released rules for javadoc, comments, indentations and etc. And by the way i am planning to implement these rules.......thanx
Related
Sonar rule for interfaces only RSPEC-1213
I want to modify / make the rule target only public interfaces (not public classes etc). Is this possible ? Im using this rule in Java code but its too strict for my project and I would love to know if there is a way to change it a little bit. Link for rule: https://rules.sonarsource.com/java/RSPEC-1213
For an existing ruleset on SonarQube, talk to your sonar administrator to change the rules that are enforced on the code and remove that particular one from global enforcement.
There have been a few times I've gone to the admins of the tool for the install that I use and said "this rule isn't one that I care about or will enforce and only makes it confusing" and had them remove that rule from the globally run ruleset.
Is it possible to write your own rule?
Yes, it is possible. From SonarQube's docs: Adding coding rules you have some options. Either you can write a plugin for SonarQube and add that to your instance (docs), or you can write an external application that analyzes the code which SonarQube consumes.
If you don't have your own instance of sonarqube or aren't up to writing the associated plugin or external tooling... you might want to instead lookout PMD (site).
For PMD, writing a custom rule can be much simpler (docs). One of the ways that PMD works is by 'compiling' the Java code into an XML representation of the abstract syntax tree for Java and then running xpath queries against that XML (tutorial).
The xpath rule can then be included in a project's configuration.
What about turning it off for the code that I'm working on?
If a specific rule is one that you don't want to invoke, you could suppress it with #SuppressWarnings("java:S106") (that particular spares warnings is for System.out.println use, but the same structure can be used for other warnings) or by adding // NOSONAR too strict on the line. There are spots where I have such comments where following the rule for a particular set of code is problematic and suppress it for that line, method, or class - with the comment about why that is done.
That particular rule... I'm gonna agree with the Java (and now Oracle) guidelines and follow it. The reason is that if anyone else works on the code, they'll expect it to follow that convention. Having a consistent understanding of what things should be where in code so that another developer doesn't need to go dig through an entire file to find the constructor when it is expected to be at the top (under the field definition) is a good thing. What's more, it limits the future cases where a developer goes through to make things consistent with conventions and results in a lot of style: updating code to follow style guide commits later.
Java formatting convention for new project
I am starting a new project which might be open-sourced later on and/or at least get some external contributors during its life-time. I am now thinking about what the best approach to code-style / auto-formatting would be. I am a strong supporter of only having auto-formatted code committed to a project, as this eliminates the differences between individual developers and helps keeping individual commits clutter-free of reformatting issues. My first approach was to use Eclipse built-in style for the project, but I really don't like the default style, because I think line-break at 80 characters is way out-dated for today's screen resolutions. Also, as the name suggests, it's available only for people using Eclipse as IDE. So I was also thinking about using my own formatter settings and checking the exported settings into the project's repository so that any contributor can pick them up. Again, this would force most people to use Eclipse, as I am not aware of any formatting definition that can be read by multiple IDEs. Any hint how this is handled in other projects? I searched some github repositories, but to me it seems that this issue is more or less ignored by a lot of projects. I do understand that this question may be border-line for Stack Overflow, as I don't know if a definite answer is possible and if this triggers a discussion, but it is something I often struggle with when starting a new project.
While screens grow wider, they don't seem to grow taller.
Whatever you other drivers are, preserve vertical space. Put { and } on lines containing other language key words, if you can.
In any case, use a maven plugin or other automated tool in your compile chain to enforce the rules that you care about. That way they are unambiguous.
Also don't create too many rules that don't matter. Each rule costs time to make the code comply.
I understand your concern and in my opinion the best approach is to create code formatting preference file which can be shared along with the project. For example in eclipse Using a file explorer, navigate to //.settings and copy org.eclipse.jdt.core.prefs to a new location. This file contains all your formatting settings. Hence this can be shared to maintain the code formatting consistencies. If not that then you might have to rely on the editor specific code formatting. I definitely look forward to other expert opinion on the same if what I have shared is not optimal as per the requirement.
Why java method name should not contain underscore?
I am using PMD for checking java coding violation.
I am writing automation test cases, and given method name like this:
public void testCaseMethod_4_2_16(){
//some implementation
}
where 4_2_16 is test case number 4.2.16
and when I check for PMD violation it is showing method name should not contain underscore, and this violation is shown as blocker according to PMD rules.
So here are my question:
Why method name should not contain underscore is define as a blocker or PMD error?
What kind of PMD violation we should avoid and
What kind of violation we should try to fixed?
As for my understanding at least we should avoid first two level of violation shown by PMD.
Any help on the PMD rule will be appreciated. thanks
It is a standard that has been set so that people can easily read each other's code, therefore making code more maintainable. The quote below is from Oracle's website on Java code conventions: 80% of the lifetime cost of a piece of software goes to maintenance. Hardly any software is maintained for its whole life by the original author. Code conventions improve the readability of the software, allowing engineers to understand new code more quickly and thoroughly.
Java prefers camelCase notation. Java code conventions say that you should try to avoid underscores in method or variable name whenever it is possible. In your case PMD's default prioritization fails. This rule should be on a "nice-to-have" level, but definitely not a blocker issue. I would recommend you to use SonarQube, which did a re-prioritization of PMD's rules, which works better is most cases. The rule you found is just on Major level (below levels Blocker and Critical). If you are still not satisfied with the results, you can use the SourceMeter plugin for SonarQube, which has a different (possible better) prioritization of PMD rules as well.
Although there are lots of opinions out there, I like to separate quality-assurance for production and the one for test code. If that is not possible for whatever reason, I'd rather deactivate (most of) the warnings for test-code. I think it is not worth the time making the test-code pretty (and underscores in method names is just styling to me), while there are serious warnings for the production code left open.
In java underscore is allowed while naming identifiers. I don't know why PMD is showing violation but you can just deactivate the warnings and go on. UNDERSCORE is allowed in java while naming identifiers. I think it may help you...
CheckStyle, FindBugs, and PMD configurations without overlap
At my company we started using CheckStyle, FindBugs, and PMD to check our code quality and unify our programming styles. These tools are very effective but unfortunately they have some overlapping rule sets. Are there configurations available online which have removed the overlapping rules? Of course, these default rule sets are not applicable to every project but it would be a good starting point for us. We could take this default (non overlapping) configuration and adapt it to our style instead of pruning the rule sets for duplicates first.
Sonar ships with some pretty good configurations called Sonar Way and Sonar Way with FindBugs which don't appear to have any overlap between the different tools (CheckStyle, FindBugs, and PMD). They seem to fulfill my needs as a decent starting point for a configuration without all the superfluous rules. PS I was hesitant to answer my own question but nobody else had an answer or even an insightful comment. This thread on Meta also seems to think it's okay.
What code analysis tools do you use for your Java projects? [closed]
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 10 years ago. What code analysis tools do you use on your Java projects? I am interested in all kinds static code analysis tools (FindBugs, PMD, and any others) code coverage tools (Cobertura, Emma, and any others) any other instrumentation-based tools anything else, if I'm missing something If applicable, also state what build tools you use and how well these tools integrate with both your IDEs and build tools. If a tool is only available a specific way (as an IDE plugin, or, say, a build tool plugin) that information is also worth noting.
For static analysis tools I often use CPD, PMD, FindBugs, and Checkstyle.
CPD is the PMD "Copy/Paste Detector" tool. I was using PMD for a little while before I noticed the "Finding Duplicated Code" link on the PMD web page.
I'd like to point out that these tools can sometimes be extended beyond their "out-of-the-box" set of rules. And not just because they're open source so that you can rewrite them. Some of these tools come with applications or "hooks" that allow them to be extended. For example, PMD comes with the "designer" tool that allows you to create new rules. Also, Checkstyle has the DescendantToken check that has properties that allow for substantial customization.
I integrate these tools with an Ant-based build. You can follow the link to see my commented configuration.
In addition to the simple integration into the build, I find it helpful to configure the tools to be somewhat "integrated" in a couple of other ways. Namely, report generation and warning suppression uniformity. I'd like to add these aspects to this discussion (which should probably have the "static-analysis" tag also): how are folks configuring these tools to create a "unified" solution? (I've asked this question separately here)
First, for warning reports, I transform the output so that each warning has the simple format:
/absolute-path/filename:line-number:column-number: warning(tool-name): message
This is often called the "Emacs format," but even if you aren't using Emacs, it's a reasonable format for homogenizing reports. For example:
/project/src/com/example/Foo.java:425:9: warning(Checkstyle):Missing a Javadoc comment.
My warning format transformations are done by my Ant script with Ant filterchains.
The second "integration" that I do is for warning suppression. By default, each tool supports comments or an annotation (or both) that you can place in your code to silence a warning that you want to ignore. But these various warning suppression requests do not have a consistent look which seems somewhat silly. When you're suppressing a warning, you're suppressing a warning, so why not always write "SuppressWarning?"
For example, PMD's default configuration suppresses warning generation on lines of code with the string "NOPMD" in a comment. Also, PMD supports Java's #SuppressWarnings annotation. I configure PMD to use comments containing "SuppressWarning(PMD." instead of NOPMD so that PMD suppressions look alike. I fill in the particular rule that is violated when using the comment style suppression:
// SuppressWarnings(PMD.PreserveStackTrace) justification: (false positive) exceptions are chained
Only the "SuppressWarnings(PMD." part is significant for a comment, but it is consistent with PMD's support for the #SuppressWarning annotation which does recognize individual rule violations by name:
#SuppressWarnings("PMD.CompareObjectsWithEquals") // justification: identity comparision intended
Similarly, Checkstyle suppresses warning generation between pairs of comments (no annotation support is provided). By default, comments to turn Checkstyle off and on contain the strings CHECKSTYLE:OFF and CHECKSTYLE:ON, respectively. Changing this configuration (with Checkstyle's "SuppressionCommentFilter") to use the strings "BEGIN SuppressWarnings(CheckStyle." and "END SuppressWarnings(CheckStyle." makes the controls look more like PMD:
// BEGIN SuppressWarnings(Checkstyle.HiddenField) justification: "Effective Java," 2nd ed., Bloch, Item 2
// END SuppressWarnings(Checkstyle.HiddenField)
With Checkstyle comments, the particular check violation (HiddenField) is significant because each check has its own "BEGIN/END" comment pair.
FindBugs also supports warning generation suppression with a #SuppressWarnings annotation, so no further configuration is required to achieve some level of uniformity with other tools. Unfortunately, Findbugs has to support a custom #SuppressWarnings annotation because the built-in Java #SuppressWarnings annotation has a SOURCE retention policy which is not strong enough to retain the annotation in the class file where FindBugs needs it. I fully qualify FindBugs warnings suppressions to avoid clashing with Java's #SuppressWarnings annotation:
#edu.umd.cs.findbugs.annotations.SuppressWarnings("UWF_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR")
These techniques makes things look reasonably consistent across tools. Note that having each warning suppression contain the string "SuppressWarnings" makes it easy to run a simple search to find all instances for all tools over an entire code base.
I use a combination of Cobertura, Checkstyle, (Ecl)Emma and Findbugs. EclEmma is an awesome Eclipse plugin that shows the code coverage by coloring the java source in the editor (screenshot) - the coverage is generated by running a JUnit test. This is really useful when you are trying to figure out which lines are covered in a particular class, or if you want to see just which lines are covered by a single test. This is much more user friendly and useful than generating a report and then looking through the report to see which classes have low coverage. The Checkstyle and Findbugs Eclipse plugins are also useful, they generate warnings in the editor as you type. Maven2 has report plugins that work with the above tools to generate reports at build time. We use this to get overall project reports, which are more useful when you want aggregate numbers. These are generated by our CI builds, which run using Continuum.
All of the following we use and integrate easiy in both our Maven 2.x builds and Eclipse/RAD 7: Testing - JUnit/TestNG Code analysis - FindBugs, PMD Code coverage - Clover In addition, in our Maven builds we have: JDepend Tag checker (TODO, FIXME, etc) Furthermore, if you're using Maven 2.x, CodeHaus has a collection of handy Maven plugins in their Mojo project. Note: Clover has out-of-the-box integration with the Bamboo CI server (since they're both Atlassian products). There are also Bamboo plugins for FindBugs, PMD, and CheckStyle but, as noted, the free Hudson CI server has those too.
I use the static analysis built into IntelliJ IDEA. Perfect integration. I use the code coverage built into Intellij IDEA (based on EMMA). Again, perfect integration. This integrated solution is reliable, powerful, and easy-to-use compared to piecing together tools from various vendors.
Checkstyle is another one I've used at a previous company... it's mainly for style checking, but it can do some static analysis too. Also, Clover for code coverage, though be aware it is not a free tool.
We are using FindBugs and Checkstyle as well as Clover for Code Coverage. I think it's important to have some kind of static analysis, supporting your development. Unfortunately it's still not widely spread that these tools are important.
We use FindBugs and JDepend integrated with Ant. We use JUnit but we're not using any coverage tool. I'm not using it integrated to Rational Application Developer (the IDE I'm using to develop J2EE applications) because I like how neat it looks when you run javac in the Windows console. :P
I've had good luck with Cobertura. It's a code coverage tool which can be executed via your ant script as part of your normal build and can be integrated into Hudson.
Our team use PMD and Cobertura, actually our projects are maven projects and there is very simple to include plug ins for code analysis. The real question would be for specific project which analysis you need to use, my opinion is that it's you couldn't use the same plugins for each project.
in our project we use Sonar in front of checkstyle, pmd.... together with the CI (Bamboo, Hudson) we get also a nice history of our source quality and what directing we go. I do like Sonar, because you one central tool in the CI Stack that does it for you, and you can easy customize the rules for each project.
Structure 101 is good at code analysis and finding the cyclic package dependencies.
I am looking for many answers to learn about new tools and consolidate this knowledge in a one question/thread, so I doubt there will be 1 true answer to this question. My answer to my own question is that we use: Findbugs to look for common errors bad/coding - run from maven, and also integrates easily into Eclipse Cobertura for our coverage reports - run from maven Hudson also has a task-scanner plugin that will display a count of your TODO and FIXMEs, as well as show where they are in the source files. All are integrated with Maven 1.x in our case and tied into Hudson, which runs our builds on check-in as well as extra things nightly and weekly. Hudson trend graphs our JUnit tests, coverage, findbugs, as well as open tasks. There is also a Hudson plugin that reports and graphs our compile warnings. We also have several performance tests with their own graphs of performance and memory use over time using the Hudson plots plugin as well.